Using LSTM Networks To Predict Crypto Prices
Two weeks ago, I started off on my journey in using machine learning to make predictions on crypto prices. To do a short recap, I’m working on a cryptocurrency trading bot with my brother, which leverages small differences in the prices of different exchanges, such as between Ethereum and Bitcoin. In this project, I was given the task of making a prediction of the price of the next few trades based on the previous 60 trades.
For this task, I decided to use LSTM networks because of their ability to hold information from previous long-term data and use it to make accurate predictions. I also found that RNNs’ suffer from vanishing and exploding gradients, which means that the weights can either change extremely slowly or overshoot heavily, which is why I didn’t decide to stick with LSTM’s.
Here’s a short explanation of how LSTM’s work.
In essence, all an LSTM is doing is using the previous outputs to make its next prediction. In this diagram, you can see that the output A is being passed into the next prediction of the network. This allows the network to make more accurate predictions based on the previous data it’s given. Because of this, LSTM’s are wayyy more powerful than deep learning algorithms (for time-series predictions).
But, it gets a bit more complex than that. The LSTM is actually going through a number of different activation functions and steps to make predictions. This picture models what the A in the previous picture was doing. This is called a memory cell, and it has a few functionalities.
I know, I know, this may look like rocket science, but it’s really not. If you look at the picture, you can see that there are three inputs. The memory from the previous block, the input vector, and the output of the previous block. The forget gate just removes certain information from the memory, the input gate determines which information to add to the memory, and that memory is then used to determine the output and is then passed on to the next memory cell, where the same cycle occurs. Otherwise, it’s just element wise multiplication and summation to get from an input to an output.
So. Let’s get into the details of how I used an LSTM in my project. I first extracted the data from a MySQL database using pymysql.
trainset = con.cursor()trainset.execute("SELECT * FROM Trades.tx WHERE Symbol='ETHBTC' AND time >= '2019-10-14 02:23:30.437000' LIMIT 0, 40000;")
rows = trainset.fetchall()
Once I got the data into python, I got it ready for preprocessing, which essentially meant separating the price column in the DB. This allowed me to convert it into a vector that was suitable for the preprocessing.
trainingset= []for row in rows:
trainingset.append(row[2])trainingSet = np.array(trainingSet , dtype='f')
This next block of code essentially ordered all the data into a matrix that kept on increasing the element in the vector by 1, meaning that if the last value of the row in the matrix had an index of i, the next row would have an index of i + 1.
from sklearn import preprocessingtrainingSet = trainingSet.reshape(-1, 1)training_set = trainingSet
# Feature Scaling
from sklearn.preprocessing import MinMaxScaler
sc = MinMaxScaler(feature_range = (0, 1))
training_set_scaled = sc.fit_transform(training_set)X_train = []
for i in range(60, len(trainingSet)):
X_train.append(training_set_scaled[i-60:i, 0])X_train = np.array(X_train)# Reshaping to get ready for training
X_train = np.reshape(X_train, (X_train.shape[0], X_train.shape[1], 1))
This allows the LSTM to increment by 1 element, instead of 60. After I got the data preprocessed, I fed it into the network, which used keras and trained overnight for 100 epochs. I initially had a lot less, but I was getting really inaccurate results.
regressor = Sequential()regressor.add(LSTM(units = 50, return_sequences = True, input_shape = (X_train.shape[1], 1)))
regressor.add(Dropout(0.2))regressor.add(LSTM(units = 100, return_sequences = True))
regressor.add(Dropout(0.2))regressor.add(LSTM(units = 100, return_sequences = True))
regressor.add(Dropout(0.2))regressor.add(LSTM(units = 50))
regressor.add(Dropout(0.2))
regressor.add(Dense(units = 1))regressor.compile(optimizer = 'adam', loss = 'mean_squared_error')#training the LSTM
regressor.fit(X_train, y_train, epochs = 100, batch_size = 300)
For my testing phase, I separated a few thousand samples and fed it into the algorithm in the same way that I fed the training data. Here are some of my data results.
As you can see, the model accurately predicted quite a few of the samples. However, crypto prices are sometimes really unpredictable, and because of that, you can get results like this.
Or like this.
Finally, I saved the model so my overnight training wouldn’t go to waste if I accidentally ran the code again while trying to do more data visualization.
import os
model_json = regressor.to_json()with open('modelsave.json', 'w') as save_json:
save_json.write(model_json)regressor.save_weights("model.h5")print("saved model to disk!")
So. That last sample was pretty bad if you’re building a trading bot. Here are some of my next steps for the project:
- I’m working with an experienced ML developer to create ensemble networks (combining multiple networks) so that I can get better results.
- We are inputting more data points so that the neural network is able to make better predictions.
Well, that was my experience building an LSTM using Keras! From now on, this will be more of a side project, but I’ll be sure to post any updates if something interesting happens. Feel free to check out my Github Repo for this project if you’re interested.