STOCK PRICE PREDICTION USING GRID HYPER PARAMETER TUNING IN GATED RECURRENT UNIT

 

Bhavsar Shachi ¹, Ravi Gor ²

 

¹ Research Scholar, Department of Mathematics, Gujarat University, Ahmedabad-380009, India

² Department of Mathematics, Gujarat University, Ahmedabad-380009, India

 

1. INTRODUCTION

Stock price prediction is one of the matters confronting investors in everyday financial market. Researchers are working to study the market behaviour and pattern. As understanding market will help investors to obtain higher trading benefits, to know trend and avoid losses. Sometimes the decision process of trend prediction may get biased due to aspects such as trader’s psychology, investors demand of high returns, social media statements, political scenario, others opinion, influencers, etc. The trends might be predicted false or wrong if decisions were made in panic situations. Machine learning can be the best option to avoid being emotionally invested. Deep learning can provide promising results while working with highly volatile data.

Recurrent Neural Network (RNN) is one of the models of deep learning used for both classification and regression. The Gated Recurrent Unit (GRU) is an upgraded model of RNN which uses the back propagation algorithm to find optimal weights for model. RNN has mainly two sub models GRU and Long Short-Term Memory (LSTM).  Here, GRU model was selected as it contains two gates as compared to three gates in LSTM which decreases the computational cost.

One of the results of efficient market hypothesis says that stock prices can be predicted by analysing historical price movements Fama (1970) Gao et al. (2021) Hence, historical stock data of previous years is used for the prediction. Due to the pandemic situation use of social media increased. People tends to move towards social media more not only to entertain but to share their opinions and ideas to world. Increasing use of social media accounts leads to study the pattern and behaviour, as investing in social media companies would be profitable. So, stock prices of the social media companies were analysed and predicted, to understand their pattern.

 

2. LITERATURE REVIEW

Shena et al. (2018) used hybrid of Gated recurrent unit and support vector machines to overcome gradients explosion and vanishing gradient problem for financial signal forecasting. Three stock indexes S&P 500 Index, Heng Seng Index and Deutscher Aktien index of previous 27 years were used for forecasting. The hybrid approach of model consist of traditional layers with Gated recurrent unit cell and output layer was changed with support vector machine model.  The obtained results were compared with traditional machine learning models such as support vector machines and deep neural network. The comparison of results concluded that hybrid model outperformed another traditional model. Shena et al. (2018)

Raut and Sethia (2018) predicted (t+5) th day stock prices and provided buying and selling strategies. For model 50 different indicators were calculated from which independent component analysis was applied for dimension reduction. Performance of Support Vector Machine, Artificial neural network, Long Short-Term Memory, and Gated Recurrent Unit were compared to find optimal model where Long Short Term Memory model provided highest returns. Raut and Sethia (2018)

Das et al. (2019) predicted stock prices of four stock indices by means of evolutionary algorithm and Machine learning algorithm. Firefly algorithm, Genetic algorithm, Principal component analysis and Factor analysis were used as a feature reduction technique. Reduced features were applied on Extreme learning machine, online sequential extreme learning machine and Recurrent backpropagation neural network and the results were compared to check optimised model. The results shows that combination of firefly algorithm for feature reduction and online sequential extreme learning machine for prediction gives optimized results. Das et al. (2019)

Selvamuthu et al. (2019) used traditional machine learning models i.e., Support vector machines and Neural networks for forecasting stock price. In Neural networks three models Scaled conjugate gradient, Levenberg-Marquardt and Bayesian regularization were applied on 15 min dataset of India company and tick dataset. Every model provides similar accuracy but first two models take few hours to forecast, and Bayesian regularization model takes few days to forecast providing best accuracy from all the models used. Selvamuthu et al. (2019)

Polamuri and Srinivas (2019) used supervised learning technique Random Forest for stock price forecasting. Extremes randomize tree regressor is used in Random Forest model, to improve forecasting results.  This extreme randomize tree offers reduction in variance. Linear regression, Multivariate regression, Decision tree and Support vector machine algorithms were used for forecasting where Random Forest algorithm with extreme randomize regressor give best result. Polamuri and Srinivas (2019)

Pengfei and Yan (2019) used the combined PSR and Long Short-Term Memory model to predict prices for six stock indices. ARIMA, Support Vector Regressor, Multi-layer Perceptron, Long Short-Term Memory were used to compared with the proposed combined model. The compared result shows that, combined LSTM and PSR model gives better predictions as compared to others for stock indices. Pengfei and Yan (2019).

Assaf and Kolasani (2020) predicted stock movements with the help of sentiment analysis. Different traditional machine learning models were applied on different sentiments for prediction in which Support vector machine performed well. So, Support Vector Machine was selected for stock sentiment analysis based on tweets made with word stock. The sentiment values of previous day were used for next day’s prediction. Boosted regression tree and multi-layer perceptron Neural Network were compared to train sentiment values for stock index value estimation. Results showed that from both model multi-layer perceptron Neural Network performed better. Assaf and Kolasani (2020)

Shahi et al. (2020) compared Long Short-Term Memory and Gated Recurrent Unit for stock price prediction. Information such as last traded price, news headline and body were used with stock price data. Experiments were carried out with news sentiments and without news sentiments where the news sentiment model performed better. Both models with the financial news data performed equally well where Gated Recurrent Unit performed slightly better compared to Long Short-Term Memory. Shahi et al. (2020)

Touzani and Douzi (2021) studied the trading strategies of Moroccan stock market. Two models Long Short-Term Memory and Gated Recurrent Unit were used for prediction. Long Short-Term Memory model was used for short term predictions and Gated Recurrent Unit model was used for medium term predictions. Based on prediction, rules were implemented to buy or sell stocks. Transactions were simulated applying these rules to estimate returns for stocks selections. Touzani and Douzi (2021)

Ghosh and Gor (2022) used K-means clustering and Random Forest Regression algorithms for sales prediction. They used clustering methods for ad campaigning analysis. First, ad groups are created using the K-Means clustering algorithm then Random Forest Regressor algorithm is used to optimize sales conversion and predict future sales. Impressions, clicks, and spent are used as independent variables to predict total number of people that asked about the product after viewing the ad on Facebook. They also calculated Mean Absolute Error and Root Mean Square Error. The integration of two algorithms K-means clustering and Random Forest regressor gives permissive result with 75% accuracy. Ghosh and Gor (2022), Bhavsar and Gor (2022)

Srivinay et al. (2022) predicted stock prices using Prediction rule ensemble technique and Deep neural network. Initially moving average technical indicators were used with 20, 50, 200 days average. Then with the help of prediction rule ensemble technique various rules were applied for stock price prediction. Also, Hyper parameter such as layers, learning rate, epochs and neurons were hyper tuned. At the end the results of both methods were combined by average which shows that the combined results show better performance than Artificial neural network and Deep neural network model. Srivinay et al. (2022)

 

3. DATA DESCRIPTION AND DATA PRE-PROCESSING

Data Collection: Here, stock price data for five different social media companies Google (2344 days), Facebook (2344 days), Snapchat (1295 days), Pinterest (601 days) and Twitter (2127 days) was obtained namely dataset1 to 5 respectively. The dataset includes high, open, low, close, adjusted close price and volume of the stocks. (fin) (wsj)

Data cleaning: There was no null/Nan values in dataset. Also, no outliers were removed form data. The column with adjusted close price was removed as it was highly corelated with close price. (Bhavsar & Gor, Predicting Restaurant Ratings using Back Propagation Algorithm, 2022)

Train and test data: 70% / 30% of ratio was used for training and testing purpose respectively. (Bhavsar & Gor, Predicting Restaurant Ratings using Back Propagation Algorithm, 2022)       

Figure 1

                                                                             

Figure 1 Dataset description with all parameters (fin) (wsj)

 

4. MODEL DESCRIPTION OF BIDIRECTIONAL GRU WITH DENSE LAYERS Cho et al. (2014) Rajpurohit et al. (2021)

Gated Recurrent Unit (GRU) can be considered as an upgraded or improvised version of standard Recurrent Neural Network. It was introduced by Cho, et al. in 2014 and known as variation of LSTM. GRU’s main aim is to solve vanishing gradient problem which arises in typical Recurrent neural networks. Hidden layer of GRU contains mainly two gates Update gate and Reset gate that helps in how much or which information to be passed in future and which past information to be forget shown in figure (A). Rajpurohit et al. (2021)

Figure 2

                                                                            

Figure 2 GRU structure for update and reset gate Rajpurohit et al. (2021)

 

For time step t, update gate can be termed as  and defined as   Similarly reset gate can be termed as  and defined as . Correspondingly, hidden state computed as . And the final output computed as . Where,  is an input of current state,  is an output of previous hidden state and ,,,,, are weights. this notation denotes the pairwise multiplication or a Hadamard product.  Cho et al. (2014), Rajpurohit et al. (2021), Bhavsar and Gor (2022)

Adaptive moment estimation – Adam optimizer  Kingma and Ba (2015), Dozat (2015)

As GRU is a neural network, it also forward and backward propagate in similar manner to Artificial Neural Network.  Here Adaptive moment estimation optimizer was used for backpropagation. Following is the equation for Adam optimizer:

 where  and  Where V is exponential moving average and S squared moving average of gradients, initially taken as 0. Default values taken as ∝=0.001, β_1=0.9, β_2=0.999 and e=10^ (-8). Kingma and Ba (2015), Dozat (2015)

Figure 3

                                                                          

Figure 3 Proposed Neural network and Backpropagation Algorithm

 

5. PROPOSED METHODOLOGY

The model is divided into mainly three parts

·        Data preparation which includes pre-processing, scaling, and cleaning.

·        Hyper-parameter tuning with the help of Grid Search algorithm on training dataset.

·        Stock price prediction with the help of Gated Recurrent Unit.

The GRU model with following configuration was developed. Same as the number of parameters i.e., 5 neurons are taken in input and 1 neuron in output layer.

·        For hidden layer, ‘tanh’ activation function was taken.

·        Adam optimizer was used for backpropagation. (The optimizer was considered in model because it has ability to update learning rate as well as gradient at every step.)

·        Proposed Backpropagation Algorithm is shown in Figure 3

·        Truncated normal was taken as kernel initializer.

·        40 epochs were considered for model with early stopping criteria.

·        As the model was regression type, output layer contains linear activation function.

·        Optimized values for number of neurons, layers, dropout value, etc were obtained by hyper parameter tuning.

Hyper parameters help to control the process of learning for given model. Hyper parameter tuning provides optimal values of hyper parameters for the proposed model. Here, hyper parameters such as number of hidden layers, number of neurons in each layer, batch size and dropout are tuned with the help of Grid search algorithm. Where the optimal values obtained for number of hidden layers was 3, number of neurons was 64, batch size was 64 and dropout 0.2. Once the model gets tuned on training dataset, optimized valued were used in testing dataset to study the results. Model is developed in Python language.

Figure 4

                                                                         

Figure 4 Hyper parameter tuning of hidden layers, neurons, batch size and drop out

 

Figure 5

                                                                           

Figure 5 Summary of purposed model

 

6. RESULT AND DISCUSSION

The hyper parameter tuning algorithm was applied on the google stock price dataset. The obtained values of hyperparameters were applied on other four stock price dataset, to check efficiency of the proposed hyper parameter tuned GRU model.

The accuracy of the model was calculated by r2 score and loss was measured by Root mean square error (RMSE). Results showed that the model performed well with every dataset. As shown in the table dataset 2 and 3 performed bit less as compared to the other dataset, though the results were good enough.  The number of days was taken less in dataset 2 and 3, as compared to other datasets. Due to the less data fed to the model, performance of the model was decreased.

  Table 1

Table 1 Predicted stock price of test dataset
	RMSE	r2 score
Dataset 1	0.0015	0.9543
Dataset 2	0.0276	0.9488
Dataset 3	0.055	0.8996
Dataset 4	0.0466	0.8104
Dataset 5	0.0237	0.9812

 

Figure 6

                                                                               

Figure 6 Predicted & actual price of Dataset 1

  

Figure 7        

                                                                     

Figure 7 Predicted & actual price of Dataset 2

 

Figure 8

                                                                       

Figure 8 Predicted & actual price of Dataset 3

 

Figure 9

                                                                        

Figure 9 Predicted & actual price of Dataset 4

 

Figure 10

                                                                         

Figure 10 Predicted & actual price of Dataset 5

 

7. CONCLUSION

Here, stock price prediction for social media companies was carried out with the help of Deep Learning GRU model. Grid search algorithm was used to tune the hyper parameters for the GRU model. The benefit of hyper parameter tuning is to avoid the manual trial and error method. Grid search will suggest optimized hyper parameter values for model. The reason for selecting GRU was due to its faster computational efficiency. The model was developed to study the market behaviour on social media companies. The results showed that companies with a smaller amount of data performed little less as compared to the companies with large amount of data. GRU model overall achieved good performance, which shows the ability of model to perform well even with less amount of dataset. Future work can be carried out using different evolutionary algorithms for hyper parameter tuning.

 

CONFLICT OF INTERESTS

None. 

 

ACKNOWLEDGMENTS

None.

 

REFERENCES

Assaf, R. & Kolasani, S. (2020). Predicting Stock Movement Using Sentiment Analysis of Twitter Feed with Neural Networks. Journal of Data Analysis and Information Processing, 8, 309-319. https://doi.org/10.4236/jdaip.2020.84018

Baheti, R. Shirkande, G. Bodake, S. Deokar, J. & Archana, K. (2021). Stock Market Analysis from Social Media and News using Machine Learning Techniques. International, Journal on Data Science and Machine Learning with Applications, 1(1), 59-67.

Bhavsar, S. & Gor, R. (2022). Comparison of Back propagation algorithms : Bidirectional GRU and Genetic Deep Neural Network for Churn Customer (In press). International Organization of Scientific Research Journal of Computer Engineering (IOSR-JCE).

Bhavsar, S. & Gor, R. (2022). Predicting Restaurant Ratings using Back Propagation Algorithm. International Organization of Scientific Research journal of Applied Mathematics (IOSR-JM), 18(2), 5-9.

Cho, K. Merrienboer, B. V. Bahdanau, D. & Bengio, Y. (2014). On the Properties of Neural Machine Translation : Encoder-Decoder Approaches.

Das, S. Mishraa, D. & Rout, M. (2019). Stock market prediction using Firefly algorithm with evolutionary framework optimized feature reduction for OSELM method. Expert Systems with Applications, 1-24. https://doi.org/10.1016/j.eswax.2019.100016

Dozat, T. (2015). Incorporating Nesterov Momentum into Adam. 

Fama, E. F. (1970). Efficient capital markets : a review of theory and empirical work. 5e Journal of Finance, 25(2), 383-417. https://doi.org/10.2307/2325486

Gao, Y. Wang, R. & Zhou, E. (2021). Stock Prediction Based on Optimized LSTM and GRU Models. Hindawi Scientific Programming, 8. https://doi.org/10.1155/2021/4055281

Ghosh, M. & Gor, R. (2022). Ad-Campaign Analysis and Sales prediction using K-means Clustering and Random Forest Regressor. International Organization of Scientific Research Journal of Applied Mathematics, 18(2), 10-14.

Guan, C. Liu, W. & Cheng, J. C. (2021). Using Social Media to Predict the Stock Market Crash and Rebound amid the Pandemic : The Digital 'Haves' and 'Have mores'. Annals of Data Science, 9(1), 5-31. https://doi.org/10.1007/s40745-021-00353-w

Kingma, D. & Ba, J. (2015). Adam :  A Method for Stochastic Optimization. 3rd International Conference for Learning Representations, San Diego.

Mehta, P. Pandya, S. & Kotecha, K. (2021). Harvesting social media sentiment analysis to enhance stock market prediction using deep learning. PeerJ Computer Science. https://doi.org/10.7717/peerj-cs.476

Pengfei, Y. & Yan, X. (2019). Stock price predictions based on deep neural networks, Neural computing and applications. 1609-1628. https://doi.org/10.1007/s00521-019-04212-x

Polamuri, S. Srinivas, K. (2019). Stock Market Prices Prediction using Random Forest and Extra Tree Regression. International Journal of Recent Technology and Engineering (IJRTE), 8(3), 1224-1228. https://doi.org/10.35940/ijrte.C4314.098319

Rajpurohit, V. Bhavsar, S. & Gor, R. (2021). A comparision of GRU-based ETH price prediction. Proceeding of International Conference on Mathemaitcal Modelling and Simulation in Physical Sciences (MMSPS-2021), 424-431.

Raut, A. & Sethia, P. (2018). Application of LSTM, GRU and ICA for Stock Price Prediction. Proceedings of ICTIS, 2, 1-10. https://doi.org/10.1007/978-981-13-1747-7_46

Selvamuthu, D. Vineet, K. & Mishra, A. (2019). Indian Stock Market prediction using Artificial Neural Networks on tick data. Financial Innovation, 16(5), 1-12. https://doi.org/10.1186/s40854-019-0131-7

Shahi, T. Shrestha, A. Neupan, A. & Guo, W. (2020). Stock Price Forecasting with Deep Learning : A Comparative Study. Mdpi journal, 8, 1-15. https://doi.org/10.3390/math8091441

Shena, G. Tana, Q. Zhanga, H. & Ze, P. (2018). Deep Learning with Gated Recurrent Unit Networks for Financial Sequence Predictions. 8th International Congress of Information and Communication Technology, 895-903. https://doi.org/10.1016/j.procs.2018.04.298

Soni, P. Tewari, Y. & Krishnan, D. (2022). Machine Learning Approaches in Stock Price, Prediction : A Systematic Review. Journal of Physics : Conference Series. https://doi.org/10.1088/1742-6596/2161/1/012065

Srivinay, Manujakshi, B. C. Kabadi, M. G. & Naik, N. (2022). A hybrid stock price prediction model based on PRE and DeepNeural Network. MDPI. https://doi.org/10.3390/data7050051

Touzani, Y. & Douzi, K. (2021). An LSTM and GRU based trading strategy adapted to the Moroccan market, Journal of Big data, 1-16. https://doi.org/10.1186/s40537-021-00512-z