Financial time series prediction under Covid-19 pandemic crisis with Long Short-Term Memory (LSTM) network

Abstract

In this paper, we design and apply the Long Short-Term Memory (LSTM) neural network approach to predict several financial classes’ time series under COVID-19 pandemic crisis period. We use the S&P GSCI commodity indices and their sub-indices and consider the stock market indices for different regions. Based on the daily prices, the results show that the proposed LSTM network can form a robust prediction model to determine the optimal diversification strategies. Our prediction model achieved RMSEs and MAEs too small for the different selected financial assets, showing the predictive power of our LSTM network especially during the COVID-19 health crisis. In addition, our LSTM network outperforms ARIMA-type models for all selected assets.

Introduction

The outbreak of the Coronavirus brought dramatic changes to our lives. As of the mid-June 2021, this pandemic has killed more than 3.7 million people and infected over 176 million. As the virus news moves far beyond China’s borders, the COVID-19 pandemic has significantly impacted the stock markets around the world. Up until two months ago, it was assumed that the COVID-19 outbreak would be a localized problem for China and that any spillover effects to the rest of the world could be easily managed by policy easing by central banks. The supply and demand of goods and services have been largely affected by the COVID-19 pandemic. The supply of goods and services is deteriorating because factories as well as offices are closed and many of them have gone bankrupt. In addition, demand also dropped because consumers are obliged to stay at home, so they stopped spending. In addition, the pandemic crisis has psychological impacts and affects people’s ability to make the right financial and non-financial decisions (Júnior et al. 2020, World Health Organization, 2020). Adequate solutions to such a situation must be found to keep people’s well-being. Among these possible solutions, we can note the prediction and in particular that of the prices of financial assets.

The prediction of asset prices has been widely studied in the financial literature. One of the main objectives is the management of financial portfolios. In modern portfolio theory, optimal asset allocation (Markowitz, 1952, Sharpe, 1964) is based on historical data. Indeed, this theory assumes that future asset prices (or their returns) known with certainty. In reality, future prices are unknown and their estimates are linked to forecast errors which can affect considerably the portfolio investment strategies, particularly in periods of financial and economic crises. It seems interesting to predict a future investment decision by referring to the forecasting data. The prediction of financial asset prices is also used for speculative purposes. Several modeling approaches have been proposed to predict price (or return) of financial securities. These models are essentially based on fundamental or technical analysis. Fundamental analysis focuses on the economic standing of the firm, employees, the board of directors, financial status, firm’s yearly report, balance-sheets, income-reports, terrestrial and climatic circumstances and political data (Tsai and Hsiao, 2010; Ghaznavi et al. 2016). Technical analysis is an analysis strategy for predicting the trend of prices and considers that prices fluctuate in patterns that are determined by investors’ changing tendencies in the direction of various economic, commercial, financial, political, and psychological factors (Rather et al. 2014; Thanh et al. 2018; Zhou et al. 2018).

In the last few decades, Artificial Neural Networks (ANN) has become a powerful analytical tool used to predict prices assets. The application of artificial neural networks in predicting stock prices and trends, has been the subject of several works (Qiu and Song 2016; Moghaddam et al. 2016; García et al. 2018, among others). Recently, recurrent neural networks have been widely used for the analysis of time series with high time dependency, such as stock returns (Yoshihara et al. 2014; Rather et al. 2014; Ye, 2017). There are different types of recurrent neural networks. The LSTM network is the most dynamic and powerful network. The main characteristic of this network is that it contains memory modules. These modules allow to integrate the long term dependence of the sequences. This LSTM network has shown a strong performance in the prediction of financial series. This LSTM network has been applied in many areas. Among others, in the text translation (Datta et al. 2020), large vocabulary speech recognition (Li and Wu, 2015), medicine diagnostic (Gao et al. 2019), traffic control in cities or its environmental impact (Awan et al. 2020). Also for forecasting economics and financial time series (Elliot and Hsu, 2017; Zhuge et al. 2017; Siami-Namini et al. 2018; Minami, 2018; Fischer and Krauss, 2018; Ji et al. 2019; Livieris et al. 2020).

The aim of this paper is to design Long Short-Term Memory (LSTM) neural network to predict financial time series under COVID-19 pandemic crisis period. Our LSTM accommodates to multiple classes of financial assets, including stock market indices from the main zones, all commodity sectors and finally the US bond market. In addition, the network is designed and tested over a long period from 02 January 1998 to 16 September 2020 covering several events in the financial markets (stock market crashes, the great financial crisis, the European sovereign debt crisis, falls in commodity prices, COVID-19). Our paper contributes to the current literatures in three ways: First, we design and implement an LSTM network, allowing the prediction of the prices of multiple classes of financial assets. Second, the study period runs from January 1998 to September 2020, covering several turbulence events including drops in stock and commodities markets and COVID-19 pandemic crisis. To our knowledge, this is the first study that predicts the price of commodities, and stock market indices during the COVID-19 pandemic crisis. Third, we use the S&P GSCI commodity indices and their sub-indices and consider the stock market indices for different regions for a more complete analysis. Generally, results show that our LSTM network has a good ability to predict all considered financial asset prices.

The remainder of this article is as follows. Section II presents some of the main work related to this area. Section III outlines RNN-LSTM neural network. The results of data analysis and empirical results are presented in Section IV. The discussion will take place in section V. Finally, Section VI concludes the paper.

Literature review

Forecasting financial series and especially financial asset prices is a major challenge in the financial industry. Several articles in the financial literature have focused on this topic. In this context, the ability of artificial intelligence methods to predict future movements of financial assets has been the subject of an abundant literature. Using daily closing prices of 367 public companies traded on the Shanghai Stock Exchange, Cao et al. (2005) compare the predictive powers of linear models (Like Fama-French models) and neural networks. Their results show that neural networks are better suited to predicting stock prices traded on emerging stock markets, such as China. Additionally, Chen and Li (2006) compare the performance of five competing models, namely a linear AR model, the LSTAR and ESTAR smooth transition autoregressive model, and two neural networks MLP (Multi-Layer Perception) and JCN (Jump Connection Nets- in forecasting the daily returns of the Shanghai Chinese stock index. Using the daily closing prices of the Shanghai stock index from October 8, 1996 to December 31, 2004, the authors find that neural networks can improve the forecasting quality.

LSTM networks can detect correlation in nonlinear time series (such as financial time series) and produce predictions with high accuracy. Indeed, from the historical data of financial time series, LSTM networks reveal the characteristics of the data and generate predictions of trends or prices. Chen et al. (2015) presented an LSTM network for predicting returns in the Chinese stock market based on intra-day price data from 3049 companies over the period from December 1990 to September 2015. The results reveal the superiority of the LSTM network over the random method and its ability to provide accurate forecasts of stock returns. Elliot and Hsu (2017) compare the LSTM neural networks to linear models (mean, ordinary and generalized) in forecasting the SP500 index price. They find that the LSTM model outperforms the linear models in SP500 price prediction. Li and Tam (2017) find that the integration of investor attitudes and behaviors in LSTM networks improves the quality of stock market forecasts. The empirical findings show that the proposed model based on LSTM neural networks provides an accuracy of 87.86% and surpasses the SVM method by at least 6%. Zhuge et al. (2017) propose to improve the predictive power of LSTM by integrating both emotional data and daily prices as input variables to the network. By comparing its prediction performance with that of recurrent neural networks, the authors find that the proposed model improves the quality of prediction. Shah et al. (2018) compare the characteristics of the LSTM model and the DNN network to predict the closure price of two companies listed on the Indian Stock Exchange (Tech Mahindra and BSE Sensex) and show the LSTM model has greater predictive power. Skehin et al. (2018) use the linear Autoregressive Integrated Moving Average (ARIMA) model and the LSTM network to produce next-day predictions for the closing prices of five U.S. companies (Facebook Inc., Apple Inc., Amazon.com Inc., Netflix Inc., and Alphabet Inc.). In addition, the authors use the wavelet methods to decompose the series into approximation and detail components to better explain behavior over time. The combination of these techniques in a new ensemble model increases the accuracy of the predictions.

Siami-Namini et al. (2018) compare the predictive power of LSTM networks with the ARIMA statistical model and show that the LSTM model has the lowest root-mean-square error and offers the best performance. Fischer and Krauss (2018) use the LSTM networks to forecast the SP500 stock prices and reveal that the LSTM network has a very high predictive power. Minami (2018) tries to predict the future prices of a company (Tsogami Corporation) listed on the Tokyo Stock Exchange based on LSTM network. The author concludes that the LSTM network is a promising method in stock price forecasting.

More recently, Rundo et al. (2019) forecast the EUR/USD exchange rate based on neural network and reveal that LSTM networks are more appropriate for predicting financial time series. Ji et al. (2019) propose an ARIMA-CNN-LSTM model to predict the future carbon contracts prices and find that the proposed model provides better accuracy in predicting the price when compared to the ARIMA, CNN and LSTM models taken separately. He et al. (2019) propose to build a model that integrates the characteristics of LSTM, CNN and attention mechanisms to predict the price of gold. They show that the proposed model reduces the root-mean-square error (RMSE), the mean absolute error (MAE) and the mean absolute percentage error (MAPE). The authors conclude that the LSTM-CNN with attention mechanism is the appropriate model to predict the price of gold.

Nikou et al. (2019) compare the predictive power of LSTM networks with ANN, SVR (Support Vector Regression), and RF (random forests) models. They find that LSTM performs better in predicting the closing prices of iShares MSCI United Kingdom. Lakshminarayanan and McCra (2019) use an LSTM network and present a prediction of the Dow Jones Industrial Average (DJIA) price. In their LSTM network, they combine the DJIA price with other external parameters such as crude oil and gold prices. The authors find that this combination improves the prediction quality. Qi et al. (2020) predict the future price movements of Forex based on training data, features selected from historical data and technical analysis indicators utilizing advanced Artificial Intelligence techniques. The results show a great potential of this combined approach to predict the price of the forex exchange rate and to develop a successful trading strategy. Jin et al. (2020) propose the deep learning-based stock market prediction model considering investors’ emotional tendency. They adopt the revised version of the LSTM to focus more critical information. Experiment results show that the revised LSTM model improves prediction accuracy. Therefore, investors’ emotional tendency is effective to improve the predicted results. Muncharaz (2020) compares the predictive ability of the LSTM network with that of classical time series models (Exponential Smooth Time Series and ARIMA). He finds that LSTM significantly reduces the prediction errors. More recently, Livieris et al. (2020) develop a model that exploits the advantages of convolutional neural networks (CNN) and long and short memory networks (LSTM) to predict the price of gold. The authors find that the CNN-LSTM model has the lowest root-mean-square (mean) error and therefore the highest accuracy in terms of prediction of the precious metal price.

Widiputra et al. (2021) propose a multivariate CNN-LSTM network for the prediction of stock indices from four Asian stock markets (Shanghai, Japan, Singapore and Indonesia) during the COVID-19 pandemic. In contrast to CNN and LSTM, experimental results show that multivariate CNN-LSTM has the highest statistical accuracy and reliability (lowest RMSE value). These results confirm the use of multivariate CNN-LSTM to forecast the prices of various stock market indices.

Sako et al. (2022) compared the performance of three recurrent neural network models, the simple recurrent neural network, the short-term long memory and the closed recurrent unit on eight stock market indices (NYSE), NASDAQ, JSE, NSE, Euronext, FRA, SSE and JPX) and six exchange rates (ZAR/USD, NGN/USD, GBP/USD, EUR/USD, JPY/USD and RBM/USD) over the period from January 2, 2008 to May 28, 2021. Based on RMSE and MAE, the authors concluded that the GRU model is the best overall model, particularly for univariate forecasting of exchange rates and stock market indices.

Zaheer et al. (2023) propose an RNN network and compare its performance with the CNN and LSTM networks in predicting the closing price and next-day high price of the Shanghai Composite Index. The proposed network is based on six input features, namely volume, adjusted closing price, closing price, low price, opening price and high price. The result of this generated study shows that the suggested single-layer RNN model beats all other models. The experimental results validate the effectiveness of the proposed model, which will help investors increase their profits by making the right decisions.

RNN-LSTM neural network

Architecture

A recurrent neural network (RNN) is a network of artificial neurons with recurrent connections. It consists of interconnected units (neurons) that interact in a non-linear way and for which there is at least one cycle in the structure. The units are connected by arcs (synapses) that have weights. The output of a neuron is a non-linear combination of its inputs. Recurrent neural networks adapt to input data of various sizes. They are particularly suitable for time series analysis. This network has the recalling characteristic that allows previous inputs to persist in the network and thus influence the outputs, just like the human brain. Network training techniques are the same for classical networks (retro-propagation). The basic RNN model presents some difficulties in its application, particularly with regard to the learning algorithms aimed at searching for optimal connection weights. Other versions of recurrent neural networks have been advanced, notably long and short memory networks. A long and short term memory network (LSTM) is the most widely used recurrent neural network architecture in practice. This network is designed to deal with long-term addiction problems. In this type of network, information is stored over long periods of time and called up when needed. The LSTM network, proposed by Hochreiter and Schmidhuber (1997), is generally used in time series prediction, especially in finance.

The structure of repeated module of LSTM is shown in Fig. 1. The LSTM network regroups four activation functions:

• Cell state: This is the cell that stores information over time.

• Forget gate: This gate decides which information should be kept or deleted. It is a sigmoid (logistic) function that uses and returns a value between zero and one. If the value is close to zero, it means that the information should be forgotten. On the other hand, if the value is close to 1 then it must be memorized for the rest.

• Input gate: The LSTM module receives inputs from other parts of the system. The input gate is a sigmoid (logistic) function that decides which information needs to be updated.

• Output gate: It must decide what the next hidden state will be, which contains information about the previous inputs and is used for prediction. This output is based on the filtered version of the Cell state information.

Fig. 1: Repeating module of LSTM.

LSTMs have a chain-like structure with a repetitive module with a special structure with four neural network layers that interact in a very special way.

Design of an LSTM neural network

Different steps are needed to predict the price series through LSTM networks:

Step 1: Data collection.

Step 2: Data preprocessing: LSTM networks are very sensitive to the order of magnitude of the input data (time series). The data must therefore be resized to a scale of zero to one. This is a normalization of the data.

Step 3: Splitting the data set.

Step 4: Creation and adjustment of the LSTM networks: in this step the input data are transmitted to the LSTM network to design and adjust to our problem. Biases and weights are initially assigned randomly. Our LSTM network is composed of a single input layer (a single input variable), a hidden layer with four LSTM blocks and an output layer that makes a single prediction. The activation function used is the sigmoid function. In addition, the training of our LSTM network is performed over 100 epochs. At each epoch, the chosen loss function is optimized based on the ADAM optimization algorithm.

Step 5: Generation of output values (prediction): the last step consists in generating the predictions.

Performance evaluation metric (loss function)

The performance of the model was measured by standard regression metrics including root mean squared error (RMSE) and mean absolute error (MAE). The RMSE is the root of MSE, its formula is as follows:

$${\rm{RMSE}} = \sqrt {\frac{1}{N}{\sum} {\left( {\hat y_i - y_i} \right)^2} }$$

The MAE is the average of absolute error. It measures errors between paired observations. Its formula is as follows:

$${\rm{MAE}} = \frac{1}{N}{\sum} {\left| {\hat y_i - y_i} \right|}$$

where N is the number of observations, y_i is the target value (actual value) and $\hat y_i$ is the output of networks.

Empirical prediction of financial time series

Data description

We extracted historical daily financial time series from 02 January 1998 to 16 September 2020 from Datastream databases. The daily data included equity market indices, U.S. bond market indices and commodity indices. We use the MSCI indices to represent the equity market. We assume that a diversified equity portfolio is represented by the MSCI World, the European equity market is represented by the MSCI Europe, the US equity market by the MSCI US, the Asia-Pacific market by the MSCI Pacific, the MENA market by the MSCI EAFE and the emerging markets by the MSCI EM. We refer to the S&P US 5-10Y and US Benchmark 10Y Bond Indices to specify the US bond market. We consider the S&P GSCI (Total Return) family of commodity indices. Our study will be carried out on 34 commodities indices.

Table 1 presents a summary statistics of indices. Average index values range from 30,2855 (the average value of the S&P GSCI COCOA index) to 4098.407 (the average value of the S&P GSCI Commodity index). The S&P GSCI Livestock index is the raw materials index with the highest average value (2629.093), followed by the Industrial Metals (1155.503) and Precious Metals (1166.367) indices, while the average levels of the S&P GSCI Grain and Soft indices are the lowest (83.16776 and 422.456 respectively).

Table 1 Descriptive statistics.

As far as stock market indices are concerned, the MSCI Pacific index had the highest average value, while the MSCI Emerging Markets index had the lowest. What’s more, both bond market indices are at low levels compared to the stock market indices, and almost all commodity indices (with the exception of the S&P GSCI Grain).

Investing in the S&P GSCI COCOA index or in government bonds seems to be the least risky strategy. The prices of these indices have low volatility. The S&P GSCI Energy sub-indices have high standard deviations, followed by the precious metals and industrial indices. In addition, commodity indices vary considerably from one product category to another. This confirms the findings of Erb and Harvey (2006) and Kat and Omen (2007) that commodities are a heterogeneous asset class. This leads us to note that stock market indices are framed by riskier and less risky commodity indices.

Table 1 also shows that all indices have a positive excess kurtosis. This means that the price distribution of the indices is leptokurtic, with wider tails and a higher probability of extreme events. In addition, all indices are negatively skewed. These statistics reject the hypothesis of normally distributed returns for all asset classes.

The closing price at the end of each trading day is the subject of our prediction exercise. Data is normalized between 0 and 1 using min-max normalization. Then, for each index, we split the data into two parts: training and testing. The first 80% duration of each dataset is allocated for training and the rest 20% duration allocated for testing. The training period is characterized by several financial crises (stock market crash 1998, 2001, 2012, subprime crises of 2008, European Sovereign Debt Crisis 2011). This will allow our network to memorize the behavior of the markets during the different phases (calm and crisis) and to generate predictions with great precision. The test phase is marked by the COVID-19 pandemic. We will then test the predictive power of our LSTM especially during the health crisis.

Empirical results

Our results are reported in two stages. We first present the price prediction results of the indices of the main markets (stock, bond and commodities). Next, we look at the prediction of individual indices for different zones and commodities.

For each index, we compute the two selected performance evaluation measures (RMSE and MAE) in both the training set and the test set. In the training set, the network parameters and weights are adjusted to ensure consistency between the network outputs and the actual data. The curves of the outputs and actual values are normally confused and the values of RMSE and MAE are correspondingly low. However, in the test set, the values are extracted from a network whose parameters and weights are already optimized and no further adjustments are made in this set. Therefore, the performance of an LSTM network must be evaluated on the test set. For this reason, we focus our analysis on the RMSE and MAE values on the test set.

Table 2 exhibits the performance evaluation metrics of LSTM output.

Table 2 Evaluation of LSTM model prediction of global indices.

Table 2 results show that the RMSE and MAE averages using normalized data of the LSTM network for training set are 19.72 and 13.84 respectively. For test set, the RMSE and MAE averages are 18.19 and 13.60, respectively. This result reflects the strong performance of our LSTM in predicting the selected indices. To ensure the quality of prediction of our LSTM network, we propose to display the prediction figures.

Figure 2 reports the original time series data (blue). Figure 3 plots the learning (orange) and test (green) curves against the original data (blue).

Fig. 2: Original time series data of global indices.

A graphical representation of the original data for the MSCI world, US BENCHMARK 10 Y, S&PGSCI Commodity, S&PGSCI Energy, S&PGSCI Industrial Metals, S&PGSCI Precious Metal, S&PGSCI Softs, S&PGSCI Grains, and S&PGSCI Livestock indices.

Fig. 3: Learning and test curves vs original data of global indices.

A graphical representation of the original time series data (blue), the predictions on the training set (orange) and on the test set (green) of global indices.

Figure 3 exhibits that the predicted values of the LSTM network are close to the actual values, and that the price trend is also very consistent. In addition, the green part of Fig. 3, MAE’s and RMSE’s test set show that the bond index prediction has the lowest prediction error. On the other hand, the prediction on commodity indices is slightly less accurate. The prediction of the stock index has the highest RMSE and MAE. It therefore seems important to deepen the analysis and study the prediction errors for the different assets and regions considered (Fig. 4).

Fig. 4: Original time series data of stock market and US government bond indices.

A graphical representation of the original data for the MSCI WORLD, MSCI EM, MSCI EAFE, MSCI EUROPE, MSCI PACIFIC, MSCI USA, US BENCHMARK 10 Y, and SP US 5-10Y indices.

Tables 3, 4, 5, and 6 report the loss functions in the prediction of each selected index.

Table 3 Evaluation of LSTM model prediction of stock market and US government bond indices.

Table 4 Evaluation of LSTM model prediction of Energy sector indices.

Table 5 Evaluation of LSTM model prediction of Industrial and Precious Metals indices.

Table 6 Evaluation of LSTM model prediction of Agriculture indices.

As shown in Table 3 and Fig. 5, the prediction of the US market index has the highest forecast error. Indeed, the MAE and RMSE averages in the test set are 33.84 and 43.90 respectively. This relatively large error can be explained by the rapid and the speculative rise of the MSCI USA Index during the COVID-19 pandemic. Indeed, the average price of the MSCI USA index doubled from $1217 in the period 1998-2014 to $2422 in the period 1998–2015. So our model has certainly managed to anticipate the uptrend of the US stock market, but it has lost in precision due to the sharp increase in prices. Moreover, the forecast of the stock market indices for the other considered regions has a high precision (RMSE and MAE averages in the test set are 17.46 and 12.73, respectively). For these indices, our LSTM network has thus succeeded in reproducing the long-term dependencies of the sequences and in reducing the quadratic error. It has generated the same trends and accurate daily rates. As a result, our LSTM has a strong forecasting capability for MSCI EM, MSCI EAFE, MSCI EUROPE and MSCI PACIFIC even during the COVID-19 pandemic outbreak.

Fig. 5: Learning and test curves vs original data of stock market and US government bond indices.

A graphical representation of original time series data (blue), predictions on the training set (orange) and on the test set (green) of stock market and US government bond indices.

On the other hand, our model has a very high accuracy in predicting the daily prices of US government bond indices for different maturities during the crisis periods. This result can be attributed to the low variability of sovereign bond prices.

Table 4 presents the quadratic errors and absolute errors resulting from the price forecast of the sub-indexes of the energy sector. The forecast errors are generally low and reflect the strong capacity of our LSTM network to predict energy sub-indexes. The RMSE and MAE averages in the test set are 23.29 and 19.79, respectively. The widest gap between the true value and the output of the LSTM network is recorded in the SPGSCI Unleaded Gasoline index and the narrowest is in the SPGSCI Biofuel index (Fig. 6).

Fig. 6: Original time series data of Energy sector indices.

A graphical representation of the original data for the S&PGSCI Energy, S&PGSCI Brent Crude, S&PGSCI Crude Oil, S&PGSCI Gas Oil, S&PGSCI Heating Oil, S&PGSCI Natural Gas, S&PGSCI Unl. Gaso, S&PGSCI Petroleum, and S&PGSCI Biofuel indices.

Almost all price graphs (Fig. 7) of the energy sub-indices show the same pattern during the training of the network: an increase in prices followed by a decrease (exception for Unleaded Gasoline and Biofuel that is marked by two phases of increase). Besides, all the energy sectors recorded a fall in prices during our test set period. During this period, our LSTM is able to anticipate this downward trend and presents a prediction of the prices with a high accuracy. To sum up, we can confirm that our LSTM can reconstitute the dependency structure of the sequences in the case of the energy sub-indices and provide a good prediction of the daily prices especially during the health crisis.

Fig. 7: Learning and test curves vs original data of Energy sector indices.

A graphical representation of original time series data (blue), predictions on the training set (orange) and on the test set (green) of Energy sector indices.

Table 5 shows that both the forecast errors of the daily prices of the sub-indexes of industrial metals and precious metals are small. This shows the good quality of the prediction generated by our LSTM network (Fig. 8).

Fig. 8: Original time series data of Industrial and Precious Metals indices.

A graphical representation of the original data for the S&PGSCI Industrial Metals, S&PGSCI Aluminum, S&PGSCI Copper, S&PGSCI Lead, S&PGSCI Nickel, S&PGSCI Zinc, S&PGSCI Precious Metal, S&PGSCI Gold, and S&PGSCI Platinum indices.

During the training set, industrial and precious metal prices initially showed strong growth followed by a rapid decline (Fig. 9). During the test set, industrial index prices show a decrease followed by an increase and finally another decrease. Our network replicates the same sequence of trends and shows the high accuracy in daily price prediction (all RMSE and MAE are low). As a safe haven, the price of gold recorded during the period 2015–2020 (characterized by the succession of crises) a strong increase. The average price of gold rose from $430 in the period 1998–2014 to $703 in the period 2015–2021. Our LSTM network anticipates this strong growth with significant accuracy in the prediction of daily gold prices. In addition, silver and platinum prices fell during the test phase. The outputs of LSTM are identical to the real prices of silver and platinum, showing the strong predictive power of our network.

Fig. 9: Learning and test curves vs original data of Industrial and Precious Metals indices.

A graphical representation of original time series data (blue), predictions on the training set (orange) and on the test set (green) of Industrial and Precious Metals indices.

To summarize, Fig. 9 confirms the great ability of our LSTM network to forecast daily industrial and precious metal prices. Indeed, the graphs of historical data and LSTM outputs are almost identical throughout the study period. Our LSTM network, thus proves its strong performance in forecasting precious and industrial metal prices during the COVID-19 pandemic (Fig. 10).

Fig. 10: Original time series data of Agriculture indices.

A graphical representation of the original data for the S&PGSCI Cocoa, S&PGSCI Coffee, S&PGSCI Cotton, S&PGSCI Sugar, S&PGSCI Corn, S&PGSCI Soybeans, S&PGSCI Wheat (Kansas), S&PGSCI Wheat(CBOT), S&PGSCI Feeder Cattle, S&PGSCI Lean Hogs, S&PGSCI Live Cattle indices.

Table 6 reports the daily price forecast RMSE and MAE for the agricultural sub-indexes. The results are similar to the preceding ones. The LSTM neural network reveals a good ability to predict agricultural sub-indexes. The RMSEs and MAE of these indices are the smallest.

Figure 11 depicts the graph of the LSTM network output and historical data for the agricultural sub-indexes. As shown in Fig. 11, actual and LSTM predicted values are very close.

Fig. 11: Learning and test curves vs original data of Agriculture indices.

A graphical representation of original time series data (blue), predictions on the training set (orange) and on the test set (green) of Agriculture indices.

With the exception of cocoa, all agricultural products experienced a price decline during the Corona crisis. This is mainly explained by the suspension of import and export operations during the containment period. This fall in agricultural commodity prices (as well as the increase in the price of cocoa) was well predicted by our network and the forecast of daily prices is of high quality. This demonstrates the strong predictive power of our network in the agricultural sector even during the COVID-19 pandemic crisis.

In summary, it can be seen that the LSTM neutral network has a strong predictive capability both on the training and test set. It proves that our proposed LSTM method has a good predictive power. Obtaining predicted values very close to the actual observed values reveals a very high predictive power of our network on all considered assets and markets. Besides, our LSTM network has demonstrated its ability to produce good forecasts during the periods of crisis, particularly during the current health crisis (COVID-19).

To better ascertain the predictive power of our LSTM network, we propose a comparison of the results obtained from our network with those obtained from an ARIMA-type model.

We have tested several ARIMA specifications for each asset in our sample. The model selected is the one that minimizes the Akaike info criterion. Model parameters are then estimated using maximum likelihood. Once the models have been specified and the parameters estimated for each asset, we calculate the measures of prediction quality, namely RMSE and MAE. Both values are calculated during both the training and test periods. These two periods are defined in the same way as for the LSTM network.

Looking at Table 7 and Tables 1–6, we can conclude that for almost all assets (with the exception of natural gas), the valuation measures for the two periods (training and test) of our LSTM network are less important than those calculated in the ARIMA models. In fact, the RMSE and MAE values are lower for our LSTM network. This result testifies to the strong predictive capacity of our model compared with ARIMA-type models. Long- and short-term memory neural networks are therefore able to minimize prediction errors to a greater extent, and as a result the predictive values derived from the LSTM network are closer to reality than those derived from ARIMA-type models.

Table 7 Evaluation of ARIMA model prediction of indices.

Predicting the price of financial assets is a very important process that can be beneficial to investors. Indeed, choosing the optimal investment strategies has attracted the interest of many investors around the world. However, due to the dynamism that characterizes our world, decision making is a difficult and complex task. In order to make successful choices,

investors seek to predict the future state of the financial market. A good prediction system is a good decision support tool that can provide investors with additional information (such as the future direction of asset prices) and consequently make more accurate and profitable investments. In addition, during periods of crisis, the prices of financial assets become very volatile and difficult to predict. Therefore, presenting an LSTM network that can provide financial asset prices with a high degree of accuracy, during crisis periods, is of great interest to investors and is a good decision support tool. In fact, based on our LSTM outputs, investors can build their optimal investment strategies that reduce their risk exposure. In other words, the forecast prices of financial assets make it possible to identify the assets that investors should include in their diversified portfolios or in hedging strategies during periods of crisis, particularly the COVID-19 pandemic crisis.

Discussion

The aim of our study is to use our historical database to predict the future prices of stock, bond and commodity indices. Forecasting financial asset prices is a very important subject for all players in the financial sphere. In fact, a good prediction enables you to make the right decisions when choosing the best financial investments. This can lead to substantial gains. In other words, a model capable of efficiently predicting financial time series will enable us to find optimal investment strategies on the financial markets that maximize investors’ gains.

To this end, we apply long- and short-term memory recurrent neural networks (LSTM) to equity market indices, U.S. bond market indices and commodity indices. We use the MSCI indices to represent the equity market, the S&P US 5-10Y and US Benchmark 10Y Bond Indices to specify the US bond market and S&P GSCI (Total Return) family of commodity indices over the period 02 January 1998 to 16 September 2020. The use of this LSTM network has enabled us to generate future predictions for stock, bond and commodity indices during several phases, including crisis periods characterized by extremely volatile and unpredictable returns (the Covid-19 pandemic). We consider this to be the novelty of our paper. Indeed, in our opinion, predicting future index prices for the various financial asset classes and producing forecasts that are very close to reality even during stressful periods characterized by high price volatility is a considerable contribution and a revelation of our study. A good forecast of stock, bond and commodity indices will help define an investor’s optimal strategy. This investment decision will have a major impact on the investor’s wealth. In fact, if we can “beat the market” and correctly forecast future prices (or even trends), we’ll have a considerable lead over other market players.

The recurrent neural networks we have adopted, namely the LSTM network, have shown very high predictive power on our series. This strong ability to predict real observations leads us to use this network to extract predictive values for financial index prices. On the basis of this data, we can choose the strategy that maximizes diversification gains.

Conclusion

This paper focuses on the prediction of the daily prices of several financial asset using the LSTM neural network. Based on a dataset consisting of daily prices of stock market indices of several regions, US governmental bond indices, and commodity indices, the results show that our LSTM network can produce good forecasts for all the financial asset prices in several types of markets. The LSTM network allows us to capture the characteristics of stock markets in different areas, all commodity markets, and U.S. government bonds. It also detects temporal interdependencies between observations in the same series. We can then rely on the LSTM network to generate future financial series prices and then make the best decisions for the optimal investment strategy choice through the financial markets. Our results reveal great potential for the LSTM approach to develop a successful optimal investment strategy.

The main contributions of this paper are (1) we design and implement an LSTM network allowing the prediction of the multiple classes of financial asset prices, (2) we predict the price of commodities, and stock market indices during the COVID-19 pandemic crisis, (3) we use the S&P GSCI commodity indices and their sub-indices and consider the stock market indices for different regions for a more complete analysis, (4) our LSTM network has a good ability to predict all considered financial asset prices. The results show lower loss function values (both for RMSE and MAE) for our LSTM network than for ARIMA-type models.

To conclude, we have addressed one of the main concerns of the various players on the financial markets (investors, intermediaries, financial companies, rating agencies, researchers, etc.): forecasting the prices of financial assets. In fact, forecasting is a complex, if not impossible, task if the hypothesis of market efficiency is validated. It involves making future price projections (or trends) based on historical data. Accurate forecasts help decision-makers to plan for the future. In particular, a good prediction will enable a good allocation of wealth on the financial markets. These markets are considered both profitable and risky. Designing a model (based on LSTM networks in our thesis) that has the ability to predict the prices (or trends) of financial assets, belonging to several types of financial markets, with a certain degree of accuracy therefore enables investors to derive significant benefits. The empirical results of this study could have important and significant implications for the various players in the financial sphere and researchers, particularly in times of crisis (Covid-19 pandemic or any other systemic risk event) when financial markets are characterized by downturns and high volatility. Indeed, on the basis of these empirical results, we have shown that an LSTM network enables investors to predict the future prices of different asset classes with a low margin of error, even during periods of severe turbulence. We suggest that investors and portfolio managers develop hedging strategies that take into account the benefits of diversification based on predictions from our LSTM network. Our results could also encourage researchers to use neural networks to predict prices in other asset classes (cryptocurrencies, DeFI, NFT), as well as in other geographical areas and country groups.

Nevertheless, as with any research work, this study suffers from a number of weaknesses that can be improved upon in future studies, and can be seen as avenues for extension. The first is to develop an artificial neural network, combining the advantages of recurrent and convolutional networks, which can predict the prices of different financial assets with greater accuracy and over a longer investment horizon. A second avenue of extension lies in the prediction of digital asset prices (García-Medina and Aguayo-Moreno, 2023; García-Medina and Luu Duc Huynh, 2021). A third approach involves predicting the volatility of financial assets by combining recurrent neural networks and Conditionally Heteroskedastic Autoregressive models (García-Medina and Aguayo-Moreno, 2023).