Forecasting stock prices changes using long-short term memory neural network with symbolic genetic programming

This study introduces an augmented Long-Short Term Memory (LSTM) neural network architecture, integrating Symbolic Genetic Programming (SGP), with the objective of forecasting cross-sectional price returns across a comprehensive dataset comprising 4500 listed stocks in the Chinese market over the period from 2014 to 2022. Using the S&P Alpha Pool Dataset for China as basic input, this architecture incorporates data augmentation and feature extraction techniques. The result of this study demonstrates significant improvements in Rank Information coefficient (Rank IC) and IC information ratio (ICIR) by 1128% and 5360% respectively when it is applied to fundamental indicators. For technical indicators, the hybrid model achieves a 206% increase in Rank IC and an impressive surge of 2752% in ICIR. Furthermore, the proposed hybrid SGP-LSTM model outperforms major Chinese stock indexes, generating average annualized excess returns of 31.00%, 24.48%, and 16.38% compared to the CSI 300 index, CSI 500 index, and the average portfolio, respectively. These findings highlight the effectiveness of SGP-LSTM model in improving the accuracy of cross-sectional stock return predictions and provide valuable insights for fund managers, traders, and financial analysts.

Predicting the Stock return is a challenging endeavour, given the nonlinear nature of the stock market and the different approaches to predict the stock change.Though, advancements in artificial intelligence and other superior models have been used to increase forecasting accuracy, the prediction accuracy rate is still an unresolved issues 1 .
Enormous amount of attention in the empirical asset pricing literature has been directed to answer the questions of what drives the stock prices and what input features play major role in generating accurate results.In early years, Fama proposed in a weak-form market, people can make abnormal returns by mastering fundamental information, such as financial statements 2 .However, many scholars doubt the financial ratios do not consistently outperform the historical average benchmark forecast out of sample 3 .In addition other researchers started with the price trend itself, using technical indicators and found that technical indicators were efficient in predicting the market in the past 3 .Thomas Fischer's 2018 study in the US market, utilizing the LSTM model, stands as one of the most referenced papers in cross-sectional stock selection.It unveiled a 51.4% accuracy rate between 1992 and 2015.However, the assessment of alpha between 2010 and 2015 showed a stagnant cumulative alpha of zero, highlighting limitations in this strategy during that period 4 .A subsequent parallel study by Ghosh et al. expanded input variables from one to three features but omitted classification accuracy.Notably, it demonstrated significant alpha improvements with a positive trajectory from 2010 to 2015, which unfortunately turned negative from 2016 to 2019 5 .
The persistence of challenges in maintaining consistent alpha generation seems to stem from limitations in input variables.Hanauer et al. addressed this issue by integrating fundamental indicators into their machine learning (ML) models, resulting in an average risk-adjusted alpha of approximately 6% for European stocks 6 .In contrast, Liu et al. study focusing on China's stock market employed a Deep Neural Network (DNN) incorporating 36 price-related trend features and 5 fundamental factors.Despite initially achieving a validation accuracy of 55.46%, the subsequent inclusion of trend-related features led to a decrease in accuracy, reaching 49.79% and falling short of the established 50% benchmark 7 .
features.This process not only enhances computational efficiency but also contributes to improved predictive capabilities 17 .By 2023, even transformer models with fused multi-source features have been proposed, leveraging the ITD (intrinsic time-scale decomposition) method to manage feature dimensions effectively 18 .
Third supplementary approach is data augmentation.Fisher's attempt of LSTM is single LSTM module, and the attributes of overfitting was challenged by other scholars due to the limited availability of data points.Yujin presented a novel data augmentation approach to avoid the overfitting and propose ModAugNet Framework including two modules, one is overfitting prevention LSTM module, and another is prediction LSTM module.The number of data point has been increased by 252 times 19 .We could also find the Phase Space Reconstruction (PSR) method 13 or feature expansion method 20 for data augmentation.
Finally, the combination of Genetic Algorithm (GA) and Deep Neural Network (DNN) or other Machine Learning models has been utilized by many researchers to improve prediction accuracy.For the application of GA in conjunction with Deep Neural Networks (DNNs), two main applications can be observed: hyperparameter tuning and feature selection.
Hyperparameter tuning is a crucial aspect that needs to be addressed in the optimization process, including parameters setting such as the number of layers, nodes per layer, and number of time lags.GA is frequently employed to search for optimal hyperparameters for DNN.In 2018, Chung and Shin employed GA to identify the optimal number of time lags and LSTM units for hidden layers in stock prediction models 21 .In a similar study in 2019, Chung and Shin optimized the kernel size, kernel window, and pooling window size for CNN 22 .In addition, GA has been used to determine appropriate hyperparameters and input data sizes for Generative Adversarial Networks (GANs) in stock prediction by He and Kita in 2021 23 .These studies demonstrate the effectiveness of GA in optimizing the hyperparameters of various deep learning models for stock prediction.
As for the feature selection, many researchers combine GA and other DNN model to reduce input variables and enhance calculation speed by selecting appropriate factors from a large pool of candidate variables.For instance, Chen and Zhou used GA to rank factor importance and select features for a Long Short-Term Memory (LSTM) model, while Milad employed GA as a heuristic approach for selecting relevant features for an Artificial Neural Network (ANN) 24,25 .Li utilized a multilayer GA to select features and reduce high dimensionality in a stock dividend dataset 26 .Recently, Yun revised GA-based selection methods to a two-stage process, using a wrapper method to select important features to avoid the curse of dimensionality, followed by the filter method to select more critical factors 27 .
The challenges associated with the aforementioned methods are distinct: 1 Data decomposition methods are commonly utilized in stock index prediction rather than in the individual stock selection process.The unequal frequencies obtained from time series data pose a significant challenge, hindering the parallel aggregation of decomposed features for individual stocks.2 Principal Component Analysis (PCA) limitations align with theoretical expectations but often diverge from expected performance in empirical scenarios, demonstrating diminished effectiveness. 3 Existing data augmentation methods are relatively simplistic and exhibit limited efficacy in improving accuracy or alpha effects.These methods mainly expand existing features without notably enhancing their value or informativeness.4 Tuning a Deep Neural Network (DNN) faces challenges from models with numerous parameters.Even with genetic algorithm integration, computational demands persist.While the genetic algorithm only reduces factors in feature selection.
An encouraging approach is to integrate Genetic Algorithms (GA) principles into the Data Augmentation method.This innovative strategy aims to leverage GA concepts to actively evolve factors and select features from this Genetic Evolved Method.This may lead to more effective feature sets.We'll delve into this proposed novel GA-based data augmentation method in methodology part.

The proposed deep neural network
In Artificial Intelligence (AI), Deep Neural Network (DNN) falls under the subset of Machine Learning and Neural Network 28 .DNN is based on the artificial neural network (ANN) which contained one or several layers between the input and output layers.In each layer it consists of the same components, and they are neurons, synapses, weights, biases, and functions 29 .
The proposed SGP-DNN framework comprises four primary phases: data pre-processing, data augmentation, filtered factor transformation, and feature extraction.As depicted in Fig. 1, during Phase 1, we acquire the dataset from the Alpha Factor Library by S&P Global Market Intelligence, which includes raw fundamental and technical indicators.Instead of conducting feature selection at this stage, our focus is on standard data pre-processing steps such as handling missing values, deleting outliers or noise, and performing feature normalization using the z-score method.Concurrently, we define heuristic formulas to aid the subsequent SGP programming phase.
In Phase 2, SGP operates similarly to a conventional Genetic Algorithm, involving steps like selection, crossover, and mutation.However, it notably differs by utilizing both heuristic generators and normalized features from phase 1 as inputs.A comprehensive explanation of SGP will be provided in Section "data augmentation: symbolic genetic programming".The outcomes of phase 2 consist of optimal features developed through the evolutionary process of SGP, tailored to its specific customized fitness function.
After Phase 2, a set of optimal features (augmented data) is obtained.Phase 3 involves defining rules for feature filtering, considering both their fitness for the model's function and their relevance to the prediction target.These filtered features then undergo transformation to ensure they are of a suitable size and sequence for subsequent use in the LSTM model.The output of Phase 3 comprises the filtered features arranged in an appropriate sequence, in this instance, with lags of 15 days.
The last phase (4) revolves around feature extraction using a DNN model.Specifically, we utilized the Long Short-Term Memory (LSTM) method to discern non-linear patterns, aiming to enhance the accuracy of stock return prediction.Selected features, arranged in sequence from Phase 3, are inputted into a two-layered LSTM model and utilize the final hidden layer output for the final prediction.However, in the feature extraction phase, we also experiment the filtered factors with Multi-Layer Perceptron (MLP).The objective is to observe whether LSTM or MLP could handle the raw data in the extracting pattern according to the nature of feature source (fundamental or technical indicators).A detailed explanation of the DNN model selection process will be presented in Section "Feature extraction method: LSTM vs MLP".
The effectiveness for classification of the prediction from phase I to phase IV is measured using Accuracy, Precision, Recall, Rank Information Coefficient (Rank IC) and Information Ratio of IC (ICIR) as performance (sensitivity) metrics for cross-sectional price change prediction, as demonstrated in Eqs. 1 to 5 In the next Section, we present the discussion on the dataset, software and hardware used in this study, as well as the elaboration on phase I, data augmentation and phase II, feature extraction.
Rank Information Coefficient (Rank IC) serves as a pivotal tool for appraising predictive model efficacy, particularly in portfolio formation during stock selection across a range.This metric evaluates the correlation in rankings between predicted scores of diverse securities and their realized returns, prioritizing relative rankings over precise predictions.It notably facilitates cross-sectional selections aimed at securing alpha or risk-adjusted returns for portfolios.Worth noting is that the sign of Rank IC holds less significance than its magnitude.A positive Rank IC suggests that higher stock values anticipate relatively larger returns, while a negative Rank IC signifies that lower stock values predict larger returns.Additionally, the Information Ratio of Rank IC (ICIR) parallels the Sharpe ratio for a portfolio, providing further insights into its performance.
( For the data preparation and pre-processing, Python 3.8 was employed along with the numpy and pandas packages.The design of DNN models, including LSTM and MLP, was achieved using KERAS 2.4, a package based on Google TensorFlow 2.4.The Symbolic Genetic Programming (SGP) was implemented using the gplearn 0.0.2 package in Python.While the DNN network was trained on NVIDIA GPUs, the remaining models, such as SGP part, were trained on a CPU cluster.Detailed information regarding the software and hardware specifications utilized can be found in Table 2.
The main aim of this study is to anticipate and forecast changes in cross-sectional stock prices.The target variable is categorized into two statuses: a value of 1 signifies a stock return higher than the medium of crosssectional stock returns, while a value of 0 indicates a stock return lower than the medium of cross-sectional stock returns over short-term intervals.The research investigates standard timeframes frequently used in stock predictions, spanning short-term intervals of 5 days (1 week), 10 days (2 weeks), and long-term intervals of 20 days (1 month).This study integrates short-term technical indicators and long-term fundamental indicators.Consequently, the 5-day forecasting period is chosen to assess prediction accuracy.By amalgamating these varied features, the study aims to provide accurate forecasts regarding stock price changes, specifically determining whether they will surpass or fall below the medium of cross-sectional stock prices within the defined 5-day window.

Data augmentation: symbolic genetic programming
The second step of the proposed DNN framework is to investigate the Genetic Algorithm (GA) in the data augmentation phase.Based on literature, Genetic Algorithms are a type of learning algorithm, that would result in a better neural network by crossing over the weights of two good neural networks.This algorithm could also generate and evaluates consecutive generations of humans in order to achieve optimization objectives.The algorithm creates mutation from the stock related indicators by randomly changing the chromosomes or genes of the individual parents.In this situation, GA can be complicated and costly when implemented on the stock related indicators which is nonlinear and having lots of noise or outliers.Therefore, to solve the problem of nonlinear type of data, the Symbolic Genetic Programming (SGP) is employed in this study.SGP has several advantages as it evolve by building blocks.In SGP, it employed the regression analysis which is more robust to search the space in finding the best model to fit the given stock return data.Different from GA, SGP find an intrinsic relationship between two or more variables which is hidden.Typically, there are two types of genes that contribute to the generations.
The first type in the study refers to the input features, while the second type represents the processing operators, encompassing mathematical functions like addition, subtraction, division, and multiplication.Predicting stock price data can be a daunting task, given its complex, dynamic, and non-linear nature.To tackle this challenge, mainstream hedge funds like World Quant, Cubist, and Menelia employ various heuristic operators such as correlation, covariance, and variance.These operators help them analyze and interpret the data, enabling them to make informed investment decisions 31 , as depicted in Table 3, to enhance the analysis and prediction of stock price data.In this study, an improved Symbolic Genetic Programming (SGP) is proposed, which utilizes symbolic tree expressions to handle and solve complex optimization problems, providing greater flexibility.The four-step approach outlined in Fig. 3 is applied to enhance the performance of the SGP.
The first step in our proposed SGP, is to initiate the population of the genes.Here, we introduce the heuristic operators like the Table 3 shows in the reproduction of the genes.To guide the evolution of the genes, we set certain parameters.For instance, we established a probability of 40% for crossover, which involves exchanging genes between two individuals in the population.Additionally, we set a 40% probability for replacement, which involves copying an individual gene in the population.Finally, we assigned a very low probability for three types of mutation to prevent an excessive influx of new input features, which could lead to unpredictability.This helps maintain stability in the incorporation of new genetic material into the population.www.nature.com/scientificreports/Second, we designed and added rolling windows for all heuristic operators to the original SGP.To this end, we randomly generate rolling window seeds between 3 and 20 for rolling window to produce additional symbolic formulas.The third step is to design the fitness function.In this study, calculations are performed to determine the fitness target.In addition to using the original Pearson correlation (Rank IC) between the value of the symbolic formula and future price change as the fitness target, a combined formula will be used.This combined formula takes into consideration both the relatively high cumulated return of the bottom group among all cross-sectional stocks and the maintenance of monotonicity in the cumulated return of k groups based on the order of values in the symbolic formula.By incorporating these factors, the fitness target aims to optimize the performance of the symbolic formula in predicting stock price changes.
The formula is shown from Eqs. ( 6) to (8) below: After obtaining many symbolic formulas based on the above algorithms, the final amendment for SGP is the filter system for the outcomes.The success ratio of Pearson correlation (Rank IC) and the profit and loss ratio (P&L ratio) of Pearson correlation from Eqs. ( 9) to (10) will be employed to select the final synthetic symbolic formulas generated by the SGP model.These above two ratios will also be used as metrics for the experiment part

Feature extraction method: LSTM vs MLP
The final step in the proposed hybrid DNN framework involves extracting features from the augmented selected data obtained through the SGP process.Feature selection is carried out by creating a Hybrid DNN model that accommodates individual data sources based on their specific characteristics.(5)  Since the development of DNN, the Multiple Layer Perceptron (MLP) was initially introduced as a basic supervised learning algorithm with multiple layers, each consisting of several neurons.However, MLPs have a significant drawback in their ability to handle sequence or time series data effectively.This limitation poses a crucial challenge in stock return forecasting, which heavily relies on the historical states of stocks, following a Markov Chain.To address this issue, a more suitable approach is to utilize the LSTM (Long Short-Term Memory) model, which falls under the category of Recurrent Neural Networks (RNN).LSTMs are specifically designed for sequence modelling tasks and overcome the limitations of MLP.Both LSTM and MLP models are chosen for comparisons, as shown in Fig. 4.
In the first step, the original indicators are either inputted into the SGP model (as depicted in Fig. 4) to obtain selected features, which are then fed to the MLP or LSTM model.Alternatively, the original indicators can be directly fed into the MLP or LSTM model for comparison.
The performance of the four experiments is evaluated using metrics such as Rank IC and ICIR to determine the best model based on the dataset's unique characteristics.The optimization goal for all network settings is to minimize Mean Squared Error (MSE), while the performance quality is assessed using Rank IC and ICIR as metric indicators.Finally, the trained network is used to recognize feature patterns, and based on the Enhanced To ensure simulating the real stock investing and considering the 'backtest' 32 , the forward rolling window and the segmentation prediction method were followed, the specific details are illustrated in Fig. 5.The whole sample period will be divided into three parts, in the training part, the dataset length is 1020 days which is used to update the model parameters.As for validation part, we use 160 days for tunning and the test part is 20 days and as a result the rolling window is also set as 20 days.The ratio of training set, validation set is taken as 8.5:1 and the real test days is 720 days from 2019-11-30 to 2022-12-31-resulting in a total of 36 non-overlapping trading periods.

Forecasting, ranking, and trading
The SGP-DNN model utilizes available information prior to time t to forecast the future price change of each stock.Its objective is for each stock to surpass the average price changes observed in the cross-sectional market during the subsequent period t + 1.To achieve this, the model ranks all cross-sectional stocks (4500 in total) in ascending order based on the predicted return by SGP-DNN for the next period.The highest-ranked stocks form the top group, and historically, we have divided the entire cross-sectional stocks into 10 groups, each containing 450 stocks.This ranking score serves as a basis for long only portfolio construction.
Long-Only Portfolio Strategy: The Long-Only Portfolio Strategy focuses on taking long positions in the top k stock portfolios, which are then held for a single period (t + 1).To gauge the effectiveness of this strategy, we will compare its performance against the CSI 300 and CSI 500 benchmarks (denoted as Excess R above 300 and Excess R above 500).These benchmarks represent broad-based indexes in the Chinese stock market.Moreover, we will also consider the average performance of an equal-weighted portfolio as a third performance benchmark (denoted as Excess R above average), the sharp ratio of Excess R above average (Sharp Ratio) will be also measured as the metrics in experiment part.

Experiments with five classical DNN frameworks for comparison
In this study, two types of raw data-fundamental and technical indicators-are utilized to evaluate the Neural Network's performance.The objective is to experiment with both datasets and identify potential discrepancies in outcomes.
Preceding this analysis, an extensive comparative test was conducted.The subsequent metrics will illustrate comparisons among several models: classical Thomas Fischer's LSTM model, Ghosh's three features LSTM model, single LSTM model with raw features, PCA-LSTM model with raw features, and proposed SGP-LSTM model with raw features.
Figure 6 displays our comparison among these 5 models, utilizing a train/validation/test split without employing rolling window or segmentation prediction methods for direct comparisons.The entire sample period was divided into a 70% train set, a 20% validation set, and a 10% test set, enabling a thorough evaluation of the comparison metrics.The examination of metrics across five models in Table 4 unveils intriguing discoveries.Thomas Fischer's LSTM model notably improves accuracy, achieving 53.8% in the Chinese stock market, surpassing its previously reported 51.4% accuracy in the US stock market (1992 to 2015).Unexpectedly, despite incorporating two additional features into Ghosh's model, there's no observable accuracy enhancement within the Chinese stock market (53.2% for Ghosh compared to Thomas' 53.8%).
In the related work section, I highlighted the persistent trend of negative accumulated alpha observed in both Thomas and Ghosh's LSTM models over the past decade.Table 5's data reveals a compelling pattern: the ratio between forecasted positive and negative outcomes exceeds 2, notably peaking at 3.75 in Ghosh's model.This underscores a consistent bias toward predicting positive outcomes.For instance, in Ghosh's model, out of over 4700 stocks, 3727 are forecasted as positive compared to 995 forecasted as negative, showcasing a recurring inclination to predict individual stock returns surpassing the median of all sectional stocks.This trend may significantly contribute to the negative accumulated alpha observed within the top group portfolio (trading rules for the 10 groups outlined in Section "Forecasting, ranking, and trading"), to be further explored in www.nature.com/scientificreports/Section "Proposed models comparison with Fisher's LSTM model, Ghosh's three features LSTM model" through experimentation.A similar pattern is evident in Thomas's model.Conversely, the other three models exhibit a more balanced distribution between positive and negative forecasted outcomes.Table 4 showcases the SGP-LSTM model achieving the highest accuracy rate of 53.70% in the test set.However, upon comparing accuracy and Rank IC, the Single-LSTM model (52.80%, 7.3%) appears less effective compared to the PCA-LSTM model (53.50%, 9.0%), suggesting an advantage of PCA-LSTM in predictive effectiveness over Single-LSTM.Notably, substituting PCA with SGP led to an improvement in accuracy and Rank IC from (53.5%, 9.0%) to (53.7%, 9.3%).These results signify that SGP-LSTM outperforms PCA-LSTM, validating the efficacy of data augmentation or decomposition methods.
Additionally, considering both the balanced distributions for positive and negative forecasted outcomes and the Rank-IC, the Proposed SGP-LSTM model demonstrates superiority over the other four models.Further exploration of this superiority will be detailed in Section "Proposed models comparison with Fisher's LSTM model, Ghosh's three features LSTM model" through experimentation.
After completing experiments on the five models using machine learning metrics, our next step involves a more in-depth exploration of the alpha effect employing the Proposed SGP-LSTM model.This exploration will utilize two distinct sets of raw features processed through rolling windows, following the trading rules outlined in Section "Forecasting, ranking, and trading".
In our stock trading experiment, we conducted 'backtests' spanning 720 days.The prediction horizon was fixed at 5 days, and we implemented a rolling cycle of 20 days.This setup allowed the hybrid model to optimize its parameters every 20 days, utilizing the 1180 data points mentioned earlier.These optimized parameters remained consistent for the subsequent 20-day period.Additionally, every 5 days, the model ranked its stock prediction values, selecting the top 10% (450 stocks) for portfolio construction.Equal weights were assigned for buying and  www.nature.com/scientificreports/holding, and stocks with limitations were excluded to address potential trading issues.This approach ensured systematic assessment while constructing the portfolio based on the model's predictions within this timeframe.Following the procedure of Proposed SGP-DNN model in Section "The proposed deep neural network", we also nee to examine the efficacy of SGP when integrated with MLP and LSTM, thereby assessing the suitability of SGP in conjunction with both methods.The experiment will be segmented into two main sections: Section "Experiment with fundamental indicator" will elaborate on experiments using fundamental indicators, while Section "Experiment using technical indicator."will focus on experiments employing technical indicators.This structured approach aims to comprehensively explore the impact and potential of integrating SGP within different feature processing methods.

Experiment with fundamental indicator
We execute the experiment with the fundamental indicator.This experiment is to observe 8 metrics of the cross-sectional stock return prediction based on the fundamental indicator, whether the integration of SGP give improvement or vice versa.First, we experiment the fundamental indicator directly using the MLP method.Then followed by experimenting it using LSTM method.This experiment is without integration of SGP.To observe the capability of SGP, we executed an experiment based on the using the LSTM and MLP respectively with the integration of SGP method.Table 5, illustrates the results of the experiments conducted, where the LSTM or MLP is integrated with SGP, is labelled as SGP-MLP and SGP-LSTM respectively.While the results obtained without the integration of SGP is shown in the column labelled as MLP and LSTM respectively.
Table 5 shows the results executed from the experiment for data in the year of 2020 to 2022.Whereas Fig. 6 summarize the data from 2020 to 2022 based on its average mean.Based on the results shown in Fig. 8, the results indicate that when the raw fundamental indicators were used as input for LSTM or MLP models, the average IC values were − 1.85% and − 2.88%, respectively.The average value of IC in this situation is considered low whereby the ideal average value should be above 8%.While the average value for ICIR were − 1.55 and − 3.22, respectively.This value for cross-sectional stock price change prediction is considered average or acceptable.The ideal value for ICIR is above 3.However, after integrating the models with the SGP algorithm, the IC absolute values increased to 8.05% for MLP and 7.98% for LSTM which is considered as ideal outcome.
The SGP-LSTM model attained the highest average value of − 5.46 for ICIR, surpassing the performance of other models.It exhibited superior results in terms of IC-success ratio and IC-PNL, with values of 78.91% and 2.13, respectively.Furthermore, both the SGP-LSTM and SGP-MLP models showcased notable advantages over the single DNN models by employing a straightforward rule-based strategy for a long-only approach.Specifically, the SGP-LSTM model demonstrated a Excess R exceeding the CSI 300 index by 22.35% and surpassing the CSI 500 index by 16.26%.Moreover, it achieved an Excess R above the average by 9.89% per year, positioning it among the top 10% of mutual fund managers in China.

Experiment using technical indicator.
In contrast, according to the findings presented in Table 6 and Fig. 9, SGP-MLP or SGP-LSTM does not demonstrate significant advantages over single DNN models when it comes to technical indicators.On average, the single LSTM model for technical indicators produced the best results in terms of normal metrics such as IC, ICIR, and IC-success, with percentages of -8.64%, -6.561, and 85.71% respectively (the original IC mean of technical indicators is 2.82% and ICIR mean is 0.23 from Table 1 and  www.nature.com/scientificreports/which is better suited for the LSTM model, as explained in the methodology section.Notably, when considering a long-only strategy, the LSTM model exhibited a significantly higher Excess R above average at 13.28%, compared to the MLP model's 3.94%.
Based on the experiments conducted earlier, we could summarize that the fundamental indicator will achieve the best result, when the indicators are fed into SGP algorithm, while the technical indicator will achieve the best result without integrating the SGP but directly through LSTM technique.Therefore, we design a new DNN framework that could work well with both fundamental and technical indicators.Figure 10 below illustrates the proposed DNN framework where both fundamental and technical indicators are fed as the raw data.The explanation on the experiment on this proposed framework will be discussed in the next section.

Experiment using fundamental and technical features.
Based on Fig. 10, the fundamental indicators are first fed as the raw data into the framework.As mentioned earlier, the results shows better when SGP is integrated with LSTM or MLP.Therefore, the fundamental indicators are processed based on SGP and the output is being an input for the Phase I, the augmentation phase.The output for the augmentation phase is combined with the technical indicators to be an input for the Phase II, the feature selection.Here, only LSTM is utilized as from the experiment executed earlier, LSTM outperformed the MLP in terms of a better results.Two layers of LSTM are performed with 100 nodes each, where the final feature selection is only 30 notes for the price changes prediction.In Section "Experiment using fundamental and technical features.",we present the results based on the experiments conducted using the new proposed framework as shown in Fig. 10.According to Table 7 our hybrid model showcased a significant improvement of 1128% in information coefficient (IC) and an impressive surge of 5360% in IC information ratio (ICIR) when applied to fundamental indicators.For technical indicators, the hybrid model achieved a commendable 206% increase in IC and a remarkable surge of 2752% in ICIR.According to Table 8 and Fig. 11, the proposed SGP-LSTM model attained a rank IC value of 9.24% and an ICIR of 7.24 for a five-day prediction horizon.

Proposed models comparison with Fisher's LSTM model, Ghosh's three features LSTM model
This section focuses on comparing the alpha effect of the top group portfolio among the Proposed SGP-LSTM model, Thomas Fischer's, and Ghosh's models.The rolling windows test, executed with identical settings between Section "Feature extraction method: LSTM vs MLP" and Section "Forecasting, ranking, and trading", aimed to compare the performance of the proposed SGP-LSTM model against Thomas Fischer's and Ghosh's LSTM models.This comprehensive analysis involved evaluating metrics for both the DNN model and the portfolio's risk-adjusted accumulated return.
Table 9 presents the outcomes derived from the 2020 to 2022 test set, delineating the average metrics for the three models scrutinized in this study.As expounded in Section "Experiments with five classical DNN frameworks for comparison", limitations regarding the uneven distribution of forecasted positive and negative outcomes are evident for both the Thomas Fischer and Ghosh models.Specifically, as highlighted in Table 9, the precision metric exhibits poor performance for Thomas and Ghosh models, registering values of 52.1% and 51.5%, respectively, compared to 53.07% for the Proposed SGP-LSTM model.Despite their higher Rank IC values, as anticipated in Section "Experiments with five classical DNN frameworks for comparison", all models resulted in negative Excess Return (alpha).Ultimately, the Proposed SGP-LSTM model demonstrated a 16.38% Excess Return (alpha) for the test set, accompanied by an information ratio of around 2.66.www.nature.com/scientificreports/Furthermore, Fig. 12 showcases the cumulative performance comparison of Excess Returns over the average return of all stocks within three long-only portfolios: Ghosh's LSTM model, Thomas Fischer's LSTM model, and our proposed enhanced SGP-LSTM model, which yields the most favorable outcomes.Over the three-year out-of-sample period, it achieves a relative annual return of 16.38% and accumulates a total return of 57.62%.In contrast, despite exhibiting relatively high accuracy rates, the accumulated excess returns of Thomas Fischer and Ghosh remain negative.

Technical Indicators
Figure 13 presents a comparison of the cumulative return curves for the proposed SGP-LSTM model portfolio and two broad-based indices, as well as Ghosh's LSTM and Thomas Fischer's LSTM portfolio, during the period of 2020-2022.The results clearly demonstrate that the proposed model outperformed the average portfolio, as well as the CSI 300 and CSI 500 indices.Notably, the SGD-LSTM hybrid model exhibited significant outperformance compared to the CSI 300 index, the CSI 500 index as well as two benchmark portfolios, as shown in Fig. 13.Over the span of three years, the proposed SGP-LSTM model accrued an accumulated R of about 67.75%.In contrast, the CSI 500 achieved 11.32% in accumulated R, whereas both CSI 300 and the two benchmark portfolios accumulated negative returns.This study focused solely on financial time series data, which is known for its high autocorrelation.However, recent research has explored the incorporation of diverse data sources such as social media data, news, macroeconomic data, and high-frequency data.Moreover, the proposed hybrid SGP-DNN model could benefit from additional optimization targets, such as Excess Return of top groups or monotonicity of ten groups of target stocks, instead of solely relying on MSE as the optimization goal.Additionally, recent advancements in reinforcement learning or generative adversarial networks (GANs), such as ChartGPT application, have been suggested to be combined with hybrid DNN models.Therefore, it could be worthwhile to consider supplementing the suggested hybrid SGP-DNN model with GANs or reinforcement learning techniques to leverage multi-source information and improve prediction performance.

Figure 3 .
Figure 3.The structure of the proposed Symbolic Genetic Programming.

Figure 6 .Figure 7 .
Figure 6.Train/validation/test set for the whole sample period.

Figure 10 .
Figure 10.The proposed SGP-DNN framework for raw indicators.

Figure 11 .
Figure 11.The metrics based on the proposed SGP-LSTM framework for the average mean data.

Phase 1: Data Pre-Processing Phase 4: Feature Extraction Filtered Factors Transformation Phase 3: Filtered Factors Transformation Phase 2: Symbolic Genetic Programming Figure 1.
Flowchart of the proposed Deep Neural Network Framework.

Table 1 .
Rank IC mean of the dataset.

Table 2 .
Descriptions on the software and hardware.

Table 4 .
Metrics of Comparisons among different models.

Table 5 .
The metrics of fundamental indicators for DNN with MLP or LSTM.

Table 6 .
The metrics of technical indicators for DNN with MLP or LSTM.The metrics of technical indicators for DNN with MLP or LSTM.

Table 7 .
The metrics based on the proposed SGP-LSTM framework.

Table 8 .
The metrics based on the proposed SGP-DNN framework.

Table 9 .
The mean of metrics comparisons among three models.
ConclusionThis paper introduced a methodology to enhance the cross-sectional stock return prediction by utilizing Symbolic Genetic Programming (SGP) for input generation and integrating it with Deep Neural Network (DNN) models.The study demonstrated significant improvements in prediction, outperforming popular market indexes.A hybrid model combining SGP with Long Short-Term Memory (LSTM) showcased superior performance, consistently surpassing market returns a simple rule-based strategy based on the proposed hybrid SGP-LSTM model outperforms major Chinese stock indexes, generating average annualized excess returns of 31.00%,24.48%, and 16.38% compared to the CSI 300 index, CSI 500 index, and the average portfolio, respectively.The findings highlight the potential of the proposed approach in generating profitable investment strategies and provide insights into addressing challenges in data integration and feature engineering.