Bird’s Eye View feature selection for high-dimensional data

In machine learning, an informative dataset is crucial for accurate predictions. However, high dimensional data often contains irrelevant features, outliers, and noise, which can negatively impact model performance and consume computational resources. To tackle this challenge, the Bird’s Eye View (BEV) feature selection technique is introduced. This approach is inspired by the natural world, where a bird searches for important features in a sparse dataset, similar to how a bird search for sustenance in a sprawling jungle. BEV incorporates elements of Evolutionary Algorithms with a Genetic Algorithm to maintain a population of top-performing agents, Dynamic Markov Chain to steer the movement of agents in the search space, and Reinforcement Learning to reward and penalize agents based on their progress. The proposed strategy in this paper leads to improved classification performance and a reduced number of features compared to conventional methods, as demonstrated by outperforming state-of-the-art feature selection techniques across multiple benchmark datasets.


Background and literature review
In recent years, various optimization techniques have been developed to tackle complex problems across fields such as computer science, engineering, finance, machine learning, and data science.This section reviews three of the most prominent algorithms: Markov Chain, Evolutionary Algorithms (specifically Genetic Algorithm), and Reinforcement Learning.These methods have proven to be effective in addressing challenging optimization problems and have been widely used.Despite their importance, these methods have certain drawbacks, including constrained exploration, the necessity for parameter modification, the inability to handle multiple objectives, and slow or premature convergence.Thus, it is crucial to take these restrictions into account when applying them to challenging optimization problems.One can overcome these limitations by carefully characterizing the problem, selecting the best algorithm, fine-tuning the parameters, and using complementary strategies to solve the shortcomings of each approach.In the following subsections, a brief overview of each approach, its key concepts, applications, advantages, and usage in the proposed work are provided.

Markov chain.
The Markov analysis is a technique for estimating the value of a variable that is solely dependent on its current state, without taking into account prior activity 13 .It calculates a random variable based on the present state of other variables using a probability matrix.This makes it a useful tool for evaluating state transitions in various fields such as surveillance 14 , machine learning 15 , and computer vision 16 .Its popularity is due to its ease of use and good prediction accuracy, often outperforming more complex models 17 .Although widely used, few studies have applied it to feature extraction [18][19][20] , where Markov chain features are extracted to capture dynamic changes in data and used by learning algorithms to make decisions.A new concept of feature selection, based on the transition probabilities of the Markov chain, is proposed as an alternative to feature extraction in our work.
Evolutionary algorithms.An Evolutionary Algorithm (EA) is a computational method that solves problems by mimicking the behavior of living organisms using nature-inspired mechanisms 21 .The use of EAs for feature selection has received significant attention in academia, with various algorithms being proposed, including Particle Swarm Optimization (PSO) [22][23][24] , Genetic Algorithm (GA) 25,26 , Artificial Bee Colony (ABC) 27 , Genetic Programming (GP) 28 , Gravitational Search Algorithm (GSA) 29 and Ant Colony Optimization (ACO) 30,31 .One advantage of EAs is their population-based search approach, which involves a team of entities exploring the fitness landscape to find the globally optimum solution.This allows for more effective and efficient exploration of vast and challenging search areas.The sharing of information among team members also enables the discovery of potential regions of the search space and the narrowing of the search to critical areas.Additionally, these methods balance exploration and exploitation, allowing for faster convergence while avoiding local optimal solutions.These unique characteristics make EAs a promising approach for designing neural networks 32 .
Genetic algorithms are the type of evolutionary algorithms used in this work.A genetic algorithm is an optimization technique that uses a process inspired by natural evolution to find the best solution for a problem.The algorithm works by iteratively searching through a space of potential solutions, selecting and breeding the most promising candidates based on a set of rules inspired by genetics, and introducing random mutations to create new solutions.This process is repeated until either a satisfactory solution is found or a specified number of iterations have passed.Genetic algorithms are commonly used in machine learning and data analysis to find optimal model parameters [33][34][35] or identify patterns in data 36,37 .The same approach is applied to feature selection in the proposed work.Initially, a set of possible feature combinations is generated randomly, represented as pairs.These pairs are then evaluated using a fitness function that assigns a score based on their accuracy.The pairs with the highest scores are selected for reproduction, mimicking the process of natural selection.The process repeats until a satisfactory solution is found or a specified number of iterations have been reached.

Reinforcement learning.
Reinforcement learning 38,39 is a method of learning by interacting with the environment and learning from rewards received from actions taken.It aims to find the best long-term solution by balancing exploration and exploitation.This type of learning has a lot of potential for effective feature selection in the subspace of features.Feature selection can be performed through single-agent 40,41 or multi-agent 42 decision processes.In a single-agent process, only one agent decides on the selection or deselection of features, resulting in a large action space and the risk of getting stuck in a local optimum solution.On the other hand, in a multi-agent process, multiple agents are involved in feature selection, which enables easier exploration and convergence of the search space.This approach also resembles natural systems, as there are similarities between reinforcement learning and biological systems 43 .

A fitness function to better evaluation of classifiers
Classifier evaluation metrics 44,45 are used to determine the effectiveness of a classification model by comparing the predicted outcomes to the actual outcomes.Some commonly used metrics for evaluating classifiers include: • Accuracy It measures the percentage of correct predictions made by the model out of all predictions.It is defined as (TP + TN)/(TP + TN + FP + FN) , where TP (True Positives) represents the number of positive instances correctly predicted, TN (True Negatives) represents the number of negative instances correctly predicted, FP (False Positives) represents the number of negative instances incorrectly predicted as positive, and FN (False Negatives) represents the number of positive instances incorrectly predicted as negative.• Precision It is the ratio of true positive predictions to the sum of true positive and false positive predic- tions.Precision measures the ability of the classifier to avoid false positive predictions and is defined as TP/(TP + FP).• Recall (Sensitivity or True Positive Rate) It is the ratio of true positive predictions to the sum of true positive and false negative predictions.Recall measures the ability of the classifier to detect positive instances and is defined as TP/(TP + FN).• F1-Score It is the harmonic mean of precision and recall, used to balance precision and recall when they are in conflict.The F1-Score is defined as (2 • Precision • Recall)/(Precision + Recall) .It provides a balance between precision and recall, as it is a measure of the harmonic mean of these two values.The choice of evaluation metric will depend on the problem and the goals of the classifier.For example, precision may be important when false positive predictions are costly, while recall may be important when false negative predictions are costly.Note that in multiclass classification, precision, recall, and F1-Score can be calculated for each class and then averaged using macro-average or micro-average methods.The confusion matrix is a table that has C rows and C columns, where C is the number of classes.Each row of the matrix represents the instances in a predicted class, while each column represents the instances in an actual class.For example, consider a multiclass classification problem with C = 3 classes.The confusion matrix would be a 3 × 3 table, as shown below in Table 1.
Where TP i represents the number of instances of class i that are correctly predicted as class i, and FP ij repre- sents the number of instances of class j that are incorrectly predicted as class i.
From the values in the confusion matrix, various evaluation metrics such as accuracy, precision, recall, and F1-Score for each class, as well as macro-average and micro-average across all classes, can be calculated.The choice of evaluation metric will depend on the problem and the goals of the classifier.

Methods
The goal of feature selection is to identify and select the smallest possible subset of relevant features from a larger set of features, to improve the accuracy, interpretability, and computational efficiency of the model.The idea is to remove redundant, irrelevant, and noisy features that may negatively impact the model's performance.The selection of a smaller set of relevant features not only aids in mitigating overfitting but also enhances the interpretability and comprehensibility of the model for human experts.A new tree search algorithm is developed in this paper to better explore the search space representing all the possible subsets.Our algorithm starts from the root node and expands it to generate child nodes until a goal node is found.
The search algorithm begins with a randomly selected subset of features represented by a sequence of 1 s and 0 s, where 1 s indicate selected features and 0 s indicate unselected features, i.e., each leaf belongs to {0, 1} d , where the integer d is the size of the total features.
The root leaf generates A new subsets, known as children, by randomly altering the states of each pair of features.The children are formed using the transition probability of the Markov chain of each feature pair, the transition matrices reflect the likelihood of transitioning between distinct states {00, 01, 10, 11}, with initial values for the transition probabilities of 0.25.
Through the expansion, the transition matrices are updated based on a rewards function reflecting the performance of the generated children.Therefore, each new leaf generated will inherit the transition matrices of each pair of features from the parent and update them according to the concept of reward that will describe later in this section.
Updating these transition matrices in the right manner will favor certain extensions of the proposed tree to better explore the search space.After each cycle or iteration, only the highest-performing leaves are kept for further expansion.
The following definitions are crucial for a thorough explanation of the approach: • States or leaves are defined in: {0, 1} d , where the integer d is the size of the total features.
• A : number of children generated by each leaf; each offspring represents a subset of selected features.
• M A : number of top-performing leaves that are selected for further expansion at each iteration.
• t: number of iterations.
• s: number of stages.
• F t,s j : represents the status of the jth leaf (i.e., state) at time t and stage s, F t,s j ∈ {0, 1} d , j = 1, . . ., M A , which specifies whether each feature has been selected or not.The position of values of 1 shows the location of the features that have been chosen, and the position of the values of 0 indicates the position of the features that have been eliminated.• f t,s i,j : represents the value of the ith feature in the jth leaf at time t and stage s , f t,s i,j ∈ {0, 1} , i = 1, 2, . . ., d and j = 1, . . ., M A .• C t,s i,j : represents the state of ith feature pair, C t,s i,j = {f t,s 2i−1,j , f t,s 2i,j } , at time t and stage s of jth leaf.
) : transition probability from the pair C t−1,s i,j to the pair C t.s i,j , it represents the actions of the evolutional algorithm.
• d : dimension of data or number of features f t,s i,j , i = 1, 2, . . ., d; • n: number of observations of data.

Genetic algorithm.
The BEV algorithm utilizes a smart branching evolution approach that is based on dynamic Markov chains.At each new expansion, a fixed number of leaves ( M A ) are chosen.Each leaf is represented by a sequence consisting of 1 s and 0 s and they are organized in pairs within the sequence.The process begins with a root leaf and generates A children leaves, where A is less than M A .Since the number of generated leaves does not exceed M A , all of them are selected.During the next expansion, each leaf (or child) generates A leaves, resulting in a total of A • A children and A parent leaves.These children and parent leaves are evaluated, and the best M A leaves are chosen for the expansion.
In the subsequent step, each leaf from the selected A • M A leaves generates a A child, resulting in ( A • M A ) children and M A parent leaves.Again, these leaves are assessed, and only the best M A leaves are selected for the next expansion.This process continues until there is no further improvement in the quality of the solution.
Figure 2 illustrates the process of the BEV method, which involves expanding the children and selecting the most effective subset of features with A set to 3 and M A set to 9. Starting from the root leaf, three leaves are generated and all of them will be selected as they do not exceed the value A • M A .The next expansion results in 9 children and 3 parent leaves, and the 9 best leaves will be chosen based on their performance (step 1).From the selected 9 leaves, a total of 27 leaves (children) are generated, leading to a combined set of 36 leaves (including (1) min i TP i TP i + j� =i FP ij parents and children).Similarly, in the next expansion, the 9 best leaves among the 36 will be chosen (step 2), and this process continues iteratively.
Each leaf is represented by a sequence of 1 s and 0 s, where the features are grouped in pairs, as shown in Fig. 3. Every pair of features for each leaf has its transition matrix that determines the expansion process for that pair.Two scenarios must be taken into account when features are grouped two by two. Figure 4a, b demonstrate these two scenarios depending on whether the dimension d is even or odd.

Markov decision process (MDP) and reinforcement learning.
In order to determine the optimal subset of features that effectively differentiate between different classes, the BEV algorithm utilizes an smart approach to update transition probabilities during the transition from one state to another.This updating process is based on a reward and penalty mechanism.When the fitness function shows improvement, a reward value is added to the transition probability associated with the corresponding direction.At the same time, one third of the reward value is deducted from the transition probabilities of other directions.On the other hand, if the  fitness function does not improve, a penalty value is applied to the transition probability of the relevant direction, while one third of the penalty value is added to the transition probabilities of other directions.
As each Markov chain has four states {00, 01, 10, 11}, each pair of features at each leaf of F t,s j has four separate probability mass functions that govern the expansion process.Each child leaf will inherit these probability mass functions, or transition matrices, from the parent leaf and update them based on the fitness function as shown in Figs. 5 and 6.
The fitness function, denoted by f , can be interpreted as the classification accuracy at the state F t,s j , The accuracy is calculated based solely on the features chosen with a value of 1 at their positions.The fitness function f can be chosen as a minimum accuracy for each class as: where TP i represents the number of instances of class i that are correctly predicted as class i, and FP ij represents the number of instances of class j that are incorrectly predicted as class i.The value K represents the total number of classes.
In the case where A = 3 and A • M A = 9, Fig. 5 illustrates the early stages of expansion in a process, where three leaves, denoted as F t=1,s=0 j with j = 1, 2, 3, emerge from the root leaf.Another 9 leaves are generated from   www.nature.com/scientificreports/ the 3 leaves F t=1,s=0 j noted F t=2,s=0 j for j = 1 to 9. From these 12 leaves, only 9 are selected for continued expan- sion through the application of fitness functions, f F t=1,s=0 j for j = 1, 2, 3 and f F t=2,s=0 j for j = 1 to 9, which determines the most suitable leaves for growth.
The growth of each leaf is achieved through the transitions of each pair of features, represented by C t,s i,j .The progression is guided by the transition probabilities, which are visualized in Fig. 6 through the presentation of four probability mass functions.
The transition probability of the ith pair at time t and stage s can be described as follows: Figure 7 illustrates an example of how the probabilities are updated according to the fitness function values where it was initially supposed to be uniformly distributed, i.e., P 0,s i,1 C t=0,s=0 i,0 = 0.25 .When the fitness function improves, a reward in the form of a value (ε) is added to the transition probability associated with the corresponding direction.Simultaneously, a deduction of ε/3 is made from the transition probabilities of other directions.Conversely, if the fitness function fails to improve, a penalty is applied by subtracting ε from the transition probability of the relevant direction, while ε/3 is added to the transition probabilities of other directions.
Figure 8 clarifies the process of our approach, where each leaf F t,s j from A • M A leaves will be expanded to A leaves noted as follows: The selected best A • M A leaves, according to the fitness function, will be given new labels of F t+1,s j forj = 1toM A .At each stage s and iteration t, new leaves are identified by generating A independent uniform random variables, denoted α t,s j,i,r , for each leave j and each pair of features i.These variables are drawn from a uniform distribution between 0 and 1, with r = 1, …, A , as illustrated in Fig. 9.
(4) The transition pair from C t.s i,j to C t+1.s i,(j−1)A+r is controlled by the values of the random variable α t,s j,i,r as indicated by Eq. ( 8).
At every expansion, one of the four probability mass functions for each pair of features for each leave generated from the A • M A leaves must be updated after inheriting the transition matrices from the parent leaf.This process is illustrated in Figs. 7, 8 and 10.
A probability mass function (p.m.f) is a function that describes the probability distribution of a discrete random variable.The following are some of the properties of a p.m.f that need to be kept during the process of updating: {0, 0} if α t,s i,j,r < P t,s 0,i,(j−1)A+r {0, 1} if P t,s 0,i,(j−1)A+r ≤ α t,s i,j,r < P t,s 0,i,(j−1)A+r + P t,s 1,i,(j−1)A+r {1, 0} if P t,s 0,i,(j−1)A+r + P t,s 1,i,(j−1)A+r ≤ α t,s i,j,r < P t,s 0,i,(j−1)A+r + P t,s 1,i,(j−1)A+r + P t,s  www.nature.com/scientificreports/ • Non-negativity The p.m.f must be non-negative, meaning that it can take a value of 0, but it cannot be nega- tive.
• Non-exceeding 1 The p.m.f must be less than 1, meaning that it can take a value of 1, but it cannot be bigger.
• Normalization The sum of the p.m.f over all possible outcomes of the discrete random variable must equal 1, meaning that the probabilities of all outcomes add up to 100%.
Therefore, the procedure of probability of transition updating can be executed according to the following equation when the transition was performed from C t,s i,j = {0, 1} to C t,s i,1 = {1, 1} for instance.
where ε is the value given by the reward function, γ ∈ {+1, −1}, and The other three probability mass functions P t+1,s i,l (x|{0, 0}), P t+1,s i,l (x|{0, 1}), P t+1,s i,l (x|{1, 1}) are kept the same.The reward may be positive or negative depending on the evolution of the fitness function values from the leaf F t+1,s m to the leaf F t,s r , and it can be captured by the variable γ as follows: The reward function ε should be small variables depending on the progress of the fitness function, and dif- ferent functions can be proposed as follows: Or where η and τ are two parameters that can be any small values, refer to Fig. 11.
The process proceeds through stages until accuracy can no longer be improved or further dimension reduction is not possible.The next stage (s + 1) will evaluate the best features selected from the previous stage (s) as the root of the new stage (s + 1).The progression through stages is necessary when there is a progression in performance, as shown in Fig. 12.
As shown in Fig. 13, most transition probabilities will eventually converge to either 1 or 0, referred to as the equilibrium distribution, after a certain number of iterations determined by the reward value ε .At that point, it is necessary to reset the transition probabilities to 0.25 of the best leaf of the current stage as the root leaf for the next stage and repeat the branching process to see if higher accuracy can be achieved with fewer features.
(9) P t+1,s i,l The overall structure of each stage of the BEV approach is summarized in Fig. 14.

Results and discussion
This section plans to evaluate the proposed strategy by conducting experiments on a range of datasets that are commonly used for testing and comparison purposes.These datasets will serve as benchmarks to compare the performance with state-of-the-art methods and showcase the robustness of our technique.A thorough analysis of results, in terms of accuracy and size of selected features, will provide valuable insights into the strengths and weaknesses of our approach.
Datasets.The evaluation of the suggested method was conducted using 10 real-world high-dimensional datasets.These datasets are used to test the performance of the method in terms of feature selection and classification tasks.The datasets used in the evaluation of the suggested method are gene expression datasets with high dimensionality, meaning there are more features than observations.Additionally, the datasets are challenging because of the imbalanced distribution of observations across classes.Table 2 provides information on the number of observations, number of features, and other relevant details for these datasets.Experimental settings.This paragraph describes the process of evaluating the proposed strategy using tenfold cross-validation.To account for the limited number of samples in the datasets, the cross-validation technique is used to create the training and test sets (no validation set is used).One-fold is reserved as the test set and not used in the feature selection process, while the remaining nine folds are used for building the training data.The selected features are then used to update the training and test sets, which are fed into the KNN algorithm to evaluate their performance.To ensure a fair and comprehensive assessment, each dataset is subjected to ten independent tenfold cross-validation tests with different random seeds, resulting in 100 total runs for each dataset.This approach aligns with previous research and provides a current assessment of the state-of-the-art 31,46 .
Baseline methods.To demonstrate its effectiveness, the proposed work is compared with several existing feature selection algorithms that cover various techniques such as ant colony optimization, variable-length  Vol:.( 1234567890) Parameter' settings.Table 3 presents the parameters utilized in the proposed approach.The rest of the baseline methods compared are in line with those specified in prior studies 31,46 .
Results and discussion.Table 4 demonstrates the performance of the proposed methodology on 10 highdimensional real-world datasets.The comparison between the actual feature vector and the results of the proposed feature selection method is displayed for each dataset.The developed algorithm significantly improves classification accuracy and reduces the dimensionality of all datasets, as shown in Fig. 15.The graphical  comparison highlights the improvement in the performance of the proposed feature selection results compared to the original feature vectors.Table 5 provides a detailed analysis of the performance of the proposed algorithm, including the best, worst, and meaningful results.The dataset size reduction process is implemented iteratively until the accuracy and feature count remain consistent in three consecutive stages.During these initial stages, the dimensionality reduction is carried out without sacrificing precision.In the following three stages, the criteria for maintaining accuracy are relaxed, allowing for further reduction in dimensions with the possibility of fluctuating accuracy.Figures 16 and 17 summarize the results of 10 separate runs on all datasets using these additional stages.It can be seen that the number of features decreases as the stages progress.Initially, accuracy increases consistently, but in the last three stages, accuracy may decline as the feature count decreases.The results show that, while the balanced accuracy may vary among the same dataset experiments in the early stages, it eventually converges to a similar level in the later stages.
Additionally, Fig. 18 demonstrates that as the feature count decreases, the balanced accuracy for all datasets improves, highlighting the critical role of feature selection in attaining optimal accuracy and its potential for reducing the actual feature vector size.It is noteworthy that there is a trade-off between the number of features and accuracy, as reducing the feature vector size too much can result in decreased accuracy in most cases.
To showcase the versatility of our approach, we expanded our analysis by incorporating two additional classification models, Random Forest and Support Vector Machine (SVM), in addition to the KNN model.We conducted experiments on two datasets, 'brain tumor 1' and 'brain tumor 2' , to assess the accuracy of the BEV and Autoencoder algorithms.We evaluated and compared the performance of these algorithms by averaging the results obtained from 10 experiments.These datasets were intentionally selected as they offer potential for improvement beyond what the BEV algorithm achieves in terms of accuracy.The corresponding comparison is presented in Table 6.Details of the Autoencoder parameters used for these evaluations can be found in Table 7.
The results clearly demonstrate that our proposed algorithm outperforms the Autoencoder when employing different classification models on the aforementioned datasets.Notably, the BEV algorithm achieves optimal performance by selecting only 7 and 5 features for 'brain tumor 1' and 'brain tumor 2' , respectively, whereas the Autoencoder attains its best performance with 100 features on both datasets.After analyzing the results, we made several key observations.Firstly, the random forest classifier demonstrated the best performance when utilizing the autoencoder with 7 features.However, when employing 50 and 100 features, the KNN classifier outperformed other classification models.It is important to highlight that, despite the varying performance across different feature configurations, none of the results surpassed the accuracy and feature efficiency achieved by the BEV algorithm.
Furthermore, we emphasize that the BEV algorithm excels in extracting precise features, ensuring the preservation of the exact features present in the dataset.In contrast, the autoencoder learns compressed representations that may not directly align with the original features of the data.This distinction highlights the strength of the BEV algorithm in capturing relevant information from the dataset.9). Figure 19 highlights the superiority of our approach in comparison with the two best techniques TSHFS-ACO and ERM-FS in terms of accuracy.Table 10 compares the average number of selected features for various techniques.Despite a higher mean balanced accuracy, the proposed BEV approach results in a lower average number of selected features on 8 out of 10 datasets.This highlights the efficiency of the proposed BEV in identifying the optimal features while reducing dimensions.The TFSACO performed better in reducing dimensions on 2 out of 10 datasets.Table 11 presents a comparison of the average balanced accuracy and the average number of selected features of classical studies.It clearly shows that the proposed BEV approach outperforms all other techniques in overall performance.In conclusion, these results demonstrate the superiority of the proposed BEV for high-dimensional feature selection.
To assess the performance of the BEV algorithm, we conducted an evaluation with recall which is an important metric in addition to accuracy.We compared the results of the BEV algorithm with the ERM-FS algorithm, which achieved the second-highest accuracy after our proposed algorithm, as shown in Table 11.The evaluation was performed using 'macro' recall since our scenario involved multiple classes.The results for both algorithms can be found in Table 5.It is important to note that 'macro' recall was utilized to ensure a comprehensive evaluation in our multi-class setting.
Table 12 demonstrates that the BEV algorithm consistently surpasses the ERM-FS algorithm in terms of macro recall across the various datasets.This finding highlights the superior performance of the BEV algorithm in accurately capturing important information from the data.In fact, the BEV algorithm achieves a perfect macro recall score of 100% on the majority of the datasets, further emphasizing its effectiveness.However, it is important to mention that in the case of 11 Tumor, Brain Tumor 1, and 9 Tumor datasets, the BEV algorithm exhibits a comparatively lower macro recall of 86.3%, 80%, and 66.6% respectively, indicating an area with potential for improvement.• n) , assuming the number of neighbors ( k ) remains constant.Table 4 provides the computational time needed for the different algorithms.The algorithms were executed on an Intel Core i7-4770 CPU @3.4 GHz.
According to Table 13, the proposed algorithm is positioned as the third fastest in terms of average computation time across all datasets.It is noteworthy that VLPSO exhibits the highest speed, followed by ERM-FS.However, it is important to emphasize that although VLPSO excels in computational efficiency, it does not rank among the top algorithms in terms of accuracy.Conversely, the proposed algorithm demonstrates slightly slower computation time compared to ERM-FS, but it achieves significantly better accuracy performance while utilizing a reduced number of features.

Conclusion
The proposed Bird's Eye View (BEV) feature selection approach offers a solution to the challenge of selecting features in high-dimensional datasets.It combines three different paradigms and employs a rewarding scheme and collective evolution with Markov impact to iteratively reduce the feature space.The BEV algorithm draws inspiration from the genetic algorithm mechanism and implements a smart branching evolution approach that relies on dynamic Markov chains.The algorithm begins by initializing a root leaf and proceeds to generate children leaves, where the number of generated leaves is determined by a predetermined fixed value.Each leaf is represented by a sequence of 1 s and 0 s, organized in pairs.The best leaves are selected for each expansion based on evaluation.This iterative process continues until no further improvement is observed.The BEV algorithm effectively distinguishes between different classes by utilizing a reward and penalty mechanism to update transition probabilities during state transitions.This mechanism is based on the improvement or lack thereof in the fitness function.As a result, the algorithm achieves a significantly reduced feature subset while preserving high classification performance.The effectiveness of the proposed BEV approach in high-dimensional feature selection is demonstrated by its ability to generate a significantly reduced feature subset while maintaining a high fitness level.Through evaluation on 10 benchmark datasets, the BEV model outperforms current state-of-the-art methods.Furthermore, it offers advantages such as simplicity in development, ease of hyperparameter configuration, and fast execution.
However, it is important to note that our approach is a stochastic algorithm, which means it provides suboptimal solutions rather than guaranteed optimal solutions.Despite effectively exploring the search space, there is no guarantee that the selected feature subset will be the absolute best.Achieving satisfactory performance in the proposed approach depends heavily on fine-tuning various hyperparameters.One avenue for future research involves exploring the tuning of additional hyperparameters to enhance the algorithm's performance.Additionally, we plan to investigate the inclusion of sets of k-features, as opposed to limiting the selection to only two features.This modification aims to assess whether expanding the feature selection scope can further improve the approach's performance.

Figure 1 .
Figure 1.Eagle at a high altitude seeking the optimum way.

Figure 2 .
Figure 2. Process implementation in recursive levels.The process explains how the search space in an upcoming stage is reduced by considering only the best-performing features from the previous stage.We select or omit the specified features by assigning a 1 or 0 to each feature position.

Figure 3 .
Figure 3.The features are gathered two by two in the leaf F t,s j .

Figure 5 .
Figure 5. Process of expanding tree when A = 3, A • M A = 9.

Figure 7 .
Figure 7. Probability updating mechanism based on the rewarding scheme by adding ε to the appropriate direction as a reward and subtracting ε /3 to the other direction if the fitness function was improved and vice versa.

Figure 8 .
Figure 8. Expanding of the leave F t,s j to A different children leaves.

Figure 9 .Figure 10 .
Figure 9. Expanding of the leaf F t,s j to A = 3 different children leaf where α t,s i,j,k are generated from an independent identically uniform distribution between [0,1] to define new pairs in other leaves according to their p.m.f.

Figure 13 .
Figure13.The evaluation of the transition probability of the best pair of features to determine when the equilibrium distribution will be attained.

Figure 15 .
Figure 15.Performance comparison with the original feature vector of different datasets.(a) Performance in terms of dimensionality reduction.(b) Performance in terms of classification accuracy.

Figure 17 .
Figure 17.Accuracy of the proposed algorithm on ten different datasets over ten independent runs.The graphs show the improving classification accuracy with recursive stages.

Figure 18 .
Figure 18.The performance regarding the number of features versus Balanced accuracy.The representation demonstrates the effectiveness of reducing dimensionality over all ten datasets.

class 1 Actual class 2 Actual class 3
In this study, a new metric is proposed to better monitor the performance of classifiers.Our new metric will accurately measure the accuracy of each class and is suitable for use in feature selection.Therefore, this metric can be used as a fitness function in our search algorithm 52rticle swarm optimization, comprehensive learning PSO with adaptive learning probability, and correlationbased feature selection.The comparison includes evolutionary models (TSHFS-ACO (two-stage hybrid feature selection model based on ant colony optimization)31, IRRF-SACO (Relevance-redundancy feature selection based on ant colony optimization)47), particle swarm optimization [Standard PSO, VL-PSO (Variable-Length Particle Swarm Optimization)46, CLPSO (Comprehensive Learning PSO) enhanced with the adaptive learning probability48, and CSO (Competitive Swarm Optimizer)49], graph-based [TFSACO (Text feature selection using ACO)50], and classical methods [LFS (linear forward selection), CFS (correlation-based feature selection)51, and FCBF (fast correlation-based feature selection)52].

Table 2 .
Details of datasets.

Table 4 .
Results on different datasets compared to the full feature set.

Table 5 .
Best, worst, and mean results on different datasets by the proposed algorithm.

Table 9
presents the results of the proposed methodology against state-of-the-art approaches in terms of balanced classification accuracy.The proposed BEV method outperforms current state-of-the-art techniques, including the two best methods TSHFS-ACO and ERM-FS, in balanced classification accuracy.BEV achieved an average improvement of 9.21% and 4.23% over TSHFS-ACO and ERM-FS, respectively.The largest improvement was observed in the Brain Tumor 2 dataset, with 8.77% and 21.92% over ERM-FS and TSHFS-ACO.The second largest improvement was seen in Brain Tumor 1 dataset, with 5.74%

Table 6 .
Accuracy comparison of BEV and Autoencoder algorithms on two datasets: 'brain tumor 1' and 'brain tumor 2' based on the average of 10 experiments.Significant values are in bold.

Table 8 .
Autoencoder performance on different desired features on two datasets i.e., 'brain tumor 1' and 'brain tumor 2' based on AVG of 10 experiments.Significant values are in bold.

Table 9 .
Comparison in terms of average balanced accuracy with existing studies in 100 feature selection runs (mean ± std).Significant values are in bold.
Figure 19.Comparison of proposed BEV with best performing TSHFS-ACO and ERM-FS in terms of Mean Balanced Accuracy (%) on all the datasets.The datasets are ranked in numbers from the highest dimensions to the lowest dimensions.

Table 10 .
Comparison in terms of average number of features selected with existing studies in 100 feature selection runs.Significant values are in bold.

Table 11 .
Comparison in terms of average balanced accuracy and number of selected features with classical studies in 100 feature selection runs.Significant values are in bold.The BEV algorithm utilizes the KNN model as its classification model.During training, the time complexity of the KNN model is O(1) , indicating that it does not depend on the size or dimensionality of the dataset.However, during prediction, the time complexity becomes O(k• n • d) ,where k represents the number of neighbors, n denotes the number of samples/points in the data, and d represents the dimensionality of the dataset.It's important to note that the time required for distance calculations is typically insignificant compared to other algorithmic steps.The performance of the BEV algorithm is primarily affected by the dimensionality of the dataset.As the dimensionality increases, the computational time also increases.Consequently, the overall time complexity of the BEV algorithm can be expressed as O(d 2 Vol.:(0123456789) Scientific Reports | (2023) 13:13303 | https://doi.org/10.1038/s41598-023-39790-3www.nature.com/scientificreports/Algorithm complexity.

Table 12 .
Macro recall comparison of ERM-FS and BEV algorithms on multiple datasets.Significant values are in bold.

Table 13 .
Computational time comparison of various algorithms.