Hybrid selection based multi/many-objective evolutionary algorithm

In the last decade, numerous multi/many-objective evolutionary algorithms (MOEAs) have been proposed to handle multi/many-objective problems (MOPs) with challenges such as discontinuous Pareto Front (PF), degenerate PF, etc. MOEAs in the literature can be broadly divided into three categories based on the selection strategy employed such as dominance, decomposition, and indicator-based MOEAs. Each category of MOEAs have their advantages and disadvantages when solving MOPs with diverse characteristics. In this work, we propose a Hybrid Selection based MOEA, referred to as HS-MOEA, which is a simple yet effective hybridization of dominance, decomposition and indicator-based concepts. In other words, we propose a new environmental selection strategy where the Pareto-dominance, reference vectors and an indicator are combined to effectively balance the diversity and convergence properties of MOEA during the evolution. The superior performance of HS-MOEA compared to the state-of-the-art MOEAs is demonstrated through experimental simulations on DTLZ and WFG test suites with up to 10 objectives.

www.nature.com/scientificreports/ Unlike PMOEAs that compare individuals using two criteria (i.e., dominance relation and density), Indicatorbased MOEAs (IMOEAs) 12 adopt a single value referred to as an indicator to measure both convergence and diversity (IBEA 12 , I SDE +13 ). However, developing an indicator that balances both diversity and convergence is challenging. The ability of the indicator to balance the convergence and diversity gets challenging as the number of objectives increases. Some indicators 12 are biased towards convergence, while some favour diversity 14 . Therefore, in 14 , a stochastic combination of both convergence-and diversity-biased indicators are considered. IMOEA based on hypervolume 15 is effective but computationally expensive. I SDE + is computationally efficient but fails to preserve the essential corner solutions.
In Reference Vector-based MOEAs (RV-MOEAs) 16,17 , the population members are guided towards the optimal Pareto Front (PF) in the direction specified by the weight or reference vectors. In general, the reference vectors are selected by sampling a uniform set of points on a hyperplane i f i = 1 in the normalized M-objective space referred to as Normal Boundary Intersection (NBI) method 18 . In other words, it is implicitly assumed that the optimal PF is bounded by a unit simplex of reference vectors that is non-degenerate, continuous, and smooth without significant nonlinearities. However, there exist several MOPs characterized by degenerate and discontinuous PFs. As a result, several of the uniform weight vectors fail to get associated with any of the solutions and are referred to as ineffective weight vectors. In addition, it has been observed that the number of non-dominated solutions obtained by MOEA/D 19 , a primitive RV-MOEA, is often much smaller than the number of weight vectors as-(1) multiple weight vectors can share a single good solution, and (2) all solutions are not always non-dominated. Therefore, the Pareto dominance criterion has been integrated into RV-MOEAs (NSGA-III 11,20 , RVEA 21 , MOEA/DD 22 , TDEA 8 , PMEA 23 ). On the other hand, a set of uniform weight vectors may not be able to approximate the different sizes and shapes of PFs. In other words, the initialization of the weight vectors should depend on the shape and size of the PF, which may not be known in advance. Therefore, RV-MOEAs [24][25][26][27] with weight vector adaptation during the evolution were proposed to effectively handle MOPs with regular as well as irregular PFs. Instead of adapting the weight vectors, a combination of uniform weight vectors and a secondary criterion (e.g., polar-metric 23 ) in to select the solutions corresponding to the ineffective weight vectors have also been investigated. Motivated by the work in 23 , we propose a hybridized framework, referred to as HS-MOEA, that employs I SDE + as the secondary criterion to select solutions corresponding to the ineffective weight vectors, in addition to the Pareto dominance. In other words, the aim of this study is to develop a new environmental selection strategy that benefits from the advantages of Pareto-, decomposition-and indicator-based approaches. First, Pareto dominance alleviates the selection of dominated solutions. Second, weight vectors assist in the selection of well-diversified and convergent solutions in each generation. Third, if the weight vector fail to differentiate the high-quality parent solutions then the indicator assists the selection process by considering both convergence and diversity. The ability of I SDE + to select a set of converged and diverse solutions from unselected ones with respect to a set of already selected solutions is expected to aid the uniform weight vectors in achieving better convergence and diversity.
The rest of this paper is organized as follows. The second section presents the preliminaries. The third section introduces related work and motivation for the current study. The fourth section contains details of HS-MOEA. The fifth section presents experimental setup and comparison results of HS-MOEA with a number of state-ofthe-art MOEAs. The last section presents the conclusions and future directions.

Preliminaries
Generally, MOP is formulated as: where x represents an D dimensional decision vector in , and M is the number of objectives.
In multi-objective optimization, the following concepts have been well defined and widely applied.

Pareto Dominance:
For any two solutions x and y , x is said to dominate y, denoted as . , M} and f j (x) < f j (y) for at least one j ∈ {1, 2, . . . , M}.

Pareto Optimality:
A solution x * is said to be Pareto-optimal if there is no other solution x ∈ such that x ≺ x * . 3. Pareto-optimal Set (PS): It is the set of all Pareto-optimal solutions and is defined as PS = {x ∈ |xisParetooptimal}. 4. Pareto-optimal Front (PF): It is the set of all Pareto-optimal solutions and is defined as PF = f (x) ∈ R M x ∈ PS}.

Ideal point:
Ideal point is a vector z * = (z * 1 , z * 2 , . . . , z * M ) , which is the infimum of f i for each i ∈ {1, 2, . . . , M} in the PF. 6. Nadir point: Nadir point is a vector z nad = (z nad 1 , z nad 2 , . . . , z nad M ) , which is the supremum of f i for each i ∈ {1, 2, . . . , M} in the PF. 7. Weight vector: A weight vector w is a M dimensional vector w = (w 1 , w 2 , . . . , w M ) such that M i=1 w i = 1 and w i ≥ 0∀i ∈ {1, 2, . . . , M} . The Normal Boundary Intersection (NBI) method is a systematic approach that places points on a normalized hyper-plane, i.e., on a (M − 1)-dimensional unit simplex. It generates H+M−1 M−1 number of finite weights where M is the number of objective of problem and H is the number of divisions where � · � denotes L 2 norm and θ is a penalty parameter.

Related work and motivation
In RV-MOEAs, the objective vector corresponding to each solution is converted into a scalar value based on a series of uniformly distributed weight vectors. To maintain population diversity during the evolution, RV-MOEAs assign the same search space preference to each direction vector. However, the performance of RV-MOEAs strongly depends on the shape of the PF 28 , which is not known in advance. In addition, the size and shape of PF vary over generators. Therefore, it is essential to adapt the weight vectors during the evolution process or employ a secondary selection criterion to aid the uniform weight vectors. In the literature, attempts have been made to improve the performance of RV-MOEAs on MaOPs with both the regular and irregular PFs. In NSGA-III 11 , significant changes to the selection operator were performed compared to its predecessor NSGA-II, where the diversity promotion among population members is achieved by a set of well-spread reference points. The employment of reference vectors improves the scalability of the algorithm by reducing the computational complexity that arises due to the increase in the number of objectives. NSGA-III is further modified (referred to as A-NSGA-III 20 ) where ineffective reference points are re-allocated based on the distribution and association of the solution. Ineffective reference vectors are the ones that do not have any population members associated with them. RVEA 21 employs a scalarization approach, termed as Angle Penalized Distance, that assesses convergence by calculating the distance between the candidate solution and the weight vector. In 27 , a weight vector adaptation strategy was employed to enhance the performance of RVEA. TDEA 8 enhances the convergence of NSGA-III in high dimensional objective spaces by-(1) incorporating a new dominance scheme and (2) employing the aggregation function-based fitness evaluation scheme of MOEA/D. In Polar Metric based Evolutionary Algorithm (PMEA) 23 , a metric inspired from the PBI operator referred to as polar-metric (p-metric) is proposed to measure the convergence and diversity. During environmental selection, a weight vector adjustment strategy is employed to select the well-diversified solutions. The environmental selection of PMEA is demonstrated in Fig. 1 where x i and w i represent the solutions and weight vectors, respectively. The values on the perpendicular lines represent the p-metric values of the solutions to the corresponding weight vectors. As in Fig. 1a, PMEA assigns x 1 , x 4 , x 5 to w 2 , w 3 , w 1 , respectively and selects for future evolution. Solutions x 2 and x 3 are not selected. In addition, w 4 does not have any associated solutions and is considered as ineffective. The ineffective weight vector ( w 4 ) is re-initialized ( w 5 ) to pass through the nearest non-selected solution (x 3 ) as shown in Fig. 1b. After the re-initialization, solution x 3 associated with weight vector w 5 is selected. In other words, starting with uniform weight vectors, at first, PMEA selects solutions based on p-metric. Later, a weight vector adaptation is made to select the rest of the solutions necessary to create the population for the next generation. However, during the next generation, the weight vectors are uniformly initialized before evaluating the p-metric. From Fig. 1b, it can be noticed that the weight vectors are not well diversified can affect the population www.nature.com/scientificreports/ diversity during the evolution process. In other words, adapting weights to select the solution in each generation based on the distribution of population is not appropriate. Motivated by these observations, we propose to combine the advantages of Pareto, decomposition, and indicator methods in a single framework. In the proposed framework, the uniform weight vectors are assisted by I SDE + indicator, which serves as a secondary criterion.

Hybrid selection based multi/many-objective evolutionary algorithm (HS-MOEA)
In this section, Hybrid Selection based Multi/Many-Objective Evolutionary Algorithm (HS-MOEA) is introduced. The different steps of HS-MOEA are detailed below.

Initialization.
A set of uniform weight vectors ( w ) are generated using the NBI method, then subsequently, a population of size N ( |w| ) is initialized within the permissible boundaries as shown in Line 01 of Algorithm 1.

Mating selection and offspring generation. I SDE + indicator values corresponding to each solution in
the population (P t ), corresponding to the tth iteration is evaluated using Eq. (3) (Line 04, Algorithm 1). The solutions with the highest I SDE + values are considered to be better. Using I SDE + values, binary tournament selection is performed to generate the mating pool (Line 05, Algorithm 1). Then, the offspring population is generated using the variation operators such as polynomial mutation 6 and simulated binary crossover 6 (Line 06, Algorithm 1).
In HS-MOEA, the mating selection is performed using I SDE + indicator. The indicator values corresponding to each solution in the population is evaluated using Eq. (3) (Line 04, Algorithm 1).
where P SB(x) ∈ P t and y ∈ P SB (x) , such that SB y < SB(x). SB represent the sum of normalized objectives. N SB(x) represents the size of P SB(x) .
The solutions with the highest I SDE + values are considered to be better. Then, the binary tournament selection is performed based on I SDE + values to generate the matting pool (Line 05, Algorithm 1). After the mating selection, the offspring population is generated using the variation operators such as mutation and crossover, as shown in Line 06, Algorithm 1. In the current, the mutation and crossover operators employed are Polynomial Mutation (PM) 2 and Simulated Binary Crossover (SBX) 2 .
Normalization. Normalization (Line 07, Algorithm 1) is an essential tool to map the unscaled search space to scaled one so as to characterize the badly scaled objectives. In HS-MOEA, the normalization (of the j th population member is given in Eq. (5).
where, z * i and z nad i are considered as the lowest and highest values of i th objective function. www.nature.com/scientificreports/ Environmental Selection. The environmental selection selects a set of N converged but diversified solutions from a combined population (R) of size 2N (Line 08, Algorithm 1). The working mechanism is detailed in Algorithm 2. Non-dominated sorting 6 (Lines 01 ~ 02, Algorithm 2) is performed to classify the population R into several fronts (Fr) and identify the population P ND = l=1:L Fr l (where L satisfies l=1:L Fr l ≥ N and l=1:L−1 Fr l < N).
Association. In HS-MOEA, the association procedure (Line 04, Algorithm 2) is performed in the normalized objective space, where the ideal point z * is shifted to origin. At each generation, the individuals of the P ND t population is associated with the reference vectors ( w) . For the association operator, the norm of each solution x in P ND t is evaluated as: Then, the angle between F(x) and w i is defined as: . During association, each solution is assigned to its closest reference vector. K i is the number of solutions associated with the weight vector w i during the association process, which can range from 0 to N . Figure 2 represents the association operator, where the filled circles are the associated solutions with the corresponding nearest weight vector.
between the origin and the projected point of x over w , θ = penalty parameter. PBI refers to the penalty-based boundary intersection 16 . First, select one solution from each non-empty K i ∀i = 1, 2, . . . , N based on PBI value and save them in S (referred to as a set of selected solutions). The remaining solutions are stored in U (referred to as a set of unselected solutions). If the size of S is N then the whole set S is declared as a parent population of the next generation (Line 06, Algorithm 2); otherwise, go for the second round of selection. In the second round, (N − |S|) solutions are to be selected from U using I SDE + indicator. For each x ∈ U the values of I SDE + referred to as I U SDE + is calculated (Line 08, Algorithm 2). To evaluate the indicator, the solutions in U are sorted in the ascending order of the normalized sum of objectives (SB) (Line 09, Algorithm 2). The solution with the least SB is assigned the highest possible indicator value of one. To evaluate the I U SDE + of a given solution x ∈ U , the solutions in U that are better in convergence with the least SB compared to x and solutions in set S are shifted as in Eq. (8). Then (N − |S|) solutions from U with the largest I U SDE + are selected (P 1 ) (Line 10, Algorithm 2) and added to S (Line 11, Algorithm 2), which becomes the population (P) for the next generation.
where A is ( U SB (x) + S) . U SB (x) ∈ U and y ∈ A. For all y ∈ U SB (x) such that SB y < SB(x) , S and U represent a set of selected and unselected solutions respectively by weight vector association.
In other words, the use of I SDE + indicator enables the selection of converged, yet diverse solutions with respect to the already selected solutions (S). First, Pareto dominance alleviates the selection of dominated solutions. Second, weight vectors assist in the selection of well-diversified and convergent solutions in each generation. Third, if the weight vector fails to differentiate the high-quality parent solutions, then the indicator assists the selection process by considering both convergence and diversity. The advantage of employing I SDE + is that it enables the selection of the solutions considering the solutions that are already selected through weight vector association in the second step. In other words, HS-MOEA gets benefitted from both the reference vectors and indicators.

Experimental setup, results and discussion
Experiments were conducted on 16 scalable test problems from DTLZ 29 and WFG 30 , test suites comprising of 7 and 9 problems, respectively. For each test problem, 2-, 4-, 6-, 8-and 10-objectives are considered. The parameter values employed are present in Table 1. On each instance, 30 independent runs were performed for each algorithm on a PC with a 3.30 GHz Intel (R) Core (TM) i7-8700 CPU and Windows 10 Pro 64-bit operating system with 16 GB RAM. As a stopping criterion, the maximum number of generations for DTLZ1 and WFG2 is set to 700 and for DTLZ3 and WFG1 it is set as 1000.  Table 2. In addition, the statistical tests (t-test) at a 5% significance level were conducted to compare the significance of the difference between the mean metric values yielded by HS-MOEA and state-of-the-art algorithms. The signs " + ", "−" and "≈" against the HV values indicate that the HS-MOEA is statistically "better", "worse" and "comparable" with the corresponding algorithm, respectively. The last row of Table 2 represents the overall performance of HS-MOEA in terms of the number of instances it is better (Win-W), comparable (Tie-T) and worst (Loss-L) with respect to the corresponding algorithm. As shown in Table 2 and Fig. 3 and WFG2 test problems. However, the improvements seems to be minimal compared to the advantages HS-MOEA achieves on other problems such as DTLZ5, DTLZ7, WFG3 ~ 9. MOEA/D and MOEA/DD seem to perform similarly compared to HS-MOEA, performing slightly better on DTLZ1. On WFG3, MOEA/D performs better than all the state-of-the-art algorithms, including HS-MOEA. However, the degraded performance of MOEA/D on the remaining 15 problems seems to outweigh the superior performance on WFG3.
Among the state-of-the-art algorithms, I SDE + exhibits competitive performance compared to HS-MOEA. The superiority of I SDE + compared to HS-MOEA can be seen on DTLZ1, WFG1, WFG6 and WFG9. The performance improvement is significant; however, HS-MOEA is also close. HS-MOEA has a slight advantage over I SDE + on problems such as DTLZ7 and WFG2 that have disconnected PF.
To demonstrate the effectiveness of HS-MOEA, a more detailed analyse corresponding to DTLZ1 and DTLZ7 is presented. GD and Delta indicators that indicate the convergence and diversity performance of MOEAs are summarized in Tables 3 and 4, respectively. Lower values of both GD and Delta values indicate the superiority of the algorithm. The convergence (GD) of HS-MOEA is consistently better than PMEA. However, the convergence of HS -MOEA lags behind I SDE + on DTLZ1, which was designed to test the convergence performance of MOEAs. However, the diversity (Delta) of HS -MOEA is consistently better than I SDE + . On the other hand, HS -MOEA fails to maintain the diversity with respect to PMEA on DLTZ7, which has discontinuous PF. In other words, the convergence of HS-MOEA is better or comparable to PMEA, while the diversity is better or comparable to I SDE + . The improved performance of HS-MOEA is because it gets benefitted from both the best qualities of each component-(1) Pareto dominance's ability to eliminate low-quality solutions, (2) Uniform weight vectors maintain the diversity, and (3) I SDE + indicator enable both convergence and diversity in problems with MOPs with irregular or discontinuous PFs. Therefore, the performance of HS-MOEA is competitive or better in most cases. The significance is more visible in problems with discontinuous PFs such as DTLZ7. Figures 4 and 5 present the parallel coordinates that describe the distribution of the solutions corresponding to PMEA, I SDE + and HS-MOEA on 8-objective instances of DTLZ1 and DTLZ7. From the figures, it is evident that HS-MOEA is able to provide a well converged and diverse set of solutions compared to PMEA on both the DTLZ1 and DTLZ7 instances. However, the parallel coordinates of I SDE + and proposed HS-MOEA seem nearly identical on both DTLZ1 and DTLZ7. From the results in Table 4, it is evident that I SDE + slightly outperforms HS-MOEA on DTLZ1, which has continuous linear PF, while HS-MOEA performs better on DTLZ7 that has discontinuities in the PF.

Conclusion
This paper proposes a Hybrid Selection based Multi/Many-objective optimization, named HS-MOEA. In HS-MOEA, a new environmental selection that benefits from the advantages of Pareto dominance, reference vectors and an indicator is proposed. HS-MOEA is compared with seven state-of-the-art MOEAs on a number of widely used test instances. Experimental results demonstrate the superiority of HS-MOEA among all compared algorithms, mainly on problems with discontinuous PFs such as DTLZ7. In the future, we would like to investigate the possibility of weight vector adaptation using the I SDE + indicator. In other words, new positions of the ineffective weight vectors and the consequent adjustment of the effective weight vectors can be estimated by employing the indicator values.    Table 2. Comparison of HV and statistical results on DTLZ and WFG test problems ("+"-win, "≈"-TIE, "−"-loss).