Evidence-based recommender system for high-entropy alloys

Existing data-driven approaches for exploring high-entropy alloys (HEAs) face three challenges: numerous element-combination candidates, designing appropriate descriptors, and limited and biased existing data. To overcome these issues, here we show the development of an evidence-based material recommender system (ERS) that adopts Dempster–Shafer theory, a general framework for reasoning with uncertainty. Herein, without using material descriptors, we model, collect and combine pieces of evidence from data about the HEA phase existence of alloys. To evaluate the ERS, we compared its HEA-recommendation capability with those of matrix-factorization- and supervised-learning-based recommender systems on four widely known datasets of up-to-five-component alloys. The k-fold cross-validation on the datasets suggests that the ERS outperforms all competitors. Furthermore, the ERS shows good extrapolation capabilities in recommending quaternary and quinary HEAs. We experimentally validated the most strongly recommended Fe–Co-based magnetic HEA (namely, FeCoMnNi) and confirmed that its thin film shows a body-centered cubic structure.


I. ILLUSTRATIVE EXAMPLES
The following examples provide explanations of how the evidence theory work to learn the similarity and infer the HEA formation for new element combinations, identifying equiatomic alloys.
We consider each pair as a source of evidence support that {D} is similar to {E} in term of substitutability to form the HEA phase.Each evidence is modeled using mass function as ({similar, dissimilar}) = 0.9 The three pieces of evidence are combined using the Dempster' rule of combination to accumulate the believe that {D} is similar to {E}: Next, if we observed (included in the data) that the HEA phase exists for alloy {G, H, I, D}, the ERS (which focuses on finding some chance for discovering new combination of elements that the HEA phase exist and ignores the belief regarding ¬HEA) will consider that there is some believe that the HEA phase also exists for {G, H, I, E} (by substituting {D} with {E}).The evidence is modeled using mass function as follows: Consequently, if we observed (included in the data) that the HEA phase exists for {G, H, I, C}, the algorithm (which focuses on finding some chance for discovering new combination of elements that the HEA phase exist and ignores the belief regarding ¬HEA) will consider that there is some believe that the HEA phase also exists for {G, H, I, D, E} (by substituting {C} with {D, E}).

II. ALLOYS DATA SETS
In the evaluation experiments, we use eight data sets consisting of binary, ternary, quaternary, and quinary alloys comprising multiple equiatomically combined elements.The data sets consist of data from experiments and calculations.In this section, we will follow Ref.
35 to describe the data sets.The alloys contained in the data sets comprise E = { Fe, Co, Ir, Cu, Ni, Pt, Pd, Rh, Au, Ag, Ru, Os, Si, As, Al, Tc, Re, Mn, Ta, Ti, W, Mo, Cr, V, Hf, Nb, and Zr}.Supplementary Figure 2 shows the proportion of 27 elements in the data sets.
Any alloy contained in the following data sets is considered as an HEA if its order-disorder transition temperature is below its melting temperature.

III. DIFFERENCES BETWEEN SIMILARITY MATRICES LEARNED FROM D CALPHAD AND D AFLOW
There are some notable differences between these results obtained from experiments with D CALPHAD and D AFLOW .The similarity matrix learned from D AFLOW shows that Au and Ag are very similar (Supplementary Figure 3 b).Furthermore, both are similar to V, Mn, and Al but not to other late transition metals (Supplementary Figure 3 a).Mn is also similar to Tc, Re, and Cr but not to the other early transition metals.However, Tc and Re are somewhat similar to the other early transition metals.Furthermore, Zr is somewhat similar to the late transition metals, but different from the early transition metals.Clearly, these results are different from that obtained from D CALPHAD owing to the difference between the predicted label (HEA or ¬HEA) for the Zr-containing alloys recommended based on CALPHAD and AFLOW calculations, as listed in Supplementary Table 1.Al, Si, and As are all similar to each other and to Fe and Co (Supplementary Figure 3 a).However, Al is similar to V, Cr, and Mn but not to Ti, whereas Si and As are very similar to Ti but not to V or Cr.

IV. MONITORING HEA RECALL RATIOS IN TEST SET A. Evaluation of HEA-recommendation capability by cross-validation
In the experiment with D ASMI16 , the result shows that the ERS can significantly reduce the number of trials required to recall all the HEAs in the test set compared to the competitor systems (Supplementary Figure 4 a).The proposed ERS requires less than 12, 25, and 80% of all the possible trials to recall one-half, three-quarters, and all the HEAs in the test set, respectively (Supplementary Table 2).In the D CALPHAD experiment, the ERS requires less than 2 and 5% of all the possible trials to recall one-half and three-quarters of the HEAs in the test set, respectively, which are the fewest trials required among all the recommender systems (Supplementary Figure 4 b and Supplementary Table 2).Interestingly, in the D ASMI16 and D CALPHAD experiments, the supervised-method-based recommender systems either approximately randomly selected possible HEAs (Naïve Bayes and decision tree) or could not rank any (logistic regression and SVM) at all because these data sets contain only positively labeled HEAs.
The result in D AFLOW experiment demonstrates that the ERS also outperforms the competitor systems in recalling one-half of the HEAs in the test set.However, the ERS cannot reliably recall the one-quarter of the HEAs remaining in the test set because not enough evidence is available in the training data to make inferences about the remaining HEAs (Supplementary Figure 4 c and Supplementary Table 2).The D LTVC and D AFLOW experimental results are identical(Supplementary Figure 4 d).Although the ERS performs better than the other recommendation systems in recovering one-half of the test HEAs in the D LTVC data set (requiring only less than 3% of the number of possible trials), it cannot reliably recover the remaining one-quarter of the test HEAs owing to the lack of evidence in the training data (Supplementary Table 2).

B. Evaluation of HEA-recommendation capability by extrapolation
In the D quaternary AFLOW experiment, the ERS performs significantly better than the NMF-based recommender system, requiring less than 5 and 19% of the total number of possible HEA candidates to recall 50 and 75% of the HEAs in the test set, respectively (Supplementary

•
D ASMI16 : The order-disorder transition temperatures (T exp c ) and melting temperatures (T exp m ) of the alloys are both experimentally evaluated 1 .All of the alloys contained in D ASMI16 show an order-disorder transition temperature below their melting temperature (T exp c < T exp m ).• D CALPHAD : The order-disorder transition temperatures (T c ) and melting temperatures (T m ) of the alloys are both predicted using calculated-phase-diagram (CALPHAD) calculations 2-4 based on the temperatures for some binary alloys (three possible for each ternary alloy) found in the Thermo-Calc software SSOL5 database 5 .Similar to the D ASMI16 data set, the D CALPHAD data set only contains the alloys satisfying T c < T m .• D AFLOW , D quaternary AFLOW , and D quinary AFLOW : The order-disorder transition temperatures (T AFLOW c ) of the alloys contained in these data sets are estimated using the automatic flow (AFLOW) convex-hull database 6 .The melting temperatures T exp m and T m are applied to the binary and ternary alloys, respectively.The alloy is considered as an HEA if T AFLOW c < T exp m for binary alloys and T AFLOW c < T m for ternary, quaternary, and quinary alloys).• D LTVC , D quaternary LTVC , and D quinary LTVC : These data sets contain the same alloys as those contained in data sets D AFLOW , D quaternary AFLOW , and D quinary AFLOW , respectively.However, the properties of the alloys contained in these data sets are predicted using the method of Lederer, Toher, Vecchio, and Curtarolo (LTVC) 7 .Ab-initio calculations are used to estimate the order-disorder transition temperatures (T LTVC c ) of the alloys contained in these data sets.In addition, the T m values are the same as those of the alloys contained in the AFLOW data sets.Any alloy in these data sets is predicted as an HEA if T LTVC c < T exp m for binary alloys and T LTVC c < T m for ternary, quaternary, and quinary alloys Note that D ASMI16 and D CALPHAD only contain confirmed and predicted HEAs, respectively.Therefore, although we assume that the properties of all the other binary or ternary alloys (not included in the data set) have not yet been confirmed, we do not assume that those alloys are not HEAs.

Table 3 )
. In the D quaternary LTVCexperiment, the ERS and competitor matrix-based system devel-