key: cord-0692556-omv13qki
authors: Hou, Shuai; Li, Yujiao; Bai, Meijuan; Sun, Mengyue; Liu, Weiwei; Wang, Chao; Tetik, Halil; Lin, Dong
title: Phase Prediction of High-Entropy Alloys by Integrating Criterion and Machine Learning Recommendation Method
date: 2022-05-05
journal: Materials (Basel)
DOI: 10.3390/ma15093321
sha: b33b1e2f1deda55be13ad4f77ccd7cc4956131fc
doc_id: 692556
cord_uid: omv13qki

The comprehensive properties of high-entropy alloys (HEAs) are highly-dependent on their phases. Although a large number of machine learning (ML) algorithms has been successfully applied to the phase prediction of HEAs, the accuracies among different ML algorithms based on the same dataset vary significantly. Therefore, selection of an efficient ML algorithm would significantly reduce the number and cost of the experiments. In this work, phase prediction of HEAs (PPH) is proposed by integrating criterion and machine learning recommendation method (MLRM). First, a meta-knowledge table based on characteristics of HEAs and performance of candidate algorithms is established, and meta-learning based on the meta-knowledge table is adopted to recommend an algorithm with desirable accuracy. Secondly, an MLRM based on improved meta-learning is engineered to recommend a more desirable algorithm for phase prediction. Finally, considering poor interpretability and generalization of single ML algorithms, a PPH combining the advantages of MLRM and criterion is proposed to improve the accuracy of phase prediction. The PPH is validated by 902 samples from 12 datasets, including 405 quinary HEAs, 359 senary HEAs, and 138 septenary HEAs. The experimental results shows that the PPH achieves performance than the traditional meta-learning method. The average prediction accuracy of PPH in all, quinary, senary, and septenary HEAs is 91.6%, 94.3%, 93.1%, and 95.8%, respectively.

High-entropy alloys are composed of multiple (not less than five) main elements [1, 2] . They typically possess high hardness, high strength, high temperature-softening resistance, superior wear resistance, and corrosion resistance [3] . HEAs have broad application prospects in the nuclear power industry, biochemistry, chemical industry, etc. [4] . The phases of HEAs mainly include solid solution (SS), intermetallic compound (IM), solid solution and intermetallic compound (SS+IM), and amorphous phase (AM) [5] [6] [7] . Because these phases are key factors determining the performance of materials, the accurate prediction of the phases in HEAs is crucial for material design [8] .

In recent decades, many phase prediction methods for HEAs have been applied in the materials field. However, the quantity of element combination of HEAs is much larger than single-principal component alloys, so it is more difficult to predict the phases of HEAs. The traditional trial-and-error method is an approach to detect the phases of HEAs that has low efficiency, long cycle time, and high cost [9] . The functional density theory and calculation of phase diagram method (CALPHAD) are other methods to predict the phases

HEAs have excellent properties that are considerably affected by their phase structures [35] . The phases of HEAs include SS, IM, SS+IM, and AM. Some scholars have adopted the parameter method to determine the phase formation of HEAs in the materials field [5, [11] [12] [13] .

To date, the main parameters of HEAs include δ, ∆H mix , Ω, ∆χ, and ∆S mix . The formulae of these parameters are shown in Equations (1)-(5) [11, 12] : 2 (1)

where δ is the mean square deviation of atomic size of all elements in multi-principal alloy, c i is the atomic percentage of the i-th element, r i is the atomic radius of the i-th principal element, and r is the weighted average atomic radius of all principals, r = ∑ n i=1 c i r i .

where ∆H mix is the enthalpy of mixing; H ij is the mixing enthalpy of i-th and j-th binary liquid alloy; and c i and c j are the atomic percentage of the i-th and j-th element, respectively.

where ∆S mix is the entropy of mixing; c i is the atomic percentage of the i-th element; and R is the gas constant, the value of which is 8.314 J·K −1 ·mol −1 .

where Ω is the number of states of multi-principal element alloy system molecules; and T m is the weighted average melting point for all elements, T m = ∑ n i=1 c i T i m .

where ∆χ is the electronegativity difference of the HEA system, and χ = ∑ n i=1 c i χ i , χ i is the electronegativity of constituent elements.

Meta-learning is utilized to solve the problem of algorithm selection [36, 37] . The meta-learning method establishes the relationship between different datasets and the performance of algorithms to recommend the most appropriate algorithm. A schematic diagram of meta-learning is shown in Figure 1 . 

Meta-learning is utilized to solve the problem of algorithm selection [36, 37] . The meta-learning method establishes the relationship between different datasets and the performance of algorithms to recommend the most appropriate algorithm. A schematic diagram of meta-learning is shown in Figure 1 . In Figure 1 , the data library contains N datasets, including D1, D2, …, DN. The selection of meta-features significantly affects the performance of meta-learning. The simple, statistical, and information theoretic meta-features can reflect the characteristic of several datasets [24, 26, 29, 38] . In Figure 1 , there are E candidate algorithms. The meta-knowledge table is constructed by meta-features and performance of candidate algorithms. The metalearner is a very important part of meta-learning. The meta-learner is trained based on the meta-knowledge table, which takes the meta-features of datasets as input variables and yields the performance of algorithms as output variables. The KNN method is often used as a meta-learner in the literature [28, 29] . The meta-model is a trained meta-learner that maps relationships between the meta-features of datasets and the performance of candidate algorithms. The algorithm steps for meta-learning are as follows:

Step 1: Compute the meta-features of each dataset.

Step 2: Train each algorithm on each dataset to evaluate its performance.

Step 3: Construct the meta-knowledge table with the meta-features of datasets and the performance of candidate algorithms.

Step 4: Train the meta-learner based on the meta-knowledge table to map the relationship between meta-features and performance of the candidate algorithms. The trained meta-learner is also called the meta-model.

Step 5: Compute the meta-features of the new dataset and predict the performance of candidate algorithms on the new dataset by the meta-model. Select the algorithm with the highest predictive performance as the recommended algorithm.

The SNN is a neighbor-instance selection method for classification and regression problems. Given an instance, its shelly nearest neighbors refer to the nearest neighbors that make up the shell to encapsulate the instance [39] . The SNN method considers its left In Figure 1 , the data library contains N datasets, including D 1 , D 2 , . . . , D N . The selection of meta-features significantly affects the performance of meta-learning. The simple, statistical, and information theoretic meta-features can reflect the characteristic of several datasets [24, 26, 29, 38] . In Figure 1 , there are E candidate algorithms. The metaknowledge table is constructed by meta-features and performance of candidate algorithms. The meta-learner is a very important part of meta-learning. The meta-learner is trained based on the meta-knowledge table, which takes the meta-features of datasets as input variables and yields the performance of algorithms as output variables. The KNN method is often used as a meta-learner in the literature [28, 29] . The meta-model is a trained metalearner that maps relationships between the meta-features of datasets and the performance of candidate algorithms. The algorithm steps for meta-learning are as follows:

Step 1: Compute the meta-features of each dataset.

Step 2: Train each algorithm on each dataset to evaluate its performance.

Step 3: Construct the meta-knowledge table with the meta-features of datasets and the performance of candidate algorithms.

Step 4: Train the meta-learner based on the meta-knowledge table to map the relationship between meta-features and performance of the candidate algorithms. The trained meta-learner is also called the meta-model.

Step 5: Compute the meta-features of the new dataset and predict the performance of candidate algorithms on the new dataset by the meta-model. Select the algorithm with the highest predictive performance as the recommended algorithm.

The SNN is a neighbor-instance selection method for classification and regression problems. Given an instance, its shelly nearest neighbors refer to the nearest neighbors that make up the shell to encapsulate the instance [39] . The SNN method considers its left and right nearest neighbors of each attribute in a given dataset [31] . The specific steps of the SNN algorithm are as follows [40] .

Let

. . , f R } be the set of R class labels and D = {(x 1 , y 1 ), (x 2 , y 2 ), . . . , (x N , y N )} be the dataset consisting of N instances, where x i is a vector of M attributes, and y i ⊆ F is the label of the i-th instance, x i . For the j-th attribute (1 ≤ j ≤ M), the left nearest neighbor of a query instance, x t , within D refers to the instances whose value on the j-th attribute is smaller than x t but larger than the rest. The left nearest neighbor of a query instance, x t , on the j-th attribute is defined as follows (Equation (6)):

where x ij is the j-th attribute value of x i . According to this definition, if x tj was not the smallest one, x t has at least one left nearest neighbor, x i ∈ D, on the j-th attribute, such that x ij ≤ x tj , whereas x kj ≤ x ij for the remaining instances, x k , within D. Based on Equation (6), the left nearest neighbors of x t over all attributes within D are represented as Equation (7):

In a similar way, the right nearest neighbors of x t over all attributes within D are represented as Equation (8):

where x + t (D, j) is the right nearest neighbors of x t with respect to the j-th attribute, as shown in Equation (9):

The SNN of x t within D refers to its left and right nearest neighbors, as shown in Equation (10):

Generally speaking, there are about 2 × M shelly nearest neighbors for x t if D has M attributes. For ease of understanding, the SNN of the query instance is shown in Figure 2 . and right nearest neighbors of each attribute in a given dataset [31] . The specific steps of the SNN algorithm are as follows [40] .

be the set of R class labels and For the j-th attribute ( 1 j M ≤ ≤ ), the left nearest neighbor of a query instance, xt, within D refers to the instances whose value on the j-th attribute is smaller than xt but larger than the rest. The left nearest neighbor of a query instance, xt, on the j-th attribute is defined as follows (Equation (6)):

where xij is the j-th attribute value of xi. According to this definition, if xtj was not the smallest one, xt has at least one left nearest neighbor, i x D ∈ , on the j-th attribute, such

for the remaining instances, xk, within D. Based on Equation (6), the left nearest neighbors of xt over all attributes within D are represented as Equation (7):

In a similar way, the right nearest neighbors of xt over all attributes within D are represented as Equation (8):

is the right nearest neighbors of xt with respect to the j-th attribute, as shown in Equation (9):

The SNN of xt within D refers to its left and right nearest neighbors, as shown in Equation (10): Figure 2 shows an instances diagram of two-dimensional distribution. The red triangle is the query instance, marked as Query. The blue quadrangle is the neighbor instances of the query instance, which are selected by the SNN. x t is the query instance. The right nearest neighbor set, x + t (D, j), is composed of all instances in dataset D whose values are greater than or equal to x tj . The left nearest neighbor set, x − t (D, j), is composed of all instances in dataset D whose values are less than or equal to x tj . The total dimension number is M = 2, where j = 1 represents the horizontal axis variable and j = 2 represents the vertical axis variable. x t1 and x t2 represents the horizontal and vertical axis value of the query instance, respectively. The selection process of neighbor instances of x t is as follows: First, the horizontal axis value and the vertical axis value of all instances in the dataset, D, are compared with x t1 and x t2 , respectively. Secondly, if the value of the horizontal axis variable of the instances is greater or less than x t1 , these instances form the instance set x + t (D, 1) or x − t (D, 1). If the value of the vertical axis variable of the instances is greater or less than x t2 , these instances form the instance set x + t (D, 2) or x − t (D, 2). Thirdly, the left and right nearest neighbor of query instance x t along the horizontal axis are x 1 and x 2 , which are the maximum or minimum in x − t (D, 1) or x + t (D, 1). Fourthly, the instance with a maximum value in x − t (D, 2) is x 3 , and the instance with a minimum value in x + t (D, 2) is the same instance, x 3 . The left and right nearest neighbor of query instance x t along the vertical axis is the same instance, x 3 . Thus, the shelly nearest neighbor set of the query sample x t instance includes instances x 1 , x 2 , and x 3 .

Decremental instance selection for KNN regression was first proposed by Song [30] . DISKR is an effective instance selection algorithm for KNN regression that removes outliers and the samples with less effect in the training set for KNN.

The KNN regressor is learned by comparing the given test instances with the training set. Let D = {(x 1 , y 1 ), (x 2 , y 2 ), . . . , (x N , y N )} be the training set, where x i = (x i1 , x i2 , . . . , x iM ) is the i-th instance with M attributes, and y i is the output of x i . N is the number of instances. When a query instance, x t , is given, the distance, d i , between x t and each instance, x i , in D is calculated first, and then d i is sorted by ascending order. The first k instances whose d i ranks ahead are selected as k nearest neighbors of x t , and the predicted output,ŷ t , of x t is the average value of y i of k nearest neighbors.

The specific steps of DISKR are as follows:

Output: The subset S ⊆ D.

Step 1: Remove outliers. If PD(x i ) =|y i −ŷ i |> (1 − θ)y i , then the instance x i is an outlier instance and removed; otherwise, it is not an outlier instance and is retained. y i is the output of x i ;ŷ i is the predicted output of x i .

Step 2: Sort the remaining instances after removing outliers. Sort instances, x i , by the absolute difference, PD(x i ) =|y i −ŷ i |, in descending order.

Step 3: Delete instances with less effect on the KNN regressor. The effect of x i could be estimated by the change in performance of KNN over D and D − {x i }. The training error is used to approximately estimate the performance of KNN, which is expressed by the residual sum of squares (RSS). (11) and (12):

The effect of x i on KNN is represented as Equation (13):

After an instance, x i , is removed, the following rule is adopted to avoid a significant negative change in the performance of the regressor, as shown in Equation (14):

where θ ∈ (0, 1) is the significant coefficient.

Step 4: Output the subset S; the remaining samples in the subset S are more relative samples that remove outliers and points that have less effect on KNN.

In this section, the phase prediction of high-entropy alloys (PPH) is proposed by integrating machine learning recommendation method and criterion to predict the phases of HEAs.

An MLRM based on improved meta-learning is proposed. The schematic diagram shown in Figure 3 illustrates MLRM, which can guide material designers to recommend an ideal algorithm.

The effect of xi on KNN is represented as Equation (13):

After an instance, xi, is removed, the following rule is adopted to avoid a significant negative change in the performance of the regressor, as shown in Equation (14):

where (0,1) θ ∈ is the significant coefficient.

Step 4: Output the subset S; the remaining samples in the subset S are more relative samples that remove outliers and points that have less effect on KNN.

In this section, the phase prediction of high-entropy alloys (PPH) is proposed by integrating machine learning recommendation method and criterion to predict the phases of HEAs.

An MLRM based on improved meta-learning is proposed. The schematic diagram shown in Figure 3 illustrates MLRM, which can guide material designers to recommend an ideal algorithm.

Step 4:

Step6:

Step 1: Calculate the parameters by parameter method

Step 2: Select the metafeatures and calculate the values of meta-features

The values of meta-features (MF)

Step 3: Get the accuracies of candidate algorithms on data library L

Step 5: Train the new meta-learner ISD

Step 8: Output the recommended algorithm

Calculate the values of metafeatures MFnew Calculate the parameters by parameter method As shown in Figure 3 , the data library,

and Ω of each sample in each dataset, Di, are calculated by the parameter method [11, 12] . The meta-features should reflect the characteristic of the datasets. The meta-feature set, As shown in Figure 3 , the data library, L = {D 1 , D 2 , . . . , D N }, contains N datasets. D i is the i-th dataset of L. The parameters ∆S mix , ∆H mix , δ, ∆χ, and Ω of each sample in each dataset, D i , are calculated by the parameter method [11, 12] . The meta-features should reflect the characteristic of the datasets. The meta-feature set,

and Ω are the mean value of ∆S mix , ∆H mix , δ, ∆χ, and Ω, respectively; and σ 2

Ω are the variance of ∆S mix , ∆H mix , δ, ∆χ, and Ω, respectively, in each dataset. The candidate algorithm set, C = {C 1 , C 2 , . . . , C M }, includes the decision tree (DT), KNN, SVM, random forest (RF), and bagging. The meta-knowledge table describes the relationship between the values of meta-features (MF) and the accuracy of candidate algorithms (CA).

The new meta-learner based on SNN and DISKR is proposed to improve the performance of meta-learning and is called improved SNN and DISKR (ISD). Therefore, an MLRM based on improved meta-learning is proposed. The meta-model is the trained meta-learner, ISD.

The steps of the MLRM are as follows:

Output: Recommendation algorithm.

Step 1: Compute the parameters ∆S mix , ∆H mix , δ, ∆χ, and Ω of each sample in each dataset, D i , by Equations (1)-(5) in Section 2.1. Step 2: Construct meta-features and calculate the values of meta-features:

Step 3: Based on the values of meta-features and the real classification results on each dataset, train candidate algorithms DT, KNN, SVM, RF, and bagging in candidate algorithm set C = {C 1 , C 2 , . . . , C M } to evaluate their accuracies.

Step 4: The meta-knowledge table is established based on meta-features of datasets and accuracies of candidate algorithms. The values of meta-features are used as the input variable (MF). The accuracies of candidate algorithms are used as the output variable (CA).

Step 5: Train the new meta-learner, ISD, based on the input variable, MF, and output variable, CA, in Step 4. Thus, obtain the trained meta-learner ISD, also known as the meta-model.

Step 6: Compute parameters of each sample in the new dataset, D new , by the parameter method: ∆S mix , ∆H mix , δ, ∆χ, and Ω. Compute the values of meta-features, MF new , of the new dataset,

Step 7: Input meta-features, MF new , of the new dataset, D new , to the meta-model. The realization process of the meta-model is as follows. (15):

where the nearest neighbor set, SD(MF new ), is obtained by the intersection of SNN(MF new ) and DI(MF new ).

Step 8: Output the average accuracies of candidate algorithms by SD(MF new ). Select the algorithm with the highest average accuracy as the recommended algorithm.

The criterion of the SS phase has been verified by a large number of experiments in the material field. In the criterion of SS, if δ and Ω fall within the range of δ ≤ 6.6% and Ω ≥ 1.1, the SS phase is easily formed [5, 11, 13] . The decision process of the criterion is easy to understand, saves computation time, and has strong interpretability and generalization. In this paper, the criterion of SS is integrated into the phase prediction of HEAs.

Combining the advantages of strong learning ability of MLRM and strong interpretability and generalization of the criterion of SS, the phase prediction of HEAs is proposed by integrating criterion and MLRM. The PPH is a serial model. First, the criterion of the SS phase is used to determine the phases of the test dataset, T. Secondly, when the remaining samples cannot be determined by the criterion of SS, the MLRM is adopted to recommend the appropriate ML algorithm for the remaining samples. Finally, the recommended algorithm is used to predict the phase of HEAs.

A schematic diagram of the PPH is shown in Figure 4 . In Figure 4 , the blue frame is the criterion of SS, and the green frame is the MLRM. The specific details of the PPH are illustrated as follows:

Input: Test dataset T = {t 1 , t 2 , . . . t N }. Output: Phases of samples in T.

Step 1: Compute the features δ and Ω of samples in test dataset T. Utilize the criterion of SS to judge the phase of samples. The dataset, P = p 1 , p 2 , . . . , p J , is composed of samples satisfying δ ≤ 6.6% and Ω ≥ 1.1. Output: the judgement results of samples in the dataset, P, are SS.

Step 2: In addition to the samples of dataset P, the remaining samples of the test dataset, T, constitute dataset A = a 1 , a 2 , . . . , a Q , that is, A = T − P. When the criterion of SS cannot judge the samples of dataset A, the machine learning recommendation method is adopted to predict phases of HEAs.

Step 3: For dataset A, compute the parameters of each sample in dataset A by Equations (1)-(5) in Section 2.1, and compute the values of meta-features of dataset A. Then, input the values of meta-features into the meta-model.

Step4: Predict the accuracies of the candidate algorithms by meta-model, and select an ideal algorithm as the recommended algorithm in dataset A.

Step5: The recommended algorithm is used to predict the phase of dataset A.

Step6: Output the phases of test dataset T, combining results of dataset P and dataset A. 

The criterion of the SS phase has been verified by a large number of experiments in the material field. In the criterion of SS, if δ and Ω fall within the range of 6.6% δ ≤ and 1.1 Ω ≥ , the SS phase is easily formed [5, 11, 13] . The decision process of the criterion is easy to understand, saves computation time, and has strong interpretability and generalization. In this paper, the criterion of SS is integrated into the phase prediction of HEAs.

Combining the advantages of strong learning ability of MLRM and strong interpretability and generalization of the criterion of SS, the phase prediction of HEAs is proposed by integrating criterion and MLRM. The PPH is a serial model. First, the criterion of the SS phase is used to determine the phases of the test dataset, T. Secondly, when the remaining samples cannot be determined by the criterion of SS, the MLRM is adopted to recommend the appropriate ML algorithm for the remaining samples. Finally, the recommended algorithm is used to predict the phase of HEAs.

A schematic diagram of the PPH is shown in Figure 4 . In Figure 4 , the blue frame is the criterion of SS, and the green frame is the MLRM. The specific details of the PPH are illustrated as follows:

Input: Test dataset

Output: Phases of samples in T.

Step Step 2: In addition to the samples of dataset P, the remaining samples of the test dataset, T, constitute dataset

, that is, A = T-P. When the criterion of SS cannot judge the samples of dataset A, the machine learning recommendation method is adopted to predict phases of HEAs.

Step 3: For dataset A, compute the parameters of each sample in dataset A by Equations (1)-(5) in Section 2.1, and compute the values of meta-features of dataset A. Then, input the values of meta-features into the meta-model.

Step4: Predict the accuracies of the candidate algorithms by meta-model, and select an ideal algorithm as the recommended algorithm in dataset A.

Step5: The recommended algorithm is used to predict the phase of dataset A. 

In order to verify the effectiveness of the proposed PPH, numerous experiments are carried out. The accuracy of the proposed method is compared with several traditional ML algorithms, including DT, KNN, SVM, RF, and bagging [41] [42] [43] . In this paper, the parameters of the algorithms in this experiment are introduced. In DT, the hyperparameter 'maximum depth' is set to '25'. The hyperparameter 'k' of KNN is set to '5'. The support vector classification (SVC) method is used in SVM. In bagging, the hyperparameter 'number of weak learners' is set to '11'. In ISDISKR, the hyperparameter 'θ' is set as '0.05'. The other parameters of all ML algorithms above are set as default.

The experiments are carried out by the sklearn library of Python (version 3.7.4). Python was created by Guido van Rossum in Amsterdam, the Netherlands. The hardware device is a laptop, and the brand of the computer is Lenovo. The CPU is an Intel(R) Core(M) I7-8700 CPU @ 3.20 GHz 3.19 GHz, the RAM is 16.0 GB, the manufacturer is Lenovo Group Limited, and the location is Beijing, China.

In our study, the 12 HEAs datasets were collected from several papers [4, 6, 7, 10, 14, [44] [45] [46] [47] [48] [49] . For convenience, the 12 datasets are sequentially marked by D 1 , D 2 , D 3 , D 4 , D 5 , D 6 , D 7 , D 8 , D 9 , D 10 , D 11 , and D 12 . The 12 datasets contain 902 samples, including 405 quinary HEAs, 359 senary HEAs, and 138 septenary HEAs. The phases of 902 samples include SS, IM, SS+IM, and AM. The quantity of SS, IM, SS+IM, and AM of the quinaries, senaries, and septenaries in 12 datasets is shown in Figure 5 . As shown in Figure 5 , the number of SS is the largest, and the number of IM is the least among the 12 datasets. Figure 5 shows that each dataset contains different quantities of quinaries, senaries, and septenaries. The 12 datasets include 24 elements, which contain Al, Ag, Be, Ce, Co, Cr, Cu, Dy, Fe, Gd, Hf, Mg, Mn, Mo, Nb, Ni, Pd, Si, Sn, Ta, Ti, V, Y, and Zr. The occurrence time of each element in each dataset is shown in Figure 6 . As shown in Figure 6 , the occurrence time of different elements varies. The elements Al, Co, Cr, Cu, Fe, Ni, and Ti appear more frequently, and the element Gd appears only once in 12 datasets. The 12 datasets include 24 elements, which contain Al, Ag, Be, Ce, Co, Cr, Cu, Dy, Fe, Gd, Hf, Mg, Mn, Mo, Nb, Ni, Pd, Si, Sn, Ta, Ti, V, Y, and Zr. The occurrence time of each element in each dataset is shown in Figure 6 . As shown in Figure 6 , the occurrence time of different elements varies. The elements Al, Co, Cr, Cu, Fe, Ni, and Ti appear more frequently, and the element Gd appears only once in 12 datasets.

The maximum and minimum values of the five input parameters, ∆S mix , ∆H mix , δ, ∆χ, and Ω, are shown in Table 1 .

A snapshot of part of the samples in the HEA dataset according to the Pandas data frame format is shown as Table 2 . In Table 2 , the first column is the number of HEAs, and the second column provides the names of the HEAs. The columns ranging from three to seven are the parameters of HEAs, i.e., δ, ∆H mix , Ω, ∆χ, and ∆S mix . The final column is the phases of HEAs. Table 2 shows a snapshot of the first six samples taken from all HEAs.

The correlation between any two parameters in Table 2 plays an important role in phase prediction of HEAs, so the Pearson correlation coefficient, C ab , is selected to describe the correlations of HEA parameters, as shown in Equation (16) [14] :

where a i and b i are the sample values of parameters a and b, respectively; a and b are the mean values of parameters a and b, respectively; s a and s b are the standard deviation of parameters a and b, respectively; and n is the number of samples. If C ab = 1 or C ab = −1, the parameters a and b are almost completely relevant or completely irrelevant, respectively. The correlation among the 5 parameters of 12 datasets is shown in Figure 7 . Table 1 . Table 2 , the first column is the number of HEAs, and In Figure 7 , the values of the correlation coefficient between the two corresponding parameters is shown in every grid. Color intensity is proportional to the correlation coefficients. In the left side of the correlogram, the legend color shows the relationship between the correlation coefficients and the corresponding color. The darker the blue color, the higher the correlation; the darker the orange color, the lower the correlation. The range of Cab in 12 datasets is between−0.88 and 0.96. Among them, ∆Hmix and Ω in D5 have the highest correlation.

The meta-learning based on meta-knowledge table can recommend an ideal algorithm for material designers so as to predict the phases of HEAs. In order to validate the performance of meta-learning, the experimental results of meta-learning are compared with the DT, KNN, SVM, RF, and bagging [41] [42] [43] . The fivefold cross-validation method is also carried out 20 times to avoid the overfitting problem of ML algorithms [50] . The comparison experimental results between meta-learning and traditional ML algorithms for all, quinaries, senaries, and septenaries are shown in Figure 8 . In Figure 8 , the experimental results of all, quinaries, senaries, and septenaries between meta-learning and other traditional ML algorithms are shown in (a), (b), (c), and (d), respectively. In Figure 8a , In Figure 7 , the values of the correlation coefficient between the two corresponding parameters is shown in every grid. Color intensity is proportional to the correlation coefficients. In the left side of the correlogram, the legend color shows the relationship between the correlation coefficients and the corresponding color. The darker the blue color, the higher the correlation; the darker the orange color, the lower the correlation. The range of C ab in 12 datasets is between−0.88 and 0.96. Among them, ∆H mix and Ω in D 5 have the highest correlation.

The meta-learning based on meta-knowledge table can recommend an ideal algorithm for material designers so as to predict the phases of HEAs. In order to validate the performance of meta-learning, the experimental results of meta-learning are compared with the DT, KNN, SVM, RF, and bagging [41] [42] [43] . The fivefold cross-validation method is also carried out 20 times to avoid the overfitting problem of ML algorithms [50] . The comparison experimental results between meta-learning and traditional ML algorithms for all, quinaries, senaries, and septenaries are shown in Figure 8 . In Figure 8 , the experimental results of all, quinaries, senaries, and septenaries between meta-learning and other traditional ML algorithms are shown in (a), (b), (c), and (d), respectively. In Figure 8a , "all" represents samples containing quinaries, senaries, and septenaries. The horizontal axis represents the ID of these datasets, and the vertical axis represents the accuracies of meta-learning and traditional ML algorithms. The red line with pentagonal stars represents the accuracy of algorithms recommended by meta-learning. The black line with squares shows the accuracy of DT. The navy line with circles shows the accuracy of KNN. The blue line with upward triangles represents the accuracy of SVM. The pink line with downward triangles shows the accuracy of RF, and the olive line with diamonds represents the accuracy of bagging. Based on the results presented in Figure 8 , the algorithm with the highest accuracy can be recommended by meta-learning on some datasets. The blue line with upward triangles represents the accuracy of SVM. The pink line with downward triangles shows the accuracy of RF, and the olive line with diamonds represents the accuracy of bagging. Based on the results presented in Figure 8 , the algorithm with the highest accuracy can be recommended by meta-learning on some datasets. In order to facilitate the reader's observation, the results of all, quinaries, senaries, and septenaries are shown in Tables 3-6. The bold part of the tables is the algorithm with the highest accuracy on the corresponding dataset. In order to facilitate the reader's observation, the results of all, quinaries, senaries, and septenaries are shown in Tables 3-6. The bold part of the tables is the algorithm with the highest accuracy on the corresponding dataset. 0.910 ± 0.020 0.890 ± 0.007 0.890 ± 0.009 0.935 ± 0.005 0.880 ± 0.027 0.880 ± 0.027 Table 6 . Accuracy comparison between meta-learning and traditional ML algorithms in septenaries. In Table 3 , the first column is the ID of 12 datasets, and the second column to the seventh column are the accuracies of DT, KNN, SVM, RF, bagging, and meta-learning, respectively. As shown in Table 3 , meta-learning recommends the algorithm with the highest accuracy; in D 9 , D 11 , and D 12 , the algorithms with the highest accuracy are bagging, bagging, and bagging, and the accuracies are 0.983, 0.995, and 0.912, respectively, for all. In D 4 , D 5 , and D 7 , although meta-learning does not recommend the highest-accuracy algorithm, it recommends the algorithm with the second highest accuracy; the algorithms with the second highest accuracy are bagging, bagging, and bagging, and the accuracies are 0.833, 0.871, 0.911, respectively, for all.

It can be seen from Table 4 that meta-learning recommends the algorithm with the highest accuracy in D 2 , D 3 , D 6 , D 7 , D 9 , and D 12 , and the accuracies are 0.927, 0.894, 0.994, 0.898, 0.968, 0.853, respectively, for quinaries. Meta-learning recommends the algorithm with the second highest accuracy in D 1 and D 11 ; the algorithms with the second highest accuracy are bagging and RF in D 1 and D 11 , respectively.

As shown in Table 5 , meta-learning recommends the algorithm with the highest accuracy in D 1 , D 3 , D 6 , and D 10 for senaries, and the accuracies are 0.872, 0.900, 0.972, 0.817, respectively. Meta-learning recommends the algorithm with the second highest accuracy in D 2 for senaries; the algorithm with the second highest accuracy is bagging in D 2 .

In Table 6 , the algorithm with the highest accuracy is recommended by meta-learning in D 2 , D 3 , and D 12 for septenaries; the algorithms with the highest accuracy are RF, RF, and RF in D 2 , D 3 , and D 12 , respectively. Meta-learning recommends the algorithm with the second highest accuracy in D 4 and D 7 for septenaries; the accuracies are 0.969 and 0.964, respectively.

Meta-learning recommends the algorithm with the high highest accuracy in some datasets. The reason should be that meta-learning can mine the relationship between mathematical statistical characteristics of HEA datasets and the performance of algorithms. The mathematical statistical characteristics of datasets include the mean value and variance of these parameters presented in Section 2.1. It is similarly reported in the literature in other fields meta-learning can recommend the desirable algorithm in most cases [24, 27] . The experimental results also show that RF and bagging have higher accuracies than DT, KNN, and SVM [41] [42] [43] . The reason could be that RF and bagging are both ensemble algorithms, which are composed of multiple classifiers and integrate all the classification results.

However, meta-learning cannot recommend the algorithm with the highest accuracy in some datasets. The reasons could be as follows: firstly, the meta-learner is the crucial part in meta-learning method, and the meta-learner of meta-learning is KNN. The performance of KNN is not ideal in ML algorithms. Secondly, the KNN does not fully consider the information of the left and right nearest neighbors of every sample by every attribute. Finally, the meta-learner, KNN, does not consider how to exclude the noise samples and make up for missing information, which influences the decision result of different algorithms. If the performance of the meta-learner can be improved, the recommendation of meta-learning will be more accurate.

To alleviate the problem presented in Section 4.2, a novel MLRM method is proposed based on improved meta-learning. In order to validate the performance of MLRM, a comparison between MLRM and meta-learning is shown in Figure 9 . In Figure 9 , the accuracies of MLRM and meta-learning of all, quinaries, senaries, and septenaries are shown in (a), (b), (c), and (d), respectively. The horizontal axis is the ID of these datasets, and the vertical axis is the accuracy. The red line with circles shows the accuracy of algorithms recommended by MLRM. The black line with squares represents the accuracy of algorithms recommended by meta-learning. It can be seen from Figure 9 that all experimental results of MLRM are better than those of meta-learning. In order to facilitate the reader's observation, the rankings of algorithms recommended by meta-learning and MLRM, as wells real rankings are listed in Tables 7-10. The accuracies of the optimal algorithms recommended by meta-learning and MLRM, as well as the accuracies of real optimal algorithms are listed in Tables 11-14. In order to facilitate the reader's observation, the rankings of algorithms recommended by meta-learning and MLRM, as wells real rankings are listed in Tables 7-10. The accuracies of the optimal algorithms recommended by meta-learning and MLRM, as well as the accuracies of real optimal algorithms are listed in Tables 11-14. Bagging, DT Bagging, RF Bagging, RF D 10 Bagging, RF DT, Bagging DT, SVM D 11 Bagging, RF Bagging, RF Bagging, RF D 12 Bagging, RF Bagging, RF Bagging, DT In Table 7 , the first column is the ID of datasets. The second column and the third column are the ranking of the algorithms recommended by meta-learning and MLRM, respectively. The fourth column is the real algorithm ranking, which is also called TURE. The previous two best algorithms recommended by meta-learning and MLRM are listed from column 2 to column 3. The fourth column represents the actual previous two best algorithms. The first algorithm name and the second algorithm name appearing from column 2 to column 4 represent the highest and the second highest algorithm. In Table 11 , the first column is the ID of datasets, and the second column to the third column are the accuracies of the optimal algorithms recommended by meta-learning and MLRM. The fourth column is the accuracy of the real optimal algorithm. As seen in Tables 7 and 11 , the MLRM can recommend the algorithm with the highest accuracy in 12 datasets for all. However, meta-learning only recommends the algorithm with the highest accuracy in D 9 , D 11 , and D 12 ; the second highest accuracy in D 4 , D 5 , and D 7 ; and neither the highest nor the second highest accuracy algorithm in other datasets.

According to Tables 8 and 12 , meta-learning only recommends the algorithm with the highest accuracy in D 2 , D 3 , D 6 , D 7 , D 9 , and D 12 ; and the second highest accuracy in D 1 and D 11 . The MLRM can recommend the algorithm with the highest accuracy in every dataset for quinaries.

As shown in Tables 9 and 13 , MLRM recommends the algorithm with the highest accuracy for senaries. However, meta-learning only recommends the algorithm with the highest accuracy in D 1 , D 3 , D 6 , and D 10 ; and the second highest accuracy in D 2 . The quantity of ideal algorithms recommended by meta-learning is less than that recommended by MLRM.

As shown in Tables 10 and 14 , meta-learning only recommends the algorithm with the highest accuracy in D 2 , D 3 , and D 12 ; and the second highest accuracy in D 4 and D 7 . The MLRM recommends the algorithm with the highest accuracy in 12 datasets for septenaries.

In summary, MLRM can recommend the algorithm with the highest accuracy in every dataset, whereas meta-learning can only recommend the algorithm with the highest accuracy on several datasets. The reasons may be that the meta-learner in traditional metalearning is KNN, and the meta-learner in MLRM is ISD based on DISKR and SNN. KNN performs poorly when there are noise points and does not consider the useful neighbor information of each sample. In other fields, the DISKR in MLRM has been shown to remove noise points and reduce the size of the datasets to improve performance [30] . The SNN in MLRM is adopted to obtain more reliable and useful left and right neighbor information [31] .

A PPH is proposed based on the criterion of SS and MLRM. The criterion of SS is as follows: if the δ and Ω of samples are in the scope of δ ≤ 6.6% and Ω ≥ 1.1, the phases of samples are recognized as SS. To better illustrate the criterion of SS, a scatter plot based on the δ − Ω coordinate system is shown in Figure 10 , which shows the distribution of SS and non-SS of HEAs. In Figure 10 , the horizontal axis is δ%, and the vertical axis is Ω. The horizontal virtual line is Ω = 1.1, the vertical virtual line is δ = 6.6%, the upper left corner of the dotted line represents the scope of δ ≤ 6.6%, and Ω ≥ 1.1. The red circle represents the samples for which the phase is SS, and the blue triangle represents the samples for which the phase is non-SS. Figure 10 shows that the phase of most samples in the upper left corner are SS in each dataset, and most samples of SS fall within the range of δ ≤ 6.6% and Ω ≥ 1.1. The experimental results show that the criterion δ ≤ 6.6% and Ω ≥ 1.1 has good effect on SS phase determination. Therefore, the criterion of SS is integrated to the PPH to improve the phase prediction accuracy of HEAs.

In order to verify the performance of the PPH, a set of comparison experiments between PPH and MLRM are carried out. The comparison of the results is shown in Figure 11 . In Figure 11 , the accuracies of PPH and MLRM in all, quinaries, senaries, and septenaries are shown in (a), (b), (c), and (d), respectively. The horizontal axis is the ID of these datasets, and the vertical axis is the accuracy. The red line with circles shows the accuracy of PPH. The black line with squares represents the accuracy of algorithms recommended by MLRM. As shown in Figure 11 , PPH has higher prediction accuracy than MLRM in all, quinaries, senaries, and septenaries.

The specific results are listed in Table 15 . The first column is the ID of datasets. Column 2, column 4, column 6, and column 8 are the accuracies of MLRM in all, quinaries, senaries, and septenaries, respectively. Column 3, column 5, column 7, and column 9 are the accuracies of PPH in all, quinaries, senaries, and septenaries, respectively. NULL represents no accuracy of MLRM and PPH. From Table 15 , it can be concluded that the accuracy of PPH is higher than that of MLRM. The reason could be that PPH combines the advantages of the criterion of SS and MLRM. The MLRM is a pure ML method, and it does not make full use of the professional knowledge in the material field. The criterion of SS has strong interpretability and generalization and requires only a small amount of training samples. Therefore, the prediction accuracy of PPH is higher than that of MLRM [33, 34, 51] . In order to verify the performance of the PPH, a set of comparison experiments between PPH and MLRM are carried out. The comparison of the results is shown in Figure  11 . In Figure 11 , the accuracies of PPH and MLRM in all, quinaries, senaries, and septenaries are shown in (a), (b), (c), and (d), respectively. The horizontal axis is the ID of these datasets, and the vertical axis is the accuracy. The red line with circles shows the accuracy of PPH. The black line with squares represents the accuracy of algorithms recommended by MLRM. As shown in Figure 11 , PPH has higher prediction accuracy than MLRM in all, quinaries, senaries, and septenaries. The specific results are listed in Table 15 . The first column is the ID of datasets. Column 2, column 4, column 6, and column 8 are the accuracies of MLRM in all, quinaries, senaries, and septenaries, respectively. Column 3, column 5, column 7, and column 9 are the accuracies of PPH in all, quinaries, senaries, and septenaries, respectively. NULL represents no accuracy of MLRM and PPH. From Table 15 , it can be concluded that the accuracy of PPH is higher than that of MLRM. The reason could be that PPH combines the advantages of the criterion of SS and MLRM. The MLRM is a pure ML method, and it does not make full use of the professional knowledge in the material field. The criterion of SS has strong interpretability and generalization and requires only a small amount of training samples. Therefore, the prediction accuracy of PPH is higher than that of MLRM [33, 34, 51] . 

In order to predict the phases of HEAs, in this paper, we propose the phase prediction of HEAs by integrating the criterion and machine learning recommendation method. First, a meta-knowledge table based on characteristics of HEAs and performance of candidate algorithms is established, and the experiments show that meta-learning can recommend the algorithm with ideal accuracy on some datasets for material designers. Second, in order to guide material designers to select an algorithm with higher accuracy, an MLRM based on SNN and DISKR is proposed. The experimental results show that the recommendation accuracy of the MLRM is higher than that of meta-learning based on KNN on all, quinary, senary, and septenary HEAs. Third, the PPH consisting of the criterion of SS and MLRM is proposed. Compared with other ML algorithms in the HEA field, the experimental results show that the PPH can achieve performance than the traditional meta-learning method. The average prediction accuracy of the PPH in all, quinary, senary, and septenary HEAs is 91.6%, 94.3%, 93.1%, and 95.8%, respectively. The method proposed in this paper can reduce the burden of material designers in selecting algorithms, effectively improve the prediction accuracy of HEAs phase, and considerably reduce the experimental time and cost. In addition, the PPH can also provide a research foundation in other material fields. 

Entropy contributions to phase stability in binary random solid solutions

BCC + C14) High-Entropy Alloy Multiphase Structure Analysis Based on the Theory of Molecular Orbitals. Materials

Theoretical and Experimental Studies of the Structural, Phase Stability and Elastic Properties of AlCrTiFeNi Multi-Principle Element Alloy

A critical review of high entropy alloys and related concepts

Prediction of high-entropy stabilized solid-solution in multi-component alloys

A new thermodynamic parameter to predict formation of solid solution or intermetallic phases in high entropy alloys

Thermodynamics of concentrated solid solution alloys

Tuning element distribution, structure and properties by composition in high-entropy alloys

Metastable high-entropy dual-phase alloys overcome the strengthductility trade-off

High-entropy alloy: Challenges and prospects

Alloy Design and Properties Optimization of High-Entropy Alloys

Phase stability in high entropy alloys: Formation of solid-solution phase or amorphous phase

Design of high-entropy alloys with a single solid-solution phase: Average properties vs. their variances

Using machine learning and feature engineering to characterize limited material datasets of high-entropy alloys

Machine learning guided appraisal and exploration of phase design for high entropy alloys

Machine-learning informed prediction of high-entropy solid solution formation: Beyond the Hume-Rothery rules

Machine Learning-Based Strength Prediction for Refractory High-Entropy Alloys of the Al-Cr-Nb-Ti-V-Zr System

Dual-Rate Adaptive Optimal Tracking Control for Dense Medium Separation Process Using Neural Networks

Machine learning for phase selection in multi-principal element alloys

Machine-learning phase prediction of high-entropy alloys

The phase selection via machine learning in high entropy alloys

Remaining useful life estimation for proton exchange membrane fuel cells using a hybrid method

No Free Lunch Theorems for Search

A Literature Survey and Empirical Study of Meta-Learning for Classifier Selection

A meta-learning approach for selecting image segmentation algorithm

Adaptive recommendation model using meta-learning for populationbased algorithms

A recommendation system for meta-modeling: A meta-learning based approach

A new data characterization for selecting clustering algorithms using meta-learning

Clustering algorithm selection by meta-learning systems: A new distance-based problem characterization and ranking combination methods

An efficient instance selection algorithm for k nearest neighbor regression

Shell-neighbor method and its application in missing data imputation

Multi-kernel learnt partial linear regularization network and its application to predict the liquid steel temperature in ladle furnace

Pruned Bagging Aggregated Hybrid Prediction Models for Forecasting the Steel Temperature in Ladle Furnace

Flotation Height Prediction under Stable and Vibration States in Air Cushion Furnace Based on Hard Division Method

Phase Engineering of High-Entropy Alloys

Metalearning: A survey of trends and technologies

Meta-learning in multivariate load demand forecasting with exogenous meta-features. Energy Effic

Cross-disciplinary perspectives on meta-learning for algorithm selection

Parimputation: From Imputation and Null-Imputation to Partially Imputation

Neighbor selection for multilabel classification

Phase prediction in high entropy alloys with a rational selection of materials descriptors and machine learning models

An integrated framework with machine learning and radiomics for accurate and rapid early diagnosis of COVID-19 from Chest X-ray

Development and validation of consensus machine learning-based models for the prediction of novel small molecules as potential anti-tubercular agents

A criterion for the formation of high entropy alloys based on lattice distortion

Electronegativity and enthalpy of mixing biplots for High Entropy Alloy solid solution prediction

Electronic and thermodynamic criteria for the occurrence of high entropy alloys in metallic systems

Predicting the formation and stability of single phase high-entropy alloys

The elastic-strain energy criterion of phase formation for complex concentrated alloys

Preoperative Ultrasound Radiomics Signatures for Noninvasive Evaluation of Biological Characteristics of Intrahepatic Cholangiocarcinoma

Multiple predicting K-fold cross-validation for model selection

Multi-Model-and Soft-Transition-Based Height Soft Sensor for an Air Cushion Furnace

Informed Consent Statement: Not applicable.

The data presented in this study are available upon request from the corresponding author.

The authors declare no conflict of interest.