Machine learning and interactive GUI for concrete compressive strength prediction

Elshaarawy, Mohamed Kamel; Alsaadawi, Mostafa M.; Hamed, Abdelrahman Kamal

doi:10.1038/s41598-024-66957-3

Download PDF

Article
Open access
Published: 19 July 2024

Machine learning and interactive GUI for concrete compressive strength prediction

Scientific Reports volume 14, Article number: 16694 (2024) Cite this article

1283 Accesses
2 Citations
1 Altmetric
Metrics details

Subjects

Abstract

Concrete compressive strength (CS) is a crucial performance parameter in concrete structure design. Reliable strength prediction reduces costs and time in design and prevents material waste from extensive mixture trials. Machine learning techniques solve structural engineering challenges such as CS prediction. This study used Machine Learning (ML) models to enhance the prediction of CS, analyzing 1030 experimental CS data ranging from 2.33 to 82.60 MPa from previous research databases. The ML models included both non-ensemble and ensemble types. The non-ensemble models were regression-based, evolutionary, neural network, and fuzzy-inference-system. Meanwhile, the ensemble models consisted of adaptive boosting, random forest, and gradient boosting. There were eight input parameters: cement, blast-furnace-slag, aggregates (coarse and fine), fly ash, water, superplasticizer, and curing days, with the CS as the output. Comprehensive performance evaluations include visual and quantitative methods and k-fold cross-validation to assess the study’s reliability and accuracy. A sensitivity analysis using Shapley-Additive-exPlanations (SHAP) was conducted to understand better how each input variable affects CS. The findings showed that the Categorical-Gradient-Boosting (CatBoost) model was the most accurate prediction during the testing stage. It had the highest determination-coefficient (R²) of 0.966 and the lowest Root-Mean-Square-Error (RMSE) of 3.06 MPa. The SHAP analysis showed that the age of the concrete was the most critical factor in the predictive accuracy. Finally, a Graphical User Interface (GUI) was offered for designers to predict concrete CS quickly and economically instead of costly computational or experimental tests.

Ensemble learning based compressive strength prediction of concrete structures through real-time non-destructive testing

Article Open access 21 January 2024

Forecasting the strength of preplaced aggregate concrete using interpretable machine learning approaches

Article Open access 10 April 2024

Unboxing machine learning models for concrete strength prediction using XAI

Article Open access 14 November 2023

Introduction

Escalating emissions of Greenhouse Gases (GHG) have led to the decline of polar ice stores in the Arctic and Antarctic, presenting grave ecological challenges for the Earth¹. The building industry currently stands as the predominant emitter of GHGs, contributing nearly half of the global emissions². A key component of concrete, Portland cement (PC), plays a significant role in these emissions³. The fabrication of PC is responsible for approximately 7% of worldwide CO₂ emissions, with half of this figure arising from the calcination process of calcium oxide (CaO)⁴. Current records show that PC production is at 4000 million tons annually, with projections suggesting an increase to 6000 million tons by 2060⁵. Such figures underscore the urgent need for effective substitutes that can satisfy the surging demand for concrete while reducing energy consumption and CO₂ emissions⁶. One of the proposed scientific and viable solutions is to incorporate waste and recycled materials into concrete production, as it can offer a sustainable approach to meet the growing demand for concrete and simultaneously diminish environmental hazards and the degradation of our natural resources^7,8.

Various alternative materials, sourced from by-products of industrial processes (such as ground granulated blast furnace slag (GGBS)⁹, granite powder¹⁰, and fly ash¹¹), can serve as substitutes for PC. The incorporation of these alternative raw materials can enhance the qualities of concrete while simultaneously diminishing its ecological footprint. Research has demonstrated that replacing PC with materials like GGBS, granite powder, and fly ash in concrete can lead to a reduction of up to 80% in GHG emissions¹². Typically, these industrial by-products are accumulated in large quantities, posing additional environmental concerns. Therefore, their utilization as a partial cement substitute represents an environmentally and industrially advantageous strategy¹³.

Concrete stands as the most utilized construction material on a global scale, surpassed only by water in terms of global consumption, owing to its valuable properties such as the availability of raw materials, ease of molding from liquid to solid states, and the capacity to bear loads post-hardening^14,15,16,17. Its global utilization in structures is also due to its diverse advantageous characteristics, like compressive strength (CS), durability, stiffness, hardness, porosity, density, and resistance to fire and heat. Among these properties, compressive strength is considered paramount, as it significantly influences the safety and durability of concrete structures¹⁸. The variance in concrete’s compressive strength is attributed to its heterogeneous nature, which consists of a binder, aggregates, water, and admixtures¹⁹. These components and how they are combined influence the material’s ability to endure compressive stress, with factors such as the water-to-binder ratio, aggregate size, and binder type playing significant roles²⁰. The complexity of this mixture makes it challenging to determine the compressive strength accurately.

Laboratory testing involves crushing samples of standard dimensions after a specific curing period, is the conventional method for evaluating compressive strength²¹, and is uniformly accepted worldwide. However, such tests are now deemed inefficient and not cost-effective, both expensive and time-intensive. To circumvent these limitations, some studies have introduced various empirical regression techniques to predict concrete’s compressive strength from its mix proportions^22,23,24. Nonetheless, the intricate and highly non-linear relationships between the concrete’s constituents and its strength render the creation of exact regression models exceptionally difficult²⁵.

In the past few years, significant progress has been made in the artificial intelligence (AI) sector, highlighted by the evolution of machine learning (ML) models, including Artificial Neural Network (ANN), Support Vector Machines (SVM), Gene Expression Programming (GEP), and deep learning (DL). This advancement has enabled the extensive use of these techniques across different areas, thereby equipping researchers with powerful tools to address and resolve intricate challenges more efficiently. Compared to conventional regression methods, ML is capable of developing predictive models through certain models that effectively manage regression issues and demonstrate remarkably high accuracy^{26,27,28,29,30}.

The prediction of strength properties of fly ash-based concrete holds significant importance due to the environmental benefits and improved performance characteristics it offers. Fly ash, a by-product of coal combustion is a promising alternative. Its incorporation into concrete mitigates waste disposal issues and enhances the concrete’s mechanical properties. Predicting the compressive strength of fly ash-based concrete using ML models is crucial. It allows for optimizing mix designs, ensuring the concrete meets the required performance standards while minimizing environmental impact. The main factors affecting the compressive strength of concrete include the water-to-binder ratio, aggregate size, binder type, curing age, and the specific characteristics of the alternative materials used, such as fly ash. Addressing these factors by applying ML models can significantly improve the accuracy of strength predictions and optimize concrete mix designs, thereby enhancing performance and sustainability.

Analyzing extensive data from previous concrete mixtures using ML models allows for predicting concrete properties, optimizing material proportions, enhancing quality control, and projecting long-term durability. These models are adept at identifying patterns and connections within data, which leads to more accurate predictions and improved decision-making in the design of concrete mixes. Additionally, machine learning offers opportunities to increase the sustainability of concrete mix designs. Through ML models, optimal material combinations can be identified, which contribute to reducing waste, decreasing carbon emissions, and improving concrete structures’ overall lifespan and performance, considering factors like cost, availability, and environmental impact. Several researchers have investigated the application of machine learning in structural engineering, especially in predicting the compressive strength of concrete.

Chithra et al.³¹ effectively employed the ANN algorithm to forecast the compressive strength of high-performance concrete (HPC) that incorporates silica nanoparticles and copper slag. Similarly, a study by Nguyen et al.³² introduced four distinct machine-learning models to anticipate the compressive and tensile strength of HPC, highlighting the superior output accuracy of models based on gradient boosting regressor (GBR) and extreme gradient boosting (XGBoost). Furthermore, Kumar et al.³³ gathered data from 120 sets to craft several predictive models for assessing the compressive strength of lightweight concrete (LWC), identifying the SVM model as the most efficient. Ashrafian³⁴ successfully applied heuristic regression techniques to estimate fibrous concrete’s strength and ultrasonic pulse velocity. In another work, Zhang et al.³⁵ utilized the random forest (RF) algorithm to foretell the uniaxial compressive strength of lightweight self-compacting concrete. They also conducted a significant analysis of eight input variables to determine their importance. Additionally, Aslam et al.³⁶ built a GEP-based model to predict the compressive strength of high-strength concrete (HSC), utilizing a substantial dataset comprising 357 points. An overview of the different ML models adopted in past literature and the approaches used in the present investigation is delineated in Table 1.

Table 1 ML models adopted in the past literature.

Full size table

The novelty of this study lies in its comprehensive approach to utilizing machine learning models for predicting concrete CS, offering significant improvements in accuracy and practical applicability over traditional methods. The literature review confirms their successful application in similar engineering contexts, validating their effectiveness and appropriateness for the study’s objectives. The following are the authors’ specific contributions:

1.
To introduce diverse ML models, including non-ensemble (regression-based, genetic programming, neural network, fuzzy-inference-system) and ensemble models (adaptive boosting, random forest, gradient boosting, categorical boosting), and assess their effectiveness in predicting concrete CS.
2.
Using Bayesian Optimization (BO) for hyperparameter tuning significantly enhances predictive accuracy.
3.
To conduct a detailed performance analysis across low, moderate, and high CS ranges.
4.
To perform a sensitivity analysis using Shapley-Additive-exPlanations (SHAP) to determine the most influential parameters affecting CS prediction, offering insights for improving predictive accuracy.
5.
To close the gap between complex computational predictions and practical real-world applications by creating a user-friendly GUI, highlighting the practical value of the study and its comprehensive approach to integrating theoretical model development with professional usability.

Research objectives

This research builds upon prior work by employing machine learning models to predict the CS of concrete across a broad spectrum of data, varying from 2.33 to 82.60 MPa. The main objective was to evaluate the efficacy of different ML models for predicting the CS of concrete. Figure 1 shows the flowchart of the methodological approach used in this study to predict the compressive strength of concrete. Initially, data from 1030 datasets are collected, including various components like cement and aggregates, and their properties are analyzed through histograms and heatmaps. Then, two types of predictive models are applied: non-ensemble models and ensemble models. The models’ performance is evaluated by comparing actual and predicted values, using metrics like R2 and RMSE, and through k-fold cross-validation. Sensitivity analysis is conducted, and the results are benchmarked against previous studies to identify the best predictive model. This approach aims to facilitate the researcher’s ability to gauge the effect of different variables on the prediction of CS in a more time-efficient and cost-effective manner compared to extensive experimental studies.

Database collection

To develop a model for predicting outcomes and to analyze the data statistically, researchers can use data from laboratory experiments or gather information from previously published studies. For this research, a substantial dataset consisting of 1030 data points related to the CS of concrete was assembled by reviewing past scholarly articles: Song et al.¹⁹, Song et al.³⁷, and Yeh⁵⁷. The data analysis of the study focused on eight principal attributes, which were used as the input variables: cement (C), blast furnace slag (Slag), fly ash (FA), water (W), superplasticizer (SP), coarse aggregate (Cagg), fine aggregate (Fagg), and the number of days of curing (Age). These were all considered to predict the final compressive strength, the outcome variable. Table 2 provides a concise overview of the statistical description of the collected data, presenting a comprehensive summary of its characteristics. Each row refers to a distinct variable, while the columns contain specific statistical measures for these variables.

Table 2 Descriptive statistics for the collected database.

Full size table

Furthermore, the frequency distribution of the dataset is visually represented in Fig. 2 through histogram plots. These plots are invaluable for understanding the distribution patterns of each variable, such as normality, skewness, and the presence of outliers, which align with the statistics presented in Table 2. The x-axis represents each variable, while the y-axis indicates the frequency of occurrences. This visualization enables a thorough assessment of these variables. The general observations include:

Most variables (i.e., X2, X3, X5, and X8) show a robust positive skewness, indicating a higher concentration of lower values and fewer higher values.
The X4, X6, X7, and Y variables display more balanced distributions with central tendencies.
Outliers are more prominent in features with positive skewness, where higher values occur less frequently.

Correlation analysis

Examining the correlation between variables is crucial for comprehending the connections between dependent features and the target strength factor, as this analysis seeks to determine the most effective prediction model. This method’s most widely used measure is the Pearson correlation coefficient (r), which helps to understand these relationships^58,59. It can be calculated as the ratio of the covariance (cov) of two variables (x, y) to the product of their standard deviations, as represented in Eq. (1).

$$r = \frac{{cov \left( {x,\;y} \right)}}{{\sigma_{x} \sigma_{y} }} = \frac{{\mathop \sum \nolimits_{i = 1}^{n} \left( {x_{i } - \overline{x}} \right)\left( {y_{i } - \overline{y}} \right)}}{{\sqrt {\mathop \sum \nolimits_{i = 1}^{n} \left( {x_{i } - \overline{x}} \right)^{2} } \sqrt {\mathop \sum \nolimits_{i = 1}^{n} \left( {y_{i } - \overline{y}} \right)^{2} } }}$$

(1)

where $\overline{x}$ and $\overline{y}$ are the mean of two variables x and y; n is the number of a dataset.

Figure 3 presents a heatmap that demonstrates the influence of each variable on all other variables. Notably, the strongest positive correlations between X1, X5, X8, and Y are observed, with r-values of 0.50, 0.37, and 0.33, respectively. This indicates that the CS of concrete is significantly influenced by adding cement, followed by superplasticizer, and finally by the number of days of curing, as evidenced by their higher values. Conversely, the strongest negative correlation between X4 and X5 (− 0.66) suggests an inverse relationship between water and superplasticizers. Also, there is a substantial negative correlation between water and CS of concrete, with an r-value of − 0.29. The remaining variables show a weak correlation between concrete’s CS and each other, indicating that variables do not have linear solid relationships with each other. The absence of uncorrelated features implies that all eight input parameters are relevant and can be effectively employed in predicting the CS of concrete.

Figure 4 presents a scatterplot matrix that provides a comprehensive visual analysis of the eight input variables and their relationship with the output variable (Y). The matrix includes histograms on the diagonal, illustrating the distribution of each variable individually, as previously discussed. The off-diagonal cells in the matrix contain scatter plots that show the pairwise relationships between variables. Each scatter plot provides a visual representation of the correlation between two variables. The input parameters X1 and X5 exhibit a clear positive linear relationship with the output variable Y in Positive Linear Relationships. These findings suggest a positive correlation between the increase in X1 and X5 and the increase in Y, indicating a direct relationship. In addition, the variables X3 and X5 exhibited a positive linear relationship, indicating that as the value of X3 increases, the value of X5 also tends to increase.

A strong negative linear relationship is evident between the input parameters X3 and X4, indicating that as X3 increases, X4 consistently decreases. Furthermore, the correlation between X4 and X5 is highly negative, suggesting that higher values of X4 correspond to lower values of X5. In contrast, the X2 input exhibits weak or unclear linear associations with the majority of other variables, suggesting a low level of correlation. Similar to X2, the input X6 does not exhibit distinct linear patterns with the majority of its variables. Regarding the input parameter X7, it is worth noting that while there is a minor correlation with X5, overall, X7 does not exhibit significant linear associations with most other variables. The input X8 exhibits clear clusters when plotted against other input parameters, indicating the existence of sub-groups within the data. Finally, a subtle non-linear pattern can be observed regarding the correlation between inputs X6 and X7. To gain a better understanding of the underlying relationship, additional investigation may be necessary.

Data normalization

Some machine learning models may not function optimally when there is a variation in the scale of input data. As indicated in Table 2, the cement range lies between 102 and 540 kg/m³, while the range for superplasticizers is between 0.0 and 32.2%, highlighting the disparate magnitudes of different input features. To address this, data normalization or rescaling is employed, which adjusts all input variables to a uniform scale. This process utilizes the max–min mapping function, as outlined in Eq. (2).

$$X_{n} = \frac{{X - X_{min} }}{{X_{max} - X_{min} }}$$

(2)

In this equation, X_n represents the normalized data, X_min and X_max denote the minimum and maximum values of each input variable, and X refers to the original dataset undergoing rescaling. The primary benefit of data rescaling lies in its ability to expedite computations and enhance the accuracy and stability of the machine learning-based prediction model.

Non-ensemble models

In this research, six different non-ensemble models were utilized, namely Multiple Linear Regression (MLR), Multiple Nonlinear Regression (MNLR), Support Vector Regression (SVR), Gene Expression Programming (GEP), Artificial Neural Networks (ANN), and Adaptive Neuro-Fuzzy Inference System (ANFIS). These models were developed using the Python programming environment within the ANACONDA software, MATLAB, and SPSS programs. Concise explanations of each model are provided in the subsequent sub-sections.

MLR model

The MLR model is an extension of simple linear regression to predict a single output variable using multiple input variables^60,61,62. This method assumes a linear relationship between the inputs and the output. It’s particularly valuable in situations where various factors influence the response variable, allowing for the assessment of the relative contribution of each predictor. The MLR model is straightforward, interpretable, and widely used in various fields for its ability to provide insights into relationships between variables. Multivariate Linear Regression is effective for predicting and understanding the underlying data structure.

MNLR model

Nonlinear models are straightforward, easy to understand, and effective for making predictions⁶³. These models are versatile in terms of the range of average outcomes they can express. However, they might not be as adaptable as linear models when describing different data types. Nonetheless, if the nonlinear model is well-suited for a particular situation, it could be more efficient, use fewer parameters, and be simpler to understand. This clarity is often due to how parameters relate to significant, meaningful processes.

The process of using the MNLR model involves several steps: firstly, identifying the variable we want to predict; secondly, creating a nonlinear equation that represents how this variable is affected by other variables; thirdly, inputting initial guesses for the parameters of this equation, with the Levenberg–Marquardt method being the chosen technique for estimation; and finally, initiating the MNLR analysis to generate and review the results in the output log.

SVR model

The SVR model is an extension of Support Vector Machines (SVMs) used for regression problems^64,65. SVR effectively finds the best-fit hyperplane in a high-dimensional space that can predict continuous values, maintaining a balance between the complexity of the model and the amount of error tolerated. It’s especially useful for datasets with many features and is known for its robustness against overfitting. This study uses the linear kernel to model relationships between input variables and the target variable linearly. The linear kernel in SVR essentially represents a straight line in the feature space. It assumes that the relationship between the input features and the target variable is linear, meaning that a change in the input features results in a proportional change in the predicted value.

GEP model

The GEP model was developed to create computer programs and is similar to Genetic Algorithms (GAs) and Genetic Programming (GP)^61,66. Figure 5 shows a flowchart of the GEP model. The GEP model follows a structured flow that begins with creating an initial chromosome population, representing potential solutions. These chromosomes are then expressed as computer programs. Following this, each program is executed, and its performance is evaluated based on a predefined fitness function.

If the termination condition is met, the process ends. Otherwise, it iterates to produce a new generation. This involves selecting the best-performing programs to continue to the next cycle and using genetic operators to create a new generation of chromosomes. These genetic operators include mutation, inversion, one-point recombination, two-point recombination, gene recombination, and insertion sequence (IS) transposition rate. The cycle repeats, continually evaluating the fitness of programs and generating new ones until the best possible solution is found or another termination condition is met, at which point the model concludes.

ANN model

ANNs are a cornerstone of machine learning, inspired by the structure and function of the human brain. They are particularly adept at identifying complex, non-linear relationships within large datasets. ANNs consist of interconnected nodes or neurons, which collectively learn to perform tasks like regression and classification by considering examples. Their flexibility and adaptability make them suitable for various applications, from image recognition to natural language processing^67,68,69. A typical neural multilayer perceptron in an ANN consists of three layers: an input layer, one or more hidden layers, and an output layer, as illustrated in a three-layered architecture. In predicting new data sets, a model employs numerous neurons organized into a network to process information. These neurons are interconnected through weights and biases, crucial determinants of a machine learning model’s precision. Networks can be categorized into basic ANNs with a single hidden layer or deep neural networks with multiple layers. Utilizing additional hidden layers augments the ANN’s ability to identify the connections between inputs and outputs, thereby enhancing model accuracy. Figure 6 shows that the variable Y, represented by the CS of concrete, was set as the output from the ANN model, while the eight variables were assigned as the inputs to the ANN model.

ANFIS model

The ANFIS model, initially introduced by Jang⁷⁰ and subsequently elaborated upon by Jang et al.⁷¹, constitutes a universal approximation methodology. In this capacity, it can approximate any real continuous function defined on a compact set with arbitrary precision. The ANFIS structure closely resembles an ANN, featuring five layers, each comprised of nodes, including rules. Notably, the Sugeno fuzzy model, as proposed by Takagi and Sugeno⁷², is frequently employed in ANFIS. A prototypical rule set for a first-order Sugeno fuzzy model, embodying two fuzzy If–Then rules, can be succinctly expressed as follows:

$${\text{Rule}}\;1:\;\;{\text{If}}\;x\;{\text{is}}\;A_{1} \;\;{\text{and}}\;\;y\;{\text{is}}\;B_{1} ,\;\;{\text{then}}\;f_{1} = p_{1} x + q_{1} y + r_{1}$$

(3)

$${\text{Rule}}\; 2:\;\;{\text{ If}}\; x\;{\text{ is}}\; A_{2} \;{\text{ and}}\; y\;{\text{ is}}\; B_{2} , \;{\text{then}}\; f_{2} = p_{2} x + q_{2 } y + r_{2}$$

(4)

Here, $(A_{1} ,\;A_{2} ,\;B_{1} ,\;B_{2} )$ represent fuzzy sets, and $(p_{ij} ,\;q_{ij} ,\;r_{ij} )$ denote parameters associated with the consequent part of each rule. This formulation describes the basic configuration of a Sugeno fuzzy model as it applies to an ANFIS. Figure 7 depicts the equivalent ANFIS framework. In this ANFIS setup, nodes within the same level perform similar functions. The structure is composed of five layers: the first is the input layer, followed by the rule layer, then the normalization layer, the consequent layer, and finally, the output layer. For an in-depth explanation of the ANFIS framework, one can refer to Chang and Chang⁷³.

Ensemble models

In this research, four different ensemble models were utilized, namely Adaptive Boosting (AdaBoost), Random Forest (RF), eXtreme Gradient Boosting (XGBoost), and Categorical Gradient Boosting (CatBoost). The development of these models was carried out using the Python programming environment within the ANACONDA program. Concise explanations of each model are provided in the subsequent sub-sections.

AdaBoost model

Boosting is a well-known algorithm in machine learning, first suggested by Schapire⁷⁵. Subsequently, Freund⁷⁶ developed AdaBoost. This method focuses on combining several basic classifiers created during training into a single strong classifier. Additionally, it enhances the training process to improve the formation of these basic classifiers. The AdaBoost model is an ensemble technique that combines multiple weak learners to form a strong learner. In regression, it sequentially fits a model to adjust the weights of instances based on the errors of the previous model, focusing more on difficult-to-predict instances. The AdaBoost model is often used to improve the accuracy of decision trees and is known for its simplicity and effectiveness in reducing bias and variance.

RF model

The RF model is an ensemble learning method that operates by constructing many decision trees during training and outputting the average prediction of the individual trees. Breiman⁷⁷ first developed the RF model, which combines the ideas of randomly selecting features and grouping data samples together. The RF model is widely used for both classification and regression tasks. It is particularly well-known for its ability to handle large datasets with higher dimensionality and provides estimates of feature importance, which can be very insightful.

XGBoost model

The XGBoost model implements gradient-boosted decision trees designed for speed and performance. It is a highly flexible and versatile algorithm known for its efficiency in handling sparse data and its ability to perform well on a wide range of regression and classification problems. The XGBoost model has been used successfully in numerous machine learning competitions due to its scalability and ability to produce highly competitive predictive models⁶².

The choice to use XGBoost for this research is based on its useful characteristics. It uses regularization to avoid overfitting and uses second-order gradients for quicker convergence. It can handle missing data when finding splits and uses stochastic gradient descent to increase variety and reduce overfitting. Reduction is also employed to minimize overfitting. Furthermore, the XGBoost is designed with system-level enhancements such as parallel processing and cache optimization, which make it both fast and capable of handling large datasets.

CatBoost model

Categorical Gradient Boosting (CatBoost) represents a recent advancement in gradient-boosting algorithms designed to handle categorical features while minimizing information loss⁷⁸. CatBoost distinguishes itself through two key characteristics: the utilization of ordered boosting to mitigate target leakage and its effectiveness, particularly on small datasets. Within the CatBoost model, the computation of the sample average for $x_{{\sigma_{i,k} }}$ involves considering the target values of preceding samples in a random permutation: $\sigma$ = ($\sigma_{1}$, $\sigma_{2}$, …, $\sigma_{N}$) of the dataset, thereby guarding against overfitting. This process is illustrated by Eq. (5).

$$x_{{\sigma_{i,k} }} = \mathop \sum \limits_{j = 1}^{i - 1} \left[ {x_{{\sigma_{i,k} }} = x_{{\sigma_{j,k} }} } \right]y_{{\sigma_{j} }} + \overline{a} P/\mathop \sum \limits_{j = 1}^{i - 1} \left[ {x_{{\sigma_{i,k} }} = x_{{\sigma_{j,k} }} } \right]y_{{\sigma_{j} }} + \overline{a}$$

(5)

where $x_{{\sigma_{i,k} }} = x_{{\sigma_{j,k} }}$ denotes a condition being met, indicated by a value of 1. P represents a predetermined value, and $\overline{a}$ is the coefficient to determine the significance of P.

Hyperparameters tuning

To optimize the hyperparameter settings in the current study, BO is employed. This technique contrasts with traditional methods like Grid-Search by initially modeling the prior distribution of the objective function and iteratively refining the search within the hyperparameter space for the optimal configuration. Initially, each model’s range and prior distribution of hyperparameters are specified. BO then identifies the configuration that maximizes performance within this predefined hyperparameter space. This approach enhances the efficiency of hyperparameter tuning, minimizing redundant experiments and accelerating the identification of the most effective hyperparameter combinations^79,80,81,82. Figure 8 illustrates a framework example of BO-XGBoost.

Comprehensive performance evaluation of models

The dataset was methodically partitioned into two distinct sets: training and testing. The training set is used to fit the model, and the testing set is used to evaluate the model’s predictive performance. The split ratio was carefully chosen to balance the need for sufficient training data against the necessity of a robust evaluation. Hence, the collected dataset was divided into 80% for training and 20% for testing. This ensures that the model is trained on a representative sample of the data while still subject to rigorous testing on data it has not previously encountered. However, two commonly used approaches involve quantitative and visual methods to evaluate and compare the adopted ML models⁸⁴.

Visual methods

Visual methods include scatter plots, violin boxplots, and Taylor diagrams. These methods offer a quick and informative way to compare models, providing insights into accurate predictions for various statistical measures like maximum, minimum, median, and quartiles. They may not capture information about the performance and ranking of models. Scatter plots are used to visualize the relationship between two variables. Violin plots provide a full distribution of the data. This is crucial when comparing models because it shows not only the central tendency (i.e., mean or median) but also the spread and density of model performance metrics. Taylor diagrams are a specialized graphical representation that quantifies the similarity between actual and predicted values. These diagrams plot the correlation, the standard deviation, and the root mean square error of predictions on a single chart. This provides a comprehensive view of a model’s accuracy, variability, and overall performance compared to the actual observations.

Quantitative methods

Quantitative methods including seven performance indices: Determination Coefficient (R²), Willmott Index (WI), Root Mean Square Error (RMSE), Scatter Index (SI), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE) and Mean Bias Error (MBE). The ideal values for these indices are as follows: R² and WI should ideally be 1, indicating perfect prediction accuracy, while RMSE, SI, MAE, MAPE, and MBE should ideally be 0, indicating no error in the predictions. In summary, a predictive model is ideal if its performance indicators are close to or strictly at these values. The equations for calculating these indices are presented in Eqs. (6–12) as follows⁸⁵:

$$R^{2} = 1 - \frac{{\mathop \sum \nolimits_{i = 1}^{n} \left( {x_{i} - y_{i} } \right)^{2} }}{{\mathop \sum \nolimits_{i = 1}^{n} \left( {x_{i} - \overline{x}} \right)^{2} }}$$

(6)

$${\text{WI}} = 1 - \frac{{\mathop \sum \nolimits_{i = 1}^{n} \left( {x_{i} - y_{i} } \right)^{2} }}{{\mathop \sum \nolimits_{i = 1}^{n} \left( {\left| {x_{i} - \overline{x}} \right| + \left| {y_{i} - \overline{x}} \right|} \right)^{2} }}$$

(7)

$${\text{RMSE}} = \sqrt {\frac{{\mathop \sum \nolimits_{i = 1}^{ n} \left( {x_{i} - y_{i} } \right)^{2} }}{n} }$$

(8)

$${\text{SI}} = \frac{RMSE}{{\overline{{x_{i} }} }}$$

(9)

$${\text{MAE}} = \frac{1}{n} \mathop \sum \limits_{i = 1}^{ n} \left| {x_{i} - y_{i} } \right|$$

(10)

$${\text{MAPE}} = \frac{1}{n} \mathop \sum \limits_{i = 1}^{ n} \left| {\frac{{x_{i} - y_{i} }}{{x_{i} }}} \right|$$

(11)

$${\text{MBE}} = \frac{1}{n} \mathop \sum \limits_{i = 1}^{n} (x_{i} - y_{i} )$$

(12)

where $x_{i}$ is the actual CS values; $\overline{{x_{i} }}$ is the mean of the actual CS dataset; $y_{i}$ is the predicted CS value.

k-fold cross-validation

K-fold cross-validation (Fig. 9) is a widely used method to check the performance of ML models. It involves dividing the dataset into several parts, typically ten, known as “folds.” In this tenfold system, the dataset splits into ten subsets. For each test, nine groups are used to train the model, and one group is kept for testing. This approach is suitable for understanding variability within the data and doesn’t take too much time to compute⁶². Each of the ten subsets becomes the test set, with the others being used for training. A reliable measure of model accuracy is obtained by averaging results from all ten tests. This way of testing helps ensure effective training of the adopted ML models and reduces the chance of missing out on essential data in the dataset.

SHAP feature importance analysis

To analyze the sensitivity and interpret ML models on both a wide-scale and a more detailed level, researchers use the SHAP approach, which draws on principles from cooperative game theory⁴⁷. The SHAP method was employed to gauge the comparative impact of input variables on the prediction process. As an advanced method within the realm of explainable artificial intelligence, SHAP helps clarify the complex interactions between the input variables and the model predictions, as shown in Fig. 10. It offers critical insights by identifying which features are most influential on predictions and how they modify the predicted results^87,88. Equation (13) shows the Shapley value $\phi_{i}$ for feature i is determined by calculating the average marginal contribution of that feature across all possible permutations of features. In this equation, N represents the set of all features, S represents a subset of features that excludes feature i, ∣S∣ denotes the cardinality of set (S), v(S) represents the model’s prediction when only features in set (S) are considered, and v(S ∪ {i}) represents the model’s prediction when feature i is added to set S⁸⁹.

$$\phi_{i} = \mathop \sum \limits_{{S{ \subsetneq }N\left\{ i \right\}}} \frac{{\left| s \right|!\left( {\left| N \right| - \left| S \right| - 1} \right)!}}{\left| N \right|!}\left[ {v\left( {S \cup \left\{ i \right\}} \right) - v\left( S \right)} \right]$$

(13)

Results and discussion

Hyperparametric configuration of ML models

The optimal form of the equations developed by the MLR and MNLR models was the result of extensive testing. Considering the developed MLR model, the derived linear equation (Eq. 14) from the MLR model can be written as follows:

$${\mathbf{Y}} = - 23.331 + 0.120 {\mathbf{X1}} + 0.104 {\mathbf{X2}} + 0.088 {\mathbf{X3}} - 0.150 {\mathbf{X4}} + 0.292 {\mathbf{X5}} + 0.018 {\mathbf{X6}} + 0.020 {\mathbf{X7}} + 0.114 {\mathbf{X8}}$$

(14)

Regarding the developed MNLR model, the derived nonlinear equation (Eq. 15) from the MNLR model can be written as follows:

$${\mathbf{Y}} = \left[ {\left( {{ }{\mathbf{X1}}^{0.097} - 0.89 {\mathbf{X2}}} \right){*}\left( {{\mathbf{X2}}*{\mathbf{X5}}*{\mathbf{X7}}} \right)^{{ - 1.17 \times 10^{ - 4} }} { }} \right]{* }\left[ {\left( {{ }{\mathbf{X1}}{*}{\mathbf{X3}}{*}{\mathbf{X8}}^{{4.4 \times 10^{ - 25} }} } \right){*}\left( {\frac{{{\mathbf{X8}}}}{{{\mathbf{X4}}}}} \right)^{0.94} { }} \right]{*}\left[ {\frac{{\left( {{ }{\mathbf{X6}}^{5.9} {* }{\mathbf{X3}}^{ - 0.88} } \right){*}\left( {{\mathbf{X1}} + {\mathbf{X3}} - 34.37} \right)}}{{\left( {14.3{ }{\mathbf{X4}}} \right)^{0.94} + {\mathbf{X2}}}}{ }} \right]$$

(15)

On the other hand, a trial-and-error approach was used to determine the hyper-parameters, architectures, and functions of the remaining ML models during the training stage. The model with the highest average prediction accuracy across the entire training set was chosen based on these results. The grid search hyper-parameter tuning for the developed ML models is shown in Table 3.

Table 3 Optimized primary hyperparameters for the adopted ML models.

Full size table