Outcome prediction for acute kidney injury among hospitalized children via eXtreme Gradient Boosting algorithm

Acute kidney injury (AKI) is common among hospitalized children and is associated with a poor prognosis. The study sought to develop machine learning-based models for predicting adverse outcomes among hospitalized AKI children. We performed a retrospective study of hospitalized AKI patients aged 1 month to 18 years in the Second Xiangya Hospital of Central South University in China from 2015 to 2020. The primary outcomes included major adverse kidney events within 30 days (MAKE30) (death, new renal replacement therapy, and persistent renal dysfunction) and 90-day adverse outcomes (chronic dialysis and death). The state-of-the-art machine learning algorithm, eXtreme Gradient Boosting (XGBoost), and the traditional logistic regression were used to establish prediction models for MAKE30 and 90-day adverse outcomes. The models’ performance was evaluated by split-set test. A total of 1394 pediatric AKI patients were included in the study. The incidence of MAKE30 and 90-day adverse outcomes was 24.1% and 8.1%, respectively. In the test set, the area under the receiver operating characteristic curve (AUC) of the XGBoost model was 0.810 (95% CI 0.763–0.857) for MAKE30 and 0.851 (95% CI 0.785–0.916) for 90-day adverse outcomes, The AUC of the logistic regression model was 0.786 (95% CI 0.731–0.841) for MAKE30 and 0.759 (95% CI 0.654–0.864) for 90-day adverse outcomes. A web-based risk calculator can facilitate the application of the XGBoost models in daily clinical practice. In conclusion, XGBoost showed good performance in predicting MAKE30 and 90-day adverse outcomes, which provided clinicians with useful tools for prognostic assessment in hospitalized AKI children.


Scientific Reports
| (2022) 12:8956 | https://doi.org/10.1038/s41598-022-13152-x www.nature.com/scientificreports/ learning approaches, is an efficient implementation of the gradient boosting framework 21 . The machine learning algorithm has many advantages, such as high predictive accuracy, automatic modeling of non-linearities and high-order interactions, and robustness to multicollinearity. XGBoost has been shown to outperform traditional statistical methods, such as logistic regression, in diverse fields 15,22,23 and has the potential to improve outcome prediction in hospitalized AKI children. Therefore, the study aimed to use the XGBoost algorithm to develop outcome prediction models in hospitalized AKI children.

Methods
Study design. We performed a retrospective study of admissions from January 1, 2015 to December 31, 2020 in the Second Xiangya Hospital of Central South University in China. Pediatric AKI patients were identified from hospitalized children aged between 1 month and 18 years, with at least two serum creatinine (SCr) measurements in any 7-day window during the first 30 days of hospitalization. AKI was determined according to the SCr criteria of the 2012 Kidney Disease: Improving Global Outcomes (KDIGO) Clinical Practice Guideline 24 . The time of AKI diagnosis was identified as the earliest time when the change in SCr met the KDIGO criteria. Baseline SCr was defined as the lowest SCr in the 7 days before AKI diagnosis, or the minimum inpatient SCr value for patients who met the criteria of community-acquired AKI 5 . We analyzed only the first hospitalization when a patient had multiple admissions during the study period. We excluded patients with end-stage renal disease (CKD stage 5, maintenance dialysis, and renal transplantation, identified by diagnosis codes) and hospital stay < 48 h. The study was approved by the Medical Ethics Committee of the Second Xiangya Hospital of Central South University (No. 2013-S061) and registered in the Chinese Clinical Trial Registry (ChiCTR-1800019857).
Informed consent was waived due to the retrospective nature. The study was performed in accordance with the Declaration of Helsinki.
Data collection. We extracted data on patients' demographics, diagnoses, clinical laboratory tests and treatments from the electronic medical record system and laboratory information system. AKI stage was defined based on the KDIGO criteria and determined using the highest SCr value during the first 7 days after AKI diagnosis. We excluded initiation of renal replacement therapy (RRT) when determining AKI stage 3 but recorded it as a separate variable. Patients were recognized as community-acquired AKI when the increase in SCr on the first day met the KDIGO criteria, or the SCr value on admission was ≥ 1.5 times the standardized SCr reference value and ≥ 1.5 times the lowest SCr value during hospitalization 5 . Patients who did not meet the criteria for community-acquired AKI were categorized as hospital-acquired AKI. Comorbidities were identified by the diagnosis codes (International Classification of Diseases, 10th Edition) on admission and at discharge. We analyzed surgery operations and exposure to nephrotoxic drugs in the 7 days before the time of AKI diagnosis. Surgery operations were determined based on the procedure codes and the surgery date, and both were recorded at hospital discharge. The use of nephrotoxic drugs was determined in accordance with the list of nephrotoxic drugs presented in a recent study 5 . We also collected laboratory data and clinical interventions within 7 days after AKI diagnosis. Laboratory data included hemoglobin, white blood cells, platelets, proteinuria (urinary protein dipstick values ≥ 1 +), total bilirubin, albumin, serum potassium, and serum sodium. If multiple measurements of a laboratory parameter were available during the period, we used the one taken closest to the time of AKI diagnosis. Clinical interventions included the use of loop diuretics, mechanical ventilation, and RRT. To establish and validate prediction models for MAKE30 and 90-day adverse outcomes, we randomly allocated pediatric AKI patients to the training and the test sets by the ratio of 7 to 3. In the training set, both XGBoost and logistic regression were used for model construction. The list of all predictor variables included in the prediction models is shown in Supplementary Table S1. XGBoost is an optimized distributed gradient boosting method with high efficiency, flexibility and portability 21 . It implements machine learning algorithms under the Gradient Boosting framework. The final output is obtained by weighting multiple decision trees and decreasing the gradient of the loss function. XGBoost provides a variety of hyper-parameters for different settings. This study used grid search and five-fold cross-validation to identify optimal hyper-parameters. The training set was randomly split into 5 equal-sized subsets, and 4 of them were used for model training, while the remaining one served as the validation set. This process was repeated 5 times, using one subset for model validation each www.nature.com/scientificreports/ time. After parameter-tuning, seven hyper-parameters (eta, max_depth, min_child_weight, subsample, colsam-ple_bytree, gamma and lambda) were optimized, and they were set in the final model. Feature importance of the XGBoost model was calculated using the gain as the measure, representing each feature's fractional contribution to the model based on the total gain of this feature's splits. Finally, the performance of the prediction models was further evaluated in the test set. Evaluation metrics included the area under the receiver operating characteristic curve (AUC), the area under the precision-recall curve (AUPRC), and the Brier score, in which AUC was selected as the primary metric. The optimal cutoff points were determined based on the maximum Youden index in the training set. In addition, we conducted 5 random splits to test the robustness of the findings.

Outcomes
In sensitivity analysis, we examined the performance of the XGBoost models and the logistic regression models in predicting MAKE30 and 90-day adverse outcomes in pediatric AKI patients in different age groups. We also examined the models' performance in AKI children in the ICU and those in other units.

Results
Patient characteristics. During the study period, 18,194 of 93,040 hospitalized children had at least two times SCr measurements in a 7-day window during the first 30 days of hospitalization. Of them, 1394 pediatric AKI patients who met all eligibility criteria were included in our study (Fig. 1). The overall occurrence of AKI was 7.7% (1394/18,194) among hospitalized children. The incidence of AKI was 16.6%, 7.1% and 4.8% in patients aged 1 month to 1 year (infancy), aged 2 to 10 years (childhood), and aged 11 to 18 years (adolescence), respectively.
Baseline characteristics of the study cohort are shown in Table 1. The study cohort consisted of 504 (36.2%) AKI patients in infancy, 502 (36.0%) in childhood, and 388 (27.8%) in adolescence. Hospital-acquired AKI accounted for 75.4% of pediatric AKI patients. Most patients (60.5%) were diagnosed with AKI stage 1, while 22.9% were diagnosed with AKI stage 2 and 16.6% with AKI stage 3. For hospitalized AKI children, the top three most common clinical settings were nephrotoxic drugs (48.3%), congenital heart disease or cardiac surgery (32.4%) and sepsis (13.0%).
Outcomes. Outcomes of the study cohort are shown in Table 2 . 2 and Supplementary  Fig. S3). Table 5 describes the performance of the prediction models for MAKE30 and 90-day adverse outcomes. At the optimal cutoff points, XGBoost achieved a sensitivity of 72.0% and a specificity of 77.4% for MAKE30 and a sensitivity of 73.0% and a specificity of 84.0% for 90-day adverse outcomes in the test set. The precision-recall curves of the models are provided in Fig. 3 and Supplementary Fig. S4. In the test set, the AUPRC of the XGBoost model was 0.521 for MAKE30 and 0.409 for 90-day adverse outcomes. The Brier score and calibration plots of the models are provided in Fig. 4 and Supplementary Fig. S5. The Brier scores of the two models were lower than that of the null model. The results of the 5 random splits are shown in Supplementary Table S6.      6 show the top 15 most important features derived from the XGBoost model. Feature importance reflects the contribution of each variable to the results during the learning process. AKI stage 3 was the most important variable for the prediction of MAKE30, followed by AKI stage 2, serum albumin, platelet count, and serum potassium. For 90-day adverse outcomes, the top 5 most important predictors were serum albumin, platelet count, shock, age, and serum potassium. The partial dependence plots and individual conditional expectation plots of the XGBoost models were provided in Supplementary Figs. S6 and S7.
In sensitivity analysis, we examined the models' performance in pediatric AKI patients stratified by age groups. We also evaluated the models' performance in AKI children in ICU and those in other units. The results are shown in Supplementary Figs. S8-S11. The XGBoost models were superior to the logistic models for predicting MAKE30 and 90-days adverse outcomes in the subgroups of pediatric AKI patients.
We further developed a web-based risk calculator (http:// xydsb AKIte am. xyeyy. com) to promote the application of the XGBoost models, which can automatically calculate the risk of MAKE30 and 90-day adverse outcomes in hospitalized AKI children.

Discussion
The present study found that the incidence of MKAE30 and 90-day adverse outcomes was 24.1% and 8.1% among hospitalized AKI children, respectively. AKI stage was the major risk factor for MAKE30. CKD was the major risk factor for 90-day adverse outcomes. Additionally, we established and validated machine learning-based models using the XGBoost algorithm for predicting MAKE30 and 90-day adverse outcomes. A web-based calculator was established to apply the XGBoost models in daily clinical practice.
Several recent studies have examined the incidence and outcomes of AKI among hospitalized children [3][4][5]13,25,26 . The incidence of AKI varies with clinical settings and age. A large multicenter study reported a 20% overall  www.nature.com/scientificreports/ incidence of AKI among 101,836 pediatric inpatients in China 5 . In the study, AKI occurred in 28% of infants, higher than 17% of childhood patients and 12% of adolescents. Additionally, one study showed that the occurrence of AKI increased in parallel with age and was greatest in patients aged 15 to 18 years old 25 . Our study found that AKI occurred in 7.7% of all hospitalized children. The incidence of AKI in infants was approximately twice that in childhood patients and three times that in adolescents (4.8%). Differences may depend on the diverse causes of AKI and the distribution of comorbidities. Although the incidence of AKI differs between patient populations, it is consistently related to poorer prognosis in hospitalized children 3 . Previous studies have reported that the incidence of MAKE30 was 9.6% in children with sepsis 11 and 5.2% in critically ill children 12 .
Our study showed that MKAE30 and 90-day outcomes occurred in 24.1% and 8.1% of hospitalized AKI children, www.nature.com/scientificreports/ respectively. The results suggested that continuous monitoring during hospitalization and frequent follow-up after discharge are essential for pediatric AKI patients.
The study identified risk factors associated with MAKE30 and 90-day adverse outcomes. As expected, AKI stage and CKD were the major risk factors of MAKE30 and 90-day adverse outcomes, respectively. A higher AKI stage reflects more severe renal dysfunction, while a history of CKD suggests decreased glomerular reserve at baseline 27 . Another important finding is that the risk of 90-day adverse outcomes increased with age. The reasons may be distinct developmental status and repairability in hospitalized AKI children of different ages. In addition, hospital-acquired AKI was independently associated with MAKE30. Although differences in outcomes between hospital-and community-acquired AKI have been investigated in adults 28 , studies focusing on pediatric patients www.nature.com/scientificreports/ are still limited. Finally, baseline variables associated with systemic diseases and multiorgan dysfunction were also risk factors for poor prognosis in hospitalized AKI children 3,5 .
Our study used machine learning methods to predict adverse outcomes in hospitalized AKI children. The results showed that the XGBoost models achieved good performance in predicting MAKE30 and 90-day adverse outcomes. Previous studies have also shown the applicability of the XGBoost algorithm in predicting complications after pediatric cardiac surgery 29 , multiple organ dysfunction in pediatric ICU 30 , and volume responsiveness in oliguric AKI patients 15 . Compared with traditional logistic regression, there are several strengths of the XGBoost algorithm. Firstly, XGBoost has a strong non-linear fitting capability. In the logistic regression model, a linear relationship between the continuous independent variables and the logit conversion values of the dependent variables is needed. Instead, XGBoost makes flexible assumptions and has the ability to learn the complex relationship between the input variables. Secondly, XGBoost is robust to outliers and multicollinearity among the predictors. By contrast, logistic regression requires that there is no multicollinearity between the independent variables. Thirdly, XGBoost can achieve better predictive performance by applying ensemble learning, which integrates the results of multiple weak learners to obtain the strong learner. Fourthly, the XGBoost algorithm can identify important predictors of the outcome by calculating the contribution of each feature to each tree in the learning process 20 , which clinicians may ignore in clinical practice.
Early prediction of adverse outcomes is critical for risk stratification and clinical decision-making in hospitalized AKI children. To promote the clinical application of the XGBoost models, we further established a webpage risk calculator for prognostic assessment of pediatric AKI patients. The risk calculator can help clinicians identify high-risk patients at an early stage for individualized management, such as discussions of goal-of-care, decisions about resource allocation, evaluations of the quality of care, and suggestions of follow-up frequency, and may improve the prognosis of hospitalized AKI children.
Our study has several limitations. Firstly, because it was a single-center retrospective study conducted in an academic hospital, the results may not be generalizable to patients in other medical centers. Secondly, the sample size was relatively small, resulting in a limited number of positive individuals of some baseline variables. An essential variable was CKD, which was determined based on admission or discharge diagnosis codes. Because of the lack of body height data, we were unable to identify it according to the estimated glomerular filtration rate. Thirdly, urine output criteria were not used for AKI diagnosis because hourly urine output rate was not routinely measured in hospitalized AKI patients outside the ICU. Future multi-center prospective studies are required to externally validate the robustness and clinical effectiveness of the prediction models in a larger cohort of hospitalized AKI children.

Conclusions
In conclusion, we determined the incidence and outcomes of AKI among hospitalized children and developed machine learning-based prediction models for MAKE30 and 90-day adverse outcomes using the XGBoost algorithm. The XGBoost models showed good predictive performance in all hospitalized AKI children and in different subgroups. We further established a web-based risk calculator to promote the clinical application of the XGBoost models, which provided clinicians with useful tools for prognostic assessment in hospitalized AKI children. Future multi-center prospective studies are required to demonstrate the robustness and clinical effectiveness of the prediction models.