The value of five scoring systems in predicting the prognosis of patients with sepsis-associated acute respiratory failure

Our study aimed to identify the optimal scoring system for predicting the prognosis of patients with sepsis-associated acute respiratory failure (SA-ARF). All data were taken from the fourth version of the Markets in Intensive Care Medicine (MIMIC-IV) database. Independent risk factors for death in hospitals were confirmed by regression analysis. The predictive value of the five scoring systems was evaluated by receiving operating characteristic (ROC) curves. Kaplan‒Meier curves showed the impact of acute physiology score III (APSIII) on survival and prognosis in patients with SA-ARF. Decision curve analysis (DCA) identified a scoring system with the highest net clinical benefit. ROC curve analysis showed that APS III (AUC: 0.755, 95% Cl 0.714–0.768) and Logical Organ Dysfunction System (LODS) (AUC: 0.731, 95% Cl 0.717–0.7745) were better than Simplified Acute Physiology Score II (SAPS II) (AUC: 0.727, 95% CI 0.713–0.741), Oxford Acute Severity of Illness Score (OASIS) (AUC: 0.706, 95% CI 0.691–0.720) and Sequential Organ Failure Assessment (SOFA) (AUC: 0.606, 95% CI 0.590–0.621) in assessing in-hospital mortality. Kaplan‒Meier survival analysis patients in the high-APS III score group had a considerably poorer median survival time. The DCA curve showed that APS III may provide better clinical benefits for patients. We demonstrated that the APS III score is an excellent predictor of in-hospital mortality.


Data extraction and management
We used Navicat Premium software (version 15.0) to extract relevant data from the MIMIC-IV database.MIMIC-IV provided all of the needed data, which included APSIII, SAPSII, OASIS, LODS, and SOFA scores.We also included covariates that might influence the relationship between these five scoring systems and in-hospital mortality, extracting the following basic data: age, sex, race, and whether the patients were mechanically ventilated.The laboratory parameters included hematocrit (HCT), platelet (PLT), hemoglobin (HB), white blood cell (WBC), prothrombin time (PT), creatinine (Cr), blood urea nitrogen (BUN), the international normalized ratio (INR), lactate, the ratio of partial pressure of O 2 in arterial blood to the fraction of inspired oxygen (PaO 2 /FiO 2 ), vital signs, such as heart rate (HR), mean arterial pressure (MBP), the respiratory rate (RR) and temperature (T).The comorbidities included hypertension, liver disease, renal disease, chronic pulmonary disease (CPD), and diabetes.If a laboratory test or vital sign was measured more than once on the first day of hospitalization, the median value was obtained.

Statistical analysis
We used R (version 4.2.2),SPSS (version 27) and MedCalc (version 20.0.22) to analyze the data.Normally distributed data were expressed as mean ± standard deviation using the independent samples t-test; non-normally distributed data were expressed as median (interquartile spacing) [M (QL, QU)] using the Mann-Whitney test.Categorical variables were analyzed using t-test or chi-square test and expressed as numbers and percentages.Regression analyses were performed to identify independent risk factors for death in the hospital.Variables with P < 0.05 in the univariate analysis were incorporated into the multivariate analysis.The AUC of the ROC curves were compared using the method of Delong et al. 27 to determine the predictive ability of each scoring system for hospital death.After the optimal cutoff value was obtained from the ROC curve, Kaplan-Meier curves were plotted to assess the impact of APSIII on the survival prognosis of patients in the high and low subgroups.The log rank test was used to evaluate the difference in survival between the high-stage and low-stage patients.Finally, DCA was applied to assess the net benefits of the five scoring systems for patients with SA-ARF 28 .The net benefit is a decision analysis metric that combines the assessment of harms and benefits.It is a clinical judgment of the relative value of the benefits and harms of a certain examination.A series of net benefits can be obtained by plotting via a DCA, which can be used to assess diagnosis and prognosis 29 .The larger the area under the DCA curve is, the greater the clinical benefit of a scoring system will be.

Baseline characteristics
We extracted data on 7648 patients with SA-ARF from the MIMIC-IV database.After further screening and elimination, 3874 eligible patients were included.We divided them into 1260 non-survival and 2641 survival groups.The flow chart of the data extraction process is available in Fig. 1.The average age of the survival patients was 65 years, and the non-survival group was 67 years.The average age was lower in the survivor group than in the non-survival group (P < 0.05).The length of hospital stay (LOS Hos) and length of ICU stay (LOS ICU) were longer in the survival group than in the non-survival group (P < 0.001).Among the laboratory results on the day of admission, the WBC, Cr, BUN, INR, PT, lactate, HR, and RR were significantly higher in the non-survival group than the survival group (P < 0.05).A higher PLT levels, Hb, T, BP, and MAP were found in the survival group than in the non-survival group (P < 0.001).The non-survival group was more likely to have the following coexisting comorbidities than was the survivor group: renal disease (P < 0.05).All five scoring systems were high in the non-survival group compared to the survival group.Information related to sex, age, laboratory test results, and the scoring systems used for patients with SA-ARF is shown in Table 1.

Comparison of Receiver operating characteristic curves for five scoring systems
The predictive value of the scoring system for in-hospital mortality was compared using AUC.The ROC curves are shown in Fig. 2, APSIII (AUC: 0.755, 95% Cl 0.714-0.768),LODS (AUC: 0.731, 95% Cl 0.717-0.7745),SAPSII (AUC: 0.727, 95% CI 0.713-0.741),OASIS (AUC: 0.706, 95% CI 0.691-0.720)and SOFA (AUC: 0.606, 95% CI 0.590-0.621)(Table 4).The AUCs of LODS, OASIS, and APS III were more than 0.7, which were significantly Figure 1.Flow chart of this study.PaO 2 /FiO 2 the ratio of partial pressure of O 2 in arterial blood to the fraction of inspired oxygen; Hb haemoglobin, INR the international normalized ratio, WBC white blood cell, BUN blood urea nitrogen, BP blood pressure, T temperature, RR the respiratory rate.higher than the AUC value of SOFA.The AUC of APS III was greater than that of LODS.APSIII had the highest AUC and was more reliable in predicting death in the hospital.The threshold of the scoring system corresponding to Youden's index was chosen as the best threshold for predicting death in the hospital.APSIII had the highest Youden's index (0.367) and sensitivity (73.53%), its specificity (63.2%) was within tolerable bounds, and OASIS had the highest specificity (67.06%).

Kaplan-Meier curves of the APSIII scoring system
The ideal cutoff value for the APSIII score was 69 according to the ROC curve and Youden's index calculations, splitting the patients in the APSIII subgroup into two subgroups: high-and low-scoring subgroups (Fig. 3).In the low subgroup (APSIII < 69), the median survival was 102.824 days (95% CI 101.030-104.619),while in the high subgroup, it was 64.278 days (95% CI 61.773-66.783).There was a statistically significant difference in survival time between the high and low subgroups (χ 2 = 539.6405,P < 0.001).The hazard ratio (HR) for the low subgroup compared to the high subgroup was 0.263 (95% Cl 0.235-0.294),indicating that the risk of death in www.nature.com/scientificreports/ the low subgroup was 0.263 times greater than that in the high subgroup.Therefore, the prognosis of the low subgroup was superior to that of the high subgroup.

Comparison of decision curve analysis curves
As shown in Fig. 4. The results of the DCA curve showed that the red line representing APSIII always above the other lines (LODS, SAPSII, OASIS, SOFA in descending order).Therefore, APS III has the best net benefit and provides the best clinical benefit.We can utilize the APS III score for timely clinical interventions to achieve better clinical benefits.

Discussion
Early identification of patients with SA-ARF is important for prognosis.We can implement mechanical ventilation, infection control and various urgent interventional treatments as soon as possible, which will help reduce the risk of death.We aim to find a scoring system that will help us predict prognosis at an early clinical stage, so that physicians can intervene early in this group of high-risk patients to reduce in-hospital mortality.APSIII, SAPSII, LODS, OASIS, and SOFA are some typical scoring systems, and the prognostic value of these scoring systems in patients with SA-ARF is worth investigating.
We have come to the following conclusions: (1) both logistic regression and Cox regression results indicated that APSIII and LODS were independent risk factors for predicting in-hospital mortality in patients with SA-ARF.
(2) ROC curves suggested that the AUCs of APSIII, SAPSII, LODS, and OASIS were above 0.7, which showed some clinical predictive value.APSIII had the best predictive value for assessing in-hospital mortality, while SOFA had the poorest performance.(3) In-hospital mortality was associated with LODS, OASIS, SOFA and APSIII scores, with higher scores representing more severe organ failure and worse prognosis.(4) Kaplan-Meier survival curves showed that patients with APSIII values greater than 69 had lower median survival days and a higher in-hospital mortality rate.The log rank test of two survival curves were P(sig.)< 0.01, and the Breslow test were P(sig.)< 0.01, suggesting that there was a difference between the APSIII score and prognosis of patients.( 5) DCA curves revealed that APS III had the greatest beneficial effect on patients within the maximum threshold, with timely clinical intervention may improve clinical benefit.The other four scoring systems, in descending order of clinical benefit, were LODS, SAPSII, OASIS, and SOFA.
In addition, several laboratory indicators, lactate levels, and the RR are also independent risk factors for inhospital death in patients with SA-ARF.The release of metabolites and lactic acid increases, and causes respiratory acidosis 30 .Therefore, the lactate concentration can be used to evaluate the overall outcome of patients with sepsis 13 .When SA-ARF occurs, it may cause acidosis, which is negative for the patient's prognosis.Some indicators can be used to document the degree of acidosis, such as the RR and lactate level 31 .When sepsis develops to ARF, the body is deprived of oxygen due to impaired gas exchange, causing shortness of breath and a buildup of lactic acid.The accumulation of lactic acid is associated with mortality 32 .In addition, we discovered that age was an important risk factor associated with death in the hospital.This might be because the risk of death from sepsis increases with age 33 .
APSIII is a common scoring system in the ICU.APSIII is part of the acute physiological score of the APACHE II score, which contains 12 indicators, namely, T, MAP, HR, RR, PaO 2 , pulmonary artery oxygen differential (A-aDO 2 ), hematocrit (HCT), serum potassium (K + ), serum sodium (Na + ), pH, Cr, and WBC count; it is simpler than the APACHE II score 16 .APSIII can be used in pediatric ICU, neurological ICU, and acute pancreatitis for determining prognosis and predict risk of death [34][35][36] .Fernández et al. 37 found that the APSIII score was related to 90-day fatality in patients with sepsis-related acute kidney injury (Cox regression: hazard ratio (HR): 1.01, 95% confidence interval (Cl): 1.0-1.0,p < 0.048).We found that the APSIII had the highest sensitivity and Youden's index, and it had the largest AUC of 0.755 (95% CI 0.741-0.768).The DCA curve also showed that the APSIII provide the best clinical benefit to patients.Kaplan-Meier curves also revealed that higher APSIII scores were associated with poorer prognosis.The reason why APSIII has the best predictive value may be related to its indicators, pH and RR, which allow it to identify disturbances in acid-base balance.As mentioned above, ARF can lead to respiratory acidosis through deepened and accelerated breathing and accumulate lactic acid in the body 31 .An imbalance in alkaline balance leads to disturbances in electrolyte metabolism, which can be reflected by indicators such as Na + and K + in APSIII.APSIII is also an indicator of lung ventilation.When ARF occurs, the prognosis is related to oxygen uptake in the lungs.When pulmonary ventilation is impaired, this can exacerbate the progression of the disease.This is an overlooked component of other scoring systems.There are no studies on the value of the APSIII scoring system for in-hospital mortality in patients with SA-ARF.We found that the APSIII score has the optimal predictive value.
Zhu et al. 38 discovered in a study of scores in patients with sepsis that the AUC for SAPS II is (AUC: 0.754, 95% CI 0.743-0.765),for OASIS is (AUC: 0.753, 95% CI 0.742-0.764),and for LODS is (AUC: 0.822, 95% CI 95.0-743.0),which is comparable to our study that LODS (AUC: 0.731, 95% CI 0.717-0.745),OASIS (AUC: 0.706, 95% CI 0.691-0.720),and SAPSII (AUC: 0.727, 95% CI 0.713-0.741).The indicators of LODS involve several systems, including the neurological, circulatory, renal, respiratory, and hepatic systems 39 .In severe ARF, the PaO 2 /FiO 2 ratio can be used to indicate pulmonary ventilation, but it may be biased when accompanied by increased PaCO 2 .Finally, the LODS score demonstrated good predictive value in this investigation.The regression analysis showed that it was an independent risk factor for death in the hospital, and the clinical benefit of the AUC and DCA curves was second only to that of the APS III score.SAPSII includes seventeen variables, including age, physiological variables, and chronic diseases, and it also contains indicators reflecting electrolyte disturbances such as Na + , k + , HCO 3 − and PaO 2 /FiO 2 .However, it is not as accurate as APSIII for predicting SA-ARF, and its predictive value lies at an average level.As previously described 21 , OASIS is a machine learning algorithm-based model that does not have as many for determining organ failure as does APSIII.None of these three scoring systems has significant advantages over the APSIII.Moreover, these methods do not perform as well as APSIII on the DCA curve and have poorer clinical benefits than APSIII.
Notably, the SOFA score performed the worst in terms of predictive value.The SOFA score is a common indicator used to assess organ failure in patients with sepsis 40 .Ferreira et al. 41 found that when SOFA score were assessed in the first 96 h in the ICU, the mortality increased with increasing SOFA score.Zeng et al. 42 discovered that the SOFA score was related to mortality from ARF in patients with lung cancer (HR: 1.142, 95% CI 1.012-1.288,p = 0.031).However, both logistic and Cox regression analyses suggested that the SOFA score is not an independent risk factor for predicting death in the hospital.ROC curves also showed that it had the lowest AUC (AUC: 0.606, 95% CI 0.590-0.621).The DCA curve showed that the score was at the bottom of the curve.The SOFA score has the lowest predictive value and clinical benefit.It may be that it has considerably fewer indicators than a valuable scoring system for appeals.
The strength of our study is that our study is retrospective and based on the large database-MIMIC-IV.We studied a large sample size.The results of our study are also convincing.The data extracted from the database provides credibility to this study.The data extracted from the database lend credibility to this study.
This study has several limitations.First, our study was only a single-center retrospective analysis of the MIMIC-IV database, and we did not use external data to validate the conclusions.Second, the database we used has limitations.We strictly followed the ICD diagnostic codes to identify sepsis and ARF, and we ultimately obtained data on the SA-ARF.The database did not contain a specific classification of the site of infection, nor was it detailed enough to categorize the type of ARF; therefore, we were unable to perform subgroup analyses (e.g., whether it originated from pneumonia, hospital acquired pneumonia (HAP), or acute respiratory distress syndrome (ARDS)) to draw more convincing conclusions.Third, we were unable to determine whether inhospital deaths were caused by SA-ARF or by the patient's ultimate decision to abandon treatment.Therefore, additional large clinical cohort studies are needed to validate the accuracy of these findings.

Conclusion
APSIII and LODS are independent risk factors for mortality in patients who develop SA-ARF.ROC and DCA curves showed that APSIII had the best predictive value.APSIII is an excellent predictor of prognosis in patients with SA-ARF, and it can predict in-hospital mortality more accurately.Early assessment of a patient's risk of death allows physicians to make the best clinical decisions and helps to improve the prognosis of patients.

Figure 4 .
Figure 4. Comparison of decision curve analysis curves.The clinical benefit of the scoring system can be assessed by assessing the net benefit (y-axis) over a range of threshold probabilities (x-axis).The grey line represents the assumption that all patients with SA-ARF present with in-hospital death, and the black line indicates the assumption that no patients with SA-ARF present with in-hospital death.It can be concluded that the APSIII curve (red) shows the greatest benefit compared to the other curves.APSIII acute physiology score III, SAPSII Simplified acute physiology score, OASIS Oxford acute severity of illness score, LODS Logistic organ dysfunction system, SOFA Sequential organ failure assessment.

Table 1 .
Demographic and clinical characteristics of patients (admission variable only).Hb Hemoglobin, WBC White blood cell, PLT Blood platelet, BUN Blood urea nitrogen, Cr Creatinine, INR International normalized ratio, PT Prothrombin time, PaO 2 Arterial oxygen pressure, FiO 2 Inspired oxygen fraction, T Temperature, HR Heart rate, RR Respiratory rate, MAP Mean arterial pressure, APSIII acute physiology score III, SAPSII Simplified acute physiology score, OASIS Oxford acute severity of illness score, LODS Logistic organ dysfunction system, SOFA Sequential organ failure assessment.

Table 2 .
Binomial logistic regression analysis of in-hospital mortality in patients with SA-ARF in ICU.PaO 2 Arterial oxygen pressure, FiO 2 Inspired oxygen fraction, RR Respiratory rate, APSIII Acute physiology score III, SAPSII Simplified acute physiology score, OASIS Oxford acute severity of illness score, LODS Logistic organ dysfunction system, SOFA Sequential organ failure assessment.

Table 3 .
Cox regression analysis of risk factors for in-hospital death in patients with SA-ARF in ICU.PaO 2 Arterial oxygen pressure, FiO 2 Inspired oxygen fraction, RR Respiratory rate, APSIII Acute physiology score III, SAPSII Simplified acute physiology score, OASIS Oxford acute severity of illness score, LODS Logistic organ dysfunction system, SOFA Sequential organ failure assessment.

Table 4 .
Comparisons of different predictive index.APSIII Acute physiology score III, SAPSII Simplified acute physiology score, OASIS Oxford acute severity of illness score, LODS Logistic organ dysfunction system, SOFA Sequential organ failure assessment, AUC area under the ROC curve, Cl confidence interval, P-value/Z-value compared with APSIII.Factor AUC 95%Cl Optimal cut-off Sensitivity Specificity Youden's index P-value Z-value