Introduction

The health condition of intensive care unit (ICU) patients can vary radically depending on many factors, including previous health history, underestimation of illness severity, efficiency of care, and response to treatment1. Although great strides have been made in recent years in the field of critical care medicine, the mortality rate of ICU patients has seen only a small decrease and still remains around 20–40% due to the highly complex and heterogeneous diseases of these patients2,3.

In clinical practice, ICU prognostic models are critical for correctly evaluating and identifying high-risk ICU patients. This information helps clinicians to make appropriate medical judgements and prevent ICU deaths while also ensuring proper utilization of limited healthcare resources, especially in low- and middle-income countries (LMICs)4,5,6. These systems use bedside and digital distinguishing tools to identify the risk of serious aggravation and death in critical patients and can be used to help capture the intensity of resource utility and gain a better understanding of what constitutes true ICU-acquired organ dysfunction7. The predictive efficiencies of the commonly used scoring systems are reported in Table 18,9,10. Among these systems, the Acute Physiology and Chronic Health Evaluation (APACHE)-II combines three critical domains to predict the mortality of patients: demographic features, such as age and sex; an evaluation of the patient’s chronic health status and admission diagnosis; and the worst values of 12 physiological variables during the first 24 h following ICU admission. However, use of the APACHE-II is time consuming and requires considerable medical expenses, because more than 20 clinical variables are needed to complete the scoring, which should be finished within 24 h after admission. Similar to the APACHE-II, a sequential organ failure assessment focusing on multiple-organ dysfunction is also inconvenient for rapidly assessing ICU patients. Therefore, it may be impractical to apply these scoring systems widely in resource-restricted settings as in LMICs11.

Table 1 Comparison of scoring systems for predicting ICU mortality.
Full size table

The Modified Early Warning Score (MEWS) is a simple and efficient track-and-trigger system for identifying patients with acute illness. It is derived from five common and vital physiological signs: respiratory rate, body temperature, systolic blood pressure, pulse rate, and level of consciousness. This score is helpful for predicting ICU admission and in-hospital mortality through the detection of physiological abnormalities12. The MEWS has advantages in application as it uses easily measurable and available parameters, does not increase the burden of disease, and is suitable for resource-limited settings. In the Surviving Sepsis campaign guidelines, the MEWS is recommended as a screening tool for identifying and managing critically ill patients13. However, the MEWS has been determined to be inferior to the APACHE II in terms of predictive efficacy for ICU mortality. A comparative study regarding the predictive efficacy for 28-day mortality in shock patients reported area under the receiver operating characteristic curve (AUROC) values of 0.785 for the APACHE-II and 0.614 for the MEWS10.

An ideal risk scoring system for critically ill patients should be easy to use, with accurate and informative performance as well as a low cost in order to improve the treatment of ICU patients. However, development of such a system due has been difficult due to the highly complex and heterogeneous diseases of ICU patients. Some convenient laboratory indexes such as the neutrophil-to-lymphocyte ratio (NLR), red cell distribution width (RDW), lactate (lac) concentration, and osmolarity have been widely applied for the prediction of ICU mortality in multiple patient populations in the past few years14,15,16,17. Some reports also have shown that the combination of a scoring system with simple laboratory indexes can improve the predictive efficacy of traditional scoring systems. For example, in a 28-day mortality analysis of 292 shock patients, an innovative MEWS based on the conventional MEWS, age, and transcutaneous oxygen saturation (AUROC = 0.696) was shown to be superior to the conventional MEWS (AUROC = 0.614; p < 0.05)10. Therefore, the integration of a traditional scoring system and simple laboratory indexes might offer a scoring system with a high predictive efficacy suitable for utility in LMICs.

In this study, we developed an improved MEWS scoring system using convenient data, including the MEWS, NLR, lac concentration, international normalized ratio (INR), osmolarity level, and presence of metastatic cancer, by analyzing the correlation of each variable with 28-day ICU mortality. We then compared the predictive efficacies of different scoring systems for 28-day mortality in a development group and verified our developed model in a validation group using clinical data of ICU patients included in the Medical Information Mart for Intensive Care III (MIMIC-III) database.

Methods

Study design

This study analyzed a retrospective cohort of patients admitted to the ICU (aged 14 years or older). A new MEWS scoring system was developed with the aim of better predicting the 28-day all-cause mortality of critically ill patients with a validation display in a nomogram. The datasets used in this study were derived from the publicly available database MIMIC-III (version 1.4), which contains high-quality health-related data from patients who were admitted to the ICU of the Beth Israel Deaconess Medical Center between 2001 and 2012. After completing the National Institutes of Health web-based training course, we obtained approval to access the database (Certification Number: 37764466). Informed consent was not required because all protected health information had been de-identified.

Study population

We reviewed the discharge summaries of all patients in the MIMIC-III database admitted to the ICU between 2001 and 2012. All ICU patients aged > 14 years old with a measured MEWS within 24 h after ICU admission were included in this study. Patients who met any of the following criteria were excluded: (1) age less than 14 years; (2) readmission in the same hospitalization (only data from the first ICU admission were included in forming the final cohort); (3) unavailability of MEWS due to omission of a measurement within 24 h after ICU admission. The screened ICU patients were eligible for subsequent analysis.

Data extraction, management, and processing

Demographic, clinical, and laboratory data and risk scoring results were extracted with structured query language using PostgreSQL tools (version 9.6) or calculated from the following tables: ADMISSIONS, ICUSTAYS, CHARTEVENTS, DIAGNOSIS_ICD, d_items, d_icd_diagnoses, LABEVENTS, PATIENTS, prescriptions, and Materialized Views. The extracted items for demographic, clinical, and laboratory data and risk scoring results in the database are listed in Table 2. The data processing, including missing data imputation and Winsorizing, was only performed on the development set, and the validation set was used to validate the predictive performance of the developed model. The worst values for lab parameters were selected if they were measured multiple times within 48 h before and after ICU admission. The body mass index (BMI) was calculated as weight (kg)/height (m)2, and osmolarity was calculated as (2 × sodium + potassium) + (glucose/18) + (blood urea nitrogen/2.8). The risk scoring systems including the APACHE-II and MEWS. The APACHE II scoring system is based on 12 physiological variables (temperature, mean arterial pressure, heart rate, respiratory rate, alveolar-arterial oxygen tension difference [fraction of inspired oxygen (FiO2 > 50%)] or partial pressure of oxygen in the arterial blood (PaO2; FiO2 < 50%), arterial pH or HCO3, serum sodium, serum potassium, serum creatinine, hematocrit, white blood cell count, and the Glasgow Coma Scale score, a chronic health evaluation and age adjustment score. Each variable was calculated using the worst values for these parameters recorded during the first 24 h in the ICU; the range for the total Apache II score is 0–71 points. The APACHE-II scores for all ICU patients were acquired with the SQL code from the Materialized Views of the MIMIC-III database. The MEWS was calculated according to Table 37 with the worst values within 24 h after ICU admission selected for the parameter used for the MEWS evaluation.

Table 2 Demographic, clinical, laboratory and risk scoring systems extracted from the database.
Full size table
Table 3 Details of modified early warning score.
Full size table

Because the true ages of patients over 89 years old were omitted due to the privacy policy of the MIMIC database, we selected age × 90/300 as a surrogate age for those patients. In data processing, we used multiple imputation to fulfill missing values based on patients with known values that were most similar to those patients with missing values. The missing data were predicted by the relationship between variables, and multiple complete datasets were generated by the Monte Carlo method. After analyzing these datasets, the analysis results were summarized. After imputations, we selected Winsor means to duplicate outliers with the command of winsor2 with replace cuts (1,99). To further examine the effect of osmolarity, the data were additionally categorized into different levels for analysis in the logistic regression models, which could facilitate quick individualized scoring for further validation and clinical utility as follows: osmolarity level 1, < 300 mmoL/L; level 2, ≥ 300 mmoL/L.

Development of the risk prediction model and model validation

The eligible patients were randomly assigned at a ratio of 2:1 to either the derivation cohort for model development or the internal validation cohort for model verification. We performed an initial analysis of all available variables between survivors and nonsurvivors in the development and validation cohorts. Univariate and multivariate logistic analyses were used to identify independent predictors for 28-day all-cause mortality of critically ill patients and to develop the predictive model. Collinearity analysis was used to avoid potential multicollinearity of the predictive model. The discriminative performance of the obtained predictive model was compared with that of the APACHE-II, MEWS, RDW, NLR, lac, and osmolarity in the development and validation groups based on AUROC and 95% confidence interval (CI) values. Calibration of the constructed model was assessed by the H/L C-statistic and calibration curves, and the accuracy of the constructed model was evaluated by the Brier score. Precision-recall area under the curve (PR-AUC) values were calculated for the constructed model and the APACHE-II using the validation cohort.

k-Fold cross validation of the mionl-MEWS score

We performed k-fold cross-validation with five random folds for the total of 51,121 patients. We compared the AUROC, positive predictive value (PPV), and negative predictive value (NPV) values between the model and cross-validation to show the robustness of our model.

Nomogram development for the simplified prediction model

A nomogram is a graphical tool that can be easily used by clinicians in a resource-limited environment, as no statistical software or online electronic calculator is required. In this study, a nomogram was formulated with clinical practicability based on the results for the obtained predictive model.

Statistical analysis

All patients were divided into two cohorts (development vs. validation) with complete randomization. The distributions of continuous variables were assessed by the Kolmogorov–Smirnov test, and data with skewed distributions were log normalized. Normally distributed continuous variables were expressed as mean ± standard deviation (SD), and non-normally distributed continuous variables were expressed as median (interquartile range). Categorical variables were expressed as absolute values (percentages). Descriptive statistics from the development and validation cohorts were used to compare the baseline data between survivors and nonsurvivors with the t test for normally distributed data, the Mann–Whitney U test for non-normally distributed data, and the chi-squared test for categorical variables. The covariates associated with 28-day all-cause mortality were further identified with univariate and multivariate logistic regression analyses. For each variable, the unadjusted and adjusted odds ratios (ORs) were assessed and reported with p-values and 95% CIs. The multivariate logistic regression model (mionl-MEWS) was built using a forward selection modeling process with a significance level of 0.05. The variables independently associated with 28-day mortality (metastatic cancer, MEWS, lac concentration, NLR, INR, and osmolarity level) were included in the final model. Furthermore, potential multicollinearity was tested using a mean variance inflation factor (VIF), where a value ≥ 10 indicated multicollinearity. Additionally, we assessed the discriminative abilities of the different models based on AUROC values. We then applied the obtained model generated from the development dataset to the validation dataset and assessed the discriminative ability based on the AUROC and the calibration capacity based on the H/L C-statistic. We also generated the calibration curves and calculated the Brier scores for predicting mortality among both the development and validation cohorts. The PR-AUC was applied to evaluate the predictive performance considering clinical application with the validation cohort. The robustness of the developed model was evaluated via k-fold cross validation. To enhance the clinical utility of the model, a nomogram was constructed based on the results of the multivariate analysis. All analyses were performed using Stata software (StataCorp. 2017, Stata Statistical Software: Release 15, College Station, TX: StataCorp LLC, version 14.0). A two-sided p < 0.05 was considered statistically significant.

Ethical approval and consent to participate

Informed consent was not required because all protected health information had been de-identified.

Results

Study population and baseline characteristics

According to the inclusion and exclusion criteria, 1,444,795 ICU patients were selected from the MIMIC-III database. Of these, we excluded 1,301,179 cases as repeated ICU admissions, 32,089 patients with an age < 14 years, and 60,406 patients because the MEWS was not measured within 24 h before or after ICU admission. In total, 51,121 cases with sufficient data were included in the final analysis, including 28,742 male patients (55.52%) and 22,379 female patients (44.48%). The mean age of all patients was 74.80 ± 55.04 years. A total of 6825 patients died within 28 days, establishing an initial 28-day mortality rate of 13.35%. The detailed process of study population selection is shown in Fig. 1. Hypertension (54.68%) was the most common comorbidity, followed by cardiac arrhythmia (30.00%), diabetes (28.15%), and congestive heart failure (28.05%). In our study, 34,081 patients (66.67%) were randomly assigned to the development cohort, and 17,040 patients (33.33%) were assigned to the validation cohort.

Figure 1
figure 1

Study population and protocol flowchart. Flow chart illustrating the major steps in the development and validation of the mionl-MEWS model.

Full size image

Development of a risk prediction model for 28-day all-cause mortality of ICU patients

The 28-day all-cause mortality percentages among critically ill patients were 13.39% in the development cohort (4069/30,399) and 13.61% in the validation cohort (2319/17,040). Significant differences in baseline clinical features, risk scores, and laboratory data were observed between survivors and nonsurvivors, as summarized in Tables 4 and 5. In the development cohort, nonsurvivors were predominantly male and compared with survivors, they had a significantly higher incidence of chronic medical conditions or comorbidities such as congestive heart failure, cardiac arrhythmia, pulmonary circulation disease, vasoactive drug use, liver disease, renal failure, hypothyroidism, paralysis, other neurological disease, solid tumor, metastatic cancer, lymphoma, and coagulopathy; and had a significantly lower incidence of valvular disease, hypertension, diabetes, psychoses, depression, and alcohol or drug abuse. Compared with survivors, nonsurvivors also were older and had significantly higher values for length of ICU stay, APACHE-II score, MEWS, white blood cell count, RDW, NLR, platelet (PLT) count, total bilirubin level, INR, aspartate transaminase level, alanine transaminase level, prothrombin time, activated partial thromboplastin time, blood urea nitrogen level, serum creatinine level, blood glucose level, lac concentration, osmolarity, and sodium level. In addition, the nonsurvivors had significantly lower hemoglobin, pH, PaO2, PaCO2, SO2, and PaO2/FiO2 values compared with the survivors. No significant differences in the rates of peripheral vascular disease, chronic pulmonary disease, peptic ulcer, anemia due to blood loss, deficiency anemia, rheumatoid arthritis, or acquired immune deficiency syndrome were observed between survivors and nonsurvivors in the development cohort. However, significant differences in the incidence rates like congestive heart failure, valvular disease, peripheral vascular disease, anemia due to blood loss, rheumatoid arthritis, and acquired immune deficiency syndrome etc. were also observed between the survivors and nonsurvivors in the validation cohort. The baseline characteristics showed similar distributions between the development and validation cohorts, indicating the successful randomization in the present study.

Table 4 Baseline characteristics and comparisons of demographics data, chronic medical conditions or comorbidities, risk scores and laboratory parameters of the study population between different survival status in development group.
Full size table
Table 5 Baseline characteristics and comparisons of demographics data, chronic medical conditions or comorbidities, risk scores and laboratory parameters of the study population between different survival status in validation group.
Full size table

Next, we included the variables that differed significantly between survivors and nonsurvivors of the development cohort in univariate logistic regression analysis. The results presented in Table 6 demonstrated that all selected variables were significantly associated with 28-day mortality in the univariate logistic regression analysis, similar to the results of the abovementioned univariate analyses. The demographic characteristics with the three largest OR values were: age, OR = 1.033, p < 0.001; BMI, OR = 0.995, p < 0.001; and sex, OR = 0.880, p < 0.001. The three chronic medical conditions or comorbidities with the largest OR values were: metastatic cancer, OR = 2.833, p < 0.001; coagulopathy, OR = 2.100, p < 0.001; and requirement of vasoactive drug therapy, OR = 1.812, p < 0.001. For risk scores and laboratory parameters, we selected the indicators with a low cost and a high frequency of use in the ICU. For example, the MEWS can be obtained by simple calculation with the parameters on the nursing record sheet; the RDW and NLR can be obtained via routine blood tests; and the lac concentration, INR, and osmolarity can be obtained using portable testing tools. Regarding the lac concentration, INR, RDW, NLR, and osmolarity, significantly increasing 28-day mortality rates were observed in patients with a lower BMI or a higher age, INR, RDW, and osmolarity level (osmolarity ≥ 300: nonsurvivors, 83.68% vs. survivors, 71.35%, p < 0.001; Fig. 2a–g). Therefore, we selected age, BMI, sex, metastatic cancer, coagulopathy, vasoactive drug use, MEWS, lac concentration, RDW, NLR, INR, and osmolarity level for inclusion in the initial multivariate logistic regression analysis. The present selection strategy is more convenient to use in the clinic than the selection strategy in which all variables based on the results of the univariate analyses in the development cohort are included. The multivariate logistic analyses identified age, metastatic cancer, coagulopathy, MEWS, lac concentration, RDW, NLR, INR, and osmolarity level as independent risk factors for 28-day mortality. The adjusted OR values with 95% CIs for these variables are presented in Table 7. Furthermore, we evaluated the potential multicollinearity of the model above based on the VIF. The VIFs for the RDW, age, and osmolarity level in the prediction model for 28-day mortality were 12.39, 12.23, and 12.34, respectively, thus indicating the multicollinearity of the initial predictive model. To acquire an ideal model, we removed the RDW and age due to multicollinearity as well as coagulopathy given that the INR can simply reflect abnormal coagulation. Finally, we selected metastatic cancer, MEWS, lac concentration, NLR, INR, and osmolarity level for multivariate logistic regression analysis again to build a simplified model. The adjusted ORs together with the 95% CIs and VIF values for the simplified predictive model for 28-day mortality are listed in Table 7.

Table 6 Univariate analyses of factors associated with 28-day ICU mortality rate in development cohort.
Full size table
Figure 2
figure 2

Comparisons of age, body mass index (BMI), red cell distribution width (RDW), neutrophil-to-lymphocyte ratio (NLR), international normalized ratio (INR), lactate (lac) concentration, and osmolarity between survivors and nonsurvivors in the development cohort. (a) Comparison of age by the U test, p < 0.001; (b) comparison of BMI by the U test, p < 0.001; (c) comparison of RDW by the t test, p < 0.001; (d) comparison of NLR by the U test, p < 0.001; (e) comparison of INR by the U test, p < 0.001; (f) comparison of lac concentration by the t test, p < 0.001; and (g) comparison of osmolarity by the chi-squared test, p < 0.001.

Full size image
Table 7 Multivariate analyses and VIF assessment of factors associated with 28-day mortality rate in development cohort.
Full size table

Considering that this predictive model was constructed based on the MEWS, NLR, lac concentration, INR, osmolarity level, and presence of metastatic cancer, the model was named the “mionl-MEWS” model. The AUROC for 28-day mortality using the mionl-MEWS for critically ill patients was 0.717 (95% CI 0.708–0.726, p < 0.001). The calculated H/L C-statistic was equal to 11.27 (p = 0.187), and the calibration plot of the observed versus expected probabilities for assessment of model performance is displayed in Fig. 3. The AUROC values for the APACHE-II, MEWS, RDW, NLR, lac concentration, and osmolarity were 0.743, 0.667, 0.639, 0.603, 0.594, and 0.622, respectively (Table 8). Statistical differences were detected among these AUROC (p < 0.001; Fig. 4). The Brier scores, which indicate model accuracy for measuring prediction at an individual level, were 0.097 (p = 0.575) for the mionl-MEWS, 0.102 (p = 0.673) for the APACHE-II, 0.108 (p = 0.575) for the MEWS, 0.110 (p = 0.492) for RDW, 0.109 (p = 0.574) for lac concentration, 0.112 (p = 0.507) for the NLR, and 0.111 (p = 0.671) for osmolarity (Table 8).

Figure 3
figure 3

Calibration plot of observed versus expected probabilities for assessment of the predictive performance of the mionl-MEWS model in the development cohort.

Full size image
Table 8 Performance of mionl-MEWS, APACHE-II, MEWS, RDW, NLR, and lac for predicting 28 day-mortality in critically ill patients in the development and validation cohorts.
Full size table
Figure 4
figure 4

Predictive performance of the mionl-MEWS, APACHE-II, MEWS, neutrophil-to-lymphocyte ratio (NLR), red cell distribution width (RDW), lactate (lac) concentration, and osmolarity level for 28-day mortality in critically ill patients in the development cohort.

Full size image

Internal validation of the mionl-MEWS score

Next, we internally validated the mionl-MEWS model in the validation group. All VIF values for the mionl-MEWS model are listed in Table 9. The H/L C-statistic in the validation group was equal to 12.33 (p = 0.518), and the calibration plot is displayed in Fig. 5. The AUROC for the mionl-MEWS model for predicting 28-day mortality among ICU patients demonstrated good discriminative power in the validation group (0.908, 95% CI 0.883–0.933, p < 0.001). The AUROC values for the APACHE-II, MEWS, RDW, NLR, lac concentration, and osmolarity in the validation group were 0.883 (0.853–0.915), 0.877 (0.846–0909), 0.712 (0.662–0.761), 0.630 (0.577–0.682), 0.729 (0.682–0.775), and 0.751 (0.705–0.797), respectively (Table 8). Similarly, statistical differences were also detected among these AUROC values (p < 0.001). Although the AUROC for the mionl-MEWS appeared to be greater than that for the APACHE-II, the difference was not found to be significant (p = 0.120; Fig. 6). The PR-AUCs for the mionl-MEWS and APACHE-II were 0.907 and 0.899, respectively (Fig. 7). The Brier scores were as follows: mionl-MEWS, 0.122 (p = 0.540); APACHE-II, 0.102 (p = 0.287); MEWS, 0.111 (p = 0.538); RDW, 0.138 (p = 0.326); lac, 0.138 (p = 0.512); NLR, 0.157 (p = 0.421); and osmolarity, 0.163 (p = 0.890) (Table 8). These results indicate that the mionl-MEWS had good predictive ability with great calibration abilities. Importantly, the mionl-MEWS was not found to be inferior to the APACHE-II and was shown to be superior to other risk scores in the validation group.

Table 9 VIF assessment of factors associated with 28-day mortality rate in validation cohort.
Full size table
Figure 5
figure 5

Calibration plot of observed versus expected probabilities for assessment of the predictive performance of the mionl-MEWS model in the development cohort.

Full size image
Figure 6
figure 6

Predictive performance of the mionl-MEWS, APACHE-II, MEWS, neutrophil-to-lymphocyte ratio (NLR), red cell distribution width (RDW), lactate (lac) concentration, and osmolarity level for 28-day mortality in critically ill patients in the validation cohort.

Full size image
Figure 7
figure 7

Comparison of precision-recall area under the curves (PR-AUCs) between the mionl-MEWS and APACHE-II in the validation cohort.

Full size image

k-Fold cross validation of the mionl-MEWS score

To further illustrate the robustness of the developed mionl-MEWS model, we used repetitive randomization and k-fold cross validation (k = 5) to analyze the total of 51,121 patients. The AUROC for our model was 0.898 and that with k-fold cross-validation was 0.895 (Fig. 8). Under k-fold cross validation, the PPV and NPV were similar between our model and k-fold cross-validation (PPV 0.842 vs. 0.847 and NPV 0.805 vs. 0.810, respectively).

Figure 8
figure 8

Comparison of the area under the receiver operating characteristic curve (AUROC) values between the mionl-MEWS and cross validation.

Full size image

Nomogram for the mionl-MEWS score

Because the AUROC value provides limited information regarding how a prediction score works in clinical practice, a nomogram is needed to visualize the prognostic model for clinicians, and this graph is useful in resource-limited settings such as those without statistical software or electronic calculators. We translated the model with integrated independent factors into a nomogram using Stata statistical software. The prognostic nomogram derived from the mionl-MEWS score for clinical application is shown in Fig. 9.

Figure 9
figure 9

Nomogram for the mionl-MEWS model. On the nomogram, an individual patient’s predicted mortality risk according to the mionl-MEWS model is located on each variable axis, and a line is drawn upward to determine the corresponding score for each variable state. The sum of these numbers indicates the total score, and a line is drawn to the probability axis to determine the likelihood of 28-day mortality (INR international normalized ratio, Lac lactate, MEWS Modified Early Warning Score, NLR neutrophil-to-lymphocyte ratio).

Full size image

Discussion

To the best of our knowledge, this retrospective study is the first to propose a simple prognostic model (mionl-MEWS) combining metastatic cancer, MEWS, lac, NLR, INR, and osmolarity level for the prediction of 28-day mortality in critically ill patients with internal validation. Based on the AUROC and PR-AUC values, the predictive efficacy of the mionl-MEWS for 28-day mortality in critically ill patients was superior to that of the traditional MEWS, NLR, RDW, lac, or osmolarity alone. Hence, the mionl-MEWS could be used to assist with clinical decision-making in the management of ICU patients.

Considering the likelihood of long in-hospital stays and high medical resource consumption, early identification of mortality risk using prognostic scoring systems is important for timely and effective management and intervention in critically ill patients in the ICU. In addition, patterns of ICU admissions have changed due to advances in the treatment of solid malignancies with immunotherapy and targeted therapies. For example, the proportion of patients with metastatic diseases increased from 48.6% in 2007–2008 to 60.2% in 2017–2018 in France18. Although many scoring systems for critical illness have been proposed to translate the complexity of patients’ conditions into a single measure based on quantitative survival probabilities in current clinical practice, the drawbacks and flaws of these individual systems cannot be ignored. For instance, some assessment tools require many blood tests and/or scoring items, which can be time-consuming and lead to delayed interventions and/or a high financial burden for patients. Thus, fast, convenient, and inexpensive evaluation tools are needed in clinical practice.

Our study retrospectively collected variables that could predict the 28-day mortality in critically ill patients. These variables, such as the MEWS, lac, NLR, INR, etc., were chosen from the literature and used in previous ICU risk assessment models. In our study, we demonstrated that compared with survivors, nonsurvivors tended to be older; male; have a higher incidence of metastatic cancer, coagulopathy, and vasopressor drug use within 48 h; have a lower BMI; and have higher MEWS, RDW, NLR, lac, INR, and osmolarity values, indicating that these factors might serve as potential prognostic markers in critically ill patients. Next, we investigated the factors that independently predicted 28-day mortality in critically ill patients. Our initial multivariate logistic regression analysis also showed that age, metastatic cancer, coagulopathy, MEWS, lac concentration, NLR, RDW, INR, and osmolarity level were independent predictors for 28-day mortality. Unfortunately, multicollinearity was detected among age, RDW, and osmolarity level. However, a series of studies have demonstrated that RDW has predictive value for mortality in patients with heart failure, septic shock, acute respiratory distress syndrome, etc.14,19,20. In addition, age ≥ 80 years was shown to be associated with higher ICU and hospital death compared with younger ages21. In our study, RDW and age also showed a correlation with the mortality of critically ill patients (OR 1.512; 95% CI 1.377–1.660; OR 1.038; 95% CI 1.036–1.041, p < 0.001, respectively). Nevertheless, in a previous cohort study of 8089 individuals analyzing the effect of age and RDW, the age-dependency of RDW seemed to be a universal biological feature22. Therefore, we removed age and RDW from our model to avoid multicollinearity in subsequent modeling.

Among the three underlying disease variables, metastatic cancer was previously shown to be an important predictor of a high 30-day mortality in the ICU23 along with mechanical ventilation and vasopressor use24. In the present study, the OR value for metastatic cancer as a predictor of 28-day mortality was 2.791 (95% CI 2.474–3.150; p < 0.001), which is similar to that reported by Barth et al. for the outcome of patients with metastatic lung cancer admitted to the ICU (OR 4.22 [1.4–12.4]; p = 0.008)24. Therefore, tumor metastasis should be considered in the decision-making process in the ICU. Coagulopathy also is a common cause for a poor prognosis in critically ill patients in the ICU, and its severity has been shown to predict hospital mortality standardized by INR25. Therefore, we only selected INR for inclusion in the final model analysis. Finally, vasopressors are commonly administered to ICU patients with hypotension to raise patients’ blood pressure26. Decision-making regarding the timing of vasopressor initiation as well as balancing the risks and benefits of vasopressor use remains challenging. In the dataset used in our study, the proportion of patients who required treatment with a vasopressor within 48 h was significantly higher in the nonsurvivor group than in the survivor group (27.8% vs. 17.52% p < 0.001). Interestingly, vasopressor use was not found to be an influencing factor in our multiple regression analysis though. In a cohort study regarding the mortality of septic shock patients, only a weak correlation between the timing of vasopressor initiation and hospital mortality was found (adjusted OR 1.02, 95% CI 1.01–1.03, p < 0.001)27. These results also indirectly corroborate the finding in the present study that the timing of vasopressor initiation might not be associated with 28-day mortality in the ICU.

Among the indexes, MEWS was developed as a practical tool that can rapidly and effectively estimate clinical death risk using only five simple and basic physiological parameters without increasing the economic burden, since these parameters can be acquired from patient’s electronic medical records automatically. In a previous observational study, Moon et al. found that the introduction of MEWS charts significantly reduced the number of in-hospital cardiac arrest calls (2% vs. 3%; p = 0.004) and in-hospital mortality rates (42% vs. 52%; p = 0.05)28. In addition, in a study predicting the 28-day mortality rate of ICU patients with severe septic shock, the MEWS was associated with the 28-day mortality rate (OR 1.462; 95% CI 1.122–1.905; p = 0.005)29, which was consistent with the finding in our study (OR 1.250; 95% CI 1.232–1.269; p < 0.001). However, another study found that the MEWS had limited ability to estimate sudden disease aggravation in patients, such as the occurrence of cardiac shock30. Therefore, the predictive value of the MEWS alone for the mortality rate in critically ill patients required further investigation.

Sepsis is well-recognized major health problem in the ICU globally. One study found that the proportion of ICU patients with ICU-acquired sepsis was 24.4% and that the mortality of hospitalized sepsis patients was very high at 25–30%31. Whether patients had sepsis was an important factor affecting the mortality of ICU patients. NLR, as an immune-related biomarker, was shown to serve as a convenient prognostic marker in sepsis patients. In their study predicting 28-day mortality in sepsis patients, Liu et al. reported that the NLR was associated with the 28-day mortality rate (OR 1.340; 95% CI 1.253–1.434; p < 0.001)32. However, in the present study, the OR value for the NLR was only 1.045 (95% CI 1.038–1.053; p < 0.001). This consistency might be due to differences in the study populations, as Liu et al. only selected patients with sepsis, and the present population was based on all ICU patients, not only those with sepsis. Previously, the lac concentration has been associated with mortality in different groups of critically ill patients, such as those with cardiogenic, hypovolemic, or septic shock. Relative hyperlactatemia (1.36–2.00 mmol/L) within the first 24 h of ICU admission was reported to be an independent predictor for in-hospital and ICU mortality in critically ill patients16. In addition, osmolarity with a threshold of 300 mmol/L was shown to be associated with increased mortality in critically ill patients with cardiac, cerebral, vascular, or gastrointestinal diagnoses at admission33, and these findings are consistent with those of our study (OR 1.669; 95% CI 1.517–1.836; p < 0.001).

Due to the complexity and heterogeneity in disease among critically ill patients, combination of different indexes can more accurately reflect the prognosis of ICU patients than any single index34. Thus, we included metastatic cancer, MEWS, lac concentration, NLR, INR, and osmolarity level in our model to predict 28-day mortality. Using the ROC curves to evaluate the 28-day mortality of critically ill patients, a higher AUROC values in the development cohort (0.717) and the validation cohort (0.908) were found upon combination of these six parameters as a composite index compared with each parameter separately. Notably, the mionl-MEWS had the greatest AUROC value, superior to those of the MEWS, RDW, osmolarity, NLR, and lac alone, indicating that the mionl-MEWS can provide a more comprehensive reflection of each patient’s condition from six dimensions, including metastatic cancer for the distribution characteristics of ICU patients, MEWS for patients’ general condition, lac concentration for microcirculation, NLR for sepsis, INR for coagulopathy, and osmolarity for the internal environment.

Furthermore, we used the Brier score to assess the accuracy of our developed model. Among the evaluated indexes, the mionl-MEWS had the smallest Brier score in the development cohort and the third lowest score in the validation cohort, indicating that the mionl-MEWS offered good accuracy for prediction at an individual level. Additionally, we calculated the H/L C-statistic to assess consistent agreement between the observed ICU mortality and the actual ICU mortality. The mionl-MEWS showed adequate calibration, suggesting the assignment of the correct probability at all levels of predicted risk. Finally, the mionl-MEWS model provided stable evaluation with excellent calibration in the validation group (AUROC: 0.908 and PR-AUC: 0.907).

Our study has some strengths. First, to our knowledge, this study is the first to demonstrate enhanced prognostic ability for 28-day mortality in ICU patients via the combination of metastatic cancer, MEWS, lac concentration, NLR, INR, and osmolarity level. Second, the sample size in our study was relatively large, which reduced selection bias. Furthermore, we applied different probability models to evaluate the mionl-MEWS model in order to ensure the scientific nature and credibility of the results. Third, the parameters included in the mionl-MEWS model are objective and easily accessible among laboratory parameters that are widely available to clinicians. Fourth, the constructed nomogram makes 28-day mortality prediction easy and rapid in clinical practice.

Nevertheless, it is important to recognize the limitations of our study. Our data were collected retrospectively from the MIMIC-III database, and because this was a single-center retrospective study, it might be difficult to extend the findings to other hospitals. External validation in cohorts from other countries is needed to generalize our findings. Additionally, due to incomplete data collection and inaccurate data elements from the MIMIC-III database, the potential for bias cannot be excluded.

Conclusion

In the present study, we developed a prediction model, the mionl-MEWS, for the 28-day mortality of critically ill patients admitted to the ICU, demonstrated internal validation, and ensured the included clinical variables can be easily obtained in resource-limited settings. Our results showed that the mionl-MEWS offered higher predictive value for the 28-day mortality of critically ill patients compared with other scoring variables and/or systems. However, additional research is required to demonstrated whether the mionl-MEWS can be applied widely and extensively.