Introduction

Acute myocardial infarction (AMI) is one of the leading causes of global cardiovascular burden, among which ST elevation myocardial infarction (STEMI) has attracted much attention of clinicians due to its high mortality and morbidity1,2. In the past decade, the widespread use of percutaneous coronary intervention (PCI) has significantly increased the survival chances of STEMI patients3, however, the survival rate is still not within the ideal range. According to reports from recent studies4,5, the residual in-hospital mortality of STEMI ranges from 5–10%. Risk classification is critical for STEMI and can be used for selecting treatment regimens as well as planning hospital discharge, especially for patients admitted to the intensive care unit (ICU). But currently, there is no recognized tool for predicting in-hospital mortality for intensive care patients with STEMI. Thus, it is particularly important to find an effective tool.

Several scoring systems have been proven effective for predicting mortality in intensive care patients. The Logistic Organ Dysfunction System (LODS) is an organ dysfunction scoring system and permits the calculation of predicted mortality based on the organ dysfunction score on the day of ICU admission6. The LODS score has been effectively used to predict the mortality of intensive care patients with sepsis and patients in neurological ICU7,8. The Oxford Acute Severity of Illness Score (OASIS) was proposed in 2013 by machine-learning algorithms, and predictive models of ICU mortality using OASIS achieved an area under the receiver operating characteristic curve of 0.88 and calibrated well9. The Simplified Acute Physiology Score (SAPS II) provides an estimate of the risk of mortality without having to specify a primary diagnosis and was regarded as a starting point for evaluation of the efficiency of ICU10. In this study, we aimed to explore the performance of the above three scoring systems in predicting the in-hospital mortality of intensive care patients with STEMI, to provide valuable clues to clinical practice.

Methods

Database

This retrospective study was designed based on the critical care database, Medical Information Mart for Intensive Care IV (MIMIC-IV, https://mimic.mit.edu/). The database contains real hospital stays between 2008 and 2019 for patients admitted to a tertiary academic medical center in Boston, MA, USA. According to the official requirements of the database, investigators must complete the "Protect Human Research Participants" exam on the National Institutes of Health website and sign a data use agreement before granting access11. One author (Tianyang Hu) passed the exam (record ID: 37474354) and obtained access to the database.

This study was conducted in accordance with the Declaration of Helsinki. The establishment of the MIMIC-IV database was approved by the Massachusetts Institute of Technology and Beth Israel Deaconess Medical Center, and consent was obtained for the original data collection. The patients in the database are anonymous, therefore, the ethical approval statement and the need for informed consent were waived for this study.

Study population and data extraction

All intensive care patients diagnosed with STEMI were screened and identified by the “long_title” in the “d_icd_diagnoses” table of MIMIC-IV database. Since one patient may be admitted to the ICU multiple times, we only included the first ICU stay for each patient. Navicat Premium software (v15.0) was used to extract the following data of the included patients from MIMIC-IV database (v1.0): age, gender, length of ICU stay, length of hospital stay, coexisting comorbidities (congestive heart failure, peripheral vascular disease, cerebrovascular disease, chronic pulmonary disease, rheumatic disease, peptic ulcer disease, liver disease, diabetes, renal disease, malignant cancer, hypertension, and obesity), laboratory tests (red blood cell, white blood cell, platelets, hemoglobin, anion gap, blood urea nitrogen, creatinine, and INR), vital signs (heart rate, systolic blood pressure, diastolic blood pressure, mean blood pressure, respiratory rate, temperature, and SpO2), and scoring systems (LODS, OASIS, and SAPS II). PCI is the standard interventional treatment modality for patients with STEMI12, also was proved to be effective on one-year mortality in non-ST-segment elevation myocardial infarction (NSTEMI) patients13. Approximately 20–30% of patients are not eligible for PCI and require surgical intervention, that is, coronary artery bypass grafting (CABG)14,15. Therefore, we additionally investigated whether the included patients underwent PCI or CABG.

Statistical analysis

The Kolmogorov–Smirnov test was used to assess the normality of continuous variables. Normally distributed variables were expressed as mean ± standard deviation (M ± SD), and the independent sample t-test was used for comparison; if the distribution was abnormal, continuous variables were expressed as the median with interquartile range (IQR), and Wilcoxon rank-sum test was used for comparison. Categorical variables were expressed as numbers and percentages and compared using the Chi-square test. Binomial logistic regression analysis of the three scoring systems for in-hospital mortality in intensive care patients with STEMI was conducted to adjust the results of the statistical analysis for potential confounding factors by selecting appropriate variables closely related to the condition of STEMI. Variables with P values < 0.1 in univariate analysis were included in multivariate analysis. Z test was used to compare the predictive value of the three scoring systems by comparing the area under curves (AUC) of the receiver operating characteristic curves (ROC).

Propensity score matching (PSM) analysis was performed to reduce bias between the in-hospital death group and survival group based on the following variables: age, gender, length of hospital stay, and underwent PCI/CABG or not. The propensity scores were calculated by a logistic regression model, and the PSM analysis was performed using a 1:1 nearest neighbor matching algorithm with a caliper of 0.02 without replacement.

We also performed the decision curve analysis16,17,18 (DCA), a method for evaluating alternative diagnostic and prognostic strategies, to explore the net benefits between the three scoring systems and intensive care patients with STEMI. The concept of the DCA is generally combining accuracy measures and clinical applicability by integrating clinical consequences associated with a test result. The “net benefit” is calculated by the difference between the proportion of relative harms of false positives and false negatives weighted by the odds of the selected threshold for high-risk designation, therefore the difference between the expected benefit and the expected harm. If it is predicted that the outcome event may occur with a certain probability and the emergency intervention measures (for patients with STEMI, they should be revascularization, that is, PCI or CABG) are immediately carried out at the same time, the use of the scoring system corresponding to the largest area under the decision curve will have the greatest clinical benefit.

All analyses were conducted with SPSS software (v26.0; IBM, Armonk, NY), MedCalc statistical software (v19.6.1; MedCalc Software Co., Ltd., Ostend, Belgium), and R software (v4.0.3, CRAN). The Z test was performed using MedCalc statistical software by the method of Delong et al.19. DCA was performed by the "rmda" package of R software. P value < 0.05 was considered statistically significant.

Results

Baseline characteristics

76,540 ICU admissions were contained in MIMIC-IV database, and finally, 522 patients were enrolled in this study (of which 104 died and 418 survived in hospital, Fig. 1). The length of hospital stay in the survival group death group was longer than in the death group (P < 0.001). Age, the level of white blood cell, anion gap, blood urea nitrogen, creatinine, INR, heart rate, and respiratory rate, SAPS II score, LODS score, and OASIS score in the death group was significantly higher than in the survival group (P < 0.001 for all), while the level of red blood cell, platelets, hemoglobin, systolic/diastolic/mean blood pressure, and temperature in the survival group was significantly higher than in the death group (P < 0.05 for all). There were no significant differences in length of ICU stay and the level of saturation of peripheral oxygen between the two groups. The prevalence of peripheral vascular disease, diabetes, and renal disease was significantly higher in the death group (P < 0.05 for all), while the prevalence of hypertension and the proportion of patients who underwent PCI/CABG were significantly higher in the survival group (P < 0.05 for all). The baseline characteristics of the study population are presented in Table 1. After propensity score matching, the matching variables are balanced and comparable between the two groups (P > 0.05 for all, see Ppsm in Table 1 for details), resulting 94 patients in the death group and 94 patients in the survival group.

Figure 1
figure 1

Flowchart of study cohort. ICU intensive care unit, STEMI ST elevation myocardial infarction, ICD International Classification of Disease, 10th Revision.

Full size image
Table 1 Demographic data of the study population.
Full size table

Logistic regression analysis

The three scoring systems (LODS, OASIS, and SAPS II) were all risk factors for in-hospital mortality in intensive care patients with STEMI (P < 0.001 for all) before adjustment. After adjustment, gender, heart rate, and underwent PCI/CABG or not were all independent risk factors for in-hospital mortality; In addition, LODS score and SAPS II were independent risk factors (OR: 1.447, 95% CI 1.208–1.732, P = 0.002; OR: 1.060, 95% CI 1.018–1.103, P = 0.004) for in-hospital mortality, while OASIS was not correlated with the mortality (OR: 0.962, 95% CI 0.912–1.015, P = 0.158) (Table 2).

Table 2 Binomial Logistic regression analysis for in-hospital mortality among intensive care patients with STEMI including the three scoring systems.
Full size table

Comparison of ROC curves

Before PSM, the AUCs of LODS, OASIS, SAPS II were 0.867, 0.827, and 0.894 for the study cohort, respectively (Fig. 2A). The AUCs were compared, resulting in LODS vs OASIS (Z = 2.365, P = 0.018), LODS vs SAPS II (Z = 1.846, P = 0.065), and OASIS vs SAPS II (Z = 3.551, P = 0.0004). The cut-off value corresponding to Youden’s index was selected as the optimal cut-off value for predicting in-hospital mortality. SAPS II had the highest sensitivity (89.42%) and Youden’s index (0.6382), while LODS had the highest specificity (87.80%). The results are presented in Table 3. After PSM, the AUCs of LODS, OASIS, SAPS II were 0.877, 0.821, and 0.881, respectively (Fig. 2B). The results of AUCs comparisons were LODS vs OASIS (Z = 2.477, P = 0.013), LODS vs SAPS II (Z = 0.210, P = 0.833), and OASIS vs SAPS II (Z = 2.613, P = 0.009). LODS had the highest sensitivity (73.40%) and Youden’s index (0.6064), while SAPS II had the highest specificity (90.43%). The results are presented in Table 4.

Figure 2
figure 2

(A) ROC curves of the scoring systems for the study cohort (before propensity score matching); (B) ROC curves of the scoring systems for the study cohort (after propensity score matching); (C) ROC curves of the scoring systems for the patients underwent PCI/CABG; (D) ROC curves of the scoring systems for the patients without PCI/CABG.

Full size image
Table 3 Comparison of ROC curves (before PSM).
Full size table
Table 4 Comparison of ROC curves (after PSM).
Full size table

We conducted a stratified analysis of the patients who underwent PCI/CABG or not. In the PCI/CABG group, the AUCs of LODS, OASIS, SAPS II were 0.853, 0.825, and 0.867, respectively (Fig. 2C). The results of AUCs comparisons were LODS vs OASIS (Z = 0.966, P = 0.334), LODS vs SAPS II (Z = 0.571, P = 0.568), and OASIS vs SAPS II (Z = 1.402, P = 0.161). LODS had the highest sensitivity (82.93%) and Youden’s index (0.6024), while OASIS had the highest specificity (83.39%). In the non-PCI/CABG group, the AUCs of LODS, OASIS, SAPS II were 0.857, 0.804, and 0.897, respectively (Fig. 2D). The results of AUCs comparisons were LODS vs OASIS (Z = 2.234, P = 0.026), LODS vs SAPS II (Z = 1.868, P = 0.062), and OASIS vs SAPS II (Z = 3.457, P = 0.0005). SAPS II had the highest sensitivity (95.24%) and Youden’s index (0.6190), while LODS had the highest specificity (92.38%). The results are presented in Tables 5 and 6.

Table 5 Comparison of ROC curves (patients underwent PCI/CABG).
Full size table
Table 6 Comparison of ROC curves (patients not underwent PCI/CABG).
Full size table

Comparison of Decision curves

The three decision curves (the study cohort before propensity score matching, Fig. 3; after propensity score matching, Fig. 4) all showed that the red line representing LODS were almost always above the yellow line representing SAPS II and the blue line representing OASIS (in descending order were LODS, SAPS II, and OASIS), which means LODS had the greatest net benefit ranges among the three scoring systems.

Figure 3
figure 3

Decision curve analysis of the three scoring systems for the study cohort (before propensity score matching). X-axis indicates the threshold probability for in-hospital mortality and Y-axis indicates the net benefits.

Full size image
Figure 4
figure 4

Decision curve analysis of the study cohort (after propensity score matching).

Full size image

Discussion

Patients with STEMI remain at increased risk of mortality if they survive the initial ischaemic event20. Rapid and accurate assessment of the severity affects the treatment and prognosis of STEMI critically21, especially for the patients in ICU. To the best of our knowledge, our study explored the predictive value of the three scoring systems (LODS, OASIS, and SAPS II) for in-hospital mortality of intensive care STEMI patients for the first time. We confirmed that both LODS and SAPS II have good predictive value, and all AUCs were greater than 0.85. Retrospective studies may have potential biases, therefore we performed a PSM analysis and the results still suggested that LODS and SAPS II have excellent predictive value. Considering whether underwent PCI/CABG or not may have a great impact on the prognosis of the study cohort, we conducted a stratified analysis, and the results still supported the above conclusions. Comparing LODS and SAPS II, there was no significant statistical difference between the two scoring systems (P > 0.05 for all), and in most cases, both were higher than OASIS in predictive value.

The OASIS scoring system contains 10 variables9, which is the least among the three scoring systems (Fig. 5). Actually, most of the intensive care patients with STEMI require emergency surgery, and the variable "elective surgery" may not apply to these patients. Meanwhile, the widespread use of PCI/CABG has significantly increased the survival rates of STEMI patients in the past decade, and most STEMI patients may not have the "pre-length of ICU stay". In addition, the focus of the treatment of STEMI patients is the recanalization of the coronary arteries and the rescue of the myocardium, while ventilation therapy is often used in the treatment of STEMI in an auxiliary form. The above factors ultimately lead to the lowest predictive value of the OASIS scoring system.

Figure 5
figure 5

Details of the three scoring systems.

Full size image

The sophisticated scoring systems require the collection of numerous physiologic measurements, making their applications in clinical practice difficult. The LODS scoring system contains 11 variables, and SAPS II contains 15 variables (Fig. 5). The common variables of LODS and SAPS II are mostly related to outcomes of STEMI. For example, the GCS score was confirmed to be higher in acute myocardial infarction patients than in acute trauma patients22. Furthermore, heart rate and blood pressure are undoubtedly closely related to myocardial infarction (especially in intensive care patients with STEMI), which characterizes the severity of the disease. Acute myocardial infarction is the most common cause of cardiogenic shock, and cardiogenic shock can lead to end-organ hypoperfusion (such as the kidney) and tissue hypoxia23. As a result, urine output and PaO2/FiO2 can reflect the severity and prognosis of myocardial infarction (including STEMI) to a certain extent correspondingly. The LODS scoring system has 4 fewer variables than SAPS II, but its predictive value is still comparable to SAPS II, which may be largely contributed by platelets and prothrombin time. Platelet activation and thrombin generation play key roles in intracoronary thrombus formation24. Platelets and prothrombin time largely reflect the conditions of platelet activation and thrombin generation, thus LODS is suitable for predicting the prognosis of STEMI. Although SAPS II contains many variables, electrolyte variables such as potassium, sodium, and bicarbonate tend to reflect the degree of metabolic acidosis in the form of anion gap (research showed that about 72% of critically ill patients have increased initial anion gap levels when they are admitted to the ICU, and the in-hospital mortality rate of patients with an elevated anion gap is much higher than that of patients without elevated anion gap25). For "comorbidity", this variable not only increases the workload when calculating scores but also may be more suitable for the prediction of long-term mortality; in addition, patients with STEMI are often admitted to the emergency department, therefore the variable "admission type" may not be sensitive and specific. In summary, LODS and SAPS II have similar predictive values, but LODS is more concise and maybe more acceptable to clinicians for application.

DCA could guide the decisions of treatment in clinical work. In our study, before/after PSM, the decision curves showed that LODS has the largest range under the curves (mainly in the high-risk threshold area between 0.4 and 0.8, Figs. 3 and 4), indicating that LODS had a slightly better net benefit in the threshold area. This is a large range of clinically reasonable preferences, therefore, considering that LODS not only has excellent predictive value for in-hospital mortality of intensive care patients with STEMI (as we discussed above) and greater clinical benefit, it may be recommended for clinical use.

Prior to this, researchers have made some explorations on the prediction of in-hospital mortality of STEMI patients. Koonsiripaiboon et al. used the Global Registry of Acute Coronary Events (GRACE) risk score to predict in-hospital mortality for 209 STEMI patients and found that the scoring system was validated26. Ugalde et al. predicted in-hospital mortality of ST elevation acute myocardial infarction using the Thrombolysis In Myocardial Infarction (TIMI) risk score and found that TIMI score was acceptably useful (the AUC for the ROC curve was 0.7)27. It should be noted that the above two scoring systems are commonly used to predict medium and long-term mortality, such as 30-day mortality, 180-day mortality, and 1-year mortality. In terms of short-term (such as in-hospital mortality) predictive value, however, there is still a lack of multi-center large sample clinical evidence. Moreover, GRACE and TIMI score are more difficult to score and require repeated evaluations, which rely heavily on dynamic electrocardiogram (ECG) changes and myocardial injury markers. The three scoring systems in this study are all applicable to ICU, and the predictive value of LODS has been confirmed. Therefore, the LODS scoring system may be more valuable for the prediction of in-hospital mortality for intensive care patients with STEMI.

There are several limitations to this study. Firstly, since it is currently impossible to extract imaging data such as ECG and coronary angiography from the MIMIC-IV database, it is difficult to obtain an accurate GRACE score and TIMI risk score. This study did not include the above two scoring systems for comparison; secondly, the database has too many missing values for some myocardial injury markers such as troponin on the first day of admission. We did not take these factors into consideration, which may affect the results; thirdly, the results of before/after PSM and stratified analysis found that the optimal cut-off values of each group were slightly different, which may be related to the variation between groups. Both patient characteristics and center characteristics may affect the results, and may even cause the scoring system to overestimate the mortality rate28; last but not least, the clinical strategies for patients with STEMI have changed over this relatively long time period (from 2008 to 2019), for example, from early revascularization of all vessels to revascularization of the culprit vessel only, indication for CABG etc. However, we cannot evaluate the biases caused by the changes. Since this study is a single-center study based on the American population, the conclusions of this study still need to be confirmed by well-designed prospective, multi-center clinical trials.

Conclusions

Among the three scoring systems, LODS and SAPS II have excellent predictive value, and in most cases, both were higher than OASIS. With a more concise composition and greater clinical benefit, LODS may be a better predictor of in-hospital mortality for intensive care patients with STEMI.