Introduction

Despite recent therapeutic advances, the World Health Organization assesses that injury causes 5.8 million death annually, and that this number is increasing worldwide1,2. Trauma-related mortality and disability is a major public health issue2,3. Moreover, the distribution of the timing of traumatic deaths is tri-modal3: traumas owing to non-salvageable injuries result in immediate deaths4; haemorrhagic or neurologic conditions, including haemorrhagic injuries of abdominal organs or expanding intracranial mass lesions, cause early death within the first 24 h following severe trauma4,5; and complications such as disseminated intravascular coagulopathy, sepsis, acute respiratory distress syndrome, and multiple organ dysfunction syndrome (MODS) cause late deaths days or weeks after injury in many trauma victims4,5. A strong immune response to traumatic injury is considered a critical factor in eliciting life-threatening conditions5. MODS remains the primary cause of late death after severe trauma, including up to 30% of deaths that are deemed possibly preventable6,7. Patents with MODS experience increased lengths of stay in intensive care units (ICUs) and may require renal replacement therapy6,7. Therefore, clinical improvement and survival in patients with major traumatic injuries require early recognition of their conditions and interventions to prevent the development of MODS6,7.

Technological advances in automated haematological analysers have enabled the acquisition of the delta neutrophil index (DNI) using leukocyte differentials obtained from two independent channels, the myeloperoxidase tungsten-halogen channel and the lobularity/nuclear density channel8,9. The DNI reflects the proportion of circulating immature granulocytes and is calculated as the difference between the leukocyte differentials measured in the myeloperoxidase channel and the differentials detected in the lobularity channel8,9. A previous study showed that the DNI (as calculated using an automated blood cell analyser) was strongly correlated with the manual immature granulocyte counts (r = 0.75, p < 0.05)10; an increase in the number of immature granulocytes in circulation is an important criterion in the diagnosis of systemic inflammatory response syndrome11. Several studies have found that a higher DNI value is closely associated with a positive blood culture, septic shock, disseminated intravascular coagulation, and mortality in critical care patients with suspected sepsis10,11,12,13. Recently, an increased DNI was also shown to reflect the severity of sterile inflammation-linked diseases such as out-of-hospital cardiac arrest, upper gastrointestinal haemorrhage, pulmonary embolism, and acute myocardial infarction8,14,15.

In patients with traumatic injury, systemic inflammatory response syndrome is abruptly initiated within 30 minutes of major traumatic injury, and is an inflammatory response to hypoperfusion and reperfusion owing to blood loss and tissue damage rather than infection2. As a result, the systemic response to severe injury leads to multiorgan failure (MOF) and sepsis that compounds the original traumatic injury2. Tissue injury after trauma releases damage-associated molecular patterns from necrotic cells, which activate immune cells such as neutrophils2. Increased pro-inflammatory cytokines activate neutrophils that elicit the destruction of healthy organ tissues with proteases and reactive oxygen species, and also sequestrate neutrophils in bystander organs by migrating across damaged endothelium2. Major loss of blood components, including neutrophils, occurs following an injury owing to vascular damage and massive haemorrhage16,17. To compensate for the tremendous depletion and demand for activated neutrophils following their marked loss owing to consumption, sequestration, and destruction of mature cells, the switch to emergency granulopoiesis considerably enhances the release of immature granulocytes by rapid production and turnover of cells into the peripheral blood from the bone marrow16,17,18,19. As a consequence of severe inflammation, the dysregulated neutrophil function, known as neutrophil paralysis, further impairs the migration of neutrophils, and causes unwanted antimicrobial responses and neutrophil sequestration in remote organs20. In patients with severe traumatic injuries such as haemorrhage and damage to organs and tissues, the release of immature granulocytes into circulation may contribute to the failure to protect against infection and the risk of MOF21,22.

To the best of our knowledge, no study has assessed the ability of serial DNI measurements to predict the clinical severities of MODS and short-term mortality, nor have any DNI cut-off values that predict the severity of trauma patients admitted to an ICU been derived. Hence, in this study, we investigated the clinical utility of the DNI as a predictor of severity in trauma patients admitted to an ICU.

Results

Study population and clinical evaluation

Figure 1 shows the enrolment and clinical outcomes of trauma patients admitted to the ICU. 366 patients (mean age 50.5 ± 18.8 years) with a mean ISS of 28.7 ± 11.5 were enrolled. Thirty-seven patients (10.1%) had died by day 30. The ISS was significantly greater in the non-survival group (41.5 ± 13.8) than in the survival group (27.3 ± 10.3; p < 0.001). Sixty patients (17.2%) developed MODS in the ICU (Table 1). The linear mixed model suggested that changes in DNI values from ED admission to 48 h later were different between the groups over time; there were significant changes in DNI values as correlated with 30-day mortality and MODS (Fig. 2A,B).

Figure 1
figure 1

Flow diagram of patient enrolment (A) and the criteria for trauma critical pathway activation (B).

Full size image
Table 1 Clinical characteristics of patients stratified by 30-day mortality and development of multiple organ dysfunction syndrome.
Full size table
Figure 2
figure 2

Linear mixed model of the delta neutrophil index (DNI) to estimate significant differences between groups over time according to 30-day mortality (A) and development of multiple organ dysfunction syndrome (MODS) (B). The receiver operating characteristic curves for predictability of the DNI at 12 h (C) and 24 h (D) after admission according to MODS development.

Full size image

DNI as predictor of 30-day mortality and MODS development

Univariate Cox and logistic regression analyses revealed that DNI values were significantly different at 12 h in patients who died within 30 days or experienced MODS versus those who did not (Supplement 1). The multivariable Cox regression model demonstrated that an increased DNI value at 12 h after ED admission (HR, 1.051; 95% CI, 1.024–1.079; p < 0.001) was a strong independent predictor of short-term mortality among severe trauma patients admitted to an ICU (Table 2). On multivariable logistic regression analysis, higher DNI values at 12 and 24 h after ED admission were also strong independent predictors of the development of MODS (Table 3). The AUCs for predicting MODS using the DNI at 12 and 24 h among patients admitted to an ICU with severe trauma were 0.776 (p- < 0.001) and 0.762 (p < 0.001), respectively (Fig. 2C,D).

Table 2 Multivariable Cox proportional hazards regression analysis for predictors of 30-day mortality.
Full size table
Table 3 Multivariable logistic regression analysis for predictors of the development of multiple organ dysfunction syndrome.
Full size table

Comparison of DNI and conventional clinical markers as predictors of 30-day mortality and MODS in critically ill trauma patients

To predict MODS development, ROC comparisons showed that the AUC of the DNI at 12 h was not significantly inferior to those of other markers (APACHE II, ISS, total transfused packed red blood cell volumes, and lactate on ED admission, and 24 h) except for the SOFA. Moreover, the DNI value at 24 h was not significantly inferior to those of other markers, including SOFA, APACHE II, and lactate, at 24 h. However, it was significantly superior to lactate and ISS on ED admission (Supplement 2). Using Youden’s method, the optimal cut-offs for the DNI values at 12 and 24 h were found to be 3.25% (sensitivity: 83.1 [71–91.6]; specificity: 72.2 [66.4–77.4]) and 2.85% (sensitivity: 75.4 [62.2–85.9]; specificity: 76.9 [71.5–81.8]), respectively. A DNI value > 3.25% was closely associated with the probability of MODS after trauma at 12 h after ED admission (OR, 12.7; 95% CI: 6.12–26.35; p < 0.001), as was a DNI value > 2.85% at 24 h (OR, 10.24; 95% CI: 5.26–19.92; p < 0.001) (Fig. 2C,D). To estimate the optimal cut-off values based on time-to-event data, we created Kaplan-Meier curves for 30-day mortality according to DNI values (Fig. 3); a log-rank test at 12 h after ED admission showed that these DNI cut-off values were independent predictors of 30-day mortality (p < 0.001). Harrell’s C-index for the prediction of 30-day mortality using the DNI was 0.876 (95% CI 0.831–0.916, p < 0.001) at 12 h after ED admission (Fig. 4A). The optimal DNI cut-off value for predicting 30-day mortality, the identification of which was a primary aim of this study, was 5.3% at 12 h after ED admission (p < 0.001). On further analysis using the Contal and O’Quigley technique, a DNI cut-off of >5.3% at 12 h after ED admission was significantly associated with an increased risk of 30-day mortality (HR, 18.11; 95% CI, 6.99–46.94; p < 0.001) among patients admitted to an ICU with severe trauma. Compared to the derived Harrell C-index, the C-statistic of the DNI at 12 h was not statistically inferior to those of other markers (i.e., it was similar to the SOFA, APACHE II, ISS, lactate at 24 h, and PT at 12 h). Conversely, the C-statistic of the DNI at 12 h was statistically superior to those of lactate, PT, and tCO2 on ED admission (Fig. 4B and Supplement 3).

Figure 3
figure 3

The delta neutrophil index (DNI) as a predictor of 30-day mortality. Higher DNI values at 12 h (A) and 24 h (B) after admission were significantly associated with an increased risk of 30-day mortality among trauma patients admitted to an ICU.

Full size image
Figure 4
figure 4

Comparison of Harrell’s C-index for the delta neutrophil index (DNI) for the prediction of 30-day mortality according to time (A); and comparison of the area under the curve (AUC) for the DNI when predicting the development of multiple organ dysfunction syndrome (MODS) according to time (B). The Harrell’s C-index and AUC showed discriminative abilities for the risk stratification of 30-day mortality and development of MODS (statistical information is shown in supplements 2 and 3). (A) *Harrell’s C-index of DNI at 12 h was statistically superior to lactate, prothrombin time, and tCO2 on emergency department admission (p < 0.05). (B) *AUC of DNI at 12 h was statistically superior to lactate on emergency department admission and Injury Severity Score (p < 0.05). AUC of DNI at 12 h was statistically inferior to the Sequential Organ Failure Assessment score at 24 h after emergency department admission (p < 0.05).

Full size image

The prognostic value of DNI when combined with conventional risk factors

The IDI and NRI are indicators of improvements following reclassification in a nested model, and can thus determine whether the predictive power of a combination of traditional factors is improved upon DNI inclusion. The addition of the DNI at 12 h after ED admission yielded a significantly positive IDI with respect to predicting 30-day mortality; the continuous NRI was also positive and was significant for the DNI at 12 h. Additionally, the diagnostic performance was significantly improved by adding DNI to the reference model for predicting MODS according to the IDI and NRI (Table 4).

Table 4 Comparison of the performance of the survival models (A) and multiple organ dysfunction syndrome (B) with and without the delta neutrophil index (DNI) by Harrell’s C-index, area under the curve operating characteristic (AUROC), integrated discrimination improvement (IDI), and continuous net reclassification improvement (NRI).
Full size table

The effect of early significant intervention on the predictability of DNI values

Of the 366 patients, 213 (58.2%) underwent at least one of the early significant interventions (transfusion, n = 173, embolization, n = 66, and surgery, n = 101) within 12 hours after ED admission. We compared the predictability of DNI values for the development of MODS and 30-day mortality between patients who underwent at least one of the early significant interventions within 12 hours after ED admission and patients who received conservative management. There was no significant difference of the predictability of DNI values for the development of MODS between intervention (0.806 (95% CI: 0.734–0.879)) and conservative (0.846 (95% CI: 0.732–0.960)) groups (p = 0.563)). There was also no significant difference for predictability of DNI values for the 30-day mortality between the intervention (0.841 (95% CI: 0.784–0.884)) group and the conservative (0.917(95% CI: 0.883–0.98)) group (p = 0.157). Consequently, the predictability of DNI values in patients who underwent early significant intervention was similar to the total enrolled patient population and patients who received conservative management (Supplements 4 and 5).

Discussion

We demonstrated that an increased DNI value, which reflects the fraction of immature granulocytes, was a significant independent predictor of the development of MODS and 30-day mortality. Despite advances in scoring systems and laboratory markers, the early recognition of severe complications such as the development of MODS and risk stratification for late death in traumatic injury remains challenging. The SOFA was originally developed as a descriptive score23; its final score is based on the degree of dysfunction of six organ systems as measured via several physiologic and laboratory results23. Although the SOFA is used to determine multiple organ failure (MOF) in trauma, it does not propose a cut-off value for MOF development23. The APACHE II score estimates mortality in the ICU based on a number of laboratory values and indicators of both acute and chronic diseases24. The ISS as an anatomical scoring system that provides an overall score for patients with multiple injuries and correlates linearly with mortality24. However, it is often difficult to fully determine the extent of injury before thorough investigation (and if required, surgery).

Neither the SOFA nor APACHE II scoring system predicts MODS after injury24. Durham et al. demonstrated that MOF was associated only with APACHE III; total blood products transfused in the first 24 h, and lactate levels at 24 h25. Our data showed that the AUCs of the DNI at 12 and 24 h were not statistically inferior to those of the APACHE II, total transfused packed red blood cell volumes, or lactate level at 24 h. Considering the efforts required to predict severity, including serial measurements over time, these scoring systems are excessively complicated for patients with trauma26.

As a screening tool for severely traumatized patients, we found that the DNI at 12 h after ED admission could predict the development of MODS and short-term mortality earlier than several other scoring systems. Inflammatory cytokines and C-reactive protein are also able to predict the development of organ failure with a high level of accuracy in traumatized patients27; even before clear clinical symptoms develop, interleukin-6 (IL-6) kinetics in the first days of hospitalization may predict the development of MOF27. However, it is difficult to estimate the development of MOF using levels of C-reactive protein (that is increased in response to (IL-6 secretion) 24 h after admission27. Although several pro-inflammatory cytokines like IL-6 can be usefully applied in clinical situations, additional costs, time, and new equipment must be considered for single or repeated measurements to predict the severity of the patients’ conditions. Recent studies have explored the DNI as an adjunct predictor because inflammatory markers that can be measured rapidly, easily, serially, and inexpensively are an unmet need9,26. The automatically calculated DNI is a more prompt and accurate method than manually counting immature granulocytes10,14. Several studies have shown that an increased DNI is associated with poor outcomes in specific disease conditions and can predict the severity of infection and sterile inflammation9,10,11,14,15. Although the pathophysiologic mechanisms by which DNI values are associated with severity of disease are not fully understood, several studies have elucidated the mechanisms by which immature granulocytes are released early and rapidly during severe inflammation9,15,26.

In the present study, we found that DNI values of >3.25% and >5.3% at 12 h post-admission were significant predictors of the development of MODS and 30-day mortality, respectively, in our trauma patients. Park et al. reported that severe sepsis and septic shock were significantly increased in patients with DNI > 6.5% in the first 24 h after ICU admission11. Yune et al. also demonstrated that a DNI > 10.5% on day 1 was associated with a higher 30-day mortality rate after surviving out-of-hospital cardiac arrest14. Taken together, the data show that the DNI value reflects the severity of diseases associated with systemic and sterile inflammation. The use of the DNI, which can be measured rapidly, easily, and inexpensively, can be used to assess the severity of several diseases because of the benefit of being automatically analysed as part of the complete blood count. Moreover, the present study revealed that the DNI value at 12 h post-ED admission predicted 30-day mortality with a reliability similar to those of the SOFA, APACHE II, ISS score, and lactate level at 24 h. Additionally, the DNI value 24 h post-admission was not inferior to the other factors (except for the SOFA) in predicting 30-day mortality. The DNI reflects the severity that is associated with MODS and short-term mortality, and produces predictive values that are statistically similar to several other scoring systems. Therefore, we cautiously propose that DNI values can be an ancillary test for the early prediction of injury/disease severity 12 h before other scoring systems such as the ISS, SOFA, and APACHE II, and is a serially measurable marker for monitoring severity over time in patients with trauma.

This study had several limitations. First, despite using the prospective TRAUMA CP registry that had a standardized and predetermined protocol at our institution, our data from a single, tertiary, academic hospital were analysed retrospectively. We could not mandatorily and serially measure certain classification systems such as the Trauma and Injury Severity Score (TRISS), New Injury Severity Score, and APACHE III, nor could we investigate other inflammatory mediators such as tissue necrosis factor and other pro-inflammatory cytokines that are known indicators of trauma severity in our TRAUMA CP protocol28. Therefore, it was difficult to control for confounding factors, which increased the possibility of selection bias. However, we did investigate the TRISS (ISS plus the revised trauma score including systolic blood pressure, respiratory rate, and Glasgow Coma Scale) at limited time points. We also constructed a multivariable regression model considering the risk of multicollinearity and the possibility of overestimation or underestimation. We used the ISS, SOFA, and APACHE II scores as factors in this model because these scoring systems inherently introduced variability by not being applied consistently in our trauma patients. Sudarsanam et al. demonstrated that the APACHE II and APACHE III scores were strongly associated, with a correlation score of 0.829. In future studies, to validate the prognostic potential of DNI, a prospective study is needed to compare the predictive power of DNI values and other reliable indicators of organ dysfunction (such as APACHEIII, New Injury Severity Score, and cytokines) as an independent risk factor to add to reference prediction models. A second limitation was that we excluded patients admitted to the NCU to achieve homogeneity; hence, patients with severe brain injury may not have been included in this study. However, severe injury isolated to the brain is closely associated with higher mortality. Third, in an evaluation of the usefulness of a new marker when the marker is added to reference models with sufficient predictive power or standard risk factors, the application of AUC may be too conservative and the NRI and IDI may be overestimated. The statistical interpretation needs to be carefully considered due to the limitations of the two statistical methods.

Finally, our study did not clarify the precise mechanisms by which the DNI is associated with severity after traumatic injury. Large, prospective, randomized clinical trials, as well as additional targeted investigations, are required to address each of these limitations.

In conclusion, an increased DNI value is an independent predictor of MODS development and 30-day mortality in trauma patients admitted to an ICU. The DNI is routinely and easily measurable as part of the complete blood count without additional cost or time, and can therefore be considered a suitable parameter for the early estimation of the severity of a traumatic injury.

Methods

Study population and the TRAUMA critical pathway (CP)

A retrospective cohort single-centre study was performed in the emergency department (ED) of an urban, university-affiliated, tertiary referral medical centre with an annual census of approximately 85,000 visits. The study was approved by the institutional review board (No. 3-2017-0188) of Yonsei University Health System, which waived the requirement for written informed consent because of the retrospective nature of the study. All methods were performed in accordance with the relevant guidelines and regulations. This study enrolled eligible adult patients who were admitted to the ICU after enrolment in the TRAUMA CP of the ED between January 1, 2011, and May 31, 2017. Figure 1 shows the enrolment, TRAUMA CP alerting criteria, and clinical outcome data of patients suspected of having sustained severe trauma. To validate the usefulness of serial DNI measurements over time, we excluded patients who transferred from another hospital before admission to our institution’s ED, those who transferred out to another hospital within the first 96 h, those discharged against medical advice, and those who died in the ED. Additional exclusion criteria were a history of haematological malignancy or autoimmune disease, admission to the independent neurosurgical care unit (NCU) for isolated head injuries, and current pregnancy; moreover, those who died within the first 96 h after injury were not included in MODS-related analyses based on predetermined guidelines1,23. Considering the differences in intensive management, we excluded trauma patients treated in the NCU to maintain the homogeneity of the study cohort.

Our institution schedules specialized traumatologists and interventional radiologists to be on call 24 h/day, 7 days/week, to treat patients who arrive at the ED requiring emergency surgical and radiological interventions. In 2011, our TRAUMA CP was designed and implemented for the effective management of adult patients (≥18 years) suspected of having sustained severe trauma to reduce unnecessary in-hospital time delays through a computerized provider order entry-based alert system, short message service, and simple standing orders through the activation stage28. TRAUMA CP records were prospectively obtained based on a predetermined protocol. In the triage area of the ED, physicians, nurses, or emergency medical technicians screened candidates for the TRAUMA CP program as soon as feasible using the Advanced Trauma Life Support (ATLS) guidelines28. When a patient had at least one of seven warning mechanisms and one of 10 clinical suspicions of severe trauma (known as the TRAUMA CP alerting criteria) on ED admission, an ED physician in the triage area activated the TRAUMA program by selecting the activation icon on the order-entry window (Fig. 1). The computerized provider order entry system highlights the name of the patient in purple and simultaneously submits medical orders according to a predetermined program. According to the predetermined protocol based on the ATLS guidelines, the traumatologists and emergency physicians immediately evaluate the patient and simultaneously administer treatment28. Even if the system was activated, we excluded patients with minor injuries from our program to preserve the cost-effective use of medical resources and limit radiation hazards. In the emergency setting, medical staff reassessed each patient by performing a physical examination and ‘focused assessment with sonography for trauma’, and decided whether to include the patient in this program28. This protocol was applied for 24 h/7 days, whereupon severe trauma patients received standard management from the ED to the ICU.

Data collection

We collected baseline characteristics including sex, age, previous medical history, mechanism of injury, mental status on arrival, vital signs, laboratory values, Injury Severity Score (ISS), and follow-up data for all patients based on a predetermined protocol. We also investigated whether packed red blood cells, fresh frozen plasma, and platelets were transfused in the first 24 h, and noted the main types of emergency treatment performed within 24 h of ED admission (including emergency surgery, angioembolization, and conservative care). The Sequential Organ Failure Assessment (SOFA) and Acute Physiology and Chronic Health Evaluation (APACHE) II were obtained from the TRAUMA CP records to determine the clinical severity of each patient’s condition. Additional data were extracted from electronic sources on radiology, ICU progress notes, nursing charts, and discharge summaries.

The DNI values were obtained for all patients within 10 minutes of ED admission as well as 12 ± 3 h, 24 ± 3 h, and 48 ± 6 h after admission. The DNI for each patient was determined using venous blood in ethylenediaminetetraacetic acid (EDTA)-containing vacutainers on presentation to the ED. To assess the DNI, we used the same type of haematology analyser (ADVIA 2120; Siemens, Forchheim, Germany) used for the analysis of the complete blood count. We also conducted other laboratory tests including measuring white blood cell counts and levels of blood urea nitrogen, creatinine, electrolytes, alanine transaminase, and lactate at the time of ED admission.

DNI measurement

The analysers used to determine DNI were optical systems based on a cytochemical myeloperoxidase tungsten-halogen channel (which measures and differentiates neutrophils, eosinophils, lymphocytes, monocytes, and large unstained cells based on size and myeloperoxidase staining intensity), and a laser-diode channel (that calculates, classifies, and counts cell types based on lobularity/nuclear density and size). The DNI was then calculated by subtracting the fraction of mature polymorphonuclear neutrophils from the sum of the myeloperoxidase-reactive cells, thus deriving the circulating immature granulocytes as the leukocyte sub-fraction10,12,14.

Clinical endpoints

Our primary endpoint was the development of MODS during in-hospital admission, which was defined as a Sequential Organ Failure Assessment Score of 6 or more on two or more consecutive days at least 48 h after ED admission5. Additionally, the secondary endpoint was all-cause mortality within 30 days of ED admission after sustaining trauma.

Statistical analysis

Demographic and clinical data are presented as percentages or frequencies, medians (interquartile ranges), and means ± standard deviations, as appropriate. We compared categorical variables using the χ2 test or Fisher’s exact test and continuous variables using a two-sample t-test or the Mann–Whitney U-test. Using a linear mixed model and repeated measures covariance pattern with unstructured covariance within patients, we evaluated significant differences between groups over time. Two fixed effects represented the clinical effect (level: survival vs. death by day 30 and development vs. non-development of MODS) and time effect (time: DNI determined on admission and at 12 h, 24 h, and 48 h after ED admission). Differences in clinical effect over time were analysed by the formula “clinical effect × time”.

We also measured significant differences concerning the development of MODS. Univariate Cox proportional hazards regression and logistic regression analyses were used to evaluate relationships among demographic characteristics and clinical data for all-cause mortality at 30 days and the development of MODS. Multivariable Cox proportional hazards regression that integrated the major covariates (variables with a p-value < 0.05) identified by our univariate Cox proportional hazards regression analysis was used to determine potential independent predictors of 30-day mortality by considering time-to-event data. Additionally, multivariable logistic regression analysis was performed to assess potential independent predictors of the development of MODS. The results of multivariable Cox proportional hazards regression analyses are expressed as hazard ratios (HRs) and 95% confidence intervals (CIs). Those of multivariable logistic regression analysis are represented as odds ratios (ORs) and 95% CIs. We determined the area under the curve (AUC) to assess the ability of the DNI to predict the development of MODS using receiver operating characteristic (ROC) curves. We assessed the optimal cut-off for the DNI using Youden’s method for discriminating the development of MODS in trauma patients admitted to an ICU. Next, ROC curves were constructed to assess the diagnostic performance of the DNI and other clinical parameters, and the AUCs were calculated and compared. We also performed net reclassification improvement (NRI) and integrated discrimination improvement (IDI) to analyse the degree to which the addition of DNI to the reference model improved its predictive ability30.

We constructed Kaplan-Meier survival curves using 30-day mortality data, and compared groups using the log-rank test. The optimal cut-off values for the dichotomization of the clinical outcome variable were estimated based on time-to-event data using the technique devised by Contal and O’Quigley14,31. The optimal cut-off point was selected by maximizing the HR. To investigate the additional predictive power of the DNI at each time point, Harrell’s C-index was assessed to examine the time-dependent discriminatory ability against SOFA, APACHE II, and ISS on ICU admission; lactate, total CO2 and prothrombin time on ED admission; and lactate and prothrombin time 24 h after ED admission versus the ability of DNI to predict 30-day mortality32. To assess the 95% CI and p-values for the IDI and continuous NRI, as well as the C-index and differences between models, a standard bootstrap method was used with resampling performed 1,000 times. We also assessed the continuous NRI and IDI at the final follow-up visit (30 days) to assess the improvement in performance of the survival model when DNI (as an additional parameter) is included9,33,34. To identify the effect of early significant intervention including transfusion, embolization, and surgery on the predictability of DNI values, we reviewed medical records to identify patients who received significant intervention before obtaining DNI within 12 h after ED admission. We compared the predictability of DNI values for the development of MODS and the 30-day mortality between those that received early intervention and those that had more conservative management using ROC curves with Delong method and Harrell’s C-index. Statistical analyses were performed using SAS, version 9.2 (SAS Institute Inc., Cary, NC); R software, version 3.2.5 for Windows (the R foundation for statistical computing, Vienna, Austria; http://www.R-project.org/); and MedCalc, version 12.7.0 (MedCalc Software, Ostend, Belgium). A p-value < 0.05 was considered significant.

Ethics approval and consent to participate

The study was approved by the institutional review board (No. 3-2017-0188) of Yonsei University Health System, which waived the requirement for written informed consent because of the retrospective nature of the study. All methods were performed in accordance with the relevant guidelines and regulations.