Introduction

Neonatal sepsis is a common, deadly problem with global impact including significant morbidity and mortality, even in high-resource countries.1,2 The greatest incidence and impact of neonatal sepsis is seen in the smallest, most prematurely born infants. In a cohort of 5100 extremely preterm infants (born before 29 completed weeks of gestation), 34% experienced late-onset sepsis (LOS, a positive blood culture with a bacterial or fungal organism after 72 h of life), which was associated with 18% mortality.3 LOS survivors demonstrate significant neurodevelopmental impact, which is superimposed on the impact of premature birth, and extends into the second decade of life.4,5

The distinction of infection from sepsis is not widely recognized in the neonatal intensive care unit (NICU). In contrast to sepsis definitions in adults and children, definitions of sepsis commonly used in neonatology are variable and heavily predicated on the isolation of pathogens from blood and/or the associated length of prescribed antimicrobial treatment.6 However, establishing the definition of sepsis on bacteremia and the accompanying treatment alone will not further advance either research or clinical decisions to reduce mortality.7 The definition of sepsis is “life-threatening organ dysfunction caused by a dysregulated host response to infection.”8 Organ dysfunction indicates a pathobiology more complex than simply infection plus an accompanying inflammatory response. The presence of life-threatening organ dysfunction is demonstrated using a sequential organ failure assessment (SOFA) to determine risk of ICU admission or mortality.8 Because admission to a NICU for preterm infants is not optional, ICU admission is not applicable in this population. Therefore, mortality, the most severe outcome, must be chosen as the initial outcome of interest. To define sepsis in neonates therefore requires an operational definition of organ dysfunction applicable specifically to this population [neonatal SOFA (nSOFA)] that predicts mortality in the setting of presumed infection. This deficiency specifically prevents the establishment of a consensus definition for neonatal sepsis that is a prerequisite for improved sepsis outcomes. While defining infection is not the objective of this work, identification of patients with infection that have an increased risk of mortality based on objective organ dysfunction is the goal and is the prerequisite to redefining sepsis in this population.

We recently showed the progression of organ failure in neonates with lethal LOS in a large retrospective cohort.9 The need for mechanical ventilation, oxygen requirement, requirement for cardiovascular support in the form of vasoactive drugs, and the prevalence of thrombocytopenia all significantly increased during the progression to mortality with LOS. Guided by those data, we developed and tested an objective, electronic health record (EHR)-automated, nSOFA scoring system to predict mortality from LOS in premature, very low birth weight (VLBW) infants at our center.

Methods

Study population

This retrospective case–control study was approved by the University of Florida (UF) Institutional Review Board (IRB; waiver of consent) prior to the collection of any data. Following IRB approval, an integrated data repository of all VLBW newborns born at <33 weeks gestation (by obstetric estimate) admitted to the UF Health, level IV NICU between January 2012 and September 2016 without patient identifiers was created via an honest broker. This repository included all data recorded in the EHR and was used for this work. Infants with LOS were identified. Infants that were transferred to the UF Health NICU from outside hospitals that developed signs of sepsis prior to transfer were excluded because the variables of interest were not all available in the EHR.

Definitions

LOS was defined as an event that met all of the following criteria before data were extracted to reduce potential bias: (1) the event occurred after the third day of life,10 (2) subjective clinician concern for serious infection (demonstrated by a blood culture being performed, empiric antimicrobial treatment started at evaluation and continued for at least 7 days or until death), and (3) a single bacterial/fungal pathogen isolated from blood or death with necrotizing enterocolitis (NEC). All C-reactive protein (CRP) measurements made during the episode (<7 days from the blood culture or until death) were collected to objectively demonstrate the presence of inflammation. Coagulase-negative Staphylococcus isolates were included only if there was evidence of significant CRP elevation (≥45 mg/L) during the episode.11

Components of the nSOFA score

The organ systems selected and scoring for the nSOFA score were guided by our large retrospective study that chronicled the progressive organ dysfunction that occurred among 679 infants hospitalized in the NICU with lethal LOS.9 The nSOFA utilizes categorical scores (total score range 0–15) to objectively describe dynamic changes in: (1) the need for mechanical ventilation and oxygen requirement (score range 0–8), (2) the need for inotropic support including the use of corticosteroid support (for presumed adrenal insufficiency or catecholamine-resistant shock) (score range 0–4), and (3) the presence and degree of thrombocytopenia (score range 0–3) (Table 1). A description of the nSOFA components is presented in detail in Supplemental Data. nSOFA scores were calculated for each infant at 9 time points surrounding the sepsis evaluation (−48 h, −24, −12, −6, evaluation, +6, +12, +24, +48).

Table 1 Neonatal sequential organ failure assessment (nSOFA) components and scoring (0-15 point range)
Full size table

Statistics

Continuous variables were summarized as a median with quartiles (25th and 75th percentiles). Categorical variables were summarized using percentages. Fisher’s exact tests were used for categorical variables and Wilcoxon signed-rank test or Mann–Whitney was used for continuous variables. Areas under the receiver operating characteristic (ROC) curves (AUCs) were calculated. Graph Pad Prism (version 7) was used for all calculations.

Results

We identified 44 sepsis survivors and 16 non-survivors who met our definition of LOS (Table 2). The groups were similar with respect to birth weight, gestational age, delivery mode, sex, and race. Non-survivors experienced sepsis earlier after birth compared with survivors (median day of life 19 vs 11, p = 0.002). Sepsis survivors exhibited a greater CRP peak compared to non-survivors (152 mg/L vs 91 mg/L, p = 0.003), which was likely affected by either the time from evaluation to death for non-survivors (median 54 h) or that providers may not have repeated CRP in non-survivors due to a perceived lack of clinical indication. Among LOS survivors, 25 (57%) had blood cultures positive for Gram-negative bacteria; there were two cases of Candida and the remaining infants (n = 17, 39%) had blood cultures positive for Gram-positive bacteria (Supplemental Table S1). Among non-survivors, 6 (38%) patients had Gram-negative bacteremia, 5 (31%) had Gram-positive bacteremia, 4 (25%) had negative cultures (all had Modified Bell’s stage ≥2 NEC), and one had a blood culture positive for Candida albicans.

Table 2 Demographics of the study population with late-onset sepsis
Full size table

Median nSOFA scores between survivors and non-survivors prior to the LOS evaluation were different at −48 [0 (quartiles 0, 2) vs 2 (0, 3); p = 0.001], −24 [0 (0, 2) vs 2 (0, 3); p = 0.042], and −12 h time points [0 (0, 2) vs 2 (0, 6); p = 0.020] but not at −6 h (Fig. 1). nSOFA scores at evaluation and at all points measured after evaluation were different between survivors and non-survivors [evaluation: median 1 (quartiles: 0, 2) vs 6 (2, 9); +6: 1 (0, 4) vs 6 (2, 10), +12 h: 1 (0, 3) vs 8 (6, 11); +24: 2 (0, 3) vs 11 (10, 13); +48: 2 (0, 3) vs 10 (7, 14); all p < 0.001]. Significant changes between survivors and non-survivors within nSOFA score component scores occurred in the respiratory [−48 (p = 0.016); −12 (p = 0.018); evaluation, +6, +12, +24, and +48 (all p < 0.001, Supplemental Fig. S1A), cardiovascular (all time points except −48, p < 0.05; Supplemental Fig. S1B), and platelet component scores (+12 and +24, p < 0.05; Supplemental Fig. S1C). Examination of all nSOFA scores from survivors at all time points showed 91% of nSOFA scores calculated were ≤4. Mortality was higher at evaluation (13% vs 67%, p < 0.001), +6 h (15% vs 64%, p = 0.002), and +12 h (7% vs 71%, p < 0.001) among patients with an nSOFA score of >4 as compared to patients with a score of ≤4. Mortality correlated with the maximum nSOFA score (nSOFAmax) at evaluation or +6 h (form 13% for nSOFAmax <2 to 88% for nSOFAmax >8) (Fig. 2). Because mortality among non-survivors reached 25% by 24 h and 50% by 48 h, mortality risk was not calculated at these later time points.

Fig. 1
figure 1

Neonatal sequential organ failure assessment (nSOFA) total scores among survivors and non-survivors around sepsis episode. All data points shown for both groups at all time points. Median values and interquartile ranges are shown

Full size image
Fig. 2
figure 2

Mortality risk based on the neonatal sequential organ failure assessment (nSOFA) maximum at evaluation or +6 h. Mortality (%) was determined by using the maximum nSOFA value measured at evaluation or +6 h among the 44 survivors and 16 non-survivors

Full size image

A change in nSOFA score (nSOFAΔ) over time may also provide useful information on the risk of sepsis mortality (Fig. 3). Prior to evaluation, non-survivors experienced a significant median nSOFAΔ compared to survivors [−48 to evaluation: 4 (0, 5) vs 0 (0, 2); p = 0.002; −24 to evaluation: 3 (0, 5) vs. 0 (0, 2), p < 0.001] (Fig. 3a). No differences in median nSOFAΔ between survivors and non-survivors were seen in the intervals −48 to −24, −24 to −12, or −12 to −6 h. However, a difference in median nSOFAΔ was found from −6 to evaluation for non-survivors as compared to survivors [2 (0, 5) vs 0 (0, 2); p = 0.021]. As expected, a significant increase in median nSOFAΔ occurred in non-survivors compared to survivors over multiple time intervals surrounding evaluation [−6 to +6; 2 (0, 4) vs. 0 (0, 0), −12 to +12; 5 (2, 9) vs 0 (0, 0), −24 to +24; 9 (3, 12) vs 2 (0, 3), −48 to +48; 9 (3, 12) vs 1 (0, 2); all p < 0.001] (Fig. 3b). Six patients among the non-survivors experienced little change in the nSOFA scores during the 96-h interval we examined. All six demonstrated clinical decline and died after 48 h. Three patients had NEC totalis and one had pulmonary hypoplasia.

Fig. 3
figure 3

The change in neonatal sequential organ failure assessment (nSOFA) scores among sepsis survivors and non-survivors. a Change in patient nSOFA scores in the interval shown prior to the sepsis evaluation. b Change in patient nSOFA scores in the interval shown surrounding the sepsis evaluation. Each symbol represents the intra-individual difference in nSOFA scores between the respective time points (nSOFAΔ). Median values and interquartile ranges shown. If mortality occurred prior to a timed score, then the most proximal nSOFA score was substituted to allow calculations

Full size image

To examine the utility of the nSOFA at the individual patient level for mortality, an ROC curve showed an AUC of 0.77 at evaluation (95% confidence interval 0.62–0.92; p = 0.001), 0.79 at +6 h (0.66–0.92; p < 0.001) and 0.93 at +12 h (0.86–0.997; p < 0.001) (Fig. 4). Combined sensitivity and specificity were maximized using an nSOFA score of ≥3 [evaluation (75%, 77%); +6 (75%, 68%); +12 (100%, 70%)]. The median time to death was 54 h (quartiles: 22–188) and all infants who died received parenteral antibiotics from the time of sepsis evaluation until death. No deaths occurred prior to 12 h.

Fig. 4
figure 4

Receiver operating characteristics curve for neonatal sequential organ failure assessment to predict mortality at the specified time points. Area under the curve (AUC) was 0.77 at evaluation (95% confidence interval 0.62–0.92; p = 0.001), 0.79 at +6 h (0.66–0.92; p < 0.001), and 0.93 at +12 h (0.86–0.997; p < 0.001)

Full size image

Although the nSOFA scores for the nine time points we measured highlight the dynamics of sepsis-induced organ dysfunction, these time points only represent a fraction of the possible granularity when nSOFA scores are calculated from data collected as part of routine care and saved in the EHR. As representative examples, longitudinal profiles of nSOFA scores for a survivor (Supplemental Fig. S2A) and non-survivor (Supplemental Fig. S2B) with LOS caused by Klebsiella pneumoniae demonstrate the individual dynamics and complexity of this disease. We integrated the nSOFA score algorithm into our EHR system (EPIC), so real-time nSOFA score profiles can be added to a patient list for any neonatal care provider (Supplemental Fig. S3A). A color-coded designation reflects the severity of the current nSOFA score, and a mouse-hover-over function allows the user to quickly see the individual component scores for each patient. The nSOFA-related data fields (SpO2, FiO2, mechanical ventilation, vasoactive drugs, steroids, platelet count) are updated in real time as any new data are made available and nSOFA scores, including individual component scores, are recorded in the EHR every 15 min (Supplemental Fig. S3B). nSOFA scores are also plotted in an accordion view to allow graphic representation of the change over a selected interval of time (Supplemental Fig. S3C). Thus, as new real-time data are acquired, the nSOFA updates automatically eliminating the need for nursing to chart or calculate nSOFA scores. For geographic areas or health-care systems that do not use EPIC, we made the web-app nSOFA calculator available online: (http://www.peds.ufl.edu/apps/nsofa/default.aspx).

Discussion

Our goal was to develop an objective, automated, evidence-based organ dysfunction scoring system dependent on routinely collected data and recorded in an EHR that can identify NICU patients at a high risk of mortality among those with LOS. The nSOFA is applicable specifically to this patient population and predicts mortality in the setting of LOS. We found comparable trajectories of organ dysfunction for respiratory, cardiovascular, and platelet components in this single-center, retrospective cohort of VLBW preterm infants as we saw in the prior study of progressive organ failure with lethal LOS.9 We were able to successfully integrate and automate the nSOFA scoring system into EPIC, a commonly used EHR platform.

In adults, a rise in the SOFA score of ≥2 points in the setting of presumed infection has an AUC 0.76 for mortality.12 We found similar AUCs for the nSOFA to predict mortality at the time of sepsis evaluation and at +6 h that rose to 0.93 by +12 h in this cohort of neonates with LOS. Although validation of the nSOFA in another cohort of neonates with presumed infection is necessary, these data suggest that the nSOFA could perform similarly to the adult SOFA and have utility at the individual patient level. If it is shown that the nSOFA score successfully predicts sepsis mortality in subsequent neonatal cohorts, use of the nSOFA may facilitate a reduction in the number of patients required in interventional randomized clinical trials aimed at reducing sepsis-related mortality.

While the goal is to develop an organ dysfunction system that is: (1) based only on objective criteria that predicts sepsis mortality risk, (2) applicable to both EOS and LOS, and (3) is useful across all gestational ages, this as an initial step may be unrealistic. Significant organ dysfunction is frequently present at birth among VLBW infants that are not infected secondary to physiologic immaturity. Infections that begin in utero, and present immediately after birth (early-onset), are superimposed upon the significant physiologic and molecular alterations that accompany preterm birth and the transition to extrauterine life.11 Machine-based learning approaches on much larger cohorts would be desirable. However, the lack of a consensus definition in this population, in conjunction with the strong emphasis on microbiologic criteria alone rather than life-threatening organ dysfunction, places the validity of the primary exposure (sepsis) for those patients in question. This limitation would be expected to decrease the signal-to-noise ratio and mask any potential benefit. The majority of microbiologically confirmed sepsis episodes and mortality in the NICU are due to LOS and occur in the VLBW population, which is why we have focused our initial efforts in these areas.

As we have proposed, the nSOFA score could and likely will expand to include other measures of organ dysfunction as they prove meaningful to sepsis mortality in evidence-based studies.13 For example, in the adult SOFA score, central nervous system (CNS), liver, and renal dysfunction are included. Currently, measures of these systems in neonates are more problematic. For example, in adults a total bilirubin of 5 mg/dL represents hepatic dysfunction8 and yet would be absolutely within normal limits for neonates for several days after birth. Changes in renal function including creatinine and urine output may be likely additions to nSOFA scoring in the future, but these have many possible caveats (fluid intake, diuretics) that require more interpretation than the objective data used in the current iteration of the nSOFA. CNS dysfunction scores in adults and children are based on the Glasgow coma scale system.8,14 CNS function assessments in neonates, and particularly in preterm neonates, are broad and primarily subjective (decreased activity, lethargic). Incorporation of an objective CNS component, for example, a change in regional cerebral oxygen saturation, if predictive of mortality in preterm neonates as in preliminary studies in adults with septic shock,15 will be a positive addition to the nSOFA. The addition of platelet transfusion in response to low values and the response to transfusion may also add predictive value.16

One of the significant achievements of this work was the successful implementation of the nSOFA scoring system into the EHR. The integration of nSOFA scores automatically recorded every 15 min into a commonly used EHR system without burdening the bedside nurse to document should have a significant impact on multiple unmet needs related to neonatal sepsis. The real-time automatically updating nSOFA score has great potential to help covering clinicians and nursing leadership with shift-based unit staffing needs and may improve patient safety outcomes. Notification of clinical providers when a threshold score is reached or a rate of change that is concerning occurs may provide additional utility, especially for large NICUs.

We acknowledge that this study has several limitations inherent to a single-center, retrospective case–control study. One advantage of a single-center study is the ability to achieve adequate data granularity, which was critical to defend the primary exposure (LOS). Confirmation of the utility of the nSOFA to predict mortality risk will require testing in another cohort of infants. A study of the utility of prospectively recorded nSOFA scores, including the determination of the association between nSOFA scores and mortality risk, as well as with survivor outcomes, is underway at our institution. We did not have adequate numbers of patients to compare the utility of the nSOFA by primary sites of infection (peritonitis, pneumonia, meningitis, urinary tract infection, central line-associated bloodstream infection, primary bacteremia). The site of origin will be considered in future studies. Surgical peritonitis is a definitive state of infection and could be associated with higher nSOFA scores as compared to non-surgical LOS. To address this potential concern, we identified 11 additional patients with surgical peritonitis (and negative blood cultures) who survived, calculated their nSOFA scores, and re-analyzed the critical time points (−6, 0, +6, +12) with these new patients (n = 55 survivors, 16 non-survivors) (Supplemental Fig. S4). None of the main results (nSOFA comparisons at −6, 0, +6, +12; AUC ROC) were affected by the inclusion of these 11 survivors with surgical peritonitis. We intentionally excluded examination of early-onset sepsis (EOS) because the degree of organ dysfunction immediately after birth may be high for VLBW preterm infants in the absence of infection. Microbiologically confirmed EOS is very rare, especially in term infants, so to test the utility of the nSOFA for EOS mortality will require a multicenter or large database approach to address other confounding conditions that lead to organ dysfunction and early death (pulmonary hypoplasia, severe intraventricular hemorrhage, hypoxia–ischemic encephalopathy).

The nSOFA scoring system has the potential to align neonatal sepsis investigators globally by ensuring neonatal sepsis-related data derived from cohorts studied in different settings by different investigators can be successfully integrated into the literature and interpreted. The nSOFA, if validated in additional cohorts, would address a critical unmet need for a useful, objective operational definition of organ dysfunction applicable to this unique population that can align neonatal sepsis investigators, allow longitudinal disease stratification including prognostication, and facilitate predictive enrichment for future prospective interventional clinical trials.6,9,17