OBJECTIVE: To determine the clinical utility of SNAP score versus the highest oxygen index (OI) in first 24 hours of admission in predicting outcome of HRF.
STUDY DESIGN: All admissions (1991 to 1999) ≥36 weeks gestation, ventilated for ≥12 hours with FiO2≥0.50, without congenital anomalies were reviewed. Primary outcome measure was survival (without ECMO) versus ECMO and/or death.
RESULTS: From 184 infants with HRF, 148 survived (without ECMO) versus 36 died and/or received ECMO. SNAP score and highest OI were similar in predicting outcome of HRF (area under ROC curve: 0.813±0.037 versus 0.814±0.041; P=0.72). Death and/or ECMO requirement were best predicted by a SNAP score of 19 (Sensitivity 75.0%, Specificity 71%) or an OI of 28 (Sensitivity 75.0%, Specificity 76.4%).
CONCLUSION: Although both, the SNAP score and highest OI, are useful and similar in predicting outcome of HRF, OI is preferable because of its ease of use. We believe the predictive value of these parameters should be evaluated in a multicenter setting.
In the term and near-term infants, persistent pulmonary hypertension of the newborn (PPHN) is often the primary cause for hypoxic respiratory failure (HRF) resulting from various underlying pathologic conditions (meconium aspiration syndrome, pneumonia, ARDS, etc). PPHN has high mortality (11 to 48%)1,2,3,4,5 with the most severely affected requiring extracorporeal membrane oxygenation (ECMO).6,7 A timely transfer for inhaled nitric oxide (iNO) or ECMO may improve the outcome of this condition.
The oxygen index (OI) is one measure for quantifying the severity of HRF. In PPHN, an OI >40 has an associated mortality rate of 60 to 80% and hence is a criterion for ECMO.5,6,7,8 The score for neonatal acute physiology (SNAP) is a physiological severity index recorded in first 24 hours of admission and is calculated based on 34 objective physiologic measurements including OI, blood pressure, and blood chemistries.9,10 SNAP score correlates well with nursing workload, length of stay and physician estimates of overall mortality.9,10,11 SNAP score has been validated for predicting mortality, most often in sick low-birth-weight infants.9,10,12
Although both the SNAP score and the highest OI within the first 24 hours of admission, are useful tools in predicting the outcome of sick neonates, neither has been evaluated in predicting the outcome of HRF. Early identification of infants, prior to their development of refractory respiratory failure, may facilitate a more timely transfer to a center that provides ECMO or iNO. The purpose of this study was to evaluate the utility of SNAP score and the highest OI obtained in the first 24 hours of admission in predicting death and/or requirement of ECMO in HRF.
DESIGN AND METHODS
This is a retrospective cohort study of term and near-term infants admitted to the neonatal intensive care unit (NICU) at the MetroHealth Medical Center (MHMC, Cleveland, OH). This NICU is a level III unit with over 700 admissions per year. Besides receiving admissions from our high-risk perinatal service, the unit serves as a referral center for several community hospitals in the surrounding geographical area. This unit is similar to the 12 units comprising the NICHD consortium for prevalence of PPHN 4.6 versus 1.9% (range 0.43 to 6.82%), survival 91.9% versus 88% (67 to 96%) and need for ECMO 13% versus 34% (0 to 85%), respectively.1 Infants who fail nonconventional ventilation (jet ventilation or high frequency ventilation) are subsequently transferred to our regional ECMO center located at the Rainbow Babies and Children's Hospital (RB&C, Cleveland, OH).
All infants admitted to the NICU at MHMC with a diagnosis of HRF between July 1991 and June 1999 were eligible for this study. The inclusion criteria were infants ≥36 weeks gestation with HRF. HRF was defined as requirement of endotracheal intubation followed by mechanical ventilation≥12 hours with an FiO2 ≥0.50. The exclusion criterion was the presence of any life threatening congenital anomaly (i.e., diaphragmatic hernia, cyanotic congenital heart disease, chromosomal anomalies, abdominal wall defects). NICU admission logbook, NICU electronic database and MHMC electronic database were used to generate a list of all candidates. From a potential list of 249 infants, 57 (22.9%) were excluded for a congenital anomaly (abdominal wall defect, n=26; diaphragmatic hernia, n=13; multiple congenital anomalies, n=9; congenital heart disease, n=9). An additional 8 (3.3%) were excluded for unavailable medical records.
The following information was abstracted from the medical records: demographic variables, perinatal history, postnatal treatment modalities, SNAP score and highest OI in the first 24 hours of admission to NICU, primary outcome measures (death and/or ECMO) and secondary outcome measures (length of stay, duration of mechanical ventilation). For infants transported for ECMO, medical records from both institutions were reviewed.
SNAP score is a summation of 34 objective physiologic variables obtained within the first 24 hours of admission, which includes OI, blood pressure, and blood chemistries.9 Each variable is recorded on an ordinal scale and then summated. The OI is computed from the following formula: OI=[(Mean Airway Pressure) (FiO2) (100)]/(postductal PaO2). The primary outcome was defined a priori as survival without ECMO versus ECMO and/or death. The secondary outcomes were duration of mechanical ventilation (hours), days in supplemental oxygen and hospital length of stay. The SNAP scores and highest OI were collapsed into the following categories: SNAP (group 1: 0 to 9; group 2: 10 to 19; and group 3: >19) and highest OI (group 1: <14; group 2: 14 to 25; and group 3: >25). These arbitrarily chosen categories were based upon our preliminary data analysis, and are comparable to the criteria used in previous studies in predicting neonatal outcome.12,13,14 The purpose of the categorical analysis was to describe our patients in relationship to the severity of illness. This analysis illustrates the relationship of these markers for severity of illness with regards to the intensity of other interventions and to the development of adverse outcomes.
Parametric, interval level data are reported as mean ±SD, whereas the ordinal and nonparametric interval data are reported as the median with the 25th to 75th percentile. Interval level data were considered to be nonparametric if the kurtosis or skewness was >3.0 or <−3.0. χ2-test was used for categorical variables. Ordinal and nonparametric interval data were analyzed with Kruskal–Wallis H or Mann–Whitney U tests. If the p-value from the Kruskal–Wallis test was <0.05, the Mann–Whitney U test was then performed with the Bonferroni correction to reduce the testwise error. Statistical significance was defined a priori as p<0.05 (two-tail). The data were analyzed using the SPSS for Windows V 10.0 (SPSS Inc., Chicago, IL).
The receiver operating characteristics curves (ROC) for SNAP and highest OI were produced by plotting the sensitivity (true positive rate on the y-axis) versus 100-specificity (false-positive rate on the x-axis) for each discrete value of the SNAP score and highest OI using the outcome of survival (without ECMO) versus ECMO and/or death. A scale with an excellent cutoff value for predicting the outcome would produce an ROC curve with a sharp breakpoint in the left upper corner, whereas one with a nondiscriminate cutoff point would generate a diagonal line through the origin. The best cutoff value for a tertiary care NICU would be represented on the ROC curve at the middle of the shoulder (cutoff value with the best possible sensitivity and specificity), whereas the best cutoff value level for a Level I nursery would be the point at which the shoulder begins (highest possible sensitivity with an acceptable specificity).15 The area and standard error under each ROC curve were computed by the maximum likelihood estimates and compared to each other by the paired critical ratio Z-test.16 The ROC curve analysis was performed using “the ROC Curve Analyze, V6.0”.17 In developing and evaluating clinical scoring systems for predicting outcome, the goal is for the area under the ROC curve to exceed 0.70.18 Most ICU scoring systems have areas in the 0.80 to 0.90 range.19,20 The 95% confidence intervals (CI) are also reported for sensitivity and specificity.13,18
Of the 184 infants with HRF, 148 infants survived without ECMO intervention versus 36 who died and/or received ECMO (12 died without ECMO, three died with ECMO and 21 survived with ECMO). In all, 68 (36.9%) of the 184 infants were outborn and were transferred to our NICU at a median age of 7 (range 3.5 to 8) hours. The outborn and inborn infants were similar (p>0.05) for gestational age (38.7±1.8 versus 39.1±1.9 weeks), birth weight (3.3±0.7 versus 3.3±0.6 kg), gender (64.7 versus 53.4% male), race (61.8 versus 47.4% white), 5 minute Apgar 8 [(5 to 9) versus 8 (6 to 9), 25th to 75th percentile], maternal age (26.4±5.9 versus 25.1±6.4 years) and adequate prenatal care (71.4 versus 65.7%), respectively. The mean SNAP scores and highest OI were also similar between the outborn and inborn infants (SNAP: 15.7±7.7 versus 17.9±7.0; OI: 30.0±31.0 versus 30.8±33.5; p>0.05, respectively).
Of the 57/184 (31%) infants transferred to RB&C (median age 22 hours; range 2 to 145) for consideration of iNO/ECMO, 32 were inborn versus 25 outborn. The median age of initiation of ECMO in the 24 infants was 22 (range 7.5 to 150) hours. The 12 infants who died without receiving ECMO (including four during or soon after transport to RB&C and before initiation of ECMO), died at 26 (range 5 to 168) hours from the following etiology: six with meconium aspiration syndrome and multiorgan failure including renal failure, two with primary PPHN, two with overwhelming Group B Streptococcus sepsis, one pulmonary hemorrhage with seizures, and one with severe hypoxic ischemic encephalopathy, pneumonia, pneumothorax and pneumopericardium. The infant and maternal demographics were similar between those who survived without ECMO and those who died and/or received ECMO (Table 1).
Infants who underwent ECMO and/or died in contrast to the survivors (without ECMO) were more likely to be ventilated with oscillator/jet ventilator (33.4 versus 9.5%), to receive neuromuscular blockade (80.6 versus 36.5%), bicarbonate infusions (50.0% versus 24.4%) and inotropic support (91.75 versus 53.4%) and undergo an echocardiogram to confirm the diagnosis of PPHN (91.7 versus 56.8%), respectively (p<0.005). The surfactant usage was similar in both groups (33.4 versus 27.8%, respectively).
Outcome by SNAP Score
Among the three SNAP groups, the mortality rate and need for ECMO increased as the SNAP score increased (p<0.003). The duration of mechanical ventilation and highest OI increased with increase in SNAP (p<0.01) (Table 2). Patients who died had higher SNAP scores than the survivors (24.8 ±8.1 versus 16.5±6.9, p<0.001). They died early in their hospital course as reflected in their shorter length of stay in comparison to survivors (7.0 ± 9.4 versus 18.3 ± 14.9 days, p<0.001). Among the survivors, the SNAP increased in parallel with duration of mechanical ventilation (group 1, 71.5±64.5; group 2, 138.2±122.2; group 3, 187.1±146.9 hours; p<0.05 between groups 1 and 3), days to room air (group 1, 7.4±5.7; group 2, 10.3±10.0; group 3, 15.9±16.8; p<0.05 between groups 1 and 3) and length of stay (group 1, 13.1± 6.4; group 2, 16.5± 11.4; group 3, 24.6± 21.1 days; p<0.05 between group 1 versus 3, and 2 versus 3).
Outcome by Highest OI in first 24 hours of admission
Among the three OI categories, the mortality rate and the usage of ECMO increased as the highest OI in the first 24 hours increased (p<0.001, Table 3). Patients who died had higher OI in first 24 hours than the survivors (60.1±44.7 versus 25.7±21.4, p<0.001). None of the infants with OI<14 died, while 80% of total deaths and 78% of infants requiring ECMO and/or death had an OI>25. Among the survivors, the OI category increased in parallel with their duration of mechanical ventilation (group 1, 87.0±73.3; group 2, 169.4±145.1; group 3, 180.8±140.9 hours; p<0.05 between group 1 versus 2, and 1 versus 3), days to room air (group 1, 7.3±6.0; group 2, 12.0±11.6; group 3, 15.6±15.9; p<0.05 between groups 1 versus 2, and 1 versus 3) and length of stay (group 1, 13.8±7.4; group 2, 18.3±12.8; group 3, 23.2±20.1 days; p<0.05 between groups 1 and 3).
SNAP Score and OI Predicting Outcome
The area under the ROC curve for SNAP score and highest OI in first 24 hours was similar (Figure 1). The best SNAP score (highest sensitivity with best specificity) for predicting death and/or ECMO was 19 (sensitivity 75%, 95% CI: 57.5 to 87.2%; specificity 71%, 95% CI: 62.8 to 78), where as the highest OI in the first 24 hours was 28 (sensitivity 75.0%, 95% CI: 57.2 to 87.2; specificity 76.4%, 95% CI: 68.6 to 82.8). The positive predictive value for a SNAP score of 19 and highest OI of 28 were 38.6 and 43.5%, respectively. The best SNAP and OI values used as a screening test (highest sensitivity with reasonable specificity) were 14 (sensitivity 97.2%, 95% CI: 83.9 to 99.8; specificity 39.9%, 95% CI: 32.0 to 48.2) and 15 (sensitivity 94.4%, 95% CI: 80.1 to 99.0; specificity 41.9, 95% CI: 33.9 to 50.3).
The key finding of this study is that both the SNAP score and the highest OI in the first 24 hours of admission can identify most term and near-term infants with hypoxic respiratory failure who will subsequently die or require ECMO. Infants with a SNAP score greater than 19 or a highest OI greater than 28 have a two-fold increase in risk for dying and/or requiring ECMO (pretest risk 19.6%, post-test risk with a positive test or positive predictive value of 38.6 and 43.5%, respectively). The SNAP score and the highest OI in the first 24 hours were similar in their ability to predict those who may develop these adverse outcome.
This is the first study to our knowledge evaluating clinical scoring systems early in the course of HRF for predicting outcome. Prior studies have used the highest OI during the entire hospital course for HRF for predicting mortality or have used clinical scoring systems (i.e., SNAP, SNAPPE-II) in the first day of life for predicting in hospital mortality irrespective of disease states. The SNAPPE-II Score when performed during the first 12 hours of life had a higher area under the curve than reported in this manuscript (0.87 versus 0.81).13 This may be secondary to our exclusion of infants with severe congenital anomalies (diaphragmatic hernia) and to the difference in severity of illness between the two cohorts. In the SNAPPE-II cohort, 77% of the infants ≥1500 g had a SNAP score ≤9 in contrast to 14% in our cohort. The inclusion of severe birth defects with HRF would have probably improved the sensitivity secondary to a higher “true positive rate,” whereas studying a less severe population would have increased the specificity secondary to a higher “true negative rate”.
In the categorical analysis of SNAP scores and highest OI in the first 24 hours, as the category increased, so did the mortality rate, requirement for ECMO, length of stay and use of medical interventions (ventilator support, inotropic medications). As in the low-birth-weight infant studies predicting outcome regardless of disease processes, these scoring systems can also be used to predict mortality and utilization of health care resources for a specific disease process.
Potential limitations of this study include: the clinical diagnosis of PPHN in some infants, the lack of other markers for respiratory severity, the clinical use of OI in initiating ECMO, the effect of therapy on OI/SNAP scores and the limitation of scoring systems to the first 24 hours of admission. The diagnosis of PPHN was confirmed by echocardiography in 63.6% of the infants with the remainder receiving a clinical diagnosis of PPHN upon discharge from the NICU. Even though the majority of patients had the diagnosis confirmed by echocardiography, the term HRF was used instead of PPHN. The OI was chosen over other markers of respiratory severity (PO2/fraction of inspired oxygen, arterial–alveolar oxygen difference) because they were statistically equivalent in the development of the SNAPPE-II13 and the OI is one of the criteria in deciding on the initiation of iNO14,19 or ECMO therapy. The clinical use of OI in the decision-making process may lead to the potential of incorporation bias. The potential for this bias would not be present in the patients who died prior to ECMO therapy, in those who received ECMO beyond the first day of life and in the evaluation of the SNAP score. As the patient illness becomes worse, reflected as a higher OI, they would have received additional therapy including surfactant, HFV, and inotropic support. These additional therapies could have slowed the rate of rise of their OI/SNAP scores, which could have had a lowering (or dampening) effect on the ROC curve. Extending the data collection period for the scoring systems beyond the first day of admission would have enhanced the ability of the two scoring systems to detect adverse outcomes; however, it would have detracted from the objective of this study, which is the early identification of infants who die and or require ECMO.
One of the strengths of this study is that it consists of almost all of the admissions (97.6%) for HRF over an 8-year period to a level III NICU. Even though this unit is comparable in prevalence and outcome of PPHN to the other 12 units comprising the NICHD consortium for prevalence of PPHN, before these findings can be implemented at the bedside, they should be confirmed in a multicenter study.
Recent published data suggest that the use of alternative therapies like iNO may obviate the need for ECMO.21 Also, ECMO use in infants with severe HRF decreases mortality.22 Centers managing infants with HRF, but without these therapies, need to assess the risk of death early in the infant's hospital course in order to safely transport these infants to appropriate facilities. The SNAP score and highest OI in the first 24 hours of admission are effective and similar in predicting death and/or ECMO requirement. The best overall cutoff values for SNAP score (19) and OI (28) and the screening test values for SNAP score (14) and OI (15) could be easily utilized in management and transport decisions. In practical terms, levels I and II units may use the lower screening values of SNAP and OI (which have a higher sensitivity) for decision to transfer these infants to a tertiary care unit. For tertiary NICU's without ECMO, they may elect to use the higher values (i.e., the SNAP of 19 or OI of 28, which have a higher specificity) in deciding when to transfer infants to ECMO centers. Since both the SNAP score and highest OI in first 24 hours of admission are comparable in predicting outcomes, we would recommend OI because it is simple to use. Unlike previous studies on SNAP and OI, this study provides the physician with cutoff values obtained during the first 24 hours of admission, which would assist him in making an earlier triage decision to the appropriate neonatal care facility.
In infants with HRF, SNAP score and the highest OI in the first 24 hours of admission can identify infants at risk for death and/or ECMO requirement. In addition, they also predict duration of assisted ventilation, days of supplemental oxygen and length of hospital stay. These scoring systems are similar in their ability to predict mortality and requirement for ECMO. Both the SNAP score and highest OI in the first 24 hours of admission may be useful tools for allocation of resources, early introduction of alternative therapies like iNO and ECMO or counseling parents. Following confirmation of these findings, preferably at other centers, would then allow centers without iNO or ECMO, caring for critically ill newborn infants, to use the SNAP score and highest OI in first 24 hours of admission to identify infants that likely require transfer.
Presented in part at the Pediatric Academic Societies and American Academy of Pediatrics Joint Meeting in Boston, MA, May 12–16, 2000.
Supported in part by the General Clinical Research Center (GCRC) Grant from NIH (MO1RR00080) awarded to the MetroHealth Medical Center, CWRU, Cleveland, OH, USA.
Walsh-Sukys MC, Tyson JE, Wright LL . et al. Persistent pulmonary hypertension of the newborn: Trends in era before nitric oxide: practice variation and outcomes. Pediatrics 2000;105:14–20.
Goetzman BW, Riemenschneider TR . Persistence of the fetal circulation. Pediatr Res 1980;2:37–40.
John E, Roberts V, Burnard E . Persistent pulmonary hypertension of the newborn treated with hyperventilation: clinical features and outcome. Aust Paediatr J 1988;24:357–361.
Heymann JR, Adams A, Gardner TH . Persistent pulmonary hypertension of the newborn: trends in incidence, diagnosis and management. Am J Dis Child 1984;138:592–595.
Dworetz AR, Moya FR, Sabo B, Gladstone I, Gross I . Survival of infants with persistent pulmonary hypertension without extracorporeal membrane oxygenation. Pediatrics 1989;84:1–6.
Bartlett RH, Gazzaniga AB, Toomasian J, Corwin AG, Roloff D, Rucker R . Extra-corporeal membrane oxygenation (ECMO) in neonatal respiratory failure—100 cases. Ann Surg 1986;204:236–245.
Walsh MC, Stork E . Persistent pulmonary hypertension of the newborn. Rational therapy based on pathophysiology. Clin Perinatol 2001;28:609–627.
Beck R, Anderson KD, Pearson GD, Cronin J, Miller MK, Short BL . Criteria for extracorporeal membrane oxygenation in a population of infants with persistent pulmonary hypertension of the newborn. J Pedeatr Surg 1986;4:297–302.
Richardson DK, Gray JE, McCormick MC, Workman K, Goldmann DA . Score for neonatal acute physiology: a physiological severity index for neonatal intensive care. Pediatrics 1993;91:617–623.
Escobar GJ, Fischer A, Li DK, Kremers R, Armstrong MA . Score for neonatal acute physiology: validation in three kaiser permanente neonatal intensive care units. Pediatrics 1995;96:918–922.
Stevens SM, Richardson DK, Gray JE, Goldmann DA, McCormick MC . Estimating neonatal mortality risk: an analysis of clinicians' judgments. Pediatrics 1994;93:945–950.
Richardson DK, Phibbs CS, Gray JE, McCormick MC, Workman K, Goldmann DA . Birth weight and illness severity: independent predictors of neonatal mortality. Pediatrics 1993;91:969–975.
Richardson DK, Corcoran JD, Escobar GJ, Lee SK . SNAP-II and SNAPPE-II: Simplified newborn illness severity and mortality risk scores. J Pediatr 2001;138:92–100.
Clark RH, Kueser TJ, Walker MW . et al. Low-dose nitric oxide therapy for persistent pulmonary hypertension of the newborn. Clinical inhaled nitric oxide research group. N Engl J Med 2000;342:469–474.
Fletcher RH, Fletcher SW, Wagner EH, editors. Clinical Epidemiology, the Essentials. Baltimore, MD: Williams & Wilkins; 1996 p. 62.
Dorfman DD, Alf E . Maximum likelihood estimation of parameters of signal detection theory and determination of confidence intervals-rating method data. J Math Psychol 1969;6:487–496.
Centor RM, Keightley J . ROC Curve Analyzer, Version 6 0 Richmond, VA, 1990.
Lemeshow S, Le Gall JR . Modeling the severity of illness of ICU patients. A systems update. JAMA 1994;272:1049–1055.
Diamond GA . Limited assurances. Am J Cardiol 1989;63:99–100.
Fleiss JL . Statistical Methods for Rates and Proportions. NY, New York: John Wiley & Sons; 1981 p. 1–18.
The neonatal inhaled nitric oxide study group (NINOS). Inhaled nitric oxide in full-term and nearly full-term infants with hypoxic respiratory failure. N Engl J Med 1997;336:597–604.
The collaborative UK ECMO (extracorporeal membrane oxygenation) trial: follow-up to 1 year of age. Pediatrics 1998;101(4):E1.
About this article
A Comprehensive Study of Clinical Biomarkers, Use of Inotropic Medications and Fluid Resuscitation in Newborns with Persistent Pulmonary Hypertension
Pediatric Cardiology (2015)
Addition of SNAP to perinatal risk factors improves the prediction of bronchopulmonary dysplasia or death in critically ill preterm infants
BMC Pediatrics (2013)
Pediatric Nephrology (2013)
Pediatric Nephrology (2012)
Patch repair is an independent predictor of morbidity and mortality in congenital diaphragmatic hernia
Pediatric Surgery International (2011)