Introduction

Neonatal early-onset sepsis (EOS), defined as sepsis occurring within 72 h after birth, has high morbidity and mortality.1 The overall incidence of EOS is 0.1% and increases in certain subgroups such as very low birth weight (VLBW) infants (birth weight <1500 g) or infants born after a gestational age <28 weeks to 1.4 and 1.8%, respectively.2 Diagnosis of EOS is challenging given the subtle and non-specific signs and symptoms. Since timely commencement of antibiotics could prevent sepsis-related morbidity and mortality, the threshold to initiate empiric antibiotic therapy before diagnostic confirmation is low.3,4 Consequently, 5% of all late preterm and term infants and up to 75% of VLBW infants are exposed to antibiotics empirically for suspected EOS.3,4 Exposure to antibiotics early in life increases the risk of antibiotic resistance and impacts microbial gut colonization by decreasing its diversity and increasing the abundance of pathogens.5 This may increase the risk of both immediate and long-term adverse effects, such as growth retardation and auto-immune disorders.5,6,7,8,9 In order to reduce the risk for sepsis-related morbidity and mortality on one hand and to prevent overtreatment with antibiotics on the other hand, a diagnostic test with high sensitivity and specificity is needed.

The currently considered gold standard for EOS diagnosis is a bacterial blood culture drawn from a peripheral vein.10 The exact sensitivity of a peripheral blood culture (PBC) for EOS is unknown; however, clinicians have questioned the accuracy since cultures obtained from patients with clinical illness often remain sterile. The sensitivity of a PBC decreases with sample volume, while collecting an adequate blood volume from neonates can be challenging.11 Furthermore, maternal intrapartum antibiotic use might further decrease the sensitivity, although advances in blood culture techniques limit this risk nowadays.12,13 Besides, PBCs typically require phlebotomy, which is associated with pain1,10 and it contributes to iatrogenic anemia, especially in VLBW infants.14

The use of umbilical cord blood culture (UCBC) has been suggested as an alternative diagnostic test if EOS is suspected at the time of birth. Collection of umbilical cord blood is not painful, it is technically easy to perform, and sufficient sample volume can be obtained circumventing the risk for iatrogenic anemia.15 However, studies on the diagnostic accuracy of UCBC compared to PBC included low sample sizes and the results are conflicting. To date, no systematic review or meta-analyses has been performed. Therefore, we aimed to systematically identify, appraise, and evaluate the diagnostic test accuracy (DTA) of UCBC for the diagnosis of EOS compared to PBC including a meta-analysis.

Methods

Study objectives

To investigate the primary aim of this review, we first compared results of UCBC as index test directly with results of the gold standard, PBC, as reference test. Second, because of uncertainty about the true sensitivity of either tests, we compared the results of UCBC and PBC as separate index tests with the previous papers’ definition of clinical EOS as reference. For this comparison, only studies with paired UCBC and PBC were included. Third, the DTA of PBC and UCBC combined as index test (if one or both tests were positive, the outcome was regarded positive) for clinically diagnosed sepsis was evaluated.

Protocol and registration

The protocol for this systematic review was registered prospectively with Prospero (ID-number CRD42021238106). The manuscript was written in accordance with the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) of DTA studies checklist (Supplement 1).16

Study eligibility criteria

Studies investigating the diagnostic accuracy of UCBC for EOS were eligible. Studies comparing the accuracy of UCBC with either PBC proven or clinically diagnosed EOS as the gold standard were included. Since there is currently no uniform definition of clinically diagnosed EOS, we did not include a strict definition.17 Articles including conventional and/or non-conventional culture techniques were included. Animal studies and case reports were excluded. If no full text was available, the same was requested from the author. If original authors did not respond, studies were excluded. No date or language restrictions were applied.

Information sources and search strategy

A literature search was performed based on the PRISMA statement.16 To identify eligible studies, systematic searches were performed in collaboration with a medical information specialist in the bibliographic databases PubMed, Embase, Web of Science (Core Collection), and Wiley/Cochrane Library from inception up to January 21, 2021. The following terms were used (including synonyms and closely related words) as index terms or free-text words: “Neonates,” “Early-onset sepsis,” and “Umbilical cord blood.” The full search strategies for all databases can be found in Supplement 2.

Study selection and data collection

After removal of duplicates, two reviewers (T.H.D. and D.H.V.) independently screened all potentially relevant titles and abstracts for eligibility. The full text of the selected articles was obtained for further review of the eligibility criteria. Differences in judgment were resolved through a consensus procedure. Data from the included articles was extracted by the two reviewers (T.H.D. and D.H.V.) and verified by the other authors. Articles found through references and other sources were also included if eligible. The following data were extracted if available: year of study, country, study design including study setting, inclusion and exclusion criteria, characteristics of the study population, number of participants, incidence of culture-proven and clinically diagnosed EOS, cultured pathogens, definition of clinically diagnosed sepsis, DTA of UCBC for PBC, DTA of both UCBC and PBC for clinically diagnosed EOS, maternal intrapartum antibiotic use, and collection technique of umbilical cord blood.

Risk of bias and quality assessment

Two reviewers (T.H.D. and D.H.V.) independently evaluated the methodological quality and the risk of bias of the articles included in the final analysis, using the QUADAS-2, a tool for the quality assessment of diagnostic accuracy studies.18

Meta-analysis

The true and false positive and negative values for each individual study were entered into RevMan Version 5.4.1.19 This software was used to create forest plots and summary receiver operating characteristics (sROC). Subsequently, a bivariate random effects model20 was used to estimate the pooled summary sensitivity and specificity including 95% confidence intervals (CIs). This was done using Proc NLMIXED in SAS version 9.4.21,22 If no variance in sensitivity or specificity was observed between the studies, the delta method was used to calculate CIs.23 The calculated parameter estimates were imported to RevMan to visualize the calculated summary operation points in the sROC.

Forest plots and sROCs were visually inspected to identify heterogeneity. We planned to explore potential sources of heterogeneity, such as the incidence of culture-proven and/or clinically diagnosed EOS, number of inclusions, year of publication, gestational age, volume of blood used for UCBC, and the reporting of well-defined protocol for sterilization of the umbilical cord. If sufficient studies were available, these potential sources were added to the model as a covariate.

Results

Study selection

The literature search generated a total of 3830 references: 983 in PubMed, 1709 in Embase, 944 in Web of Science, and 194 in the Cochrane Library. Three additional articles were identified through other sources. After removing duplicates, 1908 references remained. The abstract and titles of these articles were screened, excluding 1856 studies. The full texts of the remaining 52 articles were further checked for eligibility. A total of 33 were excluded based on the inclusion and exclusion criteria. The other 19 articles (17 on conventional culture, 2 on non-conventional molecular cultures) were included in this systematic review. The flow chart of the search and selection process is presented in Fig. 1.

Fig. 1: Flow diagram of the study selection.
figure 1

Overview of the study selection process from identification of records to study inclusion.

Full size image

Study characteristics

The selected studies included a total of 2385 infants, with sample sizes ranging from 30 up to 323 participants. Publication dates of the included studies ranged from 1976 to 2020. One case–control study24 and 18 observational cohort studies were included.25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42 From the latter, 1 included all admitted infants32 and the other 17 included only infants at higher risk of EOS based on the presence of one or more risk factors.25,26,27,28,29,30,31,33,34,35,36,37,38,39,40,41,42 One study included only term born infants,25 four only preterm born infants,24,26,27,28 and the other studies included both term and preterm born infants.29,30,31,32,33,34,35,36,37,38,39,40,41,42 Umbilical cord blood samples were collected directly after birth and peripheral blood samples as soon as possible postpartum but before the initiation of antibiotics in all studies. Not all studies collected a paired sample of cord blood and peripheral blood from every individual participant. Umbilical cord blood and peripheral blood was collected from 2152 and 1519 infants for conventional culture, respectively. The 2 studies on molecular culturing techniques included a total of 123 infants and collected paired cord blood and peripheral blood of all 123 infants. Characteristics of the included studies and the main outcomes are described in Table 1.

Table 1 Overview of the study characteristics.
Full size table

Risk of bias and quality of evidence

The risk of bias due to patient selection was regarded as low. All cohort studies included a consecutive number of patients, based on predetermined eligibility criteria. Two studies excluded patients with contaminated cultures,24,29 possibly introducing bias in patient selection. None of the studies reported whether the results for the index test and reference test were interpret blinded from the other test. However, due to the standard laboratory protocols and evident results from blood cultures, it was deemed unlikely that this introduced bias. Risk of bias for two studies in the domain of the reference standard was unclear, since it was not reported whether umbilical cord blood was collected under sterile conditions.31,42 Due to the study objectives, all studies used either PBC and/or clinically diagnosed sepsis (defined as the presence of a set of clinical symptoms and/or laboratory values indicating the presence of sepsis) as a reference standard. Since both PBC and clinically diagnosed sepsis are imperfect reference standard to detect EOS,1 this might affect the validity of results. Therefore, risk of bias and applicability concerns were estimated to be high in all studies for the reference standard. It was estimated that the flow and timing of the participants did not introduce bias in most studies. Few studies, however, were unable to collect paired samples from both umbilical cord blood and peripheral blood from all infants, which might introduce partial verification bias.33,35,40,41,42 A summary of the risk of bias for individual studies and overall summary of the risk of bias per domain is, respectively, demonstrated in Figs. 2 and 3. In general, studies were qualitatively well performed. However, due to low sample sizes and low incidence of EOS cases in combination with the imperfect reference standards, the overall quality of evidence was regarded as low.

Fig. 2: Summary of risk of bias and applicability concerns.
figure 2

Judgment of authors about each of the four domains in the QUADAS-2 for every included study.

Full size image
Fig. 3: Risk of bias and applicability concerns graph.
figure 3

Judgment of authors about each of the four domains in the QUADAS-2 presented as percentages across the included studies. Each bar shows the number of studies in each category.

Full size image

UCBC results compared to PBC results

A total of 13 studies, including a total of 1213 patients, compared the outcomes of conventional UCBC with paired PBC as the gold standard.25,26,27,29,30,31,32,34,36,37,38,39,40 Most studies showed a high rate of negative PBC and UCBC, resulting in high specificity of UCBC for PBC. The number of patients with a positive PBC was low and the reported sensitivity of UCBC for PBC showed considerable heterogeneity across the different studies (Fig. 4). Meta-analysis of the study results showed a pooled sensitivity of 75.0% (95% CI 44.1-91.9) and specificity of 91.3% (95% CI 83.4–95.6) of UCBC for PBC. Supplemental Table 1 demonstrates the cultured micro-organisms in both PBC and UCBC.

Fig. 4: Forest plot of umbilical cord blood culture consistency with paired peripheral blood culture results.
figure 4

Umbilical cord blood culture had a pooled sensitivity of 44.1% (95% CI 75.0–91.9) and specificity of 91.3% (95% CI 83.4–95.6) for peripheral blood culture results. CI confidence interval, FP false positive, FN false negative, TN true negative, TP true positive.

Full size image

DTA for clinically diagnosed sepsis

The definition of false positive (contamination) culture results differed between the studies. A positive culture was determined as false positives based on the cultured pathogen,27,32 laboratory values,25,26,29,36,37 clinical symptoms,40 or a combination of these factors.30,31,34,38,39 A total of 17 studies reported true positive and false positive rates for UCBC (range 0–24% and 0–12%, respectively).25,26,27,29,30,31,32,33,34,35,36,37,38,39,40,41,42 From these studies, 13 also reported these rates for PBC (range 0–27% and 0–27%, respectively).25,26,29,30,32,33,34,35,36,37,38,40,41 Supplemental Table 1 demonstrates the number contaminated cultures for PBC and for UCBC including the cultured micro-organisms. Sensitivity and specificity could not be calculated in all studies, because true and false negative values were not reported. A total of eight studies reported the number of true and false negative outcomes for UCBC (Fig. 5).26,29,31,35,36,37,39,42 Four of these studies also collected paired samples for PBC and reported the DTA of both tests for clinically diagnosed EOS.26,29,36,37 In these four studies, clinical diagnosis of sepsis was defined as the presence of two or more risk factors for EOS in combination with two or more laboratory values indicating sepsis according to Evidence Based Practice guideline on the Management of Neonatal Sepsis by the National Neonatology Forum.43,44 The summary operation points from the meta-analyses of these 4 studies demonstrated a pooled sensitivity of 20.4% (95% CI 0.0–40.9) and specificity of 100.0% (95% CI 100.0–100.0) for PBC to detect a clinical diagnosis of EOS. The meta-analysis for UCBC including the 4 studies collecting paired samples from the cord and a peripheral vein yielded a pooled sensitivity of 42.6% (95% CI 12.7–72.4%) and specificity of 97.8% (95% CI 93.1–100.0) for clinical EOS as demonstrated in the sROC plot in Fig. 6. The DTA for PBC and UCBC combined as one index test (if one or both cultures were positive, the outcome was regarded as positive) yielded a pooled sensitivity and specificity of 44.0% (95% CI 20.5–70.5) and 97.8 (95% CI 89.6–99.6) for clinical EOS, respectively.

Fig. 5: Forest plot of peripheral blood culture and umbilical cord blood culture for diagnosis of clinically diagnosed sepsis.
figure 5

CI confidence interval, FP false positive, FN false negative, TN true negative, TP true positive.

Full size image
Fig. 6: Summary of the receiver operating characteristic plot of the diagnostic test accuracy of peripheral blood culture and the diagnostic test accuracy of umbilical cord blood culture for diagnosis of clinically diagnosed early-onset sepsis.
figure 6

Only studies collecting paired blood samples from the umbilical cord and a peripheral vein are included. The solid circles represent the pooled sensitivity and specificity for each test.

Full size image

Due to the low number of studies, the low numbers of inclusions in the individual studies and the following wide 95% CI, statistical comparison of the summary operation points of the sensitivity and specificity was not possible. There was large heterogeneity in the year of publication, number of EOS cases, and region of conduction between the studies. Given the low number of studies reporting the DTA of both PBC and UCBC for clinical EOS, we were also unable to statistically assess the influence of these sources of heterogeneity on the results of the meta-analyses. Since only two studies with low sample sizes determined the DTA of non-conventional molecular culturing techniques using cord blood for EOS including different techniques (Sanger sequencing and 16S rRNA gene sequencing),24,28 we decided to not include them in a meta-analysis.

Discussion

This is the first systematic review investigating the DTA of UCBC for diagnosing neonatal EOS at the time of birth. The currently used gold standard for diagnosing EOS, a conventional PBC, is a painful procedure for the infant and it is often a challenge for the attending clinicians to obtain an adequate sample volume. Besides, it contributes to the risk for iatrogenic anemia, especially in VLBW infants. UCBC circumvents the above-mentioned challenges and risks, but the DTA of UCBC for EOS has not been studied thoroughly before. We demonstrated that, compared to the DTA of PBC, UCBC has a higher sensitivity (20 versus 43%) and comparable specificity (100 versus 97%) for clinically diagnosed EOS.

In the majority of infants with negative PBC, also a negative UCBC was found. We demonstrated a pooled specificity of 91.3% and a varying, but lower pooled sensitivity of 75.0% of UCBC for paired PBC outcomes. However, it is known that PBC results for the diagnosis of EOS can be false negative, especially when an inadequate sample volume is obtained, impairing the sensitivity.45,46,47 This demonstrates the necessity to evaluate the accuracy of new diagnostic tests for EOS using “clinically diagnosed EOS” besides “culture-proven EOS” as target outcome. In our study, the pooled sensitivity for clinically diagnosed EOS of UCBC was higher compared to PBC. Combining both UCBC and PBC as one index test did not further increase the sensitivity.

Due to the risk for iatrogenic anemia, it is not feasible to collect a large amount of neonatal peripheral blood.14. However, often the bacterial load in blood of septic neonates is low,48,49 and consequently, a larger sample volume is required for adequate sensitivity of blood culture in this population. One of the advantages of UCBC over PBC includes the opportunity to collect larger sampling volume more easily,15 which might explain the increase in sensitivity. Meanwhile, collecting blood from an unsterile umbilical cord may introduce the risk for contamination and false positive results, possibly decreasing the specificity. One study reported a false positive rate for UCBC of 26.1%31, but definition for contamination, (sterile) collecting technique, nor a false positive rate for PBC were not reported. Two studies reported no false positives for UCBC36,37. The other 4 studies reported a false positive rate ranging between 1.9 and 9.8%26,29,35,42; 2 of these did not report on their (sterile) collection technique35,42. The 4 studies included in the meta-analyses for clinically diagnosed EOS did use well-defined protocols for cord sterilizing prior to collection of cord blood samples,26,29,36,37 thereby reducing risk of contamination as demonstrated by the low pooled false positive rate of 2.2% (i.e., pooled specificity of 97.8%).

Strengths and limitations

Outcomes of individual studies with small number of EOS cases lack power and random errors may have a large influence, especially on the sensitivity. This is the first meta-analysis pooling the results of these small studies. Given the imperfect gold standard (PBC), it is valuable not only to compare the results of UCBC with those of paired PBC but also to compare the accuracy of both tests for clinically diagnosed EOS.

There are also some limitations that need to be addressed. First, due to the sparse available data on paired samples of PBC with UCBC for clinically diagnosed sepsis, only four studies with a limited number of participants were included in the meta-analysis comparing the DTA of UCBC with the DTA of PBC for clinical EOS. Second, a clinical diagnosis of sepsis was defined as the presence of two or more risk factors in combination with two or more laboratory values indicating EOS in these four studies. This imperfect reference standard for EOS might have classified infants without bacterial or fungal sepsis as clinical sepsis cases and consequently the sensitivity of both tests might have been underestimated. Third, there was large heterogeneity in the year and country of publication and the study populations. Besides, not all studies reported whether umbilical cord blood was obtained under sterile conditions, possibly influencing the DTA.

Currently, there is an enormous overtreatment with antibiotics in newborns with a suspicion or increased risk for EOS due to a lack of accurate tests. Withholding antibiotics in non-septic infants could prevent antibiotic-related adverse events.1,3,4 Based on the pooled sensitivity of 43% of an UCBC, it might be unlikely that clinicians will discontinue antibiotics in case of a negative UCBC, while a strong clinical suspicion for EOS exists. However, the increased sensitivity in combination with low risk for false positives (i.e., high specificity) will guide clinicians for pathogen-specific targeted therapy more often when using UCBC. Sensitivity of both conventional PBC and UCBC conventional culture might be impaired by low bacterial load and intrapartum maternal antibiotic use, although the risk on the latter is decreasing nowadays by the use of specialized culture media removing antibiotics from the sample.12 Since non-conventional molecular cultures can also detect and amplify DNA of dead bacteria and may detect bacterial DNA even with lower bacterial loads in a sample, these techniques may further increase sensitivity. We identified that only two studies were investigating the accuracy of different non-conventional culturing techniques,24,28 limiting the possibility to draw conclusions.

Given the low cumulative number of EOS events in the meta-analysis, the limited number of studies investigating the accuracy of both conventional UCBC as well as PBC for clinically diagnosed EOS and the heterogeneity between studies in country and year of publication, the results from the meta-analyses should be interpreted cautiously. Larger prospective studies including higher numbers of EOS cases are warranted. These studies should collect paired samples of the umbilical cord and a peripheral vein from the same infant and define the target outcome clinically diagnosed sepsis according to internationally accepted and validated methods, such as proposed by Vergnano et al.50 When implementing UCBC in clinical care, a (slight) increase in the false positive rate cannot be excluded and unnecessary prolongation of antibiotics in false positive cases should be taken into account. As demonstrated for PBC,51 quality improvement initiatives might reduce the risk for false positives and might improve adoption of UCBC in future studies and in clinical care. These initiatives include staff education on aseptic collecting techniques and the preparation and availability of pre-made collection kits in the delivery room. Whether these strategies improve the diagnostic accuracy of UCBC for EOS needs to be assessed in future studies. Whether rapid culture-independent molecular diagnostic procedures such as PCR-based techniques can further increase the sensitivity for EOS diagnosis using umbilical cord blood also needs to be elucidated in future studies.

In conclusion, this systematic review demonstrated that UCBC has higher sensitivity and comparable specificity for clinical EOS, compared to PBC. Considering the larger blood volume that can be obtained from the umbilical cord via a painless procedure, the low risk of iatrogenic anemia, and low risk of false positives, UCBC might be considered as reference test in the diagnosis of EOS. However, given the limitations of the current available studies, future high-quality studies on the accuracy of UCBC for EOS diagnosis are needed to validate these findings.