Introduction

Recently, remarkable progress has been achieved in reducing overall incidence of infectious diseases1,2. However, infectious diseases remain a leading cause of childhood morbidity and mortality globally3,4. Specifically, mortality of up to 20% has been reported in children with sepsis5, largely because the lack of specific signs and symptoms in children makes early diagnosis and treatment challenging6. In addition, rapid deterioration can occur when compensation fails in pediatric patients7,8. Thus, early recognition of sepsis is crucial to ensure timely management and to reduce mortality9,10.

Pediatric sepsis has been defined by the International Pediatric Sepsis Consensus Conference (IPSCC) as the presence of systemic inflammatory response syndrome (SIRS) with suspected or proven infection in a child11. However, the SIRS criteria lack specificity for identifying infected patients at high risk of mortality because children with non-infectious conditions can also meet the SIRS criteria; moreover, children who meet the SIRS criteria for sepsis often do not have organ dysfunction12,13.

In the case of adults, the Sepsis-1 and Sepsis-2 consensus definitions also adopted the SIRS criteria14,15. However, the Third International Consensus Definitions for Sepsis and Septic Shock (SEPSIS-3) task force revised the sepsis definition as the presence of a dysregulated host response that manifests as detrimental organ dysfunction, thus enabling the differentiation of sepsis from uncomplicated infections or non-infectious conditions16,17. Organ dysfunction is specifically defined by acute changes in the Sequential Organ Failure Assessment score (SOFA) with ≥ 2 points16,17.

The SEPSIS-3 task force also recommended the bedside tool, the quick SOFA (qSOFA), for the early recognition of adult patients with suspected infection at risk for poor outcomes. The qSOFA is a simplified version of the SOFA, comprising only three components: low systolic blood pressure (≤ 100 mmHg), high respiratory rate (≥ 22 breaths/min), and altered mental status (Glasgow Coma Scale [GCS] < 15)16,17. The SOFA and qSOFA showed better prognostic performance than the former sepsis criteria among adult patients in various clinical settings17,18,19,20. However, the SEPSIS-3 criteria were developed and validated for adult patients and did not consider age-dependent physiologic variables16.

Recently, there have been attempts to adapt the SEPSIS-3 criteria for pediatric patients21,22. The age-adjusted or age-adapted qSOFA, which is adjusted according to age-specific cutoffs, has shown promising results among pediatric patients in emergency department (ED)23,24 and intensive care unit (ICU) settings13,25. However, there is no systematic review and meta-analysis reporting the predictive value of the age-adjusted qSOFA in children. Therefore, we aimed to evaluate the performance of the age-adjusted qSOFA in predicting outcomes, including mortality and disease severity, among pediatric patients with suspected or confirmed infection.

Materials and methods

This systematic review and meta-analysis is reported according to the Preferred Reporting Items for Systematic reviews and Meta-Analyses statement (PRISMA)26 and is registered with PROSPERO (CRD42021232257).

Study selection, eligibility, and data extraction

Two authors (SHY and HK) independently conducted literature searches of PubMed, EMBASE, the Cochrane Library, and Web of Science, without language or time restrictions, on January 6, 2021, with the aim of finding eligible studies assessing the performance of age-adjusted qSOFA in predicting mortality and/or disease severity in pediatric patients with suspected or confirmed infection. Various combinations of the following key words were used in the systematic search: “Quick Sequential Organ Failure Assessment,” “qSOFA,” “quick SOFA,” “q-SOFA,” “quick-SOFA,” and “pediatric,” “child,” “adolescent,” “infant,” and “neonate.”

Studies were eligible if they aimed to assess the performance of age-adjusted qSOFA to predict mortality or disease severity in pediatric patients (aged < 18 years) with suspected or confirmed infection. We used the following as indicators reflecting disease severity: admission or transfer to an ICU (including a critical care unit), development of severe sepsis11, or prolonged hospital stay (dependent on the authors’ definition, regardless of duration). If enrolled patients received a diagnostic code (e.g., International Classification of Diseases code) indicative of an infection or were diagnosed with sepsis/septic shock via consensus definition, we accepted them as patients with confirmed infection. In addition, if enrolled patients had signs or symptoms of infection (e.g., fever), or were treated for a bacterial infection (e.g., treated with therapeutic antibiotics), we inferred suspected infection. Studies were included if they reported sufficient data to construct a 2 × 2 contingency tables. Reviews, editorials, expert opinions, animal experiments, or studies presenting duplicate data were excluded.

The following information was retrieved from each study: first author, publication year, sample size, patient source (e.g., ED or ICU), time of age-adjusted qSOFA assessment, cutoff criteria of age-adjusted qSOFA, true positives, false positives, true negatives, and false negatives derived from the sensitivity and specificity of the age-adjusted qSOFA in predicting mortality and disease severity. When studies comprised multiple groups, each group was considered as an individual study.

Quality assessment

Currently, there is no widely used assessment tool for assessing the quality of studies of predictive risk scores. This study used a revised seven-item quality assessment scale27,28, which was derived from the Quality Assessment of Diagnostic Accuracy Studies tool29 and Newcastle–Ottawa Scale30. It comprises seven criteria: unbiased patient selection; representative of a wide spectrum of disease severity; predictor variables assessed blinded to outcome; outcome assessed blinded to the predictor variables; accurate definition of outcomes; availability of the same clinical data; and adequate follow-up27,28. We defined adequate follow-up as a follow-up of > 90%. Two reviewers (SE and SHY) independently performed the methodological quality assessment. Any disagreements were resolved by discussion.

Statistical analyses

Summary estimates of sensitivity, specificity, positive and negative likelihood ratios (LR+ and LR–), and pooled diagnostic odds ratio (DOR) were calculated using a bivariate random-effects model31. The DOR of a test (or score) is the ratio of the odds of positivity among patients versus the odds among healthy individuals or a control group32,33. When the DOR increases to greater than 1, the discriminative power of the outcome becomes greater32. We used summary receiver-operating characteristic (SROC) curves to calculate the area under the curve (AUC), which assisted in estimating the discriminative power of a test or score33. The AUC takes values between 0 and 1, with higher values indicating better test (or score) performance34. Heterogeneity of sensitivity and specificity were evaluated from the forest plots of the studies’ estimates and using a χ2 test (P < 0.1, significant). In the presence of significant heterogeneity, we conducted meta-regression analysis and a priori planned subgroup analysis to explore the sources of heterogeneity using the following as covariates with 95% confidence interval (CI): patient source (ED vs. ICU); sample size (< 10,000 vs. ≥ 10,000); outcome (mortality vs. disease severity); scales for assessing mental status in age-adjusted qSOFA (GCS vs. Alert, Voice, Pain, Unresponsive [AVPU] scale); age-specific vital signs criteria (2005 IPSCC definition vs. others); center (single center vs. multicenter); and cut-off value (≥ 2 vs. ≥ 1). We excluded studies in the meta-regression analysis if they used both the GCS and AVPU scale for mental status checks, or if their primary outcome was both in-hospital mortality and disease severity concomitantly. In addition, we also performed pooled analysis using one study population per study to examine whether the results were biased by including the same populations multiple times. As the reason for separation into different datasets from a study varies (e.g., outcomes, cutoff, age-specific vital signs criteria), we selected the data using the cutoff value of ≥ 2 and 2005 IPSCC definition as age-specific vital sign criteria. We measured publication bias with visualization of funnel plots and Egger’s test. Statistical analyses and meta-analyses were conducted using R program, version 3.6.3 (R Foundation for Statistical Computing, Vienna, Austria); P-values < 0.05 were considered statistically significant.

Results

PubMed, EMBASE, Web of Science and Cochrane database searches as per the predefined search words revealed 81 articles. After removing duplicates and screening abstracts, 20 full-text articles were read, resulting in 11 articles that met the inclusion criteria for the systematic review and meta-analysis. Reasons for exclusion are shown in the flow diagram (Fig. 1). Data from 172,569 patients of 11 observational studies13,23,24,25,35 were finally included. The general characteristics of the included studies are presented in Table 1 and Supplementary Table S1.

Figure 1
figure 1

Flow diagram of the search and selection process.

Full size image
Table 1 Characteristics of the included studies.
Full size table

Study characteristics

The majority of studies were retrospective, and only one study13 was prospective. Six studies13,24,25 were designed to evaluate the value of the age-adjusted qSOFA in predicting mortality. Four studies24,35 evaluated the performance of the age-adjusted qSOFA in predicting disease severity; three studies24,35 evaluated the value of qSOFA in predicting ICU admission, and one study35 evaluated the value of qSOFA in predicting the development of severe sepsis11. Only one study23 was designed to evaluate the ability of the age-adjusted qSOFA to predict both ICU transfer and/or mortality within 30 days as the primary outcome.

Patient sources were as follows: 122,943 ICU patients from four studies13,25, 49,448 ED patients from five studies23,24, and 178 pediatric tertiary referral center patients from two studies35. The majority of the studies were single-center studies (n = 7, 63.6%)23,24,35 and chose cut-off criteria as ≥ 2 (n = 9, 81.8%)13,23,24,25,35. Most studies (n = 9, 81.8%)13,23,24,25,35 adopted the 2005 IPSCC definition for age-specific vital signs criteria. Six (54.5%) 13,25,35 studies used GCS, four studies (36.4%)24 used AVPU, and one study23 used either GCS or AVPU to assess mental status. All of the studies were published between 2018 and 2020 (Table 1 and Supplementary Table S1).

Quality assessment of the included studies

Three studies (27.3%) enrolled patients consecutively25, and one single-center study23 defined suspected bacterial infection as the commencement of antibiotics within 24 h after ED arrival at the non-academic facility and excluded surgical diagnoses; thus, the study was deemed not representative of a wide spectrum of disease severity. Although all studies assessed the predictor variables that constituted the age-adjusted qSOFA blinded to outcomes, no studies clearly reported that the outcomes were assessed blindly to the age-adjusted qSOFA. Overall, outcomes were clearly defined and the same clinical data was available in all studies. All included studies showed adequate follow-up of patients (Supplementary Table S2).

Age-adjusted qSOFA for predicting mortality and disease severity

Included studies showed a wide range of sensitivities (0.29–1.00) and specificities (0.05–0.99) (Fig. 2 and Supplementary Table S3). Pooled sensitivity and specificity of qSOFA for predicting mortality and disease severity are 0.685 (95% CI 0.527–0.809) and 0.706 (95% CI 0.362–0.911), respectively. Pooled LR+, LR–, and DOR were 2.919 (95% CI 2.186–3.898), 0.519 (95% CI 0.428–0.630), and 6.565 (95% CI 4.459–9.667), respectively. The AUC was 0.733 (95% CI 0.683–0.768) (Fig. 3). Significant heterogeneity was noted in terms of sensitivity (χ2 = 233.079; P < 0.001) and specificity (χ2 = 47,040.96; P < 0.001). No significant publication bias was found in the funnel plot and Egger’s test (P = 0.8671) (Supplementary Fig. S1). When including one study population per study, five studies with a total of 55,301 patients were included and analyzed. Compared with the original results, the pooled sensitivity was slightly lower (0.685 vs. 0.571), and pooled specificity was higher (0.706 vs. 0.851). However, the pooled results of DOR (6.565 vs. 6.816) and AUC (0.733 vs. 0.711) were similar (Supplementary Tables S4 and S5 and Supplementary Figs. S2–S4).

Figure 2
figure 2

Coupled forest plots of the sensitivity and specificity of the age-adjusted quick Sequential Organ Failure Assessment score for predicting mortality and disease severity in pediatric patients with suspected or confirmed infection.

Full size image
Figure 3
figure 3

Summary receiver-operating characteristic (SROC) curve of the predictive performance of age-adjusted quick Sequential Organ Failure Assessment score for mortality and disease severity in pediatric patients with suspected or confirmed infection. The area under the curve of the SROC was 0.733 (95% CI 0.683–0.768). conf.region, 95% confidence region for SROC curve.

Full size image

Heterogeneity exploration and subgroup analysis

A meta-regression analysis revealed that patient source, sample size, outcome, center, and scales for assessing mental status were significant factors affecting heterogeneity (Supplementary Table S6). When comparing summary estimates of the DOR between subgroups, significant differences were only found in relation to patient source and scales for assessing mental status (Table 2). Age-adjusted qSOFA showed a significantly higher pooled DOR in the studies including ED patients than in the studies including ICU patients [22.214 (95% CI 7.115–69.360) vs. 4.092 (95% CI 3.058–5.474), P = 0.005]; Studies that used the AVPU to assess mental status also showed a significantly higher pooled DOR compared with studies that used the GCS [23.009 (95% CI 4.559–116.123) vs. 3.968 (95% CI 3.015–5.224), P = 0.036].

Table 2 Subgroup analysis.
Full size table

Age-adjusted qSOFA for mortality

Six studies assessed the performance of age-adjusted qSOFA for predicting mortality. Pooled sensitivity and specificity of age-adjusted qSOFA for predicting mortality were 0.729 (95% CI 0.655–0.792) and 0.626 (95% CI 0.206–0.915), respectively. Pooled LR+, LR– and DOR were 1.925 (95% CI 1.629–2.275), 0.452 (95% CI 0.381–0.535), and 4.433 (95% CI 3.223–6.097), respectively. AUC was found to be 0.735 (95% CI 0.677–0.780) (Table 3). Further detailed accuracy estimates, coupled forest plots, SROC curves, and heterogeneity test results of studies evaluating predictive accuracy of the age-adjusted qSOFA for mortality are provided in the Supplementary Tables S7 and S8 and Supplementary Figs. S5–S7.

Table 3 Summary estimates of the predictive accuracy of the age-adjusted quick Sequential Organ Failure Assessment score according to the outcome.
Full size table

Age-adjusted qSOFA for disease severity

Four studies reported the performance of age-adjusted qSOFA for predicting disease severity. Pooled sensitivity and specificity of age-adjusted qSOFA for predicting disease severity was 0.731 (95% CI 0.207–0.966) and 0.724 (95% CI 0.113–0.982). Pooled LR+, LR–, and DOR were 3.341 (95% CI 0.498–22.416), 0.708 (95% CI 0.618–0.811), and 8.866 (95% CI 1.355–58.035), respectively. The AUC was 0.786 (95% CI 0.518–0.905) (Table 3). Detailed accuracy estimates, coupled forest plots, SROC curves, and heterogeneity test results of studies evaluating the predictive accuracy of the age-adjusted qSOFA for disease severity are provided in the Supplementary Tables S9 and S10 and Supplementary Figs. S8–S10.

Discussion

In this review, we assessed the performance of the age-adjusted qSOFA in predicting mortality and disease severity in pediatric patients with suspected or confirmed infection. We identified 11 studies, including 172,569 patients from the ED, pediatric tertiary referral center, and ICU. We found that the age-adjusted qSOFA had a moderate performance for predicting in-hospital mortality and disease severity in pediatric patients.

The qSOFA was initially recommended by the SEPSIS-3 task force as a readily available bedside tool16,36, and the age-adjusted qSOFA has the same advantages: it does not require laboratory tests and enables prompt and repeatable assessment of patients. However, as a screening tool to identify ‘at-risk patients’, the age-adjusted qSOFA satisfies the requirements for convenience and feasibility, but does not satisfy the requirement for high sensitivity37. In clinical practice, screening tools typically require high sensitivity to safely rule out those at low risk of adverse outcomes38.

Determining which patients are at high risk of severe illness or mortality is essential for appropriate clinical decision making. When clinicians initially encounter pediatric patients with suspected infection, the specific outcomes (e.g. mortality, ICU admission or prolonged hospital admission itself) would be not matter at that moment, only whether this patient has a potential to become a severe, critical patient requiring close observation, and intensive treatment will be of more interest to clinicians. Thus, we intended to assess the predictive performance of age-adjusted qSOFA as a quick, easy, bedside screening tool for identifying these ‘at risk patients’. Then, we demonstrated the individual performance of age-adjusted qSOFA according to the specific outcomes, such as mortality and disease severity, for clinicians to consider further prognostic aspects.

As described in previous studies39,40,41, we assessed the discriminative power of the prediction score (age-adjusted qSOFA) for identifying at-risk pediatric patients by calculating AUC. An AUC above 0.7 was considered to be acceptable and useful34,40. In our results, aged-adjusted qSOFA achieved an AUC of 0.733, indicating a useful discrimination for pediatric patients at risk who need close monitoring and intensive treatment.

Likewise, the DOR was also calculated as another single indicator of age-adjusted qSOFA performance for discrimination of at-risk patients42. DOR of 6.57 in our result means that the odds of positivity (above cutoff value of age-adjusted qSOFA) in at risk patients is about six times higher than the odds of positivity in non-risk patients. DOR does not depend on disease prevalence33. However, it depends on what criteria are used to define disease or pathological conditions of the study population (e.g., comorbidity, disease severity)33. Because considerable heterogeneity existed in our analysis, we conducted the subgroup analysis of DOR of age-adjusted qSOFA according to the various factors that can affect the results and also the causes of the heterogeneity in the pooled analysis.

Regarding patient sources, qSOFA has reported a better predictive power to that of the full SOFA for in-hospital mortality in adult patients outside the ICU16,17. However, the full SOFA showed higher predictive validity when compared with the qSOFA among patients in the ICU16,17. The majority of patients in ICU are administered vasopressor support and/or mechanical ventilation, thus the qSOFA may not have a reasonable clinical value for patients in this setting17. Our results also found that the age-adjusted qSOFA has a better DOR for predicting mortality and disease severity in ED patients than in ICU patients. These results showed that the age-adjusted qSOFA is more useful for screening pediatric patients outside the ICU.

Scales assessing mental status is a significant source of heterogeneity in this meta-analysis. Currently, the qSOFA in adults uses the GCS16. In our analysis, studies using the AVPU to assess mental status showed higher predictive performance than studies using the GCS. The AVPU scale is less complex than the GCS, and uses only four categories (Alert; Verbal response; response to Pain; Unresponsive). The AVPU scale can be used quickly and easily43 and has been reported to correlate well with the GCS44. According to the results of this study, it is reasonable to use the AVPU scale to assess mental status in the age-adjusted qSOFA.

However, there are important limitations to the application of the age-adjusted qSOFA in the pediatric field. First, there is a global tendency not to measure blood pressure in pediatric acute care settings45. In addition, hypotension presents at a late stage of septic shock in pediatric patients7,46. Unlike adults, blood pressure is typically maintained in children in the early stage of septic shock, compensated by increased heart rate and systemic vascular resistance47,48,49. Thus, it may not be a valuable measure in frontline health care facilities such as ED47.

To address these limitations, Romaine et al.24 suggested a novel scale, the “Liverpool quick Sequential Organ Failure Assessment (LqSOFA)” score. The LqSOFA score, which ranges 0–4, comprises respiratory rate, age-adjusted heart rate, capillary refill time, and level of consciousness assessed using the AVPU scale. Romaine et al.24 reported that LqSOFA with ≥ 2 criteria showed equal sensitivity (0.6) and specificity (0.988) for predicting sepsis-related mortality when compared with an age-adjusted qSOFA with ≥ 2 criteria. In addition, when compared with age-adjusted qSOFA ≥ 2 criteria, LqSOFA ≥ 2 criteria showed low but better sensitivity (0.392 vs. 0.289) and similar high specificity (0.992 vs. 0.991) for the prediction of critical care admission within 48 h.

When comparing SIRS to qSOFA criteria, recent meta-analyses have consistently presented a higher sensitivity but lower specificity of SIRS criteria than those of the qSOFA for the prediction of in-hospital mortality among adult patients in various clinical settings39,50,51,52,53. In this review, we could not compare the pooled predictive performance of the age-adjusted qSOFA with that of the SIRS criteria, because there were few pediatric studies that provided the required data. Schlapbach et al.13 compared the predictive performance of the age-adjusted qSOFA with that of SIRS criteria for in-hospital mortality and showed that the age-adjusted qSOFA had a lower sensitivity but higher specificity and LR+ than those of SIRS criteria. Higher specificity and LR+ indicate that the age-adjusted qSOFA is a better scale for ruling in pediatric patients at risk of mortality. Regarding disease severity, van Nassau et al.23 reported that age-adjusted qSOFA ≥ 2 criteria showed lower sensitivity and LR+ but higher specificity than those of SIRS ≥ 2 criteria in predicting prolonged hospitalization (length of stay ≥ 7 days).

The present study has several strengths. As far as we are aware, our review is the first systematic review and meta-analysis to evaluate the predictive performance of the age-adjusted qSOFA in pediatric patients. Our meta-analyses used data from favorable quality studies with a large sample size. This may provide the substrate for future guidelines for screening infectious pediatric patients who are likely to progress to severe disease, or who are at risk of death.

This study has some limitations. First, there is a significant heterogeneity in this meta-analysis. We investigated the factors affecting heterogeneity by meta-regression and subgroup analysis, although we could not investigate factors related to differing diagnostic criteria and specific clinical settings. Second, if the predictive ability of the age-adjusted qSOFA was assessed with different outcomes or with different criteria in a same cohort, we consider them as separate studies54. Because the pooled results of AUC and DOR were similar between results using one study population per study and results using several datasets from the same study population, the overall predictive power of the age-adjusted qSOFA for mortality and morbidity can be considered similar in both analyses. This also indicates that the results would not to be strongly biased by including the same populations multiple times. Nevertheless, our results still need to be interpreted and applied cautiously because the same study population were pooled. Third, most of the included studies are retrospective studies, which were not devised to the validate the age-adjusted qSOFA. Fourth, most of the studies were conducted in western countries. Further studies are required to ensure the applicability of the results of studies of the age-adjusted qSOFA to other countries. Fifth, we did not search gray literature, as we aimed to review the characteristics of published literature. Incorporating a gray literature search may help to minimize the effects of publication bias55; however, we found no significant publication bias in this analysis. Sixth, long-term outcomes or healthcare costs were not available in the literature that was included; thus we could not evaluate these in this analysis. Finally, we could not compare the overall predictive performance of the age-adjusted qSOFA with other predictive biomarkers, due to the limited number of clinical studies.

Conclusions

Current evidence suggests that the age-adjusted qSOFA has a moderate predictive value for mortality and disease severity in pediatric patients with suspected or confirmed infection. The age-adjusted qSOFA is a simple and feasible tool to use in settings outside the ICU such as ED, and in resource-limited settings. However, a screening tool with higher sensitivity for pediatric patients is needed.