Age-adjusted quick Sequential Organ Failure Assessment score for predicting mortality and disease severity in children with infection: a systematic review and meta-analysis

We assessed the diagnostic accuracy of the age-adjusted quick Sequential Organ Failure Assessment score (qSOFA) for predicting mortality and disease severity in pediatric patients with suspected or confirmed infection. We conducted a systematic search of PubMed, EMBASE, the Cochrane Library, and Web of Science. Eleven studies with a total of 172,569 patients were included in the meta-analysis. The pooled sensitivity, specificity, and diagnostic odds ratio of the age-adjusted qSOFA for predicting mortality and disease severity were 0.69 (95% confidence interval [CI] 0.53–0.81), 0.71 (95% CI 0.36–0.91), and 6.57 (95% CI 4.46–9.67), respectively. The area under the summary receiver-operating characteristic curve was 0.733. The pooled sensitivity and specificity for predicting mortality were 0.73 (95% CI 0.66–0.79) and 0.63 (95% CI 0.21–0.92), respectively. The pooled sensitivity and specificity for predicting disease severity were 0.73 (95% CI 0.21–0.97) and 0.72 (95% CI 0.11–0.98), respectively. The performance of the age-adjusted qSOFA for predicting mortality and disease severity was better in emergency department patients than in intensive care unit patients. The age-adjusted qSOFA has moderate predictive power and can help in rapidly identifying at-risk children, but its utility may be limited by its insufficient sensitivity.

Study selection, eligibility, and data extraction. Two authors (SHY and HK) independently conducted literature searches of PubMed, EMBASE, the Cochrane Library, and Web of Science, without language or time restrictions, on January 6, 2021, with the aim of finding eligible studies assessing the performance of age-adjusted qSOFA in predicting mortality and/or disease severity in pediatric patients with suspected or confirmed infection. Various combinations of the following key words were used in the systematic search: "Quick Sequential Organ Failure Assessment, " "qSOFA, " "quick SOFA, " "q-SOFA, " "quick-SOFA, " and "pediatric, " "child, " "adolescent, " "infant, " and "neonate. " Studies were eligible if they aimed to assess the performance of age-adjusted qSOFA to predict mortality or disease severity in pediatric patients (aged < 18 years) with suspected or confirmed infection. We used the following as indicators reflecting disease severity: admission or transfer to an ICU (including a critical care unit), development of severe sepsis 11 , or prolonged hospital stay (dependent on the authors' definition, regardless of duration). If enrolled patients received a diagnostic code (e.g., International Classification of Diseases code) indicative of an infection or were diagnosed with sepsis/septic shock via consensus definition, we accepted them as patients with confirmed infection. In addition, if enrolled patients had signs or symptoms of infection (e.g., fever), or were treated for a bacterial infection (e.g., treated with therapeutic antibiotics), we inferred suspected infection. Studies were included if they reported sufficient data to construct a 2 × 2 contingency tables. Reviews, editorials, expert opinions, animal experiments, or studies presenting duplicate data were excluded.
The following information was retrieved from each study: first author, publication year, sample size, patient source (e.g., ED or ICU), time of age-adjusted qSOFA assessment, cutoff criteria of age-adjusted qSOFA, true positives, false positives, true negatives, and false negatives derived from the sensitivity and specificity of the age-adjusted qSOFA in predicting mortality and disease severity. When studies comprised multiple groups, each group was considered as an individual study. Quality assessment. Currently, there is no widely used assessment tool for assessing the quality of studies of predictive risk scores. This study used a revised seven-item quality assessment scale 27,28 , which was derived from the Quality Assessment of Diagnostic Accuracy Studies tool 29 and Newcastle-Ottawa Scale 30 . It comprises seven criteria: unbiased patient selection; representative of a wide spectrum of disease severity; predictor variables assessed blinded to outcome; outcome assessed blinded to the predictor variables; accurate definition of outcomes; availability of the same clinical data; and adequate follow-up 27,28 . We defined adequate follow-up as a follow-up of > 90%. Two reviewers (SE and SHY) independently performed the methodological quality assessment. Any disagreements were resolved by discussion.
Statistical analyses. Summary estimates of sensitivity, specificity, positive and negative likelihood ratios (LR+ and LR-), and pooled diagnostic odds ratio (DOR) were calculated using a bivariate random-effects model 31 . The DOR of a test (or score) is the ratio of the odds of positivity among patients versus the odds among healthy individuals or a control group 32,33 . When the DOR increases to greater than 1, the discriminative power of the outcome becomes greater 32 . We used summary receiver-operating characteristic (SROC) curves to calculate the area under the curve (AUC), which assisted in estimating the discriminative power of a test or score 33 . The AUC takes values between 0 and 1, with higher values indicating better test (or score) performance 34 . Heterogeneity of sensitivity and specificity were evaluated from the forest plots of the studies' estimates and using a χ 2 test (P < 0.1, significant). In the presence of significant heterogeneity, we conducted meta-regression analysis and a priori planned subgroup analysis to explore the sources of heterogeneity using the following as covariates with 95% confidence interval (CI): patient source (ED vs. ICU); sample size (< 10,000 vs. ≥ 10,000); outcome (mortality vs. disease severity); scales for assessing mental status in age-adjusted qSOFA (GCS vs. Alert, Voice, Pain, Unresponsive [AVPU] scale); age-specific vital signs criteria (2005 IPSCC definition vs. others); center (single center vs. multicenter); and cut-off value (≥ 2 vs. ≥ 1). We excluded studies in the meta-regression analysis if they used both the GCS and AVPU scale for mental status checks, or if their primary outcome was both in-hospital mortality and disease severity concomitantly. In addition, we also performed pooled analysis using www.nature.com/scientificreports/ one study population per study to examine whether the results were biased by including the same populations multiple times. As the reason for separation into different datasets from a study varies (e.g., outcomes, cutoff, age-specific vital signs criteria), we selected the data using the cutoff value of ≥ 2 and 2005 IPSCC definition as age-specific vital sign criteria. We measured publication bias with visualization of funnel plots and Egger's test. Statistical analyses and meta-analyses were conducted using R program, version 3.6.3 (R Foundation for Statistical Computing, Vienna, Austria); P-values < 0.05 were considered statistically significant.

Results
PubMed, EMBASE, Web of Science and Cochrane database searches as per the predefined search words revealed 81 articles. After removing duplicates and screening abstracts, 20 full-text articles were read, resulting in 11 articles that met the inclusion criteria for the systematic review and meta-analysis. Reasons for exclusion are shown in the flow diagram ( Fig. 1). Data from 172,569 patients of 11 observational studies 13,[23][24][25]35 were finally included. The general characteristics of the included studies are presented in Table 1 and Supplementary Table S1.

Study characteristics.
The majority of studies were retrospective, and only one study 13 was prospective.
Six studies 13,24,25 were designed to evaluate the value of the age-adjusted qSOFA in predicting mortality. Four studies 24,35 evaluated the performance of the age-adjusted qSOFA in predicting disease severity; three studies 24,35 evaluated the value of qSOFA in predicting ICU admission, and one study 35 evaluated the value of qSOFA in predicting the development of severe sepsis 11 . Only one study 23 was designed to evaluate the ability of the ageadjusted qSOFA to predict both ICU transfer and/or mortality within 30 days as the primary outcome. Patient sources were as follows: 122,943 ICU patients from four studies 13,25 , 49,448 ED patients from five studies 23,24 , and 178 pediatric tertiary referral center patients from two studies 35 . The majority of the studies were single-center studies (n = 7, 63.6%) 23,24,35 and chose cut-off criteria as ≥ 2 (n = 9, 81.8%) 13,[23][24][25]35 . Most studies (n = 9, 81.8%) 13,23-25,35 adopted the 2005 IPSCC definition for age-specific vital signs criteria. Six (54.5%) 13,25,35 studies used GCS, four studies (36.4%) 24 used AVPU, and one study 23 used either GCS or AVPU to assess mental status. All of the studies were published between 2018 and 2020 (Table 1 and Supplementary Table S1).

Quality assessment of the included studies.
Three studies (27.3%) enrolled patients consecutively 25 , and one single-center study 23 defined suspected bacterial infection as the commencement of antibiotics within 24 h after ED arrival at the non-academic facility and excluded surgical diagnoses; thus, the study was deemed not representative of a wide spectrum of disease severity. Although all studies assessed the predictor variables that constituted the age-adjusted qSOFA blinded to outcomes, no studies clearly reported that the outcomes www.nature.com/scientificreports/ were assessed blindly to the age-adjusted qSOFA. Overall, outcomes were clearly defined and the same clinical data was available in all studies. All included studies showed adequate follow-up of patients (Supplementary  Table S2).

Heterogeneity exploration and subgroup analysis.
A meta-regression analysis revealed that patient source, sample size, outcome, center, and scales for assessing mental status were significant factors affecting heterogeneity (Supplementary Table S6). When comparing summary estimates of the DOR between subgroups, significant differences were only found in relation to patient source and scales for assessing mental status (

Discussion
In this review, we assessed the performance of the age-adjusted qSOFA in predicting mortality and disease severity in pediatric patients with suspected or confirmed infection. We identified 11 studies, including 172,569 patients from the ED, pediatric tertiary referral center, and ICU. We found that the age-adjusted qSOFA had a moderate performance for predicting in-hospital mortality and disease severity in pediatric patients. The qSOFA was initially recommended by the SEPSIS-3 task force as a readily available bedside tool 16,36 , and the age-adjusted qSOFA has the same advantages: it does not require laboratory tests and enables prompt and repeatable assessment of patients. However, as a screening tool to identify 'at-risk patients' , the age-adjusted qSOFA satisfies the requirements for convenience and feasibility, but does not satisfy the requirement for high sensitivity 37 . In clinical practice, screening tools typically require high sensitivity to safely rule out those at low risk of adverse outcomes 38 .
Determining which patients are at high risk of severe illness or mortality is essential for appropriate clinical decision making. When clinicians initially encounter pediatric patients with suspected infection, the specific outcomes (e.g. mortality, ICU admission or prolonged hospital admission itself) would be not matter at that moment, only whether this patient has a potential to become a severe, critical patient requiring close observation, and intensive treatment will be of more interest to clinicians. Thus, we intended to assess the predictive performance of age-adjusted qSOFA as a quick, easy, bedside screening tool for identifying these 'at risk patients' . Then, we demonstrated the individual performance of age-adjusted qSOFA according to the specific outcomes, such as mortality and disease severity, for clinicians to consider further prognostic aspects.
As described in previous studies [39][40][41] , we assessed the discriminative power of the prediction score (ageadjusted qSOFA) for identifying at-risk pediatric patients by calculating AUC. An AUC above 0.7 was considered to be acceptable and useful 34,40 . In our results, aged-adjusted qSOFA achieved an AUC of 0.733, indicating a useful discrimination for pediatric patients at risk who need close monitoring and intensive treatment.  Table 3. Summary estimates of the predictive accuracy of the age-adjusted quick Sequential Organ Failure Assessment score according to the outcome. AUC, area under the curve; CI, confidence interval; SROC, summary receiver-operating characteristic. www.nature.com/scientificreports/ Likewise, the DOR was also calculated as another single indicator of age-adjusted qSOFA performance for discrimination of at-risk patients 42 . DOR of 6.57 in our result means that the odds of positivity (above cutoff value of age-adjusted qSOFA) in at risk patients is about six times higher than the odds of positivity in nonrisk patients. DOR does not depend on disease prevalence 33 . However, it depends on what criteria are used to define disease or pathological conditions of the study population (e.g., comorbidity, disease severity) 33 . Because considerable heterogeneity existed in our analysis, we conducted the subgroup analysis of DOR of age-adjusted qSOFA according to the various factors that can affect the results and also the causes of the heterogeneity in the pooled analysis.
Regarding patient sources, qSOFA has reported a better predictive power to that of the full SOFA for inhospital mortality in adult patients outside the ICU 16,17 . However, the full SOFA showed higher predictive validity when compared with the qSOFA among patients in the ICU 16,17 . The majority of patients in ICU are administered vasopressor support and/or mechanical ventilation, thus the qSOFA may not have a reasonable clinical value for patients in this setting 17 . Our results also found that the age-adjusted qSOFA has a better DOR for predicting mortality and disease severity in ED patients than in ICU patients. These results showed that the age-adjusted qSOFA is more useful for screening pediatric patients outside the ICU.
Scales assessing mental status is a significant source of heterogeneity in this meta-analysis. Currently, the qSOFA in adults uses the GCS 16 . In our analysis, studies using the AVPU to assess mental status showed higher predictive performance than studies using the GCS. The AVPU scale is less complex than the GCS, and uses only four categories (Alert; Verbal response; response to Pain; Unresponsive). The AVPU scale can be used quickly and easily 43 and has been reported to correlate well with the GCS 44 . According to the results of this study, it is reasonable to use the AVPU scale to assess mental status in the age-adjusted qSOFA.
However, there are important limitations to the application of the age-adjusted qSOFA in the pediatric field. First, there is a global tendency not to measure blood pressure in pediatric acute care settings 45 . In addition, hypotension presents at a late stage of septic shock in pediatric patients 7,46 . Unlike adults, blood pressure is typically maintained in children in the early stage of septic shock, compensated by increased heart rate and systemic vascular resistance [47][48][49] . Thus, it may not be a valuable measure in frontline health care facilities such as ED 47 .
To address these limitations, Romaine et al. 24 suggested a novel scale, the "Liverpool quick Sequential Organ Failure Assessment (LqSOFA)" score. The LqSOFA score, which ranges 0-4, comprises respiratory rate, ageadjusted heart rate, capillary refill time, and level of consciousness assessed using the AVPU scale. Romaine et al. 24 reported that LqSOFA with ≥ 2 criteria showed equal sensitivity (0.6) and specificity (0.988) for predicting sepsis-related mortality when compared with an age-adjusted qSOFA with ≥ 2 criteria. In addition, when compared with age-adjusted qSOFA ≥ 2 criteria, LqSOFA ≥ 2 criteria showed low but better sensitivity (0.392 vs. 0.289) and similar high specificity (0.992 vs. 0.991) for the prediction of critical care admission within 48 h.
When comparing SIRS to qSOFA criteria, recent meta-analyses have consistently presented a higher sensitivity but lower specificity of SIRS criteria than those of the qSOFA for the prediction of in-hospital mortality among adult patients in various clinical settings 39,[50][51][52][53] . In this review, we could not compare the pooled predictive performance of the age-adjusted qSOFA with that of the SIRS criteria, because there were few pediatric studies that provided the required data. Schlapbach et al. 13 compared the predictive performance of the age-adjusted qSOFA with that of SIRS criteria for in-hospital mortality and showed that the age-adjusted qSOFA had a lower sensitivity but higher specificity and LR+ than those of SIRS criteria. Higher specificity and LR+ indicate that the age-adjusted qSOFA is a better scale for ruling in pediatric patients at risk of mortality. Regarding disease severity, van Nassau et al. 23 reported that age-adjusted qSOFA ≥ 2 criteria showed lower sensitivity and LR+ but higher specificity than those of SIRS ≥ 2 criteria in predicting prolonged hospitalization (length of stay ≥ 7 days).
The present study has several strengths. As far as we are aware, our review is the first systematic review and meta-analysis to evaluate the predictive performance of the age-adjusted qSOFA in pediatric patients. Our meta-analyses used data from favorable quality studies with a large sample size. This may provide the substrate for future guidelines for screening infectious pediatric patients who are likely to progress to severe disease, or who are at risk of death.
This study has some limitations. First, there is a significant heterogeneity in this meta-analysis. We investigated the factors affecting heterogeneity by meta-regression and subgroup analysis, although we could not investigate factors related to differing diagnostic criteria and specific clinical settings. Second, if the predictive ability of the age-adjusted qSOFA was assessed with different outcomes or with different criteria in a same cohort, we consider them as separate studies 54 . Because the pooled results of AUC and DOR were similar between results using one study population per study and results using several datasets from the same study population, the overall predictive power of the age-adjusted qSOFA for mortality and morbidity can be considered similar in both analyses. This also indicates that the results would not to be strongly biased by including the same populations multiple times. Nevertheless, our results still need to be interpreted and applied cautiously because the same study population were pooled. Third, most of the included studies are retrospective studies, which were not devised to the validate the age-adjusted qSOFA. Fourth, most of the studies were conducted in western countries. Further studies are required to ensure the applicability of the results of studies of the age-adjusted qSOFA to other countries. Fifth, we did not search gray literature, as we aimed to review the characteristics of published literature. Incorporating a gray literature search may help to minimize the effects of publication bias 55 ; however, we found no significant publication bias in this analysis. Sixth, long-term outcomes or healthcare costs were not available in the literature that was included; thus we could not evaluate these in this analysis. Finally, we could not compare the overall predictive performance of the age-adjusted qSOFA with other predictive biomarkers, due to the limited number of clinical studies.