Development and validation of a new risk scoring system for solid tumor patients with suspected infection

This study aimed to develop a new prognostic model for predicting 30-day mortality in solid tumor patients with suspected infection. This study is a retrospective cohort study and was conducted from August 2019 to December 2019 at a single center. Adult active solid tumor patients with suspected infection were enrolled among visitors to the emergency room (ER). Logistic regression analysis was used to identify potential predictors for a new model. A total of 899 patients were included; 450 in the development cohort and 449 in the validation cohort. Six independent variables predicted 30-day mortality: Eastern Cooperative Oncology Group (ECOG) performance status (PS), peripheral oxygen saturation (SpO2), creatinine, bilirubin, C-reactive protein (CRP), and lactate. The C-statistic of the new scoring system was 0.799 in the development cohort and 0.793 in the validation cohort. The C-statistics in the development cohort was significantly higher than those of SOFA [0.723 (95% CI: 0.663–0.783)], qSOFA [0.596 (95% CI: 0.537–0.655)], and SIRS [0.547 (95% CI: 0.483–0.612)]. The discriminative capability of the new cancer-specific risk scoring system was good in solid tumor patients with suspected infection. The new scoring model was superior to SOFA, qSOFA, and SIRS in predicting mortality.

Data collection and evaluation. Data were collected retrospectively from the hospital's electronic medical records. Clinical variables included demographics, comorbidities, type of cancer, cancer stage, Eastern Cooperative Oncology Group (ECOG) performance status (PS) 13 , and initially measured vital signs, including mental status on arrival the ER. Comorbidities, including hypertension, diabetes mellitus, chronic renal disease, chronic liver disease, chronic lung disease, cardiovascular disease, and cerebrovascular disease, were analyzed based on the medical records at ER presentation. Chronic renal disease is defined as the glomerular filtration rate (GFR) has last less than 60 ml/min/1.73 m 2 through 3 more months. Patients who could not check the previous GFR value were excluded. According to the Alert/responsive to Voice/responsive to Pain/Unresponsive (AVPU) scale, mental status was assessed. The AVPU values could be substituted to Glasgow Coma Scale (GCS) scores of 15,13,8, and 6, respectively 14 . A score of < 15 on the GCS was defined as an indication of altered mental status. Three sets of blood cultures were obtained before the administration of IV antibiotics during the ER stay; if patients had a catheter, one set of the three was drawn from the catheter. Laboratory data included complete blood count, chemistry, electrolytes, coagulation battery, inflammatory markers such as C-reactive protein (CRP), and serum lactate. The first obtained value was used in cases with multiple test results. For lactate, the values higher than 15.0 mmol/L were reported as ' > 15.0 mmol/L' . The SIRS, qSOFA, and SOFA scores were calculated based on the ER's physiological and laboratory data. Statistical analysis. The primary outcome was 30-day mortality. Data were reported as the mean ± standard deviation (SD) or median and inter-quartile range for continuous variables. They were compared between groups using the Student's t-test. Categorical variables were presented as the number and percentage and were compared using a chi-squared test.
To develop a new scoring system for 30-day mortality, univariate and multivariate logistic regression analyses were performed with an entering procedure in the development cohort. Results were summarized as odds ratios (OR) and respective 95% confidence intervals (CI). Variables with a P-value of < 0.1 in the univariate analysis and clinical relevance were considered for the multivariate analysis. A simple risk score was devised using the penalized maximum likelihood estimates of the predictors in the multivariable model. The constant of the scoring system was defined as one-third of the highest regression coefficient. The score was the weighted sum of those predictors. The weights were defined as the rounded integer value of the regression coefficients' quotient value divided by the constant. The risk score's discrimination capability was assessed using the C-statistic. Continuous variables were categorized in the new prediction model. The cutoff values of creatinine and total bilirubin were determined based on the normal values of our institution. The cutoff values of SpO 2 (94%), lactate (2 mmol/L), and CRP (10 mg/dL) were set by referring to previous studies.
The calibration capability of the risk score was assessed by the Hosmer-Lemeshow test and calibration plot. Internal validation was performed by bootstrapping with 1000 iterations and calculated optimism-corrected C-statistic. The risk score was then categorized into low, intermediate, and high-risk groups based on the likelihood of 30-day mortality. A P-value of < 0.05 was considered statistically significant 15,16 .
The predictive performances of the SOFA, qSOFA, SIRS scores, and the new scoring system were analyzed by using the C-statistics values and compared with Delong's test. A C-statistics of 1.0 denotes perfect, whereas a value close to 0.50 indicates no apparent accuracy. All statistical analyses were performed using SAS software version 9.4 (SAS Institute Inc., Cary, NC), R version 3.6.1 (R Foundation for Statistical Computing, Vienna, Austria, http:// www.R-proje ct. org), and IBM SPSS Statistics for Windows, version 21.0 (IBM Corp., Armonk, NY, USA).
Ethics approval. This retrospective chart review study involving human participants was in accordance with the ethical standards of the institutional and national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. The Institutional Review Board of Asan medical center approved this study.

Results
Baseline characteristics of the total population. As shown in Fig. 1, 1331 cancer patients were initially screened at ER during the study period, and 126 with no suspected infection or low probability of infection were excluded. Among the remaining 1205 patients, 111 with hematologic malignancies, 37 with no adequate workup, 22 with used antibiotics before ER arrival, 128 with multiple visits, and 8 with follow-up loss were excluded additionally. A total of 899 patients were finally included in the study: 450 in the development cohort and 449 in the validation cohort. Table 1 shows the baseline characteristics in total study subjects. The mean age was 63.0 (SD 11.8) years, and the male was 55.6%. The most common cancer was lung cancer (19.2%), followed by biliary cancer (16.1%) and pancreatic cancer (13.9%). Lung and hepatobiliary infections were the most common in 24.4% and 25.1%. The overall 30-day mortality was 22.5%. There was a significant difference in the 30-day mortality rate between development and validation cohorts (19.3% and 25.6%, P = 0.024). Table 2 shows the comparison of characteristics between survivors and nonsurvivors in 30 days in the development cohort. Nonsurvivors were   (Fig. 2).

Discussion
We have developed a new cancer-specific scoring system for solid cancer patients with suspected infection. This new system had a strong discriminative power to predict 30-day mortality, showing a C-statistic of 0.799 in the development cohort and 0.793 in the validation cohort. In comparison with existing scoring systems, the new scoring system was superior to SOFA (C-statistic, 0.723), qSOFA (C-statistic, 0.596), and SIRS (C-statistic, 0.547). This new scoring system was developed and validated in two groups of similar size, recruited at different times in the same hospital. There was no significant difference in baseline characteristics between the two development and validation cohorts. In our study, the overall 30-day mortality rate was 22.5%. There was a difference in 30-day Table 2. Comparison of characteristics between survivors and non-survivors in 30 days in the development cohort. Values are expressed as the mean ± standard deviation and the number (%). CRP C-reactive protein, ECOG Eastern Cooperative Oncology Group, PS performance status, SBP systolic blood pressure, SpO 2 peripheral oxygen saturation, WBC white blood cells. a Infection was documented by radiologic studies, including chest x-ray, thorax or abdomen and pelvis computed tomography. b Infection was documented by body fluid culture: blood in 16.0%, urine in 15.8%, and sputum in 13.3%. c Infection was documented by molecular assay included polymerase chain reaction (PCR) for respiratory virus or antigen tests for Pneumococcus or Legionella, Aspergillus, etc.  25.6%, P = 0.024). However, the C-statistic (0.799 in the development cohort, 0.793 in the validation cohort) of our new scoring system showed a good discriminative capability to predict 30-day mortality in both groups. The Hosmer-Lemeshow goodness to fit test suggested that predicted mortality reflects true mortality, and thus our scoring system is well-calibrated. Several prognostic scoring systems have been used to predict prognosis in patients with infection [17][18][19] . SOFA is one of the most frequently validated systems and is an excellent predictor of mortality 20 . However, most of the SOFA studies were on non-cancer patients, and studies on those with cancer were limited with inconsistent results 8,9,21 . In a recent study, SOFA had good discriminative power in patients with cancer and was superior to qSOFA 8 . In two previous studies, both SOFA and qSOFA showed weak discriminative ability with a C-statistic of < 0.7 in predicting mortality for cancer patients with infection 9,21 . As shown in this study, our new scoring system can be a good alternative to predict mortality in cancer patients with suspected infection.
The new cancer-specific scoring system consisted of six components: ECOG PS, SpO 2 , creatinine, total bilirubin, CRP, and lactate. These six components reflect the underlying condition of patients with cancer and acute responses to infection. The ECOG PS of these patients has been considered as an essential prognostic factor 22,23 . ECOG PS had the highest score distribution among the new scoring system variables in our study. Performance status is affected by many factors such as the patients' age, cancer stage, and side effects of anti-cancer treatment. Patients who have a poor PS and limited functional capacity tend to have more difficulty tolerating rigorous cancer treatments. These patients have less favorable outcomes than those with a better PS, regardless of distant metastasis or treatments given 24 . In this study, ECOG PS was a significant prognostic factor in cancer patients with suspected infection, whereas advanced stage or anti-cancer treatment were not. Lactate represents tissue hypoperfusion, and lactate > 2 mmol/L was introduced as diagnostic criteria of septic shock 25 . Furthermore, lactate has been shown to have prognostic power in cancer patients with sepsis 8 . Increased CRP is an indicator of inflammation in patients with sepsis. A recent study reported that CRP carries significant independent prognostic information 26 , which was consistent with the results of our study; elevated CRP was associated with increased 30-day mortality in cancer patients with suspected infection. Creatinine and total bilirubin are indicators of  www.nature.com/scientificreports/ hepatic and renal function, which are also components of SOFA 27 . SOFA included PaO 2 /FiO 2 as a respiratory indicator. We used SpO 2 in place of PaO 2 /FiO 2 given the ease to continuously and noninvasively obtain at the ER without drawing arterial blood. A previous study reported that SpO 2 was consistently associated with mortality in patients with septic shock 28 . This study has several limitations. First, it was conducted in a single hospital, which was a tertiary referral cancer center. There may have been a high proportion of severe disease. In our study, the 30-day mortality rate was 22.5%; however, in the US population report, cancer-related sepsis hospitalizations had high in-hospital mortality of 27.9% 1 . Second, as this study used a retrospective design, there were some limitations in data collection. However, we have a specialized area for cancer patients in ER, Cancer ER, ER for cancer patients 29 , and cancer patients are treated according to standardized protocols for their conditions and chief complaints. In this study, we tried to create a prognosis model with basic clinical variables included in the protocols for the treatment of patients with suspected infection. For this reason, we had very little missing data. Lastly, we included only solid tumors and excluded hematologic malignancies in this study. Therefore, it is difficult to apply the new scoring system to patients with hematologic malignancies. Many studies have shown that prognosis of patients with solid tumors and hematologic malignancies is quite different 30,31 . We consider that the prognostic model would have to be different to assess these two groups appropriately.
In conclusion, the new scoring system had a robust discriminative capability to predict prognosis in cancer patients with suspected infection. This new scoring system can be a good alternative for patients with solid tumor compared with existing scoring systems.

Conclusion
A new risk scoring system in active cancer patients with suspected infection consisted of six components: ECOG PS, SpO 2 , creatinine, total bilirubin, CRP, and lactate. The new scoring system was superior to the existing scoring systems of SIRS, qSOFA, and SOFA in predicting 30-day mortality. We believe that our system can help physicians at the ER to predict prognosis for cancer patients more accurately and inform treatment decisions.

Data availability
The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.