To investigate the value of artificial intelligence (AI) assisted quantification on initial chest CT for prediction of disease progression and clinical outcome in patients with coronavirus disease 2019 (COVID-19). Patients with confirmed COVID-19 infection and initially of non-severe type were retrospectively included. The initial CT scan on admission was used for imaging analysis. The presence of ground glass opacity (GGO), consolidation and other findings were visually evaluated. CT severity score was calculated according to the extent of lesion involvement. In addition, AI based quantification of GGO and consolidation volume were also performed. 123 patients (mean age: 64.43 ± 14.02; 62 males) were included. GGO + consolidation was more frequently revealed in progress-to-severe group whereas pure GGO was more likely to be found in non-severe group. Compared to non-severe group, patients in progress-to-severe group had larger GGO volume (167.33 ± 167.88 cm3 versus 101.12 ± 127 cm3, p = 0.013) as well as consolidation volume (40.85 ± 60.4 cm3 versus 6.63 ± 14.91 cm3, p < 0.001). Among imaging parameters, consolidation volume had the largest area under curve (AUC) in discriminating non-severe from progress-to-severe group (AUC = 0.796, p < 0.001) and patients with or without critical events (AUC = 0.754, p < 0.001). According to multivariate regression, consolidation volume and age were two strongest predictors for disease progression (hazard ratio: 1.053 and 1.071, p: 0.006 and 0.008) whereas age and diabetes were predictors for unfavorable outcome. Consolidation volume quantified on initial chest CT was the strongest predictor for disease severity progression and larger consolidation volume was associated with unfavorable clinical outcome.
Coronavirus disease 2019 (COVID-19) is a worldwide pandemic which firstly outbroke in Wuhan, China in December 2019 and spread to hundreds of nations since then1. Although a large proportion of patients with COVID-19 disease were mild cases, the rate of severe and critical cases was not low and the mortality could be as high as 21%2,3.
Chest computed tomography (CT) has been used as the first-line imaging modality for prompt diagnosis and monitoring disease course of COVID-19 pneumonia4. Compared to reverse transcription polymerase chain reaction (RT-PCR), chest CT has very high sensitivity to identify COVID-19 pneumonia5. The extent of involved pulmonary lesions was also found to be associated with unfavorable clinical outcome6.
In addition, artificial intelligence (AI) has emerged as a useful tool in terms of quantitative evaluation of pulmonary diseases, such as emphysema7. Compared to visual analysis, it allows more precise quantification of pulmonary lesions based on different attenuation thresholds. Moreover, according to recent studies, AI was found to be helpful for distinguishing COVID-19 from pneumonia of other origin through Chest CT with high diagnostic accuracy, and predict respiratory complications of COVID-198,9,10. However, whether AI-derived lesion quantification on chest CT was related to disease progression remained unclear. Therefore, the current study aims to investigate the value of AI assisted quantification for prediction of disease severity progression and clinical outcome, and compare it with conventional CT parameters evaluated by radiologist.
Material and methods
The institutional review board of Shanghai Jiao Tong University Affiliated Sixth People’s Hospital approved this retrospective study with a waiver of informed consent. All methods in the current study was carried out in accordance with relevant guidelines and regulations. From February 10th, 2020 to April 9th, 2020, hospitalized patients with confirmed COVID-19 infection (diagnosed by RT-PCR assay with throat swab) were retrospectively included. The RT-PCR tests were performed by using TaqMan One-Step RT-PCR Kits from Shanghai Huirui Biotechnology Co., Ltd. The diagnosis was made if any of the RT-PCR result was positive when multiple tests were performed. The exclusion criteria were: (1) the image quality of chest CT was significantly impaired so that AI based quantification was not feasible; (2) patients had superimposed infection of other pathogens; (3) patients were initially classified as severe or critical type on admission. The initial CT scan on admission was used for further analysis.
Chest CT protocol
All patients were scanned with a 128-slice multi-detector CT (Revolution Maxima, GE Medical Systems, Milwaukee, US) in supine position with inspiratory breath hold. The main acquisition parameters were listed as follow: tube voltage = 120 kVp, tube current was automatically modulated (150 mA–350 mA), slice thickness = 1.25 mm, slice interval = 1 mm, matrix = 512 * 512, field of view = 350 mm * 350 mm. Two datasets with different kernels were reconstructed for the image interpretation of lung (sharp kernel, Lung, GE Medical Systems) and mediastinum (smooth kernel, Stnd, GE Medical Systems).
Visual analysis of chest CT
All datasets were reconstructed with lung kernel and soft kernel for evaluation of pulmonary parenchyma and mediastinum. Lung window was set as a level of − 600HU and width of 1200HU whereas mediastinal window was set a level of 40HU and width of 350HU. The presence of following lesions were visually assessed: (1) ground glass opacity (GGO), which was defined as hazy increased attenuation with preserved margins of bronchus and vasculature11; (2) consolidation, which was defined as opacification with obscured margins of bronchus and vasculature; (3) reticulation; (4) nodules; (5) lymphadenopathy; (6) pleural effusion; (7) other abnormalities.
For semi-quantitative analysis, one previously reported CT score system was employed12. In brief, each lung was divided into three zones: upper (above the carina), middle (between carina and inferior pulmonary vein), and lower (below the inferior pulmonary vein) zones. Each lung zone was assigned a score that was based on the extent of involvement: score 0, 0% involvement; score 1, < 25% involvement; score 2, 25–50% involvement; score 3, 50–75% involvement; and score 4, ≥ 75% involvement. The summed score on per-patient level indicated overall lung involvement (maximal CT score for both lungs was 24)12.
Two chest radiologists (with 6-year and 14-year experience of chest imaging) independently evaluated all patients without the knowing of clinical characteristics and prognosis. Any disagreement between two observers was resolved by consensus.
AI assisted lesion quantification
Lesion quantification was performed using a commercially available software (LungDoc, version 1.19.1, ShuKun Network Technology, Beijing, China). In the recognition of pneumonia regions, the semantic segmentation technology based on deep learning was used to perform one-time segmentation extraction of the pneumonia regions in the input lung parenchyma. The network used a deep convolutional neural network based on residual structure UNet, of which details were given in online supplemental materials. The volumes of GGO and consolidation were automatically measured according to different attenuation thresholds (− 750HU ~ − 300HU for GGO, − 300HU ~ 50HU for consolidation)13. The percentages of GGO volume as well as consolidation volume versus whole lung volume were also recorded.
Two chest radiologists (with 6-year and 14-year experience of chest imaging) independently supervised the lesion quantification and manual adjustment of lesion regions was made when necessary. Any disagreement between two observers was resolved by consensus.
Disease severity and clinical outcome
Disease severity was evaluated according to the 6th edition of diagnosis and treatment protocols of pneumonia caused by novel coronavirus, issued by Chinese centers for disease control and prevention14. Patients with confirmed diagnosis of COVID-19 were classified into four types as described below: (1) mild, patients with mild symptoms and no imaging finding of pneumonia; (2) moderate, patients with fever, respiratory symptoms and imaging findings of pneumonia; (3) severe, patients met any of the following conditions (a. respiratory distress, respiratory rate ≥ 30 times/min; b. SpO2 < 93% at rest; c. PaO2/FiO2 ≤ 300 mmHg; d. rapid progression of disease involvement [more than 50%] on chest CT within 24 ~ 48 h ); (4) critical, patients met any of the following conditions (a. respiratory failure and need mechanical assistance; b. shock; c. “extra pulmonary” organ failure, d. intensive care unit [ICU] is needed). For further analysis, patients were grouped as “non-severe” (classified as mild or moderate type) and “progress-to-severe” (classified as severe or critical type) according to the most severe classification during hospitalization.
In addition, the mortality rate and rate of critical conditions (a. respiratory failure and need mechanical assistance; b. shock; c. “extra pulmonary” organ failure, d. ICU stay) were also recorded. Patients were considered event-free if they survived without occurrence of the above critical conditions.
Statistical analysis was performed with a commercially available statistical software (MedCalc Statistical Software version 15.2.2, MedCalc Software BVBA, Ostend, Belgium). Quantitative variables were expressed as mean ± standard deviation (SD). Student’s t test was used for normally distributed data, and the Mann–Whitney U test for non-normally distributed data. Categorical variables were reported as count (%), and compared by the Fisher exact test or chi-square test. Interobserver agreement of CT semi-quantitative score and AI assisted quantification was assessed by intraclass correlation coefficient (ICC). The Dice coefficient was used to evaluate the performance of the deep learning algorithms. Logistic regression model was used to calculate the area under the receiver operating characteristic (ROC) curve for different CT parameters. ROC analysis was performed by the method developed by Hanley and McNeil15. The area under curve (AUC) of each parameter was calculated and the optimal cutoff values for all parameters were determined by the Youden index, the maximum sum of sensitivity and specificity. To ascertain the associations of CT-derived semi-quantitative and quantitative parameters with disease severity progression and clinical outcome, we performed univariate and multivariate logistic regression analyses, which was performed by using the “stepwise” approach. The model included variables with p < 0.20 in the univariate analysis. A two-tailed P < 0.05 was considered statistically significant.
From February 10th, 2020 to April 9th, 2020, 148 patients with confirmed COVID-19 infection were retrospectively reviewed. Three patients were excluded due to significantly impaired image quality of chest CT whereas 4 patients were excluded because of having superimposed infection of other pathogens. Moreover, another 18 patients who were classified as severe or critical type were also ruled out (Fig. 1). Finally, 123 patients [mean age: 64.43 ± 14.02 (range 30–93) years, 62 males] were included in the further analysis. All patients were followed up at a mean time of 23.2 ± 11.7 days. For patients from progressed to severe group, the mean interval between admission and disease progression was 11.3 ± 3.4 days. Detailed clinical characteristics were given in Table 1.
Initial chest CT findings between non-severe and progress-to-severe group
During hospitalization, 39 patients progressed to severe group while 84 patients remained in non-severe group. Overall, pure GGO and GGO + consolidation were the two predominant findings of initial chest CT (Table 2). Other findings, such as reticulation, nodule, cavitation, were infrequently presented. As for subgroup analysis, GGO + consolidation was more frequently revealed in progress-to-severe group whereas pure GGO was more likely to be found in non-severe group (Table 2). Moreover, patients in progress-to-severe group had higher incidence of pleural effusion, nodule and larger number of involved lobes (Table 2).
In terms of CT semi-quantitative score and AI assisted quantification, both methods had excellent inter-observer agreement (ICC for CT score: 0.897, 95% CI 0.875–0.921; ICC for GGO quantification: 0.931, 95% CI 0.899–0.958; ICC for consolidation quantification: 0.972, 95% CI 0.951–0.998; all p < 0.001). The regions of lesion (pneumonia) were reliably segmented by AI assisted quantification, with a Dice coefficient of 0.845 (CI 0.751–0.944, p = 0.005). Compared to non-severe group, patients in progress-to-severe group had larger GGO volume as well as consolidation volume. The GGO volume percentage and consolidation volume percentage was also significantly higher in severe group (Table 2, Figs. 2 and 3).
Correlation of CT-derived parameters with disease severity progression and clinical outcome
ROC curve analysis and logistic regression was performed to determine the predictive value of CT-derived parameters for prediction of disease severity progression. As shown by ROC analysis, consolidation volume and consolidation volume percentage had the largest AUC in discriminating non-severe from progress-to-severe group (Fig. 4). AI assisted quantification outperformed CT semi-quantitative score for identifying patients progressing to severe group with higher diagnostic accuracy, sensitivity and specificity (Table 3). According to univariate analysis, consolidation volume and consolidation volume percentage were the strongest predictors for disease severity progression among all imaging parameters, followed by CT severity score and GGO volume percentage. After the adjustment by multivariate regression, consolidation volume and age remained as the strongest predictors (hazard ratio: 1.053 and 1.071, p: 0.006 and 0.008) (Table 4).
In terms of prognosis evaluation, 34 patients experienced the occurrence of at least one critical event. Similar to the above results, consolidation volume and consolidation volume percentage had the largest AUC in discriminating patients with and without critical events (Table 3). However, according to multivariate analysis, only age and diabetes were the significant predictors for unfavorable clinical outcomes (Table 4).
The main finding of the current study confirmed the value of AI assisted lesion quantification for prediction of disease severity progression. The consolidation volume on initial chest CT was the strongest predictor among all CT-derived parameters and larger consolidation volume was associated with unfavorable clinical outcome.
Chest CT is considered one pivotal diagnostic approach in the management of COVID-19 infection for detection of pulmonary involvement and serial follow-up of disease course16. In addition to visual diagnosis, latest technical development in the field of AI enables automatic lesion quantification using preset attenuation thresholds17. According to the current findings, this AI based quantification was not only beneficial for disease diagnosis but also for prognosis evaluation. Compared to conventional clinical characteristics, consolidation volume on initial chest CT was the strongest predictor for progress-to-severe group. Although GGO is the commonest finding on chest CT, consolidation has been reported to be more frequently presented in cases of severe and critical type18,19. Similar to other types of viral pneumonia, the underlying pathology of consolidation in the setting of COVID-19 could correlated to the complete filling of alveoli by inflammatory exudation20. When necrotizing bronchitis and diffuse alveolar damage occurs in the setting of viral pneumonia, the ventilation function is seriously compromised21. Therefore, it is conceivable that patients with larger area of consolidation on initial chest CT are more likely to progress to severe or critical clinical conditions.
Another important finding of the present study was the superiority of AI assisted quantification over conventional CT severity score for prediction of disease progression. The semi-quantitative CT score was first introduced in the evaluation of disease severity of severe acute respiratory syndrome12. However, this score does not take different lesion components, such as consolidation and GGO, into account. According to the current results, consolidation volume carried more significant predictive value that did GGO volume. Further, CT severity score roughly assesses the overall extent of disease according to involved zones. It is unable to distinguish mildly discrepant lesion severity when involvement difference is less than 25% within one zone. The latter one can only be evaluated by AI-assisted quantification. Thus, precise evaluation of absolute quantification based on different lesion characteristics is more helpful than CT semi-quantitative score in terms of risk stratification in patients with COVID-19. Similar findings can also be observed from one previous AI-based chest CT study, which found that percentage of consolidation volume and other CT quantified lesion parameters can early and non-invasively predict the progression to severe illness22.
In terms of the clinical outcome evaluation, consolidation volume had the highest diagnostic accuracy among all imaging parameters to identify patients who eventually experienced critical clinical events. This was in line with one previous semi-quantitative chest CT study that the severe extent of parenchymal involvement was associated with poor clinical outcome23. However, according to multivariate analysis, only age and diabetes remained as the significant predictor for poor prognosis in our study. This could be potentially ascribed to the limited number of total included patients. Future studies with larger sample size may help to determine the predictive value of consolidation volume for critical clinical events.
Despite of the above promising findings, the present study has several limitations. First, the current study did not employ radiomics analysis, which allows extraction of a large number of quantitative features from medical images for diagnosis and prognosis evaluation24,25. In addition, the sample size of the present study was not large enough to include more clinical characteristics and imaging features into multivariate analysis. Future studies with more included patients are warranted to validate the current results. Finally, the AI algorithm used in the current study was not able to identify other radiological findings, such as “crazy paving” and “bronchial wall thickening” in the analysis. It would be helpful to include those findings in future upgraded AI approach to further improve the performance of risk stratification.
In conclusion, consolidation volume quantified on initial chest CT was the strongest predictor for disease severity progression and larger consolidation volume was associated with unfavorable clinical outcome. AI-assisted lesion quantification was helpful for risk stratification and prognosis evaluation in patients with COVID-19.
Anonymous chest CT data will be available for review only upon reasonable request from the corresponding author. However, the data cannot be made public to maintain patients’ privacy and local legal reasons.
Coronavirus disease 2019
Ground glass opacity
Reverse transcription polymerase chain reaction
Li, Q. et al. Early transmission dynamics in Wuhan, China, of novel coronavirus-infected pneumonia. N. Engl. J. Med. 382, 1199–1207 (2020).
Guan, W. J. et al. Clinical characteristics of coronavirus disease 2019 in China. N. Engl. J. Med. 382, 1708–1720 (2020).
Richardson, S. et al. Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the New York city area. JAMA 323, 2052–2059 (2020).
Zu, Z. Y. et al. Coronavirus disease 2019 (COVID-19): A perspective from China. Radiology 296, E15–E25 (2020).
Ai, T. et al. Correlation of chest CT and RT-PCR testing in coronavirus disease 2019 (COVID-19) in China: A report of 1014 cases. Radiology 296, E32–E40 (2020).
Yang, R. et al. Chest CT severity score: An imaging tool for assessing severe COVID-19. Radiology 2, e200047 (2020).
Fischer, A. M. et al. Comparison of artificial intelligence-based fully automatic chest CT emphysema quantification to pulmonary function testing. AJR Am. J. Roentgenol. 214, 1065–1071 (2020).
Bai, H. X. et al. Augmentation of radiologist performance in distinguishing COVID-19 from pneumonia of other origin at chest CT. Radiology 296, E156–E165 (2020).
Abbasian Ardakani, A. et al. COVIDiag: A clinical CAD system to diagnose COVID-19 pneumonia based on CT findings. Eur. Radiol. https://doi.org/10.1007/s00330-020-07087-y (2020).
Burdick, H. et al. Prediction of respiratory decompensation in Covid-19 patients using machine learning: The READY trial. Comput. Biol. Med. https://doi.org/10.1016/j.compbiomed.2020.103949 (2020).
Hansell, D. M. et al. Fleischner society: Glossary of terms for thoracic imaging. Radiology 246, 697–722 (2008).
Ooi, G. C. et al. Severe acute respiratory syndrome: Temporal lung changes at thin-section CT in 30 patients. Radiology 230, 836–844 (2004).
Kauczor, H. U. et al. Automatic detection and quantification of ground-glass opacities on high-resolution CT using multiple neural networks: Comparison with a density mask. AJR Am. J. Roentgenol. 175, 1329–1334 (2000).
China NHC. Diagnosis and treatment protocols of pneumonia caused by novel coronavirus (trial version 6). 2020.
Hanley, J. A. & McNeil, B. J. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143, 29–36 (1982).
Shi, H. et al. Radiological findings from 81 patients with COVID-19 pneumonia in Wuhan, China: A descriptive study. Lancet Infect Dis. 20, 425–434 (2020).
Huang, L. et al. Serial quantitative chest CT assessment of COVID-19: Deep-learning approach. Radiology 2, e200075 (2020).
Li, K. et al. The clinical and chest CT features associated with severe and critical COVID-19 pneumonia. Invest. Radiol. 55, 327–331 (2020).
Lyu, P. et al. The performance of chest CT in evaluating the clinical severity of COVID-19 pneumonia: Identifying critical cases based on CT characteristics. Invest. Radiol. 55, 412–421 (2020).
Barton, L. M. et al. COVID-19 autopsies, Oklahoma, USA. Am. J. Clin. Pathol. 153, 725–733 (2020).
Koo, H. J. et al. Radiographic and CT features of viral pneumonia. Radiographics. 38, 719–739 (2018).
Liu, F. J. et al. CT quantification of pneumonia lesions in early days predicts progression to severe illness in a cohort of COVID-19 patients. Theranostics. 10, 5613–5622 (2020).
Meiler, S. et al. Can CT performed in the early disease phase predict outcome of patients with COVID 19 pneumonia? Analysis of a cohort of 64 patients from Germany. Eur. J. Radiol. https://doi.org/10.1016/j.ejrad.2020.109256 (2020).
Jia, T. Y. et al. Identifying EGFR mutations in lung adenocarcinoma by noninvasive imaging using radiomics features and random forest modeling. Eur. Radiol. 29, 4742–4750 (2019).
Thawani, R. et al. Radiomics and radiogenomics in lung cancer: A review for the clinician. Lung Cancer. 115, 34–41 (2018).
This study is supported by Medicine and Engineering Interdisciplinary Research Fund of Shanghai Jiao Tong University (Grant No.: YG2020YQ18).
The authors declare no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Li, Y., Shang, K., Bian, W. et al. Prediction of disease progression in patients with COVID-19 by artificial intelligence assisted lesion quantification. Sci Rep 10, 22083 (2020). https://doi.org/10.1038/s41598-020-79097-1
This article is cited by
Increased lactate dehydrogenase reflects the progression of COVID-19 pneumonia on chest computed tomography and predicts subsequent severe disease
Scientific Reports (2023)
Leukocyte glucose index as a novel biomarker for COVID-19 severity
Scientific Reports (2022)
A meta-analysis on the risk factors adjusted association between cardiovascular disease and COVID-19 severity
BMC Public Health (2021)
Quantitative evaluation of COVID-19 pneumonia severity by CT pneumonia analysis algorithm using deep learning technology and blood test results
Japanese Journal of Radiology (2021)
Clinical and radiological characteristics of COVID‑19 patients without comorbidities
Wiener klinische Wochenschrift (2021)
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.