Abstract
Tumor recurrence affects up to 70% of early-stage hepatocellular carcinoma (HCC) patients, depending on treatment option. Deep learning algorithms allow in-depth exploration of imaging data to discover imaging features that may be predictive of recurrence. This study explored the use of convolutional neural networks (CNN) to predict HCC recurrence in patients with early-stage HCC from pre-treatment magnetic resonance (MR) images. This retrospective study included 120 patients with early-stage HCC. Pre-treatment MR images were fed into a machine learning pipeline (VGG16 and XGBoost) to predict recurrence within six different time frames (range 1–6 years). Model performance was evaluated with the area under the receiver operating characteristic curves (AUC–ROC). After prediction, the model’s clinical relevance was evaluated using Kaplan–Meier analysis with recurrence-free survival (RFS) as the endpoint. Of 120 patients, 44 had disease recurrence after therapy. Six different models performed with AUC values between 0.71 to 0.85. In Kaplan–Meier analysis, five of six models obtained statistical significance when predicting RFS (log-rank p < 0.05). Our proof-of-concept study indicates that deep learning algorithms can be utilized to predict early-stage HCC recurrence. Successful identification of high-risk recurrence candidates may help optimize follow-up imaging and improve long-term outcomes post-treatment.
Similar content being viewed by others
Introduction
Hepatocellular carcinoma (HCC) is the second most common cause of cancer-related death worldwide1,2. Although early-stage HCC patients are treated with curative intent, the recurrence rates remain high. Recurrence risk has been noted to be as high as 70% for patients treated with surgical resection3 or thermal ablation4 and about 20% with orthotopic liver transplantation (OLT)5 within 5 years post-treatment. Although patients with recurrence may undergo image-guided locoregional therapy or adjuvant systemic treatment with targeted agents such as sorafenib, supporting evidence for these approaches is limited6,7,8.
The diagnosis and staging of HCC are mainly based on cross-sectional body imaging, such as magnetic resonance (MR) imaging and computer tomography (CT). The size and number of tumors are the main criteria forming the foundation of various staging systems, for instance, the Barcelona Clinic Liver Cancer (BCLC) system9. But, the lesion number and size do not adequately characterize the tumor biology or its characteristic imaging appearance. Thus, these systems cannot accurately estimate the patient’s post-treatment recurrence risk.
Some recent studies have studied the value of machine learning algorithms to predict HCC recurrence10,11,12,13. So far, these studies have used a predefined catalog of manually hand-crafted features to predict HCC recurrence10,12,14. This approach has some drawbacks, since manually hand-crafted features may not capture cancer’s pathophysiological properties entirely. That means, they are created by humans and may be subject to selection bias. In contrast, deep learning algorithms can automatically extract the most relevant and predictive imaging features, without human engineering. Deep learning algorithms can also be utilized in combination with medical imaging data, to support a wide range of applications, including automated segmentation, lesion classification, and abnormality detection15. Its hallmark lies in the advanced ability to deal with unstructured data, such as medical images16, and to automatically extract relevant imaging features that may not be apparent to even the most experienced human eyes17.
Even though there are already studies on HCC recurrence prediction utilizing deep learning algorithms, most of these studies use CT imaging as input18. In contrast, we ask whether using MR imaging instead of CT is more promising due to the higher soft tissue contrast of MR imaging compared to CT. To the best of our knowledge, there is no study utilizing deep learning for HCC recurrence prediction in combination with MR imaging for early-stage HCC.
This study aims to build, validate, and test a deep learning approach to predict the post-treatment (surgical, ablative, or OLT) recurrence risk in early-stage HCC patients from pre-treatment MR imaging.
Methods
Patient population
This was a single-center retrospective study compliant with the Health Insurance Portability and Accountability Act (HIPAA). This study was approved by the institutional review board of Yale University and informed consent was waived. The study was performed in accordance with the Declaration of Helsinki. One hundred and twenty adults (≥ 18 years) from 2005 to 2018 were included. From a total of 1190 patients treated from January 2005 to December 2018 for HCC at our institution, 120 patients were included according to the following criteria: (1) either histopathologically or imaging-criteria confirmed HCC, (2) either stage 0 or stage A, according to the most recent BCLC staging system19, (3) surgical resection, thermal ablation, or OLT as first-line, stand-alone treatment after initial listing/consideration for transplant in all three scenarios, (4) available multi-parametric contrast-enhanced pre-treatment MR imaging, and (5) complete response after treatment, defined as the disappearance of all signs of viable tumor at first post-interventional follow-up imaging. Exclusion criteria were as follows: (1) imaging confirmed macrovascular invasion or metastatic HCC, (2) presence of active co-malignancies from the time of HCC diagnosis through the entire follow-up period, and (3) excessive motion artifact on pre-treatment MR imaging. Pre-treatment clinical variables were collected. Patient data were extracted from the Electronic Health Records. Further information regarding patient characteristics can be found in Table 1.
To ensure all patients were diagnosed according to current imaging diagnostic criteria, all MR images were read by three board-certified radiologists (two of them with 7, and one of them with 8 years of experience in body MR Imaging, respectively) according to the most recent Liver Reporting and Data Systems (LI-RADS v2018)20 criteria. The dataset was split into three portions. Each radiologist read a portion in independent reads. Patients with at least one LI-RADS (LR)-5 lesion were considered diagnosed with HCC. For patients not meeting LR-5 imaging criteria, biopsy or postoperative pathological confirmation of HCC was considered sufficient for diagnosis.
Clinical endpoint
Recurrence was defined as the intra- or extrahepatic appearance of HCC after treatment, confirmed by multi-parametric contrast-enhanced imaging or histology. Recurrence included intrahepatic local (within the liver, at the place of the original tumor), intrahepatic distant (within the liver, distant from the original tumor), and extrahepatic (outside the liver) recurrence. Recurrence-free survival (RFS) was defined as the time interval between curative treatment to first imaging evidence of recurrence. Multi-parametric contrast-enhanced MR imaging or computer tomography (CT) of the abdomen was used for follow-up imaging, including pre-contrast, arterial, portal venous, and delayed phase imaging. Patients received imaging every 3 months in the first post-procedure year and every 6 months after that. To monitor for extrahepatic recurrence, patients received a non-contrast chest CT every 6 months.
Imaging acquisition
Acquisitions from multiple MR imaging scanners with different contrast agents were used to allow for a more robust and generalizable machine learning model. Scanners were 1.5 T and 3 T models produced by Siemens (Aera, Avanto, Espree, TrioTim, Skyra, Verio), General Electric (SIGNA HDx), and Philips (Achieva). Studies were performed with several contrast agents, including Gadavist (Bayer), Magnevist (Bayer), Eovist (Bayer), Multihance (Bracco Diagnostics), Prohance (Bracco Diagnostics), Optimark (Covidien), and Dotarem (Guerbet). Contrast agents were administered at a dose of 1–1.5 mmol/kg with an injection speed of 2–5 mL/s. The input for our model consisted of multi-parametric, gadolinium-enhanced T1-weighted gradient echo acquisitions including arterial phase (35 s post-injection), portal venous phase (70 s post-injection), and delayed phase (3–5 min post-injection) imaging. Repetition and echo times ranged from 2.5–5.5 ms and 1–3 ms, respectively. Pixel bandwidth ranged from 250 to 650 Hz, pixel spacing from 0.5 to 1.7 mm, slice thickness from 3 to 5 mm, number of slices from 43 to 125, and image matrices from 160 × 160 to 415 × 200.
Machine learning pipeline
Image pre-processing
A rectangular, three-dimensional bounding box was placed around the liver in the pre-treatment, multi-parametric MR axial images using 3D Slicer (Version 4.10.2)21. The task was performed by a third-year radiology resident, supervised by board-certified subspecialty-trained readers with 7 and 8 years of experience, as detailed above. The whole liver was chosen as the region of interest to capture most of the potentially predictive imaging features. The bounding box was created based on the arterial phase and then adapted to the portal venous and delayed phase images. An exemplary representation of a bounding box can be found in Supplementary Fig. 2. After that, corresponding slices of all three phases were resized to 224 × 224 pixels and combined into one stack. The resizing steps were performed to create a consistent input matrix size of 224 × 224 × 3 pixels required for feature extraction.
Feature extraction
Feature extraction and voxel intensity normalization were performed using VGG1622. This convolutional neural network (CNN) was pre-trained on a subset of the ImageNet database. The original pre-trained weights were used for feature extraction without fine-tuning the neural network. It was chosen due to its robust performance in various domains23, and it is an effective method when only a limited amount of imaging data is available. The voxel intensities were normalized using z-score normalization, as per given the formulae: xnʹ = (xn − µ)/σ, where xn is the original voxel intensity value, µ the mean of all voxel intensity values per patient, σ the standard deviation of all voxel intensity values per patient, and xnʹ the normalized voxel intensity value. In total, 4096 features per stack were extracted from the 2nd fully connected layer of the CNN. Finally, the maximum value per feature across all MR image stacks was pooled to obtain a single feature vector of 4096 features per patient. A more detailed overview of the pre-processing and feature extraction can be found in Supplementary Fig. 1.
Recurrence prediction
The final imaging feature vector was fed into XGBoost24, a state-of-the-art tree-based ensemble classifier. To validate the algorithm, nested cross-validation (CV)25 was performed, consisting of an outer- and an inner loop. In the outer loop, leave-one-out CV was used to evaluate the model performance, meaning that all patients but one were used to train, and the remaining one was used to validate the model. In the inner loop, tenfold stratified Monte Carlo CV was used to optimize the model hyperparameters. For this, patients were randomly assigned to training (90%) and validation (10%) sets over six iterations. Figure 1 depicts an overview of our machine learning workflow.
Outcome endpoint
The Kaplan–Meier method was used to investigate our machine learning pipeline’s clinical relevance and efficiency, with RFS as the outcome of interest. Patients were stratified by the predictions of the models into “high-risk” and “low-risk” recurrence. As the machine learning model only provides a recurrence probability, a specific threshold had to be chosen, against which the patients are stratified. For each time frame, the threshold was selected by dividing the number of all recurrent by the number of all cases. Patients who recurred during the analyzed time frame were marked as events, whereas patients who remained cancer-free were censored. Survival curves were compared using the log-rank test. A p-value below 0.05 was considered significant.
Statistical analysis
The median and interquartile ranges were used to summarize patient characteristics. Recurrent and non-recurrent patients were compared using the Chi-square test (for categorical and ordinal variables) and the Mann–Whitney U test (for continuous variables). Bonferroni correction was used to adjust the significance level for the family-wise error rate in the baseline characteristics. The area under the receiver operating characteristic curve (AUC–ROC) was used to evaluate the machine learning model’s performance. All statistical analysis was performed in Python (Version 2.7.3).
Results
Patients characteristics
Table 1 summarizes the patients’ baseline characteristics. Among 120 patients diagnosed with HCC, 88 (73.3%) were male. The underlying liver disease was distributed as follows, with HCV being the most prevalent cause (n = 61, 50.8). While non-alcoholic steatohepatitis (NASH) (n = 14, 11.7%), alcoholic liver disease (n = 11, 9.2%), HBV (n = 5, 4.2%), multiple etiologies (n = 14, 11.6%) and other pathologies (n = 15, 12.5%) were the other half. The majority of patients (n = 102, 85%) had one imaging-confirmed lesion at the time of diagnosis, followed by two (n = 15, 12.5%), and three (n = 3, 2.5%) lesions. The median (IQR) Meld-Na score without HCC exception points was 9 (5).
Among all patients, treatment modalities included surgical resection (n = 32, 26.7%), thermal ablation (n = 29, 24.2%), and OLT (n = 59, 49.1%). Disease recurrence was present in 45 (37.5%) patients and occurred more frequently in resected (n = 18, 56.3%) or ablated (n = 19, 65.5%) patients than in patients treated with OLT (n = 8, 13.6%). The mean time to recurrence was 26.8 months. Recurrent HCC appeared as intrahepatic local recurrence at the margin of the ablated/resected area (n = 10, 22.7%), intrahepatic distant recurrence remote from the original tumor site (n = 22, 50%), extrahepatic recurrence (n = 5, 11.4%) as well as at multiple locations (n = 7, 15.9%). The median (IQR) time from treatment to the first follow-up imaging was 166 days (326). There was no statistically significant difference in the clinical characteristics between recurrence and no-recurrence patients.
All MR images were re-examined according to the most recent LI-RADS criteria. In total, 93 patients (77.5%) were confirmed as being HCC-diagnosed with at least one LR-5 lesion on the pre-treatment MR imaging. The remaining 27 (22.5%) patients presented at most with one LR-4 (n = 17, 14.2%), LR-3 (n = 7, 5.8%), or LR-M (n = 3, 2.5%) lesion. Among these subjects, HCC was confirmed in 17 (14.2%) patients with liver biopsy and 10 (8.3%) patients with postoperative pathology. Table 2 summarizes the results of the LI-RADS reading.
Prediction of recurrence
Table 3 depicts the predictive performance for all models. Six machine learning models were built to predict each patient’s recurrence probability within six different time frames. With the increasing length of the investigated time frame, the number of patients included in each model decreased, reducing the number of patients included from 120 in the 1-year model to 66 in the 6-year model.
The test-AUC classification performance across all models ranged between (0.71–0.85). The variation in performance across all models was low, apparent by the standard deviation of their respective average test-AUC (AUC = 0.76, SD = 0.05). The test-AUC of each model is formed from one sample of predictions; hence standard deviations for these values are not provided.
Recurrence-free survival
Figure 2 shows the Kaplan–Meier plots for RFS. To investigate the algorithm’s clinical relevance the final recurrence predictions were analyzed with the Kaplan–Meier method. According to the model’s predictions, patients were stratified into “high-risk”- and “low-risk”-recurrence groups. The number of patients in each analysis corresponds to the number of patients in the corresponding recurrence prediction models, namely 120 (12), 116 (26), 98 (36), 82 (40), 74 (43), and 66 (44) for the 1–6 years models, respectively.
The analysis revealed a significant difference between predicted low- and high-risk patients in five of the six different recurrence time frames, namely for 2-, 3-, 4-, 5-, and 6-year recurrence time points. (log-rank test, p < 0.05). No statistical difference was obtained for the 1-year analysis (p = 0.06). A Kaplan–Meier analysis for each time frame and treatment can be found in Supplementary Fig. 3.
Discussion
This study aimed to build, validate and test a deep learning approach to predict the post-treatment early-stage HCC recurrence risk from pre-treatment MR imaging. For all pre-defined time frames (ranging from 1 to 6 years), our model was able to predict tumor recurrence using baseline imaging data with moderate to high accuracy and test AUCs between 0.71 to 0.85. The subsequent Kaplan–Meier analysis showed that such predictions allow patients to be stratified into risk groups with significant differences in expected RFS at five recurrence time frames, with greater levels of certainty for the later time points. Overall, this study demonstrates that machine learning algorithms, specifically CNNs in combination with decision trees, can detect MR imaging features of HCC and background liver parenchyma associated with tumor recurrence in treated early-stage disease.
The relatively high prevalence of early disease recurrence following curative or potentially curative therapies of HCC continues to impede HCC, with 5-year recurrence rates being as high as 80%3,26,27,28. Among other unmet needs, current predominantly non-functional and lesion-size-based imaging criteria have proven to be of limited value when differentiating between patients with high-risk and low-risk for recurrence. There is scientific consensus that novel biomarkers for improved prediction of outcome are an unmet need with the hope to develop improved liver transplantation criteria that would incorporate factors beyond basic tumor morphology on cross-sectional imaging (number and size)29. Along those lines, our study was designed to apply currently available deep learning techniques to this unmet need. Our data demonstrated that this neural network generated reliable outcome prediction based on disease morphology on baseline imaging with a performance similar to or better than previously reported in related works12,13,30,31,32,33. While previous studies mainly used manually hand-crafted features based on human expert knowledge, our approach relies on automated feature extraction using a CNN. Overall, the clinical applicability of deep learning-based prediction of recurrences using baseline MR imaging will remain the subject of further study.
A direct comparison with prior works on HCC recurrence prediction is limited by the fact that most of the published studies use CT34 instead of MR imaging which is far less available for staging or screening purposes, particularly in Asia. Lv et al.35 proposed a deep-learning-based radiomics model to predict the 3-year recurrence of resected HCC from CT imaging, with a testing AUC of 0.83, notably higher than ours (0.71). Wang et al.18 presented a comparable study design to predict early HCC recurrence with an AUC of 0.825. Nevertheless, it remains difficult to deduct which modality is preferable given the use of different machine learning approaches, inclusion criteria, and number of patients.
A major technical advantage of the presented work is the use of the whole liver volume for algorithmic input rather than relying on segmented regions of the tumor or single slices. In comparison, Kim et al.10 attempted to predict early recurrences of HCC using quantitative imaging features, with the best results being obtained using a model that included the peritumoral environment (3 mm) rather than the tumor alone, impressively underlining the need to look beyond the malignancy itself. Additionally, Ji et al.36 demonstrated that in patients with cirrhosis, texture-specific imaging features of the liver background could be used to predict HCC recurrence, confirming that the underlying hepatic microenvironment as a whole may be reflective of the risk for recurrence. As HCC can be considered a secondary disease arising on top of a cirrhotic background, recurrent cancer may not only develop at the treated site but also de-novo. Therefore, whole liver input may overall be useful to account for underlying parenchymal conditions as an additional source of disease recurrence.
In the study, we chose the investigated recurrence period to be up to 6 years, as most post-operative HCC recurrences occur within the first 5–6 years. Our Kaplan–Meier analysis showed a statistically significant difference between low- and high-risk patients in five different time-to-recurrence (TTR) time frames with AUCs between 0.75 and 0.85. But our model was not statistically significant within 1 year, which is likely due to the small number of recurrent cases (n = 12) and the large data imbalance between recurrent and nonrecurrent cases (12 vs. 108).
Our study has some limitations. First, this is a retrospective single-center study with a relatively small cohort. The lack of external validation can be a source of bias and external algorithmic validation would be necessary for larger datasets before clinical deployment. Therefore, in light of the relatively small cohort and to prevent overfitting the data, we did not further pursue an independent test set at this point (e.g. 20% of the patients). However, the use of nested CV gives some confidence in the robustness of the learned models. As demonstrated in Table 1, we included nine HCC patients with end-stage liver cirrhosis (Child–Pugh-class C). According to former BCLC versions, these patients would haven been classified as terminal stage (BCLC D) disease patients and therefore no longer meet the inclusion criteria of our study. However, the most recent updates of the BCLC staging system19 allows for patients to be classified as early-stage and therefore be considered for transplant if their tumor burden meets transplant criteria. In our study, all nine patients were treated with OLT and therefore met the BCLC 2022 guidelines. In observance of the new BCLC 2022 algorithm, one patient in our cohort is formally staged as BCLC B but was downstaged to early-stage and ultimately met transplant criteria. Last, the LI-RADS reading was done as a split, with each of the radiologists reviewing a subset of the cases. A design where each radiologist reviewed all cases would have been preferable.
As we further our research and its application we intend to include various clinical features and modify the deep neural network to investigate whether there is an improvement in prediction accuracy. Some clinical parameters are already known to be associated with the recurrence of HCC, including gender, age, AFP37, NLR, or PLR38. We are hopeful that Ensemble methods that combine several predictors may potentially yield higher accuracy and add to the algorithm’s utility.
In conclusion, our work serves as proof of principle that deep learning-based algorithms can predict recurrence from pre-treatment MR imaging in patients with early-stage HCC. This study suggests that deep learning-based extraction of imaging features from baseline diagnostic imaging could be used to individualize treatment options with prognostications and adjust post-treatment follow-up in the sense of precision medicine. Future work should focus on enhancing the reliability of these algorithms with multi-center prospective cohort studies and the incorporation of clinical data.
Data availability
Data can be made available upon reasonable request to the corresponding author.
References
El-Serag, H. B. & Rudolph, K. L. Hepatocellular carcinoma: Epidemiology and molecular carcinogenesis. Gastroenterology 132, 2557–2576. https://doi.org/10.1053/j.gastro.2007.04.061 (2007).
GBD 2015 Mortality and Causes of Death Collaborators. Global, regional, and national life expectancy, all-cause mortality, and cause-specific mortality for 249 causes of death, 1980–2015: A systematic analysis for the Global Burden of Disease Study 2015. Lancet 388, 1459–1544. https://doi.org/10.1016/S0140-6736(16)31012-1 (2016).
Imamura, H. et al. Risk factors contributing to early and late phase intrahepatic recurrence of hepatocellular carcinoma after hepatectomy. J. Hepatol. 38, 200–207. https://doi.org/10.1016/s0168-8278(02)00360-4 (2003).
Doyle, A. et al. Outcomes of radiofrequency ablation as first-line therapy for hepatocellular carcinoma less than 3 cm in potentially transplantable patients. J. Hepatol. 70, 866–873. https://doi.org/10.1016/j.jhep.2018.12.027 (2019).
Yao, F. Y. et al. Liver transplantation for hepatocellular carcinoma: Expansion of the tumor size limits does not adversely impact survival. Hepatology 33, 1394–1403. https://doi.org/10.1053/jhep.2001.24563 (2001).
Ko, H. K., Ko, G. Y., Yoon, H. K. & Sung, K. B. Tumor response to transcatheter arterial chemoembolization in recurrent hepatocellular carcinoma after living donor liver transplantation. Korean J. Radiol. 8, 320–327. https://doi.org/10.3348/kjr.2007.8.4.320 (2007).
Zhai, H. et al. Microwave ablation in treating intrahepatic recurrence of hepatocellular carcinoma after liver transplantation: An analysis of 11 cases. Int. J. Hyperth. 31, 863–868. https://doi.org/10.3109/02656736.2015.1091953 (2015).
Mancuso, A. et al. Survival of patients treated with sorafenib for hepatocellular carcinoma recurrence after liver transplantation: A systematic review and meta-analysis. Dig. Liver Dis. 47, 324–330. https://doi.org/10.1016/j.dld.2015.01.001 (2015).
Forner, A., Reig, M. & Bruix, J. Hepatocellular carcinoma. Lancet 391, 1301–1314. https://doi.org/10.1016/S0140-6736(18)30010-2 (2018).
Kim, S. et al. Radiomics on gadoxetic acid-enhanced magnetic resonance imaging for prediction of postoperative early and late recurrence of single hepatocellular carcinoma. Clin. Cancer Res. 25, 3847–3855. https://doi.org/10.1158/1078-0432.ccr-18-2861 (2019).
Hui, T. C. H., Chuah, T. K., Low, H. M. & Tan, C. H. Predicting early recurrence of hepatocellular carcinoma with texture analysis of preoperative MRI: A radiomics study. Clin. Radiol. 73(1056), e1011-1056.e1016. https://doi.org/10.1016/j.crad.2018.07.109 (2018).
Shan, Q. Y. et al. CT-based peritumoral radiomics signatures to predict early recurrence in hepatocellular carcinoma after curative tumor resection or ablation. Cancer Imaging 19, 11. https://doi.org/10.1186/s40644-019-0197-5 (2019).
Zhang, Z. et al. Hepatocellular carcinoma: Radiomics nomogram on gadoxetic acid-enhanced MR imaging for early postoperative recurrence prediction. Cancer Imaging 19, 22. https://doi.org/10.1186/s40644-019-0209-5 (2019).
Xu, X. et al. Radiomic analysis of contrast-enhanced CT predicts microvascular invasion and outcome in hepatocellular carcinoma. J. Hepatol. 70, 1133–1144. https://doi.org/10.1016/j.jhep.2019.02.023 (2019).
Soffer, S. et al. Convolutional neural networks for radiologic images: A radiologist’s guide. Radiology 290, 590–606. https://doi.org/10.1148/radiol.2018180547 (2019).
Chartrand, G. et al. Deep learning: A primer for radiologists. Radiographics 37, 2113–2131. https://doi.org/10.1148/rg.2017170077 (2017).
LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444. https://doi.org/10.1038/nature14539 (2015).
Wang, W. et al. Deep learning-based radiomics models for early recurrence prediction of hepatocellular carcinoma with multi-phase CT images and clinical data. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. 2019, 4881–4884. https://doi.org/10.1109/EMBC.2019.8856356 (2019).
Reig, M. et al. BCLC strategy for prognosis prediction and treatment recommendation Barcelona Clinic Liver Cancer (BCLC) staging system. The 2022 update. J. Hepatol. https://doi.org/10.1016/j.jhep.2021.11.018 (2021).
Chernyak, V. et al. Liver Imaging Reporting and Data System (LI-RADS) version 2018: Imaging of hepatocellular carcinoma in at-risk patients. Radiology 289, 816–830. https://doi.org/10.1148/radiol.2018181494 (2018).
Fedorov, A. et al. 3D slicer as an image computing platform for the quantitative imaging network. Magn. Reson. Imaging 30, 1323–1341. https://doi.org/10.1016/j.mri.2012.05.001 (2012).
Simonyan, K. & Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv e-prints, arXiv:1409.1556 (2014).
Russakovsky, O. et al. ImageNet large scale visual recognition challenge. arXiv e-prints, arXiv:1409.0575 (2014).
Chen, T. & Guestrin, C. XGBoost: A scalable tree boosting system. arXiv e-prints, arXiv:1603.02754 (2016).
Varma, S. & Simon, R. Bias in error estimation when using cross-validation for model selection. BMC Bioinform. 7, 91. https://doi.org/10.1186/1471-2105-7-91 (2006).
Sapisochin, G. et al. Benefit of treating hepatocellular carcinoma recurrence after liver transplantation and analysis of prognostic factors for survival in a large Euro-American series. Ann. Surg. Oncol. 22, 2286–2294. https://doi.org/10.1245/s10434-014-4273-6 (2015).
Bodzin, A. S. et al. Predicting mortality in patients developing recurrent hepatocellular carcinoma after liver transplantation: Impact of treatment modality and recurrence characteristics. Ann. Surg. 266, 118–125. https://doi.org/10.1097/SLA.0000000000001894 (2017).
Hasegawa, K. et al. Comparison of resection and ablation for hepatocellular carcinoma: A cohort study based on a Japanese nationwide survey. J. Hepatol. 58, 724–729. https://doi.org/10.1016/j.jhep.2012.11.009 (2013).
Batra, R. K. Utility of liver transplantation within the bounds of non-futility. Curr. Transplant. Rep. 7, 187–193. https://doi.org/10.1007/s40472-020-00288-w (2020).
Ji, G. W. et al. Radiomic features at contrast-enhanced CT predict recurrence in early stage hepatocellular carcinoma: A multi-institutional study. Radiology 294, 568–579. https://doi.org/10.1148/radiol.2020191470 (2020).
Guo, D. et al. Radiomics analysis enables recurrence prediction for hepatocellular carcinoma after liver transplantation. Eur. J. Radiol. 117, 33–40. https://doi.org/10.1016/j.ejrad.2019.05.010 (2019).
Ning, P. et al. Application of CT radiomics in prediction of early recurrence in hepatocellular carcinoma. Abdom. Radiol. (NY) 45, 64–72. https://doi.org/10.1007/s00261-019-02198-7 (2020).
Yuan, C. et al. Prediction early recurrence of hepatocellular carcinoma eligible for curative ablation using a Radiomics nomogram. Cancer Imaging 19, 21. https://doi.org/10.1186/s40644-019-0207-7 (2019).
Zhu, H. B. et al. Radiomics-based nomogram using CT imaging for noninvasive preoperative prediction of early recurrence in patients with hepatocellular carcinoma. Diagn. Interv. Radiol. 26, 411–419. https://doi.org/10.5152/dir.2020.19623 (2020).
Lv, C. et al. Prediction of 3-year recurrence rate of hepatocellular carcinoma after resection based on contrast-enhanced CT: A single-centre study. Br. J. Radiol. 96, 20220702. https://doi.org/10.1259/bjr.20220702 (2023).
Ji, G. W. et al. Machine-learning analysis of contrast-enhanced CT radiomics predicts recurrence of hepatocellular carcinoma after resection: A multi-institutional study. EBioMedicine 50, 156–165. https://doi.org/10.1016/j.ebiom.2019.10.057 (2019).
Tsilimigras, D. I. et al. Synergistic impact of alpha-fetoprotein and tumor burden on long-term outcomes following curative-intent resection of hepatocellular carcinoma. Cancers (Basel). https://doi.org/10.3390/cancers13040747 (2021).
Zheng, J. et al. Neutrophil to lymphocyte ratio and platelet to lymphocyte ratio as prognostic predictors for hepatocellular carcinoma patients with various treatments: A meta-analysis and systematic review. Cell Physiol. Biochem. 44, 967–981. https://doi.org/10.1159/000485396 (2017).
Funding
This work was supported by the NIH/NCI [R01CA206180].
Author information
Authors and Affiliations
Contributions
A.K.; collected the clinical and imaging data, designed study, wrote paper. T.Z.; designed study, co-wrote paper. N.C.; co-wrote paper. R.R; collected clinical data, annotated data. J.S., A.P., M.E., S.H.; collected clinical data, annotated data. M.L.; helped with data collection. J.O.; supervised the machine learning, proof-read paper. M.N., K.C., E.T.; proof-read imaging data. B.G., J.C.; supervised the research, proof-read paper. L.S.; supervised the statistical analysis, proof-read paper. R.B., D.M.; supervised the patient collection, proof-read paper.
Corresponding author
Ethics declarations
Competing interests
MingDe Lin receives a salary from Visage Imaging, Inc. Other than that, the authors of the study declare no conflict of interest.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Kucukkaya, A.S., Zeevi, T., Chai, N.X. et al. Predicting tumor recurrence on baseline MR imaging in patients with early-stage hepatocellular carcinoma using deep machine learning. Sci Rep 13, 7579 (2023). https://doi.org/10.1038/s41598-023-34439-7
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-023-34439-7
This article is cited by
-
Latest advances in hepatocellular carcinoma management and prevention through advanced technologies
Egyptian Liver Journal (2024)
-
Deep Learning-Based Prediction of Post-treatment Survival in Hepatocellular Carcinoma Patients Using Pre-treatment CT Images and Clinical Data
Journal of Imaging Informatics in Medicine (2024)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.