Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Prediction and recommendation by machine learning through repetitive internal validation for hepatic veno-occlusive disease/sinusoidal obstruction syndrome and early death after allogeneic hematopoietic cell transplantation

Bone Marrow Transplantation volume 57, pages 538–546 (2022)Cite this article

Subjects

Abstract

Using traditional statistical methods, we previously analyzed the risk factors and treatment outcomes of veno-occlusive disease/sinusoidal obstruction syndrome (VOD/SOS) after allogeneic hematopoietic cell transplantation. Within the same cohort, we applied machine learning to create prediction and recommendation models. We analyzed 2572 transplants using eXtreme Gradient Boosting (XGBoost) to predict post-transplant VOD/SOS and early death. Using the XGBoost and SHapley Additive exPlanations (SHAP), we found influential factors and devised recommendation models, which were internally verified by repetitive ten-fold cross-validation. SHAP values suggested that gender, busulfan dosage, age, forced expiratory volume, and Disease Risk Index were significant factors for VOD/SOS. The areas under the receiver operating characteristic curves and the areas under the precision-recall curve of the models were 0.740, 0.144 for all VOD/SOS, 0.793, 0.793 for severe to very severe VOD/SOS, and 0.746, 0.304 for early death. According to our single feature recommendation, following the busulfan dosage was the most effective for preventing VOD/SOS. The recommendation method for six adjustable feature sets was also validated, and a subgroup corresponding to five to six features showed significant preventive power for VOD/SOS and early death. Our personalized treatment set recommendation showed reproducibility in repetitive internal validation, but large external cohorts should prospectively validate our model.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1
Fig. 2: Probability distributions of XGBoost prediction model outputs.
Fig. 3: The cumulative incidence curves in one validation for group stratification of the prediction models.
Fig. 4: The cumulative incidence curves of feature set recommendation analysis.

Similar content being viewed by others

References

  1. Kong SG, Jeong S, Lee S, Jeong J-Y, Kim DJ, Lee HS. Early transplantation-related mortality after allogeneic hematopoietic cell transplantation in patients with acute leukemia. BMC Cancer. 2021;21:177. https://doi.org/10.1186/s12885-021-07897-3.

  2. Mohty M, Malard F, Abecassis M, Aerts E, Alaskar AS, Aljurf M, et al. Revised diagnosis and severity criteria for sinusoidal obstruction syndrome/veno-occlusive disease in adult patients: a new classification from the European Society for Blood and Marrow Transplantation. Bone Marrow Transplant. 2016;51:906–12. https://doi.org/10.1038/bmt.2016.130.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Yoon JH, Yoo KH, Sung KW, Jung CW, Kim JS, Hahn SM, et al. Validation of treatment outcomes according to revised severity criteria from European Society for Blood and Marrow Transplantation (EBMT) for sinusoidal obstruction syndrome/veno-occlusive disease (SOS/VOD). Bone Marrow Transplant. 2019;54:1361–1368. https://doi.org/10.1038/s41409-019-0492-6.

    Article  PubMed  Google Scholar 

  4. Yoon JH, Min GJ, Park SS, Park S, Lee SE, Cho BS, et al. Incidence and risk factors of hepatic veno-occlusive disease/sinusoidal obstruction syndrome after allogeneic hematopoietic cell transplantation in adults with prophylactic ursodiol and intravenous heparin or prostaglandin E1. Bone Marrow Transplant. 2021;56:1063–13. https://doi.org/10.1038/s41409-021-01215-y.

  5. McDonald GB, Hinds MS, Fisher LD, Schoch HG, Wolford JL, Banaji M, et al. Veno-occlusive disease of the liver and multiorgan failure after bone marrow transplantation: a cohort study of 355 patients. Ann Intern Med. 1993;118:255–67.

    Article  CAS  PubMed  Google Scholar 

  6. Strouse C, Zhang Y, Zhang M-J, Digilio A, Pasquini M, Horowitz MM, et al. Risk score for the development of veno-occlusive disease after allogeneic hematopoietic cell transplant. Biol Blood Marrow Transplant. 2018;24:2072–80. https://doi.org/10.1016/j.bbmt.2018.06.013.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Richardson PG, Smith AR, Triplett BM, Kernan NA, Grupp SA, Antin JH, et al. Earlier defibrotide initiation post-diagnosis of veno-occlusive disease/sinusoidal obstruction syndrome improves Day +100 survival following haematopoietic stem cell transplantation. Br J Haematol. 2017;178:112–8. https://doi.org/10.1111/bjh.14727.

  8. Richardson PG, Riches ML, Kernan NA, Brochstein JA, Mineishi S, Termuhlen AM, et al. Phase 3 trial of defibrotide for the treatment of severe veno-occlusive disease and multi-organ failure. Blood. 2016;127:1656–65. https://doi.org/10.1182/blood-2015-10-676924.

  9. Armand P, Kim HT, Logan BR, Wang Z, Alyea EP, Kalaycio ME, et al. Validation and refinement of the Disease Risk Index for allogeneic stem cell transplantation. Blood. 2014;123:3664–71. https://doi.org/10.1182/blood-2014-01-552984.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Gupta V, Braun TM, Chowdhury M, Tewari M, Choi SW. A systematic review of machine learning techniques in hematopoietic stem cell transplantation (HSCT). Sensors. 2020;20:6100 https://doi.org/10.3390/s20216100.

    Article  PubMed Central  Google Scholar 

  11. Daoud M, Mayo M. A survey of neural network-based cancer prediction models from microarray data. Artif Intell Med. 2019;97:204–14. https://doi.org/10.1016/j.artmed.2019.01.006.

    Article  PubMed  Google Scholar 

  12. Senanayake S, White N, Graves N, Healy H, Baboolal K, Kularatna S. Machine learning in predicting graft failure following kidney transplantation: a systematic review of published predictive models. Int J Med Inform. 2019;130:103957 https://doi.org/10.1016/j.ijmedinf.2019.103957.

    Article  PubMed  Google Scholar 

  13. Shouval R, Labopin M, Bondi O, Mishan-Shamay H, Shimoni A, Ciceri F, et al. Prediction of allogeneic hematopoietic stem-cell transplantation mortality 100 days after transplantation using a machine learning algorithm: a European Group for Blood and Marrow Transplantation Acute Leukemia Working Party Retrospective Data Mining Study. J Clin Oncol. 2015;33:3144–51. https://doi.org/10.1200/jco.2014.59.1339.

    Article  PubMed  Google Scholar 

  14. Lee C, Haneuse S, Wang H-L, Rose S, Spellman SR, Verneris M, et al. Prediction of absolute risk of acute graft-versus-host disease following hematopoietic cell transplantation. PLoS One. 2018;13:e0190610 https://doi.org/10.1371/journal.pone.0190610.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Arai Y, Kondo T, Fuse K, Shibasaki Y, Masuko M, Sugita J, et al. Using a machine learning algorithm to predict acute graft-versus-host disease following allogeneic transplantation. Blood Adv. 2019;3:3626–34. https://doi.org/10.1182/bloodadvances.2019000934.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Gandelman JS, Byrne MT, Mistry AM, Polikowsky HG, Diggins KE, Chen H, et al. Machine learning reveals chronic graft-versus-host disease phenotypes and stratifies survival after stem cell transplant for hematologic malignancies. Haematologica. 2019;104:189–96. https://doi.org/10.3324/haematol.2018.193441.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. McDonald GB, Sharma P, Matthews DE, Shulman HM, Thomas ED. Venocclusive disease of the liver after bone marrow transplantation: diagnosis, incidence, and predisposing factors. Hepatology. 1984;4:116–22.

  18. Jones RJ, Lee KS, Beschorner WE, Vogel VG, Grochow LB, Braine HG, et al. Venoocclusive disease of the liver following bone marrow transplantation. Transplantation. 1987;44:778–83.

    Article  CAS  PubMed  Google Scholar 

  19. XGBoost: a scalable tree boosting system. 2016. ACM; 2016.

  20. Inoue T, Ichikawa D, Ueno T, Cheong M, Inoue T, Whetstone WD, et al. XGBoost, a machine learning method, predicts neurological recovery in patients with cervical spinal cord injury. Neurotrauma Rep. 2020;1:8–16. https://doi.org/10.1089/neur.2020.0009.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Budholiya K, Shrivastava SK, Sharma V. An optimized XGBoost based diagnostic system for effective prediction of heart disease. J King Saud Univ – Comput Inf Sci. (In press) 2020. https://doi.org/10.1016/j.jksuci.2020.10.013.

  22. Breiman L. Random forests. Mach Learn. 2001;45:5–32. https://doi.org/10.1023/a:1010933404324.

    Article  Google Scholar 

  23. Schapire RE. A brief introduction to boosting. In: Proceedings of the 16th international joint conference on Artificial intelligence – Volume 2. Stockholm, Sweden: Morgan Kaufmann Publishers Inc.; 1999. p. 1401–6.

  24. Stoltzfus JC. Logistic regression: a brief primer. Academic Emerg Med. 2011;18:1099–104. https://doi.org/10.1111/j.1553-2712.2011.01185.x.

    Article  Google Scholar 

  25. Lundberg SM, Erion G, Chen H, Degrave A, Prutkin JM, Nair B, et al. From local explanations to global understanding with explainable AI for trees. Nat Mach Intell. 2020;2:56–67. https://doi.org/10.1038/s42256-019-0138-9.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Breiman L. Statistical modeling: the two cultures (with comments and a rejoinder by the author). Stat Sci. 2001;16:199–231. https://doi.org/10.1214/ss/1009213726.

    Article  Google Scholar 

  27. Raschka S. Model evaluation, model selection, and algorithm selection in machine learning. arXiv. 2020:1811.12808v3.

  28. Rodriguez JD, Perez A, Lozano JA. Sensitivity analysis of k-fold cross validation in prediction error estimation. IEEE Trans Pattern Anal Mach Intell. 2010;32:569–75. https://doi.org/10.1109/tpami.2009.187.

    Article  PubMed  Google Scholar 

  29. Kohavi R. A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Proceedings of the 14th International Joint Conference on Artificial Intelligence – Volume 2. Montreal, Quebec, Canada: Morgan Kaufmann Publishers Inc.; 1995. p. 1137–43.

  30. Steyerberg EW, Harrell FE Jr, Borsboom GJJM, Eijkemans MJC, Vergouwe Y, Habbema JDF. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol. 2001;54:774–81. https://doi.org/10.1016/S0895-4356(01)00341-9.

    Article  CAS  PubMed  Google Scholar 

  31. Lundberg SM, Nair B, Vavilala MS, Horibe M, Eisses MJ, Adams T, et al. Explainable machine-learning predictions for the prevention of hypoxaemia during surgery. Nat Biomed Eng. 2018;2:749–60. https://doi.org/10.1038/s41551-018-0304-0.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Amann J, Blasimme A, Vayena E, Frey D, Madai VI. Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC Med Inform Decis Mak. 2020;20:310. https://doi.org/10.1186/s12911-020-01332-6.

  33. Singh A, Sengupta S, Lakshminarayanan V. Explainable deep learning models in medical image analysis. J Imaging. 2020;6:52. https://doi.org/10.3390/jimaging6060052.

  34. Chen J, Li K, Rong H, Bilal K, Yang N, Li K. A disease diagnosis and treatment recommendation system based on big data mining and cloud computing. Inf Sci. 2018;435:124–49. https://doi.org/10.1016/j.ins.2018.01.001.

    Article  Google Scholar 

  35. Sahoo AK, Pradhan C, Barik RK, Dubey H. DeepReco: deep learning based health recommender system using collaborative filtering. Computation. 2019;7:25 https://doi.org/10.3390/computation7020025.

    Article  Google Scholar 

  36. Katzman JL, Shaham U, Cloninger A, Bates J, Jiang T, Kluger Y. DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural network. BMC Med Res Methodol. 2018;18:24. https://doi.org/10.1186/s12874-018-0482-1.

  37. Leclerc V, Ducher M, Bleyzac N. Bayesian networks: a new approach to predict therapeutic range achievement of initial cyclosporine blood concentration after pediatric hematopoietic stem cell transplantation. Drugs R D. 2018;18:67–75. https://doi.org/10.1007/s40268-017-0223-7.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  38. Krakow EF, Hemmer M, Wang T, Logan B, Arora M, Spellman S, et al. Tools for the precision medicine era: how to develop highly personalized treatment recommendations from cohort and registry data using Q-learning. Am J Epidemiol. 2017;186:160–72. https://doi.org/10.1093/aje/kwx027.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

This study was supported by the Research Fund for Big Data Analysis of Seoul St. Mary’s Hospital, The Catholic University of Korea. It was permitted and conducted in accordance with the Institutional Review Board and Ethics Committee guidelines of the Catholic Medical Center (KC20RISI0132/199). HJH was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (NRF-2017R1E1A1A03070105, NRF-2019R1A5A1028324). The current study data are available from the corresponding author on reasonable request.

Author information

Author notes

  1. These authors contributed equally: Seungjoon Lee, Eunsaem Lee.

Authors and Affiliations

  1. AMSquare Corp., Pohang, Gyeongbuk, Korea

    Seungjoon Lee, Jaewoo Jung & Hyung Ju Hwang

  2. Department of Mathematics, Pohang University of Science and Technology (POSTECH), Pohang, Gyeongbuk, Korea

    Eunsaem Lee, Min Sue Park & Hyung Ju Hwang

  3. Department of Hematology, Catholic Hematology Hospital and Leukemia Research Institute, Seoul St. Mary’s Hospital, College of Medicine, The Catholic University of Korea, Seoul, Korea

    Sung-Soo Park, Gi June Min, Silvia Park, Sung-Eun Lee, Byung-Sik Cho, Ki-Seong Eom, Yoo-Jin Kim, Seok Lee, Hee-Je Kim, Chang-Ki Min, Seok-Goo Cho, Jong Wook Lee & Jae-Ho Yoon

Authors
  1. Seungjoon Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eunsaem Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sung-Soo Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Min Sue Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jaewoo Jung
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Gi June Min
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Silvia Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Sung-Eun Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Byung-Sik Cho
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Ki-Seong Eom
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Yoo-Jin Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Seok Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Hee-Je Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Chang-Ki Min
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Seok-Goo Cho
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Jong Wook Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Hyung Ju Hwang
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Jae-Ho Yoon
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

SL and EL analyzed data and created the models and wrote the manuscript; MSP and JJ analyzed data and supported creating the models; S-SP, GJM, SP, S-EL, B-SC, K-SE, Y-JK, SL, H-JK, C-KM, S-GC, and JWL provided patients and materials and reviewed the manuscript; HJH supervised all data management and model creation; J-HY designed and conducted the study, provided patients and materials, analyzed data, and wrote the manuscript.

Corresponding authors

Correspondence to Hyung Ju Hwang or Jae-Ho Yoon.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, S., Lee, E., Park, SS. et al. Prediction and recommendation by machine learning through repetitive internal validation for hepatic veno-occlusive disease/sinusoidal obstruction syndrome and early death after allogeneic hematopoietic cell transplantation. Bone Marrow Transplant 57, 538–546 (2022). https://doi.org/10.1038/s41409-022-01583-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41409-022-01583-z

This article is cited by

Search

Advanced search

Quick links