Explainable machine-learning predictions for the prevention of hypoxaemia during surgery


Although anaesthesiologists strive to avoid hypoxaemia during surgery, reliably predicting future intraoperative hypoxaemia is not possible at present. Here, we report the development and testing of a machine-learning-based system that predicts the risk of hypoxaemia and provides explanations of the risk factors in real time during general anaesthesia. The system, which was trained on minute-by-minute data from the electronic medical records of over 50,000 surgeries, improved the performance of anaesthesiologists by providing interpretable hypoxaemia risks and contributing factors. The explanations for the predictions are broadly consistent with the literature and with prior knowledge from anaesthesiologists. Our results suggest that if anaesthesiologists currently anticipate 15% of hypoxaemia events, with the assistance of this system they could anticipate 30%, a large portion of which may benefit from early intervention because they are associated with modifiable factors. The system can help improve the clinical understanding of hypoxaemia risk during anaesthesia care by providing general insights into the exact changes in risk induced by certain characteristics of the patient or procedure.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.


All prices are NET prices.

Fig. 1: Prescience integrates many data sources into a single risk, which is explained through a succinct visual summary.
Fig. 2: Patient and procedure characteristics.
Fig. 3: Pooled comparison of the prediction performance of five anaesthesiologists with and without the assistance of Prescience.
Fig. 4: Sample real-time prediction during a procedure.
Fig. 5: Comparison of averaged feature-importance estimates between Prescience and anaesthesiologists for both initial and real-time prediction.
Fig. 6: Effect of varying individual feature values for both preoperative and real-time features.

Data availability

Owing to patient-privacy considerations, the operating-room datasets from participating hospitals are not publicly available. The raw data from the anaesthesiologist comparisons in Fig. 3 are available in Supplementary Tables 6 and 7, and data from Fig. 5 are available in Supplementary Tables 8 and 9.


  1. 1.

    Weiser, T. G. et al. Estimate of the global volume of surgery in 2012: an assessment supporting improved health outcomes. Lancet 385, S11 (2015).

    Article  Google Scholar 

  2. 2.

    Gawande, A. A., Thomas, E. J., Zinner, M. J. & Brennan, T. A. The incidence and nature of surgical adverse events in Colorado and Utah in 1992. Surgery 126, 66–75 (1999).

    CAS  Article  Google Scholar 

  3. 3.

    Kable, A. K., Gibberd, R. W. & Spigelman, A. D. Adverse events in surgical patients in Australia. Int. J. Qual. Health Care 14, 269–276 (2002).

    CAS  Article  Google Scholar 

  4. 4.

    Nair, B. G., Gabel, E., Hofer, I., Schwid, H. A. & Cannesson, M. Intraoperative clinical decision support for anesthesia. Anesth. Analg. 124, 603–617 (2017).

    Article  Google Scholar 

  5. 5.

    Maier-Hein, L. et al. Surgical data science for next-generation interventions. Nat. Biomed. Eng. 1, 691–696 (2017).

    Article  Google Scholar 

  6. 6.

    Dunham, C. M., Hileman, B. M., Hutchinson, A. E., Chance, E. A. & Huang, G. S. Perioperative hypoxemia is common with horizontal positioning during general anesthesia and is associated with major adverse outcomes: a retrospective study of consecutive patients. BMC Anesthesiol. 14, 43 (2014).

    Article  Google Scholar 

  7. 7.

    Strachan, L. & Noble, D. W. Hypoxia and surgical patients--prevention and treatment of an unnecessary cause of morbidity and mortality. J. R. Coll. Surg. Edinb. 46, 297–302 (2001).

    CAS  PubMed  Google Scholar 

  8. 8.

    Ehrenfeld, J. M. et al. The incidence of hypoxemia during surgery: evidence from two institutions. Can. J. Anaesth. 57, 888–897 (2010).

    Article  Google Scholar 

  9. 9.

    Kooij, F. O., Klok, T., Hollmann, M. W. & Kal, J. E. Decision support increases guideline adherence for prescribing postoperative nausea and vomiting prophylaxis. Anesth. Analg. 106, 893–898 (2008).

    Article  Google Scholar 

  10. 10.

    Garg, A. X. et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA 293, 1223–1238 (2005).

    CAS  Article  Google Scholar 

  11. 11.

    ElMoaqet, H., Tilbury, D. M. & Ramachandran, S. K. Multi-step ahead predictions for critical levels in physiological time series. IEEE Trans. Cybern. 46, 1704–1714 (2016).

    Article  Google Scholar 

  12. 12.

    Lipton, Z. C., Kale, D. C. & Wetzell, R. C. Phenotyping of clinical time series with LSTM recurrent neural networks. Preprint at http://arxiv.org/abs/1510.07641 (2015).

  13. 13.

    Henry, K. E., Hager, D. N., Pronovost, P. J. & Saria, S. A targeted real-time early warning score (TREWScore) for septic shock. Sci. Transl. Med. 7, 299ra122 (2015).

    Article  Google Scholar 

  14. 14.

    Saria, S., Rajani, A. K., Gould, J., Koller, D. & Penn, A. A. Integration of early physiological responses predicts later illness severity in preterm infants. Sci. Transl. Med. 2, 48ra65 (2010).

    Article  Google Scholar 

  15. 15.

    Caruana, R. et al. Intelligible models for healthcare: predicting pneumonia risk and hospital 30-day readmission. In Proc. 21st ACM SIGKDD Int. Conf. Knowl. Discov. Data Min. 1721–1730 (ACM, 2015).

  16. 16.

    Deo, R. C. Machine learning in medicine. Circulation 132, 1920–1930 (2015).

    Article  Google Scholar 

  17. 17.

    Memarian, N., Kim, S., Dewar, S., Engel, J. & Staba, R. J. Multimodal data and machine learning for surgery outcome prediction in complicated cases of mesial temporal lobe epilepsy. Comput. Biol. Med. 64, 67–78 (2015).

    Article  Google Scholar 

  18. 18.

    Štrumbelj, E. & Kononenko, I. Explaining prediction models and individual predictions with feature contributions. Knowl. Inf. Syst. 41, 647–665 (2014).

    Article  Google Scholar 

  19. 19.

    Ribeiro, M. T., Singh, S. & Guestrin, C. “Why should I trust you?” Explaining the predictions of any classifier. In Proc. 22nd ACM SIGKDD Int. Conf. Know. Disc. Data Min. 1135–1144 (ACM, 2016).

  20. 20.

    Lundberg, S. & Lee, S.-I. A unified approach to interpreting model predictions. In Adv. Neural Information Processing 4765–4774 (Curran Associates, 2017).

  21. 21.

    Lundberg, S. M., Erion, G. G. & Lee, S.-I. Consistent individualized feature attribution for tree ensembles. Preprint at http://arxiv.org/abs/1802.03888 (2018).

  22. 22.

    Tarassenko, L., Hann, A. & Young, D. Integrated monitoring and analysis for early warning of patient deterioration. Br. J. Anaesth. 97, 64–68 (2006).

    CAS  Article  Google Scholar 

  23. 23.

    Summers, R. L., Pipke, M., Wegerich, S., Conkright, G. & Isom, K. C. Functionality of empirical model-based predictive analytics for the early detection of hemodynamic instabilty. Biomed. Sci. Instrum. 50, 219–224 (2014).

    PubMed  Google Scholar 

  24. 24.

    Current Procedural Terminology: CPT (American Medical Association, 2007).

  25. 25.

    Friedman, J. H. Greedy function approximation: a gradient boosting machine. Ann. Stat. 29, 1189–1232 (2001).

    Article  Google Scholar 

  26. 26.

    Chen, T. & Guestrin, C. XGBoost: a scalable tree boosting system. In Proc. 22nd ACM SIGKDD Int. Conf. Know. Disc. Data Min. 785–794 (ACM, 2016).

  27. 27.

    Lumachi, F. et al. Relationship between body mass index, age and hypoxemia in patients with extremely severe obesity undergoing bariatric surgery. In Vivo 24, 775–777 (2010).

  28. 28.

    Kendale, S. M. & Blitz, J. D. Increasing body mass index and the incidence of intraoperative hypoxemia. J. Clin. Anesth. 33, 97–104 (2016).

    Article  Google Scholar 

  29. 29.

    Defining Adult Overweight and Obesity (Centers for Disease Control and Prevention, 2016); https://www.cdc.gov/obesity/adult/defining.html

  30. 30.

    Myles, P. S., Leslie, K., McNeil, J., Forbes, A. & Chan, M. T. V. Bispectral index monitoring to prevent awareness during anaesthesia: the B-Aware randomised controlled trial. Lancet 363, 1757–1763 (2004).

    CAS  Article  Google Scholar 

  31. 31.

    Avidan, M. S. et al. Anesthesia awareness and the bispectral index. N. Engl. J. Med. 358, 1097–1108 (2008).

    CAS  Article  Google Scholar 

  32. 32.

    Epstein, R. H., Dexter, F. & Patel, N. Influencing anesthesia provider behavior using anesthesia information management system data for near real-time alerts and post hoc reports. Anesth. Analg. 121, 678–692 (2015).

    CAS  Article  Google Scholar 

  33. 33.

    Guay, J. & Ochroch, E. A. Intraoperative use of low volume ventilation to decrease postoperative mortality, mechanical ventilation, lengths of stay and lung injury in patients without acute lung injury. Cochrane Datab. Syst. Rev. J. 2018, CD011151 (2018).

  34. 34.

    Pulse Oximetry Training Manual (World Health Organization, 2011).

  35. 35.

    Dyagilev, K. & Saria, S. Learning (predictive) risk scores in the presence of censoring due to interventions. Mach. Learn. 102, 323–348 (2016).

    Article  Google Scholar 

  36. 36.

    Roth, A. E. (ed.) The Shapley Value: Essays in Honor of Llloyd S. Shapley (Cambridge Univ. Press, Cambridge, 1988).

    Google Scholar 

  37. 37.

    Health, United States, 2016: With Chartbook on Long-term Trends in Health 314–317 (National Center for Health Statistics, Hyattsville, 2017).

Download references


We thank G. Erion, M. T. Ribeiro, J. Schreiber and members of the Lee laboratory for feedback and suggestions that improved the manuscript and experiments. This work was supported by National Science Foundation grant nos. DBI-135589 and DBI-1552309, National Institutes of Health grant no. 1R35GM128638, NSF Graduate Research Fellowship grant no. DGE-1256082 and a UW eScience/ITHS seed grant Machine Learning in Operating Rooms.

Author information




S.-I.L., S.M.L., B.N. and J.K. initiated the study. S.-I.L. and S.M.L. developed the Prescience algorithms and designed data analyses and experiments. S.M.L. performed data analyses, experiments and data preprocessing. B.N. and S.-F.N. provided the electronic medical record data. J.K. recruited anaesthesiologists and helped design the anaesthesiologist test and survey. M.H., M.J.E., T.A., D.E.L. and D.K.-W.L. performed the web-based anaesthesiologist experiments and provided survey data. M.S.V. provided clinical assessment, interpretation of feature importances and connections with anaesthesiologists’ workflow. S.-I L. and S.M.L. wrote the paper in conjunction with B.N., J.K. and M.S.V. who wrote the sections on clinical interpretation and integration with current practices. M.H. provided manuscript feedback.

Corresponding author

Correspondence to Su-In Lee.

Ethics declarations

Competing interests

B.N. is an advisor for Perimatics LLC and holds equity in the company. D.K.-W.L. is a Chief Medical Officer for MDmetrix, Inc. The other authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Figures 1–12, Supplementary Tables 1–3 and Supplementary References 1–2.

Reporting Summary

Supplementary Table 4

Initial features used by Prescience. An enumeration of all the 3,797 features used for preoperative predictions.

Supplementary Table 5

Intraoperative features used by Prescience. An enumeration of all the 3,905 features used for intraoperative predictions.

Supplementary Table 6

Data for Fig. 3a.

Supplementary Table 7

Data for Fig. 3b.

Supplementary Table 8

Data for Fig. 5a.

Supplementary Table 9

Data for Fig. 5b.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Lundberg, S.M., Nair, B., Vavilala, M.S. et al. Explainable machine-learning predictions for the prevention of hypoxaemia during surgery. Nat Biomed Eng 2, 749–760 (2018). https://doi.org/10.1038/s41551-018-0304-0

Download citation

Further reading