Skip to main content

Thank you for visiting You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Comment
  • Published:

Guidelines for reinforcement learning in healthcare

In this Comment, we provide guidelines for reinforcement learning for decisions about patient treatment that we hope will accelerate the rate at which observational cohorts can inform healthcare practice in a safe, risk-conscious manner.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Buy this article

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Sequential decision-making tasks.

Debbie Maizels/Springer Nature

Fig. 2: Effective sample size in off-policy evaluation.

Debbie Maizels/Springer Nature


  1. Obermeyer, Z. & Emanuel, E. J. N. Engl. J. Med. 375, 1216 (2016).

    Article  Google Scholar 

  2. Parbhoo, S., Bogojeska, J., Zazzi, M., Roth, V. & Doshi-Velez, F. AMIA Summits on Translational Science Proceedings 2017, 239 (2017).

    PubMed Central  Google Scholar 

  3. Guez, A., Vincent, R. D., Avoli, M. & Pineau, J. Treatment of epilepsy via batch-mode reinforcement learning. In Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence 1671–1678 (AAAI, 2008).

  4. Komorowski, M., Celi, L. A., Badawi, O., Gordon, A. & Faisal, A. Nat. Med. 24, 1716–1720 (2018).

  5. Chakraborty, B., Moodie, E. & Erica, E. M. Statistical Methods for Dynamic Treatment Regimes (Springer, New York, 2013).

  6. Simpson, N., Lamontagne, F. & Shankar-Hari, M. Curr Opin Crit Care. 23, 561–566 (2017).

    Article  Google Scholar 

  7. Johansson, F., Shalit, U. & Sontag, D. Learning representations for counterfactual inference. In Proceedings of the 33th International Conference on Machine Learning (ICML, 2016).

  8. Precup, D., Sutton, R. S. & Singh, S. P. Eligibility traces for off-policy policy evaluation. In Proceedings of the Seventeenth International Conference on Machine Learning 759–766 (ICML, 2000).

  9. Gottesman, O. et al. Evaluating Reinforcement Learning Algorithms in Observational Health Settings. Preprint at (2018).

  10. Doshi-Velez, F. & Kim, B. Towards a rigorous science of interpretable machine learning. Preprint at (2017).

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Leo Anthony Celi.

Ethics declarations

Competing interests

A.A.F. has received funding from Fresenius-KABI in the past.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gottesman, O., Johansson, F., Komorowski, M. et al. Guidelines for reinforcement learning in healthcare. Nat Med 25, 16–18 (2019).

Download citation

  • Published:

  • Issue Date:

  • DOI:

This article is cited by


Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing