Guidelines for reinforcement learning in healthcare

Article metrics

In this Comment, we provide guidelines for reinforcement learning for decisions about patient treatment that we hope will accelerate the rate at which observational cohorts can inform healthcare practice in a safe, risk-conscious manner.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.


All prices are NET prices.

Fig. 1: Sequential decision-making tasks.

Debbie Maizels/Springer Nature

Fig. 2: Effective sample size in off-policy evaluation.

Debbie Maizels/Springer Nature


  1. 1.

    Obermeyer, Z. & Emanuel, E. J. N. Engl. J. Med. 375, 1216 (2016).

  2. 2.

    Parbhoo, S., Bogojeska, J., Zazzi, M., Roth, V. & Doshi-Velez, F. AMIA Summits on Translational Science Proceedings 2017, 239 (2017).

  3. 3.

    Guez, A., Vincent, R. D., Avoli, M. & Pineau, J. Treatment of epilepsy via batch-mode reinforcement learning. In Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence 1671–1678 (AAAI, 2008).

  4. 4.

    Komorowski, M., Celi, L. A., Badawi, O., Gordon, A. & Faisal, A. Nat. Med. 24, 1716–1720 (2018).

  5. 5.

    Chakraborty, B., Moodie, E. & Erica, E. M. Statistical Methods for Dynamic Treatment Regimes (Springer, New York, 2013).

  6. 6.

    Simpson, N., Lamontagne, F. & Shankar-Hari, M. Curr Opin Crit Care. 23, 561–566 (2017).

  7. 7.

    Johansson, F., Shalit, U. & Sontag, D. Learning representations for counterfactual inference. In Proceedings of the 33th International Conference on Machine Learning (ICML, 2016).

  8. 8.

    Precup, D., Sutton, R. S. & Singh, S. P. Eligibility traces for off-policy policy evaluation. In Proceedings of the Seventeenth International Conference on Machine Learning 759–766 (ICML, 2000).

  9. 9.

    Gottesman, O. et al. Evaluating Reinforcement Learning Algorithms in Observational Health Settings. Preprint at (2018).

  10. 10.

    Doshi-Velez, F. & Kim, B. Towards a rigorous science of interpretable machine learning. Preprint at (2017).

Download references

Author information

Correspondence to Leo Anthony Celi.

Ethics declarations

Competing interests

A.A.F. has received funding from Fresenius-KABI in the past.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Further reading