In this Comment, we provide guidelines for reinforcement learning for decisions about patient treatment that we hope will accelerate the rate at which observational cohorts can inform healthcare practice in a safe, risk-conscious manner.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

References

  1. 1.

    Obermeyer, Z. & Emanuel, E. J. N. Engl. J. Med. 375, 1216 (2016).

  2. 2.

    Parbhoo, S., Bogojeska, J., Zazzi, M., Roth, V. & Doshi-Velez, F. AMIA Summits on Translational Science Proceedings 2017, 239 (2017).

  3. 3.

    Guez, A., Vincent, R. D., Avoli, M. & Pineau, J. Treatment of epilepsy via batch-mode reinforcement learning. In Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence 1671–1678 (AAAI, 2008).

  4. 4.

    Komorowski, M., Celi, L. A., Badawi, O., Gordon, A. & Faisal, A. Nat. Med. 24, 1716–1720 (2018).

  5. 5.

    Chakraborty, B., Moodie, E. & Erica, E. M. Statistical Methods for Dynamic Treatment Regimes (Springer, New York, 2013).

  6. 6.

    Simpson, N., Lamontagne, F. & Shankar-Hari, M. Curr Opin Crit Care. 23, 561–566 (2017).

  7. 7.

    Johansson, F., Shalit, U. & Sontag, D. Learning representations for counterfactual inference. In Proceedings of the 33th International Conference on Machine Learning (ICML, 2016).

  8. 8.

    Precup, D., Sutton, R. S. & Singh, S. P. Eligibility traces for off-policy policy evaluation. In Proceedings of the Seventeenth International Conference on Machine Learning 759–766 (ICML, 2000).

  9. 9.

    Gottesman, O. et al. Evaluating Reinforcement Learning Algorithms in Observational Health Settings. Preprint at https://arxiv.org/abs/1805.12298 (2018).

  10. 10.

    Doshi-Velez, F. & Kim, B. Towards a rigorous science of interpretable machine learning. Preprint at https://arxiv.org/abs/1702.08608 (2017).

Download references

Author information

Author notes

  1. These authors contributed equally: Omer Gottesman, Fredrik Johansson.

Affiliations

  1. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA, USA

    • Omer Gottesman
    •  & Finale Doshi-Velez
  2. Institute for Medical Engineering and Science, MIT, Cambridge, MA, USA

    • Fredrik Johansson
    •  & David Sontag
  3. Laboratory for Computational Physiology, Harvard-MIT Health Sciences & Technology, MIT, Cambridge, MA, USA

    • Matthieu Komorowski
    •  & Leo Anthony Celi
  4. Department of Surgery and Cancer, Faculty of Medicine, Imperial College London, London, UK

    • Matthieu Komorowski
  5. Department of Bioengineering, Imperial College London, London, UK

    • Aldo Faisal
  6. Department of Computing, Imperial College London, London, UK

    • Aldo Faisal
  7. Data Science Institute, London, UK

    • Aldo Faisal
  8. MRC London Institute of Clinical Sciences, London, UK

    • Aldo Faisal
  9. Division of Pulmonary, Critical Care and Sleep Medicine, Beth Israel Deaconess Medical Center, Boston, MA, USA

    • Leo Anthony Celi
  10. MIT Critical Data, Cambridge, MA, USA

    • Leo Anthony Celi

Authors

  1. Search for Omer Gottesman in:

  2. Search for Fredrik Johansson in:

  3. Search for Matthieu Komorowski in:

  4. Search for Aldo Faisal in:

  5. Search for David Sontag in:

  6. Search for Finale Doshi-Velez in:

  7. Search for Leo Anthony Celi in:

Competing interests

A.A.F. has received funding from Fresenius-KABI in the past.

Corresponding author

Correspondence to Leo Anthony Celi.

About this article

Publication history

Published

DOI

https://doi.org/10.1038/s41591-018-0310-5

Newsletter Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing