Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

DIGITAL HEALTH

Diagnosing bias in data-driven algorithms for healthcare

Nature Medicine volume 26, pages 25–26 (2020)Cite this article

Subjects

A recent analysis highlighting the potential for algorithms to perpetuate existing racial biases in healthcare underscores the importance of thinking carefully about the labels used during algorithm development.

The commercial algorithm analyzed by Obermeyer et al. was trained in a supervised learning framework to predict a patient’s total medical expenditures in the following year, using their age, sex, prescribed medications, medical encounters and billed amounts during the current year; notably, the algorithm did not include race4. Patients predicted by the algorithm to be in the 97th percentile or above in terms of future expenditures were automatically enrolled in a care program in which patients with complex health needs receive additional resources, and the primary care physicians of patients in the 55th percentile and above were asked to consider enrolling the patients in this care program. As the top 5% of patients account for 50% of health spending5, many health systems now use similar algorithms to guide care-management interventions.

This is a preview of subscription content

Access options

Buy article

Get time limited or full article access on ReadCube.

$32.00

Buy

All prices are NET prices.

References

  1. Nemati, S. et al. Crit. Care Med. 46, 547–553 (2018).

    Article  Google Scholar 

  2. Caruana, R. et al. in Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 1721–1730 (ACM, 2015).

  3. Bayati, M. et al. PLoS One 9, e109264 (2014).

    Article  Google Scholar 

  4. Obermeyer, Z., Powers, B., Vogeli, C. & Mullainathan, S. Science 366, 447–453 (2019).

    CAS  Article  Google Scholar 

  5. Schoenman, J. A. & Chockley, N. Understanding US health care spending. NICHM Foundation Data Brief (2011).

  6. National Academy of Medicine. Effective care for high-need patients. https://nam.edu/HighNeeds/highNeedPatients.html (2017).

  7. Benjamin, R. People’s Science: Bodies and Rights on the Stem Cell Frontier (Stanford University Press, 2013).

  8. Oh, J. et al. Infect. Control Hosp. Epidemiol. 39, 425–433 (2018).

    Article  Google Scholar 

  9. Liu, V. X. et al. Am. J. Respir. Crit. Care Med. 196, 856–863 (2017).

    Article  Google Scholar 

  10. Silver, D. et al. Nature 550, 354–359 (2017).

    CAS  Article  Google Scholar 

  11. Komorowski, M., Celi, L. A., Badawi, O., Gordon, A. C. & Faisal, A. A. Nat. Med. 24, 1716–1720 (2018).

    CAS  Article  Google Scholar 

  12. Schulam, P. & Saria, S. in Advances in Neural Information Processing Systems 30 (eds Guyon, I. et al.) 1697–1708 (Curran Associates, 2017).

Download references

Author information

Affiliations

  1. Division of Computer Science and Engineering, University of Michigan, Ann Arbor, MI, USA

    Jenna Wiens

  2. Law School, University of Michigan, Ann Arbor, MI, USA

    W. Nicholson Price II

  3. Department of Internal Medicine, University of Michigan, Ann Arbor, MI, USA

    Michael W. Sjoding

Authors
  1. Jenna Wiens
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. W. Nicholson Price II
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael W. Sjoding
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jenna Wiens.

Ethics declarations

Competing interests

The authors declare no competing interests.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Wiens, J., Price, W.N. & Sjoding, M.W. Diagnosing bias in data-driven algorithms for healthcare. Nat Med 26, 25–26 (2020). https://doi.org/10.1038/s41591-019-0726-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41591-019-0726-6

Further reading

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing