Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Derivation and external validation of risk stratification models for severe maternal morbidity using prenatal encounter diagnosis codes

Journal of Perinatology volume 41, pages 2590–2596 (2021)Cite this article

Subjects

Abstract

Objective

We sought to develop a prediction model using prenatal diagnosis codes that could help clinicians objectively stratify a women’s risk for delivery-related morbidity.

Study design

We performed a prospective cohort study of women delivering at a single academic medical center between 2016 and 2019. Diagnosis codes from outpatient encounters were extracted from the electronic health record. Standard and common machine-learning methods for variable selection were compared. The performance characteristics from the selected model in the training data set—a LASSO model with a lambda that minimized the Bayes information criteria—were compared in a testing and external validation set.

Results

The model identified a group of women, those in the highest decile of predicted risk, who were at a two to threefold increased risk of maternal morbidity.

Conclusion

As EHR data becomes more ubiquitous, other data types generated from the prenatal period may improve the model’s performance.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Frequency of individual 3-digit ICD-10 codes.
Fig. 2: Receiver operating curve comparisons.
Fig. 3: Calibration plot comparisons.

Similar content being viewed by others

References

  1. Creanga AA, Berg CJ, Syverson C, Seed K, Bruce FC, Callaghan WM. Pregnancy-related mortality in the United States, 2006–10. Obstet Gynecol. 2015;125:5–12.

    Article  Google Scholar 

  2. Creanga AA, Syverson C, Seed K, Callaghan WM. Pregnancy-related mortality in the United States, 2011–3. Obstet Gynecol. 2017;130:366–73.

    Article  Google Scholar 

  3. Creanga AA, Berg CJ, Ko JY, Farr SL, Tong VT, Bruce FC, et al. Maternal mortality and morbidity in the United States: where are we now? J Women’s Health. 2014;23:3–9.

    Article  Google Scholar 

  4. Severe Maternal Morbidity in the United States Pregnancy Reproductive Health CDC. https://www.cdc.gov/reproductivehealth/maternalinfanthealth/severematernalmorbidity.html Accessed 4 Dec 2017.

  5. Bateman BT, Bansil P, Hernandez-Diaz S, Mhyre JM, Callaghan WM, Kuklina EV. Prevalence, trends, and outcomes of chronic hypertension: a nationwide sample of delivery admissions. Am J Obstet Gynecol. 2012;206:134.e1–8.

    Article  Google Scholar 

  6. DeSisto CL, Kim SY, Sharma AJ. Prevalence estimates of gestational diabetes mellitus in the United States, pregnancy risk assessment monitoring system (PRAMS), 2007–10. Prev Chronic Dis. 2014;11:E104.

    Article  Google Scholar 

  7. Mogos MF, Salemi JL, Ashley M, Whiteman VE, Salihu HM. Recent trends in placenta accreta in the United States and its impact on maternal-fetal morbidity and healthcare-associated costs, 1998–2011. J Matern Fetal Neonatal Med. 2016;29:1077–82.

    Article  Google Scholar 

  8. Bardenheier BH, Imperatore G, Gilboa SM, Geiss LS, Saydah SH, Devlin HM, et al. Trends in gestational diabetes among hospital deliveries in 19 U.S. States, 2000–10. Am J Prev Med. 2015;49:12–19.

    Article  Google Scholar 

  9. Bauer ME, Bateman BT, Bauer ST, Shanks AM, Mhyre JM. Maternal sepsis mortality and morbidity during hospitalization for delivery: temporal trends and independent associations for severe sepsis. Anesth Analg. 2013;117:944–50.

    Article  Google Scholar 

  10. American College of Obstetricians and Gynecologists and Society for Maternal–Fetal Medicine, Menard MK, Kilpatrick S, Saade G, Hollier LM, Joseph GF et al. Levels of maternal care. Am J Obstet Gynecol. 2015;212:259–71.

    Article  Google Scholar 

  11. Easter SR, Robinson JN, Menard MK, Creanga AA, Xu X, Little SE, et al. Potential effects of regionalized maternity care on U.S. hospitals. Obstet Gynecol. 2019;134:545–52.

    Article  Google Scholar 

  12. Bukowski R, Schulz K, Gaither K, Stephens KK, Semeraro D, Drake J, et al. Computational medicine, present and the future: obstetrics and gynecology perspective. Am J Obstet Gynecol. 2021;224:16–34.

    Article  Google Scholar 

  13. Venkatesh KK, Strauss RA, Grotegut CA, Heine RP, Chescheir NC, Stringer JSA, et al. Machine learning and statistical models to predict postpartum hemorrhage. Obstet Gynecol. 2020;135:935–44.

    Article  Google Scholar 

  14. Stamilio DM, Beckham AJ, Boggess KA, Jelovsek JE, Venkatesh KK. Risk factors for postpartum readmission for preeclampsia or hypertension before delivery discharge among low-risk women: a case-control study. Am J Obstet Gynecol MFM. 2021;3:100317.

    Article  Google Scholar 

  15. Maric T, Singh N, Duncan K, Thorpe-Beeston GJ, Savvidou MD. First-trimester fetal growth discordance and development of preeclampsia in dichorionic twin pregnancies. J Perinat Med. 2015;43:755–9.

    Article  Google Scholar 

  16. Bateman BT, Mhyre JM, Hernandez-Diaz S, Huybrechts KF, Fischer MA, Creanga AA, et al. Development of a comorbidity index for use in obstetric patients. Obstet Gynecol. 2013;122:957–65.

    Article  Google Scholar 

  17. Easter SR, Bateman BT, Sweeney VH, Manganaro K, Lassey SC, Gagne JJ, et al. A comorbidity-based screening tool to predict severe maternal morbidity at the time of delivery. Am J Obstetr Gynecol. 2019. https://doi.org/10.1016/j.ajog.2019.06.025.

  18. Hoffman MK, Ma N, Roberts A. A machine learning algorithm for predicting maternal readmission for hypertensive disorders of pregnancy. Am J Obstet Gynecol MFM. 2021;3:100250.

    Article  Google Scholar 

  19. Rosenbloom JI, Tuuli MG, Stout MJ, Young OM, Woolfolk CL, López JD et al. A Prediction Model for Severe Maternal Morbidity in Laboring Patients at Term. Am J Perinatol. 2018. https://doi.org/10.1055/s-0038-1626716.

  20. ICD—ICD-10-CM—International Classification of Diseases, ICD-10-CM/PCS Transition. 2019.https://www.cdc.gov/nchs/icd/icd10cm_pcs_background.htm Accessed 20 Jul 2020.

  21. How Does CDC Identify Severe Maternal Morbidity? CDC. 2019.https://www.cdc.gov/reproductivehealth/maternalinfanthealth/smm/severe-morbidity-ICD.htm Accessed 21 Apr 2020.

  22. HCUP-US NIS Overview. https://www.hcup-us.ahrq.gov/nisoverview.jsp Accessed 18 Jan 2017.

  23. Steyerberg EW. Clinical prediction models. Second Edition. Switzerland: Springer Nature. 2019.

  24. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Mak. 2006;26:565–74.

    Article  Google Scholar 

  25. Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ. 2015;350:g7594.

    Article  Google Scholar 

  26. Metcalfe A, Lix LM, Johnson J-A, Currie G, Lyon AW, Bernier F, et al. Validation of an obstetric comorbidity index in an external population. BJOG. 2015;122:1748–55.

    Article  CAS  Google Scholar 

Download references

Funding

Dr. Clapp’s work on this project was funded by a grant from the American Association of Obstetricians and Gynecologists Foundation and the American Board of Obstetricians and Gynecologists.

Author information

Authors and Affiliations

  1. Department of Obstetrics and Gynecology, Massachusetts General Hospital, Boston, MA, USA

    Mark A. Clapp, Kaitlyn E. James & Anjali J. Kaimal

  2. Center for Quantitative Health, Massachusetts General Hospital, Boston, MA, USA

    Thomas H. McCoy Jr &  Roy H. Perlis

  3. Department of Psychiatry, Massachusetts General Hospital, Boston, MA, USA

    Thomas H. McCoy Jr &  Roy H. Perlis

  4. Department of Population Medicine, Harvard Medical School, Boston, MA, USA

    Anjali J. Kaimal

Authors
  1. Mark A. Clapp
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas H. McCoy Jr
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kaitlyn E. James
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anjali J. Kaimal
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Roy H. Perlis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mark A. Clapp.

Ethics declarations

Conflict of interest

Dr. Perlis has received fees for consulting or service on scientific advisory boards for Genomind, Psy Therapeutics, Outermost Therapeutics, RID Ventures, and Takeda. He has received patent royalties from Massachusetts General Hospital. He holds equity in Psy Therapeutics and Outermost Therapeutics.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Clapp, M.A., McCoy Jr, T.H., James, K.E. et al. Derivation and external validation of risk stratification models for severe maternal morbidity using prenatal encounter diagnosis codes. J Perinatol 41, 2590–2596 (2021). https://doi.org/10.1038/s41372-021-01072-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41372-021-01072-z

Search

Advanced search

Quick links