Prediction of gestational diabetes based on nationwide electronic health records

Abstract

Gestational diabetes mellitus (GDM) poses increased risk of short- and long-term complications for mother and offspring14. GDM is typically diagnosed at 24–28 weeks of gestation, but earlier detection is desirable as this may prevent or considerably reduce the risk of adverse pregnancy outcomes5,6. Here we used a machine-learning approach to predict GDM on retrospective data of 588,622 pregnancies in Israel for which comprehensive electronic health records were available. Our models predict GDM with high accuracy even at pregnancy initiation (area under the receiver operating curve (auROC) = 0.85), substantially outperforming a baseline risk score (auROC = 0.68). We validated our results on both a future validation set and a geographical validation set from the most populated city in Israel, Jerusalem, thereby emulating real-world performance. Interrogating our model, we uncovered previously unreported risk factors, including results of previous pregnancy glucose challenge tests. Finally, we devised a simpler model based on just nine questions that a patient could answer, with only a modest reduction in accuracy (auROC = 0.80). Overall, our models may allow early-stage intervention in high-risk women, as well as a cost-effective screening approach that could avoid the need for glucose tolerance tests by identifying low-risk women. Future prospective studies and studies on additional populations are needed to assess the real-world clinical utility of the model.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: Data and cohort characteristics.
Fig. 2: Predictive model evaluation.
Fig. 3: Shapley values-based interpretation of the model.
Fig. 4: Questionnaire-based prediction, and efficiency of the predictor as a GDM screening tool.

Data availability

The data that support the findings of this study originate from Clalit Health Services. Restrictions apply to the availability of these data and they are therefore not publicly available. Due to restrictions, these data can be accessed only by request to the authors and/or Clalit Health Services.

Code availability

The code that supports the findings of this study is tailored to the data and the fields of the Clalit Health Services database, and is thus not provided since it is of no use as a standalone without access to the data per se. The algorithmic models used the standard Python code package scikit-learn, which is publicly available.

References

  1. 1.

    Lowe, L. P. et al. Hyperglycemia and Adverse Pregnancy Outcome (HAPO) Study: associations of maternal A1C and glucose with pregnancy outcomes. Diabetes Care 35, 574–580 (2012).

  2. 2.

    Lowe, W. L. et al. Association of gestational diabetes with maternal disorders of glucose metabolism and childhood adiposity. JAMA 320, 1005–1016 (2018).

  3. 3.

    Scholtens, D. M. et al. Hyperglycemia and Adverse Pregnancy Outcome Follow-up Study (HAPO FUS): maternal glycemia and childhood glucose metabolism. Diabetes Care 42, 381–392 (2019).

  4. 4.

    Zhao, P. et al. Maternal gestational diabetes and childhood obesity at age 9–11: results of a multinational study. Diabetologia 59, 2339–2348 (2016).

  5. 5.

    Koivusalo, S. B. et al. Gestational diabetes mellitus can be prevented by lifestyle intervention: the Finnish gestational diabetes prevention study (RADIEL): a randomized controlled trial. Diabetes Care 39, 24–30 (2016).

  6. 6.

    Wang, C. et al. A randomized clinical trial of exercise during pregnancy to prevent gestational diabetes mellitus and improve pregnancy outcome in overweight and obese pregnant women. Am. J. Obstet. Gynecol. 216, 340–351 (2017).

  7. 7.

    Donovan, P. J. & McIntyre, H. D. Drugs for gestational diabetes. Aust. Prescr. 33, 141–144 (2010).

  8. 8.

    American Diabetes Association. 2. Classification and diagnosis of diabetes: standards of medical care in diabetes—2018. Diabetes Care 41, S13–S27 (2018).

  9. 9.

    Hunt, K. J. & Schuller, K. L. The increasing prevalence of diabetes in pregnancy. Obstet. Gynecol. Clin. N. Am. 34, 173–199 (2007).

  10. 10.

    Bain, E. et al. Diet and exercise interventions for preventing gestational diabetes mellitus. Cochrane Database Syst. Rev. CD010443 https://doi.org/10.1002/14651858.CD010443.pub2 (2015).

  11. 11.

    Avati, A. et al. Improving palliative care with deep learning. BMC Med. Inform. Decis. Mak. 18(Suppl 4), 122 (2018).

  12. 12.

    Silva, I., Moody, G., Scott, D. J., Celi, L. A. & Mark, R. G. Predicting in-hospital mortality of ICU patients: the PhysioNet/Computing in Cardiology Challenge 2012. Comput. Cardiol. (2010) 39, 245–248 (2012).

  13. 13.

    Razavian, N., Marcus, J. & Sontag, D. Multi-task prediction of disease onsets from longitudinal lab tests. Preprint at arXiv https://arxiv.org/abs/1608.00647 (2016).

  14. 14.

    Oh, J. et al. A generalizable, data-driven approach to predict daily risk of Clostridium difficile infection at two large academic health centers. Infect. Control Hosp. Epidemiol. 39, 425–433 (2018).

  15. 15.

    Miotto, R., Li, L., Kidd, B. A. & Dudley, J. T. Deep patient: an unsupervised representation to predict the future of patients from the electronic health records. Sci. Rep. 6, 26094 (2016).

  16. 16.

    Danilenko-Dixon, D. R., Van Winter, J. T., Nelson, R. L. & Ogburn, P. L. Universal versus selective gestational diabetes screening: application of 1997 American Diabetes Association recommendations. Am. J. Obstet. Gynecol. 181, 798–802 (1999).

  17. 17.

    Qiu, H. et al. Electronic health record-driven prediction for gestational diabetes mellitus in early pregnancy. Sci. Rep. 7, 16417 (2017).

  18. 18.

    Syngelaki, A. et al. First-trimester screening for gestational diabetes mellitus based on maternal characteristics and history. Fetal Diagn. Ther. 38, 14–21 (2015).

  19. 19.

    US Department of Health and Human Services, National Institutes of Health & Eunice Kennedy Shriver National Institute of Child Health and Human Development. Am I at Risk for Gestational Diabetes? https://www.nichd.nih.gov/sites/default/files/publications/pubs/Documents/gestational_diabetes_2012.pdf (2012).

  20. 20.

    Steyerberg, E. W. et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 21, 128–138 (2010).

  21. 21.

    Vickers, A. J. & Elkin, E. B. Decision curve analysis: a novel method for evaluating prediction models. Med. Decis. Making 26, 565–574 (2006).

  22. 22.

    Lundberg, S. & Lee, S.-I. A unified approach to interpreting model predictions. Adv. Neural Inf. Proc. Syst. 30, 4765–4774 (2017).

  23. 23.

    Chu, S. Y. et al. Maternal obesity and risk of gestational diabetes mellitus. Diabetes Care 30, 2070–2076 (2007).

  24. 24.

    Williams, M. A., Qiu, C., Dempsey, J. C. & Luthy, D. A. Familial aggregation of type 2 diabetes and chronic hypertension in women with gestational diabetes mellitus. J. Reprod. Med. 48, 955–962 (2003).

  25. 25.

    van Leeuwen, M. et al. Glucose challenge test for detecting gestational diabetes mellitus: a systematic review. BJOG 119, 393–401 (2012).

  26. 26.

    Donovan, L. et al. Screening tests for gestational diabetes: a systematic review for the US Preventive Services Task Force. Ann. Intern. Med. 159, 115–122 (2013).

  27. 27.

    Lamain-de Ruiter, M. et al. External validation of prognostic models to predict risk of gestational diabetes mellitus in one Dutch cohort: prospective multicentre cohort study. BMJ 354, i4338 (2016).

  28. 28.

    Lao, T. T., Ho, L.-F., Chan, B. C. P. & Leung, W.-C. Maternal age and prevalence of gestational diabetes mellitus. Diabetes Care 29, 948–949 (2006).

  29. 29.

    Di Cianni, G. et al. Prevalence and risk factors for gestational diabetes assessed by universal screening. Diabetes Res. Clin. Pract. 62, 131–137 (2003).

  30. 30.

    Teh, W. T. et al. Risk factors for gestational diabetes mellitus: implications for the application of screening guidelines. Aust. N. Z. J. Obstet. Gynaecol. 51, 26–30 (2011).

  31. 31.

    Shepherd, E. et al. Combined diet and exercise interventions for preventing gestational diabetes mellitus. Cochrane Database Syst. Rev. 11, CD010443 (2017).

  32. 32.

    Davey, R. X. Selective versus universal screening for gestational diabetes mellitus: an evaluation of predictive risk factors. Medical J. Aust. 174, 118–121 (2001).

  33. 33.

    Kalter-Leibovici, O. et al. Screening and diagnosis of gestational diabetes mellitus: critical appraisal of the new International Association of Diabetes in Pregnancy Study Group recommendations on a national level. Diabetes Care 35, 1894–1896 (2012).

  34. 34.

    Phelan, M., Bhavsar, N. A. & Goldstein, B. A. Illustrating informed presence bias in electronic health records data: how patient interactions with a health system can impact inference. EGEMS (Wash DC) 5, 22 (2017).

  35. 35.

    Zhang, C. & Ning, Y. Effect of dietary and lifestyle factors on the risk of gestational diabetes: review of epidemiologic evidence. Am. J. Clin. Nutr. 94, 1975S–1979S (2011).

  36. 36.

    Dudley, D. J. Diabetic-associated stillbirth: incidence, pathophysiology, and prevention. Clin. Perinatol. 34, 611–626 (2007). vii.

  37. 37.

    Data. Clalit Research Institute; http://clalitresearch.org/about-us/our-data/ (accessed 23 July, 2019).

  38. 38.

    Vandorsten, J. P. et al. NIH consensus development conference: diagnosing gestational diabetes mellitus. NIH Consens. State Sci. Statements 29, 1–31 (2013).

  39. 39.

    State of Isreal Ministry of Health. Monitoring of Pregnancy and Medical Examinations During Pregnancy https://www.health.gov.il/English/Topics/Pregnancy/during/examination/Pages/permanent.aspx (accessed 23 July, 2019).

  40. 40.

    Hastie, T., Tibshirani, R. & Friedman, J. The Elements of Statistical Learning (Springer, 2009).

  41. 41.

    Fernández-Delgado, M., Cernadas, E., Barro, S. & Amorim, D. Do we need hundreds of classifiers to solve real world classification problems? J. Mach. Learn. Res. 15, 3133–3181 (2014).

  42. 42.

    Omar, K. XGBoost and LGBM for Porto Seguro’s Kaggle Challenge: A Comparison Semester Project (ETH, 2018).

  43. 43.

    Biendata Competitions. KDD Cup of Fresh Air https://biendata.com/competition/kdd_2018/winners/ (accessed 23 July 2019).

  44. 44.

    Josse, J., Prost, N., Scornet, E. & Varoquaux, G. On the consistency of supervised learning with missing values. Preprint at arXiv https://arxiv.org/abs/1902.06931 (2019).

  45. 45.

    Chen, T. & Guestrin, C. XGBoost: A scalable tree boosting system. in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD 2016 (eds Krishnapuram, B. et al.) 785–794 (ACM Press, 2016).

  46. 46.

    Ke, G. et al. LightGBM: A Highly Efficient Gradient Boosting Decision Tree https://papers.nips.cc/paper/6907-lightgbm-a-highly-efficient-gradient-boosting-decision-tree.pdf (2017).

  47. 47.

    CBS. Regional Statistics Section https://www.cbs.gov.il/EN/settlements/Pages/default.aspx?mode=Yeshuv (accessed 10 July 2018).

  48. 48.

    Lundberg, S. M. et al. Explainable machine-learning predictions for the prevention of hypoxaemia during surgery. Nat. Biomed. Eng. 2, 749–760 (2018).

Download references

Acknowledgements

We thank G. Barabash, E. Barkan, I. Kalka and members of the Segal group for discussions. E.S. is supported by the Crown Human Genome Center, by D. L. Schwarz, J. N. Halpern and L. Steinberg, and by grants funded by the European Research Council and the Israel Science Foundation.

Author information

N.S.A., S.S. and E.H. conceived the project, designed and conducted the analyses, interpreted the results and wrote the manuscript, and are listed in random order. H.R. conducted the analyses and wrote the manuscript. S.B.-H., A.B.-H., R.D.B. and B.F. interpreted the results. A.W. and E.S. conceived and directed the project and analyses, designed the analyses, interpreted the results, wrote the manuscript and supervised the project.

Correspondence to Arnon Wiznitzer or Eran Segal.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information Joao Monteiro was the primary editor on this article and managed its editorial process and peer review in collaboration with the rest of the editorial team.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Baseline prediction, based on Baseline Risk Score.

a: Odds ratio for the risk score composing parameters. Adjusted odds ratios were derived from a logistic regression model, both values are presented on the training set. b: Prevalence among women grouped by risk score. Error bars represent 90% confidence intervals on the train set. c: Histogram of risk scores in the training set. d: ROC curve for NIH Risk Score and for a logistic regression model trained on its constructing parameters. Results are reported on the future validation set. Logistic regression model does not suppress the Naive summation in the risk score. (n = 82,678 for all panels).

Extended Data Fig. 2 Evaluation of the model on the geographical validation set.

a: Receiver Operating Characteristic (ROC) curve, comparing our model (solid) and the Baseline Risk Score (dashed). Lighter colored lines are ROC curves of stratified partition of the validation set (not shown in ROC); bracketed values are 95% confidence intervals calculated through a normal fit of those curves. b: Precision-Recall (PR) curve, with the same properties as in A. c: The fraction of GDM-positive samples in every decile of the predicted probability. d: Predictions on different subsets of the cohort. auPR is shown for each subset, for our model (blue) and the baseline score (orange). Error bars show 95% confidence intervals, and dark blue lines show the prevalence in each subset. Shaded area is the distribution of the relevant score. e: Performance by gestational age at prediction. Every point is the evaluation score of a model built only with features available at this time point. (n = 46,002 for panels A-C. Subset sample sizes are listed in panel D).

Extended Data Fig. 3 Evaluation of the model on the geo-temporal validation set.

a: Receiver Operating Characteristic (ROC) curve, comparing our model (solid) and the Baseline Risk Score (dashed). Lighter colored lines are ROC curves of stratified partition of the validation set; bracketed values are 95% confidence intervals calculated through a normal fit of those curves. b: Precision-Recall (PR) curve, with the same properties as in A. c: The fraction of GDM-positive samples in every decile of the predicted probability. d: Predictions on different subsets of the cohort. auPR is shown for each subset, for our model (blue) and the baseline score (orange). Error bars show 95% confidence intervals, and dark blue lines show the prevalence in each subset. Shaded area is the distribution of the relevant score. e: Performance by gestational age at prediction. Every point is the evaluation score of a model built only with features available at this time point. (n = 8,540 for panels A-C. Subset sample sizes are listed in panel D).

Extended Data Fig. 4

Evaluation results in different validation sets.

Extended Data Fig. 5 Basic utility of the predictor.

a: Calibration curve, showing the fraction of positive samples per bin versus the mean predicted probability of the bin. Blue and red bars represent the ratio of negative/positive samples in the bin, respectively. b: Decision curve, showing the net benefit versus the threshold probability, for both predictor and baseline. The predictor outperforms the baseline at all thresholds. (n = 82,678 for all panels).

Extended Data Fig. 6 Additional dependence plots.

Top 20 features are shown (ordered left to right, top to bottom). In each the mean predicted relative risk is plotted versus feature value. Bands represent SD area of the population per bin, which is connected to interactions between input features. (n = 82,678).

Extended Data Fig. 7 Histogram of lab tests during pregnancy, showing the window definition of F0, F1 and F2.

The peaks showing are weekly, and represents the fact that patients tend to see a doctor in the same day of the week.

Supplementary information

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Artzi, N.S., Shilo, S., Hadar, E. et al. Prediction of gestational diabetes based on nationwide electronic health records. Nat Med 26, 71–76 (2020). https://doi.org/10.1038/s41591-019-0724-8

Download citation