Abstract
Artificial intelligence (AI)-based methods have emerged as powerful tools to transform medical care. Although machine learning classifiers (MLCs) have already demonstrated strong performance in image-based diagnoses, analysis of diverse and massive electronic health record (EHR) data remains challenging. Here, we show that MLCs can query EHRs in a manner similar to the hypothetico-deductive reasoning used by physicians and unearth associations that previous statistical methods have not found. Our model applies an automated natural language processing system using deep learning techniques to extract clinically relevant information from EHRs. In total, 101.6 million data points from 1,362,559 pediatric patient visits presenting to a major referral center were analyzed to train and validate the framework. Our model demonstrates high diagnostic accuracy across multiple organ systems and is comparable to experienced pediatricians in diagnosing common childhood diseases. Our study provides a proof of concept for implementing an AI-based system as a means to aid physicians in tackling large amounts of data, augmenting diagnostic evaluations, and to provide clinical decision support in cases of diagnostic uncertainty or complexity. Although this impact may be most evident in areas where healthcare providers are in relative shortage, the benefits of such an AI system are likely to be universal.
This is a preview of subscription content, access via your institution
Relevant articles
Open Access articles citing this article.
-
Acute renal injury after aortic arch reconstruction with cardiopulmonary bypass for children: prediction models by machine learning of a retrospective cohort study
European Journal of Medical Research Open Access 08 November 2023
-
The long-term health outcomes, pathophysiological mechanisms and multidisciplinary management of long COVID
Signal Transduction and Targeted Therapy Open Access 01 November 2023
-
Real-time detection of laryngopharyngeal cancer using an artificial intelligence-assisted system with multimodal data
Journal of Translational Medicine Open Access 07 October 2023
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Rent or buy this article
Prices vary by article type
from$1.95
to$39.95
Prices may be subject to local taxes which are calculated during checkout
Data availability
We have made available the Jupyter notebook that we used in constructing and validating the hierarchical logistic regression models: https://s3.cn-north-1.amazonaws.com.cn/ped.emr/Data/hierachical_logistic_regression.ipynb. To protect patient confidentiality, we have deposited de-identified aggregated patient data in a secured and patient confidentiality compliant cloud in China in concordance with data security regulations. Data access can be requested by writing to the corresponding authors. All data access requests will be reviewed and (if successful) granted by the Data Access Committee.
References
Hu, J., Perer, A. & Wang, F. Data Driven Analytics for Personalized Healthcare. (Springer Internatonal Publishing, Switzerland, Healthcare Information Management Systems: Cases, Strategies, and Solutions, 2016).
Nezhad, M.Z., Zhu, D.X., Sadati, N., Yang, K. & Levy, P. SUBIC: A supervised bi-clustering approach for precision medicine. 2017 16th Ieee International Conference on Machine Learning and Applications (Icmla). Preprint at https://arxiv.org/pdf/1709.09929.pdf (2017).
Hornberger, J. Electronic health records: a guide for clinicians and administrators. JAMA 301, 110–110 (2009).
Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118 (2017).
Kermany, D. S. et al. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172, 1122–1131 (2018).
Gulshan, V. et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316, 2402–2410 (2016).
Erickson, B. J., Korfiatis, P., Akkus, Z. & Kline, T. L. Machine learning for medical imaging. Radiographics 37, 505–515 (2017).
Wang, F., Zhang, P., Qian, B., Wang, X. & Davidson, I. Clinical risk prediction with multilinear sparse logistic regression. In Proc. 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.145–154 (2014).
Turchin, A. et al. Using regular expressions to abstract blood pressure and treatment intensification information from the text of physician notes. J. Am. Med. Inform. Assoc. 13, 691–695 (2006).
Halevy, A., Norvig, P. & Pereira, F. The unreasonable effectiveness of data. IEEE Intelligent Systems 24, 8–12 (2009).
Banko, M. & Brill, E. Scaling to very very large corpora for natural language disambiguation. In Proc. 39th Annual Meeting Association for Computational Linguistics. 26–33 (Association for Computational Linguistics, Stroudsburg, 2001).
Tsui, B. Y., et al. Creating a scalable deep learning based named entity recognition model for biomedical textual data by repurposing biosample free-text annotations. Preprint at https://www.biorxiv.org/content/biorxiv/early/2018/09/12/414136.full.pdf (2018).
Rajkomar, A. et al. Scalable and accurate deep learning with electronic health records. NPJ Digital Medicine 1, 18 (2018).
Wilkinson, M. D. et al. Comment: the fair guiding principles for scientific data management and stewardship. Sci. Data 3, 160018 (2016).
Liang, Y., Chen, Z., Huang, X. & Zeng, L. Analysis of the disease spectrum of hospitalized children in guangdong province. Chin. Med. J. (Engl) 1, 414–418 (2013).
WHO. International Statistical Classification of Diseases and Related Health Problems. (World Health Organization, 2004).
English–Chinese Medical Dictionary (英汉医学大词典) (Shanghai Scientific and Technical Publishers (上海科学技术出版社), 2015).
Lindberg, D. A. B., Humphreys, B. L. & Mccray, A. T. The unified medical language system. Methods Inf. Med. 32, 281–291 (1993).
Tweedie, F. J., Singh, S. & Holmes, D. I. Neural network applications in stylometry: the federalist papers. Computers and the Humanities 30, 1–10 (1996).
Luong, M.-T., Pham, H. & Manning, C. D. Effective approaches to attention-based neural machine translation. Preprint at https://arxiv.org/abs/1508.04025 (2015).
Lipton, Z.C., Kale, D.C. & Wetzel, R.C. Phenotyping of clinical time series with LSTM recurrent neural networks. Preprint at https://arxiv.org/pdf/1510.07641.pdf (2015).
Peng, X.B., Andrychowicz, M., Zaremba, W. & Abbeel, P. Sim-to-real transfer of robotic control with dynamics randomization. IEEE International Conference on Robotics and Automation (ICRA) 3803–3810 (2018).
Graves, A. & Schmidhuber, J. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Networks 18, 602–610 (2005).
Yeung, K.Y. & Ruzzo, W.L. Details of the adjusted rand index and clustering algorithms supplement to the paper ‘an empirical study on Principal Component Analysis for clustering gene expression data. Available at http://faculty.washington.edu/kayee/pca/supp.pdf (2011).
Acknowledgements
This study was funded by the National Key Research and Development Program of China (2017YFC1104600 to H.L.), National Natural Science Foundation of China (81771629 to H.X. and 81700882 to J.X.), Guangzhou Women and Children’s Medical Center, Guangzhou Regenerative Medicine and Health Guangdong Laboratory (Innovation and Startup Talents Program 2018GZR031001 to L.Z. and R.H.).
Author information
Authors and Affiliations
Contributions
H.L., B.T., H.N., W.C., S.L.B., G. Liu, D.S.K., X. S., C.C.S.V., P.T., H.S., J.C., L. H., J.Z., L.Z., R.H., S.H., G. Li, P.L., X.Z., Z.Z., L.P., H.C., R.L., S.L., Y.C., S.T., H.Y., X.H., W. He, W.L., Q.Z., J.J., W.Y., J.G., W.O., Y. Deng, Q.H., B. Wang, C.Y., Y.L., S.Z., Y. Duan, R.Z., S.G., C.L.Z., O.L., E.D.Z., G.K., X.W., C.W., N.N., J.X., W.X., B. Wang, W.W., J.L., B.P., C.B., D.X., W. He, S.H., Y.Z., W. Haw, M.G., A.T., C.-N.H., H.C., L.Z., H.X. and K.Z. collected and analyzed the data. X.H. and K.Z. conceived the project. K.Z., S.L.B., B.T., H.L., and H.X. wrote the manuscript. All authors discussed the results and reviewed the manuscript.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended data
Extended Data 1 Unsupervised clustering of NLP extracted textual features from pediatric diseases.
The diagnostic system analyzed the EHRs in the absence of a defined classification system. This grouping structure reflects the detection of trends in clinical features without pre-defined labeling or human input. The clustered blocks are marked with the boxes with grey lines.
Extended Data 2 Design of the natural language processing information extraction model.
Segmented sentences from the raw text of the EHR were embedded using word2vec. The LSTM model then generated the structured records in a query–answer format. This schematic illustrates the process using the free-text ‘lesion in the upper left lobe of patient’s lung’ as an example.
Supplementary Information
Supplementary Tables
Supplementary Tables 1–9
Rights and permissions
About this article
Cite this article
Liang, H., Tsui, B.Y., Ni, H. et al. Evaluation and accurate diagnoses of pediatric diseases using artificial intelligence. Nat Med 25, 433–438 (2019). https://doi.org/10.1038/s41591-018-0335-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41591-018-0335-9
This article is cited by
-
Artificial intelligence in the diagnosis of dental diseases on panoramic radiographs: a preliminary study
BMC Oral Health (2023)
-
Machine learning-based models to predict one-year mortality among Chinese older patients with coronary artery disease combined with impaired glucose tolerance or diabetes mellitus
Cardiovascular Diabetology (2023)
-
Real-time detection of laryngopharyngeal cancer using an artificial intelligence-assisted system with multimodal data
Journal of Translational Medicine (2023)
-
Weight progression and adherence to weight gain target in women with vs. without gestational diabetes: a retrospective cohort study
BMC Pregnancy and Childbirth (2023)
-
Acute renal injury after aortic arch reconstruction with cardiopulmonary bypass for children: prediction models by machine learning of a retrospective cohort study
European Journal of Medical Research (2023)
