Abstract
Artificial intelligence (AI)-based methods have emerged as powerful tools to transform medical care. Although machine learning classifiers (MLCs) have already demonstrated strong performance in image-based diagnoses, analysis of diverse and massive electronic health record (EHR) data remains challenging. Here, we show that MLCs can query EHRs in a manner similar to the hypothetico-deductive reasoning used by physicians and unearth associations that previous statistical methods have not found. Our model applies an automated natural language processing system using deep learning techniques to extract clinically relevant information from EHRs. In total, 101.6 million data points from 1,362,559 pediatric patient visits presenting to a major referral center were analyzed to train and validate the framework. Our model demonstrates high diagnostic accuracy across multiple organ systems and is comparable to experienced pediatricians in diagnosing common childhood diseases. Our study provides a proof of concept for implementing an AI-based system as a means to aid physicians in tackling large amounts of data, augmenting diagnostic evaluations, and to provide clinical decision support in cases of diagnostic uncertainty or complexity. Although this impact may be most evident in areas where healthcare providers are in relative shortage, the benefits of such an AI system are likely to be universal.
This is a preview of subscription content, access via your institution
Relevant articles
Open Access articles citing this article.
-
Real-time artificial intelligence assisted insulin dosage titration system for glucose control in type 2 diabetic patients: a proof of concept study
Current Medicine Open Access 03 February 2023
-
Artificial intelligence for strengthening healthcare systems in low- and middle-income countries: a systematic scoping review
npj Digital Medicine Open Access 28 October 2022
-
Materials discovery of ion-selective membranes using artificial intelligence
Communications Chemistry Open Access 20 October 2022
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 per month
cancel any time
Subscribe to this journal
Receive 12 print issues and online access
$189.00 per year
only $15.75 per issue
Rent or buy this article
Get just this article for as long as you need it
$39.95
Prices may be subject to local taxes which are calculated during checkout
Data availability
We have made available the Jupyter notebook that we used in constructing and validating the hierarchical logistic regression models: https://s3.cn-north-1.amazonaws.com.cn/ped.emr/Data/hierachical_logistic_regression.ipynb. To protect patient confidentiality, we have deposited de-identified aggregated patient data in a secured and patient confidentiality compliant cloud in China in concordance with data security regulations. Data access can be requested by writing to the corresponding authors. All data access requests will be reviewed and (if successful) granted by the Data Access Committee.
References
Hu, J., Perer, A. & Wang, F. Data Driven Analytics for Personalized Healthcare. (Springer Internatonal Publishing, Switzerland, Healthcare Information Management Systems: Cases, Strategies, and Solutions, 2016).
Nezhad, M.Z., Zhu, D.X., Sadati, N., Yang, K. & Levy, P. SUBIC: A supervised bi-clustering approach for precision medicine. 2017 16th Ieee International Conference on Machine Learning and Applications (Icmla). Preprint at https://arxiv.org/pdf/1709.09929.pdf (2017).
Hornberger, J. Electronic health records: a guide for clinicians and administrators. JAMA 301, 110–110 (2009).
Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118 (2017).
Kermany, D. S. et al. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172, 1122–1131 (2018).
Gulshan, V. et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316, 2402–2410 (2016).
Erickson, B. J., Korfiatis, P., Akkus, Z. & Kline, T. L. Machine learning for medical imaging. Radiographics 37, 505–515 (2017).
Wang, F., Zhang, P., Qian, B., Wang, X. & Davidson, I. Clinical risk prediction with multilinear sparse logistic regression. In Proc. 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.145–154 (2014).
Turchin, A. et al. Using regular expressions to abstract blood pressure and treatment intensification information from the text of physician notes. J. Am. Med. Inform. Assoc. 13, 691–695 (2006).
Halevy, A., Norvig, P. & Pereira, F. The unreasonable effectiveness of data. IEEE Intelligent Systems 24, 8–12 (2009).
Banko, M. & Brill, E. Scaling to very very large corpora for natural language disambiguation. In Proc. 39th Annual Meeting Association for Computational Linguistics. 26–33 (Association for Computational Linguistics, Stroudsburg, 2001).
Tsui, B. Y., et al. Creating a scalable deep learning based named entity recognition model for biomedical textual data by repurposing biosample free-text annotations. Preprint at https://www.biorxiv.org/content/biorxiv/early/2018/09/12/414136.full.pdf (2018).
Rajkomar, A. et al. Scalable and accurate deep learning with electronic health records. NPJ Digital Medicine 1, 18 (2018).
Wilkinson, M. D. et al. Comment: the fair guiding principles for scientific data management and stewardship. Sci. Data 3, 160018 (2016).
Liang, Y., Chen, Z., Huang, X. & Zeng, L. Analysis of the disease spectrum of hospitalized children in guangdong province. Chin. Med. J. (Engl) 1, 414–418 (2013).
WHO. International Statistical Classification of Diseases and Related Health Problems. (World Health Organization, 2004).
English–Chinese Medical Dictionary (英汉医学大词典) (Shanghai Scientific and Technical Publishers (上海科学技术出版社), 2015).
Lindberg, D. A. B., Humphreys, B. L. & Mccray, A. T. The unified medical language system. Methods Inf. Med. 32, 281–291 (1993).
Tweedie, F. J., Singh, S. & Holmes, D. I. Neural network applications in stylometry: the federalist papers. Computers and the Humanities 30, 1–10 (1996).
Luong, M.-T., Pham, H. & Manning, C. D. Effective approaches to attention-based neural machine translation. Preprint at https://arxiv.org/abs/1508.04025 (2015).
Lipton, Z.C., Kale, D.C. & Wetzel, R.C. Phenotyping of clinical time series with LSTM recurrent neural networks. Preprint at https://arxiv.org/pdf/1510.07641.pdf (2015).
Peng, X.B., Andrychowicz, M., Zaremba, W. & Abbeel, P. Sim-to-real transfer of robotic control with dynamics randomization. IEEE International Conference on Robotics and Automation (ICRA) 3803–3810 (2018).
Graves, A. & Schmidhuber, J. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Networks 18, 602–610 (2005).
Yeung, K.Y. & Ruzzo, W.L. Details of the adjusted rand index and clustering algorithms supplement to the paper ‘an empirical study on Principal Component Analysis for clustering gene expression data. Available at http://faculty.washington.edu/kayee/pca/supp.pdf (2011).
Acknowledgements
This study was funded by the National Key Research and Development Program of China (2017YFC1104600 to H.L.), National Natural Science Foundation of China (81771629 to H.X. and 81700882 to J.X.), Guangzhou Women and Children’s Medical Center, Guangzhou Regenerative Medicine and Health Guangdong Laboratory (Innovation and Startup Talents Program 2018GZR031001 to L.Z. and R.H.).
Author information
Authors and Affiliations
Contributions
H.L., B.T., H.N., W.C., S.L.B., G. Liu, D.S.K., X. S., C.C.S.V., P.T., H.S., J.C., L. H., J.Z., L.Z., R.H., S.H., G. Li, P.L., X.Z., Z.Z., L.P., H.C., R.L., S.L., Y.C., S.T., H.Y., X.H., W. He, W.L., Q.Z., J.J., W.Y., J.G., W.O., Y. Deng, Q.H., B. Wang, C.Y., Y.L., S.Z., Y. Duan, R.Z., S.G., C.L.Z., O.L., E.D.Z., G.K., X.W., C.W., N.N., J.X., W.X., B. Wang, W.W., J.L., B.P., C.B., D.X., W. He, S.H., Y.Z., W. Haw, M.G., A.T., C.-N.H., H.C., L.Z., H.X. and K.Z. collected and analyzed the data. X.H. and K.Z. conceived the project. K.Z., S.L.B., B.T., H.L., and H.X. wrote the manuscript. All authors discussed the results and reviewed the manuscript.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended data
Extended Data 1 Unsupervised clustering of NLP extracted textual features from pediatric diseases.
The diagnostic system analyzed the EHRs in the absence of a defined classification system. This grouping structure reflects the detection of trends in clinical features without pre-defined labeling or human input. The clustered blocks are marked with the boxes with grey lines.
Extended Data 2 Design of the natural language processing information extraction model.
Segmented sentences from the raw text of the EHR were embedded using word2vec. The LSTM model then generated the structured records in a query–answer format. This schematic illustrates the process using the free-text ‘lesion in the upper left lobe of patient’s lung’ as an example.
Supplementary Information
Supplementary Tables
Supplementary Tables 1–9
Rights and permissions
About this article
Cite this article
Liang, H., Tsui, B.Y., Ni, H. et al. Evaluation and accurate diagnoses of pediatric diseases using artificial intelligence. Nat Med 25, 433–438 (2019). https://doi.org/10.1038/s41591-018-0335-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41591-018-0335-9
This article is cited by
-
Engaging children and young people on the potential role of artificial intelligence in medicine
Pediatric Research (2023)
-
Integrating structured and unstructured data for timely prediction of bloodstream infection among children
Pediatric Research (2023)
-
Artificial intelligence-based clinical decision support in pediatrics
Pediatric Research (2023)
-
Real-time artificial intelligence assisted insulin dosage titration system for glucose control in type 2 diabetic patients: a proof of concept study
Current Medicine (2023)
-
Role of artificial intelligence–guided esophagogastroduodenoscopy in assessing the procedural completeness and quality
Indian Journal of Gastroenterology (2023)
