Accumulation of abnormal tau in neurofibrillary tangles (NFT) occurs in Alzheimer disease (AD) and a spectrum of tauopathies. These tauopathies have diverse and overlapping morphological phenotypes that obscure classification and quantitative assessments. Recently, powerful machine learning-based approaches have emerged, allowing the recognition and quantification of pathological changes from digital images. Here, we applied deep learning to the neuropathological assessment of NFT in postmortem human brain tissue to develop a classifier capable of recognizing and quantifying tau burden. The histopathological material was derived from 22 autopsy brains from patients with tauopathies. We used a custom web-based informatics platform integrated with an in-house information management system to manage whole slide images (WSI) and human expert annotations as ground truth. We utilized fully annotated regions to train a deep learning fully convolutional neural network (FCN) implemented in PyTorch against the human expert annotations. We found that the deep learning framework is capable of identifying and quantifying NFT with a range of staining intensities and diverse morphologies. With our FCN model, we achieved high precision and recall in naive WSI semantic segmentation, correctly identifying tangle objects using a SegNet model trained for 200 epochs. Our FCN is efficient and well suited for the practical application of WSIs with average processing times of 45 min per WSI per GPU, enabling reliable and reproducible large-scale detection of tangles. We measured performance on test data of 50 pre-annotated regions on eight naive WSI across various tauopathies, resulting in the recall, precision, and an F1 score of 0.92, 0.72, and 0.81, respectively. Machine learning is a useful tool for complex pathological assessment of AD and other tauopathies. Using deep learning classifiers, we have the potential to integrate cell- and region-specific annotations with clinical, genetic, and molecular data, providing unbiased data for clinicopathological correlations that will enhance our knowledge of the neurodegeneration.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.


All prices are NET prices.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.


  1. 1.

    Hoglinger, GU, G Respondek and GG Kovacs. New classification of tauopathies. Rev Neurol (Paris) 2018. https://doi.org/10.1016/j.neurol.2018.07.001.

  2. 2.

    Lebouvier T, Pasquier F, Buee L. Update on tauopathies. Curr Opin Neurol. 2017;30:589–98.

  3. 3.

    Morris M, Maeda S, Vossel K, Mucke L. The many faces of tau. Neuron. 2011;70:410–26.

  4. 4.

    Höglinger GU, Melhem NM, Dickson DW, et al. Identification of common variants influencing risk of the tauopathy progressive supranuclear palsy. Nat Genet. 2011;43:699–705.

  5. 5.

    Bennett DA, Schneider JA, Buchman AS, Barnes LL, et al. Overview and findings from the rush memory and aging project. Curr Alzheimer Res. 2012;9:646–63.

  6. 6.

    Ghetti B, Oblak AL, Boeve BF, et al. Invited review: Frontotemporal dementia caused by microtubule-associated protein tau gene (MAPT) mutations: a chameleon for neuropathology and neuroimaging. Neuropathol Appl Neurobiol. 2015;41:24–46.

  7. 7.

    Cox PA, Davis DA, Mash DC, et al. Dietary exposure to an environmental toxin triggers neurofibrillary tangles and amyloid deposits in the brain. Proc R Soc B. 2016;283:20152397

  8. 8.

    McKee AC, Cairns NJ, Dickson DW, et al. The first NINDS NIBIB consensus meeting to define neuropathological criteria for the diagnosis of chronic traumatic encephalopathy. Acta Neuropathol. 2016;131:75–86.

  9. 9.

    Ferrer I, Lopez-Gonzalez I, Carmona M, et al. Glial and neuronal tau pathology in tauopathies: characterization of disease-specific phenotypes and tau pathology progression. J Neuropathol Exp Neurol. 2013;73:81–97.

  10. 10.

    Kahlson MA, Colodner KJ. Glial tau pathology in tauopathies: functional consequences. J Exp Neurosci. 2015;9(Suppl 2):43–50.

  11. 11.

    Kovacs GG. Tauopathies (Kovacs GG, Alafuzoff I, eds) Vol. 145, 355–68, Handb Clin Neurol., Elsevier, 2017.

  12. 12.

    Murray ME, Kouri N, Lin W-L, et al. Clinicopathologic assessment and imaging of tauopathies in neurodegenerative dementias. Alzheimer’s Res Ther. 2014;6:1.

  13. 13.

    Braak H, Braak E. Neuropathological staging of Alzheimer-related changes. Acta Neuropathol. 1991;82:239–59.

  14. 14.

    Jellinger KA. Different patterns of hippocampal tau pathology in Alzheimer’s disease and PART. Acta Neuropathol. 2018. https://doi.org/10.1007/s00401-018-1894-z.

  15. 15.

    Salloway S, Sperling R. Understanding conflicting neuropathological findings in patients clinically diagnosed as having Alzheimer dementia. JAMA Neurol. 2015;72:1106–8.

  16. 16.

    Al-Janabi S, Huisman A, Van Diest PJ. Digital pathology: current status and future perspectives. Histopathology. 2012;61:1–9.

  17. 17.

    Litjens G, Kooi T, Bejnordi BE. et al. A survey on deep learning in medical image analysis. Med Image Anal. 2017;42:60–88.

  18. 18.

    Stead WW. Clinical implications and challenges of artificial intelligence and deep learning. JAMA. 2018;320:1107–8.

  19. 19.

    Hinton G. Deep learning—a technology with the potential to transform health care. JAMA. 2018;320:1101–2.

  20. 20.

    LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;28:436–44.

  21. 21.

    Naylor CD. On the prospects for a (deep) learning health care system. JAMA. 2018;320:1099–1100.

  22. 22.

    Aresta G, Araújo T, Kwok S, et al. BACH: Grand Challenge on Breast Cancer Histology Images. arXiv Prepr. 2018;arXiv:1808.04277.

  23. 23.

    Donovan MJ, Fernandez G, Scott R, et al. Development and validation of a novel automated Gleason grade and molecular profile that define a highly predictive prostate cancer progression algorithm-based test. Prostate Cancer Prostatic Dis. 2018. https://doi.org/10.1038/s41391-018-0067-4

  24. 24.

    Crary JF, Trojanowski TQ, Schneider JA, et al. Primary age-related tauopathy (PART): a common pathology associated with human aging. Acta Neuropathol. 2014;128:755–66.

  25. 25.

    Scott R, Khan FM, Zeineh J, Donovan M, Fernandez G. Gland ring morphometry for prostate cancer prognosis in multispectral immunofluorescence images. In: International Conference on Medical Image Computing and Computer-Assisted Intervention 2014 LNCS 8673. Cham: Springer; 2014. p. 585–92.

  26. 26.

    Badrinarayanan V, Kendall A, Cipolla R. SegNet:a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell. 2017;39:2481–95.

  27. 27.

    Paszke A, Gross S, Chintala S, Chanan G, et al. Automatic differentiation. In: PyTorch 2017. Long Beach: NIPS-W; 2017.

  28. 28.

    Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542:115–8.

  29. 29.

    Gulshan V, Peng L, Coram M, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316:2402–10.

Download references


The authors would like to acknowledge NIH grants R01AG054008 (JFC), R01NS095252 (JFC), R01AG062348 (ACM/JFC), RF1AG060961 (JFC), F32AG056098 (KF), Department of Defense W81XWH-13-MRPRA-CSRA, the Tau Consortium (Rainwater Charitable Trust), and the Alzheimer’s Association (NIRG-15-363188). The first author was also supported by a Career Development Award funded by NIH-NOA 3P50AG005138 (MS). We thank Jill Gregory for the illustration. The authors also would like to thank Ping Shang, HT, Jeff Harris, HTL, and Chan Foong, MS, for technical assistance, and Javed and Shahnaz Iqbal Family Trust for the generous donation.

Author information


  1. Department of Pathology, Icahn School of Medicine at Mount Sinai, New York, NY, 10029, USA

    • Maxim Signaevsky
    • , Marcel Prastawa
    • , Kurt Farrell
    • , Nabil Tabish
    • , Elena Baldwin
    • , Natalia Han
    • , Megan A. Iida
    • , John Koll
    • , Clare Bryce
    • , Dushyant Purohit
    • , Russell Hanson
    • , Michael J. Donovan
    • , Carlos Cordon-Cardo
    • , Jack Zeineh
    • , Gerardo Fernandez
    •  & John F. Crary
  2. Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York, NY, 10029, USA

    • Maxim Signaevsky
    • , Kurt Farrell
    • , Nabil Tabish
    • , Elena Baldwin
    • , Natalia Han
    • , Megan A. Iida
    • , Clare Bryce
    • , Dushyant Purohit
    • , Russell Hanson
    •  & John F. Crary
  3. Ronald M. Loeb Center for Alzheimer’s Disease, Icahn School of Medicine at Mount Sinai, New York, NY, 10029, USA

    • Maxim Signaevsky
    • , Kurt Farrell
    • , Nabil Tabish
    • , Elena Baldwin
    • , Natalia Han
    • , Megan A. Iida
    • , Clare Bryce
    • , Russell Hanson
    •  & John F. Crary
  4. Center for Computational and Systems Pathology, Icahn School of Medicine at Mount Sinai, New York, NY, 10025, USA

    • Marcel Prastawa
    • , John Koll
    • , Michael J. Donovan
    • , Carlos Cordon-Cardo
    • , Jack Zeineh
    •  & Gerardo Fernandez
  5. Departments of Psychiatry and Neuroscience, Icahn School of Medicine at Mount Sinai, New York, NY, 10029, USA

    • Dushyant Purohit
    •  & Vahram Haroutunian
  6. J. James Peters VA Medical Center, Bronx, NY, USA

    • Vahram Haroutunian
  7. Department of Neurology, Boston University School of Medicine, Boston, MA, 02118, USA

    • Ann C. McKee
  8. Department of Pathology, Boston University School of Medicine, Boston, MA, 02118, USA

    • Ann C. McKee
    •  & Thor D. Stein
  9. Alzheimer’s Disease Center, CTE Program, Boston University School of Medicine, Boston, MA, 02118, USA

    • Ann C. McKee
    •  & Thor D. Stein
  10. Mental Illness Research, Education and Clinical Center, James J. Peters VA Boston Healthcare System, Boston, MA, 02130, USA

    • Ann C. McKee
    •  & Thor D. Stein
  11. Department of Veteran Affairs Medical Center, Bedford, MA, 01730, USA

    • Ann C. McKee
    •  & Thor D. Stein
  12. Neuropathology Laboratory, Department of Pathology, UT Southwestern Medical Center, Dallas, TX, 75390, USA

    • Charles L. White III
    • , Jamie Walker
    •  & Timothy E. Richardson


  1. Search for Maxim Signaevsky in:

  2. Search for Marcel Prastawa in:

  3. Search for Kurt Farrell in:

  4. Search for Nabil Tabish in:

  5. Search for Elena Baldwin in:

  6. Search for Natalia Han in:

  7. Search for Megan A. Iida in:

  8. Search for John Koll in:

  9. Search for Clare Bryce in:

  10. Search for Dushyant Purohit in:

  11. Search for Vahram Haroutunian in:

  12. Search for Ann C. McKee in:

  13. Search for Thor D. Stein in:

  14. Search for Charles L. White III in:

  15. Search for Jamie Walker in:

  16. Search for Timothy E. Richardson in:

  17. Search for Russell Hanson in:

  18. Search for Michael J. Donovan in:

  19. Search for Carlos Cordon-Cardo in:

  20. Search for Jack Zeineh in:

  21. Search for Gerardo Fernandez in:

  22. Search for John F. Crary in:

Conflict of interest

The authors declare that they have no conflict of interest.

Corresponding author

Correspondence to John F. Crary.

About this article

Publication history