Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Quantitative analysis of abnormalities in gynecologic cytopathology with deep learning

Laboratory Investigation (2021)Cite this article

Subjects

Abstract

Cervical cancer is one of the most frequent cancers in women worldwide, yet the early detection and treatment of lesions via regular cervical screening have led to a drastic reduction in the mortality rate. However, the routine examination of screening as a regular health checkup of women is characterized as time-consuming and labor-intensive, while there is lack of characteristic phenotypic profile and quantitative analysis. In this research, over the analysis of a privately collected and manually annotated dataset of 130 cytological whole-slide images, the authors proposed a deep-learning diagnostic system to localize, grade, and quantify squamous cell abnormalities. The system can distinguish abnormalities at the morphology level, namely atypical squamous cells of undetermined significance, low-grade squamous intraepithelial lesion, high-grade squamous intraepithelial lesion, and squamous cell carcinoma, as well as differential phenotypes of normal cells. The case study covered 51 positive and 79 negative digital gynecologic cytology slides collected from 2016 to 2018. Our automatic diagnostic system demonstrated its sensitivity of 100% at slide-level abnormality prediction, with the confirmation with three pathologists who performed slide-level diagnosis and training sample annotations. In the cellular-level classification, we yielded an accuracy of 94.5% in the binary classification between normality and abnormality, and the AUC was above 85% for each subtype of epithelial abnormality. Although the final confirmation from pathologists is often a must, empirically, computer-aided methods are capable of the effective extraction, interpretation, and quantification of morphological features, while also making it more objective and reproducible.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

Rent or Buy

All prices are NET prices.

Fig. 1: The overall architecture of the proposed computer-aided cytology image diagnostic system.
Fig. 2: A couple of example image patches cropped from the collected slides that were used as training input in the deep-learning system.
Fig. 3: Performance of the 12-category classification in cervical cells.
Fig. 4: An example of image patch of 2,500 × 2,500 pixels cropped from a whole-slide image.
Fig. 5: The automatic quantitative profile of abnormal cells in whole slide images.
Fig. 6: Quantitative analysis on nuclear pixels of the annotated multiple-class cells at the magnification rate of 40×.

References

  1. 1.

    Arbyn M, Weiderpass E, Bruni L, de Sanjosé S, Saraiya M, Ferlay J, et al. Estimates of incidence and mortality of cervical cancer in 2018: a worldwide analysis. Lancet Glob Health. 2020;8:e191–203.

    Article  Google Scholar 

  2. 2.

    Nandini NM, Nandish SM, Pallavi P, Akshatha SK, Chandrashekhar AP, Anjali S, et al. Manual liquid based cytology in primary screening for cervical cancer—a cost effective preposition for scarce resource settings. Asian Pac J Cancer Prev. 2012;13:3645–51.

    CAS  Article  Google Scholar 

  3. 3.

    Gibb RK, Martens MG. The impact of liquid-based cytology in decreasing the incidence of cervical cancer. Rev Obstet Gynecol. 2011;4:S2.

    PubMed  PubMed Central  Google Scholar 

  4. 4.

    Wilbur DC. Digital cytology: current state of the art and prospects for the future. Acta Cytol. 2011;55:227–38.

    CAS  Article  Google Scholar 

  5. 5.

    Khalbuss WE, Pantanowitz L, Parwani AV. Digital imaging in cytopathology. Patholog Res Int. 2011;2011:1–10.

    Article  Google Scholar 

  6. 6.

    Cibas ES, Ducatman BS. Cytology E-Book: diagnostic principles and clinical correlates. 4th ed.  Elsevier Health Sci, Canda. 2013.

  7. 7.

    Nayar R, Wilbur DC. The Bethesda system for reporting cervical cytology: definitions, criteria, and explanatory notes. 3rd ed. Springer, Cham. 2015.

  8. 8.

    Carozzi F, Negri G, Sani C Molecular Cytology Applications on Gynecological Cytology. Springer, Cham. 2018;127–49.

  9. 9.

    Campanella G, Hanna MG, Geneslaw L, Miraflor AP, Silva VWK, Busam KJ, et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat Med. 2019;25:1301–9.

    CAS  Article  Google Scholar 

  10. 10.

    Kather JN, Krisam J, Charoentong P, Luedde T, Herpel E, Weis CA, et al. Predicting survival from colorectal cancer histology slides using deep learning: a retrospective multicenter study. PLoS Med. 2019;16:e1002730.

    Article  Google Scholar 

  11. 11.

    Kather JN, Pearson AT, Halama N, Jager D, Krause J, Loosen SH, et al. Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nat Med. 2019;25:1054–6.

    CAS  Article  Google Scholar 

  12. 12.

    Bejnordi BE, Veta M, Van Diest PJ, Van Ginneken B, Karssemeijer N, Litjens GJS, et al. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA. 2017;318:2199–210.

    Article  Google Scholar 

  13. 13.

    Coudray N, Ocampo PS, Sakellaropoulos T, Narula N, Snuderl M, Fenyo D, et al. Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning. Nature Med. 2018;24:1559–67.

    CAS  Article  Google Scholar 

  14. 14.

    Zhang L, Lu L, Nogues I, Summers RM, Liu S, Yao J. DeepPap: deep convolutional networks for cervical cell classification. IEEE J Biomed Health Inform. 2017;21:1633–43.

    Article  Google Scholar 

  15. 15.

    Song Y, Zhang L, Chen S, Ni D, Lei B, Wang T. Accurate segmentation of cervical cytoplasm and nuclei based on multiscale convolutional network and graph partitioning. IEEE Trans Biomed Eng. 2015;62:2421–33.

    Article  Google Scholar 

  16. 16.

    Byju NB, Sujathan VK, Malm P, Kumar RR. A fast and reliable approach to cell nuclei segmentation in pap stained cervical smears. CSI Trans on ICT. 2013;1:309–15.

    Article  Google Scholar 

  17. 17.

    Volerman A, Cifu A. Cervical cancer screening. JAMA. 2014;312:2279–80.

    CAS  Article  Google Scholar 

  18. 18.

    Song Y, Tan EL, Jiang X, Cheng JZ, Ni D, Chen S, et al. Accurate cervical cell segmentation from overlapping clumps in pap smear images. IEEE Trans Med Imaging. 2016;36:288–300.

    Article  Google Scholar 

  19. 19.

    Kuko M, Pourhomayoun M. Single and clustered cervical cell classification with ensemble and deep learning methods. Inf Syst Front. 2020;22:1039–51.

    Article  Google Scholar 

  20. 20.

    Tareef A, Song Y, Cai W, Huang H, Chang H, Wang Y, et al. Automatic segmentation of overlapping cervical smear cells based on local distinctive features and guided shape deformation. Neurocomputing. 2017;221:94–107.

    Article  Google Scholar 

  21. 21.

    Zhou Y, Chen H, Xu J, Dou Q, Heng PA. Irnet: instance relation network for overlapping cervical cell segmentation. MICCAI. 2019;LNCS(11764):640–8.

  22. 22.

    Bao H, Sun X, Zhang Y, Pang B, Li H, Zhou L, et al. The artificial intelligence-assisted cytology diagnostic system in large-scale cervical cancer screening: a population-based cohort study of 0.7 million women. Cancer Med. 2020;9:6896–906.

    Article  Google Scholar 

  23. 23.

    Cruzroa A, Gilmore H, Basavanhally A, Feldman M, Ganesan S, Shih N, et al. High-throughput adaptive sampling for whole-slide histopathology image analysis (HASHI) via convolutional neural networks: application to invasive breast cancer detection. PLoS ONE. 2018;13:e0196828.

    Article  Google Scholar 

  24. 24.

    Hou L, Samaras D, Kurc TM, Gao Y, Davis JE, Saltz JH. Patch-based convolutional neural network for whole slide tissue image classification. CVPR. 2016;2424–33.

  25. 25.

    Aeffner F, Wilson K, Martin NT, Black JC, Hendriks CLL, Bolon B, et al. The gold standard paradox in digital image analysis: manual versus automated scoring as ground truth. Arch Pathol Lab Med. 2017;141:1267–75.

    Article  Google Scholar 

  26. 26.

    Rączkowski Ł, Możejko M, Zambonelli J, Szczurek E. ARA: accurate, reliable and active histopathological image classification framework with Bayesian deep learning. Sci. Reports. 2019;9:1–12.

    Google Scholar 

  27. 27.

    Abels E, Pantanowitz L, Aeffner F, Zarella MD, van der Laak J, Bui MM, et al. Computational pathology definitions, best practices, and recommendations for regulatory guidance: a white paper from the digital pathology association. J Pathol. 2019;249:286–94.

    Article  Google Scholar 

  28. 28.

    Koliopoulos G, Nyaga VN, Santesso N, Bryant A, Martin-Hirsch PP, Mustafa RA, et al. Cytology versus HPV testing for cervical cancer screening in the general population. Cochrane Database Syst Rev. 2017;8:1–105.

    Google Scholar 

  29. 29.

    He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. CVPR. 2016;770–8.

  30. 30.

    Ronneberger OP, Fischer Brox T. U-net: convolutional networks for biomedical image segmentation. MICCAI. 2015;LNCS(9351):234–41.

  31. 31.

    Yosinski J, Clune J, Bengio Y, Lipson H. How transferable are features in deep neural networks? Adv Neural Inf Process Syst. 2014;2:3320–8.

    Google Scholar 

  32. 32.

    Shen Y, Ke J. A deformable CRF model for histopathology whole-slide image classification. MICCAI. 2020;LNCS(12265):500–8.

  33. 33.

    Tosun AB, Pullara F, Becich MJ, Taylor DL, Chennubhotla SC, Fine JL. HistoMaprTM: An explainable AI (xAI) platform for computational pathology solutions. Patent App. 2020;LNCS(12090):204–7.

  34. 34.

    Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J. Unet++: a nested u-net architecture for medical image segmentation. Deep Learn Med Image Anal Multimodal Learn Clin Decis Support. 2018;11045:3–11.

    Article  Google Scholar 

Download references

Author information

Affiliations

  1. Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, China

    Jing Ke & Xiaoyao Liang

  2. School of Computer Science and Engineering, University of New South Wales, Sydney, NSW, Australia

    Jing Ke

  3. School of Mathematical Sciences, Shanghai Jiao Tong University, Shanghai, China

    Yiqing Shen

  4. School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai, China

    Yizhou Lu

  5. School of Information, University of Michigan, Ann Arbor, MI, USA

    Junwei Deng

  6. Department of Obstetrics and Gynecology, Columbia University, New York, NY, USA

    Jason D. Wright

  7. Department of Pathology, Shanghai Tongshu Medical Laboratory Co.Ltd, Shanghai, China

    Yan Zhang

  8. Department of Endocrinology and Metabolism, Shanghai Jiao Tong University Affiliated Sixth People’s Hospital, Shanghai Clinical Center for Diabetes, Shanghai, China

    Qin Huang & Fusong Jiang

  9. Quantitative Imaging, Data61 CSIRO, Sydney, NSW, Australia

    Dadong Wang

  10. Department of Micro-Nano Electronics, Shanghai Jiao Tong University, Shanghai, China

    Naifeng Jing

  11. Biren Research, Shanghai, China, Shanghai, China

    Xiaoyao Liang

Authors
  1. Jing Ke
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yiqing Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yizhou Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Junwei Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jason D. Wright
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Qin Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Dadong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Naifeng Jing
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Xiaoyao Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Fusong Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

JK conceived and designed the system, performed the analysis, interpreted the results, and wrote the paper. YS constructed the mathematical models. YS and YL worked on the figures and graphs. YL and JD implemented the system and profiled the experiment results. JDW, YZ, QH, and FJ made the pathological annotation and diagnosed the slides. NJ and DW suggested the overall architecture. XL and JK collected the data.

Corresponding author

Correspondence to Jing Ke.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethics approval

The data used in this study were collected from Shanxi Tumor Hospital, China. All experiments were conducted in accordance with the Ethical Guidelines for Shanxi Tumor Hospital. An ethics commitment from Shanxi Tumor Hospital granted this dataset to the corresponding author for this research on July 29, 2019, and the ethical approval ID was 201903.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ke, J., Shen, Y., Lu, Y. et al. Quantitative analysis of abnormalities in gynecologic cytopathology with deep learning. Lab Invest (2021). https://doi.org/10.1038/s41374-021-00537-1

Download citation

Search

Quick links