Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Review Article
  • Published:

Automated molecular-image cytometry and analysis in modern oncology

Abstract

Diagnostic methods for initial diagnosis and patient stratification for treatment are key to modern oncology, but many challenges remain. In developed countries, advances in early diagnosis and therapeutics have led to challenges in the sampling of sub-centimetre lesions, with repeat biopsies straining accuracy and throughput of pathological assessment. Conversely, low-income and middle-income countries face extremely limited pathology and imaging resources, large caseloads, convoluted and inefficient workflows, and a lack of specialists. Advances in material sciences, chemistry, engineering and artificial intelligence have led to the introduction of a new class of affordable image cytometers that enable automated cell phenotyping, with ongoing clinical testing indicating that these systems can alleviate existing bottlenecks and improve diagnostic efficiency. Ultimately, these diagnostic methods are likely to surpass current pathology approaches on the basis of the richness of molecular measurements and the fact that they require only scant cellular material, rather than tissue sections. As these methods can be miniaturized and are low-power, they can also be used in point-of-care settings. In this Review, we focus on new devices and approaches for the integrated analysis of scant cancer samples, particularly those obtained by fine-needle aspiration.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Overview of automated molecular-image cytometry.
Fig. 2: Cyclic labelling technologies for multiplexed cancer-marker and host-cell-marker assessment.
Fig. 3: LDIH.
Fig. 4: FPM.
Fig. 5: Miniaturized fluorescent cytometers.
Fig. 6: Machine learning in imaging analyses.

Similar content being viewed by others

References

  1. Weissleder, R., Schwaiger, M. C., Gambhir, S. S. & Hricak, H. Imaging approaches to optimize molecular therapies. Sci. Transl Med. 8, 355ps16 (2016).

    Article  CAS  Google Scholar 

  2. Frenk, N. E. et al. High-content biopsies facilitate molecular analyses and do not increase complication rates in patients with advanced solid tumors. JCO Precis. Oncol. 1, 1–9 (2017).

    Google Scholar 

  3. Capitanio, A., Dina, R. E. & Treanor, D. Digital cytology: A short review of technical and methodological approaches and applications. Cytopathology 29, 317–325 (2018).

    Article  CAS  Google Scholar 

  4. Asthana, V. et al. An inexpensive, customizable microscopy system for the automated quantification and characterization of multiple adherent cell types. PeerJ 6, e4937 (2018).

    Article  CAS  Google Scholar 

  5. Balsam, J., Bruck, H. A. & Rasooly, A. Mobile flow cytometer for mHealth. Methods Mol. Biol. 1256, 139–153 (2015).

    Article  CAS  Google Scholar 

  6. Molnár, B. et al. Circulating cell-free nucleic acids as biomarkers in colorectal cancer screening and diagnosis — an update. Expert Rev. Mol. Diagn. 19, 477–498 (2019).

    Article  CAS  Google Scholar 

  7. Shao, H. et al. New technologies for analysis of extracellular vesicles. Chem. Rev. 118, 1917–1950 (2018).

    Article  CAS  Google Scholar 

  8. Sandlin, R. D. et al. Ultra-fast vitrification of patient-derived circulating tumor cell lines. PLOS ONE 13, e0192734 (2018).

    Article  CAS  Google Scholar 

  9. Sarioglu, A. F. et al. A microfluidic device for label-free, physical capture of circulating tumor cell clusters. Nat. Methods 12, 685–691 (2015).

    Article  CAS  Google Scholar 

  10. Cristofanilli, M. et al. The clinical use of circulating tumor cells (CTCs) enumeration for staging of metastatic breast cancer (MBC): International expert consensus paper. Crit. Rev. Oncol. Hematol. 134, 39–45 (2019).

    Article  Google Scholar 

  11. No authors listed. First comprehensive companion diagnostic OK’d. Cancer Discov. 8, OF4 (2018).

    Google Scholar 

  12. Van Hoeck, A., Tjoonk, N. H., van Boxtel, R. & Cuppen, E. Portrait of a cancer: mutational signature analyses for cancer diagnostics. BMC Cancer 19, 457 (2019).

    Article  Google Scholar 

  13. Hannouf, M. B. et al. Cost-effectiveness analysis of multigene expression profiling assays to guide adjuvant therapy decisions in women with invasive early-stage breast cancer. Pharmacogenomics J. 20, 27–46 (2020).

    Article  CAS  Google Scholar 

  14. Saha, M., Mukherjee, R. & Chakraborty, C. Computer-aided diagnosis of breast cancer using cytological images: a systematic review. Tissue Cell 48, 461–474 (2016).

    Article  Google Scholar 

  15. Filipczuk, P., Fevens, T., Krzyzak, A. & Monczak, R. Computer-aided breast cancer diagnosis based on the analysis of cytological images of fine needle biopsies. IEEE Trans. Med. Imaging 32, 2169–2178 (2013).

    Article  Google Scholar 

  16. Dey, P., Logasundaram, R. & Joshi, K. Artificial neural network in diagnosis of lobular carcinoma of breast in fine-needle aspiration cytology. Diagn. Cytopathol. 41, 102–106 (2013).

    Article  Google Scholar 

  17. Landau, M. A. & Pantanowitz, L. Artificial intelligence in cytopathology: a review of the literature and overview of commercial landscape. J. Am. Soc. Cytopathol. 8, 230–241 (2019).

    Article  Google Scholar 

  18. Stephens, L., Bevins, N. J., Bengtsson, H. I. & Broome, H. E. Comparison of different small clinical hematology laboratory configurations with focus on remote smear imaging. Arch. Pathol. Lab. Med. 143, 1234–1245 (2019).

    Article  Google Scholar 

  19. Andrade, A. R. et al. Recent computational methods for white blood cell nuclei segmentation: a comparative study. Comput. Methods Prog. Biomed. 173, 1–14 (2019).

    Article  Google Scholar 

  20. Prieto, S. P., Powless, A. J., Boice, J. W., Sharma, S. G. & Muldoon, T. J. Proflavine hemisulfate as a fluorescent contrast agent for point-of-care cytology. PLOS ONE 10, e0125598 (2015).

    Article  CAS  Google Scholar 

  21. Lin, J. R., Fallahi-Sichani, M. & Sorger, P. K. Highly multiplexed imaging of single cells using a high-throughput cyclic immunofluorescence method. Nat. Commun. 6, 8390 (2015).

    Article  CAS  Google Scholar 

  22. Gerdes, M. J. et al. Highly multiplexed single-cell analysis of formalin-fixed, paraffin-embedded cancer tissue. Proc. Natl Acad. Sci. USA 110, 11982–11987 (2013).

    Article  CAS  Google Scholar 

  23. Schubert, W. et al. Analyzing proteome topology and function by automated multidimensional fluorescence microscopy. Nat. Biotechnol. 24, 1270–1278 (2006).

    Article  CAS  Google Scholar 

  24. Giedt, R. J. et al. Single-cell barcode analysis provides a rapid readout of cellular signaling pathways in clinical specimens. Nat. Commun. 9, 4550 (2018).

    Article  CAS  Google Scholar 

  25. Ullal, A. V. et al. Cancer cell profiling by barcoding allows multiplexed protein analysis in fine-needle aspirates. Sci. Transl Med. 6, 219ra9 (2014).

    Article  CAS  Google Scholar 

  26. Agasti, S. S., Liong, M., Peterson, V. M., Lee, H. & Weissleder, R. Photocleavable DNA barcode–antibody conjugates allow sensitive and multiplexed protein analysis in single cells. J. Am. Chem. Soc. 134, 18499–18502 (2012).

    Article  CAS  Google Scholar 

  27. Kishi, J. Y. et al. SABER amplifies FISH: enhanced multiplexed imaging of RNA and DNA in cells and tissues. Nat. Methods 16, 533–544 (2019).

    Article  CAS  Google Scholar 

  28. De Wit, S. et al. Classification of cells in CTC-enriched samples by advanced image analysis. Cancers 10, 377 (2018).

    Article  CAS  Google Scholar 

  29. Haun, J. B. et al. Micro-NMR for rapid molecular analysis of human tumor samples. Sci. Transl Med. 3, 71ra16 (2011).

    Article  Google Scholar 

  30. Yang, K. S. et al. Multiparametric plasma EV profiling facilitates diagnosis of pancreatic malignancy. Sci. Transl Med. 9, eaal3226 (2017).

    Article  Google Scholar 

  31. Min, J. et al. Computational optics enables breast cancer profiling in point-of-care settings. ACS Nano 12, 9081–9090 (2018).

    Article  CAS  Google Scholar 

  32. Im, H. et al. Design and clinical validation of a point-of-care device for the diagnosis of lymphoma via contrast-enhanced microholography and machine learning. Nat. Biomed. Eng. 2, 666–674 (2018).

    Article  Google Scholar 

  33. Ghazani, A. A. et al. Molecular characterization of scant lung tumor cells using iron-oxide nanoparticles and micro-nuclear magnetic resonance. Nanomedicine 10, 661–668 (2014).

    Article  CAS  Google Scholar 

  34. Peterson, V. M. et al. Ascites analysis by a microfluidic chip allows tumor-cell profiling. Proc. Natl Acad. Sci. USA 110, E4978–E4986 (2013).

    Article  CAS  Google Scholar 

  35. Marquard, A. N., Carlson, J. C. T. & Weissleder, R. Glass chemistry to analyze human cells under adverse conditions. ACS Omega 4, 11515–11521 (2019).

    Article  CAS  Google Scholar 

  36. Lohmann, A. W., Dorsch, R. G., Mendlovic, D., Ferreira, C. & Zalevsky, Z. Space–bandwidth product of optical signals and systems. J. Opt. Soc. Am. A 13, 470–473 (1996).

    Article  Google Scholar 

  37. Van Es, S. L. et al. Constant quest for quality: digital cytopathology. J. Pathol. Inform. 9, 13 (2018).

    Article  Google Scholar 

  38. Hu, B., Bolus, D. & Brown, J. Q. Improved contrast in inverted selective plane illumination microscopy of thick tissues using confocal detection and structured illumination. Biomed. Opt. Express 8, 5546–5559 (2017).

    Article  CAS  Google Scholar 

  39. Wang, M. et al. Gigapixel surface imaging of radical prostatectomy specimens for comprehensive detection of cancer-positive surgical margins using structured illumination microscopy. Sci. Rep. 6, 27419 (2016).

    Article  CAS  Google Scholar 

  40. Garcia-Sucerquia, J. et al. Digital in-line holographic microscopy. Appl. Opt. 45, 836–850 (2006).

    Article  Google Scholar 

  41. Gurkan, U. A. et al. Miniaturized lensless imaging systems for cell and microorganism visualization in point-of-care testing. Biotechnol. J. 6, 138–149 (2011).

    Article  CAS  Google Scholar 

  42. Zheng, G., Lee, S. A., Antebi, Y., Elowitz, M. B. & Yang, C. The ePetri dish, an on-chip cell imaging platform based on subpixel perspective sweeping microscopy (SPSM). Proc. Natl Acad. Sci. USA 108, 16889–16894 (2011).

    Article  CAS  Google Scholar 

  43. Kim, S. B. et al. A cell-based biosensor for real-time detection of cardiotoxicity using lensfree imaging. Lab. Chip 11, 1801–1807 (2011).

    Article  CAS  Google Scholar 

  44. Greenbaum, A. et al. Imaging without lenses: achievements and remaining challenges of wide-field on-chip microscopy. Nat. Methods 9, 889–895 (2012).

    Article  CAS  Google Scholar 

  45. Im, H. et al. Digital diffraction analysis enables low-cost molecular diagnostics on a smartphone. Proc. Natl Acad. Sci. USA 112, 5613–5618 (2015).

    Article  CAS  Google Scholar 

  46. Rostykus, M., Soulez, F., Unser, M. & Moser, C. Compact in-line lensfree digital holographic microscope. Methods 136, 17–23 (2018).

    Article  CAS  Google Scholar 

  47. Rappaz, B. et al. Comparative study of human erythrocytes by digital holographic microscopy, confocal microscopy, and impedance volume analyzer. Cytometry A 73, 895–903 (2008).

    Article  Google Scholar 

  48. Seo, S. et al. High-throughput lens-free blood analysis on a chip. Anal. Chem. 82, 4621–4627 (2010).

    Article  CAS  Google Scholar 

  49. Mudanyali, O., Bishara, W. & Ozcan, A. Lensfree super-resolution holographic microscopy using wetting films on a chip. Opt. Express 19, 17378–17389 (2011).

    Article  CAS  Google Scholar 

  50. Lee, S. A. et al. Color capable sub-pixel resolving optofluidic microscope and its application to blood cell imaging for malaria diagnosis. PLOS ONE 6, e26127 (2011).

    Article  CAS  Google Scholar 

  51. Kim, S. J. et al. Deep transfer learning-based hologram classification for molecular diagnostics. Sci. Rep. 8, 17003 (2018).

    Article  CAS  Google Scholar 

  52. Rivenson, Y., Zhang, Y., Günaydın, H., Teng, D. & Ozcan, A. Phase recovery and holographic image reconstruction using deep learning in neural networks. Light Sci. Appl. 7, 17141 (2018).

    Article  CAS  Google Scholar 

  53. Greenbaum, A. & Ozcan, A. Maskless imaging of dense samples using pixel super-resolution based multi-height lensfree on-chip microscopy. Opt. Express 20, 3129–3143 (2012).

    Article  Google Scholar 

  54. Luo, W., Zhang, Y., Feizi, A., Göröcs, Z. & Ozcan, A. Pixel super-resolution using wavelength scanning. Light Sci. Appl. 5, e16060 (2016).

    Article  CAS  Google Scholar 

  55. Bao, P., Situ, G., Pedrini, G. & Osten, W. Lensless phase microscopy using phase retrieval with multiple illumination wavelengths. Appl. Opt. 51, 5486–5494 (2012).

    Article  Google Scholar 

  56. Ou, X., Horstmeyer, R., Zheng, G. & Yang, C. High numerical aperture Fourier ptychography: principle, implementation and characterization. Opt. Express 23, 3472–3491 (2015).

    Article  CAS  Google Scholar 

  57. Tian, L. et al. Computational illumination for high-speed in vitro Fourier ptychographic microscopy. Optica 2, 904–911 (2015).

    Article  CAS  Google Scholar 

  58. Nguyen, T., Xue, Y., Li, Y., Tian, L. & Nehmetallah, G. Deep learning approach for Fourier ptychography microscopy. Opt. Express 26, 26470–26484 (2018).

    Article  Google Scholar 

  59. Zheng, G., Horstmeyer, R. & Yang, C. Wide-field, high-resolution Fourier ptychographic microscopy. Nat. Photonics 7, 739–745 (2013).

    Article  CAS  Google Scholar 

  60. Horstmeyer, R., Chung, J., Ou, X., Zheng, G. & Yang, C. Diffraction tomography with Fourier ptychography. Optica 3, 827–835 (2016).

    Article  CAS  Google Scholar 

  61. Dong, S. et al. Aperture-scanning Fourier ptychography for 3D refocusing and super-resolution macroscopic imaging. Opt. Express 22, 13586–13599 (2014).

    Article  Google Scholar 

  62. Ghosh, K. K. et al. Miniaturized integration of a fluorescence microscope. Nat. Methods 8, 871–878 (2011).

    Article  CAS  Google Scholar 

  63. Liberti, W. A., Perkins, L. N., Leman, D. P. & Gardner, T. J. An open source, wireless capable miniature microscope system. J. Neural Eng. 14, 045001 (2017).

    Article  Google Scholar 

  64. Jacob, A. D. et al. A compact head-mounted endoscope for in vivo calcium imaging in freely behaving mice. Curr. Protoc. Neurosci. 84, e51 (2018).

    Article  Google Scholar 

  65. Aharoni, D. & Hoogland, T. M. Circuit investigations with open-source miniaturized microscopes: past, present and future. Front. Cell. Neurosci. 13, 141 (2019).

    Article  Google Scholar 

  66. Helmchen, F., Fee, M. S., Tank, D. W. & Denk, W. A miniature head-mounted two-photon microscope. High-resolution brain imaging in freely moving animals. Neuron 31, 903–912 (2001).

    Article  CAS  Google Scholar 

  67. Skocek, O. et al. High-speed volumetric imaging of neuronal activity in freely moving rodents. Nat. Methods 15, 429–432 (2018).

    Article  CAS  Google Scholar 

  68. Adams, J. K. et al. Single-frame 3D fluorescence microscopy with ultraminiature lensless FlatScope. Sci. Adv. 3, e1701548 (2017).

    Article  Google Scholar 

  69. Ah Lee, S., Ou, X., Lee, J. E. & Yang, C. Chip-scale fluorescence microscope based on a silo-filter complementary metal-oxide semiconductor image sensor. Opt. Lett. 38, 1817–1819 (2013).

    Article  CAS  Google Scholar 

  70. Almada, P. et al. Automating multimodal microscopy with NanoJ-Fluidics. Nat. Commun. 10, 1223 (2019).

    Article  CAS  Google Scholar 

  71. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).

    Article  CAS  Google Scholar 

  72. LeCun, Y., Bottou, L., Bengio, Y. & Haffner, P. Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998).

    Article  Google Scholar 

  73. Cao, C. et al. Deep learning and its applications in biomedicine. Genom. Proteom. Bioinform. 16, 17–32 (2018).

    Article  Google Scholar 

  74. Abadi, M. et al. TensorFlow: a system for large-scale machine learning. Preprint at arXiv https://arxiv.org/abs/1605.08695 (2016).

  75. Ronneberger, O., Fischer, P. & Brox, T. U-Net: convolutional networks for biomedical image segmentation. Med. Image Comput. Comput. Assist. Interv. 9351, 234–241 (2015).

    Google Scholar 

  76. Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K. & Yuille, A. L. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40, 834–848 (2018).

    Article  Google Scholar 

  77. Krizhevsky, A., Sutskever, I. & Hinton, G. E. ImageNet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 1, 1097–1105 (2012).

    Google Scholar 

  78. He, K., Zhang, X., Ren, S. & Sun, J. in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 770–778 (IEEE, 2016).

  79. Simonyan, K. & Zisserman, A. Very deep convolutional networks for large-scale image recognition. Preprint at arXiv https://arxiv.org/abs/1409.1556 (2014).

  80. Szegedy, C. et al. in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 1–9 (IEEE, 2015).

  81. Schindelin, J. et al. Fiji: an open-source platform for biological-image analysis. Nat. Methods 9, 676–682 (2012).

    Article  CAS  Google Scholar 

  82. Kamentsky, L. et al. Improved structure, function and compatibility for CellProfiler: modular high-throughput image analysis software. Bioinformatics 27, 1179–1180 (2011).

    Article  CAS  Google Scholar 

  83. Held, M. et al. CellCognition: time-resolved phenotype annotation in high-throughput live cell imaging. Nat. Methods 7, 747–754 (2010).

    Article  CAS  Google Scholar 

  84. Murphy, R. F. CellOrganizer: image-derived models of subcellular organization and protein distribution. Methods Cell Biol. 110, 179–193 (2012).

    Article  Google Scholar 

  85. Sommer, C., Straehle, C., Koethe, U. & Hamprecht, F. A. in Proc. IEEE Int. Symp. Biomed. Imaging 230–233 (IEEE, 2011).

  86. Kvilekval, K., Fedorov, D., Obara, B., Singh, A. & Manjunath, B. S. Bisque: a platform for bioimage analysis and management. Bioinformatics 26, 544–552 (2010).

    Article  CAS  Google Scholar 

  87. Thrall, M. J. Automated screening of Papanicolaou tests: a review of the literature. Diagn. Cytopathol. 47, 20–27 (2019).

    Article  Google Scholar 

  88. Kashyap, A., Jain, M., Shukla, S. & Andley, M. Study of nuclear morphometry on cytology specimens of benign and malignant breast lesions: a study of 122 cases. J. Cytol. 34, 10–15 (2017).

    Article  Google Scholar 

  89. Subbaiah, R. M., Dey, P. & Nijhawan, R. Artificial neural network in breast lesions from fine-needle aspiration cytology smear. Diagn. Cytopathol. 42, 218–224 (2014).

    Article  CAS  Google Scholar 

  90. Ouyang, W., Aristov, A., Lelek, M., Hao, X. & Zimmer, C. Deep learning massively accelerates super-resolution localization microscopy. Nat. Biotechnol. 36, 460–468 (2018).

    Article  CAS  Google Scholar 

  91. Buggenthin, F. et al. Prospective identification of hematopoietic lineage choice by deep learning. Nat. Methods 14, 403–406 (2017).

    Article  CAS  Google Scholar 

  92. Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118 (2017).

    Article  CAS  Google Scholar 

  93. Gulshan, V. et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316, 2402–2410 (2016).

    Article  Google Scholar 

  94. Van Valen, D. A. et al. Deep learning automates the quantitative analysis of individual cells in live-cell imaging experiments. PLOS Comput. Biol. 12, e1005177 (2016).

    Article  CAS  Google Scholar 

  95. Heo, Y. J., Lee, D., Kang, J., Lee, K. & Chung, W. K. Real-time image processing for microscopy-based label-free imaging flow cytometry in a microfluidic chip. Sci. Rep. 7, 11651 (2017).

    Article  CAS  Google Scholar 

  96. Christiansen, E. M. et al. In silico labeling: predicting fluorescent labels in unlabeled images. Cell 173, 792–803.e19 (2018).

    Article  CAS  Google Scholar 

  97. Wang, C. et al. Deconvolution of subcellular protrusion heterogeneity and the underlying actin regulator dynamics from live cell imaging. Nat. Commun. 9, 1688 (2018).

    Article  CAS  Google Scholar 

  98. Varmus, H. & Kumar, H. S. Addressing the growing international challenge of cancer: a multinational perspective. Sci. Transl Med. 5, 175cm2 (2013).

    Article  Google Scholar 

  99. Livingston, J. Cancer in the shadow of the AIDS epidemic in southern Africa. Oncologist 18, 783–786 (2013).

    Article  Google Scholar 

  100. Chabner, B. A. et al. Cancer in Botswana: the second wave of AIDS in Sub-Saharan Africa. Oncologist 18, 777–778 (2013).

    Article  Google Scholar 

  101. Naresh, K. N. et al. Lymphomas in sub-Saharan Africa – what can we learn and how can we help in improving diagnosis, managing patients and fostering translational research. Br. J. Haematol. 154, 696–703 (2011).

    Article  Google Scholar 

  102. Mwamba, P. M. et al. AIDS-related non-Hodgkin’s lymphoma in sub-Saharan Africa: current status and realities of therapeutic approach. Lymphoma https://doi.org/10.1155/2012/904367 (2012).

    Article  Google Scholar 

  103. Pai, S. I. & Westra, W. H. Molecular pathology of head and neck cancer: implications for diagnosis, prognosis, and treatment. Annu. Rev. Pathol. 4, 49–70 (2009).

    Article  CAS  Google Scholar 

  104. Pai, S. I. et al. Comparative analysis of the phase III clinical trials of anti-PD1 monotherapy in head and neck squamous cell carcinoma patients (CheckMate 141 and KEYNOTE 040). J. Immunother. Cancer 7, 96 (2019).

    Article  Google Scholar 

  105. Carney, B. et al. Target engagement imaging of PARP inhibitors in small-cell lung cancer. Nat. Commun. 9, 176 (2018).

    Article  CAS  Google Scholar 

  106. Kossatz, S. et al. Detection and delineation of oral cancer with a PARP1 targeted optical imaging agent. Sci. Rep. 6, 21371 (2016).

    Article  CAS  Google Scholar 

  107. Kossatz, S., Weber, W. & Reiner, T. Detection and delineation of oral cancer with a PARP1-targeted optical imaging agent. Mol. Imaging 16, 1536012117723786 (2017).

    Article  CAS  Google Scholar 

  108. Kossatz, S. et al. PARP1 as a biomarker for early detection and intraoperative tumor delineation in epithelial cancers–first-in-human results. Preprint at bioRxiv https://doi.org/10.1101/663385 (2019).

  109. Couzin-Frankel, J. Medicine contends with how to use artificial intelligence. Science 364, 1119–1120 (2019).

    Article  CAS  Google Scholar 

  110. Goodsaid, F. M. The labyrinth of product development and regulatory approvals in liquid biopsy diagnostics. Clin. Transl. Sci. 12, 431–439 (2019).

    Article  Google Scholar 

  111. Pantanowitz, L. et al. Twenty years of digital pathology: an overview of the road travelled, what is on the horizon, and the emergence of vendor-neutral archives. J. Pathol. Inform. 9, 40 (2018).

    Article  Google Scholar 

  112. Kozma, P., Kehl, F., Ehrentreich-Förster, E., Stamm, C. & Bier, F. F. Integrated planar optical waveguide interferometer biosensors: a comparative review. Biosens. Bioelectron. 58, 287–307 (2014).

    Article  CAS  Google Scholar 

  113. He, Z., Lee, Y. H., Chanda, D. & Wu, S. T. Adaptive liquid crystal microlens array enabled by two-photon polymerization. Opt. Express 26, 21184–21193 (2018).

    Article  CAS  Google Scholar 

  114. Kuznetsov, A. I. et al. Laser-induced transfer of metallic nanodroplets for plasmonics and metamaterial applications. J. Opt. Soc. Am. B 26, B130–B138 (2009).

    Article  CAS  Google Scholar 

  115. Lin, R. J. et al. Achromatic metalens array for full-colour light-field imaging. Nat. Nanotechnol. 14, 227–231 (2019).

    Article  CAS  Google Scholar 

  116. Wolfer, T., Bollgruen, P., Mager, D., Overmeyer, L. & Korvink, J. G. Printing and preparation of integrated optical waveguides for optronic sensor networks. Mechatronics 34, 119–127 (2016).

    Article  Google Scholar 

  117. Hui, H. Y. L. et al. “Immuno-flowFISH” for the assessment of cytogenetic abnormalities in chronic lymphocytic leukemia. Cytometry A 95, 521–533 (2019).

    Article  CAS  Google Scholar 

  118. Mazzini, G. & Danova, M. Fluorochromes for DNA staining and quantitation. Methods Mol. Biol. 1560, 239–259 (2017).

    Article  CAS  Google Scholar 

  119. Ng, B. L., Fu, B., Graham, J., Hall, C. & Thompson, S. Chromosome analysis using benchtop flow analysers and high speed cell sorters. Cytometry A 95, 323–331 (2019).

    Article  CAS  Google Scholar 

  120. Smith, P. J., Darzynkiewicz, Z. & Errington, R. J. Nuclear cytometry and chromatin organization. Cytometry A 93, 771–784 (2018).

    Article  Google Scholar 

  121. Gupta, A. et al. Deep learning in image cytometry: a review. Cytometry A 95, 366–380 (2019).

    Article  Google Scholar 

  122. Jarrett, K., Kavukcuoglu, K., Ranzato, M. A. & LeCun, Y. in IEEE 12th Int. Conf. Comput. Vis. 2146–2153 (IEEE, 2009).

  123. Nair, V. & Hinton, G. E. in Proc. 27th Int. Conf. Mach. Learn. 807–814 (2010).

  124. Bridle, J. S. Training stochastic model recognition algorithms as networks can lead to maximum mutual information estimation of parameters. Adv. Neural Inf. Process. Syst. 2, 211–217 (1990).

    Google Scholar 

  125. Zhou, Y. T. & Chellappa, R. in IEEE Int. Conf. Neural Netw. 71–78 (IEEE, 1988).

  126. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I. & Salakhutdinov, R. Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929–1958 (2014).

    Google Scholar 

  127. Yosinski, J., Clune, J., Bengio, Y. & Lipson, H. How transferable are features in deep neural networks? Adv. Neural Inf. Process. Syst. 27, 3320–3328 (2014).

    Google Scholar 

Download references

Acknowledgements

The authors acknowledge extensive discussions with L.K. Chin, J. Min, J. Oh, H. Im, T. Rainer and J. Carlson, and thank C. Landeros for discussions on machine learning and K. Joyes for editing the manuscript. The authors are indebted to J. Higgins and C. Vinegoni for the critical review of the manuscript. The authors are supported by the following grants: NIH-UH3 CA202637, NIH-U01CA206997, NIH-R01CA204019 and NIH-R01CA206890 (R.W.); NIH-R01CA229777, NIH-U01CA233360, DoD-W81XWH1910199, DoD-W81XWH1910194 and MGH Scholar Fund (H.L.).

Author information

Authors and Affiliations

Authors

Contributions

Both authors contributed equally to the preparation of this review.

Corresponding author

Correspondence to Ralph Weissleder.

Ethics declarations

Competing interests

R.W. declares that he has received consultancy payments from Tarveda Pharmaceuticals, ModeRNA, Alivio Therapeutics and Accure Health, and that he is a shareholder of T2 Biosystems, Lumicell, Accure Health and Aikili Biosystems. H.L. declares that he has received consultancy payments from Exosome Diagnostics and Accure Health, and that he is a shareholder of Accure Health and Aikili Biosystems. Patents: all patents associated with R.W. and H.L. have been assigned to and handled by Massachusetts General Hospital.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Weissleder, R., Lee, H. Automated molecular-image cytometry and analysis in modern oncology. Nat Rev Mater 5, 409–422 (2020). https://doi.org/10.1038/s41578-020-0180-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41578-020-0180-6

This article is cited by

Search

Quick links

Nature Briefing: Cancer

Sign up for the Nature Briefing: Cancer newsletter — what matters in cancer research, free to your inbox weekly.

Get what matters in cancer research, free to your inbox weekly. Sign up for Nature Briefing: Cancer