Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Artificial intelligence-enabled quantitative phase imaging methods for life sciences

Nature Methods volume 20, pages 1645–1660 (2023)Cite this article

Subjects

Abstract

Quantitative phase imaging, integrated with artificial intelligence, allows for the rapid and label-free investigation of the physiology and pathology of biological systems. This review presents the principles of various two-dimensional and three-dimensional label-free phase imaging techniques that exploit refractive index as an intrinsic optical imaging contrast. In particular, we discuss artificial intelligence-based analysis methodologies for biomedical studies including image enhancement, segmentation of cellular or subcellular structures, classification of types of biological samples and image translation to furnish subcellular and histochemical information from label-free phase images. We also discuss the advantages and challenges of artificial intelligence-enabled quantitative phase imaging analyses, summarize recent notable applications in the life sciences, and cover the potential of this field for basic and industrial research in the life sciences.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: AI-integrated QPI methods applied to biomedical research.
Fig. 2: Principles and advantages of QPI.
Fig. 3: Image reconstruction and enhancement of QPI using AI.
Fig. 4: Using deep learning for organelle segmentation in QPI images of biological cells.
Fig. 5: Diverse classification applications in life science using QPI and AI.
Fig. 6: AI-enabled image translation from QPI images to other imaging modalities for various biological systems.

Similar content being viewed by others

References

  1. Milestones in light microscopy. Nat. Cell Biol. 11, 1165 (2009).

  2. Lee, K. et al. Quantitative phase imaging techniques for the study of cell pathophysiology: from principles to applications. Sensors 13, 4170–4191 (2013).

    PubMed  PubMed Central  Google Scholar 

  3. Kim, K. et al. Optical diffraction tomography techniques for the study of cell pathophysiology. J. Biomed. Photonics Eng. 2, 020201 (2016).

    Google Scholar 

  4. Popescu, G. Quantitative phase imaging of cells and tissues (McGraw-Hill Education, 2011).

  5. Park, Y., Depeursinge, C. & Popescu, G. Quantitative phase imaging in biomedicine. Nat. Photonics 12, 578–589 (2018).

    CAS  Google Scholar 

  6. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).

    CAS  PubMed  Google Scholar 

  7. Zernike, F. How I discovered phase contrast. Science 121, 345–349 (1955).

    PubMed  Google Scholar 

  8. Nomarski, G. Microinterféromètre différentiel à ondes polarisées. J. Phys. Rad. 16, 9S–13S (1955).

    Google Scholar 

  9. Marquet, P. et al. Digital holographic microscopy: a noninvasive contrast imaging technique allowing quantitative visualization of living cells with subwavelength axial accuracy. Opt. Lett. 30, 468–470 (2005).

    PubMed  Google Scholar 

  10. Popescu, G., Ikeda, T., Dasari, R. R. & Feld, M. S. Diffraction phase microscopy for quantifying cell structure and dynamics. Opt. Lett. 31, 775–777 (2006).

    PubMed  Google Scholar 

  11. Ikeda, T., Popescu, G., Dasari, R. R. & Feld, M. S. Hilbert phase microscopy for investigating fast dynamics in transparent systems. Opt. Lett. 30, 1165–1167 (2005).

    PubMed  Google Scholar 

  12. Zuo, C. et al. Transport of intensity equation: a tutorial. Opt. Lasers Eng. 135, 106187 (2020).

    Google Scholar 

  13. Soto, J. M., Rodrigo, J. A. & Alieva, T. Optical diffraction tomography with fully and partially coherent illumination in high numerical aperture label-free microscopy [Invited]. Appl. Opt. 57, A205–A214 (2018).

    PubMed  Google Scholar 

  14. Baek, Y. & Park, Y. Intensity-based holographic imaging via space-domain Kramers–Kronig relations. Nat. Photonics 15, 354–360 (2021).

    CAS  Google Scholar 

  15. Tian, L. & Waller, L. Quantitative differential phase contrast imaging in an LED array microscope. Opt. Express 23, 11394–11403 (2015).

    CAS  PubMed  Google Scholar 

  16. Zheng, G., Horstmeyer, R. & Yang, C. Wide-field, high-resolution Fourier ptychographic microscopy. Nat. Photonics 7, 739–745 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  17. Abbe, E. Beiträge zur Theorie des Mikroskops und der mikroskopischen Wahrnehmung. Arch. für. Mikroskop. Anat. 9, 413–468 (1873).

    Google Scholar 

  18. Skylaki, S., Hilsenbeck, O. & Schroeder, T. Challenges in long-term imaging and quantification of single-cell dynamics. Nat. Biotechnol. 34, 1137–1144 (2016).

    CAS  PubMed  Google Scholar 

  19. Magidson, V. & Khodjakov, A. Circumventing photodamage in live-cell microscopy. Methods Cell. Biol. 114, 545–560 (2013).

    PubMed  Google Scholar 

  20. Huisken, J., Swoger, J., Del Bene, F., Wittbrodt, J. & Stelzer, E. H. Optical sectioning deep inside live embryos by selective plane illumination microscopy. Science 305, 1007–1009 (2004).

    CAS  PubMed  Google Scholar 

  21. Weber, M., Mickoleit, M. & Huisken, J. Light sheet microscopy. Methods Cell. Biol. 123, 193–215 (2014).

    PubMed  Google Scholar 

  22. Denk, W., Strickler, J. H. & Webb, W. W. Two-photon laser scanning fluorescence microscopy. Science 248, 73–76 (1990).

    CAS  PubMed  Google Scholar 

  23. Helmchen, F. & Denk, W. Deep tissue two-photon microscopy. Nat. Methods 2, 932–940 (2005).

    CAS  PubMed  Google Scholar 

  24. Axelrod, D., Burghardt, T. P. & Thompson, N. L. Total internal reflection fluorescence. Annu. Rev. Biophys. Bioeng. 13, 247–268 (1984).

    CAS  PubMed  Google Scholar 

  25. Axelrod, D. Total internal reflection fluorescence microscopy in cell biology. Traffic 2, 764–774 (2001).

    CAS  PubMed  Google Scholar 

  26. Subedi, N. R. et al. Integrative quantitative-phase and airy light-sheet imaging. Sci. Rep. 10, 20150 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  27. Icha, J., Weber, M., Waters, J. C. & Norden, C. Phototoxicity in live fluorescence microscopy, and how to avoid it. BioEssays 39, 1700003 (2017).

    Google Scholar 

  28. Lee, A. J. et al. Label-free monitoring of 3D cortical neuronal growth in vitro using optical diffraction tomography. Biomed. Opt. Express 12, 6928–6939 (2021).

    CAS  PubMed  PubMed Central  Google Scholar 

  29. Ryu, D. et al. Label-free white blood cell classification using refractive index tomography and deep learning. BME Front. 2021, 9893804 (2021).

    PubMed  PubMed Central  Google Scholar 

  30. Kim, G. et al. Rapid species identification of pathogenic bacteria from a minute quantity exploiting three-dimensional quantitative phase imaging and artificial neural network. Light Sci. Appl. 11, 190 (2022). Rapid label-free classification of bacterial species using QPI and deep learning. The framework utilizes a single 3D QPI image to accurately classify different bacterial species, without the need for labeling.

    CAS  PubMed  PubMed Central  Google Scholar 

  31. Carrigan, S. D., Scott, G. & Tabrizian, M. Toward resolving the challenges of sepsis diagnosis. Clin. Chem. 50, 1301–1314 (2004).

    CAS  PubMed  Google Scholar 

  32. Barer, R. Interference microscopy and mass determination. Nature 169, 366–367 (1952).

    CAS  PubMed  Google Scholar 

  33. Davies, H. & Wilkins, M. Interference microscopy and mass determination. Nature 169, 541–541 (1952).

    CAS  PubMed  Google Scholar 

  34. Gabor, D. A new microscopic principle. Nature 161, 777–778 (1948). A demonstration of holographic microscopy.

    Google Scholar 

  35. Bruning, J. H. et al. Digital wavefront measuring interferometer for testing optical surfaces and lenses. Appl. Opt. 13, 2693–2703 (1974).

    CAS  PubMed  Google Scholar 

  36. Yamaguchi, I. & Zhang, T. Phase-shifting digital holography. Opt. Lett. 22, 1268–1270 (1997).

    CAS  PubMed  Google Scholar 

  37. Yamaguchi, I., Kato, J.-i, Ohta, S. & Mizuno, J. Image formation in phase-shifting digital holography and applications to microscopy. Appl. Opt. 40, 6177–6186 (2001).

    CAS  PubMed  Google Scholar 

  38. Takeda, M., Ina, H. & Kobayashi, S. Fourier-transform method of fringe-pattern analysis for computer-based topography and interferometry. J. Opt. Soc. Am 72, 156–160 (1982).

    Google Scholar 

  39. Leith, E. N. & Upatnieks, J. Reconstructed wavefronts and communication theory. J. Opt. Soc. Am. 52, 1123–1130 (1962).

    Google Scholar 

  40. Wang, Z. et al. Spatial light interference microscopy (SLIM). Opt. Express 19, 1016–1026 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  41. Bhaduri, B., Pham, H., Mir, M. & Popescu, G. Diffraction phase microscopy with white light. Opt. Lett. 37, 1094–1096 (2012).

    PubMed  Google Scholar 

  42. Zheng, G., Shen, C., Jiang, S., Song, P. & Yang, C. Concept, implementations and applications of Fourier ptychography. Nat. Rev. Phys. 3, 207–223 (2021).

    Google Scholar 

  43. Oh, C., Isikman, S. O., Khademhosseinieh, B. & Ozcan, A. On-chip differential interference contrast microscopy using lensless digital holography. Opt. Express 18, 4717–4726 (2010).

    CAS  PubMed  PubMed Central  Google Scholar 

  44. Bon, P., Maucort, G., Wattellier, B. & Monneret, S. Quadriwave lateral shearing interferometry for quantitative phase microscopy of living cells. Opt. Express 17, 13080–13094 (2009).

    CAS  PubMed  Google Scholar 

  45. Nguyen, T. H., Kandel, M. E., Rubessa, M., Wheeler, M. B. & Popescu, G. Gradient light interference microscopy for 3D imaging of unlabeled specimens. Nat. Commun. 8, 210 (2017).

    PubMed  PubMed Central  Google Scholar 

  46. Seo, S., Su, T. -W., Tseng, D. K., Erlinger, A. & Ozcan, A. Lensfree holographic imaging for on-chip cytometry and diagnostics. Lab Chip 9, 777–787 (2009).

    CAS  PubMed  Google Scholar 

  47. Greenbaum, A. et al. Imaging without lenses: achievements and remaining challenges of wide-field on-chip microscopy. Nat. Methods 9, 889–895 (2012).

    CAS  PubMed  PubMed Central  Google Scholar 

  48. Su, T. -W., Xue, L. & Ozcan, A. High-throughput lensfree 3D tracking of human sperms reveals rare statistics of helical trajectories. Proc. Natl Acad. Sci. USA 109, 16018–16022 (2012).

    CAS  PubMed  PubMed Central  Google Scholar 

  49. Zuo, C., Sun, J., Zhang, J., Hu, Y. & Chen, Q. Lensless phase microscopy and diffraction tomography with multi-angle and multi-wavelength illuminations using a LED matrix. Opt. Express 23, 14314–14328 (2015).

    CAS  PubMed  Google Scholar 

  50. Teague, M. R. Irradiance moments: their propagation and use for unique retrieval of phase. J. Opt. Soc. Am. 72, 1199–1209 (1982).

    Google Scholar 

  51. Teague, M. R. Deterministic phase retrieval: a Green’s function solution. J. Opt. Soc. Am. 73, 1434–1441 (1983).

    Google Scholar 

  52. Tian, L. & Waller, L. 3D intensity and phase imaging from light field measurements in an LED array microscope. Optica 2, 104–111 (2015).

    Google Scholar 

  53. Chen, M., Tian, L. & Waller, L. 3D differential phase contrast microscopy. Biomed. Opt. Express 7, 3940–3950 (2016).

    PubMed  PubMed Central  Google Scholar 

  54. Kandel, M. E. et al. Phase imaging with computational specificity (PICS) for measuring dry mass changes in sub-cellular compartments. Nat. Commun. 11, 6256 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  55. Lee, M. et al. Deep-learning-based three-dimensional label-free tracking and analysis of immunological synapses of CAR-T cells. Elife 9, e49023 (2020). Study on dynamics of immunological synapses of CAR-T cells through automatic segmentation of the synapses from label-free QPI images using deep learning.

    CAS  PubMed  PubMed Central  Google Scholar 

  56. Goswami, N. et al. Label-free SARS-CoV-2 detection and classification using phase imaging with computational specificity. Light Sci. Appl. 10, 176 (2021).

    CAS  PubMed  PubMed Central  Google Scholar 

  57. Jo, Y. et al. Label-free multiplexed microtomography of endogenous subcellular dynamics using generalizable deep learning. Nat. Cell Biol. 23, 1329–1337 (2021). Study on highly multiplexed virtual staining of label-free QPI images using deep learning. The framework transforms a single label-free cellular QPI image into multiple fluorescence-labeled images, providing multiple subcellular specificities simultaneously.

    CAS  PubMed  Google Scholar 

  58. Wolf, E. Three-dimensional structure determination of semi-transparent objects from holographic data. Opt. Commun. 1, 153–156 (1969).

    Google Scholar 

  59. Isikman, S. O., Bishara, W. & Ozcan, A. Partially coherent lensfree tomographic microscopy. Appl. Opt. 50, H253–H264 (2011).

    PubMed  PubMed Central  Google Scholar 

  60. Isikman, S. O. et al. Lens-free optical tomographic microscope with a large imaging volume on a chip. Proc. Natl Acad. Sci. USA 108, 7296–7301 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  61. Hugonnet, H., Lee, M. & Park, Y. Optimizing illumination in three-dimensional deconvolution microscopy for accurate refractive index tomography. Opt. Express 29, 6293–6301 (2021).

    PubMed  Google Scholar 

  62. Hugonnet, H., Lee, M. J. & Park, Y. K. Quantitative phase and refractive index imaging of 3D objects via optical transfer function reshaping. Opt. Express 30, 13802–13809 (2022).

    PubMed  Google Scholar 

  63. Kamilov, U. S. et al. Learning approach to optical tomography. Optica 2, 517–522 (2015).

    CAS  Google Scholar 

  64. Lim, J., Ayoub, A. B., Antoine, E. E. & Psaltis, D. High-fidelity optical diffraction tomography of multiple scattering samples. Light Sci. Appl. 8, 82 (2019).

    PubMed  PubMed Central  Google Scholar 

  65. Fan, S., Smith-Dryden, S., Li, G. & Saleh, B. Reconstructing complex refractive-index of multiply-scattering media by use of iterative optical diffraction tomography. Opt. Express 28, 6846–6858 (2020).

    PubMed  Google Scholar 

  66. Lee, M., Hugonnet, H. & Park, Y. Inverse problem solver for multiple light scattering using modified Born series. Optica 9, 177–182 (2022).

    CAS  Google Scholar 

  67. Merola, F. et al. Tomographic flow cytometry by digital holography. Light Sci. Appl. 6, e16241 (2017).

    CAS  PubMed  PubMed Central  Google Scholar 

  68. Lee, M. et al. High-resolution assessment of multidimensional cellular mechanics using label-free refractive-index traction force microscopy. Preprint at bioRxiv https://doi.org/10.1101/2023.02.15.528626 (2023).

  69. Vasdekis, A. E. et al. Eliciting the impacts of cellular noise on metabolic trade-offs by quantitative mass imaging. Nat. Commun. 10, 848 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  70. Kandel, M. E., Teng, K. W., Selvin, P. R. & Popescu, G. Label-free imaging of single microtubule dynamics using spatial light interference microscopy. ACS Nano 11, 647–655 (2017).

    CAS  PubMed  Google Scholar 

  71. Lee, C. et al. Label-free three-dimensional observations and quantitative characterisation of on-chip vasculogenesis using optical diffraction tomography. Lab Chip 21, 494–501 (2021).

    CAS  PubMed  Google Scholar 

  72. Park, J. et al. Quantification of structural heterogeneity in H&E stained clear cell renal cell carcinoma using refractive index tomography. Biomed. Opt. Express 14, 1071–1081 (2023).

    PubMed  PubMed Central  Google Scholar 

  73. Bokemeyer, A. et al. Quantitative phase imaging using digital holographic microscopy reliably assesses morphology and reflects elastic properties of fibrotic intestinal tissue. Sci. Rep. 9, 19388 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  74. Chetlur, S. et al. cudnn: efficient primitives for deep learning. Preprint at https://doi.org/10.48550/arXiv.1410.0759 (2014).

  75. Paszke, A. et al. PyTorch: an imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems 32, 8026–8037 (2019). Introduction of Pytorch, which enabled efficient access to deep learning frameworks, making it one of the most widely used machine learning libraries in the current era.

  76. Abadi, M. et al. TensorFlow: a system for large-scale machine learning. OSDI 16, 265–283 (2016). Introduction of TensorFlow, one of the most extensively utilized machine learning libraries, which has substantially enhanced the usability of machine learning algorithms.

  77. Lim, J. et al. Comparative study of iterative reconstruction algorithms for missing cone problems in optical diffraction tomography. Opt. Express 23, 16933–16948 (2015).

    CAS  PubMed  Google Scholar 

  78. Petrou, M. M. & Petrou, C. Image Processing: the Fundamentals (John Wiley & Sons, 2010).

  79. Rivenson, Y., Zhang, Y., Gunaydin, H., Teng, D. & Ozcan, A. Phase recovery and holographic image reconstruction using deep learning in neural networks. Light Sci. Appl. 7, 17141 (2018). An early study on rapid image reconstruction in holographic imaging using deep learning.

    CAS  PubMed  PubMed Central  Google Scholar 

  80. Nguyen, T., Xue, Y., Li, Y., Tian, L. & Nehmetallah, G. Deep learning approach for Fourier ptychography microscopy. Opt. Express 26, 26470–26484 (2018).

    PubMed  Google Scholar 

  81. Barbastathis, G., Ozcan, A. & Situ, G. On the use of deep learning for computational imaging. Optica 6, 921–943 (2019).

    Google Scholar 

  82. Wu, Y. et al. Extended depth-of-field in holographic imaging using deep-learning-based autofocusing and phase recovery. Optica 5, 704–710 (2018).

    Google Scholar 

  83. Wu, Y. et al. Bright-field holography: cross-modality deep learning enables snapshot 3D imaging with bright-field contrast using a single hologram. Light Sci. Appl. 8, 25 (2019).

    PubMed  PubMed Central  Google Scholar 

  84. Huang, L. et al. Holographic image reconstruction with phase recovery and autofocusing using recurrent neural networks. ACS Photonics 8, 1763–1774 (2021).

    CAS  Google Scholar 

  85. Chen, H., Huang, L., Liu, T. & Ozcan, A. Fourier Imager Network (FIN): a deep neural network for hologram reconstruction with superior external generalization. Light Sci. Appl. 11, 254 (2022).

    CAS  PubMed  PubMed Central  Google Scholar 

  86. Pirone, D. et al. Speeding up reconstruction of 3D tomograms in holographic flow cytometry via deep learning. Lab Chip 22, 793–804 (2022).

    CAS  PubMed  Google Scholar 

  87. Xue, Y., Cheng, S., Li, Y. & Tian, L. Reliable deep-learning-based phase imaging with uncertainty quantification. Optica 6, 618–629 (2019).

    PubMed  PubMed Central  Google Scholar 

  88. Dardikman-Yoffe, G. et al. PhUn-Net: ready-to-use neural network for unwrapping quantitative phase images of biological cells. Biomed. Opt. Express 11, 1107–1121 (2020).

    PubMed  PubMed Central  Google Scholar 

  89. Ryu, D. et al. DeepRegularizer: rapid resolution enhancement of tomographic imaging using deep learning. IEEE Trans. Med. Imaging 40, 1508–1518 (2021).

    PubMed  Google Scholar 

  90. Lim, J., Ayoub, A. B. & Psaltis, D. Three-dimensional tomography of red blood cells using deep learning. Adv. Photonics 2, 026001 (2020).

  91. Zhu, J.-Y., Park, T., Isola, P. & Efros, A. A. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proc. IEEE International Conference on Computer Vision 2223–2232 (IEEE, 2017).

  92. Yin, D. et al. Digital holographic reconstruction based on deep learning framework with unpaired data. IEEE Photonics J. 12, 1–12 (2019).

    Google Scholar 

  93. Zuo, C., Chen, Q., Qu, W. & Asundi, A. Phase aberration compensation in digital holographic microscopy based on principal component analysis. Opt. Lett. 38, 1724–1726 (2013).

    PubMed  Google Scholar 

  94. Zhang, Y. et al. PhaseGAN: a deep-learning phase-retrieval approach for unpaired datasets. Opt. Express 29, 19593–19604 (2021).

    PubMed  Google Scholar 

  95. Chung, H., Huh, J., Kim, G., Park, Y. K. & Ye, J. C. Missing cone artifact removal in odt using unsupervised deep learning in the projection domain. IEEE Trans. Comput. Imaging 7, 747–758 (2021).

    Google Scholar 

  96. Huang, L. et al. Self-supervised learning of hologram reconstruction using physics consistency. Nat. Mach. Intell. 5, 895–907 (2023).

    Google Scholar 

  97. Ulyanov, D., Vedaldi, A. & Lempitsky, V. Deep image prior. In Proc. IEEE Conf. Computer Vision and Pattern Recognition 9446–9454 (IEEE. 2018).

  98. Wang, F. et al. Phase imaging with an untrained neural network. Light Sci. Appl. 9, 77 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  99. Zhou, K. C. & Horstmeyer, R. Diffraction tomography with a deep image prior. Opt. Express 28, 12872–12896 (2020).

    PubMed  PubMed Central  Google Scholar 

  100. Huang, L., Yang, X., Liu, T. & Ozcan, A. Few-shot transfer learning for holographic image reconstruction using a recurrent neural network. APL Photonics 7, 070801 (2022).

    Google Scholar 

  101. Chang, T. et al. Calibration-free quantitative phase imaging using data-driven aberration modeling. Opt. Express 28, 34835–34847 (2020).

    CAS  PubMed  Google Scholar 

  102. Park, D.-Y. & Park, J.-H. Hologram conversion for speckle free reconstruction using light field extraction and deep learning. Opt. Express 28, 5393–5409 (2020).

    PubMed  Google Scholar 

  103. Chen, L., Chen, X., Cui, H., Long, Y. & Wu, J. Image enhancement in lensless inline holographic microscope by inter-modality learning with denoising convolutional neural network. Opt. Commun. 484, 126682 (2021).

    CAS  Google Scholar 

  104. Goy, A., Arthur, K., Li, S. & Barbastathis, G. Low photon count phase retrieval using deep learning. Phys. Rev. Lett. 121, 243902 (2018).

    CAS  PubMed  Google Scholar 

  105. Deng, M., Li, S., Goy, A., Kang, I. & Barbastathis, G. Learning to synthesize: robust phase retrieval at low photon counts. Light Sci. Appl. 9, 36 (2020).

    Google Scholar 

  106. Choi, G. et al. Cycle-consistent deep learning approach to coherent noise reduction in optical diffraction tomography. Opt. Express 27, 4927–4943 (2019).

    PubMed  Google Scholar 

  107. Wu, Z. et al. SIMBA: scalable inversion in optical tomography using deep denoising priors. IEEE J. Sel. Top. Signal Process. 14, 1163–1175 (2020).

    Google Scholar 

  108. Rivenson, Y. et al. Deep learning microscopy. Optica 4, 1437–1443 (2017).

    Google Scholar 

  109. Liu, T. et al. Deep learning-based super-resolution in coherent imaging systems. Sci. Rep. 9, 3926 (2019). Resolution enhancement of both amplitude and phase images in a holographic imaging system using deep learning.

    PubMed  PubMed Central  Google Scholar 

  110. Yuan, Y. et al. Unsupervised image super-resolution using cycle-in-cycle generative adversarial networks. In Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops, 701–710 (IEEE, 2018).

  111. Rizwan, I., Haque, I. & Neubert, J. Deep learning approaches to biomedical image segmentation. Inform. Med. Unlocked 18, 100297 (2020).

    Google Scholar 

  112. Minaee, S. et al. Image segmentation using deep learning: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 44, 3523–3542 (2021).

    Google Scholar 

  113. Wang, P., Bista, R., Bhargava, R., Brand, R. E. & Liu, Y. Spatial-domain low-coherence quantitative phase microscopy for cancer diagnosis. Opt. Lett. 35, 2840–2842 (2010).

    CAS  PubMed  PubMed Central  Google Scholar 

  114. Pal, N. R. & Pal, S. K. A review on image segmentation techniques. Pattern Recognition 26, 1277–1294 (1993).

    Google Scholar 

  115. Nguyen, T. H. et al. Automatic Gleason grading of prostate cancer using quantitative phase imaging and machine learning. J. Biomed. Opt. 22, 36015 (2017).

    PubMed  Google Scholar 

  116. Kandel, M. E. et al. Multiscale assay of unlabeled neurite dynamics using phase imaging with computational specificity. ACS Sens. 6, 1864–1874 (2021).

    CAS  PubMed  PubMed Central  Google Scholar 

  117. Kandel, M. E. et al. Reproductive outcomes predicted by phase imaging with computational specificity of spermatozoon ultrastructure. Proc. Natl Acad. Sci. USA 117, 18302–18309 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  118. Lee, J. et al. Deep-learning-based label-free segmentation of cell nuclei in time-lapse refractive index tomograms. IEEE Access 7, 83449–83460 (2019).

    Google Scholar 

  119. Choi, J. et al. Label-free three-dimensional analyses of live cells with deep-learning-based segmentation exploiting refractive index distributions. Preprint at bioRxiv https://doi.org/10.1101/2021.05.23.445351 (2021).

  120. Pirone, D. et al. Stain-free identification of cell nuclei using tomographic phase microscopy in flow cytometry. Nat. Photonics 16, 851–859 (2022).

    CAS  PubMed  PubMed Central  Google Scholar 

  121. Hugonnet, H. et al. Multiscale label-free volumetric holographic histopathology of thick-tissue slides with subcellular resolution. Adv. Photonics 3, 026004 (2021).

    Google Scholar 

  122. Nitta, N. et al. Intelligent image-activated cell sorting. Cell 175, 266–276 (2018).

    CAS  PubMed  Google Scholar 

  123. Chen, C. L. et al. Deep learning in label-free cell classification. Sci. Rep. 6, 21471 (2016). An early study on the utilization of deep learning integrated with QPI for label-free cell type classification.

    CAS  PubMed  PubMed Central  Google Scholar 

  124. Jo, Y. et al. Holographic deep learning for rapid optical screening of anthrax spores. Sci. Adv. 3, e1700606 (2017).

    PubMed  PubMed Central  Google Scholar 

  125. Pavillon, N., Hobro, A. J., Akira, S. & Smith, N. I. Noninvasive detection of macrophage activation with single-cell resolution through machine learning. Proc. Natl Acad. Sci. USA 115, E2676–E2685 (2018).

    PubMed  PubMed Central  Google Scholar 

  126. Go, T., Kim, J. H., Byeon, H. & Lee, S. J. Machine learning‐based in‐line holographic sensing of unstained malaria‐infected red blood cells. J. Biophotonics 11, e201800101 (2018).

    PubMed  Google Scholar 

  127. Javidi, B. et al. Sickle cell disease diagnosis based on spatio-temporal cell dynamics analysis using 3D printed shearing digital holographic microscopy. Opt. Express 26, 13614–13627 (2018).

    PubMed  Google Scholar 

  128. Kim, G., Jo, Y., Cho, H., Min, H. S. & Park, Y. Learning-based screening of hematologic disorders using quantitative phase imaging of individual red blood cells. Biosens. Bioelectron. 123, 69–76 (2019).

    CAS  PubMed  Google Scholar 

  129. Ozaki, Y. et al. Label-free classification of cells based on supervised machine learning of subcellular structures. PLoS ONE 14, e0211347 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  130. Karandikar, S. H. et al. Reagent-free and rapid assessment of T cell activation state using diffraction phase microscopy and deep learning. Anal. Chem. 91, 3405–3411 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  131. Rubin, M. et al. TOP-GAN: Stain-free cancer cell classification using deep learning with a small training set. Med. Image Anal. 57, 176–185 (2019).

    PubMed  Google Scholar 

  132. Park, S. et al. Label-free tomographic imaging of lipid droplets in foam cells for machine-learning-assisted therapeutic evaluation of targeted nanodrugs. ACS Nano 14, 1856–1865 (2020).

    CAS  PubMed  Google Scholar 

  133. Belashov, A. V. et al. In vitro monitoring of photoinduced necrosis in HeLa cells using digital holographic microscopy and machine learning. J. Opt. Soc. Am. A Opt. Image Sci. Vis. 37, 346–352 (2020).

    CAS  PubMed  Google Scholar 

  134. Lam, V. et al. Quantitative scoring of epithelial and mesenchymal qualities of cancer cells using machine learning and quantitative phase imaging. J. Biomed. Opt. 25, 1–17 (2020).

    PubMed  Google Scholar 

  135. Ryu, D. et al. Label-free 3D quantitative phase imaging cytometry with deep learning: identifying naive, memory, and senescent T cells. J. Immunol. 204, 86.5 (2020).

    Google Scholar 

  136. Singla, N. & Srivastava, V. Deep learning enabled multi-wavelength spatial coherence microscope for the classification of malaria-infected stages with limited labelled data size. Opt. Laser Technol. 130, 106335 (2020).

  137. O’Connor, T., Anand, A., Andemariam, B. & Javidi, B. Deep learning-based cell identification and disease diagnosis using spatio-temporal cellular dynamics in compact digital holographic microscopy. Biomed. Opt. Express 11, 4491–4508 (2020).

    PubMed  PubMed Central  Google Scholar 

  138. Butola, A. et al. High spatially sensitive quantitative phase imaging assisted with deep neural network for classification of human spermatozoa under stressed condition. Sci. Rep. 10, 13118 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  139. Nissim, N., Dudaie, M., Barnea, I. & Shaked, N. T. Real‐time stain‐free classification of cancer cells and blood cells using interferometric phase microscopy and machine learning. Cytom. Part A 99, 511–523 (2021).

    Google Scholar 

  140. Paidi, S. K. et al. Raman and quantitative phase imaging allow morpho-molecular recognition of malignancy and stages of B-cell acute lymphoblastic leukemia. Biosens. Bioelectron. 190, 113403 (2021).

    CAS  PubMed  PubMed Central  Google Scholar 

  141. Işıl, C. A. A. et al. Phenotypic analysis of microalgae populations using label-free imaging flow cytometry and deep learning. ACS Photonics 8, 1232–1242 (2021). High-throughput classification of microalgae species using QPI and deep learning.

    Google Scholar 

  142. Shu, X. et al. Artificial‐intelligence‐enabled reagent‐free imaging hematology analyzer. Adv. Intell. Syst. 3, 2000277 (2021). Rapid label-free classification of white blood cells using QPI and deep learning techniques. The framework achieves accurate classification of various types of white blood cells including subtypes of lymphocytes.

    Google Scholar 

  143. Pirone, D. et al. Identification of drug-resistant cancer cells in flow cytometry combining 3D holographic tomography with machine learning. Sens. Actuators B 375, 132963 (2023).

    CAS  Google Scholar 

  144. Mirsky, S. K., Barnea, I., Levi, M., Greenspan, H. & Shaked, N. T. Automated analysis of individual sperm cells using stain-free interferometric phase microscopy and machine learning. Cytom. Part A 91, 893–900 (2017).

    Google Scholar 

  145. Roitshtain, D. et al. Quantitative phase microscopy spatial signatures of cancer cells. Cytom. Part A 91, 482–493 (2017).

    CAS  Google Scholar 

  146. Zhang, J. K., He, Y. R., Sobh, N. & Popescu, G. Label-free colorectal cancer screening using deep learning and spatial light interference microscopy (SLIM). APL Photonics 5, 040805 (2020).

    PubMed  Google Scholar 

  147. Wang, H. et al. Early detection and classification of live bacteria using time-lapse coherent imaging and deep learning. Light Sci. Appl. 9, 118 (2020).

    PubMed  PubMed Central  Google Scholar 

  148. Ben Baruch, S., Rotman-Nativ, N., Baram, A., Greenspan, H. & Shaked, N. T. Cancer-cell deep-learning classification by integrating quantitative-phase spatial and temporal fluctuations. Cells 10, 3353 (2021).

    PubMed  PubMed Central  Google Scholar 

  149. Rotman-Nativ, N. & Shaked, N. T. Live cancer cell classification based on quantitative phase spatial fluctuations and deep learning with a small training set. Front. Physics https://doi.org/10.3389/fphy.2021.754897 (2021).

  150. Noy, L. et al. Sperm-cell DNA fragmentation prediction using label-free quantitative phase imaging and deep learning. Cytometry A 103, 470–478 (2022).

  151. Liu, T. et al. Rapid and stain-free quantification of viral plaque via lens-free holography and deep learning. Nat. Biomed. Eng 7, 1040–1052 (2023).

    PubMed  PubMed Central  Google Scholar 

  152. Rivenson, Y. et al. PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning. Light Sci. Appl. 8, 23 (2019). A key study of virtual staining of label-free QPI images of tissue slides using deep learning. Deep learning enabled the automatic transformation of label-free phase images of tissue slides into stained images, eliminating the need for chemical staining.

    PubMed  PubMed Central  Google Scholar 

  153. Wang, R. et al. Virtual brightfield and fluorescence staining for Fourier ptychography via unsupervised deep learning. Opt. Lett. 45, 5405–5408 (2020).

    PubMed  Google Scholar 

  154. Nygate, Y. N. et al. Holographic virtual staining of individual biological cells. Proc. Natl Acad. Sci. USA 117, 9223–9231 (2020). An early study on virtual staining of label-free QPI images of sperm cells using deep learning. By transforming label-free phase images into histochemically labeled images, the study achieves virtual staining of the nucleus and other structures, facilitating rapid evaluation of the quality of individual sperm cells.

    CAS  PubMed  PubMed Central  Google Scholar 

  155. Ben-Yehuda, K. et al. Simultaneous morphology, motility, and fragmentation analysis of live individual sperm cells for male fertility evaluation. Adv. Intell. Syst. 4, 2100200 (2022).

    Google Scholar 

  156. Guo, S.-M. et al. Revealing architectural order with quantitative label-free imaging and deep learning. Elife 9, e55502 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  157. Abdar, M. et al. A review of uncertainty quantification in deep learning: techniques, applications and challenges. Inf. Fusion 76, 243–297 (2021).

    Google Scholar 

  158. Angelopoulos, A. N. et al. Image-to-image regression with distribution-free uncertainty quantification and applications in imaging. In Proc. 39th International Conference on Machine Learning, 162, 717–730 (2022).

  159. Borhani, N., Kakkava, E., Moser, C. & Psaltis, D. Learning to see through multimode fibers. Optica 5, 960–966 (2018).

    Google Scholar 

  160. Li, S., Deng, M., Lee, J., Sinha, A. & Barbastathis, G. Imaging through glass diffusers using densely connected convolutional networks. Optica 5, 803–813 (2018).

    Google Scholar 

  161. Li, Y., Xue, Y. & Tian, L. Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media. Optica 5, 1181–1190 (2018).

    Google Scholar 

  162. Kang, I., Pang, S., Zhang, Q., Fang, N. & Barbastathis, G. Recurrent neural network reveals transparent objects through scattering media. Opt. Express 29, 5316–5326 (2021).

    PubMed  Google Scholar 

  163. Kang, I., Goy, A. & Barbastathis, G. Dynamical machine learning volumetric reconstruction of objects’ interiors from limited angular views. Light Sci. Appl. 10, 74 (2021).

    CAS  PubMed  PubMed Central  Google Scholar 

  164. Kellman, M. R., Bostan, E., Repina, N. A. & Waller, L. Physics-based learned design: optimized coded-illumination for quantitative phase imaging. IEEE Trans. Comput. Imaging 5, 344–353 (2019).

    Google Scholar 

  165. Cheng, Y. F. et al. Illumination pattern design with deep learning for single-shot Fourier ptychographic microscopy. Opt. Express 27, 644–656 (2019).

    PubMed  Google Scholar 

  166. Kim, K., Konda, P. C., Cooke, C. L., Appel, R. & Horstmeyer, R. Multi-element microscope optimization by a learned sensing network with composite physical layers. Opt. Lett. 45, 5684–5687 (2020).

    PubMed  Google Scholar 

  167. Mengu, D. & Ozcan, A. All-optical phase recovery: diffractive computing for quantitative phase Imaging. Adv. Opt. Mater. 10, 2200281 (2022).

    CAS  Google Scholar 

  168. Lin, X. et al. All-optical machine learning using diffractive deep neural networks. Science 361, 1004–1008 (2018).

    CAS  PubMed  Google Scholar 

  169. Mengu, D., Luo, Y., Rivenson, Y. & Ozcan, A. Analysis of diffractive optical neural networks and their integration with electronic neural networks. IEEE J. Sel. Top. Quantum Electron. 26, 3700114 (2020).

    CAS  PubMed  Google Scholar 

  170. Li, J. et al. Spectrally encoded single-pixel machine vision using diffractive networks. Sci. Adv. 7, eabd7690 (2021).

    PubMed  PubMed Central  Google Scholar 

  171. Sakib Rahman, M. S. & Ozcan, A. Computer-free, all-optical reconstruction of holograms using diffractive networks. ACS Photonics 8, 3375–3384 (2021).

    CAS  Google Scholar 

  172. Yan, H. et al. Virtual optofluidic time-stretch quantitative phase imaging. APL Photonics 5, 046103 (2020).

    CAS  Google Scholar 

  173. Lee, K. C. et al. Quantitative phase imaging flow cytometry for ultra‐large‐scale single‐cell biophysical phenotyping. Cytometry A 95, 510–520 (2019).

    PubMed  Google Scholar 

  174. Rodenburg, J. & Maiden, A. Ptychography. in Springer Handbook of Microscopy (eds. P. W. Hawkes & J. C. H. Spence) 819–904 (Springer, 2019).

  175. Park, C., Lee, K., Baek, Y. & Park, Y. Low-coherence optical diffraction tomography using a ferroelectric liquid crystal spatial light modulator. Opt. Express 28, 39649–39659 (2020).

    CAS  PubMed  Google Scholar 

  176. Wu, J.-L. et al. Ultrafast laser-scanning time-stretch imaging at visible wavelengths. Light Sci. Appl. 6, e16196 (2017).

    CAS  PubMed  PubMed Central  Google Scholar 

  177. Lai, Q. T. K. et al. High-speed laser-scanning biological microscopy using FACED. Nat. Protoc. 16, 4227–4264 (2021).

    CAS  PubMed  Google Scholar 

  178. Yip, G. G. K. et al. Multimodal FACED imaging for large-scale single-cell morphological profiling. APL Photonics 6, 070801 (2021).

    Google Scholar 

Download references

Acknowledgements

This work was supported by the National Research Foundation of Korea (2015R1A3A2066550, 2022M3H4A1A02074314, RS-2023-00241278), an Institute of Information & Communications Technology Planning & Evaluation (IITP; 2021-0-00745) grant funded by the Korea government (MSIT), a KAIST Institute of Technology Value Creation, Industry Liaison Center (G-CORE Project) grant funded by MSIT (N11230131), the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Korea (HI21C0977, HR22C1605), the US National Science Foundation (NSF) Biophotonics Program, NSF PATHS-UP Engineering Research Center and Koç Group.

Author information

Author notes

  1. DongHun Ryu

    Present address: Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA

  2. These authors contributed equally: Juyeon Park, Bijie Bai.

Authors and Affiliations

  1. Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea

    Juyeon Park, DongHun Ryu, Chungha Lee, Geon Kim & YongKeun Park

  2. KAIST Institute for Health Science and Technology, KAIST, Daejeon, Republic of Korea

    Juyeon Park, DongHun Ryu, Chungha Lee, Mahn Jae Lee, Jeongwon Shin, Geon Kim & YongKeun Park

  3. Electrical and Computer Engineering Department, University of California, Los Angeles, Los Angeles, CA, USA

    Bijie Bai, Tairan Liu, Yi Luo, Luzhe Huang, Yijie Zhang, Yuzhu Li & Aydogan Ozcan

  4. Bioengineering Department, University of California, Los Angeles, Los Angeles, CA, USA

    Bijie Bai & Aydogan Ozcan

  5. Graduate School of Medical Science and Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea

    Mahn Jae Lee

  6. Department of Biological Sciences, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea

    Jeongwon Shin

  7. Tomocube, Daejeon, Republic of Korea

    Dongmin Ryu, Hyun-seok Min & YongKeun Park

Authors
  1. Juyeon Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bijie Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. DongHun Ryu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tairan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chungha Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yi Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Mahn Jae Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Luzhe Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Jeongwon Shin
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Yijie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Dongmin Ryu
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Yuzhu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Geon Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Hyun-seok Min
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Aydogan Ozcan
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. YongKeun Park
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Aydogan Ozcan or YongKeun Park.

Ethics declarations

Competing interests

D.H.R., M.J.L., D.R., H.-s.M. and Y.K.P. have financial interests in Tomocube, a company that commercializes holotomography and quantitative phase imaging instruments. A.O. has financial interests in Lucendi and Pictor Labs, companies that commercialize deep learning-enhanced microscopy and sensing systems for water quality and pathology applications, respectively. All other authors declare no competing interests.

Peer review

Peer review information

Nature Methods thanks Keisuke Goda, Jianglei Di and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Primary Handling Editor: Rita Strack, in collaboration with the Nature Methods team.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Park, J., Bai, B., Ryu, D. et al. Artificial intelligence-enabled quantitative phase imaging methods for life sciences. Nat Methods 20, 1645–1660 (2023). https://doi.org/10.1038/s41592-023-02041-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41592-023-02041-4

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing