Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • News & Views
  • Published:

MICROSCOPY

The promise and peril of deep learning in microscopy

Nature Methods volume 18, pages 131–132 (2021)Cite this article

Subjects

Ever since van Leeuwenhoek peered into his homemade microscope and revealed a world inhabited by “small animals,” scientists have been pushing the limits of microscopy to see ever finer details in living cells and organisms. Though current methods would be science fiction to van Leeuwenhoek, we’ve entered an era of diminishing returns: camera sensors are 95% efficient, modern lasers can evaporate samples, fluorescent molecules reliably emit thousands of photons, and objectives lenses hit their fundamental physical performance limit over a century ago. Nevertheless, a “resolution revolution” seeded by Lukosz in the 1960s and heralded by Hell, Gustafsson and Betzig in the 2000s introduced “super-resolution” (SR) to the scientific vernacular. In this issue of Nature Methods, Qiao et al.1 continue the revolution by riding the tidal wave of deep learning (DL) — a framework rooted in the 1940s2 that has only recently enjoyed the computational muscle required by any but the simplest tasks3 — and present alchemic results: transforming low-resolution, low-contrast, noisy images into super-resolved, high contrast, clean micrographs.

Qiao et al.’s achievement is threefold. First, they collect an exceptional training dataset1, an invaluable public resource for new method development, consisting of matched noisy, low-resolution images and high-quality, super-resolved, structured illumination microscopy (SIM, a variant of SR microscopy) reconstructions. Second, they introduce two DL architectures, termed deep Fourier channel attention networks (DFCAN) and deep Fourier generative adversarial networks (DFGAN), which, as their names imply, learn feature representations in the Fourier domain; and finally, they apply the networks to both the perennial problem of SIM reconstruction from nine low-quality images and the more fantastical concept of single-image SR (SISR)4, in which an SR image is inferred entirely from a single diffraction-limited, or lower resolution, image.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Do androids dream of electric cells?

References

  1. Qiao, C. Nat. Methods https://doi.org/10.1038/s41592-020-01048-5 (2021).

  2. McCulloch, W. S. & Pitts, W. Bull. Math. Biophys. 5, 115–133 (1943).

    Article  Google Scholar 

  3. Silver, D. et al. Nature 550, 354–359 (2017).

    Article  CAS  Google Scholar 

  4. Dong, C., Loy, C. C., He, K. & Tang, X. IEEE Trans. Pattern Anal. Mach. Intell. 38, 295–307 (2016).

    Article  Google Scholar 

  5. McElreath, R. Statistical Rethinking: A Bayesian Course with Examples in R and Stan (Taylor and Francis, 2020).

  6. Citron, D. Deep fakes: a looming challenge for privacy, democracy, and national security. Center for Internet and Society https://cyberlaw.stanford.edu/publications/deep-fakes-looming-challenge-privacy-democracy-and-national-security (2018).

  7. Wang, H. & Yeung, D.-Y. A survey on Bayesian deep learning. ACM Comput. Surv. 53, 1–37 (2020).

    Google Scholar 

  8. Eisenstein, M. Nat. Methods 17, 1075–1079 (2020).

    Article  CAS  Google Scholar 

  9. Hutson, M. Science https://doi.org/10.1126/science.aau0577 (2018).

  10. D’Amour, A. et al. Underspecification presents challenges for credibility in modern machine learning. Preprint at arXiv https://arxiv.org/abs/2011.03395 (2020).

Download references

Author information

Authors and Affiliations

  1. 10x Genomics, Pleasanton, CA, USA

    David P. Hoffman

  2. DrivenData Inc, Denver, CO, USA

    Isaac Slavitt & Casey A. Fitzpatrick

Authors
  1. David P. Hoffman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Isaac Slavitt
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Casey A. Fitzpatrick
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David P. Hoffman.

Ethics declarations

Competing interests

The authors declare no competing interests.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hoffman, D.P., Slavitt, I. & Fitzpatrick, C.A. The promise and peril of deep learning in microscopy. Nat Methods 18, 131–132 (2021). https://doi.org/10.1038/s41592-020-01035-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41592-020-01035-w

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing