Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Field-dependent deep learning enables high-throughput whole-cell 3D super-resolution imaging

Nature Methods volume 20, pages 459–468 (2023)Cite this article

Subjects

Abstract

Single-molecule localization microscopy in a typical wide-field setup has been widely used for investigating subcellular structures with super resolution; however, field-dependent aberrations restrict the field of view (FOV) to only tens of micrometers. Here, we present a deep-learning method for precise localization of spatially variant point emitters (FD-DeepLoc) over a large FOV covering the full chip of a modern sCMOS camera. Using a graphic processing unit-based vectorial point spread function (PSF) fitter, we can fast and accurately model the spatially variant PSF of a high numerical aperture objective in the entire FOV. Combined with deformable mirror-based optimal PSF engineering, we demonstrate high-accuracy three-dimensional single-molecule localization microscopy over a volume of ~180 × 180 × 5 μm3, allowing us to image mitochondria and nuclear pore complexes in entire cells in a single imaging cycle without hardware scanning; a 100-fold increase in throughput compared to the state of the art.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Schematic of FD-DeepLoc.
Fig. 2: Field-dependent aberration maps for an astigmatic PSF induced by a cylindrical lens.
Fig. 3: Performance of FD-DeepLoc on simulated datasets with field-dependent aberrations.
Fig. 4: Performance of FD-DeepLoc on experimental astigmatic 3D data of NPCs in a large FOV.
Fig. 5: FD-DeepLoc enables 3D super-resolution imaging of mitochondria within a large FOV (180 × 180 µm2) and DOF (5 μm).
Fig. 6: FD-DeepLoc with flexible DMO PSF engineering enables high quality 3D super-resolution imaging of thick neurites over a large FOV.

Similar content being viewed by others

Data availability

The data that support the findings of this study are publicly available at Zenodo. For Fig. 3, raw data, aberration maps, ground-truth list and example network models of the dataset with normal aberration and medium SNR are available at https://doi.org/10.5281/zenodo.7310921; For Fig. 4, raw data of two cropped ROIs, aberration maps and example network model are available at https://doi.org/10.5281/zenodo.7299483; the raw bead stacks are available at https://doi.org/10.5281/zenodo.7302064. For Fig. 6, the first 10,000 frames of the raw data, aberration maps and example network model are available at https://doi.org/10.5281/zenodo.7299064; the raw bead stacks are available at https://doi.org/10.5281/zenodo.7300138. For Supplementary Fig. 5, the raw data, uniform aberration maps and example network model are available at https://doi.org/10.5281/zenodo.7299197. The statistical source data for Figs. 1–4 and Extended Data Figs. 1, 5, 8 and 9 are attached. Other datasets are available from the corresponding author upon reasonable request. Source data are provided with this paper.

Code availability

Source code of FD-DeepLoc is provided in Supplementary Code 1 and updated versions are available at: https://github.com/Li-Lab-SUSTech/FD-DeepLoc. The microscope acquisition software was developed with Micro-manager v.2.0 (http://micro-manager.org) and EMU v.1.1 (https://github.com/jdeschamps/htSMLM). The custom localization data analysis and rendering software is available at https://github.com/jries/SMAP.

References

  1. Chi, W. et al. Descriptor Δ G C‐O Enables the quantitative design of spontaneously blinking rhodamines for live‐cell super‐resolution imaging. Angew. Chem. Int. Ed. 59, 20215–20223 (2020).

    Article  CAS  Google Scholar 

  2. Pegoraro, G. & Misteli, T. High-throughput imaging for the discovery of cellular mechanisms of disease. Trends Genet. 33, 604–615 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Holden, S. J. et al. High-throughput 3D super-resolution microscopy reveals Caulobacter crescentus in vivo Z-ring organization. Proc. Natl Acad. Sci. USA 111, 4566–4571 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Mahecic, D. et al. Homogeneous multifocal excitation for high-throughput super-resolution imaging. Nat. Methods 17, 726–733 (2020).

    Article  CAS  PubMed  Google Scholar 

  5. Beghin, A. et al. Localization-based super-resolution imaging meets high-content screening. Nat. Methods 14, 1184–1190 (2017).

    Article  CAS  PubMed  Google Scholar 

  6. Barentine, A. E. S. et al. An integrated platform for high-throughput nanoscopy. Preprint at bioRxiv https://doi.org/10.1101/606954 (2022).

  7. Du, Y. et al. Computational framework for generating large panoramic super-resolution images from localization microscopy. Biomed. Opt. Express 12, 4759 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  8. Zhao, Z., Xin, B., Li, L. & Huang, Z. High-power homogeneous illumination for super-resolution localization microscopy with large field-of-view. Opt. Express 25, 13382 (2017).

    Article  CAS  PubMed  Google Scholar 

  9. Diekmann, R. et al. Chip-based wide field-of-view nanoscopy. Nat. Photonics 11, 322–328 (2017).

    Article  CAS  Google Scholar 

  10. Douglass, K. M., Sieben, C., Archetti, A., Lambert, A. & Manley, S. Super-resolution imaging of multiple cells by optimized flat-field epi-illumination. Nat. Photonics 10, 705–708 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Mau, A., Friedl, K., Leterrier, C., Bourg, N. & Lévêque-Fort, S. Fast widefield scan provides tunable and uniform illumination optimizing super-resolution microscopy on large fields. Nat. Commun. 12, 3077 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Lohmann, A. W. & Paris, D. P. Space-variant image formation. J. Opt. Soc. Am. 55, 1007 (1965).

    Article  Google Scholar 

  13. Diezmann, A., von, Lee, M. Y., Lew, M. D. & Moerner, W. E. Correcting field-dependent aberrations with nanoscale accuracy in three-dimensional single-molecule localization microscopy. Optica 2, 985 (2015).

    Article  Google Scholar 

  14. Yan, T., Richardson, C. J., Zhang, M. & Gahlmann, A. Computational correction of spatially variant optical aberrations in 3D single-molecule localization microscopy. Opt. Express 27, 12582 (2019).

    Article  CAS  PubMed  Google Scholar 

  15. Möckl, L., Roy, A. R. & Moerner, W. E. Deep learning in single-molecule microscopy: fundamentals, caveats, and recent developments. Biomed. Opt. Express 11, 1633 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  16. Nehme, E. et al. DeepSTORM3D: dense 3D localization microscopy and PSF design by deep learning. Nat. Methods 17, 734–740 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Speiser, A. et al. Deep learning enables fast and dense single-molecule localization with high accuracy. Nat. Methods 18, 1082–1090 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Kim, T., Moon, S. & Xu, K. Information-rich localization microscopy through machine learning. Nat. Commun. 10, 1996 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  19. Nehme, E. et al. Learning optimal wavefront shaping for multi-channel imaging. IEEE Trans. Pattern Anal. Mach. Intell. 43, 2179–2192 (2021).

    Article  PubMed  Google Scholar 

  20. Zhang, P. et al. Analyzing complex single-molecule emission patterns with deep learning. Nat. Methods 15, 913–916 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. Islam, M. A., Jia, S. & Bruce, N. D. B. How much position information do convolutional neural networks encode? ? In Proceedings of International Conference on Learning Representations (ICLR, 2020).

  22. Liu, R. et al. in Advances in Neural Information Processing Systems (NeurIPS), Vol. 31, 9605–9616 (Curran Associates, 2018).

  23. Siemons, M., Hulleman, C. N., Thorsen, R. Ø., Smith, C. S. & Stallinga, S. High-precision wavefront control in point spread function engineering for single emitter localization. Opt. Express 26, 8397 (2018).

    Article  CAS  PubMed  Google Scholar 

  24. Ferdman, B. et al. VIPR: vectorial implementation of phase retrieval for fast and accurate microscopic pixel-wise pupil estimation. Opt. Express 28, 10179 (2020).

    Article  PubMed  Google Scholar 

  25. Leutenegger, M., Rao, R., Leitgeb, R. A. & Lasser, T. Fast focus field calculations. Opt. Express 14, 4897–4903 (2006).

    Article  Google Scholar 

  26. Li, Y. et al. Global fitting for high-accuracy multi-channel single-molecule localization. Nat. Commun. 13, 3133 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Möckl, L., Roy, A. R., Petrov, P. N. & Moerner, W. E. Accurate and rapid background estimation in single-molecule localization microscopy using the deep neural network BGnet. Proc. Natl Acad. Sci. USA 117, 60–67 (2020).

    Article  PubMed  Google Scholar 

  28. Novák, T., Gajdos, T., Sinkó, J., Szabó, G. & Erdélyi, M. TestSTORM: versatile simulator software for multimodal super-resolution localization fluorescence microscopy. Sci. Rep. 7, 951 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  29. Sage, D. et al. Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software. Nat. Methods 16, 387–395 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Li, Y. et al. Real-time 3D single-molecule localization using experimental point spread functions. Nat. Methods 15, 367–369 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Ries, J. SMAP: a modular super-resolution microscopy analysis platform for SMLM data. Nat. Methods 17, 870–872 (2020).

    Article  CAS  PubMed  Google Scholar 

  32. Thevathasan, J. V. et al. Nuclear pores as versatile reference standards for quantitative superresolution microscopy. Nat. Methods 16, 1045–1053 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  33. Pavani, S. R. P. et al. Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function. Proc. Natl Acad. Sci. USA 106, 2995–2999 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. Shechtman, Y., Sahl, S. J., Backer, A. S. & Moerner, W. E. Optimal point spread function design for 3D imaging. Phys. Rev. Lett. 113, 133902 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  35. Fu, S. et al. Deformable mirror-based optimal PSF engineering for 3D super-resolution imaging. Opt. Lett. 47, 3031 (2022).

    Article  CAS  PubMed  Google Scholar 

  36. Chaudhry, A., Shi, R. & Luciani, D. S. A pipeline for multidimensional confocal analysis of mitochondrial morphology, function, and dynamics in pancreatic β-cells. Am. J. Physiol. Endocrinol. Metab. 318, E87–E101 (2020).

    Article  CAS  PubMed  Google Scholar 

  37. Arthur, D. & Vassilvitskii, S. in Proceedings of the Annual ACM-SIAM Symposium on Discrete Algorithms, 1027–1035 (SODA, 2007).

  38. Xu, K., Zhong, G. & Zhuang, X. Actin, Spectrin, and associated proteins form a periodic cytoskeletal structure in axons. Science 339, 452–456 (2013).

    Article  CAS  PubMed  Google Scholar 

  39. Yanny, K., Monakhova, K., Shuai, R. W. & Waller, L. Deep learning for fast spatially varying deconvolution. Optica 9, 96 (2022).

    Article  Google Scholar 

  40. Xu, F. et al. Three-dimensional nanoscopy of whole cells and tissues with in situ point spread function retrieval. Nat. Methods 17, 531–540 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  41. El Beheiry, M. & Dahan, M. ViSP: representing single-particle localizations in three dimensions. Nat. Methods 10, 689–690 (2013).

    Article  CAS  PubMed  Google Scholar 

  42. Zhang, B., Zerubia, J. & Olivo-Marin, J.-C. Gaussian approximations of fluorescence microscope point-spread function models. Appl. Opt. 46, 1819 (2007).

    Article  PubMed  Google Scholar 

  43. Ronneberger, O. et al. in Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, Vol. 9351, 234–241 (Springer, 2015).

  44. Annibale, P., Vanni, S., Scarselli, M., Rothlisberger, U. & Radenovic, A. Quantitative photo-activated localization microscopy: unraveling the effects of photoblinking. PLoS ONE 6, e22678 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  45. Perlin, K. Improving noise. ACM Trans. Graph. 21, 681–682 (2002).

    Article  Google Scholar 

  46. Schröder, D., Deschamps, J., Dasgupta, A., Matti, U. & Ries, J. Cost-efficient open source laser engine for microscopy. Biomed. Opt. Express 11, 609 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  47. Deschamps, J. & Ries, J. EMU: reconfigurable graphical user interfaces for Micro-Manager. BMC Bioinform. 21, 456 (2020).

    Google Scholar 

  48. Zhang, Y. et al. Rapid single-step induction of functional neurons from human pluripotent stem cells. Neuron 78, 785–798 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  49. Thoma, E. C. et al. Ectopic expression of neurogenin 2 alone is sufficient to induce differentiation of embryonic stem cells into mature neurons. PLoS ONE 7, e38651 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  50. Zhang, Y. et al. Nanoscale subcellular architecture revealed by multicolor three-dimensional salvaged fluorescence imaging. Nat. Methods 17, 225–231 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  51. Zhou, R. et al. Proteomic and functional analyses of the periodic membrane skeleton in neurons. Nat. Commun. 13, 3196 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

This work was supported by the Key Technology Research and Development Program of Shandong (2021CXGC010212), Guangdong Natural Science Foundation Joint Fund (2020A1515110380), Shenzhen Science and Technology Innovation Commission (grant nos. KQTD20200820113012029 and KQTD20210811090115021), Guangdong Provincial Key Laboratory of Advanced Biomaterials (2022B1212010003), China National Postdoctoral Program for Innovative Talents (BX20220141) and a startup grant from Southern University of Science and Technology.

Author information

Author notes

  1. These authors contributed equally: Shuang Fu, Wei Shi

Authors and Affiliations

  1. Guangdong Provincial Key Laboratory of Advanced Biomaterials, Department of Biomedical Engineering, Southern University of Science and Technology, Shenzhen, China

    Shuang Fu, Wei Shi, Tingdan Luo, Yingchuan He, Lulu Zhou, Jie Yang, Zhichao Yang, Zhiyong Guo, Chengyu Yang, Chao Liu, Peng Xi, Dayong Jin & Yiming Li

  2. School of Life Sciences, Southern University of Science and Technology, Shenzhen, China

    Jiadong Liu, Xiaotian Liu & Mingjie Zhang

  3. Key Laboratory of Biomedical Engineering of Hainan Province, School of Biomedical Engineering, Hainan University, Haikou, China

    Zhen-li Huang

  4. European Molecular Biology Laboratory, Cell Biology and Biophysics, Heidelberg, Germany

    Jonas Ries

  5. Department of Biomedical Engineering, College of Future Technology, Peking University, Beijing, China

    Peng Xi

  6. Institute for Biomedical Materials and Devices (IBMD), Faculty of Science, University of Technology Sydney, Sydney, New South Wales, Australia

    Dayong Jin

Authors
  1. Shuang Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tingdan Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yingchuan He
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lulu Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jie Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Zhichao Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jiadong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Xiaotian Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Zhiyong Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Chengyu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Chao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Zhen-li Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Jonas Ries
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Mingjie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Peng Xi
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Dayong Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Yiming Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Y.L. conceived the concept and supervised the project. S.F. and W.S. developed the deep-learning-based algorithms. Y.L. and T.L. developed the GPU-based vectorial PSF fitter. W.S. developed the field-dependent PSF calibration software. J.Y., Y.L., Y.H., Z.H., Z.Y. and Z.G. designed the optical setup. W.S., L.Z., C.Y. and C.L. prepared the U2OS and COS-7 cell samples. J.L., X.L. and M.Z. contributed the induced neuronal cells from mESCs. W.S. and L.Z. labeled the cells and acquired the data. Y.H. and S.W. set up the microscope hardware. S.F., S.W. and T.L. analyzed the data. Y.L., S.F., W.S., J.R., P.X. and D.J. wrote the manuscript with the input from all other authors.

Corresponding author

Correspondence to Yiming Li.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Methods thanks Thanh-An Pham and the other, anonymous, reviewers for their contribution to the peer review of this work. Primary Handling Editor: Rita Strack, in collaboration with the Nature Methods team. Peer reviewer reports are available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Uniform illumination of large FOV.

a, Full frame (1608 ×1608 pixels) imaging of BG-AF647 dye solution (S9136S, New England Biolabs, 1:1000 dilution from stock in Milli-Q water) on coverslip. b, Intensity profile of the yellow dashed line in a. c, Example image of beads (T7279, Invitrogen, TetraSpeck) on coverslip (defocused by 300 nm) under uniform illumination. d, Zoomed image of the yellow rectangle indicated in c. Representative results are shown from 7 experiments. Scale bars, 20μm.

Source data

Extended Data Fig. 2 Comparison of experimental and fitted PSF models across the FOV.

Experiment and fitted astigmatism PSF models (a-e) and DMO Tetrapod PSF models (f-j) acquired from different positions in the full frame image (1608 × 1608 pixels). a1, Fitted pupil function of bead stacks locate at the left top position (79, 136). a2, Comparison of the experimental and fitted PSF model within ± 600 nm axial range. b-e are the same as a, but with different locations: b right top (1417, 22); c left bottom (246, 1588); d right bottom (1439, 1503); e middle (812, 805). f-j are experiment and fitted DMO Tetrapod PSF models within ± 3 μm axial range acquired from different positions: f left top (139, 75); g right top (1329, 206); h left bottom (177, 1422); i right bottom (1444, 1577); j middle (855, 810). Scale bars, 1 µm.

Extended Data Fig. 3 FD-DeepLoc architecture.

a, Overall network architecture of FD-DeepLoc. Similar to the DECODE network, FD-DeepLoc contains two U-Nets as indicated by the gray box. For each input frame, features will be extracted by the frame analysis module and then concatenated together to the temporal context module. Finally, a ten-channel map will be output to form the emitter predictions of each frame. The output of each layer is depicted with colorful blocks, where h and w represent the height and width of the channels. b, CoordConv channels. The CoordConv layer is placed in the first convolution layer of each module and has two convolution operations. One convolution (3×3) is operated on the input image. The other one (1×1) is operated on two coordinate channels. The results are then added as the output of the Coordconv layer.

Extended Data Fig. 4 Effects of PSF model mismatch between training and test dataset.

The training dataset is generated by PSF model with astigmatism aberration. Reconstructed images for test data using PSF models with astigmatism aberration (a, b, c), astigmatism plus spherical aberration (d, e, f) and astigmatism plus spherical and coma aberration (g, h, i) are shown. The performance of network with different loss function are compared: original DECODE loss function (a, d, g), original DECODE loss function plus cross entropy term (b, e, h) and original DECODE loss function plus cross entropy term and robust training (c, f, i). For robust training, small random aberrations were added (Methods). The signal to background ratio for test data is set as 5,500 average photons per emitter (sampled from a uniform distribution within [1,000, 10,000]) and 50 background photons per pixel. The rms wavefront error of spherical aberration and coma aberration are both set as 20 nm (λ = 660 nm). The ground truth coordinates are taken from the SMLM challenge training dataset MT029. Scale bar, 2 µm.

Extended Data Fig. 5 Comparison of localization accuracy for single molecules with field dependent aberrations analyzed by FD-DeepLoc and DECODE.

a, Localization accuracy of a left top single molecule at different axial positions analyzed by FD-DeepLoc and DECODE. Both networks were trained using the same training data containing all spatially variant PSFs. The bottom panel of a shows the corresponding 3D PSF. b-e are the same as a, but at different locations: b right top; c left bottom; d right bottom; e middle. The data is simulated using the aberration maps in Supplementary Fig. 6. 5,000 photons and 50 background were used for each single molecule. 3,000 single-emitter images are generated with random x, y positions at each axial position (x, y is random within a pixel, z is random within a 50 nm step, r.m.s.e. is averaged over 50 nm bins).

Source data

Extended Data Fig. 6 FD-DeepLoc enables large FOV 3D super-resolution imaging of nuclear pores with high fidelity.

a, Overview of the panoramic 3D super-resolution image of Nup96-SNAP-Alxea Fluor 647. b, Top views of the NPCs reconstructed with FD-DeepLoc, DECODE and Cspline. b1, Zoomed view of the region indicated by the white dashed box in a. b2, b3 and b4 are the zoomed images of the rectangle region indicated in b1 reconstructed by FD-DeepLoc, DECODE and Cspline separately. c, Side view images of the region bounded by the 500 nm dashed lines in b1 reconstructed by FD-DeepLoc, Cspline and DECODE separately. Representative results are shown from 3 experiments. Scale bars, 50 μm (a), 5 μm (b1,) and 1 μm (b2, c1).

Extended Data Fig. 7 Performance of FD-DeepLoc on 3D astigmatism imaging of mitochondria in a large FOV.

a, Overview of the panoramic 3D super-resolution image of immunolabeled TOM 20 in mitochondria. b1, c1, Zoomed images of the center and marginal areas as indicated by the dashed boxes b, c in a. b2, b3, Side-view cross-section of the region bounded by the dashed lines in b1 reconstructed by FD-DeepLoc and Cspline, separately. c2, c3 the same as b2, b3, but for the region bounded by the dashed lines in c1. Representative results are shown from 2 experiments. Scale bars, 50 μm (a), 10 μm (b1, c1) and 1 μm (b2, c2).

Extended Data Fig. 8 FD-DeepLoc enables large FOV whole cell 3D super-resolution imaging of NPC.

a, Overview of the panoramic whole cell 3D super-resolution image of NPC. b, Different axial sections of a. b1-3 are top views of axial range of (0 μm–2 μm), (2 μm–3 μm) and (3 μm – 5 μm), respectively. c, Magnified views of areas denoted by box c in a. c1, Top view of the box c area denoted in a. c2, Side-view cross-section of region denoted by the dashed line in c1. c3-4 are zoomed views of the top and bottom surface of the boxed area denoted in c1. d, Same as c for magnified views of region denoted by box d in a. e, The distribution of the fitted radii of the NPCs for the cell in c. e1 and e2 are for the top and bottom nucleus surface, respectively. f is the same as e, but for the cell in d. Representative results are shown from 7 experiments. Scale bars, 50 μm (a, b1), 5 μm (c1, c2, d1, d2) and 1 μm (c3, d3).

Source data

Extended Data Fig. 9 Quantitative analysis of whole cell 3D super-resolved mitochondria images.

a, Overview of 16 full frame images of whole cell 3D super-resolved mitochondria. b, Magnified angled views of 3 representative cells denoted by the boxes in a. c, Quantitative analysis of mitochondria morphology and network connectivity using Mitochondria Analyzer36. n = 121 cells imaged from 16 independent experiments and then classified by k-means++ into 3 groups. The numbers of cells for type 1/2/3 are 47/22/52, respectively. One-way ANOVA with two-sided Bonferroni multiple comparison test, the significance level was set as 0.05. The adjusted P-value for pairwise comparison is shown in each sub-panel of c. Boxplots are drawn from the 25th to 75th percentile with the horizontal line denoted the median and the whiskers extending to the minimum and maximum. Values outside 1.5 times interquartile range are defined as outliers which are shown as magenta circle. d, Principal component analysis was performed on the features used in c with the first 3 principal components plotted, color indicates the categorization results. Scale bars, 50 μm (a) and 10 μm (b1).

Source data

Extended Data Fig. 10 Layout of custom-built microscope used for this study.

M: mirror, DiM: dichroic mirror, FM: flip mirror, L: lens, TS: translation stage, FC: fiber coupler, MM fiber: multi-mode fiber, SM fiber: single-mode fiber, AP: aperture, BFP: back focal plane, FW: filter-wheel, TBL: tube lens, QPD: quadrant photodiode, DM: deformable mirror. a, Optical path of the microscope. Our setup has two excitation options: single-mode (SM) excitation and multi-mode (MM) excitation. In this work, we mainly use the MM mode for large FOV excitation. Excitation lasers are firstly reflected by dichroic mirrors DiM1-DiM3 (DMLP 425/505/605, Thorlabs) and then coupled by lens L1 into a multi-mode fiber (WFRCT200x200-230×230-440-620-1100 N, NA = 0.22, CeramOptec). The fiber is bound with a vibrator to generate a homogeneous illumination8,46. There are two imaging paths for the fluorescence detection which is separated by a flip mirror FM3. In the reflection path of FM3 (KSHM90, Owis), cylindrical lens (LJ1516L1-A, Thorlabs) based astigmatism 3D super-resolution imaging was performed. For the other imaging path without FM3, DM engineered PSFs were used for 3D super-resolution imaging. The focal length for each lens: L1 (f = 19 mm, Ø0.5 inch), L2 (f = 75 mm, Ø1 inch), L3 (PLN 10X/NA0.25, Olympus), L4(f = 400 mm, Ø1 inch), L5 (f = 35 mm, Ø1 inch), L6 (f = 400 mm, Ø75 mm), L7 (f = 400 mm, Ø75 mm), TBL (f = 180 mm, TTL180-A, Thorlabs), L8 (f = 150 mm, Ø2 inch), L9 (f = 150 mm, Ø2 inch), L10 (f = 75 mm, Ø1 inch), L11 (f = 150 mm, Ø2 inch), L12 (f = 150 mm, Ø2 inch), L13 (f = 1000 mm, LJ1516L1-A, Thorlabs). b, The rendered mechanical design using SolidWorks (Dassault Systèmes). Inset is the ray-tracing diagram of the imaging path rendered in OpticStudio (Zemax).

Supplementary information

Supplementary Information

Supplementary Figs. 1–14 and Supplementary Notes 1–6.

Reporting Summary

Peer Review File

A tour view animation of the large-FOV 3D super-resolution image of NPC in Fig. 4.

A tour view animation of the large-FOV 3D super-resolution image of mitochondria in Extended Data Fig. 7.

Panoramic view animation of the large-FOV and DOF 3D super-resolution image of mitochondria in Fig. 5.

Animation of a single cell in the large-FOV and DOF 3D super-resolution image of mitochondria in Fig. 5.

Panoramic view animation of the large-FOV and DOF 3D super-resolution image of NPC in Extended Data Fig. 8.

Animation of a single cell in the large-FOV and DOF 3D super-resolution image of NPC in Extended Data Fig. 8.

A tour view animation of the large-FOV and DOF 3D super-resolution image of neurites in Fig. 6.

Supplementary Code 1

Example and source code for FD-DeepLoc.

Source data

Source Data Fig. 1

Statistical Source Data.

Source Data Fig. 2

Statistical Source Data.

Source Data Fig. 3

Statistical Source Data.

Source Data Fig. 4

Statistical Source Data.

Source Data Extended Data Fig. 1

Statistical Source Data.

Source Data Extended Data Fig. 5

Statistical Source Data.

Source Data Extended Data Fig. 8

Statistical Source Data.

Source Data Extended Data Fig. 9

Statistical Source Data.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fu, S., Shi, W., Luo, T. et al. Field-dependent deep learning enables high-throughput whole-cell 3D super-resolution imaging. Nat Methods 20, 459–468 (2023). https://doi.org/10.1038/s41592-023-01775-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41592-023-01775-5

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing