Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Real-time multi-angle projection imaging of biological dynamics

Nature Methods volume 18, pages 829–834 (2021)Cite this article

Subjects

Abstract

We introduce a cost-effective and easily implementable scan unit that converts any camera-based microscope with optical sectioning capability into a multi-angle projection imaging system. Projection imaging reduces data overhead and accelerates imaging by a factor of >100, while also allowing users to readily view biological phenomena of interest from multiple perspectives on the fly. By rapidly interrogating the sample from just two perspectives, our method also enables real-time stereoscopic imaging and three-dimensional particle localization. We demonstrate projection imaging with spinning disk confocal, lattice light-sheet, multidirectional illumination light-sheet and oblique plane microscopes on specimens that range from organelles in single cells to the vasculature of a zebrafish embryo. Furthermore, we leverage our projection method to rapidly image cancer cell morphodynamics and calcium signaling in cultured neurons at rates up to 119 Hz as well as to simultaneously image orthogonal views of a beating embryonic zebrafish heart.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Principle of multi-angle projection imaging.
Fig. 2: Projection imaging applied to various microscopes.
Fig. 3: Dynamic projection imaging.
Fig. 4: Simultaneous multi-angle viewing.

Data availability

The data presented in this paper are publicly available on Zenodo at https://doi.org/10.5281/zenodo.4720252.

Code availability

The MATLAB scripts used in this paper are available at https://github.com/AdvancedImagingUTSW.

References

  1. Quirin, S. et al. Calcium imaging of neural circuits with extended depth-of-field light-sheet microscopy. Opt. Lett. 41, 6080 (2016).

    Article  Google Scholar 

  2. Shechtman, Y., Weiss, L. E., Backer, A. S., Sahl, S. J. & Moerner, W. E. Precise three-dimensional scan-free multiple-particle tracking over large axial ranges with tetrapod point spread functions. Nano Lett. 15, 4194–4199 (2015).

    Article  CAS  Google Scholar 

  3. Abrahamsson, S., Usawa, S. & Gustafsson, M. A new approach to extended focus for high-speed high-resolution biological microscopy. Proc. SPIE 6090, Three-Dimensional and Multidimensional Microscopy: Image Acquisition and Processing XIII, 60900N (eds Conchello, J.-A. et al.) https://www.spiedigitallibrary.org/conference-proceedings-of-spie/6090/60900N/A-new-approach-to-extended-focus-for-high-speed-high/10.1117/12.647022.short (2006).

  4. Shain, W. J., Vickers, N. A., Goldberg, B. B., Bifano, T. & Mertz, J. Extended depth-of-field microscopy with a high-speed deformable mirror. Opt. Lett. 42, 995–998 (2017).

    Article  CAS  Google Scholar 

  5. Tomer, R. et al. SPED light sheet microscopy: fast mapping of biological system structure and function. Cell 163, 1796–1806 (2015).

    Article  CAS  Google Scholar 

  6. Lu, R. et al. Video-rate volumetric functional imaging of the brain at synaptic resolution. Nat. Neurosci. 20, 620–628 (2017).

    Article  CAS  Google Scholar 

  7. Olarte, O. E., Andilla, J., Artigas, D. & Loza-Alvarez, P. Decoupled illumination detection in light sheet microscopy for fast volumetric imaging. Optica 2, 702–705 (2015).

    Article  Google Scholar 

  8. Woringer, M., Darzacq, X., Zimmer, C. & Mir, M. Faster and less phototoxic 3D fluorescence microscopy using a versatile compressed sensing scheme. Opt. Express 25, 13668 (2017).

    Article  Google Scholar 

  9. Liu, L. et al. Superresolution along extended depth of focus with binary-phase filters for the Gaussian beam. J. Opt. Soc. Am. A. 25, 2095–2101 (2008).

    Article  Google Scholar 

  10. Hu, X., Jalal, S., Sheetz, M., Bakke, O. & Margadant, F. Micro-stepping extended focus reduces photobleaching and preserves structured illumination super-resolution features. J. Cell Sci. 133, jcs240796 (2020).

    Article  CAS  Google Scholar 

  11. Sharpe, J. Optical projection tomography as a tool for 3D microscopy and gene expression studies. Science 296, 541–545 (2002).

    Article  CAS  Google Scholar 

  12. Chen, L. et al. Simultaneous angular multiplexing optical projection tomography at shifted focal planes. Opt. Lett. 38, 851–853 (2013).

    Article  Google Scholar 

  13. Chen, L. et al. Mesoscopic in vivo 3-D tracking of sparse cell populations using angular multiplexed optical projection tomography. Biomed. Opt. Express 6, 1253–1261 (2015).

    Article  Google Scholar 

  14. Lacroute, P. & Levoy, M. Fast volume rendering using a shear-warp factorization of the viewing transformation. in Proc. 21st Annual Conference on Computer Graphics and Interactive Techniques: SIGGRAPH ‘94 451–458 (1994).

  15. Chen, B.-C. et al. Lattice light-sheet microscopy: imaging molecules to embryos at high spatiotemporal resolution. Science 346, 1257998 (2014).

    Article  Google Scholar 

  16. Dunsby, C. Optically sectioned imaging by oblique plane microscopy. Opt. Express 16, 20306–20316 (2008).

    Article  CAS  Google Scholar 

  17. Sapoznik, E. et al. A versatile oblique plane microscope for large-scale and high-resolution imaging of subcellular dynamics. eLife 9, e57681 (2020).

    Article  CAS  Google Scholar 

  18. Haslehurst, P., Yang, Z., Dholakia, K. & Emptage, N. Fast volume-scanning light sheet microscopy reveals transient neuronal events. Biomed. Opt. Express 9, 2154–2167 (2018).

    Article  CAS  Google Scholar 

  19. Bingen, P., Reuss, M., Engelhardt, J. & Hell, S. W. Parallelized STED fluorescence nanoscopy. Opt. Express 19, 23716–23726 (2011).

    Article  CAS  Google Scholar 

  20. Chang, B.-J. et al. Universal light-sheet generation with field synthesis. Nat. Methods 16, 235–238 (2019).

    Article  CAS  Google Scholar 

  21. Charras, G. T., Coughlin, M., Mitchison, T. J. & Mahadevan, L. Life and times of a cellular bleb. Biophys. J. 94, 1836–1853 (2008).

    Article  CAS  Google Scholar 

  22. Kumar, M., Kishore, S., Nasenbeny, J., McLean, D. L. & Kozorovitskiy, Y. Integrated one- and two-photon scanned oblique plane illumination (SOPi) microscopy for rapid volumetric imaging. Opt. Express 26, 13027–13041 (2018).

    Article  CAS  Google Scholar 

  23. Voleti, V. et al. Real-time volumetric microscopy of in vivo dynamics and large-scale samples with SCAPE 2.0. Nat. Methods 16, 1054–1062 (2019).

    Article  CAS  Google Scholar 

  24. Delarue, M. et al. mTORC1 controls phase separation and the biophysical properties of the cytoplasm by tuning crowding. Cell 174, 338–349.e320 (2018).

    Article  CAS  Google Scholar 

  25. Taylor, J. M., Girkin, J. M. & Love, G. D. High-resolution 3D optical microscopy inside the beating zebrafish heart using prospective optical gating. Biomed. Opt. Express 3, 650–660 (2012).

    Article  Google Scholar 

  26. Liebling, M., Forouhar, A. S., Gharib, M., Fraser, S. E. & Dickinson, M. E. Four-dimensional cardiac imaging in living embryos via postacquisition synchronization of nongated slice sequences. J. Biomed. Opt. 10, 054001 (2005).

    Article  Google Scholar 

  27. Mickoleit, M. et al. High-resolution reconstruction of the beating zebrafish heart. Nat. Methods 11, 919–922 (2014).

    Article  CAS  Google Scholar 

  28. Levoy, M., Ng, R., Adams, A., Footer, M. & Horowitz, M. Light field microscopy. in Proc. ACM SIGGRAPH 2006 (2006).

  29. Yang, B. et al. Epi-illumination SPIM for volumetric imaging with high spatial-temporal resolution. Nat. Methods 16, 501–504 (2019).

    Article  CAS  Google Scholar 

  30. Millett-Sikking, A. et al. High NA single-objective light-sheet. Zenodo https://doi.org/10.5281/zenodo.3376243 (2019).

  31. Murali, V. S. et al. An image-based assay to quantify changes in proliferation and viability upon drug treatment in 3D microenvironments. BMC Cancer 19, 502 (2019).

    Article  Google Scholar 

  32. Venkateswaran, N. et al. MYC promotes tryptophan uptake and metabolism by the kynurenine pathway in colon cancer. Genes Dev. 33, 1236–1251 (2019).

    Article  CAS  Google Scholar 

  33. Dean, K. M. et al. Imaging subcellular dynamics with fast and light-efficient volumetrically parallelized microscopy. Optica 4, 263–271 (2017).

    Article  Google Scholar 

  34. Miura, K. Bleach correction ImageJ plugin for compensating the photobleaching of time-lapse sequences. F1000Res. 9, 1494 (2020).

    Article  Google Scholar 

  35. Schindelin, J. et al. Fiji: an open-source platform for biological-image analysis. Nat. Methods 9, 676–682 (2012).

    Article  CAS  Google Scholar 

Download references

Acknowledgements

We thank D. Saucier for the help with the preparation of the zebrafish samples, and L. Perez-Castro and the Sorrell laboratory for providing the colorectal cancer cells. We thank the Cancer Prevention Research Institute of Texas (grant no. RR160057 to R.F.), and the National Institutes of Health (grant nos. T32CA080621 to M.K., F32GM117793 to K.M.D., K25CA204526 to E.S.W. and R35GM133522 to R.F.). P.R. acknowledges support through R35GM136428 to Gaudenz Danuser. J.D.M. thanks E. Derivery for access to the spinning disk microscope, R. Mittal for HeLa cells and A. Fowle for fabricating the custom mirror mount. J.D.M. acknowledges support from Fitzwilliam College, Cambridge, through a Research Fellowship.

Author information

Author notes

  1. These authors contributed equally: Bo-Jui Chang, James D. Manton.

Authors and Affiliations

  1. Department of Cell Biology, University of Texas Southwestern Medical Center, Dallas, TX, USA

    Bo-Jui Chang, Etai Sapoznik, Erik S. Welf, Vasanth S. Murali, Kevin M. Dean & Reto Fiolka

  2. MRC Laboratory of Molecular Biology, Cambridge, UK

    James D. Manton

  3. Department of Molecular Genetics, UT Southwestern Medical Center, Dallas, TX, USA

    Theresa Pohlkamp & Tamara S. Terrones

  4. Lyda Hill Department of Bioinformatics, University of Texas Southwestern Medical Center, Dallas, TX, USA

    Erik S. Welf, Vasanth S. Murali, Philippe Roudot, Kevin M. Dean & Reto Fiolka

  5. Calico Life Sciences LLC, South San Francisco, CA, USA

    Kayley Hake & Andrew G. York

  6. Walter and Eliza Hall Institute of Medical Research, Parkville, Victoria, Australia

    Lachlan Whitehead

  7. Department of Medical Biology, University of Melbourne, Parkville, Victoria, Australia

    Lachlan Whitehead

Authors
  1. Bo-Jui Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. James D. Manton
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Etai Sapoznik
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Theresa Pohlkamp
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tamara S. Terrones
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Erik S. Welf
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Vasanth S. Murali
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Philippe Roudot
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Kayley Hake
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Lachlan Whitehead
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Andrew G. York
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Kevin M. Dean
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Reto Fiolka
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

R.F. conceived and built the scan unit. R.F., B-J.C. and J.D.M. designed, built and operated the microscopes. R.F. and A.G.Y. designed multi-angle research. E.S. and E.S.W. provided MV3 cells. V.S.M. prepared tumor spheroids, T.P. and T.S.T. provided primary neurons. R.F. and J.D.M derived the relationship between rotated projections and sheared projections of volumes. L.W. and K.H. developed animations. J.D.M. derived equations for localization precision. P.R. performed the 3D reconstruction from two projections. R.F. and K.M.D. wrote the manuscript. All authors read and provided feedback on the final manuscript.

Corresponding author

Correspondence to Reto Fiolka.

Ethics declarations

Competing interests

R.F. filed a patent for the scan unit and its applications to microscopy, and K.M.D. has an investment interest in Discovery Imaging Systems, LLC.

Additional information

Peer review information Nature Methods thanks Ulrich Kubitscheck, Manish Kumar and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Rita Strack was the primary editor on this article and managed its editorial process and peer review in collaboration with the rest of the editorial team.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Comparison of simulations of projection method and extended-depth-of-field techniques.

Given a raw volume of the mitochondrial membranes within a HeLa cell (sum projection and depth colour-coding in a), simulations show the corresponding image obtained through b OPM-Pro, as described in the main text, c wide-field-Pro, d reducing the numerical aperture of the objective from 1.35 to 0.5, e using a 12-zone layer cake mask, f using a 6-alpha cubic phase plate. Beneath each ‘raw’ image, the result of deconvolving the image with 200 iterations of the Richardson-Lucy algorithm is shown, labelled ‘RL200’. Cyan insets show a 3× magnified view, while magenta insets show the sum projection of the point spread function used.

Extended Data Fig. 2 Spectral analysis of simulated projection method and extended-depth-of-field techniques.

Logarithmically scaled, normalized Fourier power spectra for corresponding raw images in Extended Data Fig. 1: a cubic phase mask, b layer cake mask, c projection method applied to a wide-field microscope, d projection method applied to the oblique plane microscope described in the main text, e a wide-field microscope of lower numerical aperture (0.5). f shows radially averaged profiles of all spectra, up to the diffraction limit.

Extended Data Fig. 3 Deskewing of LLSM and OPM data.

Schematic representation of sample scanning in a lattice light-sheet microscopy and b OPM. The z-axis represents the optical axis of the detection objective in LLSM and the optical axis of the primary objective in OPM. In both cases the scan direction is diagonal to the detection axis. As such, the data is sheared when a stack is acquired, which is usually compensated computationally by applying the opposite amount of shearing (also known as ‘deskewing’). c shows raw image data from LLSM obtained using sample scanning. d shows the same data after numerical deskewing. For the work presented here, the tilt angle of the coverslip in LLSM was 30 degree relative to the nominal focal plane, For OPM, the imaging plane was tilted 30 degrees relative to the nominal focal plane of the primary objective.

Extended Data Fig. 4 Equivalence of projections from rotated and sheared volumes.

Numerical simulation of the rotation of a 2D image and subsequent sum-projection, and comparison to numerical shearing of a 2D image and subsequent sum-projection. To compare the two line-profiles, the projection from the sheared image was compressed to the same length as the projection from the rotated image. Small differences between the two profiles may have arisen from numerical errors during the shearing and rotation operations.

Extended Data Fig. 5 Design of the scan unit.

CAD rendering of the galvanometric scan unit (gray), which attaches to the c-mount of a sCMOS camera (black).

Extended Data Fig. 6 Point spread functions under different viewing angles.

a point spread function for LLSM-Pro using 100 nm beads in agarose. LLSM was operated in sample scanning mode, with the scan direction along the optical axis of the detection objective. b Point spread function for OPM-Pro, using 100 nm beads on a coverslip. c Point spread function for mSPIM-Pro using 100 nm beads embedded in agarose. mSPIM was operated using mechanical scanning of the objective and galvo scanning of the light-sheet. d Point spread function for SDCM-Pro using 100 nm beads on a coverslip. The SDCM was operated using sample scanning with a piezo stage. Angle definition: a, c, d: Angle is indicated relative to the optical axis of the detection objective. b: Angle is indicated relative to the detection axis of the OPM system (see also Extended Data Fig. 3). At −30°, the viewing direction is normal to the coverslip. Brightness/contrast adjustment has been applied to the data as described in the method. Scale bar: 1 micron.

Extended Data Fig. 7 Comparison of our projection method and numerical projections.

a Single projection obtained with LLSM-Pro. b Sum projection of a conventional 3D stack. c Maximum intensity projection of a conventional 3D stack. d 2D deconvolution of a projection obtained with LLSM-Pro. e Sum projection of conventional 3D stack after 3D deconvolution. f Maximum intensity projection of conventional 3D stack after 3D deconvolution. Scale bars: 10 microns.

Extended Data Fig. 8 3D localization from two views compared with ground truth data from a conventional 3D stack.

Maximum intensity projection of a ground truth data set of genetically encoded monomeric nanoparticles in a fixed MV3 cell (gray) and the localizations of these particles in the 3D stack (blue circles). Yellow circles mark where the reconstruction from two projections overlapped, whereas red circles mark reconstructions that do not match up with the ground truth. Scale bar: 10 microns.

Extended Data Fig. 9 Zebrafish heart imaging by acquiring conventional 3D stack.

The same zebrafish heart shown in Fig. 4f was imaged with a conventional 3D z-stacking. A maximum intensity projection (a) and a sum projection (b) of the stack in xy (top row) and xz (bottom row) are shown. The exposure time is 25ms (40Hz frame rate), the z step size is 0.312 microns, and there are 322 z slices in the stack. Scale bars: 20 microns.

Extended Data Fig. 10 Extended depth of focus vs projection method.

a Schematic representation of an imaging volume of height Z, imaged with an extended depth of focus (EDF) PSF. The EDF PSF is n-fold extended in the axial direction. b Imaging the same volume while rapidly sweeping a diffraction limited PSF and a light-sheet through the same volume. Since the PSF in (a) is n-fold stretched compared to the diffraction limited PSF in b, the Strehl ratio (that is the peak intensity of the PSF) is at least n-fold reduced compared to b. For a, the whole sample is illuminated during the exposure time. In b, any part of the sample is illuminated 1/n-th of the exposure time. Assuming linear fluorescence response to excitation light-intensity, the lower Strehl ratio of the EDF PSF is compensated by the longer effective exposure time of any part of the sample.

Supplementary information

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chang, BJ., Manton, J.D., Sapoznik, E. et al. Real-time multi-angle projection imaging of biological dynamics. Nat Methods 18, 829–834 (2021). https://doi.org/10.1038/s41592-021-01175-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41592-021-01175-7

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing