We introduce a cost-effective and easily implementable scan unit that converts any camera-based microscope with optical sectioning capability into a multi-angle projection imaging system. Projection imaging reduces data overhead and accelerates imaging by a factor of >100, while also allowing users to readily view biological phenomena of interest from multiple perspectives on the fly. By rapidly interrogating the sample from just two perspectives, our method also enables real-time stereoscopic imaging and three-dimensional particle localization. We demonstrate projection imaging with spinning disk confocal, lattice light-sheet, multidirectional illumination light-sheet and oblique plane microscopes on specimens that range from organelles in single cells to the vasculature of a zebrafish embryo. Furthermore, we leverage our projection method to rapidly image cancer cell morphodynamics and calcium signaling in cultured neurons at rates up to 119 Hz as well as to simultaneously image orthogonal views of a beating embryonic zebrafish heart.
This is a preview of subscription content, access via your institution
Open Access articles citing this article.
npj Imaging Open Access 28 November 2023
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 print issues and online access
$259.00 per year
only $21.58 per issue
Rent or buy this article
Prices vary by article type
Prices may be subject to local taxes which are calculated during checkout
The data presented in this paper are publicly available on Zenodo at https://doi.org/10.5281/zenodo.4720252.
The MATLAB scripts used in this paper are available at https://github.com/AdvancedImagingUTSW.
Quirin, S. et al. Calcium imaging of neural circuits with extended depth-of-field light-sheet microscopy. Opt. Lett. 41, 6080 (2016).
Shechtman, Y., Weiss, L. E., Backer, A. S., Sahl, S. J. & Moerner, W. E. Precise three-dimensional scan-free multiple-particle tracking over large axial ranges with tetrapod point spread functions. Nano Lett. 15, 4194–4199 (2015).
Abrahamsson, S., Usawa, S. & Gustafsson, M. A new approach to extended focus for high-speed high-resolution biological microscopy. Proc. SPIE 6090, Three-Dimensional and Multidimensional Microscopy: Image Acquisition and Processing XIII, 60900N (eds Conchello, J.-A. et al.) https://www.spiedigitallibrary.org/conference-proceedings-of-spie/6090/60900N/A-new-approach-to-extended-focus-for-high-speed-high/10.1117/12.647022.short (2006).
Shain, W. J., Vickers, N. A., Goldberg, B. B., Bifano, T. & Mertz, J. Extended depth-of-field microscopy with a high-speed deformable mirror. Opt. Lett. 42, 995–998 (2017).
Tomer, R. et al. SPED light sheet microscopy: fast mapping of biological system structure and function. Cell 163, 1796–1806 (2015).
Lu, R. et al. Video-rate volumetric functional imaging of the brain at synaptic resolution. Nat. Neurosci. 20, 620–628 (2017).
Olarte, O. E., Andilla, J., Artigas, D. & Loza-Alvarez, P. Decoupled illumination detection in light sheet microscopy for fast volumetric imaging. Optica 2, 702–705 (2015).
Woringer, M., Darzacq, X., Zimmer, C. & Mir, M. Faster and less phototoxic 3D fluorescence microscopy using a versatile compressed sensing scheme. Opt. Express 25, 13668 (2017).
Liu, L. et al. Superresolution along extended depth of focus with binary-phase filters for the Gaussian beam. J. Opt. Soc. Am. A. 25, 2095–2101 (2008).
Hu, X., Jalal, S., Sheetz, M., Bakke, O. & Margadant, F. Micro-stepping extended focus reduces photobleaching and preserves structured illumination super-resolution features. J. Cell Sci. 133, jcs240796 (2020).
Sharpe, J. Optical projection tomography as a tool for 3D microscopy and gene expression studies. Science 296, 541–545 (2002).
Chen, L. et al. Simultaneous angular multiplexing optical projection tomography at shifted focal planes. Opt. Lett. 38, 851–853 (2013).
Chen, L. et al. Mesoscopic in vivo 3-D tracking of sparse cell populations using angular multiplexed optical projection tomography. Biomed. Opt. Express 6, 1253–1261 (2015).
Lacroute, P. & Levoy, M. Fast volume rendering using a shear-warp factorization of the viewing transformation. in Proc. 21st Annual Conference on Computer Graphics and Interactive Techniques: SIGGRAPH ‘94 451–458 (1994).
Chen, B.-C. et al. Lattice light-sheet microscopy: imaging molecules to embryos at high spatiotemporal resolution. Science 346, 1257998 (2014).
Dunsby, C. Optically sectioned imaging by oblique plane microscopy. Opt. Express 16, 20306–20316 (2008).
Sapoznik, E. et al. A versatile oblique plane microscope for large-scale and high-resolution imaging of subcellular dynamics. eLife 9, e57681 (2020).
Haslehurst, P., Yang, Z., Dholakia, K. & Emptage, N. Fast volume-scanning light sheet microscopy reveals transient neuronal events. Biomed. Opt. Express 9, 2154–2167 (2018).
Bingen, P., Reuss, M., Engelhardt, J. & Hell, S. W. Parallelized STED fluorescence nanoscopy. Opt. Express 19, 23716–23726 (2011).
Chang, B.-J. et al. Universal light-sheet generation with field synthesis. Nat. Methods 16, 235–238 (2019).
Charras, G. T., Coughlin, M., Mitchison, T. J. & Mahadevan, L. Life and times of a cellular bleb. Biophys. J. 94, 1836–1853 (2008).
Kumar, M., Kishore, S., Nasenbeny, J., McLean, D. L. & Kozorovitskiy, Y. Integrated one- and two-photon scanned oblique plane illumination (SOPi) microscopy for rapid volumetric imaging. Opt. Express 26, 13027–13041 (2018).
Voleti, V. et al. Real-time volumetric microscopy of in vivo dynamics and large-scale samples with SCAPE 2.0. Nat. Methods 16, 1054–1062 (2019).
Delarue, M. et al. mTORC1 controls phase separation and the biophysical properties of the cytoplasm by tuning crowding. Cell 174, 338–349.e320 (2018).
Taylor, J. M., Girkin, J. M. & Love, G. D. High-resolution 3D optical microscopy inside the beating zebrafish heart using prospective optical gating. Biomed. Opt. Express 3, 650–660 (2012).
Liebling, M., Forouhar, A. S., Gharib, M., Fraser, S. E. & Dickinson, M. E. Four-dimensional cardiac imaging in living embryos via postacquisition synchronization of nongated slice sequences. J. Biomed. Opt. 10, 054001 (2005).
Mickoleit, M. et al. High-resolution reconstruction of the beating zebrafish heart. Nat. Methods 11, 919–922 (2014).
Levoy, M., Ng, R., Adams, A., Footer, M. & Horowitz, M. Light field microscopy. in Proc. ACM SIGGRAPH 2006 (2006).
Yang, B. et al. Epi-illumination SPIM for volumetric imaging with high spatial-temporal resolution. Nat. Methods 16, 501–504 (2019).
Millett-Sikking, A. et al. High NA single-objective light-sheet. Zenodo https://doi.org/10.5281/zenodo.3376243 (2019).
Murali, V. S. et al. An image-based assay to quantify changes in proliferation and viability upon drug treatment in 3D microenvironments. BMC Cancer 19, 502 (2019).
Venkateswaran, N. et al. MYC promotes tryptophan uptake and metabolism by the kynurenine pathway in colon cancer. Genes Dev. 33, 1236–1251 (2019).
Dean, K. M. et al. Imaging subcellular dynamics with fast and light-efficient volumetrically parallelized microscopy. Optica 4, 263–271 (2017).
Miura, K. Bleach correction ImageJ plugin for compensating the photobleaching of time-lapse sequences. F1000Res. 9, 1494 (2020).
Schindelin, J. et al. Fiji: an open-source platform for biological-image analysis. Nat. Methods 9, 676–682 (2012).
We thank D. Saucier for the help with the preparation of the zebrafish samples, and L. Perez-Castro and the Sorrell laboratory for providing the colorectal cancer cells. We thank the Cancer Prevention Research Institute of Texas (grant no. RR160057 to R.F.), and the National Institutes of Health (grant nos. T32CA080621 to M.K., F32GM117793 to K.M.D., K25CA204526 to E.S.W. and R35GM133522 to R.F.). P.R. acknowledges support through R35GM136428 to Gaudenz Danuser. J.D.M. thanks E. Derivery for access to the spinning disk microscope, R. Mittal for HeLa cells and A. Fowle for fabricating the custom mirror mount. J.D.M. acknowledges support from Fitzwilliam College, Cambridge, through a Research Fellowship.
R.F. filed a patent for the scan unit and its applications to microscopy, and K.M.D. has an investment interest in Discovery Imaging Systems, LLC.
Peer review information Nature Methods thanks Ulrich Kubitscheck, Manish Kumar and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Rita Strack was the primary editor on this article and managed its editorial process and peer review in collaboration with the rest of the editorial team.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended Data Fig. 1 Comparison of simulations of projection method and extended-depth-of-field techniques.
Given a raw volume of the mitochondrial membranes within a HeLa cell (sum projection and depth colour-coding in a), simulations show the corresponding image obtained through b OPM-Pro, as described in the main text, c wide-field-Pro, d reducing the numerical aperture of the objective from 1.35 to 0.5, e using a 12-zone layer cake mask, f using a 6-alpha cubic phase plate. Beneath each ‘raw’ image, the result of deconvolving the image with 200 iterations of the Richardson-Lucy algorithm is shown, labelled ‘RL200’. Cyan insets show a 3× magnified view, while magenta insets show the sum projection of the point spread function used.
Extended Data Fig. 2 Spectral analysis of simulated projection method and extended-depth-of-field techniques.
Logarithmically scaled, normalized Fourier power spectra for corresponding raw images in Extended Data Fig. 1: a cubic phase mask, b layer cake mask, c projection method applied to a wide-field microscope, d projection method applied to the oblique plane microscope described in the main text, e a wide-field microscope of lower numerical aperture (0.5). f shows radially averaged profiles of all spectra, up to the diffraction limit.
Schematic representation of sample scanning in a lattice light-sheet microscopy and b OPM. The z-axis represents the optical axis of the detection objective in LLSM and the optical axis of the primary objective in OPM. In both cases the scan direction is diagonal to the detection axis. As such, the data is sheared when a stack is acquired, which is usually compensated computationally by applying the opposite amount of shearing (also known as ‘deskewing’). c shows raw image data from LLSM obtained using sample scanning. d shows the same data after numerical deskewing. For the work presented here, the tilt angle of the coverslip in LLSM was 30 degree relative to the nominal focal plane, For OPM, the imaging plane was tilted 30 degrees relative to the nominal focal plane of the primary objective.
Numerical simulation of the rotation of a 2D image and subsequent sum-projection, and comparison to numerical shearing of a 2D image and subsequent sum-projection. To compare the two line-profiles, the projection from the sheared image was compressed to the same length as the projection from the rotated image. Small differences between the two profiles may have arisen from numerical errors during the shearing and rotation operations.
CAD rendering of the galvanometric scan unit (gray), which attaches to the c-mount of a sCMOS camera (black).
a point spread function for LLSM-Pro using 100 nm beads in agarose. LLSM was operated in sample scanning mode, with the scan direction along the optical axis of the detection objective. b Point spread function for OPM-Pro, using 100 nm beads on a coverslip. c Point spread function for mSPIM-Pro using 100 nm beads embedded in agarose. mSPIM was operated using mechanical scanning of the objective and galvo scanning of the light-sheet. d Point spread function for SDCM-Pro using 100 nm beads on a coverslip. The SDCM was operated using sample scanning with a piezo stage. Angle definition: a, c, d: Angle is indicated relative to the optical axis of the detection objective. b: Angle is indicated relative to the detection axis of the OPM system (see also Extended Data Fig. 3). At −30°, the viewing direction is normal to the coverslip. Brightness/contrast adjustment has been applied to the data as described in the method. Scale bar: 1 micron.
a Single projection obtained with LLSM-Pro. b Sum projection of a conventional 3D stack. c Maximum intensity projection of a conventional 3D stack. d 2D deconvolution of a projection obtained with LLSM-Pro. e Sum projection of conventional 3D stack after 3D deconvolution. f Maximum intensity projection of conventional 3D stack after 3D deconvolution. Scale bars: 10 microns.
Extended Data Fig. 8 3D localization from two views compared with ground truth data from a conventional 3D stack.
Maximum intensity projection of a ground truth data set of genetically encoded monomeric nanoparticles in a fixed MV3 cell (gray) and the localizations of these particles in the 3D stack (blue circles). Yellow circles mark where the reconstruction from two projections overlapped, whereas red circles mark reconstructions that do not match up with the ground truth. Scale bar: 10 microns.
The same zebrafish heart shown in Fig. 4f was imaged with a conventional 3D z-stacking. A maximum intensity projection (a) and a sum projection (b) of the stack in xy (top row) and xz (bottom row) are shown. The exposure time is 25ms (40Hz frame rate), the z step size is 0.312 microns, and there are 322 z slices in the stack. Scale bars: 20 microns.
a Schematic representation of an imaging volume of height Z, imaged with an extended depth of focus (EDF) PSF. The EDF PSF is n-fold extended in the axial direction. b Imaging the same volume while rapidly sweeping a diffraction limited PSF and a light-sheet through the same volume. Since the PSF in (a) is n-fold stretched compared to the diffraction limited PSF in b, the Strehl ratio (that is the peak intensity of the PSF) is at least n-fold reduced compared to b. For a, the whole sample is illuminated during the exposure time. In b, any part of the sample is illuminated 1/n-th of the exposure time. Assuming linear fluorescence response to excitation light-intensity, the lower Strehl ratio of the EDF PSF is compensated by the longer effective exposure time of any part of the sample.
Supplementary Notes 1–9, References and Tables 1 and 2.
About this article
Cite this article
Chang, BJ., Manton, J.D., Sapoznik, E. et al. Real-time multi-angle projection imaging of biological dynamics. Nat Methods 18, 829–834 (2021). https://doi.org/10.1038/s41592-021-01175-7
This article is cited by
npj Imaging (2023)
Nature Methods (2022)
Nature Methods (2022)
Nature Methods (2021)