Light-field microscopy (LFM) is a scalable approach for volumetric Ca2+ imaging with high volumetric acquisition rates (up to 100 Hz). Although the technology has enabled whole-brain Ca2+ imaging in semi-transparent specimens, tissue scattering has limited its application in the rodent brain. We introduce seeded iterative demixing (SID), a computational source-extraction technique that extends LFM to the mammalian cortex. SID can capture neuronal dynamics in vivo within a volume of 900 × 900 × 260 μm located as deep as 380 μm in the mouse cortex or hippocampus at a 30-Hz volume rate while discriminating signals from neurons as close as 20 μm apart, at a computational cost three orders of magnitude less than that of frame-by-frame image reconstruction. We expect that the simplicity and scalability of LFM, coupled with the performance of SID, will open up a range of applications including closed-loop experiments.
This is a preview of subscription content
Subscribe to Nature+
Get immediate online access to the entire Nature family of 50+ journals
Subscribe to Journal
Get full journal access for 1 year
only $9.92 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Get time limited or full article access on ReadCube.
All prices are NET prices.
Chen, T.-W. et al. Ultrasensitive fluorescent proteins for imaging neuronal activity. Nature 499, 295–300 (2013).
Schrödel, T., Prevedel, R., Aumayr, K., Zimmer, M. & Vaziri, A. Brain-wide 3D imaging of neuronal activity in Caenorhabditis elegans with sculpted light. Nat. Methods 10, 1013–1020 (2013).
Nguyen, J.P. et al. Whole-brain calcium imaging with cellular resolution in freely behaving Caenorhabditis elegans. Proc. Natl. Acad. Sci. USA 113, E1074–E1081 (2016).
Prevedel, R. et al. Simultaneous whole-animal 3D imaging of neuronal activity using light-field microscopy. Nat. Methods 11, 727–730 (2014).
Ahrens, M.B., Orger, M.B., Robson, D.N., Li, J.M. & Keller, P.J. Whole-brain functional imaging at cellular resolution using light-sheet microscopy. Nat. Methods 10, 413–420 (2013).
Denk, W., Strickler, J.H. & Webb, W.W. Two-photon laser scanning fluorescence microscopy. Science 248, 73–76 (1990).
Ji, N., Freeman, J. & Smith, S.L. Technologies for imaging neural activity in large volumes. Nat. Neurosci. 19, 1154–1164 (2016).
Grewe, B.F., Langer, D., Kasper, H., Kampa, B.M. & Helmchen, F. High-speed in vivo calcium imaging reveals neuronal network activity with near-millisecond precision. Nat. Methods 7, 399–405 (2010).
Botcherby, E.J. et al. Aberration-free three-dimensional multiphoton imaging of neuronal activity at kHz rates. Proc. Natl. Acad. Sci. USA 109, 2919–2924 (2012).
Kirkby, P.A., Srinivas Nadella, K.M.N. & Silver, R.A. A compact acousto-optic lens for 2D and 3D femtosecond based 2-photon microscopy. Opt. Express 18, 13721–13745 (2010).
Kim, K.H. et al. Multifocal multiphoton microscopy based on multianode photomultiplier tubes. Opt. Express 15, 11658–11678 (2007).
Cheng, A., Gonçalves, J.T., Golshani, P., Arisaka, K. & Portera-Cailliau, C. Simultaneous two-photon calcium imaging at different depths with spatiotemporal multiplexing. Nat. Methods 8, 139–142 (2011).
Stirman, J.N., Smith, I.T., Kudenov, M.W. & Smith, S.L. Wide field-of-view, multi-region, two-photon imaging of neuronal activity in the mammalian brain. Nat. Biotechnol. 34, 857–862 (2016).
Yang, W. et al. Simultaneous multi-plane imaging of neural circuits. Neuron 89, 269–284 (2016).
Packer, A.M., Russell, L.E., Dalgleish, H.W.P. & Häusser, M. Simultaneous all-optical manipulation and recording of neural circuit activity with cellular resolution in vivo. Nat. Methods 12, 140–146 (2015).
Duemani Reddy, G., Kelleher, K., Fink, R. & Saggau, P. Three-dimensional random access multiphoton microscopy for functional imaging of neuronal activity. Nat. Neurosci. 11, 713–720 (2008).
Katona, G. et al. Fast two-photon in vivo imaging with three-dimensional random-access scanning in large tissue volumes. Nat. Methods 9, 201–208 (2012).
Fernández-Alfonso, T. et al. Monitoring synaptic and neuronal activity in 3D with synthetic and genetic indicators using a compact acousto-optic lens two-photon microscope. J. Neurosci. Methods 222, 69–81 (2014).
Prevedel, R. et al. Fast volumetric calcium imaging across multiple cortical layers using sculpted light. Nat. Methods 13, 1021–1028 (2016).
Song, A. et al. Volumetric two-photon imaging of neurons using stereoscopy (vTwINS). Nat. Methods 14, 420–426 (2017).
Lu, R. et al. Video-rate volumetric functional imaging of the brain at synaptic resolution. Nat. Neurosci. 20, 620–628 (2017).
Huisken, J., Swoger, J., Del Bene, F., Wittbrodt, J. & Stelzer, E.H. Optical sectioning deep inside live embryos by selective plane illumination microscopy. Science 305, 1007–1009 (2004).
Wu, Y. et al. Inverted selective plane illumination microscopy (iSPIM) enables coupled cell identity lineaging and neurodevelopmental imaging in Caenorhabditis elegans. Proc. Natl. Acad. Sci. USA 108, 17708–17713 (2011).
Chhetri, R.K. et al. Whole-animal functional and developmental imaging with isotropic spatial resolution. Nat. Methods 12, 1171–1178 (2015).
Bouchard, M.B. et al. Swept confocally-aligned planar excitation (SCAPE) microscopy for high speed volumetric imaging of behaving organisms. Nat. Photonics 9, 113–119 (2015).
Abrahamsson, S. et al. Fast multicolor 3D imaging using aberration-corrected multifocus microscopy. Nat. Methods 10, 60–63 (2013).
Levoy, M., Ng, R., Adams, A., Footer, M. & Horowitz, M. Light field microscopy. ACM Trans. Graph. 25, 924 (2006).
Broxton, M. et al. Wave optics theory and 3-D deconvolution for the light field microscope. Opt. Express 21, 25418–25439 (2013).
Cohen, N. et al. Enhancing the performance of the light field microscope using wavefront coding. Opt. Express 22, 24817–24839 (2014).
Pégard, N.C. et al. Compressive light-field microscopy for 3D neural activity recording. Optica 3, 517 (2016).
Liu, H.-Y. et al. 3D imaging in volumetric scattering media using phase-space measurements. Opt. Express 23, 14461–14471 (2015).
Helmchen, F. & Denk, W. Deep tissue two-photon microscopy. Nat. Methods 2, 932–940 (2005).
Mukamel, E.A., Nimmerjahn, A. & Schnitzer, M.J. Automated analysis of cellular signals from large-scale calcium imaging data. Neuron 63, 747–760 (2009).
Pnevmatikakis, E.A. et al. Simultaneous denoising, deconvolution, and demixing of calcium imaging data. Neuron 89, 285–299 (2016).
Dombeck, D.A., Harvey, C.D., Tian, L., Looger, L.L. & Tank, D.W. Functional imaging of hippocampal place cells at cellular resolution during virtual navigation. Nat. Neurosci. 13, 1433–1440 (2010).
Kaifosh, P., Lovett-Barron, M., Turi, G.F., Reardon, T.R. & Losonczy, A. Septo-hippocampal GABAergic signaling across multiple modalities in awake mice. Nat. Neurosci. 16, 1182–1184 (2013).
Graves, A.R. et al. Hippocampal pyramidal neurons comprise two distinct cell types that are countermodulated by metabotropic receptors. Neuron 76, 776–789 (2012).
Waller, L. & Tian, L. Computational imaging: machine learning for 3D microscopy. Nature 523, 416–417 (2015).
Zhou, P., Resendez, S.L., Stuber, G.D., Kass, R.E. & Paninski, L. Efficient and accurate extraction of in vivo calcium signals from microendoscopic video data. Preprint at http://arxiv.org/abs/1605.07266 (2016).
Apthorpe, N.J. et al. Automatic neuron detection in calcium imaging data using convolutional networks. In Advances in Neural Information Processing Systems 29 (eds. Lee, D.D., Sugiyama, M., Luxburg, U.V., Guyon, I. & Garnett, R.) 3270–3278 (Curran Associates, Inc., 2016).
We thank W. Haubensak and his lab members for sharing their animal facility and reagents, M. Colombini and the IMP workshop for manufacturing of mechanical components, and F. Schlumm and Q. Lin for help with zebrafish imaging. The computational results presented here were achieved in part through use of the Vienna Scientific Cluster (VSC). T.N. acknowledges the Leon Levy Foundation (Leon Levy Fellowship in Neuroscience). This work was supported in part through funding from the Vienna Science and Technology Fund (WWTF; project VRG10-11), the Human Frontiers Science Program (Project RGP0041/2012), the Institute of Molecular Pathology, and the Kavli Foundation, all to A.V.; and from the Intelligence Advanced Research Projects Activity (IARPA) via the Department of Interior/Interior Business Center (DoI/IBC; contract number D16PC00002). The US government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of IARPA, DoI/IBC or the US government.
The authors declare no competing financial interests.
Integrated supplementary information
Supplementary Figure 1 Illustration of the effects of scattering in conventional wide-field and light-field microscopy, illustrating the prior state of the art.
(a) Ground truth: neurons in the mouse cortex.
(b) Wide field image of ground truth in the presence of scattering.
(c)-(d) LFM raw image of ground truth without scattering (c) and with scattering (d).
(e) Zoom into the indicated region in (d), solid white line is intensity profile along dashed white line. Black arrow highlights brightness gradients originating from directionality information retained in scattered light, while white arrow highlights peak due to ballistic light.
(f) Axial projection of volumetric reconstruction of the LFM image without scattering shown in (c).
(g) Un-mixed temporal signals extracted from an LFM movie without scattering, from the regions-of-interested indicated as dashed circles in (f).
(h) Axial projection of volumetric reconstruction of the LFM image with scattering shown in (d).
(i) Temporal signals extracted from an LFM movie with scattering, from dashed circles in (h). Black rectangles highlight regions with crosstalk from one neuronal signal to the other in the presence of scattering.
(a) Standard deviation image of a raw LFM movie from mouse cortex (no background subtraction)
(b) Example of spatial background component extracted using rank-1 matrix factorization of a raw LFM recording from mouse cortex
(c) Example of temporal background component extracted using rank-1 matrix factorization of a raw LFM recording from mouse cortex
(d) Standard deviation image of a background-subtracted LFM movie from mouse cortex
(e) Maximum-intensity projection of LFM reconstruction of the standard deviation image with background subtraction shown in (d). Red circles: Centers of neuron candidates as identified using volume segmentation by local maxima search
(f) Example of initial forward model for a particular neuron candidate. White trace: corresponding initial temporal signal estimate
(g) Convergence plot of alternating spatial and temporal non-negative least squares optimization: Norm of residual vs. iteration count
(h) Example of refined forward model after 20 iterations. White trace: corresponding refined temporal signal estimate
Supplementary Figure 3 Simulation of SID performance versus ground truth: SID of overlapping neurons with small axial distances.
(a) Detail from standard deviation image of a synthetic LFM raw movie generated as described in Suppl. Note 3 for a simulated depth of 450 μm).
(b) + (c) Two demixed spatial components (neuron footprints) detected by SID within the encircled region shown in (a). The detected neurons are spaced 3 μm apart laterally and 8 μm axially in the ground truth.
(d) Composite color overlay of the spatial components (b) and (c), showing considerable overlap (yellow).
(e) - (h) Comparisons of ground truth temporal signals extracted using SID and spatial region-of-interest extraction. Black: Ground truth signals for the two neurons whose LFM images fall into the area encircled in (a). Green: demixed signals corresponding to the spatial components (b) and (c), as indicated by the arrows. Violet: signal extracted by summing over the encircled area in (a). Red circles highlight intervals where time series of neuron 1 (b) and neuron 2 (c) are mixed for a circular region-of-interest extraction (e, f) and demixed for SID (g, h) as evident by comparing the corresponding signals to the ground truth.
Heat map of 5505 neuronal signals detected by Seeded Iterative Demixing (SID) in a four minute, 20 Hz LFM recording of spontaneous activity in the larval zebrafish brain. Corresponding neuron locations are shown in Fig. 2 d-e of the main text.
Supplementary Figure 5 A comparison of the extraction performance of SID and an ICA-based method in larval zebrafish.
(a) Zoom into a frontal slice of the reconstructed standard deviation image shown in Fig. 2c of the main text, overlaid with spatial component centroids (neuron positions) detected by an ICA-based method (orange circles) and Seeded Iterative Demixing (SID) (green circles). Circle radii are standard deviations of spatial components (orange circles), or a fixed value of 3 μm (green circles).
(b) Example signals detected at the locations encircled in (a). Numbers in (a) and (b) identify corresponding locations and signals. Signals are grouped into three groups (indicated by curly braces) according to their spatial proximity and overlapping spatial components. (i) Example of components found by SID (1) and ICA (2), respectively, that match well both in location and activity. (ii) Example of signals where a single component from Seeded Iterative Demixing (SID) (3) is split up into several overlapping and temporally correlated components by ICA (4, 5). (iii) Examples of components where ICA resulted in signals (9, 10) that appear to be mixtures of signals detected by SID (6, 8), due to ICA detecting “ghost” components in scatter. Vertical arrows indicate time points where peaks from SID-signals (6, 8) appear in ICA-signals (9, 10).
Heatmap of neuron activity traces obtained by Seeded Iterative Demixing (SID), corresponding to positions shown in Fig. 3c-d of the main text. Upper panel: 296 active, GCaMP6m-expressing neurons found in 0-170 μm depth range. Lower panel: 208 neurons found in 120-380 μm in a subsequent recording.
Supplementary Figure 7 Neuron detection scores for different sensitivity settings of SID and CaImAn, and quality of SID-extracted Ca2+ transients.
(a)-(f) Neuron detection scores "Sensitivity", "Precision" and "F-score" of SID and CaImAn for three different sensitivity settings, as described in Suppl. Note 5
(g) Neuron detection scores versus depth as achieved by SID (green traces), in comparison to scores achieved by the analysis package CaImAn applied to the 2PM data (blue traces), both evaluated with respect to a ground truth. (i) Sensitivity score (ratio of number of detected to actual neurons), (ii) Precision score (ratio of number of true positives to sum of true and false positives), (iii) F-Score (harmonic mean of sensitivity and precision). n = 4.
(h) Correlation of SID-extracted signals to ground truth (see Suppl. Note 5), computed across peaks only, for four different tissue depths
(a) Mean image of 2PM movie of a single plane in mouse cortex, same data as in Fig. 5c
(b)Standard deviation image of 2PM movie of a single plane in mouse cortex, same data as in Fig. 5c. White arrows highlight neurites that are identified by a local maximum search and segmentation of the LFM standard deviation image (c, d), in addition to active somata.
(c) Maximum-intensity projection of reconstructed standard deviation image of LFM movie from mouse cortex, recorded by sending part of the 2P-excited emission light to an LFM camera in a hybrid 2PM-LFM setup. Corresponding 2PM signal shown in (a) and (b).
(d) Result of volume segmentation of LFM-reconstructed standard deviation image shown in (c). Comparison with (b) shows that in addition to the active somata, some large and active neurites (corresponding pairs highlighted with white arrows in (b) and (d)) are identified by the local maximum search and segmentation algorithm and thus can be subtracted.
(e) + (f): Examples of axial maximum intensity projections of LFM-reconstructed footprints classified as neurons
(g) + (h): Examples of axial maximum intensity projections of LFM-reconstructed footprints classified as neuropil
(a) Motion metric based on image autocorrelation as described in Suppl. Note 7, evaluated for a 2PM recording (red trace) and a simultaneously acquired SID recording (green trace) from mouse cortex (PPC) at 200 μm depth. Black trace indicates rotational speed [a.u.] of treadmill disk due to walking behavior of a head-fixed mouse, as measured by a high-speed optical computer mouse in proximity to the treadmill disk
(b) Top panel: Heatmap of SID-extracted neuronal activity traces from simultaneous 2P-SID recording (frame rate 3 Hz) from mouse cortex (PPC) at 200 μm depth. Bottom panels: Motion metrics and treadmill tracking data as in (a). Column marked in yellow in top panel indicates frame excluded from further analysis due to high motion metric value in corresponding time bin, as described in Suppl Note 7
Supplementary Figure 10 SID-extracted positions and signals of jRGECO-labeled neurons in mouse hippocampus CA1.
(a) Heatmap of SID-extracted neuronal activity traces for 54 neurons recorded from mouse hippocampus CA1 at a frame rate of 5 Hz. Cranial window surgery and imaging as described in the main text and Methods section. (The recordings were performed under suboptimal conditions in which the expression of jRGECO was not fully developed and the transfection protocol sub-optimal)
(b) SID-extracted 3D neuron positions corresponding to activity traces in (a). Top panel: isometric view. Bottom panels: Top, side and front views. Depth is given from bottom surface of cranial window placed onto corpus callosum as shown in Fig. 4a of the main text
Supplementary Figures 1–10 and Supplementary Notes 1–7 (PDF 3503 kb)
Matlab code implementing seeded iterative demixing. The code requires additional functions published online as Supplementary Software with the paper by Prevedel et al. (Simultaneous whole-animal 3D imaging of neuronal activity using light-field microscopy. Nat. Methods 11, 727–730 (2014)). Updated versions of the code will be made available online at http://vaziria.com and http://github.com/vazirilab. (ZIP 16 kb)
3D rendering of naive reconstructions of LFM Ca2+ imaging data of mouse cortex in vivo: 0–170 μm below dura.
Rendering of an LFM recording of spontaneous GECI activity in GCaMP6m-labeled mouse posterior parietal cortex, reconstructed by naive deconvolution with a simulated, ballistic PSF, showing considerable blur due to scattering. Recording duration: 1 min. Recording frame rate: 30 f.p.s. Playback at 60 f.p.s. (MP4 250 kb)
3D rendering of naive reconstructions of LFM Ca2+ imaging data of mouse cortex in vivo: 170–380 μm below dura.
Rendering of an LFM recording of spontaneous GECI activity in GCaMP6m-labeled mouse posterior parietal cortex, reconstructed by naive deconvolution with a simulated, ballistic PSF, showing considerable blur due to scattering. Recording duration: 1 min. Recording frame rate: 30 f.p.s. Playback at 60 f.p.s. (MP4 313 kb)
3D rendering of cortical neuronal activity extracted using Seeded Source Extraction (SID) from LFM Ca2+ imaging of mouse cortex in vivo: 0–170 μm below dura.
Rendering of neuron positions and normalized temporal signals revealed using Seeded Source Extraction (SID) in a 1-min LFM recording of GECI activity in GCaMP6m-labeled mouse posterior parietal cortex, showing spontaneous activity. Depth range: 30 μm above to 170 μm below dura. Neuron positions indicated as spheres with 15-μm diameter. Recording frame rate: 30 f.p.s. Playback at 60 f.p.s. Data correspond to Figure 3c,d and Supplementary Figure 6. (MP4 774 kb)
3D rendering of cortical neuronal activity extracted by Seeded Source Extraction (SID) from LFM Ca2+ imaging of mouse cortex in vivo: 170–380 μm below dura.
Rendering of neuron positions and normalized temporal signals revealed using Seeded Source Extraction (SID) in a 1-min LFM recording of GECI activity in GCaMP6m-labeled mouse posterior parietal cortex, showing spontaneous activity. Depth range: 170–380 μm below dura. Neuron positions indicated as spheres with 15-μm diameter. Recording frame rate: 30 f.p.s. Playback at 60 f.p.s. Data correspond to Figure 3c,d and Supplementary Figure 6. (MP4 516 kb)
About this article
Cite this article
Nöbauer, T., Skocek, O., Pernía-Andrade, A. et al. Video rate volumetric Ca2+ imaging across cortex using seeded iterative demixing (SID) microscopy. Nat Methods 14, 811–818 (2017). https://doi.org/10.1038/nmeth.4341
Experiments in Fluids (2022)
Real-time volumetric reconstruction of biological dynamics with light-field microscopy and deep learning
Nature Methods (2021)
Computational optical sectioning with an incoherent multiscale scattering model for light-field microscopy
Nature Communications (2021)
Light: Science & Applications (2021)
Imaging volumetric dynamics at high speed in mouse and zebrafish brain with confocal light field microscopy
Nature Biotechnology (2021)