Optimal image quality in light-sheet microscopy requires a perfect overlap between the illuminating light sheet and the focal plane of the detection objective. However, mismatches between the light-sheet and detection planes are common owing to the spatiotemporally varying optical properties of living specimens. Here we present the AutoPilot framework, an automated method for spatiotemporally adaptive imaging that integrates (i) a multi-view light-sheet microscope capable of digitally translating and rotating light-sheet and detection planes in three dimensions and (ii) a computational method that continuously optimizes spatial resolution across the specimen volume in real time. We demonstrate long-term adaptive imaging of entire developing zebrafish (
Danio rerio) and Drosophila melanogaster embryos and perform adaptive whole-brain functional imaging in larval zebrafish. Our method improves spatial resolution and signal strength two to five-fold, recovers cellular and sub-cellular structures in many regions that are not resolved by non-adaptive imaging, adapts to spatiotemporal dynamics of genetically encoded fluorescent markers and robustly optimizes imaging performance during large-scale morphogenetic changes in living organisms.
View full text
At a glance
: Spatiotemporally adaptive light-sheet microscopy.
a) Fundamental optical challenges associated with long-term live imaging of large biological specimens often lead to loss of spatial overlap between illuminating light sheets and detection focal planes in light sheet microscopy. The most severe problems are caused by spatial and temporal heterogeneity of the refractive index across the live specimen, the surrounding support matrix and the medium in the sample chamber (left). Thermal, mechanical and electronic drifts in microscope components during live imaging can further contribute to a degradation of spatial resolution ( Supplementary Video 10). When imaging developing organisms, such as early zebrafish ( D. rerio) embryos during epiboly (top right), one also needs to consider that optical conditions change continuously as a function of time and spatial location in the sample. Live imaging of genetically encoded fluorescent markers, such as a pan-neural fluorescent marker tracking the developing nervous system in Drosophila (bottom right), is further complicated by spatiotemporal dynamics in marker expression. Recovering optimal resolution in the imaging experiment thus requires spatiotemporal adaptation of the microscope to the dynamic optical conditions while tracking dynamic fluorescent signals. ( b) Overview of the fully automated light-sheet microscopy framework for spatiotemporally adaptive imaging, which addresses the challenges outlined in a. Our framework consists of (i) a multi-view light-sheet microscope with ten digitally adjustable degrees of freedom that control 3D offsets and 3D angles between light sheets and detection focal planes, and (ii) a real-time software layer that autonomously monitors image quality throughout the imaging volume and automatically and continuously adjusts these degrees of freedom to optimize spatial resolution and image quality across the sample in space and time. Scale bar, 5 μm.
: Spatiotemporally adaptive imaging of
Drosophila embryonic development.
a) Dorsoventral maximum-intensity projections of a D. melanogaster embryo expressing RFP in all cell nuclei (w;His2Av-mRFP1;+), representing a 21-h time-lapse experiment using spatiotemporally adaptive imaging ( Supplementary Video 2). Landmark developmental processes are annotated on the experiment time axis. Imaging started in the blastoderm stage (“0 h” on the time axis), corresponding to 3 h after egg laying (h AEL). The embryo is ~500 μm long and ~200 μm wide. ( b) Plots visualizing real-time corrections of the positions of light sheets 1 (green) and 2 (orange) relative to the respective detection focal planes as a function of time and spatial location in the embryo. These corrections were computed by the real-time software layer of the adaptive imaging framework to maximize spatial resolution throughout the specimen. Using a fully automated workflow, image quality in the embryo was sampled, evaluated and optimized at six reference planes ( z 0–z 5, top right inset). ( c) Improvements in spatial resolution and image quality achieved by spatiotemporally adaptive imaging. Example image data are shown for the spatial location marked in b at 5 h. Fourier analysis of the microscopy data acquired with (top) and without (bottom) microscope state corrections computed by the adaptive imaging framework demonstrates a 39% increase in the cut-off radius in frequency space across the entire image plane. Enlarged views (right) and line profiles (below) show that spatiotemporally adaptive imaging recovered cellular and sub-cellular features that were not resolved with non-adaptive imaging. Images labeled “not corrected” were acquired using the optimized microscope parameter settings determined by the AutoPilot framework at the beginning of the experiment (“Initial System optimization”, Online Methods). Thus the difference between “corrected” and “not corrected” settings shown here and in d is a lack of continuous microscope adaptation over the course of the experiment for images labeled as “not corrected.” A comprehensive side-by-side comparison is shown as a function of space and time in Supplementary Videos 3 and 4. ( d) Side-by-side comparison of image quality and spatial resolution in representative image regions for adaptively corrected (top row) and uncorrected (middle row) microscope states at 21 h. Increase in spatial resolution (factors in green) was quantified by comparative analysis of the derivatives of intensity line profiles crossing sharp edges in the image data, corresponding to boundaries of fluorescently labeled cell nuclei (bottom row). The computational procedure and its mathematical derivation are described in Supplementary Methods, part 6. The complete set of Drosophila example image data is presented in Supplementary Figure 14. Scale bars, 20 μm ( c, left), 5 μm ( c, right; d).
: Spatiotemporally adaptive imaging of zebrafish embryonic development.
a) Lateral maximum-intensity projections of a D. rerio embryo expressing GFP in all cell nuclei (H2B–eGFP), representing a 12-h time-lapse experiment using spatiotemporally adaptive imaging with degrees of freedom D i and I i ( i = 1, 2; Supplementary Video 5). Imaging started in the 30%-epiboly stage (“0 h” on the time axis), corresponding to 5 h post fertilization. The embryo is ~700 μm in diameter. ( b) As fast, coordinated cell movements spread the blastoderm across the large, central yolk cell (see a), the adaptive light-sheet-based imaging framework continuously adjusts the microscope system state to maintain optimal image quality. To facilitate this spatiotemporal adaption in imaging experiments with dynamic fluorescence signals, the framework automatically flags reference locations lacking fluorescence signal (thin gray lines) and monitors the emergence of fluorescence signal as a function of time and spatial location in the specimen (thick blue lines). Note the continuous spreading of the blastoderm across the yolk cell and the concomitant detection of fluorescence signal in corresponding reference locations z 4– z 6 during the first 4 h of the experiment. ( c) Plots visualizing real-time corrections of the positions of the two light sheets (green and orange) relative to the respective detection focal planes as a function of time and spatial location in the embryo (reference planes z 0– z 6; see a). Corrections in regions lacking fluorescent signals are guided by neighboring reference planes until local fluorescent signal emerges and is used to determine region-specific microscope state corrections. ( d) Improvements in spatial resolution and image quality achieved by spatiotemporally adaptive imaging. Example image data are shown for the spatial location marked in c at 6 h. Fourier analysis of data (second column) acquired with (top) and without (bottom) microscope corrections computed by the adaptive imaging framework demonstrates a 27% increase in cut-off radius in frequency space. Enlarged views and line profiles (right) show that adaptive imaging recovered cellular and sub-cellular features that were not resolved by non-adaptive imaging. Defocus aberrations up to 6 μm occur without adaptive imaging (bottom right, DCTS values for AutoPilot image defocus series). ( e) Side-by-side comparison of image quality and spatial resolution in two representative image regions for adaptively corrected (degrees of freedom D i, I i, Y i, α i and β i with i = 1, 2) and uncorrected microscope states at the end of epiboly. Locations of image planes are indicated in illustrations to the left of each image panel. Increase in spatial resolution (factors in green) was quantified using derivatives of line profiles crossing sharp edges in the images corresponding to boundaries of fluorescently labeled cell nuclei. See Supplementary Videos 6 and 7 for a systematic side-by-side comparison of images in corrected and uncorrected microscope states. The procedure and its mathematical derivation are described in Supplementary Methods, part 6. The complete set of zebrafish example image data is presented in Supplementary Figure 15. Scale bars, 50 μm ( d, left), 10 μm ( d, right), 5 μm ( e).
: Spatiotemporally adaptive imaging of dynamic gene expression patterns.
a) Dorsoventral maximum-intensity projections of a D. melanogaster embryo expressing RFP in all cell nuclei and GFP in the nervous system (deadpanEE–Gal4, UAS–myr::GFP, His2Av–RFP), representing a 20-h time-lapse experiment using spatiotemporally adaptive imaging ( Supplementary Video 8). Imaging started in the blastoderm stage (“0 h” on the time axis), corresponding to 3 h AEL. Expression of the pan-neural marker starts at around 10 h. ( b) The onset of expression of the pan-neural marker is automatically detected by the adaptive imaging framework, which optimizes all parameters associated with this color channel in response to the emerging signal. Note that the onset of expression occurs slightly earlier in ventral regions (reference planes z 0, z 1, and z 2). ( c) Improvements in spatial resolution and image quality achieved by spatiotemporally adaptive imaging. Example image data are shown for the spatial location marked in b at 18.5 h. Fourier analysis of the microscopy data (second column) acquired with (top) and without (bottom) microscope state corrections computed by the adaptive imaging framework demonstrates a 32% increase in cut-off radius in frequency space. Enlarged views and line profiles to the right show that adaptive imaging recovered cellular and sub-cellular features that were not resolved with non-adaptive imaging. Plot to the bottom right shows DCTS values determined by AutoPilot for a defocus series acquired at the image location shown to the left, indicating optimal image quality in the corrected system state. ( d) The adaptive imaging framework automatically corrects for focal shifts between different color channels arising from chromatic aberrations inherent to the design of the detection objectives. For the Nikon 16×/0.8 objectives used in this experiment, the framework compensated for a focal shift of 0.84 μm between GFP and RFP detection bands. ( e) The adaptive imaging framework automatically optimizes the position of the beam waist of the illuminating Gaussian laser beams (position of minimal light-sheet thickness) by real-time adjustment of the positions Y 1 and Y 2 of the illumination objectives during volumetric imaging (left). In multi-color imaging experiments, the illumination focus trajectory is analyzed for each color channel separately and optimally adapted to the respective spatial distribution of each fluorescent marker (middle: blue, ubiquitous nuclear RFP; orange, pan-neural GFP). To maximize resolution, different illumination focus trajectories are needed for the ubiquitous and pan-neural markers used in this experiment: switching illumination focus trajectories assigned to the two-color channels degrades spatial resolution substantially, leading to a loss of cellular resolution (see images labeled “Switched” vs. “Optimal” and corresponding line profiles shown at right). Scale bars, 20 μm ( c, left), 10 μm ( c, right).
: Spatiotemporally adaptive optimization of the 3D light-sheet path
a) In addition to the positions of detection focal planes ( D), lateral light-sheet offsets ( I) and axial positions of light-sheet waists ( Y), the adaptive imaging framework also optimizes the 3D orientation of light sheets by adjusting angular degrees of freedom and α β. ( b) On first principles, the light-sheet angle β inside a live specimen is expected to change between image planes as a result of refraction at the interface between mounting matrix ( 36 n m ≈1.339) and specimen ( 37 n e ≈1.35 for cytosol). By contrast, the light-sheet angle is not expected to vary across ovoid-shaped samples if their short axis is aligned with the illumination axis. ( α c) If light-sheet and detection focal planes are co-planar outside the sample but tilted with respect to each other inside the sample, not all sample regions illuminated by the light sheet are in focus simultaneously. At a depth of 50 μm in a Drosophila embryo, optimal focus settings change continuously across the image plane (see regions a, b, c and d), leading to a 2-μm focus spread that corresponds to β = 0.6°. ( d) The 3D orientation of the light sheet in the sample is automatically determined with a three-step algorithm: first, acquisition of a symmetric defocus stack; second, division of stack into sub-regions, DCTS focus curve computation for each sub-region, and determination of points ( x, y, d) characterizing the 3D light-sheet path; third, detection of outliers and robust reconstruction of angles and α β between light-sheet and detection focal plane. ( e) Measuring and correcting angular mismatches and α β between light sheets and detection focal planes improves spatial resolution beyond the level achieved by spatiotemporally adaptive imaging restricted to degrees of freedom D, I and Y. Representative examples of superficial and deep image regions in a Drosophila embryo are shown as enlarged views (purple, green) acquired with (top) and without (bottom) adaptive optimization of and α β. Line profiles (bottom) reveal sub-cellular features that are not resolved by correcting only D, I and Y. ( f) Experimentally measured and theoretically predicted (black and gray lines in β-plot) correction angles β across the volume of a D. melanogaster embryo. Predictions were obtained with a ray optics model that assumes average refractive indices of 1.339 and 1.35 of matrix and surface regions in the embryo 36 , respectively. The good agreement between experiment and model suggests that two main optical effects are responsible for angular mismatches of light sheets and detection focal planes inside the sample: (i) light-sheet refraction at the interface between embryo and surrounding matrix/medium, and (ii) curvature of detection focal planes inside the sample as a result of sample-induced lensing along the optical detection path. Scale bars, 5 μm ( 37 e).
: Spatiotemporally adaptive whole-brain functional imaging in larval zebrafish.
a) Geometrical outline of dorsal half of a zebrafish larval brain viewed from a dorsal perspective. Magenta and green boxes indicate the locations of the image data shown in b and d, respectively. ( b) Side-by-side comparison of image quality and spatial resolution in adaptively corrected and uncorrected image data of a representative midbrain region after 11 h of whole-brain functional imaging in a 4-d-old Tg(elavl3:GCaMP6f) zebrafish larva ( Supplementary Table 10). A detailed side-by-side comparison of multiple brain regions captured in this spatiotemporally adaptive whole-brain functional imaging experiment is shown as a function of time in Supplementary Video 9. ( c) Top, enlarged view of the image regions marked by orange boxes in b. Bottom, intensity line profile across three adjacent neurons, corresponding to the cyan lines in the image data shown above. Black arrows indicate the location of cell boundaries. Non-adaptive imaging fails to resolve individual cell identities, whereas adaptive imaging recovers and maintains single-cell resolution. ( d) Adaptive whole-brain imaging was performed for a total period of 20 h using an interleaved imaging scheme that acquires one complete brain volume every 375 ms and alternates between corrected (blue) and uncorrected (red) microscope states in subsequent volumetric scans. Both versions of the experiment start with the same initial (optimized) microscope state, i.e., all microscope parameters are identical at time point 0. Single-neuron activity traces are shown for two pairs of neurons in the forebrain region highlighted by a green box in a. One set of activity traces (A,B) shows high-speed functional data for a 3-min period at the 1-h mark of the experiment, whereas the other set (C,D) shows data at the 11-h mark. The fidelity of single-neuron activity traces is substantially improved by adaptive imaging already in the early phase of the time-lapse recording (1 h). In the late phase (11 h), further degradation of image quality and data fidelity affects multiple brain regions in the uncorrected image data, for which high resolution and image contrast is restored by AutoPilot-mediated microscope adaptation ( Supplementary Video 9). Scale bars, 20 μm ( b), 5 μm ( c), 10 μm ( d).
: Perturbation benchmark of spatiotemporally adaptive imaging performance
: Spatiotemporally adaptive imaging of
: Recovery of high spatial resolution in
Drosophila adaptive imaging
: Recovery of cellular resolution in deep tissue layers by adaptive imaging
: Spatiotemporally adaptive imaging of zebrafish embryogenesis
: Recovery of high spatial resolution in zebrafish adaptive imaging
: Quantification of resolution improvements in zebrafish adaptive imaging
: Spatiotemporally adaptive two-color imaging of neural development
: Spatiotemporally adaptive whole-brain functional imaging in larval zebrafish
: Example of system drift during non-adaptive long-term imaging