The ever-increasing speed and resolution of modern microscopes make the storage and post-processing of images challenging and prevent thorough statistical analyses in developmental biology. Here, instead of deploying massive storage and computing power, we exploit the spherical geometry of zebrafish embryos by computing a radial maximum intensity projection in real time with a 240-fold reduction in data rate. In our four-lens selective plane illumination microscope (SPIM) setup the development of multiple embryos is recorded in parallel and a map of all labelled cells is obtained for each embryo in <10 s. In these panoramic projections, cell segmentation and flow analysis reveal characteristic migration patterns and global tissue remodelling in the early endoderm. Merging data from many samples uncover stereotypic patterns that are fundamental to endoderm development in every embryo. We demonstrate that processing and compressing raw image data in real time is not only efficient but indispensable for image-based systems biology.
Understanding how biological tissues grow and sculpt organs of varied size, shape and function is one of the most intriguing questions in developmental biology today1. Non-invasive fluorescence microscopy is the primary tool to observe, track and quantify the development of biological specimens, but the imaging of living vertebrate embryos has been a challenge due to their size and opacity. Recently, light-sheet microscopy2,3, such as selective plane illumination microscopy (SPIM)4, has been shown to be a valuable technology that performs especially well in samples that are too large for conventional techniques, for example, confocal microscopy. In SPIM, instantaneous optical sectioning is achieved by illuminating the sample with a sheet of light and generating fluorescence in a thin slice, which is then imaged with a fast camera. Millimetre-sized specimens can be reconstructed by rotating and imaging them from different sides (multi-view imaging)5. Photo-toxicity in SPIM has been shown to be negligible even at high acquisition rates2,6. As a result, the imaging speed is becoming less dictated by how much light the sample can tolerate, rather, it is more determined by the camera’s speed.
With the development of fast, high-resolution sCMOS cameras, the rate and amount of data generated by SPIM is approximately three orders of magnitude higher than that of conventional confocal microscopes (Supplementary Fig. S1). SPIM setups with one or more sCMOS cameras7,8,9 may deliver important new information but at the same time present a major burden of data transfer, storage and management. Therefore, in many high-resolution microscope setups the number of experiments is limited by the available storage, which precludes the rigorous statistical analysis of image data needed for quantitative developmental biology. The issue of long-term storage can be tackled by compression of data8; however, data processing, visualization and analysis are still challenging as the dimensions of the images and the number of voxels do not change. Raw data rates may exceed 1 GB s−1, though may be far less dense in information content. Consequently, there is a need to process, condense and analyse these data ‘on-the-fly’.
One way to efficiently reduce the data stream would be to exploit knowledge about the shape of the sample. Microscope data sets are usually a series of flat images of planar sections and therefore cuboidal, independent of the object’s shape. For flat cell cultures or fixed tissue sections this may not be an issue. Many organisms however exhibit a spherical or ellipsoidal shape, such as embryos of major model organisms (zebrafish, frog, fruit fly and so on), and their cuboidal images are unnecessarily large and offer no dedicated way for visualization. We found that image transformations tailored to the shape of the sample can be performed in real time and efficiently compress the data stream from the microscope. Analysis and visualization of the desired scientific information become straightforward even when many samples are imaged.
The endoderm, one of the three germ layers, which is distributed as a monolayer on the surface of the spherical yolk during early stages of zebrafish development, exemplifies that it is essential to image the entire tissue with high spatial and temporal resolution. Whereas most studies on endoderm development have primarily focused on fate specification and organ formation10,11, little is known about the coordination of cells in space and time that patterns the tissue. Early endoderm migration is of immense importance for organ formation, as defects in cell movement during gastrulation result in duplication or mis-positioning of organs12. In the past, the endoderm has been analysed in isolated axial maximum intensity projections of small areas, where the projection is approximately orthogonal to the surface13,14.
These limitations are now overcome in our system: To quantify and understand cell behaviour in gastrula-stage zebrafish embryos, we designed a four-lens SPIM system with integrated real-time image processing that delivers radial maximum intensity projections at high speed. These undistorted high-resolution images of the entire endoderm are computed during the acquisition, in contrast to existing visualization techniques for three-dimensional (3D) data sets of fixed15 and live7 Drosophila embryos. In our approach, a typical data set requires Gigabytes instead of Terabytes and offers visualization and evaluation of cell migration in the context of entire embryos right out of the microscope. Taking insights from cartography, two-dimensional (2D) map projections are generated to visualize the entire tissue in a single image. Real-time data registration eliminates any movement of the embryo in its chorion and registers all embryos to a common, predefined coordinate system. Finally, cell tracks are computed to visualize cell localization, density and global migration dynamics in the zebrafish endoderm. We demonstrate that dedicated real-time image transformations efficiently reduce the data stream from the microscope and provide visualizations of complex morphological processes in whole embryos for the systematic analysis of a large sample size.
Four-lens SPIM setup for panoramic endoderm imaging
Light-sheet microscopy offers a number of advantages over conventional fluorescence microscopy techniques, in particular low photo-toxicity, fast and sensitive detection, and good penetration in scattering tissues2,3,4. Multi-view acquisition makes SPIM the ideal tool for in vivo imaging of millimetre-sized samples, which cannot be imaged entirely from a single side5,16. Combined with a fast ultrasonic rotational motor, a classical two-lens SPIM setup with one illumination and one detection lens can provide multiple views in a few seconds. Extending this approach, multidirectional SPIM uses a second illumination lens to uniformly excite the focal plane with superior penetration17. Furthermore, adding a second detection lens simplifies the acquisition as two opposing views are captured when scanning the sample through the light sheet7,8,9. We built a four-lens SPIM system (Fig. 1a) that delivers multiple views at high-speed and is optimized for capturing the entire zebrafish endoderm.
The central unit of our setup consists of four identical objective lenses (Fig. 1b). The two detection lenses and their associated optics image the common focal plane onto two sCMOS cameras. A linear stage moves the sample along the detection axis through the focal plane, triggering the acquisition of the cameras every two microns. The two illumination lenses alternately illuminate the focal plane with laser light at every position during the scan. The cameras capture an image of the entire field of view of ~1.2 mm2 at 62.5 frames per second (f.p.s.). As the cameras had been aligned to provide identical images from two opposing views, no image registration is required to merge the two.
Zebrafish embryos (5 h post fertilization (h.p.f.)) were embedded in 1.5% low-melting agarose in an fluorinated ethylene propylene (FEP) tube6. To visualize the endoderm, the transgenic zebrafish line Tg(sox17:EGFP)13,18, which expresses green fluorescent protein in all endodermal cells as well as the dorsal forerunner cells (DFC), was used. The chorion was left intact to provide ideal conditions for the embryo to develop6. The tube was mounted onto the stage, dipping into the medium-filled imaging chamber from the top. As the sample was moved through the light sheets, the cameras acquired the well-resolved sectors of the embryo (Fig. 1c,d). This process took <10 s.
We found that the image quality improved further when the embryo was rotated by 45° with a high-speed ultrasonic piezo motor at each time point to acquire a second, complementary view (Fig. 1e–h). A total of eight image stacks (two views for each illumination-detection lens pair) were acquired (Supplementary Movie 1). The additional view improved image quality mainly in the parts of the embryo directly facing the illumination lenses, which were not very well detected due to the larger optical path through the embryo and the limited axial resolution, and in the parts directly facing the detection lenses, which were obliquely illuminated. Owing to the fast motors and the high acquisition rate no developmental changes between the individual views were observed, ensuring that the embryo could be reconstructed in its entirety to visualize the process of endoderm patterning.
The shape of an early zebrafish embryo can be approximated by a sphere. On its surface, the endoderm forms a single-cell layer. Projecting the acquired 3D image volume onto the surface of a sphere captures all information about this layer in a single 2D image (Fig. 2, Supplementary Movie 2). To prepare for a radial projection in real time, a number of steps were performed before each experiment. First, fluorescent beads embedded in agarose were imaged from different angles. The resulting stacks were computationally registered, providing the transformation matrix to align both views (Supplementary Methods). Stage drifts were found to be negligible for the typical duration of an experiment. We therefore used the same transformation matrix throughout the time-lapse.
Transmission light through one of the detection lenses was used to position the embryo in the chamber and to determine the start and end of the stack for both angles. A transmission stack was captured, showing in each plane a different part of the embryo’s surface in focus. A sphere was fitted to these in-focus regions, resembling the surface of the embryo (Fig. 2a, Supplementary Fig. S2, Supplementary Methods). The surface of the sphere was evenly sampled, resulting in a list of ~2.7 million vertices (Fig. 2b).
Rays were sampled from the centre of the sphere to all vertices. A lookup table was prepared that associated voxels within a shell of 140 μm around the embryo surface with the corresponding vertex index (Fig. 2a,b, Supplementary Fig. S3). For the second view, the fitted sphere was transformed with the known registration matrix before sampling the rays. Eight tables were calculated this way, for each combination of illumination side, camera and acquisition angle. These initial steps to prepare for the time-lapse acquisition took about 10 min.
During the actual acquisition, the values of the relevant pixels of every image were assigned to the vertices according to the lookup table. For each vertex, only the maximum value was saved at the end of the stack. At every time point, the eight projections obtained in this way were fused to yield a complete picture of the embryo (Fig. 2c–e, Supplementary Movie 3). We registered the data of consecutive time points to eliminate global movements of the embryo in the chorion such as the turning at the end of gastrulation (Supplementary Movies 4 and 5). Fusion and registration were run in parallel on a separate computer. Our experiments ran for 12 h, one time point every 30 s. Performance analysis using offline data revealed that our radial projection implementation runs with up to 250 f.p.s. and can thus easily keep up with the acquisition speed of the cameras (62.5 f.p.s.). The amount of data was reduced by a factor of ~240 compared to the raw data stream, from 13 TB to 56 GB.
The challenge of visualizing and analysing spherical data has been well studied over decades in the geographical sciences and a number of projections have been developed19, each with well-defined properties. We applied several of these projection methods to the spherical data of the zebrafish endoderm (Fig. 3, Supplementary Movie 6). The conformal (angle-preserving) Mercator projection was calculated parallel to the acquisition as instant visual quality control and later to depict the orientation of cell flow during a time-lapse experiment (Supplementary Movie 7). The area-preserving Bonne and Gall-Peters projections were calculated to quantify the distribution of cell densities. The Winkel Tripel projection gave a good overview of the data with minimal overall distortion. In addition, the raw vertex data was kept for further processing such as cell tracking.
A single radial projection as described above can be applied to monolayered tissues such as the early endoderm. To make our approach more generally applicable we extended our software to multi-layer projections. The shell around the surface of the sphere, considered for the projections, was now subdivided into thin concentric spherical shells with Ri+1=Ri+dR. For each shell the maximum along each ray was computed separately yielding a set of projections, one for each radius Ri=R1,...,RN (Fig. 4a). The thickness of the shells dR and the number of layers N can be adjusted depending on the morphology of the sample.
In the radial multi-layered projections, the raw 3D data is converted into another 3D representation for the sample, which includes only the relevant parts of the sample in a form that can be easily visualized and analysed. For example, the layers can be colour-coded and overlayed (Fig. 4b–e). The multi-layer projections proved to be important at later stages during development when the tissue undergoes invagination at the dorsal midline of the embryo (Fig. 4d,e, Supplementary Movie 8) as well as in recordings where more than just the endoderm is fluorescently labelled (Supplementary Fig. S4). At the same time, the amount of data was still reduced by a factor of ~240/N.
Cell dynamics in the zebrafish endoderm
A complete understanding of the molecular and cell biological mechanisms underlying cell dynamics and patterning of the endoderm requires long-term imaging of the entire tissue combined with thorough quantitative analyses. During gastrulation, endoderm cells perform a random walk to rapidly spread on the yolk surface, followed by convergence and extension movements to gather along the embryonic axis12,13,14 (Supplementary Movie 7). The chemokine receptor Cxcr4a is expressed in the gastrula-stage endoderm and has been implicated in the regulation of endodermal cell migration. Two previous studies using morpholino-mediated (morphant) knockdown of Cxcr4a function described distinct endodermal phenotypes and proposed very different models for the role of Cxcr4a in endoderm migration. Mizoguchi et al.13 showed that the Cxcr4a ligand, Cxcl12b, can act as a chemoattractant for endodermal cells, and proposed that Cxcl12/Cxcr4a are required for chemotaxis and dorsal convergence movements of the endoderm13. In contrast, Nair and Schilling12 showed that Cxcr4a is required for transcriptional activation of integrin β1b and endodermal cell adhesion, and proposed a model where Cxcr4a is required for the tethering of endodermal cells to mesoderm in the early gastrula-stage embryo. cxcr4a morphant endodermal cells displayed a marked shift towards the animal pole12, a phenotype which was also observed in the cxcr4as421 mutant (Supplementary Fig. S5, unpublished observation by P.A.), and which might obscure defects in dorsal convergence movements as assessed by in situ hybridization for sox17 (refs 12, 13). To investigate the role of Cxcr4a in controlling endodermal cell migrational behaviour, we used our four-lens SPIM setup to image the entire endoderm in control and cxcr4a morphant Tg(sox17:EGFP) embryos, starting at 6 h.p.f. However, due to the low density of endodermal cells and the inevitable variance in cell location between embryos, a comprehensive map of global tissue flow can only be established by gathering and merging data from multiple samples.
We adapted our setup so that several embryos can be imaged in parallel. Up to six embryos were embedded in the same tube and one embryo after the other was positioned in front of the objectives using the vertical stage (Fig. 4f). This way all samples were imaged sequentially within 1 min and the development of different genotypes, for example, wild-type and cxcr4a morphant embryos, can be directly compared over time. Now an embryo can be reconstructed in its entirety in one of two possible recording modes: For ultimate image quality the embryos are rotated once for a total of eight data stacks as described above. Alternatively, the additional view is omitted to save time and image more embryos in parallel.
We recorded the development of four embryos in parallel using the four-lens SPIM setup without rotation (Supplementary Movie 9). To analyse the cellular movements that pattern the endoderm, we segmented the cells (Supplementary Movie 10) and quantified various parameters using the original spherical representation and appropriate map projections. Flow maps were used to depict the path of cells and their angle with respect to the dorsal midline on Mercator projections (Supplementary Movie 11). While each embryo exhibits a slightly different flow pattern (Fig. 4g–j), the convergence movement of cells, with their tracks orthogonal to the dorsal midline, is present in all embryos and was visualized by overlaying all the tracks (Fig. 4k).
Endodermal cell migration in WT and cxcr4a morphants
Time-lapse images of wild-type embryos showed cells on both lateral surfaces converging towards the dorsal midline while the embryonic axis extended in the A-P-direction (Fig. 5a–d, Supplementary Movie 7). Anteriorly located cells began to converge during early gastrulation (Fig. 6a) and progressed later to more lateral regions (Fig. 6c). The laterally located cells migrated in a directed manner, orthogonal to the midline (as seen by the red tracks in Fig. 6a,c, Supplementary Movie 11). In contrast, in cxcr4a morphants, the cells failed to converge properly at the midline (Figs. 5e–h and 6b,d; Supplementary Fig. S6 and Supplementary Movies 12 and 13), consistent with the report of Mizoguchi et al.13
The chemoattractant model predicts that loss of Cxcr4a function should affect the directionality of endodermal cells. To quantify directionality independent of cell distribution, we extracted a general profile of tissue flows by computing the mean orientation of flow through each point in space (averaged across time and samples). The resulting flow fields were visualized with streamlines, indicating the general patterns of cell motion present during gastrulation (Fig. 6e,f). We found that the averaged streamlines show a consistent flow pattern, which becomes more complete with increasing sample size (Supplementary Fig. S7). Surprisingly, the streamlines of cxcr4a morphant endodermal cells were indistinguishable from those of wild-type cells (Fig. 6g; n=12) indicating that the direction of movement is not affected in the absence of Cxcr4a function.
An alternative model is that the early mis-positioning of endodermal cells affects later dorsal convergence movements. To visualize how the spatial distribution of endoderm cells change over time, we calculated cell densities on the area-preserving Bonne projections. The distribution of cells in cxcr4a morphants during early gastrulation appeared more asymmetric with a clear bias towards the anterior side as compared with wild type (Fig. 6h–k, Supplementary Movie 14). Again, to merge data from multiple embryos and reveal common characteristics the density maps were averaged. For a more precise quantification of this asymmetric distribution in a well-defined reference frame, we calculated cell densities in annular regions of increasing radii around the DFCs for each time point (Supplementary Fig. S8). The asymmetry was consistent with the previously observed anterior shift of endoderm cells12, resulting in a larger gap between the DFCs and the endoderm margin. In both wild-type and cxcr4a morphant embryos, the mean distance between DFCs and the endoderm margin was found similar at 60% epiboly. However, in cxcr4a morphants this gap increased up to a maximal width of 250 μm (65 μm higher than that in the wild type) before decreasing, further demonstrating the anterior shift of endoderm cells in cxcr4a morphants (Fig. 6l, n=12 each). The higher variation in the gap sizes of morphant compared with control embryos reflects a higher embryo-to-embryo variability that is commonly seen with morpholino injections.
Our analysis shows that loss of Cxcr4a function affects the early distribution and subsequently the end-point position of endodermal cells, without affecting directionality, indicating that the dorsal convergence defects observed in cxcr4a morphant endoderms are due not to a loss of directionality but rather to the relatively longer distance that these cells have to migrate to reach the dorsal midline by the end of gastrulation. These results suggest that Cxcr4a is not required for chemotaxis, and illustrate the requirement for global imaging and multi-sample analysis to understand the true nature of cell migration defects in mutants and morphants such as cxcr4a.
We have built a dedicated four-lens SPIM setup providing multiple views at high speed and combined it with a novel, real-time processing engine. In contrast to common 3D imaging approaches, we took advantage of the spherical geometry of the zebrafish embryo to extract the desired information while efficiently reducing the amount of data. The raw data from the cameras were not saved at any point. On the one hand, this means that any additional 3D information that is not captured in the projection gets lost and the projection cannot be repeated at a later time point. On the other hand, the radial projection is an effective compression that immediately delivers preprocessed data for further analysis. Therefore, even though the presented real-time processing will not eliminate the need for slow 3D imaging techniques in more complex shapes, it is a very effective strategy to streamline further analysis and increase throughput in this and other applications.
In contrast to axial projections, our radial maximum intensity projection preserved the topology of the tissue. The entire endoderm was imaged quickly with high image quality and displayed for fast visual inspection and precise quantification without distortions irrespective of the actual radius of the sample. The data were conveniently displayed using appropriate map projections, which not only facilitate the analysis of tissue-wide cell movements but also eliminate the potential errors in conventional axial projections. The comprehensive data sets are essential to understand the complex cell behaviours during endoderm development and the dissection of the molecular mechanisms that control these behaviours, such as the chemokine receptor Cxcr4a. Our results suggest that the anterior shift in the endoderm cell distribution in cxcr4a morphant embryos accounts for the defects observed in dorsal convergence during late gastrulation.
Imaging an entire tissue with high spatial and temporal resolution may reveal details of developmental processes in the embryo under observation. At the same time, the information from an individual sample needs to be compared to a large ensemble to quantify the variability and the significance of the findings. Therefore, the goal in quantitative developmental biology should be to combine the ideal microscopy technique with appropriate image processing that allows fast visualization and direct analysis of the data. In light sheet microscopy, due to its low photo-toxicity and fast acquisition, data processing has become a serious bottleneck. The presented imaging pipeline gives an example of efficient image acquisition and processing in real time, compromising neither on acquisition speed nor on lateral resolution. Single-cell dynamics and processes at the level of whole tissues can now be simultaneously visualized and correlated. We have combined our multi-sample mounting and acquisition protocols to image many samples in parallel with data processing routines to merge the data from individual embryos, for example, into streamlines or averaged density maps. Further work is now needed to apply this novel toolbox to high-throughput microscope setups to obtain the high sample rates essential for developmental systems biology.
The solution presented here is applicable not just for the zebrafish embryo but also for other spherical samples and can be adapted to other convex shapes such as fly embryos, worms and so on. As shown, it can also be extended to multi-layered tissue by replacing the single radial maximum intensity projection with a multi-layered stack. Multi-layer projections can preserve the surface topology and account for, quantify and recover non-spherical samples.
Even though our image-processing pipeline runs very efficiently in Fiji20 on a common workstation computer, the calculations could be performed even faster on dedicated hardware, ideally directly on the frame grabber card or even inside the camera. Instead of storing and post-processing large volumes of image data, the automated extraction of relevant information in real time opens the door for quantification of features in a large number of samples at high speed.
The four-lens SPIM setup consists of four identical water-dipping Olympus UMPLFLN 10x/0.3 objectives, two for illumination and two for detection. The sample chamber was custom-made from acrylic with four openings sealed with rubber o-rings. The sample position and the light sheets are monitored with a webcam through a glass window on the bottom of the chamber. Two Toptica iBeam smart (488 nm, 200 mW) lasers were externally triggered for alternating double-sided illumination. Each beam was guided with an optical fibre onto a continuously running galvanometric mirror (1 kHz, EOPC) for even fluorescence illumination (multidirectional SPIM17). Light sheets were generated with cylindrical lenses and projected with telescopes and the illumination objective lenses onto the common focal plane of both detection lenses. The average excitation power in the entire object plane was ~6 mW per arm. The focal planes of the two detection objectives were imaged onto two Andor Zyla sCMOS cameras through emission filters (Chroma ET BP525/50). Both cameras were precisely aligned via a focusable tube lens and an adjustable mirror to acquire images from the same focal plane, which were streamed to separate computers. The sample was moved and rotated with three motorized linear stages (M-404.1PD/6PD, M-112.1DG) and one rotational stage (M-660.55, Physik Instrumente) using a custom-made software (Supplementary Methods).
Zebrafish were handled in accordance with EU directive 2011/63/EU as well as the German Animal Welfare Act. The transgenic zebrafish line Tg(sox17:EGFP)13,18 was used for visualizing endodermal cells during gastrulation. Embryos were collected after fertilization and incubated at 32 °C to speed up development while the temperature during imaging was kept constant at 24 °C. For morphant experiments, embryos were injected with 10 ng of cxcr4a morpholino (5′-AGACGATGTGTTCGTAATAAGCCAT-3′) at the 1-cell stage12.
Low-melting agarose (1.5%) solution was prepared in E3 medium and maintained at 37 °C. At 30% epiboly stage, embryos with chorion were transferred into low-melting agarose and sucked into a cleaned FEP tube6 (inner diameter: 2.0 mm, wall thickness: 0.5 mm), using a syringe and needle. The embryos are positioned on top of each other with minimum gap between them. The agarose was allowed to solidify at room temperature for 5 min and the FEP tube was mounted on the stage, dipping from the top into the sample chamber filled with E3 medium for time-lapse acquisition.
The zebrafish embryo was moved along the detection axis through the alternating light sheets to acquire a z-stack of 402 planes. The two cameras were triggered simultaneously by the stage every 2 μm, acquiring one projection for each combination of illumination side and camera. The laser light was triggered when the entire chip was exposed avoiding any artefacts due to the rolling shutter. The sample was then rotated by 45° to acquire a second view, generating a total of eight projections at each time point. These steps were repeated at an interval of 30 s for a period of 12 h. In parallel, the eight projections were fused using nonlinear blending to generate a single data set based on the transformations pre-calculated with the bead sample. For imaging multiple samples in parallel, we omitted the rotation of the sample. Instead, a z-stack was acquired for one embryo and the stage translated the tube along the y axis to position the next embryo in front of the objective lenses. By repeating this, a time point is acquired for each sample and the stage moves back to bring the first embryo in the field of view. This entire process is repeated for time-lapse acquisition of multiple samples.
To confirm that the laser light does not induce any developmental defects we compared two embryos that were embedded in the same tube. One was imaged for 12 h, whereas the other one was not exposed to any laser light. We found no morphological differences (Supplementary Fig. S9) and both embryos were raised without defects. We also performed a timelapse with reduced speed, in which an embryo was imaged only every 15 min. We could not observe any difference to an embryo that was imaged every 30 s, as usually done in single-sample experiments (Supplementary Movie 15). At the end of an experiment we captured another transmission stack to verify that the initially fitted sphere model had been valid during the course of the entire timelapse.
Registration of time points
When the embryonic axis forms towards the end of gastrulation, the embryo inside the chorion turns by 90°. To separate this global movement from local cell movements, images of consecutive time points were registered rigidly (Supplementary Movies 4 and 5). For this purpose, the Fuller projection was calculated at each time point and smoothed using a Gaussian kernel with σ=1.5. ImageJ’s MaximumFinder plugin was used to detect local image maxima with an intensity tolerance of 10, which detected the centres of the majority of cells. The best rigid transformation between the cell centres of consecutive time points was then estimated using the Iterative Closest Point algorithm21,22.
The software for radial projection can be applied both in real-time (images obtained directly from the camera) and post-processing (images read from the hard drive). To ensure that our software is working correctly we performed a time-lapse experiment with reduced temporal resolution, allowing us to save the raw image data. We then calculated radial projections post-processing and compared it to the raw data for consistency. Using offline data we found that our processing framework can deal with data rates of up to 250 f.p.s.
Acquisition computers used for real-time processing were two Dell T7500 workstations (2 × 2.3 GHz Dual-Core, 60 GB RAM). As soon as projections were created on the acquisition computers, they are copied via a gigabit Ethernet connection to a Mac Pro workstation (2 × 2.26 GHz Quad-Core Intel Xeon, 32 GB RAM) for fusion, registration and map projections. The source code for radial maximum projection, fusion, registration and map projection is publicly available (Supplementary Software 1).
2D map projections
Generally, map projections aim to find a planar projection of a spherical surface such that angles, areas, distances and shapes are preserved. However, it is impossible to meet all of these requirements in a single projection, and therefore, many different types of projections have been invented. Applied to our biological data, the distortions introduced by a particular projection type influence the interpretation of the data, so that the choice for a projection needs to be taken with care. In our application we are quantifying and visualizing cellular movement of the endoderm. For analysing directionality of cell flow, the Mercator projection is ideal as it is conformal, that is, angle-preserving. To study the density of cells, area-preserving projections such as the Bonne or Gall-Peters projections are preferred. Finally, for the visualization of the global cellular flow projections reducing the overall shape distortions are preferable, such as the Fuller and Winkel Tripel projections. Map projections are ideal for visualization, as they depict the entire surface in a single 2D image. For further processing and quantification, however, we chose to use the raw vertex data where possible to avoid distortions. The final projections depict the data in polar coordinates instead of Cartesian coordinates. This way slight size differences between embryos are eliminated and their cellular pattern can be directly compared. In our projections, the commonly used scale bar is not applicable any more. The radius of the sphere provides the necessary information about the dimensions of the embryo. Distortions in the Mercator projection resulting from an incorrectly determined centre or a non-spherical embryo have been quantified (Supplementary Fig. S10) and would allow us to identify and reject such data.
A Java library that was originally designed for cartography and implements all common map projections ( http://www.jhlabs.com/java/maps/proj) was used to map the spherical data on to 2D planar images.
Cell segmentation and tracking
The projected raw images were processed with a Retinex filter23 to enhance the local image contrast and reduce spatial in homogeneity in image intensities. A total-variation filter was applied to reduce noise while preserving image structures24. Both filters were implemented in C++.
Cells were detected in the Mercator projections by applying an active contours method, which allowed us to control spatial regularity of detected cell shapes25. An adapted size constraint was used to eliminate small, noisy artefacts, where the local area distortions of the mapping were explicitly taken into account by using an adaptive size constraint with respect to the latitude position of each cell. Each cell was tagged with an individual ID with connected component labelling. Dense deformation fields between consecutive frames were obtained by applying a fast fluid image registration26. Cells were finally tracked by propagating the detected cell masks with the obtained flow fields from the registration and subsequent consistency check27 (Supplementary Methods). All methods were implemented in Mathematica v8.0 (Wolfram Research) except for the registration and raw segmentation (C++).
Feature quantification and visualization
All relevant features were evaluated either in the suitable projections or in spherical coordinates: directions of cell migration on the conformal Mercator map, densities on the area-preserving Bonne projection and distances were evaluated as great circle distances in the original spherical coordinates.
Overlays of cell tracks from multiple samples were computed after alignment of all samples to the position of the DFCs at tailbud stage. Streamline plots, that is, lines of flow that are tangential to the flow vectors at each point, were computed from a static global flow field obtained by estimating the mean flow direction at each point in space. Flow directions were estimated by computing the average direction (vector sum of normalized directions) of trajectory samples (length n=10 data points) in a spatial box around each point.
The margin around the DFCs was analysed by calculating the distance of each cell from the DFC and subsequent binning of the obtained distances. Radial histograms from each time point were concatenated to produce the kymographs (Supplementary Fig. S8). The radii of the margin around the DFC were estimated by the 0.1 quantile of the annular distribution for each time point. The planar visualizations were computed in Mathematica, and volume renderings were done with custom Python scripts and Blender ( http://www.blender.org/).
Analysis of movies (fluid registration, cell detection, tracking and visualization) was done on a MacBook Pro with 8GB of RAM and a QuadCore 2.3 GHz Intel Core i7. To further speedup post-processing and analysis of multi-sample data, several movies were processed in parallel on a compute server with 24 Intel Xeon CPUs running at 2.67 GHz.
How to cite this article: Schmid, B. et al. High-speed panoramic light-sheet microscopy reveals global endodermal cell dynamics. Nat. Commun. 4:2207 doi: 10.1038/ncomms3207 (2013).
Lecuit, T. Tissue Remodeling and Epithelial Morphogenesis. Current Topics in Developmental Biology Academic Press (2009).
Huisken, J. & Stainier, D. Y. R. Selective plane illumination microscopy techniques in developmental biology. Development 136, 1963–1975 (2009).
Weber, M. & Huisken, J. Light sheet microscopy for real-time developmental biology. Curr. Opin. Genet. Dev. 21, 566–572 (2011).
Huisken, J., Swoger, J., Del Bene, F., Wittbrodt, J. & Stelzer, E. H. K. Optical sectioning deep inside live embryos by selective plane illumination microscopy. Science 305, 1007–1009 (2004).
Swoger, J., Verveer, P. J., Greger, K., Huisken, J. & Stelzer, E. H. K. Multi-view image fusion improves resolution in three-dimensional microscopy. Opt. Express 15, 8029–8042 (2007).
Kaufmann, A., Mickoleit, M., Weber, M. & Huisken, J. Multilayer mounting enables long-term imaging of zebrafish development in a light sheet microscope. Development 139, 3242–3247 (2012).
Krzic, U., Gunther, S., Saunders, T. E., Streichan, S. J. & Hufnagel, L. Multiview light-sheet microscope for rapid in toto imaging. Nat. Meth. 9, 730–733 (2012).
Tomer, R., Khairy, K., Amat, F. & Keller, P. J. Quantitative high-speed imaging of entire developing embryos with simultaneous multiview light-sheet microscopy. Nat. Meth. 9, 755–763 (2012).
Weber, M. & Huisken, J. Omnidirectional microscopy. Nat. Meth. 9, 656–657 (2012).
Kikuchi, Y. et al. casanova encodes a novel Sox-related protein necessary and sufficient for early endoderm formation in zebrafish. Genes Dev. 15, 1493–1505 (2001).
Warga, R. M. & Nüsslein-Volhard, C. Origin and development of the zebrafish endoderm. Development 126, 827–838 (1999).
Nair, S. & Schilling, T. F. Chemokine signaling controls endodermal migration during zebrafish gastrulation. Science 322, 89–92 (2008).
Mizoguchi, T., Verkade, H., Heath, J. K., Kuroiwa, A. & Kikuchi, Y. Sdf1/Cxcr4 signaling controls the dorsal migration of endodermal cells during zebrafish gastrulation. Development 135, 2521–2529 (2008).
Pézeron, G. et al. Live analysis of endodermal layer formation identifies random walk as a novel gastrulation movement. Curr. Biol. 18, 276–281 (2008).
Luengo Hendriks, C. L. et al. Three-dimensional morphology and gene expression in the Drosophila blastoderm at cellular resolution I: data acquisition pipeline. Genome Biol. 7, R123 (2006).
Preibisch, S., Saalfeld, S., Schindelin, J. & Tomancak, P. Software for bead-based registration of selective plane illumination microscopy data. Nat. Meth. 7, 418–419 (2010).
Huisken, J. & Stainier, D. Y. R. Even fluorescence excitation by multidirectional selective plane illumination microscopy (mSPIM). Opt. Lett. 32, 2608–2610 (2007).
Sakaguchi, T., Kikuchi, Y., Kuroiwa, A., Takeda, H. & Stainier, D. Y. R. The yolk syncytial layer regulates myocardial migration by influencing extracellular matrix assembly in zebrafish. Development 133, 4063–4072 (2006).
Snyder, J. P. Flattening the Earth: Two Thousand Years of Map Projections University of Chicago Press (1997).
Schindelin, J. et al. Fiji: an open-source platform for biological-image analysis. Nat. Meth. 9, 676–682 (2012).
Besl, P. J. & McKay, N. D. A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell 14, 239–256 (1992).
Rusinkiewicz, S. & Levoy, M. Efficient variants of the ICP algorithm. inProceedings of the third international conference on 3-D Digital Imaging and Modeling 145–152 (2001).
Land, E. H. & McCann, J. J. Lightness and retinex theory. J. Opt. Soc. Am. 61, 1–11 (1971).
Chan, T. F., Osher, S. & Shen, J. The digital TV filter and nonlinear denoising. IEEE Trans. Image process. 10, 231–241 (2001).
Chan, T. F. & Vese, L. A. Active contours without edges. IEEE Trans. Image Process. 10, 266–277 (2001).
Kuska, J.-P., Scheibe, P. & Braumann, U.-D. Fast fluid extensions for image registration algorithms. Proc. Int. Conf. Image Proc. 2408–2411 (2008) doi:10.1109/ICIP.2008.4712278.
Scherf, N. et al. Imaging, quantification and visualization of spatio-temporal patterning in mESC colonies under different culture conditions. Bioinformatics 28, i556–i561 (2012).
Schmid, B., Schindelin, J., Cardona, A., Longair, M. & Heisenberg, M. A high-level 3D visualization API for Java and ImageJ. BMC Bioinformatics 11, 274 (2010).
This project was initiated in the laboratory of Didier Stainier at UCSF, and J.H. and P.A. thank Didier Stainier for generous support. We also thank Johannes Schindelin for his Fiji Plugin, Tom Stückemann for discussions and the reviewers for their constructive criticism. The work in the lab of J.H. was supported by the Max Planck Society and the Human Frontier Science Program (HFSP). The work in the group of I.R. was supported by the European Commission project EuroSyStem (200270) and the HFSP-grant RGP0051/2011.
The authors declare no competing financial interests.
Supplementary Figures S1-S10, Supplementary Methods and Supplementary References (PDF 1836 kb)
Source code for radial maximum projection, fusion, registration and map projection. (ZIP 1607 kb)
Illustration of multi-view acquisition. During acquisition, the sample is moved through the light sheet that is generated alternately by two illumination lenses (top and bottom). Each of the two detection lenses (left and right) captures one hemisphere. Indicated are the parts of the sample that are imaged under ideal conditions. A second view is acquired after turning the sample by 45°. The movie demonstrates the interplay of illumination, detection and sample orientation to cover the entire embryo with uniform image quality. (MOV 3000 kb)
Flattening the embryo. The entire embryo is assembled from eight parts, each of which is imaged best by a specific combination of illumination, detection and sample orientation. This movie shows how the different parts get mapped onto a 2D plane, in this case using the Gall-Peters projection. The four colors highlight the parts that are acquired with two cameras for both orientations of the sample. (MOV 5741 kb)
Illustration of real-time projection. While the sample is moved through the light sheet the projection is updated in real-time. A specific combination of illumination side, detection camera and rotation angle fills in the values for one of the eight parts (see also Supplementary Movie 1). Once all eight parts are obtained, they are fused via non-linear blending. The Mercator projection is shown here for illustration, while the actual real-time projection is performed using sphere vertices instead of a regular pixel grid. (MOV 4233 kb)
: The benefit of registration (orthogonal projection). Two consecutive images are aligned with each other to eliminate any global movement of the embryo itself and to preserve local cell movements. The data is then arranged such that the axis of the embryo lands in the middle of the image at the final time point. This movie compares an unregistered (left) and registered (right) projection. An orthogonal projection is used here since it provides the most intuitive illustration of global rotations to the human eye. (MOV 6086 kb)
The benefit of registration (Mercator projection). This movie demonstrates the effect of registration on the Mercator projection (same data as in Supplementary Movie 3). (MOV 5633 kb)
Overview of different projection types. The same data set is shown using different map projections: (a) Winkel Tripel and (b) Fuller (distortion reducing), (c) Mercator (angle-preserving), (d) Bonne (area-preserving) and (e) orthogonal projection. (MOV 6750 kb)
Projected data of a wildtype embryo. Focusing on the first hours of development, this movie shows a Winkel Tripel projection of the raw registered data of a wild-type embryo in full temporal resolution, acquired every 30 s. Frame rate: 30 frames per second (fps), i.e. 1 second in the movie corresponds to 15 minutes experimental time. (MOV 12492 kb)
Multi-layer projection of a wildtype embryo. In order to visualize the invagination of endoderm post gastrulation, multilayer projection was generated. The movie shows different radial layers in different colors to visualize the 3D information in a single image. Frame rate: 15 frames per second (fps), i.e. 1 second in the movie correspond to 15 minutes experimental time. (MOV 1187 kb)
Multi-sample time-lapse of wildtype embryos. Four wildtype embryos were mounted in the same FEP tube and imaged simultaneously for 12h. This movie shows the raw registered data on a Mercator projection, acquired every minute. Frame rate: 20 frames per second (fps), i.e. 1 second in the movie correspond to 20 minutes experimental time. (MOV 7338 kb)
Cell segmentation. This movie demonstrates the results of cell detection and tracking. A wild-type dataset is shown in Mercator projection. Detected cell masks are indicated as colored overlay on the original image sequence. Individual cells are shown in different colors. Cell detection starts at 1 h, when the GFP signal is bright enough to be reliably detected and ends at 5 h, when individual cells are no longer discernible around the dorsal midline. Frame rate: 30 fps, i.e. 1 second in the movie corresponds to 15 minutes experimental time. (MOV 3589 kb)
Orientation of cell movements in a wildtype embryo. This movie shows the orientation of cell movement in a wildtype embryo over time. The Mercator projection is chosen to preserve angles. Cell tracks are shown for each detected cell. Color scale indicates the unsigned angle between the cell track (end-to-end vector) and the horizontal axis (dorsal midline) from blue (0) to red (p/2). Cell detection begins at 1h, when the GFP signal is bright enough to be reliably detected and ends at 5h, when individual cells are no longer discernible around the dorsal midline. Frame rate: 60 fps, i.e. 1 second in the movie corresponds to 30 minutes experimental time. (MOV 553 kb)
Projected data of a cxcr4a morphant embryo. Focusing on the first hours of development, this movie shows a Winkel Tripel projection of the raw registered data of a morphant embryo in full temporal resolution, acquired every 30 s. Frame rate: 30 fps, i.e. 1 second in the movie corresponds to 15 minutes experimental time. (MOV 11613 kb)
Orientation of cell movements in a cxcr4a morphant embryo. This movie shows the orientation of cell movement in a cxcr4a morphant embryo over time. The Mercator projection is chosen to preserve angles. Cell tracks are shown for each detected cell. Color scale indicates the unsigned angle between the cell track (end-to-end vector) and the horizontal axis (dorsal midline) from blue (0) to red (p/2). Cell detection begins at 1 h, when the GFP signal is bright enough to be reliably detected and ends at 5 h, when individual cells are no longer discernible around the dorsal midline. Frame rate: 60 fps, i.e. 1 second in the movie corresponds to 30 minutes experimental time. (MOV 704 kb)
Comparison of cell densities in a wildtype and cxcr4a morphant embryo. This movie compares the cell density distribution in a wildtype and cxcr4aMO embryo over time. The sinusoidal Bonne projection is chosen to preserve areas. The color code for densities range from blue (0) to white (>8 cells/1000 μm2). Frame rate: 60 fps, i.e. 1 second in the movie corresponds to 30 minutes experimental time. (MOV 4001 kb)
Wildtype embryos imaged at different rates. To check for the phototoxic effect on embryos due to high speed imaging, two wildtype embryos were imaged at different intervals. One imaged every 30 seconds and the other was sampled only every 15 minutes. This movie compares the dynamics of the two on an orthogonal projection. (MOV 2702 kb)
About this article
Cite this article
Schmid, B., Shah, G., Scherf, N. et al. High-speed panoramic light-sheet microscopy reveals global endodermal cell dynamics. Nat Commun 4, 2207 (2013). https://doi.org/10.1038/ncomms3207
Annual Review of Cell and Developmental Biology (2019)
A Numerical Framework for Efficient Motion Estimation on Evolving Sphere-Like Surfaces Based on Brightness and Mass Conservation Laws
SIAM Journal on Imaging Sciences (2019)
Lightsheet localization microscopy enables fast, large-scale, and three-dimensional super-resolution imaging
Communications Biology (2019)
Journal of Neuroscience Methods (2019)
Biomedical Engineering Letters (2019)