ClearVolume: open-source live 3D visualization for light-sheet microscopy

To the Editor:

Current state-of-the-art light-sheet microscopes rely on sophisticated control software to perform the acquisition of gigabytes of image data per second over the course of hours or even days. Typically the microscopes acquire data in a first step, and in a second step these data are processed and visualized offline. The delay between data acquisition and data assessment wastes time and storage space. Technology that enables viewing and assessing data during imaging would offer significant advantages. However, even the most advanced microscopes display only the latest image plane acquired or projection while the raw volumetric data are saved to disk1,2,3,4.

Ideally, one would be able to instantly visualize and analyze time-resolved multichannel three-dimensional (3D+t) data during acquisition, virtually rotate and zoom into the sample, go back in time to replay dynamic processes of interest, and even access the live data feed remotely. From a biological perspective this would shorten the time needed to tune imaging parameters and assess a sample's viability or relevance. Experimentally, the ability to remotely visualize a 3D+t data stream would facilitate the monitoring of long-term time-lapse imaging. From an optics point of view, live visualization would also be helpful when performing system design and alignment. Indeed, irrespective of the experimentalist's goal, the earlier one can visualize and assess the acquired data in their spatiotemporal entirety, the more effective live light-sheet microscopy becomes in the discovery process.

In this work, we present ClearVolume, an open-source package for live graphical processing unit (GPU)-accelerated 3D+t multichannel visualization. It is designed to easily augment any custom-built light-sheet microscope with real-time visualization capabilities. For that purpose ClearVolume has a simple interface for all major programming languages and frameworks used for controlling light-sheet microscopes: Java, C/C++, Python, LabVIEW and OpenSPIM (μManager) (Fig. 1a, Supplementary Videos 1, 2, 3 and Supplementary Note 2). Key features especially relevant to light-sheet microscopy are the capability to create instant multiview and multicolor renderings (Fig. 1d) as well as instant rewind and replay of time-lapse recordings (Fig. 1c). Furthermore, ClearVolume provides real-time GPU-based image processing capabilities, such as image sharpness estimation and sample drift tracking (Fig. 1c). Finally, ClearVolume enables live streaming of 3D data in real time over the Internet (Fig. 1b). This effectively decouples the physical location of the microscope from the experimenter's location. For example, in the case of long-term developmental imaging, it is possible to remotely inspect the health of the embryo and the quality of the recorded volumes. Overall, ClearVolume enhances the microscope's ergonomics and permits the immediate 3D perception of the imaged processes.

Figure 1: ClearVolume: an open-source real-time multichannel 3D visualization toolkit for microscopy.
figure1

(a) ClearVolume can interface with a number of software systems, such as LabVIEW or μManager. It also features language bindings for C/C++, Java and Python. (b) The volumetric data acquired by a connected microscope can either be rendered locally or sent to a different computer for remote inspection. (c) In addition to live volumetric rendering, ClearVolume features real-time sharpness and sample drift evaluation, instant replay and arbitrary 2D and 3D overlays. (d) ClearVolume can selectively visualize multiview, multichannel data sets.

The value of integrating ClearVolume into a high-speed light-sheet microscope optimized for cellular imaging is demonstrated with a volumetric acquisition of a 12-cell-stage Caenorhabditis elegans embryo undergoing mitosis (Supplementary Video 4 and Supplementary Note 1). The embryo could be virtually rotated, and regions of interest, such as cell divisions, could be inspected while 3D data were acquired at six stacks per second.

Further, we demonstrated the use of ClearVolume for long-term time-lapse imaging with an OpenSPIM microscope5. ClearVolume was readily integrated into the μManager (www.micro-manager.org/) plug-in to allow live 3D visualization. We used ClearVolume to remotely monitor a developing Drosophila melanogaster embryo to check on sample drift, photobleaching and other artifacts (Supplementary Video 5). Time-shifting allows inspection of the data at any given point in time during acquisition (Supplementary Video 6).

In addition, ClearVolume can be used for aligning and calibrating light-sheet microscopes. The overall 3D point spread function of the optical system and the full 3D structure of the beam (for example, Gaussian or Bessel beam) can be visualized (Supplementary Videos 7 and 8). To aid manual alignment of the microscope, ClearVolume computes image sharpness in real time and provides audiovisual feedback Supplementary Video 9). Finally, ClearVolume computes and visualizes sample drift trajectories and makes that information available through an interface for stabilization of the sample (Supplementary Video 10).

ClearVolume can also be used on other types of microscopes such as confocal microscopes (Supplementary Video 11). Going beyond microscopy, ClearVolume is integrated with the popular Fiji6 (http://fiji.sc) and KNIME (http://www.knime.org/) software packages, bringing real-time 3D+t multichannel volume visualization to a larger community (Supplementary Videos 12,13,14). Furthermore, ClearVolume's modularity allows any user to implement additional CPU- or GPU-based modules for image analysis and visualization.

In summary, ClearVolume makes live imaging truly live by enabling direct real-time inspection of the specimen imaged in light-sheet microscopes. The source code of ClearVolume is available at http://clearvolume.github.io.

References

  1. 1

    Tomer, R., Khairy, K., Amat, F. & Keller, P.J. Nat. Methods 9, 755–763 (2012).

    CAS  Article  Google Scholar 

  2. 2

    Krzic, U., Gunther, S., Saunders, T.E., Streichan, S.J. & Hufnagel, L. Nat. Methods 9, 730–733 (2012).

    CAS  Article  Google Scholar 

  3. 3

    Chen, B.-C. et al. Science 346, 1257998 (2014).

    Article  Google Scholar 

  4. 4

    Mickoleit, M. et al. Nat. Methods 11, 919–922 (2014).

    CAS  Article  Google Scholar 

  5. 5

    Pitrone, P.G. et al. Nat. Methods 10, 598–599 (2013).

    CAS  Article  Google Scholar 

  6. 6

    Schindelin, J. et al. Nat. Methods 9, 676–682 (2012).

    CAS  Article  Google Scholar 

Download references

Acknowledgements

Thanks to M. Sarov of the TransgeneOmics facility (MPI-CBG) for providing C. elegans lines. Thanks to C. Schmied (MPI-CBG) for providing us with D. melanogaster embryo samples. Thanks to P. Tomancak, E. Betzig and S. Saalfeld for advice. Thanks to S. Bundschuh and the Light Microscopy Facility (MPI-CBG). This work was supported by the German Federal Ministry of Research and Education (BMBF) under the funding code 031A099.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Loic A Royer.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Texts and Figures

Supplementary Notes 1 and 2 (PDF 4120 kb)

Supplementary Video 1

C/C++ interface. Demonstrating how to use ClearVolume binding to C/C++ to visualize a live stream of volumetric data. (MP4 2213 kb)

Supplementary Video 2

LabVIEW interface. Demonstrating how to use ClearVolume binding to LabVIEW to visualize volumetric data. (MP4 5368 kb)

Supplementary Video 3

Python interface. Demonstrating how to use ClearVolume binding to Python to visualize volumetric data. (MP4 7072 kb)

Supplementary Video 4

Live 3D acquisition and view of a C. elegans embryo (His-GFP+BTub-GFP). The embryo is at the AB8 stage, 50 minutes after first division of the AB founder cell. We see 12 cells, two of which are undergoing mitosis (MS and E founder cells) . We used our high-speed light sheet microscope capable acquiring 6 stacks per second. In a second part the video shows a full screen view of the same embryo where we center, zoom and rotate around a cell division. In a third part the video shows the same embryo imaged at a later stage. (MP4 7137 kb)

Supplementary Video 5

Remote connection to an OpenSPIM microscope augmented with ClearVolume. The microscope is recording a time lapse of the development of a Drosophila melanogaster (histone-YFP) embryo (time point period: 5 min). We show it is possible to remotely access and visualize a 3D stack. (MP4 4406 kb)

Supplementary Video 6

Time-shift feature on an OpenSPIM microscope. The screen recording shows a time lapse recording of the development of a Drosophila melanogaster (Histone-YFP) embryo (acquisition time step: 5 min), and the replay using the time-shift feature of ClearVolume. (MP4 4840 kb)

Supplementary Video 7

Visualizing the 3D point spread function. Visualization and inspection of the 3D PSF of a light-sheet microscope using ClearVolume. (MP4 6912 kb)

Supplementary Video 8

Gaussian and Bessel beam visualization. Visualization and inspection of the full 3D structure of Gaussian and Bessel beams in a digital scanned light sheet microscope (DSLM). (MP4 8747 kb)

Supplementary Video 9

Live sharpness estimation. ClearVolume utilizing the Tenengrad image sharpness measure to provide audiovisual feedback on PSF quality. (MP4 4548 kb)

Supplementary Video 10

Live drift correction. ClearVolume visualizing sample drift inside the imaging volume using simulated drift data. (MP4 5757 kb)

Supplementary Video 11

Beyond light-sheet microscopes. ClearVolume visualizing live data acquired on a commercial Zeiss LSM 780 NLO confocal microscope via the pyclearvolume extension. (MP4 5611 kb)

Supplementary Video 12

Fiji interface. Demonstrating the integration of ClearVolume into Fiji. (MP4 10692 kb)

Supplementary Video 13

Remote connection inside Fiji. Demonstrating how to fetch remote data via ClearVolume for processing inside Fiji. (MP4 5405 kb)

Supplementary Video 14

KNIME interface. Demonstrating the integration of ClearVolume into KNIME. (MP4 6842 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Royer, L., Weigert, M., Günther, U. et al. ClearVolume: open-source live 3D visualization for light-sheet microscopy. Nat Methods 12, 480–481 (2015). https://doi.org/10.1038/nmeth.3372

Download citation

Further reading

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing