Efficient Bayesian-based multiview deconvolution

Abstract

Light-sheet fluorescence microscopy is able to image large specimens with high resolution by capturing the samples from multiple angles. Multiview deconvolution can substantially improve the resolution and contrast of the images, but its application has been limited owing to the large size of the data sets. Here we present a Bayesian-based derivation of multiview deconvolution that drastically improves the convergence time, and we provide a fast implementation using graphics hardware.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: Principles and performance.
Figure 2: Deconvolution of simulated 3D multiview data.
Figure 3: Application to biological data.

References

  1. 1

    Huisken, J., Swoger, J., Del Bene, F., Wittbrodt, J. & Stelzer, E.H.K. Science 305, 1007–1009 (2004).

    CAS  Article  Google Scholar 

  2. 2

    Keller, P.J., Schmidt, A.D., Wittbrodt, J. & Stelzer, E.H.K. Science 322, 1065–1069 (2008).

    CAS  Article  Google Scholar 

  3. 3

    Truong, T.V., Supatto, W., Koos, D.S., Choi, J.M. & Fraser, S.E. Nat. Methods 8, 757–760 (2011).

    CAS  Article  Google Scholar 

  4. 4

    Swoger, J., Verveer, P., Greger, K., Huisken, J. & Stelzer, E.H.K. Opt. Express 15, 8029–8042 (2007).

    Article  Google Scholar 

  5. 5

    Shepp, L.A. & Vardi, Y. IEEE Trans. Med. Imaging 1, 113–122 (1982).

    CAS  Article  Google Scholar 

  6. 6

    Hudson, H.M. & Larkin, R.S. IEEE Trans. Med. Imaging 13, 601–609 (1994).

    CAS  Article  Google Scholar 

  7. 7

    Verveer, P.J. et al. Nat. Methods 4, 311–313 (2007).

    CAS  Article  Google Scholar 

  8. 8

    Bonettini, S., Zanella, R. & Zanni, L. Inverse Probl. 25, 015002 (2009).

    Article  Google Scholar 

  9. 9

    Krzic, U. Multiple-View Microscopy with Light-Sheet Based Fluorescent Microscope. PhD thesis, Univ. Heidelberg (2009).

  10. 10

    Temerinac-Ott, M. et al. IEEE Trans. Image Process. 21, 1863–1873 (2012).

    Article  Google Scholar 

  11. 11

    Richardson, W.H. J. Opt. Soc. Am. 62, 55–59 (1972).

    Article  Google Scholar 

  12. 12

    Lucy, L.B. Astron. J. 79, 745–754 (1974).

    Article  Google Scholar 

  13. 13

    Dempster, A.P., Laird, N.M. & Rubin, D.B. J. R. Stat. Soc. Series B Stat. Methodol. 39, 1–38 (1977).

    Google Scholar 

  14. 14

    Preibisch, S., Saalfeld, S., Schindelin, J. & Tomancak, P. Nat. Methods 7, 418–419 (2010).

    CAS  Article  Google Scholar 

  15. 15

    Tikhonov, A.N. & Arsenin, V.Y. Solutions of Ill-Posed Problems (Winston, 1977).

  16. 16

    Long, F., Peng, H., Liu, X., Kim, S. & Myers, E. Nat. Methods 6, 667–672 (2009).

    CAS  Article  Google Scholar 

  17. 17

    Pitrone, P.G. et al. Nat. Methods 10, 598–599 (2013).

    CAS  Article  Google Scholar 

  18. 18

    Keller, P.J. et al. Nat. Methods 7, 637–642 (2010).

    CAS  Article  Google Scholar 

  19. 19

    Schindelin, J. et al. Nat. Methods 9, 676–682 (2012).

    CAS  Article  Google Scholar 

  20. 20

    Pietzsch, T., Preibisch, S., Tomancak, P. & Saalfeld, S. Bioinformatics 28, 3009–3011 (2012).

    CAS  Article  Google Scholar 

  21. 21

    Uddin, M.S., Lee, H.K., Preibisch, S. & Tomancak, P. Microsc. Microanal. 17, 607–613 (2011).

    CAS  Article  Google Scholar 

Download references

Acknowledgements

We thank T. Pietzsch (Max Planck Institute of Molecular Cell Biology and Genetics (MPI-CBG)) for helpful discussions, proofreading and access to his unpublished software; N. Clack, F. Carrillo Oesterreich and H. Bowne-Anderson for discussions; N. Maghelli for two-photon imaging; P. Verveer (MPI Dortmund) for source code and helpful discussions; M. Weber for imaging the Drosophila time series; S. Jaensch for preparing the C. elegans embryo; J.K. Liu (Cornell University) for the LW698 strain; S. Saalfeld for help with 3D rendering; P.J. Keller for supporting F.A. and for the DSLM-SI data set; A. Cardona for access to his computer; and Carl Zeiss Microimaging for providing us with the SPIM prototype. S.P. was supported by MPI-CBG in P.T.'s lab, Howard Hughes Medical Institute (HHMI) in E.M.'s lab and the Human Frontier Science Program (HFSP) Postdoctoral Fellowship LT000783/2012 in R.H.S.'s lab, with additional support from US National Institutes of Health (NIH) GM57071. F.A. was supported by HHMI in P.J. Keller's lab. E.S. and M.S. were supported by MPI-CBG. R.H.S. was supported by NIH grants GM057071, EB013571 and NS083085. E.M. was supported by HHMI and MPI-CBG. P.T. was supported by The European Research Council Community's Seventh Framework Program (FP7/2007-2013) grant agreement 260746 and the HFSP Young Investigator grant RGY0093/2012. M.S., E.M. and P.T. were additionally supported by the Bundesministerium für Bildung und Forschung grant 031A099.

Author information

Affiliations

Authors

Contributions

S.P. and F.A. derived the equations for multiview deconvolution. S.P. implemented the software and performed all analysis, and F.A. implemented the GPU code. E.S. generated and imaged the H2Av-mRFPruby fly line. M.S. prepared, and M.S. and S.P. imaged, the C. elegans L1 sample. S.P. and P.T. conceived the idea and wrote the manuscript. R.H.S. provided support and encouragement, E.M. and P.T. supervised the project.

Corresponding authors

Correspondence to Stephan Preibisch or Pavel Tomancak.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–17, Supplementary Table 1 and Supplementary Notes 1–8 (PDF 12761 kb)

Visualization of the multiview deconvolution process on synthetic data

The movie illustrates the core idea underlying the multiview deconvolution process. The deconvolution is finished once each view is identical to the deconvolved image convolved with the respective PSF, i.e. the deconvolved image together with the PSF's completely explains each observed view. To achieve this, the deconvolution first computes the difference between the input image and the convolution of the current guess of the underlying image with the PSF. Based on this difference each view suggests an update that is then combined into one update of the deconvolved image. In this movie the iterations continue until a cross correlation of 0.99 is achieved. For this synthetic data picturing lines, 21944 iterations were required. (MOV 1264 kb)

Deconvolution of simulated dataset (lateral orientation, XY)

Shows all slices in lateral orientation of view 1 through the simulated ground truth, view 1, content-based fused image, MAPG deconvolution, Optimization II deconvolution and Optimization II deconvolution with Tikhonov regularization (lamdba = 0.004). (MOV 3200 kb)

Deconvolution of simulated dataset (orientation along rotation axis, YZ)

Shows all slices oriented along the rotation axis of the virtual microscope. Cut planes are shown through the simulated ground truth, view 1, content-based fused image, MAPG deconvolution, Optimization II deconvolution and Optimization II deconvolution with Tikhonov regularization (lamdba = 0.004). (MOV 3235 kb)

Fixed L1 C. elegans larvae orientation 1

Fixed C. elegans larvae in L1 stage expressing LMN-1::GFP (green) and stained with Hoechst (magenta) imaged on Zeiss Lightsheet Z1 (Carl Zeiss Microimaging) with a 20x/1.0 detection objective using four views (0, 45, 90, 135 degrees). The specimen is oriented so that the required bounding box is minimized. The top panel shows a stack of input view 0 close to its original orientation; the lower panel shows the corresponding deconvolved stack. (MOV 2035 kb)

Fixed L1 C. elegans larvae orientation 2

Fixed C. elegans larvae in L1 stage expressing LMN-1::GFP (green) and stained with Hoechst (magenta) imaged on Zeiss Lightsheet Z.1 (Carl Zeiss Microimaging) with a 20x/1.0 detection objective using four views (0, 45, 90, 135 degrees). The specimen is oriented so that the required bounding box is minimized. The top view pictures the stack of input view 0 in an almost axial orientation (orthogonal to Supplementary Video 4), the lower panel shows the corresponding deconvolved stack. (MOV 1983 kb)

Fixed L1 C. elegans larvae along rotation axis

Fixed C. elegans larvae in L1 stage expressing LMN-1::GFP (green) and stained with Hoechst (magenta) imaged on Zeiss Lightsheet Z1 (Carl Zeiss Microimaging) with a 20x/1.0 detection objective using four views (0, 45, 90, 135 degrees). The specimen is oriented so that the required bounding box is minimized. The top part pictures the stack of input view 0 approximately along the rotation axis of the microscope (orthogonal to Supplementary Video 4 & 5), the lower panel shows the corresponding deconvolved stack. Note that none of the input views offers high resolution in this orientation. (MOV 2821 kb)

Segmentation of fixed L1 C. elegans larvae

3D maximal projection of the manual segmentation of the deconvolved fixed C. elegans larvae in L1 stage expressing LMN-1::GFP and stained with Hoechst. 558 nuclei were manually identified using the ImgLib219 based viewing and annotation software developed by Tobias Pietzsch (unpublished software). (MOV 3565 kb)

Deconvolution of Drosophila embryogenesis time and series orientation 1

Drosophila melanogaster embryo expressing His-YFP in all nuclei imaged on a Zeiss SPIM prototype with a 20x/0.5NA detection objective using 7 views. The top panel shows a 3d rendering of the side view of the deconvolved volume, the lower panel the result of the content-based fusion. (MOV 6223 kb)

Deconvolution of Drosophila embryogenesis time and series orientation 2

Drosophila melanogaster embryo expressing His-YFP in all nuclei imaged on a Zeiss SPIM prototype with a 20x/0.5NA detection objective using 7 views. The top panel shows a 3d rendering of dorsal side of the deconvolved volume, the lower panel the result of the content-based fusion. (MOV 6006 kb)

Deconvolution of Drosophila embryogenesis time and series view along rotation axis

Drosophila melanogaster embryo expressing His-YFP in all nuclei imaged on a Zeiss SPIM prototype with a 20x/0.5NA detection objective using 7 views. The top panel shows a 3d rendering of the anterior part, the lower panel of the posterior part of the embryo. The left side pictures the deconvolved volume, on the right side the result of the content-based fusion. Note that these renderings of the sample reconstitute a view along the rotation axis of the microscope, i.e. no views were acquired by the microscope in this orientation and therefore the increase in resolution by the deconvolution is maximal. (MOV 6549 kb)

Deconvolution of Drosophila embryogenesis time and series acquired with Lightsheet Z1

Drosophila melanogaster embryo expressing Histone-H2Av-mRFPruby fusion in all cells was imaged on Zeiss Lightsheet Z1 (Carl Zeiss Microimaging) with a 20x/1.0 detection objective using dual sided illumination from six angles every 1.5 minutes for 17 hours (715 timepoints). The recording starts right before initiation of gastrulation movements and ends when the embryo hatches into a first instar larva and crawls out of the imaged volume. The data were registered using Fiji's bead based multiview SPIM registration plugin15. The two times downsampled registered data were fused into a single output image using content based fusion (top) and multiview deconvolution with 10 iterations and Tikhonov regularization parameter set to 0.0006 (bottom). The fused and deconvolved data were independently rendered using Fiji/ImgLib219 based cpu renderer (Stephan Saalfeld unpublished software). All processing was performed on a cluster computer using Fiji launched with custom shell scripts (http://fiji.sc/SPIM_Registration_on_cluster). The deconvolution took advantage of four nodes with NVIDIA Tesla graphics card and ran for the 715 time points for about 35 hours (i.e 4 more graphics cards would make the processing real time, 17 hours of processing for 17 hour acquisition). (MOV 33862 kb)

Supplementary Software 1

Software for the deconvolution of multiview microscopy data (Fiji plugin) (ZIP 361 kb)

Supplementary Software 2

Software simulating multiview selective plane microscopy data including signal attenuation, convolution, reduced axial sampling and Poisson noise. (ZIP 421 kb)

Supplementary Software 3

CUDA implementation of the Fourier convolution for fast processing (additional library for Fiji plugin) (ZIP 66 kb)

Source data

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Preibisch, S., Amat, F., Stamataki, E. et al. Efficient Bayesian-based multiview deconvolution. Nat Methods 11, 645–648 (2014). https://doi.org/10.1038/nmeth.2929

Download citation

Further reading