Biological imaging software tools

Journal name:
Nature Methods
Volume:
9,
Pages:
697–710
Year published:
DOI:
doi:10.1038/nmeth.2084
Published online
Corrected online
Corrected online

Abstract

Few technologies are more widespread in modern biological laboratories than imaging. Recent advances in optical technologies and instrumentation are providing hitherto unimagined capabilities. Almost all these advances have required the development of software to enable the acquisition, management, analysis and visualization of the imaging data. We review each computational step that biologists encounter when dealing with digital images, the inherent challenges and the overall status of available software for bioimage informatics, focusing on open-source options.

At a glance

Figures

  1. Overview of imaging workflow.
    Figure 1: Overview of imaging workflow.

    Modern bioimaging requires the use of software tools for most stages of the workflow.

  2. Image acquisition spans a range of complexity and variation.
    Figure 2: Image acquisition spans a range of complexity and variation.
  3. Options for image storage vary in complexity and size.
    Figure 3: Options for image storage vary in complexity and size.
  4. Image analysis and visualization span a range of complexity and variation.
    Figure 4: Image analysis and visualization span a range of complexity and variation.
  5. Screenshots illustrating the image-analysis steps starting from a multichannel, multiphoton time-lapse movie culminating in a bioinformatics profiling of the extracted spatiotemporal data, using the FARSIGHT toolkit.
    Figure 5: Screenshots illustrating the image-analysis steps starting from a multichannel, multiphoton time-lapse movie culminating in a bioinformatics profiling of the extracted spatiotemporal data, using the FARSIGHT toolkit.

    (a) This movie (courtesy of E. Robey, University of California Berkeley) recorded 3D movements of thymocytes in an ex vivo preparation of a live developing mouse thymus at 2-min intervals, with wild-type thymocytes displayed in cyan, F5 thymocytes in green, and dendritic cells in violet. The first step is cell segmentation, shown as an orthogonal (x, y, z and time (t)) view. Cells are delineated and identified with numbers that correspond to rows of a table of cell measurements (data not shown). (b,c) The cell-tracking results are displayed in a 'beads on strings' view, showing the 3D movement paths of cells for detecting anomalies (b), and a '3D kymograph view', showing the same movement paths overlaid on a spatiotemporal (x, y and t) projection for convenience of assessing cell-tracking accuracy (c). (d) Histogram of cell-morphological measurements (size). (e) Scatter plots provide a visual cytometric summary of pairs of measurements. (f) Coifman bi-cluster plots organize the cell data into groups based on the cytometric data. (g) Histogram of cell tracking measurements (track tortuosity). (h) Scatter plot view of pairs of cell-track measurements. (i) Coifman bi-cluster plot organizing the cell tracks into groups based on the track-based measurements. Bi-cluster modules are courtesy of R. Coifman (Yale University) and L. Carin (Duke University).

  6. Bioimaging libraries and toolkits are available to cover a range of functionalities.
    Figure 6: Bioimaging libraries and toolkits are available to cover a range of functionalities.
  7. Machine learning.
    Figure 7: Machine learning.

    Application areas in which machine learning is used in bioimaging.

  8. Workflow systems.
    Figure 8: Workflow systems.

    Benefits of using a workflow system.

Change history

Corrected online 20 July 2012
In the version of this article initially published, Nico Stuurman's last name was incorrect. The error has been corrected in the HTML and PDF versions of the article.
Corrected online 29 August 2012
In the version of this article initially published, the disclaimer was omitted. The error has been corrected in the HTML and PDF versions of the article.

References

  1. Peng, H. Bioimage informatics: a new area of engineering biology. Bioinformatics 24, 18271836 (2008).
  2. Gustafsson, M.G. Nonlinear structured-illumination microscopy: wide-field fluorescence imaging with theoretically unlimited resolution. Proc. Natl. Acad. Sci. USA 102, 1308113086 (2005).
  3. Huang, B., Wang, W., Bates, M. & Zhuang, X. Three-dimensional super-resolution imaging by stochastic optical reconstruction microscopy. Science 319, 810813 (2008).
  4. Hess, S.T., Girirajan, T.P. & Mason, M.D. Ultra-high resolution imaging by fluorescence photoactivation localization microscopy. Biophys. J. 91, 42584272 (2006).
  5. Jones, S.A., Shim, S.H., He, J. & Zhuang, X. Fast, three-dimensional super-resolution imaging of live cells. Nat. Methods 8, 499508 (2011).
  6. Planchon, T.A. et al. Rapid three-dimensional isotropic imaging of living cells using Bessel beam plane illumination. Nat. Methods 8, 417423 (2011).
  7. Edelstein, A., Amodaj, N., Hoover, K., Vale, R. & Stuurman, N. Computer control of microscopes using μManager. Curr. Protoc. Mol. Biol. 92, 14.20.1114.20.17 (2010).
  8. Lin, H.P., Vincenz, C., Eliceiri, K.W., Kerppola, T.K. & Ogle, B.M. Bimolecular fluorescence complementation analysis of eukaryotic fusion products. Biol. Cell 102, 525537 (2010).
  9. Pologruto, T.A., Sabatini, B.L. & Svoboda, K. ScanImage: flexible software for operating laser scanning microscopes. Biomed. Eng. Online 2, 13 (2003).
  10. Conrad, C. et al. Micropilot: automation of fluorescence microscopy-based imaging for systems biology. Nat. Methods 8, 246249 (2011).
  11. Allan, C. et al. OMERO: flexible, model-driven data management for experimental biology. Nat. Methods 9, 245253 (2012).
  12. Kvilekval, K., Fedorov, D., Obara, B., Singh, A. & Manjunath, B.S. Bisque: a platform for bioimage analysis and management. Bioinformatics 26, 544552 (2010).
  13. Wu, L., Faloutsos, C., Sycara, K.P. & Payne, T.R. Feedback adaptive loop for content-based retrieval. in Proceedings of the 26th International Conference on Very Large Data Bases (Morgan Kaufmann Publishers Inc., 2000).
  14. Goff, S.A. et al. The iPlant Collaborative: cyberinfrastructure for plant biology. Frontiers in Plant Science 2, 34 (2011).
  15. Glory, E. & Murphy, R.F. Automated subcellular location determination and high-throughput microscopy. Dev. Cell 12, 716 (2007).
  16. Ljosa, V. & Carpenter, A.E. Introduction to the quantitative analysis of two-dimensional fluorescence microscopy images for cell-based screening. PLoS Comput. Biol. 5, e1000603 (2009).
  17. Lakowicz, J.R. Principals of Fluorescence Spectroscopy. (Academic Press, New York, 1999).
  18. Kankaanpää, P. et al. BioImageXD: an open, general-purpose and high-throughput image-processing platform. Nat. Methods 9, 683689 (2012).
  19. de Chaumont, F. et al. Icy: an open bioimage informatics platform for extended reproducible research. Nat. Methods 9, 690696 (2012).
  20. Schindelin, J. et al. Fiji: an open-source platform for biological-image analysis. Nat. Methods 9, 676682 (2012).
  21. Peng, H., Ruan, Z., Long, F., Simpson, J.H. & Myers, E.W. V3D enables real-time 3D visualization and quantitative analysis of large-scale biological image data sets. Nat. Biotechnol. 28, 348353 (2010).
  22. Carpenter, A.E. et al. CellProfiler: image analysis software for identifying and quantifying cell phenotypes. Genome Biol. 7, R100 (2006).
  23. Fiala, J.C. Reconstruct: a free editor for serial section microscopy. J. Microsc. 218, 5261 (2005).
  24. Feng, D. et al. Stepping into the third dimension. J. Neurosci. 27, 1275712760 (2007).
  25. Rosset, A., Spadola, L., Ratib, O. & Osiri, X. An open-source software for navigating in multidimensional DICOM images. J. Digit. Imaging 17, 205216 (2004).
  26. Kremer, J.R., Mastronarde, D.N. & McIntosh, J.R. Computer visualization of three-dimensional image data using IMOD. J. Struct. Biol. 116, 7176 (1996).
  27. Collins, T.J. ImageJ for microscopy. Biotechniques 43, 2530 (2007).
  28. Abramoff, M., Magalhaes, P. & Ram, S. Image processing with ImageJ. Biophotonics International 11, 3642 (2004).
  29. Schneider, C.A., Rasband, W.S. & Eliceiri, K.W. NIH Image to ImageJ: 25 years of image analysis. Nat. Methods 9, 671675 (2012).
  30. Kamentsky, L. et al. Improved structure, function and compatibility for CellProfiler: modular high-throughput image analysis software. Bioinformatics 27, 11791180 (2011).
  31. Preibisch, S., Saalfeld, S., Schindelin, J. & Tomancak, P. Software for bead-based registration of selective plane illumination microscopy data. Nat. Methods 7, 418419 (2010).
  32. Tsai, C.L. et al. Robust, globally consistent and fully automatic multi-image registration and montage synthesis for 3-D multi-channel images. J. Microsc. 243, 154171 (2011).
  33. Preibisch, S., Saalfeld, S. & Tomančák, P. Globally optimal stitching of tiled 3D microscopic image acquisitions. Bioinformatics 25, 14631465 (2009).
  34. Saalfeld, S., Fetter, R., Cardona, R. & Tomancak, P. Elastic volume reconstruction from series of ultrathin microscopy sections. Nat. Methods 9, 717720 (2012).
  35. Walter, T. et al. Visualization of image data from cells to organisms. Nat. Methods 7, S26S41 (2010).
  36. Saalfeld, S., Cardona, A., Hartenstein, V. & Tomanččák, P. CATMAID: collaborative annotation toolkit for massive amounts of image data. Bioinformatics 25, 19841986 (2009).
  37. Qu, L. et al. Simultaneous recognition and segmentation of cells: application in C. elegans. Bioinformatics 27, 28952902 (2011).
  38. Long, F., Peng, H., Liu, X., Kim, S.K. & Myers, E. A 3D digital atlas of C. elegans and its application to single-cell analyses. Nat. Methods 6, 667672 (2009).
  39. Pau, G., Fuchs, F., Sklyar, O., Boutros, M. & Huber, W. EBImage–an R package for image processing with applications to cellular phenotypes. Bioinformatics 26, 979981 (2010).
  40. Shamir, L., Delaney, J.D., Orlov, N., Eckley, D.M. & Goldberg, I.G. Pattern recognition software and techniques for biological image analysis. PLoS Comput. Biol. 6, e1000974 (2010).
  41. Murphy, R.F. An active role for machine learning in drug development. Nat. Chem. Biol. 7, 327330 (2011).
  42. Murphy, R.F., Velliste, M. & Porreca, G. Robust numerical features for description and classification of subcellular location patterns in fluorescence microscope images. J. VLSI Signal Process. 35, 311321 (2003).
  43. Nattkemper, T.W., Twellmann, T., Ritter, H. & Schubert, W. Human vs machine: evaluation of fluorescence micrographs. Comput. Biol. Med. 33, 3143 (2003).
  44. Johnston, J., Iser, W.B., Chow, D.K., Goldberg, I.G. & Wolkow, C.A. Quantitative image analysis reveals distinct structural transitions during aging in Caenorhabditis elegans tissues. PLoS ONE 3, e2821 (2008).
  45. Huang, K. & Murphy, R.F. From quantitative microscopy to automated image understanding. J. Biomed. Opt. 9, 893912 (2004).
  46. Shamir, L. et al. Wndchrm – an open source utility for biological image analysis. Source Code Biol. Med. 3, 13 (2008).
  47. Loo, L.H., Wu, L.F. & Altschuler, S.J. Image-based multivariate profiling of drug responses from single cells. Nat. Methods 4, 445453 (2007).
  48. Perlman, Z.E. et al. Multidimensional drug profiling by automated microscopy. Science 306, 11941198 (2004).
  49. Chen, X. & Murphy, R.F. Objective clustering of proteins based on subcellular location patterns. J. Biomed. Biotechnol. 2005, 8795 (2005).
  50. Jones, T.R. et al. Scoring diverse cellular morphologies in image-based screens with iterative feedback and machine learning. Proc. Natl. Acad. Sci. USA 106, 18261831 (2009).
  51. Jackson, C., Glory-Afshar, E., Murphy, R.F. & Kovacevic, J. Model building and intelligent acquisition with application to protein subcellular location classification. Bioinformatics 27, 18541859 (2011).
  52. Peng, T. et al. Determining the distribution of probes between different subcellular locations through automated unmixing of subcellular patterns. Proc. Natl. Acad. Sci. USA 107, 29442949 (2010).
  53. Coelho, L.P., Peng, T. & Murphy, R.F. Quantifying the distribution of probes between subcellular locations using unsupervised pattern unmixing. Bioinformatics 26, i7i12 (2010).
  54. Carpenter, A.E., Kamentsky, L. & Eliceiri, K.W. A call for bioimaging software usability. Nat. Methods 9, 666670 (2012).
  55. Cardona, A. & Tomancak, P. Current challenges in open-source bioimage informatics. Nat. Methods 9, 661665 (2012).
  56. Nielsen, M. Reinventing Discovery: The New Era of Networked Science. (Princeton University Press, 2011).
  57. Linkert, M. et al. Metadata matters: access to image data in the real world. J. Cell Biol. 189, 777782 (2010).
  58. Larson, S.D. & Martone, M.E. Ontologies for neuroscience: what are they and what are they good for? Front. Neurosci. 3, 6067 (2009).
  59. Plant, A.L., Elliott, J.T. & Bhat, T.N. New concepts for building vocabulary for cell image ontologies. BMC Bioinformatics 12, 487 (2011).
  60. Swedlow, J.R. Finding an image in a haystack: the case for public image repositories. Nat. Cell Biol. 13, 183 (2011).

Download references

Author information

Affiliations

  1. Laboratory for Optical and Computational Instrumentation, University of Wisconsin at Madison, Madison, Wisconsin, USA.

    • Kevin W Eliceiri
  2. Department of Computer and Information Science, Universität Konstanz, Konstanz, Germany.

    • Michael R Berthold
  3. National Institute on Aging, National Institutes of Health, Baltimore, Maryland, USA.

    • Ilya G Goldberg
  4. Kitware Inc., New York, New York, USA.

    • Luis Ibáñez
  5. Department of Electrical and Computer Engineering, Center for Bio-image Informatics, University of California Santa Barbara, Santa Barbara, California, USA.

    • B S Manjunath
  6. National Center for Microscopy and Imaging Research, University of California San Diego, La Jolla, California, USA.

    • Maryann E Martone
  7. Lane Center for Computational Biology, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA.

    • Robert F Murphy
  8. Janelia Farm Research Campus, Howard Hughes Medical Institute, Ashburn, Virginia, USA.

    • Hanchuan Peng
  9. Biochemical Science Division, National Institute of Standards and Technology, Gaithersburg, Maryland, USA.

    • Anne L Plant
  10. Department of Electrical and Computer Engineering, University of Houston, Houston, Texas, USA.

    • Badrinath Roysam
  11. Department of Cellular and Molecular Pharmacology, University of California, San Francisco, San Francisco, California, USA.

    • Nico Stuurman
  12. Wellcome Trust Centre for Gene Regulation and Expression, University of Dundee, Dundee, UK.

    • Jason R Swedlow
  13. Max Planck Institute of Molecular Cell Biology and Genetics, Dresden, Germany.

    • Pavel Tomancak
  14. Imaging Platform, Broad Institute of Harvard and the Massachusetts Institute of Technology, Cambridge, Massachusetts, USA.

    • Anne E Carpenter

Competing financial interests

J.R.S. is affiliated with Glencoe Software, Inc., a company that contributes to OMERO. M.R.B. is co-founder and co-owner of KNIME.com AG, a company that contributes to the development of the KNIME platform.

Corresponding authors

Correspondence to:

Author details

Additional data