Imagining the future of bioimage analysis

Modern biological research increasingly relies on image data as a primary source of information in unraveling the cellular and molecular mechanisms of life. The quantity and complexity of the data generated by state-of-the-art microscopes preclude visual or manual analysis and require advanced computational methods to fully explore the wealth of information. In addition to making bioimage analysis more efficient, objective, and reproducible, the use of computers improves the accuracy and sensitivity of the analyses and helps to reveal subtleties that may be unnoticeable to the human eye. Many methods and software tools have already been developed to this end, but there is still a long way to go before biologists can blindly trust automated measurements. Here, we summarize the current state of the art in bioimage analysis and provide a perspective on likely future developments.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: Common steps in bioimage analysis.
Figure 2: Examples of bioimage analysis in various applications.

References

  1. 1

    Frisby, J.P. & Stone, J.V. Seeing: The Computational Approach to Biological Vision (The MIT Press, Cambridge, MA, USA, 2010).

    Google Scholar 

  2. 2

    Ji, N., Shroff, H., Zhong, H. & Betzig, E. Advances in the speed and resolution of light microscopy. Curr. Opin. Neurobiol. 18, 605–616 (2008).

    CAS  Article  Google Scholar 

  3. 3

    Prewitt, J.M.S. & Mendelsohn, M.L. The analysis of cell images. Ann. NY Acad. Sci. 128, 1035–1053 (1966).

    CAS  Article  Google Scholar 

  4. 4

    Peng, H. et al. Bioimage informatics for big data. Adv. Anat. Embryol. Cell Biol. 219, 263–272 (2016).

    Article  Google Scholar 

  5. 5

    Eliceiri, K.W. et al. Biological imaging software tools. Nat. Methods 9, 697–710 (2012).

    CAS  Article  Google Scholar 

  6. 6

    Szeliski, R. Computer Vision: Algorithms and Applications (Springer, London, UK, 2011).

    Google Scholar 

  7. 7

    Sarder, P. & Nehorai, A. Deconvolution methods for 3-D fluorescence microscopy images. IEEE Signal Process. Mag. 23, 32–45 (2006).

    Article  Google Scholar 

  8. 8

    Qu, L., Long, F. & Peng, H. 3-D registration of biological images and models: registration of microscopic images and its uses in segmentation and annotation. IEEE Signal Process. Mag. 32, 70–77 (2015).

    Article  Google Scholar 

  9. 9

    Wu, Q., Merchant, F.A. & Castleman, K.R. Microscope Image Processing (Academic Press, Burlington, MA, USA, 2008).

    Google Scholar 

  10. 10

    Meijering, E. Cell segmentation: 50 years down the road. IEEE Signal Process. Mag. 29, 140–145 (2012).

    Article  Google Scholar 

  11. 11

    Meijering, E., Dzyubachyk, O., Smal, I. & van Cappellen, W.A. Tracking in cell and developmental biology. Semin. Cell Dev. Biol. 20, 894–902 (2009).

    Article  Google Scholar 

  12. 12

    Pincus, Z. & Theriot, J.A. Comparison of quantitative methods for cell-shape analysis. J. Microsc. 227, 140–156 (2007).

    CAS  Article  Google Scholar 

  13. 13

    Depeursinge, A., Foncubierta-Rodriguez, A., Van De Ville, D. & Müller, H. Three-dimensional solid texture analysis in biomedical imaging: review and opportunities. Med. Image Anal. 18, 176–196 (2014).

    Article  Google Scholar 

  14. 14

    Shamir, L., Delaney, J.D., Orlov, N., Eckley, D.M. & Goldberg, I.G. Pattern recognition software and techniques for biological image analysis. PLoS Comput. Biol. 6, e1000974 (2010).

    Article  Google Scholar 

  15. 15

    Walter, T. et al. Visualization of image data from cells to organisms. Nat. Methods 7 (Suppl.), S26–S41 (2010).

    CAS  Article  Google Scholar 

  16. 16

    Buck, T.E., Li, J., Rohde, G.K. & Murphy, R.F. Toward the virtual cell: automated approaches to building models of subcellular organization “learned” from microscopy images. BioEssays 34, 791–799 (2012).

    CAS  Article  Google Scholar 

  17. 17

    Neumann, B. et al. Phenotypic profiling of the human genome by time-lapse microscopy reveals cell division genes. Nature 464, 721–727 (2010).

    CAS  Article  Google Scholar 

  18. 18

    Takemura, S.Y. et al. A visual motion detection circuit suggested by Drosophila connectomics. Nature 500, 175–181 (2013).

    CAS  Article  Google Scholar 

  19. 19

    Ponti, A., Machacek, M., Gupton, S.L., Waterman-Storer, C.M. & Danuser, G. Two distinct actin networks drive the protrusion of migrating cells. Science 305, 1782–1786 (2004).

    CAS  Article  Google Scholar 

  20. 20

    Spanjaard, E. et al. Quantitative imaging of focal adhesion dynamics and their regulation by HGF and Rap1 signaling. Exp. Cell Res. 330, 382–397 (2015).

    CAS  Article  Google Scholar 

  21. 21

    Danuser, G. Computer vision in cell biology. Cell 147, 973–978 (2011).

    CAS  Article  Google Scholar 

  22. 22

    Cardona, A. & Tomancak, P. Current challenges in open-source bioimage informatics. Nat. Methods 9, 661–665 (2012).

    CAS  Article  Google Scholar 

  23. 23

    Prins, P. et al. Toward effective software solutions for big biology. Nat. Biotechnol. 33, 686–687 (2015).

    CAS  Article  Google Scholar 

  24. 24

    Carpenter, A.E., Kamentsky, L. & Eliceiri, K.W. A call for bioimaging software usability. Nat. Methods 9, 666–670 (2012).

    CAS  Article  Google Scholar 

  25. 25

    Marr, D. Vision: A Computational Investigation into the Human Representation and Processing of Visual Information (The MIT Press, Cambridge, MA, USA, 2010).

    Google Scholar 

  26. 26

    Ter Haar Romeny, B.M. Front-End Vision and Multi-Scale Image Analysis (Springer, Berlin, Germany, 2003).

    Google Scholar 

  27. 27

    Pridmore, T.P., French, A.P. & Pound, M.P. What lies beneath: underlying assumptions in bioimage analysis. Trends Plant Sci. 17, 688–692 (2012).

    CAS  Article  Google Scholar 

  28. 28

    Dudai, Y. How big is human memory, or on being just useful enough. Learn. Mem. 3, 341–365 (1997).

    CAS  Article  Google Scholar 

  29. 29

    Brady, T.F., Konkle, T. & Alvarez, G.A. A review of visual memory capacity: beyond individual items and toward structured representations. J. Vis. 11, 4 (2011).

    Article  Google Scholar 

  30. 30

    Bishop, C.M. Pattern Recognition and Machine Learning (Springer, New York, NY, USA, 2006).

    Google Scholar 

  31. 31

    Sommer, C. & Gerlich, D.W. Machine learning in cell biology - teaching computers to recognize phenotypes. J. Cell Sci. 126, 5529–5539 (2013).

    CAS  Article  Google Scholar 

  32. 32

    LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).

    CAS  Article  Google Scholar 

  33. 33

    Price, K. Anything you can do, I can do better (no you can't). Comput. Vis. Graph. Image Process. 36, 387–391 (1986).

    Article  Google Scholar 

  34. 34

    Gillette, T.A., Brown, K.M., Svoboda, K., Liu, Y. & Ascoli, G.A. DIADEMChallenge.Org: a compendium of resources fostering the continuous development of automated neuronal reconstruction. Neuroinform. 9, 303–304 (2011).

    Article  Google Scholar 

  35. 35

    Chenouard, N. et al. Objective comparison of particle tracking methods. Nat. Methods 11, 281–289 (2014).

    CAS  Article  Google Scholar 

  36. 36

    Maška, M. et al. A benchmark for comparison of cell tracking algorithms. Bioinformatics 30, 1609–1617 (2014).

    Article  Google Scholar 

  37. 37

    Sage, D. et al. Quantitative evaluation of software packages for single-molecule localization microscopy. Nat. Methods 12, 717–724 (2015).

    CAS  Article  Google Scholar 

  38. 38

    Roux, L. et al. Mitosis detection in breast cancer histological images: An ICPR 2012 contest. J. Pathol. Inform. 4, 8 (2013).

    Article  Google Scholar 

  39. 39

    Veta, M. et al. Assessment of algorithms for mitosis detection in breast cancer histopathology images. Med. Image Anal. 20, 237–248 (2015).

    Article  Google Scholar 

  40. 40

    Peng, H. et al. BigNeuron: large-scale 3D neuron reconstruction from optical microscopy images. Neuron 87, 252–256 (2015).

    CAS  Article  Google Scholar 

  41. 41

    Ljosa, V., Sokolnicki, K.L. & Carpenter, A.E. Annotated high-throughput microscopy image sets for validation. Nat. Methods 9, 637 (2012).

    CAS  Article  Google Scholar 

  42. 42

    Ince, D.C., Hatton, L. & Graham-Cumming, J. The case for open computer programs. Nature 482, 485–488 (2012).

    CAS  Article  Google Scholar 

  43. 43

    Scherf, N. & Huisken, J. The smart and gentle microscope. Nat. Biotechnol. 33, 815–818 (2015).

    CAS  Article  Google Scholar 

  44. 44

    Long, B., Li, L., Knoblich, U., Zeng, H. & Peng, H. 3D image-guided automatic pipette positioning for single cell experiments in vivo. Sci. Rep. 5, 18426 (2015).

    CAS  Article  Google Scholar 

  45. 45

    Fernández-González, R., Muñoz-Barrutia, A., Barcellos-Hoff, M.H. & Ortiz- de-Solorzano, C. Quantitative in vivo microscopy: the return from the 'omics'. Curr. Opin. Biotechnol. 17, 501–510 (2006).

    Article  Google Scholar 

  46. 46

    Swedlow, J.R., Zanetti, G. & Best, C. Channeling the data deluge. Nat. Methods 8, 463–465 (2011).

    CAS  Article  Google Scholar 

  47. 47

    Lahat, D., Adali, T. & Jutten, C. Multimodal data fusion: an overview of methods, challenges, and prospects. Proc. IEEE 103, 1449–1477 (2015).

    Article  Google Scholar 

  48. 48

    Ljosa, V. et al. Comparison of methods for image-based profiling of cellular morphological responses to small-molecule treatment. J. Biomol. Screen. 18, 1321–1329 (2013).

    CAS  Article  Google Scholar 

  49. 49

    Peng, H., Ruan, Z., Long, F., Simpson, J.H. & Myers, E.W. V3D enables real-time 3D visualization and quantitative analysis of large-scale biological image data sets. Nat. Biotechnol. 28, 348–353 (2010).

    CAS  Article  Google Scholar 

  50. 50

    Kreshuk, A., Koethe, U., Pax, E., Bock, D.D. & Hamprecht, F.A. Automated detection of synapses in serial section transmission electron microscopy image stacks. PLoS One 9, e87351 (2014).

    Article  Google Scholar 

Download references

Acknowledgements

The authors thank their group members and collaborators for insightful discussions that have helped shape their thoughts and research in bioimage analysis over the years. They also thankfully acknowledge support from Erasmus University Medical Center (E.M.), the National Science Foundation (CAREER DBI 1148823 to A.E.C.), the Allen Institute for Brain Science and the Janelia Research Campus of Howard Hughes Medical Institute (H.P.), the German Research Foundation (DFG SFB 1129/1134) and a Weston Visiting Professorship (F.A.H.), and Agence Nationale de la Recherche (ANR-10-INBS-04-06-France- BioImaging) and Institut Pasteur (J.C.O.M.). Raw data for the bioimage analysis examples shown are courtesy of Graham Knott (Fig. 2c) and Anna Akhmanova (Fig. 2d).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Erik Meijering.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Meijering, E., Carpenter, A., Peng, H. et al. Imagining the future of bioimage analysis. Nat Biotechnol 34, 1250–1255 (2016). https://doi.org/10.1038/nbt.3722

Download citation

Further reading

Search

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing