Modern biological research increasingly relies on image data as a primary source of information in unraveling the cellular and molecular mechanisms of life. The quantity and complexity of the data generated by state-of-the-art microscopes preclude visual or manual analysis and require advanced computational methods to fully explore the wealth of information. In addition to making bioimage analysis more efficient, objective, and reproducible, the use of computers improves the accuracy and sensitivity of the analyses and helps to reveal subtleties that may be unnoticeable to the human eye. Many methods and software tools have already been developed to this end, but there is still a long way to go before biologists can blindly trust automated measurements. Here, we summarize the current state of the art in bioimage analysis and provide a perspective on likely future developments.
Your institute does not have access to this article
Open Access articles citing this article.
Scientific Data Open Access 11 August 2020
Scientific Reports Open Access 24 September 2019
BMC Bioinformatics Open Access 14 September 2019
Subscribe to Journal
Get full journal access for 1 year
only $8.25 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Get time limited or full article access on ReadCube.
All prices are NET prices.
Frisby, J.P. & Stone, J.V. Seeing: The Computational Approach to Biological Vision (The MIT Press, Cambridge, MA, USA, 2010).
Ji, N., Shroff, H., Zhong, H. & Betzig, E. Advances in the speed and resolution of light microscopy. Curr. Opin. Neurobiol. 18, 605–616 (2008).
Prewitt, J.M.S. & Mendelsohn, M.L. The analysis of cell images. Ann. NY Acad. Sci. 128, 1035–1053 (1966).
Peng, H. et al. Bioimage informatics for big data. Adv. Anat. Embryol. Cell Biol. 219, 263–272 (2016).
Eliceiri, K.W. et al. Biological imaging software tools. Nat. Methods 9, 697–710 (2012).
Szeliski, R. Computer Vision: Algorithms and Applications (Springer, London, UK, 2011).
Sarder, P. & Nehorai, A. Deconvolution methods for 3-D fluorescence microscopy images. IEEE Signal Process. Mag. 23, 32–45 (2006).
Qu, L., Long, F. & Peng, H. 3-D registration of biological images and models: registration of microscopic images and its uses in segmentation and annotation. IEEE Signal Process. Mag. 32, 70–77 (2015).
Wu, Q., Merchant, F.A. & Castleman, K.R. Microscope Image Processing (Academic Press, Burlington, MA, USA, 2008).
Meijering, E. Cell segmentation: 50 years down the road. IEEE Signal Process. Mag. 29, 140–145 (2012).
Meijering, E., Dzyubachyk, O., Smal, I. & van Cappellen, W.A. Tracking in cell and developmental biology. Semin. Cell Dev. Biol. 20, 894–902 (2009).
Pincus, Z. & Theriot, J.A. Comparison of quantitative methods for cell-shape analysis. J. Microsc. 227, 140–156 (2007).
Depeursinge, A., Foncubierta-Rodriguez, A., Van De Ville, D. & Müller, H. Three-dimensional solid texture analysis in biomedical imaging: review and opportunities. Med. Image Anal. 18, 176–196 (2014).
Shamir, L., Delaney, J.D., Orlov, N., Eckley, D.M. & Goldberg, I.G. Pattern recognition software and techniques for biological image analysis. PLoS Comput. Biol. 6, e1000974 (2010).
Walter, T. et al. Visualization of image data from cells to organisms. Nat. Methods 7 (Suppl.), S26–S41 (2010).
Buck, T.E., Li, J., Rohde, G.K. & Murphy, R.F. Toward the virtual cell: automated approaches to building models of subcellular organization “learned” from microscopy images. BioEssays 34, 791–799 (2012).
Neumann, B. et al. Phenotypic profiling of the human genome by time-lapse microscopy reveals cell division genes. Nature 464, 721–727 (2010).
Takemura, S.Y. et al. A visual motion detection circuit suggested by Drosophila connectomics. Nature 500, 175–181 (2013).
Ponti, A., Machacek, M., Gupton, S.L., Waterman-Storer, C.M. & Danuser, G. Two distinct actin networks drive the protrusion of migrating cells. Science 305, 1782–1786 (2004).
Spanjaard, E. et al. Quantitative imaging of focal adhesion dynamics and their regulation by HGF and Rap1 signaling. Exp. Cell Res. 330, 382–397 (2015).
Danuser, G. Computer vision in cell biology. Cell 147, 973–978 (2011).
Cardona, A. & Tomancak, P. Current challenges in open-source bioimage informatics. Nat. Methods 9, 661–665 (2012).
Prins, P. et al. Toward effective software solutions for big biology. Nat. Biotechnol. 33, 686–687 (2015).
Carpenter, A.E., Kamentsky, L. & Eliceiri, K.W. A call for bioimaging software usability. Nat. Methods 9, 666–670 (2012).
Marr, D. Vision: A Computational Investigation into the Human Representation and Processing of Visual Information (The MIT Press, Cambridge, MA, USA, 2010).
Ter Haar Romeny, B.M. Front-End Vision and Multi-Scale Image Analysis (Springer, Berlin, Germany, 2003).
Pridmore, T.P., French, A.P. & Pound, M.P. What lies beneath: underlying assumptions in bioimage analysis. Trends Plant Sci. 17, 688–692 (2012).
Dudai, Y. How big is human memory, or on being just useful enough. Learn. Mem. 3, 341–365 (1997).
Brady, T.F., Konkle, T. & Alvarez, G.A. A review of visual memory capacity: beyond individual items and toward structured representations. J. Vis. 11, 4 (2011).
Bishop, C.M. Pattern Recognition and Machine Learning (Springer, New York, NY, USA, 2006).
Sommer, C. & Gerlich, D.W. Machine learning in cell biology - teaching computers to recognize phenotypes. J. Cell Sci. 126, 5529–5539 (2013).
LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).
Price, K. Anything you can do, I can do better (no you can't). Comput. Vis. Graph. Image Process. 36, 387–391 (1986).
Gillette, T.A., Brown, K.M., Svoboda, K., Liu, Y. & Ascoli, G.A. DIADEMChallenge.Org: a compendium of resources fostering the continuous development of automated neuronal reconstruction. Neuroinform. 9, 303–304 (2011).
Chenouard, N. et al. Objective comparison of particle tracking methods. Nat. Methods 11, 281–289 (2014).
Maška, M. et al. A benchmark for comparison of cell tracking algorithms. Bioinformatics 30, 1609–1617 (2014).
Sage, D. et al. Quantitative evaluation of software packages for single-molecule localization microscopy. Nat. Methods 12, 717–724 (2015).
Roux, L. et al. Mitosis detection in breast cancer histological images: An ICPR 2012 contest. J. Pathol. Inform. 4, 8 (2013).
Veta, M. et al. Assessment of algorithms for mitosis detection in breast cancer histopathology images. Med. Image Anal. 20, 237–248 (2015).
Peng, H. et al. BigNeuron: large-scale 3D neuron reconstruction from optical microscopy images. Neuron 87, 252–256 (2015).
Ljosa, V., Sokolnicki, K.L. & Carpenter, A.E. Annotated high-throughput microscopy image sets for validation. Nat. Methods 9, 637 (2012).
Ince, D.C., Hatton, L. & Graham-Cumming, J. The case for open computer programs. Nature 482, 485–488 (2012).
Scherf, N. & Huisken, J. The smart and gentle microscope. Nat. Biotechnol. 33, 815–818 (2015).
Long, B., Li, L., Knoblich, U., Zeng, H. & Peng, H. 3D image-guided automatic pipette positioning for single cell experiments in vivo. Sci. Rep. 5, 18426 (2015).
Fernández-González, R., Muñoz-Barrutia, A., Barcellos-Hoff, M.H. & Ortiz- de-Solorzano, C. Quantitative in vivo microscopy: the return from the 'omics'. Curr. Opin. Biotechnol. 17, 501–510 (2006).
Swedlow, J.R., Zanetti, G. & Best, C. Channeling the data deluge. Nat. Methods 8, 463–465 (2011).
Lahat, D., Adali, T. & Jutten, C. Multimodal data fusion: an overview of methods, challenges, and prospects. Proc. IEEE 103, 1449–1477 (2015).
Ljosa, V. et al. Comparison of methods for image-based profiling of cellular morphological responses to small-molecule treatment. J. Biomol. Screen. 18, 1321–1329 (2013).
Peng, H., Ruan, Z., Long, F., Simpson, J.H. & Myers, E.W. V3D enables real-time 3D visualization and quantitative analysis of large-scale biological image data sets. Nat. Biotechnol. 28, 348–353 (2010).
Kreshuk, A., Koethe, U., Pax, E., Bock, D.D. & Hamprecht, F.A. Automated detection of synapses in serial section transmission electron microscopy image stacks. PLoS One 9, e87351 (2014).
The authors thank their group members and collaborators for insightful discussions that have helped shape their thoughts and research in bioimage analysis over the years. They also thankfully acknowledge support from Erasmus University Medical Center (E.M.), the National Science Foundation (CAREER DBI 1148823 to A.E.C.), the Allen Institute for Brain Science and the Janelia Research Campus of Howard Hughes Medical Institute (H.P.), the German Research Foundation (DFG SFB 1129/1134) and a Weston Visiting Professorship (F.A.H.), and Agence Nationale de la Recherche (ANR-10-INBS-04-06-France- BioImaging) and Institut Pasteur (J.C.O.M.). Raw data for the bioimage analysis examples shown are courtesy of Graham Knott (Fig. 2c) and Anna Akhmanova (Fig. 2d).
The authors declare no competing financial interests.
About this article
Cite this article
Meijering, E., Carpenter, A., Peng, H. et al. Imagining the future of bioimage analysis. Nat Biotechnol 34, 1250–1255 (2016). https://doi.org/10.1038/nbt.3722
A robust unsupervised machine-learning method to quantify the morphological heterogeneity of cells and nuclei
Nature Protocols (2021)
Nature Protocols (2021)
Nature Methods (2021)
Scientific Data (2020)