Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

An objective comparison of cell-tracking algorithms

Nature Methods volume 14, pages 1141–1152 (2017)Cite this article

Subjects

Abstract

We present a combined report on the results of three editions of the Cell Tracking Challenge, an ongoing initiative aimed at promoting the development and objective evaluation of cell segmentation and tracking algorithms. With 21 participating algorithms and a data repository consisting of 13 data sets from various microscopy modalities, the challenge displays today's state-of-the-art methodology in the field. We analyzed the challenge results using performance measures for segmentation and tracking that rank all participating methods. We also analyzed the performance of all of the algorithms in terms of biological measures and practical usability. Although some methods scored high in all technical aspects, none obtained fully correct solutions. We found that methods that either take prior information into account using learning strategies or analyze cells in a global spatiotemporal video context performed better than other methods under the segmentation and tracking scenarios included in the challenge.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Rent or buy this article

Get just this article for as long as you need it

$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Concept of cell segmentation and tracking.
Figure 2: Concept of the main factors that determine the quality of cell images and videos.
Figure 3
Figure 4: Sample images of the challenge data sets.
Figure 5: Top-three technical performance values (SEG, TRA and OP) obtained by the competing algorithms.
Figure 6: Top-three performing methods of the three challenge editions.
Figure 7: Top-three biological performance values (CT, TF, BC(i) and CCA) measures obtained by the competing algorithms.
Figure 8: Top-three performing methods of the three challenge editions in terms of the CT, TF, BC(i) and CCA scores.

References

  1. Franz, C.M., Jones, G.E. & Ridley, A.J. Cell migration in development and disease. Dev. Cell 2, 153–158 (2002).

    Article  CAS  Google Scholar 

  2. Bullen, A. Microscopic imaging techniques for drug discovery. Nat. Rev. Drug Discov. 7, 54–67 (2008).

    Article  CAS  Google Scholar 

  3. Walter, R.J. & Berns, M.W. Digital image processing and analysis. in Video Microscopy (ed. Inoué, S.) 327–392 (Springer Sciences, 1986).

  4. Schneider, C.A., Rasband, W.S. & Eliceiri, K.W. NIH Image to ImageJ: 25 years of image analysis. Nat. Methods 9, 671–675 (2012).

    Article  CAS  Google Scholar 

  5. Meijering, E. Cell segmentation: 50 years down the road. IEEE Signal Process. Mag. 29, 140–145 (2012).

    Article  Google Scholar 

  6. Dufour, A.C. et al. Signal processing challenges in quantitative 3-D cell morphology: more than meets the eye. IEEE Signal Process. Mag. 32, 30–40 (2015).

    Article  Google Scholar 

  7. Zimmer, C. et al. On the digital trail of mobile cells. IEEE Signal Process. Mag. 23, 54–62 (2006).

    Article  Google Scholar 

  8. Wuttisarnwattana, P., Gargesha, M., van't Hof, W., Cooke, K.R. & Wilson, D.L. Automatic stem cell detection in microscopic whole mouse cryo-imaging. IEEE Trans. Med. Imaging 35, 819–829 (2016).

    Article  Google Scholar 

  9. Lerner, B., Clocksin, W.F., Dhanjal, S., Hultén, M.A. & Bishop, C.M. Automatic signal classification in fluorescence in situ hybridization images. Cytometry 43, 87–93 (2001).

    Article  CAS  Google Scholar 

  10. Chen, X., Zhou, X. & Wong, S.T.C. Automated segmentation, classification, and tracking of cancer cell nuclei in time-lapse microscopy. IEEE Trans. Biomed. Eng. 53, 762–766 (2006).

    Article  Google Scholar 

  11. Henry, K.M. et al. PhagoSight: an open-source MATLAB package for the analysis of fluorescent neutrophil and macrophage migration in a zebrafish model. PLoS One 8, e72636 (2013).

    Article  CAS  Google Scholar 

  12. Wählby, C., Sintorn, I.M., Erlandsson, F., Borgefors, G. & Bengtsson, E. Combining intensity, edge and shape information for 2D and 3D segmentation of cell nuclei in tissue sections. J. Microsc. 215, 67–76 (2004).

    Article  Google Scholar 

  13. Cicconet, M., Geiger, D. & Gunsalus, K. Wavelet-based circular hough-transform and its application in embryo development analysis. in Proc. of the International Conference on Computer Vision Theory and Applications 669–674 (Science and Technology Publications, 2013).

  14. Türetken, E., Wang, X., Becker, C.J., Haubold, C. & Fua, P. Network flow integer programming to track elliptical cells in time-lapse sequences. IEEE Trans. Med. Imaging 36, 942–951 (2017).

    Article  Google Scholar 

  15. Malpica, N. et al. Applying watershed algorithms to the segmentation of clustered nuclei. Cytometry 28, 289–297 (1997).

    Article  CAS  Google Scholar 

  16. Ortiz de Solórzano, C. et al. Segmentation of confocal microscope images of cell nuclei in thick tissue sections. J. Microsc 193, 212–226 (1999).

    Article  Google Scholar 

  17. Cliffe, A. et al. Quantitative 3D analysis of complex single border cell behaviors in coordinated collective cell migration. Nat. Commun. 8, 14905 (2017).

    Article  CAS  Google Scholar 

  18. Ronneberger, O., Fisher, P. & Brox, T. U-net: convolutional networks for biomedical image segmentation. in Proc. MICCAI 2015 LNCS 9351, 234–241 (Spring, Cham, 2015).

    Google Scholar 

  19. Schiegg, M. et al. Graphical model for joint segmentation and tracking of multiple dividing cells. Bioinformatics 31, 948–956 (2015).

    Article  CAS  Google Scholar 

  20. Zimmer, C., Labruyère, E., Meas-Yedid, V., Guillén, N. & Olivo-Marin, J.-C. Segmentation and tracking of migrating cells in videomicroscopy with parametric active contours: a tool for cell-based drug testing. IEEE Trans. Med. Imaging 21, 1212–1221 (2002).

    Article  Google Scholar 

  21. Dufour, A., Thibeaux, R., Labruyère, E., Guillén, N. & Olivo-Marin, J.C. 3-D active meshes: fast discrete deformable models for cell tracking in 3-D time-lapse microscopy. IEEE Trans. Image Process. 20, 1925–1937 (2011).

    Article  Google Scholar 

  22. Maška, M. et al. Segmentation and shape tracking of whole fluorescent cells based on the Chan-Vese model. IEEE Trans. Med. Imaging 32, 995–1006 (2013).

    Article  Google Scholar 

  23. De Solorzano, C.O., Malladi, R., Lelièvre, S.A. & Lockett, S.J. Segmentation of nuclei and cells using membrane related protein markers. J. Microsc. 201, 404–415 (2001).

    Article  CAS  Google Scholar 

  24. Dzyubachyk, O., van Cappellen, W.A., Essers, J., Niessen, W.J. & Meijering, E. Advanced level-set-based cell tracking in time-lapse fluorescence microscopy. IEEE Trans. Med. Imaging 29, 852–867 (2010).

    Article  Google Scholar 

  25. Dufour, A. et al. Segmenting and tracking fluorescent cells in dynamic 3-D microscopy with coupled active surfaces. IEEE Trans. Image Process. 14, 1396–1410 (2005).

    Article  Google Scholar 

  26. Bensch, R. & Ronneberger, O. Cell segmentation and tracking in phase contrast images using graph cut with asymmetric boundary costs. In Proc. 2015 IEEE Int. Symp. Biomed. Imaging (ISBI) 1120–1123 (2015).

  27. Harder, N. et al. Automatic analysis of dividing cells in live cell movies to detect mitotic delays and correlate phenotypes in time. Genome Res. 19, 2113–2124 (2009).

    Article  CAS  Google Scholar 

  28. Bise, R., Yin, Z. & Kanade, T. Reliable cell tracking by global data association. in Proc. 2011 IEEE Int. Symp. Biomed. Imaging (ISBI) 1004–1010 (2011).

  29. Magnusson, K.E.G., Jaldén, J., Gilbert, P.M. & Blau, H.M. Global linking of cell tracks using the Viterbi algorithm. IEEE Trans. Med. Imaging 34, 911–929 (2015).

    Article  Google Scholar 

  30. Maška, M. et al. A benchmark for comparison of cell tracking algorithms. Bioinformatics 30, 1609–1617 (2014).

    Article  Google Scholar 

  31. Svoboda, D. & Ulman, V. MitoGen: A framework for generating 3D synthetic time-lapse sequences of cell populations in fluorescence microscopy. IEEE Trans. Med. Imaging 36, 310–321 (2017).

    Article  Google Scholar 

  32. Murray, J.I. et al. Automated analysis of embryonic gene expression with cellular resolution in C. elegans. Nat. Methods 5, 703–709 (2008).

    Article  CAS  Google Scholar 

  33. Amat, F. et al. Fast, accurate reconstruction of cell lineages from large-scale fluorescence microscopy data. Nat. Methods 11, 951–958 (2014).

    Article  CAS  Google Scholar 

  34. Chenouard, N. et al. Objective comparison of particle tracking methods. Nat. Methods 11, 281–289 (2014).

    Article  CAS  Google Scholar 

  35. Schindelin, J. et al. Fiji: an open-source platform for biological-image analysis. Nat. Methods 9, 676–682 (2012).

    Article  CAS  Google Scholar 

  36. Klette, R. & Zamperoni, P. Handbook of Image Processing Operators (New York, Wiley, 1996).

  37. Lin, C.L. & Miller, J.D. 3D characterization and analysis of particle shape using X-ray microtomography (XMT). Powder Technol. 154, 61–69 (2005).

    Article  CAS  Google Scholar 

  38. Matula, P. et al. Cell tracking accuracy measurement based on comparison of acyclic oriented graphs. PLoS One 10, e0144959 (2015).

    Article  Google Scholar 

  39. Li, K. et al. Cell population tracking and lineage construction with spatiotemporal context. Med. Image Anal. 12, 546–566 (2008).

    Article  Google Scholar 

  40. Brown, M.R. et al. Flow-based cytometric analysis of cell cycle via simulated cell populations. PLOS Comput. Biol. 6, e1000741 (2010).

    Article  Google Scholar 

Download references

Acknowledgements

We acknowledge the work of A. Urbiola, C. Ederra, T. España, S. Venkatesan, D.M.W. Balak, P. Karas, T. Bolcková, M. Štreitová, M. Charousová and L. Zátopková, who manually annotated the data sets to create the ground truths used to evaluate the performance of the algorithms. We also would like to thank F. Prósper (CIMA-University of Navarra), E. Bártová (Institute of Biophysics, Academy of Sciences of the Czech Republic), J. Essers (Erasmus University Medical Center), the Mitocheck consortium, A. Rouzaut (CIMA-University of Navarra), R. Kamm (Massachussets Institute of Technology), the Waterston Lab (The George Washington University), P. Keller (Howard Hughes Medical Institute), S. Kumar (University of California at Berkeley), G. van Cappellen (Erasmus University Medical Center) and T. Becker (Fraunhofer Institution for Marine Biology), who provided the data sets used in the three challenge editions. Finally, we thank R. Stoklasa for technical support. The participants would like to acknowledge the contributions of M. Schiegg, D. Stöckel, J. Crowe, M. Temerinac-Ott and P. Fischer. This work was funded by Spanish Ministry of Economy MINECO grants DPI2012-38090-C03-02 (C.O.-d.-S.) and DPI2015-64221-C2-2 (C.O.-d.-S.), TEC2013-48552-C2-1-R (A.M.B.), TEC2015-73064-EXP (A.M.B), and TEC2016-78052-R (A.M.B.); Netherlands Organization for Scientific Research (NWO) grants 612.001.018 (M.R. and E.M.) and 639.021.128 (I.S.); Dutch Technology Foundation (STW) grant 10443 (I.S. and E.M.); Czech Science Foundation (GACR) grant P302/12/G157 (M.K. and Pavel Matula); the Czech Ministry of Education, Youth and Sports grant LTC17016 in the frame of EU COST NEUBIAS project (M.M., Pavel Matula, Petr Matula, D.S. and M.K.); Helmholtz Association (J.S. and R.M.) and DFG grant MI 1315/4-1 (J.S. and R.M.); the Excellence Initiative of the German Federal and State Governments EXC 294 (O.R., T.B. and R.B.); the Swiss Commission for Technology and Innovation, CTI project 16997 (Ö.D. and L.M.); the BMBF, projects ENGINE (NGFN+), RNA-Code (e:Bio) and de.NBI, as well as the DFG, SFB 1129 and RTG 1653 (N.H. and K.R.); the HGS MathComp Graduate School, the SFB 1129 for integrative analysis of pathogen replication and spread, the RTG 1653 for probabilistic graphical models, and the CellNetworks Excellence Cluster/EcTop (C.H., S.W. and F.H.); the Baxter Foundation and US National Institutes of Health grant AG020961 (H.M.B.) and the Swedish Research Council VR Grant 2015-04026 (K.M. and J.J.); and the BMBF, project de.NBI, grant 031L0102 (V.U. and F.J.).

Author information

Author notes

  1. Vladimír Ulman

    Present address: Max Planck Institute of Molecular Cell Biology and Genetics, Dresden, Germany

  2. Olaf Ronneberger

    Present address: DeepMind, London, UK

  3. Nathalie Harder

    Present address: Definiens AG, Munich, Germany

  4. Pengdong Xiao

    Present address: National Heart Research Institute Singapore (NHRIS), National Heart Centre Singapore (NHCS), Singapore

  5. Yuexiang Li

    Present address: Computer Vision Institute, College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China

  6. Vladimír Ulman and Martin Maška: These authors contributed equally to this work.

Authors and Affiliations

  1. Centre for Biomedical Image Analysis, Masaryk University, Brno, Czech Republic

    Vladimír Ulman, Martin Maška, Pavel Matula, Petr Matula, David Svoboda & Michal Kozubek

  2. ACCESS Linnaeus Centre, KTH Royal Institute of Technology, Stockholm, Sweden

    Klas E G Magnusson & Joakim Jaldén

  3. Computer Science Department and BIOSS Centre for Biological Signaling Studies University of Freiburg, Frieburg, Germany

    Olaf Ronneberger, Robert Bensch & Thomas Brox

  4. Heidelberg Collaboratory for Image Processing, IWR, University of Heidelberg, Heidelberg, Germany

    Carsten Haubold, Steffen Wolf & Fred A Hamprecht

  5. Department of Bioinformatics and Functional Genomics, Biomedical Computer Vision Group, BIOQUANT, IPMB, University of Heidelberg and DKFZ, Heidelberg, Germany

    Nathalie Harder & Karl Rohr

  6. Departments of Medical Informatics and Radiology, Biomedical Imaging Group Rotterdam, Erasmus University Medical Center Rotterdam, Rotterdam, The Netherlands

    Miroslav Radojevic, Ihor Smal & Erik Meijering

  7. Department of Microbiology and Immunology, Baxter Laboratory for Stem Cell Biology, and Institute for Stem Cell Biology and Regenerative Medicine, Stanford University School of Medicine, Stanford, California, USA

    Helen M Blau

  8. Division of Image Processing, Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands

    Oleh Dzyubachyk & Boudewijn Lelieveldt

  9. Intelligent Systems Department, Delft University of Technology, Delft, The Netherlands

    Boudewijn Lelieveldt

  10. Institute of Molecular and Cell Biology, A*Star, Singapore

    Pengdong Xiao

  11. Department of Engineering, University of Nottingham, Nottingham, UK

    Yuexiang Li

  12. Faculty of Engineering, University of Nottingham, Ningbo, China

    Siu-Yeung Cho

  13. BioImage Analysis Unit, Institut Pasteur, Paris, France

    Alexandre C Dufour & Jean-Christophe Olivo-Marin

  14. Research Centre in Biomedical Engineering, School of Mathematics, Computer Science and Engineering, City University of London, London, UK

    Constantino C Reyes-Aldasoro & Jose A Solis-Lemus

  15. Group for Automated Image and Data Analysis, Institute for Applied Computer Science, Karlsruhe Institute of Technology, Eggenstein-Leopoldshafen, Germany

    Johannes Stegmaier & Ralf Mikut

  16. i3S - Instituto de Investigação e Inovação em Saúde, Universidade do Porto, Porto, Portugal

    Tiago Esteves & Pedro Quelhas

  17. Facultade de Engenharia, Universidade do Porto, Porto, Portugal

    Tiago Esteves

  18. S3IT, University of Zurich, Zurich, Switzerland

    Ömer Demirel & Lars Malmström

  19. Max Planck Institute of Molecular Cell Biology and Genetics, Dresden, Germany

    Florian Jug & Pavel Tomancak

  20. Bioengineering and Aerospace Engineering Department, Universidad Carlos III de Madrid, Getafe, Spain

    Arrate Muñoz-Barrutia

  21. Instituto de Investigación Sanitaria Gregorio Marañon, Madrid, Spain

    Arrate Muñoz-Barrutia

  22. CIBERONC, IDISNA and Program of Solid Tumors and Biomarkers, Center for Applied Medical Research, University of Navarra, Pamplona, Spain

    Carlos Ortiz-de-Solorzano

  23. Bioengineering Department, TECNUN School of Engineering, University of Navarra, San Sebastián, Spain

    Carlos Ortiz-de-Solorzano

Authors
  1. Vladimír Ulman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Martin Maška
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Klas E G Magnusson
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Olaf Ronneberger
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Carsten Haubold
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Nathalie Harder
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Pavel Matula
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Petr Matula
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. David Svoboda
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Miroslav Radojevic
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Ihor Smal
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Karl Rohr
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Joakim Jaldén
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Helen M Blau
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Oleh Dzyubachyk
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Boudewijn Lelieveldt
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Pengdong Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Yuexiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Siu-Yeung Cho
    View author publications

    You can also search for this author in PubMed Google Scholar

  20. Alexandre C Dufour
    View author publications

    You can also search for this author in PubMed Google Scholar

  21. Jean-Christophe Olivo-Marin
    View author publications

    You can also search for this author in PubMed Google Scholar

  22. Constantino C Reyes-Aldasoro
    View author publications

    You can also search for this author in PubMed Google Scholar

  23. Jose A Solis-Lemus
    View author publications

    You can also search for this author in PubMed Google Scholar

  24. Robert Bensch
    View author publications

    You can also search for this author in PubMed Google Scholar

  25. Thomas Brox
    View author publications

    You can also search for this author in PubMed Google Scholar

  26. Johannes Stegmaier
    View author publications

    You can also search for this author in PubMed Google Scholar

  27. Ralf Mikut
    View author publications

    You can also search for this author in PubMed Google Scholar

  28. Steffen Wolf
    View author publications

    You can also search for this author in PubMed Google Scholar

  29. Fred A Hamprecht
    View author publications

    You can also search for this author in PubMed Google Scholar

  30. Tiago Esteves
    View author publications

    You can also search for this author in PubMed Google Scholar

  31. Pedro Quelhas
    View author publications

    You can also search for this author in PubMed Google Scholar

  32. Ömer Demirel
    View author publications

    You can also search for this author in PubMed Google Scholar

  33. Lars Malmström
    View author publications

    You can also search for this author in PubMed Google Scholar

  34. Florian Jug
    View author publications

    You can also search for this author in PubMed Google Scholar

  35. Pavel Tomancak
    View author publications

    You can also search for this author in PubMed Google Scholar

  36. Erik Meijering
    View author publications

    You can also search for this author in PubMed Google Scholar

  37. Arrate Muñoz-Barrutia
    View author publications

    You can also search for this author in PubMed Google Scholar

  38. Michal Kozubek
    View author publications

    You can also search for this author in PubMed Google Scholar

  39. Carlos Ortiz-de-Solorzano
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

V.U. actively participated in the organization and management of the CTC challenges by handling submissions, producing synthetic data sets, evaluating the submitted results and globally analyzing the participant's contributions, and creating annotations for data set evaluation. V.U. contributed to the writing of the manuscript and produced the tables and plot results, as well as the Fiji plugin with the evaluation suite. M.M. actively participated in the organization and management of the CTC challenges by handling and evaluating submissions, providing evaluation and annotation software, supervising annotations, and creating consensual ground truths for the evaluation of the submitted results. M.M. contributed to the writing of the manuscript and was a challenge participant. K.E.G.M., O.R. and C.H. were top ranked challenge participants and contributed to the writing of the manuscript. N.H. was a top ranked challenge participant. Pavel Matula actively participated in the organization of the CTC challenges by leading the development of a suitable tracking measure and assessing the behavior of various measures on challenge data sets. Petr Matula, M.R. and I.S. actively participated in the organization of the CTC challenges by preparing data and supervising data annotation. D.S. actively participated in the organization of the CTC challenges by leading the development of synthetic data generator and creation of suitable collection of synthetic time-lapse sequences with absolute ground truth. K.R., J.J., H.M.B., O.D., B.L., P.X., Y.L., S.-Y.C., A.C.D., J.-.C.O.-M., C.C.R.-A., J.A.S.-L., R.B., T.B., J.S., R.M., S.W., F.A.H., T.E., P.Q., Ö.D. and L.M. were challenge participants. F.J. contributed to the revision of the manuscript and supported V.U. with the related data processing. P.T., E.M., A.M.-B. and M.K. were challenge organizers and contributed to the revision of the manuscript. C.O.-d.-S. was a challenge organizer, coordinated the work of the committee that organized the challenges and wrote the manuscript with input from all of the authors.

Corresponding author

Correspondence to Carlos Ortiz-de-Solorzano.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Integrated supplementary information

Supplementary Figure 1 Representative frame of DIC-C2DH-HeLa videos.

This dataset presents low SNR and CR values characteristic of phase-enhancement microscopy techniques mostly because the average intensity of the cells is very similar to the intensity of the background. The signal inside the cells is highly heterogeneous due to DIC-highlighted internal structures and organelles. The cells are heterogeneous in intensity and shape since in most frames co-exist well spread, cuboidal, low intensity interphase cells (B) with rounded, bright cells undergoing mitosis (A). The cells are highly packed and show very low intensity changes between neighboring cells (black arrows).

Supplementary Figure 2 Representative frame of Fluo-C2DL-MSC videos.

The SNR and CR values are low, due to the low level of emission of the fluorescent cytoplasmic reporter, especially in the long, thin filopodial extensions of the cell (white arrow). The intensity is also quite variable in different parts of the cell (see the different intensity of the nucleus and cytoplasm in cell A), and the cells present different levels of intensity (compare cells A and B), possibly due to different expression of the transfected fluorescent reporter. The shape of the cells is highly irregular due to the long filopodial extensions (white arrow). The cells show a significant degree of bleaching and move fast, which leads to low cell overlaps between consecutive frames.

Supplementary Figure 3 Representative xy (top) and xz (bottom) slices of a frame of Fluo-C3DH-H157 videos.

This dataset displays reasonably good values for most properties, with the exception of some signal decay due to photobleaching. The presence of prominent blebs and some heterogeneity between cell intensities can also complicate accurately segmenting and delineating the cell boundaries.

Supplementary Figure 4 xy (top left and right) slices from two consecutive frames and one xz (bottom) slice of a frame of Fluo-C3DL-MDA321 videos.

The SNR and CR of this dataset are relatively low due to both low signal intensity and increased background, which affects the quality of the signal especially in the long migration-related filopodial extensions (white arrow). This noisy signal efficiency causes high internal heterogeneity. The images were acquired at low resolution, especially in the axial direction (see bottom panel), and also in the temporal dimension (compare the same cell, A and B in two consecutive frames) and suffer from significant photobleaching, which complicates the segmentation and tracking tasks even further.

Supplementary Figure 5 Representative frame of Fluo-N2DH-GOWT1 videos.

The signal inside the cell nuclei varies due to the existence of prominent, unlabeled nucleoli (white arrows) and the heterogeneity of the average cell intensities (see for instance the difference between the cells labeled A and B).

Supplementary Figure 6 Representative frame of Fluo-N2DL-HeLa videos.

The image shows some of the challenges posed by this dataset, including signal heterogeneity between cells as shown by the presence of a large range of nuclear intensity, the low spatial resolution, high cell density, and the presence of division events (white arrows).

Supplementary Figure 7 xy (top left and right) slices from two consecutive frames and one xz (bottom) slice of a frame of Fluo-N3DH-CE videos.

The most significant problems of this dataset are high cell density, the low resolution in the axial direction (see xz bottom slice), low cell overlap between frames caused by large temporal resolution, and the abundance of mitotic cells (white arrows) typical of a developing embryo.

Supplementary Figure 8 Representative frame of Fluo-N3DH-CHO videos.

The main challenges of this dataset are the internal heterogeneity of the staining, clearly visible in the images and caused by the fact that the nuclear staining does not label the nucleoli of the cells (white arrows), and a relatively high cell density.

Supplementary Figure 9 Representative xy (top) and xz (bottom) slices of a frame of Fluo-N3DL-DRO videos.

The low spatial (visible in this image) and temporal resolution characteristic of light sheet microscopy, and the presence of frequent mitoses typical for a developing embryo render this dataset the most difficult of the challenge datasets.

Supplementary Figure 10 Representative frame of PhC-C2DH-U373 videos.

At this level of resolution, the SNR, CR, Heti, and Hetb values are deficient, as expected for a contrast enhancement microscopy data. All other characteristics are either average or good, which seems to compensate the deficient values for the segmentation and tracking task. Among them, high spatial and temporal resolutions, and a relatively low cells density (visible in the image) seem to be especially beneficial.

Supplementary Figure 11 Representative frame of PhC-C2DL-PSC videos.

Most of the parameters are in the average to low range, especially those already mentioned for brightfield modalities. The very low spatial resolution (visible in the image), to some extent compensated by the high temporal resolution, and significant number of division events (black arrows) make the cells in this dataset difficult to segment and track.

Supplementary Figure 12 Example of the initial image (technically a labeled mask) used for one of the Fluo-N3DH-SIM+ competition videos.

The image is displayed using maximum intensity projection of labels. Note the presence of an extended boundary to allow cells to move away from the cell population. Cutting this border away introduces the effect of cells leaving and entering the field of view of the final image sequence.

Supplementary Figure 13 Overall performance (OP) of the top-three performing algorithms, if available, binned per dataset across the three CTC editions.

For all but the synthetic datasets (Fluo-N2DH-SIM+ and Fluo-N3DH-SIM+), the target scores correspond to the OP scores of the three individual manual annotations. For the two embryonic datasets (Fluo-N3DH-CE and Fluo-N3DL-DRO), however, there were three segmentation annotations and only one tracking annotation available. Accordingly, the target scores for those datasets were calculated by considering 1.0 (i.e., perfect match to the provided tracking ground truth) as the tracking scores of the three individual manual annotations. Note that missing dots correspond to datasets not offered in a particular CTC edition.

Supplementary Figure 14 Robustness of the weighting used for SEG and TRA.

Robustness of the weighting used for SEG and TRA. The image shows the number of rank changes in the top-three ranked methods, as the weights given to SEG and TRA change from 0 to 1, in 0.001 steps.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–14, Supplementary Tables 1–4 and Supplementary Notes 1–3 (PDF 4572 kb)

Life Sciences Reporting Summary (PDF 158 kb)

Supplementary Data 1

Configuration file used for creating the synthetic competition videos (PDF 509 kb)

Supplementary Data 2

Parameter configurations used by the competing cell tracking methods across all applicable videos (XLSX 81 kb)

Supplementary Data 3

Technical measures calculated for all competing cell tracking methods across all applicable competition videos (XLSX 66 kb)

Supplementary Data 4

Biologically-oriented measures calculated for all competing cell tracking methods across all applicable competition videos (XLSX 113 kb)

Supplementary Data 5

Usability measures calculated for all competing cell tracking methods across s all applicable videos in the training and competition datasets (XLSX 79 kb)

Dataset DIC-C2DH-HeLa

Rendering of a representative fragment of datasets DIC-C2DH-HeLa (AVI 7660 kb)

Dataset Fluo-C2DL-MSC

Rendering of a representative fragment of datasets Fluo-C2DL-MSC (AVI 5553 kb)

Dataset Fluo-C3DH-H157

Rendering of a representative fragment of datasets Fluo-C3DH-H157 (AVI 1835 kb)

Dataset Fluo-C3DL-MDA321

Rendering of a representative fragment of datasets Fluo-C3DL-MDA321 (AVI 1778 kb)

Dataset Fluo-N2DH-GOWT1

Rendering of a representative fragment of datasets Fluo-N2DH-GOWT1 (AVI 15466 kb)

Dataset Fluo-N2DL-HeLa

Rendering of a representative fragment of datasets Fluo-N2DL-HeLa (AVI 3733 kb)

Dataset Fluo-N3DH-CE

Rendering of a representative fragment of datasets Fluo-N3DH-CE (AVI 5669 kb)

Dataset Fluo-N3DH-CHO

Rendering of a representative fragment of datasets Fluo-N3DH-CHO (AVI 8201 kb)

Dataset Fluo-N3DL-DRO

Rendering of a representative fragment of datasets Fluo-N3DL-DRO (AVI 5771 kb)

Dataset PhC-C2DH-U373

Rendering of a representative fragment of datasets PhC-C2DH-U373 (AVI 7024 kb)

Dataset PhC-C2DL-PSC

Rendering of a representative fragment of datasets PhC-C2DL-PSC (AVI 5487 kb)

Dataset Fluo-N2DH-SIM+

Rendering of a representative fragment of datasets Fluo-N2DH-SIM+ (AVI 3330 kb)

Dataset Fluo-N3DH-SIM+

Rendering of a representative fragment of datasets Fluo-N3DH-SIM+ (AVI 3026 kb)

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ulman, V., Maška, M., Magnusson, K. et al. An objective comparison of cell-tracking algorithms. Nat Methods 14, 1141–1152 (2017). https://doi.org/10.1038/nmeth.4473

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nmeth.4473

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing