We present a combined report on the results of three editions of the Cell Tracking Challenge, an ongoing initiative aimed at promoting the development and objective evaluation of cell segmentation and tracking algorithms. With 21 participating algorithms and a data repository consisting of 13 data sets from various microscopy modalities, the challenge displays today's state-of-the-art methodology in the field. We analyzed the challenge results using performance measures for segmentation and tracking that rank all participating methods. We also analyzed the performance of all of the algorithms in terms of biological measures and practical usability. Although some methods scored high in all technical aspects, none obtained fully correct solutions. We found that methods that either take prior information into account using learning strategies or analyze cells in a global spatiotemporal video context performed better than other methods under the segmentation and tracking scenarios included in the challenge.
We acknowledge the work of A. Urbiola, C. Ederra, T. España, S. Venkatesan, D.M.W. Balak, P. Karas, T. Bolcková, M. Štreitová, M. Charousová and L. Zátopková, who manually annotated the data sets to create the ground truths used to evaluate the performance of the algorithms. We also would like to thank F. Prósper (CIMA-University of Navarra), E. Bártová (Institute of Biophysics, Academy of Sciences of the Czech Republic), J. Essers (Erasmus University Medical Center), the Mitocheck consortium, A. Rouzaut (CIMA-University of Navarra), R. Kamm (Massachussets Institute of Technology), the Waterston Lab (The George Washington University), P. Keller (Howard Hughes Medical Institute), S. Kumar (University of California at Berkeley), G. van Cappellen (Erasmus University Medical Center) and T. Becker (Fraunhofer Institution for Marine Biology), who provided the data sets used in the three challenge editions. Finally, we thank R. Stoklasa for technical support. The participants would like to acknowledge the contributions of M. Schiegg, D. Stöckel, J. Crowe, M. Temerinac-Ott and P. Fischer. This work was funded by Spanish Ministry of Economy MINECO grants DPI2012-38090-C03-02 (C.O.-d.-S.) and DPI2015-64221-C2-2 (C.O.-d.-S.), TEC2013-48552-C2-1-R (A.M.B.), TEC2015-73064-EXP (A.M.B), and TEC2016-78052-R (A.M.B.); Netherlands Organization for Scientific Research (NWO) grants 612.001.018 (M.R. and E.M.) and 639.021.128 (I.S.); Dutch Technology Foundation (STW) grant 10443 (I.S. and E.M.); Czech Science Foundation (GACR) grant P302/12/G157 (M.K. and Pavel Matula); the Czech Ministry of Education, Youth and Sports grant LTC17016 in the frame of EU COST NEUBIAS project (M.M., Pavel Matula, Petr Matula, D.S. and M.K.); Helmholtz Association (J.S. and R.M.) and DFG grant MI 1315/4-1 (J.S. and R.M.); the Excellence Initiative of the German Federal and State Governments EXC 294 (O.R., T.B. and R.B.); the Swiss Commission for Technology and Innovation, CTI project 16997 (Ö.D. and L.M.); the BMBF, projects ENGINE (NGFN+), RNA-Code (e:Bio) and de.NBI, as well as the DFG, SFB 1129 and RTG 1653 (N.H. and K.R.); the HGS MathComp Graduate School, the SFB 1129 for integrative analysis of pathogen replication and spread, the RTG 1653 for probabilistic graphical models, and the CellNetworks Excellence Cluster/EcTop (C.H., S.W. and F.H.); the Baxter Foundation and US National Institutes of Health grant AG020961 (H.M.B.) and the Swedish Research Council VR Grant 2015-04026 (K.M. and J.J.); and the BMBF, project de.NBI, grant 031L0102 (V.U. and F.J.).
Integrated supplementary information
Rendering of a representative fragment of datasets DIC-C2DH-HeLa
Rendering of a representative fragment of datasets Fluo-C2DL-MSC
Rendering of a representative fragment of datasets Fluo-C3DH-H157
Rendering of a representative fragment of datasets Fluo-C3DL-MDA321
Rendering of a representative fragment of datasets Fluo-N2DH-GOWT1
Rendering of a representative fragment of datasets Fluo-N2DL-HeLa
Rendering of a representative fragment of datasets Fluo-N3DH-CE
Rendering of a representative fragment of datasets Fluo-N3DH-CHO
Rendering of a representative fragment of datasets Fluo-N3DL-DRO
Rendering of a representative fragment of datasets PhC-C2DH-U373
Rendering of a representative fragment of datasets PhC-C2DL-PSC
Rendering of a representative fragment of datasets Fluo-N2DH-SIM+
Rendering of a representative fragment of datasets Fluo-N3DH-SIM+