Skip to main content

Thank you for visiting You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Learning biophysical determinants of cell fate with deep neural networks

A preprint version of the article is available at bioRxiv.


Deep learning is now a powerful tool in microscopy data analysis, and is routinely used for image processing applications such as segmentation and denoising. However, it has rarely been used to directly learn mechanistic models of a biological system, owing to the complexity of the internal representations. Here, we develop an end-to-end machine learning approach capable of learning an explainable model of a complex biological phenomenon, cell competition, directly from a large corpus of time-lapse microscopy data. Cell competition is a quality control mechanism that eliminates unfit cells from a tissue, during which cell fate is thought to be determined by the local cellular neighbourhood over time. To investigate this, we developed a new approach (τ-VAE) by coupling a probabilistic encoder to a temporal convolution network to predict the fate of each cell in an epithelium. Using the τ-VAE’s latent representation of the local tissue organization and the flow of information in the network, we decode the physical parameters responsible for correct prediction of fate in cell competition. Remarkably, the model autonomously learns that cell density is the single most important factor in predicting cell fate—a conclusion that is in agreement with our current understanding from over a decade of scientific research. Finally, to test the learned internal representation, we challenge the network with experiments performed in the presence of drugs that block signalling pathways involved in competition. We present a novel discriminator network, which using the predictions of the τ-VAE can identify conditions that deviate from the normal behaviour, paving the way for automated, mechanism-aware drug screening.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Get just this article for as long as you need it


Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Learning a meaningful representation of cell competition to predict the fate of cells.
Fig. 2: Architecture, training and inference using the τ-VAE.
Fig. 3: An explainable internal representation of cell competition.
Fig. 4: The predictive model of cell behaviour enables drug evaluation.

Data availability

Training data including cell images, single-cell trajectories and metadata are available from the UCL data repository37. For other enquiries contact the corresponding author. Source data are provided with this paper.

Code availability

A reference implementation of the τ-VAE is available at and the UCL software repository38. For other enquiries contact the corresponding author.


  1. Levayer, R. & Moreno, E. Mechanisms of cell competition: themes and variations. J. Cell Biol. 200, 689–698 (2013).

    Article  Google Scholar 

  2. Morata, G. & Ripoll, P. Minutes: mutants of Drosophila autonomously affecting cell division rate. Dev. Biol. 42, 211–221 (1975).

    Article  Google Scholar 

  3. Parker, T., Madan, E., Gupta, K., Moreno, E. & Gogna, R. Cell competition spurs selection of aggressive cancer cells. Trends Cancer 6, 732–736 (2020).

    Article  Google Scholar 

  4. Levayer, R., Hauert, B. & Moreno, E. Cell mixing induced by myc is required for competitive tissue invasion and destruction. Nature 524, 476–480 (2015).

    Article  Google Scholar 

  5. Vincent, J.-P., Fletcher, A. G. & Baena-Lopez, L. A. Mechanisms and mechanics of cell competition in epithelia. Nat. Rev. Mol. Cell Biol. 14, 581–591 (2013).

    Article  Google Scholar 

  6. Hogan, C. et al. Characterization of the interface between normal and transformed epithelial cells. Nat. Cell Biol. 11, 460–467 (2009).

    Article  Google Scholar 

  7. Wagstaff, L. et al. Mechanical cell competition kills cells via induction of lethal p53 levels. Nat. Commun. 7, 11373 (2016).

    Article  Google Scholar 

  8. Bove, A. et al. Local cellular neighborhood controls proliferation in cell competition. Mol. Biol. Cell 28, 3215–3228 (2017).

    Article  Google Scholar 

  9. Gradeci, D. et al. Cell-scale biophysical determinants of cell competition in epithelia. eLife 10, e61011 (2021).

  10. Moen, E. et al. Deep learning for cellular image analysis. Nat. Methods 16, 1233–1246 (2019).

    Article  Google Scholar 

  11. Belthangady, C. & Royer, L. A. Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction. Nat. Methods 16, 1215–1225 (2019).

    Article  Google Scholar 

  12. Ren, E. et al. Deep learning-enhanced morphological profiling predicts cell fate dynamics in real-time in hPSCs. Preprint at bioRxiv (2021).

  13. Kingma, D. P. & Welling, M. Auto-encoding variational bayes. Preprint at (2013).

  14. Chan, C. K., Hadjitheodorou, A., Tsai, T. Y.-C. & Theriot, J. A. Quantitative comparison of principal component analysis and unsupervised deep learning using variational autoencoders for shape analysis of motile cells. Preprint at bioRxiv (2020).

  15. Zaritsky, A. et al. Interpretable deep learning uncovers cellular properties in label-free live cell images that are predictive of highly metastatic melanoma. Cell Syst. 12, 733–747.e6 (2021).

    Google Scholar 

  16. Wu, Z. et al. DynaMorph: self-supervised learning of morphodynamic states of live cells. Mol. Biol. Cell 33, e21110561 (2022).

    Article  Google Scholar 

  17. Yang, K. D. et al. Predicting cell lineages using autoencoders and optimal transport. PLoS Comput. Biol. 16, e1007828 (2020).

    Article  Google Scholar 

  18. Buggenthin, F. et al. Prospective identification of hematopoietic lineage choice by deep learning. Nat. Methods 14, 403–406 (2017).

    Article  Google Scholar 

  19. Norman, M. et al. Loss of Scribble causes cell competition in mammalian cells. J. Cell Sci. 125, 59–66 (2012).

    Article  Google Scholar 

  20. Ronneberger, O., Fischer, P. & Brox, T. U-Net: convolutional networks for biomedical image segmentation. Lect. Notes Comput. Sci. 9351, 234–241 (2015).

    Article  Google Scholar 

  21. Ulicna, K., Vallardi, G., Charras, G. & Lowe, A. R. Automated deep lineage tree analysis using a Bayesian single cell tracking approach. Front. Comput. Sci. 3 (2021).

  22. Mnih, V., Heess, N., Graves, A. & Kavukcuoglu, K. Recurrent models of visual attention. In Advances in Neural Information Processing Systems Vol. 27, 2204-2212 (eds Ghahramani, Z. et al.) (Curran Associates, 2014);

  23. Higgins, I. et al. beta-VAE: learning basic visual concepts with a constrained variational framework. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proc. (2017);

  24. van den Oord, A. et al. Wavenet: a generative model for raw audio. Preprint at (2016).

  25. Bai, S., Kolter, J. Z. & Koltun, V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. Preprint at (2018).

  26. Hochreiter, S. & Schmidhuber, J. Long short-term memory. Neural Comput. 9, 1735–1780 (1997).

    Article  Google Scholar 

  27. Levayer, R., Dupont, C. & Moreno, E. Tissue crowding induces caspase-dependent competition for space. Curr. Biol. 26, 670–677 (2016).

    Article  Google Scholar 

  28. Smilkov, D., Thorat, N., Kim, B., Viégas, F. B. & Wattenberg, M. Smoothgrad: removing noise by adding noise. Preprint at (2017).

  29. Kuma, Y. et al. BIRB796 inhibits all p38 MAPK isoforms in vitro and in vivo. J. Biol. Chem. 280, 19472–19479 (2005).

    Article  Google Scholar 

  30. Chandrasekaran, S. N., Ceulemans, H., Boyd, J. D. & Carpenter, A. E. Image-based profiling for drug discovery: due for a machine-learning upgrade? Nat. Rev. Drug Discov. 20, 145–159 (2020).

    Article  Google Scholar 

  31. Kucinski, I., Dinan, M., Kolahgar, G., & Piddini, E. Chronic activation of JNK JAK/STAT and oxidative stress signalling causes the loser cell status. Nat. Commun. 8, 136 (2017).

    Article  Google Scholar 

  32. Parmar, N. et al. Image transformer. Preprint at (2018).

  33. Abnar, S. & Zuidema, W. H. Quantifying attention flow in transformers. Preprint at (2020).

  34. Hetzel, L., Fischer, D. S., Günnemann, S. & Theis, F. J. Graph representation learning for single-cell biology. Curr. Opin. Syst. Biol. 28, 100347 (2021).

    Article  Google Scholar 

  35. Burgess, C. P. et al. Understanding disentangling in β-VAE. Preprint at (2018).

  36. Long, J., Shelhamer, E. & Darrell, T. Fully convolutional networks for semantic segmentation. Preprint at (2015).

  37. Lowe, A. R., Soelistyo, C. J., Vallardi, G. & Charras, G. cellX-predict datasets. UCL Research Data Repository (2022).

  38. Lowe, A. R. & Soelistyo, C. J. cellX-predict software. UCL Software Database (2022).

Download references


This work was supported by a BBSRC LIDo artificial intelligence PhD studentship for C.J.S. G.V. was supported by BBSRC grant BB/S009329/1. We thank N. Day, J. Michalowska and D. Smaje for help with annotating data, and M. Kelkar for additional supporting data. We thank Y. Fujita for the kind gift of cell lines used in this work. We also thank members of the Lowe and Charras laboratories for discussions and technical support during the project. A.R.L. acknowledges the Turing Fellowship from the Alan Turing Institute. A.R.L. and G.C. acknowledge the support of BBSRC grant BB/S009329/1.

Author information

Authors and Affiliations



A.R.L. and G.C. conceived and designed the research. G.V. performed experiments. C.J.S. developed and performed computational analysis. A.R.L. wrote the image processing and cell tracking code. C.J.S., G.V., G.C. and A.R.L. evaluated the results and wrote the paper.

Corresponding author

Correspondence to Alan R. Lowe.

Ethics declarations

Competing interests

A UK provisional patent application (patent application no. GB2116864.6) filed in relation to these results by applicant UCL Business Ltd remains pending. The remaining authors declare no competing interests.

Peer review

Peer review information

Nature Machine Intelligence thanks Shalin Mehta and Chris Bakal for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Data flow in the model.

(a) Example time-lapse microscopy data showing a mixed population of MDCKWT (green) and scribkd(magenta) cells. (b) Single-cell tracking is used to build a detailed training dataset of trajectories. The single-cell track is used to extract a glimpse of the cell over time, that becomes the input data for the machine learning models. (c) The data preparation and inference pipeline. A CNN/LSTM network classifies the fate of the cell and determines the cutoff point to truncate the track to remove images that encode the fate of the cell. The goal of the machine learning model is then to learn a representation that can predict the fate of a cell (circled in white) given the local configuration during interphase. Importantly, the model does not actually observe the fate since these data fall beyond the cutoff. Images are taken at 4-minute intervals, MDCKWT cells appear in green and scribkdin magenta.

Extended Data Fig. 2 Glimpse extraction and cell masking to determine the best image input scale for prediction.

Three different scale windows are extracted, Small, Mid and Large, corresponding to 21 × 21 μm, 42 × 42 μm and 84 × 84 μm FOV respectively. For the mid-scale, we also perform masking, by removing either the neighbor cells or the central cell to determine the important features for prediction.

Extended Data Fig. 3 Generative modeling of ‘synthetic’ trajectories.

For each synthetic trajectory we start by encoding a real image as a starting point. Next, we take a random walk in the latent space. These trajectories in latent space are used as inputs to the TCN. Here, we also use the decoder to generate image sequences that represent the random walks in latent space.

Supplementary information

Supplementary Information

Supplementary Methods, Figs. 1–19 and Tables 1–3.

Reporting Summary

Supplementary Video 1

Time-lapse acquisition and tracking of single cell, showing three different spatial scales extracted to form the glimpse.

Supplementary Video 2

Glimpse extracted from Supplementary Video 1.

Supplementary Video 3

Example cell detection and tracking for MDCKWT:scribkd dataset.

Supplementary Video 4

Example cell detection and tracking for MDCKWT:scribkd,tet− dataset.

Supplementary Video 5

Example cell detection and tracking for MDCKWT:scribkd + 2 μM BIRB 796 dataset.

Supplementary Video 6

Example τ -VAE output for MDCKWT:scribkd dataset.

Supplementary Video 7

Example τ -VAE output for MDCKWT:scribkd,tet− dataset.

Supplementary Video 8

Example τ -VAE output for MDCKWT:scribkd + 2 μM BIRB 796 dataset.

Source data

Source Data Fig. 2

Statistical source data.

Source Data Fig. 3

Statistical source data.

Source Data Fig. 4

Statistical source data.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Soelistyo, C.J., Vallardi, G., Charras, G. et al. Learning biophysical determinants of cell fate with deep neural networks. Nat Mach Intell 4, 636–644 (2022).

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI:


Quick links

Nature Briefing: Translational Research

Sign up for the Nature Briefing: Translational Research newsletter — top stories in biotechnology, drug discovery and pharma.

Get what matters in translational research, free to your inbox weekly. Sign up for Nature Briefing: Translational Research