Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Review Article
  • Published:

Bridging live-cell imaging and next-generation cancer treatment

Nature Reviews Cancer volume 23, pages 731–745 (2023)Cite this article

Subjects

Abstract

By providing spatial, molecular and morphological data over time, live-cell imaging can provide a deeper understanding of the cellular and signalling events that determine cancer response to treatment. Understanding this dynamic response has the potential to enhance clinical outcome by identifying biomarkers or actionable targets to improve therapeutic efficacy. Here, we review recent applications of live-cell imaging for uncovering both tumour heterogeneity in treatment response and the mode of action of cancer-targeting drugs. Given the increasing uses of T cell therapies, we discuss the unique opportunity of time-lapse imaging for capturing the interactivity and motility of immunotherapies. Although traditionally limited in the number of molecular features captured, novel developments in multidimensional imaging and multi-omics data integration offer strategies to connect single-cell dynamics to molecular phenotypes. We review the effect of these recent technological advances on our understanding of the cellular dynamics of tumour targeting and discuss their implication for next-generation precision medicine.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Live-cell imaging demonstrates heterogeneity in treatment response and uncovers T cell therapy mechanism of action.
Fig. 2: Live-cell imaging dimensionality augmentation uncovers actionable targets and biomarkers.
Fig. 3: Envisioned opportunities and challenges of live 3D imaging for preclinical and clinical implementation.

Similar content being viewed by others

References

  1. Jerby-Arnon, L. et al. A cancer cell program promotes T cell exclusion and resistance to checkpoint blockade. Cell 175, 984–997.e24 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  2. Tirosh, I. et al. Dissecting the multicellular ecosystem of metastatic melanoma by single-cell RNA-seq. Science 352, 189–196 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Patel, A. P. et al. Single-cell RNA-seq highlights intratumoral heterogeneity in primary glioblastoma. Science 344, 1396–1401 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Marx, V. Method of the year: spatially resolved transcriptomics. Nat. Methods 18, 9–14 (2021).

    Article  CAS  PubMed  Google Scholar 

  5. Pelka, K. et al. Spatially organized multicellular immune hubs in human colorectal cancer. Cell 184, 4734–4752.e20 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Hickey, J. W. et al. Spatial mapping of protein composition and tissue organization: a primer for multiplexed antibody-based imaging. Nat. Methods 19, 284–295 (2022).

    Article  CAS  PubMed  Google Scholar 

  7. van Ineveld, R. L. et al. Revealing the spatio-phenotypic patterning of cells in healthy and tumor tissues with mLSR-3D and STAPL-3D. Nat. Biotechnol. 39, 1239–1245 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  8. Entenberg, D., Oktay, M. H. & Condeelis, J. S. Intravital imaging to study cancer progression and metastasis. Nat. Rev. Cancer 23, 25–42 (2023).

    Article  CAS  PubMed  Google Scholar 

  9. Wadman, M. FDA no longer has to require animal testing for new drugs. Science 379, 127–128 (2023).

    Article  CAS  PubMed  Google Scholar 

  10. Drost, J. & Clevers, H. Organoids in cancer research. Nat. Rev. Cancer 18, 407–418 (2018).

    Article  CAS  PubMed  Google Scholar 

  11. Dagogo-Jack, I. & Shaw, A. T. Tumour heterogeneity and resistance to cancer therapies. Nat. Rev. Clin. Oncol. 15, 81–94 (2018).

    Article  CAS  PubMed  Google Scholar 

  12. Vitale, I., Shema, E., Loi, S. & Galluzzi, L. Intratumoral heterogeneity in cancer progression and response to immunotherapy. Nat. Med. 27, 212–224 (2021).

    Article  CAS  PubMed  Google Scholar 

  13. Rios, A. C. & Clevers, H. Imaging organoids: a bright future ahead. Nat. Methods 15, 24–26 (2018).

    Article  CAS  PubMed  Google Scholar 

  14. You, R. et al. Active surveillance characterizes human intratumoral T cell exhaustion. J. Clin. Invest. 131, e144353 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Salmon, H. et al. Matrix architecture defines the preferential localization and migration of T cells into the stroma of human lung tumors. J. Clin. Invest. 122, 899–910 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Irvine, D. J., Maus, M. V., Mooney, D. J. & Wong, W. W. The future of engineered immune cell therapies. Science 378, 853–858 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Stephens, D. J. & Allan, V. J. Light microscopy techniques for live cell imaging. Science 300, 82–86 (2003).

    Article  CAS  PubMed  Google Scholar 

  18. Specht, E. A., Braselmann, E. & Palmer, A. E. A critical and comparative review of fluorescent tools for live cell imaging. Annu. Rev. Physiol. 79, 93–117 (2016).

    Article  PubMed  Google Scholar 

  19. Huang, Q. et al. The frontier of live tissue imaging across space and time. Cell Stem Cell 28, 603–622 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  20. Borlinghaus, R. T. MRT letter: high speed scanning has the potential to increase fluorescence yield and to reduce photobleaching. Microsc. Res. Tech. 69, 689–692 (2006).

    Article  CAS  PubMed  Google Scholar 

  21. Datta, R., Heaster, T. M., Sharick, J. T., Gillette, A. A. & Skala, M. C. Fluorescence lifetime imaging microscopy: fundamentals and advances in instrumentation, analysis, and applications. J. Biomed. Opt. 25, 071203–071203 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Nobis, M. et al. Intravital FLIM-FRET imaging reveals dasatinib-induced spatial control of src in pancreatic cancer. Cancer Res. 73, 4674–4686 (2013).

    Article  CAS  PubMed  Google Scholar 

  23. Pettinato, G. et al. Spectroscopic label-free microscopy of changes in live cell chromatin and biochemical composition in transplantable organoids. Sci. Adv. 7, eabj2800 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. Edlund, C. et al. LIVECell—a large-scale dataset for label-free live cell segmentation. Nat. Methods 18, 1038–1045 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Christiansen, E. M. et al. In silico labeling: predicting fluorescent labels in unlabeled images. Cell 173, 792–803.e19 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Hu, C. et al. Live-dead assay on unlabeled cells using phase imaging with computational specificity. Nat. Commun. 13, 713 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Zaritsky, A. et al. Interpretable deep learning uncovers cellular properties in label-free live cell images that are predictive of highly metastatic melanoma. Cell Syst. 12, 733–747.e6 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. Ko, J. et al. Spatiotemporal multiplexed immunofluorescence imaging of living cells and tissues with bioorthogonal cycling of fluorescent probes. Nat. Biotechnol. 40, 1654–1662 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Ko, J. et al. In vivo click chemistry enables multiplexed intravital microscopy. Adv. Sci. 9, 2200064 (2022).

    Article  CAS  Google Scholar 

  30. Ponsioen, B. et al. Quantifying single-cell ERK dynamics in colorectal cancer organoids reveals EGFR as an amplifier of oncogenic MAPK pathway signalling. Nat. Cell Biol. 23, 377–390 (2021). Using ERK biosensor live-cell imaging in colorectal cancer organoids, Ponsioen et al. revealed a remarkable level of spatiotemporal heterogeneity in response to inhibitor treatment, with a high variation in ERK reactivation peaks between neighbouring cells.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Ohta, Y. et al. Cell–matrix interface regulates dormancy in human colon cancer stem cells. Nature 608, 784–794 (2022). Intravital imaging of a fluorescent Rainbow lineage tracing reporter in subcutaneously engrafted colorectal cancer organoids identified a dormant tumour cell population that persists during chemotherapy and leads to tumour outgrowth.

    Article  CAS  PubMed  Google Scholar 

  32. Yan, C. et al. Visualizing engrafted human cancer and therapy responses in immunodeficient zebrafish. Cell 177, 1903–1914.e14 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  33. Kaufman, T. et al. Visual barcodes for clonal-multiplexing of live microscopy-based assays. Nat. Commun. 13, 2725 (2022). These authors applied multiplexed fluorescent reporter imaging to drug screening, demonstrating that drugs targeting different pathways converge in their tumour cell outcome.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. Yang, J.-M. et al. Deciphering cell signaling networks with massively multiplexed biosensor barcoding. Cell 184, 6193–6206.e14 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Kudo, T., Lane, K. & Covert, M. W. A multiplexed epitope barcoding strategy that enables dynamic cellular phenotypic screens. Cell Syst. 13, 376–387.e8 (2022).

    Article  CAS  PubMed  Google Scholar 

  36. Chen, C. et al. Multiplexed live-cell profiling with Raman probes. Nat. Commun. 12, 3405 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  37. Wei, L. et al. Super-multiplex vibrational imaging. Nature 544, 465–470 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  38. Shou, J. et al. Super-multiplex imaging of cellular dynamics and heterogeneity by integrated stimulated Raman and fluorescence microscopy. iscience 24, 102832 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Strack, R. Imaging without the labels. Nat. Methods 19, 30–30 (2022).

    Article  CAS  PubMed  Google Scholar 

  40. Martino, N. et al. Wavelength-encoded laser particles for massively multiplexed cell tagging. Nat. Photonics 13, 720–727 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  41. Bray, M.-A. et al. Cell Painting, a high-content image-based assay for morphological profiling using multiplexed fluorescent dyes. Nat. Protoc. 11, 1757–1774 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  42. Freckmann, E. C. et al. Traject3d allows label-free identification of distinct co-occurring phenotypes within 3D culture by live imaging. Nat. Commun. 13, 5317 (2022). Here, Freckmann et al. identified rare drug resistant cell populations exclusively defined by morpho-dynamic analysis, leading to the proposition of combinatorial drug treatment that reduced the presence of these populations.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  43. Lefebvre, A. E. Y. T., Ma, D., Kessenbrock, K., Lawson, D. A. & Digman, M. A. Automated segmentation and tracking of mitochondria in live-cell time-lapse images. Nat. Methods 18, 1091–1102 (2021).

    Article  CAS  PubMed  Google Scholar 

  44. Longden, J. et al. Deep neural networks identify signaling mechanisms of ErbB-family drug resistance from a continuous cell morphology space. Cell Rep. 34, 108657 (2021).

    Article  CAS  PubMed  Google Scholar 

  45. Wu, Z. et al. DynaMorph: self-supervised learning of morphodynamic states of live cells. Mol. Biol. Cell 33, ar59 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  46. You, L. et al. Linking the genotypes and phenotypes of cancer cells in heterogenous populations via real-time optical tagging and image analysis. Nat. Biomed. Eng. 6, 667–675 (2022).

    Article  CAS  PubMed  Google Scholar 

  47. Driscoll, M. K. et al. Robust and automated detection of subcellular morphological motifs in 3D microscopy images. Nat. Methods 16, 1037–1044 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  48. Eddy, C. Z. et al. Morphodynamics facilitate cancer cells to navigate 3D extracellular matrix. Sci. Rep.-uk 11, 20434 (2021).

    Article  CAS  Google Scholar 

  49. Crainiciuc, G. et al. Behavioural immune landscapes of inflammation. Nature 601, 415–421 (2022). In this work, the authors identified functional immune cell subsets solely based on morpho-dynamic profiling, leading to the discovery of a new target that could specifically prevent unwanted cell behaviour instead of ablating overall function.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  50. Davenport, A. J. et al. CAR-T cells inflict sequential killing of multiple tumor target cells. Cancer Immunol. Res. 3, 483–494 (2015).

    Article  CAS  PubMed  Google Scholar 

  51. Bandey, I. N. et al. Designed improvement to T-cell immunotherapy by multidimensional single cell profiling. J. Immunother. Cancer 9, e001877 (2021). Live-cell imaging and molecular profiling of CAR-T cell activity uncovered a target that could be exploited for next generation therapy design, and in combination with mathematical modelling demonstrated that targeting inefficacy can arise from various steps in the lysosome polarization and degranulation process.

    Article  PubMed  PubMed Central  Google Scholar 

  52. Dekkers, J. F. et al. Uncovering the mode of action of engineered T cells in patient cancer organoids. Nat. Biotechnol. 41, 60–69 (2022). Live-cell imaging of patient-derived tumour organoid and T cell therapy co-cultures integrated with transcriptomic profiling highlighted the dynamic and molecular signature of serial killing.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Romain, G. et al. Multidimensional single-cell analysis identifies a role for CD2-CD58 interactions in clinical antitumor T cell responses. J. Clin. Invest. 132, e159402 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  54. Larson, R. C. et al. CAR T cell killing requires the IFNγR pathway in solid but not liquid tumours. Nature 604, 563–570 (2022).

    Article  CAS  PubMed  Google Scholar 

  55. Khazen, R. et al. Functional heterogeneity of cytotoxic T cells and tumor resistance to cytotoxic hits limit anti‐tumor activity in vivo. EMBO J. 40, e106658 (2021). Here, Khazen et al. observed using live-cell imaging that most T cell anti-tumour responses in vitro and in vivo led to reversible damage instead of tumour cell death.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  56. Weigelin, B. et al. Cytotoxic T cells are able to efficiently eliminate cancer cells by additive cytotoxicity. Nat. Commun. 12, 5217 (2021). Weigelin et al. identified additive cytotoxicity as a killing mechanism of cytotoxic T cells to promote tumour death.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  57. Ronteix, G. et al. High resolution microfluidic assay and probabilistic modeling reveal cooperation between T cells in tumor killing. Nat. Commun. 13, 3111 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  58. Hamieh, M. et al. CAR T cell trogocytosis and cooperative killing regulate tumour antigen escape. Nature 568, 112–116 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  59. Cazaux, M. et al. Single-cell imaging of CAR T cell activity in vivo reveals extensive functional and anatomical heterogeneity. J. Exp. Med. 216, 1038–1049 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  60. Mulazzani, M. et al. Long-term in vivo microscopy of CAR T cell dynamics during eradication of CNS lymphoma in mice. Proc. Natl Acad. Sci. USA 116, 24275–24284 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  61. Liu, L., Dai, B., Li, R., Liu, Z. & Zhang, Z. Intravital molecular imaging reveals the restrained capacity of CTLs in the killing of tumor cells in the liver. Theranostics 11, 194–208 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  62. Kantari-Mimoun, C. et al. CAR T-cell entry into tumor islets is a two-step process dependent on IFNγ and ICAM-1. Cancer Immunol. Res. 9, 1425–1438 (2021).

    Article  CAS  PubMed  Google Scholar 

  63. Davenport, A. J. et al. Chimeric antigen receptor T cells form nonclassical and potent immune synapses driving rapid cytotoxicity. Proc. Natl Acad. Sci. USA 115, E2068–E2076 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  64. Yan, C. et al. Single-cell imaging of T cell immunotherapy responses in vivo. J. Exp. Med. 218, e20210314 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  65. Melenhorst, J. J. et al. Decade-long leukaemia remissions with persistence of CD4+ CAR T cells. Nature 602, 503–509 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  66. Turtle, C. J. et al. CD19 CAR–T cells of defined CD4+:CD8+ composition in adult B cell ALL patients. J. Clin. Invest. 126, 2123–2138 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  67. Liadi, I. et al. Individual motile CD4+ T cells can participate in efficient multikilling through conjugation to multiple tumor cells. Cancer Immunol. 3, 473–482 (2015).

    Article  CAS  Google Scholar 

  68. Boulch, M. et al. A cross-talk between CAR T cell subsets and the tumor microenvironment is essential for sustained cytotoxic activity. Sci. Immunol. 6, eabd4344 (2021).

    Article  CAS  PubMed  Google Scholar 

  69. Konishi, Y. et al. Live-cell fret imaging reveals a role of extracellular signal-regulated kinase activity dynamics in thymocyte motility. Iscience 10, 98–113 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  70. Cuny, A. P., Ponti, A., Kündig, T., Rudolf, F. & Stelling, J. Cell region fingerprints enable highly precise single-cell tracking and lineage reconstruction. Nat. Methods 19, 1276–1285 (2022).

    Article  CAS  PubMed  Google Scholar 

  71. Ravindran, S. Five ways deep learning has transformed image analysis. Nature 609, 864–866 (2022).

    Article  CAS  PubMed  Google Scholar 

  72. Soelistyo, C. J., Vallardi, G., Charras, G. & Lowe, A. R. Learning biophysical determinants of cell fate with deep neural networks. Nat. Mach. Intell. 4, 636–644 (2022). The application of an explainable deep learning model to intravital time-lapse imaging data uncovered how microenvironmental cues contribute to cellular death or proliferation.

    Article  Google Scholar 

  73. Burger, G. A., van de Water, B., Dévédec, S. E. L. & Beltman, J. B. Density-dependent migration characteristics of cancer cells driven by pseudopod interaction. Front. Cell Dev. Biol. 10, 854721 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  74. Beck, R. J., Bijker, D. I. & Beltman, J. B. Heterogeneous, delayed-onset killing by multiple-hitting T cells: stochastic simulations to assess methods for analysis of imaging data. PLoS Comput. Biol. 16, e1007972 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  75. Han, H., Wu, G., Li, Y. & Zi, Z. eDetect: a fast error detection and correction tool for live cell imaging data analysis. iScience 13, 1–8 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  76. Matthews, J. M. et al. OrganoID: a versatile deep learning platform for tracking and analysis of single-organoid dynamics. PLoS Comput. Biol. 18, e1010584 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  77. Deben, C. et al. OrBITS: label-free and time-lapse monitoring of patient derived organoids for advanced drug screening. Cell Oncol. 46, 299–314 (2023).

    Article  CAS  Google Scholar 

  78. Buck, M. D. et al. Mitochondrial dynamics controls t cell fate through metabolic programming. Cell 166, 63–76 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  79. Lisci, M. et al. Mitochondrial translation is required for sustained killing by cytotoxic T cells. Science 374, eabe9977 (2021).

    Article  CAS  PubMed  Google Scholar 

  80. Nguyen, P. et al. Unsupervised discovery of dynamic cell phenotypic states from transmitted light movies. PLoS Comput. Biol. 17, e1009626 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  81. Sekh, A. A. et al. Physics-based machine learning for subcellular segmentation in living cells. Nat. Mach. Intell. 3, 1071–1080 (2021).

    Article  Google Scholar 

  82. Berg, S. et al. ilastik: interactive machine learning for (bio)image analysis. Nat. Methods 16, 1226–1232 (2019).

    Article  CAS  PubMed  Google Scholar 

  83. Ershov, D. et al. TrackMate 7: integrating state-of-the-art segmentation algorithms into tracking pipelines. Nat. Methods 19, 829–832 (2022).

    Article  CAS  PubMed  Google Scholar 

  84. Gómez-de-Mariscal, E. et al. DeepImageJ: a user-friendly environment to run deep learning models in ImageJ. Nat. Methods 18, 1192–1195 (2021).

    Article  PubMed  Google Scholar 

  85. Belevich, I. & Jokitalo, E. DeepMIB: user-friendly and open-source software for training of deep learning network for biological image segmentation. PLoS Comput. Biol. 17, e1008374 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  86. Chamier, Lvon et al. Democratising deep learning for microscopy with ZeroCostDL4Mic. Nat. Commun. 12, 2276 (2021). This is an exemplary publication of efforts to develop user-friendly platforms for code-free implementation of various deep learning models for imaging data.

    Article  Google Scholar 

  87. Medeiros, Gde et al. Multiscale light-sheet organoid imaging framework. Nat. Commun. 13, 4864 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  88. Konen, J. et al. Image-guided genomics of phenotypically heterogeneous populations reveals vascular signalling during symbiotic collective cancer invasion. Nat. Commun. 8, 15078 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  89. Summerbell, E. R. et al. Epigenetically heterogeneous tumor cells direct collective invasion through filopodia-driven fibronectin micropatterning. Sci. Adv. 6, eaaz6197 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  90. Haase, C. et al. Image-seq: spatially resolved single-cell sequencing guided by in situ and in vivo imaging. Nat. Methods 19, 1622–1633 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  91. Kwok, S. J., Montoro, D. T., Haber, A. L., Yun, S.-H. & Vinarsky, V. Single-cell transcriptomics of a dynamic cell behavior in murine airways. Elife 12, e76645 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  92. Chen, W. et al. Live-seq enables temporal transcriptomic recording of single cells. Nature 608, 733–740 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  93. Hu, K. H. et al. ZipSeq: barcoding for real-time mapping of single cell transcriptomes. Nat. Methods 17, 833–843 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  94. Lane, K. et al. Measuring signaling and RNA-Seq in the same cell links gene expression to dynamic patterns of NF-κB activation. Cell Syst. 4, 458–469.e5 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  95. Yuan, J., Sheng, J. & Sims, P. A. SCOPE-Seq: a scalable technology for linking live cell imaging and single-cell RNA sequencing. Genome Biol. 19, 227 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  96. Genshaft, A. S. et al. Live cell tagging tracking and isolation for spatial transcriptomics using photoactivatable cell dyes. Nat. Commun. 12, 4995 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  97. van der Leun, A. M. et al. Single-cell analysis of regions of interest (SCARI) using a photosensitive tag. Nat. Chem. Biol. 17, 1139–1147 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  98. Medaglia, C. et al. Spatial reconstruction of immune niches by combining photoactivatable reporters and scRNA-seq. Science 358, 1622–1626 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  99. Flores, B. C. T. et al. Microfluidic live tracking and transcriptomics of cancer-immune cell doublets link intercellular proximity and gene regulation. Commun. Biol. 5, 1231 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  100. Su, P.-R. et al. Microscopy-based single-cell proteomic profiling reveals heterogeneity in DNA damage response dynamics. Cell Rep. Methods 2, 100237 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  101. Liu, Z. et al. Integrating single-cell RNA-seq and imaging with SCOPE-seq2. Sci. Rep.-uk 10, 19482 (2020).

    Article  CAS  Google Scholar 

  102. Daniel, P. T. et al. CD95/Fas-triggered apoptosis of activated T lymphocytes is prevented by dendritic cells through a CD58-dependent mechanism. Exp. Hematol. 27, 1402–1408 (1999).

    Article  CAS  PubMed  Google Scholar 

  103. Mahecic, D. et al. Event-driven acquisition for content-enriched microscopy. Nat. Methods 19, 1262–1267 (2022). Implementation of smart microscopy by on-the-fly analysing fluorescence and morphological parameters to detect events of interest and trigger a change in microscope acquisition speed.

    Article  CAS  PubMed  Google Scholar 

  104. Alvelid, J., Damenti, M., Sgattoni, C. & Testa, I. Event-triggered STED imaging. Nat. Methods 19, 1268–1275 (2022). 

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  105. Lan, T.-H., He, L., Huang, Y. & Zhou, Y. Optogenetics for transcriptional programming and genetic engineering. Trends Genet. 38, 1253–1270 (2022).

    Article  CAS  PubMed  Google Scholar 

  106. Havrylyuk, D., Hachey, A. C., Fenton, A., Heidary, D. K. & Glazer, E. C. Ru(II) photocages enable precise control over enzyme activity with red light. Nat. Commun. 13, 3636 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  107. Wu, Y. et al. Wireless multi-lateral optofluidic microsystems for real-time programmable optogenetics and photopharmacology. Nat. Commun. 13, 5571 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  108. Wong, C. H., Siah, K. W. & Lo, A. W. Estimation of clinical trial success rates and related parameters. Biostatistics 20, 273–286 (2019).

    Article  PubMed  Google Scholar 

  109. Morgan, P. et al. Impact of a five-dimensional framework on R&D productivity at AstraZeneca. Nat. Rev. Drug. Discov. 17, 167–181 (2018).

    Article  CAS  PubMed  Google Scholar 

  110. Kramer, B. A., del Castillo, J. S. & Pelkmans, L. Multimodal perception links cellular state to decision making in single cells. Science 377, 642–648 (2022).

    Article  CAS  PubMed  Google Scholar 

  111. Zhang, J. et al. Spiking transformers for event-based single object tracking. 2022 IEEE/CVF Conf. Comput. Vis. Pattern Recognit. Cvpr 8791–8800 (IEEE, 2022).

  112. Sharir, G., Noy, A. & Zelnik-Manor, L. An image is worth 16x16 words, what is a video worth? Arxiv https://doi.org/10.48550/arxiv.2103.13915 (2021).

    Article  Google Scholar 

  113. Pineda, J. et al. Geometric deep learning reveals the spatiotemporal features of microscopic motion. Nat. Mach. Intell. 5, 71–82 (2023).

    Article  Google Scholar 

  114. Shi, Y., Daugird, T. A. & Legant, W. R. A quantitative analysis of various patterns applied in lattice light sheet microscopy. Nat. Commun. 13, 4607 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  115. Voleti, V. et al. Real-time volumetric microscopy of in vivo dynamics and large-scale samples with SCAPE 2.0. Nat. Methods 16, 1054–1062 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  116. Delgado-Rodriguez, P., Brooks, C. J., Vaquero, J. J. & Muñoz-Barrutia, A. Innovations in ex vivo light sheet fluorescence microscopy. Prog. Biophysics Mol. Biol. 168, 37–51 (2022).

    Article  CAS  Google Scholar 

  117. Marquart, J., Chen, E. Y. & Prasad, V. Estimation of the percentage of US patients with cancer who benefit from genome-driven oncology. JAMA Oncol. 4, 1093 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  118. Ding, S. et al. Patient-derived micro-organospheres enable clinical precision oncology. Cell Stem Cell 29, 905–917.e6 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  119. Voabil, P. et al. An ex vivo tumor fragment platform to dissect response to PD-1 blockade in cancer. Nat. Med. 27, 1250–1261 (2021).

    Article  CAS  PubMed  Google Scholar 

  120. Patel, K. B. et al. High-speed light-sheet microscopy for the in-situ acquisition of volumetric histological images of living tissue. Nat. Biomed. Eng. 6, 569–583 (2022). The development of a high-speed, label-free intravital imaging technique (MediSCAPE) that, owing to its real-time and non-invasive nature, was applied in a healthy volunteer.

    Article  PubMed  PubMed Central  Google Scholar 

  121. Ao, Z. et al. Microfluidics guided by deep learning for cancer immunotherapy screening. Proc. Natl Acad. Sci. USA 119, e2214569119 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  122. Nguyen, M. et al. Dissecting effects of anti-cancer drugs and cancer-associated fibroblasts by on-chip reconstitution of immunocompetent tumor microenvironments. Cell Rep. 25, 3884–3893.e3 (2018).

    Article  CAS  PubMed  Google Scholar 

  123. Schnalzger, T. E. et al. 3D model for CAR‐mediated cytotoxicity using patient‐derived colorectal cancer organoids. EMBO J. 38, e100928 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  124. Ao, Z. et al. Evaluation of cancer immunotherapy using mini-tumor chips. Theranostics 12, 3628–3636 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  125. Clevers, H. Modeling development and disease with organoids. Cell 165, 1586–1597 (2016).

    Article  CAS  PubMed  Google Scholar 

  126. Sachs, N. et al. A living biobank of breast cancer organoids captures disease heterogeneity. Cell 172, 373–386.e10 (2018).

    Article  CAS  PubMed  Google Scholar 

  127. Vlachogiannis, G. et al. Patient-derived organoids model treatment response of metastatic gastrointestinal cancers. Science 359, 920–926 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  128. Ooft, S. N. et al. Patient-derived organoids can predict response to chemotherapy in metastatic colorectal cancer patients. Sci. Transl. Med 11, eaay2574 (2019).

    Article  CAS  PubMed  Google Scholar 

  129. Yao, Y. et al. Patient-derived organoids predict chemoradiation responses of locally advanced rectal cancer. Cell Stem Cell 26, 17–26.e6 (2020).

    Article  CAS  PubMed  Google Scholar 

  130. Betge, J. et al. The drug-induced phenotypic landscape of colorectal cancer organoids. Nat. Commun. 13, 3135 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  131. Lukonin, I. et al. Phenotypic landscape of intestinal organoid regeneration. Nature 586, 275–280 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  132. Spiller, E. R. et al. Imaging-based machine learning analysis of patient-derived tumor organoid drug response. Front. Oncol. 11, 771173 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  133. Artegiani, B. et al. Fast and efficient generation of knock-in human organoids using homology-independent CRISPR–Cas9 precision genome editing. Nat. Cell Biol. 22, 321–331 (2020).

    Article  CAS  PubMed  Google Scholar 

  134. Kersten, K., Visser, K. E., Miltenburg, M. H. & Jonkers, J. Genetically engineered mouse models in oncology research and cancer medicine. Embo Mol. Med. 9, 137–153 (2017).

    Article  CAS  PubMed  Google Scholar 

  135. Arlauckas, S. P. et al. In vivo imaging reveals a tumor-associated macrophage–mediated resistance pathway in anti–PD-1 therapy. Sci. Transl. Med 9, eaal3604 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  136. Manning, C. S., Hooper, S. & Sahai, E. A. Intravital imaging of SRF and Notch signalling identifies a key role for EZH2 in invasive melanoma cells. Oncogene 34, 4320–4332 (2015).

    Article  CAS  PubMed  Google Scholar 

  137. Muta, Y. et al. Composite regulation of ERK activity dynamics underlying tumour-specific traits in the intestine. Nat. Commun. 9, 2174 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  138. Asrir, A. et al. Tumor-associated high endothelial venules mediate lymphocyte entry into tumors and predict response to PD-1 plus CTLA-4 combination immunotherapy. Cancer Cell 40, 318–334.e9 (2022).

    Article  CAS  PubMed  Google Scholar 

  139. Wang, Z. et al. Rapid tissue prototyping with micro-organospheres. Stem Cell Rep. 17, 1959–1975 (2022).

    Article  CAS  Google Scholar 

  140. Wen, C. et al. 3DeeCellTracker, a deep learning-based pipeline for segmenting and tracking cells in 3D time lapse images. Elife 10, e59187 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  141. Robitaille, M. C., Byers, J. M., Christodoulides, J. A. & Raphael, M. P. Self-supervised machine learning for live cell imagery segmentation. Commun. Biol. 5, 1162 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  142. Ulicna, K., Vallardi, G., Charras, G. & Lowe, A. R. Automated deep lineage tree analysis using a bayesian single cell tracking approach. Front. Comput. Sci. 3, 734559 (2021).

    Article  Google Scholar 

  143. Sugawara, K., Çevrim, Ç. & Averof, M. Tracking cell lineages in 3D by incremental deep learning. Elife 11, e69380 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  144. Qiao, C. et al. Rationalized deep learning super-resolution microscopy for sustained live imaging of rapid subcellular processes. Nat. Biotechnol. 41, 367–377 (2022).

    Article  PubMed  Google Scholar 

  145. Cascarano, P. et al. Recursive deep prior video: a super resolution algorithm for time-lapse microscopy of organ-on-chip experiments. Med. Image Anal. 72, 102124 (2021).

    Article  PubMed  Google Scholar 

  146. Zhou, Z. et al. Joint multi-frame detection and segmentation for multi-cell tracking. Arxiv https://doi.org/10.48550/arxiv.1906.10886 (2019).

  147. Hayashida, J., Nishimura, K. & Bise, R. MPM: joint representation of motion and position map for cell tracking. Arxiv https://doi.org/10.48550/arxiv.2002.10749 (2020).

  148. Sità, L. et al. A deep-learning approach for online cell identification and trace extraction in functional two-photon calcium imaging. Nat. Commun. 13, 1529 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  149. Payer, C., Štern, D., Feiner, M., Bischof, H. & Urschler, M. Segmenting and tracking cell instances with cosine embeddings and recurrent hourglass networks. Med. Image Anal. 57, 106–119 (2019).

    Article  PubMed  Google Scholar 

  150. Ottl, S., Amiriparian, S., Gerczuk, M. & Schuller, B. W. motilitAI: a machine learning framework for automatic prediction of human sperm motility. iscience 25, 104644 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  151. Kimmel, J. C., Brack, A. S. & Marshall, W. F. Deep convolutional and recurrent neural networks for cell motility discrimination and prediction. IEEE/ACM Trans. Comput. Biol. Bioinform 18, 562–574 (2018).

    Article  Google Scholar 

  152. Buggenthin, F. et al. Prospective identification of hematopoietic lineage choice by deep learning. Nat. Methods 14, 403–406 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  153. Moen, E. et al. Accurate cell tracking and lineage construction in live-cell imaging experiments with deep learning. Biorxiv 803205 https://doi.org/10.1101/803205 (2019).

  154. Li, X. et al. Real-time denoising enables high-sensitivity fluorescence time-lapse imaging beyond the shot-noise limit. Nat Biotechnol 41, 282–292 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  155. Mencattini, A. et al. Discovering the hidden messages within cell trajectories using a deep learning approach for in vitro evaluation of cancer drug treatments. Sci. Rep.-uk 10, 7653 (2020).

    Article  CAS  Google Scholar 

  156. Chudakov, D. M., Lukyanov, S. & Lukyanov, K. A. Tracking intracellular protein movements using photoswitchable fluorescent proteins PS-CFP2 and Dendra2. Nat. Protoc. 2, 2024–2032 (2007).

    Article  CAS  PubMed  Google Scholar 

  157. Ando, R., Hama, H., Yamamoto-Hino, M., Mizuno, H. & Miyawaki, A. An optical marker based on the UV-induced green-to-red photoconversion of a fluorescent protein. Proc. Natl Acad. Sci. USA 99, 12651–12656 (2002).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  158. Vahabi, N. & Michailidis, G. Unsupervised multi-omics data integration methods: a comprehensive review. Front. Genet. 13, 854752 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

A.C.R. is supported by a European Research Council (ERC) starting grant (no. 804412).

Author information

Author notes

  1. These authors contributed equally: Maria Alieva, Amber K. L. Wezenaar.

Authors and Affiliations

  1. Princess Máxima Center for Pediatric Oncology, Utrecht, The Netherlands

    Maria Alieva, Amber K. L. Wezenaar, Ellen J. Wehrens & Anne C. Rios

  2. Instituto de Investigaciones Biomedicas Sols-Morreale (IIBM), CSIC-UAM, Madrid, Spain

    Maria Alieva

  3. Oncode Institute, Utrecht, The Netherlands

    Amber K. L. Wezenaar, Ellen J. Wehrens & Anne C. Rios

Authors
  1. Maria Alieva
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amber K. L. Wezenaar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ellen J. Wehrens
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anne C. Rios
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The authors contributed equally to all aspects of the article.

Corresponding authors

Correspondence to Ellen J. Wehrens or Anne C. Rios.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Reviews Cancer thanks Jakob Kather, Jan Grimm and David Entenberg for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Glossary

Behaviour-guided transcriptomics

Using live-cell imaging to select cells with behaviour of interest for subsequent sequencing.

Computer vision

A subfield of artificial intelligence dedicated to interpreting images and videos.

High-numerical aperture objectives

A microscope objective equipped with a lens featuring a wide opening angle, enabling enhanced light collection and improved imaging resolution.

Lattice light-sheet microscopy

Light-sheet microscope technique that uses ultrathin light sheets derived from 2D optical lattices to provide fast and high axial resolution.

Light-sheet microscopy

Fluorescence imaging technique that uses a thin sheet of laser light to illuminate samples, enabling fast 3D imaging with reduced phototoxicity and high imaging speed.

Multiphoton microscopy

Fluorescence microscope technique that uses the simultaneous absorption of multiple long wave photons to excite fluorescent molecules, allowing for deep tissue imaging with reduced photodamage, increased penetration depth and resolution.

On-the-fly analysis

Real-time data analysis during microscope acquisition.

Optogenetics

Genetic engineering to allow light-driven tagging, perturbation or functional manipulation of cells.

Photopharmacology

Light-based induction of drug activity.

Phototoxicity

Cellular damage caused by prolonged or intense laser illumination during imaging of living cells.

Raman spectroscopy

An imaging technique that utilizes light scattering to provide information on the molecular and chemical properties of structures on the basis of their vibrational and rotational characteristics. It can also be combined with Raman probes for multiplexed imaging.

Resonant scanners

A type of galvanometric mirror scanner that allows fast image acquisition with single-point scanning microscopes.

Smart microscopy

A microscopy technique using computer vision for the on-the-spot image analysis providing real-time feedback to improve the resolution of intended parameters during imaging acquisition.

Swept confocally aligned planar excitation

Single-objective light-sheet imaging technology suitable for high-speed volumetric imaging.

T cell serial killing

Tumour targeting process in which cytotoxic T cells kill multiple tumour cells in sequence.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alieva, M., Wezenaar, A.K.L., Wehrens, E.J. et al. Bridging live-cell imaging and next-generation cancer treatment. Nat Rev Cancer 23, 731–745 (2023). https://doi.org/10.1038/s41568-023-00610-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41568-023-00610-5

Search

Advanced search

Quick links

Nature Briefing: Cancer

Sign up for the Nature Briefing: Cancer newsletter — what matters in cancer research, free to your inbox weekly.

Get what matters in cancer research, free to your inbox weekly. Sign up for Nature Briefing: Cancer