Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Smart lattice light-sheet microscopy for imaging rare and complex cellular events

Nature Methods volume 21, pages 301–310 (2024)Cite this article

Subjects

Abstract

Light-sheet microscopes enable rapid high-resolution imaging of biological specimens; however, biological processes span spatiotemporal scales. Moreover, long-term phenotypes are often instigated by rare or fleeting biological events that are difficult to capture with a single imaging modality. Here, to overcome this limitation, we present smartLLSM, a microscope that incorporates artificial intelligence-based instrument control to autonomously switch between epifluorescent inverted imaging and lattice light-sheet microscopy (LLSM). We apply this approach to two unique processes: cell division and immune synapse formation. In each context, smartLLSM provides population-level statistics across thousands of cells and autonomously captures multicolor three-dimensional datasets or four-dimensional time-lapse movies of rare events at rates that dramatically exceed human capabilities. From this, we quantify the effects of Taxol dose on spindle structure and kinetochore dynamics in dividing cells and of antigen strength on cytotoxic T lymphocyte engagement and lytic granule polarization at the immune synapse. Overall, smartLLSM efficiently detects rare events within heterogeneous cell populations and records these processes with high spatiotemporal four-dimensional imaging over statistically significant replicates.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: SmartLLSM training and performance.
Fig. 2: smartLLSM workflow.
Fig. 3: Population statistics of Taxol-induced mitotic defects and the effect of antigen strength on immune synapses.
Fig. 4: High-throughput imaging of Taxol-induced mitotic defects and the effect of antigen strength on immune synapses.
Fig. 5: High-throughput imaging of kinetochore dynamics during mitosis.
Fig. 6: Longitudinal tracking of kinetochore motion during mitosis.

Similar content being viewed by others

Data availability

Due to the inordinate size of the image data (∼40 TB), it is not currently feasible to deposit this into a central repository; however, all datasets underlying the results in this paper are available from the corresponding author upon request. To the extent possible, the authors will try to meet all requests for data sharing within 2 weeks from the original request. Source data are provided with this paper.

Code availability

The source code, annotation GUI, the library of annotated training data and the trained YOLOv5 network generated in the current study are available at https://github.com/legantlab/smartLLSM. Code is provided under The MIT License for open source software, a permissive license approved by the Open Source Initiative. Specific terms can be found at https://opensource.org/licenses/MIT.

References

  1. Power, R. M. & Huisken, J. A guide to light-sheet fluorescence microscopy for multiscale imaging. Nat. Methods 14, 360–373 (2017).

    Article  CAS  PubMed  Google Scholar 

  2. Hobson, C. M. et al. Practical considerations for quantitative light sheet fluorescence microscopy. Nat. Methods 19, 1538–1549 (2022).

    Article  CAS  PubMed  Google Scholar 

  3. Liu, Z. et al. 3D imaging of Sox2 enhancer clusters in embryonic stem cells. eLife 3, e04236 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  4. Gustavsson, A.-K., Petrov, P. N., Lee, M. Y., Shechtman, Y. & Moerner, W. E. 3D single-molecule super-resolution microscopy with a tilted light sheet. Nat. Commun. 9, 123 (2018).

    Article  PubMed  PubMed Central  ADS  Google Scholar 

  5. Legant, W. R. et al. High-density three-dimensional localization microscopy across large volumes. Nat. Methods 13, 359–365 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  6. Chen, B.-C. et al. Lattice light-sheet microscopy: imaging molecules to embryos at high spatiotemporal resolution. Science 346, 1257998 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  7. Chen, J. et al. Single-molecule dynamics of enhanceosome assembly in embryonic stem cells. Cell 156, 1274–1285 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Keller, P. J., Schmidt, A. D., Wittbrodt, J. & Stelzer, E. H. K. Reconstruction of zebrafish early embryonic development by scanned light sheet microscopy. Science 322, 1065–1069 (2008).

    Article  CAS  PubMed  ADS  Google Scholar 

  9. McDole, K. et al. In toto imaging and reconstruction of post-implantation mouse development at the single-cell level. Cell 175, 859–876.e33 (2018).

    Article  CAS  PubMed  Google Scholar 

  10. Huisken, J., Swoger, J., Del Bene, F., Wittbrodt, J. & Stelzer, E. H. K. Optical sectioning deep inside live embryos by selective plane illumination microscopy. Science 305, 1007–1009 (2004).

    Article  CAS  PubMed  ADS  Google Scholar 

  11. Liu, T.-L. et al. Observing the cell in its native state: Imaging subcellular dynamics in multicellular organisms. Science 360, eaaq1392 (2018).

    Article  PubMed  PubMed Central  ADS  Google Scholar 

  12. Reynaud, E. G., Peychl, J., Huisken, J. & Tomancak, P. Guide to light-sheet microscopy for adventurous biologists. Nat. Methods 12, 30–34 (2015).

    Article  CAS  PubMed  Google Scholar 

  13. Laissue, P. P., Alghamdi, R. A., Tomancak, P., Reynaud, E. G. & Shroff, H. Assessing phototoxicity in live fluorescence imaging. Nat. Methods 14, 657–661 (2017).

    Article  CAS  PubMed  Google Scholar 

  14. Sapoznik, E. et al. A versatile oblique plane microscope for large-scale and high-resolution imaging of subcellular dynamics. eLife 9, e57681 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Kumar, A. et al. Dual-view plane illumination microscopy for rapid and spatially isotropic imaging. Nat. Protoc. 9, 2555–2573 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Dean, K. M., Roudot, P., Welf, E. S., Danuser, G. & Fiolka, R. Deconvolution-free subcellular imaging with axially swept light sheet microscopy. Biophys. J. 108, 2807–2815 (2015).

    Article  CAS  PubMed  PubMed Central  ADS  Google Scholar 

  17. Conrad, C. et al. Micropilot: automation of fluorescence microscopy-based imaging for systems biology. Nat. Methods 8, 246–249 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. André, O., Ahnlide, J. K., Norlin, N., Swaminathan, V. & Nordenfelt, P. Data-driven microscopy allows for automated context-specific acquisition of high-fidelity image data. Cell Rep. Methods. 3, 100419 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  19. Barentine, A. E. S. et al. An integrated platform for high-throughput nanoscopy. Nat. Biotechnol. https://doi.org/10.1038/s41587-023-01702-1 (2023).

  20. Mahecic, D. et al. Event-driven acquisition for content-enriched microscopy. Nat. Methods. https://doi.org/10.1038/s41592-022-01589-x (2022).

  21. Alvelid, J., Damenti, M., Sgattoni, C. & Testa, I. Event-triggered STED imaging. Nat. Methods. https://doi.org/10.1038/s41592-022-01588-y (2022).

  22. Bochkovskiy, A., Wang, C.-Y. & Liao, H.-Y. M. YOLOv4: Optimal speed and accuracy of object detection. Preprint at https://doi.org/10.48550/arXiv.2004.10934 (2020).

  23. Redmon, J., Divvala, S., Girshick, R. & Farhadi, A. You only look once: unified, real-time object detection. in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 779–788 https://doi.org/10.1109/CVPR.2016.91 (IEEE, 2016).

  24. Jocher, G. YOLOv5 SOTA realtime instance segmentation. Ultralytics https://doi.org/10.5281/zenodo.3908559 (2020).

  25. Yamashita, N. et al. Three-dimensional tracking of plus-tips by lattice light-sheet microscopy permits the quantification of microtubule growth trajectories within the mitotic apparatus. J. Biomed. Opt. 20, 101206 (2015).

    Article  PubMed  ADS  Google Scholar 

  26. Pamula, M. C. et al. High-resolution imaging reveals how the spindle midzone impacts chromosome movement. J. Cell Biol. 218, 2529–2544 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  27. David, A. F. et al. Augmin accumulation on long-lived microtubules drives amplification and kinetochore-directed growth. J. Cell Biol. 218, 2150–2168 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. Ritter, A. T. et al. Actin depletion initiates events leading to granule secretion at the immunological synapse. Immunity 42, 864–876 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Ikui, A. E., Yang, C.-P. H., Matsumoto, T. & Horwitz, S. B. Low concentrations of taxol cause mitotic delay followed by premature dissociation of p55CDC from Mad2 and BubR1 and abrogation of the spindle checkpoint, leading to aneuploidy. Cell Cycle 4, 1385–1388 (2005).

  30. Brito, D. A. & Rieder, C. L. The ability to survive mitosis in the presence of microtubule poisons differs significantly between human nontransformed (RPE-1) and cancer (U2OS, HeLa) cells. Cell Motil. Cytoskelet. 66, 437–447 (2009).

    Article  CAS  Google Scholar 

  31. Denton, A. E. et al. Affinity thresholds for naive CD8+ CTL activation by peptides and engineered influenza A viruses. J. Immunol. 187, 5733–5744 (2011).

    Article  CAS  PubMed  Google Scholar 

  32. Frazer, G. L., Gawden-Bone, C. M., Dieckmann, N. M. G., Asano, Y. & Griffiths, G. M. Signal strength controls the rate of polarization within CTLs during killing. J. Cell Biol. 220, e202104093 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  33. Yachi, P. P., Ampudia, J., Zal, T. & Gascoigne, N. R. J. Altered peptide ligands induce delayed CD8–T cell receptor interaction—a role for CD8 in distinguishing antigen quality. Immunity 25, 203–211 (2006).

    Article  CAS  PubMed  Google Scholar 

  34. Jenkins, M. R., Tsun, A., Stinchcombe, J. C. & Griffiths, G. M. The strength of T cell receptor signal controls the polarization of cytotoxic machinery to the immunological synapse. Immunity 31, 621–631 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Magidson, V. et al. The spatial arrangement of chromosomes during prometaphase facilitates spindle assembly. Cell 146, 555–567 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  36. Chaly, N. & Brown, D. L. The prometaphase configuration and chromosome order in early mitosis. J. Cell Sci. 91, 325–335 (1988).

    Article  PubMed  Google Scholar 

  37. Sen, O., Harrison, J. U., Burroughs, N. J. & McAinsh, A. D. Kinetochore life histories reveal an Aurora-B-dependent error correction mechanism in anaphase. Dev. Cell https://doi.org/10.1016/j.devcel.2021.10.007 (2021).

  38. Klaasen, S. J. et al. Nuclear chromosome locations dictate segregation error frequencies. Nature https://doi.org/10.1038/s41586-022-04938-0 (2022).

  39. Royer, L. A. et al. Adaptive light-sheet microscopy for long-term, high-resolution imaging in living organisms. Nat. Biotechnol. 34, 1267–1278 (2016).

    Article  CAS  PubMed  Google Scholar 

  40. Eisenstein, M. Smart solutions for automated imaging. Nat. Methods 17, 1075–1079 (2020).

    Article  CAS  PubMed  Google Scholar 

  41. Moore, R. P. et al. A multi-functional microfluidic device compatible with widefield and light sheet microscopy. Lab Chip 22, 136–147 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  42. Ding, S. et al. Efficient transposition of the piggyBac (PB) transposon in mammalian cells and mice. Cell 122, 473–483 (2005).

    Article  CAS  PubMed  Google Scholar 

  43. Yeo, N. C. et al. An enhanced CRISPR repressor for targeted mammalian gene regulation. Nat. Methods 15, 611–616 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  44. Kirk, J. M. et al. Functional classification of long non-coding RNAs by k-mer content. Nat. Genet. 50, 1474–1482 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  45. Shao, L., Milkie, D. & Lambert, T. cudadecon. Zenodo https://doi.org/10.5281/zenodo.7659013 (2023).

  46. Jaqaman, K. et al. Robust single-particle tracking in live-cell time-lapse sequences. Nat. Methods 5, 695–702 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We thank K. Heath, M. Clynes and V. Augoustides for assistance with annotating images. We thank G. Upadhyayula for assistance with the single particle tracking code and T. Kapoor, M. Emanuele and A. Palmer for helpful discussions and feedback on the manuscript. This work was funded in part by grants from the National Institutes of Health (1DP2GM136653) awarded to W.R.L. W.R.L. acknowledges additional support from the Searle Scholars program, the Beckman Young Investigator Program and the Packard Fellowship for Science and Engineering.

Author information

Author notes

  1. These authors contributed equally: Yu Shi, Jimmy S. Tabet.

Authors and Affiliations

  1. Joint Department of Biomedical Engineering, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Yu Shi, Jimmy S. Tabet, Andrea Giovannucci & Wesley R. Legant

  2. Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, USA

    Daniel E. Milkie

  3. Department of Pharmacology, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Timothy A. Daugird & Wesley R. Legant

  4. Department of Biochemistry and Biophysics, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Chelsea Q. Yang

  5. Altos Labs, Redwood City, CA, USA

    Alex T. Ritter

Authors
  1. Yu Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jimmy S. Tabet
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel E. Milkie
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Timothy A. Daugird
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chelsea Q. Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Alex T. Ritter
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Andrea Giovannucci
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Wesley R. Legant
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

W.R.L. conceived the project. A.G. and J.S.T. lead the development of the DL-based cell detection and classification pipeline with input and contribution from Y.S. and W.R.L. D.E.M assisted with integrating the YOLO network together with the microscope control software. T.A.D, C.Q.Y. and A.T.R assisted with sample preparation. Y.S. and W.R.L performed the imaging experiments, analyzed data and wrote the paper with feedback from all authors. W.R.L. supervised and directed the project.

Corresponding authors

Correspondence to Andrea Giovannucci or Wesley R. Legant.

Ethics declarations

Competing interests

W.R.L. and D.E.M. are authors on patents related to lattice light-sheet microscopy and its applications including US Patent numbers: US 11,221,476 B2 and US 10,795,144 B2 issued to W.R.L., D.E.M. and co-authors, and assigned to Howard Hughes Medical Institute. The other authors declare no competing interests.

Peer review

Peer review information

Nature Methods thanks Mathieu Ducros, Ilaria Testa and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Primary Handling Editor: Rita Strack, in collaboration with the Nature Methods team. Peer reviewer reports are available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

List of Supplementary Movies and Captions 1–6, Tables 1–2, Figs. and Captions 1–21, Notes 1–3 and references.

Reporting Summary

Peer Review File

Supplementary Data 1

Source data for supplementary figures.

Supplementary Video 1

Supplementary Movie 1.

Supplementary Video 2

Supplementary Movie 2.

Supplementary Video 3

Supplementary Movie 3.

Supplementary Video 4

Supplementary Movie 4.

Supplementary Video 5

Supplementary Movie 5.

Supplementary Video 6

Supplementary Movie 6.

Source data

Source Data

Source data for all the main figures.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shi, Y., Tabet, J.S., Milkie, D.E. et al. Smart lattice light-sheet microscopy for imaging rare and complex cellular events. Nat Methods 21, 301–310 (2024). https://doi.org/10.1038/s41592-023-02126-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41592-023-02126-0

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing