Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Brief Communication
  • Published:

DeepCell Kiosk: scaling deep learning–enabled cellular image analysis with Kubernetes

Nature Methods volume 18, pages 43–45 (2021)Cite this article

Subjects

A Publisher Correction to this article was published on 13 January 2021

This article has been updated

Abstract

Deep learning is transforming the analysis of biological images, but applying these models to large datasets remains challenging. Here we describe the DeepCell Kiosk, cloud-native software that dynamically scales deep learning workflows to accommodate large imaging datasets. To demonstrate the scalability and affordability of this software, we identified cell nuclei in 106 1-megapixel images in ~5.5 h for ~US$250, with a cost below US$100 achievable depending on cluster configuration. The DeepCell Kiosk can be downloaded at https://github.com/vanvalenlab/kiosk-console; a persistent deployment is available at https://deepcell.org/.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Architecture and performance of the DeepCell Kiosk.

Similar content being viewed by others

Data availability

All data that were used to generate the figures in this paper are available at https://deepcell.org/data and at https://github.com/vanvalenlab/deepcell-tf under the deepcell.datasets module.

Code availability

We used Kubernetes and TensorFlow, along with the scientific computing stack for Python. A persistent deployment of the software described can be accessed at https://deepcell.org/. All source code, including version requirements and explicit usage, is under a modified Apache license and is available at https://github.com/vanvalenlab. Detailed instructions are available at https://deepcell-kiosk.readthedocs.io.

Change history

References

  1. Ouyang, W. et al. Analysis of the Human Protein Atlas Image Classification competition. Nat. Methods 16, 1254–1261 (2019).

    Article  CAS  Google Scholar 

  2. Falk, T. et al. U-Net: deep learning for cell counting, detection, and morphometry. Nat. Methods 16, 67–70 (2019).

    Article  CAS  Google Scholar 

  3. Van Valen, D. A. et al. Deep learning automates the quantitative analysis of individual cells in live-cell imaging experiments. PLoS Comput. Biol. 12, e1005177 (2016).

    Article  Google Scholar 

  4. Schmidt, U., Weigert, M., Broaddus, C. & Myers, G. Cell detection with star-convex polygons. In Medical Image Computing and Computer Assisted Intervention—MICCAI 2018 (eds. Frangi, A. F. et al.) 265–273 (Springer International Publishing, 2018).

  5. Moen, E. et al. Accurate cell tracking and lineage construction in live-cell imaging experiments with deep learning. Preprint at bioRxiv https://doi.org/10.1101/803205 (2019).

  6. Anjum, S. & Gurari, D. CTMC: Cell Tracking with Mitosis Detection dataset challenge. in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR) Workshops 982–983 (2020).

  7. Gómez-de-Mariscal, E. et al. DeepImageJ: a user-friendly plugin to run deep learning models in ImageJ. Preprint at bioRxiv https://doi.org/10.1101/799270 (2019).

  8. Ouyang, W., Mueller, F., Hjelmare, M., Lundberg, E. & Zimmer, C. ImJoy: an open-source computational platform for the deep learning era. Nat. Methods 16, 1199–1200 (2019).

    Article  CAS  Google Scholar 

  9. von Chamier, L. et al. ZeroCostDL4Mic: an open platform to simplify access and use of deep-learning in microscopy. Preprint at bioRxiv https://doi.org/10.1101/2020.03.20.000133 (2020).

  10. McQuin, C. et al. CellProfiler 3.0: next-generation image processing for biology. PLoS Biol. 16, e2005970 (2018).

    Article  Google Scholar 

  11. Haberl, M. G. et al. CDeep3M—Plug-and-Play cloud-based deep learning for image segmentation. Nat. Methods 15, 677–680 (2018).

    Article  CAS  Google Scholar 

  12. Emami Khoonsari, P. et al. Interoperable and scalable data analysis with microservices: applications in metabolomics. Bioinformatics 35, 3752–3760 (2019).

    Article  Google Scholar 

  13. Novella, J. A. et al. Container-based bioinformatics with Pachyderm. Bioinformatics 35, 839–846 (2019).

    Article  CAS  Google Scholar 

  14. Capuccini, M. et al. On-demand virtual research environments using microservices. PeerJ Comput. Sci. 5, e232 (2019).

    Article  Google Scholar 

  15. Peters, K. et al. PhenoMeNal: processing and analysis of metabolomics data in the cloud. GigaScience 8, giy149 (2019).

  16. Schindelin, J. et al. Fiji: an open-source platform for biological-image analysis. Nat. Methods 9, 676–682 (2012).

    Article  CAS  Google Scholar 

  17. Sofroniew, N. et al. napari/napari: 0.3.5. Zenodo https://doi.org/10.5281/zenodo.3900158 (2020).

  18. Altschul, S. F. et al. Gapped BLAST and PSI-BLAST: a new generation of protein database search programs. Nucleic Acids Res. 25, 3389–3402 (1997).

    Article  CAS  Google Scholar 

  19. Keren, L. et al. A structured tumor-immune microenvironment in triple negative breast cancer revealed by multiplexed ion beam imaging. Cell 174, 1373–1387 (2018).

    Article  CAS  Google Scholar 

Download references

Acknowledgements

We thank numerous colleagues including A. Anandkumar, M. Angelo, J. Bois, I. Brown, A. Butkovic, L. Cai, I. Camplisson, M. Covert, M. Elowitz, J. Freeman, C. Frick, L. Geontoro, A. Ho, K. Huang, K. C. Huang, G. Johnson, L. Keren, D. Litovitz, D. Macklin, U. Manor, S. Patel, A. Raj, N. Pelaez Restrepo, C. Pavelchek, S. Shah and M. Thomson for helpful discussions and contributing data. We gratefully acknowledge support from the Shurl and Kay Curci Foundation, the Rita Allen Foundation, the Paul Allen Family Foundation through the Allen Discovery Center at Stanford University, the Rosen Center for Bioengineering at Caltech, Google Research Cloud, Figure 8’s AI For Everyone award, and a subaward from NIH U24-CA224309-01.

Author information

Authors and Affiliations

  1. Department of Computing and Mathematical Sciences, California Institute of Technology, Pasadena, CA, USA

    Dylan Bannon, Enrico Borba & Vibha Vijayakumar

  2. Division of Biology and Bioengineering, California Institute of Technology, Pasadena, CA, USA

    Erick Moen, Morgan Schwartz, Brian Chang, Edward Pao, William Graf & David Van Valen

  3. Department of Chemical and Systems Biology, Stanford University, Stanford, CA, USA

    Takamasa Kudo

  4. Department of Cancer Biology, Stanford University, Stanford, CA, USA

    Noah Greenwald

  5. Cloud Posse, LLC, Pasadena, CA, USA

    Erik Osterman

Authors
  1. Dylan Bannon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Erick Moen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Morgan Schwartz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Enrico Borba
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Takamasa Kudo
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Noah Greenwald
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Vibha Vijayakumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Brian Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Edward Pao
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Erik Osterman
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. William Graf
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. David Van Valen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

D.B., W.G. and D.V.V. conceived the project; D.B., W.G., E.O. and D.V.V. designed the software architecture; D.B., E.O. and W.G. wrote the core components of the software; D.B., E.M., M.S., E.B., V.V., B.C., E.O., W.G. and D.V.V. contributed to the code base; T.K. and E.P. collected data for annotation; E.M., M.S., N.G., D.B., W.G. and D.V.V. wrote documentation; D.B., E.M., W.G. and D.V.V. wrote the paper; D.V.V. supervised the project.

Corresponding author

Correspondence to David Van Valen.

Ethics declarations

Competing interests

The authors have filed a provisional patent for the described work; the software described here is available under a modified Apache license and is free for non-commercial uses.

Additional information

Peer review information Nature Methods thanks Ola Spjuth and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Rita Strack was the primary editor on this article and managed its editorial process and peer review in collaboration with the rest of the editorial team.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Notes 1–8. This Supplemental Information describes in further detail the software architecture of the DeepCell Kiosk, presents all our benchmarking data, and outlines the reasoning behind several of our design choices. Because we use terminology that is common in the cloud computing literature but may be unfamiliar to readers, we have included a glossary of common terms.

Reporting Summary

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bannon, D., Moen, E., Schwartz, M. et al. DeepCell Kiosk: scaling deep learning–enabled cellular image analysis with Kubernetes. Nat Methods 18, 43–45 (2021). https://doi.org/10.1038/s41592-020-01023-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41592-020-01023-0

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing