Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

DeepImageJ: A user-friendly environment to run deep learning models in ImageJ

Nature Methods volume 18, pages 1192–1195 (2021)Cite this article

Subjects

Abstract

DeepImageJ is a user-friendly solution that enables the generic use of pre-trained deep learning models for biomedical image analysis in ImageJ. The deepImageJ environment gives access to the largest bioimage repository of pre-trained deep learning models (BioImage Model Zoo). Hence, nonexperts can easily perform common image processing tasks in life-science research with deep learning-based tools including pixel and object classification, instance segmentation, denoising or virtual staining. DeepImageJ is compatible with existing state of the art solutions and it is equipped with utility tools for developers to include new models. Very recently, several training frameworks have adopted the deepImageJ format to deploy their work in one of the most used softwares in the field (ImageJ). Beyond its direct use, we expect deepImageJ to contribute to the broader dissemination and reuse of deep learning models in life sciences applications and bioimage informatics.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: DeepImageJ environment and scope.
Fig. 2: Functionalities of deepImageJ.

Data availability

All data that were used to generate the figures in this paper are available at https://deepimagej.github.io/deepimagej/.

Code availability

We used TensorFlow and PyTorch libraries for Python to create the in-house models shown in the figures. The deepImageJ plugin can be used in ImageJ and Fiji. The source code for the plugin together with its releases is provided at https://github.com/deepimagej/deepimagej-plugin. The pydeepimagej Python package is provided at https://github.com/deepimagej/pydeepimagej. The ImageJ macro files can be accessed at https://github.com/deepimagej/imagej-macros. All source code is under a BSD 2-Clause License. The web page https://deepimagej.github.io/deepimagej/provides free access to the ImageJ plugin, along with the bundled models and user guide for image processing.

References

  1. Meijering, E. A bird’s-eye view of deep learning in bioimage analysis. Computational Struct. Biotechnol. J. 18, 2312 (2020).

    Article  CAS  Google Scholar 

  2. Moen, E. et al. Deep learning for cellular image analysis. Nat. Methods 16, 1233–1246 (2019).

    Article  CAS  Google Scholar 

  3. Schroeder, A. B. et al. The ImageJ ecosystem: Open-source software for image visualization, processing, and analysis. Protein Sci. 30, 234–249 (2020).

    Article  Google Scholar 

  4. Deep learning gets scope time. Nat. Methods 16, 1195 (2019).

  5. Lucas, A. M. et al. Open-source deep-learning software for bioimage segmentation. Mol. Biol. Cell 32, 823–829 (2021).

    Article  CAS  Google Scholar 

  6. Weigert, M. et al. Content-aware image restoration: pushing the limits of fluorescence microscopy. Nat. Methods 15, 1090–1097 (2018).

    Article  CAS  Google Scholar 

  7. Inés, A., Domínguez, C., Heras, J., Mata, E. & Pascual, V. DeepClas4Bio: Connecting bioimaging tools with deep learning frameworks for image classification. Computers Biol. Med. 108, 49–56 (2019).

    Article  Google Scholar 

  8. Berg, S. et al. Ilastik: interactive machine learning for (bio)image analysis. Nat. Methods 16, 1226–1232 (2019).

    Article  CAS  Google Scholar 

  9. Ouyang, W., Mueller, F., Hjelmare, M., Lundberg, E. & Zimmer, C. ImJoy: an open-source computational platform for the deep learning era. Nat. Methods 16, 1199–1200 (2019).

    Article  CAS  Google Scholar 

  10. von Chamier, L. et al. Democratising deep learning for microscopy with ZeroCostDL4Mic. Nat. Commun. 12, 2276 (2021).

    Article  Google Scholar 

  11. Fäßler, F. et al. Cryo-electron tomography workflows for quantitative analysis of actin networks involved in cell migration. Microsc. Microanalysis 26, 2518–2519 (2020).

    Article  Google Scholar 

  12. Midtvedt, B. et al. Quantitative digital microscopy with deep learning. Appl. Phys. Rev. 8, 011310 (2021).

    Article  CAS  Google Scholar 

  13. Schneider, C. A., Rasband, W. S. & Eliceiri, K. W. NIH Image to ImageJ: 25 years of image analysis. Nat. Methods 9, 671–675 (2012).

    Article  CAS  Google Scholar 

  14. Schmidt, U., Weigert, M., Broaddus, C. & Myers, G. Cell detection with star-convex polygons. In Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 – 21st International Conference, Granada, Spain, September 16–20, 2018, Proceedings, Part II 265–273 (Springer, 2018).

  15. Rivenson, Y. et al. Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning. Nat. Biomed. Eng. 3, 466–477 (2019).

    Article  CAS  Google Scholar 

  16. Falk, T. et al. U-Net: deep learning for cell counting, detection, and morphometry. Nat. Methods 16, 67–70 (2019).

    Article  CAS  Google Scholar 

  17. Nehme, E., Weiss, L. E., Michaeli, T. & Shechtman, Y. Deep-STORM: super-resolution single-molecule microscopy by deep learning. Optica 5, 458–464 (2018).

    Article  CAS  Google Scholar 

  18. Caicedo, J. C. et al. Nucleus segmentation across imaging experiments: the 2018 data science bowl. Nat. Methods 16, 1247–1253 (2019).

    Article  CAS  Google Scholar 

  19. Gómez-de-Mariscal, E. et al. Deep-learning-based segmentation of small extracellular vesicles in transmission electron microscopy images. Sci. Rep. 9, 13211 (2019).

    Article  Google Scholar 

  20. Tsai, H.-F., Gajda, J., F.W. Sloan, T., Rares, A. & Shen, A. Q. Usiigaci: Instance-aware cell tracking in stain-free phase contrast microscopy enabled by machine learning. SoftwareX 9, 230–237 (2019).

    Article  Google Scholar 

  21. Gómez-de-Mariscal, E., Franco, D., Muñoz-Barrutia, A. & Arganda-Carreras, I. in Bioimage Analysis Components and Workflows (eds Sladoje, N. & Miura, K.) (Springer, 2021).

  22. Schindelin, J. et al. Fiji: an open-source platform for biological-image analysis. Nat. Methods 9, 676–682 (2012).

    Article  CAS  Google Scholar 

  23. Rueden, C. T. et al. ImageJ2: ImageJ for the next generation of scientific image data. BMC Bioinformatics 18, 529 (2017).

    Article  Google Scholar 

Download references

Acknowledgements

We thank P. M. Gordaliza, I. Arganda-Carreras (tested the beta-versions), D. F. González-Obando (tested the beta-versions), C. Rueden, S. Tosi, T. Pengo (tested the beta-versions), R. Henriques (ZeroCostDL4Mic), R. F. Laine (ZeroCostDL4Mic), G. Jacquemet (ZeroCostDL4Mic), D. Krentzel (ZeroCostDL4Mic) and C. Möhl (YAPIC) for the fruitful discussions and enriching feedback about the deepImageJ project. We would also like to thank NEUBIAS for supporting the project, the NEUBIAS symposium and NEUBIAS Academy@Home and P. Rasti and S. Bollmann for including deepImageJ in their tutorials. We would like specially to mention all the contributors and community partners at the BioImage Model Zoo for the time they have spent to get a cross-compatible model format. We acknowledge the support of Ministerio de Ciencia, Innovación y Universidades, Agencia Estatal de Investigación, under grant nos TEC2016-78052-R and PID2019-109820RB-I00, MINECO/FEDER, UE, co-financed by European Regional Development Fund (ERDF), ‘A way of making Europe’ (E.G.M., C.G.L.H., A.M.B.), and a 2017 Leonardo Grant for Researchers and Cultural Creators, BBVA Foundation (A.M.B.). This work was also supported by the EPFL Center for Imaging (C.G.L.H., L.D., D.S., M.U.). We would like to thank the Science for Life Laboratory, Erling-Persson Foundation and Knut and Alice Wallenberg foundation (grant no. 2018.0172) (W.O., E.L.). We thank the program ‘Short Term Scientific Missions’ of NEUBIAS (network of European bioimage analysts) (E.G.M., C.G.L.H.). We also want to acknowledge the support of NVIDIA Corporation with the donation of the Titan X (Pascal) GPU card used for this research (A.M.B.).

Author information

Author notes

  1. These authors contributed equally: E. Gómez-de-Mariscal, C. García-López-de-Haro.

Authors and Affiliations

  1. Bioengineering and Aerospace Engineering Department, Universidad Carlos III de Madrid and Instituto de Investigación Sanitaria Gregorio Marañón, Madrid, Spain

    Estibaliz Gómez-de-Mariscal, Carlos García-López-de-Haro & Arrate Muñoz-Barrutia

  2. Science for Life Laboratory, School of Engineering Sciences in Chemistry, Biotechnology and Health, KTH – Royal Institute of Technology, Stockholm, Sweden

    Wei Ouyang & Emma Lundberg

  3. EPFL Center for Imaging, École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland

    Laurène Donati

  4. Biomedical Imaging Group and EPFL Center for Imaging, École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland

    Michael Unser & Daniel Sage

Authors
  1. Estibaliz Gómez-de-Mariscal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carlos García-López-de-Haro
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei Ouyang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Laurène Donati
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Emma Lundberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Michael Unser
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Arrate Muñoz-Barrutia
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Daniel Sage
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

E.G.-M. and C.G.-L.-H. contributed to the design of the experimental framework, and reviewed, trained and exported existing image processing methods. C.G.-L.-H. and D.S. developed and implemented the toolbox and worked on the supporting documentation with input from the rest of the authors. C.G.-L.-H. and W.O. built the connection between the toolbox and ImJoy. E.G.-M. wrote the code lines of the supplementary Python notebooks, Python library and ImageJ macros. E.G.M. and W.O. worked on the synchronization with the BioImage Model Zoo. E.G.-M., W.O. and L.D. wrote the manuscript with help from E.L., M.U., A.M.-B. and D.S. E.G.-M., A.M.-B. and D.S. created the website of deepImageJ. M.U., A.M.-B. and D.S. initiated the project. A.M.-B. and D.S. supervised the project. All the authors contributed to the conception of the study, the design of the experimental framework and took part in the literature review. All authors revised the manuscript.

Corresponding authors

Correspondence to Arrate Muñoz-Barrutia or Daniel Sage.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information Rita Strack was the primary editor on this article and managed its editorial process and peer review in collaboration with the rest of the editorial team.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Figs. 1–6, Methods, User guide and Examples.

Reporting Summary

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gómez-de-Mariscal, E., García-López-de-Haro, C., Ouyang, W. et al. DeepImageJ: A user-friendly environment to run deep learning models in ImageJ. Nat Methods 18, 1192–1195 (2021). https://doi.org/10.1038/s41592-021-01262-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41592-021-01262-9

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing