Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Plantorganelle Hunter is an effective deep-learning-based method for plant organelle phenotyping in electron microscopy

Nature Plants volume 9, pages 1760–1775 (2023)Cite this article

Subjects

Abstract

Accurate delineation of plant cell organelles from electron microscope images is essential for understanding subcellular behaviour and function. Here we develop a deep-learning pipeline, called the organelle segmentation network (OrgSegNet), for pixel-wise segmentation to identify chloroplasts, mitochondria, nuclei and vacuoles. OrgSegNet was evaluated on a large manually annotated dataset collected from 19 plant species and achieved state-of-the-art segmentation performance. We defined three digital traits (shape complexity, electron density and cross-sectional area) to track the quantitative features of individual organelles in 2D images and released an open-source web tool called Plantorganelle Hunter for quantitatively profiling subcellular morphology. In addition, the automatic segmentation method was successfully applied to a serial-sectioning scanning microscope technique to create a 3D cell model that offers unique views of the morphology and distribution of these organelles. The functionalities of Plantorganelle Hunter can be easily operated, which will increase efficiency and productivity for the plant science community, and enhance understanding of subcellular biology.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Overview of Plantorganelle Hunter for segmenting and characterizing organelle morphology.
Fig. 2: Dataset information and OrgSegNet segmentation results.
Fig. 3: Detailed computational comparison of OrgSegNet and other algorithms.
Fig. 4: Representative TEM images segmented by OrgSegNet and other algorithms.
Fig. 5: Visualization of CAMs from different layers of OrgSegNet.
Fig. 6: Application of our AI system for quantitatively evaluating plant chloroplast changes in 2D and 3D views.

Similar content being viewed by others

Data availability

The plant organelle dataset for the current study is available in the Science Data Bank repository at https://cstr.cn/31253.11.sciencedb.01335. Source data are provided with this paper.

Code availability

The code is available on GitHub at https://github.com/yzy0102/OrgSegNet.

References

  1. Kirchhoff, H. Chloroplast ultrastructure in plants. New Phytol. 223, 565–574 (2019).

    Article  PubMed  Google Scholar 

  2. Shimada, T. et al. Plant vacuoles. Annu. Rev. Plant Biol. 69, 123–145 (2018).

    Article  CAS  PubMed  Google Scholar 

  3. Broda, M., Millar, A. H. & Van Aken, O. Mitophagy: a mechanism for plant growth and survival. Trends Plant Sci. 23, 434–450 (2018).

    Article  CAS  PubMed  Google Scholar 

  4. Goswami, R. et al. Mechanical shielding in plant nuclei. Curr. Biol. 30, 2013–2025 (2020).

    Article  CAS  PubMed  Google Scholar 

  5. Healy, S., McMahon, J., Owens, P., Dockery, P. & FitzGerald, U. Threshold-based segmentation of fluorescent and chromogenic images of microglia, astrocytes and oligodendrocytes in FIJI. J. Neurosci. Methods 295, 87–103 (2018).

    Article  PubMed  Google Scholar 

  6. Greenwald, N. F. et al. Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning. Nat. Biotech. 40, 555–565 (2022).

    Article  CAS  Google Scholar 

  7. Stringer, C., Wang, T., Michaelos, M. & Pachitariu, M. Cellpose: a generalist algorithm for cellular segmentation. Nat. Methods 18, 100–106 (2021).

    Article  CAS  PubMed  Google Scholar 

  8. Moen, E. et al. Deep learning for cellular image analysis. Nat. Methods 16, 1233–1246 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. Fetter, K. C., Eberhardt, S., Barclay, R. S., Wing, S. & Keller, S. R. StomataCounter: a neural network for automatic stomata identification and counting. New Phytol. 223, 1671–1681 (2019).

    Article  PubMed  Google Scholar 

  10. Li, S. et al. LeafNet: a tool for segmenting and quantifying stomata and pavement cells. Plant Cell 34, 1171–1188 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  11. Huang, L. et al. Interlaced sparse self-attention for semantic segmentation. Preprint at https://doi.org/10.48550/arXiv.1907.12273 (2019).

  12. Zhao, H., Shi, J., Qi, X., Wang, X. & Jia, J. Pyramid scene parsing network. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 2881–2890 (IEEE, 2017).

  13. Liu, Z. et al. Swin transformer: hierarchical vision transformer using shifted windows. In Proc. IEEE/CVF International Conference on Computer Vision 10012–10022 (IEEE, 2021).

  14. Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F. & Adam, H. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proc. European Conference on Computer Vision (ECCV) 801–818 (Springer, 2018).

  15. Pérez-Pelegrí, M. et al. PSPU-net for automatic short axis cine MRI segmentation of left and right ventricles. In 2020 IEEE 20th International Conference on Bioinformatics and Bioengineering (BIBE) 1048–1053 (IEEE, 2020).

  16. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A. & Torralba, A. Learning deep features for discriminative localization. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 2921–2929 (IEEE, 2016).

  17. Nowak, J. et al. A network-based framework for shape analysis enables accurate characterization of leaf epidermal cells. Nat. Commun. 12, 458 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Graham, L. & Orenstein, J. M. Processing tissue and cells for transmission electron microscopy in diagnostic pathology and research. Nat. Protoc. 2, 2439–2450 (2007).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Liu, Y. et al. Phenotypic characterization and gene mapping of a thermo-sensitive albino leaf mutant tsa1 in rice. Acta Agron. Sin. 42, 1754–1763 (2016).

    Article  Google Scholar 

  20. Zhou, K. et al. Young Leaf Chlorosis 1, a chloroplast-localized gene required for chlorophyll and lutein accumulation during early leaf development in rice. Planta 237, 279–292 (2013).

    Article  CAS  PubMed  Google Scholar 

  21. Oi, T. et al. Three-dimensional ultrastructural change of chloroplasts in rice mesophyll cells responding to salt stress. Ann. Bot. 125, 833–840 (2020).

    Article  CAS  PubMed  Google Scholar 

  22. Cui, Y. et al. A whole-cell electron tomography model of vacuole biogenesis in Arabidopsis root cells. Nat. Plants 5, 95–105 (2019).

    Article  CAS  PubMed  Google Scholar 

  23. Zeng, Y. et al. The plant unique ESCRT component FREE1 regulates autophagosome closure. Nat. Commun. 14, 1768 (2023).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. Liang, Z. et al. Electron tomography of prolamellar bodies and their transformation into grana thylakoids in cryofixed Arabidopsis cotyledons. Plant Cell 34, 3830–3843 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  25. Heinrich, L. et al. Whole-cell organelle segmentation in volume electron microscopy. Nature 599, 141–146 (2021).

    Article  CAS  PubMed  Google Scholar 

  26. Gallusser, B. et al. Deep neural network automated segmentation of cellular structures in volume electron microscopy. J. Cell Biol. 222, e202208005 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  27. 2018 data science bowl. Kaggle https://kaggle.com/competitions/data-science-bowl-2018 (2018).

  28. Kumar, N. et al. A dataset and a technique for generalized nuclear segmentation for computational pathology. IEEE Trans. Med. Imaging 36, 1550–1560 (2017).

    Article  PubMed  Google Scholar 

  29. Ljosa, V., Sokolnicki, K. L. & Carpenter, A. E. Annotated high-throughput microscopy image sets for validation. Nat. Methods 9, 637 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Valen, D. A. V. et al. Deep learning automates the quantitative analysis of individual cells in live-cell imaging experiments. PLoS Comput. Biol. 12, e1005177 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  31. Bannon, D. et al. DeepCell 2.0: automated cloud deployment of deep learning models for large-scale cellular image analysis. Preprint at bioRxiv https://doi.org/10.1101/505032 (2018).

  32. Cutler, K. J. et al. Omnipose: a high-precision morphology-independent solution for bacterial cell segmentation. Nat. Methods 19, 1438–1448 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  33. Hay, E. A. & Parthasarathy, R. Performance of convolutional neural networks for identification of bacteria in 3D microscopy datasets. PLoS Comput. Biol. 14, e1006628 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  34. Tschuchnig, M. E., Oostingh, G. J. & Gadermayr, M. Generative adversarial networks in digital pathology: a survey on trends and future potential. Patterns 1, 100089 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  35. Zechmann, B. Ultrastructure of plastids serves as reliable abiotic and biotic stress marker. PLoS One 14, e0214811 (2019).

  36. Chen, P. & Levy, D. L. Regulation of organelle size and organization during development. Semin. Cell Dev. Biol. 133, 53–64 (2023).

    Article  CAS  PubMed  Google Scholar 

  37. Bernal, M., Ramiro, M. V., Cases, R., Picorel, R. & Yruela, I. Excess copper effect on growth, chloroplast ultrastructure, oxygen-evolution activity and chlorophyll fluorescence in Glycine max cell suspensions. Physiol. Plant. 127, 312–325 (2006).

    Article  CAS  Google Scholar 

  38. Lou, L., Shen, Z. & Li, X. The copper tolerance mechanisms of Elsholtzia haichowensis, a plant from copper-enriched soils. Environ. Exp. Bot. 51, 111–120 (2004).

    Article  CAS  Google Scholar 

  39. Hameed, A. et al. Effects of salinity stress on chloroplast structure and function. Cells 10, 2023 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Wada, K. Labelme: image polygonal annotation with Python. Zenodo https://doi.org/10.5281/zenodo.5711226 (2021).

  41. Zhou, Z., Siddiquee, M. M. R., Tajbakhsh, N. & Liang, J. UNet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans. Med. Imaging 39, 1856–1867 (2020).

    Article  PubMed  Google Scholar 

  42. Ronneberger, O., Fischer, P. & Brox, T. U-Net: convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention 234–241 (Springer, 2015).

  43. Falk, T. et al. U-Net: deep learning for cell counting, detection, and morphometry. Nat. Methods 16, 67–70 (2019).

    Article  CAS  PubMed  Google Scholar 

  44. Wollmann, T. et al. GRUU-Net: integrated convolutional and gated recurrent neural network for cell segmentation. Med. Image Anal. 56, 68–79 (2019).

    Article  CAS  PubMed  Google Scholar 

  45. Lee, J. et al. Deep-learning-based label-free segmentation of cell nuclei in time-lapse refractive index tomograms. IEEE Access 7, 83449–83460 (2019).

    Article  Google Scholar 

  46. Christiansen, E. M. et al. In silico labeling: predicting fluorescent labels in unlabeled images. Cell 173, 792–803.e19 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  47. Gómez-de-Mariscal, E. et al. Deep-learning-based segmentation of small extracellular vesicles in transmission electron microscopy images. Sci. Rep. 9, 13211 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  48. Yang, L. et al. NuSeT: a deep learning tool for reliably separating and analyzing crowded cells. PLoS Comput. Biol. 16, e1008193 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  49. Caicedo, J. C. et al. Evaluation of deep learning strategies for nucleus segmentation in fluorescence images. Cytom. A 95, 952–965 (2019).

    Article  Google Scholar 

  50. Al-Kofahi, Y., Zaltsman, A., Graves, R., Marshall, W. & Rusu, M. A deep learning-based algorithm for 2-D cell segmentation in microscopy images. BMC Bioinformatics 19, 365 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  51. Majurski, M. et al. Cell image segmentation using generative adversarial networks, transfer learning, and augmentations. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops 1114–1122 (IEEE, 2019).

  52. Guay, M. D. et al. Dense cellular segmentation for EM using 2D–3D neural network ensembles. Sci. Rep. 11, 2561 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  53. Lugagne, J.-B., Lin, H. & Dunlop, M. J. DeLTA: automated cell segmentation, tracking, and lineage reconstruction using deep learning. PLoS Comput. Biol. 16, e1007673 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  54. Hoorali, F., Khosravi, H. & Moradi, B. Automatic Bacillus anthracis bacteria detection and segmentation in microscopic images using UNet++. J. Microbiol. Methods 177, 106056 (2020).

    Article  CAS  PubMed  Google Scholar 

  55. He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 770–778 (IEEE, 2016).

  56. Xiao, T., Liu, Y., Zhou, B., Jiang, Y. & Sun, J. Unified perceptual parsing for scene understanding. In Proc. European Conference on Computer Vision (ECCV) 418–434 (Springer, 2018).

  57. He, T. et al. Bag of tricks for image classification with convolutional neural networks. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 558–567 (IEEE, 2019).

  58. Cordts, M. et al. The cityscapes dataset for semantic urban scene understanding. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 3213–3223 (IEEE, 2016).

  59. Cao, Y., Xu, J., Lin, S., Wei, F. & Hu, H. GCNet: Non-local networks meet squeeze-excitation networks and beyond. In Proc. IEEE/CVF International Conference on Computer Vision Workshops 1971–1980 (IEEE, 2019).

  60. Long, J., Shelhamer, E. & Darrell, T. Fully convolutional networks for semantic segmentation. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 3431–3440 (IEEE, 2015).

  61. MMSegmentation Contributors. MMSegmentation: OpenMMLab Semantic Segmentation Toolbox and Benchmark (GitHub, 2020).

  62. Isensee, F., Jaeger, P. F., Kohl, S. A. A., Petersen, J. & Maier-Hein, K. H. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18, 203–211 (2021).

    Article  CAS  PubMed  Google Scholar 

  63. Zechmann, B., Möstl, S. & Zellnig, G. Volumetric 3D reconstruction of plant leaf cells using SEM, ion milling, TEM, and serial sectioning. Planta 255, 118 (2022).

    Article  CAS  PubMed  Google Scholar 

  64. Zechmann, B., Müller, M., Möstl, S. & Zellnig, G. Three-dimensional quantitative imaging of Tobacco mosaic virus and Zucchini yellow mosaic virus induced ultrastructural changes. Protoplasma 258, 1201–1211 (2021).

    Article  CAS  PubMed  Google Scholar 

  65. Guo, J. et al. Three-dimensional analysis of membrane structures associated with tomato spotted wilt virus infection. Plant Cell Environ. 46, 650–664 (2023).

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

This work was supported by the Key R&D Program of Zhejiang (2021C02023 to Y.H. and 2022C02032 to X.F.), the National Natural Science Foundation of China (32071895 to Y.H.) and the Key Laboratory of Traceability for Agricultural Genetically Modified Organisms, Ministry of Agriculture and Rural Affairs, P. R. China (2022KF03 to X.F.).

We thank B.-H. Kang (Centre for Cell and Developmental Biology, Chinese University of Hong Kong), M. Taniguchi and T. Oi (Graduate School of Bioagricultural, Nagoya University, Japan) and K. Yamane (Graduate School of Agriculture, Kindai University, Japan) for their kindly and useful suggestions and valuable data.

Author information

Author notes

  1. These authors contributed equally: Xuping Feng, Zeyu Yu.

Authors and Affiliations

  1. College of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou, China

    Xuping Feng, Zeyu Yu, Hui Fang, Guofeng Yang, Yongqiang Shi & Yong He

  2. College of Life Sciences, Nanjing Agricultural University, Nanjing, China

    Xuping Feng, Liting Chen, Xinran Zhou, Bing Hu, Chun Qin, Gang Hu, Guipei Xing, Boxi Zhao & Feng Liu

  3. The Rural Development Academy & Agricultural Experiment Station, Zhejiang University, Huzhou, China

    Xuping Feng, Zeyu Yu & Guofeng Yang

  4. Huzhou Institute of Zhejiang University, Hangzhou, China

    Hui Fang

  5. Center for Data Science, Zhejiang University, Hangzhou, China

    Hangjin Jiang

  6. Biological Experiment Teaching Center, College of Life Sciences, Nanjing Agricultural University, Nanjing, China

    Bing Hu, Chun Qin, Gang Hu & Guipei Xing

  7. Center of Cryo-Electron Microscopy, Zhejiang University School of Medicine, Hangzhou, China

    Jiansheng Guo

  8. School of Mathematics and Statistics, University of Melbourne, Parkville, Australia

    Feng Liu

  9. Department of Computer Science, Hong Kong Baptist University, Hong Kong, China

    Bo Han

  10. Center for Microscopy and Imaging, Baylor University, Waco, TX, USA

    Bernd Zechmann

Authors
  1. Xuping Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zeyu Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hui Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hangjin Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Guofeng Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Liting Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Xinran Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Bing Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Chun Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Gang Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Guipei Xing
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Boxi Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Yongqiang Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Jiansheng Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Feng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Bo Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Bernd Zechmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Yong He
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Feng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

X.F. and F.L. (China) designed and conceptualized the study. Z.Y. implemented the algorithm and analysed the image data. X.F., Z.Y. and F.L (China). wrote the paper. H.F., F.L. (Australia) and B. Han deployed the OrgSegNet model on the web portal. H.J., Y.H. and G.Y. reviewed the paper. L.C., X.Z., B.Z., Y.S. and F.L. (China) labelled the dataset. B. Hu, C.Q., G.H. and G.X. prepared microscopy images. J.G. and B.Z. provided 3D cell data and recommendations.

Corresponding authors

Correspondence to Yong He or Feng Liu.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Plants thanks Ralf Mikut, Jean Molinier and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Discussion, Methods, Figs. 1–22, Tables 1–9 and References.

Reporting Summary

Supplementary Table 10–12

Table 10 Diversity value of shape complexity, electron density and cross-sectional area of organelles from different plant species across the training, testing and validation datasets. Table 11 Plant species descriptions of each 2D TEM image. Table 12 3D FIB-SEM data included in the dataset.

Supplementary Video 1

Slice images segmented with OrgSegNet and the 3D reconstructed chloroplasts in a rice mesophyll cell. a, Reconstruction results of all chloroplasts in the visual field. b, Automated prediction of slice results with OrgSegNet. c, Results of reconstruction of all chloroplasts in a single mesophyll cell. d, Results of reconstruction of all chloroplasts annotated manually.

Supplementary Data 1

Statistical source data for supplementary figures.

Source data

Source Data Fig. 2

Statistical source data.

Source Data Fig. 3

Statistical source data.

Source Data Fig. 4

Statistical source data.

Source Data Fig. 5

Statistical source data.

Source Data Fig. 6

Statistical source data.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Feng, X., Yu, Z., Fang, H. et al. Plantorganelle Hunter is an effective deep-learning-based method for plant organelle phenotyping in electron microscopy. Nat. Plants 9, 1760–1775 (2023). https://doi.org/10.1038/s41477-023-01527-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41477-023-01527-5

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing