Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Automated synaptic connectivity inference for volume electron microscopy

Nature Methods volume 14, pages 435–442 (2017)Cite this article

Subjects

Abstract

Teravoxel volume electron microscopy data sets from neural tissue can now be acquired in weeks, but data analysis requires years of manual labor. We developed the SyConn framework, which uses deep convolutional neural networks and random forest classifiers to infer a richly annotated synaptic connectivity matrix from manual neurite skeleton reconstructions by automatically identifying mitochondria, synapses and their types, axons, dendrites, spines, myelin, somata and cell types. We tested our approach on serial block-face electron microscopy data sets from zebrafish, mouse and zebra finch, and computed the synaptic wiring of songbird basal ganglia. We found that, for example, basal-ganglia cell types with high firing rates in vivo had higher densities of mitochondria and vesicles and that synapse sizes and quantities scaled systematically, depending on the innervated postsynaptic cell types.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Steps in connectomic analysis and skeleton-to-volume reconstruction conversion.
Figure 2: CNN-based identification of synapses and other ultrastructure in different SBEM data sets.
Figure 3: Mapping objects to skeletons.
Figure 4: Inference of subcellular parts.
Figure 5: Automated cell type classification.
Figure 6: Automatically generated connectivity matrix.

Similar content being viewed by others

References

  1. Briggman, K.L. & Bock, D.D. Volume electron microscopy for neuronal circuit reconstruction. Curr. Opin. Neurobiol. 22, 154–161 (2012).

    CAS  PubMed  Google Scholar 

  2. Denk, W., Briggman, K.L. & Helmstaedter, M. Structural neurobiology: missing link to a mechanistic understanding of neural computation. Nat. Rev. Neurosci. 13, 351–358 (2012).

    CAS  PubMed  Google Scholar 

  3. Briggman, K.L., Helmstaedter, M. & Denk, W. Wiring specificity in the direction-selectivity circuit of the retina. Nature 471, 183–188 (2011).

    CAS  PubMed  Google Scholar 

  4. Helmstaedter, M. et al. Connectomic reconstruction of the inner plexiform layer in the mouse retina. Nature 500, 168–174 (2013).

    CAS  PubMed  Google Scholar 

  5. Takemura, S.-Y. et al. A visual motion detection circuit suggested by Drosophila connectomics. Nature 500, 175–181 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  6. Kim, J.S. et al. Space-time wiring specificity supports direction selectivity in the retina. Nature 509, 331–336 (2014).

    CAS  PubMed  PubMed Central  Google Scholar 

  7. Wanner, A.A., Genoud, C., Masudi, T., Siksou, L. & Friedrich, R.W. Dense EM-based reconstruction of the interglomerular projectome in the zebrafish olfactory bulb. Nat. Neurosci. 19, 816–825 (2016).

    CAS  PubMed  Google Scholar 

  8. Turaga, S.C. et al. Convolutional networks can learn to generate affinity graphs for image segmentation. Neural Comput. 22, 511–538 (2010).

    PubMed  Google Scholar 

  9. Jain, V. et al. Boundary learning by optimization with topological constraints. in Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conf. 2488–2495 (2010).

  10. Berning, M., Boergens, K.M. & Helmstaedter, M. SegEM: efficient image analysis for high-resolution connectomics. Neuron 87, 1193–1206 (2015).

    CAS  PubMed  Google Scholar 

  11. Pallotto, M., Watkins, P.V., Fubara, B., Singer, J.H. & Briggman, K.L. Extracellular space preservation aids the connectomic analysis of neural circuits. Elife 4, e08206 (2015).

    PubMed  PubMed Central  Google Scholar 

  12. Plaza, S.M. et al. Annotating synapses in large em datasets. Preprint available at https://arxiv.org/abs/1409.1801 (2014).

  13. Huang, G.B., Scheffer, L.K. & Plaza, S.M. Fully-automatic synapse prediction and validation on a large data set. Preprint available at https://arxiv.org/abs/1604.03075 (2016).

  14. Bergstra, J. et al. Theano: a CPU and GPU math expression compiler. in Proceedings of the Python for Scientific Computing Conference (SciPy) 4, 3 (2010).

    Google Scholar 

  15. Giusti, A., Cires¸an, D.C., Masci, J., Gambardella, L.M. & Schmidhuber, J. Fast image scanning with deep max-pooling convolutional neural networks. Preprint available at https://arxiv.org/abs/1302.1700 (2013).

  16. Kasthuri, N. et al. Saturated reconstruction of a volume of neocortex. Cell 162, 648–661 (2015).

    CAS  PubMed  Google Scholar 

  17. Takemura, S.-Y. et al. Synaptic circuits and their variations within different columns in the visual system of Drosophila. Proc. Natl. Acad. Sci. USA 112, 13711–13716 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  18. Kreshuk, A. et al. Automated detection and segmentation of synaptic contacts in nearly isotropic serial electron microscopy images. PLoS One 6, e24899 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  19. Kreshuk, A., Koethe, U., Pax, E., Bock, D.D. & Hamprecht, F.A. Automated detection of synapses in serial section transmission electron microscopy image stacks. PLoS One 9, e87351 (2014).

    PubMed  PubMed Central  Google Scholar 

  20. Becker, C., Ali, K., Knott, G. & Fua, P. Learning context cues for synapse segmentation. IEEE Trans. Med. Imaging 32, 1864–1877 (2013).

    PubMed  Google Scholar 

  21. Huang, G.B. & Plaza, S. Identifying synapses using deep and wide multiscale recursive networks. Preprint available at https://arxiv.org/abs/1409.1789 (2014).

  22. Márquez Neila, P. et al. A Fast method for the segmentation of synaptic junctions and mitochondria in serial electron microscopic images of the brain. Neuroinformatics 14, 235–250 (2016).

    PubMed  PubMed Central  Google Scholar 

  23. Roncal, W.G. et al. VESICLE: volumetric evaluation of synaptic interfaces using computer vision at large scale. Preprint available at https://arxiv.org/abs/1403.3724 (2014).

  24. Perez, A.J. et al. A workflow for the automatic segmentation of organelles in electron microscopy image stacks. Front. Neuroanat. 8, 126 (2014).

    PubMed  PubMed Central  Google Scholar 

  25. Schüz, A. & Palm, G. Density of neurons and synapses in the cerebral cortex of the mouse. J. Comp. Neurol. 286, 442–455 (1989).

    PubMed  Google Scholar 

  26. Colonnier, M. Synaptic patterns on different cell types in the different laminae of the cat visual cortex. An electron microscope study. Brain Res. 9, 268–287 (1968).

    CAS  PubMed  Google Scholar 

  27. Gray, E.G. Axo-somatic and axo-dendritic synapses of the cerebral cortex: an electron microscope study. J. Anat. 93, 420–433 (1959).

    CAS  PubMed  PubMed Central  Google Scholar 

  28. Peters, A., Palay, S.L. & Webster H. deF. The Fine Structure of the Nervous System: Neurons and Their Supporting Cells (Oxford University Press, NY, 1991).

  29. Lenn, N.J. & Reese, T.S. The fine structure of nerve endings in the nucleus of the trapezoid body and the ventral cochlear nucleus. Am. J. Anat. 118, 375–389 (1966).

    CAS  PubMed  Google Scholar 

  30. Carrillo, G.D. & Doupe, A.J. Is the songbird Area X striatal, pallidal, or both? An anatomical study. J. Comp. Neurol. 473, 415–437 (2004).

    PubMed  Google Scholar 

  31. Reiner, A., Laverghetta, A.V., Meade, C.A., Cuthbertson, S.L. & Bottjer, S.W. An immunohistochemical and pathway tracing study of the striatopallidal organization of area X in the male zebra finch. J. Comp. Neurol. 469, 239–261 (2004).

    PubMed  Google Scholar 

  32. Goldberg, J.H. & Fee, M.S. Singing-related neural activity distinguishes four classes of putative striatal neurons in the songbird basal ganglia. J. Neurophysiol. 103, 2002–2014 (2010).

    PubMed  PubMed Central  Google Scholar 

  33. Goldberg, J.H., Adler, A., Bergman, H. & Fee, M.S. Singing-related neural activity distinguishes two putative pallidal cell types in the songbird basal ganglia: comparison to the primate internal and external pallidal segments. J. Neurosci. 30, 7088–7098 (2010).

    CAS  PubMed  PubMed Central  Google Scholar 

  34. Farries, M.A. & Perkel, D.J. A telencephalic nucleus essential for song learning contains neurons with physiological characteristics of both striatum and globus pallidus. J. Neurosci. 22, 3776–3787 (2002).

    CAS  PubMed  PubMed Central  Google Scholar 

  35. Farries, M.A., Ding, L. & Perkel, D.J. Evidence for “direct” and “indirect” pathways through the song system basal ganglia. J. Comp. Neurol. 484, 93–104 (2005).

    PubMed  Google Scholar 

  36. Zhao, T. & Plaza, S.M. Automatic neuron type identification by neurite localization in the Drosophila medulla. Preprint available at https://arxiv.org/abs/1409.1892 (2014).

  37. Jonas, E. & Kording, K. Automatic discovery of cell types and microcircuitry from neural connectomics. eLife 4, e04250 (2015).

    PubMed  PubMed Central  Google Scholar 

  38. Tanaka, M., Singh Alvarado, J., Murugan, M. & Mooney, R. Focal expression of mutant huntingtin in the songbird basal ganglia disrupts cortico-basal ganglia networks and vocal sequences. Proc. Natl. Acad. Sci. USA 113, E1720–E1727 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  39. Leblois, A., Bodor, A.L., Person, A.L. & Perkel, D.J. Millisecond timescale disinhibition mediates fast information transmission through an avian basal ganglia loop. J. Neurosci. 29, 15420–15433 (2009).

    CAS  PubMed  PubMed Central  Google Scholar 

  40. Koós, T. & Tepper, J.M. Inhibitory control of neostriatal projection neurons by GABAergic interneurons. Nat. Neurosci. 2, 467–472 (1999).

    PubMed  Google Scholar 

  41. Bennett, B.D. & Bolam, J.P. Synaptic input and output of parvalbumin-immunoreactive neurons in the neostriatum of the rat. Neuroscience 62, 707–719 (1994).

    CAS  PubMed  Google Scholar 

  42. Jaeger, D., Kita, H. & Wilson, C.J. Surround inhibition among projection neurons is weak or nonexistent in the rat neostriatum. J. Neurophysiol. 72, 2555–2558 (1994).

    CAS  PubMed  Google Scholar 

  43. Oorschot, D.E. et al. Synaptic connectivity between rat striatal spiny projection neurons in vivo: Unexpected multiple somatic innervation in the context of overall sparse proximal connectivity. Basal Ganglia 3, 93–108 (2013).

    Google Scholar 

  44. Alexander, G.E. & Crutcher, M.D. Functional architecture of basal ganglia circuits: neural substrates of parallel processing. Trends Neurosci. 13, 266–271 (1990).

    CAS  PubMed  Google Scholar 

  45. Ahrens, M.B., Orger, M.B., Robson, D.N., Li, J.M. & Keller, P.J. Whole-brain functional imaging at cellular resolution using light-sheet microscopy. Nat. Methods 10, 413–420 (2013).

    CAS  PubMed  Google Scholar 

  46. Zhang, K. & Sejnowski, T.J. A universal scaling law between gray matter and white matter of cerebral cortex. Proc. Natl. Acad. Sci. USA 97, 5621–5626 (2000).

    CAS  PubMed  PubMed Central  Google Scholar 

  47. Naumann, E.A., Kampff, A.R., Prober, D.A., Schier, A.F. & Engert, F. Monitoring neural activity with bioluminescence during natural behavior. Nat. Neurosci. 13, 513–520 (2010).

    CAS  PubMed  PubMed Central  Google Scholar 

  48. Herculano-Houzel, S., Mota, B. & Lent, R. Cellular scaling rules for rodent brains. Proc. Natl. Acad. Sci. USA 103, 12138–12143 (2006).

    CAS  PubMed  PubMed Central  Google Scholar 

  49. Mikula, S. & Denk, W. High-resolution whole-brain staining for electron microscopic circuit reconstruction. Nat. Methods 12, 541–546 (2015).

    CAS  PubMed  Google Scholar 

  50. Helmstaedter, M., Briggman, K.L. & Denk, W. High-accuracy neurite reconstruction for high-throughput neuroanatomy. Nat. Neurosci. 14, 1081–1088 (2011).

    CAS  PubMed  Google Scholar 

  51. Pedregosa, F. et al. Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011).

    Google Scholar 

Download references

Acknowledgements

We thank W. Denk for generous support and many helpful discussions. We thank M.S. Fee, K. Doroschak, S. Mikula, L. Heinrich and W. Denk for comments on the manuscript. We thank R. Illichmann, D. Dimitrov, O. Fedorashko, K. Kiesl and G. Patzer for ground truth annotations and many others for the skeleton annotations, and the KNOSSOS development team for continuous software support. We thank J. Buhmann for neurite segmentation training data, L. Heinrich for optimizing the watershed parameters, L. Schott for myelin training data, L. Burchartz for spine ground truth, C. Roome and M. Hilpert for IT support, and J. Kuhl for support with the figures. J.K. and F.S. were supported by the Boehringer Ingelheim Fonds.

Author information

Authors and Affiliations

  1. Max Planck Institute of Neurobiology, Planegg-Martinsried, Germany

    Sven Dorkenwald, Philipp J Schubert, Marius F Killinger, Shawn Mikula, Fabian Svara & Joergen Kornfeld

  2. Max Planck Institute for Medical Research, Heidelberg, Germany

    Sven Dorkenwald, Philipp J Schubert, Marius F Killinger & Gregor Urban

Authors
  1. Sven Dorkenwald
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Philipp J Schubert
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marius F Killinger
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gregor Urban
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shawn Mikula
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Fabian Svara
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Joergen Kornfeld
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

S.D., P.J.S. and J.K. developed and evaluated SyConn. M.F.K. and G.U. developed ElektroNN. S.M., F.S. and J.K. contributed data sets. S.D., P.J.S. and J.K. wrote the manuscript.

Corresponding author

Correspondence to Joergen Kornfeld.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Integrated supplementary information

Supplementary Figure 1 Failure modes of ray casting and watershed for an erroneous barrier prediction probability map

(a) Raw data and boundary prediction map (b) with a clear hole (arrow). (c) The watershed segmentation floods the adjacent process while (d) rays cast through the hole were removed during the filtering stage. Scale bar: 1 μm.

Supplementary Figure 2 Efficient generation of barrier ground truth on the mouse dataset

(a) Raw image data of mouse dataset. (b) Corresponding barrier prediction of a recursive 3D CNN, trained only on the bird dataset training data. (c) Corresponding barrier prediction after (d) an iterative post-training procedure with focused generation of minimal additional ground truth. Scale bars: 1 μm.

Supplementary Figure 3 Parameter search for optimal size filter and probability threshold

(a) Crosshair indicates probability map threshold and size filter parameters after connected-component analysis with maximum F1 score on training set. (b-d) Representative synapse examples of different sizes. Many found synaptic junction objects below the size filter threshold were false positives. Scale bars: 1 μm.

Source data

Supplementary Figure 4 Analysis of cell type classification

(a) Classification performance of an RFC spatially restricted to a local feature context, illustrating that neurites can be classified with good performance locally. (b) Left: MSN reconstruction with soma, dendrite and axon, identified by typical spiny dendritic morphology. Right: MSN axon fragment, identified by typical ultrastructural appearance of MSN synapses. (c) Separation of neurite reconstructions in principal component 1 vs. 2 space. Data points colored by ground truth annotations, which shows that cell types do not form easily separable clusters. Markers indicate MSN neurites from (b), that fall into different clusters. (d) and (e) components 1 vs. 3 and 2 vs 3. (f) Explained data variance by principal component.

Source data

Supplementary Figure 5 Examples of undetected tip contacts

(a)-(c) Two neurite reconstructions (red and blue shaded) with a tip contact that was missed during automatic contact site detection. Scale bar: 500 nm.

Supplementary Figure 6 Size verification of the automatically annotated synapses

(a) Comparison of synapse sizes, labeled by 3 annotators vs SyConn, at the same synapse locations. Per synapse mean with s.d. of annotators on y-axis. (b) Comparison of synapse size distributions for SyConn and the 3 annotators. (c) Correlation-matrix of synapse sizes, reported are pairwise Pearson’s r-values. Note that some annotators correlate particularly well (annotator 1 and 3), while there is overall variability in the annotation of synapse sizes on the level of individual synapses.

Source data

Supplementary Figure 7 Counted binarized connectivity matrix

(a) Connectivity matrix of Area X dataset; rows describe outgoing connections, columns incoming connections. Each matrix entry (enlarged for better visualization) is colored by synaptic type and shaded according to the number of synapses between the two neurons. (b) Average number of synapses of all connections between cell type pairs divided by the total number of cell type pair matrix elements.

Source data

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–7 and Supplementary Tables 1, 2 and 5 (PDF 1426 kb)

Supplementary Table 3

Parameters of the CNNs used in this study. (XLSX 57 kb)

Supplementary Table 4

Boundaries used for the volume annotations in this study. (XLSX 15 kb)

Supplementary Software

Source code and documentation. (ZIP 14646 kb)

Source data

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dorkenwald, S., Schubert, P., Killinger, M. et al. Automated synaptic connectivity inference for volume electron microscopy. Nat Methods 14, 435–442 (2017). https://doi.org/10.1038/nmeth.4206

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nmeth.4206

This article is cited by

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics