Teravoxel volume electron microscopy data sets from neural tissue can now be acquired in weeks, but data analysis requires years of manual labor. We developed the SyConn framework, which uses deep convolutional neural networks and random forest classifiers to infer a richly annotated synaptic connectivity matrix from manual neurite skeleton reconstructions by automatically identifying mitochondria, synapses and their types, axons, dendrites, spines, myelin, somata and cell types. We tested our approach on serial block-face electron microscopy data sets from zebrafish, mouse and zebra finch, and computed the synaptic wiring of songbird basal ganglia. We found that, for example, basal-ganglia cell types with high firing rates in vivo had higher densities of mitochondria and vesicles and that synapse sizes and quantities scaled systematically, depending on the innervated postsynaptic cell types.
Subscribe to Journal
Get full journal access for 1 year
only $20.17 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
Briggman, K.L. & Bock, D.D. Volume electron microscopy for neuronal circuit reconstruction. Curr. Opin. Neurobiol. 22, 154–161 (2012).
Denk, W., Briggman, K.L. & Helmstaedter, M. Structural neurobiology: missing link to a mechanistic understanding of neural computation. Nat. Rev. Neurosci. 13, 351–358 (2012).
Briggman, K.L., Helmstaedter, M. & Denk, W. Wiring specificity in the direction-selectivity circuit of the retina. Nature 471, 183–188 (2011).
Helmstaedter, M. et al. Connectomic reconstruction of the inner plexiform layer in the mouse retina. Nature 500, 168–174 (2013).
Takemura, S.-Y. et al. A visual motion detection circuit suggested by Drosophila connectomics. Nature 500, 175–181 (2013).
Kim, J.S. et al. Space-time wiring specificity supports direction selectivity in the retina. Nature 509, 331–336 (2014).
Wanner, A.A., Genoud, C., Masudi, T., Siksou, L. & Friedrich, R.W. Dense EM-based reconstruction of the interglomerular projectome in the zebrafish olfactory bulb. Nat. Neurosci. 19, 816–825 (2016).
Turaga, S.C. et al. Convolutional networks can learn to generate affinity graphs for image segmentation. Neural Comput. 22, 511–538 (2010).
Jain, V. et al. Boundary learning by optimization with topological constraints. in Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conf. 2488–2495 (2010).
Berning, M., Boergens, K.M. & Helmstaedter, M. SegEM: efficient image analysis for high-resolution connectomics. Neuron 87, 1193–1206 (2015).
Pallotto, M., Watkins, P.V., Fubara, B., Singer, J.H. & Briggman, K.L. Extracellular space preservation aids the connectomic analysis of neural circuits. Elife 4, e08206 (2015).
Plaza, S.M. et al. Annotating synapses in large em datasets. Preprint available at https://arxiv.org/abs/1409.1801 (2014).
Huang, G.B., Scheffer, L.K. & Plaza, S.M. Fully-automatic synapse prediction and validation on a large data set. Preprint available at https://arxiv.org/abs/1604.03075 (2016).
Bergstra, J. et al. Theano: a CPU and GPU math expression compiler. in Proceedings of the Python for Scientific Computing Conference (SciPy) 4, 3 (2010).
Giusti, A., Cires¸an, D.C., Masci, J., Gambardella, L.M. & Schmidhuber, J. Fast image scanning with deep max-pooling convolutional neural networks. Preprint available at https://arxiv.org/abs/1302.1700 (2013).
Kasthuri, N. et al. Saturated reconstruction of a volume of neocortex. Cell 162, 648–661 (2015).
Takemura, S.-Y. et al. Synaptic circuits and their variations within different columns in the visual system of Drosophila. Proc. Natl. Acad. Sci. USA 112, 13711–13716 (2015).
Kreshuk, A. et al. Automated detection and segmentation of synaptic contacts in nearly isotropic serial electron microscopy images. PLoS One 6, e24899 (2011).
Kreshuk, A., Koethe, U., Pax, E., Bock, D.D. & Hamprecht, F.A. Automated detection of synapses in serial section transmission electron microscopy image stacks. PLoS One 9, e87351 (2014).
Becker, C., Ali, K., Knott, G. & Fua, P. Learning context cues for synapse segmentation. IEEE Trans. Med. Imaging 32, 1864–1877 (2013).
Huang, G.B. & Plaza, S. Identifying synapses using deep and wide multiscale recursive networks. Preprint available at https://arxiv.org/abs/1409.1789 (2014).
Márquez Neila, P. et al. A Fast method for the segmentation of synaptic junctions and mitochondria in serial electron microscopic images of the brain. Neuroinformatics 14, 235–250 (2016).
Roncal, W.G. et al. VESICLE: volumetric evaluation of synaptic interfaces using computer vision at large scale. Preprint available at https://arxiv.org/abs/1403.3724 (2014).
Perez, A.J. et al. A workflow for the automatic segmentation of organelles in electron microscopy image stacks. Front. Neuroanat. 8, 126 (2014).
Schüz, A. & Palm, G. Density of neurons and synapses in the cerebral cortex of the mouse. J. Comp. Neurol. 286, 442–455 (1989).
Colonnier, M. Synaptic patterns on different cell types in the different laminae of the cat visual cortex. An electron microscope study. Brain Res. 9, 268–287 (1968).
Gray, E.G. Axo-somatic and axo-dendritic synapses of the cerebral cortex: an electron microscope study. J. Anat. 93, 420–433 (1959).
Peters, A., Palay, S.L. & Webster H. deF. The Fine Structure of the Nervous System: Neurons and Their Supporting Cells (Oxford University Press, NY, 1991).
Lenn, N.J. & Reese, T.S. The fine structure of nerve endings in the nucleus of the trapezoid body and the ventral cochlear nucleus. Am. J. Anat. 118, 375–389 (1966).
Carrillo, G.D. & Doupe, A.J. Is the songbird Area X striatal, pallidal, or both? An anatomical study. J. Comp. Neurol. 473, 415–437 (2004).
Reiner, A., Laverghetta, A.V., Meade, C.A., Cuthbertson, S.L. & Bottjer, S.W. An immunohistochemical and pathway tracing study of the striatopallidal organization of area X in the male zebra finch. J. Comp. Neurol. 469, 239–261 (2004).
Goldberg, J.H. & Fee, M.S. Singing-related neural activity distinguishes four classes of putative striatal neurons in the songbird basal ganglia. J. Neurophysiol. 103, 2002–2014 (2010).
Goldberg, J.H., Adler, A., Bergman, H. & Fee, M.S. Singing-related neural activity distinguishes two putative pallidal cell types in the songbird basal ganglia: comparison to the primate internal and external pallidal segments. J. Neurosci. 30, 7088–7098 (2010).
Farries, M.A. & Perkel, D.J. A telencephalic nucleus essential for song learning contains neurons with physiological characteristics of both striatum and globus pallidus. J. Neurosci. 22, 3776–3787 (2002).
Farries, M.A., Ding, L. & Perkel, D.J. Evidence for “direct” and “indirect” pathways through the song system basal ganglia. J. Comp. Neurol. 484, 93–104 (2005).
Zhao, T. & Plaza, S.M. Automatic neuron type identification by neurite localization in the Drosophila medulla. Preprint available at https://arxiv.org/abs/1409.1892 (2014).
Jonas, E. & Kording, K. Automatic discovery of cell types and microcircuitry from neural connectomics. eLife 4, e04250 (2015).
Tanaka, M., Singh Alvarado, J., Murugan, M. & Mooney, R. Focal expression of mutant huntingtin in the songbird basal ganglia disrupts cortico-basal ganglia networks and vocal sequences. Proc. Natl. Acad. Sci. USA 113, E1720–E1727 (2016).
Leblois, A., Bodor, A.L., Person, A.L. & Perkel, D.J. Millisecond timescale disinhibition mediates fast information transmission through an avian basal ganglia loop. J. Neurosci. 29, 15420–15433 (2009).
Koós, T. & Tepper, J.M. Inhibitory control of neostriatal projection neurons by GABAergic interneurons. Nat. Neurosci. 2, 467–472 (1999).
Bennett, B.D. & Bolam, J.P. Synaptic input and output of parvalbumin-immunoreactive neurons in the neostriatum of the rat. Neuroscience 62, 707–719 (1994).
Jaeger, D., Kita, H. & Wilson, C.J. Surround inhibition among projection neurons is weak or nonexistent in the rat neostriatum. J. Neurophysiol. 72, 2555–2558 (1994).
Oorschot, D.E. et al. Synaptic connectivity between rat striatal spiny projection neurons in vivo: Unexpected multiple somatic innervation in the context of overall sparse proximal connectivity. Basal Ganglia 3, 93–108 (2013).
Alexander, G.E. & Crutcher, M.D. Functional architecture of basal ganglia circuits: neural substrates of parallel processing. Trends Neurosci. 13, 266–271 (1990).
Ahrens, M.B., Orger, M.B., Robson, D.N., Li, J.M. & Keller, P.J. Whole-brain functional imaging at cellular resolution using light-sheet microscopy. Nat. Methods 10, 413–420 (2013).
Zhang, K. & Sejnowski, T.J. A universal scaling law between gray matter and white matter of cerebral cortex. Proc. Natl. Acad. Sci. USA 97, 5621–5626 (2000).
Naumann, E.A., Kampff, A.R., Prober, D.A., Schier, A.F. & Engert, F. Monitoring neural activity with bioluminescence during natural behavior. Nat. Neurosci. 13, 513–520 (2010).
Herculano-Houzel, S., Mota, B. & Lent, R. Cellular scaling rules for rodent brains. Proc. Natl. Acad. Sci. USA 103, 12138–12143 (2006).
Mikula, S. & Denk, W. High-resolution whole-brain staining for electron microscopic circuit reconstruction. Nat. Methods 12, 541–546 (2015).
Helmstaedter, M., Briggman, K.L. & Denk, W. High-accuracy neurite reconstruction for high-throughput neuroanatomy. Nat. Neurosci. 14, 1081–1088 (2011).
Pedregosa, F. et al. Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011).
We thank W. Denk for generous support and many helpful discussions. We thank M.S. Fee, K. Doroschak, S. Mikula, L. Heinrich and W. Denk for comments on the manuscript. We thank R. Illichmann, D. Dimitrov, O. Fedorashko, K. Kiesl and G. Patzer for ground truth annotations and many others for the skeleton annotations, and the KNOSSOS development team for continuous software support. We thank J. Buhmann for neurite segmentation training data, L. Heinrich for optimizing the watershed parameters, L. Schott for myelin training data, L. Burchartz for spine ground truth, C. Roome and M. Hilpert for IT support, and J. Kuhl for support with the figures. J.K. and F.S. were supported by the Boehringer Ingelheim Fonds.
The authors declare no competing financial interests.
Integrated supplementary information
Supplementary Figure 1 Failure modes of ray casting and watershed for an erroneous barrier prediction probability map
(a) Raw data and boundary prediction map (b) with a clear hole (arrow). (c) The watershed segmentation floods the adjacent process while (d) rays cast through the hole were removed during the filtering stage. Scale bar: 1 μm.
(a) Raw image data of mouse dataset. (b) Corresponding barrier prediction of a recursive 3D CNN, trained only on the bird dataset training data. (c) Corresponding barrier prediction after (d) an iterative post-training procedure with focused generation of minimal additional ground truth. Scale bars: 1 μm.
(a) Crosshair indicates probability map threshold and size filter parameters after connected-component analysis with maximum F1 score on training set. (b-d) Representative synapse examples of different sizes. Many found synaptic junction objects below the size filter threshold were false positives. Scale bars: 1 μm. Source data
(a) Classification performance of an RFC spatially restricted to a local feature context, illustrating that neurites can be classified with good performance locally. (b) Left: MSN reconstruction with soma, dendrite and axon, identified by typical spiny dendritic morphology. Right: MSN axon fragment, identified by typical ultrastructural appearance of MSN synapses. (c) Separation of neurite reconstructions in principal component 1 vs. 2 space. Data points colored by ground truth annotations, which shows that cell types do not form easily separable clusters. Markers indicate MSN neurites from (b), that fall into different clusters. (d) and (e) components 1 vs. 3 and 2 vs 3. (f) Explained data variance by principal component. Source data
(a)-(c) Two neurite reconstructions (red and blue shaded) with a tip contact that was missed during automatic contact site detection. Scale bar: 500 nm.
(a) Comparison of synapse sizes, labeled by 3 annotators vs SyConn, at the same synapse locations. Per synapse mean with s.d. of annotators on y-axis. (b) Comparison of synapse size distributions for SyConn and the 3 annotators. (c) Correlation-matrix of synapse sizes, reported are pairwise Pearson’s r-values. Note that some annotators correlate particularly well (annotator 1 and 3), while there is overall variability in the annotation of synapse sizes on the level of individual synapses. Source data
(a) Connectivity matrix of Area X dataset; rows describe outgoing connections, columns incoming connections. Each matrix entry (enlarged for better visualization) is colored by synaptic type and shaded according to the number of synapses between the two neurons. (b) Average number of synapses of all connections between cell type pairs divided by the total number of cell type pair matrix elements. Source data
Supplementary Figures 1–7 and Supplementary Tables 1, 2 and 5 (PDF 1426 kb)
Parameters of the CNNs used in this study. (XLSX 57 kb)
Boundaries used for the volume annotations in this study. (XLSX 15 kb)
Source code and documentation. (ZIP 14646 kb)
About this article
Cite this article
Dorkenwald, S., Schubert, P., Killinger, M. et al. Automated synaptic connectivity inference for volume electron microscopy. Nat Methods 14, 435–442 (2017). https://doi.org/10.1038/nmeth.4206
Brain Sciences (2020)
Small-world networks of neuroblastoma cells cultured in three-dimensional polymeric scaffolds featuring multi-scale roughness
Neural Regeneration Research (2020)
What Can Computational Models Learn From Human Selective Attention? A Review From an Audiovisual Unimodal and Crossmodal Perspective
Frontiers in Integrative Neuroscience (2020)
Nature Methods (2020)
PLOS Computational Biology (2020)