High-accuracy neurite reconstruction for high-throughput neuroanatomy

Journal name:
Nature Neuroscience
Year published:
Published online


Neuroanatomic analysis depends on the reconstruction of complete cell shapes. High-throughput reconstruction of neural circuits, or connectomics, using volume electron microscopy requires dense staining of all cells, which leads even experts to make annotation errors. Currently, reconstruction speed rather than acquisition speed limits the determination of neural wiring diagrams. We developed a method for fast and reliable reconstruction of densely labeled data sets. Our approach, based on manually skeletonizing each neurite redundantly (multiple times) with a visualization-annotation software tool called KNOSSOS, is ~50-fold faster than volume labeling. Errors are detected and eliminated by a redundant-skeleton consensus procedure (RESCOP), which uses a statistical model of how true neurite connectivity is transformed into annotation decisions. RESCOP also estimates the reliability of consensus skeletons. Focused reannotation of difficult locations promises a rather steep increase of reliability as a function of the average skeleton redundancy and thus the nearly error-free analysis of large neuroanatomical datasets.

At a glance


  1. Comparison of volume and skeleton annotation.
    Figure 1: Comparison of volume and skeleton annotation.

    (a,b) Examples of volume labeling (a) and skeletonization (b) for the same two neurite fragments; cell-surface labeled data (data set E1088; Online Methods). Scale bars represent 250 nm. (c) Sketch of a neurite skeleton. (d) Rate of time consumption for volume labeling10 and for skeleton annotation (data from this study; annotated using KNOSSOS, see Supplementary Movie 1), for both cell surface–labeled data (black) and conventionally stained data set (K0563, gray; see Fig. 5d). Error bars represent range for volume labeling and s.d. for skeletonization. (e) Outline of RESCOP.

  2. Skeletonization by expert annotators.
    Figure 2: Skeletonization by expert annotators.

    (a) Two complete skeletons of the same amacrine cell annotated independently by M.H. and K.L.B., starting at the soma. (b) Same skeletons shown looking onto the plane of the retina. Green, agreement among annotators; black, disagreement; numbers, disagreement locations. Stacks of original data surrounding disagreement locations are shown in Supplementary Image Stacks 1,2,3,4,5,6,7,8,9,10,11 and 12. INL, inner nuclear layer; IPL, inner plexiform layer; GCL, ganglion cell layer. Scale bars, 5 μm.

  3. RESCOP step 1, skeleton-to-skeleton agreement measurement.
    Figure 3: RESCOP step 1, skeleton-to-skeleton agreement measurement.

    (a) Overlay of seven independent skeletons of the same neurite (bipolar cell axon) annotated by slightly trained nonexperts, all starting at the soma (red cross). (be) Schematic of procedure for measuring agreement among multiple annotators for one skeleton edge (dashed line) in skeleton A. (f) Histograms of edge votes for 50-fold annotation of one cell (left) and dense skeletonization of 98 neurites (right). Bottom, vote count versus total votes (log scale). Histograms were corrected for multiple counting of the same location; see Online Methods. (g,h) Predicted vote histograms for single cell (left) and for dense skeletons (right) (g), using the distribution of edge detectabilities pfit(pe) (h) that best predicted the respective histograms in f. (i,j) Schematic of how the truth (top) is converted to detection probability (middle). Bottom, probabilities for different T (number of agreeing votes) for one edge (i, binomial distribution for pe = 0.7 and N = 10 annotators) and for all edges combined (j, schematic).

  4. RESCOP steps 2 and 3, edge elimination and skeleton recombination.
    Figure 4: RESCOP steps 2 and 3, edge elimination and skeleton recombination.

    (a) Probability that edge detectability pe has a certain value, given different edge votes, without prior knowledge (blue) and for the fitted distribution of edge detectabilities pfit(pe) (red). Whether an edge is kept or eliminated depends on whether the integral of p(pe|T,N) for pe > 0.5 (green shading) is larger or smaller than that for pe < 0.5 (red shading). In this example, edges with T = 1 and N = 4 would be eliminated and those with T = 2 to 4 would be kept. (b) Decision error, perr(T,N), with optimal (stepped line) and majority vote (dashed straight line) decision boundaries for the single-cell (top) and dense skeletonization data (bottom). (c) Elimination procedure illustrated at a branch point. Red, eliminated edges. Green, discarded skeleton pieces. (d) Variation of annotator performance reflected in average total number of votes per edge and average ratio of agreeing to total votes for each annotator. Circle, worst-performing annotator who skeletonized black skeleton in e. (e) Fifty skeletons of one amacrine cell before (left) and after (right) edge validation and consensus computation. Scale bar, 5 μm.

  5. RESCOP step 4, estimating error rate of RESCOPed skeletons.
    Figure 5: RESCOP step 4, estimating error rate of RESCOPed skeletons.

    (a) Stereo view of two superimposed sets (red and blue) of five-fold consensus skeletons. Black asterisks, disagreements. Total neurite path length, 600 μm. (b) Estimated detectability distribution for one edge for a fixed ratio of agreeing to total votes (T/N) of 0.33, but different numbers of total votes (N). Probabilities are given that the edge was erroneously kept. (c) Top, mean path length between errors as a function of number of annotators. Solid lines, estimates using equation (11) for dense neurites (red) and single cell (green); crosses, errors detected by visual comparison with the 50-fold consensus skeleton for the consensus of 1, 5 (includes a), 10 and 25 skeletons (error bars, s.e.m.). Dashed lines, average redundancy as a function of the target error rate for focused reannotation (Monte Carlo simulations). Bottom, same analysis for a conventionally stained data set annotated using the original data (blue, K0563, mag1, s.d.), data with added noise (magenta, K0563, mag1, noise) and data at half the resolution (cyan, K0563, mag2). (d) Examples from the original and degraded data sets. Scale bar, 250 nm.

  6. Doubly annotated skeletons of 114 putative rod bipolar cells in a block of mouse retina.
    Figure 6: Doubly annotated skeletons of 114 putative rod bipolar cells in a block of mouse retina.

    (a) View onto the block face. INL, inner nuclear layer; IPL, inner plexiform layer; GCL, ganglion cell layer. Dashed lines indicate bounding boxes for bd. (b) Two skeletons of a single rod bipolar cell. (c,d) View onto the plane of the retina confined to the dendrites (c) and axons (d) of bipolar cells, respectively. Cells are colored randomly in c,d. Scale bars, 10 μm.


  1. Ramón y Cajal, S. Textura del Sistema Nervioso del Hombre y de los Vertebrados (Moya, Madrid, 1899).
  2. Golgi, C. Sulla struttura della sostanza grigia del cervelo. Gazzetta Medica Italiana Lombardia 33, 244246 (1873).
  3. Harris, K.M. & Stevens, J.K. Dendritic spines of CA 1 pyramidal cells in the rat hippocampus: serial electron microscopy with reference to their biophysical characteristics. J. Neurosci. 9, 29822997 (1989).
  4. Horikawa, K. & Armstrong, W.E. A versatile means of intracellular labeling: injection of biocytin and its detection with avidin conjugates. J. Neurosci. Methods 25, 111 (1988).
  5. Stretton, A.O. & Kravitz, E.A. Neuronal geometry: determination with a technique of intracellular dye injection. Science 162, 132134 (1968).
  6. Wickersham, I.R. et al. Monosynaptic restriction of transsynaptic tracing from single, genetically targeted neurons. Neuron 53, 639647 (2007).
  7. Livet, J. et al. Transgenic strategies for combinatorial expression of fluorescent proteins in the nervous system. Nature 450, 5662 (2007).
  8. Sporns, O., Tononi, G. & Kotter, R. The human connectome: A structural description of the human brain. PLoS Comput. Biol. 1, e42 (2005).
  9. Lichtman, J.W., Livet, J. & Sanes, J.R. A technicolour approach to the connectome. Nat. Rev. Neurosci. 9, 417422 (2008).
  10. Helmstaedter, M., Briggman, K.L. & Denk, W. 3D structural imaging of the brain with photons and electrons. Curr. Opin. Neurobiol. 18, 633641 (2008).
  11. White, J.G., Southgate, E., Thomson, J.N. & Brenner, S. The structure of the nervous system of the nematode Caenorhabditis elegans. Phil. Trans. R. Soc. Lond. B 314, 1340 (1986).
  12. Denk, W. & Horstmann, H. Serial block-face scanning electron microscopy to reconstruct three-dimensional tissue nanostructure. PLoS Biol. 2, e329 (2004).
  13. Hayworth, K.J., Kasthuri, N., Schalek, R. & Lichtman, J.W. Automating the collection of ultrathin serial sections for large volume TEM reconstructions. Microsc. Microanal. 12, 8687 (2006).
  14. Knott, G., Marchman, H., Wall, D. & Lich, B. Serial section scanning electron microscopy of adult brain tissue using focused ion beam milling. J. Neurosci. 28, 29592964 (2008).
  15. Briggman, K.L. & Denk, W. Towards neural circuit reconstruction with volume electron microscopy techniques. Curr. Opin. Neurobiol. 16, 562570 (2006).
  16. Briggman, K.L., Helmstaedter, M. & Denk, W. Wiring specificity in the direction-selectivity circuit of the retina. Nature (in the press) (2011).
  17. Fiala, J.C. Reconstruct: a free editor for serial section microscopy. J. Microsc. 218, 5261 (2005).
  18. Jeong, W.K. et al. Ssecrett and NeuroTrace: interactive visualization and analysis tools for large-scale neuroscience data sets. IEEE Comput. Graph. Appl. 30, 5870 (2010).
  19. Trachtenberg, J.T. et al. Long-term in vivo imaging of experience-dependent synaptic plasticity in adult cortex. Nature 420, 788794 (2002).
  20. Stevens, J.K., McGuire, B.A. & Sterling, P. Toward a functional architecture of the retina: serial reconstruction of adjacent ganglion cells. Science 207, 317319 (1980).
  21. Chen, B.L., Hall, D.H. & Chklovskii, D.B. Wiring optimization can relate neuronal structure and function. Proc. Natl. Acad. Sci. USA 103, 47234728 (2006).
  22. Chklovskii, D.B., Vitaladevuni, S. & Scheffer, L.K. Semi-automated reconstruction of neural circuits using electron microscopy. Curr. Opin. Neurobiol. 20, 667675 (2010).
  23. Mishchenko, Y. et al. Ultrastructural analysis of hippocampal neuropil from the connectomics perspective. Neuron 67, 10091020 (2010).
  24. Jain, V. et al. Supervised learning of image restoration with convolutional networks. IEEE 11th Int. Conf. Comput. Vis. 18 (2007).
  25. Andres, B., Köthe, U., Helmstaedter, M., Denk, W. & Hamprecht, F. Segmentation of SBFSEM volume data of neural tissue by hierarchical classification. in Pattern Recognition: Lecture Notes in Computer Science (ed. Rigoll, G.) 142152 (Springer-Verlag, 2008).
  26. Turaga, S.C. et al. Convolutional networks can learn to generate affinity graphs for image segmentation. Neural Comput. 22, 511538 (2010).
  27. Martin, D., Fowlkes, C., Tal, D. & Malik, J. A Database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. IEEE 8th Int. Conf. Comput. Vis. 2, 416423 (2001).
  28. Warfield, S.K., Zou, K.H. & Wells, W.M. Validation of image segmentation by estimating rater bias and variance. Philos. Transact. A Math. Phys. Eng. Sci. 366, 23612375 (2008).
  29. Warfield, S.K., Zou, K.H. & Wells, W.M. Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation. IEEE Trans. Med. Imaging 23, 903921 (2004).
  30. Wang, W. et al. A classifier ensemble based on performance level estimation. IEEE Int. Symposium on Biomed. Imaging: from Nano to Micro, 342345 (2009).
  31. Körding, K. Decision theory: what “should” the nervous system do? Science 318, 606610 (2007).
  32. Crook, S., Gleeson, P., Howell, F., Svitak, J. & Silver, R.A. MorphML: level 1 of the NeuroML standards for neuronal morphology data and model specification. Neuroinformatics 5, 96104 (2007).

Download references

Author information


  1. Max Planck Institute for Medical Research, Heidelberg, Germany.

    • Moritz Helmstaedter,
    • Kevin L Briggman &
    • Winfried Denk


M.H. and W.D. designed the study and devised the analysis algorithms; K.L.B. carried out the SBEM experiments; M.H., K.L.B. and W.D. specified the KNOSSOS software; M.H. analyzed the data; M.H., K.L.B. and W.D. wrote the paper.

Competing financial interests

Moritz Helmstaedter and Winfried Denk have applied for a patent (Published Patent Application US 20100183217). Winfried Denk receives IP license income from Gatan Inc. for serial blockface imaging.

Corresponding author

Correspondence to:

Author details

Supplementary information

PDF files

  1. Supplementary Text and Figures (594K)

    Supplementary Figures 1–5

Zip files

  1. Supplementary Movie 1 (68M)

    Skeleton tracing with KNOSSOS. 3 orthogonal views are displayed (xy, top left, yz, top right, xz, bottom left) and a 3-dimensional view of the dataset bounding box and the skeleton being created (bottom right). The 19MB version requires a video player compatible with DivX-encoded files. The 68MB version can be viewed with a generic player but is shorter.

  2. Supplementary Image Stack 1 (3M)
  3. Supplementary Image Stack 2 (3M)
  4. Supplementary Image Stack 3 (3M)
  5. Supplementary Image Stack 4 (3M)
  6. Supplementary Image Stack 5 (3M)
  7. Supplementary Image Stack 6 (3M)
  8. Supplementary Image Stack 7 (3M)
  9. Supplementary Image Stack 8 (3M)
  10. Supplementary Image Stack 9 (3M)
  11. Supplementary Image Stack 10 (3M)
  12. Supplementary Image Stack 11 (3M)
  13. Supplementary Image Stack 12 (3M)
  14. Supplementary Image Stack 13 (54M)
  15. Supplementary Image Stack 14 (56M)
  16. Supplementary Image Stack 15 (6M)

Additional data