Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Correspondence
  • Published:

Automatic tracing of ultra-volumes of neuronal images

Nature Methods volume 14, pages 332–333 (2017)Cite this article

Subjects

To the Editor:

The core algorithm of UltraTracer (Fig. 1) reconstructs a neuron structure from the available image data on the basis of a formulation of maximum-likelihood estimation. The underlying assumption is that the occurrence of a specific neuron structure could be modeled using the joint probability of all of its subparts given the image. Briefly, UltraTracer iteratively factorizes the joint probability based on progressive maximization of conditional probabilities of the occurrence of salient and continuous subparts of a neuron (Supplementary Note). Therefore, UltraTracer explores an image by following where the neurite signal goes, on the basis of either adaptive windows generated based on the already reconstructed neuron structure or certain domain knowledge (prior information or statistics) of neuron morphology, to help refine the choice of the next tracing subarea (Supplementary Note). This process repeats until the neuron structure grows as completely as possible. We designed the UltraTracer software to quickly extract an arbitrary subvolume of interest from large neuron image files (Supplementary Note), and thus smoothly traced an image archive without the need to load a large number of image voxels into computer memory.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Workflow of UltraTracer for tracing a large 3D image volume.

References

  1. Helmstaedter, M. Nat. Methods 10, 501–507 (2013).

    Article  CAS  Google Scholar 

  2. Acciai, L., Soda, P. & Iannello, G. Neuroinformatics 4, 353–367 (2016).

    Article  Google Scholar 

  3. Peng, H. et al. Neuron 87, 252–256 (2015).

    Article  CAS  Google Scholar 

  4. Zhao, T. et al. Neuroinformatics 9, 247–261 (2011).

    Article  Google Scholar 

  5. Wu, J. et al. Neuroimage 87, 199–208 (2014).

    Article  Google Scholar 

  6. Xiao, H. & Peng, H. Bioinformatics 29, 1448–1454 (2013).

    Article  CAS  Google Scholar 

Download references

Acknowledgements

We thank the Allen Institute for Brain Science and data contributors to the BigNeuron project for providing neuron data sets. This work was funded by the Allen Institute for Brain Science. The authors wish to thank the Allen Institute founders, P.G. Allen and J. Allen, for their vision, encouragement and support.

Author information

Author notes

  1. Hanchuan Peng and Zhi Zhou: These authors contributed equally to this work.

Authors and Affiliations

  1. Allen Institute for Brain Science, Seattle, Washington, USA

    Hanchuan Peng, Zhi Zhou & Michael Hawrylycz

  2. Erasmus University Medical Center Rotterdam, Rotterdam, the Netherlands

    Erik Meijering

  3. Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, Virginia, USA

    Ting Zhao

  4. Krasnow Institute for Advanced Study, George Mason University, Fairfax, Virginia, USA

    Giorgio A Ascoli

Authors
  1. Hanchuan Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhi Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Erik Meijering
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ting Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Giorgio A Ascoli
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Michael Hawrylycz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hanchuan Peng.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Integrated supplementary information

Supplementary Figure 1 UltraTracer extends and improves various base tracers to reconstruct large image volumes.

A. UltraTracer with four base tracers APP1, APP2, NeuTube, and MOST (Supplementary Note) applied to image regions R1 and R2. B. Comparison of UltraTracer and the direct use of base tracers. TR: traditional method (i.e. using a base tracer directly to reconstruct the entire 3-D image volume); UT: UltraTracer; BASDM: Best Average Spatial Distance compared with Manual reconstructions; PM: Peak computer-Memory; TT: Tracing Time. The image volume used has 2111×3403×291 voxels. Two independent human manual reconstructions were used for comparison; their BASD (Best Average Spatial Distance) is 3.56 voxels.

Supplementary Figure 2 Comparison of UltraTracer and the direct use of 6 additional base tracers on a human neuron image stack.

These 6 base tracers including Snake (Narayanaswamy, et al, 2011), Minimum Spanning Tree (MST as used by a number of groups independently; the original idea could be referred as Dijkstra, 1959), NeuroGPSTree (Quan, et al, 2016), Rivulet (Liu, et al, 2016), TReMAP (Zhou, et al, 2016), and nctuTW(Lee, et al, 2012). Two base tracers, Snake and Rivulet, were not able to generate the reconstruction using TR, since their usage of computer-memory exceeded the available memory of the testing computer (128 GB). One base tracer, nctuTW, failed to generate the reconstruction using TR or UT because it was too slow (in fact, it could not even produce a reconstruction for a 768×768×291 voxel sub-volume of the human neuron image stack within three hours). The image stack is the same one used in Supplementary Figure 1B.

Supplementary Figure 3 UltraTracer (with base tracer APP2) is scalable with respect to ultra-volumes of neuron images, without compromising the tracing accuracy in terms of spatial distance, morphological and topological features.

TR: Traditional approach. UT: UltraTracer. Testing data: neurons 1, 2, 3, and 4 are confocal image stacks of human pyramidal neurons, neurons 5 and 7 are confocal image stacks of mouse pyramidal neurons, neuron 6 is a brightfield image stack of human pyramidal neuron. In reconstruction-consistency testing of TR and UT based on various features, the "percentage of structure difference" of two reconstructions measures the portion of their visible difference (the nearest matching reconstruction nodes in two tracings are more than 2-voxel apart), the "percentage of matched bifurcation pairs" is defined as the portion of reciprocally best matching bifurcation points divided by the average number of bifurcation points of two reconstructions, the "total length", "total surface", and "total volume" are the length, the surface, and the volume of all neuronal compartments in reconstructions, the "average diameter" is the average diameters of all compartments in a reconstruction, the "Hausdorff dimension" (Falconer, 2004) measures the fractal dimension of reconstructions. In parentheses, the statistics (mean +/- s.d.) derived from TR-reconstructions using 59 rotated images (every 6 degrees around the center of XY-plane) for each neuron are shown as controls. Bottom-right inset: Regression analysis of peak memory and tracing time versus the image volume tested on 31 brightfield images.

Supplementary Figure 4 Application of UltraTracer to brightfield imaging image stacks of mouse V1 neurons.

A. An example of brightfield image. B. Enhanced image using an adaptive approach (Zhou, et al, 2015). C. UltraTracer reconstruction based on the enhanced image in B. Different colors indicate reconstructions from different image regions.

Supplementary Figure 5 UltraTracer enhanced by incorporating prior knowledge of the adaptive subarea (window) size in tracing, which was learned from largescale statistics of mammalian neuron reconstructions.

A. The estimated window size (in x, y, and z) as a function of the distance of a neuron compartment to the soma. The maximum window size was set to be 1024 voxels. B. Comparison results of two tracings, one with the TDAW method (magenta) and another with PTDAW (green), where the prior is the estimated window size in A. In each zoom-in region (R1 ~ R3), the gray-scale image voxels are also displayed. The two reconstructions are slightly offset in x-direction for better visualization.

Supplementary Figure 6 Average neuron-compartment density as a function of the distance between the neuron-compartment and the soma.

This information was used as a look-up table for PTDAW to avoid potential skewed estimation based on any extreme cases.

Supplementary Figure 7 Tracing results of TDAW (magenta) and PTDAW (green) for a mouse pyramidal cell (voxel size 0.143μm×0.143μm×0.28μm).

Reconstructions are slightly offset in x-direction for better visibility.

Supplementary Figure 8 Combination scheme 1: UltraTracer combines different base tracers to achieve better performance on a 3-D confocal image stack of a Lucifer Yellow labeled human pyramidal neuron.

APP2+NeuTube: the soma region is traced by APP2, and the rest is traced by NeuTube. APP2+MOST: the soma region is traced by APP2, and the rest is traced by MOST. APP2+NeuTube explores 2.80 billion voxels areas, but needs 2158.69s for tracing. APP2+MOST generates a relatively complete reconstruction (1.90 billion voxels scanned areas) with a much faster tracing speed (89.18s tracing time). Neuron data used here is the neuron 4 in Supplementary Figure 3.

Supplementary Figure 9 Combination scheme 2: UltraTracer real-time selects suitable tracing algorithm on a confocal image stack of human pyramidal neuron.

For each explored image region, two reconstructions (APP2 and NeuTube) were generated first. For the "best candidate" result, the contrast-to-background ratio in the image region around the reconstruction was used to choose the suitable algorithm. For the "consensus" result, the union of two reconstructions is used as the result for the current image region. Both two real-time selection results had similar BASDM scores (2.62 voxels and 3.42 voxels in the best candidate result, and 3.73 voxels and 4.18 voxels in the consensus result) to APP2 (2.78 voxels and 3.57 voxels) and NeuTube (5.17 voxels and 5.42 voxels). Neuron data used is the same neuron in Supplementary Figure 1B.

Supplementary Figure 10 An application example of UltraTracer for tracing multiple biocytin-filled human neurons with axons.

The images were from 3 sections, each of which was imaged separately (voxel size 0.114μm×0.114μm× 0.28μm). UltraTracer was used to reconstruct automatically based on multiple starting locations on these separate image stacks. The final reconstruction (red) was assembled using the NeuronAssembler tool in Vaa3D (vaa3d.org). The reconstruction, including axons and dendrites, was also manually validated (blue in zoom-in views, slightly offset in x-direction for better visibility), with some substructures of the reconstruction edited (addition or deletion of some structures based on visual inspection). Overall more than 90% portion of the automatic reconstruction could be easily validated manually for this example, while the 10% were too difficult even for manual reconstruction (e.g. the manual deletion in location d of region R1 seemed to be a problematic deletion in the manual correction). The total lengths of the automatic and manually curated reconstructions were 22.51 and 20.15 mm, respectively.

Supplementary Figure 11 UltraTracer workflow.

Supplementary Figure 12 Fixed versus adaptive tile size.

A. Based on the boundary tips of the left tile (containing the soma), three image tiles (1.1 billion voxels) have been loaded to trace the right side of the neuron with fixed tile size. B. A much smaller part (0.55 billion voxels) of the image volume has been loaded with the adaptive tile size. The purpose of this figure is to show the comparison of loading areas to trace the right side of the neuron using fixed and adaptive tile sizes. So only the part of the traced neuron structures is shown.

Supplementary Figure 13 One reconstruction fusion example.

A. Over-tracing due to overlap between adjacent tiles. B. The over-tracing error has been fixed.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–13 and Supplementary Note 1 (PDF 11006 kb)

Supplementary Software

UltraTracer Test Image Data and Binary Program (ZIP 40044 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Peng, H., Zhou, Z., Meijering, E. et al. Automatic tracing of ultra-volumes of neuronal images. Nat Methods 14, 332–333 (2017). https://doi.org/10.1038/nmeth.4233

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nmeth.4233

This article is cited by

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics