Article | Published:

An active texture-based digital atlas enables automated mapping of structures and markers across brains

Abstract

Brain atlases enable the mapping of labeled cells and projections from different brains onto a standard coordinate system. We address two issues in the construction and use of atlases. First, expert neuroanatomists ascertain the fine-scale pattern of brain tissue, the ‘texture’ formed by cellular organization, to define cytoarchitectural borders. We automate the processes of localizing landmark structures and alignment of brains to a reference atlas using machine learning and training data derived from expert annotations. Second, we construct an atlas that is active; that is, augmented with each use. We show that the alignment of new brains to a reference atlas can continuously refine the coordinate system and associated variance. We apply this approach to the adult murine brainstem and achieve a precise alignment of projections in cytoarchitecturally ill-defined regions across brains from different animals.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Code availability

All analysis was done following the algorithms detailed in the Methods. The code was written in Python and is available as Supplementary Material and, together with updates, at https://github.com/ActiveBrainAtlas/MouseBrainAtlas through the GNU General Public License (GPL). Organization of the code is in a ReadMe file.

Data availability

All raw data are publicly available. They may be downloaded, with a listing of files found in Supplementary Table 1 and at https://github.com/ActiveBrainAtlas/MouseBrainAtlas/blob/master/doc/Brain_stack_directories.md, through the Amazon Web Service Storage S3 at the bucket named mousebrainatlas-rawdata.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  1. 1.

    Roland, P. E. & Zilles, K. Brain atlases: a new research tool. Trends Neurosci. 17, 458–467 (1994).

  2. 2.

    Jones, E. G., Stone, J. M. & Karten, H. J. High-resolution digital brain atlases: a Hubble telescope for the brain. Ann. N.Y. Acad. Sci. 1225S1, E147E159 (2011).

  3. 3.

    MacKenzie-Graham, A. et al. A multimodal, multidimensional atlas of the c57bl/6j mouse brain. J. Anat. 204, 93102 (2004).

  4. 4.

    Majka, P. & Wojcik, D. K. Possuma framework for three-dimensional reconstruction of brain images from serial sections. Neuroinformatics 14, 265278 (2016).

  5. 5.

    Kuan, L. et al. Neuroinformatics of the allen mouse brain connectivity atlas. Methods 73, 4–17 (2015).

  6. 6.

    Pauli, W. M., Nil, A. N. & Tyszka, J. M. A high-resolution probabilistic in vivo atlas of human sub-cortical brain nuclei. Sci. Data 5, 180063 EP (2018).

  7. 7.

    Toga, A. W. et al. Postmortem cryosectioning as an anatomic reference for human brain mapping. Comput. Med. Imaging Graph. 21, 131–141 (1997).

  8. 8.

    Swanson, L. W. & Bota, M. Foundational model of structural connectivity in the nervous system with a schema for wiring diagrams, connectome, and basic plan architecture. Proc. Natl Acad. Sci. USA 107, 20610–20617 (2010).

  9. 9.

    Jones, E. G. Viewpoint: the core and matrix of thalamic organization. Neurosci. 85, 331–345 (1998).

  10. 10.

    Braitenberg, V. On the Texture of Brains, An Introduction to Neuroanatomy for the Cybernetically Minded (Springer, Heidelberg, 1977).

  11. 11.

    Gong, H. et al. High-throughput dual-colour precision imaging for brain-wide connectome with cytoarchi-tectonic landmarks at the cellular level. Nat. Commun. 7, 12142 (2016).

  12. 12.

    Economo, M. N. et al. A platform for brain-wide imaging and reconstruction of individual neurons. eLife 5, e10566 (2016).

  13. 13.

    Richardson, D. S. & Lichtman, J. W. Clarifying tissue clearing. Cell 162, 246257 (2015).

  14. 14.

    Wilt, B. A. et al. Advances in light microscopy for neuroscience. Ann. Rev. Neurosci. 32, 435–506 (2009).

  15. 15.

    Gray, P. A. Transcriptional factors define the neu-roanatomical organization of the medullary reticular formation. Front. Neuroanat. 7, 1–21 (2013).

  16. 16.

    McElvain, L. E. et al. Circuits in the rodent brainstem that control whisking in concert with other orofacial motor actions. Neurosci. 368, 152–170 (2018).

  17. 17.

    Chiang, A.-S. et al. Three-dimensional reconstruction of brain-wide wiring networks in Drosophila at single-cell resolution. Curr. Biol. 21, 1–11 (2011).

  18. 18.

    Peng, H. et al. Brainaligner: 3D registration atlases of Drosophila brains. Nat. Methods 8, 493–498 (2011).

  19. 19.

    Ronneberger, O. et al. Vibe-z: a framework for 3D virtual colocalization analysis in zebrafish larval brains. Nat. Methods 9, 735–742 (2012).

  20. 20.

    Randlett, O. et al. Whole-brain activity mapping onto a zebrafish brain atlas. Nat. Methods 12, 1039–1046 (2015).

  21. 21.

    Pinskiy, V. et al. High-throughput method of whole-brain sectioning, using the tape-transfer technique. PLoS ONE 10, e0102363 (2015).

  22. 22.

    Ioffe, S. & Szegedy, C. Batch normalization: accelerating deep network training by reducing internal co-variate shift. Proceedings of the 32nd International Conference on International Conference on Machine Learning 37, 448–456 (2015).

  23. 23.

    Fay, R. A. & Norgren, R. Identification of rat brain-stem multisynaptic connections to the oral motor nuclei using pseudorabies virus. i. Masticatory muscle motor systems. Brain Res Brain Res. Rev. 25, 255–275 (1997).

  24. 24.

    Yasui, Y. et al. Non-dopaminergic neurons in the substantia nigra project to the reticular formation around the trigeminal motor nucleus in the rat. Brain Res. 585, 361–366 (1992).

  25. 25.

    Li, Y., Takada, M., Kaneko, T. & Mizuno, N. Premo-tor neurons for trigeminal motor nucleus neurons in-nervating the jaw-closing and jaw-opening muscles: differential differential in the lower brainstem of the rat. J. Comp. Neurol. 365, 563–579 (1995).

  26. 26.

    Mizuno, N. et al. A light and electron microscopic study of premotor neurons for the trigeminal motor nucleus. J. Comp. Neurol. 215, 290–298 (1983).

  27. 27.

    Travers, J. B. & Norgen, R. Afferent projections to the oral motor nuclei in the rat. J. Comp. Neurol. 220, 280–298 (1983).

  28. 28.

    Stanek, E., Rodriguez, E., Zhao, S., Han, B. X. & Wang, F. Supratrigeminal bilaterally projecting neurons maintain basal tone and enable bilateral phasic activation of jaw-closing muscles. J. Neurosci. 36, 7663–7675 (2016).

  29. 29.

    Wickersham, I. R., Finke, S., Conzelmann, K.-K. & Callaway, E. M. Retrograde neuronal tracing with a deletion-mutant rabies virus. Nat. Methods 4, 47–49 (2007).

  30. 30.

    Takatoh, J. et al. New modules are added to vibrissal premotor circuitry with the emergence of exploratory whisking. Neuron 77, 346–360 (2013).

  31. 31.

    Johnson, G. A. et al. Waxholm space: An image-based reference for coordinating mouse brain research. Neuroimage 53, 365–372 (2010).

  32. 32.

    Roland, P. E. et al. Human brain atlas: for high-resolution functional and anatomical mapping. Hum. Brain Mapp. 1, 173184 (1994).

  33. 33.

    Pollack, J. D., Wu, D.-Y. & Satterlee, J. S. Molecular neuroanatomy: a generation of progress. Trends Neurosci. 37, 106–123 (2014).

  34. 34.

    Gonzlez-Vill, S. et al. A review on brain structures segmentation in magnetic resonance imaging. Artif. Intell. Med. 73, 45–69 (2016).

  35. 35.

    Papp, E. A., Leergaard, T. B., Calabrese, E. & Johnson, G. A. Waxholm space atlas of the Sprague Dawley rat brain. Neuroimage 97, 374–386 (2014).

  36. 36.

    MacKenzie-Graham, A. et al. The informatics of a c57bl/6j mouse brain atlas. Neuroinformatics 1, 397–410 (2003).

  37. 37.

    Yushkevich, P. A. et al. Using MRI to build a 3D reference atlas of the mouse brain from histology images. In Proc. International Society of Magnetic Resonance in Medicine Vol. 13 (Wiley, 2005).

  38. 38.

    Oh, S. W. et al. A mesoscale connectome of the mouse brain. Nature 508, 201–214 (2014).

  39. 39.

    Renier, N. et al. Mapping of brain activity by automated volume analysis of immediate early genes. Cell 165, 1789–1802 (2016).

  40. 40.

    Feng, D. et al. Exploration and visualization of connectivity in the adult mouse brain. Methods 73, 9097 (2015).

  41. 41.

    Lau, C. et al. Exploration and visualization of gene expression with neuroanatomy in the adult mouse brain. BMC Bioinformatics 9, 153 (2008).

  42. 42.

    Dempsey, B. et al. Mapping and analysis of the con-nectome of sympathetic premotor neurons in the ros-tralventrolateral medulla of the rat using a volumetric brain atlas. Front. Neural Circ. 11, 9 (2017).

  43. 43.

    Senyukova, O. V., Lukin, A. S. & Vetro, D. P. Automated atlas-based segmentation of Nissl-stained mouse brain slices. Programmi. Comput. Soft. 37, 245–251 (2011).

  44. 44.

    Amunts, K. & Zilles, K. Architectonic mapping of the human brain beyond Brodmann. Neuron 88, 1086–1107 (2015).

  45. 45.

    Frth, D. et al. An interactive framework for whole-brain maps at cellular resolution. Nat. Neurosci. 21, 139149 (2018).

  46. 46.

    Bakker, R., Tiesinga, P. & Ktter, R. The scalable brain atlas: instant web-based access to public. Neuroinformatics 13, 353366 (2013).

  47. 47.

    Zingg, B. et al. Neural networks of the mouse neocortex. Cell 156, 10961111 (2014).

  48. 48.

    Ng, L. et al. An anatomic gene expression atlas ofthe adult mouse brain. Nat. Neurosci. 12, 356–362 (2009).

  49. 49.

    Mazziotta, J. et al. A probabilistic atlas and reference system for the human brain: International Consortium for Brain Mapping (ICBM). Philos. Trans. Royal Soc. B 356, 1293–1322 (2001).

  50. 50.

    Parekh, R. & Ascoli, G. A. Neuronal morphology goes digital: a research hub for cellular and system neuroscience. Neuron 77, 1017–1038 (2013).

  51. 51.

    Miller, M. I., Beg, M. F., Ceritoglu, C. & Stark, C. Increasing the power of functional maps of the medial temporal lobe by using large deformation diffeo-morphic metric mapping. Proc. Natl Acad. Sci. USA 102, 9685–9690 (2005).

  52. 52.

    Tsai, P. S. et al. Correlations of neuronal and microvascular densities in cortex revealed by direct counting and colocalization of nuclei and vessels. J. Neurosci. 29, 12455314570 (2009).

  53. 53.

    Ragan, T. et al. Serial two-photon tomography for au-tomated ex vivo mouse brain imaging. Nat. Methods 9, 255–258 (2012).

  54. 54.

    Ren, J., Choi, H., Chung, K. & Bouma, B. E. Label-free volumetric optical imaging of intact murine brains. Sci. Rep. 7, 46306 (2017).

  55. 55.

    Tsai, P. S. et al. All-optical histology using ultrashort laser pulses. Neuron 39, 27–41 (2003).

  56. 56.

    Klein, S., Staring, M., Murphy, K., Viergever, M. A. & Pluim, J. P. Elastix: a toolbox for intensity-based medical image registration. IEEE Trans. Med. Imag. 29, 196–205 (2010).

  57. 57.

    Maes, F., Collignon, A., Vandermeulen, D., Marchal, G. & Suetens, P. Multimodality image registration by maximization of mutual information. IEEE Trans. Med. Imag. 16, 187198 (1997).

Download references

Acknowledgements

The idea for this project was catalyzed at the 2008 meeting on ‘The Architectural Logic of Mammalian CNS’ at the Banbury Center, Cold Spring Harbor Laboratory. We thank N.M. Lindsay for help with annotating the trigeminus, A. Brzozowska-Prechtl and H. Liechty for assistance with the histology, A. Newberry for assistance with coding, and X. Ji and K. Svoboda for timely discussions. This work was funded by NIH BRAIN awards (U01 grant nos. MH105971 and NS0905905 and U19 grant nos. MH114821 and NS107466), a Mathers Charitable Foundation award, and funds from the Dr. George Feher Experimental Biophysics Endowed Chair. We thank L. Enquist and the Center for Neuroanatomy with Neurotropic Viruses (NIH grant no. OD010996) for supplying the PRV.

Author information

Y.F., H.J.K. and D.K. conceived the project. Y.C. and Y.F. designed the algorithm. D.F., B.F., D.K., L.E.M., P.P.M. and A.S.T. contributed data. Y.C., Y.F., B.F., H.J.K., D.K. and L.E.M. planned experiments and analyzed data. Y.C., Y.F., B.F. and D.K. wrote the manuscript. D.K. and P.P.M. dealt with the many institutional organizations that govern animal health and welfare, surgical procedures and laboratory health and safety issues that include specific oversight of chemicals, controlled substances, cutting tools and viruses.

Competing interests

The authors declare no competing interests.

Correspondence to Yoav Freund or David Kleinfeld.

Integrated supplementary information

Supplementary Figure 1 Reconstruction of one expertly annotated brain out of three used to bootstrap the atlas.

a, Two-dimensional contours mapped to three-dimensional subject space. The gray level volume is shown for reference. b, Reconstructed landmark structures for this particular annotated brain.

Supplementary Figure 2 Graphical interface for annotation of landmarks.

The main panel shows the original full-resolution section image and structure contours. The side panels show virtual sections of the reconstructed gray level volume in three orthogonal planes. All panels are synchronized.

Supplementary Figure 3 Classification performance.

The receiver operator characteristic (ROC) curves for the classifiers of all 28 structures, using two annotated brains for training and the other annotated brain for test. The area under an ROC measures the performance of a classifier.

Supplementary Figure 4 Estimation of locations and shapes for the bootstrapped reference atlas.

a, Illustration of three annotated brains (blue, green, red) brought in registration in atlas space by global affine transforms. b, Top-down view of the atlas space. Each structure is represented by a color. Circles are instance centroids. Stars are the nominal centroids. Shaded plane is the common mid-sagittal plane. Note the symmetry of the nominal centroids of paired structure with respect to the mid-sagittal plane. c, Reconstructions of three annotated brains with facial motor nucleus (7N) in both sides highlighted. d, An example of the shape estimation for 7N using all six instances of the facial motor nucleus, one from each hemisphere. e, All six instances of the facial motor nucleus aligned using rigid transforms. f, Probabilistic average shape of the facial motor nucleus obtained by voxel averaging.

Supplementary Figure 5 Consistency between automatic and manual annotation.

a, Jaccard index, which ranges from 0 for completely disjoint to 1 for an exact overlap. b,c, The deviation of the centroid of a registered landmark from the expert annotation in absolute units (b) and normalized to the size of the landmark (c). The online source data for this figure include a listing of all data points. Source data

Supplementary Figure 6 Mapping of fluorescent intensities from Neurotrace-blue-stained sections to thionin-stained sections.

Data were obtained from brains with alternate sections stained with Neurotrace blue or thionin. The sections are rigidly aligned by Elastix, with mutual information as criteria. We then randomly sample ten pairs of regions, each 500 µm by 500 µm, from many pairs of adjacent sections and match the intensity histograms of corresponding regions. We match histograms of moderately sized regions rather than entire images because the global tissue content is likely to be different even for adjacent sections, while using regions of limited extent reduces this variance. a, Histogram of the pixel intensity of a region from a thionin section; histology shown as an inset. b, Histogram of the pixel intensity of a region from a Neurotrace blue section at the same level in the brain; histology shown as an inset. c, The estimated nonlinear mapping between the intensities of Neurotrace blue to those of thionin. We collected 1,000 such curves from five brains. The thick black line is for the section in a and b, and the other lines are for other pairs of sections. d, The histogram and image of the Neurotrace blue data in b after correction.

Supplementary Figure 7 Consistency between ChAT- and texture-based annotation.

This analysis is over two ChAT brains. a, Jaccard index. b,c, The deviation of the centroid of a registered landmark from the expert annotation in absolute units (b) and normalized to the size of the landmark (c). The online source data for this figure include a listing of all data points. Source data

Supplementary Figure 8 Quantification of human correction.

This analysis is over 13 brains. Only selected landmarks required corrections. a, Corrections in absolute units. b, Corrections normalized to the size of the landmark. The online source data for this figure include a listing of all data points. Source data

Supplementary Figure 9 Measures of registration confidence.

a, Landscape of the objective function for a particular registration in one mouse. Magnitude is normalized to yield z-scores. Significance metrics are the z-score of the estimated maximum, and the margin, that is, the distance from the maximum where the z-score drops to unity. b, The z-scores of all structure-specific registrations across 12 mice, as indicated. c,d, Lower bound of all structure-specific registrations in absolute (c) and normalized (d) coordinates across 12 mice, as indicated. e,f, Upper bound of all structure-specific registrations in absolute (e) and normalized (f) coordinates across 12 mice, as indicated. The online source data for this figure include a listing of data points in bf. Source data

Supplementary Figure 10 The variation in positions of structures around respective nominal centroids.

Different brains are represented by different colors. a, The s.d. in three dimensions across 12 mice, as indicated. The sample-averaged root-mean-square s.d. is 160 ± 40 µm. bd, The s.d. across 12 mice, as indicated, projected along the medial–lateral (b), dorsal–ventral (c), and rostral–caudal (d) axes. The online source data for this figure include a listing of data points in bd. Source data

Supplementary Figure 11 Deformation fields derived from global registration and landmark-specific registration for an example section.

Contours are the cross-sections of 0.5-level iso-surfaces of the aligned atlas structures. Grid lines represent the transformed result of a regular grid defined in atlas space. a, Results after global registration. Structures are placed reasonably close to the correct positions, but individual adjustment is still necessary. Grid lines exhibit an affine transformation. b, Results after structure-specific registration. Structure pose and locations are improved. Warped grid lines demonstrate the final deformation field.

Supplementary Figure 12 Results for the area under the receiver operator characteristic curve (ROC) for three different patch sizes.

Classifiers were trained using two brains, MD585 and MD589, and the accuracy was measured against a third brain, MD594. The online source data for this figure include a listing of all data points. Source data

Supplementary information

Supplementary Information

Supplementary Figures 1–12 and Supplementary Tables 1 and 2

Reporting Summary

Source data

Source Data for Fig. 4

Source Data for Supplementary Fig. 5

Source Data for Supplementary Fig. 7

Source Data for Supplementary Fig. 8

Source Data for Supplementary Fig. 9

Source Data for Supplementary Fig. 10

Source Data for Supplementary Fig. 12

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark
Fig. 1: Structure of an automated atlas.
Fig. 2: Workflow for training the atlas, which consists of annotating brain sections followed by computation.
Fig. 3: Workflow to align a new brain with the current reference atlas.
Fig. 4: Reliability and variability in estimates of landmark position for new brains.
Fig. 5: Application of the texture-based alignment to fluorescent imaging within and across brains.
Fig. 6: Defining a landmark on the basis of texture versus grayscale.
Supplementary Figure 1: Reconstruction of one expertly annotated brain out of three used to bootstrap the atlas.
Supplementary Figure 2: Graphical interface for annotation of landmarks.
Supplementary Figure 3: Classification performance.
Supplementary Figure 4: Estimation of locations and shapes for the bootstrapped reference atlas.
Supplementary Figure 5: Consistency between automatic and manual annotation.
Supplementary Figure 6: Mapping of fluorescent intensities from Neurotrace-blue-stained sections to thionin-stained sections.
Supplementary Figure 7: Consistency between ChAT- and texture-based annotation.
Supplementary Figure 8: Quantification of human correction.
Supplementary Figure 9: Measures of registration confidence.
Supplementary Figure 10: The variation in positions of structures around respective nominal centroids.
Supplementary Figure 11: Deformation fields derived from global registration and landmark-specific registration for an example section.
Supplementary Figure 12: Results for the area under the receiver operator characteristic curve (ROC) for three different patch sizes.