The field of connectomics is pulling neuroscience into a speedy, high-throughput lane that is generating vast amounts of data.
Researchers seeking to understand the brain want big data. And they are getting them. Just as geneticists have moved from genes to genomes to the interacting network of factors that regulate and modify the genome, neuroscientists are going from studying single neurons to tracing how vast neuronal networks connect and interact.
“I think this is a really exciting field,” says neuroscientist Moritz Helmstaedter at the Max Planck Institute for Neurobiology in Martinsried, Germany, who is working to obtain a cell-level overview of the neuronal connections — the connectome — of the mammalian cortex. “Many people are pretty ambitious about breaking the next barrier in understanding how the brain works by using this new field of connectomics.”
Sceptics argue that current methods lack the power to map the massively interconnected web of around 100 billion neurons in the human brain. Even if technology can rise to the challenge, they say, it is impossible to decipher so much data.
Clay Reid, a neuroscientist at Harvard Medical School in Boston, Massachusetts, and recently appointed as a senior investigator at the Allen Institute for Brain Science in Seattle, Washington, counters detractors by pointing to recent progress in neuroscience. A few years ago, it was nearly impossible to collect data on networks of neurons. “It's not routine now but it's easier,” he says. And, he adds, even without a map of the entire brain, charting just a fraction of a neural circuit is an important advance.
The way the neuroscience community is embracing connectomics and its big-data harvest is part of a technical and cultural shift. “We're moving away from artisan cottage-industry science and going after bigger, harder, more complex problems,” says geneticist Geoffrey Duyk, who is partner and managing director at the venture investment company TPG Biotech in San Francisco, California.
Advances in high-throughput technology and automation are giving neuroscientists both speed and scale. By combining whole-brain magnetic resonance imaging and computational methods, some teams are mapping the major routes of traffic in the human brain. Other researchers are concentrating on animal models to chart the brain's neuronal circuitry on a much smaller scale, at the level of individual neurons and their projections of axons, dendrites and synaptic connections. The goal is to add the information gleaned to the animal and human connectomes, building them up as resources for understanding behaviour in health and disease.
Studying the neuronal web at every possible scale demands the integration of every available method. Old-fashioned staining techniques are used alongside new methods of tissue preparation and methods taken from genomics (the study of genomes), and traditional and new approaches are used in microscopy and image analysis. Automation pipelines and computational methods are essential to handling the data — but so is skilled, manual artistry.
The brain is the only organ for which the number and types of cells it contains has not been determined. Just being able to differentiate these cells from one another under a microscope is an important advance. Jeff Lichtman and his colleagues at Harvard University's Center for Brain Science in Cambridge, Massachusetts, apply light and electron microscopy to study how neural circuits change over the course of development. Using a genomic technique called Brainbow, the group was able to make specific DNA modifications in transgenic mice to label neurons fluorescently in more than 90 colours1. The researchers could then distinguish individual neurons in the brain's dense tangles of otherwise identical neurons. Separately, the Brainstorm Consortium, which is composed of scientists from Harvard and the Massachusetts Institute of Technology (MIT), in Cambridge, Massachusetts, and from Stanford University, California, is also working on new methods for application in connectomics, in areas such as brain-tissue preparation, imaging and image analysis.
Because sample preparation in neuroscience is labour and time intensive, several groups are working on improving it. Lichtman's team has developed the automatic tape-collection mechanism for ultramicrotomes (ATM), which automatically sections brain tissue and collects thousands of slices on a moving tape. The slices can then be stained and imaged on a scanning electron microscope. “It vastly improves sectioning methodology,” says neuroscientist Scott Emmons, who has just set up an ATM in his lab at the Albert Einstein College of Medicine in New York. Classically, Emmons explains, sections are floated on water and then gathered manually. “They got lost, they get scrambled up, you cannot make long series,” he says.
At the moment, imaging and image analysis can be only partially automated. But, for connectomics researchers to obtain the massive amounts of imaging data they want, technology will need to be expanded. Lichtman and Winfried Denk of the Max Planck Institute for Medical Research in Heidelberg, Germany, note that this approach will necessitate “the automation, even industrialization of imaging”2.
High-throughput neuroscience reaches industrial scale at the Allen Institute for Brain Science. When a project is explored for feasibility, the institute looks for a 'brute force' method to convert it to managed, pipelined programmes with milestones and schedules, says Amy Bernard, director of structured science at the institute. This approach has been adapted from industry, says Chinh Dang, the institute's chief technology officer (see 'Neuroscience goes industrial').
Large-scale efforts at a number of other labs take on circuits in big ways. At Harvard Medical School, Reid and his colleagues have been mapping and studying neural connections in the mouse visual cortex. To do this, they first looked at a brain region in vivo, using two-photon microscopy — a technique that is good for imaging live tissue — and indicator molecules that show when neurons fire and release calcium. They then captured images from fixed tissue with a custom-built serial section transmission electron microscope camera array3.
The team captured more than 3 million digital images of the mouse cortex and, in collaboration with the Pittsburgh Supercomputing Center in Pennsylvania, stitched them together into 1,200 montages comprising 10 billion pixels apiece, and aligned them in three dimensions. The computing support was key. “We would not have been able to do the work that we did last year without the help of that team,” says Reid.
Denk, Helmstaedter and Kevin Briggman from the National Institute of Neurological Disorders and Stroke in Bethesda, Maryland, used a labour-intensive approach in their recent study of a mouse retinal circuit. They, too, used two-photon imaging with calcium indicators to capture brain functional patterns in vivo and then imaged fixed tissue to obtain ultrastructure information4.
But for that last step they used serial block-face scanning electron microscopy, an automated technique in which a block of tissue is imaged and then the top slice is shaved off to image the next slice. With their own visualization-annotation software tools, such as KNOSSOS (www.knossostool.org), they annotated the images to show how neurons branch and interconnect.
Reaching out to crowds
Both teams performed precise segmentation on the images they obtained: tracing the branching structure of neurons and the synapses between them. This approach has revealed links between structure and function — illuminating, for example, how the many types of neurons in the retinal circuit compute visual signals. “The way we got there was entirely segmentation by hand,” says Reid, who says that the human eye remains best suited for the task, given our innate skill at recognizing patterns.
Ultimately, however, researchers such as Sebastian Seung at MIT and Dmitri Chklovskii at the Howard Hughes Institute's Janelia Farm research campus in Ashburn, Virginia, hope to teach machines how to perform segmentation. “Image processing is currently the bottleneck,” says Viren Jain, a neuroscientist at Janelia Farm.
One component of that bottleneck is the fact that circuits can be metres long. Contouring a path just one-third of a metre long would take a human annotator some 60,000 hours, or around 30 years assuming a normal work week. However, if manual reconstruction has its challenges, so too does automation, note Denk, Briggman and Helmstaedter5,6. Speeding up image analysis is hindered by stains that can emphasize the cell surface but obscure the visibility of synapses. And tracing a circuit is dogged by errors even with experts doing the work, which makes it hard to imagine how a computer program could do as well or better.
Ultimately, new tools are needed to determine the 'volume' of neurons — that is, when different pathways touch each other and may connect. In the meantime, Reid's Harvard team continues to collect electron microscopy data and analyse them using open-source tools such as TrakEM2 (go.nature.com/kgsalt) and the Collaborative Annotation Toolkit for Massive Amounts of Image Data (CATMAID) (www.catmaid.org). “Once the computers take over,” he says, wiring diagrams will capture more connections in a circuit. “All of us are convinced that we can get tens of thousands or maybe hundreds of thousands of connections, and then it will really become circuit science.”
Until then, one way of speeding up segmentation is to use crowds to sift through the data and then reconcile the many results. The software for tracing and annotating electron micrographs initially ran mainly on expensive lab computers, which limited access, says Helmstaedter. But in 2006, he and his colleagues converted KNOSSOS into distributable software and began recruiting students, of whom they now have about 200, to annotate electron micrographs at home or in the dorm. “This particular combination of being able to browse these extremely large data sets and still do it on a laptop is really so far unique,” he says.
The team uses different software, the redundant-skeleton consensus procedure (RESCOP), to reconcile the many annotations generated by the students, which overlap in some ways and conflict in others.
Helmstaedter and his colleagues are now looking to expand the crowd working on their data to include members of the public. To entice participation, the team has hired gaming programmers to add an element of play. “The idea is basically to fly through the brain, it has the feel of a flight simulator,” he says of the prototype. Similarly, scientists at MIT have launched Wired Differently (www.wireddifferently.org) to engage crowds in neuron tracing.
In the long term, Helmstaedter hopes that automation will become so good that people are only needed for annotating complex brain regions. Having just launched his own lab, he is gearing up to pursue his goal of a cell-level connectome of the mouse cortex, which will involve tracing the connections between billions of neurons. “That is really an issue of scale,” he says. “We need a factor of 100 in terms of annotation efficiency.”
Small labs think big
High-throughput neuroanatomy might seem to be for large labs only, but the methods they use will enable smaller ones to follow in their path, Helmstaedter says. A few smaller labs are already scaling up, using new methods to pull more data out of optical and electron microscopy images than was possible a few years ago.
In 2004, fresh from his postdoc at the University of Southern California in Los Angeles, where he studied connectivity in the rat brain, neuroscientist Hong-Wei Dong was among the first scientists to be recruited to the Allen Institute to create the Allen Brain Atlas, a map of gene expression for the entire mouse brain.
“I never thought science can be done on this kind of scale,” says Dong, who has now left the Allen Institute to start his own lab at the University of California, Los Angeles (UCLA). “If I didn't have that kind of experience, I probably would have never thought of mapping the connectivity for the entire brain.” At UCLA he launched iConnectome, a large-scale optical-imaging project, the aim of which is to create a three-dimensional connectome of the mouse brain. He uses optical microscopy and fluorescent markers to capture pathway information; the images are on a coarser scale than with electron microscopy but they show how brain regions interact.
Dong's methods include classic neuroanatomical techniques of surgery, preparing and mounting tissue on slides, using tracers to show neuronal inputs and outputs, and classic staining techniques such as Nissl staining, which shows the architecture of brain regions. “That is all manual labour,” he says.
In neuroanatomy, tracers are usually injected into a brain region one at a time. In Dong's project, animals receive two injections in two sites, allowing scientists to examine input and output pathways concurrently, and yielding four times more data than in studies with single tracers. It also cuts down on the cost, time and number of animals needed, he says.
In the decade since his postdoc, Dong notes that scanning tissue slides has sped up dramatically, a trend he expects to continue. Microscopes can automatically scan slides one by one, each of which holds 50–100 tissue sections. Olympus, headquartered in Tokyo, Hamamatsu in Hamamatsu City, Japan, and Aperio, which was recently acquired by Leica Biosystems in Nussloch, Germany, all make high-throughput slide scanners. But speed is not everything, because analysis must be done with care. For example, Dong says that minor wrong-way tracer transport can lead to false conclusions.
And not all image-processing steps gain speed through automation. For example, tissue slices are registered automatically to a corresponding image in the Allen Reference Brain Atlas, but contrast and brightness require manual adjustment.
Even the process of automatic registration can be difficult, because slices differ slightly from one another, causing distortion. To match each image to the reference, Dong's team keeps enhancing the registration algorithm. “That is a difficult part, but a key part for the future if we want to evolve high-throughput analysis of the data,” he says. It is also not easy to keep good computing staff, who often give in to the lure of Silicon Valley.
The three-dimensional visualization in iConnectome allows users to navigate through the brain's pathways, and layers images of tracer-labelled neurons against the backdrop of the atlas reference images, giving them a geographical context. “We put this map behind each individual brain section, and as people change the opacity, the atlas can give them a reference,” Dong says. The combination of high-throughput approaches and online presentation enables brain-structure information to be shared beyond neuroanatomy circles, he says, to researchers focusing on physiology or behaviour.
While Dong finds ways to offer data-laden images to others, in his garage sit dozens of boxes of his own slides — around 6,000 of them, containing a total of 60,000 individual tissue slices from 200 rat brains. He hopes to image them so that he can share his work on the stria terminalis brain region with others.
Brendan Brinkman, senior product manager of the Olympus America Scientific Equipment Group, based in Center Valley, Pennsylvania, has worked with Dong and other neuroanatomists who use slide-scanning and point-scanning confocal technology. To hasten data capture in microscopy further, the company is expanding its multi-point mapping software, Brinkman says. In scanning confocal imaging, a raster scan moves across a sample. To focus only on differences in fluorescence, the scanner can jump from one area to another and capture bursts of data. “You can adjust the scanning path to make it as fast as possible,” he says. Olympus has also tailored the software to let scientists capture multiple-channel fluorescent signals. “Certainly, neurobiology is the key group for this kind of technology,” he says.
And connectome projects are increasing sales of slide scanners, as researchers seek the quick generation of data from large sample sets. TPG Biotech's Duyk agrees that high-throughput approaches to neuroanatomy could be a commercial boon. “It certainly creates opportunities for the life-science tools companies to push the cutting edge of their technology,” he says.
For those lacking these tools, services are emerging. Earlier this year, for example, Renovo Neural, a spin-out from the Cleveland Clinic in Ohio, launched an electron microscopy service. Customers deliver samples, which the company sections with an ultramicrotome from Gatan in Pleasanton, California, then images with an automated serial block-face scanning microscope from Zeiss in Jena, Germany, to return hundreds of ultrastructure images.
The first complete connectome was obtained for the worm Caenorhabditis elegans in 1986. Sydney Brenner and his colleagues at the MRC Laboratory of Molecular Biology in Cambridge, UK, completed their wiring diagram of the hermaphrodite worm nervous system by tracing images of neurons on electron micrographs by hand7. The hermaphrodite is one of the animal's two genders; but the male worm has proven to be tougher to pin down, neuronally speaking.
Teams tried to map the male's nervous system as Brenner's group had for the hermaphrodite, but assembling its branching structure by hand was too hard. “The field of connectomics kind of got stuck,” Emmons says. Electron microscopist David Hall at the Albert Einstein College of Medicine is the custodian of material from the Brenner lab: notebooks, embedded tissue blocks, thin sections, negatives and the manually annotated C. elegans electron micrographs. This year, he, Emmons and colleagues revisited 5,000 of the historic images, analysing them in a new way8.
They translated their analysis of the micrographs into a map of all the connections and their strengths in the male C. elegans posterior nervous system. Of the 170 neurons they studied, 144 were involved in the circuit controlling mating behaviour, allowing the team to make a link between connectivity at the cellular level and behaviour.
Previous researchers had counted synapses but had not included synapse size, Emmons says, which the team now did. Then they applied a mathematical model to use size as a proxy for the functional strength of each neural connection. The result is a map of the neural network's connectivity that includes quantitative cell-biology information. Across the connectome, they found that interaction strengths between neurons varied more than 100-fold.
Using the existing maps from Brenner's lab made neuron tracing easier. “Somebody has already gone before you and put coloured numbers on them,” Emmons says. Elegance (go.nature.com/nvsfnn), the group's software tool, accelerated neuron tracing by translating mouse clicks into map coordinates as the team scrutinized the digitized micrographs.
Emmons thinks that his methods can speed up connectomics efforts in other small organisms. His group will expand its range in several ways; for example, by using their new ATM machine to automatically collect a series of slices for later imaging. Ultimately, says Emmons, using the ATM, “we're hoping for a 20,000 unbroken series, which would cover an entire worm, which has never been done before”. Technology like this gives a small lab the tools to tackle large projects such as finishing the male worm's connectome, comparing it to the hermaphrodite or mapping the developing nervous system. The C. elegans community is “back in the connectomics business”, he says.
Emmons will move on to analyse synaptic connections in the mouse brain, although, for now, his focus is on the worm. The ATM will deliver brain sections aplenty for imaging, but he is confident that his new scanning electron microscope with its bigger field of view is up to the task. “With one shot we can cover a whole worm cross-section,” he says.
Applying these techniques to mammalian brains takes more than automation. The human brain has over 80 billion neurons and the mouse brain has around 70 million. And both have more densely woven webs of neurons than in the worm. “So you can't just scale up from a little lab and make a big lab and do it,” Emmons says.
Big labs mapping large circuits on the single-neuron level are trying new scale-up approaches. Deciphering mammalian neural circuits is Reid's goal in his new position as senior investigator at the Allen Institute, to which he was appointed as part of the institute's ten-year US$300-million move to map connectomes and to use them to reach a broader understanding of brain function that integrates genes, circuits and behaviour.
This project, called MindScope, is an attempt to go beyond the anatomy and wiring of the brain to how things are computed in the cortex, by having scientists work side by side to study cell types, neural coding, modelling analysis and theory. “It's a dream come true,” says Reid.
He will use transgenic mice to identify the different cell types in the cortex and thalamus, and will then focus on deciphering the neural coding in the visual parts of the brain using a combination of techniques: behavioural analysis, physiology, imaging with calcium indicators and electron microscopy. The results will be compiled into what he calls network anatomy, which is a wiring diagram with information piled onto it to map and understand the connectome's dizzying array of functionalities.
As the wealth of data from using different imaging modalities and from integrated large-scale projects comes in and is collected and annotated, labs large and small will still need to put their heads and computing power together for data analysis. “Astronomical amounts of connectomics data are being generated at an exponential rate; extracting meaning from it is the bottleneck that hasn't been broken,” says Larry Swanson, neuroscientist at the University of Southern California and president elect of the Society for Neuroscience in Washington DC.
Livet, J. et al. Nature 450, 56–62 (2007).
Lichtman, J. W. & Denk, W. Science 334, 618–623 (2011).
Bock, D. D. et al. Nature 471, 177–182 (2011).
Briggman, K. L., Helmstaedter, M. & Denk, W. Nature 471, 183–188 (2011).
Denk, W., Briggman, K. L. & Helmstaedter, M. Nature Rev. Neurosci. 13, 351–358 (2012).
Helmstaedter, M., Briggman, K. L. & Denk, W. Nature Neurosci. 14, 1081–1088 (2011).
White, J. G., Southgate, E., Thomson, J. N. & Brenner, S. Phil. Trans. R. Soc. Lond. B 314, 1–340 (1986).
Jarrell, T. A. et al. Science 337, 437–444 (2012).
Related links in Nature Research
Related external links
About this article
Automated Brain Region Segmentation for Single Cell Resolution Histological Images Based on Markov Random Field
Identifying Weak Signals in Inhomogeneous Neuronal Images for Large-Scale Tracing of Sparsely Distributed Neurites
PLOS Computational Biology (2017)
Biology of the Cell (2016)
Serial block face-scanning electron microscopy: A tool for studying embryonic development at the cell-matrix interface
Birth Defects Research Part C: Embryo Today: Reviews (2015)