Several projects have generated electron microscopy data sets of regions of mammalian brain such as the cortex and the retina. Although extremely useful and widely used, these data sets have not been reconstructed to such a level that each individual synapse and each fine neurite branch are accounted for. To generate an exquisitely detailed reconstruction of a volume of the mouse neocortex, Jeff Lichtman at Harvard University has spearheaded a collaborative effort involving his own lab and labs at Johns Hopkins University, Massachusetts Institute of Technology and Duke University. “When it comes to the nitty-gritty of the brain, its finest details, there was really still quite a paucity of knowledge,” says Lichtman, explaining his motivation for this six-year project.

Multiple synapses between one axon (blue) and the spines of one dendrite (green). Reprinted from Cell, Vol. 162, Kasthuri, N. et al., Saturated reconstruction of a volume of neocortex, 648–661, Copyright 2015, with permission from Elsevier.

Even though the reconstructed volume of about 1,500 square micrometers is only a small piece of the whole mouse brain, the effort required the automation of many of the steps in the data acquisition and analysis pipeline. Lichtman says that his goal was to “get on with the interesting biology and have machines do most of the data acquisition.” For example, his team developed an automatic tape-collecting ultramicrotome that produces a thousand 30-nanometer-thick sections of tissue, of nearly flawless quality and without the loss of any sections, within 24 hours. Furthermore, image acquisition is almost fully automated and requires only minimal human oversight. Lichtman explains that at the beginning of the project they could automatically image only two or three sections; now, they can complete hundreds of sections without human intervention.

But acquiring the images is just the beginning. For reconstruction of the entire tissue volume, images need to be aligned and segmented. Automation of the alignment process is not too difficult, but automatic segmentation is more challenging. “Humans...do not have trouble to speak of in getting a reconstruction, but the computers continue to make mistakes,” concedes Lichtman. While improving the automatic segmentation algorithms, Lichtman and his team developed tools to assist researchers in correcting segmentation errors that arise.

Connectomic reconstructions are usually visualized using skeletons, meaning that neuronal processes are represented as thin lines. Lichtman's data set differs from others obtained with these approaches. He explains that he and his collaborators were “filling in every little nook and cranny of every little piece of every section.” They wanted to obtain a saturated reconstruction, in which every pixel was accounted for. In this way, they did not miss any small side branches or other structures. This reconstruction process took a lot of time, but they essentially ended up with what Lichtman calls a “digital piece of brain.”

Even though generating the data set was technically challenging, “far more time went into the analysis than into the acquisition,” says Lichtman. He mentions that it was hard to turn raw data into a minable data set that could be interrogated with biological questions; in fact, this part of the work required a substantial amount of manual analysis. The researchers generated a list of every synapse and dendritic spine and their morphological properties. This spreadsheet is currently maintained at the Open Connectome Project at Johns Hopkins University and should be a useful resource for other researchers to explore and query.

Lichtman plans to further optimize the imaging and analysis pipeline. Multibeam microscopes have already helped increase the speed of data acquisition by almost two orders of magnitude. And troubleshooting errors is an area where the team is constantly making improvements. Although they have already found ways to deal with commonly occurring problems, there is still a need to fix the less common problems. “Now we are in a mode of tens of thousands of sections, and again, we are seeing things that were so rare we never saw them before, but they shut you down until you fix them,” says Lichtman.

Lichtman is currently gearing up to tackle a larger volume of the mouse cortex, which will be on the order of a cubic millimeter. Given that this project will generate about 2,000 terabytes of data, the question remains of how to handle these huge data sets. Luckily, prices for data storage are constantly on the decline. “The gigabytes of my middle age have become the terabytes of today. And the terabytes of today are soon going to be the petabytes of the next generation,” muses Lichtman. But transferring the data between labs remains a problem. “At the moment, if you have to send somebody a thousand terabytes of data, the fastest way to get it is by truck or by air,” explains Lichtman. As data transfer is not just an issue for connectomics projects, Lichtman expects commercial solutions to help.