Alan Rudolph sounds like an over-excited sci-fi fan. In his vision of future warfare, the neural circuits of military personnel will be wired into the silicon circuits of the equipment that they control. The brains of fighter pilots, for example, could be connected direct to the controls of their planes. By sending signals to the plane by radio waves, and using cameras on the front of the plane to relay images into the visual areas of the brain, pilots could even control fighter jets from the safety of the ground.

But Rudolph has the clout to do more than fantasize. A project coordinator at the US Defense Advanced Research Projects Agency (DARPA), last year he allocated US$24 million — almost 10% of DARPA's basic research budget — to a two-year project involving mathematicians, biologists and materials engineers charged with developing technologies to interface brain and machine. Many stumbling blocks, both ethical and technological, will have to be overcome. But agency officials have no doubt about the project's potential. “In the long run, we could have brain-to-brain communication; we could improve the performance of normal healthy individuals,” says Rudolph.

Mind reader: electrodes are inserted into the cortex of a macaque to monitor neural activity.

DARPA began dabbling in neuroscience in the early 1990s, when the agency grew neurons on silicon circuits and used them to detect chemical nerve agents. In 1998 the programme expanded, developing projects that included research into how honeybees manage to be so manoeuvrable at high speeds. But the agency's neuroscience work didn't attract much public interest until last May, when Sanjiv Talwar, a bioengineer at the State University of New York in Brooklyn, published work on what came to be known as Roborat1.

Using funding from DARPA, Talwar's team implanted electrodes in the brains of five rats: one in the medial forebrain bundle, which is associated with pleasurable reward feelings linked to actions such as eating or drinking, and two others in the parts of the brain that process signals from the rat's left and right whiskers. Using signals to the reward area as an incentive, the researchers were able to train the rat to move left or right in response to signals to the appropriate whisker area. The rats could then be guided through a maze, along ledges and even be instructed to climb and jump.

The experiment showed a crude degree of communication, using just three electrodes, but it proved that useful information could be sent directly into an animal's brain. What would be possible, wondered Rudolph, if more connections could be made? Could detailed information, such as images or sound, be beamed into the brain? Could such information be extracted by recording neural signals? And, most important, could it work in humans, so that images, speech and messages could be exchanged between brain and machine, or even between brains?

These hopes led to last year's formation of the Brain Machine Interface programme. The scheme is loosely structured into three areas — motor skills, sensory perception and memory. Rudolph identified the neuroscientists, materials engineers and mathematicians who he thought were best suited to the programme. After soliciting proposals from these researchers, a DARPA expert team selected the projects that it believed would bring the agency closer to achieving brain-to-machine and brain-to-brain communication.

Motor motivation

At Duke University in Durham, North Carolina, neuroscientist Miguel Nicolelis and his colleagues are tackling the motor cortex, a brain area that controls movement. The team made a name for themselves, and attracted some DARPA funding, during the mid-1990s, when they taught a rat to control a lever via electrodes implanted in its brain2. The rat first learnt to press the lever to earn a drink of water. As the rat pressed, Nicolelis recorded signals from 46 neurons in the motor cortex. He then cut the connection between the lever and the drink. The frustrated rat pressed the bar repeatedly, but received water only when the 46 neurons repeated the same firing pattern that Nicolelis had originally recorded. After a few hours, the rat learned to earn water through thought alone.

When Nicolelis moved his experiments into a tiny owl monkey named Belle at the end of the decade, the implant had grown in complexity and the activity of around 100 neurons could be monitored. As Belle used a joystick to follow a cursor across a screen, a computer compared the movement of the joystick with the signals from the implanted electrodes. After learning how different signals related to commands for speed, direction and force of movement, the computer was, unknown to Belle, able to use her brain activity to drive a robot arm in another room3.

Since then, Nicolelis has started using macaque monkeys, because their brain morphology more closely resembles that of humans. This time, the aim is for the monkey to learn to move a robot arm using signals sent directly from its motor cortex. Seven hundred microelectrodes are positioned in ten different regions of the motor cortex, recording signals from close to 300 neurons. At the same time, tiny sensors monitor signals from the monkey's arm muscles, which compare the monkey's intended trajectory with that of the robot arm. “There are a lot of signals being monitored together,” says Nicolelis. “On a loudspeaker, it is like listening to a symphony.”

The work is currently undergoing peer review for publication, so Nicolelis cannot reveal many details. But, he says, he has been able to send back signals from the robot arm to the monkey's brain. Rather than learning to move the arm by watching it, the monkey can now direct an unseen robot arm using feedback transmitted direct to its brain. Suddenly Rudolph's dream doesn't seem so unrealistic.

While Nicolelis is extracting movement information, other DARPA-funded neuroscientists are working out how to transmit sounds and images into brains. Tomaso Poggio and James DiCarlo at the Massachusetts Institute of Technology, together with Christof Koch at the California Institute of Technology, are attempting to unravel the brain functions that underlie visual object recognition. DiCarlo is targeting a part of the brain called the inferotemporal cortex, thought to be critical for object recognition, and monitoring the activity of neurons in the brains of monkeys performing visual tasks.

Auditory brain functions are being tackled by Jon Kaas at Vanderbilt University in Nashville, Tennessee, who is trying to decipher the codes that auditory areas use to represent sounds. Kaas is currently recording the responses of different populations of neurons in the auditory cortex of macaques. He says that he can reliably record from about 30 neurons, but that the potential to record from hundreds of neurons exists. “We haven't yet been able to record in large numbers or in the locations we want,” he says.

Eventually, both groups hope to have developed enough understanding of how images and sounds are represented in the areas they are studying to be able to send signals back into those areas, creating the feeling that the animals are perceiving real stimuli. “Ultimately the goal would be to electrically stimulate the neurons and see if the animals recognize the sound,” says Kaas.

Finally, neuroscientists Sam Deadwyler of Wake Forest University and Ted Berger at the University of Southern California have teamed up to tackle the hippocampus, an area of the brain involved in the storage of memories. They aim to test whether silicon can replace parts of our brain. Their focus is a three-stage chain of regions within the hippocampus that signals are passed through during memory storage. The researchers want to see if they can build a microchip that can take signals from the first region and relay them to the final stage, bypassing the middle part of the chain.

Microprocessor memory

Thought control: Miguel Nicolelis with Belle and the robotic arm that can mimic her movements. Credit: J. WALLACE/DUKE UNIVERSITY NEWS

Berger's team has started by developing a picture of what the hippocampus does. By stimulating the neurons in slices of rat hippocampus and monitoring the output that the stimulation produces, they hope to build a mathematical model that mimics processing in the hippocampus. “We'll have a model that will predict how the hippocampus will respond to any input,” says Berger. From there, they hope to develop a microchip that recreates that same processing.

Deadwyler has complemented Berger's research by recording the activity of individual neurons in the hippocampus of rats. This work should help the pair decide where best to insert the microchip when it is ready in around two years. One of the hurdles is being able to build the electronics on a micro-scale so that the chip can fit beneath the skull.

Despite the military funding, all the projects have potential medical benefits. DiCarlo's work, for example, will improve our knowledge of the little-understood inferotemporal cortex. Lesions in this region cause agnosia — a rare condition that prevents people from recognizing or identifying objects. He is targeting a part of the brain that has not been tested before. “It's not a new technique: it just hasn't been used in this area,” he says. We know very little about the organization of the neurons; that is what makes it a high-risk project.“

A synthetic hippocampus, on the other hand, could help those suffering from Alzheimer's disease, stroke and epilepsy, says Berger. ”The malfunctioning hippocampus is difficult to care for,“ he points out.

Scanning signals

But Berger is well aware that DARPA's goals go beyond substituting a chip for neurons. Although the researchers involved seek therapeutic devices, each device could, in theory, be used to enhance normal human function. In the long run, Rudolph hopes that these different strands of work will give birth to technologies that allow images to be relayed direct to the brains of military personnel. When they decide how to act on the information they receive, electrodes in their brains will decipher their decisions and relay them to the minds of their colleagues.

Pilots, for example, could receive sensory input and manoeuvre their plane through thought alone. Berger envisages a scene reminiscent of The Matrix, in which a complex evasive manoeuvre could even be programmed into future versions of his memory implants, allowing the pilots to perform moves they may not actually have learned through traditional training. Berger adds that the system could detect that a pilot was about to make an error and override his actions. ”When you are flying an F-18 you can't afford to make a mistake,“ he says.

But are such goals anything more than the fantasies of military and scientific futurists? In the immediate future, technological problems may block progress. The brain views an implanted electrode as a foreign invader and sends cells to encapsulate the electrodes in tissue, preventing signal transmission. By using flexible, teflon-coated electrodes, Nicolelis has managed to keep electrodes implanted in the brains of the monkeys for two years and maintain the integrity of the recorded signal. Even so, he says that more work needs to be done on the biocompatibility of different materials.

In the longer term, developing a non-invasive method of recording brain signals could be a bigger stumbling block. ”We don't have any plan or interest in taking the devices we have now and implanting them into healthy people,“ says Rudolph. Existing non-invasive systems are not up to the job. Medical scanning procedures such as magnetic resonance imaging (MRI) and positron emission tomography (PET) have provided enormous insight into brain functions. But MRI measures blood flow and PET traces the movement of molecules tagged with a radioactive isotope — neither has the resolution to monitor the activity of individual brain cells. ”There is nothing that directly measures the electrical signals that flow back and forth between neurons,“ says Rudolph.

The solution may come from future projects headed by materials engineers. ”DARPA is good at mixing up communities of disciplines,“ says Rudolph, who adds that if a microchip and electrodes could be implanted into the brain without harm or pain, there might not be a need for a non-invasive device. ”But these are things that we can't envisage right now,“ he says.

While DARPA wrestles with these long-term problems, the neuroscientists involved are simply pleased to be able to carry out such audacious research. Most say that if it weren't for DARPA, they would not be doing the work they are now. ”The military has always been quite visionary when funding neuroscience, whereas the National Institutes of Health (NIH) has been conservative,“ says Koch.

Whose responsibility?

In effect, each agency approaches neuroscience from the opposite perspective. The NIH pursues brain science from the ground up, and funded many of the early projects of these researchers. ”At the NIH you do incremental science, taking little bitty steps,“ says Berger. DARPA, by contrast, prefers riskier projects with higher payoffs. ”We hope that by building the interfaces we will learn how it can be done,“ says Rudolph.

But money from DARPA can come with different problems. Unlike NIH projects, which may be funded for five years at a time, researchers working for DARPA could find their funding pulled out from under them less than two years later, leaving employees and graduate students stranded. ”You begin to feel insecure,“ says Kaas.

More important, some feel that using money from the military is ethically problematic. Many of the researchers funded by DARPA feel that the agency's goals are a little fanciful, and none believe that the military is close to achieving its ultimate aim. Nonetheless, several of the researchers that Nature spoke to were reluctant to discuss military applications, saying they would rather see their research used benevolently. For some observers, this is an abdication of the responsibility that researchers have for their work.

Martha Farah is a neuroethicist and director of the Center for Cognitive Neuroscience at the University of Pennsylvania, Philadelphia. She argues that a researcher who accepts funding from the Department of Defense, but does not believe in the goals behind the funding programme, is compromising their ethics. A great deal of basic research will be beneficial for some and detrimental to others, especially if the information can be used by the military. ”Researchers need to take some responsibility,“ says Farah.