At 9:15 in the morning two or three times a week, Jan Scheuermann maneuvers her electric wheelchair into a research laboratory at the University of Pittsburgh, where she plugs her head into a highly sophisticated piece of equipment. Two ports in her scalp connect to a prosthetic limb, a sleek, black anthropomorphic arm that extends from a metal scaffold in the lab. She is one of a dozen or so volunteers worldwide who have received brain implants as part of multiyear experiments on how to manipulate objects with their thoughts alone. More than any other user of brain-controlled prostheses, Scheuermann has learned to wield the arm with exquisite dexterity, articulating individual fingers to shake hands and rearrange objects at a wide range of speeds. “Every day I go to work, I think, this is the coolest thing,” she says.
Scheuermann began losing control of her muscles in 1996. As her genetic disorder—spinocerebellar degeneration—took its toll, she gave up her successful business as a planner of murder-mystery-themed events. By 2002 her disease had confined her to a wheelchair, which she now operates by flexing her chin up and down. She retains control of the muscles only in her head and neck. “The signals are not getting from my brain to my nerves,” she explains. “My brain is saying, ‘Lift up!’ to my arm, and my arm is saying, ‘I caaaan't heeeear you.’”
Yet technology now exists to extract those brain commands and shuttle them directly to a robotic arm, bypassing the spinal cord and limbs. Inside Scheuermann's brain are two grids of electrodes roughly the size of a pinhead that were surgically implanted in her motor cortex, a band of tissue on the surface of the brain that controls movement. The electrodes detect the rate at which about 150 of her neurons fire. Thick cables plugged into her scalp relay their electrical activity to a lab computer.
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
As she thinks about moving the arm, she produces patterns of electrical oscillations that software on the computer can interpret and translate into digital commands to position the robotic limb. Maneuvering the arm and hand, she can clasp a bar of chocolate or a piece of string cheese before bringing the food to her mouth. When she succeeds in performing a task with a new level of fluency, the researchers in the room break into applause. “Any time I did something faster we'd all say, “Ah, a new world record!”” she says.
Scheuermann calls herself a “guinea pig extraordinaire.” Her story—and that of other paraplegics fitted with brain-activated prostheses—often gets featured on television news shows or in the science pages of popular magazines. Once perfected, these fledgling technologies hold obvious appeal for letting the wheelchair-bound reach for an object or even get up and walk.
Less attention has gone to another critical contribution made by Scheuermann and others fitted with paraphernalia that jack into the brain—and by the primates and rodents that also participate in these experiments. This select group has given neuroscientists an unprecedented view of how the brain proceeds from thought to action and how it develops a new skill. Numerous experiments with this group are now documenting how brain circuits rewire as a neophyte evolves from bumbling ineptitude to fluid proficiency. Implants that eavesdrop on dozens of neurons provide scientists a window through which to watch how practice breeds mastery at the level of an individual brain cell—not only in a paraplegic but in anyone honing a new ability.
Forming Machine Memories
When neuroscientists first set out to develop brain-controlled prostheses, they assumed they would simply record neural activity passively, as if taping a speech at a conference. The transcript produced by the monitored neurons would then be translated readily into digital commands to manipulate a prosthetic arm or leg. “Early on there was this thought that you could really decode the mind,” says neuroscientist Karunesh Ganguly of the University of California, San Francisco.
Yet the brain is not static. This extraordinarily complex organ evolved to let its owner react swiftly to changing conditions related to food, mates and predators. The electrical activity whirring inside an animal's head morphs constantly to integrate new information as the external milieu shifts.
Ganguly's postdoctoral adviser, neuroscientist Jose M. Carmena of the University of California, Berkeley, wondered whether the brain might adapt to a prosthetic device as well. That an implant could induce immediate changes in brain activity—what scientists call neuroplasticity—was apparent even in 1969, when Eberhard Fetz, a young neuroscientist at the University of Washington, reported on an electrode placed in a monkey's brain to record a single neuron. Fetz decided to reward the animal with a banana-flavored pellet every time that neuron revved up. To his surprise, the creature quickly learned how to earn itself more bites of fake banana. This revelation—that a monkey could be trained to control the firing rate of an arbitrary neuron in its brain—is what Stanford University neuroscientist Krishna Shenoy calls the “Nobel Prize moment” in the field of brain-computer interfaces.
Of course, neurons adjust their behavior any time a person learns, whether it is a student becoming fluent in French or a skater finally landing a triple Axel. Yet by training an animal to control a particular cell or set of cells, scientists can observe the process unfolding in those exact neurons. Specifically, researchers monitor the bespoke firing pattern, or tuning, of each neuron under surveillance in an implantee's motor cortex.
A neuron's tuning contributes a bread crumb of information on how to execute a movement. One neuron might fire lethargically to command a movement upward, for example. That same cell firing at its maximum rate might signal a leftward tilt. This pattern of activity is called a neuron's preferred direction. Engineers developed software that aggregates all the preferred directions of the recorded neurons firing at a given moment to produce an individual's intended bearing. Later, when someone imagines a movement, the software knows which way to move the robotic limb.
Scientists were beginning to discover, however, that neurons can adjust their tuning in response to the software. In a 2009 study Carmena and Ganguly detailed two key ways that neurons begin to learn. Two monkeys spent several days practicing with a robotic arm. As their dexterity improved, their neurons changed their preferred direction (to point down rather than to the right, for example) and broadened the range of firing rates they were capable of emitting. These tuning adjustments gave the neurons the ability to issue more precise commands when they dispatched their missives.
The neuroscientists then pushed their experiments a step further, testing just how far neuroplasticity could be extended. They scrambled the computer software's control scheme so that the arm now reacted differently to the same inputs of neural data—cells that previously swung the arm to the left now might send it soaring upward. The monkeys had no trouble learning the new rules, and their neurons even reacted by adopting a new firing scheme. In fact, they could switch easily between these firing schemes to control the arm in either mode. “The brain can form something that looks like a natural motor memory for a disembodied device,” Carmena says. “To me, that is pretty remarkable.”
Back in the Pittsburgh lab, Scheuermann, too, has helped shed light on neuroplasticity. She discovered that she performed better when she was relaxed or slightly distracted, so she would tell the experimenters jokes or regale them with stories of her family while she directed the prosthetic arm to stack cups or move blocks. Biomedical engineer Jennifer Collinger of Pittsburgh and her colleagues, in their first major paper on Scheuermann's work with the robotic arm, published in 2013, documented how the neurons in her motor cortex coordinated themselves to better reach the target. “Because of the feedback she was getting about her errors, her neurons appeared to be changing their tuning,” Collinger says.
Learning from Mistakes
As Scheuermann observed the arm and the ways in which it missed its mark, she made mental adjustments. Somehow her brain could identify the specific neurons that contributed to her errors. Correcting mistakes in perception or action—the neural equivalent of software bug fixes—is one of the reasons the brain needs to be so readily changeable. Learning itself is nothing more than repeated error correction. Here, too, the brain-computer interface can help.
In work published in 2012 neuroscientist Steven M. Chase of Carnegie Mellon University and his colleagues implanted electrodes next to neurons in the motor cortex of two monkeys. They trained them to use the recorded neurons to control a cursor on a computer screen. Software identified the preferred direction of each recorded cell, as is typical. This time, however, the neuroscientists forced the monkeys to make small mistakes.
They split each monkey's controlling neurons into two groups. The scientists then tweaked the software so that when one group of neurons fired, the directions they moved the cursor were rotated (say, 30 or 60 degrees). The remaining neurons were left alone. Now when the monkey tried to reach a target on the screen, the cursor came up short. Think of pushing a shopping cart with a bum wheel that steers to the right when you want to go straight ahead. To compensate, you push the cart to the left to cancel the rightward bias.
The monkeys' neurons performed a similar realignment when they fired. The animals oversteered by recruiting neurons that pointed past the target. In a crude overcorrection, they also dampened all the neurons that earlier had fired to point directly at the target, including the cells that had not been digitally rotated. In addition, a small number of neurons changed their tuning to point in a different direction, a more permanent shift akin to remodeling the wheel.
In follow-up work, Chase and his colleagues have found that when the monkeys spend weeks practicing with rotated neurons, their brains more aggressively pursue the latter adaptation. Chase suggests that this might occur because permanent retuning simply takes longer. “It requires the network to restructure,” he says, whereas a technique such as oversteering uses neurons' existing capabilities and can happen immediately.
Chase's work provides insight into the changes in tuning that solidify motor learning. The new firing patterns become locked into place through alterations in the connection points between neurons, called synapses. Over the course of several days, new synapses grow or old ones weaken, a process that brings about microscopic but lasting shifts to the brain's networks. These subtle tweaks represent the physical basis of a new skill at the level of individual cells and molecules.
Opening a Window
Learning takes place in stages, during which a new task becomes gradually integrated with existing memories. Sleep, in fact, may be crucial in the formation of memories. It has long been noted that people perform better and faster on a wide range of tasks after slumber—whether they are solving differential equations or playing Bach études. Yet what happens during sleep has remained largely guesswork.
Brain-computer interfaces provide a means to follow what happens as sleep helps to solidify recall. In one experiment published in July, Ganguly and his colleagues trained rats to use a brain implant in their motor cortex to move a mechanical tube that dispensed drops of water.
Many of the animals first discovered that they could control the tube after they twitched and saw it move. Soon the twitches died down as most of the rodents figured out that they could activate their motor neurons and trigger a drink without moving a hair. To do so, the animals had to perform an impressive feat of neuronal detective work. The scientists had configured the neuron-reading software to make the tube harder to move whenever one subset of cells fired. To earn a drop of water, the rodents had to sort out which neurons helped them to get a sip and which ones worked against them.
After the rats fell asleep at the end of a day of practice, their implants continued recording from neurons. When the rodents that had aced the task reached deep sleep—characterized by extremely slow, synchronized waves of electrical activity—the neurons that controlled the mechanical tube fired in lockstep. If the rats had performed poorly, these same neurons were slightly out of sync.
The neural processing taking place during the animals' repose appeared to be reinforcing the firing abilities of the mission-critical neurons. Moreover, the longer the rodents spent in deep sleep, the more their performance subsequently improved.
Another study, published in June, also explored how the brain winnows down the neurons it needs to the most critical players during learning. Carmena and his colleagues developed a new technique that allows them to visually survey neurons rather than recording them, as with implanted electrodes. The researchers worked with genetically altered mice whose neurons glowed green when they fired. The scientists installed a glass plate into the rodents' skull through which they could observe neural activity using a microscope. “You're watching live as these cells flash on and off,” explains neuroscientist Daniel Feldman, one of the co-authors of the paper.
The researchers could assign any neurons in their field of view to controlling some aspect of the outside environment—in this case, the pitch of an auditory tone. The experimenters played a tone, and the animals could learn to activate their neurons to make it rise or fall. In one part of this study, the scientists picked two small groups of neurons. When the first group fired, the pitch rose. When the second set came on, it dropped. The rats were rewarded with a drop of water whenever they managed to hit a high-pitch target. “It couldn't be more abstract for the rat. It has no idea what you want it to do,” Carmena says. “Getting a rat to increase the firing rate in one ensemble and decrease the other—that is an extremely abstract thing to learn.”
As each rat practiced modulating the pitch, the neuroscientists watched through the glass window in the animal's head. Early in training, neighboring neurons glowed alongside the ones that controlled pitch. Yet within an hour the firing pattern became more precise, and these adjacent cells had gone dim.
The rodents seemed to be picking the individual cells that really mattered to form a concise and well-organized memory of their new skill. Out of the millions of neurons in their brains, the rats could identify the couple of cells that stood between them and their coveted drink.
Your Pliant Brain
Carmena and Feldman continued their experiments by adding yet another twist. They and their colleagues not only granted rats control over neurons in the motor cortex, as most studies on brain-computer interfaces do. They also grafted windows into a chunk of the somatosensory cortex, an area nestled behind the motor cortex that typically handles sensory information. The rats aced the same pitch-control test with cells in this region, too.
“Some voluntary process is reaching up into the sensory part of the brain and making it spike,” Feldman says. Impulses specific to movement, it seems, may not be all that vital to prosthetic control. Carmena's team is now investigating neurons in other parts of the brain and finding that rodents can learn to control them with ease.
These discoveries imply that the brain is perhaps more pliable than anyone may have realized. They raise the question of whether any neuron, with the right kind of feedback, might be trained to do an animal's bidding. “For brain-machine interfaces, you just want cells you can volitionally control,” Fetz says. “And I'm thinking they're all over the place, not just in motor areas. You could open up a lot of territory for gaining control.”
That would be good news for stroke patients or others who have lost mobility because of a damaged motor cortex. Brain areas that were spared when blood flow was interrupted might be able to pick up the slack.
Fully implantable devices that restore movement to paralyzed patients probably are still a couple of decades away. Scheuermann, for one, is well aware that she is unlikely to ever get her own personal robotic limb. Her interest in the project stems more from the altruistic streak that tends to emerge as a natural consequence of illness or disability. “It's given me such a sense of purpose,” she says. Although her goal is to eventually help other people with disabilities, she has also come to appreciate the contribution her brain can make in revealing the inner workings of the mind. She even nicknamed the ports in her scalp Lewis and Clark, for their vital role in exploring the brain's cryptic landscape.