Download a PDF of this story.

Peggy Willocks was 44 when she was diagnosed with Parkinson's disease. It progressed quickly, forcing her to retire four years later from her job as a primary-school principal in Elizabethton, Tennessee. Soon, her condition had deteriorated so much that she was often unable to dress and feed herself, take care of basic hygiene or walk unaided across a room.

Willocks enrolled in a trial for an experimental therapy called Spheramine, developed by Titan Pharmaceuticals, a biotechnology company in South San Francisco, California. Spheramine consists of cultured human retinal epithelial cells bound to specialized man-made carrier molecules. The cells are implanted into the brain, where it is hoped that they will produce the dopamine precursor levodopa, which can reduce the symptoms of Parkinson's disease. In August 2000, Willocks became the second person ever to receive the treatment. After having a steel halo — a stereotactic frame — bolted to her skull, she was put under general anaesthesia. Surgeons then used the frame and coordinates obtained from numerous magnetic resonance imaging (MRI) scans to pinpoint the location at which to drill. They then snaked a catheter through her brain's white matter to deliver the cells into the striatum.

At first there was no effect, but Willocks says that after 6–8 months she began to feel better. The changes were always moderate and gradual, except for once, about nine months after her surgery, when she showed what her doctor called a "radical" improvement in balance. By a year after the treatment, she and the five other patients in the phase I trial showed an improvement in motor ability of 48%, and those gains largely held 4 years later1.

Ten years on, she says she notices her condition worsening, but is still doing much better than she was before her operation. She has no doubt that the treatment works. Investigators disagree: Spheramine was shelved in 2008 after a follow-up phase II, double-blind study found that it was no more effective than placebo2. This time, the researchers compared the treatment with a 'sham' brain surgery that copied almost every aspect of the procedure Willocks received, short of injecting cells into the brain.

For many investigators aiming to treat Parkinson's and other neurological diseases invasively, using sham brain surgery as a control is, well, a no-brainer. And the practice is likely to expand in coming years, as researchers continue to develop experimental tissue transplants, gene therapies and stem-cell treatments. Small safety trials such as the one in which Willocks was enrolled may hint at the efficacy of a treatment, but they are not designed to prove it. And because they are 'open label' — both the investigators and the participants know that the drug is being administered — they are riddled with biases that can skew results. "It is so clear that open-label studies provide information that is not reliable," says Warren Olanow, a neurologist at New York's Mount Sinai Medical Center who has worked on cell-based neurosurgical therapies in Parkinson's for more than two decades. "It's almost impossible for me to imagine how a serious scientist can not desire their data or hypothesis to be tested in double-blind studies."

Other scientists, however, say that sham brain surgery is an expensive, potentially dangerous and possibly unethical bit of biomedical theatrics. It may also be unnecessary. Clinical neuroscientist Roger Barker at the University of Cambridge, UK, contends that because there is huge variation in how these therapies are administered and in how patients respond, the protocols need to be refined in an open-label setting before going on to the next stage of development. And because cost, complexity and the small number of people eligible for such invasive therapies limit the size of the studies, a sham control provides results of limited statistical utility. Barker and his colleagues across Europe are currently enrolling patients in a €12-million (US$17-million) multicentre trial of a fetal dopaminergic nerve-cell treatment for Parkinson's disease. The treatment may never be tested against a sham-surgery control. "There's a sort of historical precedent" for using placebo controls, but it may not apply to neurosurgical trials, he says. Willocks and other patients go further. Placebo-controlled studies aren't just unnecessary, they say, they are actually causing the downfall of potentially valuable treatments.

A complicating control


Click for larger version.

During the past 25 years, surgical therapies for Parkinson's disease have travelled a rocky road. In 1987, a report by Mexican surgeons3 described seemingly miraculous effects in two patients with severe Parkinson's who received transplants of tissue from the adrenal gland, which produces dopamine. In the next several years, hundreds of patients received the treatment, but some autopsies later showed that the cells didn't in fact survive4. Around the same time, researchers started to test fetal nerve-cell transplants (similar to those in Barker's trial) in small-scale studies, finding mixed but promising results. Two studies5,6 comparing the treatment to sham surgery concluded, however, that the transplants were not only ineffective, but also often caused dyskinesia — the movement disorder that plagues people with Parkinson's disease. In the past seven years, three experimental treatments (including Spheramine) that showed promise in small, open-label studies1,7,8 failed in phase II trials2,4,9 comparing them with a sham control (see 'The sham wall').

Sham brain surgery is no sugar pill. After the stereotactic frame is affixed to the skull, the patient is usually anaesthetized and surgeons drill into the skull. In most cases, the burr holes stop at the dura mater, a protective membrane covering the brain, but they sometimes go deeper: in a phase II trial testing the nerve growth factor GDNF, investigators catheterized the brain in all participants but infused saline, rather than GDNF, into the controls9.

"We have to stage the whole thing such that from the outside it's completely indistinguishable" from the real thing, says Joao Siffert, chief medical officer of Ceregene, a company in San Diego, California, that is working on a therapy that delivers a gene for another nerve growth factor, called neurturin, using a viral vector. For many sham treatments, everyone in the operating room, from surgeons to nurses' assistants, must pretend that they are busily performing the complete operation — in some cases, turning on machines to elicit appropriate noises. An extremely complex protocol ensures that no one outside the surgical team knows who got what treatment. "It's very complicated, there are a lot of moving parts," Siffert says. All that ratchets up the cost of a trial; Siffert estimates that between operating-theatre costs, follow-up and the unwieldy infrastructure required for data management, a 50-patient study would cost more than $10 million.

Still, at least in North America, Parkinson's disease investigators overwhelmingly support the use of sham surgery — at a rate of 94%, according to a 2004 survey10. Around 20% said that penetrating the brain is justifiable. And proponents say the procedure is relatively safe. Although sham brain surgery has definite risks, most notably those associated with general anaesthesia, supporters note that adverse events are almost unheard of, unlike the risks of the actual treatments. And participants in the sham groups are generally promised the treatment if it is ultimately approved; in that event, they will already have the burr holes in their skulls through which it would be administered.

Sham treatments help to tease out the placebo effect and biases. In Parkinson's disease, the placebo effect is especially strong. One reason is that patients' expectations that they will benefit from a treatment induce the release of dopamine11, the neurotransmitter that is lacking in the disease. "The placebo effect is real, it's huge and it's got a physiological basis," says Jon Stoessl, a neurologist at the University of British Columbia in Vancouver, Canada, who studies Parkinson's and the placebo effect. In one double-blind study of fetal nerve-cell transplants, patient improvement correlated with whether they believed they had received the treatment, irrespective of whether they actually had12. And the effect can last as long as two years, Stoessl says, citing an unpublished study by his colleagues.

There's a historical precedent for using placebo controls, but it may not apply to neurosurgical trials. ,

Many regard bias as a more significant confounder. "Investigators have a tremendous vested interest in seeing that their treatment is effective," says Anthony Lang, a neurologist at the University of Toronto in Canada who has participated in several neurosurgical trials for experimental Parkinson's therapies. In any trial, bias can affect how researchers assess patient responses and may inflate the patients' expectations, further enhancing the placebo effect. Compounding the problem for Parkinsons' research is the fact that there are no objective measures for how well a patient is doing. "It's just a sort of perfect storm conspiring against our ability to see definitive changes in the underlying disease," says Steven Piantadosi, a clinical-trials methodologist at Cedars-Sinai Medical Center in Los Angeles, California. "Sham surgery, properly done, can control for that."

Barker counters that it is possible to control for investigator bias in an open-label trial by taking steps such as having blinded raters assess patients. His position is in some ways unsurprising; in Europe, sham surgery is deemed much less acceptable than it is in the United States. It has never been used in the United Kingdom. Barker is categorical about his belief that transplantation of fetal tissue works, at least for some people. "I don't need sham surgery to show that," he says, pointing instead to a paper13 published last year describing two patients treated 13 and 16 years previously who were still benefiting from the treatment, and whose brains showed functional dopamine-producing neurons at the transplant site. He attributes the mixed results in past studies to variation in the patients selected for treatment, the characteristics of the tissue being implanted and the methods used to implant it. His trial will have to demonstrate efficacy without eliciting some of the side effects found in the two sham-controlled studies. That will require some type of control study, he says, but it might take the form of comparison to an approved therapy that is known to work, such as deep brain stimulation.

But time, says Barker, will best establish efficacy. In most trials the end point is no more than a year after the treatment. That may not be long enough: implanted cells or injected growth factors might take longer than this to become fully functional, and the placebo effect may not have had time to dissipate. "We want a 3–5-year endpoint," says Barker.

There are hints from some of the failed phase II trials that patients followed up beyond study endpoints might tell a more positive story4. Some say, therefore, that sham controls are sinking the prospects of valuable drugs. Anders Björklund, a neuroscientist at Lund University in Sweden who is collaborating with Barker, says that sham surgery can lead researchers to throw out a strategy prematurely if the trial fails because of technical or methodological glitches rather than a true lack of efficacy.

Advocacy and frustration

According to Perry Cohen, who leads a network of patient activists called the Parkinson Pipeline Project, that's exactly what is happening. He had always questioned the need for sham surgery, he says, but after the string of phase II failures, "We started saying, 'Hey, this is a problem. These trials failed, but we know they are working for some people.'"

For researchers, it is easy to dismiss patients' concerns as being driven by emotion. "Patients want cures," says Lang, "and they will often be convinced that the more aggressive, surgical therapies are more likely to be curative." But Cohen counters that patients have different priorities and that researchers must take these into account. Researchers use placebo controls to weed out false positives. But for patients, the real ogre is the false negatives — which can sink a therapy before it has been optimized. The better a trial is at stamping out the former, the higher the rate of the latter — which means at best delays, and at worst dead ends. Spheramine, for example, "is still on a shelf somewhere", Cohen says. Then there's Amgen's phase II trial of GDNF. The trial was halted in 2004 amid lacklustre results and potential safety concerns, which some have attributed to Amgen's procedure, rather than to the therapy itself. Now researchers are taking a renewed interest in the molecule, but although Cohen is glad it is getting a second chance, "we lost 6 years on it", he says.

Patients also have different perspectives on risk from researchers, Cohen says. He offers the story of Tom Intili, who had had Parkinson's for 10 years when, at the age of 50, he signed on to the double-blind, placebo-controlled trial of neurturin. At first, Intili improved dramatically. But when the results were unblinded, he learned that he had received the sham. His condition plummeted, leaving him more debilitated than he had been before the trial. "We just don't know what the psychological effects of unblinding are," Cohen says.

Moreover, trying to exclude the placebo effect is simply misguided, Cohen argues. "I don't want to subtract out the placebo effect — I want to keep it, because in real life it's part of the treatment," he insists. Because psychological factors are so salient in Parkinson's, a placebo response might actually potentiate a therapy, he explains. "I want to be convinced that sham surgery is necessary. I'm looking for arguments that might change my mind, but I haven't found any yet," he says.

Willocks says that she is living proof that many of the recently shelved therapies are in fact salvageable. Of course from a scientific perspective, her story is an anecdote, not data. In May, the failed phase II study of Spheramine — the therapy she received a decade ago — was finally published2. The paper closes with a warning about the dangers of the placebo effect and stresses the importance of controlling for it with a double-blind design. "That last paragraph bothered me," Willocks says. "I just don't see how they can call it a placebo effect after ten years."

figure a