To the Editor:

The Editorial in your September issue, entitled “Off the grid,” highlighted the risks of direct-to-consumer stem cell clinics and attempts by the US Food and Drug Administration (FDA; Rockville, MD) to enhance oversight of the sector1. There is a dearth of clinical data collected on the thousands of patients undergoing treatments marketed as 'stem cells' around the world in unlicensed clinics. As researchers and advocate members of the amyotrophic lateral sclerosis (ALS) community, we believe in patients' right to self-determination, but we have also despaired at how ALS patients have sometimes been exploited by individuals practicing bad medicine.

Some clinics that purport to have treated hundreds of ALS patients have not even taken the basic step of quantifying changes in progression by administering a 12-question clinical outcome measure like the ALS functional rating scale (Revised; ALSFRS-R)2, which could be completed in minutes. Several researchers3,4,5 have now shown that comparing an experimental group against historical norms can establish whether dramatic post-treatment changes are real or just part of the standard noise of ALS progression, which can include temporary plateaus and transient reversals6.

Nearly 10 years after we published the first legitimate and ethically approved report of a stem cell trial in ALS7, we were able to revisit the ALSFRS-R data from one of the patients, J.H.'s brother Stephen Heywood, with a matched control set of data from ALS patient volunteers on PatientsLikeMe (https://www.patientslikeme.com/). The matching algorithm showed that Stephen progressed as expected when compared with a tightly matched control group of ten other patients like him. Although the error bars on our predictive algorithms are large for an n of 1, this finding provides at least some quantitative evidence to support what we believed at the time8—that the treatment lacked efficacy (Fig. 1).

Figure 1: Carefully matched historical control data using the ALSFRS-R on PatientsLikeMe demonstrates that a stem cell transplant failed to slow the progress of Stephen Heywood's illness (solid blue line).
figure 1

Control patients (thin red lines) were selected to match Stephen's progression prior to the transplant, which is represented by the vertical green line. Yellow line shows average progression of matched controls.

Readers should note that this same technique has been used successfully to rapidly disprove claims of efficacy in ALS for a small-molecule treatment. The study by Wicks and colleagues disproved the notion that lithium carbonate halted the progression of ALS in a group of patients who opted to seek self-treatment off label3.

Part of the motivation for PatientsLikeMe and for observing clinical benefit and risk in the real world came from our experience dealing with borderline stem cell clinics. More than a decade ago, we were advising families affected by ALS not to fly to China for treatments by Huang Hongyun with olfactory ensheathing cells apparently harvested from aborted fetuses9. Alongside miraculous claims of neurological recovery shared between patients were stories of poor surgical hygiene and postoperative complications, including meningitis. When, in 2004, Hongyun was invited by the ALS Therapy Development Institute to present at the Massachusetts General Hospital ALS clinic, his data were “shockingly thin—indeed, insultingly so,” according to one reporter9. Researchers who subsequently investigated the claims in more detail found worrying trends for severe complications and a lack of objective improvement10,11,12.

Although dispiriting, it is instructive to note that a handful of skeptical journal articles is toothless without regulatory action. Today, Huang is Editor-in-Chief of Dove Medical Press's Journal of Neurorestoratology. The journal features several uncritical 'reviews' on ALS, Parkinson's disease, multiple sclerosis and Duchenne muscular dystrophy, which each present a summary of case series and poorly controlled experiments scattered around the literature. These 'publications' will undoubtedly be provided to future customers as evidence that they should sacrifice their life savings for a shot at therapy. Against scientific convention, we have chosen not to cite these reviews here.

Although we are staunch advocates of the open access publishing movement and supporters of pioneering new therapies and early access to them, there is a risk of a 'shadow literature' emerging alongside what we would consider rigorously conducted and peer-reviewed science. It is not simply that actors in this space don't know how to conduct science rigorously. Huang was quoted by the British newspaper The Daily Telegraph in 2004 as saying, “Really, I don't know how people benefit from our procedures, but I am not concerned with this. It is nonsense to say that I should hold up my work to conduct trials. This is surgery, not a drug. It does not need testing, it has a benefit. About half my patients have not suffered a reverse. That is all the results I need”13.

When it comes to stem cell clinics, we have found a spectrum of groups working in the space, ranging from traditional academic researchers to nontraditional (but well intentioned and reasonable) clinics trying to help patients without profit and finally to the deluded. Frauds exist, but they are rarer, are only in it for the money, know their treatments are bogus and can be uncovered through investigative journalism, such as the 2010 investigation of surgeons Francisco Morales and Lawrence Stowe by US television network CBS's documentary '60 Minutes'. After charging numerous ALS patients upwards of $100,000 for 'stem cell treatments' and being indicted on multiple counts of mail fraud and illegal manufacture and distribution of unapproved new drugs, Morales was sentenced to 5 years in federal prison, while Stowe received 6.5 years14.

But such miscreants as Morales and Stowe are the tip of a larger iceberg; providers who are not malicious, but simply ill informed and well meaning, can in aggregate be worse because they do not trigger the same degree of skepticism among patients. They genuinely want to provide treatment for the same reasons many of us do: to help those in need, to feel good about being seen as a provider and to earn the prestige of making an important discovery. Both groups may even wear the fact that they do their work outside the traditional system as a badge of honor and a form of social proof that what they are doing must be 'cutting edge'. This is not to dismiss the legitimate, thoughtful and nontraditional physicians who work outside the academic research infrastructure to provide options to desperate patients. Across the board, if real-world experimentation could be supported with good methods and a supportive ethical regulatory framework, it is easy to imagine that we could learn much together.

Furthermore, we should not lightly dismiss the efforts of patients who work to learn and gain utility from treatments that are unlikely to undergo formal rigorous study. To this end, we have partnered with ALSUntangled (http://www.alsuntangled.com/), a consortium of over 100 researchers, clinicians and patients lead by Rick Bedlack from the Duke ALS Center (Duke University, Durham, South Carolina), to investigate claims for complementary and alternative treatments for ALS. To date, the consortium has systematically investigated and graded the evidence for over 35 treatments, ranging from several specific stem cell clinics to vitamin supplementation, cannabis and even fecal transplants15.

By listening to patients' interests and concerns and systematically reviewing the preclinical, scientific and clinical claims made in marketing materials and medical notes and reported by patients online, the consortium hopes to help patients make better decisions. To date, no treatment has received a positive endorsement by ALSUntangled, but that could change in the future. The reports are published open access in the main ALS field journal, Amyotrophic Lateral Sclerosis and Frontotemporal Degeneration, and several of them have been downloaded tens of thousands of times. We have also recently undertaken work using clinical databases to establish just how frequently ALS may appear to spontaneously plateau or transiently reverse itself, as a bulwark against claims that any minor improvement following a 'stem cell' treatment can be taken as evidence of success6. Finally, we are piloting a unique hybrid virtual clinical trial for a nutritional supplement to establish that high-quality clinical data can be collected much faster, more cheaply and more conveniently than in traditional trials, demonstrating that it is possible to systematically collect data rapidly in the case of experimental medicines.

Although it has taken some 15 years since the first human ALS stem cell publication7, we are now seeing the emergence of a body of scientifically rigorous transplantation of stem cells in ALS being reported in high-quality journals3. Sadly, the treatment is clearly still not the hoped-for silver bullet. In the latest phase 2 clinical trial data to be reported, several patients suffered adverse events and no clinical benefit was apparent.

When a loved one suffers from a serious illness with no effective treatment, it is understandable that patients and their families seek alternative routes. A skeptical observer might say that, when it comes to marketing and providing treatments aggressively to patients on the basis of weak evidence, failing to collect follow-up data, failing to share data openly, or using treatments off-label with little theoretical basis, some of the dubious actors we decry differ from the rest of our mainstream medical system only in degree. The evidence we collect about whether treatments really work is done in an artificial context of randomized clinical trials against placebo, not head-to-head in the real world, with extremely patchy reporting16. As many as one in five US prescriptions may be written for off-label purposes with no follow-up data reported on efficacy or safety17. Readers in the United States will be familiar with direct-to-consumer advertising that stretches credulity in comparison to relatively small effect sizes in the literature. It's not just patients experimenting with exotic treatments that need to be on 'the grid' of a scientifically rigorous data-collection mechanism underlying a continually learning health system. We all do.