The Illusion of Evidence-Based Medicine: Exposing the crisis of credibility in clinical research Jon Jureidini & Leemon B. McHenry Wakefield (2020)
In the race to find treatments and a vaccine for COVID-19, it’s more essential than ever that society can trust drug companies seeking regulatory approval. The Illusion of Evidence-Based Medicine is the latest in a long line of books that caution us not to hold out much hope.
Child psychiatrist Jon Jureidini and philosopher Leemon McHenry dispute the assumption that all approved drugs and medical devices are safe and effective. They warn that when clinical science is hitched to the pharmaceutical industry’s dash for profits, the scientific method is undermined by marketing spin and cherry-picking of data. They propose a solution inspired by philosopher of science Karl Popper: take drug testing out of the hands of manufacturers.
The authors were afraid that academic publishers with ties to the pharmaceutical industry would demand unacceptable changes to their work, so they chose to publish with a small, independent press. To be fair, similar exposés have been produced by mainstream publishers; these include The Truth About the Drug Companies (2004) by Marcia Angell, former editor-in-chief of The New England Journal of Medicine, and Bad Pharma (2012) by the crusading clinical epidemiologist Ben Goldacre.
Little has changed since these works were published, say Jureidini and McHenry. Academics still lend their names to ghost-written papers paid for by drug companies. The companies still pressure journals to publish the papers; on the basis of these, regulators approve drugs. Because the industry controls every aspect of this process — and the all-important data — the pair refer to it as “organized crime”, following Peter Gøtzsche’s 2013 book Deadly Medicines and Organised Crime.
Jureidini and McHenry have witnessed these practices at close quarters, and spent more than ten years sifting through documents released by drug companies. In 2007, they were taken on as consultants by a California law firm that has represented plaintiffs in suits against the industry. The duo leave it to readers to decide whether this conflict of interest compromises their position. I am inclined to applaud their determination. “At stake,” they write, “is the integrity of one of the greatest achievements of modern science — evidence-based medicine.”
‘Evidence-based medicine’, some might be surprised to learn, was coined as recently as the early 1990s, to highlight the fact that doctors based much of their practice on an unscientific hotchpotch of research, experience, anecdote and custom. It has produced stunning successes, such as treating high blood pressure to reduce the risk of cardiovascular disease, and personalizing the treatment of liver cancer. Yet distortion of evidence threatens those gains, these authors warn, and risks further eroding the public’s already fragile trust in academic medicine, manifesting, for example, in the rising distrust of vaccines.
They discuss two trials for psychiatric drugs: GlaxoSmithKline’s Study 329, testing paroxetine; and Forest Laboratories’ Study CIT-MD-18, testing citalopram. Both aimed to gain US Food and Drug Administration (FDA) approval for the use of antidepressants in children and adolescents. Initial publications concluded that both drugs were safe and effective in that group. Paroxetine was not approved for this use; escitalopram, a variant of citalopram, was.
Analysing the clinical report for Study 329, Jureidini and others found in 2015 that paroxetine was not effective in adolescents with major depression, as the original 2001 publication had claimed. They also found it increased the risk of harms such as suicidal ideation (J. Le Noury et al. Br. Med. J. 351, h4320; 2015). A year later, Jureidini and McHenry deconstructed Study CIT-MD-18 (J. N. Jureidini et al. Int. J. Risk Safety Med. 28, 33–43; 2016). They revealed that violations of the trial protocol had been omitted from the original 2004 publication. Once these were accounted for, citalopram seemed no more effective than a placebo.
Both companies admitted that they had misrepresented safety and efficacy data, and paid heavy fines. Yet, Jureidini and McHenry point out, GlaxoSmithKline continued to claim that the findings of Study 329 had been accurately reported. And the FDA, they say, has taken no action to correct misreporting of Study CIT-MD-18 in Forest’s application to license escitalopram to treat adolescent depression.
Companies hand over raw trial data only if forced, usually in the course of litigation (which they budget for). Despite attempts to make the process more transparent, for example by mandating the preregistration of clinical trials, many of those data are not in the public domain. That’s why, the authors believe, these cases represent the tip of an iceberg.
The authors agree that the randomized, placebo-controlled trial is the best method we have for testing drugs, and they argue that every scientific theory should be tested by, in Popper’s phrase, attempting to falsify the null hypothesis. In a trial, this means trying to disprove the idea that the treatment makes no difference. Adhering to this principle, researchers can never say for sure that a treatment is effective, but they can say definitively that it is not effective.
However, the authors charge that drug companies have made even that impossible, by designing protocols that guarantee a positive outcome or by spinning a negative one. One concern is the redefinition of endpoints mid-trial — a worry that resurfaced in the context of the US National Institute of Allergy and Infectious Diseases’ ongoing trial of the potential COVID-19 drug remdesivir, made by Gilead Sciences of Foster City, California. Partial solutions, such as requiring companies to deposit trial results in public databases, haven’t worked. The commercial disincentives are just too strong.
Popper’s ideas have often been criticized. Theories are never truly falsified, critics say, just shown to be less wrong than others. But we’ve gone too far down the road to relativism, counter Jureidini and McHenry; Popper offers a standard of integrity to which we must return. The only way to ensure that, they conclude, is to have trials conducted in a public-health system or by an independent institution funded by a tax on the industry. This would work only with government support, which has been lacking. Yet models do exist. The Mario Negri Institute for Pharmacological Research in Milan, Italy, has been conducting independent clinical trials for nearly 60 years.
The current pandemic might provide the perfect opportunity to acknowledge that there is a problem: ill people need treatments and the well need a vaccine. Quoting ancient Greek historian Thucydides, the authors write: “There will be justice … when those who are not injured are as outraged as those who are.”
Nature 583, 26-28 (2020)