The drug Tamiflu is prescribed as the front-line treatment for serious cases of influenza. Credit: Pichi Chuang/Reuters/Corbis

A study that calls into question the stockpiling of billions of dollars’ worth of antiviral drugs to mitigate the threat of influenza pandemics has been criticized by flu researchers.

Images of the month: May 2014 Lianas protect trees from lightning Honey bees navigate with mental maps, not just the Sun

The analysis of Tamiflu and Relenza, drugs known as neuraminidase inhibitors, was published on 10 April by the Cochrane Collaboration1, a group that reviews the effectiveness of health-care measures. It concluded that the medicines were of little use. At the same time, the journal BMJ published a series of articles, including two that summarize the Cochrane findings2,3, and several editorials that focus on the five-year campaign by Cochrane and the BMJ to obtain the unpublished drug-company clinical-trial data later used in the review.

The results “challenge the historical assumption that neuraminidase inhibitors are effective in combating influenza”, declared a joint BMJ–Cochrane news release on the findings. The drugs have had their “effectiveness overplayed, and harms underplayed”, said Fiona Godlee, the BMJ’s editor-in-chief, at a press conference. The study generated worldwide media coverage, including headlines labelling Tamiflu as “useless” and “ineffective”.

But the review and its bottom line are vigorously contested by many flu researchers. They argue that the analysis — an update by Cochrane — is based on randomized clinical trials (RCTs) of the drugs that lack sufficient statistical power to allow reliable conclusions to be drawn about the effects on flu complications and hospitalizations. These are the key outcomes of interest during a flu pandemic.

The critics say that the review also excluded many observational studies that have found the drugs to be helpful in normal clinical settings.

Tamiflu is prescribed as the main treatment for serious cases of flu, and researchers worry that the media storm risks undermining public confidence in this class of drug. “We risk losing one of the few weapons we have, because of overly negative publicity,” says Peter Openshaw, director of the Centre for Respiratory Infection at Imperial College London.

The review by Cochrane, a non-profit organization in London, considered both Tamiflu (oseltamivir), produced by the drug company Roche in Basel, Switzerland, and Relenza (zanami­vir), marketed by GlaxoSmithKline in Brentford, UK. It is one of the first analyses to review clinical-study reports — documents that companies submit to regulators and that contain fuller trial data than published RCTs.

Both drugs have been found to shorten the duration of flu-like symptoms, and to be effective in treating severe disease; what is in dispute is whether they reduce hospitalizations and severe complications. The Cochrane authors and the BMJ claim that there was no evidence for such gains. They also say that the findings bring into question the decision by governments in the mid-2000s to stockpile the drugs against the threat of a pandemic of the H5N1 avian-flu virus, which has a mortality rate of around 60%. Carl Heneghan of the University of Oxford, UK, a co-author of the review, said that there was “no credible way these drugs could prevent a pandemic” and that stockpiles were “money thrown down the drain”.

But a dozen experts contacted by Nature’s News team said that the clinical-study reports, although they allow more detailed analyses, offer no substantial fresh findings.

“The Cochrane authors have done a thorough review, and deserve credit for their efforts to obtain raw clinical data from Roche and regulatory authorities, but ultimately its findings are not surprising,” says Peter Horby, a flu researcher at the University of Oxford Clinical Research Unit in Hanoi, Vietnam.

Jody Lanard and Peter Sandman, independent risk-communication experts in New York, say that the press release on the review omits findings that in their opinion are key. For example, it rounds down Tamiflu’s reported 17-hour reduction in symptom duration in adults to “just half a day”, and describes the reduction as “small”. It also does not report the 29-hour reduction in children. Lanard and Sandman claim that there has been “cherry-picking of the results to make them look worse for antivirals”.

Horby’s view is that the BMJ–Cochrane press release and public statements have contributed to media misinterpretation of what the study does and does not say. “The review does not state the drugs are ‘ineffective’ or ‘useless’,” as was reported in some media stories, he says.

In a joint response to Nature, BMJ–Cochrane said that they felt the press release did “a good job of presenting the main findings”. They defend its presentation of the reduction in duration of symptoms as justified, and note that the fuller outcomes were included in the review’s summary of its findings. The press coverage, they say, was “not bad in terms of balance and accuracy”.

The two organizations agree that the limited statistical power of the trial data on the numbers of hospitalizations and complications makes it difficult to draw reliable conclusions on these. But they argue that their findings of no evidence for any effects challenge past analyses of trials that found that the drugs reduced complications and hospitalizations.They claim that the review did show new findings, for example data on side effects.

RCTs are considered the gold standard for establishing the effectiveness of drugs, and Cochrane restricts itself to meta-analyses of these in its review. But critics note that these small clinical trials were carried out to gain regulatory approval for Tamiflu as treatment and prophylaxis for seasonal flu, which in most cases is mild. This meant that the healthy trial subjects rarely developed complications. In other words, the trials were not designed to test for the severe outcomes that are most relevant to pandemics.

The critics add that observational studies of how large numbers of people respond to treatments under normal medical care can also provide important information on the effectiveness of drugs. In March, for example, an observational study4 of 30,000 people hospitalized during the 2009–10 swine-flu pandemic reported that neuraminidase inhibitors reduced mortality by 25%.

But because of its policy of reviewing only RCTs, Cochrane did not include observational data. “Given the limited number of RCTs, and the considerable evidence base on the effectiveness of neuraminidase inhibitors over the past ten years or more, it is difficult to justify exclusion of the observational evidence,” says Ben Cowling, a flu epidemiologist at the University of Hong Kong.

BMJ–Cochrane said they exclude all observational studies because they are “unreliable for establishing treatment effects”.