In yet another blow to the pharmaceutical industry and to the US Food and Drug Administration (FDA), a damning meta-analysis published on 21 May showed that Avandia, GlaxoSmithKline's blockbuster drug for diabetes, increases the risk of heart attack by 43%. In the aftermath, the company claimed that the meta-analysis was based on incomplete information and could not reliably predict risk. The University of Washington's Bruce Psaty, who reviewed the meta-analysis and wrote an accompanying editorial in The New England Journal of Medicine , explains what the analysis really means and how the interests of the public could be better served.

What is a meta-analysis?

A meta-analysis is a technique for combining information from many studies. In this case Steve Nissen, who published the analysis on Avandia, pooled information from 42 randomized trials (N. Engl. J. Med. 356, 2457–2471; 2007). The method involves obtaining a within-study estimate of an effect and then averaging them across all of the studies—and that average is weighted. Larger studies get more weight, smaller studies get less.

The primary aims of meta-analysis are to obtain a summary estimate that is more precise and also to evaluate heterogeneity—whether an effect might be different in men and women, old and young, those with and without heart disease and so forth. But in this case, the numbers of events were too small and the number of trials were too large to do that.

Are meta-analyses the ideal tool to judge the risks associated with a drug?

The FDA lacks the authority that they need to insist on the proper trial design.

They are an excellent way of summarizing the cumulative evidence available from randomized trials. But there are potential problems. The study units themselves are randomized trials, which are generally good, but technically, a meta-analysis is an observational study, in which the investigator has set up, for instance, the eligibility criteria. So all the design issues of observational studies are present. Were all the trials identified and included? What about unpublished trials? Were the events in the trials correctly classified?

Nissen's meta-analysis did include the unpublished trials. The other point that I would make is that his study represents the cumulative evidence about heart attack risk from all the eligible trials that had been done to date on this drug.

What would be the ideal way to get this information?

The ideal would be a large, well-designed long-term trial evaluating the risks and benefits of Avandia compared to another drug, perhaps metformin. Many of us have long advocated that approach to evaluating drugs approved on the basis of surrogate endpoints. In 1999 my colleagues and I wrote about surrogate endpoints—when drugs are evaluated by their effects on risk factor levels such as glucose or blood pressure rather than long-term health outcomes (JAMA 282, 786–790; 1999). We saw a need in diabetes and hypertension to evaluate these drugs with large, long-term trials. That was the same year Avandia was approved.

So what should the FDA have done?

The medical officer's review at the FDA was nicely done. He recommended a study with information about liver disease, but he also saw some adverse effects on LDL cholesterol and weight, and wanted to know what the overall cardiovascular risk or benefit was. Does Avandia actually reduce the risk of cardiovascular disease? Does it increase it? He identified that as an important question.

I don't know whether the FDA asked for the right trial and the company wouldn't do it, or whether the FDA didn't ask for the right trial. I have no independent information about the process of negotiating that trial.

If Avandia increases heart disease and heart attacks, does it really work?

Avandia lowers blood glucose levels by about 25 to 30 milligrams per deciliter. This change is not going to improve anybody's quality of life. But high blood glucose is a risk factor for heart disease and we know that the higher the glucose, the greater the risk of cardiovascular disease. And what we are trying to do is reduce the risk of events by reducing the levels of risk factors such as glucose. So people taking the drug should have lower rates of heart attack, heart failure and strokes.

The effect of Avandia on blood glucose had suggested the possibility of a beneficial effect on heart disease. But we have the paradoxical finding that Avandia, in clinical trials, is associated with an increased rather than a decreased risk of myocardial infarction. Microvascular disease, blindness and renal disease are also all linked with blood glucose levels.

How significant is the increase in risk that Nissen found?

The baseline risk for a heart attack in a diabetic person is going to be something like 10 to 20 per 1,000 person-years. The rate depends on the person's age, gender and other risk factors. The Nissen meta-analysis suggests that among Avandia users, the rate would have increased by 40%—about 14 rather than 10 or about 28 rather than 20 per 1,000 person-years. It is not a huge risk. The effect size is similar, but in the opposite direction, to that of the lipid-lowering statin drugs, which reduce the risk of cardiovascular events.

So what should we take away from this?

Do the long-term trials. Let's say the drug does prevent heart disease. The delay in evaluation helps no one. It makes the FDA look as if they are not on the ball. It makes the company look bad—we are eight years out before we identify an important health risk.

GSK tried to dismiss the meta-analyses as a 'hypothesis'. It is not a hypothesis. It is the cumulative clinical trial evidence that we have about heart attack risk. The responsibility for not having more and better evidence in a timely fashion certainly rests with the company and also possibly partly with the FDA. The FDA lacks the authority that they need to insist on the proper trial design and timely completion.

The current system is set up to do a high-quality evaluation in the preapproval setting. Studies after approval are often for marketing. What would have been most valuable from the point of view of public health is the large, long-term trial. What we have instead is late, low-quality information. The current system does not serve the health of the public well.