Drug safety special

Chasing shadows

Why do current safety systems sometimes fail to notice that approved drugs are causing serious adverse effects? And can surveillance methods be improved? Simon Frantz investigates.

Credit: K. BROFSKY

It's 3 a.m. in a typical US emergency room, and doctors are bustling around a patient in his mid-sixties, one of several admitted that night with severe chest pain. He's anxious, sweating and struggling to breathe — classic symptoms of a heart attack.

By the following evening, he has stabilized, thanks to the prompt administration of streptokinase, which dissolved the clot that was starving his heart muscle of oxygen. It's time to examine factors such as obesity, smoking and blood cholesterol that might have contributed to the attack, and advise the patient about lifestyle changes that will help minimize the chances of a recurrence.

An estimated one million Americans are admitted to hospital under similar circumstances each year; about a third of them die. This is an all-too-common emergency, with a host of everyday causes.

No one thinks to consider whether the patient's attack might have been triggered by the pills, prescribed to ease the pain of arthritis, sitting on his bedside table back at home.

At least that was the case before 30 September 2004, when the pharmaceutical giant Merck withdrew its painkiller Vioxx from the market after it was linked to an increased risk of heart problems. The news was met with confusion, anxiety and outrage. How could the Food and Drug Administration (FDA), which is supposed to protect the US population from unsafe medicines, and which approved Vioxx for sale in May 1999, not have noticed the dangers?

Hidden dangers

Critics argue that the FDA needs to re-examine its priorities in light of the Vioxx scare (see page 554). But whatever the outcome of that debate, the drug's withdrawal has revealed how adverse effects such as heart attacks, which already occur commonly in the general population, can slip under the radar of current drug-safety surveillance. And in the wake of the Vioxx controversy, experts are examining ways in which methods of detecting such effects could be improved.

The problem with Vioxx was never going to emerge in the phase III clinical trials that are used to judge whether a drug should be put on the market. The heart attacks occurred in less than 2% of patients who had taken the drug for many months1, whereas phase III trials are typically carried out on a few thousand patients over a timescale of several weeks. Only when hundreds of thousands of patients had taken Vioxx for an extended period would any problems become apparent. Such events can only feasibly be investigated after a drug is in general use.

Current post-marketing surveillance systems can easily detect adverse events that are unexpected and rare. For instance, the multiple sclerosis treatment Tysabri was withdrawn from the US market in February, about four months after it was given expedited approval, when two people contracted a rare brain disorder called progressive multifocal leucoencephalopathy, or PML. “PML is a very rare event, so this was like a red flag waving,” says Steven Galson, acting director of the FDA's Center for Drug Evaluation and Research.

Invisible enemy

The difficulty with common conditions such as heart attacks is that cases that might be triggered by prescription drugs readily get lost in the noise. That's especially true for patients with ailments such as arthritis, who tend to be older, and therefore more susceptible to heart disease. Conventional post-marketing surveillance schemes, which mainly rely on doctors informing drug companies of adverse events associated with their products, and companies in turn informing regulators, are almost useless in this regard.

“Spontaneous reporting systems cannot distinguish whether adverse events such as heart attacks are the result of a drug or whether it would happen in that community anyway,” says Alasdair Breckenridge, who chairs the UK Medicines and Healthcare products Regulatory Agency.

Bitter pill: health problems linked to Vioxx exposed the frailties of post-market drug safety surveillance. Credit: M. DERR/AP

In such cases, the burden falls on researchers in the field of pharmacoepidemiology, who study clinical databases containing millions of patient records, documenting information on medicines prescribed, hospitalizations and deaths. Taking their cues from potential problems highlighted in clinical trials, case reports or lab experiments, these researchers use statistical tools to study large patient populations over extended periods.

“We look through the databases to find an exposed group and a non-exposed group that are quite similar except that one group gets Vioxx, for example, and one group doesn't,” explains Susan Jick, an epidemiologist at the Boston University School of Public Health, and co-director of the Boston Collaborative Drug Surveillance Program, based in Lexington, Massachusetts. “Or you do a case-control study where you look at similar people who have had, say, a heart attack, and those who haven't, and you see whether they have taken Vioxx or not.”

This sounds simple. But in practice, it is a painstaking and labour-intensive process. The problem is particularly acute in cases in which it is difficult to determine whether an adverse event is a consequence of the underlying disease, or the medication — for example, when studying patients with heart attacks who are taking drugs to control their blood pressure. But even in less ambiguous situations, it's tough to sort patients into groups that can fairly be compared to one another.

Class divides

Wayne Ray's team demonstrated that long-term use of Vioxx increases the risk of a heart attack. Credit: D. JOHNSON/VANDERBILT UNIV.

In the case of Vioxx, which belongs to a class of drugs called COX-2 inhibitors, concerns about an elevated risk of cardiovascular disease were first raised in 2001 by a reanalysis of a study comparing patients on drugs from this class with those on an older painkiller called naproxen2. At the time, Merck suggested this was because of the cardioprotective effect of naproxen, rather than adverse effects of Vioxx.

Researchers led by epidemiologist Wayne Ray of the Vanderbilt University School of Medicine in Nashville suspected otherwise. Their initial study, which examined patients on naproxen and other drugs of the same class, known as non-steroidal anti-inflammatory agents, found no evidence of a cardioprotective effect3. But a direct examination of the risks posed by Vioxx, conducted by a collaboration including Ray's team, and led by FDA researcher David Graham, wasn't published until 2005.

The study4 suggested that up to 140,000 excess cases of serious coronary heart disease — resulting, Graham says, in at least 26,000 deaths — could have occurred in the United States while Vioxx was on the market. But this required painstaking effort to decide which patients to include in the comparisons, examining such variables as existing cardiovascular disease and arthritis. The project drew on the skills of a cardiologist, a rheumatologist and a statistician. “You have to have all these experts who can bring their knowledge to bear,” says Ray. “It might take a few months to do the computer runs, but it takes many, many months of careful thought going into it beforehand.”

“The frustration is that we have the technology, but there is very little funding for carrying out post-approval studies. ”

Such difficulties are compounded by a dearth of support for pharmacoepidemiological studies. “The frustration is that we have the technology, but there is very little funding for carrying out these studies,” says Brian Strom, an epidemiologist at the University of Pennsylvania School of Medicine in Philadelphia.

US government agencies don't provide significant funding for such studies. The FDA's Office of Drug Safety, which is responsible for monitoring and assessing the safety of existing drugs, has extremely limited funding for epidemiological studies of its own. A small programme at the Agency for Healthcare Research and Quality allocates just $5 million for drug-safety surveillance studies — a fraction of the roughly $450 million it costs on average to do the clinical trials on a single drug before it can be brought to market.

Aside from funding for more studies of drug safety, the main need is to develop clinical databases that pharmacoepidemiologists can readily query to test their hunches about adverse drug events. These need to incorporate full records of patients' medical history, including any hospitalizations and the medicines they were prescribed.

The largest such database is the UK General Practice Research Database, which contains over 2.5 million patient records. It is a product of Britain's nationalized healthcare system, in which family doctors are patients' first point of contact. This database has proved successful, for instance, in confirming fears that antidepressants might trigger suicidal behaviour in some patients5. But ideally researchers want to extend studies to the United States, where drugs are typically approved earlier, and are taken up more quickly by larger numbers of patients.

Separate sources

Here, the problem is that such data are collated by individual Health Maintenance Organizations (HMOs), such as the California-based Kaiser Permanente, whose database was used for Graham's Vioxx study. Integrating these databases, and ensuring that the data they contain have common definitions and formats, would create a powerful resource for observational studies — but it is a major challenge. “Integration is probably the biggest stumbling block,” says Donald Berry, who chairs the Department of Biostatistics and Applied Mathematics at the M. D. Anderson Cancer Center in Houston, Texas.

Nevertheless, some attempts at database integration are now getting off the ground. Eric Larson, director of the Center for Health Studies in Seattle, part of the non-profit Group Health Cooperative, is developing the Coordinated Clinical Studies Network. Involving 13 HMOs around the United States, the aim is to create an integrated data ‘warehouse’ containing clinical information on up to 18 million people, initially focusing on cardiovascular disease. The three-year project began in November 2004 and has been funded to the tune of $2.7 million by the National Institutes of Health.

Even with the funding hurdle cleared, other problems might hamper this ambitious initiative. The biggest difficulty is the lack of will to carry out this project, says Larson. For people working in private sector HMOs, devoting up to three years to setting up a network, rather than actually using it, is a daunting prospect. “This is a labour of public service,” says Larson.


  1. 1

    Bresalier, R. S. et al. N. Engl. J. Med. 352, 1092–1102 (2005).

    CAS  Article  Google Scholar 

  2. 2

    Mukherjee, D., Nissen, S. E. & Topol, E. J. J. Am. Med. Assoc. 286, 954–959 (2001).

    CAS  Article  Google Scholar 

  3. 3

    Ray, W. A., Stein, C. M., Hall, K., Daugherty, J. R. & Griffin, M. R. Lancet 359, 118–123 (2002).

    CAS  Article  Google Scholar 

  4. 4

    Graham, D. J. et al. Lancet 365, 475–481 (2005).

    CAS  Article  Google Scholar 

  5. 5

    Jick, H., Kaye, J. A. & Jick, S. S. J. Am. Med. Assoc. 292, 338–343 (2004).

    CAS  Article  Google Scholar 

Download references

Author information



Related links

Related links

Related links in Nature Research








Related external links

Centers for Education and Research on Therapeutics

General Practice Research Database

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Frantz, S. Chasing shadows. Nature 434, 557–558 (2005). https://doi.org/10.1038/434557a

Download citation

Further reading


By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.