As healthcare systems look to save cash, many are turning to cost-effectiveness analyses that show which drugs make the most economical sense. But current methods for comparing the returns on medications might be trickier than previously thought, according to new research.

In a UK study, randomized clinical trials (RCTs) did not hold up to actual clinical practice in determining cost effectiveness (PLoS Med. 6, e1000194; 2009). In particular, researchers compared nonsteroidal anti-inflammatory drugs (NSAIDs), which carry a risk of gastrointestinal side effects, to more expensive Cox-2 inhibitors that do not have the same side effect.

Though Cox-2 inhibitors include the now-banned Vioxx, which has shown a risk of heart complications, data from the UK's General Practice Research Database (GPRD)—an anonymous repository of general practitioners' medical records, including demographic information, prescriptions and clinical events—showed the drug's true price has been underestimated. Randomized clinical trials found that switching patients from NSAIDS to Cox-2 inhibitors, for the sake of avoiding an adverse gastrointestinal event, cost an average $18,000 per person. When researchers used information from the GPRD, however, the cost of switching to Cox-2 inhibitors skyrocketed to $104,000 per person.

The comparison shows that even when a drug doesn't have years of clinical practice data behind it, there needs to be better evaluation of tested and targeted populations, says Tjeerd-Pieter van Staa, one of the study's authors. “This adds another piece to the evaluation of a drug when it enters the market,” he suggests.

Van Staa also notes that guidelines by the UK's National Institute for Health and Clinical Excellence (NICE)—which conducts drug evaluation for the country's National Health Service—do not distinguish the analyses needed for cost effectiveness, instead relying heavily on randomized trials.

According to Carole Longson, director of the Centre for Health Technology Evaluation at NICE, the new study highlights “the need to be cautious in taking randomized control trial data and accepting it at face value, without scrutiny of the applicability of the trial population to the population likely to receive the medicine in routine practice.”

Alan Garber, an American health economist, said the US can also derive lessons from the study findings. The US lacks a repository of information similar to the UK's GPRD and may need to invest in one. “It's all a matter of implementation, and this study demonstrates a need for much better monitoring of how we treat and how we administer these kinds of medications,” Garber says.