A middle-aged man appears in a US emergency room complaining of chest pains. Tests show that an arterial blockage is starving his heart of oxygen. He could have treatment A, or treatment B. Which one should he and his doctors choose?

The best treatment, presumably, is the one that is more likely to improve his health. But all too often, medical science doesn't know which one that is. This lack of information persuaded US lawmakers to give $1.1 billion to 'comparative effectiveness' research in the economic stimulus bill last month. Now, health-care policy-makers and researchers must ensure that they make the most of this money.

There is nothing really new about comparative effectiveness research. Defined broadly, it includes anything from literature reviews to head-to-head trials of treatments, drugs, diagnostic procedures or prevention methods. What is new is that the United States is beginning to acknowledge that it needs to do much more of this type of work if it is to mould an affordable, sustainable and effective health-care system.

The US health-care system is by most measures one of the most extravagant and wasteful in the Western world.

The new money will be split between the National Institutes of Health ($400 million), the Agency for Healthcare Research and Quality ($300 million) and the Department of Health and Human Services ($400 million). The Institute of Medicine, part of the National Academies, will provide recommendations on how the money should be spent. The agencies must take care to allocate the funding in the most effective way — by focusing on conditions that have the most impact on a population, say, or in which the research will have a real effect on medical practice — not just on those that have the most effective lobbyists. When a clinical trial can cost tens of millions of dollars, $1.1 billion doesn't sound like much.

A question of cost

Where the really tough questions arise, however, is in how this information will be used. Sometimes the decision is easy: treatment A is much better than B, so A should become standard practice. But what if treatment A is only a little bit better than B, yet costs ten times as much — is it worth it? Decisions about clinical effectiveness are difficult to divorce from those about cost effectiveness.

Some of these decisions are already being faced by countries with established systems for judging clinical and cost effectiveness. The United Kingdom's National Institute for Health and Clinical Excellence (NICE), which advises the country's National Health Service on what it should pay for, is often held up as an exemplar of sophisticated analyses.

In the United States, however, any such discussion has long been political anathema. Many individuals there find it unthinkable to be refused a treatment simply because it is deemed too expensive. Powerful lobbies from pharmaceutical companies and medical associations concur; they do not want to risk having profitable drugs or services excluded from coverage. But the practical effect is that the health-insurance companies are left to make those decisions by limiting how much they will pay out for any given treatment — with little good evidence about what actually works. This is one reason why the US health-care system costs more than $2 trillion a year — it is by most measures one of the most extravagant and wasteful in the Western world. At some point, US policy-makers and US citizens alike will have to follow the lead of their counterparts in the United Kingdom and elsewhere, and start making systematic, evidence-based decisions about what to pay for and what to exclude.

These decisions involve cold, hard calculations about the quality and value of life — and how to balance what is good for the individual against what is good for society. When a new cancer drug costs hundreds of thousands of dollars and extends a life by a few months, it uses up money that could pay for many thousands of check-ups or hip replacements that could eventually extend more lives for longer. These are the highly complicated estimates that researchers try to build into their models. NICE's decisions can be enormously unpopular: the agency faced a barrage of criticism in 2008, for example, when it recommended that four kidney-cancer drugs should not be used.

Shared resources

One way in which researchers could help is by improving their analytical tools and sharing their information and analyses. A shared database of existing clinical evidence with standardized end points and patient information, for instance, would relieve researchers from having to start literature reviews from scratch each time. Likewise, sophisticated models of disease built on genetic and other information could help researchers model the effects of a particular treatment.

Scientists should also factor in population subgroups when planning their effectiveness research. Treatment A might be better for the population as a whole, but there may be groups, determined by their genetic make-up or disease type, for which B is best. Ideally, these two strands of research would go hand in hand. The $1.1 billion should not be siphoned off to solve academic puzzles about human genetic variation — but variation should be incorporated where it can help answer questions about who should receive which treatment.

A or B? Evidence is the right place to look for the answer. But for all the health care that can be bought for $1.1 billion, much more could be bought if comparative effectiveness became a sustainable operation rather than part of a one-off spending spree. Policy-makers and researchers should therefore be careful to show what this money can buy.