The National Institute of Health and Clinical Excellence (NICE) is no stranger to controversy. Charged with deciding which new treatments should be made available to patients who use the National Health Service (NHS) in England and Wales, NICE has come under repeated public attack from the pharmaceutical industry and patient groups whenever it has decided against making a new therapy available.

Until now, most of these criticisms have been voiced from an ethical standpoint, questioning the moral case for denying any group of patients a potentially life-prolonging treatment. However, a new report funded by the European Commission differs in that it attacks the theoretical underpinning of the yardstick that NICE uses to assess new treatments: the 'quality-adjusted life year', or QALY.

QALYs are “by far the best way of getting a 'common currency' for assessing the health gains secured by different treatments,” says Peter Smith, a health policy researcher at Imperial College London who was not involved in the study. Yet “in practice they are hard to estimate, and they reflect only a representative individual's preferences,” he adds.

First proposed in the late 1970s and early 1980s as a way to objectively compare competing claims for finite healthcare resources, the QALY is now used to guide the decisions of NICE and similar agencies in Australia, Canada and Scotland. It's also under consideration in several European countries with publicly subsidized healthcare systems.

A QALY is calculated by taking the additional life expectancy conferred by a treatment and multiplying it by a value between 0 and 1 that represents the quality of that additional life expectancy. Thus, a drug that gave someone an extra 1.25 years of life on average with a quality rating of 0.6 would be assigned a QALY of 0.75. New therapies markedly above the given cost threshold per QALY—currently set by NICE at £30,000 ($47,000)—are unlikely to be made available on the NHS.

The difficulty lies in accurately calculating the quality-of-life part of the formula, and it is the assumptions that underlie this calculation that the authors of the new report now claim to have invalidated. “This is the largest-ever study specifically dedicated to testing the assumptions of the QALY,” says Ariel Beresniak, the lead author of the study from the European Consortium in Healthcare Outcomes and Cost-Benefit project and chief executive of Data Mining International, an independent research institution based in Geneva.

More than a number: Patients' views vary. Credit: istockphoto

When theory meets practice

The authors surveyed a representative sample composed of 1,300 members of the public in Belgium, France, Italy and the UK to test whether the assumptions that underlie the quality-of-life calculation are an accurate reflection of people's preferences. For example, one of the key assumptions of the QALY formula is that someone who wants to live longer in one health state would also want to live longer in another health state. “According to the theory, we would expect 100% would prefer ten years in a wheelchair instead of five years, but in fact only 72% said they would,” says Beresniak. “The theory assumes some consistency about people's preferences, but in fact it is totally different in the real world.”

Although the full report is still confidential, the main findings of the new report were presented in Brussels at the end of January and drew a swift response from NICE. Describing the study as “limited,” the agency rallied a phalanx of health economists and academics to their defense. One, Sheila Bird, who leads the Medical Research Council Biostatistics Unit at the University of Cambridge, UK, points out that NICE's decisions were based on valuations of quality of life “drawn up through a carefully designed survey in 1993 of 3,395 interviewees” from representative addresses within the UK, rather than the 1,300 respondents drawn from several nations upon which the study authors based their conclusions.

Franco Sassi, a senior health economist at the Paris-based Organisation for Economic Co-operation and Development, remains a defender of using QALYs as a metric. “QALYs are often used as a straw man for attacking the expanding role of economic evaluation in health care,” he says. Although alternative measures, most notably healthy year equivalents, have been touted as challengers to QALYs, Sassi says that “the failure of over 30 years of health economics research to produce an outcome measure for economic evaluation that is superior to the QALY on all grounds is a strong indication that the QALY has highly desirable properties, as well as limitations, and it will likely continue to be used.” He dismisses calls for the abandonment of QALYs as coming from “vested interests” who want to undermine the economic evaluation of pharmaceuticals and medical devices.

Beresniak, for his part, favors a more classical cost-effectiveness approach of calculating cost per clinical result. That might mean calculating the cost per pregnancy in fertility, for example, or the cost per clinical remission in arthritis. “Most English economists say 'QALY is the only way', but that is totally false,” he says. “In France we have one of the best health systems in the world, and we don't use QALYs.”