A key test to assess cognitive function in Alzheimer's patients may not be able to tease out subtle improvements with medication. Credit: BSIP MENDIL/SCIENCE PHOTO LIBRARY

The search for a drug to treat Alzheimer’s disease could be being undermined by flaws in a test used in clinical trials to assess patients. And though few experts would blame the test for the recent failures of potential new drugs, a major push is now on to produce more sensitive ways of measuring the progress of the disease.

During the past few years, a number of clinical trials have produced disappointing results for high-profile drugs, and some pharmaceutical companies have abandoned Alzheimer’s disease altogether, frustrated by the high costs and difficulty of producing a drug with a measurable effect. Jeremy Hobart, a neurologist at the Plymouth University Peninsula Schools of Medicine and Dentistry, UK, says that flaws in the ADAS-Cog test could be partly responsible.

In many trials branded ‘failures’, the ADAS-Cog (Alzheimer’s Disease Assessment Scale — Cognitive Behaviour section) has been used as the key test of whether a drug is working. The test scores patients on 11 components using a variety of tasks relating to memory, language and praxis (the planning of movement to achieve a purpose), such as word recognition and remembering instructions. Lower scores indicate better cognitive performance and so less severe disease.

There is, says Hobart, an argument that any study that has used ADAS-Cog may have underestimated changes in and differences between patients given the drug and controls. In two papers in Alzheimer’s & Dementia, Hobart and his colleagues detail flaws in the test that undermine its utility.

Ceiling effect

First, the authors used a large set of test scores to show ‘ceiling effects’ in eight out of the eleven ADAS-Cog components — meaning that there is an upper limit past which these parts of the test do not properly capture differences between individuals.

This means that for those in the very early stages of the disease the test is not detailed enough. Hobart likens it to measuring people’s height when all you have is a six-foot-long ruler — anyone taller than six foot is just lumped into one category, even though there may be significant differences between them.

In their second paper, the researchers applied a complex mathematical test to analyse how well ADAS-Cog performed as a measurement instrument. They concluded, again, that it has limitations “with potentially serious implications for clinical trials”.

If the test is not adequately measuring people's cognitive function, it will not pick up on subtle improvements produced by drugs being tested, says Hobart. He adds that there is no way of knowing how many trials could have been affected, and even goes as far as to say that some trials may have failed as a result — although other researchers disagree on this point.

Testing times

Part of the problem, say a number of scientists in the field, is that ADAS-Cog was published in 1984, when people’s concept of Alzheimer’s was more limited. Over time, researchers have become more interested in the earliest stages of the disease, when impairment is less obvious. To improve matters the test could be overhauled — that is in crude terms, made harder.

“These papers are very timely because we need new tests ... that are more sensitive,” says Stephen Salloway, director of neurology and the Memory and Aging Program at Butler Hospital in Providence, Rhode Island. Salloway notes that successors to the ADAS-Cog test that involve additional tasks are already in development.

Clive Ballard, director of research at the UK charity Alzheimer’s Society, agrees there is debate to be had over a better test. But he points out that ADAS-Cog is widely accepted by regulatory agencies and that there is a large body of work regarding its use in clinical trials. Any new test would need a huge amount of additional work before it was trusted to become the standard test. “I don’t think it’s perfect, but changing it does mean we lose all of that background,” Ballard says of ADAS-Cog.

Neither Salloway nor Ballard is convinced that problems with ADAS-Cog are the reason for recent drug-development setbacks. “The negative results are due to the small effect size of the drug being tested and not the measurement instrument,” says Salloway. “I wish I could blame it on the test.”