New York

The United Nations has set targets for the relief of human suffering, but progress is hard to measure. Credit: D. TELEMANS/PANOS

As world leaders gather in New York this week to discuss the future of the United Nations' Millennium Development Goals, critics are warning that many of the targets cannot be evaluated scientifically. Unless the goals are peer reviewed and amended to take account of what can actually be measured, they say, the project will fail.

Nearly 150 heads of state met in 2000 and pledged to fight problems ranging from poverty to infectious disease in a historic document called the Millennium Declaration. This was turned into a series of numerical targets with a final deadline of 2015 that are known as the Millennium Development Goals.

The ambitious and concrete-sounding aims captured imaginations and headlines worldwide, and have become symbolic of the global development effort. They include halving the number of people that live on less than a dollar a day and cutting deaths in children under five by two-thirds.

But analysts such as Amir Attaran of the University of Ottowa, Canada, say that without drastic revision, the targets risk being meaningless. “We haven't got the tools in place to measure them,” he says.

The UN Statistics Division currently keeps tabs on 48 ‘indicators’, which range from school enrolment numbers to malaria death rates. These are provided by agencies such as the World Health Organization(WHO) and the World Bank, but Attaran argues that the methods used to gather data are scientifically flawed. In the absence of birth and death registries in most African countries, for example, many figures on death and illness are gathered from household surveys. These often use conflicting methods, and do not always acknowledge sources of error or apply statistical tests of significance.

And although surveys may measure some indicators accurately — such as childhood deaths, which parents will remember — others are harder to estimate, such as cases of malaria or deaths in childbirth. For example, if a woman succumbs to fatal bleeding in hospital after giving birth at home, the link may not be recorded.

Attaran warns, in PLoS Medicine, that without accurate baseline figures and ongoing measurements it will be impossible to tell in 2015 whether the millennium goals have achieved anything. He wants independent peer reviewers to assess individual targets and indicators to determine whether they can be measured scientifically. If they cannot, he says, the goals should be amended or even abandoned.

Epidemiologists agree that the various indicators could be made more specific, and say leaders should take the opportunity this week to clarify them. “It needs to be talked about now,” says Robert Kim-Farley of the University of California, Los Angeles, who used to gather public-health data for the WHO.

“It is relevant to discuss whether we are investing enough in data-collection systems,” adds Stefano Bertozzi, director of health economics for the National Institute of Public Health in Cuernavaca, Mexico.

Some UN scientists have expressed concerns like Attaran's. They tried to start a debate last year about how to measure the goals more accurately. But the deputy secretary-general, Louise Fréchette, deflected them in a message sent to the expert committee in charge of goal statistics. The New York summit “should not be distracted by arguments over the measurement of the millennium development goals — or worse, over the different numbers being used by different agencies for the same indicators”, she said.

John McArthur is deputy director of the UN Millennium Project — an independent body based in New York that advises the United Nations on the goals. He agrees that the priority for this meeting is simply for nations to affirm their support. He acknowledges that some of the targets are tough to track, but adds, “I don't think that's a reason not to make a concerted effort.”