Nature | News

Poverty project opens to scrutiny

Panel set to reshape evaluation of Millennium Villages research after partial retraction of health claims.

Article tools

WpN/Photoshot

Health interventions may have reduced child death in African villages, but it isn’t clear by how much.

The Millennium Villages Project (MVP) aims to chart a course out of poverty for the most deprived people in Africa. But after being forced to partially retract the findings of a paper that included the claim that its interventions had slashed child mortality, the project is revamping its oversight to make its science more robust and determine whether its approach works.

The MVP, a joint venture between Columbia University and the non-profit organization Millennium Promise, both in New York, is simultaneously applying a raft of aid interventions — from bed nets for malaria control to fertilizers — to help impoverished villages to meet the United Nations’ eight Millennium Development Goals (MDGs), including reducing hunger, extreme poverty and the burden of major diseases, and increasing levels of primary education. The first Millennium Villages were launched in Ethiopia and Kenya in 2004 and 2005, and the project now operates in 10 African countries, reaching about 500,000 people in 80 villages over 14 sites.

In a paper published online in The Lancet last month1, the project claimed a significant milestone. It reported that after three years of interventions, child mortality was decreasing three times faster in the project’s villages than in the host nations in general. But the analysis was criticized for underestimating nationwide improvements in child mortality, and over­estimating those in the Millennium Villages2. The paper’s lead author, Paul Pronyk, later accepted that this claim had put the Millennium Village mortality reduction “in an excessively positive light”3, and retracted it. Pronyk then resigned as director of Monitoring and Evaluation at Columbia University’s Center for Global Health and Economic Development.

The MVP’s founder, Jeffrey Sachs, head of the Earth Institute at Columbia and a co-author of the partially retracted paper, says that the MVP research teams were too autonomous, and he regrets not having brought in external advisers earlier. “I don’t want such mistakes to occur again,” he says. Sachs has now created a faculty committee to oversee MVP research and increase interactions with outside researchers; he will co-chair the committee, along with Cheryl Palm, a scientist at the Earth Institute’s tropical agriculture programme.

An external advisory board called the Inter­national Scientific Expert Advisory Group will oversee how the MVP assesses the impact of its work. The panel is still being assembled, but will be chaired by Robert Black, head of the department of international health at Johns Hopkins Bloomberg School of Public Health in Baltimore, Maryland. Black says that Sachs has given him a remit “to open the study to more scrutiny, to be more transparent, to be very explicit about what are the expected outcomes of the study, and to be sure that evaluation is as well done, as rigorously, as possible”. Black also wants to respond to criticisms by giving external researchers access to as much of the MVP data as he can, as well as to detailed information on the project’s costs, so that outsiders can have greater confidence in the project’s results and cost–benefit analyses.

It is too soon to say what sort of recommendations the external panel will make on the MVP’s future research in the study villages. But given that the project is already almost seven years into its planned ten-year pilot phase, the scope for change is limited, says Black. “It is impossible to completely redesign the study, though many of us would have designed it differently,” he says.

Michael Clemens, a migration and develop­ment researcher at the Center for Global Development, an independent research institution in Washington DC, is unimpressed with the creation of the advisory panel, and argues that the MVP needs rigorous and transparent evaluation from completely independent groups. Clemens, who co-authored a letter4 to The Lancet that led to the partial retraction of the MVP’s findings, believes that the paper still has problems — in particular, a claim that the child mortality rate in Millennium Villages fell by one-third more than that in matched comparison villages over the course of the study. “The whole study should have been retracted,” he says, adding that he has submitted a new letter to The Lancet to this effect.

Black says that whether the MVP ultimately proves a success or failure, it will still provide useful information, so it is important to improve its evaluation mechanisms. “I don’t think it should be dismissed because it has flaws,” he says.

The UK government’s Department for International Development last year launched its own £3.8-million (US$5.9-million) independent evaluation to accompany an £11.5-million ten-year grant to create Millennium Villages covering 30,000 people in Ghana. “A massive effort is needed to raise millions of people in Africa out of poverty. Millennium Villages represent one approach, and they could make a major contribution,” the department notes. “Such an opportunity should not be missed merely for want of evidence. But nor should scaled-up resources be committed to a model without assessing whether it is indeed a cost-effective approach.”

Journal name:
Nature
Volume:
486,
Pages:
165–166
Date published:
()
DOI:
doi:10.1038/486165a

References

  1. Pronyk, P. M. et al. Lancet 379, 21792188 (2012).

  2. Gilbert, N. Nature 485, 158159 (2012).

  3. Pronyk, P. Lancet 379, 1946 (2012).

  4. Bump, J. B., Clemens, M. A., Demombynes, G. & Haddad, L. Lancet 379, 1945 (2012).

For the best commenting experience, please login or register as a user and agree to our Community Guidelines. You will be re-directed back to this page where you will see comments updating in real-time and have the ability to recommend comments to other users.

Comments

Commenting is currently unavailable.

sign up to Nature briefing

What matters in science — and why — free in your inbox every weekday.

Sign up

Listen

new-pod-red

Nature Podcast

Our award-winning show features highlights from the week's edition of Nature, interviews with the people behind the science, and in-depth commentary and analysis from journalists around the world.