Click for larger image

Click here for larger image

More than two-thirds of the world's population lives in countries that lack a reliable system for issuing medical death certificates, leaving the true scale and distribution of disease in serious doubt. The main tactic for filling that gap is verbal autopsy, which assigns a probable cause of death based on interviews with families about the deceased's symptoms.

But the reliability of the technique is under fresh scrutiny after a paper published in The Lancet last week1 used verbal autopsy to calculate that 125,000–277,000 people in India die from malaria every year (see 'Malaria mortality'). That is an order of magnitude larger than the 30,000 deaths per year that the World Health Organization (WHO) estimates.

The Lancet paper used the most common form of verbal autopsy, in which physicians assign the cause of death. But statisticians argue that probabilistic computer models can do a better job than doctors. The WHO also argues that verbal autopsy can be poor at differentiating malaria from other diseases that cause fever symptoms, which include septicaemia, viral encephalitis and pneumonia. Although the WHO has accepted the use of verbal autopsy to monitor malaria deaths and other diseases, Christopher Dye, a senior WHO official, says the method can easily give misleading results.

Brian Greenwood, a malaria epidemiologist at the London School of Hygiene and Tropical Medicine, who performed some of the earliest verbal autopsies for malaria in Africa, says that malaria deaths in India are probably underestimated to some extent, but shares the WHO's concern about the "very poor" performance of the technique on fever symptoms.

Greenwood is also concerned that as physicians in the study were familiar with the Indian states that they reviewed case reports from, the survey had a built-in bias. As any medic in India probably knows the most malarious states, this could lead to "a temptation to ascribe febrile cases to malaria" in such states, says Greenwood.

Prabhat Jha, an epidemiologist at the Centre for Global Health Research at the University of Toronto, Canada, and a co-author of the study, vigorously defends the results, arguing that physicians were given clear guidelines to carry out differential diagnosis to exclude malaria as the cause. The "total assignment of malaria deaths is not as biased as might be first believed", he says.

"We didn't blind as we thought it was important that coders knew where the case report came from," he adds. "It gave contextual information. If it smells like malaria, looks like malaria, and you see it in malarious regions then it probably is malaria."

But Gary King, a statistician at Harvard University in Cambridge, Massachusetts, notes that the different pairs of physicians that looked at each case in the Lancet paper often disagreed on the cause of death. "The error rates between the experts account for half the malaria deaths estimated," he says.

Bob Snow, a malaria epidemiologist at the Kenya Medical Research Institute–Wellcome Trust Research Programme in Nairobi, says that whatever the limitations of the study, its estimates are "closer to the truth than the WHO figures", and that its findings are consistent with the spatial and temporal epidemiology of malaria in India. Snow notes that the paper is in line with his own team's findings that the WHO has underestimated the clinical incidence of malaria in India by a similar order of magnitude2.

The needs of the many

Verbal autopsy is increasingly being questioned by statisticians, says Edward Fottrell, an epidemiologist at Umeå University in Sweden. Until now, verbal autopsy has been dominated by physicians, whose clinical background means that they tend to believe that diagnosing individual cases is key for accuracy, he says.

But the ultimate goal of verbal autopsy is not to make clinical diagnoses of individual cases, Fottrell points out. It is to estimate the distribution of causes of deaths, known as cause-specific mortality fractions (CSMFs), which are crucial to setting health-system and research priorities, and to monitoring the effectiveness of disease-control measures.

Pigeonholing cases into a single, accurate cause of death can amplify the errors in the CSMFs, says King. A better approach, he says, is to calculate the probabilities that various disease symptoms are associated with a death, and then aggregate those probabilities across an entire set of cases3.

Studies show that these probabilistic computer models can give CSMFs as good as or better than physician review, but are far faster and cheaper4. They also overcome the issue of physician subjectivity, providing a standardized method that makes results more comparable between different studies and countries.

Many researchers are reluctant to embrace verbal-autopsy models that dispense with physician review, but attitudes may be changing. The Swedish International Development Cooperation Agency, based in Stockholm, recently recommended that the international INDEPTH surveillance network, which records births, deaths and disease within large population cohorts in 17 African and Asian countries, adopts a probabilistic verbal-autopsy model. Fottrell predicts that computer models will eventually prevail over physician review.

The ultimate goal, however, is to ensure that verbal autopsy is no longer needed, says Dye, and the WHO is helping all countries to eventually implement the gold standard of a systematic medical death certification. "That is the end point that the WHO is working towards."