An off-the-shelf statistics package has tripped up pollution researchers in North America and Europe who are studying the effects of airborne soot on human health.
A default setting that produced erroneous results went unchecked for years, despite significant statistical expertise in all of the groups. “It was already such standard software when we started using it, I didn't even question it,” says Francesca Dominici, a public-health researcher at Johns Hopkins University in Baltimore, Maryland.
On 4 June, Dominici posted revised figures on her website after discovering that the error had doubled her group's estimate of the risk to health posed by particulates in the air. Two other groups that used the same tool, one in Canada and one in Greece, are now redoing their calculations.
The groups were looking for correlations between death rates and particulates in the air, which come mainly from diesel engines and power plants. Their data on air quality, hospitalizations and deaths from dozens of cities cover a seven-year period up to 1994.
Death rates vary throughout the year because of such factors as changes in temperature and disease outbreaks. To tease out the effects of particulates, the groups used a statistics program known as S-Plus.
S-Plus searches for correlations using an iterative process in which confounding effects are gradually peeled away. The default parameter in question determined how many times the procedure would iterate before stopping to produce a final result.
“For most applications the value is perfectly fine,” says David Smith, product manager of Seattle-based Insightful, which sells S-Plus. Smith says that the Hopkins case was exceptional, but that users should always check whether changing the parameter affects the outcome, and adjust it if necessary. Smith says that Insightful will tighten the default value of the parameter — slowing the programme slightly — on future releases of S-Plus.
Richard Burnett, a statistician with Health Canada in Ottawa, which is conducting a similar study, says that his group will probably revise its estimates of the impact of airborne soot on mortality downwards by 20–50%. The findings of a study run by a group at the University of Athens may also have to be adjusted, he says.
The health risk posed by particulates is a source of fierce environmental controversy in the United States, and the Bush administration is considering rules to restrict emissions. Opponents of tighter rules are likely to seize on the revisions as evidence that the research linking soot in the air to harmful effects on health is not to be trusted.