London

Frustrated weather forecasters are fighting back against rogue companies that sell forecasts with claims of impressive accuracy, but that have no apparent scientific basis.

Private forecast providers have proliferated in Europe and the United States over the past decade as various industries demand increasingly accurate information about the weather. Insurance companies want to be warned of wet weather, for example, which can result in claims for flood damage. And retailers need shorter-term predictions to help them judge what to stock. “If there's a heatwave coming we need barbecues, salads and lager,” says a spokeswoman for Sainsbury's supermarkets in the United Kingdom.

Most providers are large companies that generate a high level of satisfaction among their customers. But even national meteorological offices have been accused of using unclear statistics to advertise the accuracy of their forecasts. And there is nothing to stop any individual from using freely available data, such as those released by the US National Weather Service, to produce their own weather forecasts.

Meteorologists are concerned that the rise of private forecasters could affect the reputation of their trade. They are reluctant to name specific firms for fear of legal action, but say that some forecast providers use professional-looking websites to hide the fact they are using scientifically flawed methods that produce unreliable predictions. “We have companies claiming they can predict months ahead, using methods they will not mention,” says Pascal Mailier, a meteorologist at the University of Reading, UK.

So Britain's Royal Meteorological Society has decided to develop an accepted set of metrics with which to rate the accuracy of different forecasting firms. Mailier, who is leading the project, hopes the plan will clean up the industry. “I'm giving them the chance to prove themselves,” he says.

Washout: inaccurate weather forecasts can lead to huge losses for companies such as insurers. Credit: C. LUZIER/REUTERS/NEWSCOM

To come up with a fair way of rating firms, the society has asked Mailier and his colleagues to survey the methods that are available for assessing forecasts. They will present preliminary results, together with comments from users and the 20 or so forecast providers in Britain, to the society's annual conference in September. The rating system could in principle be used by an independent body to gauge the accuracy of different providers.

Finding a method that everyone can agree on, however, is likely to be less straightforward than it sounds. One way is to award points for different aspects of the forecast, such as wind speed or temperature, and add them up to give a total score. But this involves arbitrary judgments, so the score is highly dependent on how the points are awarded.

The Met Office inBritain has traditionally rated its next-day forecasts as 85% accurate. This sounds impressive, but forecasts obtained simply by assuming that tomorrow's weather will be the same as today's achieve 77% (see J. E. Thornes and E. A. J. Proctor Weather 54, 311–321; 1999). The Met Office now says that the figure of 85% attracts too much attention, and prefers instead to use standard statistical measures of error that focus on one variable at a time.

No single metric is likely to do the job, the project team warns, because the needs of users vary. Local governments might be interested in the accuracy of ground-temperature forecasts, for example, to decide whether roads need to be gritted in advance against ice. Airports want to know wind speeds, and energy suppliers care about temperature, especially longer-term events such as unusually mild winters.

Meteorologists also caution that data from some rural areas are patchy, and that some aspects of weather will be hard to quantify. “The state of the sky is the most difficult,” says John Thornes of the University of Birmingham, UK. Many providers forecast the cloud cover, but most measurements are generated visually by airport employees.

Despite these barriers, many users of forecasts contacted by Nature welcomed the society's survey plans. Major companies such as supermarkets and energy suppliers already run their own assessments of the forecasts that they buy, but say an independent set of metrics would be a useful addition.

Even forecasts from reputable providers can vary considerably in their accuracy, says Gearoid Lane of Centrica, an energy and services company based in Windsor, UK. If the forecasts are wrong, he adds, “there are big financial consequences”.