Decision makers, wrestling with thorny choices, are tapping into the collective foresight of ordinary people. Jim Giles reports.
Like all of their rivals in the pharmaceutical industry, executives at Eli Lilly routinely need to make tough predictions. How much of a new product do they expect to sell? Is a competing drug going to win approval first?
Millions of dollars of revenue hang on get-ting these predictions right. So it might come as a surprise to learn where some executives at the Indianapolis-based firm have looked for answers: they've been asking readers of USA Today.
Lilly is one of several major corporations now dabbling in ‘prediction markets’ — decision-making tools that harvest the collective wisdom of groups of ordinary people. Participants are asked to buy and sell shares in real outcomes, and are rewarded for betting on outcomes that turn out to be correct.
Advocates of the prediction markets claim that the predictions of such participants can end up being better than those made by specialists.
“The idea of prediction markets is a powerful one,” explains Thomas Malone, a management specialist at the Massachusetts Institute of Technology. “They let many people contribute to the collective assessment of a future event. It's a surprisingly effective way of integrating information.”
Lilly runs its markets in conjunction with NewsFutures, a firm based in New York that sells prediction-market software. When the two companies first collaborated in 2003, USA Today was experimenting with the idea as a game for its readers. Lilly subsequently asked 250 of these readers to make predictions about some of its business issues.
The group was invited to buy or sell shares pegged to specific predictions, such as the number of drugs that would be approved in a year by the US Food and Drug Administration. Shares in the correct prediction paid out virtual money at the end of the year, and Lilly motivated traders by stumping up $10,000 in real money to reward the winners.
Although there hasn't been much independent analysis of prediction markets, the data that are available suggest the concept carries promise. Markets aimed at predicting the outcome of US presidential elections and the names of Oscar winners have, according to some assessments, outperformed other indicators such as opinion polls. Emile Servan-Schreiber, NewsFutures' chief executive, says the same pattern emerged with the 2003 Lilly market; outside traders did better in their predictions than company experts.
Supporters say prediction markets motivate participants to carefully study information related to whatever they are trading in. And as outsiders, they lack the vested interest that can colour the predictions of internal staff, or even some consultants, who might, for example, have a historical empathy for a particular product line. “The markets provide an interesting counterpart to predictions from groups, such as lobbyists, who tend to see things in black and white,” says Servan-Schreiber.
Since its initial study, Lilly has commissioned two further prediction markets, the second of which is running this year and using its own sales staff as participants. In this market, predicted outcomes, such as the revenue that a specific Lilly drug will achieve in different quarters of the year, are assigned a price: a trader might buy a share in that outcome if, for example, they think the share price underestimates the outcome at the end of the quarter. Lilly declined to discuss the details of this market, but Servan-Schreiber claims that it has already outperformed standard internal forecasts.
Rival firm Abbott Laboratories, based in Illinois, has also purchased prediction-market software from NewsFutures, but declined to say how this is being used. And similar schemes are finding myriad uses elsewhere. The University of Iowa in Iowa City, for example, has set up a prediction market to help health authorities assess how the state will be affected by influenza in the coming year.
But how much trust can be placed in these tools? Much of the enthusiasm for the concept has been driven by the publicity surrounding their use as predictors of the outcomes of US presidential elections. But the data from these are slim, and corporate leaders may be sceptical about what prediction markets are really worth.
Charles Manski, an economist at Northwestern University in Illinois, has looked into the markets' performance, and takes issue with one of their assumptions, namely that traders' beliefs alone determine market price. He thinks that the price is also affected by differences in traders' budgets and attitudes to risk, and that unless these other two influences are understood, predictions will be misinterpreted. Manski warns against believing all the claims made about the approach. “The problem with prediction markets is that they've been hyped by the kinds of people who believe that markets solve all problems,” he says.
Malone concedes that the markets still need to earn their spurs. But he stresses that information from them should at least be considered — especially when it runs counter to mainstream thinking. “At a minimum, you should do more investigation to find out why the market said what it did,” he suggests. Once firms start to see the prediction market as just another tool, it'll become an everyday part of business management, he predicts. “Information technology has allowed the cost of doing this to fall to almost zero,” Malone says. “It will become routine.”
Related links in Nature Research
Climate sceptics place bets on world cooling down
Related external links
Rights and permissions
About this article
Cite this article
Wisdom of the crowd. Nature 438, 281 (2005). https://doi.org/10.1038/438281a
This article is cited by
Using prediction polling to harness collective intelligence for disease forecasting
BMC Public Health (2021)
How not to put the O into an OSCE
Perspectives on Medical Education (2018)