Election results are notoriously difficult to predict.© AP

Predicting the results of political elections could be transformed from a black art into a science, thanks to a new way of getting people to tell the truth. According to the mathematical psychologist who devised it, his method can elicit more truthful subjective judgements about anything from business forecasts to the quality of a piece of art.

Public polling and questionnaires are notoriously difficult to assess. Psephology, the science of voting prediction, has suffered serious embarrassments in the past. For example, when forecasting the outcome of the UK general election in 1992, the prediction of a close result was laughably far from the landslide victory that the Conservative party actually achieved.

It seems that, when there is no objectively correct answer to a question, people are likely to be biased by what they think others will say. For example, they are more likely to give an answer that agrees with what they perceive to be the majority or 'accepted' view than an answer that truly reflects their own belief.

Dražen Prelec of the Massachusetts Institute of Technology says he has come up with a way to eliminate this bias and has published it in Science¹.

He uses a method that manages to give high scores to people for a truthful answer by adding a second question. This query simply asks the person what they think other people will say. A comparison of this with the average answer can reveal how truthful they are being: if people lie, their score is reduced, even though no one else would ever know that they lied. Respondents are rewarded for a high score, in whatever way is deemed appropriate.

Truth and other people

The method takes advantage of the fact that holding an opinion is a valuable clue to how common that particular opinion is likely to be in a population. It is well established that people who hold an opinion are likely to give higher estimates of the popularity of that opinion than people who do not: because they think it is a valid belief, they think that other people will hold it.

Prelec takes this reasoning one step further, and says that given the limited information you have (that is, you know your own opinion, but not that of anyone else), you always have reason to believe that your opinion will prove more popular than others expect.

That is, you realize that you are likely to give a high estimate of your opinion's popularity and you think the rest of the population will underestimate the actual proportion of people that share it. Thus, an average of people's belief about the population will come out with a comparatively low estimate, and your (truthful) answer will be surprisingly common.

Paired tests

So to gain a true picture of what people think, Prelec proposes asking two questions. For example, to find out whether people think Picasso is a great artist, you ask people whether they like Picasso, and also how many people in the population they think are likely to answer yes or no to that question.

Respondents are rewarded if the answer they give to the first question turns out to be unexpectedly common compared with the overall prediction made by the group about how people are likely to answer.

Obviously everyone can't be right all the time. But given the information you have about your own opinion, you are more likely to get the highest score if you answer truthfully.

Prelec and his collaborators have just started testing the idea, by analysing surveys where respondents have no reason to lie. They found that in each case, changing the answers people gave to untruthful ones would have given them a lower score overall.

Coming to a consensus

Prelec says this method is likely to have several advantages over other polling techniques. It should work for very different populations. For example, it is just as effective whether the group in question is poorly informed or well informed.

The technique does not just provide more accurate polls, however. It should also help to find more accurate consensus answers. That could enable companies to make better business forecasts (where managers are encouraged not to worry about whether their opinion differs greatly from those of others), better estimates of risk, better legal judgements, and perhaps even better assessments of art.

For political polling, Prelec says the method would work best when there is not already a lot of public information about what others think. So it would be a good way of gauging public reaction in the immediate aftermath of some event. It could be used to assess, say, whether John Kerry trounced George W. Bush in last night's television debate. 

• References

  1. Prelec D., Science, 306. 462 - 466 (2004). | Article |