Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Election polling errors across time and space

Nature Human Behaviour volume 2, pages 276–283 (2018)Cite this article

Subjects

An Author Correction to this article was published on 26 June 2020

This article has been updated

Abstract

Are election polling misses becoming more prevalent? Are they more likely in some contexts than others? Here we undertake an over-time and cross-national assessment of prediction errors in pre-election polls. Our analysis draws on more than 30,000 national polls from 351 general elections in 45 countries between 1942 and 2017. We proceed in the following way. First, building on previous studies, we show how errors in national polls evolve in a structured way over the election timeline. Second, we examine errors in polls in the final week of the election campaign to assess performance across election years. Third, we undertake a pooled analysis of polling errors—controlling for a number of institutional and party features—that enables us to test whether poll errors have increased or decreased over time. We find that, contrary to conventional wisdom, the recent performance of polls has not been outside the ordinary. However, the performance of polls does vary across political contexts and in understandable ways.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Fig. 1
Fig. 2: Poll errors across electoral history.
Fig. 3: Poll errors in recent elections (2015–2017).

Change history

  • 26 June 2020

    An amendment to this paper has been published and can be accessed via a link at the top of the paper.

References

  1. Sturgis, P. et al. Report of the Inquiry into the 2015 British General Election Opinion Polls (Market Research Society and British Polling Council, 2016).

  2. An Evaluation of 2016 Election Polls in the US (AAPOR, 2017); http://www.aapor.org/Education-Resources/Reports/An-Evaluation-of-2016-Election-Polls-in-the-U-S.aspx

  3. Silver, N. The world may have a polling problem. FiveThirtyEight http://fivethirtyeight.com/live-blog/uk-general-election-2015/ (2015).

  4. Williams, J. P. The problem with polls. U.S. News and World Report (28 September 2015); https://www.usnews.com/news/the-report/articles/2015/09/28/why-public-opinion-polls-are-increasingly-inaccurate

  5. Zukin, C. What’s the matter With polling? The New York Times (21 June 2015); https://www.nytimes.com/2015/06/21/opinion/sunday/whats-the-matter-with-polling.html?_r=0

  6. Cassino, D. Why pollsters were completely and utterly wrong. Harvard Business Review (2016); https://hbr.org/2016/11/why-pollsters-were-completely-and-utterly-wrong

  7. Santos, R. Why the polls get it wrong. The Los Angeles Times (27 March 2016); http://www.latimes.com/opinion/op-ed/la-oe-0327-santos-polling-problems-20160327-story.html

  8. Barfar, A. & Padmanabhan, B. Election watch. Significance13, 30–33 (2016).

    Article  Google Scholar 

  9. Performance of the Polls in the EU Referendum (British Polling Council, 2016); http://www.britishpollingcouncil.org/performance-of-the-polls-in-the-eu-referendum/

  10. Jennings, W. & Wlezien, C. The timeline of elections: a comparative perspective. Am. J. Political Sci.60, 219–233 (2016).

    Article  Google Scholar 

  11. Erikson, R. S. & Wlezien, C. The Timeline of Presidential Elections: How Campaigns Do (and Do Not) Matter. (Univ. Chicago Press: Chicago, IL, 2012).

    Book  Google Scholar 

  12. Wlezien, C., Jennings, W. & Erikson, R. S. The ‘timeline’ method of studying electoral dynamics. Elect. Stud.48, 45–56 (2017).

    Article  Google Scholar 

  13. Keeter, S., Hatley, N., Kennedy, C. & Lau, A. What Low Response Rates Mean for Telephone Surveys (Pew Research Center, 2017); http://www.pewresearch.org/2017/05/15/what-low-response-rates-mean-for-telephone-surveys/

  14. Mellon, J. & Prosser, C. Missing nonvoters and misweighted samples: explaining the 2015 great British polling miss. Public Opin. Q.81, 661–687 (2017).

    Article  Google Scholar 

  15. Erikson, R. S. & Wlezien, C. Presidential polls as a time series: the case of 1996. Public Opin. Q.63, 163–177 (1999).

    Article  Google Scholar 

  16. An Evaluation of the Methodology of the 2008 Pre-Election Primary Polls (AAPOR, 2009); https://www.aapor.org/AAPOR_Main/media/MainSiteFiles/AAPOR_Rept_FINAL-Rev-4-13-09.pdf

  17. Pickup, M. & Johnston, R. Campaign trial heats as election forecasts: measurement error and bias in 2004 presidential campaign polls. Int. J. Forecast.24, 272–284 (2008).

    Article  Google Scholar 

  18. Ford, R., Jennings, W., Pickup, M. & Wlezien, C. From polls to votes to seats: forecasting the 2015 British general election. Elect. Stud.41, 244–249 (2016).

    Article  Google Scholar 

  19. Kriesi, H. The transformation of cleavage politics: the 1997 Stein Rokkan lecture. Eur. J. Political Res.33, 165–185 (1998).

    Google Scholar 

  20. Mair, P. Ruling The Void: The Hollowing of Western Democracy (Verso, London, 2013).

  21. Franklin, M. The Decline of Class Voting in Britain: Changes in the Basis of Electoral Choice, 1964–1983 (Clarendon Press, Oxford, 1985).

  22. Knutsen, O. Class Voting in Western Europe: A Comparative Longitudinal Study (Lexington Books, Lanham, MD, 2006).

  23. Evans, G. and Tilley, J. The New Politics of Class: The Political Exclusion of the British Working Class (Oxford Univ. Press, Oxford, 2017).

  24. Kayser, M. & Wlezien, C. Performance pressure: patterns of partisanship and the economic vote. Eur. J. Polit. Res.50, 365–394 (2011).

    Article  Google Scholar 

  25. Groves, R. M. Survey Errors and Survey Costs (John Wiley & Sons, Hoboken, NJ, 2004).

  26. Martin, E. A., Michael, W. T. & Kennedy, C. A review and proposal for a new measure of poll accuracy. Public Opin. Q.69, 342–369 (2005).

    Article  Google Scholar 

  27. Arzheimer, K. & Evans, J. A new multinomial accuracy measure for polling bias. Polit. Anal.22, 31–44 (2014).

    Article  Google Scholar 

  28. Laakso, M. & Taagepera, R. The “effective” number of parties: a measure with application to West Europe. Comp. Polit. Stud.12, 3–27 (1979).

    Article  Google Scholar 

  29. Kennedy, C. et al. An Evaluation of 2016 Election Polls in the United States: AAPOR Task Force Report (AAPOR, 2017); http://www.aapor.org/getattachment/Education-Resources/Reports/AAPOR-2016-Election-Polling-Report.pdf.aspx

  30. Our Survey Methodology in Detail (Pew Research Center, 2018); http://www.pewresearch.org/methodology/u-s-survey-research/our-survey-methodology-in-detail/

Download references

Acknowledgements

An earlier version of this article was presented at the Advertising Research Foundation’s ForecastxScience meeting, Google, Sunnyvale, California, 14–15 November 2017. We thank C. Kennedy, D. Rothschild, P. Sturgis and G. Terhanian for comments on earlier versions of this paper, and to Lord Lipsey and other members of the House of Lords Select Committee on Political Polling and Digital Media for their comments and questions. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information

Authors and Affiliations

  1. Department of Politics and International Relations, University of Southampton, Southampton, UK

    Will Jennings

  2. Department of Government, University of Texas at Austin, Austin, Texas, USA

    Christopher Wlezien

Authors
  1. Will Jennings
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christopher Wlezien
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

C.W. and W.J. developed the original study concept. W.J. and C.W. gathered and analysed the data, and drafted and revised the manuscript. W.J. wrote the computer code and generated the figures and tables in Stata.

Corresponding authors

Correspondence to Will Jennings or Christopher Wlezien.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jennings, W., Wlezien, C. Election polling errors across time and space. Nat Hum Behav 2, 276–283 (2018). https://doi.org/10.1038/s41562-018-0315-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-018-0315-6

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing