Are election polling misses becoming more prevalent? Are they more likely in some contexts than others? Here we undertake an over-time and cross-national assessment of prediction errors in pre-election polls. Our analysis draws on more than 30,000 national polls from 351 general elections in 45 countries between 1942 and 2017. We proceed in the following way. First, building on previous studies, we show how errors in national polls evolve in a structured way over the election timeline. Second, we examine errors in polls in the final week of the election campaign to assess performance across election years. Third, we undertake a pooled analysis of polling errors—controlling for a number of institutional and party features—that enables us to test whether poll errors have increased or decreased over time. We find that, contrary to conventional wisdom, the recent performance of polls has not been outside the ordinary. However, the performance of polls does vary across political contexts and in understandable ways.
This is a preview of subscription content
Subscribe to Nature+
Get immediate online access to the entire Nature family of 50+ journals
Subscribe to Journal
Get full journal access for 1 year
only $9.92 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Get time limited or full article access on ReadCube.
All prices are NET prices.
Sturgis, P. et al. Report of the Inquiry into the 2015 British General Election Opinion Polls (Market Research Society and British Polling Council, 2016).
An Evaluation of 2016 Election Polls in the US (AAPOR, 2017); http://www.aapor.org/Education-Resources/Reports/An-Evaluation-of-2016-Election-Polls-in-the-U-S.aspx
Silver, N. The world may have a polling problem. FiveThirtyEight http://fivethirtyeight.com/live-blog/uk-general-election-2015/ (2015).
Williams, J. P. The problem with polls. U.S. News and World Report (28 September 2015); https://www.usnews.com/news/the-report/articles/2015/09/28/why-public-opinion-polls-are-increasingly-inaccurate
Zukin, C. What’s the matter With polling? The New York Times (21 June 2015); https://www.nytimes.com/2015/06/21/opinion/sunday/whats-the-matter-with-polling.html?_r=0
Cassino, D. Why pollsters were completely and utterly wrong. Harvard Business Review (2016); https://hbr.org/2016/11/why-pollsters-were-completely-and-utterly-wrong
Santos, R. Why the polls get it wrong. The Los Angeles Times (27 March 2016); http://www.latimes.com/opinion/op-ed/la-oe-0327-santos-polling-problems-20160327-story.html
Barfar, A. & Padmanabhan, B. Election watch. Significance13, 30–33 (2016).
Performance of the Polls in the EU Referendum (British Polling Council, 2016); http://www.britishpollingcouncil.org/performance-of-the-polls-in-the-eu-referendum/
Jennings, W. & Wlezien, C. The timeline of elections: a comparative perspective. Am. J. Political Sci.60, 219–233 (2016).
Erikson, R. S. & Wlezien, C. The Timeline of Presidential Elections: How Campaigns Do (and Do Not) Matter. (Univ. Chicago Press: Chicago, IL, 2012).
Wlezien, C., Jennings, W. & Erikson, R. S. The ‘timeline’ method of studying electoral dynamics. Elect. Stud.48, 45–56 (2017).
Keeter, S., Hatley, N., Kennedy, C. & Lau, A. What Low Response Rates Mean for Telephone Surveys (Pew Research Center, 2017); http://www.pewresearch.org/2017/05/15/what-low-response-rates-mean-for-telephone-surveys/
Mellon, J. & Prosser, C. Missing nonvoters and misweighted samples: explaining the 2015 great British polling miss. Public Opin. Q.81, 661–687 (2017).
Erikson, R. S. & Wlezien, C. Presidential polls as a time series: the case of 1996. Public Opin. Q.63, 163–177 (1999).
An Evaluation of the Methodology of the 2008 Pre-Election Primary Polls (AAPOR, 2009); https://www.aapor.org/AAPOR_Main/media/MainSiteFiles/AAPOR_Rept_FINAL-Rev-4-13-09.pdf
Pickup, M. & Johnston, R. Campaign trial heats as election forecasts: measurement error and bias in 2004 presidential campaign polls. Int. J. Forecast.24, 272–284 (2008).
Ford, R., Jennings, W., Pickup, M. & Wlezien, C. From polls to votes to seats: forecasting the 2015 British general election. Elect. Stud.41, 244–249 (2016).
Kriesi, H. The transformation of cleavage politics: the 1997 Stein Rokkan lecture. Eur. J. Political Res.33, 165–185 (1998).
Mair, P. Ruling The Void: The Hollowing of Western Democracy (Verso, London, 2013).
Franklin, M. The Decline of Class Voting in Britain: Changes in the Basis of Electoral Choice, 1964–1983 (Clarendon Press, Oxford, 1985).
Knutsen, O. Class Voting in Western Europe: A Comparative Longitudinal Study (Lexington Books, Lanham, MD, 2006).
Evans, G. and Tilley, J. The New Politics of Class: The Political Exclusion of the British Working Class (Oxford Univ. Press, Oxford, 2017).
Kayser, M. & Wlezien, C. Performance pressure: patterns of partisanship and the economic vote. Eur. J. Polit. Res.50, 365–394 (2011).
Groves, R. M. Survey Errors and Survey Costs (John Wiley & Sons, Hoboken, NJ, 2004).
Martin, E. A., Michael, W. T. & Kennedy, C. A review and proposal for a new measure of poll accuracy. Public Opin. Q.69, 342–369 (2005).
Arzheimer, K. & Evans, J. A new multinomial accuracy measure for polling bias. Polit. Anal.22, 31–44 (2014).
Laakso, M. & Taagepera, R. The “effective” number of parties: a measure with application to West Europe. Comp. Polit. Stud.12, 3–27 (1979).
Kennedy, C. et al. An Evaluation of 2016 Election Polls in the United States: AAPOR Task Force Report (AAPOR, 2017); http://www.aapor.org/getattachment/Education-Resources/Reports/AAPOR-2016-Election-Polling-Report.pdf.aspx
Our Survey Methodology in Detail (Pew Research Center, 2018); http://www.pewresearch.org/methodology/u-s-survey-research/our-survey-methodology-in-detail/
An earlier version of this article was presented at the Advertising Research Foundation’s ForecastxScience meeting, Google, Sunnyvale, California, 14–15 November 2017. We thank C. Kennedy, D. Rothschild, P. Sturgis and G. Terhanian for comments on earlier versions of this paper, and to Lord Lipsey and other members of the House of Lords Select Committee on Political Polling and Digital Media for their comments and questions. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.
The authors declare no competing interests.
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Jennings, W., Wlezien, C. Election polling errors across time and space. Nat Hum Behav 2, 276–283 (2018). https://doi.org/10.1038/s41562-018-0315-6
The role of slant and message consistency in political advertising effectiveness: evidence from the 2016 presidential election
Quantitative Marketing and Economics (2022)
Public Choice (2021)
Nature Human Behaviour (2018)