Nature | News

Pollsters struggle to explain failures of US presidential forecasts

Most surveys did not predict Donald Trump’s victory over Hillary Clinton.

Article tools

Drew Angerer/Getty

Donald Trump outperformed Hillary Clinton in several states thought to be her strongholds.

What went wrong? That’s the question that many political pollsters in the United States are asking themselves in the aftermath of the 8 November presidential election. Republican candidate Donald Trump won in an electoral landslide, but for months most polls forecast a victory for his Democratic opponent, Hillary Clinton.

Many types of poll, including randomized telephone polls and online polls that people opt into, indicated a tightening of the gap between the two candidates in the weeks leading up to the election — but still pointed to a Clinton win.

“The industry is definitely going to be spending a lot of time doing some soul-searching about what happened and where do we go from here,” says Chris Jackson, head of US public polling at Ipsos, a global market-research and polling firm based in Paris.

The most recent national polls — including those conducted by ABC News/Washington Post, Ipsos, YouGov and Fox News — all estimated a Clinton lead of 3–4% over Trump. Minor-party candidates, such as Gary Johnson of the Libertarian Party and Jill Stein of the Green Party, were forecast to win single-digit support. Yet as this article was published, as the last votes were being counted, Clinton leads the popular vote by a razor-thin margin: just 0.2%. The majority of states have tipped for Trump, awarding him their valuable electoral-college votes and ensuring his victory.

Poll aggregators such as FiveThirtyEight and The New York Times nonetheless forecast Clinton’s chances of victory at 71% or higher, and The Huffington Post predicted a Clinton landslide. This dramatic polling failure could have been due to factors such as poorly assessed likely voters, people misreporting their voting intentions, or pollsters inadequately surveying some segments of the population.

“It’s a big surprise that such a wide variety of polls using such a wide variety of methodologies have all the errors fall in the same direction,” says Claudia Deane, vice-president of research at the Pew Research Center in Washington DC.

State of confusion

The University of Southern California Dornsife/Los Angeles Times presidential election poll, which included an online panel of nearly 3,000 people, was the only major national poll to forecast a Trump lead days before the election. “But we’re not sure we were right either,” says Jill Darling, survey director at the university’s Center for Economic and Social Research in Los Angeles. She notes that Trump did not defeat Clinton by 3%, as her group’s most recent poll predicted.

On a state level, pollsters forecast Clinton’s chances as higher than Trump’s in North Carolina, Pennsylvania and Florida — which the current US president, Barack Obama, had won in 2012 — and yet Trump managed to win them this year. They expected Clinton to easily defeat Trump in Virginia and Michigan, but the races in those states became nail-biters. In Wisconsin, Trump not only pulled off a surprise upset, but did so by a whole percentage point.

Most of the midwestern states, including Wisconsin, Michigan and Pennsylvania, tilted towards Trump, despite predictions to the contrary. Jackson says that polls might not have reached enough rural white people without university educations, who are concentrated in this ‘rust belt’, where manufacturing is on the decline. Exit polls show that this population accounted for a large number of Trump voters.

Busy signal

With each election, pollsters have a harder and harder time reaching people. Now that Americans have fewer landlines and more mobile phones with caller ID, they don’t respond to calls from unfamiliar numbers. Online surveys also struggle to recruit sufficient numbers of participants. To make a statistically significant estimate, for every poll, pollsters want at least 1,000 participants who are representative of the general population with respect to gender, race, education, income level and geographic distribution.

Pollsters strive to assess not just who supports whom, but also who will be likely to vote on Election Day, which is not a national holiday in the United States. This year, 119 million people voted, accounting for 55.6% of registered voters, according to Michael McDonald, a political scientist at the University of Florida in Gainesville. That is the lowest percentage since the 2000 presidential race between George W. Bush and Al Gore.

There were also more undecided voters this year than in previous presidential elections, perhaps because Trump and Clinton were historically unpopular candidates. Undecided voters may show up less in polls, yet may tilt towards one candidate more than others, Darling says. Only 53% of poll respondents felt comfortable disclosing the candidate they would vote for, lower than the 70% in earlier elections, she adds. Furthermore, people overestimate their own likelihood of voting.

“It seems like Trump voters were more enthusiastic about turnout and less enthusiastic about responding to polls. That’s a deadly combination,” says Andrew Gelman, a statistician and political scientist at Columbia University in New York City.

Polling experts in Britain conducted a formal inquiry following polling failures in last year’s general election, when polls underestimated the turnout of older, Conservative voters. Now, in the United States, the American Association for Public Opinion Research has already named an ad-hoc committee to dig into the data and conduct a post-mortem on the election polls. They aim to produce findings by next May, Deane says.

Journal name:
Nature
Volume:
539,
Pages:
339
Date published:
()
DOI:
doi:10.1038/nature.2016.20968

For the best commenting experience, please login or register as a user and agree to our Community Guidelines. You will be re-directed back to this page where you will see comments updating in real-time and have the ability to recommend comments to other users.

Comments

Commenting is currently unavailable.

sign up to Nature briefing

What matters in science — and why — free in your inbox every weekday.

Sign up

Listen

new-pod-red

Nature Podcast

Our award-winning show features highlights from the week's edition of Nature, interviews with the people behind the science, and in-depth commentary and analysis from journalists around the world.