The anniversary this week of the UK vote to leave the European Union comes amid the political fallout of another British election surprise two weeks ago. Witness, too, the extraordinary success of Emmanuel Macron in the French presidential elections. What do these unexpected outcomes mean for those of us who strive to understand voting behaviour?

Since the start of 2015, I have spent four nights under the bright lights of ITV News’s overnight election programmes. The first time, when the Conservative Party won a surprise majority in the UK Parliament, I had to set aside my enormous folder of prepared notes. For the Brexit referendum, I had prepared an explanation of why Britain might vote ‘Leave’, but the result still came as a shock. For the US presidential race in November 2016, I was careful to say that Donald Trump might win, but still had the sense of reliving the EU referendum vote, a supposedly predictable election. Earlier this month, when the ITV chairman asked for a prediction for the UK general election, I declined. 

That night, Prime Minister Theresa May saw her majority collapse, losing what had seemed a safe bet. She had called for an early election two months before, amid high favourability ratings. An unpopular manifesto and a lacklustre campaign partly explain her poor showing. However, the British Election Study (BES), which I co-lead, shows a broader factor at play. Volatility in the electorate has been steadily increasing, and so parties can be much less certain of their supporters.

The BES has provided a gold-standard of election data for more than 50 years. By 2015, work by BES research associate Jon Mellon shows, almost 40% of our survey respondents reported voting for a different party from in the previous general election. In 1966, that figure was just over 10%. So although votes were spread across minor parties in 2015, that pattern could easily collapse back to the two major ones. May’s high opinion ratings earlier this year were from people who had previously rejected her party and who might do so again.

Volatile electorates can amplify the consequences of political events. The 2016 referendum injected uncertainty into the 2017 election. The Conservatives — many of whom had urged a ‘Remain’ vote in the referendum — were now pursuing a hard break from the EU. This had the potential to motivate Leave voters and Remain voters differently in different constituencies, and shift them in hard-to-predict ways. 

With the Conservatives pitching to predominantly older, Eurosceptic voters, and the opposition Labour Party to mainly younger, liberal ones, the age gap between the parties became especially large, and turnout more important and consequential. Now we can see that patterns in 2017 were the inverse of those in the EU referendum, with increased participation in Remain-voting areas, and less in Leave areas.

This explains pollsters’ wide range of estimates during the election campaign. Because many citizens, especially younger ones, who had not voted in the 2015 general election (and probably not in the EU referendum) decided to vote in this election, models based on self-reported intentions to vote were more accurate than models based on past turnout, even though both relied on the same kinds of polling data. 

Despite common assumptions, opinion polling has not suddenly become less accurate. Since the 1960s, polls have shown a consistent amount of error from the final election result.

What has changed is the electorate and its political reactions. In addition to increased volatility, we see backlash against established politicians. In both Brexit and this month’s election, voters went against what many in authority advised. But populism is an incomplete explanation. With Brexit, voters with less education cast ballots consistent with wider populist waves. But in the 2017 election, we saw a reaction against the Conservative government among more highly educated voters, too.

For me, the most important task is not to foretell the outcome of an election, but to understand what is driving major political change. I feel we are at a turning point in British politics, most obviously with Brexit, but more profoundly as shifting political loyalties result in dramatic volatility. How do we explain this, and the factors destabilizing the British party system? 

After each election, the BES conducts in-person surveys of around 3,000 people — a representative sample. We probe what drove their choices, including the decision to vote. These days, between elections, we also run online panel studies of 30,000 respondents to track when people change their political attitudes and behaviour. Following the same respondents through a tumultuous time allows us to understand their reactions. This will become increasingly important as we encounter the constitutional, legal and political implications of Brexit and try to understand its impact on the electorate. We want to know how people reconcile their expectations with their experiences and beliefs. How do people think some groups in society prosper at the expense of others? How does discontent with inequity and perceived political and economic unfairness influence attitudes towards the establishment and the stability of the system? 

The clamour for confident predictions is powerful and tempting. The scientific attitude should focus much more on what we don’t know. We should not duck explanations, but instead ask big questions. A strong understanding of why elections are unpredictable is much better than an overconfident prediction.

figure a

Univ. Manchester