Looking back over 2017, I am trying to learn from my failure to see the Corbyn surge coming

I may have been in good company in getting this year’s election wrong, but it is important to learn the lessons for next time

Corbynmania swung the election in a way that I could not have predicted (Getty)

It is that time of the year to look back and learn. If you learn from your mistakes, my biggest mistake of 2017 was the failure to grasp the scale of the Corbyn surge in the election. That is why I have this headline pinned to my computer screen: “Why I don’t believe the YouGov model predicting a hung parliament.”

It is an article I wrote nine days before polling day, and I think about it a lot. It makes me feel better, of course, that YouGov didn’t believe its model either, and staked its reputation on its final conventional opinion poll, which put the Conservatives seven points ahead. When the votes were actually counted, the Tories were ahead in vote share by 2.4 points, and lost their majority in the House of Commons.

Survation was closest to the result, with a one-point lead, but the average of the final polls by members of the British Polling Council put the Tories eight points ahead.

It makes me feel better, too, that Jeremy Corbyn and his closest advisers, with the exception of Karie Murphy, his office director, expected Theresa May to increase her majority. This is some consolation, but I would rather have seen the hung parliament coming.

So what are the lessons to be learned? The first, I think, is the danger of fighting the last war. The 2015 election had been a surprise in the other direction. The Conservatives did better than expected, mainly at the expense of the Liberal Democrats.

That time, too, I was left kicking myself. I had even spent time on David Cameron’s campaign bus, asking why we were visiting safe Lib Dem seats in the Pennines.

I assumed that, if the opinion polls had got it wrong, they were likely to have underestimated Tory support, as they have tended to do since the war. As I wrote in that article pinned to my screen, “For the YouGov model to be right, it would mean that something has happened in the past two years that makes the national opinion polls more wrong in a pro-Tory direction than they have ever been.”

Now we know that several things happened to do precisely that. One was that most pollsters over-corrected for their 2015 error, but the other things were to do with the nature of this particular election.

Lesson two is the tendency to assume that elections are static, rather than fluid. Usually opinion polls don’t change much during an election campaign, although this may conceal a lot of churn that cancels itself out. We knew the 2017 election was different, as Labour surged 15 points over the seven weeks, while the Tories ended up, after an early boost, more or less where they started.

But the speed of change left me behind. A lot of commentators, and this was true of me, got out of London and spoke to voters early on in the campaign, but we didn’t update our reporting in the final week.

Lesson three is related: elections are driven by emotion. I understood that in 2015, possibly because I shared the feeling that an Ed Miliband government propped up by the Scottish National Party was bad idea.

This time the SNP were more or less forgotten, mainly because no one (apart from Karie Murphy, who bought a new dress so she could go with Corbyn to Buckingham Palace when he was invited to form a government) thought a hung parliament was likely.

The enthusiasm for Corbyn’s policies was not tempered, therefore, by questions of how they would work in practice. People liked free tuition and nationalisation and ending austerity, and felt liberated to vote for them without worrying about the stability of a Corbyn government propped up by the Scots Nats.

Elections also work on a simpler emotional level. Theresa May was arrogant in asking for a larger majority when she didn’t need to, just to make herself more powerful. The response of the people through the ages to any politician doing that is: no thanks. Ted Heath demanded a mandate in a similar tone of voice in February 1974, asking for the authority to defeat the miners, and lost his majority too.

Finally, May was a terrible campaigner, with terrible policies. We journalists didn’t fully appreciate it because we were already familiar with her and had discounted her woodenness. She is good in short bursts, good at looking prime ministerial and at sounding stern.

But in a campaign you need a personality. And you need a manifesto designed to win votes, not to get legislation through the House of Lords.

One of the striking revelations of the two excellent accounts of the 2017 election, by Tim Shipman (Fall Out) and Tim Ross and Tom McTague (Betting the House), is that May was worried about getting her policies through Parliament.

She wanted Nick Timothy, her chief of staff, and Ben Gummer, the Cabinet Office minister, to produce a manifesto that would give her the mandate to do that, rather than one that would win an election. That detail had already been assumed.

Thus the Tories ended up with a defensive, no-change manifesto with a plan to tax modest home owners if they were unlucky enough to suffer dementia in old age. I quaked at the foolishness of it at the time, and yet the Tories were so far ahead that I simply assumed that they couldn’t throw away the whole of their advantage.

Looking back, it should not have been hard to work out that turnout might be higher than before among the young, and lower than before among the old.

Memo to self; I probably have four and half years to learn this and inwardly digest it. The next election won’t be like the last; it could be fluid until the last moment; and the key is to work out the big, simple, emotional story that is unique to that election..