They really do not like him, you know. A lot of David Cameron's backbench MPs have no time for their Prime Minister, although most of them either voted for him to be leader or were elected to Parliament under his airbrushed face.

The press previews of Conservative conference are laced with poison. Andrew Gimson, Boris Johnson's biographer, in the New Statesman, quotes anonymous Tory MPs saying that Cameron is "no good with his backbenchers", holding them "in contempt", and, "Secretly I'm one of the people who hanker after Boris. He would make the thing such fun." A pseudonymous diarist in The Spectator retails more stories of Cameron's rudeness, "like a snooty ostrich", especially to his staff.

Boris himself, who will profess undying loyalty today with comic extravagance, professed undying disloyalty last week with some seriousness, saying that Cameron's failure to decide about airports meant "economic catastrophe". While Boris's leadership rivals, Liam Fox and David Davis, candidates defeated by Cameron in 2005, are re-running their campaigns.

Fox's contribution, an article in Friday's Telegraph, was clunky. "Over-borrowed, over-taxed and over-regulated." "Metropolitan elite." "Renegotiate … referendum." Davis gave an interview to The House, in which he also said: "The explicit answer to the European issue … is a big renegotiation and multiple referendums."

Fox and Davis are doomed to fail, but they could do terrible damage to the party in doing so. They will fail because they do not have the qualities – which Boris has – to be prime minister, but they will do damage because there is such resentment among Tory MPs and the dwindling numbers of party members for them to paddle in.

That is the thing about party conferences. They tend to expose what parties really care about. At the Liberal Democrats', it was – perhaps unexpectedly – power. There was remarkable discipline in Brighton because they really believe in coalition government. The only discordant notes came from supporters of Vince Cable, but even they want a change of leadership because it would give them a better chance of hanging on to power.

At the Labour conference, the deep irrationalism has long been over schools policy. I know it was only one delegate who shouted at the 15-year-old from Paddington Academy when she spoke from the rostrum, but "comprehensives offer extra-curricular art classes too" was a heckle notable not just for inaccuracy (Paddington is a comprehensive); it also reflected the mood of most delegates. They are hostile to free schools and academies, which used to be a Labour policy for raising standards.

And this week, the Tories in Birmingham will talk a lot about growth and "delivery", as if prosperity could be Fed-Exed to every household in the country. But what really gets them going is Europe. Renegotiation and referendum are the "comprehensives" of the Tory soul. They want to "change the terms of Britain's relationship" with Europe, and then have a referendum to prove that everyone agrees with them. It is quite mad. And quite likely to lose them the next election, not because the Eurosceptics were wrong about the euro (although they might be in one important respect to which I shall come), but because their rebelliousness has incited wider indiscipline. After the 81 Tory MPs rebelled in a vote for an EU referendum last October, 91 then rebelled against Lords reform. Cameron, now with a wounded chief whip, has come close to losing control of his parliamentary party.

It is possible to see how the Prime Minister might still win through, however. It does not really depend on what happens at conference this week, provided it avoids disaster. The winning scenario was painted jointly for me by a minister and a shadow minister, in separate conversations.

The Conservative minister asked: "What if, in a year to 18 months' time, the economy is growing... but the cuts really start to bite?" That is quite possible, and perhaps the parallel to which this Cameronian moderniser was pointing was Margaret Thatcher in 1983, when unemployment was still high but disposable income rising for the majority. It may seem harsh, but elections can be won even if a minority is suffering.

The Shadow Cabinet member then said what the Tory message would be in 2015: "Labour drove the car into the ditch; do you really want to let them back behind the wheel?" The higher growth is, the more effective this message is likely to be.

A big threat to this scenario would be if the eurozone broke up, plunging Europe into further recession. The sceptics think that is bound to happen soon. David Davis says that "they", the Europeans, "are just trying to find one more finger to stick in one more dyke". But what might have changed over the summer is that the European Central Bank has found a number with enough zeroes to hold the euro together for another two-and-a-half years.

That could be enough to give Cameron victory.

twitter.com/@JohnRentoul; independent.co.uk/johnrentoul