I don’t buy any of the Cameron leadership challenge stories. Yes, he’s unpopular with his backbenchers, who fail to understand why someone as patently competent as Charles Hendry should be sacked. It’s true too that they have taken to pointedly cheering any sacked minister, so the moment the Speaker called Nick Gibb, Shailesh Vara or Michael Fabricant, the huzzahing was fortissimo and deliberately sostenuto, as if to say, “We are not amused, Mr Cameron”.

You only have to talk to individual Tories for half a minute to know that many are harbouring a latent rage that could break out at any moment. Some think it despicable that George Osborne played such a role in the reshuffle, promoting friends, denigrating enemies. Others focus on policy. Even those who voted against Lords reform wonder how the Prime Minister contrived to lose the boundary changes.

So Andrew Mitchell has his work cut out as the new chief whip as nobody seriously expects another reshuffle before the general election (unless the Lib Dems walk the plank). With a clear Conservative majority looking less likely with every new pratfall, many are building their post-Cameron nest.

But the real problem is the lack of alternatives. Osborne and Hunt are far too tied in to the present regime. Liam Fox struts a lot, but the manner of his departure ill became him. And it feels as if the moment for David Davis has passed.

Which leaves Boris Johnson. The idea that he could waltz back into Parliament and salsa across the road to Downing Street without being hit by a metaphorical No 12 bus is as fanciful as a Barbara Cartland confection. For a start, this week’s Hillsborough report indicated just how cruelly inept the Johnson take-it-or-leave-it style can be. He has precisely the same bully mentality as Cameron and he takes the same short cuts with the truth.

But British politics has changed. When Arthur Balfour lost his Manchester East seat in the Liberal landslide in 1906, he persuaded another MP to vacate his seat and the voters dutifully sent Balfour back to the Commons in the by-election. Today’s voters would react more like the people of Leyton in 1965, who objected to the engineered by-election that was meant to return Patrick Gordon Walker, the Labour foreign secretary who had lost his Smethwick seat at the general election. The safe Labour seat went Tory.

Wanted: temporary home for MPs

You heard it here first, on 18 May, but now it is official. John Sinclair, the third Viscount Thurso and Lib Dem MP for Caithness Sutherland and Easter Ross (much of which his family has owned at some point or other), who is the lead member of the House of Commons Commission, announced on Thursday that the Palace of Westminster may have to up sticks for several years to install proper 21st-century heating and wiring and remove vast quantities of asbestos from its Victorian innards.

Although the old English parliament sat in Lincoln, Oxford, Shrewsbury, Northampton and York over the centuries, it won’t be moving far, simply because the palace relies on many newer ancillary buildings and ministers are in Whitehall. Thanks to ludicrous Franco-German intransigence, the European Parliament wastes millions of euros on a mass caravanserai every few weeks between Brussels and Strasbourg, but we should avoid that trap. So the QEII centre or Church House are still far more likely. Mind you, when the palace burnt down in 1834, King William IV offered us Buckingham Palace. I wonder if the offer is still open.

Lansley needs to lighten up

There are two jobs in politics you have to do with some grace: the Leader of the House and being a Tory Secretary of State for Wales. Both require the ability to butter up the natives. I fear Andrew Lansley has not twigged this. He can do Mr Serious and Mr Emollient, but so far Mr Friendly seems to elude him. So when my Valleys neighbour Ann Clwyd asked about asbestos in Parliament and inadvertently said she met “men in white coats” every day, she laughed, we laughed, but Lansley sat looking as glum as a wet day in Stavanger.

We’re not always fighting each other

The chamber of the Commons loves confrontation and the nature of electoral politics is combative, so it’s only natural that most voters think we spend most of our time at one another’s throats. But most MPs have friends in other parties, or at least respect their opponents.

In 1997, I stood for the then extremely safe Tory seat of Wycombe, where I had been the curate of the parish church in the late 1980s. Sir Ray Whitney, an able former diplomat, who died this summer, was the incumbent, sitting on a majority of 17,000 over the Lib Dems, with Labour a poor third. We knew each other a bit, as (poor soul) he had been subjected to several of my sermons. He was civil throughout the election, though wisely refused to debate with me in public until the eve of the election (by which time any gaffe on his part or coup on mine would be irrelevant).

On election night, Wycombe was hardly the centre of attention as all eyes were on other marginal seats, but my friends got a bit excited when the BBC ticker announced a recount in Wycombe. It was a false alarm. I had come second, by 2,370, but the recount was for the Referendum Party which had lost its deposit.

As the evening wore on, my then partner chatted to Lady Whitney, who asked him why he was there. “I’m Chris’s partner.” Lady Whitney’s face lit up. “I didn’t know he had a business.” “No, his partner for sex.”

Twitter:@ChrisBryantMP