It was a particularly bad week for the Prime Minister. On Monday, he had to linger in front of a post-Olympic audience that had attended politely to his own remarks and respectfully to a bracing address from the Princess Royal, and was now going bananas over the Mayor of London's stand-up routine.

On Wednesday, he discovered that, according to the opinion polls, Boris Johnson is worth a good 1.5 million more votes to the Tory party in a general election than he is himself. Sandwiched between these embarrassments came the unveiling of "Conservative Voice", a new and potentially disputatious right-wing ginger group that intends to say "whatever we want" to help secure victory in 2015.

Naturally, these two problems – the Mayor of London and the Tory right – are intimately connected, and yet one of them has a longer fuse than the other. After all, Boris's possible candidature for the Conservative leadership only becomes a difficulty if David Cameron founders at the polls. But Conservative Voice, for all the bromides about Downing Street and Central Office backing it to the hilt, means trouble now, crammed to the gills as it is with tough-minded self-made men espousing the usual old eyewash about small government, low taxes, fuel duty and "broad" rather than "deep" relations with Europe.

Somebody ought to write a history of post-war ginger groups, for its findings would be horribly revealing about the way political parties sink and swim. As a general rule, the more factions that a party accumulates, the less likely it is to hold on to power.

Back in the 1980s, for example, the Labour leadership could barely contrive a policy on office stationery without wondering how it might play with the Trots of the Campaign Group, the Tribunite soft left and the band of nervous right-wingers gathered in the Manifesto Group's beleaguered stockade. A party leader who is sure of him or herself can ignore these intrusions. Thus when Francis Pym and one or two pained Tory grandees devised a faction called "Centre Forward" in the early 1980s, there was general hilarity: Lord Pym, as he very soon became, had cooked his goose, and everyone knew it.

Going back to Liam Fox and David Davis's darling project, as with every other ginger group since the Labour "Keep Left" movement of the 1950s, its stated aims, if not disingenuous, are pretty superfluous. I was reminded of the time when Auberon Waugh used to write a weekly column for The Spectator entitled "Another Voice". This led one correspondent to complain that it was actually the same voice and, moreover, that it was saying the same thing.

Barring the death of Derek Jameson, it was a good week for cockney geezers, and in particular their principal modern ornament, Ray Winstone. "Raymondo" seems to be everywhere these days – starring in a remake of The Sweeney for the cinema, profiled in the press, featuring in Betfair's advertising campaign for Tuesday's World Cup qualifier. In interviews, Winstone always comes across as an engaging character, keen on the values of hearth and home and critical of the "villains" he portrays, and yet you wonder if some of the ambiguities of the stereotype he has helped to construct ever trouble his conscience.

No myth, it might be argued, is quite so pernicious as that of the lovable cockernee – a bit flash with his fists, maybe, but with heart indisputably in the right place – and the images of carefree communality it conjures up: from pearly royalty capering before their whelk stalls to gargoyle-faced old gentlemen wondering if you can Adam and Eve it me old China. In a recent sit-down with The Big Issue, Winstone seemed anxious to justify the use of this rough-hewn cosiness to persuade the punters to squander their money at the bookmaker's. He did betting ads, he explained, because, unlike financial services, it was an area in which you could exercise a degree of choice. On reflection, this seems quite as spurious a piece of logic as that antediluvian line about the Kray twins only killing their own.

The novelist Joan Brady could be found in several newspapers complaining about the decision of South Hams district council to allow the Costa Coffee chain to open a branch in Totnes, Devon. Piquancy was added to this stance by the reminder that, in 1993, Brady was a winner of the Whitbread Book of the Year award – a prize that Costa now sponsors.

Her argument was that the town's "rather singular" character should be protected from outside infestation, and that the council's vote takes no account of local feeling. Three-quarters of the town's population are said to support her. Much as one admires this defence of local democracy, there is a suspicion that the issues are not entirely clear-cut. A very similar row has just concluded in Southwold, Suffolk, where the district council gamely capitulated for fear of the legal costs of a subsequent appeal.

Once again, protesters dwelt on the town's "unique" character and its distrust of the identikit. Southwold habitués, on the other hand, would probably argue that the real Southwold disappeared all of 20 years ago when the first upmarket deli was allowed in. Then there is the regrettable, but unignorable fact that Costa Coffee knocks the infusions of most Olde Tea Shoppes into a cocked hat. The ruin of our national heritage works in mysterious ways.