Talk of the town: 'I stand there pushing my spade into the soil, pretending it's Boris's head

"I hate not being in power," Ken Livingstone said to me. "I'm a workaholic control freak and I hate not running London. Boris is going to mess the whole thing up, and until I stand again in four years' time, there's not a lot I can do about it."

We were standing in the Writers Room in Cheltenham town hall at last week's festival, waiting to go on stage and talk in front of 800 people. Ken is 10,000 words into his autobiography ("It's pretty boring so far," he says) and is starting to "do" the festival circuit. He makes most of his living now from after-dinner speaking or overseas consultancy, and while he enjoys the talks, he hates the travelling. "If you're young, then travelling's fun. But not at my age. I don't know anyone who likes spending time in airports."

He does a bit of broadcasting, fires off the occasional missive, and still takes a swipe at Lady Thatcher and the Daily Mail if you let him get a word in. But he's engaging company – funny, feisty and fundamentally entertaining. Having admitted he'd turned down an appearance on Celebrity Wife Swap, I asked if he'd fancy editing a national newspaper? To which Ken responded by bemoaning the fact that all papers these days are full of little but comment. "I mean who cares what [Guardian columnist] Catherine Bennett thinks about anything?" he asked.

He says that he often has great thoughts while gardening, and someone in the audience asked what these might be. "Well, I stand there, pushing my spade into the soil, pretending it's Boris's head. It gives me great satisfaction."

These days there are so many literary festivals it's possible for every wordsmith with a big mouth and a printed indiscretion to tour the country on a never-ending binge of honeyed cocktail sausages and screw-top sauvignon blanc. It's a punters' market, and although your day could be spoilt by finding yourself in a room with Harry Hill or Ann Widdecombe, you could also improve it by stumbling across Ken.

Now that Peter Mandelson, Alastair Campbell and Uncle Tom Cobley and all have been welcomed back into Gordon Brown's kitchen, would he ever accept a cabinet position? As 800 people heard, the answer wasn't no.

Dylan Jones is the editor of 'GQ'