The idea of a 76-year-old trumpet player having his finger on the comedy buzzer - even being the object of a small riot from young fans - seems a little far fetched. But that's Humphrey Lyttelton, chairman of one of radio's most popular anti-quiz shows. As he tells James Rampton, it's close enough to jazz.

The hippest person in comedy is a man old enough to be my grandfather. The predominantly young audience packing out the Almeida Theatre in north London last Sunday to watch Humphrey Lyttelton preside over a recording of I'm Sorry I Haven't a Clue, Radio 4's "antidote to panel-games," doubtless agree. He came on to the sort of whoopin' and hollerin' usually reserved for lachrymose American televangelists.

Lyttelton may have been in possession of a bus-pass for more than a decade, but that hasn't stopped him continuing to be just about the funniest thing on radio or television. For 25 years now, his deliberately bored tone as chairman of the show has been causing drivers listening to Radio 4 to pull over to avoid endangering other vehicles through excessive laughter.

I'm Sorry I Haven't a Clue has a huge cult following and is the sort of show which can fuel entire dinner-party conversations. Perhaps only The Archers and Letter from America rival it as a radio institution. The heads of both BBC radio and television comedy were spotted in the audience on Sunday. And which other show could persuade Dame Judi Dench to sacrifice a precious night off for the sake of a quick laugh at her expense? Eddie Izzard, the hottest act on the live circuit, is apparently also keen to appear.

Although obviously partial, the show's producer Jon Naismith sings Lyttelton's praises. "He gets laughs where they're not expected, and he's a brilliant ad libber - not just an absurd authority figure flung into this arena. He revels in the gentle anarchy of the show. You can hear the delight in his voice when the buzzers aren't working. He'll conjure magic out of it. He'll even milk laughs from a spelling-mistake. And his sense of timing is second to none. He's a national treasure."

Relaxing before the recording in a dressing-room at the Almeida in a rather loud red sweat-shirt, Lyttelton, an Old Etonian and former Guardsman, is a masterpiece of very British, self-deprecating charm. Catching sight of a clothes-rail of white coats in the corner as we enter, he says, quick as a flash: "they've already supplied the white coats to take us away in". For all that, he protests, "I'm not a comic. I'm a trumpet-player who's found himself in an extraordinary situation from which it is very difficult to extricate himself, hard as he tries."

Angus Deayton, Clive Anderson and Nick Hancock can teach him nothing about being a sardonic "anti-host". Lyttelton was doing it while they were still in satirical (and literal) short trousers. He patented the art of seeming uninterested in the proceedings he is supposed to be running. "Did somebody win?", he'll ask wearily at the end of the impossibly arcane Mornington Cresent game. And he'll introduce a music round with the words: "something guaranteed to have all music-lovers glued to the off-switch".

Again, Lyttelton plays down his role, preferring to deflect the glory onto the three regular panellists Barry Cryer, Graeme Garden and Tim Brooke- Taylor. (This is, after all, a man so modest that he reportedly turned down the offer of a knighthood.) "Driving to record the pilot for the show in 1972, I thought to myself, `what the hell am I doing? I've got a perfectly good career already. I'll just sound ridiculous trying to match the comics.' I've tried to have that in the back of my mind ever since. `What am I doing here?' It gives me an image of being detached. The biggest compliment I got was when a reviewer called me a `comatose presenter'. I thought, `I'm getting it across'."

But just what is it about this show that sends its fans so crazy? There was a near-riot when 30 people couldn't get into a recording in genteel Cheltenham, of all places, and the police were called to quell unrest among Clue-ites barred entry to an already full Paris Studios in London. Naismith observes that "there's an element of the Rocky Horror Show about it".

"The whole thing's a mystery to me - as it has been for twenty-five years," Lyttelton shrugs. "It may be there's a certain irreverence about it. It's also the sort of show that depends on the audience being in on it. Now we get a huge cheer every time I announce Mornington Cresent." Lyttelton has found the popularity of the show has spilt into his "proper" job as a jazz trumpeter. "I hesitate to use the word `cult', but there has been a spin-off as far as my music gigs are concerned. Noticeably more young people are now coming to them. They come up to me and make jokes about Samantha [his fictitious assistant] and Mrs Trellis from north Wales [a frequent, invented correspondent on I'm Sorry I Haven't a Clue]."

He sees some cross-pollination between his work in comedy and jazz. "I've learnt a lot about timing from doing this show," he reckons, "but the closest connection is that there's an ad lib element in both. There's something about both hats I wear which calls for an attitude of `what the hell, it's not the end of the world'. In jazz, if the audience gets the idea that you're putting formality before spirit, they get very suspicious. So I already had that attitude."

At the age of 76, Lyttelton still bubbles with enthusiasm for his work. "What a way to earn your living," he sighs with contentment, "to listen to four of the wittiest people in the land for two hours."

He cannot foresee a day when he'll want to give it up. "There may be a time when my ad lib responses come at four o'clock the following day," he muses. "But for now my age gives Barry the opportunity to say things like `nurse, the screens,' and `we're going for a walk later'. That's part of the charm of the show."

It is indeed. It is to the eternal shame of certain hotshot commissioning editors that they rejected the idea of a TV version of I'm Sorry I Haven't a Clue because, in Lyttelton's words, "they thought we were a bit old".

`I'm Sorry I Haven't a Clue' is broadcast on Radio 4 at 12.25pm on Saturdays and repeated at 6.30pm on Mondays.