Like most of the other cars on this normally tranquil Sussex road we are heading for the Charleston festival, named after, and taking place in the garden of Charleston farmhouse, once home to Vanessa Bell and now ashrine to the Bloomsbury Group. This is the sixth year of one of the more intimate literary festivals, though judging from the trail of Range Rovers, Volvos, taxis and the odd Roller, it might not be intimate for much longer.

In the marquee, Quentin Bell, 85 years old, is telling Jeremy Isaacs how his mother Vanessa painted her house-door in a snobbish London square, not battleship grey or black like all the others, but "vermilion: there you have the Bells in a nutshell". Charleston was, of course, his childhood home, and Virginia Woolf lived nearby, first in a "horrid little house in Firle" and then at Rodmell. Isaacs keeps the questions bubbling, since Bell, despite his aged air, is wonderfully concise, preceding many answers with a mock-woeful "Oh". What were the Bloomsbury Group like at home, asks Isaacs; did they really talk about ideas , or was it occasionally just "What's for lunch?". "Well," says Bell mischievously, after his customary pause for thought, "that's an idea too..."

And who, of all the eminent persons he has met, was the greatest genius? After casually mentioning Matisse, Bell opts for Picasso, pronounced Pickersoe: "because if he got up from a chair, the cushion would be a work of art". "You must have said that before," taxes Isaacs, jovially. Back comes the quip: "Not very often." Bell's advice to his grandchildren? "Be so amiable that you get whatever you want." His best life experience? After a pause: "Possibly being drunk." His worst? An abrupt change of tone, even though time has obviously worn this lump of grief quite smooth: "Oh, when my brother died." Julian, his mother's favourite, was killed in the Spanish Civil War. "I think he was wrong to go," Bell says sadly. About his mother he is unquenchingly bright-eyed and fond, admitting without rancour that he was not her favourite child.

In the evening the novelist Frances King shares his racy memories of Duncan Grant, E M Forster and the eccentric "Joe" Ackerley. Grant, who lived on alone at Charleston until 1978, was a "fly old boy", according to King, who recalls wanting to buy his study of a male nude. "You'll have to go through my gallery," said Grant peremptorily, then, relenting "because you're my friend", sold it direct, thus avoiding the gallery's mark-up. King remains delighted with his purchase, but found that when he eventually went to Grant's next show, not one of the paintings was as expensive as his.

After King's delightfully shrewish talk, a surprise figure shambles up: E M Forster himself, or rather the actor Benjamin Whitrow, last seen as Mr Bennet in Pride and Prejudice. "A very promising young man," he mumbles of the silvery-haired King. It is shockingly easy to believe, so skilful is this self-penned performance, that Forster is really here, telling gentle anecdotes about his life and friends. Only the fact that he's a bit more forthcoming on his sexuality than Forster himself could ever have been, reminds you that this is simply a cunning impersonation.

One of the great things about Charleston, especially on the next day when the weather turns inclement, is the cake and tea tent, which is why there is almost a riot when the cake shows signs of running out. In between the near-constant tours of people tramping though the fragile farmhouse, the urbane festival director, Alastair Upton, gives us a private tour, interrupted only by his walkie-talkie: "Alastair? Bloody disaster! Over." A retreating car has driven into the ditch alongside the farm track, and the AA is summoned then cancelled after a passing farmer hauls the luckless motorist out with his tractor. "Must remember to thank him," mutters Alastair.

Then, joy of joys, we are invited to a hearty high tea in the cosy farmhouse kitchen; with its shiny pans, colourful crocks and vast Aga, it's just like going to tea with Vanessa and Virginia. Alastair explains that the Charleston festival is tied, for practical reasons, to the Brighton festival, but unfortunately this coincides with the silage season. We get word that yet another car has nose-dived into the ditch. Patricia Hodge and Sam West mooch about, preparing for their evening performance as Vanessa and Julian Bell. They are skittish and don't really eat or make eye-contact, though. West swoops for the quiche - "is this vegetarian?" - which he bears into the house, scattering crumbs, no doubt to the horror of the curator. This is an exhausted-looking gentleman who keeps slamming into the kitchen, bringing in wine-glasses with a martyred air. After the third appearance he suddenly bursts out: "Any moment I'm going to wake up screaming! This is my nightmare of nightmares." "Poor Peter," mutters Alastair. "He really shouldn't be finding wine-glasses in his house."

Finally we weave down the path for the last event of the night, a dramatised reading of the letters of Vanessa and Julian Bell. "VB", she signed herself. "Dear Nessa," he'd reply. A letter from such a mother would, you feel sure, guarantee the possessor self-esteem for life: indeed Sam West plays Julian with a sort of smirking larkiness. In the audience, Quentin Bell seems not to mind being the butt of jokes between his mother and brother, cracking up with delight as the actors repeat old libels about the state of his beard and his pottery. Just as the light begins to fail, and the lamplit tent takes on a magical air, there's a croaking noise at the back of the marquee, a sudden crumple and fluster. Patricia Hodge reads on impeturbably as an elderly lady is carried out and laid gently down in the meadow. Though she is borne off in an ambulance at half-time, by the end of the evening word has returned that she is fine, and this news gets the biggest applause of the night.