Sex, tipsy Tories & Elizabeth David: Fifty years of the West End's infamous nightclub Annabel's

London’s most infamous A-list haunt, Annabel’s, has played host to everyone from Sinatra and John Wayne to the Queen and the Shah of Iran. This month it celebrates 50 years of glamorous excess. John Walsh delves into the archives – and discovers an intriguing link with Elizabeth David

Jerry Hall and Francesca von Thyssen at Annabel's in 1987 (Rex Features)

You could almost walk past Annabel's without noticing it. Nothing about the handsome Georgian building at No 44 Berkeley Square suggests that it might have been a byword for decadent, zillionaire-superstar hedonism for 50 years. The only hint of exclusivity is the small, orange-striped sentry box, just inside the railings, through which visitors descend to the basement club. Nothing marks the stone staircase down which, for half a century, the extremely posh and the obscenely rich, the stratospherically starry and the suavely well-connected have passed, for 300 evenings a year, in search of discreet, expensive fun.

Everybody in the public eye or the corridors of power seems to have dropped into Annabel's in the past five decades. It's the only nightclub to be visited by HM the Queen (she had a dry martini – gin, no lemon). Staff remember the evening when the Shah of Iran and two other Middle Eastern potentates nodded at each other by the bar. Film stars, rock stars, heads of state and upper-crust crooks rubbed shoulders beside the dining room's Satanic-red walls. The Duke of York threw his stag party there, a party that was raided by a brace of policewomen, who turned out to be Princess Diana and Sarah Ferguson. The day Margaret Thatcher made it to Downing Street in 1979, you couldn't move in Annabel's for Tory MPs and business grandees ordering vintage Champagne.

John Wayne came in one evening. In a club where real dukes are two a penny, the cinematic Duke arrived and headed for the bar. Already blearily intoxicated, he ordered whisky and crushed three cigars in his mighty hands while attempting to light them. He reportedly twiddled the right nipple of a lady at the bar as though tuning a radio, before heading for the Gents. The duty manager, Eddie Wetton, went after him to make sure all was well and found the great cowboy using a urinal. Wayne cocked an eye in his direction. "Now yuh can tell all your buddies," he said, "yuh hung out with John Wayne."

Some club members would probably reverse that story and imagine Mr Wayne returning to Los Angeles and telling all his buddies that he'd hung out in Annabel's. It was a transatlantic byword for classy company, high-grade larks and rumours of jiggery-pokery.

Frank Sinatra was one of the club's earliest members. When he was in London, Annabel's was his club. He'd telephone to ask if there was somewhere to change his clothes because he planned to come straight from a concert at the Albert Hall. Indeed there was. Wetton remembers helping the great crooner with his cufflinks, and receiving a tip of £100. Louis the maître d' recalls Sinatra asking him, "What do you think of me?" and getting the ambiguous reply, "Mr Sinatra, I think you are the greatest Italian since Mussolini".

Many such stories can be heard in Annabel's: A String of Naked Lightbulbs, an enthralling, one-hour documentary executive-produced by Ridley Scott and directed by Greg Fay. Among the talking heads are Anna Wintour, editor of US Vogue, Bryan Ferry, Naomi Campbell, Nicky Haslam, the PR man Lord Bell, Goldie Hawn, Sir David Tang, and Richard Caring, the entrepreneur who bought the club in 2006. The only obvious omission is any footage of Mark Birley himself. There are few photographs of Birley. The film focuses lingeringly on one image of him in shadow, his handsome face troubled and lined, a private, rather unknowable man who spent his life devoted to accumulating fine things and making people "comfortable".

Marcus Lecky Oswald Hornby Birley was the son of Sir Oswald Birley, a society painter. The young Mark went to Eton, where he made lifelong friends (many of whom became club members) and studied PPE at Oxford, but was sent down after failing his first-year exams. He worked in advertising at J Walter Thompson, then started a shop selling Hermès ties and scarves in London's Jermyn Street.

Inspiration hit him when his friend, John Aspinall, took advantage of Britain's relaxed gambling laws by opening a casino called The Clermont Club at 44 Berkeley Square. Mark told friends it "might be quite fun to have a bar underneath" and proceeded to dig out the back garden at No 44, to create the club's sumptuous dining room. He named it after his wife of nine years, Lady Annabel Vane-Tempest-Stewart, daughter of the Marquess of Queensbury (who later divorced Birley and took up with buccaneer capitalist James Goldsmith).

When he opened Annabel's in 1963, Birley decreed it must "smell of exclusivity and sex". He was, however, sufficiently old-fashioned to insist on a strict dress code. Eric Clapton was once turned away for not wearing a tie, Mick Jagger was obliged to borrow one from the doorman, and The Beatles were refused entry because, though they were wearing ties, they weren't wearing shoes. The code still exists: "Ladies are requested to opt for smart eveningwear, which may include tailored trousers," reads the rubric, "and gentlemen to don a suit of a jacket and tailored trousers, with a collared shirt. As we have only a limited number of jackets to lend, Members should brief their guests accordingly before arrival at the Club." In translation, this means: no jeans, no T-shirts and for God's sake tell your friends to scrub up properly.

The nightclub's discreet exterior at 44 Berkeley Square (Rex Features)

Birley's perfectionism, his attention to detail, was legendary. He would shudder if a soup spoon was laid at too oblique an angle. He spent weeks tracking down a shade of porcelain, or precisely the right davenport sofa. He delayed the opening of Annabel's by three months because he was waiting for the right chandelier to arrive. He is credited with inventing the little muslin shroud that's wrapped around halves of lemon so that, when you squeeze it over your smoked salmon, the juice doesn't go in your dinner companion's eye.

When considering what food should be served to his rich, and often discerning members, and their consorts, he didn't simply pick classic dishes from the Ritz and the Savoy, nor did he leave it to the taste of Donald Smith, his first head chef. He went to the top, and asked Elizabeth David, then the doyenne of Anglo-French food, what she thought of his menu.

The result was extraordinary. Far from congratulating Birley on his choice of dishes, Ms David replied with the 1,800-word equivalent of a rap on the knuckles. She ticked him off for using the phrases 'vintage sardines', 'crepes de crab au currie' ("Whoever thought it up? It sounds like something from a publicity handout") and 'truite au bleu' ("a nonsense, wherever you put it"). She poked fun at his ignorance of languages: "Escalope de veau Piedmontaise... There is no such word as 'Piedmontaise'. English is 'Piedmontese'. French is 'Piemontaise'. Italian is 'Piemontese'. Let's eliminate head-waiters' Franglo-Italian. It doesn't inspire confidence."

She raised an important point about the probable post-midnight requirements of the club's raffish members, too: "Reading through the menu sober and with a reasonably good, although admittedly middle-aged, appetite, I begin to feel horribly bloated long before I've reached the Caneton d'Aylesbury aux pêches, let alone the out-of-season raspberries and the fraises Romanoff. All I feel I could manage is a jellied egg and a piece of bread and butter. And I wonder what impression the menu makes on customers who come in half-sloshed at 1.30am? Should there be perhaps a different menu for after midnight? A selection of English supper and breakfast dishes? Eggs, bacon, kedgeree, liver and bacon, scrambled eggs, sausages and the like? The chef and his staff surely don't produce poussin Belle Époque and crepes Véfour from 8pm to 3am? ... A restaurant's menu is, after all, the equivalent of a window display. At the moment, Annabel's window dressing is more Mappin & Webb than Hermès."

The club's late owner Mark Birley with his wife Annabel (Rex Features)

History does not recall Birley's response, but it's likely he was impressed to encounter a kindred spirit, when it came to fastidious taste. He arranged for Ms David to dine at the club and let him know her verdict. According to Annabel's website, Birley waited by the phone the next morning and, when no call came, rang her up. What did she think of the food? "The potatoes were stale," was her reply.

The club wasn't an instant hit. It took 10 years to become profitable. But for the moneyed clientele and visiting stars, it was like Paradise. "Everything came together in a rather magical way," says Anna Wintour. "It was over a decade since the war and there was a sense of rebirth and renewal in England. The Sixties was an extraordinary time to be alive in London. And at the centre was Annabel's, standing for the old world but also representing the new."

"You have to remember it wasn't long after the war," says Nicky Haslam. "I think people had thought there would never be anything nice again – no luxury, no building wonderful rooms. And it was a time when designers and photographers were becoming very fashionable." "It was hugely spoiling, to be taken to Annabel's," remembers Nina Campbell, the designer. "You put on your prettiest dress." By her computation, "hundreds and hundreds of marriage proposals" were uttered over dinner (including her own). By its 10th anniversary in 1973, Annabel's had a waiting list approaching 2,000. The crash of Lloyd's insurance was a blow, as were IRA threats of imminent explosion, but the club survived to enjoy the wealth explosion of Thatcherism.

Guests party on the club's colourful dancefloor

Two fixtures at Annabel's were Nigel Dempster, the gossip columnist and the only journalist allowed on the premises, and Dai ('Dirty Dai') Llewellyn. On one of my very few visits, I remember watching the Welsh lothario – brother of Princess Margaret's beau Roddy – stroking the cheek of a young debutante at the bar with the words, "Why, my dear – you're as pretty as a primrose," a line he had possibly used once or twice before in similar circumstances.

As his commercial fortune soared, Mark Birley's private life was haunted by tragedy. He wasn't good with children. "We all had our work cut out for us with Pup," his daughter India Jane told Vanity Fair in 2008. "He could scowl and smile at the same time. For a child it's a very, very odd feeling. With an adult you can figure out the subtlety, but for a child it's terrifying." She once had to relinquish her small bedroom so that her father could use it to store his collection of boots and shoes. In her autobiography, Lady Annabel recalled her husband waking her up in the hospital where she'd given birth to their second child, Robin, and saying: "Darling, you must wake up. There must have been a mistake. I think you've been given the wrong baby – this one is simply hideous".

Kate Moss and Fat Tony at Annabel's 50th birthday party (Rex Features)

By 2000, Birley's empire had grown to include further clubs, bars and a 'brasserie' for thirtysomethings. But his physical condition had worsened. He had several bad falls, suffered from a broken hip, and needed an operation on his knee that left him unable to walk. In 2003, he turned over the running of the business to his children, Robin and India Jane. There followed an intense period of family feuding when Robin took exception to his sister's suitor, Robert Macdonald, a voice coach, and spent a rumoured £300,000 on private detectives to reveal him as a fraudster. Birley was furious, and dismissed his son from control of the company.

When Birley died in 2006, his will left the bulk of his £200m fortune to India Jane's son, Eben (now eight) and bequeathed a paltry £6m to Robin. Shortly before he died, Mark sold all his clubs for £120m to Richard Caring, the owner of J Sheekey, the Ivy and Le Caprice, who made his fortune in the Hong Kong rag trade.

While the image of British clubs has changed immeasurably over the years, Annabel's endures as a playground for the rich and starry, a plush rumpus-room where princes, oligarchs, marquesses and Hollywood royalty marvel at being under the same roof, bask in their privacy and see their melting loveliness reflected back at them in the beautiful faces ranged around the bar. "The reason why Annabel's is so successful and will be for a very long time," says Ben Elliott, co-founder of concierge service Quintessentially, "is that it's not trying to be something it's not. If it's trying to be modern, then it's not Annabel's".

'Annabel's: A String of Naked Lightbulbs' is on limited release at the Curzon Mayfair until 25 November