Gossip columnists: Masters of the dark arts

The five gay-rights activists had broken past front desk security and reached the inner sanctum of the Daily Mail newsroom, where they chained themselves to reporters' desks and chanted "Daily Mail? HATE MAIL!" at bewildered correspondents. Under the glares of the paper's editorial overlords, guards struggled to cut them free and throw them out.

Two of the demonstrators suddenly unlocked themselves and spread the message of love, dancing around the floor before finally being cornered in the atrium - where, from nowhere, a sharp-suited fellow smelling of claret hurled himself through the air, wrestling one intruder to the ground. Security removed the two protesters for their own safety. Such was office life with Nigel Dempster.

The disease that Dempster's nervous system developed, and his subsequent retirement four years ago, robbed British public life of a man who took gossip gathering seriously and persuaded his editors, readers, rivals and successors to do the same. And his death on Thursday casts a rare beam of light on the gentlefolk of Her Majesty's Diary Corps, who are rarely the retiring sort known for their reticence at the free bar.

Some of them hide behind pseudonyms (Pandora of this parish, Hickey in The Express, the Mail's Ephraim Hardcastle). Others' gurning photo bylines put readers off their breakfast. Whichever: all those who are successful share a modicum of social acceptability, pride in exclusive story-getting, a fat expense account and slush fund for tips, a weightier contacts book and a concerted viciousness (Dempster once beamed that "there is a holiday in my heart when I discover another marriage breaking up"). Those diarists tugged by the nagging spectre of morality try to dump on targets that deserve it.

I was improbably and unceremoniously appointed The Independent's diary editor 12 months and two days ago. My predecessor treated me to "diary writing for idiots" over a three-course lunch and Bloody Marys at the local gastropub, got me to write a couple of practice articles, then passed me the (unwashed) pink socks to fill. Unusually for a diarist, he escaped the grasping clutches of the men in white tunics and emerged with a twinkle in his eye, if carrying a slightly unhealthy pallor. I was on my own. A page to fill. Every day.

"What are you worrying about!" exclaimed Giles Coren, once gossip columnist for The Times, when I told him of the predicament. "Just make it up! The more outrageous the lie the better. My own personal favourite was 'spotting' Jack Nicholson in Kentish Town Kentucky Fried Chicken, eating a bargain bucket, with sauce all over his chin."

Your friends love it. Gossip writers get "plus-ones" to movie premieres, nightclub openings, theatre first nights, book launches, beauty contests, where they can hold your jacket and shriek excitedly as the star of their favourite teatime soap brushes past or Bruce Forsyth chokes on a canapé, while you slyly manoeuvre for a better vantage point to watch the politician's wife leaning in for an intimate conversation with that telly presenter in the corner. The job is a lot more than swanning around parties and jollies to Cannes, Edinburgh and Vegas, however. The groundwork is done by night, but at 9.30am you have to be sitting at your desk phone with a list of possible leads to check out and thrash out (finding time to lunch a contact) before the 6pm deadline. Then you grab the invitations folder and your jacket...

The great executioners of the craft, those of a bygone age of liquid lunches, brought a panache and mercenary professionalism to their writing (if not their drinking). They spawned crippling libel suits and won circulation wars. The boisterously uncontrollable raconteurs of the 18th century gave way to bland social notes about debutante parties and similar in the puritanical Victorian age, but the Age of the Gossip Column began in 1933 when Daily Express proprietor Lord Beaverbrook invented William Hickey.

Appropriately, the real Hickey was an 18th century drunken scoundrel banished to India in disgrace. To upset the establishment, Beaverbrook employed Tom Driberg, a penniless classics scholar, ex-Communist and "homosexual renegade" who savaged the social world that had given him wings, abusing his subjects for their behaviour and appearance wherever possible.

Driberg left to become a Labour MP. Successors failed to live up to him and the staff grew; office legend has it that half a dozen idle diary hacks would use industrial tape to mummify anyone working on the telephone. By the Sixties the gossip columnists had poisonously invaded the private lives of their pop millionaire subjects, leading to a backlash and an immediate retreat to blander page fodder.

Then came the superstar columnist Dempster, and his rivals: Compton Miller, the last editor of the William Hickey column; the well-informed ex-foreign correspondent, Ross Benson; and Peter McKay, who remains the legendary editor of the Mail's waspish Ephraim Hardcastle column. They could work a room, sniff a scandal, sign a tab and print what the great and good didn't want out there. McKay once told a colleague: "I'll shop anyone, apart from my wife. And she'd better behave..."

Miller says now: "You need that rat-like cunning, because you need to check that you're not being done over, being fed false information. That requires a lot of digging. ...The important qualities for any column are sex, power and money. The best example: no one realised that [former BP chairman] Lord Browne was gay. That would have been an incredible diary story."

There's a vague sense now that the glory days are over: staffing (but not workload) reduced, creating a "sausage factory" mentality that can lead to greater reliance on PR staff handouts to "feed the beast". There are impenetrable barriers thrown between the journalist and "the talent". The range of targets has broadened from the aristocracy out through sport, entertainment and the Z-lists of reality TV. New media has diluted the impact of the printed column: online addicts turn to Popbitch, Holy Moly, Perez Hilton, Iain Dale, Guido Fawkes. And gossip columns compete now more than ever with their own news pages: as Dempster said, when he started there was gossip on just one page, "now it's on pages one to 100".

Nevertheless, in this age people are acutely aware that they are often being force-fed a "line" on a policy, product or person. So the future for diaries that can cut through that is bright. Gossip columns should, by definition, be the most unpredictable and the rudest page in any newspaper: sending up, knocking down, exposing, shaming - the garbage chute of the national press. Write them well, and readers and journalists alike will soon be sifting through the trash.