Angus McLaren: Blackmail isn't what it used to be, but it still sets the pulse racing

The blackmailer has fewer victims in an age of liberal sexual attitudes. But an alleged attempt to extort money from a Royal shows hypocrisy retains its thrill for the prurient

"In the afternoon Randolph Churchill came to see me. He tells me there has been a scandal about Prince George – letters to a young man in Paris. A large sum had to be paid for their recovery." So wrote the diplomat Robert Bruce Lockhart in his diary for 28 April 1932.

Three-quarters of a century later we see the popular press fixated on the trial of two men who purportedly demanded £50,000 with the threat of making public audio recordings which refer to a member of the British Royal Family committing a homosexual act. At first glance it would appear that over the years blackmail has changed very little.

While waiting in supermarket checkout lines, I used to idly scan tabloid accounts of sexual blackmail, never thinking that such tawdry stories were worth serious scrutiny. And then, while researching the history of abortion in 19th century England, I was surprised to discover that in 1890s London a woman who sought to terminate a pregnancy was more likely to be hounded by a blackmailer than bothered by the police.

When I subsequently unearthed reports in the early 20th century of criminals trying to extort money from homosexual men and adulterous women I began to appreciate the pivotal role blackmail played in the policing of modern sexuality.

Historians of sexuality face the problem of finding sources that cast light on past generations' discussion of intimate matters. Accounts of sexual blackmail, I came to realise, provided an untapped mine of material.

Though stories of sexual blackmail continue to surface in today's media, they are no longer as revelatory as they were in the first decades of last century. Indeed past generations were far more familiar with the classic blackmail scenario – demanding money with the threat to reveal a victim's sexual secrets. But if you look up "blackmail" on the web today you find far more references to "political blackmail", "nuclear blackmail" and "emotional blackmail" than to sexual blackmail.

Sexual blackmail has a history; the crime emerged at a specific time and the "blackmail story" played an unexpected role in allowing veiled discussions of issues like homosexuality and abortion which were otherwise excluded from prudish public forums.

The blackmail story first emerged in the 18th century, flowered in the last decades of the 19th and became a full blown cultural preoccupation in the inter-war period. In the 18th century wealthy men were so alarmed by the thought of criminals extorting money by threatening to accuse them of sodomy that the government made the levelling of such menaces a felony. It was an easy accusation to make, and sodomy was at the time a capital offence. In the 19th century, with the increased centrality of an unsullied sexual reputation, the fear grew that even respectable men's heterosexual peccadilloes could put them at the mercy of a scoundrel.

Realistic middle-class males appreciated, of course, the existence of the sexual double standard. Hypocrisy created the fear of exposure and the threat of exposure was accordingly criminalised. But the well-off were not the only victims. Because of the law against "indecency between males," homosexuals of every class became the favoured prey of extortionists.

The law on blackmail poses an apparent paradox. It declares that threatening to tell the truth can be a crime. It raises the question of what is just: to pretend to be adhering to a particular moral code while leading a double life? Or to gain by exposing another's guilty secrets?

Blackmail trials revealed that that vague entity – one's sexual reputation – had, in an increasingly commercial age, a definite monetary value. Though other forms of larceny were far more common, reports of blackmail struck a nerve.

The middle-classes, regarding privacy and secrecy as both especially precious and dangerous, found such accounts hypnotically preoccupying. The nightmarish scenario of the exposure of a dangerous sexual secret haunted the anxious; it was a demon, which the court transcripts, like the detective stories of Arthur Conan Doyle and the later melodramatic films of Alfred Hitchcock, attempted to exorcise. A titillated public guiltily enjoyed blackmail stories and movies which brought to light the perilous sexual secrets of the adventurous.

Judges in blackmail trials aimed to impress upon society the necessity of respecting the power represented by wealth, age and status. Moreover, as some blackmail was a sort of private law enforcement, the law felt as much affronted by it as did the individual victim, and all the more intent on extirpating the crime.

The press appeared both to support and undermine the courts' attempts at deflecting attention away from sexual improprieties. Journalists always recognised the appeal of a story which broached the boundaries separating private and public, men and women, poor and wealthy, and which revealed complicated and at times shocking relationships.

Reporting on blackmail allowed the press to indulge in prurient presentations for consumption by a supposedly respectable readership.

After 1925 all English blackmail victims were provided with anonymity in court, but claims that decency prohibited full discussion of some cases fuelled, rather than dampened, public curiosity.

What newspaper readers thought of sexual blackmail is hard to determine, though the evidence suggests that the public was an active, not a passive participant in interpreting the statements given by witnesses and reported in the press. It wanted to see and hear what was considered so shocking.

An arrest precipitated a scandal. A court case rendered visible issues which were supposed to remain invisible. A variety of lessons would be drawn from the evidence heard.

Trials allowed the public to conceptualise a range of tabooed behaviors and provided it with insights into the ways in which both the rough and the respectable participated in such activities. The judges presiding over blackmail trials noted that laws pertaining to divorce, age of consent, prostitution, abortion and homosexuality subjected many sexual issues to public scrutiny. Sexual blackmail provided striking evidence that when morality was made the law's business it often became the criminal's business as well.

A well-managed trial was supposed to result in the "guilty" being punished and the "innocent" protected. Because of the very nature of the sexual blackmail trial – the fact that the victim often had a "guilty secret" – this satisfying conclusion could not always be achieved. Court accounts could lead to the airing of embarrassing questions. Did a man who seduced a woman "owe" her anything? Why should homosexuality be criminalised? Should women be permitted to abort?

The blackmail stories were both the cause and effect of a culture going through important transformations in attitudes towards sex and gender. Society preferred to blame the eruption of blackmail on certain "dangerous" women and men rather than admit that the unscrupulous simply exploited the gap between what people said about sex and what they actually did.

At times blackmail cases had important legal ramifications, such as sparking calls for legal reforms. For example, the fear of "gold diggers" in the 1930s led many American states to revoke laws – known as the "heart balm statutes" – which provide compensation for jilted young women. In the 1950s and 1960s British parliamentarians, citing the fact that existing laws encouraged the blackmailing of gays, began the decriminalisation of homosexuality.

With the closer alignment of sexual beliefs and practices in the latter half of the 20th century there followed a corresponding decline in the cultural significance of sexual blackmail. Starting in the 1950s and particularly after the "sexual revolution" of the 1960s the sexual blackmail story lost much of its cultural purchase.

The successes of the legal reformers in amending the laws pertaining to divorce, abortion and homosexuality and changes in sexual mores contributed to this waning in importance of the blackmail story. It had been an important, if indirect, way of addressing a number of sensitive sexual issues; once they could be directly broached it lost much of its interest.

Sexual blackmail did not, of course, disappear. There still are many stigmatised sexual acts and public voyeurism is rampant. But with "chequebook journalism" flourishing today, those seeking to peddle information on sexual indiscretions have the safer option of selling their story to the tabloids or the TV talk shows. The result is that a public, fascinated as ever by sexual secrets, no longer has to rely on that venerable narrative device which first allowed so many of them to be discussed – the sexual blackmail story.