Never mind the fatwas, I say, and don't get hyper-agitated by the PC platoon, we who do not find rape jokes hilarious and who rebuff the fond nicknames "Paki" and "retard" and believe careless language can sanction cruelty and injustice. Though accused daily of impeding freedom of speech, we do not have the muscle or money to destabilise that fundamental entitlement. The real pressures that turn to actual curtailments of free expression lie in confidentiality contracts, unspoken loyalties between power and those it seduces or buys up, thought-control in the name of anti-terrorism and the monstrous English libel laws, more draconian than in any other Western nation.

Feel shame and consternation that London, famed for its energy, creativity and colour is now the libel capital of the world and libel tourism has joined other suspect activities that draw the global wealthy to our shores and shake up the foundations of our democracy. Russian oligarchs, Arab sheikhs, multinationals, media moguls, and South African millionaires love our courts and laws that keep their dodgy affairs away from the prying eyes of the public.

Others freely use the system, too. International stars fed up with intrusive and false reports routinely turn to our high-court judges to protect their reputations. Actress Cameron Diaz sued an American newspaper, the Miami-based National Enquirer, in the English courts and won substantial damages. These victories, though in some ways justified, do set off unintended consequences. A select committee hearing took evidence from the New York Times and other US publications. These trials mean some papers – including the Wall Street Journal – may withdraw from the UK and block internet access.

Authors are being intimidated as well. There was the case taken by a super-rich Saudi businessman, Khalid Mahfouz against the academic Rachel Ehrenfeld, who had written a book on the funders of terrorism alleging Mahfouz was involved. Only 23 copies of the book were sold in the UK but the case went to a British court. Ehrenfeld refused to attend. Some American states, the Senate and House of Representatives are taking action to overturn the decisions of English libel judges.

Gagging is now commonplace in science and medicine. Experts in these fields have had to withdraw serious published concerns. Andrew Lewis, who runs a website where some medical claims are discredited, has been forced to pull unsubstantiated and criticised claims made by an osteopath which were already in the public domain. He would have faced libel action for reporting a fact. Dr Peter Wilmhurst, a British cardiologist, is being sued by an American manufacturer of new heart implants because he expressed doubts at a conference about their effectiveness. A scientist, Simon Singh, is still going through the gruelling and potentially crippling libel appeal process after being successfully sued by the British Chiropractic Association for describing as "bogus" some of the get-well promises made by practitioners.

Publishers, newspapers and broadcasters are now pre-empting libel actions by taking out material that might arouse or upset claimants. YouTube and citizen filming means you can be filmed saying something at a conference or seminar which then gets broadcast, making you vulnerable to the libel jackals. Honest views and opinions are not exempt. Alistair Brett, legal manager of the Times newspapers, believes these laws "have gone bonkers". They are disproportionate, immoral silencing instruments, widely and cynically used.

A comparative study of media law by the Oxford Centre for Socio-Legal Studies found that the cost of libel actions in England and Wales is 140 times higher than the European average. After the death of Robert Maxwell, the hope was that libel threats would no longer be allowed to stop legitimate investigations of the loaded and mighty. Incredibly, we went the other way. With a few exceptions, like the redoubtable journalist Tom Bower, who meticulously scrutinises the lives of big men like Conrad Black and Richard Desmond in his well-researched books, beating back the legal brutes who tear into him, most other journalists have had to accept severe limits on what we can broadcast or write about litigious men and women, organisations, institutions and businesses. I cannot even tell you the subjects and people I am not allowed to write about in our baleful new world. Censorship? Surely not? That happens in repressive China and Iran, not here? Yes, in our own backyard.

The slide into a hushed state is alarming the UN Human Rights Committee, which issued this statement last year: "The practical application of the law of libel has served to discourage critical reporting on matters of serious public interest, adversely affecting the ability of scholars and journalists to publish their work". A new campaign was launched this week by Index on Censorship and English PEN to halt this domination by thought and word controllers. Nobody with a concern for justice and fairness would want a society where anything could be said and anybody maligned without restraint. But the balance in our libel laws has tipped away from good sense into repression. Geoffrey Robertson spoke at the launch. Without using notes, the lawyer spoke with passion that shook the ground beneath his feet: "We have not free speech but expensive speech."

Ken McDonald QC reminded us of the iniquity of the law itself which assumes a defendant is guilty until proved innocent. The money involved is prohibitive even for the provably innocent. This must have been the feeling and energy at gatherings of wigged, zealous, conscientious reformists in earlier centuries.

So heady was the atmosphere that a representative of Carter-Ruck, the infamous libel firm that goes like a hound after complainants, succumbed and agreed we needed to bring down costs. We may have reached a point where even those who gain from our libel laws can see they suck. The campaign must be supported. We have nothing to lose but our chains and gags. Freedom of speech matters most when it is taken away by key players in the establishment. Like now.

Yasmin Alibhai-Brown will be performing her show in London this Wednesday and on 29 November ( www.alibhai-brown.com)