Nigel Hawkes: Peer-reviewed journals aren't worth the paper they're written on

Behind The Numbers

Share
Related Topics

Academic journals like to pride themselves on peer review, the process of sending out papers to other experts in the field for pre-publication checks.

"Is it in a peer-reviewed journal?" journalists are meant to ask themselves before launching into another story about rice pudding causing cancer, or chocolate prolonging life.

The truth is that peer review is largely hokum. What happens if a peer-reviewed journal rejects a paper? It gets sent to another peer-reviewed journal a bit further down the pecking order, which is happy to publish it. Peer review seldom detects fraud, or even mistakes. It is biased against women and against less famous institutions. Its benefits are statistically insignificant and its risks – academic log-rolling, suppression of unfashionable ideas, and the irresistible opportunity to put a spoke in a rival's wheel – are seldom examined.

This didn't matter much when peer review was just a private game for academics. If it made them feel better, that was nice. But in a sinister development, people who have published provocative or implausible claims in peer- reviewed journals have started arguing that they won't listen to criticism unless it has undergone the same laying-on of hands.

The most notorious example of this comes from the authors of The Spirit Level, a book that argues that everything from health to the level of crime, and from happiness to the quality of the toast, is better in societies that are more equal.

Professors Kate Pickett and Richard Wilkinson may well be right in this claim, though I dare say that we can all think of a few exceptions.

What isn't right, however, is their claim that as "almost all" the research that went into The Spirit Level was peer-reviewed, all future debate on the subject should take place in peer-reviewed publications. Come off it, profs, you're pulling our leg, surely? If you're writing a book about a hugely political subject such as inequality, you've surrendered any right to hide behind the flak-jacket of peer review.

Some of the criticism will doubtless be ill-informed, politically motivated, or just plain vulgar. But if the claim is true, it can see off such attacks. If it's untrue, peer review won't help to save it.

Pickett and Wilkinson aren't alone, though. A few months ago, I wrote in this column about a study in The Lancet of which I took a dim view. It claimed that Caesarean operations undertaken without medical cause were nearly three times as likely to cause death or complications to the mother as a normal birth. The evidence fell miles short of proving this claim.

Others shared my view, and one of them, Penny Christensen of Birth Trauma Canada, complained to The Lancet. Rebuffed, she sent them my analysis to support her claim. In reply, an editor wrote to her with the disparaging remark: "We are a scientific journal, and as such prefer to see the scientific debate continued by reference to other academic articles that have been peer-reviewed."

Ms Christensen's complaint has now gone to The Lancet's ombudsman, and we'll see if he shares the attitude that only the peer-reviewed are entitled to have their opinions properly considered.

If it's any help to him or to the authors of The Spirit Level, here's my view. Hiding behind peer review is a betrayal of all the principles of academic life, of open dialogue, of freedom of expression and, since the two profs are so keen on it, of equality as well. It's a disgrace.

Nigel Hawkes is director of Straight Statistics (straightstatistics.org)

React Now

iJobs Job Widget
iJobs General

JavaScript Developer (Angular, Web Forms, HTML5, Ext JS,CSS3)

£40000 - £45000 per annum + Benefits + Bonus: Harrington Starr: JavaScript Dev...

BC2

£50000 - £70000 per annum: Harrington Starr: Business Analyst Consultant (Fina...

SAP Data Migration Consultant

competitive: Progressive Recruitment: My client, a FTSE 100 organisation are u...

Programme Support, Coms, Bristol, £300-350p/d

£300 - £350 per day + competitive: Orgtel: My client, a leading bank, is curre...

Day In a Page

Read Next
 

i Editor's Letter: The final instalment of our WW1 series

Oliver Duff Oliver Duff
 

Simon Usborne: The more you watch pro cycling, the more you understand its social complexity

Simon Usborne
A History of the First World War in 100 Moments: Peace without magnanimity - the summit in a railway siding that ended the fighting

A History of the First World War in 100 Moments

Peace without magnanimity - the summit in a railway siding that ended the fighting
Scottish independence: How the Commonwealth Games could swing the vote

Scottish independence: How the Commonwealth Games could swing the vote

In the final part of our series, Chris Green arrives in Glasgow - a host city struggling to keep the politics out of its celebration of sport
Out in the cold: A writer spends a night on the streets and hears the stories of the homeless

A writer spends a night on the streets

Rough sleepers - the homeless, the destitute and the drunk - exist in every city. Will Nicoll meets those whose luck has run out
Striking new stations, high-speed links and (whisper it) better services - the UK's railways are entering a new golden age

UK's railways are entering a new golden age

New stations are opening across the country and our railways appear to be entering an era not seen in Britain since the early 1950s
Conchita Wurst becomes a 'bride' on the Paris catwalk - and proves there is life after Eurovision

Conchita becomes a 'bride' on Paris catwalk

Alexander Fury salutes the Eurovision Song Contest winner's latest triumph
Pétanque World Championship in Marseilles hit by

Pétanque 'world cup' hit by death threats

This year's most acrimonious sporting event took place in France, not Brazil. How did pétanque get so passionate?
Whelks are healthy, versatile and sustainable - so why did we stop eating them in the UK?

Why did we stop eating whelks?

Whelks were the Victorian equivalent of the donor kebab and our stocks are abundant. So why do we now export them all to the Far East?
10 best women's sunglasses

In the shade: 10 best women's sunglasses

From luxury bespoke eyewear to fun festival sunnies, we round up the shades to be seen in this summer
Germany vs Argentina World Cup 2014: Lionel Messi? Javier Mascherano is key for Argentina...

World Cup final: Messi? Mascherano is key for Argentina...

No 10 is always centre of attention but Barça team-mate is just as crucial to finalists’ hopes
Siobhan-Marie O’Connor: Swimmer knows she needs Glasgow joy on road to Rio

Siobhan-Marie O’Connor: Swimmer needs Glasgow joy on road to Rio

18-year-old says this month’s Commonwealth Games are a key staging post in her career before time slips away
The true Gaza back-story that the Israelis aren’t telling this week

The true Gaza back-story that the Israelis aren’t telling this week

A future Palestine state will have no borders and be an enclave within Israel, surrounded on all sides by Israeli-held territory, says Robert Fisk
A History of the First World War in 100 Moments: The German people demand an end to the fighting

A History of the First World War in 100 Moments

The German people demand an end to the fighting
New play by Oscar Wilde's grandson reveals what the Irish wit said at his trials

New play reveals what Oscar Wilde said at trials

For a century, what Wilde actually said at his trials was a mystery. But the recent discovery of shorthand notes changed that. Now his grandson Merlin Holland has turned them into a play
Can scientists save the world's sea life from

Can scientists save our sea life?

By the end of the century, the only living things left in our oceans could be plankton and jellyfish. Alex Renton meets the scientists who are trying to turn the tide
Richard III, Trafalgar Studios, review: Martin Freeman gives highly intelligent performance

Richard III review

Martin Freeman’s psychotic monarch is big on mockery but wanting in malice