Nigel Hawkes: Peer-reviewed journals aren't worth the paper they're written on

Behind The Numbers

Share
Related Topics

Academic journals like to pride themselves on peer review, the process of sending out papers to other experts in the field for pre-publication checks.

"Is it in a peer-reviewed journal?" journalists are meant to ask themselves before launching into another story about rice pudding causing cancer, or chocolate prolonging life.

The truth is that peer review is largely hokum. What happens if a peer-reviewed journal rejects a paper? It gets sent to another peer-reviewed journal a bit further down the pecking order, which is happy to publish it. Peer review seldom detects fraud, or even mistakes. It is biased against women and against less famous institutions. Its benefits are statistically insignificant and its risks – academic log-rolling, suppression of unfashionable ideas, and the irresistible opportunity to put a spoke in a rival's wheel – are seldom examined.

This didn't matter much when peer review was just a private game for academics. If it made them feel better, that was nice. But in a sinister development, people who have published provocative or implausible claims in peer- reviewed journals have started arguing that they won't listen to criticism unless it has undergone the same laying-on of hands.

The most notorious example of this comes from the authors of The Spirit Level, a book that argues that everything from health to the level of crime, and from happiness to the quality of the toast, is better in societies that are more equal.

Professors Kate Pickett and Richard Wilkinson may well be right in this claim, though I dare say that we can all think of a few exceptions.

What isn't right, however, is their claim that as "almost all" the research that went into The Spirit Level was peer-reviewed, all future debate on the subject should take place in peer-reviewed publications. Come off it, profs, you're pulling our leg, surely? If you're writing a book about a hugely political subject such as inequality, you've surrendered any right to hide behind the flak-jacket of peer review.

Some of the criticism will doubtless be ill-informed, politically motivated, or just plain vulgar. But if the claim is true, it can see off such attacks. If it's untrue, peer review won't help to save it.

Pickett and Wilkinson aren't alone, though. A few months ago, I wrote in this column about a study in The Lancet of which I took a dim view. It claimed that Caesarean operations undertaken without medical cause were nearly three times as likely to cause death or complications to the mother as a normal birth. The evidence fell miles short of proving this claim.

Others shared my view, and one of them, Penny Christensen of Birth Trauma Canada, complained to The Lancet. Rebuffed, she sent them my analysis to support her claim. In reply, an editor wrote to her with the disparaging remark: "We are a scientific journal, and as such prefer to see the scientific debate continued by reference to other academic articles that have been peer-reviewed."

Ms Christensen's complaint has now gone to The Lancet's ombudsman, and we'll see if he shares the attitude that only the peer-reviewed are entitled to have their opinions properly considered.

If it's any help to him or to the authors of The Spirit Level, here's my view. Hiding behind peer review is a betrayal of all the principles of academic life, of open dialogue, of freedom of expression and, since the two profs are so keen on it, of equality as well. It's a disgrace.

Nigel Hawkes is director of Straight Statistics (straightstatistics.org)

React Now

Latest stories from i100
Have you tried new the Independent Digital Edition apps?
iJobs Job Widget
iJobs General

Sauce Recruitment: Retail Planning Manager - Home Entertainment UK

salary equal to £40K pro-rata: Sauce Recruitment: Are you available to start a...

Ashdown Group: Front-End Developer - London - up to £40,000

£35000 - £40000 per annum: Ashdown Group: Creative Front-End Developer - Claph...

Recruitment Genius: Product Quality Assurance Technologist - Hardline & Electric

£18000 - £24000 per annum: Recruitment Genius: The role in this successful eco...

Ashdown Group: QA Tester - London - £30,000

£28000 - £30000 per annum: Ashdown Group: QA Tester - London - £30,000 QA Tes...

Day In a Page

Read Next
 

CPAC 2015: What I learnt from the US — and what the US could learn from Ukip

Nigel Farage
 

If I were Prime Minister: I would create a government that actually reflects its people

Kaliya Franklin
HIV pill: Scientists hail discovery of 'game-changer' that cuts the risk of infection among gay men by 86%

Scientists hail daily pill that protects against HIV infection

Breakthrough in battle against global scourge – but will the NHS pay for it?
How we must adjust our lifestyles to nature: Welcome to the 'Anthropocene', the human epoch

Time to play God

Welcome to the 'Anthropocene', the human epoch where we may need to redefine nature itself
MacGyver returns, but with a difference: Handyman hero of classic 1980s TV series to be recast as a woman

MacGyver returns, but with a difference

Handyman hero of classic 1980s TV series to be recast as a woman
Tunnel renaissance: Why cities are hiding roads down in the ground

Tunnel renaissance

Why cities are hiding roads underground
'Backstreet Boys - Show 'Em What You're Made Of': An affectionate look at five middle-aged men

Boys to men

The Backstreet Boys might be middle-aged, married and have dodgy knees, but a heartfelt documentary reveals they’re not going gently into pop’s good night
Crufts 2015: Should foreign dogs be allowed to compete?

Crufts 2015

Should foreign dogs be allowed to compete?
10 best projectors

How to make your home cinema more cinematic: 10 best projectors

Want to recreate the big-screen experience in your sitting room? IndyBest sizes up gadgets to form your film-watching
Manchester City 1 Barcelona 2 player ratings: Luis Suarez? Lionel Messi? Joe Hart? Who was the star man?

Manchester City vs Barcelona player ratings

Luis Suarez? Lionel Messi? Joe Hart? Who was the star man at the Etihad?
Arsenal vs Monaco: Monaco - the making of Gunners' manager Arsene Wenger

Monaco: the making of Wenger

Jack Pitt-Brooke speaks to former players and learns the Frenchman’s man-management has always been one of his best skills
Cricket World Cup 2015: Chris Gayle - the West Indies' enigma lives up to his reputation

Chris Gayle: The West Indies' enigma

Some said the game's eternal rebel was washed up. As ever, he proved he writes the scripts by producing a blistering World Cup innings
In Ukraine a dark world of hybrid warfare and murky loyalties prevails

In Ukraine a dark world of hybrid warfare

This war in the shadows has been going on since the fall of Mr Yanukovych
'Birdman' and 'Bullets Over Broadway': Homage or plagiarism?

Homage or plagiarism?

'Birdman' shares much DNA with Woody Allen's 'Bullets Over Broadway'
Broadchurch ends as damp squib not even David Tennant can revive

A damp squib not even David Tennant can revive

Broadchurch, Series 2 finale, review
A Koi carp breeding pond, wall-mounted iPads and a bathroom with a 'wellness' shower: inside the mansion of Germany's 'Bishop of Bling'

Inside the mansion of Germany's 'Bishop of Bling'

A Koi carp breeding pond, wall-mounted iPads and a bathroom with a 'wellness' shower