Nigel Hawkes: Peer-reviewed journals aren't worth the paper they're written on

Behind The Numbers

Share
Related Topics

Academic journals like to pride themselves on peer review, the process of sending out papers to other experts in the field for pre-publication checks.

"Is it in a peer-reviewed journal?" journalists are meant to ask themselves before launching into another story about rice pudding causing cancer, or chocolate prolonging life.

The truth is that peer review is largely hokum. What happens if a peer-reviewed journal rejects a paper? It gets sent to another peer-reviewed journal a bit further down the pecking order, which is happy to publish it. Peer review seldom detects fraud, or even mistakes. It is biased against women and against less famous institutions. Its benefits are statistically insignificant and its risks – academic log-rolling, suppression of unfashionable ideas, and the irresistible opportunity to put a spoke in a rival's wheel – are seldom examined.

This didn't matter much when peer review was just a private game for academics. If it made them feel better, that was nice. But in a sinister development, people who have published provocative or implausible claims in peer- reviewed journals have started arguing that they won't listen to criticism unless it has undergone the same laying-on of hands.

The most notorious example of this comes from the authors of The Spirit Level, a book that argues that everything from health to the level of crime, and from happiness to the quality of the toast, is better in societies that are more equal.

Professors Kate Pickett and Richard Wilkinson may well be right in this claim, though I dare say that we can all think of a few exceptions.

What isn't right, however, is their claim that as "almost all" the research that went into The Spirit Level was peer-reviewed, all future debate on the subject should take place in peer-reviewed publications. Come off it, profs, you're pulling our leg, surely? If you're writing a book about a hugely political subject such as inequality, you've surrendered any right to hide behind the flak-jacket of peer review.

Some of the criticism will doubtless be ill-informed, politically motivated, or just plain vulgar. But if the claim is true, it can see off such attacks. If it's untrue, peer review won't help to save it.

Pickett and Wilkinson aren't alone, though. A few months ago, I wrote in this column about a study in The Lancet of which I took a dim view. It claimed that Caesarean operations undertaken without medical cause were nearly three times as likely to cause death or complications to the mother as a normal birth. The evidence fell miles short of proving this claim.

Others shared my view, and one of them, Penny Christensen of Birth Trauma Canada, complained to The Lancet. Rebuffed, she sent them my analysis to support her claim. In reply, an editor wrote to her with the disparaging remark: "We are a scientific journal, and as such prefer to see the scientific debate continued by reference to other academic articles that have been peer-reviewed."

Ms Christensen's complaint has now gone to The Lancet's ombudsman, and we'll see if he shares the attitude that only the peer-reviewed are entitled to have their opinions properly considered.

If it's any help to him or to the authors of The Spirit Level, here's my view. Hiding behind peer review is a betrayal of all the principles of academic life, of open dialogue, of freedom of expression and, since the two profs are so keen on it, of equality as well. It's a disgrace.

Nigel Hawkes is director of Straight Statistics (straightstatistics.org)

React Now

Latest stories from i100
Have you tried new the Independent Digital Edition apps?
iJobs Job Widget
iJobs General

Marketing Manager - Leicestershire - £35,000

£30000 - £35000 per annum: Ashdown Group: Marketing Manager (CIM, B2B, MS Offi...

Marketing Executive (B2B and B2C) - Rugby, Warwickshire

£22000 - £25000 per annum: Ashdown Group: A highly successful organisation wit...

SEN Coordinator + Teacher (SENCO)

£1 per day: Randstad Education Leeds: Job Purpose To work closely with the he...

Research Manager - Quantitative/Qualitative

£32000 - £42000 Per Annum: Clearwater People Solutions Ltd: Our client is curr...

Day In a Page

Read Next
Piper Ryan Randall leads a pro-Scottish independence rally in the suburbs of Edinburgh  

i Editor's Letter: Britain survives, but change is afoot

Oliver Duff Oliver Duff
Some believe that David Cameron is to blame for allowing Alex Salmond a referendum  

Scottish referendum: So how about the English now being given a chance to split from England?

Mark Steel
Scottish referendum: The Yes vote was the love that dared speak its name, but it was not to be

Despite the result, this is the end of the status quo

Boyd Tonkin on the fall-out from the Scottish referendum
Manolo Blahnik: The high priest of heels talks flats, Englishness, and why he loves Mary Beard

Manolo Blahnik: Flats, Englishness, and Mary Beard

The shoe designer who has been dubbed 'the patron saint of the stiletto'
The Beatles biographer reveals exclusive original manuscripts of some of the best pop songs ever written

Scrambled eggs and LSD

Behind The Beatles' lyrics - thanks to Hunter Davis's original manuscript copies
'Normcore' fashion: Blending in is the new standing out in latest catwalk non-trend

'Normcore': Blending in is the new standing out

Just when fashion was in grave danger of running out of trends, it only went and invented the non-trend. Rebecca Gonsalves investigates
Dance’s new leading ladies fight back: How female vocalists are now writing their own hits

New leading ladies of dance fight back

How female vocalists are now writing their own hits
Mystery of the Ground Zero wedding photo

A shot in the dark

Mystery of the wedding photo from Ground Zero
His life, the universe and everything

His life, the universe and everything

New biography sheds light on comic genius of Douglas Adams
Save us from small screen superheroes

Save us from small screen superheroes

Shows like Agents of S.H.I.E.L.D are little more than marketing tools
Reach for the skies

Reach for the skies

From pools to football pitches, rooftop living is looking up
These are the 12 best hotel spas in the UK

12 best hotel spas in the UK

Some hotels go all out on facilities; others stand out for the sheer quality of treatments
These Iranian-controlled Shia militias used to specialise in killing American soldiers. Now they are fighting Isis, backed up by US airstrikes

Widespread fear of Isis is producing strange bedfellows

Iranian-controlled Shia militias that used to kill American soldiers are now fighting Isis, helped by US airstrikes
Topshop goes part Athena poster, part last spring Prada

Topshop goes part Athena poster, part last spring Prada

Shoppers don't come to Topshop for the unique
How to make a Lego masterpiece

How to make a Lego masterpiece

Toy breaks out of the nursery and heads for the gallery
Meet the ‘Endies’ – city dwellers who are too poor to have fun

Meet the ‘Endies’ – city dwellers who are too poor to have fun

Urbanites are cursed with an acronym pointing to Employed but No Disposable Income or Savings
Paisley’s decision to make peace with IRA enemies might remind the Arabs of Sadat

Ian Paisley’s decision to make peace with his IRA enemies

His Save Ulster from Sodomy campaign would surely have been supported by many a Sunni imam