Nigel Hawkes: Peer-reviewed journals aren't worth the paper they're written on

Behind The Numbers

Share
Related Topics

Academic journals like to pride themselves on peer review, the process of sending out papers to other experts in the field for pre-publication checks.

"Is it in a peer-reviewed journal?" journalists are meant to ask themselves before launching into another story about rice pudding causing cancer, or chocolate prolonging life.

The truth is that peer review is largely hokum. What happens if a peer-reviewed journal rejects a paper? It gets sent to another peer-reviewed journal a bit further down the pecking order, which is happy to publish it. Peer review seldom detects fraud, or even mistakes. It is biased against women and against less famous institutions. Its benefits are statistically insignificant and its risks – academic log-rolling, suppression of unfashionable ideas, and the irresistible opportunity to put a spoke in a rival's wheel – are seldom examined.

This didn't matter much when peer review was just a private game for academics. If it made them feel better, that was nice. But in a sinister development, people who have published provocative or implausible claims in peer- reviewed journals have started arguing that they won't listen to criticism unless it has undergone the same laying-on of hands.

The most notorious example of this comes from the authors of The Spirit Level, a book that argues that everything from health to the level of crime, and from happiness to the quality of the toast, is better in societies that are more equal.

Professors Kate Pickett and Richard Wilkinson may well be right in this claim, though I dare say that we can all think of a few exceptions.

What isn't right, however, is their claim that as "almost all" the research that went into The Spirit Level was peer-reviewed, all future debate on the subject should take place in peer-reviewed publications. Come off it, profs, you're pulling our leg, surely? If you're writing a book about a hugely political subject such as inequality, you've surrendered any right to hide behind the flak-jacket of peer review.

Some of the criticism will doubtless be ill-informed, politically motivated, or just plain vulgar. But if the claim is true, it can see off such attacks. If it's untrue, peer review won't help to save it.

Pickett and Wilkinson aren't alone, though. A few months ago, I wrote in this column about a study in The Lancet of which I took a dim view. It claimed that Caesarean operations undertaken without medical cause were nearly three times as likely to cause death or complications to the mother as a normal birth. The evidence fell miles short of proving this claim.

Others shared my view, and one of them, Penny Christensen of Birth Trauma Canada, complained to The Lancet. Rebuffed, she sent them my analysis to support her claim. In reply, an editor wrote to her with the disparaging remark: "We are a scientific journal, and as such prefer to see the scientific debate continued by reference to other academic articles that have been peer-reviewed."

Ms Christensen's complaint has now gone to The Lancet's ombudsman, and we'll see if he shares the attitude that only the peer-reviewed are entitled to have their opinions properly considered.

If it's any help to him or to the authors of The Spirit Level, here's my view. Hiding behind peer review is a betrayal of all the principles of academic life, of open dialogue, of freedom of expression and, since the two profs are so keen on it, of equality as well. It's a disgrace.

Nigel Hawkes is director of Straight Statistics (straightstatistics.org)

React Now

Latest stories from i100
Have you tried new the Independent Digital Edition apps?
iJobs Job Widget
iJobs General

Trainee Recruitment Consultant

£20000 - £25000 per annum + Uncapped Commission, 1st yr OTE £30-£40k : SThree:...

Middleware Support Analyst

£45000 - £50000 Per Annum: Clearwater People Solutions Ltd: Our client is curr...

Senior Java Developer/Designer

£400 - £450 Per Day: Clearwater People Solutions Ltd: My client are looking fo...

Domino Developer and Administrator

£40000 - £45000 Per Annum + benefits: Clearwater People Solutions Ltd: Domino ...

Day In a Page

Read Next
 

Daily catch-up: Cameron’s speech, his place in history, and the Pedant Club

John Rentoul
 

Why Facebook won't be feeling threatened by Ello...yet

Ed Rex
Italian couples fake UK divorce scam on an ‘industrial scale’

Welcome to Maidenhead, the divorce capital of... Italy

A look at the the legal tourists who exploited our liberal dissolution rules
Tom and Jerry cartoons now carry a 'racial prejudice' warning on Amazon

Tom and Jerry cartoons now carry a 'racial prejudice' warning on Amazon

The vintage series has often been criticised for racial stereotyping
An app for the amorous: Could Good2Go end disputes about sexual consent - without being a passion-killer?

An app for the amorous

Could Good2Go end disputes about sexual consent - without being a passion-killer?
Llansanffraid is now Llansantffraid. Welsh town changes its name, but can you spot the difference?

Llansanffraid is now Llansantffraid

Welsh town changes its name, but can you spot the difference?
Charlotte Riley: At the peak of her powers

Charlotte Riley: At the peak of her powers

After a few early missteps with Chekhov, her acting career has taken her to Hollywood. Next up is a role in the BBC’s gangster drama ‘Peaky Blinders’
She's having a laugh: Britain's female comedians have never had it so good

She's having a laugh

Britain's female comedians have never had it so good, says stand-up Natalie Haynes
Sistine Chapel to ‘sing’ with new LED lights designed to bring Michelangelo’s masterpiece out of the shadows

Let there be light

Sistine Chapel to ‘sing’ with new LEDs designed to bring Michelangelo’s masterpiece out of the shadows
Great British Bake Off, semi-final, review: Richard remains the baker to beat

Tensions rise in Bake Off's pastry week

Richard remains the baker to beat as Chetna begins to flake
Paris Fashion Week, spring/summer 2015: Time travel fashion at Louis Vuitton in Paris

A look to the future

It's time travel fashion at Louis Vuitton in Paris
The 10 best bedspreads

The 10 best bedspreads

Before you up the tog count on your duvet, add an extra layer and a room-changing piece to your bed this autumn
Arsenal vs Galatasaray: Five things we learnt from the Emirates

Arsenal vs Galatasaray

Five things we learnt from the Gunners' Champions League victory at the Emirates
Stuart Lancaster’s long-term deal makes sense – a rarity for a decision taken by the RFU

Lancaster’s long-term deal makes sense – a rarity for a decision taken by the RFU

This deal gives England a head-start to prepare for 2019 World Cup, says Chris Hewett
Ebola outbreak: The children orphaned by the virus – then rejected by surviving relatives over fear of infection

The children orphaned by Ebola...

... then rejected by surviving relatives over fear of infection
Pride: Are censors pandering to homophobia?

Are censors pandering to homophobia?

US film censors have ruled 'Pride' unfit for under-16s, though it contains no sex or violence
The magic of roundabouts

Lords of the rings

Just who are the Roundabout Appreciation Society?