Nigel Hawkes: Peer-reviewed journals aren't worth the paper they're written on

Behind The Numbers

Share
Related Topics

Academic journals like to pride themselves on peer review, the process of sending out papers to other experts in the field for pre-publication checks.

"Is it in a peer-reviewed journal?" journalists are meant to ask themselves before launching into another story about rice pudding causing cancer, or chocolate prolonging life.

The truth is that peer review is largely hokum. What happens if a peer-reviewed journal rejects a paper? It gets sent to another peer-reviewed journal a bit further down the pecking order, which is happy to publish it. Peer review seldom detects fraud, or even mistakes. It is biased against women and against less famous institutions. Its benefits are statistically insignificant and its risks – academic log-rolling, suppression of unfashionable ideas, and the irresistible opportunity to put a spoke in a rival's wheel – are seldom examined.

This didn't matter much when peer review was just a private game for academics. If it made them feel better, that was nice. But in a sinister development, people who have published provocative or implausible claims in peer- reviewed journals have started arguing that they won't listen to criticism unless it has undergone the same laying-on of hands.

The most notorious example of this comes from the authors of The Spirit Level, a book that argues that everything from health to the level of crime, and from happiness to the quality of the toast, is better in societies that are more equal.

Professors Kate Pickett and Richard Wilkinson may well be right in this claim, though I dare say that we can all think of a few exceptions.

What isn't right, however, is their claim that as "almost all" the research that went into The Spirit Level was peer-reviewed, all future debate on the subject should take place in peer-reviewed publications. Come off it, profs, you're pulling our leg, surely? If you're writing a book about a hugely political subject such as inequality, you've surrendered any right to hide behind the flak-jacket of peer review.

Some of the criticism will doubtless be ill-informed, politically motivated, or just plain vulgar. But if the claim is true, it can see off such attacks. If it's untrue, peer review won't help to save it.

Pickett and Wilkinson aren't alone, though. A few months ago, I wrote in this column about a study in The Lancet of which I took a dim view. It claimed that Caesarean operations undertaken without medical cause were nearly three times as likely to cause death or complications to the mother as a normal birth. The evidence fell miles short of proving this claim.

Others shared my view, and one of them, Penny Christensen of Birth Trauma Canada, complained to The Lancet. Rebuffed, she sent them my analysis to support her claim. In reply, an editor wrote to her with the disparaging remark: "We are a scientific journal, and as such prefer to see the scientific debate continued by reference to other academic articles that have been peer-reviewed."

Ms Christensen's complaint has now gone to The Lancet's ombudsman, and we'll see if he shares the attitude that only the peer-reviewed are entitled to have their opinions properly considered.

If it's any help to him or to the authors of The Spirit Level, here's my view. Hiding behind peer review is a betrayal of all the principles of academic life, of open dialogue, of freedom of expression and, since the two profs are so keen on it, of equality as well. It's a disgrace.

Nigel Hawkes is director of Straight Statistics (straightstatistics.org)

React Now

Latest stories from i100
Have you tried new the Independent Digital Edition apps?
iJobs Job Widget
iJobs General

Recruitment Genius: Account Manager

£20000 - £35000 per annum: Recruitment Genius: This full service social media ...

Recruitment Genius: Data Analyst - Online Marketing

£24000 - £35000 per annum: Recruitment Genius: We are 'Changemakers in retail'...

Austen Lloyd: Senior Residential Conveyancer

Very Competitive: Austen Lloyd: Senior Conveyancer - South West We are see...

Austen Lloyd: Residential / Commercial Property Solicitor

Excellent Salary: Austen Lloyd: DORSET MARKET TOWN - SENIOR PROPERTY SOLICITOR...

Day In a Page

Read Next
RIP Voicemail?  

Voicemail has got me out of some tight corners, so let's not abandon it

Simon Kelner
A sculpture illustrating the WW1 Christmas Truce football match in Liverpool  

It's been 100 years since the Christmas Truce, but football is still changing the world

Jim Murphy and Dan Jarvis
Isis in Iraq: Yazidi girls killing themselves to escape rape and imprisonment by militants

'Jilan killed herself in the bathroom. She cut her wrists and hanged herself'

Yazidi girls killing themselves to escape rape and imprisonment
Ed Balls interview: 'If I think about the deficit when I'm playing the piano, it all goes wrong'

Ed Balls interview

'If I think about the deficit when I'm playing the piano, it all goes wrong'
He's behind you, dude!

US stars in UK panto

From David Hasselhoff to Jerry Hall
Grace Dent's Christmas Quiz: What are you – a festive curmudgeon or top of the tree?

Grace Dent's Christmas Quiz

What are you – a festive curmudgeon or top of the tree?
Nasa planning to build cloud cities in airships above Venus

Nasa planning to build cloud cities in airships above Venus

Planet’s surface is inhospitable to humans but 30 miles above it is almost perfect
Surrounded by high-rise flats is a little house filled with Lebanon’s history - clocks, rifles, frogmen’s uniforms and colonial helmets

Clocks, rifles, swords, frogmen’s uniforms

Surrounded by high-rise flats is a little house filled with Lebanon’s history
Return to Gaza: Four months on, the wounds left by Israel's bombardment have not yet healed

Four months after the bombardment, Gaza’s wounds are yet to heal

Kim Sengupta is reunited with a man whose plight mirrors the suffering of the Palestinian people
Gastric surgery: Is it really the answer to the UK's obesity epidemic?

Is gastric surgery really the answer to the UK's obesity epidemic?

Critics argue that it’s crazy to operate on healthy people just to stop them eating
Homeless Veterans appeal: Christmas charity auction Part 2 - now LIVE

Homeless Veterans appeal: Christmas charity auction

Bid on original art, or trips of a lifetime to Africa or the 'Corrie' set, and help Homeless Veterans
Pantomime rings the changes to welcome autistic theatre-goers

Autism-friendly theatre

Pantomime leads the pack in quest to welcome all
The week Hollywood got scared and had to grow up a bit

The week Hollywood got scared and had to grow up a bit

Sony suffered a chorus of disapproval after it withdrew 'The Interview', but it's not too late for it to take a stand, says Joan Smith
From Widow Twankey to Mother Goose, how do the men who play panto dames get themselves ready for the performance of a lifetime?

Panto dames: before and after

From Widow Twankey to Mother Goose, how do the men who play panto dames get themselves ready for the performance of a lifetime?
Thirties murder mystery novel is surprise runaway Christmas hit

Thirties murder mystery novel is surprise runaway Christmas hit

Booksellers say readers are turning away from dark modern thrillers and back to the golden age of crime writing
Anne-Marie Huby: 'Charities deserve the best,' says founder of JustGiving

Anne-Marie Huby: 'Charities deserve the best'

Ten million of us have used the JustGiving website to donate to good causes. Its co-founder says that being dynamic is as important as being kind
The botanist who hunts for giant trees at Kew Gardens

The man who hunts giants

A Kew Gardens botanist has found 25 new large tree species - and he's sure there are more out there