Nigel Hawkes: Peer-reviewed journals aren't worth the paper they're written on

Behind The Numbers

Share
Related Topics

Academic journals like to pride themselves on peer review, the process of sending out papers to other experts in the field for pre-publication checks.

"Is it in a peer-reviewed journal?" journalists are meant to ask themselves before launching into another story about rice pudding causing cancer, or chocolate prolonging life.

The truth is that peer review is largely hokum. What happens if a peer-reviewed journal rejects a paper? It gets sent to another peer-reviewed journal a bit further down the pecking order, which is happy to publish it. Peer review seldom detects fraud, or even mistakes. It is biased against women and against less famous institutions. Its benefits are statistically insignificant and its risks – academic log-rolling, suppression of unfashionable ideas, and the irresistible opportunity to put a spoke in a rival's wheel – are seldom examined.

This didn't matter much when peer review was just a private game for academics. If it made them feel better, that was nice. But in a sinister development, people who have published provocative or implausible claims in peer- reviewed journals have started arguing that they won't listen to criticism unless it has undergone the same laying-on of hands.

The most notorious example of this comes from the authors of The Spirit Level, a book that argues that everything from health to the level of crime, and from happiness to the quality of the toast, is better in societies that are more equal.

Professors Kate Pickett and Richard Wilkinson may well be right in this claim, though I dare say that we can all think of a few exceptions.

What isn't right, however, is their claim that as "almost all" the research that went into The Spirit Level was peer-reviewed, all future debate on the subject should take place in peer-reviewed publications. Come off it, profs, you're pulling our leg, surely? If you're writing a book about a hugely political subject such as inequality, you've surrendered any right to hide behind the flak-jacket of peer review.

Some of the criticism will doubtless be ill-informed, politically motivated, or just plain vulgar. But if the claim is true, it can see off such attacks. If it's untrue, peer review won't help to save it.

Pickett and Wilkinson aren't alone, though. A few months ago, I wrote in this column about a study in The Lancet of which I took a dim view. It claimed that Caesarean operations undertaken without medical cause were nearly three times as likely to cause death or complications to the mother as a normal birth. The evidence fell miles short of proving this claim.

Others shared my view, and one of them, Penny Christensen of Birth Trauma Canada, complained to The Lancet. Rebuffed, she sent them my analysis to support her claim. In reply, an editor wrote to her with the disparaging remark: "We are a scientific journal, and as such prefer to see the scientific debate continued by reference to other academic articles that have been peer-reviewed."

Ms Christensen's complaint has now gone to The Lancet's ombudsman, and we'll see if he shares the attitude that only the peer-reviewed are entitled to have their opinions properly considered.

If it's any help to him or to the authors of The Spirit Level, here's my view. Hiding behind peer review is a betrayal of all the principles of academic life, of open dialogue, of freedom of expression and, since the two profs are so keen on it, of equality as well. It's a disgrace.

Nigel Hawkes is director of Straight Statistics (straightstatistics.org)

React Now

Latest stories from i100
Have you tried new the Independent Digital Edition apps?
iJobs Job Widget
iJobs General

BI Manager - £50,000

£49000 - £55000 per annum + competitive: Progressive Recruitment: My client is...

BI Project Manager - £48,000 - £54,000 - Midlands

£48000 - £54000 per annum + Benefits package: Progressive Recruitment: My clie...

VB.Net Developer

£35000 - £45000 per annum + competitive: Progressive Recruitment: If you're pa...

SAP Business Consultant (SD, MM and FICO), £55,000, Wakefield

£45000 - £55000 per annum + competitive: Progressive Recruitment: SAP Business...

Day In a Page

Read Next
 

The law is too hard on sexting teenagers

Memphis Barker
 

Obama must speak out – Americans are worried no one is listening to them

David Usborne
Screwing your way to the top? Good for Lana Del Rey for helping kill that myth

Screwing your way to the top?

Good for Lana Del Rey for helping kill that myth, says Grace Dent
Will the young Britons fighting in Syria be allowed to return home and resume their lives?

Will Britons fighting in Syria be able to resume their lives?

Tony Blair's Terrorism Act 2006 has made it an offence to take part in military action abroad with a "political, ideological, religious or racial motive"
Beyoncé poses as Rosie the Riveter, the wartime poster girl who became a feminist pin-up

Beyoncé poses as Rosie the Riveter

The wartime poster girl became the ultimate American symbol of female empowerment
The quest to find the perfect pair of earphones: Are custom, 3D printed earbuds the solution?

The quest to find the perfect pair of earphones

Earphones don't fit properly, offer mediocre audio quality and can even be painful. So the quest to design the perfect pair is music to Seth Stevenson's ears
US Army's shooting star: Lt-Col Steven Cole is the man Hollywood calls when it wants to borrow a tank or check a military uniform

Meet the US Army's shooting star

Lt-Col Steven Cole is the man Hollywood calls when it wants to borrow a tank or check a military uniform
Climate change threatens to make the antarctic fur seal extinct

Take a good look while you can

How climate change could wipe out this seal
Should emergency hospital weddings be made easier for the terminally ill?

Farewell, my lovely

Should emergency hospital weddings be made easier?
Man Booker Prize 2014 longlist: Crowdfunded novel nominated for first time

Crowdfunded novel nominated for Booker Prize

Paul Kingsnorth's 'The Wake' is in contention for the prestigious award
Vladimir Putin employs a full-time food taster to ensure his meals aren't poisoned

Vladimir Putin employs a full-time food taster

John Walsh salutes those brave souls who have, throughout history, put their knives on the line
Tour de France effect brings Hollywood blockbusters to Yorkshire

Tour de France effect brings Hollywood blockbusters to Yorkshire

A $25m thriller starring Sam Worthington to be made in God's Own Country
Will The Minerva Project - the first 'elite' American university to be launched in a century - change the face of higher learning?

Will The Minerva Project change the face of higher learning?

The university has no lecture halls, no debating societies, no sports teams and no fraternities. Instead, the 33 students who have made the cut at Minerva, will travel the world and change the face of higher learning
The 10 best pedicure products

Feet treat: 10 best pedicure products

Bags packed and all prepped for holidays, but feet in a state? Get them flip-flop-ready with our pick of the items for a DIY treatment
Commonwealth Games 2014: Great Scots! Planes and pipers welcome in Glasgow's Games

Commonwealth Games 2014

Great Scots! Planes and pipers welcome in Glasgow's Games
Jack Pitt-Brooke: Manchester City and Patrick Vieira make the right stand on racism

Jack Pitt-Brooke

Manchester City and Patrick Vieira make the right stand on racism
How Terry Newton tragedy made iron men seek help to tackle their psychological demons

How Newton tragedy made iron men seek help to tackle their psychological demons

Over a hundred rugby league players have contacted clinic to deal with mental challenges of game