Nigel Hawkes: Peer-reviewed journals aren't worth the paper they're written on

Behind The Numbers

Share
Related Topics

Academic journals like to pride themselves on peer review, the process of sending out papers to other experts in the field for pre-publication checks.

"Is it in a peer-reviewed journal?" journalists are meant to ask themselves before launching into another story about rice pudding causing cancer, or chocolate prolonging life.

The truth is that peer review is largely hokum. What happens if a peer-reviewed journal rejects a paper? It gets sent to another peer-reviewed journal a bit further down the pecking order, which is happy to publish it. Peer review seldom detects fraud, or even mistakes. It is biased against women and against less famous institutions. Its benefits are statistically insignificant and its risks – academic log-rolling, suppression of unfashionable ideas, and the irresistible opportunity to put a spoke in a rival's wheel – are seldom examined.

This didn't matter much when peer review was just a private game for academics. If it made them feel better, that was nice. But in a sinister development, people who have published provocative or implausible claims in peer- reviewed journals have started arguing that they won't listen to criticism unless it has undergone the same laying-on of hands.

The most notorious example of this comes from the authors of The Spirit Level, a book that argues that everything from health to the level of crime, and from happiness to the quality of the toast, is better in societies that are more equal.

Professors Kate Pickett and Richard Wilkinson may well be right in this claim, though I dare say that we can all think of a few exceptions.

What isn't right, however, is their claim that as "almost all" the research that went into The Spirit Level was peer-reviewed, all future debate on the subject should take place in peer-reviewed publications. Come off it, profs, you're pulling our leg, surely? If you're writing a book about a hugely political subject such as inequality, you've surrendered any right to hide behind the flak-jacket of peer review.

Some of the criticism will doubtless be ill-informed, politically motivated, or just plain vulgar. But if the claim is true, it can see off such attacks. If it's untrue, peer review won't help to save it.

Pickett and Wilkinson aren't alone, though. A few months ago, I wrote in this column about a study in The Lancet of which I took a dim view. It claimed that Caesarean operations undertaken without medical cause were nearly three times as likely to cause death or complications to the mother as a normal birth. The evidence fell miles short of proving this claim.

Others shared my view, and one of them, Penny Christensen of Birth Trauma Canada, complained to The Lancet. Rebuffed, she sent them my analysis to support her claim. In reply, an editor wrote to her with the disparaging remark: "We are a scientific journal, and as such prefer to see the scientific debate continued by reference to other academic articles that have been peer-reviewed."

Ms Christensen's complaint has now gone to The Lancet's ombudsman, and we'll see if he shares the attitude that only the peer-reviewed are entitled to have their opinions properly considered.

If it's any help to him or to the authors of The Spirit Level, here's my view. Hiding behind peer review is a betrayal of all the principles of academic life, of open dialogue, of freedom of expression and, since the two profs are so keen on it, of equality as well. It's a disgrace.

Nigel Hawkes is director of Straight Statistics (straightstatistics.org)

React Now

Latest stories from i100
Have you tried new the Independent Digital Edition apps?
SPONSORED FEATURES
iJobs Job Widget
iJobs General

Recruitment Genius: Office / Sales Manager

£22000 - £32000 per annum: Recruitment Genius: Established and expanding South...

Recruitment Genius: Administrative Assistant / Order Fulfilment

£14000 per annum: Recruitment Genius: An exciting opportunity to join a thrivi...

SThree: Trainee Recruitment Consultant

£18000 - £23000 per annum + Uncapped OTE: SThree: Trainee Recruitment Consulta...

SThree: Trainee Recruitment Consultant

£18000 - £23000 per annum + Uncapped OTE: SThree: Trainee Recruitment Consulta...

Day In a Page

Read Next
 

Syria's Kurds have little choice but to flee amid the desolution, ruins and danger they face

Patrick Cockburn
A bartender serves two Mojito cocktails  

For the twenty-somethings of today, growing up is hard to do

Simon Kelner
Refugee crisis: David Cameron lowered the flag for the dead king of Saudi Arabia - will he do the same honour for little Aylan Kurdi?

Cameron lowered the flag for the dead king of Saudi Arabia...

But will he do the same honour for little Aylan Kurdi, asks Robert Fisk
Our leaders lack courage in this refugee crisis. We are shamed by our European neighbours

Our leaders lack courage in this refugee crisis. We are shamed by our European neighbours

Humanity must be at the heart of politics, says Jeremy Corbyn
Joe Biden's 'tease tour': Could the US Vice-President be testing the water for a presidential run?

Joe Biden's 'tease tour'

Could the US Vice-President be testing the water for a presidential run?
Britain's 24-hour culture: With the 'leisured society' a distant dream we're working longer and less regular hours than ever

Britain's 24-hour culture

With the 'leisured society' a distant dream we're working longer and less regular hours than ever
Diplomacy board game: Treachery is the way to win - which makes it just like the real thing

The addictive nature of Diplomacy

Bullying, betrayal, aggression – it may be just a board game, but the family that plays Diplomacy may never look at each other in the same way again
Lady Chatterley's Lover: Racy underwear for fans of DH Lawrence's equally racy tome

Fashion: Ooh, Lady Chatterley!

Take inspiration from DH Lawrence's racy tome with equally racy underwear
8 best children's clocks

Tick-tock: 8 best children's clocks

Whether you’re teaching them to tell the time or putting the finishing touches to a nursery, there’s a ticker for that
Charlie Austin: Queens Park Rangers striker says ‘If the move is not right, I’m not going’

Charlie Austin: ‘If the move is not right, I’m not going’

After hitting 18 goals in the Premier League last season, the QPR striker was the great non-deal of transfer deadline day. But he says he'd preferred another shot at promotion
Isis profits from destruction of antiquities by selling relics to dealers - and then blowing up the buildings they come from to conceal the evidence of looting

How Isis profits from destruction of antiquities

Robert Fisk on the terrorist group's manipulation of the market to increase the price of artefacts
Labour leadership: Andy Burnham urges Jeremy Corbyn voters to think again in last-minute plea

'If we lose touch we’ll end up with two decades of the Tories'

In an exclusive interview, Andy Burnham urges Jeremy Corbyn voters to think again in last-minute plea
Tunisia fears its Arab Spring could be reversed as the new regime becomes as intolerant of dissent as its predecessor

The Arab Spring reversed

Tunisian protesters fear that a new law will whitewash corrupt businessmen and officials, but they are finding that the new regime is becoming as intolerant of dissent as its predecessor
King Arthur: Legendary figure was real and lived most of his life in Strathclyde, academic claims

Academic claims King Arthur was real - and reveals where he lived

Dr Andrew Breeze says the legendary figure did exist – but was a general, not a king
Who is Oliver Bonas and how has he captured middle-class hearts?

Who is Oliver Bonas?

It's the first high-street store to pay its staff the living wage, and it saw out the recession in style
Earth has 'lost more than half its trees' since humans first started cutting them down

Axe-wielding Man fells half the world’s trees – leaving us just 422 each

However, the number of trees may be eight times higher than previously thought
60 years of Scalextric: Model cars are now stuffed with as much tech as real ones

60 years of Scalextric

Model cars are now stuffed with as much tech as real ones