Scientific peer reviews are a 'sacred cow' ready to be slaughtered, says former editor of BMJ
The process is considered the gold standard of quality scientific research
The peer review process – long considered the gold standard of quality scientific research – is a “sacred cow” that should be slaughtered, the former editor of one of the country’s leading medical journals has said.
Richard Smith, who edited the British Medical Journal for more than a decade, said there was no evidence that peer review was a good method of detecting errors and claimed that “most of what is published in journals is just plain wrong or nonsense”.
Research papers considered for scientific and medical journals undergo a process of scrutiny by experts before they can be published. Hundreds of thousands of new studies are published around the world every year, and the peer review process exists to ensure that readers can have confidence that published findings are scientifically sound.
Join Independent Minds
For exclusive articles, events and an advertising-free read for just £5.99 €6.99 $9.99 a month
But Dr Smith said pre-publication peer review was slow, expensive and, perhaps ironically, lacking in evidence that it actually works in its chief goal of spotting errors.
Speaking at a Royal Society event earlier this week, he said an experiment conducted during his time at the BMJ, in which eight deliberate errors were included in a short paper sent to 300 reviewers, had exposed how easily the peer review process could fail.
“No-one found more than five, the median was two, and 20 per cent didn’t spot any,” he was quoted as saying by Times Higher Education. “If peer review was a drug it would never get on the market because we have lots of evidence of its adverse effects and don’t have evidence of its benefit.”
He said the process of peer review before publication could also work against innovative papers, was open to abuse, and should be done away with in favour of “the real peer review” of the wider scientific community post-publication.
“It’s time to slaughter the sacred cow,” he said, while acknowledging that do so would likely be “too bold a step” for a journal editor to take.
Dr Smith, who edited the BMJ between 1991 and 2004, is a longstanding critic of the pre-publication peer review process. In the past he has bemoaned the delays that the process can bring, in some cases of more than two years, between a paper being completed and its final publication.
His comments come at a time of serious soul-searching within the scientific community, over the quality of much published research.
The editor of the second of the country’s two leading medical journals, Dr Richard Horton of The Lancet, wrote in an editorial earlier this month that “much of the scientific literature, perhaps half, may simply be untrue”, blaming, among other things, studies with small sample sizes, researchers’ conflicts of interest and “an obsession” among scientists for pursuing fashionable trends of dubious importance”.
“The apparent endemicity of bad research behaviour is alarming,” he wrote. “In their quest for telling a compelling story, scientists too often sculpt their data to fit their preferred theory of the world.”
Dr Horton also suggested reform of the peer review process – but to improve it, not scrap it; potentially with the introduction of incentives for scientists who peer review more critically.
Dr Trish Groves, the current head of research at BMJ said that, while peer review wasn’t perfect, it was “still the best way to help research funders, conference organisers, and journal editors decide which studies to support and disseminate and to help readers, the public, patients, and healthcare providers decide what evidence to use in decision making.”
Comments
Share your thoughts and debate the big issues
Please be respectful when making a comment and adhere to our Community Guidelines.
You can find our Community Guidelines in full here.
Please be respectful when making a comment and adhere to our Community Guidelines.
Community Guidelines
You can find our Community Guidelines in full here.
Follow comments
Vote
Report Comment
Subscribe to Independent Minds to debate the big issues
Want to discuss real-world problems, be involved in the most engaging discussions and hear from the journalists? Try Independent Minds free for 14 days.
Already registered? Log inReport Comment
Delete Comment
Subscribe to Independent Minds to debate the big issues
Want to discuss real-world problems, be involved in the most engaging discussions and hear from the journalists? Try Independent Minds free for 14 days.
Already registered? Log inAbout The Independent commenting
Independent Minds Comments can be posted by members of our membership scheme, Independent Minds. It allows our most engaged readers to debate the big issues, share their own experiences, discuss real-world solutions, and more. Our journalists will try to respond by joining the threads when they can to create a true meeting of independent minds. The most insightful comments on all subjects will be published daily in dedicated articles. You can also choose to be emailed when someone replies to your comment.
The existing Open Comments threads will continue to exist for those who do not subscribe to Independent Minds. Due to the sheer scale of this comment community, we are not able to give each post the same level of attention, but we have preserved this area in the interests of open debate. Please continue to respect all commenters and create constructive debates.