Scientific peer reviews are a 'sacred cow' ready to be slaughtered, says former editor of BMJ

The process is considered the gold standard of quality scientific research

Richard Smith said reviews are slow, expensive and may not actually work
Richard Smith said reviews are slow, expensive and may not actually work

The peer review process – long considered the gold standard of quality scientific research – is a “sacred cow” that should be slaughtered, the former editor of one of the country’s leading medical journals has said.

Richard Smith, who edited the British Medical Journal for more than a decade, said there was no evidence that peer review was a good method of detecting errors and claimed that “most of what is published in journals is just plain wrong or nonsense”.

Research papers considered for scientific and medical journals undergo a process of scrutiny by experts before they can be published. Hundreds of thousands of new studies are published around the world every year, and the peer review process exists to ensure that readers can have confidence that published findings are scientifically sound.

But Dr Smith said pre-publication peer review was slow, expensive and, perhaps ironically, lacking in evidence that it actually works in its chief goal of spotting errors.

Speaking at a Royal Society event earlier this week, he said an experiment conducted during his time at the BMJ, in which eight deliberate errors were included in a short paper sent to 300 reviewers, had exposed how easily the peer review process could fail.

“No-one found more than five, the median was two, and 20 per cent didn’t spot any,” he was quoted as saying by Times Higher Education. “If peer review was a drug it would never get on the market because we have lots of evidence of its adverse effects and don’t have evidence of its benefit.”

He said the process of peer review before publication could also work against innovative papers, was open to abuse, and should be done away with in favour of “the real peer review” of the wider scientific community post-publication.

“It’s time to slaughter the sacred cow,” he said, while acknowledging that do so would likely be “too bold a step” for a journal editor to take.

Dr Smith, who edited the BMJ between 1991 and 2004, is a longstanding critic of the pre-publication peer review process. In the past he has bemoaned the delays that the process can bring, in some cases of more than two years, between a paper being completed and its final publication.

His comments come at a time of serious soul-searching within the scientific community, over the quality of much published research.

The editor of the second of the country’s two leading medical journals, Dr Richard Horton of The Lancet, wrote in an editorial earlier this month that “much of the scientific literature, perhaps half, may simply be untrue”, blaming, among other things, studies with small sample sizes, researchers’ conflicts of interest and “an obsession” among scientists for pursuing fashionable trends of dubious importance”.

“The apparent endemicity of bad research behaviour is alarming,” he wrote. “In their quest for telling a compelling story, scientists too often sculpt their data to fit their preferred theory of the world.”

Dr Horton also suggested reform of the peer review process – but to improve it, not scrap it; potentially with the introduction of incentives for scientists who peer review more critically.

Dr Trish Groves, the current head of research at BMJ said that, while peer review wasn’t perfect, it was “still the best way to help research funders, conference organisers, and journal editors decide which studies to support and disseminate and to help readers, the public, patients, and healthcare providers decide what evidence to use in decision making.”

Register for free to continue reading

Registration is a free and easy way to support our truly independent journalism

By registering, you will also enjoy limited access to Premium articles, exclusive newsletters, commenting, and virtual events with our leading journalists

Please enter a valid email
Please enter a valid email
Must be at least 6 characters, include an upper and lower case character and a number
Must be at least 6 characters, include an upper and lower case character and a number
Must be at least 6 characters, include an upper and lower case character and a number
Please enter your first name
Special characters aren’t allowed
Please enter a name between 1 and 40 characters
Please enter your last name
Special characters aren’t allowed
Please enter a name between 1 and 40 characters
You must be over 18 years old to register
You must be over 18 years old to register
Opt-out-policy
You can opt-out at any time by signing in to your account to manage your preferences. Each email has a link to unsubscribe.

By clicking ‘Create my account’ you confirm that your data has been entered correctly and you have read and agree to our Terms of use, Cookie policy and Privacy notice.

This site is protected by reCAPTCHA and the Google Privacy policy and Terms of service apply.

Already have an account? sign in

By clicking ‘Register’ you confirm that your data has been entered correctly and you have read and agree to our Terms of use, Cookie policy and Privacy notice.

This site is protected by reCAPTCHA and the Google Privacy policy and Terms of service apply.

Register for free to continue reading

Registration is a free and easy way to support our truly independent journalism

By registering, you will also enjoy limited access to Premium articles, exclusive newsletters, commenting, and virtual events with our leading journalists

Already have an account? sign in

By clicking ‘Register’ you confirm that your data has been entered correctly and you have read and agree to our Terms of use, Cookie policy and Privacy notice.

This site is protected by reCAPTCHA and the Google Privacy policy and Terms of service apply.

Join our new commenting forum

Join thought-provoking conversations, follow other Independent readers and see their replies

Comments

Thank you for registering

Please refresh the page or navigate to another page on the site to be automatically logged inPlease refresh your browser to be logged in