Trust me, I'm a researcher

Hardly a week goes by without another new health scare. And, as the recent linking of HRT and breast cancer demonstrated, widespread panic can be the result. Not even our doctors are immune to the uncertainty, says Dr Margaret McCartney

If you ever get fed up with the endless exhortations in the media to eat more of something one week, followed by a warning to eat less of it the next, you are not alone. As a GP, I flick through my pile of health journals for a couple of hours each week. There is a lot of information and, frankly, no time to digest it.

If you ever get fed up with the endless exhortations in the media to eat more of something one week, followed by a warning to eat less of it the next, you are not alone. As a GP, I flick through my pile of health journals for a couple of hours each week. There is a lot of information and, frankly, no time to digest it.

In truth, I prioritise the things that really matter. There are two types of essential reading. There are papers that, although drier than dust, are useful and deserving of thought. And then there are the things that I hear about on the radio, on my way to work. These I have to follow up, because headlines are the best hint as to what I am going to get asked about at my surgery.

Unfortunately, though, there is scarcely any time to arm myself with the answers. Take, for example, the recent reporting of the Million Women Study in The Lancet, reporting the risk of breast cancer in women using HRT. Despite the ubiquity of e-mail, fax and telephones, doctors seldom have any Department of Health guidelines ready on their desks when the telephones start ringing with callers looking for advice. This, however, is understandable. The Department of Health probably heard the news at the same time as the rest of us, and a delay of 24 to 48 hours is usual in getting official guidance to health professionals.

Currently, the way that research is done and presented is greatly debated. "Publish or perish", although a cliché, is fact. Doctors and scientists are under enormous pressure to obtain grants for research studies. Contrary to popular belief, most researchers are not guaranteed funding. Instead, many lead a hand-to-mouth existence, with huge amounts of time spent filling in detailed grant-application forms. The competition for some funding, eg the prestigious Medical Research Council grants, is intense. This investment in time and effort often pays little dividend.

The pharmaceutical industry is a frequent sponsor of research. Without it, some research would probably never be done, and thus, new drugs would not be developed. But such sponsorship is not without its critics. There is concern that having an industry sponsor creates an inherent bias in the work, and that the researchers, who owe their living to it, may be under confidentiality agreements and may find total objectivity difficult.

Even if researchers declare a pecuniary interest in the funding of their work, research findings are naturally resistant to neat translation into a snappy, accurate headline. Statistics, particularly those looking at risk assessments, can be difficult to interpret and can be open to wide-ranging misinterpretation. A common problem is, for example, reporting that a treatment is likely to "double" a risk. The importance of this doubling "relative risk" depends on how big the "absolute risk" was to start off with (eg, one in a thousand) - it may be tiny.

The Cochrane Collaboration aims, as it says on its website, "to make up-to-date, accurate information about the effects of healthcare readily available worldwide". Access to the Cochrane library - which is free and available through the National Electronic Library for Health, www.nelh.nhs.uk - allows examination of both the absolute and relative risks of treatments and disease.

After research headlines die down, there is usually a period of examination of the paper, followed by correspondence in the journal discussing points of contention. After that, if the results are felt to be valid, the findings may be incorporated in a Cochrane review, or be used as part of the evidence for NICE (National Institute for Clinical Excellence), or, in Scotland, SIGN (Scottish Intercollegiate Guidelines Network) guidelines. It can be a long time before published research makes an impact on clinical care, a source of frustration to patients and clinicians.

The Royal Society, concerned about how the public can be misled by prematurely enthusiastic publicity, have this week announced that they have set up a working group to investigate and improve the way in which scientific research is publicised. The chair of the group, the vice- president of the Royal Society, Professor Sir Patrick Bateson, said: "The results of scientific research can have quite profound effects on public opinion and policy, so it is important that scientists practise the highest standards of professionalism and integrity when communicating their results.

"At present, scientists rely almost exclusively on the practice of peer review, in which other experts check the quality of results, analysis and interpretation before they are made public, as a safeguard against the communication of poorly conducted research."

Currently, before a research paper is published, it is usually given to two independent individuals with expertise in the field. Usually unpaid, these "peer reviewers" are asked to check the paper for its accuracy of conclusions and correct methodology. While this method helps to ensure that no obviously erroneous research slips into publication, it is not perfect. It is still possible for a paper to be peer-reviewed, published, and then found to be statistically inadequate - or even based on false results.

Sir Patrick continues: "Peer review has been criticised for being too secretive, conducted behind closed doors, and assessed by anonymous referees. It has also been suggested that it provides a way in which the establishment can prevent unorthodox ideas, methods and views, regardless of their merit, from being made public. We want to see if any evidence supports such a claim." In any case, citing the claims about the existence of human clones earlier this year, it is possible, the Royal Society says, to attract huge amounts of publicity without either peer review, or indeed any proof of research.

But peer review is not the only hurdle to publication. Another team may get their results out first, making similar findings redundant; researchers may change jobs, move away, or funding may come to an end. Additionally, it has been known to Cochrane reviewers for some time that studies showing a positive result were more likely to be published, a phenomena known as "publication bias". Negative or equivocal study results do not make it into print so often.

By examining all the current information available about a healthcare topic, and by assessing the validity of the evidence, Cochrane groups can inform people about the best in "evidence-based medicine". However, for this to work, they need to be able to assess all the studies done, including studies that were never published.

Attempting to combat this, several years ago, a collective of medical-journal editors proposed "an amnesty for unpublished trials". The idea was that all researchers having unpublished trial results should place them in a database, which could be used to inform future researchers, and to contribute towards informing doctors and patients about the best current evidence for treatments. A year on, the British Medical Journal declared that, in terms of absolute numbers of research studies submitted, the exercise had been "a flop". To minimise this problem continuing in the future, the NHS Research Governance Framework now requires all clinical trials to be registered at inception, and there has been progress in making this information publicly available through www.controlled-trials.com.

However, most pharmaceutical companies have so far paid little attention either to registration, or to the Good Publication Practice guidelines produced from within the industry (www.gpp-guidelines.org). This is of concern to Sir Iain Chalmers, director of the UK Cochrane Centre, who has been campaigning for many years for greater openness in clinical research. "Patients have agreed to be in trials in the belief that they are furthering scientific knowledge. It is disgraceful that the industry is not reporting all the clinical trials. With electronic publication available, there can be no excuse for failing to publish work.

"What are these companies scared of? Is their concern for patients or for shareholders? I think that potential participants in clinical trials should not become involved unless they are assured - in writing - that the existence of the trial has been registered publicly, and the data to which they will contribute will be made publicly available, promptly, on completion of the study."

I was looking up the Million Women HRT paper on the internet for a lady who wanted some advice about her HRT. I read through the statistics, which confirmed my instinct that she would be better coming back to see me when I had the official guidance in hand. "That's fine, she said. "But you know, I don't believe everything I read or hear." She is probably wise; a little cynicism might be just what the doctor ordered.