Half of us carry a microbe that can lead to gastritis and cancer ? and it's been with us for millennia.

Imagine if a new kind of microbe sweeps the planet, infecting half the human population. The bug leaves many people unaffected, but for millions it causes chronic ill health. For thousands more, the illness caused by the bacteria develops into fatal cancer.

Imagine if a new kind of microbe sweeps the planet, infecting half the human population. The bug leaves many people unaffected, but for millions it causes chronic ill health. For thousands more, the illness caused by the bacteria develops into fatal cancer.

This is not a fictional "killer bug" but one that is already with us and has been for more than 150,000 years. The bacterium in question, Helicobacter pylori, was only discovered 20 years ago and only since last year has there been any hope of developing a vaccine against it. Helicobacter has the unique ability to colonise the human stomach in spite of the harsh acidity meant to disinfect incoming food. The bacterium has been accompanying human beings since they first emigrated from Africa to populate the world, and is now present in one in two people on the planet.

Although harmless to many people, for millions it causes suffering in the form of stomach ulcers. Each year many thousands die from cancer caused by the persistent presence of the bacteria. Despite this, few people take much notice of Helicobacter, and its name does not strike the same fear as Ebola virus, for instance, yet it causes far more suffering and death.

Misconceptions dating back to Roman times have something to do with this. The belief that gastric ulcers are caused by excess acid production in the stomach has held for two millennia. In the first century AD, for instance, the Roman physician Celsus recommended minimising acidic food to fight ulcers. This must be one of the longest-surviving documented scientific errors in the history of civilisation.

Today we know that excess acid contributes to the symptoms, so the efficient acid-reducing drugs available since the 1970s do indeed alleviate suffering, but as this is not the underlying cause, the symptoms return as soon as the medication is stopped. This, of course, provides continued revenues for doctors and drug companies, but leaves the patient without a cure.

It was only in the 1980s that the Australian pathologist J Robin Warren and his colleague Barry Marshall got their hands on the real culprit. They found strange bacteria, coiled up like a spiral staircase, living in the mucus layer lining the stomach. Unlike earlier observers, who dismissed these as contaminants, they developed a method of cultivating the microbes – the first step towards an understanding of their living habits and infectivity.

After they published the finding in 1983, it was quickly confirmed by many other researchers. The bacteria, later named Helicobacter pylori, appeared in the stomachs of many healthy people, and of all patients with chronic superficial gastritis, an inflammatory condition that can lead to ulcers or cancers of the stomach. By voluntarily infecting themselves with H. pylori, Marshall and another healthy volunteer demonstrated that the bacterium does indeed cause the disease. (Knowing this, they could stop it from becoming chronic in their stomachs by taking antibiotics.)

Subsequent research showed that infected people develop antibodies to the bacteria, which can be detected much more easily than the bug itself. With an approach similar to HIV testing, epidemiologists set out to test blood samples for Helicobacter antibodies – and got some frightening results. It turned out that in developing countries, between 70 and 90 per cent of people carry the infection, and most acquire it in childhood before the age of 10. Although the exact mechanisms of infection have remained unclear, it appears that living in a rich country helps to avoid infection; the rates of infection are about 30 per cent in Europe and North America.

But how can a bacterium survive in an environment as hostile as the human stomach? It has many adaptations accounting for its spectacular success in colonising people. It has to be able to swim against the tide, to avoid ending up in the bowels and ultimately the toilet. With its corkscrew shape it can dig itself into the mucus layer lining the stomach walls. It knows a special chemical trick to neutralise the acidic environment; it has enormous amounts of an enzyme that converts urea (a side product of protein metabolism) into the alkaline chemical ammonia, which can neutralise the acid, in its immediate surroundings at least.

In recognition of its significance for human health, H. pylori was chosen to be among the first 10 microbes to have their entire genomes sequenced. It was decoded by The Institute for Genomic Research (TIGR) in Rockville, Maryland. After publication of the genome in 1997, many other subtle survival mechanisms were unravelled.

Apart from swimming, winding into the mucus and producing ammonia, the microbe has the ability to stick to the cells of the stomach lining. It can produce toxic substances that weaken cells and allow them to release ions and nutrients essential for the bacterium. Take away any of these functions, and the bacterium could not set up colonies in a human stomach.

So these factors are also promising targets for new treatments and vaccines. Antibiotics or bismuth preparations have already been shown to cure the gastritis H. pylori causes. And unlike the anti-acid treatments apparently still used by some doctors, their success is permanent – the symptoms will not come back after the treatment.

However, considering the number of people infected, a full treatment with antibiotics would be highly problematic, as it would contribute to the already fast-growing problem of antibiotic resistance genes, affect some of the "good" bacteria in patients' bowels, and be too expensive for efficient use in developing countries. So a vaccination against the microbe would be highly desirable and the only realistic way to tackle the bug on a large scale, with control or eradication the ultimate goal.

Animal trials of vaccines for both protective and therapeutic use have been successful. The next challenge is to formulate a vaccine for humans. Antex Biologics in Gaithersburg, Maryland, has already steered a candidate compound called Helivax through the first clinical trials. The drug is taken orally and is designed both to clear existing infections and prevent future ones. All going well, the quest to expel the bug from three billion stomachs should begin in a few years' time. After 150,000 years of suffering and 2,000 years of misguided treatment, it's about time.