After four years of copying and scanning, the tobacco industry's eight million secrets are out

In an anonymous building on an industrial estate in Guildford, Surrey, lie clues to the causes of one of the world's worst epidemics. The two-storey concrete and glass structure, bristling with security cameras, contains eight million documents relating to British American Tobacco, which sells cigarettes in 180 countries.

The documents allegedly expose its involvement in smuggling and price-fixing, and reveal the tactics it allegedly has used to conceal the damage done by products that kill one in every two of their users. Yet, despite a 1998 court ruling in Minnesota, which ordered BAT to grant public access to the depository, researchers claim the company has obstructed access and allegedly tampered with the contents of files in a way which the researchers say amounts to concealment.

Now, for the first time, all eight million documents are to be made globally available, following a clandestine operation to copy and scan them on to the world wide web.

The four-year operation, called the Guildford Archiving Project, was revealed yesterday and has involved researchers from London, New York and San Francisco and cost almost £2m - funded by a group of medical charities including the Wellcome Trust, Cancer Research UK and the American Heart Association.

Although most of the documents have still to be examined, important insights into the working of the tobacco industry have been gained. According to the researchers, an audio tape proposed that BAT market a "cheap cigarette" to "dirt poor little black farmers". Researchers say the tape was wiped, apparently accidentally but has since been replaced. Researchers fear some documents have been tampered with and others removed. One, discussing marketing to "illiterate low income 16-year-olds", was changed to include the less controversial age of 18.

The collection dates from the company's origins in the early 1900s up to 1995 and is recognised as an invaluable resource detailing how the tobacco industry has operated in the past and its plans for expanding into new markets. But, according to researchers, 181 files containing 36,000 pages appear to have gone missing since the depository opened in January 2000.

Visitorsmust ring a bell to gain admittance and are then led upstairs to the viewing room where, after being signed in, they can order files from an index held on two computers. Three security cameras are trained on them and the BAT staff who operate the depository sit behind a one-way mirror at the end of the room. The files must be searched manually and there are no photocopying facilities. Orders for copies take more than a year to process.

The company has gathered intelligence on visitors. Richard Hurt, from the Mayo Clinic in New York, writing in The Lancet today, said: "Depository visitor reports show that BAT apparently tracked the database searches of a visitor. The company also tracked the physical movement of visitors and, in at least one instance, noted the personal mobile phone use of a visitor."

Tobacco researchers from the London School of Hygiene and Tropical Medicine, who have led the operation to copy and scan the documents to make them available on the web, said: "The documents have proved vital to revealing the underhand tactics used by BAT to sell its cigarettes around the world and to undermine public health efforts to reduce their devastating health impact." One document setting out BAT's five-year plan from 1994-98 apparently explains the scale and importance of smuggling to the company's global operations. It says 6 per cent of world cigarette sales in 1993 were "DNP" (duty not paid - i.e. smuggled), with the highest rates in eastern Europe (13 per cent of total sales) and Africa (12 per cent) followed by Latin America (9 per cent) and Western Europe (7 per cent).

The document says: "A key issue for BAT is to ensure that the group's system-wide objectives and performance are given the necessary priority through the active and effective management of such business." Other documents allegedly show that BAT colluded with rivals to fix prices in 23 countries across Africa, Asia, the Middle East, Latin America and Europe. In Zimbabwe, BAT's failure to achieve profit growth is attributed to a "50:50 sharing agreement with Rothmans of the 2.4 billion annual market, thus discouraging any real management dynamics."

Further revelations of BAT's involvement in the former Soviet Union are expected next week with the publication of research based on documents in the Guildford depository which researchers claim will show how the company exploited the lack of experience of the fledgling government in Uzbekistan.

The Guildford depository, managed by BAT, is one of two to which the public was granted access under the terms of a legal settlement with the tobacco industry in 1998, for a period of 10 years. The other, in Minnesota, is managed by a legal firmand there have been no problems with access. Researchers believe that BAT has dragged its feet at Guildford in the hope that time would run out. Dr Kelley Lee, a researcher at the London School of Hygiene, said: "Within those eight million documents are stories we are determined to tell. The clock was ticking on this archive - after 2009 the door would have been shut for good."

Now the collection will be preserved on an independent website run by the University of California, to be launched in September. Anna Gillmore, a researcher from the London School of Hygiene, said: "If you are trying to control an epidemic, you have to understand they way it spreads. Just as malaria is spread by the mosquito, the tobacco epidemic is spread by the industry. You have to understand how the industry works."

In a statement yesterday, BAT said the Guildford depository was "never designed to work like a public library". It added: "The researchers' ability to publish so many pages shows that they can't have found access that difficult." BAT had no problem with its documents appearing on a website and the company had nothing to hide, it said.

After a four-year investigation into allegations of involvement in smuggling at BAT the Department of Trade and Industry decided not to take any action against the company.