As Internet traffic reaches a peak of 220m words a second, total gridlock approaches collapse

EVERY DAY at about 5pm, Keith Mitchell watches to see if the Internet in Britain is about to collapse. That is the time when the flow of traffic between the 88 top-level Internet service providers in the UK reaches its peak - a vast 1.3 gigabites per second, equivalent to five copies of the Encyclopedia Britannica each second. Happily, every day so far the network has survived.

But Mr Mitchell, executive chairman of the London Internet Exchange (Linx), knows that it is only a matter of time before something has to be done. He has watched over the past five years as the volume of data passing has increased by a factor of 120.

Even at its quietest - at about 4am - the network is now 30 times busier than it was five years ago.

"Currently we could take 30 times more traffic than we are actually putting through," said Mr Mitchell yesterday, as Linx celebrated its fifth year. "But we're already talking to systems suppliers about getting faster stuff, because we know that in 18 months we will need something 10 times faster than we have now."

That problem is likely to intensify next year, as Microsoft and British Telecom start a new service which will allow businesses to access Microsoft programs, including its popular Word, Excel and Outlook products, via the Internet. The service, announced yesterday after months of speculation, will certainly lead to increased data traffic as companies abandon fixed locations, and fixed machines in offices with programs on hard discs, in favour of mobile computers which can dial in to dedicated centres holding data and programs.

"We can see it becoming very attractive to home users," said Jeremy Gittins, Microsoft UK's marketing manager. But how many people are likely to use it? Angus Fox, BT's product director, who is working with Microsoft on the pilot projects (starting within BT), said: "We have a group of people whose sole job is simply to estimate the required size of connections to the Internet." And what was their prediction for the takeup of this scheme by this time next year?

"We don't know," said Mr Fox. "We've seen a lot of predictions of explosive growth. But we need pilot schemes to give us some feedback."

Explosive growth is the real danger. Linx, which physically interconnects all the Internet service providers in the UK, could easily fall foul of any member's sudden success. If everyone suddenly tried to access one single site in the UK, it could prove disastrous as the "routers" that pass the data became overloaded.

But what is far more common is that the routers work fine, but that the computers at the far end, which hold the invidual files making up the web page, will collapse under the load of thousands or millions of simultaneous requests.

There are famous past cases: Genealogy.org, launched earlier this year, which expected 25 million requests a day and got 50 million per hour; Encyclopedia Britannica, whose britannica.com site opened in October and was overwhelmed by 10 million hits a day, 100 times over its capacity.

Last weekend, Callnet's website for its freephone-based Internet service, collapsed under a deluge of hits from people aiming to register online. Part of the reason for that collapse was that the site is very design- heavy, with graphics and various advanced elements.

Before its launch last week, spokesman Chris Klopper dismissed as "bullish" predictions of 200,000 users by Christmas, and was confident the company could cope with demand.

But over the weekend Callnet's computers could not get the data out of the door quickly enough. People got no page at all, or it arrived slowly, piece by piece. Only on Monday did it come back to service as people returned to work. Stunned, the company admitted quietly that the response had been five times greater than its most optimistic estimates.

Had Callnet's site instead been designed like most individuals' home pages - with a little text and minimal graphics - it would have been able to handle many more users. As it was, some were suspicious that there was no encryption when their credit card details were requested - but that would have put even more load on the web server.

One solution to the problem of growing demand is to install bigger, faster computers as web servers, able to shift files more quickly to the telephone lines (whose capacity far exceeds their present demand). "The trouble is that you might get your estimate badly wrong and pay for a huge machine which nobody visits," noted Mr Mitchell.

An alternative is "cacheing", in which computers along the network hold exact copies of pages held elsewhere. Freeserve, for example, can dedicate a machine to holding a copy of the American web pages most frequently accessed by its exclusively British users. Key the US page, and you actually get the one stored by Freeserve.

This is like "mirroring", in which the remote site organises other computers to hold copies, and tells users to visit those rather than the main one. Nasa has done this to good effect: three major companies now offer global mirroring services.

Few events are capable now of slowing down the growth in the use of the Internet. We log in more often, for longer: BT has noticed that the length of the average local phone call has increased from a couple of minutes in the 1980s (prompting it to encourage more use with the slogan "It's Good To Talk") to more than seven now. The principal increase has come from Internet usage, with the average British home user being online for 17 minutes every day.

The only time this year that Linx has seen a slowdown in web traffic beyond the normal daily ebb and flow was on August 11 - when between 11am and 11.30am data traffic fell by 25 per cent - because everyone was looking out of the window at the eclipse.

The problem is that even the best-designed site will collapse if its owners seriously underestimate the number of people trying to access it. But gauging that is impossible, points out Mr Mitchell, when the only deterrent to visitors is frustration.

"There's an important economic point: if you give something away for free, it's very difficult to regulate the demand. You only control the supply. But if the acquirer has to pay even a little amount, a tiny amount, that creates a back-pressure which limits demand naturally."

What that would need is an Internet scheme supporting "micropayments" of perhaps a hundredth of a penny to access a website. But though long promised, this has never succeeded: the web pages promoting them remain quiet.

Meanwhile, Mr Mitchell is looking forward to New Year's Eve.

"We don't expect people will want to spend Christmas in front of a computer rather than with their relatives," he said. "But it will definitely be interesting to see what happens over the millennium."

TRIED AND TESTED - SIX OF THE MOST POPULAR SITES

altavista.com

Fast and efficient search engine, reckoned to cover more than any other - although the same scientists say this only amounts to 16 per cent of the entire Web. Well laid-out home page encourages searching within defined areas of interest.

Time to download opening Web page: 50 seconds

ebay.com

The first and most famous of the auction sites - you can bid for anything from crockery to computers. Anyone with a credit card can take part, but make sure you read the instructions carefully first.

Download time: 90 seconds

britannica.com

The famous encyclopaedia turned electronic and free. Suffered from total overload in early days, but open for business now. Competent layout, numerous star-rated links to other sites. Homework will never be the same again.

Download time: 75 seconds

hotmail.com

Bought from its founders in 1997 by Microsoft, this free Web-based e-mail has done more than most to resuscitate letterwriting. Microsoft's wealth of experience shines through here.

Download time: 15 seconds

genealogy.com

Family-tree site with American slant. Looks old-fashioned and lacks features such as a hit counter on its search facility. Reputedly one of the most popular non-commercial sites in the world.

Download time: 70 seconds

bol.com

Internet bookshop, and a would-be rival to amazon.com. Can flag books that might interest you, and its easy-to-use format will be familiar to anyone who has used the Net before.

Download time: 135 seconds