The spies can't cope with information overload

The National Security Agency can no longer get the goods it once routinely procured

The crescent moon hangs high in the eastern Silicon Valley sky: Venus shines bright, unblinking just below. The air is chill as three figures step out into the pre-dawn gloom and jog up Sand Hill Road, the Wall Street of hi-tech. Cassie the dog is, as usual, in the lead, her blinking LED collar marking the way. Spouse Linda jogs a few steps behind while I'm in my customary middle position. It's 6am.

The crescent moon hangs high in the eastern Silicon Valley sky: Venus shines bright, unblinking just below. The air is chill as three figures step out into the pre-dawn gloom and jog up Sand Hill Road, the Wall Street of hi-tech. Cassie the dog is, as usual, in the lead, her blinking LED collar marking the way. Spouse Linda jogs a few steps behind while I'm in my customary middle position. It's 6am.

People who work in the Valley's start-ups often exercise either late or early - the work days are usually long, and leave little time for life's other pursuits. As we jogged in silence, I wondered what was on my companions' minds. Linda is probably thinking about her coming weekend trip to New Mexico to visit her old college room-mate. Cassie, doubtless, is contemplating the bowl of food that awaits her back at gulker.com's world headquarters.

I'm thinking about crypto. In particular, I'm thinking about the Seymour Hersh article that's just appeared in The New Yorker, which claims that America's shadowy, super-spook outfit, the National Security Agency, has totally lost it.

The NSA, which once employed 95,000 people, had famously kept generations of American leaders ahead of their foreign counterparts. They bugged everything from the Russian Embassy's Xerox machine (which handily microfilmed every copied document) to all the planet's communications satellites. They were to America in the Fifties and Sixties what Bletchley Park's codebreakers had been to the Second World War allies. US leaders often knew what the other guys were going to do before they did it thanks to the NSA's Sigint, short for "Signals Intelligence".

Our spooks were supposed to be able to pick up everything, from Fax and phone calls to e-mail, and somehow filter it down to the important stuff using powerful computers and ingenious software created by the world's best mathematicians and programmers. The NSA even recently filed for a patent on its speech recognition software.

Its Echelon system was said to be an open window on to all that transpired in commerce and politics. Or was it? Hersh claims that the advent of strong cryptography and fibre optics has rendered the NSA useless and impotent. India's nuclear test was a complete surprise to the US: Iraq routinely baffles the US and UN weapons teams and terrorist suspect Osama bin Laden dodges every effort to bring him in, mainly because the NSA can no longer get the goods it once routinely procured.

Hersh's story, reprinted online by the cryptography site Cryptome, has unleashed a buzz on the crypto-oriented Net. Some claim the story is just another piece of NSA disinformation aimed at deflecting public attention from its real skullduggery. Others aren't so sure.

Many knowledgeable observers have noted that, in the never-ending race between code-makers and code-breakers, the code-makers have lately been pulling far ahead. Simon Singh's latest tome, The Code Book, traces cryptography from the Romans to modern public-key systems, and makes the point that the technology of certain epochs has favoured one side and then another.

But never have the code breakers been at such a loss. True, there have been some well-publicised attacks on encryption in the last few months, including one Net-based project that used screensavers on thousands of computers to break the relatively low-level encryption that the US Government, for one, would have allowed in its citizen's hands.

But world governments and terrorists have no plan whatsoever to use low-level crypto. They use the real thing: industrial-strength, long-key encryption. And they send those messages over fibre-optic lines that are much harder to tap than old-fashioned copper wires and radio transmissions. It's just this inability to cope that has led to the US government's ill-starred efforts to outlaw strong crypto. If we can't break their codes, we'll tell them they can't use codes. Sort of like shaking a finger at a machine-gun-toting terrorist and saying: "Don't you dare fire that thing!"

Pandora's box is open. Anyone can download free crypto software that should defy the best efforts of code breakers for the next decade or so.

And strong crypto may not be the biggest issue the NSA faces. Bletchley Park's legendary code-breakers realised their biggest problem was not necessarily the German Enigma machines but the sheer volume of intercepts. The Germans generated thousands of messages daily. Most were routine but occasionally the haystack held a precious needle upon which the success of the Allied war effort might hang.

Today, the problem is much worse. Global prosperity has built a system where torrents of information flow. A single telco switch in London or Tokyo may hourly handle more traffic than moved in a year during the Second World War. The volume hides a single message better than the ciphers of generations past. Indeed, one new scheme that relies on hiding bits of plaintext messages in the Internet's flow, may prove as secure as any advanced cipher.

As I watch the dawn redden the sky over nearby Stanford University, home of many modern crypto breakthroughs, I can't help wondering if the world will be worse, or better, if private things stay private.

cg@gulker.com