'Terrorist Facebook' – the new weapon against al-Qa'ida

Intelligence agencies are building up a Facebook-style databank of international terrorists in order to sift through it with complex computer programs aimed at identifying key figures and predicting terrorist attacks before they happen.

By analysing the social networks that exist between known terrorists, suspects and even innocent bystanders arrested for being in the wrong place at the wrong time, military intelligence chiefs hope to open a new front in their "war on terror".

The idea is to amass huge quantities of intelligence data on people – no matter how obscure or irrelevant – and feed it into computers that are programmed to make associations and connections that would otherwise be missed by human agents, scientists said.

Related articles

• Steve Connor: Run the program, join the dots and find the terrorist
• Taliban steps up election terror campaign
• Search the news archive for more stories

The doctrine is already being actively pursued in Iraq and Afghanistan where thousands of people have been arrested and interrogated for information that could be fed into vast computerised databanks for analysis by social network programs.

In addition to information gleaned from interviews with suspects captured in the field, intelligence agencies are also mining the vast amounts of telecommunications data collected from emails and telephone calls with the same surveillance technology. In the US alone, hundreds of millions of dollars are being spent on developing the data-mining techniques.

"Social network analysis is analysing information about who knows who or who talks to whom," said Professor Kathleen Carley of Carnegie Mellon University in Pittsburgh, Pennsylvania, one of the civilian scientists hoping to benefit from the new military funding earmarked for research into social network analysis.

"Facebook and Google are doing social networking, which is the technology for helping you find out who to talk to and for finding out what your friends know about a person," Professor Carley said. "What social network analysis is about is giving me the whole of the 'Facebook-style' data and saying that I'm going to analyse it mathematically to tell you who the critical people are," she said.

The doctrine, however, has been criticised as time consuming, wasteful and counterproductive. Critics have also suggested that it has led to gross violations of human rights, with hundreds and possibly thousands of innocent people being detained and interrogated for longer than necessary to provide social network information.

In its most extreme form, the doctrine has led to what is known within US military circles as the "mosaic philosophy". The philosophy behind the mosaic theory is that a piece of intelligence data may not mean anything to the interrogator or even the person who is being interrogated but it can suddenly seem relevant and crucial when placed as a "tile on the mosaic," he said.

It has led, the critics argue, to the arrest and interrogation of many thousands of innocent people in Iraq and Afghanistan in the hope of gleaning any titbits of intelligence that could be fed into computers programmed with social-network algorithms.

"It's not a new philosophy, but computers and data processing have given it a new impetus and a new emphasis," said Professor Lawrence Wilkerson, a retired US Army colonel and former chief of staff to the US Secretary of State Colin Powell until 2005. "You fuse little bits and pieces of information, which to the interrogator in the field are basically meaningless, but they come in and you put them together to paint this bigger picture," said Professor Wilkerson, who is critical of the doctrine.

"[The mosaic philosophy] is not incredibly well-known. It's arcane, it's esoteric, it's limited to a very few people," he told The Independent.

Joseph Margulies, professor of law at Northwestern University in Chicago, who has studied the mosaic philosophy in relation to the detainees at Guantanamo, said that the technological and mathematical developments in social network analysis go hand in glove with the rationale behind the mosaic theory. "The former feeds on the latter. It's the myth that the computer can know everything, the belief in the omnipotent algorithm, encouraging you to embrace for longer than necessary the mistakes of the mosaic theory," Professor Margulies said.

And the collation of vast databanks has another downside. "It also has the potential to bury you in inane data, where quantity is substituted for quality," Professor Margulies said. Nevertheless, senior intelligence officials as well as academic experts in social network analysis believe that terrorist cells can be monitored effectively by the techniques, especially in theatres of conflict such as Iraq and Afghanistan.

Dr Ian McCulloh, a US Army major at West Point Military Academy in New York, said that he has used social network analysis to work out relationships between the many hundreds of videos of American deaths filmed by insurgents in Iraq.

"The rationale for how they were related is classified so I can't give away methods [but] the interpretation was that the cluster of videos were likely to have been done by the same group... It allowed us to look at the structure between terrorist groups and actual attacks," he said.

Dr McCulloh is collaborating with Professor Carley on "metanetwork" analysis, a more sophisticated form of social network analysis. He hopes to be able to monitor terrorist networks in real time and detect any changes to indicate that an attack is imminent.

"Before a terrorist event is going to occur there is usually a change in that organisation as it begins to prepare and plan and resource the event. In that context I can monitor a network in real time and monitor the change in behaviour before an event occurs," Dr McCulloh said.

"Social network analysis is to old-fashioned detective work what statistics is to intuition. It's applying mathematical rigour to what people have done before," he said.

"It's already taken off in the military structure. Where it's going to go or how successful it's going to be, I'd be hesitant to say. Social network analysis is included in the counter-insurgency document of the US Army. It's in the vernacular and military intelligence people are using it," Dr McCulloh said.