Paul Vallely: Fact is, the obvious is usually the truth

We take refuge in conspiracy theories, like the 'murder' of Dr David Kelly, for the most understandable of reasons

Scholastic philosophy is always a good place to start. The 14th-century English thinker William of Ockham is best known for his insistence on a bias towards simplicity in the construction of theories. His technique is known as Ockham's Razor, for its emphasis on shaving away superfluous ideas. Go for simplest explanation first, he counselled.

It is not advice that has been widely followed with regard to the death of Dr David Kelly, the British weapons inspector whose body was found in woods in Oxfordshire in 2003 – after it was revealed he was the source for a BBC story suggesting that Tony Blair's government had "sexed up" the case for the invasion of Iraq. Despite the official verdict by the Hutton Inquiry that the scientist committed suicide, theories have abounded that he might have been murdered by agents of the state.

A group of prominent doctors, disquieted by the officially reported causes of death, have repeatedly campaigned for a formal inquest into Dr Kelly's demise. Last week, the Government responded by publishing the pathologist's report that Lord Hutton had decreed should be classified for 70 years, out of sensitivity to the dead man's family. Now we have seen it there seem even fewer grounds for disputing that the scientist died after slashing his left wrist and taking an overdose whose effect was exacerbated by an undiagnosed heart condition. The new report was "convincing", said one of the campaigning doctors, Julian Bion, a professor of intensive care medicine. He was "certainly satisfied" that the cause of death given was the correct one.

Not everyone agreed. If Dr Kelly's death was consistent with a verdict of suicide, "then it must also be consistent with murder made to look like suicide", one conspiracist swiftly proclaimed. The bloggers were off: not enough blood, no fingerprints on knife, aversion to swallowing pills, missing dental records, mystery man in black and boat seen, and the rest.

Any reasonable person would concede that there are a number of loose ends. But to jump from that to death squads in the employ of the British prime minister is another matter. Ockham's law of logical parsimony recommends that, faced with a mystery, we select the hypothesis that introduces the fewest external assumptions and factors. But modern culture has a consistent reluctance to believe the obvious.

The White House was behind 9/11 as a pretext for invading Afghanistan. The Moon landings were codded up by Nasa in a studio. Princess Diana was murdered by the royal family. The Mafia shot John F Kennedy. Fifty of another US president's associates were bumped off in the Clinton Body Count. Global warming is got up by scientists to win extra research funds. Aids came from a virus genetically engineered by the US government to wipe out blacks and homosexuals.

There are conspiracies involving the Jews, Freemasons and the Catholic Church taking over the world. Or wild fantasies such as Dan Brown's Da Vinci Code theory that for 2,000 years the Church has been hiding the fact that Jesus secretly married and has a line of descendants. Or David Icke's lurid phantasmagoria that humanity is controlled by a secret group of shape-shifting reptilian aliens who include George Bush, Queen Elizabeth II, Kris Kristofferson and Boxcar Willie, all of whom drink human blood to maintain their humanoid appearance.

It is easy to dismiss conspiracy theories as the baggage of the freakish fringe. But polls suggest otherwise. A BBC survey showed that 22 per cent of Britons do not think Kelly killed himself. Some 43 per cent of Americans believe in UFOs. In our over-stimulated information age conspiracy theories are what Christopher Hitchens has called "the exhaust fumes of democracy". Some go further: the American political scientist Michael Barkun suggests that conspiracy is replacing democracy as the dominant political paradigm.

There are a number of reasons for the rise and rise of contemporary theories. The age of political deference is dead. Those in authority are no longer automatically trusted by virtue of their position. People are creating their own patterns of meaning.

In one sense we have always done this. The human mind is programmed to make shapes out of chaos, and create significance from the random. That is the origin of myth and human story. Just because conspiracy theories are factually inaccurate does not mean that they are not reaching after a different kind of truth. It is like the story of the man who wakes to find his wife hitting him because she has dreamt he has had an affair. "But I haven't done anything; it was your dream," he protests. "No," she replies, "but it's the kind of thing you would do." A conspiracy theory can tell us something about how modern men and women see the world and what alarms, or gives them comfort, in it.

The world is largely moulded by impersonal economic and institutional forces. But the human instinct is to see events shaped by individuals and their acts. That tendency is amplified when it comes to the forces we feel are ranged against us, most particularly in those, like the establishment, we feel more powerful than us.

So we personify it. "The enemy is clearly delineated: he is a perfect model of malice, a kind of amoral superman – sinister, ubiquitous, powerful, cruel, sensual, luxury-loving," wrote the historian Richard Hofstadter in his essay "The Paranoid Style in American Politics". And such enemies act in consort. They conspire to start runs on banks, cause depressions, create disasters, and then enjoy and profit from the misery they have produced. "The paranoid's interpretation of history is distinctly personal: decisive events are not taken as part of the stream of history, but as the consequences of someone's will," Hofstadter writes. Just because they're paranoid, doesn't mean someone's not out to get them, and it turns politics into an arena for angry minds. Karl Popper saw the totalitarianism of fascism and communism as rooted in the paranoid tribalism of conspiracy theory.

But it is manifest at far more trivial levels. The Rooney saga, one Man Utd fan, opined yesterday at Old Trafford, was "a conspiracy by Fergie to force the club's owners to buy some more top players". Conspiracy is fed by the powerlessness of the theorist. But real history is driven more by cock-up than conspiracy. Cartels are unnecessary when prices can be fixed by congruence of interests – which is why every chip van near Old Trafford charges an outrageous £2.50 for a small tray. Did the chip men get together to fix the price? We shall never know. Conspiracy is the lazy way out.

The poet Keats offers a more interesting option. What we need, he said is "negative capability – that is, when a man is capable of being in uncertainties, mysteries, doubts, without any irritable reaching after fact and reason". Keats, it is generally reported, died from tuberculosis. So why does his tombstone talk about "the Malicious Power of his Enemies"? Eh? I've got a bit of a theory about that.