Iraq: The state we're in

When will peace return to Iraq? When will the terrifying cycle of bombings and suicide attacks abate? When will the people have food and electricity? When will they be able to walk the streets in safety? When will our sons and daughters return home? How will it all end? In this special report from Baghdad, The Independent's Patrick Cockburn exposes the monumental series of blunders that plunged the nation into chaos - and explains why the conflict will be longer, bloodier and more profound in its consequences even than Vietnam

The American and British governments seem disconnected from the terrible reality of Iraq. Tony Blair says the time scale for withdrawal is "when the job is done." But stop any Iraqi in the streets of Baghdad and the great majority say the violence will get worse until the US and Britain start to pull out. They say the main catalyst for the Sunni Arab insurrection is the US occupation.

A deep crisis is turning into a potential catastrophe because President George Bush and Tony Blair pretend the situation in Iraq is improving. To prove to their own voters that progress is being made, they have imposed on Iraq a series of artificial milestones. These have been achieved but have done nothing to halt the ever deepening violence.

The latest milestone is the referendum on the new constitution - the rules of the game by which Iraq is to be governed - on which Iraqi will vote tomorrow. The document was rushed through with the US and British ambassadors sitting in on the negotiations. The International Crisis Group, a highly regarded think-tank, warns that because the country's 5 million Sunni Arabs see the constitution as legitimising the break up of the country, the referendum will ensure that "Iraq will slide towards full-scale civil war and dissolution".

The real dilemmas facing Britain in Iraq today have been surprisingly little discussed. Rancorous though the debate has been, it revolves largely around historical events: Tony Blair's decision to go to war and the absence of Saddam Hussein's weapons of mass destruction. Supporters and opponents of British involvement have skirted the main issue - the reasons for staying or pulling out - perhaps because neither can work out what to do, aside from keeping their fingers crossed.

"The Americans should leave the cities and then the country according to a timetable negotiated with the Iraqi government," says Mahmoud Othman, a veteran Iraqi political leader. He argues that Iraqi Arabs in general want the occupation to end. Yet the expressed intention of the US to leave will divide the fanatical sectarian killers of al-Qa'ida in Iraq from militant Sunni nationalists.

The need for the White House to produce a fantasy picture of Iraq is because it dare not admit that it has engineered one of the greatest disasters in American history. It is worse than Vietnam, because the enemy is punier and the original ambitions greater. At the time of the invasion in 2003 the US believed it could act alone, almost without allies, and win. In this it has utterly failed. About 1,950 American soldiers have been killed, 14,900 have been wounded, and its military command still has only islands of control. It is a defeat more serious than Vietnam because it is self-inflicted, like the British invasion of Egypt to overthrow Nasser in 1956. But by the time of the Suez crisis the British empire was already on its death bed. The disaster only represented the final nail in its coffin. Perhaps the better analogy is the Boer War, at the height of British imperial power, when the inability of its forces to defeat a few thousand Boer farmers damagingly exposed both Britain's real lack of military strength and its diplomatic isolation.

From the beginning, Iraq was a strange war for the US to fight. The first President George Bush led a vast UN-backed coalition to a conclusive victory in the Gulf War in 1991 largely because he fought a conservative war to return the Middle East to its situation prior to the invasion of Kuwait. It was a status quo to which the world was accustomed and whose maintenance was therefore supported internationally - and in the Middle East.

What his son, George W Bush, undertook 12 years later in 2003 was far more radical. It was nothing less than an attempt to alter the balance of power in the region and hence in the world. The US would take control of a country with great oil reserves. It would assume quasi-colonial control over a nation which 15 years previously had been the greatest Arab military power.

Why did the US do it? One Iraqi leader, who has frequently met George W Bush attributes many of the bizarre events of the last three years to him. "What a strange man," he exclaimed. "Not stupid but very, very strange."

This strangeness and separation from reality became more evident as the conflict escalated. The debate on why the US invaded Iraq has been over-sophisticated. The main motive for going to war was that the White House thought it could win such a conflict very easily and to its own great advantage. They were heady times in Washington at the beginning of 2002, as the final decisions were being taken on invading Iraq. It was the high tide of imperial self-confidence. The US had just achieved a swift victory in Afghanistan. The Taliban forces had evaporated after a few weeks of bombing by B-52s and the withdrawal of Pakistani support. Their strongholds in Kabul and Kandahar fell with scarcely a shot fired.

There seemed no reason why Saddam Hussein should not be defeated with equal ease. His army was a rabble, his heavier weapons, such as tanks and artillery, obsolete and ill-maintained. Iraq was exhausted by UN sanctions. Ironically, if Mr Bush and Tony Blair had truly believed that the Iraqi leader possessed the military strength sufficient to pose a threat to the Middle East through weapons of mass destruction, they would probably not have attacked him. They were right to suspect he could not put up a fight.

A few years earlier I had watched a military parade in Baghdad from a distance. A well disciplined column of elite infantry marched past Saddam, standing on a raised platform near the triumphal arch of crossed swords commemorating victory over Iran in 1988. All the soldiers appeared to be wearing smart white gloves. Only when I got closer did I realise that the Iraqi army was short of gloves, as it was of so many other types of equipment, and that the soldiers were wearing white sports socks on their hands.

Few governments can resist the temptation to fight a short victorious war that will boost their standing at home. It enables them to stand tall as defenders of the homeland. Domestic political opponents can be portrayed as traitors or lacking in patriotism. The Bush administration had been peculiarly successful in wrapping the flag around itself after 11 September and later during the war in Afghanistan. It intended to do the same thing in Iraq in the run-up to 2004 presidential election.

Washington made little effort to understand the country. The White House and the Pentagon felt that no great knowledge was required. The State Department, which did know something of Iraq, was pushed aside and ignored. Among US policy makers there was a mood of extraordinary arrogance. Had they looked at Iraq more closely they would have noticed that it had an unnerving resemblance to Lebanon. The country is a mosaic of communities. There is an old Iraq proverb that says: "Two Iraqis, three sects."

At first sight, Iraq under Saddam Hussein looked like an east European autocracy during the zenith of Communism. A brutal state machine appeared to monopolise power. But this was never so in Iraq. Iraqi society has a complex web of loyalties often superior to any allegiance owed to the state.

First, there are the three great communities of Sunni and Shia Arabs and the Kurds. But Iraqis also feel intense loyalty to tribe, clan, extended family, city, town and village. Saddam Hussein's regime was too brutal and its security forces too efficient to be overthrown, but his power was always contested. He was never able to suppress the Kurds, who were in almost permanent rebellion for half a century. They were repeatedly able to destabilise the country.

When US troops began to spread out into Iraq after the fall of Baghdad they made a surprising discovery. Most people were armed, often with high-powered modern weapons. Saddam Hussein was reduced to introducing a buy-back programme in the early 1990s to cut down on the number of heavy weapons on the streets. Even so, his officials in south-east Iraq were astonished when a tribe turned up with three tanks - presumably purloined during the Iran-Iraq war - which they were prepared to turn over for a sizeable sum of money.

The complexities of Iraqi politics and society do not end there. There is an old joke by the humourist Stephen Potter which provides advice on how to confound an expert on a country, say China or Argentina, about which one is ignorant. Potter says one should listen politely while the expert expounds on the country about which he knows everything. When he concludes, the listener simply asks: "All very true. But surely conditions are different in the south?" They always are. But in Iraq they are different for every quarter of the compass.

For instance, in September the government in Baghdad was trumpeting the capture by the Iraqi army, backed by US forces, of the northern city of Tal Afar, west of Mosul, from the insurgents. From a distance it sounds simple: government wins, resistance loses. The reality was more complex. The 200,000 people of Tal Afar are mostly Turkmen, distantly related to the Turks. The area in which the city stands is Kurdish. Furthermore the Turkmen of Tal Afar are 70 per cent Sunni, who sympathise with the Sunni Arab insurgents, and 30 per cent Shia who support the mostly Shia and Kurdish central government in Baghdad. The Sunni Turkmen, making up most of the 683 people detained by the US and the Iraqi army in the city, claim that the US forces are being manipulated by the Kurds and Shia Turkmen anxious to assert their power in this disputed region.

At first the US did not believe that such complexities mattered. Paul Bremer, a supposed expert on counter-terrorism, caught the eye of President Bush and was made US viceroy. Much maligned in retrospect, Bremer showed little grasp of the reality of Iraqi politics. But then neither did his masters in Washington. He took months to realise that the most powerful man in the country was an ageing Shia cleric called Grand Ayatollah Ali al-Sistani, living in a house down an alleyway in the holy city of Najaf. Already staggering under the impact of armed resistance by the five million Sunni Arabs, the US dared not provoke a rebellion by the 15-16 million Shia of Iraq.

Mr Bremer may not even have been the worst American imperial governor over the last century. In 1943 in recently captured Naples the US commander General Mark Clark dined off exotic fish looted from the city aquarium, and appointed Lucky Luciano, the New York mafia boss, as a senior civilian adviser.

In defence of Mr Bremer, he was not alone in underestimating the ethnic and sectarian divisions of Iraq. Iraqi opposition leaders in exile in the 1990s believed sincerely that Saddam Hussein had fomented such antagonisms. Deprived of the dictator's malign leadership, so they argued, Iraqis would peaceably work out understandings with each other. I always noticed, however, that optimistic Iraqi friends who downplayed sectarianism in Iraq always knew who was a Shia and who was a Sunni, just as accurately as people in Belfast are aware of which of their neighbours is a Protestant and which a Roman Catholic.

Relations between Sunni, Shia and Kurd have always shaped Iraqi politics. In 1919, two years after the British capture of Baghdad from the Turks, a far-sighted British official called Arnold Wilson, the civil commissioner, warned that the creation of a new state out of Iraq was a recipe for disaster. He said it was impossible to weld together Shia, Sunni and Kurd, three groups of people who detested each other. Wilson told the British government that the new state could only be "the antithesis of democratic government". This was because the Shia majority rejected domination by the Sunni minority, but "no form of government has yet been envisaged which does not involve Sunni domination". The Kurds in the north, whom it was intended to include in Iraq, "will never accept Arab rule".

All very true - and Wilson was certainly more clear sighted and better informed than Mr Bremer 85 years later. But Wilson, like Bremer, got Iraq wrong. The year after he wrote the insightful passage above, the tribes in the centre of the country, mostly Shia, rose in revolt. By the time it was suppressed the British and Indian troops had lost 2,269 dead and wounded - and the Iraqis an estimated 8,450 dead.

The uprising created a potent myth for Iraqi nationalists. It saw tentative joint action by Sunni and Shia. They even held joint religious services. Wilson and several highly informed British officials in Baghdad at the time had underestimated the fact that, however much Shia and Sunni disliked each other, they hated the British even more. The point is important because the fragmentation of Iraq is so evident today that it is easy to forget that Sunni and Shia Arabs, even when on the verge of civil war, also see themselves as sharing an Iraqi identity.

There is a dangerous cliché to the effect that "Iraq was never a real country". If repeated often enough it might become a reality. It has long been true for the Kurds, who want to have their own independent state, to which they have every right. But until now at least, the Sunni and Shia have had several identities, of which one is strongly Iraqi. This shared identity helps explain one of the mysteries of the last two-and-a-half years. Why did the insurrection against the US occupation gather pace so rapidly?

Self-assured to the point of folly, the US administration made mistake after mistake. Within a few months it created a sympathetic environment for the insurgents. US soldiers stood by and allowed Baghdad - and every other city - to be looted and state institutions from hospitals to libraries to be destroyed. The Iraqi army and security services were dissolved in May 2003 providing well-trained military experts for the insurgency. Members of the Baath party were forced out of their jobs. This included teachers and doctors who had to join the party to keep their jobs. Towards the end of that year I visited Hawaija, a Sunni Arab town in western Kirkuk province, where the pro-American mayor told me he would have to close the local hospital because so many of the doctors were being fired because of their party membership.

It was not until three years after the British army captured Baghdad in 1917 that the first serious rebellion took place. In the case of the US occupation in 2003 the rebellion started in three months. But the two uprisings have a point in common: Iraqis do not like foreign rule or occupation any more than the people of any other country. The vast majority of them did not support Saddam Hussein. By and large they did not fight for him. They do not feel the military victors had any rights over them as Germans or Japanese may have done in 1945. Above all, Iraqis knew that the British had promised freedom after the defeat of the Turks in 1917 and the US after the overthrow of Saddam Hussein in 2003 - and in neither case was the promise kept.

Strangely, the Americans and the British never seem to have understood the extent to which the occupation outraged Iraqi nationalism, though anger might take a different form in the Sunni and Shia communities. In Sunni areas anybody resisting the occupation - including bigoted and fanatical Sunni groups - could expect a degree of protection. Former members of the Baath party and the security services - never popular institutions in Iraq - may have provided a skeleton organisation for the resistance. But this would not have been enough to mount a widespread uprising if it had not enjoyed popular support.

A private poll conducted for the coalition, in effect the US and Britain, in February this year showed that 45 per cent of Iraqi Arabs supported armed attacks on the coalition forces. For the first time in Iraq, resistance groups sharing the same ideology as al-Qa'ida were able to flourish. Many Sunni did not like them but they loathed the Americans even more. It is significant that al-Qa'ida was not able to launch an effective guerrilla war against the government and its American allies in Afghanistan, where Osama bin Laden had long been based, after the overthrow of the Taliban in 2001. It was in Iraq, from where Saddam Hussein had long excluded them, that they found a welcome.

In August 2003 there began the most sustained suicide bombing campaign in history. The bombers may have been mostly pious young men from Saudi Arabia, Egypt, Jordan and Syria, but their organisation is Iraqi. It is they who provide the explosives, vehicles, safe houses and intelligence. Merciless and directed primarily against civilians, the suicide bombing has proved savagely effective in demoralising and destabilising the country.

Presumably to the American and British officials cut off in the Green Zone, the day-to-day friction between Iraqis and the occupation forces was not visible. But for anybody living in Baghdad in 2003 and 2004 the ferocity of Iraqi-Arab hatred for the occupation was very evident. Local people would dance and rejoice when a bomb or a rocket hit an American vehicle. The US was outraged in the spring of 2004 when the burnt bodies of four American contractors were hung from a bridge in Fallujah. But they were mutilated not by the insurgents who killed them but by townspeople, day labourers waiting by the road for a job. The same savage joy was visible on the faces of the Shia crowd setting fire to a British armoured vehicle in Basra on 19 September.

There should be nothing surprising about the unpopularity of the occupation. How many occupations have been popular? Even Robespierre, no shrinking violet when it came to inflicting violence on others, pointed out to fellow French revolutionaries occupying foreign lands that "nobody likes armed missionaries." Soldiers do not make good neighbours. People in the US and western Europe are often starry-eyed about what sort of people become soldiers. Often it is the unemployed, poorly educated or young men simply eager to stay out of prison. In garrison towns across Britain local pubs are often reluctant to allow soldiers to enter on the grounds they will get drunk and break the place up. It is seldom popular to say this openly. The Duke of Wellington frankly declared on several occasions that his soldiers were "the scum of the earth" and never lived it down. He justified flogging on the grounds that it was the only way to deter his soldiers from looting or mistreating civilians. It is not an attitude likely to appeal to US generals. Their men - along with dubious foreign contractors - enjoy legal immunity under Iraqi law if they shoot an Iraqi.

The anarchy into which Iraq plunged at the time of the fall of Saddam Hussein - and has never re-emerged - cannot all be blamed on American failures. Iraqis are suspicious of central authorities. The far-sighted Captain Wilson observed that three-quarters of the Iraqi population were tribal and unused to obeying any government. On the eve of the First World War, a tribe on the lower Euphrates had a chant which stigmatised the government in Baghdad as "a flabby serpent which has no venom; we have come and seen it. It is only in past times that it kept us in awe." Attitudes have not changed very much in the years since.

Under the impact of UN sanctions in the 1990s the Iraqi economy collapsed. Millions of people saw their standards of living plummet. They were prepared to do any job, commit any crime to survive. The bedouin tradition of looting has never died away in Iraq. Iraqis looted Kuwait in 1990 of anything they could steal, from mobile cranes to wedding dresses. I was staying in the al-Rashid hotel in Baghdad at the time, when lobster suddenly appeared on the menu - fresh from the deep freeze of a hotel in Kuwait. During the Kurdish civil war in the north in 1996, some 5,000 cars were stolen in Arbil city in a single day. The looting of Baghdad - and every other city in Iraq - in 2003 was part of this same predatory tradition.

Many Iraqis welcomed the fall of Saddam Hussein because he had ruined their lives. He had started two disastrous wars, against Iran in 1980 and Kuwait in 1990. Hundreds of thousands of Iraqis were killed and wounded. The country's great oil wealth was spent on weapons. In the 1990s UN sanctions wholly impoverished the country. Iraqis believed they should have been living like the Saudis and instead they had the standard of living of Sudan. As the US tanks rolled into Baghdad they hoped their lives would now get better. Instead they got worse.

There was no return to normality as they hoped but a continuing lack of electricity, water and, above all, personal security. The billions supposedly spent by the US - much of it Iraqi oil money - produced almost no benefits. The country became a feeding trough for politically well-connected US companies and individuals. Even the Baghdad stock exchange could not reopen because the American in charge was a bemused 24-year-old who had the job because of his family's connection with the Republican party.

Given that the Americans are probably no stupider or more crooked than anybody else, why was the occupation regime so dysfunctional? The answer is probably that the senior US officials who ran Iraq owed their positions to the exigencies of American, not Iraqi politics. They knew how to function in Washington but not Baghdad. If they failed to deliver a better life to Iraqis their careers suffered no damage; but if they displeased the White House they were fired.

The US in Iraq had a second fatal weakness. The White House had begun the war thinking it would win an easy victory. It hoped by this means to consolidate its political power at home. I do not mean that the administration launched the war purely in order to gain or keep control of the presidency and both houses of Congress. But if the Republican leaders had suspected that its venture in Iraq would dent their political dominance in the US then surely they would not have become embroiled. Having intervened in Iraq, the decisions they took made sense as part of the administration's domestic political agenda, but not in terms of Iraqi politics.

As the US intervention in Iraq soured in the summer of 2003, the White House began to worry about its impact on the presidential election the following year. It needed to show American voters that progress was being made in Iraq. The result was an artificial timetable of events which could be sold - above all to the US media - back home as a sign that the US had a policy and would stick to it.

By November 2003 the White House announced that direct US rule would end the following year. Sovereignty would be handed over to an interim Iraqi government in the summer of 2004. Later it decided that elections would be held in January 2005, a constitution drafted and submitted to a referendum and then fresh elections held by the end of the year. These plans were presented somewhat hypocritically as the long-term fruit of US policy. In reality, when Grand Ayatollah al-Sistani had first demanded free elections, US officials had objected, citing technical difficulties and saying a census would be necessary first.

The real motive was that the US did not want to see the Shia religious parties, some heavily influenced by Iran, come to power. But as the Sunni Arab guerrilla war gathered strength in late 2003 and early 2004 the US could not afford to offend the Shia as well as the Sunni. Sistani got what he wanted. The Middle East was on its way to seeing the first Arab state run by the Shia since the Fatimid dynasty in Egypt fell 800 years ago

It was extraordinary to watch the US occupation unravel. In the first year and a half of the war it was still possible to drive out of Baghdad and talk to people in Sunni Arab towns and villages. From early days they were full of rage against the American army. US generals seemed to pride themselves on their ignorance of local customs. Many innocent farmers were being shot dead. They often died because when they heard a loud knocking on their door in the middle of the night they would open it with a gun in their hand. This was because, ever since the Saddam Hussein closed the banks in 1990 and the Iraqi dinar collapsed in value, Iraqis kept their money at home and in hundred dollar bills. Even a modest household might have $20,000 in cash, perhaps the life savings of an extended family. Farmers feared robbers and were usually armed. When a US soldier knocked at the door of a house in the middle of the night and saw an armed Iraqi in front of him he would open fire.

It was typical of the cast of mind of the US Army at this time that they thought they had dealt with questions about the number of Iraqi civilian deaths by simply not counting them. It might have public relations advantages in the US - though even this was dubious - but the Iraqis themselves knew how many of their people were being killed. And this was in a country where the tribal tradition is that a man must seek vengeance against the killer of anybody related to him over five generations. American soldiers on the ground eventually came to understand if they accidentally killed an innocent Iraqi then they would be the targets of a retaliatory attack a few days later.

The US military commanders and their civilian equivalents were in a state of denial in Baghdad. Every few days they would hold press briefings in which they would describe the insurgents as either foreign fighters or the remnants of Saddam Hussein's regime, the "bitter-enders" in the words of Donald Rumsfeld, the US Defence Secretary.

Every escalation in attacks was described as the insurgents' last desperate convulsion. The chasm between this rosy picture of the war and the bloody reality became ever deeper. One day I heard a rumour that there was an uprising in Baiji, a Sunni Arab oil refining town north of Baghdad. The US military had not said anything about it. When I got there I found the police station and the mayor's office burnt out and the police fled. Thousands of people were on the streets chanting pro-Saddam slogans and setting fire to Turkish fuel trucks that they claimed were stealing Iraqi oil.

During that first year after the fall of Saddam Hussein reporting Iraq was not as dangerous as it later became. When nothing was happening in Baghdad I used to drive west along the highway to Fallujah where I ate in a restaurant on the main street called Haj Hussein because my driver said it served some of the best kebabs in Iraq. Local people were helpful. They said they saw journalists as neutral, or possible allies in their struggle against the occupation.

The war was getting more intense. I was in Fallujah in November 2003 when a man who owned a shop selling musical cassettes and also worked as a freelance cameraman told me a Chinook helicopter had been shot down near by. We drove across an iron bridge over the Euphrates past fields filled with cattle and crops asking farmers the way to the crash site. Within minutes I was nervously peering through the bushes at the smouldering wreckage of the giant Chinook in which 16 US troops had just died and 21 were wounded. "I saw two helicopters pass overhead when two missiles were fired at them," said Daoud Suleiman, a farmer working among the date palms. "One missed and the other hit a helicopter at the rear-end and flames started coming out of it before it crashed into a field."

Above us half a dozen Black Hawk helicopters swooped and dived as if to avoid more missiles. The farmers passed around pieces of twisted metal they had taken from the helicopter. One of them waved a piece of wreckage derisively at the Black Hawks. The others shouted at him: "Put it down! They might think it is a gun and fire at us." "I am not afraid," he yelled and waved the metal defiantly above his head again before other farmers wrestled it away from him.

I did not realise at the time that an important change was about to happen. It seemed to me that the absurd optimism of the US military briefings in Baghdad and statements from the White House were bound to be exposed by events. The US increasingly controlled only parcels of territory in Iraq in the months before the US presidential election in 2004. But President Bush's campaign had received an unexpected ally in the shape of Abu Musab al-Zarqawi, the leader of al Qa'ida in Iraq. His brand of religious militancy held that all foreigners including journalists were spies. I soon became the only non-Iraqi eating in the Haj Hussein. When I came into the main dining room the other diners looked more and more suspicious about who I was. The last time I visited the manager pointedly seated me in an empty room upstairs. A few months later a US plane bombed and destroyed the restaurant.

Thanks to Zarqawi the American and British media could not chronicle the deteriorating military position of the US in the lead-up to the presidential election. In September 2004 Iyad Allawi, the interim Iraqi prime minister, claimed during a visit to Washington that 14 or 15 out of 18 Iraqi provinces were "completely safe". Every foreign journalist in Iraq knew that this was untrue but we could not prove it without being kidnapped or killed.

Two experiences in April 2004 brought home to me how grim the situation was becoming. I also realised that if I went on reporting as I had done before I was not going to live very long. There were twin uprisings north and south. The US military lost control of Fallujah. Tanks closed the main highway to the city. I tried to get around them by driving close behind the trucks of an Iraqi aid convoy on its way to Fallujah. Men on the back of the trucks waved Iraqi flags and chanted patriotic slogans.

After an hour of weaving through country lanes over stagnant irrigation canals I was disappointed to find that we had made little progress towards Fallujah. As we got back on the main highway near the prison at Abu Ghraib an American convoy of fuel tankers roared past and was immediately ambushed. There were staccato bursts of fire from guerrillas on both sides of the road and the whoosh of rocket-propelled grenades overhead. US soldiers in Humvees guarding the convoy fired back with machine guns. We drove off the road along with some Iraqi cars on to a patch of waste ground, got out and lay flat. Basil al-Kaissi, my driver, shouted to other Iraqis who were lying down: "Take off your keffiyehs [Arab headdresses] or the Americans will think you are mujahedin [guerrilla fighters] and kill you."

At first the firing came from the other side of the highway and passed over our heads. Then somebody started to shoot from our side of the road and the US machine-gunners replied. There was a pause in the shooting and we jumped back into our car and drove off down a track road crossing a narrow bridge over a broad irrigation ditch. As we did so four mujahedin ran forward carrying a heavy machine-gun on a tripod and stopped on the bridge staring towards the sound of the firing. One of them shouted to us, asking: "What is happening?" Basil, not wanting them to notice that there was a foreign journalist in the car, shouted back: "We were trying to bring help to Fallujah but those pigs opened fire on us."

Back in Baghdad the US generals at their daily briefings in the Convention Centre in the Green Zone were refusing to admit that Iraq was out of control. They must have believed their own propaganda, which would explain why they were sending convoys of vulnerable fuel tankers through guerrilla-controlled territory.

A week or more later I decided to drive to Najaf, the shrine city where Muqtada al-Sadr, the radical Shia cleric, and his Mehdi Army militia were besieged by 2,500 American soldiers. All went well until we were stopped at a militia checkpoint at Kufa, a few miles from Najaf. I was sitting in the back of the car and wearing a red-and-white keffiyeh to hide my brown hair and pale skin. I hoped this would prevent anybody spotting me as a foreigner unless they got close. The disguise turned out to be a bad idea. The Mehdi Army militiamen peered into the car and recognised me as an obvious Westerner. The fact that I was partly dressed as an Arab made them even more suspicious. They examined my satellite phone, mobile and camera as if they were proof that I was a spy. Nobody seemed to be in charge of the checkpoint. Two of the angriest militiamen tried to bundle me into another car and I resisted. Finally three gunmen, clutching their Kalashnikovs and their chests covered with ammunition pouches, clambered into our car. We followed a second car, also filled with heavily armed militiamen, to their headquarters in the green-domed Imam Ali mosque in the centre of Kufa.

The militiamen gradually became less aggressive. They offered me cigarettes and, though I had given up smoking some time before, this did not seem to me to be a good moment to refuse the offer. They were poor young men mostly from Sadr City, the great Shia slum in east Baghdad. They minutely examined a copy of The New Yorker they found in the back seat, one of them muttering "haram" - forbidden - at a cartoon of a woman with a low-cut blouse. My belongings were gradually returned apart from the satellite phone. I saw a black-clad gunman stuff it into his ammunition belt but it would have been unwise to protest. At this moment a firefight broke out with US troops on the other side of the Euphrates. We saw machine-gun bullets strike the wall of the mosque above our heads, sending up little puffs of dust. In the face of this new attack our captors became much more friendly and took us to their headquarters beside the shrine in Najaf where we were freed.

The danger of reporting in Iraq made it easier for the US and the Iraqi government to pretend that progress was being made. To take one example, in November 2004 the US Marines assaulted and captured Fallujah, a victory widely covered by embedded correspondents. But as the assault began the insurgents rose up in Mosul, the northern capital of Iraq, and captured 30 police stations and great quantities of arms and ammunition. The 3,000-strong Iraqi police force went home or changed sides. The event was hardly noticed by the outside world because there were no embedded reporters to write about it. Iraqi officials studiously avoided putting to the test their claim that most of Iraq was safe. A British correspondent flew with Iyad Allawi, the Prime Minister, in his helicopter on as trip billed as a meeting with ordinary Iraqis. They took off from the Green Zone in the prime ministerial helicopter and flew two miles across Baghdad to the heavily guarded Police Academy. Here Allawi briefly addressed police recruits, and 10 minutes later they were back in the safety of the Green Zone.

A diminishing number of Iraqis were hopeful each time the US and its Iraqi allies announced that some new hurdle on the road to democracy and prosperity has been overcome. In June 2004, Mr Allawi's interim government supposedly took over from the occupation authorities, though many of the ministers were chosen by the US and UN envoys Robert Blackwill and Lakhdar Brahimi. Violence lessened for a few weeks and then intensified. The same thing happened after the elections in January this year when the Shia and the Kurds won a majority and the Sunni boycotted the poll. After prolonged negotiations a government was formed by Ibrahim al-Jaafari, the new Prime Minister. A few months later the Shia and Kurds but not the Sunni agreed a draft constitution to be put to a referendum on 15 October. None of these developments were necessarily bad in themselves but they were grossly oversold and pushed through at high speed to impress the American and British publics. They were not the solutions they purported to be. All raised excessive expectations at home and abroad.

The election in January, for instance, was a important because Shia and Kurds could at last vote. But since the Sunni, the only Iraqi community in revolt, boycotted the poll it was never likely to impress the insurgents. The constitution - the rules of the game for the new Iraqi state - contains federal status for the Kurds which they have long demanded. Grand Ayatollah Sistani said Shia should vote for it. At the same time Sunni imams in Baghdad told their congregations to vote against the constitution because it dissolves the Iraqi state. Some of them ended their sermons with cries of support for the mujahedin.

Discrediting all of these much-trumpeted political developments in the eyes of Iraqis is the fact that their lives are not getting better. The government is notoriously corrupt. Nobody seems to be in charge. There is still very high unemployment in the cities. Electricity has been running two hours on, four hours off in the capital for the past fortnight. At the height of the summer, the supply was even worse. Meanwhile kidnapping remains rife. And every few days the suicide bombers make their savage onslaughts and the mood among Iraqis is one of edgy despair.

It is not difficult to see why. Stendhal describes, in Memoirs of an Egotist, that when he visited a city or town he tried to identify the 10 prettiest girls, the 10 richest men and the 10 people who could have him executed. He would have his work cut out in Baghdad. Veils increasingly conceal girls' faces, the rich have fled the country - and almost anybody can have you killed. To give a picture of present-day Baghdad, surely the most dangerous city in the world, it is worth explaining in detail why a modern day Stendhal would be in trouble if he tried to identify any of the three categories he mentions.

Iraqi women used to enjoy more freedom than almost anywhere else in the Muslim world apart from Turkey. Iraq was a secular state after the overthrow of the monarchy in 1958. Women had equal rights in theory and this was also largely true in practice. These were eroded in the final years of Saddam Hussein as Iraqi society became increasingly Islamic. Now, under the constitution negotiated with the participation of the American and British ambassadors, women will legally become second class citizens in much of Iraq. About three-quarters of the girls leaving their school at lunch time in central Baghdad now wear headscarves. The reason is generally self-protection. Those who are genuinely religious conceal their hair, and these are in a minority. The others fear religious zealots. There is also a belief that kidnappers, the terror of every Iraqi parent, may be less likely to abduct a girl wearing a headscarf because they will believe she comes from a traditional family. This is not because of any religious scruples on the part of a kidnap gang but because they think that old-fashioned families are likely to belong to a strong tribe. This tribe will seek vengeance if one of its members is abducted, a much more frightening prospect for kidnappers than any action by the police.

The life of women has already become more restricted because of the violence in Baghdad. Waiting outside the College of Sciences in Baghdad at the beginning of the academic year was a 20-year-old biology student called Mariam Ahmed Yassin, who belonged to a well-off family. She was expecting a private car, driven by somebody she trusted, to take her home. Her fear was kidnapping. She said: "I promised my mother to go nowhere after college except home and never to sit in a restaurant." Her father, a businessman, had already moved to Germany. She volunteered: "I admire Saddam very much and I consider him a great leader because he could control security."

Mariam's father is part of a great exodus of business and professional people from Iraq. If Stendhal was looking for the 10 richest people in Iraq, he would be wise to start his search in Jordan, Syria or Egypt. The richer districts of the capital have become ghost towns inhabited only by trigger-happy security guards. In some parts of Baghdad property prices have fallen by half in the last six months. Well-off people want to keep it a secret if they do sell a house because kidnappers and robbers would know they have money. "Some 5,000 people were kidnapped between the fall of Saddam Hussein and May 2005," says the former human rights minister Bakhtiar Amin. The real figure is far higher since most people do not report kidnappings to the police.

In the aftermath of the fall of Baghdad the better-off were hopeful that Iraq's long night was coming to an end. Many rich Iraqis living abroad came home. They did not stay long. Today they see Iraq breaking up under the new constitution. "Federalism is a disaster," says Hussein Kubba, a businessman who lived in Iraq under Saddam Hussein. "It is a recipe for the disintegration of Iraq and more bloodshed." A Shia himself, he dreads the idea of legislation being vetted by the clergy. He says it is too late for him to leave, but his sons are being educated abroad.

The executioners, the last category of people on Stendhal's list, are all too clearly in over-supply in Iraq. These are the people who can order a killing and they are very active . Even during a quiet day as many as 40 bodies may turn up at Baghdad morgue, dead at the hands of US soldiers, insurgents, Iraqi army and police, bandits, kidnappers, robbers or simply neighbours who settled a dispute with a gun.

At one time assassins put up notices in a Baghdad street market advertising their services at the cost of a few hundred dollars. Ordinary US soldiers can shoot any Iraqi by whom they feel threatened without fear of the consequences. With suicide bombers on the loose the soldiers feel threatened all the time and most Iraqis feel threatened by them. The Iraqi police general in charge of the serious crimes squad was shot through the head by an American soldier who mistook him for a suicide bomber.

The distinguished head of al-Nahrain university was dangerously wounded in the stomach, probably by private security contractors, when his vehicle got too close to theirs. President Jalal Talabani's head of protocol was not with him when he visited Washington recently to see President Bush. Instead he was in a Baghdad hospital with a broken arm and leg after a US Humvee rammed his vehicle on the airport road.

So many people are being killed in Iraq every day for so many reasons that the outside world has come to ignore the slaughter and Iraqis themselves are almost inured to it. The death of a thousand people in a stampede during a religious festival in September was only a one-day-wonder abroad. It is worth looking at just three acts of violence in a small part of Baghdad to show how casual killings and kidnapping impact on the people of the city. They took place within a few days of each other in September in or close to al-Kudat, a previously prosperous district in the south-west of the city, where many doctors and lawyers once lived.

The first killing was at the hands of the Americans. Early one morning, a surgeon called Basil Abbas Hassan decided to leave his house in al-Kudat for his hospital in the centre of Baghdad at 7.15am in order to beat the morning rush hour. Because so many streets are blocked by concrete walls protecting military or police outposts Baghdad traffic is always on the verge of gridlock. Dr Hassan, a specialist in heart surgery, was the kind of man who should have been one of the building blocks of the new Iraq. He drove his car out of a side street onto the airport road without noticing that an American convoy was approaching behind him. A US soldier thought the car might be driven by a suicide bomber and shot Dr Hassan dead. Not many of his friends attended his funeral because so many of them had already left Iraq.

Mobile phone theft is common all over the world but in Baghdad people will kill for a handset. This not because they are more expensive than elsewhere in the world - in fact they are cheaper because nobody pays any tariffs on them - but because murder is so easy. No criminal expects to be caught. A few days after Dr Hassan was killed by the Americans, a young man aged 16, called Mohammed Ahmed, was making a call on his mobile as he walked down the street in the nearby district of al-Amariyah. A car drew up beside him and a man pointed a pistol. He said: "Give me your phone." Mohammed refused or hesitated to hand it over for a few seconds too long and the gunman killed him with a bullet in the neck.

The third story has a happier ending, though at one moment it seemed likely to end in tragedy. It happened in another street in al-Khudat. The mother of a friend called Ismail told him that there was a strange car parked outside their house. She wanted him to find out to whom it belonged. It did not seem likely that anybody would leave a car bomb in a residential side street because US or Iraqi army patrols never used it. But anything out of the ordinary in Baghdad may be dangerous and is routinely checked out.

Ismail spoke to two neighbours who denied any knowledge of the mysterious car.

A third neighbour admitted that he knew about it and went on to give the dramatic reason why it was there. He said there was a meeting of his extended family taking place in his house because a few hours earlier his 14-year-old grandson Akhil Hussein had been kidnapped as he returned from school. The kidnappers called, demanding $60,000 for his release and threatened to kill him. The panic-stricken family had gathered their relatives together to try to raise some money. They were asked to park their cars far away from the house in case the kidnappers were watching and got an exaggerated idea of the family's wealth. This explained what a strange car was doing outside Ismail's house.

The problem was that the kidnappers had kidnapped the wrong boy. His family was the only poor one in the street. They had moved into a large house to look after a well-off relative who was dying of cancer. They could not afford anything like $60,000. When the kidnappers called again the grandfather said: "We are a poor family. Come and look at our house. When our generator caught fire a month ago it burnt part of the house and we did not have the money to rebuild it." The kidnappers said they did not believe this, but a few hours later called again to say they had looked at the house and realised their mistake. A voice on the phone said: "We are sorry. We kidnapped the wrong boy. We meant to kidnap the son of a rich man living next to you. Even so you must pay the cost to us of our mistake which is one million dinars [$800]."

The grandfather of Akhil - his father was in a state of collapse because of shock - immediately went to his rich neighbour, a Kurd. He told him: "Be careful: they want to kidnap your son." The neighbour bundled his family into several cars and fled. For once the kidnappers kept their word and the grandson was released unharmed. He knew nothing because he had been tied up and his eyes taped from the moment he was seized. Akhil was lucky. Many kidnap victims are found tortured and dead even when the money is paid.

Iraqis are more likely to suffer violence from criminals than the US Army, the insurgents or the Iraqi army. The inability of the police to protect people from crime has led to profound cynicism about the government. Most Iraqis see the state as a machine for exploiting them. This is not just a view of those outside the government. Zuhair Hamadi, a veteran opponent of Saddam Hussein who was chief of staff to the cabinet under Iyad Allawi, says: "The ministries are beyond repair. Officials were corrupt under Saddam Hussein but frightened to death of being executed. Not any more."

Even Iraqis were shocked to find that almost the entire $1.3bn procurement budget of the defence ministry has disappeared according to Ali Allawi, the Finance Minister. He claims this was not just the commission payments common in the Middle East. "It is possibly one of the largest thefts in history," he said. "Huge amounts of money have disappeared. In exchange we got next to nothing, just scraps of metal."

The result is that the new Iraqi army is poorly armed. It is common to see young soldiers being driven through Baghdad in trucks that look as if they were built to carry hay or chickens. When insurgents attack the soldiers with machine-guns or bombs they inflict terrible casualties. Lack of security and lack of electricity are the two failings of the government most often raised by Iraqis. At the moment, Baghdad is getting four hours of electricity off, two hours on during the day. Baghdad resounds with the sound of generators large and small. Again and again Iraqis ask why, two-and-a-half years after the fall of Saddam Hussein they are getting less electricity than they did when he was in power and the country was under UN sanctions. There are excuses. Insurgents keep blowing up the hard-to-defend pylons carrying the electric cable which makes up the electric grid. Abdul Mohsin Shalash, the Minister of Electricity, says that one of the biggest mistakes was to cancel all the old projects begun or about to begun under Saddam Hussein and sign new more expensive contracts. Iraqis on the street cynically believe new projects are initiated so officials can pocket fat commissions.

Prime Minister Jaafari is being blamed for the multiple failings of his government. Privately and publicly he is denounced as personally feeble, professionally incompetent and unable to unite his cabinet. The allegations are hardly fair. The allocation of ministerial posts in the Iraqi government was decided after long negotiations between the parties. Mr Jaafari cannot fire ministers. His authority is limited. Jobs in every ministry are the preserve of the party that runs it. The interior ministry was taken over by the Supreme Council for the Islamic Revolution in Iraq when the government was formed and they are not going to give it up.

There is a great sump of misery in Iraq. And until the lives of people in general improve the political crisis will not end. Given such deprivation and corruption, why should soldiers fight for the government, particularly if they only joined the army or police for a job? Poverty fuels the insurgency. Abu Mazen, a lawyer in Baghdad, said: "I believe that all kinds of crime nowadays in Iraq are the result of unemployment so that people who have no job are forced to become criminals."

The only part of Iraq that is prospering is Kurdistan, because it has security. Even prostitutes from Baghdad have migrated to Sulaymaniyah on the grounds that the capital is too dangerous to work in. For all the billions of dollars supposedly spent on reconstruction in the rest of Iraq there are very few building sites to be seen outside the three Kurdish provinces where businessmen throng the hotels. There is a mood of nervy self-confidence that just for once the Kurds may be on the winning side.

Few other Iraqis share their optimism. "The Iraqis are suffering from corruption, terrorism and occupation," says the veteran opposition leader Mahmoud Othman. "Every year it is getting worse." Government leaders frequently travel to Washington and London to give a rosy picture of Iraq slowly emerging from the present bloody chaos. Living behind the walls of the Green Zone, protected by US troops and foreign security companies, they seldom have little idea themselves of life in Iraq. Mr Jaafari must give 24 hours' advance warning to his security detail if he leaves the Green Zone to visit President Talabani, whose heavily defended house is just five minutes' drive away. Iraqis do not enjoy many jokes, but the disappearance of so many cabinet ministers abroad on essential business has been a source of general amusement in the last two years.

In far-flung capitals across the globe they bid the insurgents defiance and tell them their days are numbered. In reality security is getting worse. Insurgents are tightening their grip in Sunni-dominated districts in south and west Baghdad. A policeman guarding a petrol station in this area explained that he was going home at 8pm "because the resistance takes over then and I will be killed if I stay".

Even while I was writing the above paragraph in my hotel room in Baghdad there was the sound of an explosion nearby, making the windows shake in their frames. Pigeons sitting on my balcony rose twittering in alarm. I thought of taking the lift to the flat roof on the sixth floor to see where the bomb had gone off but I decided against. The uncertain electricity supply has damaged the lift's machinery and I am likely to be suspended between floors. I have also learnt that one cloud of oily black smoke rising over Baghdad looks much like another.

An obstacle to the Iraqi, American and British governments resolving the crisis in Iraq is that at no stage over the last 30 months have they been willing to admit how bad things really are. This is hardly surprising. Neither Mr Bush nor Mr Blair want to reveal the depth of the quagmire into which they so confidently plunged in 2003. They also presumably believe that at any moment they may touch bottom. Iraqi governments, dependent on foreign support, parrot whatever they believe Washington or London wants to hear at the time. Iraq is full of mirages. In theory Iraqi army and security forces are being built up towards the 200,000 level but when Kurdish intelligence counted a unit entering Kirkuk they found most of the soldiers were not there. Its men stayed home or never existed while commanders pocketed the salaries of these "ghost battalions".

Much of the Iraqi government exists only on paper. It is more of a racket than an administration. Its officials turn up only on pay day. Elaborate bureaucratic procedures exist simply so a bribe has to be paid to avoid them. The US, with Britain in tow, was always going to be in trouble in Iraq. They did not see that Saddam Hussein was the product as well as the creator of fierce ethnic, religious and regional tensions. His demonic personality was not the only reason why Iraq's recent history is soaked in blood. This mistake would not have mattered so much if Washington had not sought to persuade the US public that all was going to plan in Iraq. A set of milestones were devised - the turning over of sovereignty to Iraq, the elections, the constitution - whose primary purpose was to give a spurious sense of progress to American voters. Not surprisingly they show increasing anger at being duped.

The most damaging consequence of this pretence at progress is that it has prevented the pursuit of more effective policies. The fact that so many Iraqis blame the US occupation for their ills does not mean they are right. But, having spent most of my time in Iraq since the fall of Saddam Hussein, I believe that the biggest mistake being made by the US and Britain is a very simple one: They do not realise the unpopularity of the occupation. No people wants to be ruled by another.

A foreign army provokes inevitable friction with civilians. Gibes in Britain in 1943-44 against allied US soldiers - "overpaid, over-sexed and over here" - are still remembered. It is animosity towards the occupation that provides political oxygen for murderous, bigoted and fanatical groups such as Zarqawi's al-Qa'ida in Iraq.

The Iraqi government will not stand on its own feet until it knows the US and British troops are going. At the moment most of its members, apart from the Kurds, lurk inside the Green Zone, as sealed off from the Iraqi people as if they had never left London or Detroit. Ironically Iraqi leaders probably exaggerate the strength of the insurgents. In public government members claim they are desperate men on the run in the face of resurgent Iraqi democracy. In private they say that without 140,000 US troops the insurgents would seize Sunni districts in Baghdad tomorrow.

The other justification for keeping a US army in Iraq is that it is preventing civil war between Shia and Sunni. The danger of such a conflict is growing as Sunni fanatics butcher innocent Shia. Shia police commandos and the security services kill former Baathists and Sunni in retaliation. Civil war could come but there is no sign that US troops can or will prevent it.

The US has imposed a timetable for political developments in Iraq that has more to do with American than Iraqi needs. It has carefully avoided proposing the only time table - a negotiated withdrawal of foreign forces - capable of reversing the spiral into chaos and war. To do so might be too much for the White House and Downing Street. Such a commitment would falsify the rosy and deceptive picture they have hitherto presented, though to an increasingly sceptical audience at home. It would mean admitting that the US is not the sole superpower, able to gain its ends in the world alone, and with only the UK as an important ally. But if there is no withdrawal then the war will escalate. The occupation exacerbates a crisis it purports to cure.

Mr Blair says British and American troops will stay until the job is done, but their very presence means Iraq will never be at peace.