Britain has been plunged into an urban nightmare. Parts of some of our largest cities, London in particular, have witnessed a terrifying breakdown of law and order. Shops have been smashed and looted. Properties have been vandalised. Families have been burned out of their homes. Cars have been set on fire. Running battles have been fought between gangs of youths and riot police. For days the capital has been awash with trepidation.

The political ramifications will be profound. This is beginning to look like the Government's Hurricane Katrina. George Bush's Republican administration looked clueless when the levees broke in 2005, engulfing the city of New Orleans. The same can be said of the Coalition since the violence first flared at the weekend. The impression left by Theresa May yesterday, where the Home Secretary surreally implored listeners of the BBC's Today programme to report rioters, was of anything but grip. David Cameron returned from holiday yesterday in response to the crisis, but his attempts to project authority were scarcely any more convincing.

Yet the Katrina analogy is appropriate in a more profound way. This is a Katrina moment for the political classes in general and indeed for the entire country. Just as the US government failed to shore up the levees above the city of New Orleans when it had a chance, successive British administrations have failed to repair the social levees that ought to protect our society from this kind of aggression. At the weekend, those levees burst, and we have been witnessing the ugly results.

We know enough about these riots and those perpetrating them to know what they are not. This is not a political protest. The rioters have no agenda. It is not centrally directed. The goal is acquisitive looting or brainless destruction. The original riot in Tottenham on Saturday seems to have been sparked by a community's sense of grievance against the police. But what happened in Woolwich, Toxteth and Bristol on Monday night is clearly not an anti-police protest. Much of it is copycat rioting. Criminal gangs and antisocial youths have seized on an opportunity to run amok, knowing that the police cannot be everywhere at once.

Nor is this a response to public-sector austerity. Reports of the Government's cuts might have added to the air of desperation in many poor communities. But the fact is that most cuts have not been implemented yet. This is not a riot driven by new media either. BlackBerrys and Twitter – neither of which existed during the inner-city civil disturbances of the 1980s – have doubtless played a role in fanning the flames. But new media is hardly a sufficient explanation for this antisocial spasm. This is also not a race riot, in the manner of Brixton, Toxteth, Handsworth or Broadwater Farm in the 1980s, either. The rioters of 2011 are racially mixed. And there is no overwhelming collective grievance against the police for racial harassment as there was three decades ago.

This disturbing phenomenon has to be understood as a conflagration of aggression from a socially and economically excluded underclass. A disaffected criminal fringe, made up of people who feel they have no stake in society, has decided to exert itself on the streets. Alienated young men and women, some of them barely more than children, have taken this as an opportunity to steal, riot, burn and to generally kick against authority.

Diane Abbott, the MP for Hackney North and Stoke Newington, said yesterday that the rioters have been trashing their own communities. Yet this is surely a misunderstanding. As the founder of the charity Kids Company, Camila Batmanghelidjh, pointed out on these pages yesterday, many of these jobless and under-educated youths simply do not feel that they belong to a community. They have formed parallel groupings instead, defined by a shocking lack of morality and an immunity from shame. It is this criminal, marginalised and sometimes mentally disturbed underclass that Britain has seen in action in recent days.

Riots by youths in the suburbs of French cities in 2005 were blamed on the physical segregation between rich inner cities and the deprived outer neighbourhoods. But what is clear from the events of recent days in Britain is that segregation is not just a geographic phenomenon; it can happen in the mind too. These youths live in the heart of wealthy British cities, but they do not feel part of them. The warning signs have been flashing for many years, particularly in London. They were visible in the motiveless murder in 2000 of Damilola Taylor in Peckham. It could be seen in the reports of mobs of youths "steaming" commuters on London trains and buses to relieve them of their valuables. It has been visible in the resurgence of knife assaults and gang shootings, even while the overall level of crime nationally continued to decline.

The great state departments of education, welfare, health and housing have failed these groups. Far too little has been done by successive generations of politicians and public servants to integrate these individuals into normal society. The fuse for this explosion has been burning down for years, perhaps even decades. If any good can emerge from the horrors of recent days it will be that we finally face up to the shame of our excluded underclass.