Dreda Say Mitchell: It's not only white people fleeing our city centres

It doesn't feature on Location, Location, Location but "race" is back in the news as a factor in where people choose to live. Trevor Phillips, who chairs the Equality and Human Rights Commission, has reiterated his concerns that Britain could become a segregated society. The Bishop of Rochester thinks some Muslim areas are becoming no-go areas for outsiders, while others are comparing some of England's northern towns to Belfast. But is it actually true?

That some people prefer to live with their own "sort" – the same ethnic background, religion, traditions – and not with "themuns", as they say in Northern Ireland, is incontestable. Actual figures are hard to come by (people are less than frank about this, as we know) but any comment about migration will always be followed by some feedback of the "Why do you think I left Britain to live up a mountain in Switzerland?" variety.

In fact, though, there are as many different reasons why people choose to relocate. The original "white flight" was the working class moving out of London to new towns such as Basildon and Crawley in the 1950s and 60s. Some of them may well have been unhappy with the changing colour of the capital, but most were looking to leave the slums behind and move into more spacious homes with plenty of green space and decent schools.

Many middle-class couples, meanwhile, enjoy living in the "vibrant" parts of our cities – until they remember the joys of leafy white suburbia at about the time they start thinking about their children's education. Is that because they don't want their youngsters skipping piano lessons to go rapping? Or is it because they have studied the league tables and worked out where the educational action is? The very wealthy will continue to send their children to public schools, where they mix with the sons and daughters of Russian oligarchs and Indian millionaires; the rich at least are truly multicultural these days.

My experience as a teacher and education adviser in London has shown that it is not only white, professional families who are giving the inner cities the old heave-ho. Parents from a range of ethnic backgrounds are thinking long and hard about their child's education. My sister, who lives on a housing estate, has recently sent her daughter across the city to be educated at a secondary school with a high reputation, rather than any local alternative.

The factors that lead migrants to concentrate in certain areas are many and varied, too. The most important is money. If you want to know where the migrants are in any particular town, find out where the poorest neighbourhoods are and that'll be it, at least to start with. Caribbean migrants didn't settle in Brixton, Notting Hill and parts of the East End in the 1950s because they were "vibrant" but because that was all they could afford. Equally, family, community and support networks are vitally important when you move to a new country. One of the reasons why the plan to disperse asylum-seekers flopped was because of this. Migrants tend to gravitate to areas where other people from the same background have already led the way.

Does any of this matter? On one level, it obviously does. In a minority of places, real segregation is either happening or on the cards and prejudice and tension are driven by the ignorance and fear that generates. It has often been noted that white and Asian youths in some northern towns listen to the same music, wear the same clothes, use the same slang and support the same football teams – but, in other respects, they may as well be living in different countries. Those who are warning about the long-term consequences of this are right to do so, and we urgently need to think of ways of breaking these barriers down.

But it is also true that most people in this country neither live nor want to live in segregated communities, and couldn't even if they did. When I go to visit my family in the East End, I in no way feel I am entering a London that is a no-go area. Nor is there anything new about divisions in our society, even if you take "race" out of the equation. The Hampstead leftist who walked down the road to have a drink with his comrades in working-class Kilburn was trying to cross a social barrier as pronounced as any racial one.

Perhaps the biggest irony of this debate is that it ignores the most fundamental division developing in our society; that between those who own property and those who don't. Whether house prices rise or fall in 2008, that is one problem that won't be going away any time soon.

The writer is a novelist and former teacher. Her book Killer Tune is published by Hodder