I now realise that I owe the taxpayers of this country an apology. My first home was a mansion set in 75 hectares of landscaped gardens, with a boating pond and Gothic ruins. My family would never have been able to afford it on their own – I'm not a Rothschild, as some of you may already have suspected – but we were able to live in a stately home built by the banking family. And it was all thanks to the generosity of the local authority.

Yes, I have to admit, I grew up in "social housing". Council houses, as they used to be called when I was a child, came in all shapes and forms. Social housing allowed my parents to live in a Grade II* listed building in west London – my Dad was a council gardener, and the flat came with the job – instead of a row of identical boxes on a modern estate.

Now someone has noticed this anomaly, which means that working-class people are still living in posh areas of our cities. It's a scandal, as a headline in yesterday's Daily Telegraph pointed out: "No one has a right to live in Kensington at taxpayers' expense." A right-wing think-tank has tackled this injustice head-on with a report – endorsed by the housing minister Grant Shapps – which wants councils to sell posh houses and use the money to build cheaper homes.

Inevitably, given the shortage of space in our cities, that means somewhere else: a place we might usefully designate "Not-Kensington", where people on low incomes can be with people like themselves. They'll be more comfortable, you know, with all the other Kayleighs and Duanes. Another way of achieving this species of social engineering, it now occurs to me, would be to create zones in our cities where parents who give their children certain first names are not allowed to live. But that's not (yet) the proposal to hand. Policy Exchange is urging local authorities to sell off "expensive" social housing as it becomes vacant – magnanimously allowing the current tenants to die there or move of their own accord – and build hundreds of thousands of cheaper homes with the proceeds.

The idea apparently stems from the discovery that more than a fifth of council house tenants in England live in homes worth more than the average privately owned home in their areas. (Have you ever noticed, by the way, that poor people live in houses while rich people have "homes"?) The report estimates that selling social housing could raise £4.5bn a year, and fund a programme to build houses in Not-Kensington and create up to 340,000 jobs in the construction industry.

You don't have to be a great thinker to see the problems with this proposal. What are the chances of all that money being spent on providing social housing? Very low, judging by what's happened when social housing has been sold off in the past, and now local authorities are desperate for funds for other projects. Loss of social housing stock, which began when Margaret Thatcher embarked on an ideological mission to sell it off, is one of the reasons for today's catastrophic housing shortage.

Then there's the problem of social exclusion, which will only get worse if people on lower incomes – including nurses, fire fighters and other essential workers – are denied access to public housing in nice parts of our cities. Neil O'Brien, the director of Policy Exchange, claims that the scheme wouldn't mean social housing tenants having to move from their local area – just to a less sought-after street, whatever that means. He must be thinking about all those wide, undeveloped spaces in Pimlico and Notting Hill, but the chief executive of the National Housing Federation takes a less rosy view. "It could effectively cleanse many towns of hard-working people who simply can't afford the high prices of buying or renting privately," David Orr said bluntly.

When I finally moved with my parents to a council estate, I discovered what social exclusion meant. In sixth form, I was one of a handful of girls at the local grammar school who didn't live in privately owned homes. I didn't have many friends, even though I think the other girls' condescension was mostly unconscious. I barely knew that public schools existed, and I only discovered much later that Old Etonians were running the country. As they are in 2012, of course.

One of the things I like most about the area of London where I now live is that it has what's known as "mixed" housing. Even so, class prejudice runs deep: a cop once asked if I'd had any problems with the council flats across the road, adding primly that he'd had to speak to some of the tenants about "inappropriate behaviour".

Some days, I have to pinch myself. It's the 21st century, and we're actually talking about moving working-class people out of nice areas? According to Policy Exchange, people don't have a "right" to live in expensive places – unless they're rich, of course. Thankfully, the wealthy won't ever have to face the prospect of living in Not-Kensington. And I can only offer my sincere apologies, as a working-class person who undeservedly started life in a Rothschild mansion.

www.politicalblonde.com; twitter.com/@polblonde