Our comprehensive schools are under attack and barely anyone in public life is defending them. As part of an ideologically charged campaign to strip the state of all but its most basic functions, the Tories are fragmenting the education system by building a patchwork of privately run free schools and academies. Forced into retreat, champions of the comprehensive ideal have adopted a purely defensive posture. But it's time to stage a counter-assault. The case has to be made that we need to drive all forms of segregation and selection from our education system. And it will take some courage, but the idea that differences in educational achievement are purely down to the quality of the schools has to be challenged, too. Class is largely to blame: but it is almost completely absent from the debate.

It is a case that Labour has failed to make: the Government can boast that it is merely building on the policy foundations laid by New Labour. And now, there are worrying signs the Tories may be building a new consensus. Ed Miliband has handed the shadow education brief to arch-Blairite Stephen Twigg to the obvious delight of Education Secretary Michael Gove. "On the basis of everything that he has said so far," he crowed "I think there may be a real change in the Labour Party's approach towards the issue, so I encourage him on the path of virtue and say no more."

It marks a dramatic turnaround for the party that unleashed the comprehensive schools revolution. New Labour's leading lights often claim Tony Crosland, the standard bearer of the party's post-war social democratic right, as their own. But he was a passionate advocate of comprehensive education. "If it's the last thing I do, I'm going to destroy every fucking grammar school in England," he supposedly told his wife. "And Wales and Northern Ireland."

Alongside the tireless education campaigner Fiona Millar, I took the case to enemy territory this week: City of London School, which charges over £13,000 a year in fees – or nearly two-thirds of the median pay-packet. Despite a spirited defence of private education by Eton-educated Conservative MP Jacob Rees-Mogg, a show of hands revealed that far more pupils had been swayed by our arguments. If we can win the debate there, we can win it everywhere.

Separating children on the basis of their parents' bank balances denies children the opportunity to mix with others from a whole range of backgrounds, fostering divisions at the earliest age. What's more, private schools are a form of taxpayer-subsidised class privilege: because they are absurdly granted charitable status, they save up to £100 million in tax breaks.

But the key point I made to the City of London boys was that – as well as being unable to grow up with kids from non-privileged backgrounds – their parents were wasting money. Above all else, it's class that determines how well children do at school. This August, a study by the OECD unsurprisingly found that privately educated students did far better than those at state schools. But it also found that those with middle-class backgrounds did just as well at comprehensives.

My own experience backs it up. My primary school was in the bottom 5 per cent by results: I was the only boy to end up at university. That's not because I was naturally brighter, but because I had odds stacked in my favour: university-educated parents, an environment free from the stresses that poverty can impose on everyday life, a comfortable living space, and so on. It's a horrible expression, but "cultural capital" – in part, having middle-class parents with degrees – gives you a massive head-start whichever school you go to. A recent study in Scotland revealed that, by the age of five, children with better-off, degree-educated parents have, on average, a vocabulary 18 months ahead of their poorer classmates. No wonder, then, that another study looking at middle-class children sent to inner-city comprehensives found most "performed brilliantly". Indeed, 15 per cent of those who went on to university ended up at Oxford or Cambridge. That's why academies and free schools completely miss the point. But they're also deeply damaging. Academies are not accountable to local authorities; it's claimed that their results are rising faster than other schools, but – except for maths and English – they don't release a breakdown by subject.

Free schools are an even greater threat. Drawing inspiration from Sweden, where they were introduced in the 1990s, they are taxpayer-funded schools set up by private individuals or businesses. And yet according to the OECD, Sweden has slipped back significantly in literacy, maths and science throughout the noughties.

If we're going to look to a Nordic country for inspiration, it should be Finland. It has no league tables, no selection, few private schools, and free school meals for all. It also has the best performing education system in the world. But it also has another key ingredient: far lower levels of social and economic inequality. Ending the segregation of our children would be good for them and for society. But if we really want to transform education, we have to wage war against our society's ever-growing economic divisions. The obsession of New Labour and the Tories in tinkering with school structures is a damaging distraction. If the consensus on education is to be taken on, the comprehensive dream and the struggle for equality must go hand in hand.

Owen Jones is author of 'Chavs: The Demonisation of the Working Class'