John Rentoul: Top marks, Ed. Now for Tesco schools

Ed Balls, the Children's Secretary, should be carried shoulder high by a bodyguard of Blairites to receive the cheers of New Labour modernisers. Not the kind of thing that is normally written about him. But credit where it is due: he has just done something important, brave and right.

Earlier this month, by contrast, we Blairite ultras were wailing and beating our breasts, burning effigies of Balls in elaborate ceremonies of mourning for the loss of Andrew Adonis, moved from education to transport in the reshuffle. Adonis, the driving force of the academies programme, had only days before been described to me by an eminent professor as "the single most successful minister in Brown's Government".

Now he was gone, and the inhabitants of Jurassic Park were celebrating. All the vested interests of educational conservatism, who see academies as a right-wing plot to grind the faces of the poor under the jackbooted heel of evangelical Christian moneymen, but who have almost no interest in what is actually happening in underperforming schools that are failing working-class children, wrote letters to the newspapers crowing over Adonis's departure.

I do not know why Adonis went, although I am sceptical when I am told that Ed Balls says "the Department of Transport stole him", "he asked to go" and "I would have far rather he had stayed". I can also testify that Adonis has been an enthusiastic advocate of a new high-speed north-south railway for some time. If there is one minister who could make such an ambitiously green project happen, it is he.

But his departure fills me with foreboding for the fate of academies. It is only school by school, with immense commitment from the centre, that the long tail of educational disadvantage in this country can be tackled. Until the number of academies reaches a critical mass, it depends on a tireless champion at the top of government, and Adonis was certainly that. But three things give me hope.

One: Adonis's successor, Jim Knight, is a fine minister with the right instinct – that schools exist to serve pupils and not the other way round. He is not a zealot, but it is just possible that the academy programme has passed the phase where zealotry is needed.

Two: the Conservatives have converted to the cause. Contrary to the libel that academies are the Tory policy of grant-maintained schools in another guise, the Opposition's support for academies represents a huge gain for the left. Grant-maintained schools, often in areas where deprivation means the Aga is playing up, were encouraged to become selective grammar schools. Academies are explicitly non-selective, and David Cameron and Michael Gove are now committed to the comprehensive principle.

Three: Ed Balls's decision last week to launch 12 pilot schemes for new providers to run "sin bins" for excluded pupils convinces me that the Blairite pulse of public service reform still beats at the heart of government. The crucial feature of the new policy went barely reported. The Daily Mail was taken by the idea that one of the pilots was an "Army cadet boot camp", while The Daily Telegraph was more impressed by one based at a city farm.

The really important point about the scheme, however, is that it is open to profit-making companies. Only one of the pilots is run by a company, but the principle is the thing. The Mail and Telegraph may not have noticed, but the true conservatives certainly did. Christine Blower, the acting general secretary of the National Union of Teachers, said: "This is the creeping privatisation of education." When will the hard left change the record? They have been playing that one since Thatcher's day – it wasn't even true then, certainly not since Kenneth Clarke was Education Secretary, and it has been a wilful distortion since 1997.

I won't call it the choice agenda, because that seems to put hackles up, so let us call it people power. Outside providers contracted to provide a public service usually have greater incentives to produce results, to secure customer satisfaction and to innovate.

The easy thing for Ed Balls to do would have been to invite other public sector bodies, charities and not-for-profit providers to bid to take over "sin bins" – properly known as Pupil Referral Units – from local education authorities. But why stop there? Why shouldn't profit-making companies also be allowed to offer the same or better service if they can do it at the same cost? To his credit, Balls decided that they should be allowed.

This may seem a small experiment, but Pupil Referral Units matter. They are the hidden corners of last resort where children that no school will tolerate end up when they are not simply truanting or in youth custody. Spending on them has increased under Labour and many good things have been done, but what they really need is new providers coming in with new ideas – such as boot camps and farms – about how to divert anti-social behaviour and give unmotivated young people a sense of purpose.

This principle of diversity should be applied more widely. Most schools in the country are far too similar, and we should not shut out any innovative providers on the primitivist basis that profit is a dirty word. As John Hutton, now Defence Secretary, asked at a meeting at Chequers in July 2005: "Why do trust schools have to be charities? Why can't they run schools for profit?"

I see no good reason, apart from popular anti-capitalist prejudice, why dynamic, well-managed companies such as Tesco should not run state schools. This was too much, even for Tony Blair. "I don't think so," he told Hutton. Anthony Seldon in Blair Unbound quotes an aide who commented that Blair knew that he was "artificially limiting the supply of schools" but that "politically he realised it was going too far".

That is why last week's announcement is so significant and so welcome. Clever, too, to make it after Adonis has gone, so that it isn't portrayed in the press as an ideological struggle between the Brownite boss and his Blairite junior. What a paradox if Ed Balls should finally go further than Blair dared to go.

More from John Rentoul