For many years it has been abundantly clear to those working in the criminal justice system that the slogan "prison works" is nonsense. Even some Labour ministers, who publicly subscribed to the doctrine, privately accepted that the doubling of our prison population over the past two decades has been counterproductive.

What has always been lacking is a self-confident government minister prepared to challenge the conventional wisdom and to change course. Mercifully, in the Justice Secretary, Kenneth Clarke, we now have such a minister.

The green paper he published yesterday proposes to restrict indeterminate sentences, extend the use of bail for suspects and to award greater sentence reductions for defendants who pleaded guilty early. Mr Clarke also wants to make community sentences a more credible alternative to custody, by toughening up work schemes.

The Justice Secretary has also ditched the populist Tory pledge from the election that anyone convicted of carrying a knife must face an automatic jail sentence. The relatively modest immediate target is to cut Britain's 85,000 prison population by 3,000 by 2014-15. But Mr Clarke has begun the important work of turning the penal oil tanker around.

There is immense pressure on the £8.9bn Justice Department budget, which is facing a 25 per cent cut over the next four years. The fact that it costs the public purse upwards of £25,000 to lock up a single prisoner for a year itself provides a compelling argument for sending fewer offenders to jail. But to Mr Clarke's credit, he is not making the case for his reforms purely on financial grounds. He recognises that unthinkingly cramming our jails was the wrong policy even when the state coffers were full.

Advocates of prison cite the fall in the crime rate over the period in which the prison population has been rising. Yet the crime rate has fallen in other Western countries over periods in which prison populations have been falling. Moreover, 60 per cent of inmates held in British jails for a year or less go on to re-offend. The National Audit Office estimates that this costs Britain between £7bn and 10bn a year. It is surely impossible to square this figure with the idea that "prison works".

If there are reservations about Mr Clarke's proposals, it is that the prison budget is facing such severe cuts that rehabilitation programmes risk being under-funded. Yet sending fewer offenders to prison ought to create a virtuous circle: less overcrowded prisons will mean that the ratio of staff to offenders will be lower. A smaller population should enable more of the prison's budget to be spent on tackling drug and alcohol abuse and on other rehabilitation schemes. This is one area of public spending where it is possible to envisage more being done for less.

The political fetishisation of sending offenders to prison for longer periods – which has held sway ever since the former Home Secretary, Michael Howard, came up with his notorious slogan – has been a disaster. We have ended up warehousing petty criminals, the mentally ill and those addicted to narcotics at huge public expense. A fixation on punishment has turned rehabilitation into an afterthought.

The fact that Mr Clarke is a member of a government that includes the Liberal Democrats, who have long advocated prison reform, probably strengthened the Justice Secretary's hand around the Cabinet table when it came to overturning this traditional Conservative policy. But that should not diminish the praise that is due to the Justice Secretary for having the courage and clarity of thought to begin to rectify a mistake.