Thousands of offenders will avoid jail under an overhaul of sentencing powers that will focus on cutting reoffending.
The sentencing Green Paper will aim to stop the revolving door of crime, divert criminals with mental health, alcohol or drug abuse problems into treatment and bring in a rehabilitation revolution with a series of programmes designed to stop repeat offenders from reoffending.
Other proposals include halving sentences for those who plead guilty early and curtailing judges' powers so that indefinite sentences, currently being served by more than 6,000 prisoners, will be reserved for only the most serious of offenders.
Payment by results will also be piloted along with proposals to involve the private and voluntary sector in running unpaid work sentences for offenders.
Justice Secretary Kenneth Clarke has said the Government is planning "extremely serious changes" to sentencing.
"The biggest thing that we're addressing in the current system, where we have an enormous prison population, sentences have got much longer than they used to and all the rest of it, is the rate of reoffending," he said.
Three in four criminals offend again within nine years and 40% commit another offence within 12 months, the latest figures show.
Pilot projects, such as the heron wing at Feltham young offenders institution in south-west London, are already helping 15 to 17-year-olds to turn their lives around.
Policing and Criminal Justice Minister Nick Herbert said charitable groups were brought in to help the offenders get into education, training or employment.
Some are paid by results, with cash payments made if the young person starts education or work, fulfils probation demands and stays out of trouble.
The unit has a reconviction rate of 14% over one year, one of the lowest rates nationwide, compared with a national average of 78% over two years for young offenders.
"It makes sense because it is a way of unlocking investment in rehabilitation without increasing the cost to the public purse," Mr Herbert said.
"Everybody wins. Society wins because these young offenders go straight, there is less criminality and people are safer. The taxpayer wins too because the cost of reoffending is huge."
Short-term sentences of less than 12 months are to be kept, but fewer will be handed down as they are expected to be used mainly for serious first time offenders or when other options have been exhausted.
The controversial indeterminate sentence for public protection, described by the Prison Reform Trust as "one of the least carefully planned and implemented pieces of legislation in the history of British sentencing", is expected to be restricted to only those jailed for 10 years or more.
And a Tory election pledge that anyone caught carrying a knife could expect a jail term will be scrapped, with no minimum tariffs being set for knife crimes.
Reductions in sentences of up to 50% for those who plead guilty early are also expected, along with a push for fewer defendants to be sent to custody on remand.
Prisons will also be made places of hard work as part of efforts to prepare offenders more effectively for the outside world.
There will also be a greater focus on victims along with measures designed to tackle drug and alcohol addiction and mental illness.
And Mr Herbert has said community sentences cannot be seen as a weak option if the Government's rehabilitation revolution is to succeed.
They "must be strengthened" so they punish offenders, help rehabilitate them and get them out of the criminal justice system, he said.
The Ministry of Justice has said a "more intelligent approach" is needed, with the most dangerous offenders being locked up while ensuring courts "have the power to make the right response to stop people committing crime".
It is thought the plans will cut the prison population of more than 85,000 by 3,000 by 2014.
Shadow justice secretary Sadiq Khan said: "A sentencing review that should have been about reducing re-offending and protecting the public seems to have become just an exercise in cutting costs.
"Sadly, despite (Prime Minister David) Cameron's rhetoric about putting the public first, the Government have retreated to a traditionally Tory ideological approach, setting an arbitrary target for the prison population, rather than addressing the primary concern of protecting the public."
Conservative former home secretary Lord (Michael) Howard said Mr Clarke was wrong to think that he could reduce the prison population and keep the crime rate down.
Lord Howard said he stuck by his famous 1993 claim that "prison works".
"I warmly welcome (Mr Clarke's) approach to rehabilitation and I hope I'm going to be able to agree with everything in the Green Paper," Lord Howard told the BBC Radio 4 Today programme.
"The one point where I do disagree with him is when he suggests that the remarkable fall in crime we have seen since 1993 has nothing to do with the rise in the prison population.
"We have seen a very significant increase in the prison population since 1993 and an almost halving in the rate of crime. These two things have gone together - they are connected."
Lord Howard rejected arguments that crime fell during the past two decades because of increasing prosperity.
"We have had a recession in the last two or three years and crime has continued to fall," he said.
"There is simply no correlation between the amount of crime and economic conditions. There is a direct correlation between the number of people in prison and the amount of crime.
"It is obvious - if persistent serious criminals are in prison, they can't commit crimes against the public."
Mr Clarke denied that he was at loggerheads with Lord Howard over prison numbers, telling the Today programme: "You will find that Michael and I agree, or I very much hope so".
The Justice Secretary said that it had not been proven that higher prison populations drive down crime.
"You can't prove it one way or the other," he said. "Countries like Canada and the Netherlands and the state of New York have had falling crime with falling prison populations.
"For serious criminals... of course you send them to prison and while they are in prison you have the benefit of knowing they are not going to commit crimes that they might otherwise do while they are inside.
"That is not an adequate approach to the 85,000 people we have got in prison now and it is certainly no argument for saying let's bang up another 20,000, because the bottom end of the prison population contains a lot of people who, if we devoted more sensible attention to it, probably could be stopped from committing more crime when they come out.
"That's what policy has got to concentrate on."
Mr Clarke rejected suggestions that his new policy was driven by the desire to cut the population in jail to save money.
The expected 3,000 reduction was "the best estimate", and not a target, he said.
"I didn't start from the proposition that I want 3,000, 10,000 or 20,000 fewer prisoners or 10,000 or 15,000 more," Mr Clarke told Today.
"I started from the proposition that there are big failings in our system because it is not stopping people reoffending, and I don't think the answer is to send more people to prison or to have rules which automatically send people to prison for a set tariff.
"Judges know what is a serious offender and judges know which offenders need punishment. We should leave more of that to them and we should concentrate on making the bit of the system which isn't working - the reform of offenders to stop them reoffending - more effective."
Mr Clarke accused the former Labour administration of "neglect" of rehabilitation in its pursuit of "headlines... to show they were banging more people up".
"There has been a huge increase in the number of people in prison which is not only not good value for money, but even more importantly I don't think that's the right way to keep on protecting people," he said.
"Some of my critics just think you should put more and more people in prison for longer and longer and longer.
"I'm afraid I personally don't think that's the best way of protecting society. We have got to stop having this revolving door where people go in to prison, serve their time and come out and within less than a year, half of them have committed some more crime.
"The prison system at the moment is not doing one of the jobs it's meant to do - stopping us developing a criminal underclass who just keep committing more crime as soon as they come out. That's because we are not tackling the problems of drugs, alcohol abuse and mental illness and we are not helping those who can go straight to prepare themselves for a job and settling down when they get out."
The assistant general secretary of the National Association of Probation Officers, Harry Fletcher, told Today: "The Green Paper which is going to be launched today is not primarily about value for money or building on what works.
"It is motivated by an ideological wish by the coalition to drive down costs and introduce the private sector.
"I think it will mean a new form of community service that will be rebranded and relaunched and because the groups are larger it will lead to more reoffending."
Asked whether Prime Minister David Cameron believed that reducing prison numbers was compatible with cutting crime levels, a Downing Street spokeswoman said: "The system is not delivering what really matters, and that is improved public safety through more effective punishments that reduce the prospect of criminals reoffending time and time again.
"Society has a right to expect that offenders are effectively punished and the Prime Minister's view is that prison is the right place for serious and dangerous offenders and he will ensure that there are always sufficient prison places available."
The spokeswoman added: "The crime levels now, even though they may be the lowest on record, are still too high in the public eye."
Asked whether Mr Cameron had changed his view on knife crime since the Conservative manifesto promised that "anyone convicted of a knife crime can expect to face a prison sentence", the spokeswoman said: "Ultimately it is up to judicial discretion, but clearly prison is going to be appropriate for particular cases.
"Those people who use knives in committing crimes should expect prison, but we all know that knife crime covers a wide range of offences and circumstances and courts must have that flexibility.
"We are talking about how you give the most appropriate punishment to those people who use knives when committing offences. You also have to remember that we are in a coalition Government."