Commenting on the rapidly increasing number of remand prisoners, he said that people released on bail committed at least 50,000 offences every year.
''That is at least 50,000 victims of crime, and the public deserves to be protected,' he declared.
Mr Howard spoke after John Bartell, chairman of the Prison Officers' Association, said that Britain's overcrowded prison system was 'spiralling rapidly out of control'.
Mr Bartell added: 'Disturbances in the prison system have risen from 60 a year to 147 in the last year, and I am afraid we will lose a number of prisons through rioting.'
A report published yesterday by the Prison Reform Trust showed that the number of remand prisoners had reached an all-time high of 12,100 in January.
Last October, Mr Howard told the Tory party conference his measures to tackle crime could result in more people going to prison, and he would not flinch from that.
Adam Sampson, the Prison Reform Trust's deputy director, told BBC Radio 4's PM programme last night that magistrates appeared to be reacting to the strong leads from ministers to get tough on law and order by remanding more people in custody.
'Faced with a marginal decision between bail or remanding in custody, magistrates are increasingly opting for the latter option, rather than taking a risk by releasing somebody on bail.'
But he added: 'There is no increase in the seriousness of the sorts of offences for which people are coming before the courts.
'Six out of 10 people remanded into custody, when they eventually come to court, are either acquitted or given a non-custodial penalty.'
But Mr Howard retorted: 'When Mr Sampson talked about magistrates not being prepared to take a risk by releasing people on bail, the risk, of course, would be run by the public.'
Referring to the 60 per cent of remand prisoners who are not jailed at trial, he told the PM programme: 'That may well be because the judges and magistrates have taken into account, when sentencing them, the fact that they have already spent a period in custody.'