Alan Johnson announced plans to slash police paperwork by cutting the size of the stop and search form.
But in a speech to the Police Superintendents' Association conference, he admitted the "bureaucracy dragon has yet to be slain".
He told senior officers individual forces and governing police authorities must also take responsibility for cutting paperwork.
Mr Johnson said: "Central government may have been slashed but we were never the only manufacturer of red tape.
"Local requirements are often equally, if not more, burdensome and this needs to be addressed too.
"For example, while the stop and account form has been abolished I have heard of instances where neighbourhood police officers are still filling in forms even though it is no longer required."
He added: "Making further inroads will require more action at force level and authority level."
Mr Johnson pledged to "radically slim down" some paperwork to save the time of officers.
The senior Labour politician said officers would only record the reason for stopping someone and their ethnicity.
He said: "Over the last few years, we've made huge efforts to cut the laborious and unnecessary paperwork that chains police officers to their desks.
"Currently, regardless of whether someone who is stopped and searched is arrested, police officers still have to fill in the form.
"It is obviously essential to record the ethnicity and the reason they were stopped, so any complaint can be properly considered.
"But there should be no need for the police to record anything further."
Increased paperwork for street stop and searches was introduced as a result of an inquiry in the aftermath of the murder of Stephen Lawrence.
The information officers record when they stop a suspect but do not arrest them could be reduced from more than 10 points to just two or three.
Details such as what was found, the registration of any vehicle involved, and if any property was damaged during the search, would be dropped.
Officials predict the changes could save officers up to 200,000 hours every year.
Following a pilot scheme the measures are likely to be included in the forthcoming Policing, Crime and Private Security Bill.
The change is the latest in a series of moves aimed at increasing the amount of time officers can spend not doing paperwork.
Over the last 18 months ministers have scrapped a longer stop and account form and the requirement for officers to fill in a form explaining how they spent every 15 minutes of their shift.
Paperwork linked to stop and searches conducted under counter terrorism legislation will remain unaltered.
Such searches have been criticised for being too intrusive and for undermining public confidence in police.
Mr Johnson also spoke out on Tory plans to inject more democratic accountability into the running of police forces.
The proposals, which could include directly elected commissioners with some control over police tactics, have been fiercely opposed.
Mr Johnson said senior officers should be wary of those offering apparently simple solutions to complex problems.
He said: "When the public say they want the police to be more accountable that does not mean they want the dubious delights of elected police boards.
"It certainly does not mean they want politicians pulling the strings or telling police how to do their jobs in London or elsewhere.
"Locally, they want a name and a number they can call about problems they see in their neighbourhood.
"They want the problem to be dealt with quickly, preferably by a police officer with a familiar face."
Scotland Yard boss Sir Paul Stephenson launched a staunch defence yesterday of keeping operational decisions and politics separate.
He said his colleagues must remain aloof from party politics and "no sensible politician would think otherwise".
In a rebuke to comments by an aide of Tory London Mayor Boris Johnson, he added that he was the "captain" of the Metropolitan Police.
Deputy Mayor Kit Malthouse said he and Mr Johnson "have our hands on the tiller" of the £3.5 billion force which employs some 50,000 people.
Mr Johnson also announced funding of £40 million for local police units across England and Wales.
The money will be made available to senior officers who work with other local authorities and charitable groups.
It will be used in smaller projects aimed at improving public confidence, reducing anti-social behaviour and neighbourhood policing.
Earlier, Ian Johnston, president of the Police Superintendents' Association, said politicians and police must be "honest about the hard times ahead".
He said a period of growth for forces across the country was coming to an end, with some areas suspending recruitment and looking to cut costs.
But Mr Johnston, a chief superintendent from Gwent, said critics often forgot the value of policing when they were highlighting its cost.
He said: "We all accept that in the future we will have to do more with less.
"How will we make efficiency savings? Some forces will question whether further efficiency savings can be made.
"Will less funding result in a police service with more police staff and fewer fully warranted police officers?"
Mr Johnston said police chiefs must think now about how to manage reduced budgets, with potential cuts of up to 20%, in the future.
He said: "It is becoming increasingly clear that whichever government is returned there will be a massive hole in public finances."
The senior officer said politicians could help police by pushing ahead with national IT projects to improve links between forces.
And he said recommendations from ageing reports aimed at cutting bureaucracy have still not been put into use.
Mr Johnston added that there had been a huge growth in civilian back office staff, costing money that could be spent on frontline officers.
Speaking to journalists after his speech, Mr Johnson said the stop and search form announcement is part of a "continuing war on bureaucracy".
He said a forthcoming report by former Police Federation chairwoman Jan Berry will examine ways of reducing paperwork at the local level.
Mr Johnson said many officers have raised concerns about the length of stop and search forms in his three months as Home Secretary.
And he rejected claims the changes may be a backward step on recommendations in the Macpherson report.
Sir William Macpherson investigated the botched police inquiry into the racist murder of Stephen Lawrence.
Mr Johnson said: "We have made huge strides. Hundreds and thousands of police hours are being saved.
"We need legislation to change the forms but we think we can change it and save 10 or 12 minutes for each officer filling in their form. Adding that together gives us 200,000 hours saved over a year.
"We think we can stay true to Macpherson and certainly we will be consulting widely in the run up to the legislation.
"We need to record ethnicity and certain details. Do we need the number of questions that are there at the moment? I am sure we can cut those right back."Reuse content