Mr Balls said the party had to show it could maintain "fiscal discipline" if voters were to accept it offered a credible alternative to the policies of the coalition.
His comments came as Labour leader Ed Miliband used a keynote speech to declare he was ready to make the "difficult choices" necessary in economically straitened times.
But after Mr Miliband again refused to accept the last Labour government had overspent in the run-up to the economic crisis, the Tories said he had "learned nothing from the past" and could not be trusted with the future.
In his speech the Labour leader offered the example of Government cuts to the winter fuel allowance as one change a Labour government may be unable to reverse if the party won the next general election.
However, with the structural deficit in the public finances now forecast to continue beyond 2015, Mr Balls warned the belt-tightening would have to go much further.
While he said the Government was cutting "recklessly fast" - choking off economic growth - Labour would still face difficult decisions if it succeeded in regaining office.
"We cannot win the arguments for the alternatives, short-term and long-term, unless people say that we would make the tough decisions in (the) long-term interests of the country based on fiscal discipline and probity. That is an essential argument for us to win," he told BBC Radio 4's The World at One.
"There is going to have to be cuts, there is going to have to be difficult decisions. We would have to have cuts in police, we would have to have cuts in the schools budget, we would have to have cuts in the defence budget.
"We can make no commitment to reverse any of the Government's tax rises or spending cuts because we don't know the state of the economy we are going to inherit and what the fiscal position will be."
In his speech, Mr Miliband said the worsening economic outlook meant that, unlike the last Labour government, an incoming Labour administration would not be able to rely on a buoyant economy to support its plans.
"It means that the Blair/Brown approach will not be enough," he said. "Each time New Labour won an election, it won at a time when business was prospering.
"Next time we come back to power, it will be different. We will be handed a deficit. We will have to make difficult choices that all of us wish we did not have to make.
"So we must rethink how we achieve fairness for Britain in a time when there is less money to spend. It's Labour's responsibility to find a new approach for tough times. So we will be a different party from the one we were in the past. A changed Labour Party."
He promised a new economic approach based on rewards for long-term wealth creation while tackling "vested interests" which squeezed the living standards of ordinary families.
As an example, Mr Miliband said he would get tough with the energy companies which took advantage of the elderly - requiring them to ensure the over-75s were given the lowest tariffs.
Aides strongly denied suggestions the speech was a relaunch of his leadership, amid falling opinion poll ratings and growing criticisms of his performance.
Mr Miliband said that after criticising his attacks on "predatory" capitalism, the other party leaders were now trying to move on to his political ground.
"The same Prime Minister who said I was anti-business when I called time on undeserved rewards at the top now claims he's desperate to stop them too," he said.
"I say to the Prime Minister, who are you trying to kid? No one is ever going to believe you're the man to take on crony capitalism. But now that you have accepted that this is the battleground of politics, I say: 'Bring it on'."
Earlier, during a radio interview, Mr Miliband brushed aside criticisms of his leadership by his former adviser, Lord Glasman, who said it was "all crap".
"It is what happens. You get noises off," he told the BBC Radio 4 Today programme. "I don't agree with him when he says it is all crap."
He also warned his opponents not to write him off. "Don't declare the result of the race when it is not yet half-run," he said. "I have a very strong inner belief that I will win the race."
However Conservative Party deputy chairman Michael Fallon said he still needed to produce a "credible plan" for cutting the deficit if Labour was to be taken seriously on the economy.
"If he seriously accepts there's less money to spend, he would stop making billions of new unfunded spending promises and instead tell us what Labour would cut," he said.
"Instead Ed Miliband has given us another relaunch but still no credibility."
PA
Comments