Kim Howells has launched a withering attack on "incompetent" Downing Street backroom staff.
He protested that highly paid aides at No 10 had failed to say "very hard things" to the Prime Minister about improving his public image.
Mr Howells admitted the party had been "struggling" since the failed leadership coup nine days ago. He said: "I'm an eternal optimist about our ability to win the election, but we took a bit of a blow last week – it's set people back a bit."
Mr Howells, the MP for Pontypridd for 21 years, said he had never supported ousting the Prime Minister "because I always thought things would change at No 10".
He said: "People around him self-evidently were failing to communicate Gordon's strengths, which he did manage to communicate during the most critical period after the immediate failure of the American banking system.
"They never managed to capitalise on that. They were, I think, incompetent and it seemed to me there was real evidence there were people in very important places not capable of saying very hard things to the Prime Minister."
He said Mr Brown's team should have urged him to be "more himself" and "much more open about what he believed about a whole range of issues and more accessible to the public".
The former minister said: "Some of these people who are very highly paid haven't done a very good job of projecting him. The public perception of Gordon Brown is not the perception I have of him – he is a much more solid individual than is portrayed publicly."
He said the problem of projecting the Prime Minister's strengths to the country still remained two-and-a-half years after he succeeded Tony Blair. But Mr Howells added: "There is still time for them to do it."
His comments echoed the scathing criticism of the Downing Street operation in a new book by Peter Watt, the former general secretary of the Labour Party.
Mr Howells told The Independent that he had been encouraged by Mr Brown's recent performances in the Commons. He said: "His strongest moments in Prime Minister's Questions have been over the last few weeks when he has been less defensive and more ready to attack from the ground he knows best, such as the shambles of Cameron's position on the nation's finances."
Mr Howells said that the delay by cabinet ministers in endorsing Mr Brown last week had aggravated the party's problems.
He said that he wished some had been "a bit more energetic" in supporting the Prime Minister, admitting the delays had fuelled speculation over his position.