Gordon Brown will face another challenge to his position as Prime Minister around the time of Labour's annual conference in September, Business Secretary Lord Mandelson predicted.
Lord Mandelson said a "small group" within the party would never be reconciled to Mr Brown's leadership, but insisted he would not "lose any sleep" over it.
Meanwhile, Foreign Secretary David Miliband indicated that he considered following James Purnell in quitting the Cabinet last Thursday.
Mr Miliband told The Guardian that his decision on whether to stay in the Government had been on Thursday, adding that he accepted his old friend's decision to resign as Work and Pensions Secretary was made "in good faith".
Their comments came as former Home Secretary Charles Clarke said that Mr Brown must win two upcoming by-elections in Norwich North and Glasgow North-East and improve Labour's standing in the polls if he was to escape calls for his removal.
Mr Clarke is expected to use a BBC interview today to indicate that he still wants Mr Brown to go.
And prominent backbencher Jon Cruddas is warning that Labour could be finished as a major force in British politics unless it had a fundamental rethink of its policies and approach.
In an interview with the Daily Telegraph, Lord Mandelson acknowledged that Mr Brown had not silenced his internal party critics when he faced down demands for his resignation at a meeting of the Parliamentary Labour Party in Westminster on Monday.
"There's a small group who keep coming back. They won't be reconciled to the Prime Minister's leadership," said Lord Mandelson, who was last week named First Secretary of State - effectively deputy prime minister.
However, he added that he would not "lose any sleep" over the threat they posed.
Meanwhile, Mr Miliband left no doubt that he was faced with a decision over whether to stand down as Foreign Secretary last week during a reshuffle which saw 10 ministers leave the Government.
Many commentators believe that if he had followed Mr Purnell in quitting as polls in the council and European elections closed on Thursday night, he would have forced Mr Brown out of office. He was not one of the ministers who flocked to the television studios to declare their loyalty to the Prime Minister that night, though he let it be known he would not be resigning.
He told The Guardian: "I'd made my decision on Thursday. Sometimes you can make your decisions with great planning and calculation and sometimes you have to make them rather more quickly.
"James made his decision in good faith, I made my decision in good faith ... we all have to live with our decisions."
The Labour Party has a settled view of who should be leader, he said, adding: "This divides people who are close and people feel passionately one way and people feel passionately the other way."
Speaking after a three-hour special session of Cabinet in Downing Street today, Mr Miliband said Labour's leadership had been given an "electric shock" by the disastrous European election results, which saw the party finish third with just over 15% of the vote.
He urged fellow ministers to get out and prepare for "the fight of our life", saying that all Cabinet members should be given "free rein" to go out and make the case for Labour. Voters do not know what the party and Mr Brown stand for, he admitted.
"We need to make sure the talents and energies of not just the whole of the Cabinet, but the whole of the Labour movement are given free rein because one in 20 people voted for us," said Mr Miliband. "The truth is that we are not automatons. We all have to get out there."
Mr Clarke said Mr Brown had several hurdles to overcome if he is to remain leader.
The former Home Secretary told Channel 4 News: "If, for example, our poll ratings go up or we win these by-elections, I think the issue will go away and he can be confident he leads us into the next election.
"If, on the other hand, he somehow doesn't fulfil these things or electorally we do badly or whatever, it might be that the issue will still be there."
After Mr Brown acknowledged in his speech to the PLP that he had "weaknesses", Lord Mandelson appeared to offer him hints in how to improve his performance as leader.
The Business Secretary told the Telegraph: "I believe in leadership and in being decisive. Secondly, in listening to people and respecting official advice you receive. And thirdly, introducing a bit of humour and jollity to your work."Reuse content