One senior minister close to Mr Brown said: "Our worry is that Blair's support is going to be transferred directly to Cameron. The longer that Gordon has to wait, the worse that will get. Blair will have to hand over to a new leader at the 2006 conference."
The minister added: "A few weeks ago, someone suggested that we could be facing a meltdown. Everyone laughed, but they would not be laughing now."
The Chancellor has told allies the Tories have been strengthened by a contest for their leadership, and he would prefer to win through a challenge for the Labour leadership, rather than as a "coronation".
John Reid, the Defence Secretary, is the favourite among cabinet colleagues to throw his hat in the ring, but Mr Brown's allies are growing more anxious about the inheritance, including the Prime Minister's drive for Tory-backed reform of public services such as education.
Baroness Morris of Yardley, the former education secretary, will give her backing this week to an alternative education "white paper" by Labour MPs opposed to allowing selection in secondary schools. Signatories include John Denham, a former home office minister, Angela Eagle, the former work and pensions minister, and dozens of Labour MPs.
Lady Morris said she supported the alternative proposals because the government plan would "exacerbate" class divisions and make it harder for children from poorer backgrounds to get into good schools.
The Labour peer, who resigned as an MP at the last election, warned that the education reforms, backed by David Cameron, would "muddy" the dividing line between Labour and the Conservatives on education policy. She accused the Government of, in effect, allowing schools to select pupils by setting their admissions policy and taking the power from local authorities.
"It might not be straight academic selection - but the same kids always score and the same kids always lose out," she said. Baroness Morrissaid the reforms would "make matters worse" and distract schools from improving standards.
"They will make it more difficult for us to break that link between social class and educational achievement," she said.