Andrew Grice: How first his bottle and then his brother cost David Miliband his dream

A career frustrated by crucial mistakes

In the end, his brother made a difficult decision easier for him. David Miliband was already edging towards quitting frontline politics so he did not get in his brother's hair. But he knew that leaving the Shadow Cabinet would bring some accusations of flouncing out, petulance and even disloyalty to his brother Ed and his party.

But the Labour leader unwittingly gave David some cover to make his exit from the front bench by condemning the decision to invade Iraq in his conference speech on Tuesday. "It showed everyone how impossible it would be for David to stay. He can't be accused of petulance now," one ally said.

Other friends of David insist his dramatic attempt to end the Labour psychodrama was not about policy differences but "doing what is right for Ed". One said last night: "He would have been constantly compared to his brother. If David had sneezed, the media would have said he had done it better than Ed would have done."

How did New Labour's golden boy and likely "heir to Blair", who became Tony Blair's head of policy at the age of 29 in 1994, allow the crown to be snatched from him by his younger brother at the last minute?

Some critics claim David's ill-fated leadership campaign illustrated his fatal flaw – the lack of a killer instinct. They point to his refusal to challenge Gordon Brown for the Labour leadership when Mr Blair stood down in 2007. Mr Blair wanted him to run but had not done his one-time protégé any favours by promoting him to a senior Cabinet post; he was Environment Secretary at the time.

David was worried about the impact on his family and told friends he "didn't want to be put through the mincer" by a ruthless Brown machine.

He did not believe he could win – a judgement not shared by some senior Labour figures, who think that Mr Brown's lack of a distinctive policy agenda would have been flushed out in a leadership election.

A year later, David was again accused of dithering when he wrote a newspaper article saying Labour should not succumb to "fatalism" without mentioning Mr Brown. It was meant as a rallying cry to cheer up a despondent party but was seen as a leadership bid. When there was no follow-through, it raised questions about his judgement. When his friend James Purnell quit the Cabinet in June 2009, many including Mr Brown assumed a coup was underway but he was a lone gunman. David suddenly held Mr Brown's fate in his hands. If he had resigned too, the Prime Minister might well have been toppled. But David had no intention of walking out. Some say he feared that, like Michael Heseltine in 1990 when Margaret Thatcher fell, he feared the dagger-wielder would not inherit the crown.

His hesitation in both 2008 and 2009 when Mr Brown was vulnerable led some Labour MPs to doubt his credentials for the top job. David was first out of the traps when Mr Brown resigned after Labour's election defeat in May. But the Blairite label hung around his neck like a millstone.

The former foreign secretary decided not to criticise Labour's record in power. Ed had no such qualms, most spectacularly on Iraq. "David became a magnet for all the bad feelings about New Labour. All the nasty stuff stuck to him," one ally said.

"David was spooked by Ed," one former cabinet minister said yesterday. "He couldn't attack his brother." The former Foreign Secretary rejected the advice of allies who wanted him to play the "experience" card. One said:"David should have asked: 'who would an American president want to phone at 3am?' But he wouldn't do it."

David can only reflect on what might have been as he contemplates more time with his family and in his favourite fish and chip shop in his South Shields constituency and, possibly, an international job in the long term.

At 45, some friends do not rule out another tilt at the leadership by David. Others are more brutal. "It's over for him," one said. "If Ed loses the election, it will be Yvette Cooper's turn."