About half the members of the Shadow Cabinet are ready to tell Iain Duncan Smith that he has to resign for the sake of the Conservative Party.
Most insiders expect the fatal blow to Mr Duncan Smith's leadership to be delivered after the May council elections, although some think it could come sooner – if the question of who succeeds appears to have been settled.
One Shadow Cabinet member said: "It will need something to happen outside this place to start the process, but once it happens, it could be very quick."
Another insider forecast a repeat of the process that brought down Margaret Thatcher, when her ministers trooped in to see her one by one, and seven out of 11 told her that her time was up. The first to deliver this message was Kenneth Clarke, the then Education Secretary.
One Tory said: "About half the Shadow Cabinet are prepared to go to see Iain Duncan Smith after the debacle in May to say that if he is not prepared to stand down, they will.
"Forget about trying to win the next general election. We're into survival. We're not talking about the Conservative Party as a serious fighting force. We're talking about whether the Conservative Party as an institution survives to fight a general election."
Mr Duncan Smith presides over a party that has been divided for more than 10 years over its attitude to the EU and the European single currency, and recently has been rent by differences between right-wing social liberals, grouped around Michael Portillo, and traditional Tories who believe the party should stand for old-fashioned family values.
Within weeks, Mr Duncan Smith will face the latest manifestation of this tension when the Local Government Bill goes into its committee stage and he presents already rebellious Tory MPs with his alternative to Section 28, which bans local authorities from promoting homosexuality.
John Bercow, the Tory MP who resigned from the Tory front bench to vote in favour of gay adoption, was among eight pro-adoption rebels and 35 more who abstained despite Mr Duncan Smith's three-line whip on the issue last November. Mr Bercow has already signed a Labour amendment that would scrap Section 28.
It is unclear whether Mr Duncan Smith will allow a free vote on the issue. If he does not, he faces a "far bigger rebellion", one MP said.
The major obstacle facing Tories who want Mr Duncan Smith removed is their inability to agree on who should take his place. Mr Clarke, who is more popular in the country than any other Tory MP, could still be defeated in a leadership contest.
Join our new commenting forum
Join thought-provoking conversations, follow other Independent readers and see their replies