Cameron flies home to Tory whispering campaign and threat of early election

David Cameron flies back to London today to answer the growing whispering campaign against his leadership of the Tory party.

Speaking from Kigali, the Conservative leader sent a defiant message to Tory critics, saying there would be no retreat into the "hinterland" and no "looking backwards" because of recent bad poll results. The Tory leader also intends to sweep aside criticism of his decision to go to Rwanda while his own constituency of Witney in Oxfordshire was under water.

Related articles

• Leading article: A flood of criticism

But he is facing an open challenge to the direction he is leading the Tories in over taxation, the environment and nuclear power. The Conservative leader is planning to be at Prime Minister's Questions at the Commons today to face Gordon Brown across the despatch box. But some of his biggest current critics are on his own backbenches.

There was more bad news for Mr Cameron today with a new poll, the third of its kind in recent days, showing Labour as having opened up a six point lead over the Conservatives. The ICM-Guardian poll had Labour on 38 points, the Tories on 32 and the Liberal Democrats on 20.

The Thatcherite 92 Group of Tory right-wing MPs agreed at a dinner on Monday night to directly challenge the Tory leadership over policy on nuclear power, aviation taxation, low taxation and the future grammar schools. The right-wingers intend to hijack policy reviews which are to begin delivering their findings during the summer. The process had been intended to provide ammunition against the Government, but now threatens to backfire on the Tory leadership.

Mr Cameron conceded at a meeting of the executive of the 1922 Committee last week that he will attend all future meetings of Tory backbench MPs. He also stated that he will attend special meetings of the 1922 Committee in the autumn to hear debates on policy issues. The 92 Group and the right-wing Cornerstone Group, which overlap and claim to represent 40 per cent of Tory MPs, intend to use the policy reviews to steer the Tories back on to traditional core issues of low taxation, support for nuclear power and tough border controls.

The whispering campaign grew after the polls showed a "Brown bounce" putting Labour in a commanding seven-point lead and after the by-elections in Ealing Southall and Sedgefield, Tony Blair's old constituency, where the Conservatives were beaten into third place by the Liberal Democrats.

It led to claims at the weekend that a handful of Tory MPs had written to Sir Michael Spicer, the chairman of the 1922 Committee, calling for a leadership election. But the threat of a leadership election was not being taken seriously by Tory MPs at Westminster. "We want to change the policy, not the leader," said one right-wing Tory. "We are pretty dismayed at what is going on, but we are not mad. Changing leader for a fourth time before an election would be suicide." There has also been criticism of Andy Coulson, the former News of the World editor, who was brought in as Mr Cameron's chief "spin doctor" just as he started to attract critical headlines.

Mr Cameron has had a summer even more dismal than the British weather. There were policy wobbles on a tax on holiday flights, the future of grammar schools and the abolition of free entry to museums and art galleries that led to angry protests and hasty retreats.

Since his arrival at Conservative Central Office, Mr Coulson has remained firmly in the background, but yesterday he authorised an aggressive counter-attack by party officials after Lord Kalms, the Eurosceptic former Tory donor, went on the BBC Today programme to openly challenge Mr Cameron's direction of the party. "There are a lot of gaps in policy... Europe, social cohesion, grammar schools. We seem to be chasing rather less substantive policies," Lord Kalms said.

One Tory briefer attacked Lord Kalms as being "like a cuckoo after the spring. He pops up every now and than to make his point and we know he will do it again. We are not exactly worried about it. He is almost a lone voice that should not be taken too seriously."

It was pointed out that Lord Kalms, the former boss of Dixons, had not donated to the party since 2004, a year before Mr Cameron was elected, and that he had never supported Mr Cameron for the leadership, instead backing his rival David Davis.

A senior party official also said reports of a leadership challenge were "hugely exaggerated". The official added: "They may be trying to create waves, but they are tiny ripples. David will come back from Rwanda absolutely robust and ready for an election if Brown calls one, and absolutely not planning to make any change in direction."

Labour are gearing up for an early election. A note was emailed to Labour MPs by Ed Balls, one of Mr Brown's closest cabinet allies, saying: "This has been the worst week yet for David Cameron." The problem for Mr Cameron is that many Tory MPs believe that, too.