When he was Leader of the Opposition, David Cameron used to say that when he walked down a street with Boris Johnson, then a junior frontbencher, people would ask: "Who's that guy with Boris Johnson?"

Even though he is now Prime Minister, the public's reaction might not be very different today. The irrepressible Mayor of London is in his element during the Games, a perfect launch pad for what could be his next race – for the Conservative Party leadership.

Lots of hurdles are ahead, but it is clear that Team Boris is on manoeuvres. Mr Johnson has a small but growing band of Tory backbenchers preparing the ground for his leadership bid whenever Mr Cameron steps down, even though that could be years away.

They are eyeing up three constituencies, in the hope Mr Johnson might be parachuted into one of them shortly before the 2015 election: Mole Valley in Surrey; Reigate; and Croydon South. (The Tory MPs who hold these seats may have other ideas.) The Mayor has promised to serve a full second term at City Hall, which would run to 2016. In public, he has denied that he would cut and run for Parliament before then. But he has left himself an escape hatch, sometimes using the "no foreseeable circumstances" formula that Michael Heseltine deployed before challenging Margaret Thatcher for the Tory crown in 1990. "Circumstances can change," one Johnson supporter said. "He could be an MP and Mayor for a year. [Ken] Livingstone held both jobs for a while. Boris would say he is being London's voice in Parliament for 12 months."

Mr Johnson might just get away with it. He usually does. He has survived scrapes that would have killed the leadership ambitions of other politicians – not least the revelations about his colourful private life and being sacked from the Tory front bench for misleading the party leader Michael Howard about it.

This week he defied political gravity when he got stuck on a zip-wire in Victoria Park, in Tower Hamlets. It could have been a moment from which a politician can never recover. For Boris, it is already factored into the share price. As Mr Cameron said: "If any other politician anywhere in the world was stuck on a zip-wire, it would be a disaster. For Boris, it's a triumph."

But surely the Tories would not want Mr Cameron's successor to be another product of Eton, Oxford and the university's rabble-rousing Bullingdon Club? Team Boris has already prepared its lines: "Boris was a scholarship boy at Eton. His background is very different to Cameron's."

So would his leadership be? Since winning a second term in May, Mr Johnson has differentiated himself from Mr Cameron by calling for a referendum on Europe, opposing House of Lords reform and demanding more pro-growth measures, all of which have been well received by Tory MPs.

At the same time, he manages to win plaudits from some on the left, who welcome his support for immigration and gay marriage as befitting the leader of a liberal-inclined city. Friends say he is "the only Heineken Tory" who can appeal to the parts of the electorate no one else in the party can reach, recalling that Labour voters were wooed when he won his second mayoral term in May. Some sceptics say Boris would not play well "outside the M25", noting that the Tories need to do well in the North and Midlands to win in 2015.

Yet a YouGov survey this week suggested that a six-point Labour lead over a Cameron-led Conservative Party would be cut to just one point if Mr Johnson were in charge. Several of the 2010 intake of Tory MPs who a year ago dismissed the idea of him becoming party leader as "mad" admit they are now taking a second look. But he is, for now, a Marmite candidate in Toryland. Some MPs anoint him the next leader; others declare he is a clown who will never be viewed by voters as a potential Prime Minister. One senior Home Counties Conservative said: "Being London Mayor is one thing. The powers are limited and being a joker plays well. Being Prime Minister is another. People voted for Ken Livingstone to be Mayor but they wouldn't have put him into No 10. Much as voters might love Boris, I'm not sure they would want his finger on the nuclear trigger."

Although the Mayor is milking the Olympics for all they are worth, sceptics who question his judgement seized on his decision to invite Rupert Murdoch to the Games yesterday. The media magnate might repay the favour by backing a Boris leadership bid. The critics see the invitation as inappropriate while police inquiries into phone hacking continue. Defending his move, Mr Johnson insisted that Mr Murdoch is "not under any criminal investigation". He played down the flurry of speculation about his Tory leadership aspirations, predicting it would "come crashing down" after the Games.

However, supporters say the Olympics have put him on a pedestal as high as Mr Cameron's and hope he can stay there. They insist there is a serious man behind the buffoonery, who can master a brief quickly and faultlessly. Yet, despite his skill with words, critics point to surprisingly poor Commons performances as an MP, which could deter some Tories.

The biggest hurdle to him becoming leader, some Boris allies believe, would be securing a place on the shortlist of two, which under Tory rules is chosen by MPs before a final ballot of party members is held. There is little doubt Mr Johnson could win among the members – a poll for The Independent by the ConservativeHome website this week made him the front-runner. His challenge would be to convince enough Tory MPs to give him a place in the ballot. Boris watchers expect his energies to be quietly devoted to that over the next two years.

Inevitably, his prospects will depend on the circumstances. Brave talk of toppling Mr Cameron if he fails to win an overall majority in 2015, even if he clings to office with the help of the Liberal Democrats, might fizzle out. If Labour gains power, Mr Cameron would probably jump before he was pushed. The other scenario is that he retains power in 2015 and stands down before completing a full second term, which is why Mr Johnson is keen to be back in the Commons.

Even some admirers on the Tory benches admit to private doubts that the party's Clown Prince would be taken seriously enough by the voters for him to be elected PM. Despite that, it would be foolish to bet against him becoming Tory leader.