So, David Cameron, why did you fail to seal the deal?

You started out against the most unpopular PM of recent times. The Lib Dems were running at less than 20 per cent in the opinion polls. The right-wing press was uniformly behind you and you had a double-digit lead for most of the four years you have led your party

Shortly before 5am on the morning after polling day, designer Anya Hindmarch and TV presenters Carol Vorderman and Kirstie Allsopp mixed with sharp-suited young men and fashionable twentysomething women in the River Room at the Conservative Party's Millbank headquarters.

A lady in a floor-length black fur coat and immaculate grey hair arrived with a giant Harrods bag. Only a statuesque young black woman in a bright orange body-con dress stood out from the sea of white faces.

The hundred-strong crowd of Tory party workers and supporters was boisterous and bleary-eyed. Goblets of red and white wine lined the bar, while a popcorn machine and candyfloss stand created a carnival atmosphere. Blue uplighters bathed the room in an otherworldly glow, while posters on the wall boasted "Bye bye Bureaucracy" and "Strong Families; Strong Society".

David Cameron, who had just arrived from his Witney constituency, delivered an upbeat message, assuring workers that if they achieve 300 or more seats they'd have the "moral authority" to kick Gordon Brown out of Downing Street.

But something was not quite right. When Mr Cameron arrived at the party he looked rather dejected. The eight vast TVs screening the BBC election coverage should, according to the script of the Cameron project, have brought news of the Conservatives' return to power after 13 years. Yet the images of Tory gains were interspersed with too many scenes of missed targets: there was no "Portillo moment" for Ed Balls, the Schools Secretary, who beat his Tory opponent by 1,101 votes.

Lucy, a young brunette in an eye-catching yellow and green frock, burst into tears when it was announced, around 5.15am, that Shaun Bailey had failed to win the seat of Hammersmith and Fulham. Sobbing, she said: "But he would have been so good for the party."

Mr Bailey, the 38-year-old black former youth worker, epitomised the new wave of candidates symbolising Mr Cameron's "changed" party and had been tipped as a star of the 2010 intake. He lost by 3,459 votes to Labour – not a narrow margin.

As the sun started to rise over the Thames and the crowd thinned out, waiters circulated with trays of mini croissants and pastries. Weary-looking guests drifted to tables lined with pots of strong coffee and tea, then wandered, blinking, out into the cool May morning. It was a sobering scene.

The Conservatives' final tally of seats – barring the delayed result in Thirsk and Malton later this month – is 306, 20 short of the winning post.

So why – with an unpopular Prime Minister at the head of a 13-year-old Labour government and a recession under his belt, a Liberal Democrat party that failed to improve on its 2005 performance, and the backing of the majority of newspapers, including The Sun – did Mr Cameron not seal the deal with the electorate?

The Tory operation during the four-week campaign was slick, but there were missed opportunities and strategic problems. The most obvious tactical error was Mr Cameron's agreement, struck weeks ago, to give Nick Clegg, the Liberal Democrat leader, an equal platform in the three televised leaders' debates. In the first, on 15 April, Mr Clegg's performance was revelatory, capturing the nation's "anti-politics" mood. It diluted Mr Cameron's ability to offer himself as the fresh-faced "change" candidate, because Mr Clegg was, to the audience watching at home, fresher.

Adrian Pepper, a former Tory government special adviser and now a polling and campaign strategist, said Conservative campaign chiefs had focused too much on differentiating Mr Cameron from Gordon Brown. They had been apparently taken by surprise by Mr Clegg in the first debate and were still wrong-footed in the second. Mr Pepper said: "It took until the third debate for David Cameron to expose Nick Clegg's amnesty on illegal immigrants and his policy on the euro, but by then the Conservative campaign had lost a lot of momentum."

Tim Montgomerie, the editor of the grass-roots website conservative, warned months ago that Mr Cameron should not have agreed to the debates because as the frontrunner he risked losing his lead.

Yet, in the end, the "Cleggmania" created by the debates was neutralised by polling day. The Lib Dems' poll ratings of early to mid 30s during the campaign never materialised: they achieved 23 per cent of the popular vote, the same as in 2005. This suggests something more fundamental was wrong with the Tory campaign which led to Mr Cameron's offer to the electorate falling short.

Mr Pepper questioned the strategy of putting so much emphasis on Mr Cameron's persona at the expense of other members of the Shadow Cabinet. "The party only managed to increase share of vote marginally above Michael Howard's in 2005, so serious questions need to be asked.

"It was a risk to run a campaign based around David alone. People were asking why William Hague was not more visible in the north and why Ken Clarke wasn't more prominent in the Midlands. You have to look behind the personal popularity numbers to identify which voters your politicians are popular with."

Tory strategists deserved credit for trying to fight a positive campaign with slogans such as "I've never voted Conservative, but ...". But not enough emphasis was placed on the negatives of the Labour government, Mr Pepper said. He added: "In any campaign you have to show that the other side doesn't deserve to be in office, alongside positive messages about why you need to be in office."

What of the Tory policies? Mr Cameron's flagship idea, the brainchild of his head of strategy Steve Hilton, is the "Big Society", but it left voters on the doorstep baffled. Critical insiders say it was launched too late – just a week before the election was called on 6 April – leaving not enough time to convince the public it was workable. Instead, it was reported as a "DIY government", with question marks raised over how hard-pressed people would fit in a bit of gardening or painting for their local council estate in between taking the kids to school and putting in a day's work.

The other flagship policy, the tax break for married couples, took a backseat position, despite it being so central to the Cameron message on stable family life being the answer to mending Britain's "broken society".

But the difficult economic climate meant any promise on tax could be picked apart by opponents. The party "won" the first week of the election campaign when dozens of business leaders came out to back Mr Cameron and George Osborne in opposing the government's 1 per cent rise in National Insurance contributions from next year – Labour's "jobs tax".

Yet this letter-writing could not sustain the whole campaign. There was also an obvious contradiction between this policy and the Conservatives' core message of austerity and fiscal restraint. The party was forced to tread carefully between promising immediate spending cuts to reduce the deficit and offering voters optimism and hope for the future – and so the message became confused.

Mr Pepper said the Tories should have been more honest in telling voters what cuts were coming. "Some people were scared to vote Conservative because we had not spelled out in enough detail what we were going to do. The idea of an austerity package was not a turn-off – it was the fact that people didn't know what it would mean for them. If I had to have an operation, and if the surgeon doesn't tell me what the operation is going to involve, I am going to be far more scared. But if the surgeon tells me what is involved I will most likely accept it."

Other areas were ignored for fear of repeating the catastrophic defeats of 2001 and 2005. Immigration, on the orders of senior Tories, was off the agenda, despite it being consistently the number two issue for voters after the economy. Only when it provided an opportunity – to attack Mr Clegg's migrant amnesty – was it thrust to the forefront of the Tory agenda.

There is also the balance of power between the central figures in the Tory operation. Mr Hilton and communications director, Andy Coulson, have clashed over how hard to push anti-Brown messages. There has been constant criticism from the right, particularly about the cliquey nature of Mr Cameron's team. James Forsyth of The Spectator magazine wrote last week: "The campaign has highlighted the insular nature of the Cameron operation. Talk to Tory big beasts and their most frequent complaint is that their telephone calls have not been returned. Alarmingly few people were kept in the loop during the campaign."

But it is perhaps too simplistic to go looking in the four-week campaign for reasons why Mr Cameron could not seal the deal. The Tory poll rating went beyond 40 per cent just once since the election was called on 6 April, when they reached 41 per cent, nine points clear of Labour and 23 points ahead of the Lib Dems. That poll, for YouGov/The Sun, was taken on 14 April, the day before the first televised leaders' debate.

Yet support in the polls was already dropping before last month. Mr Cameron's leadership, which began with so much hope in December 2005, needs to be looked at in its entirety to see why the project veered off course.

The Conservative modernisation strategy, it can be argued, was based not on having any radical game-changing policies but on combining changing the perception of the party – adopting "modern" candidates like Mr Bailey – with a modern economic model. Before the summer of 2008, there was a firm policy of not clashing with Labour on the economy but instead matching their spending plans.

All Mr Cameron needed to do was offer himself as a fresh start, centre-right, progressive alternative to a tired old Labour government and the election would fall into his lap, because it was the "turn" of the Conservatives. Until the economic crash of autumn 2008, this was a sensible strategy.

It is of some significance that the highest poll rating of Cameron's leadership was on 14 September 2008, when 52 per cent for Ipsos Mori gave the Tories a thumping 28-point lead over Labour. The next day, Lehman Brothers collapsed and the world's financial markets went into meltdown.

The economic crisis demanded a response that was distinct from the Government's, Tory strategists reasoned. The Conservative Party could no longer match Labour's spending plans, because the Government had got the country into the economic mess and ratcheted up a huge deficit. Instead, they went for austerity. Mr Osborne delivered the grim message to a Birmingham conference hall in October 2008: "The cupboard is bare."

Yet when this Tory austerity message was tested with voters, there was a steady decline in the polls. Mr Cameron and Mr Osborne then qualified their signals, suggesting the party was uncertain about its response.

The party was forced to sideline its modernisation agenda, as many of the policies were too costly. The right wing that Mr Cameron had been determined to avoid had to be squared with promises of cuts and low taxes.

Then a year ago yesterday, another truly game-changing crisis hit British politics: the expenses scandal. Mr Cameron's response to this was more sure-footed. Despite the individual embarrassments for Tory MPs over their moats and duck houses, the Tory leader made an early pitch as the clean-up candidate, ahead of his rivals. But it left the party suddenly fighting an anti-politics mood. It was difficult to offer change when the party was associated with some of the most memorable expenses claims.

In January this year, the Tories had a "soft launch" of their election campaign, confirming they would include a tax break for marriage, but not sketching out further details. A poster of an apparently airbrushed Mr Cameron, alongside the words "We can't go on like this; I'll cut the deficit, not the NHS" was widely criticised. Advertising experts said it was a mistake to place the words "cut" and "NHS" so closely on a Tory poster. It was also ridiculed in thousands of spoofs.

Mr Cameron then appeared to dilute his marriage tax plans in an interview with BBC's political editor, Nick Robinson. By the end of January, the strategy appeared more confused. During an interview at the World Economic Forum in Davos that month, Mr Cameron sounded a note of caution on Tory plans for savage cuts to public spending. This conflicted with a hardline message from Mr Osborne.

The electorate seemed to pick up on the uncertainty: since the new year, the Tories have been routinely scoring in the mid 30s in the polls, their lead over Labour reduced to below 10 points after hitting the mid 40s in 2009.

But something else happened in January: Britain emerged from recession, albeit only marginally. Yet it underscored the Labour argument that they had got it right on the economy, and that only Gordon Brown was to be trusted to steer Britain into recovery.

Conservative insiders concede other mishaps that did not help the Cameron project. Having relegated the need to modernise and detoxify the party, perhaps complacently taking it for granted that this had taken place, there were new signs of toxicity and that the party had not changed. There was the sleaze row following the revelation that Lord Ashcroft, Conservative deputy chairman and a key funder of the marginal seats the Tories were targeting, had been non-domiciled for UK tax purposes. There were also unfortunate comments by leading Conservatives on gay rights, including the shadow Home Secretary, Chris Grayling, who said people who ran B&Bs from their own homes had a right to turn away homosexual couples.

By contrast, Labour's election campaign was a disaster. There were behind-the-scenes clashes between Peter Mandelson, Ed Balls and Harriet Harman, who all wanted to impose their influence on the campaign. Yet still the Conservatives did not prevail.

The Prime Minister's personal campaign was defined by two contrasting moments: when he called Rochdale pensioner Gillian Duffy a "bigot" and when, last week, he gave a passionate speech on fairness and social justice to Citizens UK. The Tories' response to the Duffy affair was to rise above it. There have since been mutterings that they should have gone in for the kill. Mr Brown's rousing speech was overshadowed by mixed messages from cabinet ministers on tactical voting. Perhaps the closing stages of Labour's campaign did sway some votes.

The Lib Dem vote appeared to collapse in seats where the Conservatives were up against Labour – suggesting some tactical voting did take place. There was evidence of former Labour voters who had toyed with the Lib Dems switching back to their old party, galvanised by the looming threat of a Tory government. Ultimately, there were too many voters who felt unable to rush into Mr Cameron's arms: proof that the detoxification strategy was never completed.

On Thursday night, before Mr Cameron went for a pre-declaration drink in his local pub, he spent the evening chopping wood in his garden. Perhaps Tory spin doctors felt this made him sound like an ordinary family man. Yet it left the impression that he was taking out his frustrations on a chunk of tree. This weekend, as he tries to strike a coalition deal with the Lib Dems instead of putting together his first all-Tory cabinet, he is clearly a frustrated man.

Have you tried new the Independent Digital Edition apps?
ebooksAn evocation of the conflict through the eyes of those who lived through it
Angelina Jolie with her father Jon Voight
Bill Kerr has died aged 92
footballPremiership preview: All the talking points ahead of this weekend's matches
Arts and Entertainment
Warner Bros released a mock-up of what the new Central Perk will look like
tv'Friends' cafe will be complete with Gunther and orange couch
Keira Knightley poses topless for a special September The Photographer's issue of Interview Magazine, out now
The Ukip leader has consistently refused to be drawn on where he would mount an attempt to secure a parliamentary seat
voicesNigel Farage: Those who predicted we would lose momentum heading into the 2015 election are going to have to think again
Arts and Entertainment
Cara Delevingne made her acting debut in Anna Karenina in 2012
film Cara Delevingne 'in talks' to star in Zoolander sequel
Mario Balotelli pictured in his Liverpool shirt for the first time
Life and Style
Travel Shop
the manor
Up to 70% off luxury travel
on city breaks Find out more
Up to 70% off luxury travel
on chic beach resorts Find out more
sardina foodie
Up to 70% off luxury travel
on country retreats Find out more
Latest stories from i100
Have you tried new the Independent Digital Edition apps?
Independent Dating

By clicking 'Search' you
are agreeing to our
Terms of Use.

iJobs Job Widget
iJobs General

Java Developer - 1 year contract

£350 - £400 Per Day: Clearwater People Solutions Ltd: Our client based in Cent...

Junior Analyst - Graduate - 6 Month fixed term contract

£17000 - £20000 Per Annum Bonus, Life Insurance + Other Benefits: Clearwater P...

SAS Business Analyst - Credit Risk - Retail Banking

£450 - £500 per day: Orgtel: SAS Business Analyst, London, Banking, Credit Ris...

Project Manager - Pensions

£32000 - £38000 Per Annum Bonus, Life Insurance + Other Benefits: Clearwater P...

Day In a Page

Ukraine crisis: The phoney war is over as Russian troops and armour pour across the border

The phoney war is over

Russian troops and armour pour into Ukraine
Potatoes could be off the menu as crop pests threaten UK

Potatoes could be off the menu as crop pests threaten UK

The world’s entire food system is under attack - and Britain is most at risk, according to a new study
Gangnam smile: why the Chinese are flocking to South Korea to buy a new face

Gangnam smile: why the Chinese are flocking to South Korea to buy a new face

Seoul's plastic surgery industry is booming thanks to the popularity of the K-Pop look
From Mozart to Orson Welles: Creative geniuses who peaked too soon

Creative geniuses who peaked too soon

After the death of Sandy Wilson, 90, who wrote his only hit musical in his twenties, John Walsh wonders what it's like to peak too soon and go on to live a life more ordinary
Caught in the crossfire of a cyber Cold War

Caught in the crossfire of a cyber Cold War

Fears are mounting that Vladimir Putin has instructed hackers to target banks like JP Morgan
Salomé's feminine wiles have inspired writers, painters and musicians for 2,000 years

Salomé: A head for seduction

Salomé's feminine wiles have inspired writers, painters and musicians for 2,000 years. Now audiences can meet the Biblical femme fatale in two new stage and screen projects
From Bram Stoker to Stanley Kubrick, the British Library's latest exhibition celebrates all things Gothic

British Library celebrates all things Gothic

Forthcoming exhibition Terror and Wonder: The Gothic Imagination will be the UK's largest ever celebration of Gothic literature
The Hard Rock Café's owners are embroiled in a bitter legal dispute - but is the restaurant chain worth fighting for?

Is the Hard Rock Café worth fighting for?

The restaurant chain's owners are currently embroiled in a bitter legal dispute
Caribbean cuisine is becoming increasingly popular in the UK ... and there's more to it than jerk chicken at carnival

In search of Caribbean soul food

Caribbean cuisine is becoming increasingly popular in the UK ... and there's more to it than jerk chicken at carnival
11 best face powders

11 best face powders

Sweep away shiny skin with our pick of the best pressed and loose powder bases
England vs Norway: Roy Hodgson's hands tied by exploding top flight

Roy Hodgson's hands tied by exploding top flight

Lack of Englishmen at leading Premier League clubs leaves manager hamstrung
Angel Di Maria and Cristiano Ronaldo: A tale of two Manchester United No 7s

Di Maria and Ronaldo: A tale of two Manchester United No 7s

They both inherited the iconic shirt at Old Trafford, but the £59.7m new boy is joining a club in a very different state
Israel-Gaza conflict: No victory for Israel despite weeks of death and devastation

Robert Fisk: No victory for Israel despite weeks of devastation

Palestinians have won: they are still in Gaza, and Hamas is still there
Mary Beard writes character reference for Twitter troll who called her a 'slut'

Unlikely friends: Mary Beard and the troll who called her a ‘filthy old slut’

The Cambridge University classicist even wrote the student a character reference
America’s new apartheid: Prosperous white districts are choosing to break away from black cities and go it alone

America’s new apartheid

Prosperous white districts are choosing to break away from black cities and go it alone