Over the years, many people have said David Cameron is a man who seems comfortable in his own skin.

The first time I met him it was something else that struck me: here was a man who, despite his privileged upbringing, was comfortable in his own country. This made a pleasant change from many Tory politicians at the time. He was far from the first member of his party to see how outdated the Tories had become. But when the penny dropped, the modernising agenda came naturally. He fused an old-fashioned patrician desire to do good with a social and economic liberalism, overlaid with a refreshing spirit of optimism that has helped him through tough times.

Yet even now, after seven years as Tory leader and 29 months in Downing Street, people seem confused by their Prime Minister. To the left, angered more by his background then his policies (many of which are an amplification of their own), Cameron is a loathsome right-wing extremist. To the right, he is a loathsome left-wing entryist who seized control of their party and did a duplicitous coalition deal.

Downing Street is a lonely place to live. One adviser told me he thinks the country almost ungovernable, such is the hysterical tone of debate. Regardless, this week the Prime Minister needs to demonstrate why he holds the keys to power. He has had two career-defining party conferences already: in 2005, when he captured the leadership, and in 2007, when he saw off Gordon Brown’s election attempt. He needs t complete the hat-trick this year.

Cameron stands at a crossroads after six months in which his government, his party and his reputation have taken a tumble. The worst wounds were self-inflicted – especially the daft decision to cut income tax for the ultra-rich. It is hard to think of a better device to have destroyed any sense of unity during the downturn, inflamed the idea the Tories care only for the rich and given Labour such a sweet open goal.

Ed Miliband took full advantage last week - although unlike last year, his speech will seem less strong as time passes. It is simple to steal your enemies’ vocabulary. I suggested to Mr Cameron he purloin the word ‘progressive’; one speech and one article later, the word was rendered meaningless. It is also simple right now to attack the coalition’s competence. It is far harder to atone for past sins while shaping an alternative, especially for a man who – like Mitt Romney – has an innate disdain for many voters, although in Mr Miliband’s case it is for wealth-creators.

It would be easy for Cameron to turn right. To toot the dog whistle by talking tough on crime, on Europe, on immigration. He would win loud applause, appeasing activists, media critics and the more unruly MPs – some of whom resemble those Bennites who nearly destroyed the Labour Party over their refusal to compromise with the electorate. As one Labour blogger put it last week, are there really Tories out there who think their party is ten points behind because it is too left-wing? Tragically, the answer is yes.

These are the people that pressed the case for the top-rate tax cut so passionately. Now they hail Boris Johnson for the way he stands up for bankers, yet ignore his advocacy of an amnesty for illegal immigrants and support for a living wage. These fringe figures, stuck in the past and frothing with fury, oppose reforms that might boost growth such as building on green belt or loosening visa controls. And scream for bigger spending cuts while fighting attempts to trim budgets in criminal justice and defence.

Cameron’s response to his loss of political capital has been to mollify the right, as with the recent Cabinet reshuffle. But they will never like him. All this has done is unnerve his core supporters, who are becoming increasingly agitated as they stay silently loyal. More importantly, it confused voters, sowing doubts over credibility and resulting in a fall in his personal ratings.

Cameron remains the most prime ministerial figure in British politics and retains his personal popularity. But the chances of an outright Tory victory in 2015 are tiny; it is not enough to rely on the weirdness of Ed Miliband or the revival of the economy. The Prime Minister needs to remind people who he is – a modern and compassionate conservative – and offer a sharp narrative beyond deficit reduction.

The building blocks are there in the moulding of a more affordable and effective welfare state, one that works for the most vulnerable rather than the middle-classes. This can be seen in education, in justice, in planning, in police reform – even to some extent in health. As with New Labour, modernisers have become the real radicals in government, their desire to reshape their parties matched by a determination to renew their nation.

The Tories must face the future, not look to the past.

The Tories must face the future, not look to the past. This means focusing efforts on the young –bereft of jobs and housing – rather than protecting higher-income pensioners. Yes, they vote in droves, but they also have children and grandchildren. This means supporting the squeezed middle and strivers, showing empathy over rising living costs by attacking the cartels and corporate behemoths who distort the free market.

This means ignoring the shrill voices condemning gay marriage and green issues as “metropolitan” concerns; are they not aware nine out of ten Britons live in urban areas? No wonder the Conservatives have failed to convince young and metropolitan voters outside the south-east they are on their side. Besides, these are issues Cameron has stood for since he vowed to “inspire a new generation” seven years ago.

Finally, it means drastic action to woo ethnic minorities. As Lord Ashcroft has shown, not being white is the biggest reason for not voting Tory. Repudiation of Norman Tebbit’s cricket test would be a start, but how about apologising for the party’s support of South Africa apartheid, examining workplace quotas and cracking down on racist police procedures such as stop and search?

Cameron has proved he is not afraid to take on his party’s reactionaries. Now comes a week that will play an important role in shaping both the next election and his own legacy. A natural pragmatist, will he still try to appease the right and end up pleasing no one? Or will he fire up his modernisation project and rediscover his mission – and in so doing salvage his leadership, shore up his Coalition and safeguard his party?