When Nick Clegg responded to a pamphlet I presented at the Liberal Democrats' conference, (Imagining the New Britain), it brought back to me another era, another report, and another young and idealistic political leader. It was Tony Blair who, years ago, spoke at the launch of my book, True Colours, published by the Institute of Public Policy Research.

Ahead of its publication, I had travelled to several countries, commissioned surveys, and concluded we needed courageous and visionary leadership to make the UK more equal, at ease with diversity, Europe and immigration, and to find a common purpose.

Blair's words, as ever, were fulsome and fake. He didn't mean any of it, that smooth political transvestite, Mrs Thatcher in a suit. His Millennium Dome was an embodiment of misconceived, shallow ideas about national identity. Unloved, almost bulldozed and erased from history, the folly reinvented itself as an entertainment hub and is up and away.

By contrast, the government responsible for it is inept and dishonest, unable to lead a nation in crisis. Gordon Brown's frequent summonses to plucky Britishness echo in an empty space. Cameron, personally open, has astutely changed the image of his party to embrace some modernities. But on Europe his policies are darkly right-wing, and most old Tories grumpily go along with him only to get back in power.

We need national renewal and enlightened politics. The Liberal Democrats are principled on human and civil rights, justice, migration, taxation and unjust wars. If there is a hung parliament – I hope there is – they could have real influence. That and not any partisan reason persuaded me to introduce the paper at their conference.

In the European and local elections, vast numbers of voters rejected political parties that offered socially responsible democracy and chose xenophobes instead. Progressives have never had it so bad. In this age of uncertainty, they have a grave responsibility to pull Europe back from its penchant for fascism during hard times.

In Berlin last week, I spoke with an erudite peer who sees the perils of cumulative gloom overcoming our society as political and economic leadership loses its grip.

There are several parts to this tale of decline. First, Britain weakened its industrial base. Then came the near subsidence of the glass city, the financial sector, followed rapidly by the expenses row and collapse of faith in our precious democracy. I would add the loss of Empire and the catastrophic failure to start up a new one in Iraq. That fiasco has left the right feeling impotent and fuming, and the left loaded with guilt and shame.

Globalisation, despite the good it does, pushes people towards uncompromising identity politics, communal and national. Terrorism, tightening social and state control and ruthless market forces have generated shapeless fury. Up come the goosestep extremists and demagogues, racists and xenophobes, Stalinists and Islamicists, all offering remedies laced with arsenic. I never read the raving racists online but those who do tell me how revolting it is getting out there in the blogosphere. Ugly populism is fast food for the disillusioned.

For the centre-left leadership, the predicament is whether to pander to demagoguery or assuage genuine frustrations. Over the past three decades, governments have deepened income inequality and therefore resentment.

In high-migrant areas, extra funds are urgently needed to ease pressure on other inhabitants. Our least productive districts are those with static populations, but the most deprived should not pay for the indisputable economic benefits of immigration.

Unquestionably, multiculturalism – once an effective policy tool – is now failing to connect, to counter resurgent white racism or self-exclusion. When difference collides with equality, which one prevails? That central tension between key values has never been tackled by politicians who prefer laissez-faire and fudge, until a storm breaks, blowing away reason and judgement.

But though it is hard not to despair, I feel the sap of optimism rising too. Britain somehow finds its spirit in the most hopeless periods. Look what happens when citizens unite on an issue as they did over the 42-day detention proposal: if you unjustly incarcerate Muslims, you incarcerate us all, we said, a powerful message that broke the will of government.

Popular consent, I believe, can be galvanised for the greater good. A renewed national identity united old enemies in South Africa and Germany, and Anglo-Franco Canada promoted a one-nation consciousness which accommodates diversity. History mattered; so too national memories. The future mattered more, and these nations sought to drop the baggage that would hold them back.

Britons need to be drawn to such a collective enterprise. Who do we think we are? Who do we think we can be? What will bring our different tribes closer together? How can we become a more equal and less unhappy population? How can we deal with other countries as equals rather than sad and irrelevant 'imperialists' or unthinking US acolytes? These are the big questions encircling us. We need to find the right answers, and quickly too.