On Friday, I was in Exeter, where I met a woman who had been helped by a charity tackling domestic violence. She asked me to take a message back to David Cameron. "We are not just numbers on a piece of paper," she said. "We are real people."

She was among a group of women from every part of Exeter society – working in areas from housing to Sure Start – expressing anguish about what they saw happening to what they had worked so hard to build. Such sentiments are being echoed in every corner of our country. Having raised some people's hopes that he was a genuine believer in social justice, the Prime Minister is creating despair.

Last week we saw his big idea, the Big Society, exposed. When I asked him about it in the House of Commons, Mr Cameron blustered on about how everyone should support "volunteering and philanthropic giving". Indeed we do. And Labour took important steps to support both the charitable sector and volunteering during our time in office. But no one can volunteer at a library or a Sure Start centre if it's being closed down. And nor can this Conservative-led government build a Big Society while simultaneously undermining its foundations with billions of pounds worth of cuts to the voluntary sector. Those are not merely numbers on a piece of paper; they affect real people.

But the reason why Mr Cameron's Big Society is in such trouble is not simply because the Government is making painful cuts. The way it is doing it – so far, so fast – speaks to its ideological heart. It really believes that a small state will produce a Big Society.

There was a time when all the talk around his Big Society was meant to show Mr Cameron as a different kind of Conservative. It was part of what they called the "decontamination strategy", and involved photo-opportunities where he hugged everything from hoodies to huskies.

And yet, the person who helped inspire his early promise of "social responsibility", Danny Kruger, now says that "the coalition has embraced a neoliberal interpretation, in which the Big Society simply means free men and women taking power into their own hands, where the only job of the state is to get out of the way ... to police the laissez-faire regime where what happens, happens".

And Mr Cameron's charity sector hero, Camila Batmanghelidjh, whose Kids Company was supposed to be model for the Big Society, "is saddened that the debate is presenting itself as a division between the voluntary forces doing something at community [level] and the state". "What really works," she says, "is where there is an infrastructure that supports both."

When the man who helped produce your big idea, and the woman who was the model for it, both say Mr Cameron is going in the wrong direction, it is time for him to think again. The Prime Minister needs to recognise, as he might put it, that there is such a thing as the state, and it can help to build society.

Mr Cameron should visit the local libraries in my constituency. These are not some monolithic institutions of crude Conservative caricature. They have classes for new mums and babies, after-school activities for young people, clubs for the elderly. They are owned by government but they nurture community. And now many of them are threatened with closure.

And then there are other institutions that are owned by the voluntary sector, or run by it, which also unleash that spirit. Again, like Citizen's Advice Bureaux, they are at risk.

Much of this springs from the decision to target councils with 28 per cent cuts. The Conservative-led government thought this was a soft target and now, absurdly, it claims these cuts can be made painlessly, or are even being exaggerated for "politically motivated" reasons.

No wonder the Conservative chairman of the Local Government Association says such arguments are "detached from reality".

What does all this substance and style remind us of? The early 1980s, and the Thatcher government which appeared intent on ripping apart our social and economic fabric. All she could say was: "There is no alternative."

Thirty years on, the Government could have chosen our moderate, sensible plan to halve the deficit over the course of this Parliament. Of course, there would still have been difficult cuts, but it is the scale of change that is the problem.

The Tories, then and now, treat objections with disdain. They say we either do not understand or lack their ideological purity. The closed libraries, the bankrupt charities, the lost youth services, the threatened parks and public spaces, are dismissed with a wave of the hand.

There is a better way. It starts from a belief that our economy, our communities and our civic society are made stronger not by small government, not by "big government" but by a government which values and acts in partnership with them. So we reject the view, that our country will be stronger simply by government getting out of the way.

I have been clear that we should recognise the shortcomings of the centralised state, and understand that government must be devolved and responsive. But if we care about vibrant communities, strong civic institutions, if we value the bonds that tie us each to one another through our clubs, societies and through our families and friendships, then the Government must act to support them where they need it.

I am determined that the failure of the Big Society concept should not lead to the kind of pessimism about our shared future that we saw in the 1980s. It is Labour's task to show Britain – even after all the damage inflicted by the Government – that a stronger society, a Good Society can still be created.

As for "red Toryism", "compassionate Conservatism", that funny tree logo, the huskies and "going green" – these were the cloaks they wore to confuse the voters. What we are witnessing now is the recontamination of the Tory party.

In the past week, voters have seen more clearly than ever this Conservative-led government in its true colours: a single-issue government making huge sacrificies of the things we value on the altar of deficit reduction. Recontaminated. In deep blue. A traditional Tory government.