Joan Smith: What about those without any children?

If there is one piece of New Labour rhetoric which comes close to making me howl with rage, it is the phrase "hard-working families". Who are these people that Gordon Brown loves so much, parading them in his speeches at every opportunity?

Each time I've heard him use the formula – in his pre-budget statement in 2001, for instance, or last autumn during his first address as Prime Minister at the Labour Party conference - I've wondered about all the individuals who might not possess the entry qualifications for this happy little nation of nuclear families.

Doesn't he realise that they're an endangered species? Two-fifths of the population of London is single, around the same proportion of unmarried people as in the North-west of England, while in Wales and Yorkshire it's just over a third. I have no doubt that many of these people are just as hard-working as married couples, and some of them may actually meet the Prime Minister's criteria by dint of the fact that they're cohabiting and have children. There are lots of grey areas, not least whether a gay, childless couple who have taken advantage of Tony Blair's civil partnership legislation qualifies as a "hard-working family".

In the last few days, Labour MPs have belatedly woken up to the existence of several groups who most certainly don't, and they're in a flap about it. This week, the abolition of the 10p lower rate of income tax which they voted for last year came into force, leaving 5.3 million people on low incomes worse off. The big losers are childless people under the age of 65 earning less than £18,500 a year, some of whom will be paying a couple of hundred pounds more in tax while higher earners benefit from the restructuring of the tax regime.

The Prime Minister had a deservedly rough time at a meeting of the Parliamentary Labour Party ten days ago, but backbenchers can't say they weren't warned. A former minister, Frank Field, flagged up the anomaly last summer but his amendment to provide a relief scheme for low earners was defeated by 269 votes to 67.

Despite trying to take advantage of the Government's current disarray, the Conservatives abstained on the amendment. David Cameron's sudden conversion to the cause of the low-paid looks opportunistic, but it can't be denied that it is Labour MPs – dazzled, I assume, by a decade of the then Chancellor's unceasing pro-family rhetoric – who are actively responsible for leaving millions of nurses, teaching assistants and bar staff in the lurch.

These low earners have been made "an unreasonable target", according to the Treasury Select Committee, and they're understandably furious about it. Several of my friends are affected, including a part-time teaching assistant who hasn't been able to find a full-time job since she was made redundant by the company which employed her for 20 years. Another is self-employed, in her mid 50s and doesn't have a pension, which is even further out of reach following Labour's tax changes.

As a backbench rebellion gathered force earlier this week (better late then never, I suppose), various government ministers were put up to defend abolition of the 10p tax rate, usually with the unintended result of adding fuel to the fire. The most disappointing performance came from the Cabinet Office minister, Ed Miliband, who expressed "regret" for the losers but stuck to the official line that "overall these changes make the tax system fairer". The younger Miliband brother is one of the few unmarried, childless members of Brown's Cabinet and has more reason than most to understand the situation – and social value – of the millions of people who fall into this category. Instead, and despite his past support for gay rights, he has reinforced the impression that the Government values families with children more highly than the rest of us.

This is the kind of social conservatism I associate with the Daily Mail, not a progressive centre-left political party; for right-wing commentators, the conventional family is a bulwark against social disintegration and they ignore all the evidence of child abuse and domestic violence which suggests a less rosy picture. Even so, much of the prejudice against people who don't choose to get married and have children has dissipated, driven by the realisation that not everyone wants or is able to live in the kind of family unit praised by the Prime Minister.

It may be that his rhetoric has been unconsciously influenced by his own aspiration to be part of an almost-parodic nuclear family, but he's in a minority; marriage has never been less popular, as the Government's own statistics revealed last month, and we are fast heading for a situation in which 45 per cent of marriages will end in divorce. Millions of people live in a network of relationships which embraces close friends, blood relatives, step-families and gay couples, suggesting that the populace is considerably more sophisticated in this respect than the political class which governs us. A fifth of women born in 1961 remain childless, many of them by choice, and there is widespread acknowledgement that the maternal instinct is not universal.

In my 20s, when I said I didn't want to have children, accusations of selfishness and worse were quick to fly; people really appeared to imagine that couples sat down and decided to have kids for the good of society, instead of the dozens of more or less conscious reasons which influenced their decisions. These days, the choice not to have children is as widely respected as the desperate desire to become mothers which inspires infertile women to opt for IVF treatment.

What the Government has now done, with its inept and unfair tax change, is risk igniting a conflict between couples and single people, between parents and those of us who don't have children. The simmering anger about the abolition of the 10p tax rate will be felt in next month's local elections when people who can't count themselves among the Prime Minister's "hard-working families" have an opportunity to punish the Government at the polls, but it is a product of a larger political failure.

Who is to say that a gay man who works in an Aids hospice is a less valuable member of society than a married man with two kids who sells gas-guzzling SUVs for a living? Why should a nurse who doesn't have children pay more to fund tax credits for couples who've decided to have four kids? When Brown took over from Blair, his supporters claimed he was a different and more principled type of politician, with an unswerving commitment to social justice. He has inherited a huge task from his predecessor, who left behind shameful levels of child poverty. But Labour MPs should be equally ashamed of a tax regime which appears to have reinstated the Victorian notion of the undeserving poor.