Just a fraction of the people affected by the so-called "bedroom tax" have moved to a smaller home as a result of the policy, a survey of councils and housing associations has concluded.
The controversial scheme is designed to free up "under-occupied" social housing and cut hundreds of millions of pounds from the housing benefits bill.
The BBC research concluded that six per cent of tenants in England, Wales and Scotland had moved following the changes which came into force exactly a year ago.
Chris Bryant, the shadow work and pensions minister, said: "The Tories are so out of touch they just don't get it. This proves beyond any reasonable doubt that the bedroom tax was always designed as a tax on the poorest and most vulnerable."
Under the policy - described by the Coalition as the "spare room subsidy" - social tenants of working age lose 14 per cent of their housing benefit if they are deemed to have a surplus bedroom and 25 per cent if they have two or more spare bedrooms.
Critics claim the cuts are forcing disadvantaged tenants into rent arrears, putting them at risk of losing their homes, and reducing potential savings to the Treasury.
But Esther McVey, the Employment Minister, disputed the BBC's figures, saying around eight per cent of tenants had moved so far, and insisted the government was "on track" to reach its target of 25 to 30 per cent by 2017.
She also claimed there only an "indiscernible number" of tenants were now in arrears as a result of the policy.
Claudia Wood, chief executive of the think tank Demos, said: "The Government said the bedroom tax would encourage people to downsize and reduce housing benefit costs.
"In reality, a chronic shortage of smaller properties means many cannot move. Today's figures show over one in four households affected have fallen into arrears for the first time, and if the problems worsen then evictions come at a price. Evicting someone can cost over £5,000."
Willie Rennie, the Liberal Democrat leader in Scotland, called for the policy to be scrapped. He said he understood its principle, but did not think it was working as intended.