Asylum seekers are being crammed into overcrowded and substandard accommodation by G4S and Serco, according to a watchdog.
The companies could have to pay the government between £3m and £4m in rebates for “poor performance”.
The Home Office outsourced accommodation to the two private companies among others in 2012 as part of spending cuts of £140million over seven years.
A National Audit Office report found that despite improvements by other contracted firms, Serco and G4S were failing to meet targets of property standards and waiting times.
The chief executive of the Refugee Council, Maurice Wren, said the COMPASS contracts are responsible for housing some of the “most vulnerable in society”.
He added: “Asylum seekers are often people who have fled horrifying experiences in their own countries and have lost everything.
“They do not get a choice where they live and are totally reliant on the contractors to ensure that their properties are safe for them and their families to live in.”
But some asylum seekers were also being granted housing that they were not entitled to because of high salaries, according to the findings.
Investigators found poor quality housing with some “major defects” because homes taken from previous suppliers were not properly checked.
Some of the properties were unsuitable and the companies “lacked understanding” of people from different faiths and cultures, the report said.
Basic furniture was not always provided, action on complaints was slow and some staff were letting themselves into asylum seekers homes unannounced.
In Wakefield, near Leeds, the local council found G4S cramming more than 200 asylum seekers into one facility in breach of planning laws.
Despite an investigation and complaints, the report said G4S is still housing more than the permitted number of people.
The National Audit Office said the transition to the new contract was late and many people asked to move received “mixed messages” and correspondence that they could not understand because it was not translated.
But G4S and Serco told the watchdog initial information supplied to them by the Home Office was inadequate in some areas and has caused difficulties.
The demand for asylum accommodation is much higher than predicted, they said, and the historical information initially given by the Home Office has caused supply problems.
The report found the government had failed to enforce its own targets and must work with contractors better to ensure standards are met.
G4S and Serco have been embroiled in several scandals over their handling of government contracts and were stripped of responsibilities for electronically tagging criminals last month.