The Public Accounts Committee said hard action was needed to slash a growing backlog because officials did not know how many failed to leave the country - or where they were.
Calling for tough new targets to be set, its report said the "extremely serious" situation could take nearly two decades to sort out at present rates.
And until "significant inroads" are made into the backlog, the taxpayer would not get value for money for the £1.5 billion a year spent on the Immigration and Nationality Directorate.
The public spending watchdog accused the IND of undermining asylum policy by failing to keep track of the situation.
More than 400 released criminals were among those who had "disappeared", it said, with officials only able to put the total figure somewhere between 155,000 and 283,500.
Although around 1,350 failed applicants a month were being removed by September 2005, new applicants were failing at a faster rate, increasing the backlog, the MPs found.
As well as tougher measures such as detention and electronic tagging, the IND should also do more to encourage people to take advantage of far cheaper voluntary removal schemes, the MPs said.
They also recommended trials of a US-style sponsor system, where a member of the community takes responsibility for supervising an individual asylum seeker.
And they called for more arrests to be made at reporting centres rather than in the community and to redirect cash from areas such as human resources to pay for improved frontline enforcement work.
Committee chair Tory MP Edward Leigh said: "Failed asylum applicants are in increasing numbers staying in this country knowing that there is very little likelihood they will be apprehended and removed.
"The fact is that no one really knows how many of them remain in the UK or where most are living.
"The Government body which is supposed to know, the Home Office's Immigration and Nationality Directorate, has come up with an estimate of the size of the backlog of cases for removal - somewhere between 155,000 and 283,500 cases - but the vagueness of this fuels rather than allays our concern.
"What we can be confident about is that the Directorate is not removing failed asylum seekers anywhere near fast enough and the backlog of cases is growing.
"The situation is extremely serious and the Immigration and Nationality Directorate must take a hard look at its approach to removals.
"It must without delay establish a target for making substantial inroads on the backlog of older cases.
"And to meet that target, it must streamline its operations and deploy more staff on front-line work; vastly improve its information about the different categories of asylum seekers; and seriously examine a range of measures which might be deployed more, including detention, electronic tagging, the use of arrest at reporting centres rather than in the community and publicising voluntary removal schemes.
"Unless the Immigration and Nationality Directorate vigorously addresses itself to improving its poor performance, it will take many years to remove the backlog of failed asylum seekers.
"The integrity of the UK's asylum application process is at stake."
Conservative immigration spokesman Damian Green said: "This report is a shocking indictment of the state of the asylum system under Labour.
"It is disgraceful that the Government does not know where failed asylum seekers are or how many are leaving the country.
"This chronic lack of basic information is undermining any attempts to establish a civilised and credible asylum policy.
"It is all very well for the committee to call for tougher targets to be set, but the Government has form for setting tough asylum targets only to abandon them as soon as they become unachievable.
"Given that almost the same amount of people work in human resources and finance in the Immigration and Nationality Directorate as do on enforcement and removals, it is no wonder our asylum system is in such chaos."
Refugee Council chief executive Maeve Sherlock said improving the initial processing of claims was the key to cutting the backlog.
"One of the reasons we end up with the situation we are in is that decision-making on asylum claims at the beginning is really bad," she said on the BBC Radio 4 Today programme.
"About one in five claims that are turned down win on appeal.
"What that means is that people aren't getting the right decision at the beginning; they have to go to appeal, and that takes a long time and costs a lot of money.
"What I would like to see - and I'm pleased the Government is beginning to look at now - is a casework system where, right from the beginning, contact is managed with somebody, decisions are made quickly, fairly and well, and when you can then be in a situation where, rather than having someone who has been here years who finally gets turned down and then look to remove them, you've got someone under control."
Immigration Minister Tony McNulty said the Government was "either doing or seriously considering" many of the committee's recommendations.
The report was based on out-of-date figures, he said, insisting a target to remove more asylum seekers than were arriving was now being met.
He conceded that he had "no idea" what the exact number of those remaining in the country was, but said he was "pretty sure" it was at the lower end of the range.
"The figures in the report are based on 2003/2004 and the substance of the report is on one evidence session," he told the Today programme.
"Many of the things the report talks about, we are doing and have been doing for the last number of months.
"Over the last three of four weeks there have been more people being removed or leaving the country than coming as new asylum seekers.
"It hasn't happened as good as it should, it is getting better and better now and we are pushing now well above where it was a couple of years ago.
"A robust asylum system must be about removing people where that's possible to do so and we are doing that more and more in a better and better fashion."
An enhanced voluntary returns package was working well in increasing the numbers going home, he added.
Initial decision-making "needs improving", he accepted, but 90% of decisions were now made within two months under new rules.
"I am very confident going forward that new applicants are being dealt with in an entirely different way under our new asylum law and in a really effective way.
"But I do accept the committee's point that we do need to look at in detail the backlog and their point about segmenting that backlog so you can get to a stage where those from safer countries where they have no right of appeal in this country can be returned sooner and many of the other comments they make we are either doing or seriously considering."
If the backlog was a sign of incompetence, "it's a sign of an incompetence of British public policy over the last 20/30 years; it did not uniquely happen in 1997", he added.
Shadow home secretary David Davis said there had been "a story of serial failure and serial incompetence".
"How on earth do you deport people if you don't know where they are?" he asked.
The problem had "exploded" since Labour came to power and swept away Tory controls, he said.
Arresting failed asylum seekers on the spot was "very effective", he said, throwing doubt on the usefulness of tagging.
"If you are going to disappear into the community then cutting off a tag and running away is not so difficult."
Tony Blair said the Government was making progress on removing failed asylum seekers.
"The progress has been enormous," he said.
"The numbers of asylum seekers are down to a fraction of what it was a few years ago. We are now removing five times as many failed asylum seekers as when we came into office.
"And the measures that we have taken, for example issuing identity cards for new asylum seekers, tracking them carefully, fast tracking their applications, handling their applications in far greater time, has changed the situation dramatically over these past three or four years."
Speaking in Downing Street after talks with Swedish Prime Minister Goran Persson Mr Blair added: "However, we still have further to go and more to do. But the essence of it is to make sure that we handle the applications quickly, get the people who are failed out of the country."
Mr Blair said it was not just a matter for the Government, but it was also essential to get "the right court decisions" to remove failed asylum seekers.
He added: "In all the criticism don't forget the system is far better and more effective than it was 10 years ago and we are removing something like five times the numbers of people.
"As for the methods by which we hope to get more out of the country we have got an open mind on any of that. But the most important thing is when their application fails and it is handled quickly, get them out quickly. And that is why we have ramped up, huge amounts of efforts in this."
Mr Blair said the Government already offered failed asylum seekers free transport out of the country.Reuse content