Despite 300 new staff being taken on to ensure tourists got their passports before they went on holiday, the backlog has reached about 530,000, a Home Office spokeswoman admitted.
Unions said the average waiting time for passports had passed the seven week mark compared with a target time of two weeks because of problems with a new computer system at two of the Passport Agency's six offices.
The delays at Liverpool and Newport are exacerbated by new rules that mean children aged under 16 who are travelling abroad needed to have their own passports. Emergency measures have been taken including extending the life of thousands of expired passports to cope with the crisis.
Although the Home Office said that new staff recruited earlier in the month were having an impact, the backlog increased during June, according to Home Office figures.
A spokeswoman said: "We recognise that there still is the backlog.
"Improvements are very slow at the moment but what they are doing is to prioritise it by travel dates."
She said because of the public panic, families were applying for new passports earlier than expected, adding to the difficulties. Unions said privately that it would be September before the problems were overcome.
A spokesman for the Public and Commercial Services Union criticised the Passport Agency for failing to take on extra workers earlier.
He said: "A lot of the people in the offices have been working under constant pressure for three to four months and now are a bit punch drunk."
With the school holidays less than a month away, the agency was now expected to be hitting its busiest point.
Conservative Shadow Home Office Minister David Lidington said: "The Government must act without delay in order to ensure that this summer isn't ruined for the thousands of Britons expecting to jet off for a well-earned break in the sun."