But critics of Group 4, which won the contract to escort prisoners in the East Midlands and Humberside, warned that security would be at risk as more areas were privatised.
Derek Lewis, the head of the prison service, has launched an investigation and said the escapes were a cause for serious concern. He is to meet Group 4 managers early next week.
'The prison service has been monitoring closely the operation of the contract with Group 4 for court escort services. Significant difficulties have been experienced in the first few days of the contract,' he said.
Group 4 said it was making modifications to its prison vans after identifying faults in the vehicles, which are divided into separate cells. The design of the vans had been specified by the Home Office.
'Despite our initial difficulties we remain convinced that the procedures and systems, agreed with and monitored by the Home Office, are fundamentally sound and that our staff are properly trained to carry out the task,' the company said.
Gary Salesman, 20, became the third prisoner to escape from Group 4's custody when he kicked his way to freedom on the way to Glen Parva prison, Leicestershire, following a court appearance in Derby on Thursday. Two other prisoners have been recaptured after escaping in Hull and Lincoln on Monday and Tuesday.
David Evans, general secretary of the Prison Officers' Association, said the job should be kept in the public sector where there was a culture of security. 'The Government has said it is going to privatise all the escort system. It will be on their heads if they do. Public safety is being put at risk.'
Group 4 denied its training was inferior and said the 400 staff involved in the prisoner escort service had been recruited from a variety of backgrounds including the prison service, the police and the armed forces.
The prison escort service was divided into 10 areas before privatisation. Group 4's contract, covering 21 prisons, 71 magistrates' courts, 10 Crown Courts and 57 police stations, involves about 100,000 prisoner movements a year. The company admitted some early difficulties but claimed it had made improvements.