Scotland Yard is facing its worst corruption crisis since the 1970s, when senior police officers were found to be controlling London's pornography industry. The investigation and subsequent purge left many detectives out of a job and in some case serving prison sentences. The gloom that surrounded the Yard in those days is similar to the atmosphere that pervades it today.

Each day reveals more details of misconduct by the press and the police. The investigation is going to be looking for heads to roll, and the higher the rank the better. This extraordinary state of affairs has its roots back in the Eighties, in the days when News International was dependent on the police to protect its new premises in Wapping.

Violent demonstrations occurred each night and the police were able to assist News International in getting its product out on to the streets. This was a complete turnaround for The Times newspaper, which only a decade earlier had launched the huge inquiry into police corruption that shook Scotland Yard to its foundations. News International was now best friends with Scotland Yard, and senior executives and top policemen wined and dined together on a regular basis.

Nobody could see the potential problems of a free lunch. This mutual admiration society worked very well for a time. Information passed freely both ways. The police benefited from undercover operations run by the newspapers, and in return the papers got their exclusive stories. This comfortable arrangement was cemented by regular briefings from Scotland Yard's press bureau to the national press directly and sometimes through the Crime Writers Association.

The culture of police officers mixing with journalists was encouraged, and little thought was given to the potential of misconduct. Crime writers were expected to know lots of police officers, and there was great competition to get the inside story. If only things could have stayed the same.

The News of the World began to pursue a strategy of aggressively targeting celebrities. The use of "the Fake Sheikh", Mazher Mahmood, was very effective, and produced some exclusive exposés on the greed and stupidity of people who should have known better. They were able to obtain confidential information on individuals including criminal records but they were in too much of a hurry to research public records.

Some private detective agencies realised that there was money to be earned from celebrity stories and confidential crime stories. Some of these detective agencies were run by former Metropolitan Police officers who maintained good contacts with serving officers. Some ex-police officers set themselves up as stringers, and provided a conduit for confidential information supplied by officers directly to the press. Once the Rubicon had been crossed, it was comparatively easy for police officers to contravene the Data Protection Act and supply information from the Police National Computer.

Short cuts adopted by the News of the World put them closer to the coalface. The strategy of using several intermediaries was abandoned and they employed private detectives such as Jonathan Rees of Southern Investigations and Glenn Mulcaire. This was clearly cheaper but the drawback was that if the private detectives came unstuck so did those who hired them.

The Department of Professional Standards at Scotland Yard has not been standing idly by. A number of undercover operations were mounted against ex- and serving police officers who were suspected of receiving corrupt payments. Nobody in authority was prepared to recognise the endemic nature of this corruption and each case was dealt with as a stand-alone incident. Much the same attitude was adopted when Glenn Mulcaire and Clive Goodman were convicted of hacking messages of members of the royal family.

At this stage, the number of police officers involved is unknown. News International's attempts to switch the focus of the inquiry onto the police by releasing details of payments to officers raised more questions than answers. The obvious questions are "What about payments to intermediaries?" and "What were the payments for?" Hospitality and gifts must also be probed.

The two organisations that are carrying out the investigations are... the Metropolitan Police and News International, both of whom are the subject of these allegations.

It is with breathtaking cheek that News International announced its own investigation. It is quite clear that getting to the truth is not a goal, its real objective is damage limitation and face-saving. It is quite clear that any number of junior staff will be sacrificed in order the save the skins of the real decision-makers. The News International investigation should be laughed out of court, not that it is ever likely to get there.

The new police investigation is even more curious. Everybody wants to know why the original hacking investigation was curtailed after the convictions of Mulcaire and Goodman. It seems unlikely that this decision was made solely by the police, but it is a possibility, and if so, why?

The suspicion must be that pressure was brought to bear by either News International, the Crown Prosecution Service or a very high-ranking police officer, or perhaps a combination of all three.

The new police investigation into hacking has been running since January 2011 and the police corruption enquiry has only just begun. It seems to me that this is a classic situation whereby an outside police force should be used, under the supervision of the Independent Police Complaints Commission. There is clear precedent for using an outside force, and if the public are to be convinced that this is a fair and unbiased investigation then that should clearly point to using an independent force outside of London.

This is no reflection on the skill, determination or ability of Deputy Assistant Commissioner Sue Akers, but the pressure which killed off the first enquiry might still exist.

The Metropolitan Police had one go at this and fell very short. At risk is the reputation and integrity of the service. It cannot afford to get it wrong again. The problem is that senior officers did not recognise the extent of the corruption and were probably unwilling to upset their new found pals in the media.

They must accept their responsibility for what has happened. It is astonishing that with so many resources being spent on anti-corruption, they could not see it when it was right under their noses.

John O'Connor is former commander of the Flying Squad at Scotland Yard