Lord Sempill, a naval commander at the Admiralty, was accused of passing sensitive information to the Japanese Embassy in the lead-up to the attack on Pearl Harbor.
The documents show that British security services suspected at least five British citizens in London of providing information to the Japanese. "What this shows for the first time is the existence of a highly organised Japanese spy operation in Britain," says Dr Richard Aldrich, a historian from Nottingham University.
At one point the Attorney-General secretly considered prosecuting Lord Sempill. However, when the Admiralty confronted Sempill and wanted him to resign, Churchill interceded and only required Sempill to be "moved".
"This is a classic case of Churchill protecting himself," says Dr Aldrich. "If Sempill had been revealed as a spy, it would have been politically calamitous for Churchill at a low point in the war."
Educated at Eton, Bill Forbes-Sempill was apprenticed to Rolls-Royce in 1910. He became a distinguished aviator, joining the Royal Flying Corps at the beginning of the First World War. He later transferred to the Royal Naval Air Service where he rose to the rank of Commander. He was awarded the Air Force Cross.
Although he retired from the services in 1919, his engineering knowledge led to a life-long involvement with aviation. His first contact with the Japanese came in 1921 when he headed a official British mission to organise the Imperial Japanese Naval Air Service.
During his visit Sempill became a confirmed Japanophile, striking up close and long-standing relationships with the Japanese military. The Japanese were very impressed and awarded him the 3rd Order of the Rising Sun; 2nd Order of the Sacred Treasure and Special Medal of the Imperial Aero Society of Japan in the inter-war years.
Commander Forbes-Sempill succeeded his father in 1934 and became the 19th Baron Sempill, inheriting Craigievar Castle, Aberdeenshire.
When the Second World War broke out, he rejoined the Royal Naval Air Service. He was assigned to the Admiralty and worked in the Department of Air Material. There he had access to sensitive information about the latest aircraft.
Suspicions over Sempill were aroused in June 1940 when MI5 intercepted messages from Mitsubishi to London and the Yamagata Naval Air Force headquarters in Japan. These referred to payments being made to Sempill. It said that in light of the use "made of Lord Sempill by our military and naval attaches in London," these payments should continue.
When Sempill was suspected "of disclosure of secret information about Fleet Air Arm aircraft," the matter was discreetly referred to the Attorney- General and Director of Public Prosecutions.
"The Attorney General advised against prosecution, but Sempill was strictly cautioned," said the file. Lord Sempill denied the allegations and said he had not received payments from an "improper quarters." He told the Admiralty Board that the money had stopped on the outbreak of war.
MI5 tapped Sempill's phones and found Sempill had kept up his contacts with the Japanese. A year later he again came to the attention of the security services and was suspected of passing information about the Battle of the Atlantic - the continuing efforts to get merchant convoys to and from the US.
At the time Britain was not at war with Japan, but it was considered only a matter of time before war was declared.
A note to Churchill says: "As long ago as August 1940 the Director of Naval Intelligence drew attention to the apparently undesirable contacts of Lord Sempill's."
There was no hard evidence of a leak, but "recently, the Director of Naval Intelligence, found that Sempill had been indiscreet in talking to his wife about his work..." A memo reports that Churchill's security adviser, Lord Swinton, had "official knowledge that Lord Sempill is at the moment in a serious financial situation".
On 5 September 1941, Sempill was brought in front of the Fifth Sea Lord and given "a private warning".
Some key details from the file are still retained. It is not clear from the files whether Sempill was a paid spy or just indiscreet to his Japanese friends.
On 9 October 1941, a signed note from Churchill says: "Clear him out while time remains." The Admiralty confronted Sempill and told him he could either resign or be fired. Sempill protested.
Churchill was unhappy at the action: "I had not contemplated Lord Sempill being required to resign his commission, but only to be employed elsewhere in the Admiralty."
A note in the file from Churchill's aide, Desmond Morton, dated 17 October 1941 says: "The First Sea Lord ... proposes to offer him a post in the North of Scotland. I have suggested to Lord Swinton that MI5 should be informed in due course so they may take any precautions necessary."
Dr Aldrich believes that Churchill feared the scandal would become public. "What the files shows is that Japanese intelligence were able to recruit sources at a high level."
The Public Record Office files also show that the security service was concerned over a number of other British citizens, including the former Military Attache to Tokyo, General Piggot, and his continuing contacts with the Japanese.
Lord Swinton's memo to Churchill said: "General Piggot is a bigoted pro- Jap, but said to be honest and loyal, as he is misguided."
They were also using Professor Gerothwohl, a shadowy figure who fed false information to the Japanese. We was described as "a highly intelligent ferret, working for and paid by other Embassies and Legations, as well as Japan".
Gerothwohl was believed to be a German Jew who had appeared in London in the 1930s and had been a foreign affairs adviser to Lloyd George.
"We have used Gerothwohl, feeding him with `dud' information which he believes genuine, and which the services wanted to plant on the Japanese," said the note.
Lord Sempill retired, but continued to serve on many public bodies. He was a Scottish Peer from 1935-63, and died in 1965.
The current Lord Sempill, the 21st Baron, is a grandson who last week was selected as a Conservative candidate for the Scottish Parliament. He said the family did not know about the Japanese allegations.