Field Marshall Lord Bramall protested in the House of Lords that "special advisers carry more weight than the Prime Minister", who had given assurances that the uncertainty facing the armed forces was over.
There is growing opposition among Tory MPs to the pounds 1.6bn sale of the houses, which is seen as a privatisation too far. Mr Hart is believed to have been highly influential in persuading Mr Portillo to put the married quarters up for sale.
Senior Tory MPs are concerned about the plan to sell the whole of the estate, involving 55,000 homes, to a single bidder, reported to be a Japanese bank. A total of19 bids were made for the houses which have a rental income of pounds 107 million a year, and 2,700 are vacant.
The Defence Secretary's special adviser has been a controversial figure at the Ministry of Defence, and some Tory MPs are determined to counter his influence. They were delighted recently when the Government rejected his advice to buy US fighter jets instead of up-dating British Tornados.
Mr Hart emerged as an adviser to Ian MacGregor, the chairman of British Coal, at the height of the 1984 miners' strike. Lord Walker, the former Energy Secretary, told Mr Hart he could not handle the miners' strike from at Claridges.
Lord Bramall's attack on the sale of the houses will be reinforced by Tory rebels tomorrow when James Arbuthnot, the minister for defence procurement, is questioned about the plan.
"We are going to give James a very rough time," said one Tory member of the Committee. "It's totally outrageous."
Mr Arbuthnot is expected to try to calm down the rebels by outlining a range of assurances, possibly including exchange criteria under which the private developer could offer houses in another area. The Tory MPs dismiss the plans as "cosmetic".
Lord Bramall, who also protested at the threat to the future of the Staff College at Camberley, complained about "the dilution of military advice in Whitehall." Ministers and their special advisers moved to other jobs, or out of office but Lord Bramall said armed forces officers felt "apprehensive."