Downing Street today endorsed Commons Speaker John Bercow's call for a £30,000-a-year contract to rent trees for an MPs' office block to be ended.
Mr Bercow said he was "horrified" to find out the cost of the 12 fig trees in Portcullis House greenery and accepted the taxpayer was being "fleeced".
Prime Minister David Cameron's official spokesman today said Downing Street was in agreement with the Speaker, and that the Houses of Parliament must make economies, just as other parts of the public sector are doing.
The deal has been in place for 12 years and covers care and maintenance of the trees, which shade dining areas in the building's glass-topped atrium, which stands beside the historic Palace of Westminster.
But Mr Bercow said it should be scrapped immediately, if contractually possible without adding more costs, but no later than September.
In an interview with The House magazine, he said: "I was horrified by it.
"Inevitably and understandably it will cause people out there to think these people are living in another universe.
"I think the contract should absolutely be revisited. If we are going to have trees, they absolutely shouldn't be trees that cause us to fleece the taxpayer in this way, and that must change at the earliest opportunity.
"If there is a contract and it's going to cost us more to get out of it immediately than not, then it may well have to wait.
"But should the present arrangement continue beyond September? Absolutely not."
Mr Cameron's spokesman said: "I think we find ourselves agreeing with Speaker Bercow.
"All parts of the public sector need to be looking at where they can find savings, and I don't see any reason why Parliament shouldn't be part of that."
Mr Bercow also conceded that there were some grounds to argue that the daily prayer session in the Commons was discriminatory.
But though he said he would not stand in the way of a review if sufficient MPs wanted one, he said he wanted to keep what was a "reasonable and generally popular" tradition.
Focus was put on the prayers - which MPs must attend to secure a seat for some high-profile parliamentary sessions - after a court ruled prayers could not be part of local council business.
The ruling does not apply to the Commons, which sets its own rules.
But Mr Bercow said: "If enough Members wanted to look at it, I'm certainly not going to object or try to impose my view.
"Theoretically there is a ground for criticism. But I think that there should be a degree of reasoned, balanced common sense."
The Speaker also called for annual party conferences to be cut to a long weekend so that the Commons could sit without a break from September.
"Most people have an annual holiday entitlement and unless they are taking annual holiday, they will ordinarily be at work in September.
"And I think a lot of our electorate think, given that the MPs finished in the latter part of July, why are they not back at their place of work undertaking their scrutiny, standing up for our interests, debating our concerns, in September? They should be. And frankly I agree with that."
A House of Commons spokeswoman said discussions would soon get under way on what Parliamentary authorities should do about the fig trees.
The current contract is up for renewal in September and the review will look at "possible savings", the spokeswoman said.
The Department of Facilities will review options in consultation with the Administration Committee, which represents the views of MPs to officials running Parliament.
Multiple options could be laid out by the Department of Facilities and recommendations may be made on the way forward.
The spokeswoman added: "The opportunity for change is in September but they will probably start the review before that."
The spokeswoman said the review would likely be broad but could not confirm what options will be available when the issue is reviewed.
Administration Committee chairman Sir Alan Haselhurst said the group of MPs would now look at the issue of the trees - and said it would be the first such discussion since he joined the committee in 2010.
He said: "The House of Commons Commission will be the ultimate decision maker. The Speaker is the chairman of the commission.
"If he's getting backing from No 10, it's easy to see which way things are going to go.
"This issue has burst upon us - the building has been there for 12 years. It's an iconic building.
"You have to have some kind of respect for the thinking behind (the design) of Portcullis House. I would need to know what the alternatives are."
Labour MP Thomas Docherty (Dunfermline and West Fife) sits on the 16-member, cross party committee of MPs. He said he had lodged Parliamentary Questions this week in a bid to establish facts about how much the trees are worth and when the contract was made.
He said: "I can't believe the trees are worth that much, frankly.
"My understanding is this was a contract that was let years ago. As is too often the case, things are done by officials without any recourse to getting MPs' approval.
"The reality is our constituents are taxpayers and they expect some accountability."
Documents published on the Parliament website suggest the trees are a design feature of the Portcullis House atrium, part of the system for maintaining the environment inside the glass-roofed space.
Fig trees were selected because the species is able to stay healthy in an indoor, temperature controlled environment.