Where constitutional reform has failed, perhaps the rats and the asbestos will succeed. The prospect of a five-year shutdown of the Palace of Westminster could be just what is needed to transform the way the public looks at our politicians, and how politicians regard themselves.

One cannot blame architecture for everything, but buildings do affect behaviour. The House of Commons and House of Lords manage the twin feats of being pompous and inefficient. Most of their members jealously guard their privileges and status, while resisting all but the smallest changes. Were Health and Safety to force them out of the building that towers over SW1, perhaps they might alight at the real world.

The problem is not an excess of material comforts. There are many more glitzy places in London in which to work. Before the construction of Portcullis House, the modern structure connected by an escalator to the main palace, many MPs had pokey offices. Some still do. Portcullis is not particularly opulent. With its no-frills coffee shop as its main meeting point, it could almost be seen as normal.

The same cannot be said for the chambers and their immediate environs. The restaurants are a study in English public school naff – roasts and crumble, servile waiters and a stifling atmosphere of gossip and self-congratulation. The bars are similar, with taxpayer-subsidised alcohol to encourage lewdness and brawling (Eric Joyce was only the latest in a long line). The Pugin finery reinforces the members in their false sense of importance.

To cap it all are the officials and security. When I first started working as a lobby journalist in the mid-1990s, the only "acceptable" place to fraternise with MPs was the Members' Lobby. This is a curious anteroom in front of the Commons chamber. Doors open means you can't walk in, but doors closed means you can (work that one out for yourself); you can sit on this bench, if invited by a member, but you can't on that one. You can report what's said, but not where or by whom. And so the rules went on, dished out by usually officious men (usually men) wearing ridiculous chains around their neck.

While little of the flummery has been jettisoned (the Speaker, John Bercow, has dispensed with some of the court jester garb of his office), one aspect of the old days has been sadly lost. Even after the bombing of Airey Neave, security was relatively light. MPs, and others, could walk through the main gates with a wave to the policeman on duty. 9/11 changed that. Now it is a fortress, with gun-toting anti-terrorist squads patrolling all entrances and vantage points, and ever-intrusive officials to ensure that badges are shown at every turn. This is inevitable, perhaps, and nobody's fault. But the consequence is a further sense of "them" and "us".

Some MPs revel in the rarity of their habitat. Many find it oppressive, and even embarrassing. Some succumb to an SW1 variant of Stockholm Syndrome. Even the most hidebound have offices in their constituencies where they meet ordinary people with ordinary concerns. And yet, as soon as they breathe the stale air of the palace, they are invited to adopt a different persona.

Every summer, as the MPs and peers depart, the workmen move in. Tens of millions of pounds are spent on renovations; by the time everyone returns in the autumn, you cannot see the difference. If the reports can be believed, the wiring, plumbing and asbestos may be so bad that nothing short of a five-year removal will do. It has been suggested that they could be relocated 100 metres up the road, opposite Westminster Abbey, in the Queen Elizabeth II Conference Centre, an unprepossessing 1970s' block where companies present their annual results and think-tanks hold the odd presentation. The Chilcot Inquiry into Iraq took up residence there, and the Leveson Inquiry held some early seminars.

But why not go for something more radical? Why not leave London? In the late 1990s, some wiseacre suggested that parliament move to Milton Keynes. That is not as frivolous as it sounds. Or Manchester, or Birmingham? I'm sure a functional building could be found, with good broadband, security, enough office space and a conference centre for debating. Then what? Given Westminster and Whitehall's propensity for profligacy, the refurbishment is almost certain to come in late and over budget. After, say, 10 years, perhaps the next generation of MPs and members of the upper house might realise that, in different surroundings, they were better able to scrutinise legislation and hold ministers to account.

It is impossible to quantify the relationship between the built environment and performance. But each time I go to a more contemporary parliamentary or government building, I have a greater sense of normality. The creators of the Scottish Parliament dispensed with as much of Westminster's legacy as they could. But Holyrood should not have cost as much as it did. Perhaps a more appropriate example is the Reichstag in Germany. With its history under Nazism, then decades of neglect under communism, the building has, in its new incarnation, achieved the rare feat of blending the august with the accessible. The Norman Foster-designed cupola allows the public to look in from above – a gesture that seeks to enshrine democratic accountability with environmental sensitivity and an awareness of history.

In these straitened times, the least ambitious option is the most likely. The authorities will recoil from the idea of moving out, and will spend each summer in an ever more desperate attempt to patch and mend. Even if a brief relocation is required, they will be reluctant to change the building's ethos. Well into the middle of the 21st century, previously sensible men and women will square up to each other in the bearpit of the chamber, calling each other names by means of strange euphemisms, while lacking the infrastructure or support staff to keep a proper check on government power. But at least they'll enjoy the cheap booze on offer, while they tuck into their school lunches.

John Kampfner is the author of 'Freedom For Sale' and 'Blair's Wars'

Twitter: @johnkampfner

john@jkampfner.net