Steve Richards: The toxic air around Number 10

Even those who are loyal to Mr Brown are critical of the Downing Street operation

Gordon Brown should worry greatly about the hysteria surrounding the looming departure of the Transport Secretary, Ruth Kelly, rather than the resignation itself. After all, Transport Secretaries come and go more regularly than trains and buses. We are used to it. But as the news of Ms Kelly's departure seeped out in the dying, decadent hours of Labour's conference, all hell broke loose. The noisy din, fuelled by paranoid misunderstandings at the highest level, highlights vividly the breakdown in relations between Mr Brown and much of his cabinet.

Throughout the week in Manchester, conversations with cabinet ministers were dominated by a single theme. Forget about the economy or Afghanistan. Wary ministers were preoccupied by what they regard as a sometimes malevolent and dysfunctional Downing Street operation. Some were bruised by what they are convinced were briefings against them. One had just heard rumours of a briefing that he was to be sacked or demoted in a reshuffle and despaired over such inept, destructive activity. Others fumed on behalf of those who had been the victim of such onslaughts.

They cannot understand it. Such briefings do not help Mr Brown, as the overall impact is to convey a sense of a cabinet at war. More widely there is frustration at the crankiness of the Downing Street machine, slow to respond to the news agenda, the Conservatives and the rest.

It was into this mood of ministerial fearfulness that the sudden announcement of Ms Kelly's departure dropped like a bomb in the middle of the night.

Cabinet allies of the minister, and for a time Ms Kelly herself, suspected another Downing Street plot. In the early hours of Wednesday morning and for much of the following day there were seething questions from ministers based on this assumption. They could only hint at their anger in public.

Here was Hazel Blears: "These stories come from the shadows. You probably know more about it than any of us in the cabinet," she told a BBC reporter with a smile that these days hides a hint of menace. And here is what they really felt. Conor Ryan worked in Downing Street under Tony Blair and now writes a blog: "The people around Brown simply don't understand the extent to which they are damaging him and precipitating further revolts."

Ryan assumes, like many other angry ministers, that the Kelly resignation was a deliberate leak. In this case they are wrong. Both Ms Kelly and the Brown entourage got the shock of their lives when the story broke on Newsnight in the way it did. The news emerged as a result of a cock-up and not a co-ordinated operation.

Newsnight hinted at the origins on Wednesday night when it reported that they were more a reflection of the dysfunctional nature of Downing Street than a brutally devious plan.

It comes to something when some ministers assumed that Brown and his allies must have sat down and had a conversation along these lines: "We are enjoying the best media coverage for a year because of the speech... let's knock it off the headlines by briefing about a cabinet resignation." Evidently some ministers have such a low opinion of the operation that many of them leapt instinctively to such a misplaced assumption. It should be stressed there is nothing new in tensions within Downing Street and between prime ministerial courtiers and other ministers.

To take one example of many in recent decades, memoirs from the Harold Wilson era report of a plan to kill off Marcia Williams, Wilson's long-serving political friend.

As well as contemplating murder, Wilson's allies were also capable of briefing against ministerial colleagues. Tony Benn read that he was being moved from the department of industry in 1975 long before it happened.

Critical ministers also do not appreciate what it is like in the cauldron of Downing Street. Sometimes a Prime Minister's Press Secretary has to act fast for purely defensive reasons. In the case of Ms Kelly, given the speculation that there might be mass cabinet resignations after the Labour conference, I can understand why Mr Brown's spin doctor frantically briefed at three in the morning that she was off for family reasons. No one knows what it is like dealing with the chaotic rhythms of political news until they arrive in Number Ten.

Nonetheless there is a difference of degree in the case of the current tensions compared with the past. Things have got out of hand.

Even those who are loyal to Mr Brown are critical of the Downing Street operation and wonder whether they will awake to find they have been bashed about in some wild counter-productive newspaper briefing. They want Mr Brown to sort out how decisions are made and how he and those acting on his behalf deal with them.

This makes the looming reshuffle more important than usual. Most cabinet changes make no difference to a government's standing or performance. This one could do so. Mr Brown needs a big mediating figure to work with him and the rest of his cabinet, a figure whom ministers trust and whom he can work with too. Jack Straw has been mentioned for such a role, as has Alan Johnson. Although there are some other delicate decisions Mr Brown must take in relation to the reshuffle (ones he has not yet made), the most important will be a readiness to show that he is willing to operate differently with them.

My guess is that, unless they are sweeping, the changes will not make much difference. The tensions within government are as much a symptom of decline as a cause. Mr Brown can be thoughtless in his dealings with ministers and has presided over an amateurish media operation. Yet ministers will put up with a lot when they are 20 points ahead in the polls. During such heady times they also tend to stay in their jobs rather than choose to spend more time with their families.

With financial markets collapsing and some internal ideological tensions resurfacing there are other more fundamental issues for Mr Brown and his party to address. Nonetheless, if he does not try to sort out Number 10, it will be the immediate cause of his undoing.