This was not motivated by any animosity towards Mr Palmer, who had always seemed able and approachable when he was chief executive of Legal & General. It was purely because Mr Palmer was a career life insurance man, a representative of the very industry that has caused so many of the problems that the PIA is intended to prevent.
Our foreboding has turned out to be much better grounded than we could ever have foreseen. Unfair though it may be, Mr Palmer looks increasingly like a liability to the PIA - a body that is famously supposed to deliver a 'step change' in standards of financial regulation.
It did not help that Legal & General had a close relationship with Roger Levitt, the fraudster seen as the archetype of everything that is wrong with the life insurance industry. But Mr Palmer's problems really started in February when Legal & General was fined pounds 180,000 and pounds 220,000 costs for failing to keep proper control of its salesmen. The company's deficiencies dated back to Mr Palmer's period of control.
Mr Palmer quickly came under political fire. A 1990 memo, detailing further regulatory failings at Legal & General, became the subject of an early day motion in the House of Commons. Two days ago, Mr Palmer told the Independent that this motion prompted him to take the matter up with David Prosser, his successor at Legal & General.
Yet only eight days earlier, Mr Palmer had told the Commons' Treasury committee that he knew nothing about the memo and had not asked Legal & General about it. The very next day he admitted he was wrong and apologised to the committee for inadvertently misleading the MPs.
It stretches credibility almost to breaking point to understand how Mr Palmer could have forgotten his conversation with Mr Prosser, about such a high-profile memo, only a few weeks before. Mr Palmer says he failed to recognise the document under the pressure of facing MPs' questions. (It must be admitted that the elemental ferocity of Brian Sedgemore's questioning is extremely disconcerting.) The only alternative explanation for Mr Palmer's forgetfulness - that he deliberately misled MPs and then thought better of it - would make his position untenable.
The prospects for the PIA are very delicately poised. After months of factional in-fighting, it is still opposed by some of Britain's largest financial institutions, and by many MPs. Mr Palmer's departure at this stage would be a crippling blow to the PIA's already tarnished credibility.
Andrew Large, the SIB chairman, seems almost desperately keen to recognise the PIA and allow it to assume its powers. He, more than Mr Palmer, is responsible for the existing mess. Mr Palmer's departure would only highlight the SIB's role.
A single regulator for retail financial services is a good idea but, as we have regularly argued, the formation of the PIA has been marked by too many botched compromises between sectional interests. It is hard to see it surviving in its existing form. It needs both greater independence from the industry and better accountability.
Mr Palmer has said he will stand down if the PIA would be better off without him. The chaos that would ensue makes it unlikely that will happen in the short run. But it seems equally unlikely that he will be there this time next year.