It's impossible, of course, and nothing could be further from Jeffrey's mind. He's spending today writing a list of apologies to those he let down. And it's a very long list. But it is remarkable how so many people continue to have faith in him.
Jeffrey is now in a role that is completely new for him, that of the outcast - although all his life he has been an outsider, striving to be acceptable to the Establishment.
Superficially, he achieved it. But despite the support of two former prime ministers, Margaret Thatcher and John Major, there have always been sneers behind his back. The higher up the echelons of society you go, the higher the proportion of the sniggerers becomes.
The reality is that Jeffrey has always been most popular among ordinary people. Indeed, one of the things that surprised me as his campaign manager was that there was one group where he polled even higher than Ken Livingstone - young blacks. Maybe that's because for all his striving, Jeffrey never was part of the Establishment. He came from a poor background, had an absent father and spent much of his childhood in a bed-and-breakfast. He spent summers taking donkeys up and down the beach of Weston-super- Mare. Perhaps it was then that he started dreaming of greatness, but he knew he would have to have to reinvent himself first.
After all, in the Tory party of those days, to have any chance of progress you were expected to have attended a public school and Oxbridge - preferably followed by a stint as an officer in the Guards. Jeffrey's ability to create the future he wanted is extraordinary. At school, he was called Puny Archer. So he took a body-building course, trained and became an athlete, even running for his country.
It is this image of Jeffrey, the self-made man, that appeals to the aspiring classes. They don't sneer, but openly admire someone who can bounce back from poverty, insolvency and adversity with such energy and panache.
Week after week, I saw the same pattern emerge. At the fancy parties, people would be delighted to see him and then mutter under their breath after he'd passed. But on the streets, the warmth of feeling for him was genuine and spontaneous.
The first time I was with him, we stopped in Brixton. The door had barely opened before someone had thrust an autograph book into his hand. Then at the market, stall-holders queued up to tell him about their problems. In these situations, you see Jeffrey at his best - easy-going, open and determined to help.
But to the Conservative Party, he remained an outsider. They should remember, however, that it was outsiders - or at least those who came from untraditional backgrounds - who have been its greatest assets in recent years. Both John Major and Baroness Thatcher came from such backgrounds and had to conquer the snobbery and prejudice that dominated the Tory party of the time. Perhaps that is why they have instinctive sympathy for Jeffrey and were able to overlook his human failings. I am cheered that even as so many line up to take a kick at him, the people I most respect in the party are not among them.
As a mayoral candidate, Jeffrey was determined to find out about the problems of London at every level, to talk to everybody possible, whether it was the police commissioner or an old lady worried about graffiti. For two and half years he led the London mayoral debate by developing detailed practical policies to make the city a better place to live. But however many policy speeches he gave, there was hardly any interest from the media. All they wanted were the colourful stories.
Many of the themes that all the mayoral candidates now pursue were first championed by Jeffrey. For example, who knew that London was losing pounds 14bn a year to the rest of the UK until Jeffrey started trumpeting the fact?
Jeffrey has now been rejected by the Tory party. His greatest fear, however, is that he will also be rejected by the many charities to which he has devoted so much of his energy. So my most pleasant duty of the last few days was to tell him that many of his favourite charities have phoned to say they still want Jeffrey to work for them. One even called to ask Jeffrey and Mary to do a few TV appearances for them.
Meanwhile, the Tory party seeks to define itself as the party that kicked out Jeffrey. I don't blame the leadership, but it really hurts him - and it hurts me too when I see the look on his face. Michael Ancram, the party chairman, feels let down by Jeffrey because when he asked if there were any more skeletons, Jeffrey said "No". But see it from Jeffrey's point of view.
Thirteen years ago he asked a friend to cover for him about a dinner with a girlfriend. He didn't want Mary to know - the lie was to protect his marriage, not his career. Ted Francis yesterday confirmed that this was exactly how it was; there was no discussion of an impending trial. As it turned out, the lie was never used. So when questioned by Mr Ancram, I can't imagine Jeffrey even thinking of it as a skeleton.
Now, of course, any allegation made against Jeffrey will be believed. I have no idea what will yet emerge. But I know this: the people most hurt are his family. In the last few days, I have several times been advised to keep my head down. But I am proud to count myself a friend of Jeffrey's. I have had two and a half years of seeing him at close hand, not merely his famous energy and humour, but also his kindness and his generosity of spirit.
I am not blind to his weaknesses, but for me they simply underline his great strengths. Jeffrey is a sinner. I am a sinner. And now I have seen him in this state of deep remorse, and I know it will make him an even better man.
The author was manager of Lord Archer's mayoral campaignReuse content