I could see some people, never having met him before, thinking what an old buffer, how can he have been one of our leading political figures for the last 30 years, holder of umpteen high offices? Then he stood up, without notes, and made a lucid, witty and, yes, altogether splendid speech.
You could say he should, shouldn't he. That's what politicians are for. What else are they good at? But most like to give the impression they are cleverer, deeper, wiser than they are. Willie Whitelaw has gone through life doing the opposite - disarming people by apparent simpleness.
People smile at the thought of him. And not just because they remember Margaret Thatcher's famous remark: 'Everyone needs a Willie.' On the other hand, he came out with one of the sharpest put-downs of our times when he accused Harold Wilson 'of going round the country, stirring up apathy'. Is he cleverer than he appears? Inside, is he a seething mass of hatred and contempt? Hard to tell. In his autobiography, he was kind to everyone. No one was rubbished.
He was sitting in his slacks and leisure shirt at his country home near Penrith, Cumbria, surrounded by his dogs and political memorabilia. Going to his lavatory is a walk down parliamentary memory lane. On the walls, he has framed Cabinet photos, all signed, going back for decades.
He and his wife, born Celia Sprot, have lived there for 30 years, from the time he was MP for Penrith and the Borders. Their four daughters have all married. Their fourth daughter has four daughters herself, which brings his score of grandchildren to 10 girls and two boys. 'My mother was always going on about why I hadn't got any sons. Because she minded so much, we made a point of not minding.'
Later that day he was going to London for Mrs Thatcher's entry into the Lords, twitting on about it. No, not overtly moaning, that's not his way, just sighing and huffing. He is 74, fit after his minor stroke, his weight down to 15 stone, his days of drinking spirits long behind him, but he wasn't looking forward to the journey, even though he could always sleep. He's good at sleeping. Usually he managed to avoid it during Cabinet meetings, if only just, but he has been known to fall asleep at his own dinner parties.
Don't go then, I said. Ah, but his absence might be noted. Some newspaper would take it as a slur, her old deputy not turning up. 'It's all right me saying I'm just an ordinary person these days, my presence does not matter, but people would just laugh at that.'
Is this going to go on for ever? Come on, Willie, you are now a retired politician. I know you are still doing countless good works, chairing charities, helping the Durham University Appeal, the Cumbria Tourist Board, but surely you can please yourself for a change, say what you think. Could you, for example, force yourself to think of your best and worst times in politics? Simple reactions please, not on the one hand, on the other hand, and don't worry that a name omitted might constitute the most awful rudeness.
Of your many jobs, which did you most enjoy?
'Perhaps Chief Whip when we were in Opposition in 1964-70. It was my first major job and I found I could handle people, get on with them, get them to work. I realised I was a people person. Mind you, in their different way, all my jobs.'
Thank you. Now which did you least enjoy?
'Chairman of the Party. We'd just lost an election when I took over in 1974, and we were about to lose another. It was an awful time. I can't say I enjoyed it. I didn't have a lot of pleasure as Leader of the House either, but that was because I was longing for a ministry of my own.'
Which was the hardest?
'The most challenging was Northern Ireland. I was suddenly thrust into it, my first time on the national stage, but I found it exciting and enjoyable. My wife and I both loved Ireland. It looked like revolution when we arrived, so the strains were enormous.'
Which do you think you did best?
'The most constructive and rewarding job was Home Secretary. It came after I'd done several jobs, one building up from another. It's a fascinating department, despite having too many things to do. You can go to bed in a clear blue sky and wake up with storm clouds round your neck. I had the 1980-81 riots to contend with, in Brixton, Toxteth and elsewhere. I felt what we did was right. The police had to be trained in riot control, but the most important thing is to keep calm. British people are profoundly sensible. The popular press might clamour to beat them down, knock them out, but if you take extreme actions you make the offenders more truculent, and the public turns against you. I've never liked extremes of action.'
What do think was your single most successful action?
'In Northern Ireland, when I managed to set up the Power Sharing executive in 1973. It lasted a short time, and was destroyed by the 1974 election, but it was a great achievement. I hope that it is about to happen again, however painfully. It is the only eventual outcome in Northern Ireland.'
What was your worst time?
'The most ghastly incident was Fagan, who got into the Queen's bedroom (in Buckingham Palace) in July, 1982. I was outside Stirling when I was told about it, on a Friday afternoon, on the way to Glenalmond, the Scottish public school, to present prizes. My father had been a pupil there, but I'd never visited it, so I was looking forward to it. My private secretary at the Home Office rang me on the car phone to say 'I've got a bad story to tell you, there's been an intruder at the Palace'.
'I carried on to the school. The Queen was safe, and there was nothing I could do, but naturally I worried about it all weekend. The thought of what might have happened to her was utterly horrifying. When I got back, we discussed how to handle the news. One or two thought it might never come out, but on the Monday it did. A leak, I suspect.
'Everything that could possibly have gone wrong with Palace security had gone wrong. A million to one chance that a certain window was left open, or that when the Queen did call for help, they didn't recognise her voice and thought it was her maid. Ghastly, ghastly. I was haunted by it. I felt shame and misery and contemplated resignation, but was told that was ridiculous.
'Then, a week later, came the case of Commander Trestrail, the Queen's police officer, associating with a male prostitute. This was not connected with the Fagan case, but it came out when some paper investigated Palace security.
'Then, worst of all, three days later, came the IRA bombs in Hyde Park and the Regent's Park bandstand, which resulted in so many deaths. This was again a security matter, and again I had to make a statement in the House. You could not dream up three more awful things happening in just 10 days if you'd been writing a novel. Ghastly, ghastly time.'
Which person have you most admired in politics?
'As a character, that is easy. Alec Douglas-Home. Politically, he was extremely good on Foreign Affairs. At home, he wasn't given a proper chance, as he was seen to be no good on economics. But as a character, as a person, he was easily the most admirable. I can't fault him.'
OK, let's have the others. I can see you're bursting.
'Macmillan turned the Conservative Party round after Suez and in two years we'd won the election, which was a fabulous performance, but in his second term he was not such a good PM. Wilson is now almost forgotten, but I don't think I've ever seen a person dominate his Party, and the House, the way he did between 1964-70. He was very formidable. I'm fond of Ted Heath, but he undid himself by calling the 1974 election, which was a mistake. He showed courage over Europe and he was good to work under, always supporting me in Northern Ireland, but his standing never recovered from the loss of the '74 election.
'Margaret will probably get the most space in history of all the PMs I've known. She was inspirational and she had two great achievements. First, the reform of the trade unions, in which she was helped by Scargill. If he had not been so extreme, she might not have managed it. Second, and I still don't know how she did this, under her Britain seemed to matter more in the world. As a person, I can't pretend she was always easy to work with. She could be very domineering. And she wasn't always right. But I admire what she did. I was very close to her for a long time, but we were never close friends. Let's say politically close friends.'
The best speaker you ever heard?
'Nye Bevan, if roused. Now and again he wouldn't try, but if some fool interrupted him, then he was off and would be brilliant. Iain Macleod was very good, and Wilson could be devasting - not as an orator, but in destroying his opponents.'
Which politician have you liked least?
'Enoch Powell. Never liked him, as a person or a politician. I disagreed violently with his speeches on race and on Ireland.'
Who have been the unluckiest politicians?
'John Biffen and Francis Pym. Great politicians, but their characters were not compatible with Margaret. I also feel sorry for Neil Kinnock. He did a great job in removing the power of the left wing and making Labour electable. It was a twist of fortune that he is not PM today. I thought he'd get it. If Margaret had been Tory leader, he would have got it, though of course Margaret will always think she would have won. I was nervous. I still felt Labour's slogan 'Time for a Change' would swing it. Yes, Kinnock has been extremely unlucky.'
Do you consider yourself unlucky not to have been PM?
'I think I would have been a fatal choice if I had won the leadership election in Opposition. I'm not a rabble-rouser or inspirational. I'm not good at always being against things, which you have to be in Opposition. And the right wing of our party would have made it hard for me, picking on my weak points, such as economics. In power, with a strong Chancellor I could trust, who knows, I could have done it. But I have no regrets.'
Which political journalists have you admired?
'I always liked Jimmy Margach of the Sunday Times, Harry Boyne of the Daily Telegraph, David Wood of the Times and Francis Boyd of the Guardian. Perhaps my all-time favourite is Ian Aitken of the Guardian. Although he is a political opponent, he is always scrupulous.'
Your least favourite?
'I have not a lot of time for the modern parliamentary sketch writers, even Matthew Parris in the Times, always determined to take the mickey. They don't have to stand up and make the speeches. My number one enemy, someone I always found odious, was Andrew Alexander in the Daily Mail. Nothing I could do was any use to him. If everything you do is written as a farce produced by a fool, then the public begins to think it must be true. When I left the Commons for the Lords, I said to him 'They've shot your fox'. I like to think I triumphed. He's now City correspondent.'
Your famous jibe at Wilson - did you say it?
'It's a strange thing. I did say those words, but the real meaning has been lost. For a start, I meant to say 'spreading apathy' not 'stirring it up'. It happened in 1970 and Wilson was so sure of victory that he was going round the country calming people down, telling everyone not to worry, leave it all to him. I was really attacking him for encouraging people not to want a change, not saying people were apathetic to him.'
When you got your hereditary title was it some sort of joke, because you have no sons?
'Not at all. It was because Margaret at the time was establishing the principle that hereditary titles were not dead. Perhaps she was thinking she might be a Countess one day. I don't know. Lord Tonypandy got one, even though he was a bachelor. I could have applied to the Queen for something called a Remainder, so that the title could go through the female line. That's what happened to Lord Mountbatten and some war heroes, but then I thought that would be equating me with the likes of Mountbatten, which I certainly am not.'
What about your image? How real is it?
'I have been accused of being cleverer than I pretend to be. All I can say is that it's a mistake to be too clever in politics. A little humility goes a long way, and an ability to laugh at oneself. I've often said 'Oh, I can't understand all this' when I understood it perfectly well. I've seen my job as getting the best out of people, that's why I've gone around saying 'Splendid, splendid'. My wife tells me I don't say it as much today, now that I don't have to go round a constituency.
'When I was Home Secretary, I visited a prison and was introduced to some prisoners serving life. After our chat, I said 'Splendid, splendid, carry on the good work . . .' You can laugh, but what else can one say to prisoners?'
This is the last of Hunter Davies's current series of interviews. He will be back in the autumn.