Some of his critics would say that Mr Patten has gone out of his way to provoke opposition, especially during the last few months.
"It's not an argument one can win", he objects. "If you do nothing, you're washing your hands of Hong Kong; if you try to do it yourself, you're grandstanding. I've spent half my time in Hong Kong with people accusing Britain, and I suppose me, of selling out Hong Kong to British commercial interests and the other half dealing with people who say we should have sold out Hong Kong to commercial interests. Neither of which is true".
The recent introduction of legislation on subversion, which has taken place right at the end of colonial rule, has excited Chinese fury and a promise by Peking that the new law will be repealed. So, is he merely grandstanding in order to make Britain look good?
Mr Patten insists that Britain tried to secure the Chinese government's agreement but could not, and therefore was compelled unilaterally to draft a law that it had pledged to bring onto the statute books. He says that even if the legislation does not survive "we've at least established a benchmark against which anything else can be judged".
Mr Patten was appointed in 1992 as the last Governor of Hong Kong, just after winning a General Election for the Conservative Party leader John Major but losing his own seat.
The appointment was seen as a rejection of the "old China hands" in the Foreign Office and Cabinet Office who sometimes appeared to be ready to mollify Peking at almost any cost.
Although he maintained the colony's "executive-led" government, he outraged both the Chinese government and the business and professional elite in Hong Kong by giving ordinary people a greater say in the running of the territory.
Mr Patten went as far as he could, within the bounds set by the 1984 Sino-British Joint Declaration on the future of Hong Kong, to make the 1995 Legislative Council election the freest in the colony's history.
However, that was too far for China, which has chosen its own provisional council to replace the legislature as soon as it gains control in July.
It has also ensured that Mr Patten's term as Governor has been punctuated by a constant stream of abuse from China. In fact, he relishes some of the more colourful epithets that have been thrown his way, mockingly referring to himself as "the tango dancer" or "the triple violator".
But China's hostility has also meant that obstacles have been put in the way of several crucial projects, such as the new airport at Chek Lap Kok.
He believes that most people "found here a refuge which was characterised by the decencies and freedoms of British civil society, and even though only in the last decade has Hong Kong started to see democratic development, it's been for many years one of the freest societies in Asia. Britain has provided the infrastructure within which Chinese entrepreneurial genius and hard work could flower and flourish."
So what will he be doing for the next six months? It has been suggested that as a lame-duck head of government, he might as well stay at home and put his feet up. Yet his diary is packed and he seems determined to oversee the completion of his social and educational programmes.
"I have been committed to the issue of the protection of civil liberties and democratic development but I've set out a whole range of other social and economic items on the agenda which I have tried to ensure are implemented. Sometimes, as a consequence, I've been rather curiously described as a socialist."
Never the less, he is realistic. "It would be ludicrous for me not to recognise that the last six months is the last six months. I mean, people aren't going to be looking to me for reassurance about the future as they are going to be looking to C H," he says, referring to his Peking-designated successor, C H Tung.
The Governor says he gets on well with Mr Tung personally. "We've worked together while disagreeing," he says, referring to the time that his successor spent as a member of his cabinet.
Aside from the personal relationship, "we're going to have to minimise rather than maximise the difficulties". Among the difficulties is China's establishment of a puppet legislature to rival the existing Legislative Council. "If the provisional legislature is given a great raft of things to do before 1 July 1997, it will be just aggravating a problem deliberately for political reasons," he asserts, perhaps suggesting that the difficulties will not be inconsiderable.
Mr Patten will not be offering advice to his successor unless asked. He is very well aware that any hint of endorsement from himself will be taken in Peking as a black mark against Mr Tung. His advice includes the suggestion that "the democrats need to be involved with governing and running Hong Kong". This sort of talk is heresy in Peking's ruling circles.
And what of his own future? "I find the suggestions, sometimes made, that you can drop in and out of Westminster politics as though the House of Commons was the RAC Club, both politically naive and extremely presumptuous. I don't know whether I want to set my hat at trying to resume a career in party politics and even if I do, I recognise all the problems." Meanwhile he will be retiring to France and writing a book about Asia, "which I've got to get out of my system".
He will be leaving the luxuries of a large staff at Government House to tend his garden, to learn how to use his computer and get that book written. It is hard not to believe that he is ticking off the days.
Leading article, page 17