At 55 he wants to do something else in business, having lived and breathed Hillsdown for so long. He says the decision to go next year was entirely his own but he is going at a time of serious concern over the company's performance.
Profits have stood still for three years and are forecast to fall for the next two. Worse than that, the shares, up 8p to 130p yesterday, are almost back to where they were when the company first joined the stock market in 1984, after successive issues.
From a deal-a-week, broadly based conglomerate - spanning food, housebuilding, furniture and even stationery - it has had to cut back to its roots in food, selling much of the rest.
Shareholders may not have prompted yesterday's announcement but they have intervened in the past. Not so long ago they were concerned that Sir Harry occupied a central position as stakeholder, chairman and undisputed boss. There were no non- executive directors until last year.
Sir John Nott, the former defence secretary, was one of the first to be appointed and is now becoming executive chairman on a salary of pounds 150,000. Potentially more valuable, Sir John will receive 600,000 options, almost as many as Sir Harry, according to the last report and accounts.
Whether Sir John, a former chairman and chief executive of Lazard Brothers, is a good choice for a food company remains to be seen. But there is clearly a danger, played down by Sir John yesterday, of treading on the toes of the chief executive, David Newton, whose credentials have yet to be established in the City.
Even more perplexing is the question of how Sir Harry is going to shrink into a non- executive director. The rest of the board are so used to looking to him for approval that it is fair to assume this habit will die hard. It would have been better if Sir Harry had left the company altogether.
Shareholders can, however, draw consolation from Sir Harry's promise to hold on to his shares and the fact that David Thompson, his co-founder, gave up his place in the boardroom two years after relinquishing his executive role. Perhaps Sir Harry could be persuaded to follow suit after a shorter interval.