Last weekend he was inducted into boxing's Hall of Fame in New York, the only living Englishman to have received the honour. The limelight beckoned briefly again before he flew to Florida for a holiday with his son and grandchildren, who live there.
Soon it will be back to his elegant, if lonely, apartment overlooking Marble Arch, in central London - he is separated from, but still on amicable terms with, his wife of 50 years, Marie - to put the finishing touches to his biography. He promises it will be a belter. "I'm pulling no punches," he says. "Only the truth, mind." Duff promises a "massive story" about Frank Bruno, with whom he fell out when the ex-world heavyweight champion, whom he helped to build up, defected to rival Frank Warren, courtesy of Sky's coffers.
Both Warren and Sky will also figure as betes noires, no doubt, in Duff's racy memoirs. "Frank may not like it but he can stand on his head for all I care. I say some good things about him too, but the bad things are, well, pretty bad. Do you know he crossed himself 13 times on the way to the ring in the second fight with Tyson? Only he and his laundry will ever know how he felt. Heh, heh."
No doubt Bruno's lawyers await publication with felt tips poised but Duff is cute enough to be aware of the pitfalls in a business where more writs fly around than right-handers. He's sued five times for libel himself and won them all. There is also an outstanding case, against the Mirror, who quoted Warren as allegedly saying that he "didn't want to finish up like Mickey Duff, having to go to the fights to look for a friendly face".
Duff reckons that was malicious. He has friends in abundance, he says. Indeed 40 of them turned up at his birthday bash in a posh West End hotel, and were entertained by his long-time associate Jarvis Astaire's rendition of Sinatra favourites. Anyone who can get Astaire to sing for his supper must have something.
Duff may have friends, but what he doesn't have any more are fighters. For years he was the most productive figure in the game, as a matchmaker, promoter and manager. He has been involved with 49 world champions ("and I don't mean just having lunch with them or getting their autographs") now he's down to just one boxer, the lively lightweight Billy Schwer, whom he is steering towards a world title shot. "That would make it a nice round 50," says Duff. "That's what I'd like to call my book - 50 and out." His 14-year-old granddaughter reckons it should be titled The Ringmaster but Duff says it makes it all sound like a circus. And for him boxing has always been a very serious business, since the day he first stepped into the ring as a teenage welterweight. He had 69 fights and got out before he got hurt.
The old mashed potato nose wrinkles as he recalls the many great days, the fighters, the feuds and the fame. "It's not the same any more," he sighs. "The game has changed but I'll never knock boxing. Yes, it is a dangerous sport, but so are many others. I have always said that if my son had wanted to box I would have waited until he was asleep then broken his hands. The price of failure is probably too high but there are worse ways to make a living. You must remember this is not a sport of morals, where you take your turn to bat. You have to grab the opportunities."
Duff certainly did that. They called him an even better matchmaker than Eros. Probably his greatest accolade was an offer to become the resident matchmaker at Madison Square Garden - the biggest job in the business at the time. But the Londoner Duff never fancied living in New York.
Throughout the years he has been abrasive, but always pugnaciously engaging. "Listen," he'll say, "I want to tell you something." And what he tells is inevitably hilarious and usually outrageous. He loves a good joke, does Mickey, especially a Jewish one.
The Rabbi's son, born Monek Prager in Krakow, Poland, still possesses the sharpest tongue in boxing; verbally he takes no prisoners. Of a former fellow matchmaker he says: "He couldn't match the cheeks of his own backside." In his 55 years in boxing Duff has been involved in over 150 world title fights and thousands of promotions from Corwen in North Wales to Kuala Lumpur. He's literally been to Hull and back, dealing with the good, the bad and some very ugly customers. But whatever they say, he's never had any truck with real villains. The Krays once sent his wife a flower box containing a dead rat when he barred them from a show.
Another big-time gangster who threatened him was told: "Listen, I've got the best minders in the world and they all wear blue uniforms. I'm the biggest copper's nark you will ever meet." In their heyday Duff and his partners were labelled by the promotional tsar they deposed, Jack Solomons, as "the mob" and "the syndicate". Purple phrases to describe what in effect was an efficient cartel which broke one monopoly and established another. A "benevolent one," Duff insists. "I don't think we ever misused our powers. We never said to any manager, `If you don't do this you'll never fight for us again'. But I've had that said to me in America by the Don Kings of this world."
Ah, Don King. Another bete noire about whom Duff will have a few words to say in his book. "The cleverest in the business," says Duff of the man who traditionally hails him with the greeting, "Hey, Duffer". "He's utterly ruthless. He's screwed everyone especially the TV companies."
"I was never guilty of that. Maybe it was a mistake on my part. I never tried to bury the BBC. I stayed loyal to them for 33 years despite all sorts of offers. When Sky first came along I could have written my own ticket but I didn't. When the BBC ended our deal I believe it started the rot which has seen them lose so much major sport.
"When they say it was all the live boxing on terrestrial TV that was killing the game it's bullshit. It didn't kill boxing, it made it more popular. But Sky are killing boxing because the audience is restricted to the minority who can afford it. You need to be in the moneyed classes to follow sport these days and boxing has lost the fan in the street."
Is he still as much in love with boxing as he was? "No, as you get older so your enthusiasm diminishes. It's the same with everything. I am not as much in love with sex as I was."
For someone who has had few interests outside boxing, Duff admits he finds it hard to adjust to inactivity. "I'd be lying if I said otherwise. But I am fortunate in the twilight of my career to have a good fighter like Billy Schwer. He's keeping my interest alive." He also has what he has always referred to as "f... you" money. "I could have retired years ago but the main reason I hung on was because too many people wanted to see me go."
Duff accepts that the loss of his fiefdom to Warren was probably inevitable, although he resents the tactics involved. "What Warren did to me was only what I did to Solomons. In this respect life has turned the full circle. Maybe it would have been inevitable, irrespective of his methods. There's a better line of communication with young people if the guys dealing with them are not too old. But what I can say is that neither Solomons nor I ever poached one another's fighters."
Duff says he doesn't miss the aggravation, but he does miss the big nights. And the big wages. Like the time he bet $200,000 on the Hagler-Leonard fight and won. "But at the time it was probably 30 per cent of all I possessed. Nowadays I occasionally go into a casino, lose pounds 200 and run for my life. But I'm more aggrieved than when I used to lose 25 or 30 grand."
Duff says he is now happy to count his blessings, and his investments. He spends more time scouring the financial pages of the Times, checking the stock market than he does Boxing News. So is the last round really looming for the game's greatest hustler, now the Sunshine Boy? "You never can tell. Someone called me about a lad called Mathew Levy, a Jewish boy who's supposed to be the cat's whiskers. I had a look at him, but I have to ask myself if I have the patience to go through the whole thing again. I am not sure I've got the desire either. It takes four or five years to build up a fighter." He laughs: "Listen, I'll be glad to be alive in another four or five years."
It seems likely that, once the book is off his chest, Duff will officially retire, maybe to his sea-front flat in Israel. Septuagenarian he may be, but sad he isn't. And as King and countless others have learned to their cost, Mickey is certainly no Duffer.