They do concede that he's not the slickest Kennedy ever to run for office - he has a fuzzy educational record, and has been tagged as the dumbest guy on Capitol Hill - but they insist he is surrounded by excellent people and does a fine job both in Congress and in his constituency office. He's worked hard on low-income housing and getting banks to invest in poor neighbourhoods, and has made many brave stands on human rights.
His personal record with women is patchy, but politically they couldn't hope for a better friend. How could Sheila put her own concerns before this greater good? Why didn't she realise that she was not going to have a normal life if she married a Kennedy? They just can't understand it, but now that I have her before me, the answer is clear. She was never going to accept the full Kennedy-wife deal.
At 48, she's more handsome than pretty, and in a serious, high-minded sort of way. She talks without embarrassment about searching for the truth and struggling to do the right thing. She's no airhead - she has a degree from the Harvard School of Design and worked for many years as a city planner. She's not just a Protestant, she's a devout Protestant. She insisted that her Episcopalian minister be there at the altar right next to Joe's priest at the wedding, and she did not promise to obey.
She did make all the other usual promises and took them very seriously indeed. When their twin sons were born in 1980, she pushed her career to one side and applied her high standards to motherhood instead. And she's proud to say she has something to show for her effort. Her sons, she tells me with a Diana-like smile, are "light years ahead of their father". Unlike him, they are "very respectful of their friends who are girls". She mentioned one in particular who was very good at ice hockey and "a little heavy". "Their father would say something like, what chick would want to play ice hockey? But they don't think of her as an unattractive chick, they think of her as an excellent athlete."
It was when Joe went to Washington that the marriage began to fall apart. She was the one to file for divorce. She got only a nominal settlement and had to get a loan from her parents to find a house for herself and her boys to live in. Despite this, she claims it was an amicable split. She has nothing but good things to say about Mary Beth, Joe's new wife, who entered the scene as his secretary. She's glad they're married, glad they're happy, glad the boys have a kind stepmother. What she's up in arms about is Joe's decision to seek an annulment so that he can make the picture perfect and marry Mary Beth in church.
Even she was surprised by the revulsion she felt when she received that first letter from the Boston Archdiocese, in which they announced that they were planning to "investigate" her marriage with a view to proving it had never existed. How could anyone make such a statement about a courtship that had lasted nine years, and a 12-year marriage that had produced two children? To Joe it was just "Catholic gobbledygook" but to her it was saying the past two decades of her life had never happened. But when she told the Archdiocese she had no choice but to fight the annulment, she still expected the procedure to make sense. "I had respect for them. They were priests." Four years on, she still talks with outraged surprise about their secrecy, their Alice in Wonderland logic, and the way they called her mental health into question.
But, like all good graduate students, she saved her anger for the library, and before long she had discovered that the American Church was now granting 60,000 annulments a year, mostly to erase early, short-term mistakes, but all too often to allow men to dissolve very traditional 20 and 30- year marriages with impunity. The original motive was compassion - the divorce rate among Catholics is 40 per cent - and they wanted to find a way of giving people a second chance.
But the arguments they used to justify these annulments ran counter to the church's own teachings. "When you go to get forgiveness for murder, the priest doesn't try to convince you you didn't kill someone. If you had an abortion and try to get forgiveness, you're not told you were never pregnant." To have one set of rules for marriage and another for everything else was a slap in the face of any woman who had ever tried to be a good Catholic wife. By the time Joe got his annulment, she was adding the finishing touches to a book that showed the national boom in annulments for what it was. Although it told the bare bones of her own story, it was mostly about other women and a model of decorum. "I didn't want it to be a Kennedy book," she says. But she did use its launch to announce to the world that she was taking her case to the Vatican. Joe took stock of his falling polls and withdrew his candidacy.
Is she repentant? Not a bit, she says. "My kids are juniors in high school. Even in the best of circumstances they don't need their father running for higher office. People in Boston act like Joe's been exiled to Siberia, but really he's in a perfectly good situation." He'll run for Congress again, she says, and will be perfectly placed to run for the Senate when his uncle Teddy retires. As far as the book is concerned, he got off lightly. "He knows what I could say," she says with an almost wicked laugh. When I remind her that she did say a number of unkind things about him, she shrugs her shoulders and says, "I think Joe would think he was a wimp if someone told him he was a good listener. This is a man who has described himself as the family pitbull."
After more than 20 years of being asked to put her personal life to one side in the name of the Kennedy political machine, she's had it. This issue was important, too. "There was a whole conspiracy of silence. I was being told, 'Kennedy wives don't talk', but they were having it both ways. First Joe's saying we were never married, and then he's saying I'm supposed to be quiet because I was once a Kennedy wife! So, I had to speak. I wasn't going to tell all, but I was going to speak in an appropriate and responsible way."
But what if the Vatican decides not to annul the annulment? I ask. This is the only time I see a shadow cross her face. "I think that's the end of the line. If they don't, they don't." But I don't just see her going back to the dishes as if nothing happened. She clearly has enjoyed standing up for her rights in public. She must have worked out by now that the future belongs to wives who refuse to go quietly.
'Shattered Faith', by Sheila Rauch Kennedy, Poolbeg Press.