There was a fuss some years ago when he went on a television programme, and declared his own homosexuality; there was even more controversy when he recently produced "evidence" that Enoch Powell had been a youthful homosexual.
By taking his call for an examination of the hereditary monarchy into Westminster Abbey, along with a suggestion that Diana Princess of Wales had been crucified, the Extra Chaplain to the Queen can hardly be surprised if last night's lecture of "Spirituality, Shakespeare and Royalty" creates a row.
As a chaplain to the Queen between 1984 and 1995, when he became 70, and an "extra" chaplain since then, Canon James claims no special relationship with the Royal Family.
But he believes that his position as a chaplain does make him a part of the royal circle.
He told The Independent: "I conversed with the Queen once, when I was stroking her corgis and got caught up in the leads when all 28-or-so chaplains - is that a covey of chaplains? - were being taken around the royal apartments.
"You preach at one of the Royal Chapels each year, and 'such other duties as it shall be Her Majesty's pleasure to command'.
"In fact, you preach once a year at, almost certainly, the Chapel Royal, in St James's Palace, and she's rarely present. But she invites you to some jolly most years, up to Holyrood or Sandringham, or whatever it is."
After last night's lecture, Canon James would be lucky to be included in the next "jolly". But, then, he has never been one of the court's "toadying sycophants".
In the 1950s, he was chaplain to Trinity College, Cambridge, and he told The Independent: "I watched the grammar school boys from working class areas make some of the best contributions to the civil service. I doubt whether they would have got anywhere near the royal household."
One of the motivating forces of his life, he says, has always been an awareness of the underdog. His father was a valuer and assessor in Dagenham, but Canon James left school at 14, just before the Second World War, and worked as an office boy for seven years on the Thames-side wharf where the Globe Theatre now stands.
Those were years when he carried out duties - alongside London dockers - as a fire-watcher, and he says now: "That was when all the class barriers came down."
During that time, he also worked hard at night school and after the war, he was accepted as a student by King's College, London, where he studied theology.
In his lecture last night, Canon James recalled that when he had first started his training for ordination, he had been urged always to think "theologically", and he said that he was applying that instruction, now, to his thoughts on the monarchy. "It is worrying to me," he said yesterday, "that at such a time, when such a question is up for discussion, I cannot name you one theologian, or one higher-up, who has raised it."
He said that the implications of government proposals to end the voting and speaking rights of hereditary peers in the House of Lords were not being discussed at all.
"It raises the whole question of primogeniture," he told The Independent. "It goes right the way through the system, and through society, too.
"I still think there are quite a lot of people in working-class houses who talk proudly of their sons in a way that they don't talk of their first daughter. It is a very odd thing.
"Is the church afraid of addressing such questions? I do not want this to sound like a revolutionary bleat, as it were, but I am saying that it needs to be looked at."Reuse content