In the interview, conducted by psychologist Oliver James, Clary says the relationship during his early twenties, while he was studying at Goldsmiths' College, London, was "very satisfactory".
"I think he felt it wasn't for him, but it was certainly a very close relationship which was intimate over a period of time," says Mr James. His series The Chair also includes interviews with the former sports presenter turned guru David Icke, EastEnders actress Patsy Palmer and hypnotist Paul McKenna.
"We talked about the difference between having sex with a man and sex with a woman and I got the impression that he hadn't really found it very different. The implication was that she was rather masculine... He seemed very at ease with it."
Clary, who is 38 today, was a shy child who had known he was gay from a young age and always felt his difference from other boys at his Catholic public school, St Benedict's in Ealing, west London - alma mater of Hong Kong governor Chris Patten and the novelist Peter Ackroyd.
In an interview to be broadcast on Wednesday, Mr James speaks to Paul McKenna, who suffered at his strict Catholic school where he was regularly smacked, told he was bad and led to believe that thinking bad thoughts could send him to hell.Reuse content