Speaking in the garden of their new home in Kendal, Cumbria, Roderick Wright said he and Mrs Macphee were sorry for the suffering caused both to their families and to the church in Scotland.
But he pleaded for them to be left in peace while they proceeded with their lives, and their faith, "as best we can".
Bishop Wright, 56, ran off a month ago with Mrs Macphee, who is 15 years his junior and who has three children.
It then emerged he had also fathered a son, now 15, by another woman. After some time in hiding, he contacted the Church of Scotland to quit his post and the couple sold their story to the News of the World.
At the press conference yesterday, Bishop Wright said the couple had decided to speak publicly out of concern for the intrusion into the lives of their neighbours by waiting press. Since returning to their home a few days ago, Bishop Wright and Mrs Macphee said, they had been sitting in darkness in order not to alert the reporters and photographers outside.
Speaking from a prepared statement, Bishop Wright said: "We wish to apologise, especially to our families who have suffered so much, not only because of the manner of our leaving but because of the intense pressure placed upon them by the media. We also wish to apologise to the Catholic church, in Scotland in particular, and especially to the priests and people of Argyll and the Isles. Our message to them is heartfelt sorrow for the sadness and pain caused. Neither of us wished to cause such pain."
Bishop Wright said he and Mrs Macphee had both grown stronger through the trauma of recent weeks.
"We are in love and intend to marry," he said. "But at this moment we are living from day to day and we do not know yet when that will be."
Although their relationship was not sexual at the time when they ran off together, the couple confirmed yesterday that they have now become lovers.
Following reports of unhappiness among Mrs Macphee's children at the relationship, Bishop Wright said they were "fully supportive" of her.
And in response to questions about his and Mrs Macphee's attitude towards the Catholic church, he said: "We are both Catholics and intend living our faith as best we can. My active role as a priest can no longer continue, and that part of life I miss. We both accept the church's law and have no criticism to make."
Fr Tom Connelly, a spokesman for the Catholic church in Scotland, said it had "absolutely nothing to say about the situation regarding the former Bishop Wright, since he has not been in contact with this office". The church was not in the habit of discussing another person's business through the media, Fr Connelly added.Reuse content