The cabinet minister also implied Labour’s deputy leader launched her attack – in which she also called the Tories “homophobic, misogynistic, absolute vile” – because she had been drinking.
Speaking at the Conservative conference, Mr Gove first said he would not criticise her, telling an audience: “I like Angela Rayner, so I won’t see anything unmanly about her.”
But he then added: “If you consider the circumstances under which she’s had to grow up, and what she’s achieved, then anyone can make a mistake.”
When Mr Gove was asked if Ms Rayner perhaps meant what she said, he grinned: “It was late at night....”, without completing the sentence, amid laughter.
Labour’s deputy leader has spoken powerfully about leaving school aged 16, after becoming pregnant, with no qualifications.
She revealed she was once given dog food to eat as a child because her mother could not read labels on tins – and “jelly and shaving foam” because her mum looked at the pictures on products.
If she had grown up in “modern times” she “definitely” would have been taken away by social services, Ms Rayner said, ahead of Labour’s conference last month.
In Brighton last week, Ms Rayner told Labour activists: “We cannot get any worse than a bunch of scum, homophobic, misogynistic, absolute vile [inaudible] banana-republic, vile, nasty, Etonian [inaudible] piece of scum.”
Afterwards, she refused to apologise, calling her words something “you would hear very often in northern, working-class towns, that we’d even say jovially to other people” – and her street language.
However, speaking at a Policy Exchange fringe event, Mr Gove, the secretary of state for levelling up, told his audience: “I’m sure she realises it was a mistake.”
Mr Gove was speaking after his conference address, in which he attempted to flesh out what “levelling up” means, after widespread criticism that it is a vacuous slogan.
He outlined four “key principles”: strengthening local leadership, raising living standards, improving public services and enhancing “the pride people feel in the place they live”.
At the fringe meeting, he defended a decade of Conservative-led austerity as necessary in order to have “laid the foundations” of better economic performance for the future.
But he also admitted that savage local council cuts – leading to the closure of libraries and leisure centres, for example – had damaged “local pride”.
Local government having to “cut its coat quite finely” was one of the “forces at work” in the last decade, alongside retail moving online.
“Then you do have an effect on people’s sense of belonging and local pride and that has undoubtedly been the case,” Mr Gove said.
Join our new commenting forum
Join thought-provoking conversations, follow other Independent readers and see their replies