Mr Gove - who was himself adopted at four months - said it was "disgraceful" that black children are three times less likely to be adopted from care than white ones and "outrageous" for them to be denied a loving home because of a "misguided" belief that race is the most important factor in considering potential adoptive parents.
He will publish an Adoption Plan next month to sweep away bureaucratic hurdles and make the process of finding a permanent home for children in care in England swifter, simpler and more flexible.
And he made clear that this will include changing the law on inter-racial adoptions.
Despite guidance on the issue last year, a number of local authorities are still putting their belief in finding "a perfect match" ahead of the interests of children, said Mr Gove.
"There are still parts of the country where the wrong values reign and where we are not doing enough to make sure children are adopted quickly enough," he said.
"We will be legislating. The decision is ultimately the Prime Minister's and there are a range of options he is looking at, but he is clear that while there has been an improvement and the guidance has helped, there hasn't been a big enough improvement and there are some hold-out areas."
His comments came as he answered questions following a speech in London in which he spelt out his ambition to "radically" increase the supply of adoptive parents and reduce the length of time children stay in care.
"We are determined that adoption should happen more often and should happen more speedily," said the Education Secretary.
"By changing our attitude towards adoption, reducing the unnecessary bureaucracy of the assessment process and freeing up professionals to rely on their own judgment, I feel confident that we will be able to create a more efficient and effective adoption system.
"I will not settle for a modest, temporary uplift in adoption numbers, nor a short-lived acceleration in the process. Nothing less than a significant and sustained improvement will do.
"The most neglected, the most abused, the most damaged children in our care deserve nothing less."
Mr Gove said he was "angered" by statistics showing adoptions have fallen by 17% in England to just 3,050 over the last decade, with only 2% finding new homes before the age of one and the average adoption taking two-and-a-half years.
He blamed "bloated" assessment procedures which exclude too many parents and dishearten many others.
Along with groups of siblings and children with disabilities, those from ethnic minorities are the hardest to place, said Mr Gove.
Would-be adopters should be "welcomed with open arms".
While an ethnic match between adopters and child may be "a bonus", it was "outrageous to deny a child the chance of adoption because of a misguided belief that race is more important than any other factor", said the Education Secretary.
"If there is a loving family ready and able to adopt a child, issues of ethnicity must not stand in the way."
Mr Gove was also critical of councils which refuse or delay adoptions because parents were smokers or had pets or - in one case - needed a kettle with a shorter flex to comply with safety rules.
"We need radically to increase the supply of adoptive parents who are ready to give these children the love and stability they need," he said.
"I entirely reject the argument that there are too few people willing to adopt. I think there could be a vast supply: parents with their own children; couples - heterosexual and homosexual - unable to have children of their own; single individuals, both men or women.
"But the barrier which looms between these prospective parents and their potential children is a process of recruitment and assessment which turns enthusiasm into exhaustion and optimism into despair."
Next month's Action Plan will cut back form-filling and streamline the assessment process in the hope of increasing the number of children adopted during their first year, when they have the best chance of settling successfully in their new family.
Mr Gove accepted he may be accused of being "cavalier" with children's safety.
But he insisted: "What I believe would be cavalier would be to allow the continuation of an adoption process which is so slow, so inefficient, that we condemn thousands of children to a life without parents."
David Simmonds, chairman of the Local Government Association's Children and Young People Board, said: "Local authorities take their responsibilities towards children in their care extremely seriously and this includes seeking permanent and stable loving homes for these children wherever possible.
"We acknowledge that there is a variation in performance across councils and recognise that at times the system has been risk-averse, but we want to work with Government to change that and remove barriers that delay decisions, including tackling the significant delays in the family courts."
Barnardo's lead director for children in care, Jonathan Ewen, said: "Barnardo's agrees that the speed and quality of decision-making throughout the adoption process must be both transparent and urgently improved and we look forward to the Government's forthcoming action plan.
"From the moment a child is identified as being in neglect everyone should be mindful of the child's timescale - we must remember that 12 months is half a lifetime for a toddler.
"We also need to encourage people to come forward to adopt children - including older and black and ethnic minority children - and the availability of continuing support for both the adoptive family and the child is also critical if we are to give more families the confidence to adopt and prevent placements breaking down."
Anne Longfield, chief executive of family charity 4Children, said: "Anything that can make the adoption process faster and simpler will clearly benefit the record number of children going through the care system.
"Though there is a difficult balance to strike between safeguarding, helping families to stay together and taking decisive action, hurdles to the process of adoption are often against the best interests of children.
"Adoption tends to be a consequence of family breakdown and crisis. The Government must focus on early intervention and provide more support to prevent family breakdown in the first place."
The British Association for Adoption and Fostering (BAAF) welcomed Mr Gove's focus on adoption for those children who cannot return to their birth families and said it was looking forward to the publication of his Adoption Action Plan.
BAAF chief executive David Holmes said: "We all want to ensure that unnecessary delay in adoption is minimised both for children waiting for placement and for prospective adopters.
"Michael Gove is right to highlight the shortage of prospective adopters. We need more people to come forward who positively want to adopt the children who are currently waiting for adoption.
"In particular, we need people to come forward who want to adopt children over three, children in sibling groups, disabled children and black and minority ethnic children.
"We must also invest in adoption support to ensure that adopted children and their families have access to all the support they need when they need it."
PA
Comments