At a Cardiff conference organised by the Conservative Political Centre, aimed at 'thinking the unthinkable' about the future of the welfare state, Mr Redwood condemned couples who have children, but have no intention of getting married or having a stable relationship.
When asked later how exactly fathers might be urged back into the home, Mr Redwood told the Independent: 'Firstly, by starting a debate about it. A lot is to do with social expectations and encouragement. There are cases where it is possible to bring the father on side, or tell the couple to think first before they have children because it can be prevented.
'Wherever possible, couples should provide a stable relationship and build a loving home before they have children.'
Mr Redwood said the Government should build on its new legislation to force absent fathers to make a financial contribution towards the cost of raising their children. If the mother refuses to identify the father, she risks losing benefit.
In his speech, called 'Dependency and Dynamism', Mr Redwood said: 'One of the biggest social problems of our day is the surge in single-parent families. Everyone would wish to help the family that has suddenly lost the father through death, or if the mother has been abused or badly treated by the father and the relationship has broken down.
'What is more worrying is the trend in some places for young women to have babies with no apparent intention of even trying a marriage or stable relationship with the father of the child.
'In these circumstances, it must be right, before granting state aid, to pursue the father and see whether it is possible for the father to make a financial contribution, or even a fuller contribution by offering the normal love and support that fathers have offered down the ages to their families. It would be better for the child and better for the family and better for the state if more fathers assumed their natural responsibilities.'
Mr Redwood then described a recent visit to a housing estate where he was told that more than half the families were single-parent families. He asked what action was being taken to involve the men in helping bring up the children they had fathered.
He said: 'I was told 'there aren't many fathers around here'. In that community people had begun to accept that babies just happened and there was no presumption in favour of two adults creating a loving family background for their children. It is that we have to change.'
Today, the conference will hear a keynote speech from Peter Lilley, Secretary of State for Social Security. It will also hear from Professor Michael Novak, of the right-wing American Enterprise Institute, and Joe Kirwan, former principal of Plater College, Oxford, on 'Why Beveridge (founder of the welfare state) was wrong'.