A young mother stumbles though the streets, talking to herself, clearly in the throes of a breakdown. She is known to social services and neighbours have told the police of their concerns. But she is not sectioned until she has stabbed her two tiny children to death.

Another young mother boasts that she will be out of prison by Christmas – or so we are told. Why, though, should we find such a pathological refusal of responsibility, guilt or remorse so hard to believe? We already know that she succeeded, after all, in fobbing off professionals at least 60 times, by pretending that her baby was "just" being neglected, and not systematically tortured and slowly murdered?

And yet another mother stands in the dock, accused of staging her child's kidnapping. While her daughter was missing, many people were lambasted for suggesting that a woman who, by an array of fathers, had so many children that she could not herself materially support was probably not a brilliant maternal role model. Whatever the outcome of the trial of Karen Matthews, I prefer to maintain that those who dared to suggest that her lifestyle indicated that she might not have constructed her family very carefully had every right to their opinion.

Sharon Shoesmith, the director of Haringey children's services, endeared herself to no one when she commented, with reference to the dead child known only as Baby P: "Of course we are shocked by the details of this but no agency killed this child. This child was killed by members of his own family." She may have made this claim in an effort to exonerate herself and her employees, but her statement is true, nonetheless. And she may be too partisan to understand it. But what she is really saying is this: "Don't expect much of me or my staff."

On Newsnight the other evening, Iain Duncan Smith made the most important point there is to make about child protection. He said that we had to look beyond the surface, and start realising that the problem of neglected children is huge. Social workers and others in child protection are practically paralysed when it comes to deciding what is best for children, especially in deprived areas, even when they are doing their jobs to the best of their ability.

The life chances bestowed on young people growing up in care are well known to be grim. Likewise, adoptions of even very young children often have limited success, because the children are already too disturbed to benefit from a stable and nurturing environment. Yes, great effort is made to keep children in the homes where they are being "neglected", and that's awful. But it's not all about theory or ideology (though some of it is). There is simply nowhere else for them all to go.

As a person with conventional views about how to bring up children, I'd consider a mother who had no shame in presenting her small child to authorities covered in chocolate (as the mother of Baby P did, to hide his wounds) to be neglectful enough. I don't hold with giving chocolate to babies. I don't hold with carting them about with food or anything else smeared all over their faces.

So I couldn't possibly be a social worker. I just couldn't be culturally relative enough. Where do you stop, when you decide that the chocolate isn't bad, the lack of pride in your baby's appearance isn't bad, a shifting cast of "uncles" isn't bad, a failure to provide for yourself the basics of life isn't bad, or that teetering mental health isn't bad? Where do you stop, particularly, when you know, or think you know, that the alternative isn't much better anyway?

I do understand the bind that social workers are in. I don't know why social workers closed the book on Jael Mullings, who is under section, suspected of having killed her sons. But I'll never forget what one approved social worker told me, off the record, after she'd sectioned a mother in a psychotic state, who'd been dragged away in front of her children, held down by the police, and forcibly sedated.

She said that she sometimes thought: "I'll burn in hell for what I do." She believed, at her lowest ebbs, that the way we treat mental illness now will be looked back on some day as barbarism, as ignorant and stupid as treating the anaemic with leeches. She's in the same double bind as child protection workers. She just doesn't know if her cure is any better than their ailment.

This woman doesn't work in child protection. She says there is no way she could bear the unrelenting awfulness of such employment. That is something to remember too – that social workers see dirty homes, depressed and disengaged parents, insalubrious visitors and sickly little children all the time, and naturally adjust their expectations of what an acceptable family environment is accordingly. People should be wary of colluding with social workers in that dangerous task, as they rushed to do in the case of Karen Matthews, every bit as wary as they should be in assuming that they would do much better in the same woeful circumstances.