The letter from the agency - which sets maintenance payments for absent fathers - was opened by the man's wife while he was at work.
Janet Sheppard, 40, of Brislington, Bristol, said: 'As he walked through the door, I was ready to kill him. One minute I was screaming and shouting, then I was crying and then I didn't know what to do.
'I was saying to him that if it was a one-off, then we should talk about it.'
Her husband Steven, 38, contacted the CSA immediately but it took fours hours of frantic telephone calls before the agency, which has recently introduced a performance-related pay scheme for its staff, accepted it had made a mistake.
Mr Sheppard said: 'I was just stunned - I had never heard of the woman. I knew I hadn't done anything wrong but I had to convince my wife.
'She had that doubt and wanted to know what I had been doing and where I had been. For those four hours it was crazy here. I could understand my wife's reaction but I am so angry with the apology.'
A CSA spokesman said: 'We make very few genuine mistakes but this was clearly one of them. We apologise for any distress and embarrassment caused.'
Graffiti was daubed yesterday on the front door of Peter Lilley's home in Islington, north London. The Secretary of State for Social Security, whose department oversees the work of the Child Support Agency, was later targeted for a protest by campaigners against the agency.
The 20 protesters, who gathered outside Mr Lilley's house, said many women were being forced to authorise claims against their former partners against their wishes. The protesters' spokeswoman, Nina Lopez-Jones, said many women feared harassment from former partners, or simply did not want any contact with them. 'Women who are claiming benefit are not being told that they do not have to sign authorisation to claim from their ex-partners,' she said.
One of the protesters, single parent Kim Sparrow, 28, from Camden, north London, said she did not realise she was pregnant until after she and her boyfriend had split up.
'I am refusing to sign because I do not want anything to do with my ex-boyfriend. I want to live independently with my daughter and I do not want to be financially dependent on a man,' she said.
Ms Sparrow said she had not intentionally set out to get pregnant, but she believed the work she did as a full-time mother entitled her to state support.
Mr Lilley, defending the agency, said: 'There's been a lot of criticism and abuse but I would remind you of one fact - in 96 per cent of the cases that have been pursued by the agency so far the mother and child are dependent on benefit, though generally the father has a reasonable income.
'It seems to me that in the first instance it is his responsibility to contribute to the maintenance of his first family. And the taxpayer, who may often themselves be struggling to maintain their own family, should only come in and do so when neither parent has the means to do so.'