Kristin Hersh's songs come to her, unannounced, with their own agenda and causing pain and nausea. This, she believes, is one of the reasons for her well-documented psychic turmoil, which began at the same time as she launched her band Throwing Muses. Glyn Brown discovers the humour that belies her angst.

For one brief moment during this interview, as Kristin Hersh blinks reddening eyes, I fear it might turn into another "galumphing journalist makes sensitive artist cry" shock. It doesn't, because Ms Hersh has a fine old sense of humour to go with her Johnny Cash-deep voice.

Any hack who hadn't met her might be forgiven for being tentative, though. Hersh has been around the psychic block a time or two, and it comes out in her songs: first the ground-breaking music of her Rhode Island band Throwing Muses, which was wild and traumatic, full of shifting time signatures and distorted guitar, and then the dark acoustic ballads of her solo album, Hips and Makers, released in 1994 to commercial and critical acclaim.

Shortly before this, she admitted to suffering bipolar disorder, a cousin to schizophrenia; since then, she has been seen by some as the Mrs Rochester of indie-pop or, as she usefully puts it, "one wave short of a shipwreck".

Hersh, 31, is far from barking, as the warmth of new LP Strange Angels, with its folky laments and Wild West guitar, will confirm. Still, though it feels like a love letter to her husband and mentor, Billy, there is a hint that the badlands aren't quite history yet. I think we're here to find out where this dislocation comes from.

A starting point might be the acid-fuelled hippy commune on which she grew up in Sheffield, Massachusetts. Apparently, when the babysitters took the children to the park, they would often pull up screeching, "Kristin! Look at all the ostriches!"

Your parents put you in the care of these people? She shrugs. "I think they were on too many drugs themselves to notice."

The ceiling of the barn they called home was decorated with a parachute upon which someone painted "Be together". "Except the guy was really stoned, so he wrote "Be a tog eater". And my father still signs his letters that way - `Be a tog eater, love, Dad'."

Hersh's parents hailed from the Chattanooga Mountains, where their rather gothic family still hang out; last Christmas, Kristin noticed a dysfunctional automated Santa that would only say Happy Holidays once a knife was stuck in its back.

Her dad, a philosophy professor, lives in a self-built woodland house "with gargoyles and Greek gods stuck all over it". She sighs. "He's remarried an anorexic painter. They forgot to build a kitchen."

Ryder, Hersh's six-year-old, is growing up traditionally. "He's got quite a thing with Satan going on. We'll be driving in the car and he'll look out the window, look kinda distant ... and you know he's thinking about Satan. Say, Ry, what's up? And he'll say, I was just thinkin' that horns and a cape and a tail is a really good way to dress."

Hersh had her own childhood obsessions. When not stoned, her dad sang her Appalachian folk songs, which made her cry but got so deep in her psyche that she is thinking of turning them into her next album.

"They're actually really funny because they're so gross, all liquor, Jesus, murder. And for the women, whether or not they're married." If they weren't married? "They'd just kill themselves. When dad used to play me this one about drowning, I was like, right on, sister, go live with the mermaids, you don't have to get married if you don't want - it's called `I Never Will Marry' and I thought, y'know, that was her choice. But no, she's drowning herself because she doesn't wanna live single for the rest of her life. Now, should I really record that?"

Hersh first married (this one didn't last) at 17, three years after starting the now-defunct Muses. At 14, too, her bipolarity kicked in. Because the songs she wrote would visit her - and still do - unannounced, fully-formed and with their own agenda, causing the same nausea and labour pains as childbirth, she believed her psychic turmoil was caused by them. She was still seeing wolves leaping out of walls when she became pregnant. These days she takes lithium "to keep those things away from me and my family".

Back then, she thought the visions, embodied in songs, could get inside and hurt the baby. "So I had to cut a deal with the songs," she mumbles. "Because I realised, a few years ago, they wouldn't go away."

Surely you'd hate it if they did? Innocent question, relating to inspiration. But for Kristin, the songs are other-worldly children, occasionally malevolent Caspars.

"I guess I'd hate it. It's hard to admit that, and I've never really been able to say it, we should probably change the subject soon, because I'm [weak grin] about to start crying. But I did say, I won't censor you ever again if you'll stop knocking at my door at four in the morning and fucking with my family. And the songs are better for being told, they don't make me say such crazy shit. They often have subtle and very pretty things to say. Now they're not so mean."

"Madness" can sometimes be affected for a bit of rock biz glam, but not in this case. Hersh is contorted with embarrassment and guilt. Embarrassment because "I'd maintained, this is what normal women are like, my music can speak for anyone, don't call me crazy just because you haven't heard this sound before. Then there I am saying, ah well, I was crazy after all." And guilt because "it's other people who suffer. I won't say it's affected my children, because I can't face that. But I'm sure it has. And I know my husband spends most of his waking hours trying to make sure it has. And I know my husband spends most of his waking hours trying to make sure it doesn't happen again."

As Ryder and his baby brother, Wyatt, come bouncing into the hotel room, Hersh is telling me one of the final Throwing Muses stories. In it, she and drummer David Narcizo are skating near her Catskills home when they hear a rifle shot. Of course, it's the ice cracking, "And we thought that was very fitting, talking about music and skating on thin ice." If you're lucky, though, you can pull it off.

`Strange Angels' is released on 2 February. Kristin Hersh plays the UK in March.