Rachel Getting Married - Lights, camera, music...

From TV On The Radio to Robyn Hitchcock, Rachel Getting Married has a soundtrack to savour, says Elisa Bray

The eerie strings come in, the bass drum starts pounding; it's that moment in a horror film when you know something terrifying is about to happen.

Or it's the sweeping strings of a love scene.

The film score that plays on the viewer's emotions is all too familiar, and it was something the director Jonathan Demme (Silence of the Lambs, Philadelphia) wanted to avoid in his new film, Rachel Getting Married, starring Anne Hathaway as Kym the recovering drug-addict who returns home for her sister's wedding. Demme has created a score that is not only integral to the film, but haunting in its subtlety.

Music-loving Demme, who is also renowned for his documentary films about Neil Young and videos for Bruce Springsteen, invited musicians to compose the score live on set, to complement the Dogme style of the film. It also supported the film's storyline.

"For the longest time," Demme has said, "I've had this desire to provide the musical dimension of a movie without traditionally scored music. I thought: wait a minute; in the script, Paul [father of the bride] is a music-industry bigwig, Sidney's a record producer, many of his friends will be gifted musicians, so of course there would be non-stop music at this gathering. We have music playing live throughout the weekend, but always in the next room, out on the porch or in the garden."

Throughout the unconventional filming and loosely staged scenes, a New York-based Middle Eastern ensemble, including the Palestinian musician Zafer Tawil, and Iraqi Amir ElSaffar, who played the score of Demme's documentary Jimmy Carter Man From Plains, compose the score on set. Always present at the filming, the musicians had the freedom – and were encouraged – to play whenever they were inspired to, and to ignore the camera.

Well-known acting faces mingle anonymously on-screen with musicians, artists, dancers and acting-novice friends of the director. Among them are the saxophonist Donald Harrison Jr, and the Brooklyn band TV On The Radio's lead singer Tunde Adebimpe, whose music Demme had sampled in the soundtrack to his 2004 film The Manchurian Candidate. As the groom, Adebimpe shines when singing an a cappella version of Neil Young's "Unknown Legend".

Demme and the singer-songwriter Robyn Hitchcock go back a long time. Here, Hitchcock stars as a wedding guest. At the ceremony Hitchcock, at the request of Demme, performs the song "America" from his 1982 album Groovy Decay. He also plays "Up To Our Nex", written for the movie. "It's my micro-encapsulation of the movie. The song is trying to be a voice in Kym's head." Hitchcock will release the song as his next single with Venus 3. Filmed in one take at the wedding party, he is spontaneously joined by the hip-hop star Fab 5 Freddy, the dancehall singer Sister Carol, ElSaffar and Tawil.

"My memory of the whole thing is of being at a real wedding, although without the alcohol. A lot was shot in real time and the end result was the whole thing seemed as if it really had happened. It's as real as it gets," Hitchcock says. "By the time I did 'Up To Our Nex' in the tent I had 15 people. Amir did a horn arrangement and Demme's son Brooklyn was on electric guitar.

"We hadn't all played together before. The line between reality and fiction – it was a door you could walk in and out of as much as you liked. The idea that they are just playing live, that's the beauty of it. The thing I really liked about the music in the movie is that it all happens in real time. The moments of real tension in the film are not signposted by the score. It's not telling you how you're going to react when the music comes. The music is very organic, not manipulative."

For Demme, it was about creating evocative music in the moment. "At the top of the guest list was a group of musicians who I knew I could count on to create evocative original music in the moment, while we filmed, that would free the movie from the need to have a dramatic background score composed during post-production."

The success of the score can be pinpointed in one poignant scene. As Kym is filmed entering a child's empty, sunlit bedroom and looks out the window, a haunting violin melody wafts in. "I wanted something sad floating through there," says Demme. "Kym turns around, starts from the camera, and that was Zafer Tawil's cue downstairs to start playing. I heard that haunting music and saw Anne's face respond. I went running after Zafer and said, 'Zafer, what was that beautiful tune?' He said, 'That's what I composed for you.' So this rich musical theme was revealed to us as we were making the movie – and to Anne in character as Kym. It was all in the moment and there it is, onscreen."

'Rachel Getting Married' opens 23 January