Dogtooth, Giorgos Lanthimos, 96 mins, (18)<br/>Life During Wartime, Todd Solondz, 96 mins, (15)

Todd Solondz's follow-up to 'Happiness' has been out-weirded by a Greek film that takes dysfunction to a new level

As Tolstoy wrote in 1878: "Happy families are all alike; every unhappy family is unhappy in its own way." But that was 17 years before the invention of cinema, so he didn't know the half of it.

He would have been amazed and horrified to see the endless new twists on family agonies that film has devised. Until now, I'd have said that no one took the theme quite as far as American independent Todd Solondz, whose jaw-droppingly auda-cious Happiness (1998) is one of recent cinema's truly great provocations.

But Solondz may have met his match in a new Greek film called Dogtooth, which – as the title suggests – has a way of clamping itself on your attention and refusing to let go, however much it's hurting you. Even so, Solondz is on daring form with Life During Wartime – loosely speaking, a sequel to Happiness. That film was an unforgiving ensemble comedy about human weakness, in which the characters included three sisters, with three different modes of neurosis, and a bland suburban dad who was secretly a predatory paedophile. The scene at the end in which his young son asks him exactly what he does is the most devastatingly uncomfortable thing I've ever witnessed on screen.

Solondz's subsequent films Storytelling and Palindromes seemed a little incomplete by comparison, too formally elusive to hit home to the same extent. Life During Wartime, however, is both a reprise and something new.

It follows the Happiness characters, a few years on. Disgraced father Bill is newly out of prison, while his wife Trish has taken the children and moved to Florida; there she's met a new man (Michael Lerner), whom she falls for because he's so reassuringly normal. Meanwhile, Trish's uncrushably naive sister Joy is still having trouble with men, even the dead ones: one old lover returns from the grave to guilt-trip her.

Life During Wartime might seem a desperate move, an unnecessary follow-up to something impossible to follow; but the conceptually-minded Solondz is doing something quite singular here. The characters are the same but the cast is not, with very different actors stepping into the old roles. As Bill, Dylan Baker – so unsettlingly bland-seeming – is replaced by the brooding, hulking Ciaran Hinds, as though the character has been mentally, emotionally and physically transformed beyond recognition by his exposure. Joy, originally Jane Adams, is now Shirley Henderson, embodying a more fragile, other-worldly ditziness. As Joy's old beau Andy, now an angst-wracked revenant, Jon Lovitz's successor is Paul Reubens, aka disgraced kids' TV host Pee-Wee Herman. This is extremely provocative stunt casting for a film that muses on the possibility of forgiveness, and Reubens is poignantly troubling, his haggard face attesting to the effects of real-life trauma. And as the new Trish, Alison Janney makes a dazzlingly clueless queen of suburban complacency.

Life During Wartime may seem familiar: Solondz playfully admits as much when Joy experiences "just a little déjà vu". But by explicitly framing the film as a reworking of old material, Solondz messes with our expectations and erodes the certainties of character. There are new things too, notably Charlotte Rampling's self-proclaimed "monster" (inset below) who has a passing liaison with Bill: it's her most frightening performance yet, a study in the human heart as ravaged bombsite.

The family is the ultimate place of attrition, but parents and callous siblings aren't the only ones doing harm. The character who causes the profoundest damage is Trish's younger son Billy (a brave and compelling performance by Dylan Riley Snyder) who, after all, is doing nothing more than trying to understand the world in time for his barmitzvah. Solondz himself resembles the anxious suburban kids of his films. He too is always asking terrible questions about life and then, however dreadful the answers, pressing on and never stopping till he's unearthed even worse truths.

But even Solondz might be alarmed by the family in Dogtooth. A middle-aged couple have raised their children – now young adults – in absolute seclusion, feeding them bizarre lies about the world and about language. As far as the son and two daughters know, aeroplanes are tiny things that occasionally crash in the garden, cats are flesh-eating monsters and the sea is a sofa. The children entertain themselves with games of dare, while the son occasionally receives visits from Christina (Anna Kalaitzidou), a security guard paid by Dad (Christos Stergioglou) to keep the boy sexually serviced.

The claustrophobic comic nightmare is all the more intense because everything is made to seem routine, almost normal. The family villa is a clean pleasant place, swimming pool and all, and the sunlit photography creates an idyllic mood – all the better to offset the dominant oppressiveness. Director/co-writer Giorgos Lanthimos keeps us guessing and explains virtually nothing. He also provokes with an edge of queasy sexiness that's all the more unsettling for being utterly perverse and downright awkward.

The extraordinarily disciplined acting (Aggeliki Papoulia and Mary Tsoni are the daughters) shows an ensemble united by a spirit of adventure, a commitment to follow this wild Theatre of Cruelty farce to whatever extremes it leads. You could mention Buñuel, Haneke, Atom Egoyan and Sweetie-era Jane Campion by way of comparison, but Dogtooth is pretty much one of a kind. Its view of family life is so outré that Lanthimos may have the edge on Life During Wartime – but he and Todd Solondz would have lots to talk about. Their folks should feel very proud of them.

Next Week:

Jonathan Romney prepares to grapple with a couple of real sluggers: Iron Man 2 and Viking drama Valhalla Rising