Rosalind Nashashibi, ICA, London

Film-maker Nashashibi knows all the tricks of her dishonest trade ... and she uses them to unnerving effect

There is no star, as such, of Rosalind Nashashibi's film Jack Straw's Castle, although certain faces do recur, notably that of a middle-aged woman with glasses and a grey bob.

As it happens, this woman is Nashashibi's mother, which is odd, not because it smacks of nepotism – an error hardly unknown to the film world, after all – but because Jack Straw's Castle is a pub in an area of north London whose woods are the notorious haunt of gay men of a certain kind. Nashashibi's film begins with these men, cruising the glades of oak and birch with ritualised nonchalance. Whatever one feels about what they get up to there, there is one thing on which I think we will mostly agree: the West Heath is not the kind of place you'd take your mum.

So why has Nashashibi taken hers? The most obvious answer is because it is so inappropriate, that it plays up to the visceral embarrassment all of us feel at the conjunction of sexuality and mothers. Of course, Nashashibi's own presence in a gay cruising ground is itself incongruous: what goes on on Hampstead is not meant to be seen, and certainly not by straight women. On the face of it, Nashashibi's camera seems pretty dispassionate, as though, Attenborough-like, it is recording the mating rituals of an exotic species of animal. By enlisting her mother, though, the 2003 Beck's Prize-winner debunks the idea that the eye can ever be objective. All seeing (and, ergo, all film-making) can only ever be subjective, which, intentionally, raises the question of Nashashibi's own motives in making Jack Straw's Castle: are they anthropological, voyeuristic or confessional?

All of which is to say that her movie is not one work but two, a film about the animalism of gay promiscuity and a film about film. Nashashibi is big on dualities, on things and their shadows. At heart, her work is obsessed by the inherent dishonesty of its medium, the fact that film-making is a seamless game of smoke, mirrors and special effects. Often, Nashashibi puts in her own seams by splitting her films in two and showing them on split screens.

Such is the case with another 16mm movie in her show at the Institute of Contemporary Arts, The Prisoner. This follows a blonde woman in a black dress and clicky-clacky heels as she strides the walkways of London's Southbank Centre, past a sign that says – don't you love in-jokes? – "Welcome to the BFI" (British Film Institute). Nashashibi knows the tricks and lets us know that she knows them: the camera angles, the sudden close-ups, the out-of-synch double action, all of them filmic conventions that say "suspense". And yet there is nothing to be suspenseful about in The Prisoner, because nothing happens in it. On clacks the blonde and then it's over, an overture without an opera. Like a cinematic Pavlov, Nashashibi rings our bell and then, when we're nicely salivating, neglects to serve us our Pedigree Chum.

Downstairs, she takes this dualistic tendency further with an installation of three films shown back to back in a single room. One of these, Bachelor Machines Part 2, is on a split screen and plays alternately with Eyeballing; the third, noiseless, cuts between a middle-aged couple – Nashashibi's parents? – reading in bed and a plaster frog on the edge of a pond. How (or whether) any of these works integrate with the others is a moot point, although two of them, Bachelor Machines and Eyeballing, do share at least part of their set – the door to New York's 1st Precinct police station, with cops lolling outside looking hunky. The hunkier they look, the more Nashashibi's lens lingers on them, her voyeurism apparently candid. But then her lens also lingers on a lot of non-hunky things as well, seemingly at random.

As becomes quickly clear, nothing in Nashashibi's film-making is left to chance. As her camera trawls the façades and playgrounds and shop windows of Manhattan, a pattern emerges. We may be eyeballing them, but they are eyeballing us right back. There's a pair of black pearl earrings and a necklace, like a cartoon smile and eyes; the two Os of a shop sign, the two lenses of a pair of binoculars, a duet of stone balls, two bulls-eye windows, all of them ocular. In the game of mirrors that is Nashashibi's world, to see is to be seen, to judge is to be judged. This makes her sound moralising and, to an extent, I suspect she is. But then her main defendant is herself.

Institute of Contemporary Arts, London SW1 (020-7930 0493) to 1 Nov