Serpentine Pavilion takes gallery visitors over to the dark side

The phrase "dark matter" is usually applied to the most mysterious substances that make up our universe. But now the legendary Swiss architect Peter Zumthor, whose Serpentine Gallery Pavilion opens to the public on Friday, has brought dark matter to Earth in Hyde Park in a building whose apparent cloister-like simplicity asks a deceptively simple question: "Where do you think you are?"

The pavilion is a black box with a monopitch roof that tips down into a rectangular, beautifully planted central space open to the sky. Many will find the building puzzling, and perhaps unsettling. The plywood walls are clad in a coarse black scrim fabric more typically used to bag silage; here and there, sand has been rubbed into the surface, and into the scrim underfoot.

There's nothing to do here except sit on the Prussian blue wooden benches and look at the flowers for a few minutes, fire off a quick text message – "OMG! I'm in a monastic kit cloister from Medievalists-R-Us!" – and then leave.

Not quite. You might, for example, muse about the way Zumthor has created a building that looks both temporary and already heavily worn. You might also find your mind latching on to the smallest details of your surroundings: the rough feel of the scrim; the luscious darkness of the enfilade around the courtyard; the deliberately rough, riffled concrete of the paths leading to the pavilion. This building waits for you like a stage set lacking only actors and a script.

Zumthor calls the pavilion a hortus conclusus – an enclosed, pensive garden. It's really an architectural memory machine. "I think of gardens I have seen, that I believe I have seen, that I long to see," he says. "Sheltered places of great intimacy where I want to stay for a long time."

Yesterday, the path around the garden was a tangle of cameras, laptops, interviews and earnestly expectant faces – a postmodern version of the final scene from that great film noir, Sunset Boulevard, in which Zumthor might have declaimed: "I'm ready for my close-up, Mr DeMille!"

Behind Zumthor, at the heart of this simple but subtly provocative space, the asters, anemones, euphorbias and geraniums stood quite still in the brilliant sunlight.

The Serpentine Gallery Pavilion, in Hyde Park, London, is open to the public from 1 July to 16 October.