Peter Zumthor's experiments in space

The designer of this year's Serpentine Gallery pavilion is Peter Zumthor, a visionary whose best work is as much land art as architecture. Jay Merrick meets him

Peter Zumthor, the legendary Swiss architect who is designing this year's Serpentine Gallery pavilion in London, suggested that I visit one of his buildings.

So I walk uphill in steady rain, to St Benedict's Chapel, which lies on the Alpine slopes above the village of Sumvitg. Having started at the little railway station on the floor of the Vorderrhein valley, the yomp has taken the best part of an hour, the narrow road angling upwards in a long series of hairpin bends through steep fields tonking with cowbells and smudged here and there with small flocks of sheep.

The clouds are both above and below me and then, just past the ruinous walls of a monastery, the lane takes a final kink upwards. The chapel appears. It is so utterly other that it repels description. The chapel is shaped, in plan, like a pointed oval or a marquise-cut diamond, and covered in unusually small wooden shingles, applied with exquisite precision. The building is so singularly itself that it can't even be called a type of architecture. It simply exists, a kind of land art or a perfectly crafted glacial outcrop consecrated to the god of this particular place.

What kind of architect designs a building like this? An architect who has won the profession's Oscar, the Pritzker Prize, and is revered by legions of younger architects; an architect who's prepared to design a new building for the Los Angeles County Museum of Art, but said no to the new chapel at Magdalen College, Oxford; an architect who, with buildings like the thermal baths at Vals, Switzerland, has brought new levels of craft and atmosphere to modern architecture; and, most tellingly, an architect who is more interested in artists and memories than in architects and manifestos.

Three hours after trudging back down to Sumvitg railway station, I'm gazing at something that is more dream than architecture: Zumthor's early conceptual drawings and model shots of the house in Devon that he is designing for Alain de Botton, for one of the lifestyle guru's Living Architecture series of holiday homes.

The models are inchoate and very nearly formless – shards of slate, some overlapping, surrounded by crudely wired representations of trees. It's the kind of rough, primevally mysterious conceptual material that would probably be slaughtered in a student "crit" at London's self-consciously cutting-edge Architectural Association.

Peter Zumthor is immersed in things that are severely untrendy, and profoundly subtle: the relationships between places, memories, moods, atmospheres, films, art, theatre. It is no surprise to hear him speak, with evident pleasure, about "the conceptual guys in the 1970s, like Richard Serra. I saw these Americans coming back to biography through material. If you look at some art you can see that, in the case of some artists, this is exactly what they're trying to do."

And here he is, from his riveting book, Thinking Architecture: "I am convinced that a good building must be capable of absorbing the traces of human life and taking on a specific richness... I think of the patina of age on materials, of innumerable small scratches on surfaces, of varnish that has grown dull and brittle, and of edges polished by use. But when I close my eyes and try to forget these physical traces and my own first associations, what remains is a different impression, a deeper feeling – a consciousness of time passing and an awareness of human lives that have been acted out. At these moments, architecture's aesthetic and practical values, stylistic and historical significance are of secondary importance. What matters now is this feeling of deep melancholy. Architecture is exposed to life. If its body is sensitive enough, it can assume a quality that bears witness to past life."

This reminds me of the great Brazilian architect, Oscar Niemeyer, who told me that on Sundays he always spent the afternoon alone, listening tearfully to old tango records. Zumthor's architecture arises from the landscapes and homely objects of his own past, small but extraordinarily resonant memorabilia from 66 years of life that form a rosary of truths. In the maple-lined living room of his home and studio in Chur, he says, very quietly: "Designing is a matter of concentration. You go deep into what you want to do. It's about intensive research, really. The concentration is warm and intimate and like the fire inside the earth – intense but not distorted. You can go to a place, really feel it in your heart. It's actually a beautiful feeling."

This is what he's striving for with the Serpentine pavilion, which will be, he says, a hortus conclusus – or enclosed, secret, garden. When Zumthor released images of the pavilion model, they were absolutely un-slick, if not vague. As with St Benedict's chapel, there was the strong sense of an encounter with an object that resisted categorisation. Yes, the pavilion resembles a cloister – but no, it isn't a cloister; there is something fugitive about the form that denies that label. The only definite thing that can be said about the design is that it is clearly prompted by the rectangular courtyard garden that partially divides Zumthor's home from his studio.

There is something characteristically dense and still – Hopper is one of his favourite painters – about much of Zumthor's architecture, which he often prefers to have photographed, or perhaps memorialised, in black and white.

Buildings such as his thermal baths at Vals in Switzerland, and the Kolumba museum in Cologne, create extraordinarily pregnant atmospheres in which light, shadow and shifts in acoustics give spaces something very like the tensions of a held breath, a momentarily mysterious vision, or a sudden cinematic stop-frame.

And so he asks us to travel from fragments from his memory to the concentrated other-ness of his architecture. Zumthor muses that: "The bottom line may be that my inventing buildings is, indeed, a very private kind of activity. But it's done to be shared. It is comforting and consoling. From the reactions I get I can see I'm not doing something strange."

He considers a great deal of contemporary architecture to be wilfully ephemeral, and is equally underwhelmed by stolid architectural discourse. Architects, he tells me, are rarely artists: "Architecture has, of course, always been connected with the construction industry and money, producing status symbols." For Zumthor, there has to be something about a project that makes a connection to his hearth of memory – an innocence that will, in a finished building, trigger innocent reactions from others. If not, it's no-go.

"It's a choice," he says. "You have to know what you're doing. I'm not an implementer. I want to think. Maybe you can feel that I'm honest in what I'm trying to do – this little boy trying to do his best. He's not trying to trick you, or pretend. What you see is what you know I do."

My socks are still sodden from the two-mile walk up to St Benedict's Chapel. But in this living room, I feel like an Upmann Corona Major cigar at rest in a beautifully made humidor. And, in this single moment, I think I know what Peter Zumthor does.

The Serpentine Gallery Pavilion (www.serpentinegallery.org), Kensington Gardens, London W2, is open to the public from 1 July to 16 October

Airs and graces: four of Zumthor's best

Thermal Baths, Vals, Switzerland

In what is probably his most famous building, Peter Zumthor created a narrative of subtly fluctuating atmospheres using spatial, material, and sonic variations.

Brother Klaus Chapel, Eifel, Germany

A perfect example of the stark otherness of Zumthor's architecture, a landmark like a supersized trigonometry-point marker, whose interior is "tented" with 112 tree-trunks.

Kolumba Museum, Cologne

A brilliant architectural orchestration in which a new Modernist form, and spaces, are anchored to the bomb-damaged haunch of a Gothic church in the centre of Cologne.

Kunsthaus Bregenz, Austria

Zumthor created a concrete cube with shimmering glass walls for the Austrian art museum which overlooks Lake Constance and is held up as architectural minimalism at its best.