Four hours later, dripping wet, Cocker is still at the gate. Complaining half-seriously about getting pneumonia, the hood of his parka pulled up around his face, he is geeky and miserable, the antithesis of the groomed presenters who glide across British TV screens night after night. Finally his persistence pays off and he is permitted to go inside to look at the house Monsieur Chomo has turned into a work of art - something the rest of us will have to take on trust, since the artist is resolute in refusing to allow Cocker's cameras in.
Cocker relays this decision to the crew - and his TV audience - with an air of exhausted resignation. It is to his credit that the incident remains in the finished film, instead of ending up on the cutting-room floor, as it might have done in more polished documentaries. Recalling the frustrations of the shoot months later, when we meet in a tapas bar near Paddington station, he remains philosophical. "I hope one of the things that makes the series interesting is that it's done as a journey," he explains. "I had to be prepared for things not to work out."
Fortunately for Cocker, whose perpetual art student image remains intact despite the phenomenal success of his Indie band, Pulp, most of the people he wanted to talk to were more cooperative. Cocker and his crew were welcomed into a house in the French countryside where every surface glitters with a mosaic of broken crockery, and into Watts, a poor suburb of Los Angeles, where an artist named Simon Rodia has spent 34 years erecting 100ft towers in his backyard. In the final programme, Cocker visits the Indian city of Chandigarh to see Nek Chand's Rock Garden, constructed on the edge of town from urban detritus - oil drums, concrete, old bicycle parts.
The series is the culmination of Cocker's long interest in Outsider Art, a movement - if such disparate and far-flung individuals can be corralled together under that rubric - consisting of people on the fringe of society and especially of the art world. He stumbled upon its existence by accident 10 years ago, during his time as a mature student at Central St Martin's College of Art, when he discovered a book about it in the college library. What seems to appeal to him about the projects in the series is that they are intensely personal, incorporating art into the fabric of the artist's life - taking over the artist's life in the case of Le Palais Ideal, an extraordinary grotto-like structure created over 30 years by a French postman, Ferdinand Cheval.
When I ask Cocker whether he likes formal art, he replies quickly, "I haven't got a problem with formal art." But when I ask whether he visits places such as the National Gallery, which contain work created over hundreds of years in a vast range of circumstances, he demurs. "I don't like going to art galleries. They remind me of going to church - the formal atmosphere, people looking at things on the wall, attempting to get some kind of mystical revelation from them." I ask Cocker if he was religious as a child.
"I didn't have a religious upbringing," he says, launching into another attempt to explain what he dislikes about the kind of art which hangs on walls. "It's always been a bee in my bonnet. I'm sure that art was first invented by cavemen to communicate things that happened to them in their day-to-day existence. So you scrawl something on a cave wall. Obviously it's become a lot more complicated now because things like telly and photographs can do that kind of revelation. Now it's more people trying to communicate states of mind and moods. But I still think that the initial thing that sparks a work of art off should be a real event."
This seems a rather prescriptive view from someone whose own image is anti-establishment and iconoclastic. Remember the moment at the Brit Awards, three years ago, when Cocker leapt to his feet, mooned at Michael Jackson and became an instant national treasure? That unpremeditated act secured his reputation as not just a working-class hero from Sheffield but a scourge of pretension in a most stellar form. Witty, gifted, impulsive, Cocker seemed like a breath of fresh air in the pop industry - which is why I am unprepared for the dour, withdrawn figure who arrives for this interview. From the moment he climbs the stairs to the office where I am waiting with his publicist, he is uncommunicative, even a bit aggressive. He thinks we have arrived early, demanding to know what time we were supposed to meet. "Two-thirty," says the publicist. "What time is it now?"
Cocker scowls. "Two twenty-eight."
On the way to the restaurant, Cocker strides ahead, talking to the publicist as I struggle to match his pace. In the restaurant he sits down, removes his long knitted scarf and woolly hat, finds a space for the battered plastic suitcase and carrier bags he has unaccountably brought with him, and waits for the interview to commence. Why is he so edgy? I have already assured the publicist that I am not going to ask anything about his private life - I am not interested in who he sleeps with - and I have no idea why the atmosphere is so frosty.
I am wondering about his aesthetic sense, whether there is anything beyond an indifference to external influence and fashionable trends which draws him to the eccentric, obsessive artists in his films. He is resistant to analysis, protesting that he is not academic and insisting that his function in the series is little more than to be "a walking ruler", there to provide an idea of the scale of the projects.
He is also savvy enough to appreciate that the programmes run the risk of turning outsiders into insiders, that the stubborn personal vision he admires could easily become the next big thing - just as Pulp did in the summer of 1995, after years in the wilderness. "We won't be publishing the addresses [of the artists] on the Internet," he protests. "If people try hard enough they will be able to find them out. It wouldn't be appropriate to have Thomas Cook guided tours." One of the houses he filmed has already been used for fashion shoots, he reveals regretfully, adding: "Everything gets appropriated." For all his self-deprecation, Cocker is not unacquainted with theory, with the abstract aspects of art. It may also be, however, as I felt several times during the encounter, that he is talking obliquely about himself.
Occasionally something sparks his interest and he looks directly at me, his face transformed, but it does not happen often. When he says he is trying to watch less television and I tell him I have given my set away, he seems genuinely interested, wanting to compare our experiences of watching it as children. For a moment he relaxes and we are almost having a conversation, then the shutter comes down again; most of the time he seems ill at ease, as though the interview is a prison sentence he has agreed to serve and he can barely wait for his release. He seems to be not so much bored as in a state of endurance. What comes across, in an uncomfortable 75 minutes, is a kind of muted misanthropy whose origins seem hidden and personal.
"You can't force people to express themselves," he observes of the Frenchman who kept him standing in the rain. Then, going off at a tangent, he adds: "Everybody should be free to express themselves. Artists don't feel different emotions to everybody else. They might be able to express them. Their life is lived at the same emotional pitch. Probably there is a higher emotional pitch on a council estate than in a salon because in a way it's all you've got." Although Cocker's manner of speaking tends to be flat, his hands are jumpy and rarely still, a circumstance only partly explained by his revelation that he has recently given up smoking.
I ask him whether his decision to make films about Outsider Art could be interpreted as a rejection of the extraordinary fame which overtook him three and a half years ago. He agrees that becoming a celebrity, after a long period of obscurity, made life uncomfortable. "The thing about becoming successful is you kind of think to yourself: did you do it for money? So you could have this car? Especially when you come across so many idiots socially. People who like to talk about themselves and how brilliant they are. Is this why I did it?" He is eager to distance himself from other well-known travellers with their own TV series; he is not, he points out engagingly, "some Tory prick on a train". But Cocker's discovery about celebrity - "beyond a certain level it gets quite painful" - must have come as shock for someone who has been so ambitious since childhood.
"I was obsessed with it. I always wanted to be in a band. When I tried to do it, I didn't become famous, not for a long time. So the thing that kept me going must have been rather different." Fame, disillusionment, the identification with outsiders suggested by the films: this is the most interesting area we have got into and I ask another question about what drives him. As though he has said more than he intended, Cocker abruptly closes up. "I don't want to talk about it. No matter how you talk about it, it sounds like you're moaning. So I'd rather not talk about it."
I think he is wrong, that it is possible to discuss the motivation of an artist, in the broadest sense of the word, without being accused of whingeing or ingratitude. Cocker has already said that what he admires about the subjects of his films is their lack of self-consciousness, and I am struck by the opposite quality in him - by a kind of pained solipsism which seems to have erected, at least on the afternoon we meet, an impenetrable barrier between himself and other people. With his awkward gestures and nerdy clothes, Cocker seems not so much a rebel, cheerily showing two fingers to the world, as someone who is as miserably self-conscious as an adolescent. I have to keep reminding myself that I am not dealing with a teenager with hormonal problems but a man of 36.
I open my mouth to ask another question but I am so dismayed by Cocker's apparently deliberate charmlessness that nothing emerges. Mistaking my intention, he leans forward and says, "Go on, spit it out," obviously thinking I have an awkward question up my sleeve. The truth is that I want nothing more than to go home. "When is this article going to appear?" Cocker demands as I pay the bill a moment later. I tell him and he remarks, "I'll make a note not to get the paper." A piece of calculated rudeness? A cack-handed attempt at self-deprecation? I have no idea.
Maybe he hates interviews, although he says the ones connected with the series haven't been too bad. Maybe he is having a terrible day for reasons I don't know about. Or maybe he has not worked out a way to resolve the contradiction Marianne Faithfull pointed out in her heavily ironic cover version of John Lennon's "Working Class Hero" - that fame alienates people from the class they have never quite belonged to but are taken to represent.
Outside in the street Cocker spots a taxi and runs gawkily into the road - shouting, arms flailing, bags flying in all directions. He clambers in, looking back in astonishment when I call out "Goodbye." Whatever is on his mind, he has managed to leave the impression that he is Jarvis Cocker, cultural icon, and I'm just one of the Common People.
`Journeys into the Outside with Jarvis Cocker' begins on Channel 4 on 16 February