Stars in his eyes: Inside the unseen celebrity archive of photographer Richard Young

During the 1970s and 1980s, everyone who was anyone found themselves in front of Richard Young's lens. The photographer takes James Cusick through his trove of recently rediscovered pictures

Sitting in front of an old-fashioned light box, once a familiar object on a photographer's desk, Susan Young is systematically examining strip after strip of 35mm photo negatives which slip in and out of tidy envelopes. She shouts to her young assistant, working in another room downstairs: "73899. What year is this from? It's an absolutely unbelievable picture!"

The never-seen-never-used black-and-white image from a Los Angeles roller disco is being dated with the care an archaeologist lavishes on an excavated artefact. An answer comes back. "1977, I think." The picture shows the Academy Award-winning actor Jon Voight, looking uneasy on his skates; one hand on the wall, the other around a woman who looks very much like his then wife, the American actress and director Marcheline Bertrand, dressed in skin-tight satin trousers. Their young daughter, Angelina Jolie Voight, would have been two years old at the time.

Photograph 73899 is just one job from 86 boxes of negatives, each containing 1,500 images that are being converted into a high-resolution digital archive of the work of Susan's husband, the photographer Richard Young.

"I think I was in LA to do a Bee Gees concert and I met up with a friend going to a roller disco, the rage at the time," says Young. "And Voight. He was just there." The description of how he came to have a camera in the right place at the right time is offered as a regular, repeatable event that Young has somehow learnt to live with. "This is the magic of me. I create a magic."

The small gallery just off Kensington Church Street that the Youngs run is a house of names: some forgotten, some too big to forget, others resonating in ways few could have forecast when they passed through Young's lens. Their celebrity Tabularium contains, among many, the images of John Paul Getty III, Lee Marvin, Sophia Loren, Richard Burton and Elizabeth Taylor, Mick and Bianca Jagger, Peter Cook, Lionel Bart, Liza Minnelli, Tina Chow, a strange picture of Raquel Welch, Bob Hope and the Chinese ambassador together, Elton John (in an early wig) and Kenny Everett, Princess Caroline of Monaco, Lord Forte and Richard Nixon talking to each other, Adam Ant and Princess Margaret, and of course, Terence Stamp and Jack Nicholson, because there is always Terry and Jack, somewhere.

There's a picture of a young Rupert Everett, evidently in his glam-rock phase. So how did he… Young interrupts before I finish the question. "I had no idea who he was. I was photographing the Alternative Miss World contest. He just looked interesting."

Surprises are being thrown up as the Youngs trawl through their chronicle of the years. "Look at her in the centre, the one with the jaunty hat. Isn't that Camilla?" It is her. A shot from a hunting ball, organised by the interior designer Nicky Haslam at Cecil Beaton's country house, has the Duchess of Cornwall sweetly captured at a time, when, if given the choice, Prince Charles would have married her rather than Diana Spencer, and perhaps significantly dented the popularity Lady Di conferred on Britain's struggling monarchy. Then, in a couple of brief sentences, Young condenses the royal fickleness of our monarchy. "I had no idea who she was. She wasn't anyone then."

You can define Young as either a paparazzo or – perhaps more accurately – a highly talented documentary snapper. But whatever label is used, it should reflect Young's importance as a photographer in mapping the social DNA of Britain and America's emerging celebrity culture over the past 40 years.

Later this month, recognition of this contribution will come in the form of an honorary doctorate from London's University of the Arts. "The enduring variety of Richard's career has made him an inspiration to our students," said Stephen Marshall of UAL.

For a north London boy who left school at 15 to work in a boutique on the swinging King's Road, accidentally ending up a photographer via work in New York, Paris and a bookshop in Regent's Street, becoming "Dr Richard Young" is something he never expected. Smiling through his trademark beard, which is more salt'n'pepper than it was in the days when he could often be found somewhere near Langan's Brasserie dressed in a Schott leather jacket and smart cashmere sweater, he jokes: "Let's just say I won't be doing house calls or writing prescriptions".

The degree ceremony at UAL will come almost a year after the formal hearings at the Leveson Inquiry ended. Young says he watched "some of it", as the trade of the paparazzo was trashed after a queue of celebrities, Sienna Miller and Hugh Grant among them, told their tales of relentless harassment. Watching it, Young says, he felt the paparazzi "had betrayed themselves and their job. There was a lot of disrespect going on. I've always known there was another way of going about it".

The archive project is getting close to gathering all of Young's black-and-white phase, which began when he started out as a freelance in 1974, working mainly for The Evening Standard and the Daily Express, and ended around 1987 when he started to use colour film.

It's the unpublished black-and-white images of casual, at-ease celebrities, captured after a picture editor at the Express told him to "stick with it" at a time when he seemed to have London's stars to himself, which suggests that, if Hans Holbein had had access to a Nikon SLR instead of oil paints, this is the way Erasmus, Thomas More and Henry VIII's court would have been beautifully papped.

"Back then there were no publicists on your shoulder, no security. It was often just me and the doorman." Eventually relationships formed, and Young claims the respect he gave the stars meant they returned the favour.

Though he gatecrashed the bash Liz Taylor threw for Richard Burton's 50th at the Dorchester, quietly snapping the iconic birthday kiss between the world's most famous couple, that was not his signature modus operandi. In among a strand of pictures that includes Ronnie and Jo Wood, Freddie Mercury, Tatum O'Neal and Paul Raymond, we find a shot of the Bond actress and wife of Peter Sellers, Britt Ekland, snogging a young Sharon Osbourne. "Sharon's dad, the rock promoter Don Arden, threatened to decapitate me. He didn't want his daughter being seen kissing girls." The picture never made it into the pages of any paper. "If they said no, that's the way it was. That's why they trusted me."

Beyond trust and respect, Young had other tricks. While the paparazzi were outside waiting, he was often to be found inside. Langan's Brasserie, just off Piccadilly, opened in 1976 by the Irish entrepreneur Peter Langan, used to attract the cream of London's resident and visiting celebs, as did Regine's, now the Roof Gardens in Kensington. "I decided I needed to be a customer in Langan's, to eat there. I had the best window table. And when Roger Moore or Al Pacino left, I didn't pap them, I asked them." Occasionally, it didn't work out. When Young joined the staff of Ritz, the gossip magazine started by fashion photographer David Bailey and David Litchfield in 1976, their 'editorial' meetings took place in Langan's. "I was in there with the Ritz people when I saw Anna Ford and Mark Boxer, then editor of the Tatler, having a quiet dinner. It was at the start of their affair. I was told 'Go get a picture of Ford and Boxer'. I did, and all hell broke loose. Peter [Langan] started shouting at me. Chaos. But on the way out he whispered 'See you tomorrow'. That's the way he worked. It was all theatre." Langan at the Leveson Inquiry would have been quite a sight.

When Princess Grace of Monaco and her young son Albert were at the Piccadilly restaurant, Young was at his usual table. As they left, he and his camera bag left, too. The paparazzi? They'd been outside in the cold for two hours. Did his good luck ever need a boost? Did money ever change hands? "Look, I never, ever paid anyone. I'm a tight-fisted Jewish boy from Hackney."

Good timing is also part of Young's success. We turn to two fantastic unpublished pictures, one of the US singer-songwriter Roy Orbison, the other of Hollywood superstar Lauren Bacall. The Big O is decked out in a faux-Elvis outfit, making a call in a telephone box backstage at Wembley, back when it was called the Empire Pool. "They used to have an international country and western festival there in the Seventies. Mervyn Conn, the promoter, invited me backstage and Orbison suddenly appeared and went to the phone box. Remember, no mobiles back then," Young laughs. "Maybe he was reversing the charges to Nashville?"

The shot of the elegant Bacall was taken in the early morning inside the Majestic Hotel in Cannes – and maybe timing had nothing to do with it. "She just invited me to breakfast. That was all."

Looking back, Richard Young, now 66, can't explain why, having initially wanted nothing to do with newspapers, he's ended up spending 40 years in this business. "It just happened, and all I know is that it could never happen again." Then he sums it up for me: "What's the difference between then and now? Then it was easy. Now everyone has a camera, everyone takes pictures, everyone."

See more of Richard Young's photographs at independent.co.uk/richardyoung