A film's popularity depends on how many references are made to it in future movies, according to new study

An analysis of nearly 15,500 films found that it was possible to predict which ones would be considered among the greatest ever made by looking at how often its themes, images or overall message are picked up by other directors

Casablanca was referenced as one of the films that produced some of the most long-lived influences (AP)

Forget the critics, the box office receipts and the Oscars, what really makes a film into a masterpiece is how many passing references it generates in subsequent films made by other directors, scientists have discovered.

An analysis of nearly 15,500 films found that it was possible to predict which ones would be considered among the greatest ever made by looking at how often its themes, images or overall message are picked up by other directors many years later.

The researchers said that the “automated method” of working out the cultural, artistic or historical significance of films could even be applied to other spheres of artistic and scientific creativity, from literature to physics.

“The reason we say it’s automated is because as long as you’ve got the information in digital form you don’t need to answer questions of anyone you just load the computer code to get the answer,” said Professor Luis Amaral of Northwestern University in Evanston, Illinois.

Typical methods of assessing a film’s “impact”, such as sales, downloads or media attention, are not as good at predicting a film’s long-term success as its influence on other directors, who appear to be useful arbiters of long-term cinema quality, the study found.

“Directors keep coming back to movies that are significant. If you show a little bit of ‘Psycho’, such as referencing the shower scene, you are putting that whole movie in front of the viewer of the new movie,” Professor Amaral said.

The Wizard of Oz was one of the films that produced some of the most long-lived influences (Rex Features)

“There is something about a movie that is hidden to us, but there are measurable things, such as critic ratings, awards and referencing by other filmmakers, that hint at this hidden element – a movie’s significance,” he said.

“We find that ultimately it is the creators, the filmmakers themselves, who will determine which movies are important, not the expert critics…movie critics can be overconfident in spotting important works, and they have bias. Our method is as objective as it gets,” he added.

The scientists suggested that the longer the time lag between a film’s release and its subsequent reference in another film, the more significant it will be as a measure of long-term success.

They only included references made 25 years after a film’s release to ensure long-term influences. The films that produced some of the most long-lived influences are The Wizard of Oz, Star Wars, Psycho, Casablanca and Gone with the Wind.

The study analysed 42,794 citations in the film database that connect one film to another, such as the dialogue about not wanting to be with a man who runs a bar in Casablanca in the 1989 film When Harry Met Sally, which was a reference to the 1942 film Casablanca, starring Humphrey Bogart and Ingrid Bergman.

Meg Ryan in When Harry Met Sally (REX)

The study, published in the Proceedings of the National Academy of Sciences, analysed the 15,425 American-produced films listed in the Internet Movie Database to see whether there were any useful markers for predicting whether a film makes it onto the prestigious US National Film Registry of the US Library of Congress.

The registry is like a cinema hall of fame and currently includes just 625 films “of enduring importance to American culture” as judged by experts on the US National Film Preservation Board. Their judgment is based on whether the films are “culturally, historically or aesthetically significant”.

The scientists, who usually work in the spheres of mathematics, engineering and physics, looked at a range of factors, such as box office receipts and critics’ reviews, to see if there was a significant correlation with a film’s inclusion in the National Film Registry and other factors relating to a film’s apparent success.

They found box office receipts was not a particularly good predictor, citing for instance the commercial failure of some otherwise classic films, such as Duck Soup, the 1933 film considered to be the greatest made by the Marx Brothers.

Other factors, such as critical reviews and Oscar successes, were equally inadequate when it came to predicting long-term success, the study found. Oscars, for instance, were often a reflection of a film’s immediate popularity and the amount of time and money spent on studio promotion, the scientists said.

Film references

* ET (1982) has visual reference to The Thief of Baghdad (1924) in the silhouette of a figure flying in front of the Moon

* Blazing Sadles (1974) includes a spoof homage to the film classic Stagecoach (1939) in the final scene of wagons under siege drawing up in circle

* When Harry Met Sally (1989) referred to the film Casablanca (1942) when Sally says: “I don't want to spend the rest of my life in Casablanca married to a man who runs a bar.”

* The Coen Brothers film Raising Arizona (1987) includes a homage to The Evil Dead (1981) with the main character played by Nicolas Cage driving the same car, a 1973 Oldsmobile Delta 88