The Unbelievables: truth, lies and the myth of Eliot Ness' legendary battles with Al Capone

A new book sheds startling light on the Prohibition-era lawman

Kevin Costner starred as Eliot Ness in 'The Untouchables' (The Independent)

In 1957, a former federal law-enforcement agent published The Untouchables, which became one of the century's most famous crime stories. The rat-a-tat tommy-gun tale of its author, Eliot Ness, and his incorruptible men in the Prohibition Bureau fighting Chicago gangster Al Capone proved to be a hard-boiled gem.

It sold more than a million copies. It spawned a famous television series and a Hollywood blockbuster. It lofted Ness into the American pantheon of crime-fighting legends. He was the real-life Gary Cooper in High Noon, the Depression's Wyatt Earp – the square-jawed Hero Who Came to Save Us from the Bad Man in the Dark.

Of course, it's almost all fiction. Ness didn't have much to do with Scarface's 1931 tax-evasion prosecution. Whole gobs of the book were ginned up by a ghostwriter. Ness and his agents, hardly saints, boozed it up as much as anybody. Ken Burns, who co-directed a PBS series called Prohibition in 2011, dismissed Ness as a "PR invention".

So when two US senators recently announced plans to name the national headquarters of the Bureau of Alcohol, Tobacco, Firearms and Explosives in Washington DC as the Eliot Ness ATF Building, catcalls spewed like moonshine from a busted still. "Ridiculous," says Daniel Okrent, the author of Last Call: The Rise and Fall of Prohibition, one of the definitive volumes about the era. "Naming a building after him for his role in getting Capone? You might as well name it after Batman."

Ditto, says Jonathan Eig, a former Wall Street Journal reporter and author of Get Capone: The Secret Plot That Captured America's Most Wanted Gangster. "The movies are allowed to have their fun," he says. "But we shouldn't go naming buildings after them."

It's so bad that back in Chicago, Edward M Burke, an alderman for 45 years and author of three history books about the city, co-sponsored a resolution that – read this twice – protested against naming a federal building in Washington after one of his city's most famous sons.

Isn't this a good story? Wait, it gets better. Here's a punchy little bit about your big-shot Government Man. This one is all true: an assistant county prosecutor and a handful of gun-toting backups pull up at the Harvard Club, a high-end prostitution and gambling house. Bang, bang, bang, the prosecutor goes on the front door. We got a warrant; open up. A goon knocks the government lawyer to his knees. The Mafia boss comes to the door to check out the rumpus, then tells his boys that any cop who comes inside gets his "head knocked off".

American FBI agent Eliot Ness headed the investigation of Chicago mobster Al Capone in the 1930s (Getty Images)

The assistant prosecutor calls his boss, who comes out and also bangs on the door in a now-see-here huff – and gets the same high-hat treatment. Shaking in their wingtips, they call Ness. The dapper young cop grabs some off-duty officers and, wearing a fedora and a "long, camel's hair topcoat", rushes to the stand-off. "Let's have a fight here," he snaps. Bang, bang, bang, he goes on the door. The gangsters, awed by his appearance, let him in. Ness re-emerges moments later, swings open the doors and the raid begins.

Isn't that grand? You bet your sweet bippy... but it happened in Cleveland, not Chicago. The Mafia thug was James "Shimmy" Patton, not Al "Scarface" Capone. It was January of 1936 and Ness was Cleveland's director of public safety. It wasn't January of 1931, when he was a twentysomething Prohibition Bureau agent just appointed to dog the fearsome Capone outfit.

This seldom-heard anecdote, recounted in the new Eliot Ness: The Rise and Fall of an American Hero, is typical of the disconnect between fans and critics, says its author, Douglas Perry. Go through the actual records, Perry says, and you'll find an honest cop in an era when there weren't many; a fearless agent who would stare down gangsters; and an inventive law-enforcement executive who was years ahead of his time. He says his book, which comes out this week, presents a lot of Ness's post-Capone career for the first time. "When you look at the criticism," he says, "it's from people who don't know much about him."

You want the real Ness and Capone?

Ness was the youngest of five, born in 1902 of Norwegian immigrants on the South Side of Chicago. What you saw: steel factories burning orange through the night, meat-packing plants, the "hog butcher to the world" in the phrase of poet Carl Sandburg. A city that was exploding from 30,000 people in 1850 to nearly three million by the time that Ness was a teenager. Poles, Italians, Irish, Germans, Greeks, Russians, blacks from the South, a saloon-filled expanse of languages and accents, slurs and insults. A place where smoke "hung like drapery", writes Perry; a part of the city where the smell of butchering from the 500-acre Union Stock Yards "buckled the legs", writes Eig. Ness graduated from the University of Chicago and followed his brother-in-law into the Prohibition Bureau. Six-foot tall, quiet, honourable in his work but not in his marriages. He helped put together a rotating group of agents who spent the first six months of 1931 raiding Capone's hidden breweries. His car was stolen three times, his office phone line was tapped, he was offered bribes, and a friend/informant was shot four times in the face.

Still, he and his crew – which ranged from six to a dozen or so – put together a 5,000-count bootlegging indictment against Capone. They were dubbed "the untouchables" in a 1931 Chicago newspaper story because they refused bribes. US Attorney George E Q Johnson in Chicago loved Ness's work but opted to prosecute Capone on tax-evasion charges, as the jurors (like almost everybody else) loved to drink but loathed tax cheats. Ness gave reporters who covered his raids some of the impounded booze (cough, cough), which helped him grab headlines. The man himself was fond of Cutty Sark. He was given to spells of depression.

Capone was one of nine children, born in Brooklyn to Italian immigrants. Dropped out of school in the sixth grade. Burly, muscular, powerful, he worked as a bouncer, face slashed by a knife. Moved to Chicago to work for mobster Johnny Torrio. Worked at a sleazy club called the Four Deuces, in the sleazier Levee District. Smart, resourceful, brutal, about 5ft 10in, corpulent, married, moved in on the liquor trade after Prohibition started. Flamboyant, volatile, made millions, wore pastel-coloured suits and pearl-grey hats, loved reporters (cough, cough), who helped him grab headlines. Said things like: "When I sell liquor, it's called bootlegging; when my patrons serve it on Lake Shore Drive, it's called hospitality."

Capone beat 18 of 23 counts of tax evasion in a sensational 1931 trial, but a judge sentenced him to 11 years, far above the norm. He certainly knew who Ness was, but by all accounts the pair never saw one another in person until his court appearance.

A still from 'The Untouchables' starring Charles Martin Smith, Kevin Costner, Sean Connery and Andy Garcia (Rex Features)

Ness wasn't even 30 when Capone (who was just 32) was convicted. He was a boy wonder when he moved to Cleveland, then the nation's sixth-largest city, and quickly became the director of public safety, overseeing more than 2,000 cops and firefighters. He took down gangsters (see above). He forced the mayor to let him hire secret investigators, masked as city employees, to investigate cops on the take. Today, every law-enforcement agency has an "internal affairs" division to do the same thing.

Led by his mentor August Vollmer, he viewed drug and alcohol addictions as primarily medical problems, which today is commonplace. He emphasised ballistics tests, even soil samples, in building cases, the dawn of forensic sciences. He had squad cars painted an unusual tri-colour combination so that people would recognise the cops. He put two-way radios in those cars so that officers could communicate.

Some of these were original. All of them were cutting edge. "He was making innovations that have stayed with us ever since," says Perry.

Capone, meanwhile, served a good chunk of his sentence in Alcatraz, where his mind slowly deteriorated due to syphilis, contracted years earlier. Released, he moved to his palatial home in Palm Island, Florida, where his mind and body continued to rot. He died, a shell of himself, on 25 January, 1947, at age 48.

Ness did not last all that much longer. He ran for mayor of Cleveland and got trounced. During the Second World War, he worked for the government, cracking down on the vice trade near military bases.

Afterwards, he tried business and failed miserably, winding up in tiny Coudersport, Pennsylvania. He was depressed, deeply in debt, drank like a fish and cheated on his wife.

In 1955, on a trip to New York, he and a business partner were drinking at a hotel bar when his partner spotted a friend – Oscar Fraley, a wire-service reporter. Fraley joined them and, as the night grew on, talk turned to Ness's long-ago work against Capone. Fraley wanted to do a book. Ness eventually sent him about 20 pages of notes but wasn't enthused, Perry recounts, and reluctantly continued the collaboration until his heart quit on him. He died, a shell of himself, on 16 May 1957, at age 55. The New York Times did not note his death, Perry writes, and back home in Chicago, his obituary was less than 100 words.

The Untouchables hit bookstores later that year. It's swell, in a Mickey Spillane, Damon Runyan kind of way. Ness ducks a hitman's bullets, dives from a car that tries to run him down, punches out thugs, trades quips with saucy prostitutes and dries up Capone's massive operation. "I want Ness, and I want him dead," Capone is quoted as saying. It was pretty much all nonsense, but people loved it.

The book sold and sold and sold. The television series, starring tough-guy Robert Stack as Ness, debuted in 1959 and ran for four years. Kevin Costner, at the height of his matinee-idol appeal, played Ness on the big screen a quarter of a century later, where it was a $76m (£46m) hit. Since the man himself was dead before the book came out, "who knows if even Ness believed what was in it", says Okrent, the Last Call author. He's not being a stick in the mud – as a kid, he says, he loved watching The Untouchables. "That show was it," he says, laughing.

Considering all this over lunch at a restaurant in downtown Washington DC on a recent day is a man named Scott Sroka. An assistant US attorney in the district, he is the grandson of Joe Leeson, one of the most respected of the "untouchables". He's done a lot of research into his grandfather's history and defends the group's Capone work. "The historical record backs up Ness's reputation," he says. Perhaps this is so. But stories require dualities, and the standard turning point is when the hero meets the nemesis. In this case, though, maybe it was the power of Capone's legend that created Ness's, in order to turn the Prohibition-gangster era into a morality play, a popcorn narrative of Good vs Evil. This is the territory of Luke Skywalker and Darth Vader, Superman and Lex Luthor, Snow White and the Evil Queen.

But who could possibly be the equal and opposing nemesis of Al "Scarface" Capone, the brutal, charming villain who lorded over Chicago, perhaps the most notorious gangster in the nation's history? How about your upright G-man, gun drawn, his hard-soled footsteps clicking on the wet pavement, going down the alley alone in the dark, to protect us all? That is the myth of Eliot Ness, and so what? Hasn't he served his nation well, in the realms of fact and fiction? What more can you ask of an honest cop?

This is an edited version of an article that first appeared on washingtonpost.com

Fact or fiction? Historical hindsight

Hollywood's approach to history is summed up perfectly by the line in John Ford's The Man Who Shot Liberty Valance (1962): "When the legend becomes fact, print the legend. When you look at the criticism, it's from people who don't know much about him."

Just as Eliot Ness didn't really bring down Al Capone, James Stewart's white- collar, gun-shy, East Coast type wasn't really the one who slew the mighty outlaw Liberty Valance (Lee Marvin) – but it made a better story that he was.

Westerns have been especially culpable of mythologising events that, when (and if) they actually happened, seemed relatively inconsequential.

In Wyoming in 1882, small settlers were pitted in battle against cattle owners and the killers they hired. Writer Steven Bach referred to the skirmishes as "a little-known incident in American history called the Johnson County War by those historians who note it at all". From such little beginnings, many mighty Westerns have sprung. From George Stevens' Shane to Michael Cimino's Heaven's Gate, film after film has dramatised aspects of an episode not considered to have been of overwhelming significance at the time.

When it comes to Mafia movies, the first point of reference, even before Mario Puzo's novel The Godfather, is Joe Valachi.

Before his testimony to the US Senate in 1963, few knew that the Mafia even existed. The small-time mobster helped coax an entire new cinema genre into existence.

Cinema has a knack of reinforcing myths that were never true in the first place. Contrary to what we have learnt from films, the Titanic was not described by its owners and builders as being utterly unsinkable.

Then there are the omissions. As the Daily Mail was very quick to point out when George Clooney's historical drama The Monuments Men was released recently, the film failed even to mention Ronald Balfour, a British academic who died during the Second World War while trying to save art from the Nazis.

Feature films rarely last more than two hours. If there is a lot to cram in, another familiar trick is to create composite characters. Veteran actor Alan Arkin admitted that the film producer he played in Ben Affleck's Oscar-winning Argo was actually based on "two or three different people".