There is a moment, in the course of my meeting with Budd Schulberg, when the conversation turns to vintage Hollywood films that never achieved the recognition they deserved. I mention Nothing Sacred, the 1937 movie starring Carole Lombard, about a newspaper reporter who is duped into believing that an African-American shoe-cleaner is a wealthy tribal chief given to donating large sums to news organisations. He introduces him to his editor, who throws a highly publicised banquet in the man's honour, during which the chief's true origins emerge. Relegated to the obituaries desk, the journalist seeks to revive his career by filing crudely emotive reports about a country girl who has told him – untruthfully – that she is dying of radium poisoning. The film is dark, beautifully scripted, and has a memorably inventive finale.
"I like that film too," Schulberg observes. "It may have suffered by being ahead of its time."
"Preston Sturges directed that, didn't he?"
"No," the screenwriter says. "William Wellman."
"Are you sure?"
"Pretty sure," he tells me. "I wrote the ending."
Talking to Budd Schulberg, the same questions recur. Is there anybody he hasn't worked with; any significant figure he hasn't met? Seventy years ago, Vanity Fair magazine was running a series called "Impossible Interviews": imaginary conversations between individuals distanced by culture, geography or time. Hollywood star George Arliss was interviewed by Cardinal Richelieu; Mae West met Sigmund Freud, and Rockefeller went up against Stalin.
Listening to Schulberg recall binge-drinking with F Scott Fitzgerald, sparring with Hemingway, or describing how he restrained Sirhan Sirhan, seconds after he shot Robert Kennedy, you feel part of some similarly bizarre collision. You might have made the journey to his Westhampton bungalow not on the Long Island Railroad, travelling two hours east out of Manhattan, but by time machine. Rudolph Valentino attended Budd's fifth birthday party. Silent movie star Clara Bow, the original "It" girl, flirted with him. He once obliged the Marx Brothers to retake a scene after he burst out laughing on the set.
A well-preserved and alert 94, Schulberg sits by a log fire while his fourth wife Betsy Langman, a former actor and magazine journalist who protects his interests with formidable devotion, is discussing percentages on the phone. Budd seems more interested in looking out for the swans that visit Aspatuck Creek, the stretch of water just beyond his window. Some of them will take corn from his hand. If you didn't know, you would never guess that this gentlest of ornithologists, who speaks with a slight stammer, would be a legend even if all he'd ever produced was his boxing journalism.
"Did you ever fight, yourself?"
"I tried to box," he says. "But I had two major flaws: I never liked being hit on the nose. And I never developed a strategy to avoid being hit on the nose."
He's been close to Muhammad Ali for almost half a century, though this is not the kind of detail he would volunteer unless prompted. As a young man, Schulberg tells me, he was once in a limousine, travelling to a Hollywood premiere.
"These girls crowded round the car. They kept screaming: 'Who are you? Who are you?' I said: 'I am nobody.' As they backed off, I heard one of them say: 'Hear that? He's a nobody. Just like us.' "
He took that remark as a compliment. Fame, he says, has never been an ambition for him. That said, if you are one of the many people unfamiliar with his name, then Budd Schulberg is the greatest living American writer you've never heard of.
His novels, notably the 1941 classic What Makes Sammy Run? – a vicious indictment of the ruthlessness of studio bosses – and The Disenchanted (published in 1950, and largely inspired by his friendship with the disintegrating F Scott Fitzgerald) bear comparison with anything in modern fiction. Of his screenplays, On the Waterfront – the 1954 film, starring Marlon Brando as Terry Malloy, a longshoreman and retired boxer who testifies against the Mob's activities on the New York docks – remains Schulberg's proudest achievement. Brando's most famous speech, recalling the way he was forced to throw a fight, is delivered to his brother Charlie (Rod Steiger) in the back of a cab. The words have lodged in America's collective memory in a way that normally only occurs with a popular song lyric – by Cole Porter, say, or Bob Dylan. Brando's lines, Schulberg complains, are still recited to him by strangers, often with the moribund delivery affected by Robert De Niro, when he repeats them in Raging Bull: "I could have had class. I could have been a contender. I could have been somebody."
On the Waterfront is currently playing at the Haymarket Theatre in London, adapted and directed by Steven Berkoff, who plays mob leader Johnny Friendly.
"I can't imagine two more different characters than you and Berkoff. Did you know his work?"
"Not really. He called me out of the blue. I knew his name. I said: 'OK. We'll see what we can work out.' And he's been very good to work with: direct, and simple."
How is it that Budd Schulberg's reputation has endured less well than other, inferior, writers? It probably has something to do with his character, habitually focused on helping others, rather than on self-promotion. (In 1965, after the Watts riots, he travelled to that intimidating district of Los Angeles alone, and set up a writers' workshop.) Then there's the uncomfortable fact that, when summoned by Senator Joseph McCarthy's House Un-American Affairs Committee in 1951, Schulberg – a disillusioned former Communist – named names. Like Diego Maradona or Bill Clinton (this is a list to which his most spiteful and unforgiving enemies might add the name of Judas Iscariot), Budd Schulberg's career is an example of how a lifetime's achievement can be obliterated by one controversial act.
He was born, as he says, into Hollywood royalty. Budd's father, BP Schulberg, was a movie pioneer, in partnership with Louis B Mayer, before he founded Paramount Pictures. Unusually for a cinema executive, BP was a highly gifted writer. A gambler and alcoholic, he died in obscurity in 1957.
"As a boy, I'd wake up to find my father still playing cards. He'd be $20,000 down, to expert players like Zeppo Marx. That would be like $3m now."
"Growing up as you did, it's remarkable that you managed to write so convincingly not just about working-class life, but a particularly fierce and impenetrable strand of it, in On the Waterfront. Had you spent some time researching that society?"
"More than a year. Corruption on the waterfront became a cause for me. There was a priest on the West Side, Father Jack Corridan, who opposed the influence of organised crime. He said: 'Be our messenger.'"
"Threatening phone calls."
"How did you infiltrate that world?"
"I'd go from bar to bar with Brownie, a longshoreman who was my guide. He told me to listen but never speak. I had to drink like the workers. That meant boilermakers: beer and whiskey chasers. One night I had one too many."
Turning to a stranger, Schulberg recalls, "I asked: 'And what do you do?' Brownie pulled me off my stool and shouted: 'Run!' And we ran, for four blocks. When we stopped, he said: 'I believe I need a more discreet investigator.' "
"As conversation goes, 'What do you do?' doesn't sound that provocative." "No. But this guy had killed a dock boss a week before. He'd murdered many people."
"Did you meet a lot of fellow Jews in those bars?"
"The waterfront wasn't what you'd call mixed. It was almost 100 per cent second-generation Irish. No Italians, even. But I fitted in really well. I have a knack for that."
"Maybe the stammer helped."
"I'm certain it did. It makes you appear vulnerable. It disarms people."
"Sam Spiegel produced On the Waterfront. But he wasn't the first person you approached, was he?"
"Elia Kazan, the director, and I, had tried everybody else in Hollywood. Darryl Zanuck told me: 'There's nothing about this that I like. All you have is a load of sweaty longshoremen on a picket line.' "
"How did you feel when he said that?"
"It stabbed me in the heart."
The beauty of Budd Schulberg's most famous lines typifies his gift to identify poetry in everyday conversation. He says he first heard the phrase "I could have been a contender" used by his friend, the boxer Roger Donoghue.
"Marlon Brando wanted that scene out, didn't he?"
"He did. Brando believed it was unplayable. The first day's shooting, I was on the roof of the tenement in Hoboken, New Jersey, with Kazan and Brando. I said: 'Marlon, everybody loves that scene except you. Why?' He said: 'Steiger has a gun. If someone is pointing a gun at you, you're not going to make a long speech like that.' Kazan said, 'Why don't you just push his gun aside?' That was the end of the argument. Apart from that, Marlon was very amenable."
"You'd approached Frank Sinatra for the part?" "We had, when it looked as though we weren't going to get Brando."
"How did Sinatra handle rejection?" "He was mad as hell. God, he wanted that part. He screamed at me. He practically came to blows with Spiegel. He had his heart set on it."
"The unfortunate truth is that Sinatra couldn't have done it. He just couldn't act in that way ..."
"Could he act like Brando?" Schulberg allows himself a smile that reveals just a hint of pride, before continuing: "No. But who could?"
Seymour Wilson Schulberg was born in New York City in March 1914. BP and his wife Adeline, known as "Ad", moved to Hollywood when Budd was five and sister Sonya was three. (Their brother Stuart, who would collaborate with Budd as a producer, was born in 1922; he died in 1979.)
"Your 1982 memoir, Moving Pictures: Memoirs of a Hollywood Prince, covers your first 18 years. For all the glamour generated by your father's success, there's a terrible sadness in that book, when you describe struggling, as a boy, to protect your dad from his failings – which, as I recall, you describe as gambling and 'extra girls'."
"And drink. He used to say, 'Yes father,' when I tried to persuade him to stop. He had great intellect. But eventually his bad habits outdid his good habits. He was 65 when he died. I was supporting him. Even his jewellery had gone. So many great writers," Schulberg remarks, "have that self-destructive impulse."
"But not you?"
"No. Maybe I was forewarned by my father's experience."
"You come across, throughout your life, as somebody committed to opposing injustice."
"I believe that comes from my mother. She had a strong moral sense." Ad, he says, campaigned with Sylvia Pankhurst. She'd been to the Soviet Union years before his own first visit, which he made in 1934.
Firmly grounded though his character may have been, Schulberg has always had a talent for communicating the dramatic potential of recklessness in others. He talks perceptively about Hemingway, a man he knew well for 10 years.
"He was always trying to pick a fight. He'd say: 'What the hell do you know about war?' I remember him trying to push me around, arguing about boxers. Whatever area he worked in, he had to own."
As a young screenwriter, Schulberg collaborated with F Scott Fitzgerald. "Around 1939 I was working on a script called Winter Carnival. It was set in Dartmouth, New Hampshire, where I went to college," Schulberg says. "The studio announced they'd assigned another writer to work with me. When I asked who, they said: 'F Scott Fitzgerald.' I said: 'He's dead, isn't he?' They said: 'No. He's in the next room, reading your treatment.' I went in. He said: 'I'm afraid I don't think this is very good.' I said: 'I don't either.' So we went to lunch."
In a moment of imprudence, the studio decided to fly Schulberg and his thirsty companion from Burbank to New York. The two were supposed to improve the script en route.
"My father sent two magnums of Mumm champagne to the plane," Schulberg recalls. "That was our undoing. By the time we came down for fuel in Kansas City, we'd finished the first bottle. When we took off again, we opened the second."
"So when you landed in New York ..."
"We were well away. We checked in to the Warwick Hotel. Scott ordered a bottle of gin. The next day we had our meeting with Walter Wanger at the Waldorf. It did not go well. Because of the condition we were in."
Recollections of this trip form the basis for his masterpiece, The Disenchanted. The central character closely resembles Fitzgerald, though it seems inevitable that Schulberg exploited memories of other broken men, notably his father. Anthony Burgess included The Disenchanted in his 1984 work, 99 Novels: The Best in English since 1939.
"How much work had you done on the journey?"
At the culmination of this binge, he says, "We smelled of booze. We were unshaven. We looked terrible. No hotel would take us. Scott was very sick. In the end, I took him to the hospital."
If Budd Schulberg drew on F Scott Fitzgerald's character in his own fiction, the compliment would be returned, as he was to discover in a later meeting with the author. When Fitzgerald asked him to read from a new manuscript he'd produced, Schulberg realised that the narrative was unmistakably inspired by his own life in Hollywood. The new pages would eventually appear in the opening section of his unfinished, posthumous novel, The Last Tycoon.
Speaking to his close friend Hugh McIlvanney, for a BBC Arena programme in 2000, Schulberg said, of this experience, "I realised that Fitzgerald was feeding on all this Hollywood stuff. I thought: 'My God, this is what writers do; they feed on each other, they can't help themselves. But Scott was writing about the romantic side of Hollywood. And I wrote What Makes Sammy Run?'"
As an incendiary revelation of the crass ambition and poisonous machinations of Hollywood moguls, What Makes Sammy Run? has never been equalled.
"For what it's worth, I think your greatest novel is The Disenchanted..."
"I might believe that too."
"But I don't think anybody would dispute that What Makes Sammy Run? is one of the great American novels. It's never been filmed, has it?"
"No, because the studio bosses have always been afraid of it. Steven Spielberg said something about What Makes Sammy Run? quite recently. He said it was anti-Hollywood and that it should never be filmed."
"What do you think of Spielberg, incidentally?"
"Very able, but very mainstream. I think there's something a bit boring about him."
Schulberg received a letter from his father, after he'd read What Makes Sammy Run?
"I am terribly and literally frightened," BP wrote. "It is too honest and it means the end of you in Hollywood. I would like to emphasise my fear in such a way as to advise your destroying this letter."
The late Kurt Vonnegut came here, to Westhampton, in 2001, to write a tribute to Schulberg for the Paris Review.
"I didn't grow up in Hollywood," Vonnegut told him. "I grew up in Indianapolis. But when you wrote that book, I said: 'This guy's got to be crazy. Putting himself in such terrible danger.'"
"I realised it was dangerous," Schulberg replied, "but I couldn't help it." He recalled that Louis B Mayer demanded his deportation. "In Mayer's mind he was the king of a country, in Hollywood. The DA was on the studio payroll. People could and did commit murder and it wouldn't be in the paper."
What Makes Sammy Run? appalled a constituency broad enough to include the arch-conservative John Wayne, and the American Communist Party, to which Schulberg had belonged for three years in the mid-1930s. The party had tried to restrict him to writing material which served their own interests.
"Why did you join the Communist Party?"
"The main factor, for me, was that fascism was on the march: Hitler, Mussolini, Franco, Mosley. I saw the Party as the strongest answer to the fascists."
His flirtation with Soviet philosophy didn't prevent Schulberg being given a senior role in the Office of Strategic Services (later the CIA) during the Second World War.
"You met [Nazi propagandist] Leni Riefenstahl in Kitzbuhel."
"I was in charge of film evidence for the Nuremberg trials. I went to see Leni. I had a warrant in my pocket. When I took it out, she screamed. I explained that she would not be on trial, but a material witness."
"She always claimed she was a civilian."
"To her dying day. But look – you've seen Triumph Of The Will. I believe that film did more for the Nazi movement than almost anything I can think of."
"You had an exemplary war. You'd long abandoned the Communist Party. Who involved you in the McCarthy hearings?"
"A young writer called Richard Collins. He gave them my name."
"Ever ask him why?"
"Strangely enough, a year ago, I went to see him and I did ask him why. He said he thought he had to name people to save himself, and he chose me because he thought I was so famous that it wouldn't affect me."
A widespread perception of the McCarthy hearings is that those summoned fell into two categories: spineless stooges who panicked and blabbed; and heroic libertarians, who sacrificed their careers in order to tell the committee, to borrow the words of Howard Prince, the main character in Woody Allen's 1976 film The Front, that "You can all go fuck yourselves."
Schulberg's case is not quite so conveniently straight-forward. Nowhere in his history is there anything to indicate cowardice, or indifference to the suffering of others. His courage, and a fierce commitment to helping the oppressed, shine out of his work. One piece on boxer Joe Louis begins with a reference to Ralph Ellison's classic novel dealing with racial hatred, Invisible Man.
"An apt title," Schulberg wrote, "for the black race in America in the 1930s. In the eyes of white people it simply did not exist. It's only against the background of know-nothing racial prejudice that Joe Louis can be understood. The heart of the Joe Louis story is his historic break through the race barrier. The myopic racism of the day was nakedly expressed by Jack London, at ringside to cover [the first black world heavyweight champion] Jack Johnson fighting the hapless white Tommy Burns. 'He is a white man and ' so am I,' wrote this avowed socialist who preached inter-national understanding (apparently for whites only). 'Naturally I want the white man to win.'"
It's sometimes forgotten that it was only Budd Schulberg's contempt for despotism that brought him before Joe McCarthy's committee in the first place. Unlike Robert Kennedy, appointed by McCarthy as assistant counsel on the Senate Permanent Subcommittee on Investigations, Schulberg never expressed any affection for the rabid anti-Communist from Wisconsin.
"It would be hard to overstate the degree to which those hearings divided America," I suggest to Schulberg. "Feelings still run high. Less than 10 years ago someone offered to kill Kazan, because he testified. Is it true that you tempered your own evidence by insisting that the people you named were guilty of foolishness, not treason?" "I did."
"Which names did you give?" "Names that had already been given. I didn't give them any new names."
"Who were they?" "Young writers I'd worked with. One was [the screenwriter] Ring Lardner Jr."
"Wasn't he your friend?"
"Yes. Though we had come to differ about the Communists."
"I mentioned your experience to [the author and broadcaster] Studs Terkel last year, shortly before he died. He defied the investigators and lost his TV show, as you know. What he said about you was: 'It was a scary time. I don't blame anybody for getting scared.' McCarthy created a highly intimidating arena. How much did you co-operate out of fear, how much out of a sense of duty?"
"I might have been affected by intimidation. But at that time I really thought there were secret Communists who were misleading the people."
"Evil as Stalin was, I can't believe that all of these blacklisted guys – take Zero Mostel [Max Bialystock in Mel Brooks' The Producers] – were full-on supporters of his programme."
"I agree. Though Mostel was pretty staunch, actually."
"And you must also have had the motivation of anger, since the Party tried to stop you writing What Makes Sammy Run?"
"Yes. But what really affected me was the treatment of Soviet writers. Over there in 1934 I met a dozen authors. Ten years later, every one had been 'liquidated'. I met Isaac Babel. A wonderful short-story writer. His work is on my bedside table right now. They shot him."
"You said not so long ago that, given your time again, you might still testify."
"I believe I might. I felt I had to tell the truth."
"There's an obscenely compelling quality to that sort of adversarial hearing, isn't there? It's that mixture of bullying, humiliation and threatened exclusion: it's almost like a precursor of reality TV. In the case of McCarthy, the stigma has never quite gone away."
Schulberg remains calm when answering questions about the most painful public episode in his life, put by someone who wasn't even born at the time. He's less patient when people allege, as some have, that On the Waterfront is merely an allegorical representation of the McCarthy trials.
"There's no truth in that. My work was to reveal what was happening on the docks. I know what my motivation was."
"Yet On the Waterfront is the story of a justified informer. It's hard to see how Brando's dilemma over whether to testify can't have some resonance in terms of those hearings. As his character says, 'They're asking me to put my finger on my own brother.' I guess you'd accept that there can be autobiographical elements in a work that even the writer is not aware of, consciously at least. Could we agree that On the Waterfront wouldn't have been as it is, if not for McCarthy?"
"Absolutely not. The story was so strong I would have done it anyway."
Schulberg's Oscar for the screenplay of On the Waterfront is on the mantelpiece, behind him. He went on to write the acclaimed 1957 film A Face in the Crowd, another tale of egotism, this time set in the world of television. The previous year he'd been on the set of the movie based on his 1947 novel, The Harder They Fall. It was Humphrey Bogart's last film.
"This would have been in Bogart's mean and bitter period?"
"He could be very nasty. I found him very difficult to get along with. Very. He'd ask me questions about boxing and, when I would answer, he would ridicule what I was saying. He wasn't easy to get along with."
Schulberg's career has been an unusual one, not least because, having begun as a successful screenwriter, he increasingly found prose to be his natural voice: this at a time when novelists were falling over themselves to get a break in movies. (Even so, it's probably not fanciful to suggest that, if he'd never written What Makes Sammy Run?, subsequent decades might have brought Schulberg many more Academy Awards.)
In the 1960s, Schulberg devoted himself increasingly to sports writing. The elegance and insight of his writing on boxing is such that it makes for compelling reading, even for readers who might view that recreation with indifference or repugnance. Hugh McIlvanney wrote the introduction to Ringside, the 2006 anthology of Schulberg's boxing reportage.
"His longevity alone makes him a living archive of the fight business, and his recollections from as far back as the 1920s are informed by such perceptive observation that distant decades come alive as he evokes them," McIlvanney wrote. "And yet it is not Schulberg on the past but Schulberg on the present that I appreciate most when we are together at fights. I have come across few men with as profound an understanding of the imperatives that govern the quality of a performance in the ring. I confess to a lack of enthusiasm for sporting chroniclers who are fine with description but feeble in judgment. I do think that we can ask of reporters that they should be able to comprehend and interpret the action as it is unfolding and that, once it is over, they should have a clear idea of what they have seen. At ringside," he adds, "Budd Schulberg invariably does all of that, in a voice that is not just engaging and eloquent. It is wise."
No writer has ever been closer to Muhammad Ali. Schulberg travelled in Ali's car on the way to fights, sat in his dressing-room even after defeats, and was at the epicentre of some of the bizarre social situations the Louisville fighter liked to engineer. He was at the Hotel Concord in upstate New York when Ali was training for his third fight against Ken Norton. Schulberg was with his third wife, the actress Geraldine Brooks. "Ali," Schulberg recalls, "asked Geraldine for an acting lesson. She improvised a scene in which he'd be provoked into anger." After two unconvincing attempts, "She whispered in his ear, with utter conviction: 'I hate to tell you this, but everybody here except you appears to know that your wife is having an affair with one of your sparring partners.' I watched Ali's eyes. Rage."
Then, he recalls, Ali had another idea. "'Let's go to the middle of the hotel lobby. You turn on me and, in a loud voice, call me 'nigger'." Once in the foyer, crowded with Ali's entourage, "Gerry dropped it on him. 'You know what you are? You're just a goddamn lying nigger.' Schulberg recalls how Ali waited, restraining his advancing minders at the very last minute; a characteristic sense of timing that allowed his white guests, if only for a moment, to experience the emotions generated by the prospect of imminent lynching, yet live to tell the story.
Geraldine died in 1977. Budd was previously married to "my childhood sweetheart", Virginia Ray, a liaison that lasted seven years, then to the actress Victoria Anderson. He has been with Betsy for almost 30 years. The couple have two children, Ben and Jessica; Schulberg has two surviving children from his previous relationships. His sister Sonya, who never fulfilled early promise as a novelist, lives in Idaho. Family documents, including one emotional letter she wrote to BP, are posted on internet auction sites, and bear witness to their turbulent upbringing.
"Have your childhood memories made you a better parent?"
"I hope so. You know ... I keep coming back to my father. When I was 17 or so, he had an affair with a woman called Sylvia. One day I was driving past her house near Malibu. I knew he was there. I went in and dragged my father out. I said, 'You son of a bitch, you're coming home with me.' I dragged him back and told my parents to stay together. They did try, for a while. But the harm was done."
I end up staying the night in Westhampton. By the time I leave, we've talked for seven or eight hours, over the two days. Preparing to say goodbye to him, you have a sense of abandoning a last connection with another age. It's only when he starts to talk about politics that I realise we haven't even mentioned the death of Robert Kennedy.
"I was with Bobby at the Ambassador Hotel in Los Angeles," Schulberg says. "Sirhan shot him, then ran right into me. I remember he felt very small. I could see Kennedy at the end of the hall. J Edgar Hoover hated him so much that there was no real FBI presence. The mob hated Bobby. The mayor hated Bobby. So he had only amateur bodyguards like the [American] football player, Rosey Grier. After it happened we went back up to Bobby's suite. Grier was so upset. He kept saying, 'It's my fault, it's my fault.' He fell back on the bed; his eyes were sort of rolling back."
One of the curious things about American government, Schulberg continues, "is its extraordinary ability to right itself. In some way, Obama's victory confirmed my belief that this country has an in-built capacity to regenerate, to heal itself, after a period of darkness. After Joseph McCarthy, John Kennedy. After George W Bush, Barack Obama."
Before I leave, he shows me around his office, which is filled with boxes of papers relating to his next volume of autobiography, which will deal with the extraordinary experiences of his adult life. "Books," Schulberg once said, "are like time bombs. They go off again and again." Even though he's 95 next month, and has nothing more to prove in terms of his unique literary talent, it may be that we have yet to witness his most spectacular detonation.
'On the Waterfront' runs at London's Theatre Royal, Haymarket (0845 481 1870, www.trh.co.uk), until 25 April