Gil Scott-Heron - Another new day

Gil Scott-Heron pulled black music out of the Sixties and propelled it towards hip-hop. But he has lain in obscurity (and jail) for 15 years. Now a new album is in the can. Stephen Smith tracked him down

When I finally got hold of an address for Gil Scott-Heron and went to his apartment to meet him, not only was he not living there, there was no apartment there, either. God knows, the so-called "Godfather of Rap" has led an irregular life of late, but this was a no-show that only a virtuoso of the disappearing act could have pulled off. Having come all the way to New York City to interview the man, I was in danger of having nothing to report on.

Scott-Heron is one of the great pioneers of late-20th-century music, whose fusion of political and social-issue poetry with jazz and soul, particularly in the early 1970s, paved the way for rap, and then hip-hop. A published novelist by the age of 20, his tell-it-like-it-is message also marked a clear shift from the peace and love generation, most notably on "The Revolution Will Not Be Televised", a seminal poem and song from 1971 that opened his second album, and which ends with the statement: "The revolution will be live". His uncompromising compositions both drew on and fuelled the Black Power movement in the US.

He has has also been dubbed "the black Bob Dylan". But whereas Dylan is engaged on what appears to be a perpetual tour, Scott-Heron has done a plausible impression of a missing person in the 15 years since his last album was released. There have been stories of drugs and jail time, and rumours about his health. When I mentioned Scott-Heron to friends and colleagues, the ones who remembered him at all said, "is he still alive?"

As I watched my cab vanishing into the traffic of Harlem, I took heart from my recent sightings and soundings of the man. Hadn't I seen him with my own eyes less than 48 hours earlier, playing his old hits at a supper club off Times Square, and teasing the audience about his epic absences? "All those of you who bet I wouldn't be here, you lose!" he said.

On my mobile, I could play back a message from the same fathom-deep voice, the one that used to advertise a soft-drink the colour of spray-tan: "You know when you've been Tango-ed!" Here he was on my phone saying: "Yeah yeah, this is Gil ringing you back... Ring me back. I might be here but that doesn't guarantee anything in particular. OK. Peace." As a ringtone or app, it would make me the envy of my friends. As a shooting schedule, it left something to be desired.

When all seemed lost, a familiar spare figure was suddenly loping across a yard towards me, having chosen that very moment to go to the store for groceries. But for a rare intervention from Scott-Heron's underworked gastric juices, we might never have encountered one another. It seemed that I had come to the right place, after all, but where was number 232, the address I had been given? "When they finished putting numbers on these apartments," said Scott-Heron, indicating the properties at street level, "they just turned around and went back the way they came, putting the next numbers on the first floor."

It was the revenge of the Man on the old troublemaker, whose breakthrough album was named after an address: Small Talk at 125th Street & Lenox.

His unexpectedly auburn curls escaping from under a faded cap, Scott-Heron welcomed me to his den, or office, as it seemed to be, a minimally stocked but well-used studio apartment (his real home, if any, was somewhere else entirely). A sideboard was dominated by an outsize television set, an ironic fixture for a man whose best-known work is a clarion call to couch potatoes. This seemed to invite a question about the state of American culture today.

"America is the McDonald's of culture, they don't do things that last," said Scott-Heron, now 60. "By the time America has gone through as many different changes as England has, you can probably say its culture is here to stay."

A useful basketball player, Scott-Heron has been known to slam-dunk some of the hip-hop artists who have followed in his giant steps, accusing them of "posturing" and lacking a sense of humour. On this occasion, he confined himself to saying, "That stuff is for kids. I ain't been a kid in a long time. I'm not going to damn a 19- or 20-year-old for the first ten records he's done in his life. When I was going through that, I didn't want anyone to sum me up at 18. I ain't done yet!"

He seemed to enjoy yourself on stage the other night, I ventured. "I don't do so much of it now, so I can always enjoy it. If you only do it once in a while, you're fresh, you're glad to see them. You want to make them feel comfortable. They've heard a lot about how you are. I want to show them what I'm really like so they can get over that."

And what was he really like? A regular guy, if you can believe that. "I'm the same as anyone else: you come in, you want a cup of coffee and a sandwich, you go down to the corner and get one."

Scott-Heron's neighbourhood has proud associations with black cultural history. A less happy history of deprivation has been mitigated to some extent in recent years. Bill Clinton's office, no less, is only a block or two away. Scott-Heron told me: "This is a nice neighbourhood, but it's got the same problems as the rest of the city right now: you got no job and you've got a lot of bills."

Who's got us into this fix? "Back in the Sixties they used to say, "speed kills". Now it's "greed kills". Everybody's trying to leverage what they can. A few may make it, but there'll be nobody else around to do the jobs that are needed."

I ask about Obama? When he was first making records, would he have been surprised to see a black man in the White House? "I was surprised to see him when he got there! I was pleased and proud and happy that America had finally turned a corner it should have turned in 1865."

A little more than a century after the abolition of slavery, Scott-Heron's jeremiads against racism and poverty, uncompromising but melodic, were climbing the charts in Britain and the States. He was making catchy pop music about the iniquities of apartheid ("Johannesburg") a decade before Britain's Special AKA released Free Nelson Mandela. Other songs prophetically addressed contemporary bogeys including nuclear power ("We Almost Lost Detroit") and the hollow glamour of celebrity ("Show Bizness").

I asked if his music got him into trouble. Was he on a blacklist? "Nobody ever shows you the blacklist – 'that black guy, he's on the list'. But we weren't allowed to play everywhere, and our records were taken off the shelves. The Bill of Rights and the things that you pay for with your taxes constantly come up to be challenged, to make sure they're still there. And if the right of free speech is what it's supposed to be, then everything you say is alright."

Elsewhere in Scott-Heron's repertoire, songs dealing with substance abuse now have the terrible plangency of telephone calls to the emergency services replayed on television. "Down some dead-end street there ain't no turning back," croons the young Scott-Heron on "Angel Dust". Or consider "The Bottle": "If you see some brother on the corner, looking like a goner, it's gonna be me."

On the corner near Scott-Heron's place, I was happy to treat the singer to a nourishing cheese-steak sandwich at a fortified deli. It was at just such a store, a liquor outlet called The Log Cabin, that Scott-Heron was inspired to write "The Bottle".

"I watched these people lining up every morning. I discovered one of them was an ex-physician who'd been busted for abortions on young girls. There was an air traffic controller in the military: one day he sent two jets crashing into a mountain. He left work that day and never went back. These were not people whose ambition had been to be an alcoholic. They could have got into that career much earlier."

Scott-Heron has served time "in the joint" more than once following drugs offences. "I got caught with some stuff. That doesn't make me Pablo Escobar." He compared the experience to a man breaking his leg: the wound heals, the man puts the episode behind him.

So, I wonder, he did some cocaine once upon a time, and now he doesn't? "I didn't say that."

Rumours of a "new" Gil Scott-Heron album, like a fresh blockbuster from the similarly elusive JD Salinger, have circulated for as long as anyone can remember. But while the singer was in prison, he received a visit from a record producer from London, Richard Russell, and the pair have collaborated on a new long-player, waggishly entitled I'm New Here. The poet is also working on a memoir, for the savvy British publisher Canongate.

Scott-Heron himself admits to a weakness for thrillers, the turf novels of Dick Francis especially. It could scarcely be more nonplussing if Malcolm X himself was found to have been devoted to the Drones Club antics of Bertie Wooster.

Stephen Smith is Culture Correspondent of the BBC's 'Newsnight'. 'I'm New Here' is released on XL Recordings in February

HOW I MET GIL, BY HIS PRODUCER

On June 14 2006 I went to Rikers Island prison facility to meet Gil Scott-Heron to see if he wanted to make a new album. I'd sent him a letter explaining my intention and my ideas, and I had received a call from his friend Mimi telling me Gil was up for seeing me.

The contrast of Gil's spirit – intact and inspiring – with the bleakness of the surroundings was inspirational. It was made clear that CDs couldn't be brought in, but he said I should send any musical production references I wanted him to hear to his friend and housekeeper Mimi. I was confused, and asked how he would then actually get to hear it. Affecting a German accent he said, "ve haff ways".

In June 2007 Gil called me at XL to let me know he was out of jail. At the end of July we sat in Gil's bedroom in New York and we talked. The next day we listened to music; Robert Johnson, Smog, Bob Dylan, Kanye West. Gil was starting to grasp what I proposed we did together.

Gil is profound; he's also funny. As he relaxes his speech becomes peppered with profanities, and with puns and wordplay. I had a sense of a new context for Gil to be presented in.

I was about to leave for New York in October 2007 when Mimi told me that Gil's parole officer had detained him; he was in jail again, in downtown Manhattan. Gil refused my visit.

By January 2008 we had our first proper recording session with Gil. I think we both went into this session with some trepidation. His lyrics, timing and phrasing are still beyond compare. Everything just flows through him. We work with samples and drum machines, and have found a way for Gil to express himself within this more modern context.

The album is short, and intense and focused. Gil has been incredibly open-minded to make a record this current- sounding; it's been easy to forget he's 60 years old. As a nod to the past, we also have a whole extra disc full of Gil performing versions of some of his classic songs on piano, which might serve to lead people to his existing body of work.

Richard Russell is the producer of the Gil Scott-Heron album 'I'm New Here', and the owner of independent record label XL Recordings