They came at three in the morning, their regular calling time. He still remembers the fear of a 5 year-old boy, awoken in the dark by the shouts, the barking of police dogs and the threats. "They would come barging in turning the house and our beds upside down, searching, always searching."
Even today, he concedes, the memories are vivid, the scars remain.
Home was Soweto and the house the family still owns to this day. But even a 5 year-old quickly learned the reality of life for a family with the name 'Sisulu' in apartheid South Africa. By the time Lungi was in his teens, his mother had become the first woman to be detained under the 90-day law, his father had gone underground and his brother had also fled.
He was left at home to fend for his siblings and himself, with help from Beryl, his cousin. Yet still the apartheid government's police came. "They had not given up on getting to us as kids. They would come when we were alone, again in the middle of the night, and say 'We are going to take you all to the welfare people and they will distribute you to different families all over the country'.
"I have never been able to understand the evil of such minds. Some white people have come to me and said, 'We don't know how you forgave the evil of what people did to you'. One said 'I would never forgive them: I would make sure they were paid back'.
"But I forgave them even then because it was pointless fighting and hating when you had so much on your hands. The struggle for life, the day to day struggle to understand where the next meal would be coming from, was of far greater importance. We were just trying to survive, living from one emergency to the next."
Today, the eldest son of Walter Sisulu is a successful businessman in Johannesburg, the Chairman of Arup SA Ltd, a company of Engineers, designers and planners. His life is unrecognisable from those times; indeed, the new Johannesburg has changed beyond all measure. But the scars are never far below the surface...
"If you have experienced the torture, the crying and the dying of your comrades in the next cell…..I heard all of that and it did leave me scarred."
He concedes they were times that shaped his whole life. Even as a 10 year-old, he was running errands for the ANC, passing on literature which could be concealed within his young, bulky torso. "I was fat and sometimes my body was used to hide documents which I carried around Soweto. Later, I travelled all over."
Wasn't it frightening ? "Yes. But fear always brings doubts and I didn't want to be stymied by it so I put it aside. I had a job to do, I was not going to be stopped by fear. But while young white people of my age were growing up and having a good time, I was a young man already. Sometimes, when I look at my own grandchildren now, I wish I could go back and experience the joys of being a child………
"I had to give up my childhood and I couldn't talk about what I did, that was too dangerous. It meant I was virtually grown up by the time I was 10 or 12. I tended to bottle up a lot of things when I grew up because of the circumstances I found myself in."
Yet even now as he reflects on those times, Lungi Sisulu cannot quite believe it all ended when it did. He surprises me with his view that, far from the thought that it could have stopped years earlier, he didn't expect to see the end of apartheid in his own lifetime.
"For me, the repression could have gone on beyond my time. I was prepared to go on fighting and to die during that period, to see this country free. It could have gone on for another 50 years; I had read about some regimes in the world that had hung in there for a hell of a long time" (witness Zimbabwe ?).
"So it could have been a lot worse. 1994 was seen as a miracle by the world; they were expecting a war that could have gone on and on. All those things were there on the horizon, I could see them happening. For me, there were no short cuts to freedom."
We pause. So many memories, so many times gone, so many friends lost. For a moment, Lungi's sad eyes betray his inner emotions. But what of the present, what of the South Africa young men like him were inspired to fight for ? Do they survey the contemporary scene with pleasure and pride, or are there concerns ?
South Africa's development in the 14 years since 1994 has been tremendous, he insists. It is the one thing he always prides himself with. "The ANC has delivered, not what the people expected but we have to be realistic. It took those guys 50 years to mess up this country and although I don't think it will take us 50 years to right all that, only 14 years demands realism.
"Yet already in those 14 years we have covered huge tracks. Those who had never thought they would see roads coming to their village, saw roads arrive, water introduced and electricity, too."
Up in the mountains of the Transkei, in his own village of Encgcobo to where his father Walter had first gone upon his release from Robben Island in 1989, life could hardly have been simpler. They had to leave their cars at the bottom of the mountain and walk the 1 kilometre up to the village, carrying or using animals to drag everything. But today, there is a road right into the village, a tap with running water, electricity etc. "These things were unheard of in 1989, so we have to acknowledge things like that" he tells me.
This goes to the heart of one of the issues of modern day South Africa. It is difficult, he believes, for the white people because they have never had such deprivations. "Many have never been in a rural area and they think 'delivery' should be enjoyed by them. They want their suburbs to be patrolled; they don't remember the state perpetrated crime in the townships that happened at the time when they were comfortable in their suburbs.
"We had a death rate in Soweto alone of 45 people per weekend. Now we talk of a death rate of 42 a month in the whole of Gauteng. I am not condoning murders, robberies or attacks in any sense: nobody can defend crime at any place, anytime. But you have to balance things when you say crime is bad.
"That was why we formed committees to defend ourselves against this sort of crime. We did something about it in the townships then and it needs to be the same today. Everybody needs to be involved in trying to solve crime; you cannot have a policeman standing at every gate. People need to get organised, know their neighbours, keep in touch. We formed street committees and they helped. The power and potential of ordinary people coming together to fight this problem is great, fantastic. The power of people, whether positive or negative, is always awesome."
It is a view that underlines Sisulu's opinion of one of the key points of our conversation; namely, his disappointment at what he sees as an opportunity lost by the last Nationalist Party leader FW de Klerk, in the years since the ANC took power in 1994.
"Up to this day, I think De Klerk is still very much hung up on the old ways. He never stops trying to defend some of the things that happened in the past and that hurts me when he doesn't show complete honesty.
"As a man who was there when we planned the new South Africa, he should have come all the way. Of all the people, he is the last person to have made enemies with Mandela, but that happened."
What was the cost to the future South Africa that De Klerk didn't prove a committed fellow traveller along this road ? Sisulu is in no doubt on that score. "Sadly, it meant a lot of white people who would have come along with us otherwise, didn't do so. That is why we still have a racially divided South Africa today. It is because of people like him.
"Had he come along all the way, we would have had a more united country….by now. As it is, we are still battling to have a really integrated South Africa. It is still polarised, still very divided on colour lines."
Nor does he see that scenario changing any time soon, for he believes too many older white people are set in their ways. "The next generation might come together but only 'might' because wherever you are is informed by where you come from. If the cult of the family is such that you continue to regard blacks as inferior and still hate them, you carry it down to your kids. That is already happening right now."
He is thinking primarily of the young men at the University of the Free State, guilty of revolting crimes against blacks. His view is that they did not decide to be what they were or do what they did on their own; they were informed by where they came from. For Sisulu, that grim incident was a reinforcement that so much hate remains in this land.
"It wasn't a turning point but a reinforcement that we still had a lot of hate among the races. I believe a lot of black people are actually bending over backwards to try to accommodate the white people. But the white people are not forgiving, especially the older crowd who are very antagonistic. They still regard us as inferior people. It's as though they have never forgiven us for having taken over the country. So whatever good we do is swept under the carpet."
Much remains to be done in a variety of fields, not least education. He remembers his own school days, trying to learn with 75 in a class. By and large, he admits, the schools in the townships are still having it hard. Yet this issue must be addressed with urgency; as he puts it, South Africa needs to speed up the skilling up of its own young people. Education, he is certain, is fundamental to the whole future of not just South Africa but the entire African continent.
On Zimbabwe, he suspects that despite the widespread criticism of Thabo Mbeki, little more could realistically have been done. "The Zimbabwean people have been too acquiescent. Mugabe comes with the background of being a hero which is one of the reasons why the people haven't killed him.
"They say he fought for independence and saw them through a very tough period. They still have a certain amount of respect for him having actually liberated the country. They talk from emotion, that is the intrinsic problem. Why do you think they haven't shot and killed him ?
"It is in the nature of some people to be extremely aggressive and violent and it's in the nature of others to be very obedient. Mugabe will always win when it comes to the votes in rural areas because these are simple people. They see him having been put there almost by God, as a divine descendent of Kings."