But these nondescript buildings, next door to the President's lodge, inhabit a singularly dark place in the collective memory of Ugandans. What occurred within their walls during the 1970s and 1980s has traumatised the nation to this day.
Nakasero was the headquarters of Idi Amin's State Research Bureau (SRB), the euphemistically named security organisation whose "research" consisted of terror, torture and murder. Bureau officers' method of detention - known locally as panda gari - involved kidnapping, and those who entered Nakasero were seldom seen alive again, their bodies turning up in the Nile, in Lake Victoria or hanging from Kampala's main clock tower.
The nightmare did not end with Amin. Milton Obote instituted another reign of terror through the early 1980s, followed briefly by Tito Okello. When Yoweri Museveni's National Resistance Army emerged from the bush war in Luweero and stormed the capital in 1986, Ugandans hoped the bad old days were gone for good.
But many fear panda gari is back. Reports of night-time raids, illegal detentions and torture are increasing as the security forces hit back at a new threat of terrorism. Human-rights groups, politicians, the judiciary and press are growing concerned about methods employed by the Internal Security Organisation (ISO), whose recent move to Nakasero is seen as both insensitive and ominous. Some believe the organisation is mutating into a Nineties version of Amin's hated SRB.
The security crackdown has emerged in response to a wave of bombings which have terrorised Kampala in recent months. Grenades and bombs thrown into bars and restaurants and planted on buses have killed at least 49. Threats against the United States and British embassies in the wake of the Nairobi and Dar es Salaam bombings and the US strikes on Sudan and Afghanistan have heightened tension and led to the FBI, Ugandan CID and ISO employing some heavy-handed tactics in their hunt for terrorists.
Their main suspect is the Allied Defence Forces (ADF), a rebel group with Islamic fundamentalist tendencies, fighting in western Uganda. President Museveni has accused Sudan of aiding and arming the rebels and the security heat has been directed largely at Muslims.
In September, 20 Somalis were arrested in pre-dawn raids. They were held incommunicado in "safe houses" for more than two weeks, in violation of Uganda's 1995 constitution, under which suspects must be produced in court within 48 hours. Most were released without charge.
There have been numerous reports of Muslims being arrested, beaten and tortured using electric shocks. Some are being held on treason charges. One prominent businessman, in court last week on treason charges, displayed scars on his body, allegedly caused by torture at the hands of security agents.
One man who has spoken out is journalist Ogen Kevin Aliro. A week ago, Mr Aliro published a story in The Monitor newspaper entitled "Safe houses - return to the shadows". It detailed the month-long detention and torture of a 14-year-old Somali boy, allegedly by security agents controlled by the ISO. Mr Aliro claimed the boy was hung upside down and beaten, immersed in a tank of water and subjected to mock executions.
Four days after publication, Mr Aliro was ambushed by two carloads of men while driving to his home in a suburb of Kampala. He was severely beaten, suffering a broken collar bone, dislocated arm and cuts to his head. "They kept telling me they will shut my big mouth," he said.
The Ugandan Human Rights Commission says it is investigating an increasing number of complaints.
Mr Museveni last week admitted the existence of secret "safe houses" and blamed lack of police facilities for suspected terrorists being held in unofficial locations.
Rebel groups continue to cause havoc and are bringing their wars to the streets of Kampala.
Mr Museveni is enmeshed in an unpopular war in the Democratic Republic of Congo and, some say, obsessed with Sudan - an Islamic regime he condemns as "Arab people oppressing black people".
Known as the "blue-eyed boy" of the US and international donor community, Mr Museveni is aware that this status, with its billions of dollars of aid and future debt relief, is dependent on maintaining an acceptable human-rights record. This is balanced against the government's insecurity. The question is whether Uganda can shake off the weight of history and prevent Amin's ghosts of Nakasero returning to haunt this suffering country again.Reuse content