Up on Diamond Hill, the British Second World War pillbox looks like one of Enver Hoxha's frontier bunkers, a dome of pre-stressed concrete with rectangular gun slits, the last remnant of Britain's imperial disaster in Hong Kong, a reminder of that most terrible of Christmas Days in 1941. And here, amid the detritus of that ferocious Japanese victory, Kipling hits it to a T:

"Lo, all our pomp of yesterday

Is one with Nineveh and Tyre!"

For the Chinese are building a 17-track rail yard on Diamond Hill and the bunker is likely to be bulldozed, along with an old RAF hangar and whatever else is left of Britain's imperium in the former village of Tai Hom. Even Repulse Bay down below takes its name from the ship which so often graced its azure, overfished waters, and participated in that greatest of all naval disasters, the sinking of HMS Repulse by Japanese aircraft – along with the Prince of Wales – on 10 December 1941. It doomed Singapore. Two days earlier, the Japanese had crossed into the New Territories of Hong Kong and in just over two weeks – accompanied by the usual massacres – they had conquered another sceptred isle.

But you can still see the Gin Drinkers' Line that Major-General Christopher Maltby thought might hold his enemies for a week or two – he was wrong, of course, but the very name tells you what was wrong with the empire. The Royal Scots and two Indian units, the Punjabis and the Rajputs, held on vainly there. It was meant to be held by six battalions. Only 60 members of the Royal Scots could make it. Within 24 hours, the Brits were abandoning Kowloon for the loneliness of Hong Kong island. Churchill knew it was doomed. You only have to look at the pitiful pillboxes, chewed over by bullets 68 years ago, now overgrown with creepers, to see why.

But the battle of Hong Kong has not entirely faded away. First, there is the magnificent 1937 blockhouse that served as an ammunition store, the very last British position to hold out on Shouson Hill on 25 December 1941. It was the last stand, but the building is still intact, its original massive iron doors now protecting a magnificent wine cellar into which a certain Mr Charles Lim, who helps to runs an equally magnificent restaurant beside the walls, took me on a tour of inspection. Even the ammunition racks are in place. And there are two defused shells standing in the corner of a cosy underground restaurant to which wine connoisseurs may repair – safe, no doubt, even in the event of giant tsunami or nuclear attack. General Maltby would have approved.

Not much else about the battle of Hong Kong won his approval. The Japanese shelled the police station, the naval hospital and most of the civilian residential areas of the island and – after slaughtering 20 surrendering soldiers and the medical staff at the Salesian Mission – went on to further atrocities. They tortured and killed more than 60 injured soldiers at St Stephen's College along with, again, medical staff. By the time the Brits surrendered at 3.15pm on Christmas Day, countless thousands of civilians had been killed, along with 1,589 British, Canadian, Indian and Hong Kong soldiers, 1,000 of them in a single day. The Japanese lost 2,000 of their men.

For what? Hong Kong was as strategically useless then as it was when we finally left 12 years ago. But its fall was another symbol of the end of empire, a fact nowhere more sadly recognised than in the Stanley Military Cemetery. It's one of the most curious graveyards of conflict in the world, for it contains not only those familiar Commonwealth war memorials and stones but also large numbers of civilians who died in their own little hell in the Stanley internment camp during three years and eight months of Japanese occupation. There are elderly British ladies, Chinese volunteers, children, even the victims of a misbegotten US air raid later in the war, the Americans being famous for own goals even then.

There's Ethel Kate Wilmers, "born 19 December 1880 at Clifton in England, died 22 August, 1944". After life's fitful fever, her gravestone tells us, "She sleeps well." Perhaps. Mary Williamson's grave is even sadder. She died in her seventies in the Japanese camp on 2 August 1942, close to her 20-year old grandson, Douglas Harvey Collins Taylor, who died on that awful Christmas day but who is "buried in an unknown grave". One gravestone marks the mass grave of 25 men who died the same day. There are Chinese memorials – they fought hard for the Brits and died for them, even in the Royal Navy; HMS Dauntless and HMS Tenedos figure on their memorials and a Brit has written in the visitors' book: "Be proud of Chinese soldiers."

In the Repulse Bay Hotel is Ernest Hemingway's typewriter and his expenses – about which the less said the better – and a picture of the great man, plump on the terrace just six months before the invasion, reporting the Sino-Japanese war. A newspaper of 17 June 1941 survives. "Ernest Hemingway Says China Needs Pilots as Well as Planes to Beat Japanese in the Air," its headline records. By December, he could have said that about Hong Kong.

I suppose the names mean it is for ever England if not empire. There's St John's Cathedral and Beaconsfield House and Chater Road, Jackson Road, Cotton Tree Drive and a statue of John Osborn of the Winnipeg Grenadiers who chucked Japanese grenades back at the enemy until he was too late for one of them and threw his body on top to save his men. "Tell my wife," were his last words. He got the VC.

But my favourite relic of the war are the two bronze British lions that stand today outside the Hong Kong Shanghai banking headquarters behind Queen's Road Central. "Sculptor: W W Wagstaff, 1935," it says on the base. The lions – named Stephen and Stott after early British bank managers – were shot through with shrapnel and bullets in the last battle in the city, but remain proudly unrepaired. You can stick your finger in the bullet holes and realise that the live rounds hit them from both sides. Typical Brits. Like the street names, they survived.