There's a definite novelty factor to ordering a pizza to be delivered to your very own 13th-century tower at Dover Castle. Ideally, we would have rustled something up in our brand-new kitchen – complete with all mod-cons and probably one of the best views in Britain – but it was Sunday, we'd been out seeing the sights and hadn't made it to the supermarket in time.

As I gave our address – "Peverell's Tower, Dover Castle" – to a bemused-sounding man at Dolphin Pizza in Dover High Street, I wasn't that confident of success. English Heritage recently transformed the tower into holiday accommodation; we were clearly the first couple to arrive and order fast food.

Despite our misgivings, our Neapolitans arrived and, after reheating them in the shiny new ovens, we polished them off with a pleasant white wine from the welcome hamper.

Peverell's Tower stands at the highest point within the walls of Dover Castle, which was founded on the White Cliffs in 1066 by William the Conqueror. The tower itself was built around 800 years ago as part of the fortress's defensive system, and is named after William de Peverell, who was made constable of the castle in 1066.

In more recent times it was used as a staff flat, but the property had been empty for a few years when English Heritage came up with the idea to refurbish it and add this unusual abode to its holiday cottage portfolio.

As with all English Heritage holiday homes, the interior of Peverell's Tower has a simple, modern design which is nevertheless in harmony with its historic setting. Soft, muted colours and smart, informal furniture lend the one-bedroom tower a feeling of comfort and understated luxury. The bathroom has a fabulously powerful walk-in shower, and the oak kitchen/diner has everything you might need – from corkscrews to toast racks.

From every window the tower has tremendous views – either of the castle buildings, softly lit at night or, best of all, of the English Channel and the ferries coming and going from the docks.

In the hallway a gnarled wood door leads to a winding stone staircase, which takes you up to a large roof terrace from where you can admire the whole sweeping vista. Indeed, for all the tower's charm, quirkiness and romance, the breathtaking panorama immediately inspires you to go and explore.

The perfect place to start is on your own doorstep, as guests at Peverell's Tower enjoy free entry to the castle itself (plus unrestricted access to the grounds outside opening hours). This is two attractions in one: an ancient castle with a rich history stretching back beyond Norman times, and an incredible labyrinth of Second World War tunnels.

It was from these secret passages, built deep into the cliffs to house the wartime Military Command Centre, that Vice-Admiral Bertram Ramsay planned Operation Dynamo – the evacuation in May 1940 of 338,000 British and French troops trapped by German forces at Dunkirk. On a 45-minute tour you can see the room where Ramsay masterminded this great escape, as well as the telecommunications centre and many other hidden offices and living quarters.

Staying at Peverell's Tower entitles you to free entry to two other castles in the area: Walmer and Deal, both a few miles north-east. Walmer Castle was, like Deal, built during the reign of Henry VIII as part of a chain of defences against Catholic attack from Europe. But in contrast to Deal and Dover, it doesn't look like a castle; it's been updated over the years to become more like a small stately home. It is the official residence of the Lord Warden of the Cinque Ports, and in each room you can see the furniture, pictures and other belongings of successive dignitaries who have held the post since 1708. I was most intrigued by the bedroom of the Duke of Wellington – Lord Warden for 23 years – which is set out just as it was on September 14, 1852, when he died in his armchair.

Directly across the road is a pebble beach. We set off on the one-and-a-quarter mile journey to Deal, passing a band playing on the bandstand at Walmer Green along the way. Deal's main attraction, and the heart of all the action, is the seafront. We joined the Sunday afternoon walkers on the promenade and strolled towards the pier, before taking in our third and final castle.

Although small, Deal Castle dominates the seafront. In such a commanding position, Henry VIII's men would have had a good view of any invaders coming ashore. Today, visitors have access to the whole castle, but we found its cold, empty rooms rather dull after Dover and Walmer.

Another building on the seafront is the tall and narrow Timeball Tower, with a giant grey ball on its roof. It was built in 1821 to enable communication between ships and the mainland before the age of telephone and radio. From 1853 to 1927 the time ball was raised and dropped at 1pm daily to provide an accurate time check for observers at sea. The tower is now a museum of signalling and precision electrical timekeeping.

Before leaving Deal we made a point of looking up at the time ball on the hour to see it rise, then drop.

These days it's a computer-controlled action, which happens every hour between 9am and 5pm, but we didn't care about that; altogether on our Kentish trail, we had a ball.

Peverell's Tower, Dover Castle, sleeps two. It can be booked through English Heritage (0870 333 1187; Rental starts at £320 for a minimum three-night stay including a welcome hamper. Visit Kent: 01271 336 020;