'The world has been my office since I was 17'

First holiday memory?

Swanage in Dorset. We went to Corfe Castle every year until I was about five, closely followed by Pontins in Burnham-on-Sea in Somerset. They were fantastic days: staying in a chalet, going on tandem bike rides, running from the cold swimming pool to the hot – those memories stay with you.

Best holiday?

I went with my wife, Sophia, to Langkawi island in Malaysia in 2004. The circumstances were quite sad because my mum had just died; we decided to go to the Tanjung Rhu resort for a break. I distinctly remember a pianist arriving on the beach every morning. We would lie there, drinking coffee, thinking of Mum, and listening to this beautiful music.

Favourite place in the British Isles?

The Quantock Hills near where I grew up in Taunton. If you want to get a lungful of good air, it's a great place to go walking. Every time I see Somerset County Cricket Ground on television, I always pause it for the views of the Quantocks behind.

What have you learnt from your travels?

The world has been my office since I was 17 years old. I've travelled to places with work that I might not otherwise have seen – more than 50 different countries – but, after a time, it does wear you down. I know I was lucky, but travel becomes a joyless experience if you have to do it. I want to take time to enjoy it now.

Ideal travelling companion?

Sophia. She's got a great spirit for travel and always embraces everything that's put in front of her. We get on like a house on fire. I love escaping with her from everyday cares.

Beach bum,culture vulture or adrenalin junkie?

I try to injure myself in the first few days – doing waterskiing, tennis, yoga, running – so that I have to stop. And then there's nothing quite like being on an unspoilt beach with a bag of ice, a beer in hand and some sandwiches. I went to the state of Pernambuco in Brazil, which has miles of unspoilt coast. The beach at Porto de Galinhas blew my mind.

Greatest travel luxury?

Speedy boarding on easyJet. Admittedly, it's not the biggest extravagance, but I don't like to be separated from my family on the plane.

Holiday reading?

Jeremy Clarkson. Not everyone's cup of tea – but, as a writer, he's hilarious. I've got all his old books and articles at my house in Spain, so I'll dig them out in a few weeks when I go there. Recently, he got in trouble for a joke he made on Top Gear, but his articles from 10 years ago are a lot more outspoken.

Where has seduced you?

Cape Town. There are endless things to discover. We went to visit some friends and stayed with a gentleman called Graham Beck, who owns a big wine company out there with miles of vineyards. It's a land of contradictions: rack-and-ruin poverty on the one hand, outrageous wealth and great beauty on the other.

Better to travel or arrive?

I'm a destination man. There have been hundreds of times when – perhaps after a bad flight, with thousands of tennis rackets hanging off me – I've arrived to realise how lucky I am. Standing on the balcony of the hotel and looking out onto somewhere I didn't even know existed.

Worst travel experience?

A flight from Jakarta. It was 4am, I was exhausted from playing tennis, the plane was packed, and I had to get halfway across the world. About an hour after take-off, we encountered the most horrifying storm. People were screaming, bags were flying, and I had given up hope of being alive.

Worst hotel?

A resort in St Lucia: we went on a family holiday and there was borderline mutiny brewing among the staff. They were not in any mood to be nice to tourists.

Best hotel?

The W Hotel in Doha. I stayed recently for work and it just had everything you needed to function from that perspective. It was very stylish and nothing was too much trouble for the staff.

Favourite swim?

A swim off the coast near a beautiful beach called Calblanque in Spain. There's a little rocky headland where I dive off the cliff and swim round the bay, sometimes with my daughter. Every year, it gets a little bit more difficult, but I love doing it and it makes me feel good.

Best meal abroad?

A shack-type restaurant in Porto de Galinhas. This place is ridiculous; you retire to a hammock in between the main course and dessert for a snooze. I don't remember what I ate – it was such an idyllic setting.

Dream trip?

The Maldives. My wife longs to live in a villa surrounded by water, where she can go diving, so we're looking forward to visiting later this year.

Favourite city?

Paris. It's somewhere I've been recently and would like to get to know better. I hired one of the Vélib' bikes and cycled to the French Open every day to do the commentary. Within 15 minutes of finishing, I was outside the Eiffel Tower. It was a great way to see the city.

Where next?

Two of my daughters have just finished their exams, so we're off to Spain to celebrate and relax.

Andrew Castle is commentating this weekend during the Wimbledon Championships. He is also an ambassador for La Manga Club in Spain (lamangaclub.com).