I wasn't surprised to hear that the Lebanese minister for tourism, Fadi Abboud, was threatening to sue the makers of my current favourite TV show, Homeland, for misrepresenting Beirut. As I watched the second episode last Sunday, the action was supposedly based in the Lebanese capital. My wife kept asking: "Do you recognise where they are?" I didn't, mainly because the programme was shot in Haifa, in Israel, which looks more "Arab" to the average American viewer than cosmopolitan Beirut. Women are in the hijab, there is a smattering of camels, and Westerners are stared at suspiciously. No wonder we are wary of the Middle East when we see stuff like this.

Lebanon has always been concerned by negative depictions of it, and Friday's car bomb in Beirut is a disastrous setback. When things are going well in the country, it is a beautiful destination. I grew up an hour from the beach and an hour from the slopes. Unfortunately, there was also a dreadful civil war waging, and this limits your skiing trips. The new, rebuilt Lebanon has been desperate to bring back tourists, and three years ago Beirut was named by The New York Times as the number one place. There is a guy online who challenges anybody who uses the phrase "looks like Beirut" in a derogatory fashion.

The Lebanese are proud of their capital, and Homeland has ruffled their feathers. The fact that the scenes were shot in neighbouring Israel just rubs salt into the wounds. The solution would have been for Homeland to have shot the scenes in Beirut itself, showing the real city, as well as giving employment to local people. But nothing is ever obvious in the Levant.

Homeland is based on an original (and excellent) Israeli series called Prisoners of War, which was aired in the UK on Sky Arts. One of the co-creators, Gideon Raff, works on the US version. As he is an Israeli citizen, he is banned from entry to Lebanon – as is anybody with an Israeli entry stamp in their passport. It is this kind of restrictive madness that has kept the Middle East in an almost permanent state of suspicion and hostility for so long.

I have two UK passports, for travel to countries that are "mutually politically incompatible". When I visited Israel, I took the wrong one, and had to spend three hours with a very aggressive Shin Bet officer who simply refused to believe that I had been to Iran on a skiing trip. I had to produce my laptop and show him my photos of the trip. He was fascinated, as he had clearly never seen anything of Iran but footage of mad mullahs and burning Israeli flags.

While filming in Israel, I met several crew members who had been in the Israel Defence Force when they invaded Lebanon in 1982. One of them had even been stationed in Brummana, the hill-town above Beirut where I grew up and briefly went to school with Osama bin Laden. The guy couldn't stop going on about how beautiful the place and the women were and how much he longed to be able to go back and visit. Maybe, one day, this will be possible. Here's hoping, because both sides are seriously missing out.