When is an Englishman allowed to become passionate about Scotland? David Cameron took great trouble, during his visit to Edinburgh, to sound like a tactful husband, anxious to persuade his wife, Caledonia, that far from being a domineering brute who commands her to remain trapped in a loveless marriage, he is a humble supplicant who would be ever so grateful if she would try to make a go of the relationship.

In political terms, this was astute. Cameron has made it harder for his rival, Alex Salmond, to portray him as an arrogant Englishman. Salmond is a seducer who thrives by bringing out the worst in other people.

But from an emotional point of view, Cameron's performance was unsatisfying. He could not express the agonising sense of loss which many of the English would feel if the Union with Scotland were to be broken. I am aware that this emotion is not universally shared: that many of the English would say "good riddance" if Scotland were to go her own way.

The narrowness of this English nationalism alarms me. What a diminishment of both England and Scotland would result if we went our own ways. Each would become a narrower country. Scotland's golden age came after 1707. The Scottish Enlightenment – the Scotland of David Hume and Adam Smith – still echoes through British public life, nourishing a more generous patriotism.

As I write the word "British", I hesitate. One cannot pretend that it is a fully satisfactory term. English, Scottish, Welsh and Irish all sound more natural. We Unionists find ourselves defending something which sounds a bit artificial.

But that is one of the reasons why Britishness is such a valuable concept. Its very artificiality helps to make it more inclusive. It is much easier to become British than to become either English or Scottish. We all benefit from this wider frame of reference: from a mental hinterland that enlarges our sympathies and saves us from claustrophobia.

The British Empire gave a still wider frame of reference, and as the historian Michael Fry and others have described, it was built very largely by Scots. It offered a tremendous enterprise in which Scotland played a tremendous part. The British Empire is gone: but in an era of globalisation, it would be a shameful waste to discard also the outward-looking spirit which it fostered in both Scotland and England.

We British tend to imagine that we are less cosmopolitan than, say, the Germans or the French. I said as much to a German professor, who replied that when he had spent time in Edinburgh and London, he had been struck by how many of us have relatives living abroad – in Australia or Canada or India. This is much less often the case in Germany. We Britons are more cosmopolitan than we realise. It would be sad, and quite unnecessary, if having lost our empire, we also lost our outward-looking attitudes, and became immersed in an unnecessary project to reduce our own frontiers.

Alex Salmond will say, of course, that he wants an outward-looking and cosmopolitan Scotland. It is for the Scots to decide whether they believe him. Like David Cameron, I do not want to create needless aggro by presuming to tell the Scots whether or not Salmond is a prophet or a shyster. But this is not just an argument for the Scots to have among themselves. Nor is it just an argument between the Scots and the English. It is also an argument between the English and the English.

For an Englishman, in my opinion, it is good to be able to think of Scotland even if one seldom or never goes there. As it happens, I go there fairly often, being married to a Scotswoman. But that is not why I am writing this article. Even if I never caught the train north, I would still feel more free for the knowledge that the United Kingdom extends far beyond England.

One of the most enjoyable aspects of the Anglo-Scottish relationship is the way in which we have abused each other. These things ought not to be taken too seriously: abuse is very often a sign of affection. Dr Johnson was wonderfully rude about the Scots, but had it not been for the genius of Boswell, we would know so much less about the genius of Johnson. Boswell's Journals, in which he describes his irrepressible gift for bursting in upon famous men – Rousseau and Voltaire, as well as Johnson – are some of the greatest books of that kind ever written. Scotland can be as proud of Boswell as England is proud of Johnson. But what is the point in trying to apportion credit to these two writers? Johnson and Boswell belong together, just as England and Scotland belong together. To try to separate them is a kind of barbarism.

If England and Scotland do split, it will be like a divorce in which both partners realise too late that they were actually far happier together, before they allowed Salmond to entrance them with illusory ideas of bliss. It will be an atrocious and wilful waste of something very valuable, which has developed over centuries and could be wrecked within a few years.

There will be vulgar Englishmen who say we are better off without paying subsidies to the Scots, better off without Gordon Brown, and better off without the rest of the Scottish Labour Party. Unimaginative Tories will claim their party could enjoy permanent supremacy in an independent England: as if a new balance of power would not very soon arise once Scotland was gone.

What short-sighted reasons for tearing up the Union. It is greatly to Cameron's credit that he is not taken in by them. And perhaps it is also to Cameron's credit that his tone is so Anglican: that he sounds so rational and well-mannered and unexcitable. I, too, am a rather unexcitable Englishman, who would rather make a joke than admit to deep emotion. But I feel very deeply that to destroy the Union would be a monstrous act of vandalism.