Grey by name, passionate by nature: The famously charming Lord Gowrie, critic and Booker Prize judge, discusses of his love affair with literature

LORD GOWRIE is a hard-working and ubiquitous bee, forever pollinating the flowers of contemporary culture. Name any book, and he's read it; name any exhibition, and he's seen it; any newsworthy objet d'art or painting, and the chances are that Sotheby's (of which he is European chairman) has sold it.

Throughout the summer he has been force-reading more than 100 books in his role as chairman of this year's Booker judges and of the Cheltenham Literary Festival, which starts this week. 'I got up at 5am to read novels,' he says. 'I read for two or three hours every morning, and more than a dozen novels a day on holiday - well, not read: gutted.'

I have climbed Sotheby's narrow labyrinthine stairs to reach his eyrie overlooking Bond Street, a clubbish sort of room with a Victorian desk behind which hangs a portrait of a woman painted in 1821 by Adele Kindt, a deep sofa, a low table covered with catalogues, and a couple of comfortably squashed armchairs.

Lord Gowrie begins by referring obliquely to a novel of mine: ' . . . just as described in your own book, if I may so . . .' During our conversation he mentions the book twice more. Flattering stuff. He is famously charming, especially to women, who are rumoured to fall for him in droves.

Like many such men, he is not greatly liked by his own sex. At school, Lord Gowrie says, he was always unpopular. Even those who call themselves his friends are a bit malicious about him. 'He's far too attentive to women,' says one. 'He desperately wants to be rich,' confided another. By contrast, Josephine Hart, she of Maurice Saatchi and Damage, says that - apart from her husband, of course - 'Grey is the most elegant and intellectual man I know. He has a generous spirit and a lovely soul.' His really close friends refer to him by his proper name: Greysteil. The ancestor after whom he is named murdered Rizzio, Mary Queen of Scots' secretary.

Lord Gowrie sits in his favourite armchair and arranges a folder of notes across a rounded pin-striped lap. Although he is not fat, he enjoys his food and adores good wine, and at 54 it's just beginning to show. 'My comfort blanket,' he says, indicating a list of this year's Booker Prize submissions. 'I'm rather dyslexic. I have a pretty good memory, but 111 novels . . . '

This year's Booker shortlist created controversy by omitting Vikram Seth's huge, much-hyped A Suitable Boy. 'We got a savaging over that, me in particular, which in a way I found sympathetic because it shows that those who criticised us felt passionately about the book.

'The funny thing is, we all wanted the book to succeed because we were in a mood to be conservative, and it has many of those virtues. You see people at work, whereas in so many experimental novels nobody ever seems to earn a living. The reason we put out A Suitable Boy is that we thought it was abysmally edited and tailored. In saying this I feel like a pompous schoolmarm - 'Do not come to me with work in this state' - but promising lines of development and tension kept running out. The book needed cutting, like a movie.'

I notice that Lord Gowrie singles out the importance of earning a living. Unlike most hereditary earls, Alexander Greysteil, 2nd Earl of Gowrie, was born landless and penniless. The one thing he did inherit, a gorgeous, crumbling Georgian mansion in Dublin, saddled him with debts for its endless repairs (he finally sold it to the Heinz chairman Tony O' Reilly). Luckily for Lord Gowrie, he has a good and immensely versatile brain, which he has used to support his family. It's arguable that had he been born rich, he would have made much less of a worldly splash.

His father, a commando in the SAS, was killed in action during the Second World War. His mother worked in Intelligence with Freya Stark, so their two small sons - Gowrie and his younger brother, Malise - were left with their grandparents in Ireland.

'It was an interesting, very Republican environment: I think I spat at the Customs officers when I first arrived in Britain. Then my grandfather - a very distinguished Colonial civil servant (Governor-General of Australia, actually) was given a job at Windsor Castle, which not only provided a free house there, but enough money to support my widowed mother and two little boys. So we went from this Republican set-up to spend several childhood years living in Windsor Castle, where the King came to tea. As a corrective to that mad world, Versailles in austerity, with power cuts and tremendous hierarchy, my mother married again and off we went to live in the west of Ireland.'

From there, the young Gowrie went to prep school, thence to Eton, thence to teach in a girls' school, where he fell in love with one of his pupils and denies that the rest fell in love with him (though he admits he came home one day to find the lavatory filled with bluebells),then on to Oxford. I remember him as a brooding, darkly glamorous figure at Balliol, way beyond my ken. 'He was heavily into pictures and poetry and girls,' a male contemporary recalled enviously.

In 1962 he married Xandra, his first love from the girls' school, and took her off to the United States, to a teaching post at Buffalo, then Harvard. Marriage produced his heir and only child, a son called Brer, now aged 30. Why did he choose the academic life, not an obvious line of advancement for a young man with responsibilities but no money?

'I think I was a man who was frightened outside institutions. I know the grammar and politics of institutions: I'd been in boarding schools since I was seven and I knew my way round. So it was probably a reluctance to get out into the arena and, looking back, I regret that.' Does that cautiousness also explain his first marriage?

'I'm very glad I married both the people I did marry, even though one marriage didn't last,' he says carefully. His answers are graceful, prepared, and evasive. I notice from previous interviews that he couches his reply in exactly the same words every time.

At Balliol he had been responsible for buying pictures for his college, paying pounds 75 for a Hockney that would now be worth 100 times as much had the college not sold it as soon as he went down. The art world would have been a more obvious career. 'I had a fantastic time at Harvard, met some interesting people and wrote some good stuff, so I don't want it written off.'

In 1972 he ended a three-year stint as a lecturer in American literature at University College, London, to become a Tory politician in the House of Lords. 'I was desirous of competing,' he says quaintly, 'and the moment I'd finished my book (of poems, entitled A Postcard from Don Giovanni) I thought, that puts an end to that decade, and went into politics.'

He became a Conservative whip in the Lords, first in opposition, then in government from 1972-74 and was soon rising smoothly and steadily. He is frank about his inborn advantages. 'Having a hereditary seat means you start with a very competitive edge and can get promoted above your station earlier because there's less competition in the House of Lords; but if you are good and go on up, you then run into difficulties.' One of his unmentioned difficulties was being rather too highbrow for most Tory career politicians.

He was divorced from his first wife in 1973 and married Adelheid von der Schulenburg, a journalist whom he describes as 'very clever and sensible', the following year.

His political future was assured when he began working as an unpaid adviser to Margaret Thatcher. He must have been a glittering addition to her court; her Earl of Essex, if you like. The Earl of Gowrie is the kind of man most likely to appeal to her: charming, attentive and mellifluous. He speaks of her still with real affection and regret.

'I like her and I think she was a formidable and interesting and unusual politician and that, rather than any agreement with her policies, makes me get on with her. Her male colleagues weren't used to arguing with women, but if you argued back she was a dream to deal with. She's a natural intellectual: that's to say, she is convinced by reasoned argument. I didn't go for the view of her as a suburban woman. Even people I admire, like Michael Ignatieff or Salman Rushdie, would say she's an aggressive philistine. Absolute balls, but it got around and I wish I'd done more to counter it.'

He urges me back to the subject of the Booker. 'Privately, I care far more about the shortlist than the winner. I'd like to put them all in a six-pack and tell people, wherever you place your brow you're going to have a sensation out of these; you're going to be moved.' So it is true, I say, that you picked the final six because you were looking for passionate writing?

'Absolutely]' he says. 'There was a very strong entry this year, and we found ourselves drawn towards a sort of passionate freshness. When I say I'm an amateur critic, I mean it literally, in the sense that amateur originally meant lover. Great literature is analagous to a love affair because you lose your own self and blend with another sensibility. Literature liberates you from your ego.'

My time is up. It has been a charged encounter from which I emerge with a pounding headache. Yes, he is attractive, undeniably; but more important, he is vastly knowledgeable about literature and discusses it with enthusiasm. I have scribbled down a long list of books he has urged me to read, from Henry James's The Wings of a Dove to George V Higgins. Lord Gowrie's charm speeds my going. 'How do you manage to write novels as well as being a journalist?' he asks. 'By getting up at 5am,' I tell him.

(Photograph omitted)