The secret life of J R R Tolkien

The Lord of the Rings is the product of a magnificent imagination. Not that you'd have known it from the author's life, says D J Taylor

One of my strongest memories from childhood – it would have been September 1973 – is of walking into a room where a radio was playing and hearing a voice announce that JRR Tolkien had died. The voice was followed by a clip of Tolkien himself, curiously undulating in tone, offering the famous explanation of how he came to write The Hobbit: marking exam scripts late one night in the early 1930s and chancing on the providential blank page on which some instinct prompted him to print the words "In a hole in the ground there lived a hobbit..."

No other writer – not CS Lewis, not Henry Treece, not Rosemary Sutcliff – meant more to me as a child than John Ronald Reuel Tolkien. Even now, without turning to the bookshelf, I could tell you the names of the 12 dwarf companions of Thorin Oakenshield, explain the relationship between the orc-captains Shagrat and Grishnakh, and make a stab at reciting the riddles exchanged between Bilbo Baggins and Gollum deep in the tunnels below the Misty Mountains. One of the features of The Hobbit (1937) and The Lord of the Rings (1954-55) – and a characteristic of nearly every great book – is that, more even than the mighty deeds, one remembers the detail: the history of "pipeweed" (Middle Earth tobacco), for example, or the names of the hobbit families who turn up at Bilbo's farewell party. It is a world that convinces merely because of the mental scenery constantly shifted into place.

Amid the colossal fuss about Peter Jackson's impending three-film version of The Lord of the Rings – the first instalment, The Fellowship of the Ring, opens in mid December – the author of perhaps the grandest exercise in contemporary myth-making has been set rather to one side. The tendency, encouraged by late-career photographs of him cheerily smoking a pipe, or appraising trees in the Oxford botanical gardens, has been to regard him as an amiable old gentleman bearing a distinct resemblance to his most famous creation.

In fact, as a trawl through the standard biographical accounts soon demonstrates, Tolkien was far from being a hobbit. He was simply an exceptionally old-fashioned 20th-century Englishman (born 1892) who spent most of his time trying to detach himself from the 20th century. Even by the standards of his day and the environment in which he moved (Oxford common rooms peopled by sympathetic male cronies), Tolkien's separation from the processes of modern life can seem startling. He rarely watched a film or read a contemporary novel. His idea of a holiday was a fortnight's genteel seclusion in Bournemouth. The gap between the conventionality of this outward life and the seething torrents of his imagination – all pitched battles, towering crags and foaming cataracts – is not perhaps as great as it seems. Tolkien's imaginative life, lived out on the plains of Middle Earth, was all he needed.

And yet its splendours sit oddly beside the thoroughly dull existence that brought it into being: birth in South Africa, a poverty-haunted Midlands childhood dominated by the death of his mother, early marriage and the war, a first academic job at Leeds. Humphrey Carpenter begins the second half of his biography with the disclaimer "And after this, you might say, nothing else really happened."

And nothing really did. Tolkien went back to Oxford, his spiritual home since undergraduate days, held two professorships, busied himself with his academic work (his special subject was the early English dialects of the West Midlands) spent most of his spare time with his male friends and devoted his small hours to the creation of his own private world and the elvish languages that would sustain it. He was, in addition, a devoutly hardline Roman Catholic and a political reactionary of the most primitive cast. "Touching your cap to the squire may be damn bad for the squire," he is supposed once guilelessly to have remarked, "but it's damn good for you."

Well, yes indeed. And here again is an apparent paradox. How could someone who experienced a moment of profound spiritual exaltation every time he took Communion spend most of his life creating an alternative world governed not by a deity but by battling wizards? The answer is that Tolkien saw no distinction between the God-created world he inhabited and the dragon-scorched hills of his imagination. In the end, the second world was an extension of the first. Novelists, he argued, were "sub-creators" set to work by God to produce stories that would magnify the grandeur of the original. The myths on which Middle Earth rested were in a certain sense "true". There was a famous conversation with his friend CS Lewis along these lines. Lewis maintained that myths were self-evidently false – "lies breathed through silver", but lies nonetheless. No, Tolkien told him, they were inventions around the truth, ways in which primitive men explained things whose existence they were certain of, but which they could only partly comprehend.

All this is a substantial distance away from what, if one removes the allegorical elements that Tolkien claimed not to exist (evil coming from the East, characters returning from the dead...), remains the greatest Boy's Own Paper adventure ever written.

One can admire Tolkien but it is difficult to like him. Within five minutes of starting to read about him, one is dragged down by a piece of obduracy or an emotional block. Not, of course, that "liking" an author has – or should have – anything to do with his or her work. Whole schools of literary criticism, after all, have been founded on the necessity of separating the writer from the text. The gangs of Sixties hippies who turned Tolkien into a US campus bestseller were more interested in the environmental themes of the books – the hobbits returning finally to a country despoiled in their absence and spending the last pages putting it to rights – than the person who produced them.

Curiously enough, the author of The Lord of the Rings offers a ready way out of this dilemma. Perhaps Tolkien's most unusual characteristic is his apparent distance from the books he wrote. Read a contemporary novel by the likes of Salman Rushdie or Martin Amis, and practically the first thing you will be aware of is the figure of the author capering around in the text. But apart from a few stagey interjections in The Hobbit (many of which were cut from later editions), there is in Tolkien's work barely such a thing as an authorial presence; it's seamless narrative – "storytelling" in the elemental sense.

Three rings for the elven kings

under the sky

Seven for the dwarf lords in

their halls of stone

Nine for mortal men doomed to die

One for the dark lord on his

dark throne

In the land of Mordor where

the shadows lie

One ring to rule them all,

one ring to find them

One ring to bring them all

and in the shadows bind them

These are lines fit to be carved on stone, but one never wonders about the personality behind them. Tolkien did not believe himself to be an inventor, but that he was a conduit for what was already there. His usual response, if asked to explain a particular twist of character or language in his books, was to say that he did not know but would try to find out. Nearly 30 years after his death, The Lord of the Rings remains a fabulous but solitary book: like a great painting, alone on its easel in an empty gallery, on whose bright canvas no artist's signature can be traced.