PD James: Novelist and public servant who began as a crime writer but whose work crossed over into mainstream fiction

Phyllis Dorothy James was born in 1920, the eldest of three children of an Inland Revenue officer, Sidney Victor James, and his wife, Dorothy

Baroness James of Holland Park, better known as PD James, or the "Queen of Crime", whose novels crossed over into mainstream fiction, once said that success had come to her 30 years too late. The early years of her married life were an impoverished nightmare in which she scraped a living as a National Health clerk while bringing up two daughters and caring for a mentally ill husband.

Phyllis Dorothy James was born in Oxford in 1920, the eldest of three children of an Inland Revenue officer, Sidney Victor James, and his wife, Dorothy. She had mixed memories of childhood; she wasn't close to her younger sister and brother, and her parents didn't get on. Her father was not affectionate – indeed he sometimes frightened her, though later in life she admired his bravery and independence. "But they're not qualities that are important to a child," she said. "What a child wants is love and affection – oh, and approval and generosity. My father was a very mean man."

She was an anxious child, who thought of herself in the third person – as a character in a novel – to help herself get to sleep at night. She did remember pleasant summer holidays when the family would camp in her father's First World War army tent on the cliffs outside the fishing village of Lowestoft in East Anglia and explore the area by bus and on foot. The area drew her for the rest of her life.

Although she spent most of her time in London, she had a cottage in Southwold in Suffolk, and East Anglia inspired several of her novels: "My books hardly ever start with an ingenious method of murder. Almost always the idea for a book comes to me as a reaction to a particular place and setting. In East Anglia I love the wide skies, the marshes, the estuaries, the little villages. Not pretty, but full of character."

She was 19 at the outbreak of the Second World War and started work as a Red Cross nurse. Two years later she married Connor Bantry White, a medical student. They had their first daughter, Clare, in 1942. She vividly recalled sleeping in a friend's basement near the hospital waiting for her second daughter to be born in the summer of 1944 at the height of the V-1 rocket attacks on London.

"There were the bombs, and I would read Jane Austen, and one went into her blessed atmosphere of sanity and peace." She called the baby Jane. Many years later she cited Austen, as well as Evelyn Waugh, Graham Greene and Anthony Trollope, as major influences; her husband liked Joyce and Dickens, neither of whom she could abide.

In 1944 her husband went to serve in the Royal Army Medical Corps for the rest of the war but returned home a sick man and was diagnosed with schizophrenia. Until his death in 1964 he was regularly confined in psychiatric hospitals. Times were hard in the immediate postwar years. He received no war pension and James had realised he would almost certainly never get better. She got a job as an NHS filing clerk while taking evening classes in hospital administration, qualifying in 1949.

To be able to work full-time she sent her daughters to boarding school. For a time the family moved in with her in-laws, who then looked after the children during the holidays. James's home life was distressing, but she worked hard and advanced rapidly to become a hospital administrator overseeing various psychiatric units, experiences that formed part of the background to some of her books.

By the time she was 39 in 1959, even with a demanding job, a sick husband and teenage daughters to bring up, she felt she could afford the time to write a novel. "I feared that if I didn't settle down and write, I'd end up an old lady telling my children and grandchildren that I had always wanted to be a writer."

She was an admirer of the classical detective novel, in particular of Margery Allingham and Dorothy L Sayers. She decided to write a crime story as practice for the "proper" novel she would write the following year. In fact Cover Her Face took three years to write and was accepted by the first publisher who read it. She published it under her maiden name with those ambiguous initials because "I had no wish to write under my married name."

Her second novel, A Mind To Murder, set in a psychiatric outpatient clinic, came out a year later, in 1963. Her husband died the following year; the coroner gave an open verdict but it may have been suicide. Many years later, James told Anthony Clare on the radio: "I was very upset, very upset, but what has interested me is that I have continued to be upset. I have continued to grieve for him. It has become a very long and increasing grief."

Her third novel, Unnatural Causes, was published in 1967. By then she had begun to enjoy the restrictions imposed by the genre's conventions, regarding them as a useful discipline. She had created a central protagonist, Adam Dalgliesh, who seemed to straddle the line between the modern police procedural and the classic detective stories with their dilettante detectives. On the one hand he was a professional police officer, on the other an intellectual and a published poet.

"I wanted a very sensitive, essentially lonely and withdrawn person," she said. "I wanted him to be quite obviously very intelligent. I was chiefly concerned then with creating a detective quite unlike the Lord Peter Wimsey kind of gentlemanly amateur. I knew I wanted a real professional. But I wanted him to be something more – a complex and sensitive human being. Perhaps that's partly why I also made him a well-known poet."

Her fourth novel, Shroud for a Nightingale (1971), brought the classic detective novel into the modern age. Set in a nurses' training school, it combines the traditional elements of the genre – clever plotting, a complex puzzle to be solved by deductive means and a cast of striking characters – with realism and a powerful awareness of the complexities of the human mind. She admitted that her earlier novels had been rather formulaic but now she was focused on creating real characters whose behaviour had plausible motivations.

She had moved out of the NHS in 1968, passing the exam to become a Principal in the Civil Service. Until 1972 she worked in the Police Department and from 1972-79 as a senior civil servant in the Criminal Policy Department, focusing on juvenile crime. She was responsible for the appointment of scientists and pathologists to England's forensic research laboratories and advised ministers on juvenile crime and the law. With her full-time job she was too tired to write at night. "A lark, not an owl", she "scribbled" at weekends and early on weekday mornings, writing by hand, then putting her first drafts on tape for a professional typist.

With TA Critchley she examined the fragmentary evidence involved in the Ratcliffe Highway Murders of 1811 and published their conclusions in 1971 as The Maul and the Pear Tree. In 1972 she introduced the young private detective, Cordelia Gray, in An Unsuitable Job For A Woman.

Dalgliesh, who began as a detective chief inspector, had become a commander by her sixth novel, The Black Tower (1975), one of her three favourite novels (the others are A Taste for Death and Original Sin). In the 1970s James hit her stride, each novel more skilfully written than the last. Death of an Expert Witness (1977) was inspired by her beloved East Anglia.

"I like to create in books some kind of opposition between places and characters," she said. Expert Witness was set at a forensic science laboratory in East Anglia, and I got great pleasure out of placing a crime in that strongly institutional setting, and contrasting the discipline of the institution with the undisciplined – anarchic, if you like – nature of murder, and showing how it affected the people." This was the beginning of her breakthrough, though that came properly with her next novel, Innocent Blood, in 1980. Intriguingly this was a novel that was more "moral fiction" than mystery fiction, containing no puzzle elements at all.

The previous year she had worked out that she could afford to retire from the Home Office eight months early, at Christmas 1979 rather than the following August. But before publication her situation changed. "That was my situation on the Monday, and then they auctioned the American paperback and film rights, and by the Friday I was rich."

She returned to mystery fiction two years later with The Skull Beneath the Skin (1982) and produced two more first-rate bestselling novels in the 1980s, A Taste for Death (1986) and Devices and Desires (1989). A Taste for Death crossed over from crime into mainstream fiction, receiving long reviews and becoming a Book of the Month Club selection in the US.

The 1980s was James's decade. Her books were big-sellers, An Unsuitable Job For A Woman was filmed in 1982, and in 1983 Death Of An Expert Witness became the first Dalgliesh novel adapted for TV, with Roy Marsden as Dalgliesh. The others followed over the next 15 years.

Phyllis Dorothy James on stage during a reading of her book 'Death Comes to Pemberley' last year (EPA)

A magistrate from 1979-84, she threw herself even more into public service (she said that if she had to choose between writing and public service she would choose the latter). A staunch Anglican, she was on the Liturgical Commission involved with the service for the Church of England. She noted: "I think a religious sense is like an aesthetic sense. You're born with it or you aren't, and I don't mean that those born without it are less good people. I can only speak for myself. I have a need for the assurance that some beneficial power exists. I do believe in that."

Awarded many major prizes, she became an Associate Fellow of Downing College, Cambridge in 1986, served as a BBC Governor from 1988-93 and on Arts Council and British Council advisory panels. She was chairman of the Booker Prize judges in 1987, Chairman of the Society of Authors between 1984-86, and was made a Fellow of the Royal Society of Literature in 1987. Also in 1987 she was awarded the Diamond Dagger Award for lifetime achievement by the UK's Crimewriters' Association.

Ennobled in 1991, she took her duties in the House of Lords very seriously, and was showered with honorary degrees and fellowships. Yet people who met her testified to her lack of pretension and her friendliness.

In 1992 she moved away from mystery fiction with The Children of Men, a morality tale set in 2021, but returned to her chosen genre with Original Sin (1994) and A Certain Justice (1997). Her books were often a long time in the preparation. Before starting to write, she would spend a lot of time with a notebook. How long she spent on research varied with each book but, she said "I always seem to know" when it was time to start writing.

In 1999 she was made a Grandmaster of the Mystery Writers of America and produced a highly regarded "fragment of autobiography", A Time to be in Earnest. Her final novel was Death Comes to Pemberley, which continued Pride and Prejudice with a murder mystery.

Her reputation as a serious novelist was assured back in the 1980s. "Basically," one critic said, "James is a novelist who happens to put her character into mystery stories." James herself said, "I write detective stories. I hope they're novels, too, and I don't see any contradiction in that. But if I felt there was a contradiction, if the detective element got in the way of the novel and I had to sacrifice one or the other, then the detective element would have to go." She saw the crime novel, she said, as "in favour of rationality. That's what the form is all about. Crime fiction affirms the intelligibility of the universe; the moral norm; the sanctity of life."

Although she had her cottage in Suffolk, she lived most of the time from the start of the 1990s in her Georgian house in Notting Hill. She lived alone but saw a great deal of her daughters and her five grandchildren. Her main dislike – fear, in fact – was violence. She kept her home securely locked at all times.

"I hate the fact that I can't walk up to Notting Hill, five minutes away, in the evenings without the likelihood of being accosted. I dislike violence, I'm frightened of it. I'm very worried by the fact that the world is a much more violent place than when I was a girl. And it may be that by writing mysteries I am able, as it were, to exorcise this fear, which may very well be the same reason why so many people enjoy reading a mystery."

She always had a fascination with death. When she was 10 she and a friend saw the body of a schoolboy being pulled from a river. In the 1990s she said, "I've had an awareness of the reality of death from childhood. An awareness that life was short and sped past at this astonishing speed and that this inevitable end was there for all of us. The young often don't think about it because it's so far ahead but I always did think about it. I don't think it overshadowed my life particularly but I was just always aware that death could happen to anyone at any moment. And the huge finality of it." Late in life she said, "I am infinitely grateful to life and health, and aware of how brief and uncertain life is. We must live each day to the fullest." That PD James certainly did.

PETER GUTTRIDGE

Phyllis Dorothy James, author and public servant: born Oxford 3 August 1920; OBE 1983, cr 1991 Life Peer; married 1941 Connor Bantry White (died 1964; two daughters); died 27 November 2014.