Murder most horrid: crime writers gather to celebrate genre

I think I need to get out more. Certainly I can't be doing with the violence and gore, drug-taking, sexual deviancy and dysfunctional behaviour I've been getting at home lately. Before my family revolts against such disclosures, let me explain. I have been venturing out, an innocent abroad, into the imaginative realms of crime fiction.

This, it has to be admitted, is not the kind of book to which I naturally turn for edification or entertainment. But one of the most esteemed prizes in the genre – the Theakston's old Peculier Crime Novel of the Year – has just been awarded. So I decided it was time to take a crash course in crime fiction by skipping through the six books on the prize's shortlist. I have to confess that I did not get very far.

The problem was that the winner, Two-Way Split, a novel which, in the words of its author Allan Guthrie, is about "a psychotic concert pianist turned bank robber who's not taking his medication and finds his wife is sleeping with another member of the gang". It beat top crime writers such as Ruth Rendell and PD James, whose latest works didn't even get onto the long-list.

I could see why he had won. Guthrie is an immensely skilful writer in the tradition of Dashiell Hammett and Raymond Chandler, painting a world that is cruel but also cool, at least so far as the persona of its hard-boiled protagonist is concerned. His prose snarls, squeals and spits. The writing is tight, furious in its pace; there are very few adjectives in his steroid-pumping sentences. And no semi-colons. It is a spare but vivid novel in which the protagonist – you couldn't call him a hero – is not a detective or other puzzle-solver from outside but someone caught up in the violent action, and where all the sex is self-destructive or inadequate.

The characters are clichéd – the crooked private eye, the deranged bank-robber, the cowardly pimp, the frigid wife, the hard-faced tart with the heart of gold and the thug with a maverick morality that allows him to kill anyone, so long as they deserve it (but who is kind to the vulnerable, and loves his mother). But then subtlety of characterisation is not what you turn to crime fiction for. Most readers seem happy to sacrifice that for pace and tension. What gives Guthrie an added dimension is the way in which he makes patterns out of good and evil, damnation, redemption and salvation.

The shame of it is that he makes those patterns in blood and gore, with sharpened screwdrivers and testicles crushed in drawers. Not content with the strangulation of his principal female, he has to have the dead woman's stomach carved up with a knife. It is a world of drug addicts, loan sharks, massage parlours and domestic violence, where unthinking thuggery substitutes for any kind of examined life. There are almost as many deaths as in Hamlet but without any of the accompanying ontological scrutiny. It is a book of chill brutality and deeply unedifying fatalism. The pity was, I thought as I reached the end of the final page, that a writer of such skill can't find something better to do with his talent.

But it may not just be Guthrie. I turned to the next book in the pile, The Dead Place, by Stephen Booth, the chap who has been short-listed for the award for the last three years without winning. It was about "a sicko – a Rampton [high-security hospital] case" psychopath who phones the police to taunt them about a murder he is about to commit. "This killing will be a model of perfection. An accomplishment to be proud of. And it could be tonight or maybe next week. But it will be soon, I promise." Why, I began to wonder, did such a huge group of readers – crime novels outsell literary ones by a ratio of six to one, on average – want to absorb themselves in a world of such undiluted psychological abnormality? After about 80 pages, I turned to the third book.

I didn't even get that far in the last one. Stuart MacBride's Cold Granite begins with the finding of the body of a four-year-old child, strangled and mutilated. "Dead things had always been special to him. Their delicate coldness. The feel of the skin. The ripe, sweet smell as they decayed..." Enough. I turned from the prologue to the first chapter. It began: "It was pissing down outside..." Enough indeed.

As an ingénue in the world of crime writing I had expected something else when I arrived in Harrogate earlier this month for the 2007 annual Old Peculier Crime Writing Festival. With more than 70 writers attending, it is the largest event of its kind in Europe. The genteel spa town, the place where the doyenne of the detective story, Agatha Christie, vanished in the 1930s, seemed the perfect venue. How wrong can you be?

Perhaps I would begin to understand if I attended the Crime Novel of the Year award ceremony. Inside the 17th-century Crown Inn, several hundred crime aficionados are gathered to hear the six short-listed writers talk. They are all men, quiet, precise, with beards as neatly trimmed as that of Peter Sutcliffe, the Yorkshire Ripper.

They are here, says the chairwoman and crime writer, Natasha Cooper, "to reveal the dark secrets of their tortured psyches". I think that's irony. But what begins to emerge is the fault-line that runs through the conference: the question of whether crime fiction is primarily about "gritty realism", to use the favoured term of the purveyors of gore, or whether it's about puzzle-solving, the approach favoured by those who remain in the Agatha Christie tradition.

"Alan, do you censor yourself?" the chairwoman asks. "Yes, you should have seen the first draft," replies Guthrie.

One of the conference organisers, the agent Jane Gregory, later offers an apologia. "Modern crime fiction reflects British society today – war, violence in the streets, drug addicts, care in the community, sex-trafficking. Writers point the finger but in a way that people want to read about. The things that frighten us most are the things we don't understand."

Val McDermid, whose novels have been adapted and extended into the ITV drama series Wire in the Blood, continues: "We tend to make monsters out of people who commit terrible acts. But people aren't born evil. So as a society we need to examine how people become evil, and the ground that nurtures them."

There are, she acknowledges, limits to her grittiness. "There are things I won't write, but I don't know what they are till I get to them." She once asked a clinical psychologist about a character whom she wanted to mutilate his female victims' eyes, ears and mouth. "He said that he'd also cut out her vagina. But I decided I didn't need to do that to make my point."

And in the end, she says, "it's a moral art form". The detective story is a form of certainty in an uncertain world. "Evil is usually punished," says Cooper. Guthrie disagrees: "Is there such a thing as good or bad? I don't think so. I think it's all shades of grey. Is there a kind of morality in this immoral world? I don't really know that there is."

Set against this view of crime fiction is that of the puzzle-solvers. Simon Brett is a writer who specialises in the comedy of manners. Best known more generally for television and radio series such as After Henry and No Commitments, he also produces a cavalcade of crime fiction of a much more elegant variety. "The golden age of crime fiction was the Twenties and Thirties," he says, "with wonderful polymathic sleuths running rings around the local constabulary whose only function was to be baffled. The best of Agatha Christie's books were like clockwork toys. She did it brilliantly."

But things changed. "The Second World War made the concept of playing with death slightly distasteful. And writers began to run out of tricks." There are only so many times you can write a book in which the policeman turns out to be the murderer, or the narrator does it.

Yet in their place, he laments, writers have come up with a new string of clichés they seem happy to repeat – "the dysfunctional policeman, who is a maverick or a rugged individual, who stands out against the system". And he says that a law of diminishing returns applies. "There are only so many ways you can mutilate a body." Brett is happy to describe his books as "old-fashioned".

Others in the same camp voice the same criticisms. "Domestic violence may be terrible but it isn't very interesting," says David Roberts, who sets his stories in the 1930s with the son of a duke as his primary detective. "Why is it only violence that gets describes as 'realism'?" asks the award-winning writer Laura Wilson. "What is more realistic than stroking a cat?"

More seriously, the critic Mark Lawson tells the conference, there is a sense in which some male writers, in particular, seem to enjoy the detail of the violence they inflict on their women characters. There is a gloating and a relish in some writing. It is not gritty realism they seem to be after so much as a cheap thrill.

This, I think, is the gravest charge against much modern crime fiction. Like the Sunday gutter-press it offers a titillating, vicarious pleasure from what it -purports to expose and condemn, turning violence into a kind of pornography and contributing to the coarsening of our national sensibilities.

Ironically, the clearest evidence of this comes from the writers themselves. In person, Guthrie is not a Glaswegian ex-con but a teetotal vegetarian with a gentle demeanour who previously sold books in Waterstone's. The atmosphere at the Harrogate conference is extraordinarily "nice" – with the published writers hugely generous in the time they give to the 100 or so budding authors in the audience, who comprise retired men who look like they were once civil servants, women of a certain age, or young computer geeks.

"It's not as nice as this at a science-fiction conference, where the people have, shall we say, fewer social skills," says Gregory. And gatherings of writers of romantic fiction are far bitchier affairs, says Cooper: "Romantic writers idealise people, so everybody they meet in real life is a disappointment." Whereas "we get rid of a lot of our aggression on the page," says Jim Kelly, although his novels – he writes in the style of a latter-day rural Chandler – are far stronger on a brooding sense of place and atmosphere than they are in the gore factor.

The fans, it has to be said, lap it all up, puzzles, gore and all. "When you read mysteries it brings order out of chaos," says Barbara Hanes, 73, a retired social worker from Orange County, California, over on a "Murder on the QM2" cruise holiday. She gets through three books a week. "It's not a nice world but you get involved in the characters."

The oldest person at the conference, Doreen Adams, from Chester-le-Street in Co Durham, agrees. She is 82 but has already booked in for next year's Harrogate convention. "It's my annual holiday. I do read romantic stuff sometimes, for a change, but I prefer crime, especially the more modern stuff. I can sit up all night to finish it and read practically anything, no matter how bloody and gory it is." Well, at least it keeps her off the streets.