Is Exmoor the unfriendliest place in Britain?

Liz Jones, the confessional journalist, has been the target of a pot shot. Richard Caring, the restaurant owner, has had his pet stags killed. Their crime? Moving to the Somerset town of Dulverton. Tom Peck dons his bullet-proof vest to investigate

Something is rotten in the county of Somerset, and it's not the apples. Nestled in the vales of Exmoor, beneath two iron age hillforts, the town of Dulverton has been a pretty outpost of humanity since 1084, a tranquil shelter from the wilderness of the surrounding moors. No longer. In this cobbled warren of stone cottages, a 21st-century witch hunt is afoot.

Two of Dulverton's newer residents, people of the earth in the jet-setting, rather than rural sense, complain that they are being pursued by mad, gun-waving locals. Reports of recent events in the town owe something to the finale of the film The Wicker Man, in which a screaming outsider is burnt alive while locals chant a Middle English folk-song.

Four weeks ago, someone took a pot shot at the house of writer Liz Jones, who moved to the town in late 2007. Her letterbox was filled with lead. More recently, and seriously, the rich list entrepreneur Richard Caring, whose restaurant and property empire includes The Ivy, found two pet stags dead on his doorstep, slain by a high-velocity rifle. His estate manager, Alex Cheyne, said: "It doesn't make any sense unless it's a personal attack on Mr Caring or a member of staff. Mr Caring has plenty of money so we wonder if there is some bad feeling among locals." Mr Caring now employs former special forces to patrol the property with night scopes, and has urged: "I hope most local people will not want to be associated with this mindless violence... I would love them [the perpetrators] to try again."

How did it come to this?

The approach drive to Dulverton did not augur well. "Bastards, the lot of them!" cried a lost delivery driver, pulled over by the side of the tarmac. "You ask a farmer sitting on his gate for directions because the signposting is rubbish and satellite navigation doesn't work. He'll say 'Two miles this way turn left, mile-and-a-half down the track, left again, then straight on.' Half an hour later you'll drive back past him again and he'll be sitting there grinning at you.

"They don't mind if you're trying to find the address of one of their own, but if it's for one of the new houses, forget it." Even the sheep and Red Devon cows refuse to budge from the middle of the road.

Jones is widely disliked. Drinkers in Dulverton's Lion pub and the Courtyard Cafe denounce her – "Coming here, telling lies about us," claims one. Locals' treatment of her has perhaps been understandable, if at times nasty.

The moment she arrived in Dulverton, the metropolitan Daily Mail columnist and former Marie Claire editor, who used to vacuum her backyard when she lived in London, began syndicating her verdict on its residents to the national press. The area, she explained, is "faintly Amish" and inbred. "If men have teeth in the West Country, it's a bonus," she said, complaining that shop assistants there "have learning difficulties and have never heard of Illy coffee". Objectionable neighbours were dismissed as "small- minded", "bullying and sexist," and crippled by "insularity, boredom or fear".

At last weekend's Dulverton Carnival, a councillor entered a float called "Liz Jones's Coffee Morning". He sat alone and paraded through the town in a long black wig, jars of Illy coffee, rats and organic muesli on the table, a big "For Sale" sign at the back. It won first prize.

Jones says she is bewildered by the attacks and insists that she contributes significantly to the community, hiring "a local gardener, tree surgeon, equine vet, two chiropractors, an equine podiatrist [and] a holistic shearer." She is considering selling up.

As for Mr Caring, occupier of the berth 146 on the British rich list, he arrived in Dulverton in 2005 via helicopter. The chain of events that has led, four years later, to intruders on his estate shooting two of his adult stags dead – one of them a lingering, six-hour demise, in the 10-acre pen in front of his Grade II-listed house – is more complicated. He seems primarily to have offended some residents two years ago with his plan to build a "Winter Palace" – a glass annex filled with palm trees, a gym, swimming pool, steam room and accommodation for eight. Others were upset by the amount of pheasant shooting on the property, but that has been scaled back.

Jones and Mr Caring do come and go from their Exmoor homes, but both spend considerable tracts of time there, and cannot be trapped by the argument that they are rich incomers who buy properties and leave them empty, pricing out poorer residents.

Bring up the subject of Richard Caring at the bar in Dulverton's Rock House Inn and the mood is one of outrage...that anyone could be so callous as to kill the restaurateur's animals. "You're joking aren't you," protests landlady Diana Bartlett. "Listen to the locals, they're absolutely livid about those stags. I tell you, if they find the person who did it, I wouldn't want to be them."

Outside, it is a beautiful October day: cloudless blue skies, Dartford warblers singing in the trees – and the din of a low-flying helicopter, causing pensioners to shout morning greetings to one another outside the shop. Several explained that it was Mr Caring arriving to "sort out" this business with the stags. Even if it wasn't him, such talk is hardly great PR.

"A lot of this is just hard cheese," said Mal Vango, 54, a web designer who moved to the West Country from London's East End 20 years ago. He lives in the neighbouring village, Goathurst. "The man who owns the huge country house couldn't be nicer. He puts on amazing firework displays on Bonfire Night, does a big hog roast and it's free for the entire village. They go and eat his food, but they still won't accept him. A lot of them used to think they were rich before he arrived. Now they wish he'd take his millions and go away."

Mr Vango's friend "Chinner", a 54-year-old Dulverton born-and-raised man with resplendent beard, offers to take me out to the moor to teach me the ancient art of bolving – imitating the call of the stag, to seek a response. He doesn't appear to be packing a spade, so I agree.

Their annual bolving competition raises thousands of pounds for Devon Air Ambulance. The men cup hands to their mouths and make strange noises, watching all the time lest one of the horny beasts come charging. It is rutting season, after all, and a stag 9ft-high was photographed this week.

Whatever money is raised at this year's tournament, next week, there are concerns that not much will come from deeper pockets in the town. The collected self-styled "artisans" – brickies, gardeners, stonewall builders, electricians – complain that richer residents keep to themselves. The mention of Liz Jones brings a frisson to the air. "People here are very friendly," insists barman Dave Cowers, 47. "But you can't walk into an established culture and say, 'Here I am! Accept me! But I'm not going to accept you.'

"I work with people with learning difficulties, and when she used that term to attack people I couldn't believe it. It's just vituperative."

Up the road, back in the car and on my own again, the satellite navigation leads me lemming-like through a gate marked private and to stop in a field next to a child's trampoline. A grizzled farmer in wellies and flat cap marches up. "Don't tell me!" he cries. "Sat nav." He gives new directions and packs me off on my way – happily towards my B&B, and not straight out of town.