Telling porkies: From Jeffrey Archer to your fibbing friend, we all love to lie. But why?

Rob Hastings talks to the experts - and those whose tall tales went too far

'The worst lie I told was when I said that a friend of mine had died' says Rachel. "That was when I knew it was serious."

The Irish student, in her early twenties, had not invented this story to be awarded thousands in insurance money, or in an attempt to escape a prison sentence by covering up a crime. She had no reason other than to gain attention – and in fact, she was barely in control of what she was saying. She had become a compulsive liar.

"I was sitting with two people who I don't know that well, and it justf came out. They tried to comfort me, but as soon as I said it, I didn't want their sympathy because I realised what I had done and I felt guilty.

"The lies just kept coming out and they always took me completely by surprise. It's not like I'd be thinking about what to say in advance. It just flowed out of my mouth, and I couldn't control it. I wouldn't know when or why I was going to do it, I didn't know where they'd come from."

You might be sceptical reading Rachel's reasoning that she couldn't help what she was doing, and you would have good reason – especially in a year that Lance Armstrong finally admitted to his career being a fraud based on endless deceit, Chris Huhne ending his false protestations about a speeding ticket, and that Mick and Mairead Philpott were revealed to have consistently tried to fool the police, the courts and the nation about the house fire they started that killed their seven children. But then, that's why it's all the more important to speak to a self-confessed liar like Rachel when they seek help – even if it does open us to the same problems encountered by generations of experts.

'Pseudologia phantastica' was the term coined by a German doctor in 1891 after deciding the lies told by five of his patients were so excessive that their cases deserved a new form of classification. Since then, the labels 'mythomania' and 'morbid lying' have also come to be used, but it is compulsive lying and pathological lying that are the most common. Both are bandied around rather casually in the media and even in the legal system – judges are frequently quoted telling newly convicted criminals that they belong to one of these categories.

According to Mike Berry, who recently retired as a senior lecturer in forensic psychology at Manchester Metropolitan University, compulsive lying should only be used as a specific term for people whose lying has become so habitual that they can no longer control it. "The interesting thing about true compulsive liars is that they're lying against their wishes," he says. "They just have this urge or need to lie even if it's not necessarily to their benefit. The difference with pathological lying is that there's no known reason in the brain for it to happen. A pathological liar is someone who has learnt to lie because they see a benefit in lying, so the question is open as to why these people choose to do it."

Berry knows his stuff. He worked in prisons and special hospitals for more than 25 years – during which time he has dealt with "some of the world's best liars" – and also served as an expert witness in the courts to provide evidence on how reliable a witness is.

Of the two forms, compulsive lying is the most difficult to rationalise. As with symptoms of obsessive compulsive disorder, this bizarre psychiatric condition is the result of lying becoming so ingrained in a person's behaviour that it ends up being a biological function. Compulsive liars, it is thought, receive internal gratification simply for telling an untruth, regardless of whether they receive any external reward for the behaviour. Such cases are rare. But it seems lying can indeed become an addiction.

"I've been lying for as long as I can remember," says Rachel, who began lying for a reason, but steadily began doing it regardless. "I guess it started because I wanted people to be impressed by what I was saying, to think 'She's really cool'. So I started saying these things – and people started noticing me. People didn't challenge me, because what I was saying was pretty realistic. I got better at it, too.

"Over the years it steadily became more serious. It used to be that I'd say stuff to look cool, but after a while it wasn't even for that any more. If I was leaving my house and somebody asked where I was going, I'd say I was going to one place, even though I was just going to the shop."

There have been plenty of people famed for fabricating their entire lives around nothing but fibs, porky pies, little white lies and downright whoppers. In the 1960s, there was Frank Abagnale, the American con artist played by Leonardo DiCaprio in the 2002 film Catch Me If You Can, who gained legendary status for assuming eight separate identities in four years (he apparently passed himself off as an airline pilot, a lawyer and a doctor – though some doubt his tales of duplicity, basically accusing him of lying about lying).

Hundreds of investors were also taken in by the financial fraudsters Charles Ponzi, Allen Stanford and Bernard Madoff, whose lies have cost billions. Last year, cinema-goers were left aghast at the tale of Frédéric Bourdin in the documentary about Thef Imposter, a man whose lies led him to impersonate a missing child from another continent to be 'reunited' with his family in an attempt to find a new life.

At times, even the community of forensic psychologists – populated by experts who sometimes spend their whole careers examining those who cannot tell the truth – have themselves been fooled by a charlatan in their midst. Gene Morrison worked on more than 700 criminal cases with police and the courts in the UK over 26 years, despite holding no qualifications whatsoever, before being sent to prison for five years in 2005.

These are the headline-grabbing lies. But it seems many of us know someone like this, to varying degrees. Asking around in my own group of friends when I began researching this article, I was surprised to find just how many knew what they called "solid-gold bullshitters". And for those who realise a friend, lover or work colleague is making up almost everything they say, it can be a baffling, unsettling experience – and one that, despite all their best efforts to conceal their actions, often leaves the liar out in the cold.

For a couple of years, John suspected nothing. "I believed it all," he says, shaking his head. When his best friend, Nicholas, told him that he had done well in his A-level results, that he had begun dating a new girlfriend, and that his job as a football coach was going well, John had no reason not to trust him. Little did he know that his friend's entire life was steadily becoming wrapped up in lies.

John first began to wonder when he met some of Nicholas's old school friends. While he had told John he had completed his A-levels at a nearby school, Nicholas's old mates revealed that he had stopped attending – "disappeared", in their words – even before his GCSEs.

Then Nicholas claimed he had secured a job coaching at a Premier League football club – a team he also claimed gave him trials as a player in his teenage years. John later discovered that Nicholas, in fact, had no job at all. Instead, he was spending his time in his bedroom inventing glory-filled stories.

It was through the internet – and the discovery that Nicholas had created as many as 100 fake Facebook profiles to make himself look more popular and cover up for the gaps in his stories – that allowed John to finally realise that everything he thought he knew about his friend was part of a disturbing and rather tragic hoax to make him seem more interesting, successful and popular.

"He's conscious that he doesn't have a relationship," says John, "so he's concocted a lot of women who leave messages on his Facebook profile. But it's actually him writing them. He tells me all about these women, but I know they're not real. If you look carefully you notice their pictures never quite match up each time they change – I even started saving them to check. He'll say I should meet them and then comes up with weird excuses and it never happens." It has left John unsure of his friendship with Nicholas, a kind and fun person to be around but someone he's no longer sure he can trust.

Young people, like Nicholas, seem to be the most susceptible to compulsive and pathological lying. One review, which found it occurs equally among men and women, concluded the average age at the onset of compulsive and pathological lying to be 16. Their deceitful ways are typically outed at 22. Yet it it is still rare: one study of 1,000 juvenile repeat offenders in the US suggested that, even in this most vulnerable and troubled group of people, just 1 per cent of them could be classed as compulsive or pathological liars.

It has been suggested that a difference in the brain structure of compulsive liars could be part of the reason for their behaviour. Research by the University of Southern California in 2005 indicated that the brains of people with serious histories of lying had up to 26 per cent more white matter (tissue that transmits information, as opposed to the grey matter that processes it) than those who did not. But it is more commonly accepted that social factors, particularly in childhood, are the main origins of the problem.

"If you look at some of the kids growing up on rough estates in Liverpool or Manchester or London, their whole social culture is based around lying," says Mike Berry. "They learn the classic Bart Simpson reply to everything – 'It wasn't me' [Berry, here, may be referring to Bart's "I didn't do it" catchphrase or conflating it with the hit Shaggy song] – and are taught to not tell the police anything. It's a way of coping with life, and it comes to the stage where they lie all the time simply to protectf themselves. And if you lie for long enough, you become quite convincing at it."

Childhood influences can be a key factor at the other end of the social spectrum, too. That certainly appears to have been the case for perhaps Britain's most infamous liar, Jeffrey Archer, the peer and novelist, who not only falsified his CV time and again throughout his life, but also went to prison for perjury and perverting the course of justice and might well be described as having exhibited pathological tendencies. While researching Archer's family life, his biographer, the journalist Michael Crick, uncovered that Archer's father had been a conman and a bigamist who had falsely claimed to have served in the Army during the First World War.

"I suspect growing up with him made Jeffrey Archer a storyteller in the benign sense, but also in the malign sense," says Crick. "Most of us live in a clear world of truth and untruth, but he had the kind of upbringing in which it became totally blurred, in what he had done in life and what he imagined doing in life." He also believes Archer's experiences of school encouraged him to develop a taste for tall tales. "Archer had trouble fitting in at school, he seems to have been bullied and regarded as a bit puny. But if you're someone who can tell good stories, it's a way of ingratiating yourself with people."

Crick is in no doubt as to the extent to which lying has become part of Archer's character. "You have to assume anything he says is untrue unless you have independent corroboration that it's true," he laughs.

Among Archer's lies were his falsified CV, which he used to get into Oxford. Perhaps one of Archer's skills was not going too overboard in his academic lies, however, in contrast to many liars.

"The pathological liar has to oversell it," says Berry. "If someone says they've got a 2:2 degree from Northumbria, you believe them. But they are always brilliant – they've got a First, they got it in two years, they wrote a paper and a book, and it's always from Oxbridge."

Given the amount that ordinary people have been found to lie on a regular basis in their everyday lives, the argument that the basic desire to fit in or impress people can be the underlying cause for pathological lying seems all the more plausible. A poll by the Science Museum found that men admitted to lying three times per day on average, while women admitted they told untruths twice a day. This equates to 1,092 times per year for men and 728 times for women.

But Robert Feldman, professor of psychology at the University of Massachusetts and author of The Liar In Your Life, believes the true extent to which we all lie in everyday life is far greater. In fact, according to Feldman's research, when chatting with strangers, most people will lie three times in just one 10-minute conversation.

Feldman has carried out extensive research into the extent of these everyday lies. "One of the things I typically do in my research is ask people to talk together for a period of time and secretly videotape them," says Feldman. "Then I ask them to watch the video back and tell me every time they say something that is not entirely true. Almost all of the time, people say before watching it that they were totally accurate, totally truthful, and then they watch themselves and they're astounded to see the things they came out with. They don't self-censor themselves because they don't realise that they're doing it. It's a social strategy – we don't even think about it when we're doing it, it just happens."

With this in mind, perhaps we should all consider ourselves compulsive liars. For those who cross the boundary of social norms like Rachel, however, the path to change is hard.

"I started looking for help online, but I couldn't find that much," she says. "I thought there'd be a number I could ring up, I wanted someone to say: 'You're not a bad person for doing this'. I know in my heart I am a good person, but it just kept coming out.

"The last time I lied was at a party. Somebody asked me if I had spoken to someone recently and I said I had, but I hadn't. I'd almost stopped and then I did it again. It makes me feel guilty but you just can't control it sometimes.

"I want to stop, 100 per cent. I'm 21 now and I don't want to still have these lies when I'm 30. If I have something in my head, I just pause and think 'Don't say it, don't say it', and then I feel so much better, it's kind of like a little victory that I didn't say it."

Some names have been changed