Lies and statistics? The pollsters who make the numbers add up

Predicting who'll win an election is an inexact business, not least because voters don't always tell the truth. Rob Sharp meets the pollsters who make the numbers add up

Do you remember where you were on 10 April 1992? Robert Worcester, founder of Ipsos MORI, certainly does. It was the day after John Major led the Conservatives to their fourth consecutive election victory.

None of the pollsters had seen it coming. The day before the election their data had suggested Labour would narrowly beat the Tories by one percentage point, the equivalent of eight Commons seats, resulting in a hung parliament. Instead, Major's party took an eight-percentage-point lead, or 65 seats, resulting in an overall majority of 21. It was one of the biggest upsets in polling history.

"It was chaos," says Worcester. "There were five principal polling organisations and three refused to talk to the media. I don't believe in maintaining silence, so I was one of the few people interviewed on TV and ended up bearing the brunt of the blame ... I said we didn't know what we'd done wrong, but that we wouldn't leave a stone unturned until we found out."

The Market Research Society, representing the pollsters, looked into what went wrong in 1992, and blamed inaccurate data about the socio-economic breakdown of the population.

As the number-crunchers gear up for another general election, are they now more confident of their data? This month polling company ComRes reported that the Tories had slumped to 38 per cent of the vote, just seven percentage points ahead of Labour, on 31 per cent, with the Liberal Democrats at 19 per cent. Like the predicted result for 1992, this would result in a hung parliament.

So how have ComRes arrived at this latest prediction? The raw data is these days is gathered over the phone (pollsters claim people are more likely to tell the truth than in face-to-face interviews). At the ComRes offices on Millbank, around 30 employees sit behind rows of computer monitors, some busy on the phone, others tapping numbers into databases. Most polling organisations will speak to a sample of 1,000 people per opinion poll. The bigger the sample, the better chance of accuracy. With a sample of 1,000 people, there is a 95 per cent chance of accuracy to within the margin of error, in this case plus or minus three per cent.

Most polling organisations select these 1,000 interviewees randomly from the records of telephone numbers. Of the major polling organisations – ComRes, ICM, Ipsos MORI, Populus and YouGov – only YouGov relies instead on samples taken from its own pool of 300,000 'panel members' – and conducts its polls online. But any sample of 1,000 people may be unrepresentative because it features, say, too many habitual Labour- voters, or too many people from lower-income groups, or too many from certain regions. So the pollsters put the responses they collect through a process of "weighting".

For example, if a sample of 1,000 people is made up of 550 men and 450 women, it doesn't accurately reflect the British population (51 per cent female). To compensate, the answers of female respondents will be given slightly more "weight" (in this case they will each count as 1.133 people). The same procedure is carried out to adjust for age, social class and region. Pollsters will also take into account how people voted in the previous election, and a person's statistical likelihood of actually casting his or her vote.

Pollsters then convert their results into seats in the House of Commons. In carrying out this conversion, online calculators such as that at electoral- calculus.co.uk can also factor in the effect of marginal constituencies, where a seat is most likely to change hands, as well as the effects of tactical voting. Back in 1992, the pollsters had inaccurate information about socio-economic groups partly because the latest census results were 11 years out-of-date; we also now know that many people who in the event voted Conservative lied to the pollsters about their voting intentions. These days pollsters argue that weighting according to past vote gets around that bias.

"Historically, over the longer term, pollsters have had a mixed record," admits ComRes founder Andrew Hawkins. "But in 2005, within 48 hours of the election every eve-of-election poll was [accurate to] within three per cent. This is strikingly accurate given that the journey from poll question to results is not a straightforward one. It's not like using a dipstick on a car engine. It's more of a modelling process." Every pollster, he says, should "show a willingness to change their methods over time."

Do we have any choice but to listen to the pollsters? Not really, says John Rentoul, The Independent on Sunday's chief political commentator: "We don't have any alternative way of finding out what the electorate is thinking, apart from focus groups, but they don't deliver the numbers." But some pollsters will never lay the ghosts of 1992 to rest.

"I'm not confident that we can't have another 1992 election, in the same way I can't be confident that journalists aren't going to make any more mistakes," says Robert Worcester. "You can't be 100 per cent sure that lightning isn't going to strike twice. It's statistics, and I spend a heap of my life explaining this. It's the marriage of asking questions with the science of sampling. I'd always say that it's 95 per cent expertise and five per cent luck. And if you're not lucky, then you're in the wrong business."

PROMOTED VIDEO
Have you tried new the Independent Digital Edition apps?
News
ebooksAn unforgettable anthology of contemporary reportage
Arts and Entertainment
glastonbury
News
i100
Arts and Entertainment
Shock of the news: Jake Gyllenhaal in ‘Nightcrawler’
filmReview: Gyllenhaal, in one of his finest performances, is funny, engaging and sinister all at once
Arts and Entertainment
Shelley Duvall stars in Stanley Kubrick's The Shining
filmCritic Kaleem Aftab picks his favourites for Halloween
News
people
Arts and Entertainment
Kit Harington has been given a huge pay rise to extend his contract as Jon Snow in Game of Thrones
tv
Life and Style
Taste the difference: Nell Frizzell tucks into a fry-up in Jesse's cafe in east London
food + drinkHow a bike accident left one woman living in a distorted world in which spices smell of old socks and muesli tastes like pork fat
Sport
Luke Shaw’s performance in the derby will be key to how his Manchester United side get on
footballBeating City is vital part of life at United. This is first major test for Shaw, Di Maria and Falcao – it’s not a game to lose
Life and Style
Google's doodle celebrating Halloween 2014
tech
Arts and Entertainment
Don’t send in the clowns: masks and make-up conceal true facial expressions, thwarting our instinct to read people’s minds through their faces, as seen in ‘It’
filmThis Halloween, we ask what makes Ouija boards, demon dolls, and evil clowns so frightening?
News
peopleFarage challenges 'liberally biased' comedians to 'call him a narcissist'
Latest stories from i100
Have you tried new the Independent Digital Edition apps?
Independent Dating
and  

By clicking 'Search' you
are agreeing to our
Terms of Use.

iJobs Job Widget
iJobs General

Senior IP Opportunity at Major Firm

vary Attractive Salary: Austen Lloyd: MANCHESTER - AN OPENING AT A VERY HIGH Q...

Nursery Manager

£100 - £110 per day: Randstad Education Ilford: Nursery Manager Long term Ran...

Sales Consultant – Permanent – West Sussex – £24-£25k plus commission and other benefits

£24000 - £25000 Per Annum plus company car and commission: Clearwater People S...

SEN Teaching Assistant

£45 - £65 per day: Randstad Education Bristol: Supply SEN Support Jobs in Bris...

Day In a Page

The drugs revolution starts now as MPs agree its high time for change

The drugs revolution starts now as MPs agree its high time for change

Commons debate highlights growing cross-party consensus on softening UK drugs legislation, unchanged for 43 years
The camera is turned on tabloid editors in Richard Peppiatt's 'One Rogue Reporter'

Gotcha! The camera is turned on tabloid editors

Hugh Grant says Richard Peppiatt's 'One Rogue Reporter' documentary will highlight issues raised by Leveson
Fall of the Berlin Wall: It was thanks to Mikhail Gorbachev that this symbol of division fell

Fall of the Berlin Wall

It was thanks to Gorbachev that this symbol of division fell
Halloween 2014: What makes Ouija boards, demon dolls, and evil clowns so frightening?

What makes ouija boards and demon dolls scary?

Ouija boards, demon dolls, evil children and clowns are all classic tropes of horror, and this year’s Halloween releases feature them all. What makes them so frightening, decade after decade?
A safari in modern Britain: Rose Rouse reveals how her four-year tour of Harlesden taught her as much about the UK as it did about NW10

Rose Rouse's safari in modern Britain

Rouse decided to walk and talk with as many different people as possible in her neighbourhood of Harlesden and her experiences have been published in a new book
Welcome to my world of no smell and odd tastes: How a bike accident left one woman living with unwanted food mash-ups

'My world of no smell and odd tastes'

A head injury from a bicycle accident had the surprising effect of robbing Nell Frizzell of two of her senses

Matt Parker is proud of his square roots

The "stand-up mathematician" is using comedy nights to preach maths to big audiences
Paul Scholes column: Beating Manchester City is vital part of life at Manchester United. This is first major test for Luke Shaw, Angel Di Maria and Radamel Falcao – it’s not a game to lose

Paul Scholes column

Beating City is vital part of life at United. This is first major test for Shaw, Di Maria and Falcao – it’s not a game to lose
Frank Warren: Call me an old git, but I just can't see that there's a place for women’s boxing

Frank Warren column

Call me an old git, but I just can't see that there's a place for women’s boxing
Adrian Heath interview: Former Everton striker prepares his Orlando City side for the MLS - and having Kaka in the dressing room

Adrian Heath's American dream...

Former Everton striker prepares his Orlando City side for the MLS - and having Kaka in the dressing room
Simon Hart: Manchester City will rise again but they need to change their attitude

Manchester City will rise again but they need to change their attitude

Manuel Pellegrini’s side are too good to fail and derby allows them to start again, says Simon Hart
Isis in Syria: A general reveals the lack of communication with the US - and his country's awkward relationship with their allies-by-default

A Syrian general speaks

A senior officer of Bashar al-Assad’s regime talks to Robert Fisk about his army’s brutal struggle with Isis, in a dirty war whose challenges include widespread atrocities
‘A bit of a shock...’ Cambridge economist with Glasgow roots becomes Zambia’s acting President

‘A bit of a shock...’ Economist with Glasgow roots becomes Zambia’s acting President

Guy Scott's predecessor, Michael Sata, died in a London hospital this week after a lengthy illness
Fall of the Berlin Wall: History catches up with Erich Honecker - the East German leader who praised the Iron Curtain and claimed it prevented a Third World War

Fall of the Berlin Wall

History catches up with Erich Honecker - the East German leader who praised the Iron Curtain and claimed it prevented a Third World War
How to turn your mobile phone into easy money

Turn your mobile phone into easy money

There are 90 million unused mobiles in the UK, which would be worth £7bn if we cashed them in, says David Crookes