Lies and statistics? The pollsters who make the numbers add up

Predicting who'll win an election is an inexact business, not least because voters don't always tell the truth. Rob Sharp meets the pollsters who make the numbers add up

Do you remember where you were on 10 April 1992? Robert Worcester, founder of Ipsos MORI, certainly does. It was the day after John Major led the Conservatives to their fourth consecutive election victory.

None of the pollsters had seen it coming. The day before the election their data had suggested Labour would narrowly beat the Tories by one percentage point, the equivalent of eight Commons seats, resulting in a hung parliament. Instead, Major's party took an eight-percentage-point lead, or 65 seats, resulting in an overall majority of 21. It was one of the biggest upsets in polling history.

"It was chaos," says Worcester. "There were five principal polling organisations and three refused to talk to the media. I don't believe in maintaining silence, so I was one of the few people interviewed on TV and ended up bearing the brunt of the blame ... I said we didn't know what we'd done wrong, but that we wouldn't leave a stone unturned until we found out."

The Market Research Society, representing the pollsters, looked into what went wrong in 1992, and blamed inaccurate data about the socio-economic breakdown of the population.

As the number-crunchers gear up for another general election, are they now more confident of their data? This month polling company ComRes reported that the Tories had slumped to 38 per cent of the vote, just seven percentage points ahead of Labour, on 31 per cent, with the Liberal Democrats at 19 per cent. Like the predicted result for 1992, this would result in a hung parliament.

So how have ComRes arrived at this latest prediction? The raw data is these days is gathered over the phone (pollsters claim people are more likely to tell the truth than in face-to-face interviews). At the ComRes offices on Millbank, around 30 employees sit behind rows of computer monitors, some busy on the phone, others tapping numbers into databases. Most polling organisations will speak to a sample of 1,000 people per opinion poll. The bigger the sample, the better chance of accuracy. With a sample of 1,000 people, there is a 95 per cent chance of accuracy to within the margin of error, in this case plus or minus three per cent.

Most polling organisations select these 1,000 interviewees randomly from the records of telephone numbers. Of the major polling organisations – ComRes, ICM, Ipsos MORI, Populus and YouGov – only YouGov relies instead on samples taken from its own pool of 300,000 'panel members' – and conducts its polls online. But any sample of 1,000 people may be unrepresentative because it features, say, too many habitual Labour- voters, or too many people from lower-income groups, or too many from certain regions. So the pollsters put the responses they collect through a process of "weighting".

For example, if a sample of 1,000 people is made up of 550 men and 450 women, it doesn't accurately reflect the British population (51 per cent female). To compensate, the answers of female respondents will be given slightly more "weight" (in this case they will each count as 1.133 people). The same procedure is carried out to adjust for age, social class and region. Pollsters will also take into account how people voted in the previous election, and a person's statistical likelihood of actually casting his or her vote.

Pollsters then convert their results into seats in the House of Commons. In carrying out this conversion, online calculators such as that at electoral- calculus.co.uk can also factor in the effect of marginal constituencies, where a seat is most likely to change hands, as well as the effects of tactical voting. Back in 1992, the pollsters had inaccurate information about socio-economic groups partly because the latest census results were 11 years out-of-date; we also now know that many people who in the event voted Conservative lied to the pollsters about their voting intentions. These days pollsters argue that weighting according to past vote gets around that bias.

"Historically, over the longer term, pollsters have had a mixed record," admits ComRes founder Andrew Hawkins. "But in 2005, within 48 hours of the election every eve-of-election poll was [accurate to] within three per cent. This is strikingly accurate given that the journey from poll question to results is not a straightforward one. It's not like using a dipstick on a car engine. It's more of a modelling process." Every pollster, he says, should "show a willingness to change their methods over time."

Do we have any choice but to listen to the pollsters? Not really, says John Rentoul, The Independent on Sunday's chief political commentator: "We don't have any alternative way of finding out what the electorate is thinking, apart from focus groups, but they don't deliver the numbers." But some pollsters will never lay the ghosts of 1992 to rest.

"I'm not confident that we can't have another 1992 election, in the same way I can't be confident that journalists aren't going to make any more mistakes," says Robert Worcester. "You can't be 100 per cent sure that lightning isn't going to strike twice. It's statistics, and I spend a heap of my life explaining this. It's the marriage of asking questions with the science of sampling. I'd always say that it's 95 per cent expertise and five per cent luck. And if you're not lucky, then you're in the wrong business."

Arts and Entertainment
TVShow's twee, safe facade smashed by ice cream melting scandal
Arts and Entertainment
tv
Arts and Entertainment
Lisa Kudrow, Courtney Cox and Jennifer Anniston reunite for a mini Friends sketch on Jimmy Kimmel Live
tv
Life and Style
tech
PROMOTED VIDEO
Have you tried new the Independent Digital Edition apps?
News
ebooksAn evocation of the conflict through the eyes of those who lived through it
News
Actor, model and now record breaker: Jiff the Pomeranian
Video
News
REX/Eye Candy
science
Sport
Alexis Sanchez celebrates after scoring his first goal for Arsenal in the Champions League qualifier against Besiktas
sportChilean's first goal for the club secures place in draw for Champions League group stages
News
i100
News
Down time: an employee of Google uses the slide to get to the canteen
scienceBosses are inventing surprising ways of making us work harder
Independent
Travel Shop
the manor
Up to 70% off luxury travel
on city breaks Find out more
santorini
Up to 70% off luxury travel
on chic beach resorts Find out more
sardina foodie
Up to 70% off luxury travel
on country retreats Find out more
Latest stories from i100
Have you tried new the Independent Digital Edition apps?
Independent Dating
and  

By clicking 'Search' you
are agreeing to our
Terms of Use.

iJobs Job Widget
iJobs General

Teacher

£130 - £131 per day: Randstad Education Chelmsford: Ks1 teacher required for m...

Project Manager (infrastructure, upgrades, rollouts)

£38000 - £45000 Per Annum + excellent benefits package: Clearwater People Solu...

MI Analyst and SQL Developer (SQL, SSAS, SSRS)

£28000 - £32500 Per Annum + 28 days holiday, pension, discounts and more: Clea...

Creative Content Executive (writer, social media, website)

£30000 - £35000 Per Annum + 25 days holiday and bonus: Clearwater People Solut...

Day In a Page

Israel-Gaza conflict: No victory for Israel despite weeks of death and devastation

Robert Fisk: No victory for Israel despite weeks of devastation

Palestinians have won: they are still in Gaza, and Hamas is still there
Mary Beard writes character reference for Twitter troll who called her a 'slut'

Unlikely friends: Mary Beard and the troll who called her a ‘filthy old slut’

The Cambridge University classicist even wrote the student a character reference
America’s new apartheid: Prosperous white districts are choosing to break away from black cities and go it alone

America’s new apartheid

Prosperous white districts are choosing to break away from black cities and go it alone
Amazon is buying Twitch for £600m - but why do people want to watch others playing Xbox?

What is the appeal of Twitch?

Amazon is buying the video-game-themed online streaming site for £600m - but why do people want to watch others playing Xbox?
Tip-tapping typewriters, ripe pongs and slides in the office: Bosses are inventing surprising ways of making us work harder

How bosses are making us work harder

As it is revealed that one newspaper office pumps out the sound of typewriters to increase productivity, Gillian Orr explores the other devices designed to motivate staff
Manufacturers are struggling to keep up with the resurgence in vinyl records

Hard pressed: Resurgence in vinyl records

As the resurgence in vinyl records continues, manufacturers and their outdated machinery are struggling to keep up with the demand
Tony Jordan: 'I turned down the chance to research Charles Dickens for a TV series nine times ... then I found a kindred spirit'

A tale of two writers

Offered the chance to research Charles Dickens for a TV series, Tony Jordan turned it down. Nine times. The man behind EastEnders and Life on Mars didn’t feel right for the job. Finally, he gave in - and found an unexpected kindred spirit
Could a later start to the school day be the most useful educational reform of all?

Should pupils get a lie in?

Doctors want a later start to the school day so that pupils can sleep later. Not because teenagers are lazy, explains Simon Usborne - it's all down to their circadian rhythms
Prepare for Jewish jokes – as Jewish comedians get their own festival

Prepare for Jewish jokes...

... as Jewish comedians get their own festival
SJ Watson: 'I still can't quite believe that Before I Go to Sleep started in my head'

A dream come true for SJ Watson

Watson was working part time in the NHS when his debut novel, Before I Go to Sleep, became a bestseller. Now it's a Hollywood movie, too. Here he recalls the whirlwind journey from children’s ward to A-list film set
10 best cycling bags for commuters

10 best cycling bags for commuters

Gear up for next week’s National Cycle to Work day with one of these practical backpacks and messenger bags
Paul Scholes: Three at the back isn’t working yet but given time I’m hopeful Louis van Gaal can rebuild Manchester United

Paul Scholes column

Three at the back isn’t working yet but given time I’m hopeful Louis van Gaal can rebuild Manchester United
Kate Bush, Hammersmith Apollo music review: A preamble, then a coup de théâtre - and suddenly the long wait felt worth it

Kate Bush shows a voice untroubled by time

A preamble, then a coup de théâtre - and suddenly the long wait felt worth it
Robot sheepdog technology could be used to save people from burning buildings

The science of herding is cracked

Mathematical model would allow robots to be programmed to control crowds and save people from burning buildings
Tyrant: Is the world ready for a Middle Eastern 'Dallas'?

This tyrant doesn’t rule

It’s billed as a Middle Eastern ‘Dallas’, so why does Fox’s new drama have a white British star?