Lies and statistics? The pollsters who make the numbers add up

Predicting who'll win an election is an inexact business, not least because voters don't always tell the truth. Rob Sharp meets the pollsters who make the numbers add up

Do you remember where you were on 10 April 1992? Robert Worcester, founder of Ipsos MORI, certainly does. It was the day after John Major led the Conservatives to their fourth consecutive election victory.

None of the pollsters had seen it coming. The day before the election their data had suggested Labour would narrowly beat the Tories by one percentage point, the equivalent of eight Commons seats, resulting in a hung parliament. Instead, Major's party took an eight-percentage-point lead, or 65 seats, resulting in an overall majority of 21. It was one of the biggest upsets in polling history.

"It was chaos," says Worcester. "There were five principal polling organisations and three refused to talk to the media. I don't believe in maintaining silence, so I was one of the few people interviewed on TV and ended up bearing the brunt of the blame ... I said we didn't know what we'd done wrong, but that we wouldn't leave a stone unturned until we found out."

The Market Research Society, representing the pollsters, looked into what went wrong in 1992, and blamed inaccurate data about the socio-economic breakdown of the population.

As the number-crunchers gear up for another general election, are they now more confident of their data? This month polling company ComRes reported that the Tories had slumped to 38 per cent of the vote, just seven percentage points ahead of Labour, on 31 per cent, with the Liberal Democrats at 19 per cent. Like the predicted result for 1992, this would result in a hung parliament.

So how have ComRes arrived at this latest prediction? The raw data is these days is gathered over the phone (pollsters claim people are more likely to tell the truth than in face-to-face interviews). At the ComRes offices on Millbank, around 30 employees sit behind rows of computer monitors, some busy on the phone, others tapping numbers into databases. Most polling organisations will speak to a sample of 1,000 people per opinion poll. The bigger the sample, the better chance of accuracy. With a sample of 1,000 people, there is a 95 per cent chance of accuracy to within the margin of error, in this case plus or minus three per cent.

Most polling organisations select these 1,000 interviewees randomly from the records of telephone numbers. Of the major polling organisations – ComRes, ICM, Ipsos MORI, Populus and YouGov – only YouGov relies instead on samples taken from its own pool of 300,000 'panel members' – and conducts its polls online. But any sample of 1,000 people may be unrepresentative because it features, say, too many habitual Labour- voters, or too many people from lower-income groups, or too many from certain regions. So the pollsters put the responses they collect through a process of "weighting".

For example, if a sample of 1,000 people is made up of 550 men and 450 women, it doesn't accurately reflect the British population (51 per cent female). To compensate, the answers of female respondents will be given slightly more "weight" (in this case they will each count as 1.133 people). The same procedure is carried out to adjust for age, social class and region. Pollsters will also take into account how people voted in the previous election, and a person's statistical likelihood of actually casting his or her vote.

Pollsters then convert their results into seats in the House of Commons. In carrying out this conversion, online calculators such as that at electoral- calculus.co.uk can also factor in the effect of marginal constituencies, where a seat is most likely to change hands, as well as the effects of tactical voting. Back in 1992, the pollsters had inaccurate information about socio-economic groups partly because the latest census results were 11 years out-of-date; we also now know that many people who in the event voted Conservative lied to the pollsters about their voting intentions. These days pollsters argue that weighting according to past vote gets around that bias.

"Historically, over the longer term, pollsters have had a mixed record," admits ComRes founder Andrew Hawkins. "But in 2005, within 48 hours of the election every eve-of-election poll was [accurate to] within three per cent. This is strikingly accurate given that the journey from poll question to results is not a straightforward one. It's not like using a dipstick on a car engine. It's more of a modelling process." Every pollster, he says, should "show a willingness to change their methods over time."

Do we have any choice but to listen to the pollsters? Not really, says John Rentoul, The Independent on Sunday's chief political commentator: "We don't have any alternative way of finding out what the electorate is thinking, apart from focus groups, but they don't deliver the numbers." But some pollsters will never lay the ghosts of 1992 to rest.

"I'm not confident that we can't have another 1992 election, in the same way I can't be confident that journalists aren't going to make any more mistakes," says Robert Worcester. "You can't be 100 per cent sure that lightning isn't going to strike twice. It's statistics, and I spend a heap of my life explaining this. It's the marriage of asking questions with the science of sampling. I'd always say that it's 95 per cent expertise and five per cent luck. And if you're not lucky, then you're in the wrong business."

Start your day with The Independent, sign up for daily news emails
Have you tried new the Independent Digital Edition apps?
ebooks
ebooksAn introduction to the ground rules of British democracy
Latest stories from i100
Have you tried new the Independent Digital Edition apps?
SPONSORED FEATURES
Independent Dating
and  

By clicking 'Search' you
are agreeing to our
Terms of Use.

iJobs Job Widget
iJobs General

Recruitment Genius: Sales Executive or Senior Sales Executive - B2B Exhibitions

£18000 - £30000 per annum: Recruitment Genius: A Sales Executive or Senior Sal...

Recruitment Genius: Head of Support Services

£40000 - £55000 per annum: Recruitment Genius: This is an exciting opportunity...

Recruitment Genius: Warehouse Team Leader

£22000 per annum: Recruitment Genius: This industry leading company produces h...

Recruitment Genius: Business Development Manager / Sales - OTE £40,000

£20000 - £40000 per annum: Recruitment Genius: This IT provider for the educat...

Day In a Page

The Silk Roads that trace civilisation: Long before the West rose to power, Asian pathways were connecting peoples and places

The Silk Roads that trace civilisation

Long before the West rose to power, Asian pathways were connecting peoples and places
House of Lords: Outcry as donors, fixers and MPs caught up in expenses scandal are ennobled

The honours that shame Britain

Outcry as donors, fixers and MPs caught up in expenses scandal are ennobled
When it comes to street harassment, we need to talk about race

'When it comes to street harassment, we need to talk about race'

Why are black men living the stereotypes and why are we letting them get away with it?
International Tap Festival: Forget Fred Astaire and Ginger Rogers - this dancing is improvised, spontaneous and rhythmic

International Tap Festival comes to the UK

Forget Fred Astaire and Ginger Rogers - this dancing is improvised, spontaneous and rhythmic
War with Isis: Is Turkey's buffer zone in Syria a matter of self-defence – or just anti-Kurd?

Turkey's buffer zone in Syria: self-defence – or just anti-Kurd?

Ankara accused of exacerbating racial division by allowing Turkmen minority to cross the border
Doris Lessing: Acclaimed novelist was kept under MI5 observation for 18 years, newly released papers show

'A subversive brothel keeper and Communist'

Acclaimed novelist Doris Lessing was kept under MI5 observation for 18 years, newly released papers show
Big Blue Live: BBC's Springwatch offshoot swaps back gardens for California's Monterey Bay

BBC heads to the Californian coast

The Big Blue Live crew is preparing for the first of three episodes on Sunday night, filming from boats, planes and an aquarium studio
Austin Bidwell: The Victorian fraudster who shook the Bank of England with the most daring forgery the world had known

Victorian fraudster who shook the Bank of England

Conman Austin Bidwell. was a heartless cad who carried out the most daring forgery the world had known
Car hacking scandal: Security designed to stop thieves hot-wiring almost every modern motor has been cracked

Car hacking scandal

Security designed to stop thieves hot-wiring almost every modern motor has been cracked
10 best placemats

Take your seat: 10 best placemats

Protect your table and dine in style with a bold new accessory
Ashes 2015: Alastair Cook not the only one to be caught in The Oval mindwarp

Cook not the only one to be caught in The Oval mindwarp

Aussie skipper Michael Clarke was lured into believing that what we witnessed at Edgbaston and Trent Bridge would continue in London, says Kevin Garside
Can Rafael Benitez get the best out of Gareth Bale at Real Madrid?

Can Benitez get the best out of Bale?

Back at the club he watched as a boy, the pressure is on Benitez to find a winning blend from Real's multiple talents. As La Liga begins, Pete Jenson asks if it will be enough to stop Barcelona
Athletics World Championships 2015: Beijing witnesses new stage in the Jessica Ennis-Hill and Katarina Johnson-Thompson heptathlon rivalry

Beijing witnesses new stage in the Jess and Kat rivalry

The last time the two British heptathletes competed, Ennis-Hill was on the way to Olympic gold and Johnson-Thompson was just a promising teenager. But a lot has happened in the following three years
Jeremy Corbyn: Joining a shrewd operator desperate for power as he visits the North East

Jeremy Corbyn interview: A shrewd operator desperate for power

His radical anti-austerity agenda has caught the imagination of the left and politically disaffected and set a staid Labour leadership election alight
Isis executes Palmyra antiquities chief: Defender of ancient city's past was killed for protecting its future

Isis executes Palmyra antiquities chief

Robert Fisk on the defender of the ancient city's past who was killed for protecting its future