Lies and statistics? The pollsters who make the numbers add up

Predicting who'll win an election is an inexact business, not least because voters don't always tell the truth. Rob Sharp meets the pollsters who make the numbers add up

Do you remember where you were on 10 April 1992? Robert Worcester, founder of Ipsos MORI, certainly does. It was the day after John Major led the Conservatives to their fourth consecutive election victory.

None of the pollsters had seen it coming. The day before the election their data had suggested Labour would narrowly beat the Tories by one percentage point, the equivalent of eight Commons seats, resulting in a hung parliament. Instead, Major's party took an eight-percentage-point lead, or 65 seats, resulting in an overall majority of 21. It was one of the biggest upsets in polling history.

"It was chaos," says Worcester. "There were five principal polling organisations and three refused to talk to the media. I don't believe in maintaining silence, so I was one of the few people interviewed on TV and ended up bearing the brunt of the blame ... I said we didn't know what we'd done wrong, but that we wouldn't leave a stone unturned until we found out."

The Market Research Society, representing the pollsters, looked into what went wrong in 1992, and blamed inaccurate data about the socio-economic breakdown of the population.

As the number-crunchers gear up for another general election, are they now more confident of their data? This month polling company ComRes reported that the Tories had slumped to 38 per cent of the vote, just seven percentage points ahead of Labour, on 31 per cent, with the Liberal Democrats at 19 per cent. Like the predicted result for 1992, this would result in a hung parliament.

So how have ComRes arrived at this latest prediction? The raw data is these days is gathered over the phone (pollsters claim people are more likely to tell the truth than in face-to-face interviews). At the ComRes offices on Millbank, around 30 employees sit behind rows of computer monitors, some busy on the phone, others tapping numbers into databases. Most polling organisations will speak to a sample of 1,000 people per opinion poll. The bigger the sample, the better chance of accuracy. With a sample of 1,000 people, there is a 95 per cent chance of accuracy to within the margin of error, in this case plus or minus three per cent.

Most polling organisations select these 1,000 interviewees randomly from the records of telephone numbers. Of the major polling organisations – ComRes, ICM, Ipsos MORI, Populus and YouGov – only YouGov relies instead on samples taken from its own pool of 300,000 'panel members' – and conducts its polls online. But any sample of 1,000 people may be unrepresentative because it features, say, too many habitual Labour- voters, or too many people from lower-income groups, or too many from certain regions. So the pollsters put the responses they collect through a process of "weighting".

For example, if a sample of 1,000 people is made up of 550 men and 450 women, it doesn't accurately reflect the British population (51 per cent female). To compensate, the answers of female respondents will be given slightly more "weight" (in this case they will each count as 1.133 people). The same procedure is carried out to adjust for age, social class and region. Pollsters will also take into account how people voted in the previous election, and a person's statistical likelihood of actually casting his or her vote.

Pollsters then convert their results into seats in the House of Commons. In carrying out this conversion, online calculators such as that at electoral- calculus.co.uk can also factor in the effect of marginal constituencies, where a seat is most likely to change hands, as well as the effects of tactical voting. Back in 1992, the pollsters had inaccurate information about socio-economic groups partly because the latest census results were 11 years out-of-date; we also now know that many people who in the event voted Conservative lied to the pollsters about their voting intentions. These days pollsters argue that weighting according to past vote gets around that bias.

"Historically, over the longer term, pollsters have had a mixed record," admits ComRes founder Andrew Hawkins. "But in 2005, within 48 hours of the election every eve-of-election poll was [accurate to] within three per cent. This is strikingly accurate given that the journey from poll question to results is not a straightforward one. It's not like using a dipstick on a car engine. It's more of a modelling process." Every pollster, he says, should "show a willingness to change their methods over time."

Do we have any choice but to listen to the pollsters? Not really, says John Rentoul, The Independent on Sunday's chief political commentator: "We don't have any alternative way of finding out what the electorate is thinking, apart from focus groups, but they don't deliver the numbers." But some pollsters will never lay the ghosts of 1992 to rest.

"I'm not confident that we can't have another 1992 election, in the same way I can't be confident that journalists aren't going to make any more mistakes," says Robert Worcester. "You can't be 100 per cent sure that lightning isn't going to strike twice. It's statistics, and I spend a heap of my life explaining this. It's the marriage of asking questions with the science of sampling. I'd always say that it's 95 per cent expertise and five per cent luck. And if you're not lucky, then you're in the wrong business."

Start your day with The Independent, sign up for daily news emails
Sport
sportSo, how closely were you paying attention during 2014?
Arts and Entertainment
Dennis speaks to his French teacher
tvThe Boy in the Dress, TV review
News
One father who couldn't get One Direction tickets for his daughters phoned in a fake bomb threat and served eight months in a federal prison
people... (and one very unlucky giraffe)
Arts and Entertainment
Joel Edgerton, John Turturro and Christian Bale in Exodus: Gods and Kings
film
PROMOTED VIDEO
Have you tried new the Independent Digital Edition apps?
ebooks
ebooksA year of political gossip, levity and intrigue from the sharpest pen in Westminster
News
i100
Arts and Entertainment
The Plaza Theatre in Atlanta, Georgia was one of the 300 US cinemas screening
filmTim Walker settles down to watch the controversial gross-out satire
Arts and Entertainment
Amy Adams and Christoph Waltz in Tim Burton's Big Eyes
film reviewThis is Tim Burton’s most intimate and subtle film for a decade
Life and Style
Mark's crab tarts are just the right size
food + drinkMark Hix cooks up some snacks that pack a punch
Arts and Entertainment
Jack O'Connell stars as Louis Zamperini in Angelina Jolie's Unbroken
film review... even if Jack O'Connell is excellent
Latest stories from i100
Have you tried new the Independent Digital Edition apps?
Independent Dating
and  

By clicking 'Search' you
are agreeing to our
Terms of Use.

iJobs Job Widget
iJobs General

Ashdown Group: Senior Marketing Executive- City of London, Old Street

£40000 - £43000 per annum + benefits: Ashdown Group: Senior Marketing Executiv...

Ashdown Group: Marketing Manager

£40000 - £43000 per annum + benefits: Ashdown Group: An international organisa...

Ashdown Group: Internal Recruiter -Rugby, Warwickshire

£25000 - £30000 per annum: Ashdown Group: Internal Recruiter -Rugby, Warwicksh...

Ashdown Group: Marketing Manager/Marketing Controller (Financial Services)

£70000 - £75000 per annum + benefits: Ashdown Group: Marketing Manager/Marketi...

Day In a Page

Aren’t you glad you didn’t say that? The worst wince-and-look-away quotes of the year

Aren’t you glad you didn’t say that?

The worst wince-and-look-away quotes of the year
Hollande's vanity project is on a high-speed track to the middle of nowhere

Vanity project on a high-speed track to nowhere

France’s TGV network has become mired in controversy
Sports Quiz of the Year

Sports Quiz of the Year

So, how closely were you paying attention during 2014?
Alexander Armstrong on insulting Mary Berry, his love of 'Bargain Hunt', and life as a llama farmer

Alexander Armstrong on insulting Mary Berry and his love of 'Bargain Hunt'

From Armstrong and Miller to Pointless
Sanchez helps Gunners hold on after Giroud's moment of madness

Sanchez helps Gunners hold on

Olivier Giroud's moment of madness nearly costs them
A Christmas without hope: Fears grow in Gaza that the conflict with Israel will soon reignite

Christmas without hope

Gaza fears grow that conflict with Israel will soon reignite
After 150 years, you can finally visit the grisliest museum in the country

The 'Black Museum'

After 150 years, you can finally visit Britain's grisliest museum
No ho-ho-hos with Nick Frost's badass Santa

No ho-ho-hos with Nick Frost's badass Santa

Doctor Who Christmas Special TV review
Chilly Christmas: Swimmers take festive dip for charity

Chilly Christmas

Swimmers dive into freezing British waters for charity
Veterans' hostel 'overwhelmed by kindness' for festive dinner

Homeless Veterans appeal

In 2010, Sgt Gary Jamieson stepped on an IED in Afghanistan and lost his legs and an arm. He reveals what, and who, helped him to make a remarkable recovery
Isis in Iraq: Yazidi girls killing themselves to escape rape and imprisonment by militants

'Jilan killed herself in the bathroom. She cut her wrists and hanged herself'

Yazidi girls killing themselves to escape rape and imprisonment
Ed Balls interview: 'If I think about the deficit when I'm playing the piano, it all goes wrong'

Ed Balls interview

'If I think about the deficit when I'm playing the piano, it all goes wrong'
He's behind you, dude!

US stars in UK panto

From David Hasselhoff to Jerry Hall
Grace Dent's Christmas Quiz: What are you – a festive curmudgeon or top of the tree?

Grace Dent's Christmas Quiz

What are you – a festive curmudgeon or top of the tree?
Nasa planning to build cloud cities in airships above Venus

Nasa planning to build cloud cities in airships above Venus

Planet’s surface is inhospitable to humans but 30 miles above it is almost perfect