Nigel Hawkes: And our survey says...nothing you can rely on

Behind the numbers

Share
Related Topics

Aberdeen is the most sleep-deprived city in Britain, Bristol the unhealthiest, London the smelliest. The average person spends £2,776.75 a year on keeping healthy, and married couples achieve their peak happiness two years, 11 months and eight days after tying the knot.

These "facts" – and let us not forget the disclosure that 48 per cent of women admit faking an orgasm, and 9 per cent do it all the time – were all published by British newspapers last week. They were all the result of surveys, the quickest way to a news editor's heart.

Surveys are proliferating like Japanese knotweed. They borrow the authority of statistics to lend an air of truth to otherwise implausible claims. And, remarkably, all the surveys listed above come from the same survey factory, based in Bristol (Britain's unhealthiest place, remember?).

A regional news agency, South West News Service, offers a complete survey service. For £2,500 they will originate the questions, perform the survey, compile the results, write the press release, then report it as if it were news. That is a bargain if the company commissioning the survey gets a mention in Metro, the Daily Mail, the Daily Mirror, the Daily Express, the Evening Standard, the Press Association, The Sun, the Daily Record, or the Daily Star, all of which ran one or more of these surveys.

The surveys are organised by a subsidiary of SWNS, the PR agency 72 Point, and done by OnePoll, using a panel of volunteers who are paid (modestly) for their participation. They can turn a survey round in a couple of days. For an extra £300 you can motivate the panel to respond even more enthusiastically.

So what's wrong? In statistical terms, plenty. The samples for these online surveys look large – typically, 3,000 or so – but they are neither random nor necessarily representative of the public at large. The results are often broken down city by city, to attract interest from local papers or to compile league tables, when the samples for each city are far too small to make such comparisons valid. No confidence intervals are quoted.

And the results are often given an absurd precision, in the knowledge that journalists are easily seduced by an exact number. Claiming that the average person spends £2,776.75 a year keeping healthy sounds as if somebody counted every penny. In truth, nobody really counted anything at all, or not to an accuracy of better than the nearest £100 or so.

So these surveys are largely meaningless. But SWNS's website has testimonials from Tony Gallagher, Editor of The Daily Telegraph ("Its ability to deliver high-quality stories day after day is simply remarkable"), from Chris Pharo, head of news at The Sun, ("excellent survey-led news stories and stats") and, more worryingly, the Central Office of Information.

You might also worry, if you care for press independence, that quite such a cosy relationship between PR and news-gathering isn't healthy. But it's only a bit of fun, surely? "Poor show, chaps: survey shows that nearly one in 10 women fake it between the sheets", as the Daily Mail headlined one of the surveys. It's not meant to be taken seriously, so aren't my criticisms a bit priggish?

Not really. Almost all national statistics, from crime to the GDP, come from surveys designed with huge care to get as near as possible to the right answer. Newspaper surveys bear about as much relation to these as a chicken nugget does to a three-star Michelin lunch. Newspapers who give them space are selling themselves cheap.

Nigel Hawkes is Director of Straight Statistics

React Now

  • Get to the point
Latest stories from i100
Have you tried new the Independent Digital Edition apps?
iJobs Job Widget
iJobs General

Recruitment Genius: Junior Web Designer - Client Liaison

£6 per hour: Recruitment Genius: This is an exciting opportunity to join a gro...

Recruitment Genius: Service Delivery Manager

Negotiable: Recruitment Genius: A Service Delivery Manager is required to join...

Recruitment Genius: Massage Therapist / Sports Therapist

£12000 - £24000 per annum: Recruitment Genius: A opportunity has arisen for a ...

Ashdown Group: Practice Accountant - Bournemouth - £38,000

£32000 - £38000 per annum: Ashdown Group: A successful accountancy practice in...

Day In a Page

Read Next
 

Errors & Omissions: A widow’s tale with an unexpected twist

John Rentoul
 

For all his faults, Russell Brand is utterly sincere, something politicians should emulate

Janet Street-Porter
Election 2015: How many of the Government's coalition agreement promises have been kept?

Promises, promises

But how many coalition agreement pledges have been kept?
The Gaza fisherman who built his own reef - and was shot dead there by an Israeli gunboat

The death of a Gaza fisherman

He built his own reef, and was fatally shot there by an Israeli gunboat
Saudi Arabia's airstrikes in Yemen are fuelling the Gulf's fire

Saudi airstrikes are fuelling the Gulf's fire

Arab intervention in Yemen risks entrenching Sunni-Shia divide and handing a victory to Isis, says Patrick Cockburn
Zayn Malik's departure from One Direction shows the perils of fame in the age of social media

The only direction Zayn could go

We wince at the anguish of One Direction's fans, but Malik's departure shows the perils of fame in the age of social media
Young Magician of the Year 2015: Meet the schoolgirl from Newcastle who has her heart set on being the competition's first female winner

Spells like teen spirit

A 16-year-old from Newcastle has set her heart on being the first female to win Young Magician of the Year. Jonathan Owen meets her
Jonathan Anderson: If fashion is a cycle, this young man knows just how to ride it

If fashion is a cycle, this young man knows just how to ride it

British designer Jonathan Anderson is putting his stamp on venerable house Loewe
Number plates scheme could provide a licence to offend in the land of the free

Licence to offend in the land of the free

Cash-strapped states have hit on a way of making money out of drivers that may be in collision with the First Amendment, says Rupert Cornwell
From farm to fork: Meet the Cornish fishermen, vegetable-growers and butchers causing a stir in London's top restaurants

From farm to fork in Cornwall

One man is bringing together Cornwall's most accomplished growers, fishermen and butchers with London's best chefs to put the finest, freshest produce on the plates of some of the country’s best restaurants
Don't believe the stereotype - or should you?

Don't believe the stereotype - or should you?

We exaggerate regional traits and turn them into jokes - and those on the receiving end are in on it too, DJ Taylor
How to make your own Easter egg: Willie Harcourt-Cooze shares his chocolate recipes

How to make your own Easter egg

Willie Harcourt-Cooze talks about his love affair with 'cacao' - and creates an Easter egg especially for The Independent on Sunday
Bill Granger recipes: Our chef declares barbecue season open with his twist on a tradtional Easter Sunday lamb lunch

Bill Granger's twist on Easter Sunday lunch

Next weekend, our chef plans to return to his Aussie roots by firing up the barbecue
Joe Marler: 'It's the way I think the game should be played'

Joe Marler: 'It's the way I think the game should be played'

The England prop relives the highs and lows of last Saturday's remarkable afternoon of Six Nations rugby
Cricket World Cup 2015: Has the success of the tournament spelt the end for Test matches?

Cricket World Cup 2015

Has the success of the tournament spelt the end for Test matches?
The Last Word: Justin Gatlin knows the price of everything, the value of nothing

Michael Calvin's Last Word

Justin Gatlin knows the price of everything, the value of nothing