Nigel Hawkes: And our survey says...nothing you can rely on

Behind the numbers

Share
Related Topics

Aberdeen is the most sleep-deprived city in Britain, Bristol the unhealthiest, London the smelliest. The average person spends £2,776.75 a year on keeping healthy, and married couples achieve their peak happiness two years, 11 months and eight days after tying the knot.

These "facts" – and let us not forget the disclosure that 48 per cent of women admit faking an orgasm, and 9 per cent do it all the time – were all published by British newspapers last week. They were all the result of surveys, the quickest way to a news editor's heart.

Surveys are proliferating like Japanese knotweed. They borrow the authority of statistics to lend an air of truth to otherwise implausible claims. And, remarkably, all the surveys listed above come from the same survey factory, based in Bristol (Britain's unhealthiest place, remember?).

A regional news agency, South West News Service, offers a complete survey service. For £2,500 they will originate the questions, perform the survey, compile the results, write the press release, then report it as if it were news. That is a bargain if the company commissioning the survey gets a mention in Metro, the Daily Mail, the Daily Mirror, the Daily Express, the Evening Standard, the Press Association, The Sun, the Daily Record, or the Daily Star, all of which ran one or more of these surveys.

The surveys are organised by a subsidiary of SWNS, the PR agency 72 Point, and done by OnePoll, using a panel of volunteers who are paid (modestly) for their participation. They can turn a survey round in a couple of days. For an extra £300 you can motivate the panel to respond even more enthusiastically.

So what's wrong? In statistical terms, plenty. The samples for these online surveys look large – typically, 3,000 or so – but they are neither random nor necessarily representative of the public at large. The results are often broken down city by city, to attract interest from local papers or to compile league tables, when the samples for each city are far too small to make such comparisons valid. No confidence intervals are quoted.

And the results are often given an absurd precision, in the knowledge that journalists are easily seduced by an exact number. Claiming that the average person spends £2,776.75 a year keeping healthy sounds as if somebody counted every penny. In truth, nobody really counted anything at all, or not to an accuracy of better than the nearest £100 or so.

So these surveys are largely meaningless. But SWNS's website has testimonials from Tony Gallagher, Editor of The Daily Telegraph ("Its ability to deliver high-quality stories day after day is simply remarkable"), from Chris Pharo, head of news at The Sun, ("excellent survey-led news stories and stats") and, more worryingly, the Central Office of Information.

You might also worry, if you care for press independence, that quite such a cosy relationship between PR and news-gathering isn't healthy. But it's only a bit of fun, surely? "Poor show, chaps: survey shows that nearly one in 10 women fake it between the sheets", as the Daily Mail headlined one of the surveys. It's not meant to be taken seriously, so aren't my criticisms a bit priggish?

Not really. Almost all national statistics, from crime to the GDP, come from surveys designed with huge care to get as near as possible to the right answer. Newspaper surveys bear about as much relation to these as a chicken nugget does to a three-star Michelin lunch. Newspapers who give them space are selling themselves cheap.

Nigel Hawkes is Director of Straight Statistics

React Now

Latest stories from i100
Have you tried new the Independent Digital Edition apps?
iJobs Job Widget
iJobs General

Operational Risk Manager - Asset Management

£60,000 - £80,000: Saxton Leigh: Our client is an leading Asset Manager based...

Year 5/6 Teacher

Negotiable: Randstad Education Sheffield: Permanent Year 6 TeacherThe job:This...

KS1 & KS2 Teachers

Negotiable: Randstad Education Sheffield: KS1+KS2 Teachers required ASAP for l...

Year 2 Teacher

Negotiable: Randstad Education Sheffield: Year 2 Teacher The position is to wo...

Day In a Page

 

In Sickness and in Health: Waking up to my 4am witching hour of worry

Rebecca Armstrong
Oscar Pistorius sentencing: The athlete's wealth and notoriety have provoked a long overdue debate on South African prisons

'They poured water on, then electrified me...'

If Oscar Pistorius is sent to jail, his experience will not be that of other inmates
James Wharton: The former Guard now fighting discrimination against gay soldiers

The former Guard now fighting discrimination against gay soldiers

Life after the Army has brought new battles for the LGBT activist James Wharton
Ebola in the US: Panic over the virus threatens to infect President Obama's midterms

Panic over Ebola threatens to infect the midterms

Just one person has died, yet November's elections may be affected by what Republicans call 'Obama's Katrina', says Rupert Cornwell
Premier League coaches join the RSC to swap the tricks of their trades

Darling, you were fabulous! But offside...

Premier League coaches are joining the RSC to learn acting skills, and in turn they will teach its actors to play football. Nick Clark finds out why
How to dress with authority: Kirsty Wark and Camila Batmanghelidjh discuss the changing role of fashion in women's workwear

How to dress with authority

Kirsty Wark and Camila Batmanghelidjh discuss the changing role of fashion in women's workwear
New book on Joy Division's Ian Curtis sheds new light on the life of the late singer

New book on Ian Curtis sheds fresh light on the life of the late singer

'Joy Division were making art... Ian was for real' says author Jon Savage
Sean Harris: A rare interview with British acting's secret weapon

Sean Harris: A rare interview with British acting's secret weapon

The Bafta-winner talks Hollywood, being branded a psycho, and how Barbra Streisand is his true inspiration
Tim Minchin, interview: The musician, comedian and world's favourite ginger is on scorching form

Tim Minchin interview

For a no-holds-barred comedian who is scathing about woolly thinking and oppressive religiosity, he is surprisingly gentle in person
Boris Johnson's boozing won't win the puritan vote

Boris's boozing won't win the puritan vote

Many of us Brits still disapprove of conspicuous consumption – it's the way we were raised, says DJ Taylor
Ash frontman Tim Wheeler reveals how he came to terms with his father's dementia

Tim Wheeler: Alzheimer's, memories and my dad

Wheeler's dad suffered from Alzheimer's for three years. When he died, there was only one way the Ash frontman knew how to respond: with a heartfelt solo album
Hugh Bonneville & Peter James: 'Peter loves his classic cars; I've always pootled along fine with a Mini Metro. I think I lack his panache'

How We Met: Hugh Bonneville & Peter James

'Peter loves his classic cars; I've always pootled along fine with a Mini Metro. I think I lack his panache'
Bill Granger recipes: Our chef's heavenly crab dishes don't need hours of preparation

Bill Granger's heavenly crab recipes

Scared off by the strain of shelling a crab? Let a fishmonger do the hard work so you can focus on getting the flavours right
Radamel Falcao: How faith and love drive the Colombian to glory

Radamel Falcao: How faith and love drive the Colombian to glory

After a remarkable conversion from reckless defender to prolific striker, Monaco's ace says he wants to make his loan deal at Old Trafford permanent
Terry Venables: Premier League managers must not be allowed to dictate who plays and who does not play for England

Terry Venables column

Premier League managers must not be allowed to dictate who plays and who does not play for England
The Inside Word: Brendan Rodgers looks to the future while Roy Hodgson is ghost of seasons past

Michael Calvin's Inside Word

Brendan Rodgers looks to the future while Roy Hodgson is ghost of seasons past