Nigel Hawkes: And our survey says...nothing you can rely on

Behind the numbers

Share
Related Topics

Aberdeen is the most sleep-deprived city in Britain, Bristol the unhealthiest, London the smelliest. The average person spends £2,776.75 a year on keeping healthy, and married couples achieve their peak happiness two years, 11 months and eight days after tying the knot.

These "facts" – and let us not forget the disclosure that 48 per cent of women admit faking an orgasm, and 9 per cent do it all the time – were all published by British newspapers last week. They were all the result of surveys, the quickest way to a news editor's heart.

Surveys are proliferating like Japanese knotweed. They borrow the authority of statistics to lend an air of truth to otherwise implausible claims. And, remarkably, all the surveys listed above come from the same survey factory, based in Bristol (Britain's unhealthiest place, remember?).

A regional news agency, South West News Service, offers a complete survey service. For £2,500 they will originate the questions, perform the survey, compile the results, write the press release, then report it as if it were news. That is a bargain if the company commissioning the survey gets a mention in Metro, the Daily Mail, the Daily Mirror, the Daily Express, the Evening Standard, the Press Association, The Sun, the Daily Record, or the Daily Star, all of which ran one or more of these surveys.

The surveys are organised by a subsidiary of SWNS, the PR agency 72 Point, and done by OnePoll, using a panel of volunteers who are paid (modestly) for their participation. They can turn a survey round in a couple of days. For an extra £300 you can motivate the panel to respond even more enthusiastically.

So what's wrong? In statistical terms, plenty. The samples for these online surveys look large – typically, 3,000 or so – but they are neither random nor necessarily representative of the public at large. The results are often broken down city by city, to attract interest from local papers or to compile league tables, when the samples for each city are far too small to make such comparisons valid. No confidence intervals are quoted.

And the results are often given an absurd precision, in the knowledge that journalists are easily seduced by an exact number. Claiming that the average person spends £2,776.75 a year keeping healthy sounds as if somebody counted every penny. In truth, nobody really counted anything at all, or not to an accuracy of better than the nearest £100 or so.

So these surveys are largely meaningless. But SWNS's website has testimonials from Tony Gallagher, Editor of The Daily Telegraph ("Its ability to deliver high-quality stories day after day is simply remarkable"), from Chris Pharo, head of news at The Sun, ("excellent survey-led news stories and stats") and, more worryingly, the Central Office of Information.

You might also worry, if you care for press independence, that quite such a cosy relationship between PR and news-gathering isn't healthy. But it's only a bit of fun, surely? "Poor show, chaps: survey shows that nearly one in 10 women fake it between the sheets", as the Daily Mail headlined one of the surveys. It's not meant to be taken seriously, so aren't my criticisms a bit priggish?

Not really. Almost all national statistics, from crime to the GDP, come from surveys designed with huge care to get as near as possible to the right answer. Newspaper surveys bear about as much relation to these as a chicken nugget does to a three-star Michelin lunch. Newspapers who give them space are selling themselves cheap.

Nigel Hawkes is Director of Straight Statistics

React Now

Latest stories from i100
Have you tried new the Independent Digital Edition apps?
iJobs Job Widget
iJobs General

Recruitment Genius: Finance Director

£65000 - £80000 per annum: Recruitment Genius: Finance Director required to jo...

Recruitment Genius: Medico-Legal Assistant

£15000 - £25000 per annum: Recruitment Genius: This is a unique opportunity fo...

Ashdown Group: (PHP / Python) - Global Media firm

£50000 per annum + 26 days holiday,pension: Ashdown Group: A highly successful...

The Jenrick Group: Quality Inspector

£27000 per annum + pension + holidays: The Jenrick Group: A Quality Technician...

Day In a Page

Read Next
David Cameron faces the press as he arrives in Brussels for the EU leaders summit on Thursday reuters  

On the Tusk of a dilemma: Cameron's latest EU renegotiation foe

Andrew Grice
John Profumo and his wife Valerie Robson in 1959  

Stephen Ward’s trial was disgraceful. There can be no justification for it

Geoffrey Robertson QC
Homeless Veterans appeal: 'You look for someone who's an inspiration and try to be like them'

Homeless Veterans appeal

In 2010, Sgt Gary Jamieson stepped on an IED in Afghanistan and lost his legs and an arm. He reveals what, and who, helped him to make a remarkable recovery
Could cannabis oil reverse the effects of cancer?

Could cannabis oil reverse effects of cancer?

As a film following six patients receiving the controversial treatment is released, Kate Hilpern uncovers a very slippery issue
The Interview movie review: You can't see Seth Rogen and James Franco's Kim Jong Un assassination film, but you can read about it here

The Interview movie review

You can't see Seth Rogen and James Franco's Kim Jong Un assassination film, but you can read about it here
Serial mania has propelled podcasts into the cultural mainstream

How podcasts became mainstream

People have consumed gripping armchair investigation Serial with a relish typically reserved for box-set binges
Jesus Christ has become an unlikely pin-up for hipster marketing companies

Jesus Christ has become an unlikely pin-up

Kevin Lee Light, aka "Jesus", is the newest client of creative agency Mother while rival agency Anomaly has launched Sexy Jesus, depicting the Messiah in a series of Athena-style poses
Rosetta space mission voted most important scientific breakthrough of 2014

A memorable year for science – if not for mice

The most important scientific breakthroughs of 2014
Christmas cocktails to make you merry: From eggnog to Brown Betty and Rum Bumpo

Christmas cocktails to make you merry

Mulled wine is an essential seasonal treat. But now drinkers are rediscovering other traditional festive tipples. Angela Clutton raises a glass to Christmas cocktails
5 best activity trackers

Fitness technology: 5 best activity trackers

Up the ante in your regimen and change the habits of a lifetime with this wearable tech
Paul Scholes column: It's a little-known fact, but I have played one of the seven dwarves

Paul Scholes column

It's a little-known fact, but I have played one of the seven dwarves
Fifa's travelling circus once again steals limelight from real stars

Fifa's travelling circus once again steals limelight from real stars

Club World Cup kicked into the long grass by the continued farce surrounding Blatter, Garcia, Russia and Qatar
Frank Warren column: 2014 – boxing is back and winning new fans

Frank Warren: Boxing is back and winning new fans

2014 proves it's now one of sport's biggest hitters again
Jeb Bush vs Hillary Clinton: The power dynamics of the two first families

Jeb Bush vs Hillary Clinton

Karen Tumulty explores the power dynamics of the two first families
Stockholm is rivalling Silicon Valley with a hotbed of technology start-ups

Stockholm is rivalling Silicon Valley

The Swedish capital is home to two of the most popular video games in the world, as well as thousands of technology start-ups worth hundreds of millions of pounds – and it's all happened since 2009
Did Japanese workers really get their symbols mixed up and display Santa on a crucifix?

Crucified Santa: Urban myth refuses to die

The story goes that Japanese store workers created a life-size effigy of a smiling "Father Kurisumasu" attached to a facsimile of Our Lord's final instrument of torture
Jennifer Saunders and Kate Moss join David Walliams on set for TV adaptation of The Boy in the Dress

The Boy in the Dress: On set with the stars

Walliams' story about a boy who goes to school in a dress will be shown this Christmas