Nigel Hawkes: And our survey says...nothing you can rely on

Behind the numbers

Share
Related Topics

Aberdeen is the most sleep-deprived city in Britain, Bristol the unhealthiest, London the smelliest. The average person spends £2,776.75 a year on keeping healthy, and married couples achieve their peak happiness two years, 11 months and eight days after tying the knot.

These "facts" – and let us not forget the disclosure that 48 per cent of women admit faking an orgasm, and 9 per cent do it all the time – were all published by British newspapers last week. They were all the result of surveys, the quickest way to a news editor's heart.

Surveys are proliferating like Japanese knotweed. They borrow the authority of statistics to lend an air of truth to otherwise implausible claims. And, remarkably, all the surveys listed above come from the same survey factory, based in Bristol (Britain's unhealthiest place, remember?).

A regional news agency, South West News Service, offers a complete survey service. For £2,500 they will originate the questions, perform the survey, compile the results, write the press release, then report it as if it were news. That is a bargain if the company commissioning the survey gets a mention in Metro, the Daily Mail, the Daily Mirror, the Daily Express, the Evening Standard, the Press Association, The Sun, the Daily Record, or the Daily Star, all of which ran one or more of these surveys.

The surveys are organised by a subsidiary of SWNS, the PR agency 72 Point, and done by OnePoll, using a panel of volunteers who are paid (modestly) for their participation. They can turn a survey round in a couple of days. For an extra £300 you can motivate the panel to respond even more enthusiastically.

So what's wrong? In statistical terms, plenty. The samples for these online surveys look large – typically, 3,000 or so – but they are neither random nor necessarily representative of the public at large. The results are often broken down city by city, to attract interest from local papers or to compile league tables, when the samples for each city are far too small to make such comparisons valid. No confidence intervals are quoted.

And the results are often given an absurd precision, in the knowledge that journalists are easily seduced by an exact number. Claiming that the average person spends £2,776.75 a year keeping healthy sounds as if somebody counted every penny. In truth, nobody really counted anything at all, or not to an accuracy of better than the nearest £100 or so.

So these surveys are largely meaningless. But SWNS's website has testimonials from Tony Gallagher, Editor of The Daily Telegraph ("Its ability to deliver high-quality stories day after day is simply remarkable"), from Chris Pharo, head of news at The Sun, ("excellent survey-led news stories and stats") and, more worryingly, the Central Office of Information.

You might also worry, if you care for press independence, that quite such a cosy relationship between PR and news-gathering isn't healthy. But it's only a bit of fun, surely? "Poor show, chaps: survey shows that nearly one in 10 women fake it between the sheets", as the Daily Mail headlined one of the surveys. It's not meant to be taken seriously, so aren't my criticisms a bit priggish?

Not really. Almost all national statistics, from crime to the GDP, come from surveys designed with huge care to get as near as possible to the right answer. Newspaper surveys bear about as much relation to these as a chicken nugget does to a three-star Michelin lunch. Newspapers who give them space are selling themselves cheap.

Nigel Hawkes is Director of Straight Statistics

React Now

Latest stories from i100
Have you tried new the Independent Digital Edition apps?
iJobs Job Widget
iJobs General

Offshore Engineering Design Manager

£50000 - £60000 Per Annum: The Green Recruitment Company: The Green Recruitmen...

Offshore Wind Grid Connection Specialist

£50000 - £60000 Per Annum: The Green Recruitment Company: The Green Recruitmen...

SEN Science Teacher

£21804 - £31868 per annum + SEN allowance: Randstad Education Chelmsford: Are ...

History Teacher

£90 - £120 per day: Randstad Education Hull: Randstad Education are looking fo...

Day In a Page

Read Next
A Palestinian man carries the body of a one-year-old baby Noha Mesleh, who died of wounds sustained after a UN school in Beit Hanun was hit by an Israeli tank shell  

Gaza is too busy burying its children to report on its injured owls

Mira Bar Hillel
Abd Mubin Rahim of Malaysia falls to the floor after an unsuccessful lift during the Men's Weightlifting  

Usain Bolt was right about the Commonwealth Games, but we shouldn't blame the organisers

Teddy Cutler
Dress the Gaza situation up all you like, but the truth hurts

Robert Fisk on Gaza conflict

Dress the situation up all you like, but the truth hurts
Save the tiger: Tiger, tiger burning less brightly as numbers plummet

Tiger, tiger burning less brightly

When William Blake wrote his famous poem there were probably more than 100,000 tigers in the wild. These days they probably number around 3,200
5 News's Andy Bell retraces his grandfather's steps on the First World War battlefields

In grandfather's footsteps

5 News's political editor Andy Bell only knows his grandfather from the compelling diary he kept during WWI. But when he returned to the killing fields where Edwin Vaughan suffered so much, his ancestor came to life
Lifestyle guru Martha Stewart reveals she has flying robot ... to take photos of her farm

Martha Stewart has flying robot

The lifestyle guru used the drone to get a bird's eye view her 153-acre farm in Bedford, New York
Former Labour minister Meg Hillier has demanded 'pootling lanes' for women cyclists

Do women cyclists need 'pootling lanes'?

Simon Usborne (who's more of a hurtler) explains why winning the space race is key to happy riding
A tale of two presidents: George W Bush downs his paintbrush to pen father’s life story

A tale of two presidents

George W Bush downs his paintbrush to pen father’s life story
Restaurateur Mitch Tonks has given the Great Western Pullman dining car a makeover

The dining car makes a comeback

Restaurateur Mitch Tonks has given the Great Western Pullman dining car a makeover
Gallery rage: How are institutions tackling the discomfort of overcrowding this summer?

Gallery rage

How are institutions tackling the discomfort of overcrowding this summer?
Louis van Gaal has £500,000 video surveillance system installed to monitor Manchester United players

Eye on the prize

Louis van Gaal has £500,000 video surveillance system installed to monitor Manchester United players
Women's rugby: Tamara Taylor adds fuel to the ire in quest to land World Cup

Women's rugby

Tamara Taylor adds fuel to the ire in quest to land World Cup
Save the tiger: The day America’s love of backyard tigers led to a horrific bloodbath

The day America’s love of backyard tigers led to a horrific bloodbath

With only six per cent of the US population of these amazing big cats held in zoos, the Zanesville incident in 2011 was inevitable
Samuel Beckett's biographer reveals secrets of the writer's time as a French Resistance spy

How Samuel Beckett became a French Resistance spy

As this year's Samuel Beckett festival opens in Enniskillen, James Knowlson, recalls how the Irish writer risked his life for liberty and narrowly escaped capture by the Gestapo
We will remember them: relatives still honour those who fought in the Great War

We will remember them

Relatives still honour those who fought in the Great War
Star Wars Episode VII is being shot on film - and now Kodak is launching a last-ditch bid to keep celluloid alive

Kodak's last-ditch bid to keep celluloid alive

Director J J Abrams and a few digital refuseniks shoot movies on film. Simon Usborne wonders what the fuss is about
Once stilted and melodramatic, Hollywood is giving acting in video games a makeover

Acting in video games gets a makeover

David Crookes meets two of the genre's most popular voices