Big data is the future – but will it helps us to see the future?

The annuity business in particular has become a blatant rip-off: call it twilight robbery


Manchester has a lot to celebrate this week. Forget Robin van Persie’s hat-trick for United in the Champions League. Today the much-loved Central Library, whose rotunda has nurtured Mancunian talent since 1934, reopens after its £48m restoration. Wednesday’s Budget saw George Osborne announce further funds for research into graphene, the atom-thick super-material isolated at the city’s university by Andre Geim and Konstantin Novoselov in 2003. The Chancellor also unveiled the Alan Turing Institute for Data Science, backed by £42m of public money over five years. It will serve as a national memorial to the mathematician, code-breaker and computer pioneer who laid the intellectual foundations for modern life at the university’s Computing Laboratory.

It counts as progress of another sort that a Tory chancellor can pay tribute to Turing as a hero “persecuted for his sexuality by the country he helped to save”. Osborne wants the planned institute (which surely ought to be sited in Manchester) to act as the ultimate public-sector number-cruncher, a hive of mega-scale analytics “to ensure Britain leads the way again in the use of big data and algorithm research”. “Big Data” has become an ever-louder hi-tech buzzword since analysts such as Doug Laney began to spread the term a decade ago. It entails the gathering and organisation of the huge volumes of information spawned by a wired world to track, understand and – here’s the big “if” – possibly predict outcomes. In some utopian minds, big-data assessment promises a hard new branch of futurology. Others treat it as a smarter way of finding out what has happened to your parcel. (UPS was a leader in applying massive data sets to real-world problems of logistics.)

After the Budget, I asked Professor Viktor Mayer-Schönberger of Oxford University’s Internet Institute – co-author with Kenneth Cukier of the pioneer study Big Data – about the potential of the Turing centre. He says that “with big data we’ll have quite a novel way of making sense of reality (and derived from that, make often quite accurate predictions about the future)”. Yet “the processes by which we currently collect, analyse and make sense … are based on small data”, stuck in the time when information-harvesting was “expensive and time-consuming”. For him, “it will be like devising and learning a new language of science”. That will include “how we tell what we find – data visualisations are likely the new narrative, but how do we do that well?”. He hopes the new centre “will focus on this new language of science, much like Alan Turing helped us devise a language of logic”.

One of big data’s breakthroughs came in 2009 when it emerged that Google search records could track the likely path of new flu strains better than snail’s-pace reporting from public health laboratories. US star pollster (or poll analyst) Nate Silver delivered a masterclass in new-wave data-crunching when his cult site FiveThirtyEight forecast all 50 states correctly in the 2012 presidential election. At the time, the flaky augurers of the traditional media were still inspecting the chicken entrails and even predicting triumph for Mitt Romney (who?). By election day, FiveThirtyEight called for Obama with a 91 per cent probability.

Alan Turing, who was convicted of gross indecency in 1952, was granted a royal pardon last year Alan Turing, who was convicted of gross indecency in 1952, was granted a royal pardon last year














Yet Silver is notably cautious about the predictive power of big-data research. He argues that uncritical immersion in vast vats of poor-quality stuff can lead to “more distraction, more false positives, more bias and more reliance on machine-learning”. In understanding the present, let alone divining what’s to come, size alone does not matter. As Silver’s book The Signal and the Noise puts it, we can merely “strive to be less subjective, less irrational, and less wrong”. Although he did once do some work on dating data, and the correlation between chosen nights and preferred outcomes. “On Wednesday, apparently, you get the highest ratio of people who are just looking for something quick and dirty.”

Behind the popular fascination with this often arcane corner of science lies our perennial itch to know the future and prepare for it. That universal urge also underlay another plank in Osborne’s Budget speech. By liberating pensioners from the need to convert their savings into a lifetime annuity, he also challenged the status of the 300-year-old science – or semi-science – of actuarial prediction. This was long overdue. Here’s my forecast: we will pretty soon look back on the time when people in later life had to trust their security to Britain’s greedy, incompetent and often (look at the mis-selling scandals) corrupt pensions industry as evidence of a near-criminal conspiracy. The annuity business in particular has become a blatant rip-off: call it twilight robbery.

Yet behind this besuited con trick stands predictive mathematics not that different in its day from the soothsaying coups of the number-crunchers now. Life tables to measure survival chances go back to the 1660s. It was Edmond Halley (of comet fame) who, in 1693, worked out how to calculate the premiums for annuities from them. Actuaries became in effect the first professional guardians and analysts of “big data” sets. Their theorems still draw the outlines of retirement for millions.

The British conspiracy that Osborne broke this week had a peculiar horror. It yoked the actuarial approach to dependence on the vicissitudes of bank rates and the fluctuations of the markets. Who in their right mind would ever devise such a system as the motor for comfort in old age? British institutions – so often in the iron grip of City interests – speak with a forked tongue. On the one hand, they exhort us to plan for the future, to squirrel away funds against rainy – or idle – days, and to assume that we have the power to steer our financial destiny. On the other, they effectively require us to chuck our nest eggs at the roulette wheel of casino capitalism.

No wonder the UK savings ratio is so low. Or that millions of people in Britain believe that the “big data” sets of the past century yield one message: invest in property. Hence periodic house-price booms, and the mass faith in lavish profits from a hard-to-climb “housing ladder”. As Danny Dorling’s landmark critique of our property delirium All That Is Solid makes clear, we have fallen for a giant Ponzi swindle. “Climb on board and you too will win at the expense of some sucker below you. But all Ponzi schemes eventually fail. Everyone cannot get rich quick at the expense of future property buyers.”

Back to big data and the tasks that await the Alan Turing Institute. All the indicators that impact on everyday lives – housing costs, savings returns, pension expectations – demand that we stake our chips on a fuzzy future. No amount of number-crunching by super-computers will ever bring that picture into a perfectly clear focus. In Big Data, Mayer-Schönberger and Cukier insist that the quasi-prophetic powers of information in oceanic volumes will never cover every angle and option. We must still attend to “the empty space, the cracks in the sidewalk, the unspoken and as-yet unthought”. Many people due to retire soon will have grown up listening to Doris Day warbling “Que Sera Sera”: “the future’s not ours to see”. Indeed not.

At present, we blunder about in a fog of imperfect and partisan information. Out of this mist we have to make future-limiting decisions. The lifting of the annuities yoke will only deepen that sense of uncertainty. George Osborne announced a new “right to impartial advice” for investors. He said little about the mechanism of its delivery.

As yet, most comment about the proposed Turing Institute has spoken of its benefits to business. Why not enlist it in the public service too? Democratise big data. Recruit it not merely to target the consumer, Google-style, but to empower the citizen. In theory, government backs public access to high-value information through projects such as the NHS Hospital Episodes Statistics. It even funds an Open Data Institute based – inevitably – in Shoreditch. Yet the debate about data use always turns on commercial applications.

If a cash-strapped government wishes to step back from cradle-to-grave obligations, then at least give everybody guided access to its ample resources of data computation to inform the extra choices that devolve on us. Between 1908 and 1911, the fledgling philosopher Ludwig Wittgenstein studied aeronautical engineering in – where else? – Manchester. (He even patented the prototype for a helicopter engine.) In a gnomic maxim, Wittgenstein later said: “If a lion could speak, we could not understand him.” The digitised world now blasts waves of raw information around us in a deafening lion’s roar. A new research hub should allow us all to interpret it, act on it, and plan possible futures. As well as harnessing its power for profit, the state could help every citizen hear – and tame – the lion of big data.

React Now

  • Get to the point
Latest stories from i100
Have you tried new the Independent Digital Edition apps?
iJobs Job Widget
iJobs General

Recruitment Genius: Project Implementation Executive

£18000 - £23000 per annum: Recruitment Genius: They work with major vehicle ma...

Recruitment Genius: Chiropractic Assistant

£16500 per annum: Recruitment Genius: A Chiropractic Assistant is needed in a ...

Recruitment Genius: Digital Account Executive - Midlands

£18000 - £26000 per annum: Recruitment Genius: They work with major vehicle ma...

Recruitment Genius: Web Developer

£28000 - £30000 per annum: Recruitment Genius: This company provides coaching ...

Day In a Page

Read Next
With an eye for strategy: Stephen Fry’s General Melchett and Rowan Atkinson’s Edmund Blackadder  

What Cameron really needs is to turn this into a khaki election

Matthew Norman
An Italian policeman stands guard as migrants eat while waiting at the port of Lampedusa to board a ferry bound for Porto Empedocle in Sicily. Authorities on the Italian island of Lampedusa struggled to cope with a huge influx of newly-arrived migrants as aid organisations warned the Libya crisis means thousands more could be on their way  

Migrant boat disaster: EU must commit funds to stop many more dying

Alistair Dawber
NHS struggling to monitor the safety and efficacy of its services outsourced to private providers

Who's monitoring the outsourced NHS services?

A report finds that private firms are not being properly assessed for their quality of care
Zac Goldsmith: 'I'll trigger a by-election over Heathrow'

Zac Goldsmith: 'I'll trigger a by-election over Heathrow'

The Tory MP said he did not want to stand again unless his party's manifesto ruled out a third runway. But he's doing so. Watch this space
How do Greek voters feel about Syriza's backtracking on its anti-austerity pledge?

How do Greeks feel about Syriza?

Five voters from different backgrounds tell us what they expect from Syriza's charismatic leader Alexis Tsipras
From Iraq to Libya and Syria: The wars that come back to haunt us

The wars that come back to haunt us

David Cameron should not escape blame for his role in conflicts that are still raging, argues Patrick Cockburn
Sam Baker and Lauren Laverne: Too busy to surf? Head to The Pool

Too busy to surf? Head to The Pool

A new website is trying to declutter the internet to help busy women. Holly Williams meets the founders
Heston Blumenthal to cook up a spice odyssey for British astronaut manning the International Space Station

UK's Major Tum to blast off on a spice odyssey

Nothing but the best for British astronaut as chef Heston Blumenthal cooks up his rations
John Harrison's 'longitude' clock sets new record - 300 years on

‘Longitude’ clock sets new record - 300 years on

Greenwich horologists celebrate as it keeps to within a second of real time over a 100-day test
Fears in the US of being outgunned in the vital propaganda wars by Russia, China - and even Isis - have prompted a rethink on overseas broadcasters

Let the propaganda wars begin - again

'Accurate, objective, comprehensive': that was Voice of America's creed, but now its masters want it to promote US policy, reports Rupert Cornwell
Why Japan's incredible long-distance runners will never win the London Marathon

Japan's incredible long-distance runners

Every year, Japanese long-distance runners post some of the world's fastest times – yet, come next weekend, not a single elite competitor from the country will be at the London Marathon
Why does Tom Drury remain the greatest writer you've never heard of?

Tom Drury: The quiet American

His debut was considered one of the finest novels of the past 50 years, and he is every bit the equal of his contemporaries, Jonathan Franzen, Dave Eggers and David Foster Wallace
You should judge a person by how they peel a potato

You should judge a person by how they peel a potato

Dave Hax's domestic tips are reminiscent of George Orwell's tea routine. The world might need revolution, but we like to sweat the small stuff, says DJ Taylor
Beige is back: The drab car colours of the 1970s are proving popular again

Beige to the future

Flares and flounce are back on catwalks but a revival in ’70s car paintjobs was a stack-heeled step too far – until now
Bill Granger recipes: Our chef's dishes highlight the delicate essence of fresh cheeses

Bill Granger cooks with fresh cheeses

More delicate on the palate, milder, fresh cheeses can also be kinder to the waistline
Aston Villa vs Liverpool: 'This FA Cup run has been wonderful,' says veteran Shay Given

Shay Given: 'This FA Cup run has been wonderful'

The Villa keeper has been overlooked for a long time and has unhappy memories of the national stadium – but he is savouring his chance to play at Wembley
Timeless drama of Championship race in league of its own - Michael Calvin

Michael Calvin's Last Word

Timeless drama of Championship race in league of its own