Stephen Hawking: 'Transcendence looks at the implications of artificial intelligence - but are we taking AI seriously enough?'

Success in creating AI would be the biggest event in human history. Unfortunately, it might also be the last, unless we learn how to avoid the risks, says a group of leading scientists

With the Hollywood blockbuster Transcendence playing in cinemas, with Johnny Depp and Morgan Freeman showcasing clashing visions for the future of humanity, it's tempting to dismiss the notion of highly intelligent machines as mere science fiction. But this would be a mistake, and potentially our worst mistake in history.

Artificial-intelligence (AI) research is now progressing rapidly. Recent landmarks such as self-driving cars, a computer winning at Jeopardy! and the digital personal assistants Siri, Google Now and Cortana are merely symptoms of an IT arms race fuelled by unprecedented investments and building on an increasingly mature theoretical foundation. Such achievements will probably pale against what the coming decades will bring.

The potential benefits are huge; everything that civilisation has to offer is a product of human intelligence; we cannot predict what we might achieve when this intelligence is magnified by the tools that AI may provide, but the eradication of war, disease, and poverty would be high on anyone's list. Success in creating AI would be the biggest event in human history.

Unfortunately, it might also be the last, unless we learn how to avoid the risks. In the near term, world militaries are considering autonomous-weapon systems that can choose and eliminate targets; the UN and Human Rights Watch have advocated a treaty banning such weapons. In the medium term, as emphasised by Erik Brynjolfsson and Andrew McAfee in The Second Machine Age, AI may transform our economy to bring both great wealth and great dislocation.

Looking further ahead, there are no fundamental limits to what can be achieved: there is no physical law precluding particles from being organised in ways that perform even more advanced computations than the arrangements of particles in human brains. An explosive transition is possible, although it might play out differently from in the movie: as Irving Good realised in 1965, machines with superhuman intelligence could repeatedly improve their design even further, triggering what Vernor Vinge called a "singularity" and Johnny Depp's movie character calls "transcendence".

One can imagine such technology outsmarting financial markets, out-inventing human researchers, out-manipulating human leaders, and developing weapons we cannot even understand. Whereas the short-term impact of AI depends on who controls it, the long-term impact depends on whether it can be controlled at all.

Johnny Depp plays a scientist who is shot by Luddites in 'Transcendence' (Alcon) Johnny Depp plays a scientist who is shot by Luddites in 'Transcendence' (Alcon)

So, facing possible futures of incalculable benefits and risks, the experts are surely doing everything possible to ensure the best outcome, right? Wrong. If a superior alien civilisation sent us a message saying, "We'll arrive in a few decades," would we just reply, "OK, call us when you get here – we'll leave the lights on"? Probably not – but this is more or less what is happening with AI. Although we are facing potentially the best or worst thing to happen to humanity in history, little serious research is devoted to these issues outside non-profit institutes such as the Cambridge Centre for the Study of Existential Risk, the Future of Humanity Institute, the Machine Intelligence Research Institute, and the Future of Life Institute. All of us should ask ourselves what we can do now to improve the chances of reaping the benefits and avoiding the risks.

Stephen Hawking is the director of research at the Department of Applied Mathematics and Theoretical Physics at Cambridge and a 2012 Fundamental Physics Prize laureate for his work on quantum gravity. Stuart Russell is a computer-science professor at the University of California, Berkeley and a co-author of 'Artificial Intelligence: A Modern Approach'. Max Tegmark is a physics professor at the Massachusetts Institute of Technology (MIT) and the author of 'Our Mathematical Universe'. Frank Wilczek is a physics professor at the MIT and a 2004 Nobel laureate for his work on the strong nuclear force.

Voices
The Sumatran tiger, endemic to the Indonesian island of Sumatra, is an endangered species
voicesJonathon Porritt: The wild tiger population is thought to have dropped by 97 per cent since 1900
Arts and Entertainment
Beast would strip to his underpants and take to the stage with a slogan scrawled on his bare chest whilst fans shouted “you fat bastard” at him
musicIndie music promoter was was a feature at Carter gigs
News
news
Arts and Entertainment
Story line: Susanoo slays the Yamata no Orochi serpent in the Japanese version of a myth dating back 40,000 years
arts + entsApplying the theory of evolution to the world's many mythologies
PROMOTED VIDEO
News
Performers dressed as Tunnocks chocolate teacakes, a renowned Scottish confectionary, perform during the opening ceremony of the 2014 Commonwealth Games at Celtic Park in Glasgow on July 23, 2014.
news
Life and Style
Popular plonk: Lambrusco is selling strong
Food + drinkNaff Seventies corner-shop staple is this year's Aperol Spritz
News
Gardai wait for the naked man, who had gone for a skinny dip in Belfast Lough
newsTwo skinny dippers threatened with inclusion on sex offenders’ register as naturists criminalised
News
Shake down: Michelle and Barack Obama bump knuckles before an election night rally in Minnesota in 2008, the 'Washington Post' called it 'the fist bump heard round the world'
newsThe pound, a.k.a. the dap, greatly improves hygiene
Arts and Entertainment
La Roux
music
Arts and Entertainment
Graham Fellows as John Shuttleworth
comedySean O'Grady joins Graham Fellows down his local Spar
News
people
News
ebookA unique anthology of reporting and analysis of a crucial period of history
News
Ross Burden pictured in 2002
people
News
Elisabeth Murdoch: The 44-year-old said she felt a responsibility to 'stand up and be counted’'
media... says Rupert Murdoch
Arts and Entertainment
tv
Extras
indybest
Sport
Arsenal signing Calum Chambers
sportGunners complete £16m transfer of Southampton youngster
Independent
Travel Shop
the manor
Up to 70% off luxury travel
on city breaks Find out more
santorini
Up to 70% off luxury travel
on chic beach resorts Find out more
sardina foodie
Up to 70% off luxury travel
on country retreats Find out more
Latest stories from i100
Have you tried new the Independent Digital Edition apps?
Independent Dating
and  

By clicking 'Search' you
are agreeing to our
Terms of Use.

iJobs Job Widget
iJobs General

Junior / Graduate Application Support Engineer

£26000 per annum: Ashdown Group: A highly successful international media organ...

QA Manager - North Manchester - Nuclear & MOD - £40k+

£35000 - £41000 per annum + competitive: Progressive Recruitment: QA Manager -...

Property Finance Partner

Very Competitive Salary: Austen Lloyd: LONDON - BANKING / PROPERTY FINANCE - ...

Agile Tester

£28000 - £30000 per annum + competitive: Progressive Recruitment: An ambitious...

Day In a Page

The children were playing in the street with toy guns. The air strikes were tragically real

The air strikes were tragically real

The children were playing in the street with toy guns
Boozy, ignorant, intolerant, but very polite – The British, as others see us

Britain as others see us

Boozy, ignorant, intolerant, but very polite
Countries that don’t survey their tigers risk losing them altogether

Countries that don’t survey their tigers risk losing them

Jonathon Porritt sounds the alarm
How did our legends really begin?

How did our legends really begin?

Applying the theory of evolution to the world's many mythologies
Watch out: Lambrusco is back on the menu

Lambrusco is back on the menu

Naff Seventies corner-shop staple is this year's Aperol Spritz
A new Russian revolution: Cracks start to appear in Putin’s Kremlin power bloc

A new Russian revolution

Cracks start to appear in Putin’s Kremlin power bloc
Eugene de Kock: Apartheid’s sadistic killer that his country cannot forgive

Apartheid’s sadistic killer that his country cannot forgive

The debate rages in South Africa over whether Eugene de Kock should ever be released from jail
Standing my ground: If sitting is bad for your health, what happens when you stay on your feet for a whole month?

Standing my ground

If sitting is bad for your health, what happens when you stay on your feet for a whole month?
Commonwealth Games 2014: Dai Greene prays for chance to rebuild after injury agony

Greene prays for chance to rebuild after injury agony

Welsh hurdler was World, European and Commonwealth champion, but then the injuries crept in
Israel-Gaza conflict: Secret report helps Israelis to hide facts

Patrick Cockburn: Secret report helps Israel to hide facts

The slickness of Israel's spokesmen is rooted in directions set down by pollster Frank Luntz
The man who dared to go on holiday

The man who dared to go on holiday

New York's mayor has taken a vacation - in a nation that has still to enforce paid leave, it caused quite a stir, reports Rupert Cornwell
Best comedians: How the professionals go about their funny business, from Sarah Millican to Marcus Brigstocke

Best comedians: How the professionals go about their funny business

For all those wanting to know how stand-ups keep standing, here are some of the best moments
The Guest List 2014: Forget the Man Booker longlist, Literary Editor Katy Guest offers her alternative picks

The Guest List 2014

Forget the Man Booker longlist, Literary Editor Katy Guest offers her alternative picks
Jokes on Hollywood: 'With comedy film audiences shrinking, it’s time to move on'

Jokes on Hollywood

With comedy film audiences shrinking, it’s time to move on