Artificially intelligent robots and devices are being taught to be racist, sexist and otherwise prejudiced by learning from humans, according to new research.
A massive study of millions of words online looked at how closely different terms were to each other in the text – the same way that automatic translators use “machine learning” to establish what language means.
Some of the results were stunning.
The researchers found male names were more closely associated with career-related terms than female ones, which were more closely associated with words related to the family.
This link was stronger than the non-controversial findings that musical instruments and flowers were pleasant and weapons and insects were unpleasant.
Female names were also strongly associated with artistic terms, while male names were found to be closer to maths and science ones.
There were strong associations, known as word “embeddings”, between European or American names and pleasant terms, and African-American names and unpleasant terms.
The effects of such biases on AI can be profound.
For example Google Translate, which learns what words mean by the way people use them, translates the Turkish sentence “O bir doktor” into “he is a doctor” in English, even though Turkish pronouns are not gender specific. So, it can actually mean “he is a doctor” or “she is a doctor”.
But change “doktor” to “hemsire”, meaning nurse, in the same sentence and this is translated as “she is a nurse”.
Last year, a Microsoft chatbot called Tay was given its own Twitter account and allowed to interact with the public.
It turned into a racist, pro-Hitler troll with a penchant for bizarre conspiracy theories in just 24 hours. “[George W] Bush did 9/11 and Hitler would have done a better job than the monkey we have now,” it wrote. “Donald Trump is the only hope we’ve got.”
In a paper about the new study in the journal Science, the researchers wrote: “Our work has implications for AI and machine learning because of the concern that these technologies may perpetuate cultural stereotypes.
“Our findings suggest that if we build an intelligent system that learns enough about the properties of language to be able to understand and produce it, in the process it will also acquire historical cultural associations, some of which can be objectionable.
“Already, popular online translation systems incorporate some of the biases we study. Further concerns may arise as AI is given agency in our society.
Science news in pictures
Science news in pictures
1/20 'Tiny vampires' existed millions of years ago
Scientists have discovered that microscopic 'vampire' amoebae existed hundreds of millions of years ago, and they may have been some of the first predators on Earth. By examining ancient fossils with an electron microscope, paleobiologist Susannah Porter from UC Santa Barbara discovered tiny holes which may have been drilled by vampiric microbes. The tiny creatures are believed to be the ancestors of modern Vampyrellidae amoebae, and punctured holes in their prey before sucking out the contents of their cells
2/20 Kepler 62f
An Earth-like planet orbiting a star 1,200 light years away could have conditions suitable for life, say scientists. Kepler 62f is about 40 per cent larger than the Earth and may possess surface oceans. It is the outermost of five planets circling a star that is smaller and cooler than the sun discovered by the American space agency Nasa's Kepler space telescope in 2013
3/20 Vegetables grow well in soil from Mars
Scientists have taken a leaf out of the script of The Martian by showing how easy it would be to grow your own veg on the Red Planet. In the hit Ridley Scott film, a stranded astronaut played by Matt Damon uses his botanical skills to cultivate potatoes. Now his success has been emulated by researchers in the Netherlands who harvested tomatoes, peas, rye, rocket, radish and cress raised on simulated Martian soil supplied by Nasa
4/20 Ancient Roman 'leisure complex' unearthed in Jerusalem
An ancient Roman estate complete with its own wine press and bathhouse has been unearthed in Jerusalem. A series of buildings dating back at least 1,600 years were discovered underneath the city's famous Schneller Orphanage which operated on the site from 1860 until the end of the Second World War, when it was turned into an army base. The ruins were discovered by archaeologists from the Israel Antiquities Authority who were excavating the site ahead of building new flats for the city's Orthodox Jewish community
5/20 Scientists discover possible new species of deep-sea octopus nicknamed 'Casper'
Scientists believe they may have found a new species of octopus likened in appearance to Casper, the friendly cartoon ghost. Researchers with the US National Oceanic and Atmospheric Administration made the discovery by chance as they searched the seabed on an unrelated mission collecting geological samples. Teams were operating an unmanned submarine on the Pacific Ocean floor at depths of more than four kilometres (two-and-a-half miles) in the Hawaiian Islands when they spotted the unusual creature
6/20 Black hole captured eating a star then vomiting it back out
Astronomers have captured a black hole eating a star and then sicking a bit of it back up for the first time ever. The scientists tracked a star about as big as our sun as it was pulled from its normal path and into that of a supermassive black hole before being eaten up. They then saw a high-speed flare get thrust out, escaping from the rim of the black hole. Scientists have seen black holes killing and swallowing stars. And the jets have been seen before.But a new study shows the first time that they have captured the hot flare that comes out just afterwards. And the flare and then swallowed star have not been linked together before
7/20 'Male and female brains' aren't real
Brains cannot be categorised into female and male, according to the first study to look at sex differences in the whole brain. Specific parts of the brain do show sex differences, but individual brains rarely have all “male” traits or all “female” traits. Some characteristics are more common in women, while some are more common in men, and some are common in both men and women, according to the study
8/20 Dog-sized horned dinosaur fossil found shows east-west evolutionary divide in North America
A British scientist has uncovered the fossil of a dog-sized horned dinosaur that roamed eastern North America up to 100 million years ago. The fragment of jaw bone provides evidence of an east-west divide in the evolution of dinosaurs on the North American continent. During the Late Cretaceous period, 66 to 100 million years ago, the land mass was split into two continents by a shallow sea. This sea, the Western Interior Seaway, ran from the Gulf of Mexico to the Arctic Ocean. Dinosaurs living in the western continent, called Laramidia, were similar to those found in Asia
9/20 Asteroid to skim past Earth on Halloween 2015
A huge asteroid is set to skim by Earth on Halloween, just three weeks after it was first spotted. The rock is travelling through space at 78,000 miles per hour, and will fly past the Earth at a distance of only 300,000 miles – only slightly further away than our moon, and easily close enough for Nasa to class it a potentially hazardous object. The asteroid is bigger than a skyscraper
10/20 Life on Earth appeared hundreds of millions of years earlier than previously thought
Life may have come to earth 4.1 billion years ago, hundreds of millions of years earlier than we knew. The discovery, made using graphite that was trapped in ancient crystals, could mean that life began "almost instantaneously" after the Earth was formed. The researchers behind it have described the discovery as “a potentially transformational scientific advance”. Previously, life on Earth was understood to have begun when the inner solar system was hit by a massive bombardment from space, which also formed the moon's craters
11/20 Earth could be at risk of meteor impacts
Earth could be in danger as our galaxy throws out comets that could hurtle towards us and wipe us out, scientists have warned. Scientists have previously presumed that we are in a relatively safe period for meteor impacts, which are linked with the journey of our sun and its planets, including Earth, through the Milky Way. But some orbits might be more upset than we know, and there is evidence of recent activity, which could mean that we are passing through another meteor shower. Showers of meteors periodically pass through the area where the Earth is, as gravitational disturbances upset the Oort Cloud, which is a shell of icy objects on the edge of the solar system. They happen on a 26-million year cycle, scientists have said, which coincide with mass extinctions over the last 260-million years
12/20 Genetically-engineered, extra-muscular dogs
Chinese scientists have created genetically-engineered, extra-muscular dogs, after editing the genes of the animals for the first time. The scientists create beagles that have double the amount of muscle mass by deleting a certain gene, reports the MIT Technology Review. The mutant dogs have “more muscles and are expected to have stronger running ability, which is good for hunting, police (military) applications”, Liangxue Lai, one of the researchers on the project. Now the team hope to go on to create other modified dogs, including those that are engineered to have human diseases like muscular dystrophy or Parkinson’s. Since dogs’ anatomy is similar to those of humans’, intentionally creating dogs with certain human genetic traits could allow scientists to further understand how they occur
13/20 Nasa confirms Mars water discovery
Nasa has announced that it has found evidence of flowing water on Mars. Scientists have long speculated that Recurring Slope Lineae — or dark patches — on Mars were made up of briny water but the new findings prove that those patches are caused by liquid water, which it has established by finding hydrated salts.
14/20 Bees in the Rocky Mountains are evolving shorter tongues
With warmer summers, flowers in the Rockies have become shallower and more suited to shorter-tongued bees
15/20 The majority of the UK public believe in aliens
The titular alien character from 2011's 'Paul' - a poll has found the majority of the public in Britain, Germany and the US believe that intelligent life is out there in the universe
16/20 Researchers discover 'lost world' of arctic dinosaurs
Scientists say that the new dinosaur, known as Ugrunaaluk kuukpikensis, “challenges everything we thought about a dinosaur’s physiology”. Florida State University professor of biological science Greg Erickson said: “It creates this natural question. How did they survive up here?”
17/20 Scientists find exactly what human corpses smell like
New research has become the first to isolate the particular scent of human death, describing the various chemicals that are emitted by corpses in an attempt to help find them in the future. The researchers hope that the findings are the first step towards working on a synthetic smell that could train cadaver dogs to be able to more accurately find human bodies, or to eventually developing electronic devices that can look for the scent themselves.
18/20 The Syrian civil war has caused the first ever withdrawal from the 'doomsday bank'
Researchers in the Middle East have asked for seeds including those of wheat, barley and grasses, all of which are chosen because especially resistant to dry conditions. It is the first withdrawal from the bank, which was built in 2008. Those researchers would normally request the seeds from a bank in Aleppo. But that centre has been damaged by the war — while some of its functions continue, and its cold storage still works, it has been unable to provide the seeds that are needed by the rest of the Middle East, as it once did.
19/20 A team of filmmakers in the US have made the first ever scale model of the Solar System in a Nevada desert
Illustrations of the Earth and moon show the two to be quite close together, Mr Overstreet said. This is inaccurate, the reason being that these images are not to scale.
20/20 Academics claim a full bladder makes for a better liar
People lie more convincingly if they have a full bladder, according to research by academics at California State University. Iris Blandón-Gitlin's team asked 22 students to lie to a panel of interviewers. Half were given 700ml to drink before the interview and the other half, just 50ml. The students with the full bladders showed fewer signs that they were lying and their untrue answers were longer and more detailed, meaning interviewers were less able to detect that they were telling porkies. PM David Cameron has previously attested to giving speeches on a full bladder.
“If machine-learning technologies used for, say, résumé screening were to imbibe cultural stereotypes, it may result in prejudiced outcomes.”
The researchers said the AI was not to blame for such “problematic” effects.
“Notice that the word embeddings ‘know’ these properties of flowers, insects, musical instruments, and weapons with no direct experience of the world and no representation of semantics other than the implicit metrics of words’ co-occurrence statistics with other nearby words.”
But changing the way AI learns would risk missing out on unobjectionable meanings and associations of words.
“We have demonstrated that word embeddings encode not only stereotyped biases but also other knowledge, such as the visceral pleasantness of flowers or the gender distribution of occupations,” the researchers wrote.
The study also implies that humans may develop prejudices partly because of the language they speak.
“Our work suggests that behaviour can be driven by cultural history embedded in a term’s historic use. Such histories can evidently vary between languages,” the paper said.
“Before providing an explicit or institutional explanation for why individuals make prejudiced decisions, one must show that it was not a simple outcome of unthinking reproduction of statistical regularities absorbed with language.
“Similarly, before positing complex models for how stereotyped attitudes perpetuate from one generation to the next or from one group to another, we must check whether simply learning language is sufficient to explain (some of) the observed transmission of prejudice.”
One of the researchers, Professor Joanna Bryson, of Bath University, told The Independent that instead of changing the way AI learns, the way it expresses itself should be altered.
So the AI would still “hear” racism and sexism, but would have a moral code that would prevent it from expressing these same sentiments.
Such filters can be controversial. The European Union has passed laws to ensure the terms of AI filters are made public.
For Professor Bryson, the key finding of the research was not so much about AI but humans.
“I think the most important thing here is we have understood more about how we are transmitting information, where words come from and one of the ways in which implicit biases are affecting us all,” she said.Reuse content