Artificially intelligent robots and devices are being taught to be racist, sexist and otherwise prejudiced by learning from humans, according to new research.
A massive study of millions of words online looked at how closely different terms were to each other in the text – the same way that automatic translators use “machine learning” to establish what language means.
Some of the results were stunning.
The researchers found male names were more closely associated with career-related terms than female ones, which were more closely associated with words related to the family.
This link was stronger than the non-controversial findings that musical instruments and flowers were pleasant and weapons and insects were unpleasant.
Female names were also strongly associated with artistic terms, while male names were found to be closer to maths and science ones.
There were strong associations, known as word “embeddings”, between European or American names and pleasant terms, and African-American names and unpleasant terms.
The effects of such biases on AI can be profound.
For example Google Translate, which learns what words mean by the way people use them, translates the Turkish sentence “O bir doktor” into “he is a doctor” in English, even though Turkish pronouns are not gender specific. So, it can actually mean “he is a doctor” or “she is a doctor”.
But change “doktor” to “hemsire”, meaning nurse, in the same sentence and this is translated as “she is a nurse”.
Last year, a Microsoft chatbot called Tay was given its own Twitter account and allowed to interact with the public.
It turned into a racist, pro-Hitler troll with a penchant for bizarre conspiracy theories in just 24 hours. “[George W] Bush did 9/11 and Hitler would have done a better job than the monkey we have now,” it wrote. “Donald Trump is the only hope we’ve got.”
In a paper about the new study in the journal Science, the researchers wrote: “Our work has implications for AI and machine learning because of the concern that these technologies may perpetuate cultural stereotypes.
“Our findings suggest that if we build an intelligent system that learns enough about the properties of language to be able to understand and produce it, in the process it will also acquire historical cultural associations, some of which can be objectionable.
“Already, popular online translation systems incorporate some of the biases we study. Further concerns may arise as AI is given agency in our society.
Science news in pictures
Science news in pictures
1/20 Nasa releases stunning images of Jupiter's great red spot
The storm bigger than the Earth, has been swhirling for 350 years. The image's colours have been enhanced after it was sent back to Earth.
Pictures by: Tom Momary
2/20 A 3D reconstruction of an African grey parrot post euthanasia
Included in Wellcome Image Awards, this 3D image of an African grey parrot shows the highly intricate system of blood vessels.
Scott Birch. Wellcome Images
3/20 Baby Hawaiian bobtail squid
Another Wellcome Images Award winner, this time of baby Hawaiian bobtail squid. The black ink sac and light organ in the centre of the squid’s mantle cavity can be clearly seen.
Macroscopic Solutions. Wellcome Images
4/20 Skeletons of 5,000-year-old Chinese ‘giants’ discovered by archaeologists
The people are thought to have been unusually tall and strong. The tallest of the skeletons uncovered measured at 1.9m
5/20 Nasa discovers 75,000 mile-wide hole in the Sun
Sunspots are caused by interactions with the Sun’s magnetic field and are cooler areas on the star’s surface.
6/20 View(active tab) Apple News Breaking news email Edit Revisions Workflow Clear Cache NewsScience 132 million-year-old dinosaur fossil found at factory in Surrey
Paleontologists Sarah Moore and Jamie Jordan believe they have discovered a Iguanodon dinosaur, a herbivore that was around three metres tall and 10 metres long
7/20 Discovering life on Mars is less likely as researchers find toxic chemicals on its surface
The Echus Chasma, one of the largest water source regions on Mars
8/20 The Grand Prismatic Spring, the largest in the United States and third largest in the world, is seen in Yellowstone National Park.
The park is famous for its geothermal activity – which includes its spectacular, flowing springs as well as the famous "Old Faithful" geyser that sprays water out every hour or so.
9/20 An iris clip fitted onto the eye
This images is apart of the Wellcome Images Awards and shows how an artificial intraocular lens is fitted onto the eye. Used for conditions such as myopia and cataracts.
Cambridge University Hospitals NHS FT. Wellcome Images
10/20 The Syrian civil war has caused the first ever withdrawal from the 'doomsday bank'
Researchers in the Middle East have asked for seeds including those of wheat, barley and grasses, all of which are chosen because especially resistant to dry conditions. It is the first withdrawal from the bank, which was built in 2008. Those researchers would normally request the seeds from a bank in Aleppo. But that centre has been damaged by the war — while some of its functions continue, and its cold storage still works, it has been unable to provide the seeds that are needed by the rest of the Middle East, as it once did.
11/20 Scientists find exactly what human corpses smell like
New research has become the first to isolate the particular scent of human death, describing the various chemicals that are emitted by corpses in an attempt to help find them in the future. The researchers hope that the findings are the first step towards working on a synthetic smell that could train cadaver dogs to be able to more accurately find human bodies, or to eventually developing electronic devices that can look for the scent themselves.
12/20 Black hole captured eating a star then vomiting it back out
Astronomers have captured a black hole eating a star and then sicking a bit of it back up for the first time ever. The scientists tracked a star about as big as our sun as it was pulled from its normal path and into that of a supermassive black hole before being eaten up. They then saw a high-speed flare get thrust out, escaping from the rim of the black hole. Scientists have seen black holes killing and swallowing stars. And the jets have been seen before.But a new study shows the first time that they have captured the hot flare that comes out just afterwards. And the flare and then swallowed star have not been linked together before
13/20 Dog-sized horned dinosaur fossil found shows east-west evolutionary divide in North America
A British scientist has uncovered the fossil of a dog-sized horned dinosaur that roamed eastern North America up to 100 million years ago. The fragment of jaw bone provides evidence of an east-west divide in the evolution of dinosaurs on the North American continent. During the Late Cretaceous period, 66 to 100 million years ago, the land mass was split into two continents by a shallow sea. This sea, the Western Interior Seaway, ran from the Gulf of Mexico to the Arctic Ocean. Dinosaurs living in the western continent, called Laramidia, were similar to those found in Asia
14/20 'Male and female brains' aren't real
Brains cannot be categorised into female and male, according to the first study to look at sex differences in the whole brain. Specific parts of the brain do show sex differences, but individual brains rarely have all “male” traits or all “female” traits. Some characteristics are more common in women, while some are more common in men, and some are common in both men and women, according to the study
15/20 Life on Earth appeared hundreds of millions of years earlier than previously thought
Life may have come to earth 4.1 billion years ago, hundreds of millions of years earlier than we knew. The discovery, made using graphite that was trapped in ancient crystals, could mean that life began "almost instantaneously" after the Earth was formed. The researchers behind it have described the discovery as “a potentially transformational scientific advance”. Previously, life on Earth was understood to have begun when the inner solar system was hit by a massive bombardment from space, which also formed the moon's craters
16/20 Nasa confirms Mars water discovery
Nasa has announced that it has found evidence of flowing water on Mars. Scientists have long speculated that Recurring Slope Lineae — or dark patches — on Mars were made up of briny water but the new findings prove that those patches are caused by liquid water, which it has established by finding hydrated salts.
17/20 Earth could be at risk of meteor impacts
Earth could be in danger as our galaxy throws out comets that could hurtle towards us and wipe us out, scientists have warned. Scientists have previously presumed that we are in a relatively safe period for meteor impacts, which are linked with the journey of our sun and its planets, including Earth, through the Milky Way. But some orbits might be more upset than we know, and there is evidence of recent activity, which could mean that we are passing through another meteor shower. Showers of meteors periodically pass through the area where the Earth is, as gravitational disturbances upset the Oort Cloud, which is a shell of icy objects on the edge of the solar system. They happen on a 26-million year cycle, scientists have said, which coincide with mass extinctions over the last 260-million years
18/20 Genetically-engineered, extra-muscular dogs
Chinese scientists have created genetically-engineered, extra-muscular dogs, after editing the genes of the animals for the first time. The scientists create beagles that have double the amount of muscle mass by deleting a certain gene, reports the MIT Technology Review. The mutant dogs have “more muscles and are expected to have stronger running ability, which is good for hunting, police (military) applications”, Liangxue Lai, one of the researchers on the project. Now the team hope to go on to create other modified dogs, including those that are engineered to have human diseases like muscular dystrophy or Parkinson’s. Since dogs’ anatomy is similar to those of humans’, intentionally creating dogs with certain human genetic traits could allow scientists to further understand how they occur
19/20 Researchers discover 'lost world' of arctic dinosaurs
Scientists say that the new dinosaur, known as Ugrunaaluk kuukpikensis, “challenges everything we thought about a dinosaur’s physiology”. Florida State University professor of biological science Greg Erickson said: “It creates this natural question. How did they survive up here?”
20/20 A team of filmmakers in the US have made the first ever scale model of the Solar System in a Nevada desert
Illustrations of the Earth and moon show the two to be quite close together, Mr Overstreet said. This is inaccurate, the reason being that these images are not to scale.
“If machine-learning technologies used for, say, résumé screening were to imbibe cultural stereotypes, it may result in prejudiced outcomes.”
The researchers said the AI was not to blame for such “problematic” effects.
“Notice that the word embeddings ‘know’ these properties of flowers, insects, musical instruments, and weapons with no direct experience of the world and no representation of semantics other than the implicit metrics of words’ co-occurrence statistics with other nearby words.”
But changing the way AI learns would risk missing out on unobjectionable meanings and associations of words.
“We have demonstrated that word embeddings encode not only stereotyped biases but also other knowledge, such as the visceral pleasantness of flowers or the gender distribution of occupations,” the researchers wrote.
The study also implies that humans may develop prejudices partly because of the language they speak.
“Our work suggests that behaviour can be driven by cultural history embedded in a term’s historic use. Such histories can evidently vary between languages,” the paper said.
“Before providing an explicit or institutional explanation for why individuals make prejudiced decisions, one must show that it was not a simple outcome of unthinking reproduction of statistical regularities absorbed with language.
“Similarly, before positing complex models for how stereotyped attitudes perpetuate from one generation to the next or from one group to another, we must check whether simply learning language is sufficient to explain (some of) the observed transmission of prejudice.”
One of the researchers, Professor Joanna Bryson, of Bath University, told The Independent that instead of changing the way AI learns, the way it expresses itself should be altered.
So the AI would still “hear” racism and sexism, but would have a moral code that would prevent it from expressing these same sentiments.
Such filters can be controversial. The European Union has passed laws to ensure the terms of AI filters are made public.
For Professor Bryson, the key finding of the research was not so much about AI but humans.
“I think the most important thing here is we have understood more about how we are transmitting information, where words come from and one of the ways in which implicit biases are affecting us all,” she said.Reuse content