Facebook's artificial intelligence agents creating their own language is more normal than people think, researchers say

The messages sent by chatbots might look a little bizarre, but they aren't unusual or sinister

Andrew Griffin
Thursday 03 August 2017 10:46
Comments
Robotics student Gildo Andreoni interacts with a Dexmart robotic hand built at the University of Bologna in the Robotville exhibition at the Science Museum on November 29, 2011 in London, England
Robotics student Gildo Andreoni interacts with a Dexmart robotic hand built at the University of Bologna in the Robotville exhibition at the Science Museum on November 29, 2011 in London, England

Fears that computers were taking over swept the world this week when stories emerged about Facebook's AI creating its own language that researchers couldn't understand. But they might be a little misplaced.

But artificial intelligence experts have looked to calm worries that robots are becoming sentient or that we are living through the prelude to Terminator.

The messages might seem strange, they agree. But they are explicable and fairly normal in the world of artificial intelligence research.

Some of the discussion between the bots is seen below:

Bob: i can i i everything else . . . . . . . . . . . . . .

Alice: balls have zero to me to me to me to me to me to me to me to me to

Bob: you i everything else . . . . . . . . . . . . . .

Alice: balls have a ball to me to me to me to me to me to me to me

Bob: i i can i i i everything else . . . . . . . . . . . . . .

Alice: balls have a ball to me to me to me to me to me to me to me

Bob: i . . . . . . . . . . . . . . . . . . .

Alice: balls have zero to me to me to me to me to me to me to me to me to

Bob: you i i i i i everything else . . . . . . . . . . . . . .

Alice: balls have 0 to me to me to me to me to me to me to me to me to

Bob: you i i i everything else . . . . . . . . . . . . . .

Alice: balls have zero to me to me to me to me to me to me to me to me to

The messages didn't seem to be especially sinister. But the worrying nature of not being able to understand what an AI was saying or why it was saying it concerned many, and led to worries about such systems becoming sentient or conducting decisions without us being able to hold them accountable.

The story came after repeated warnings from many of the most respected minds in the world: people including Stephen Hawking have suggested that artificial intelligence could potentially bring about the end of humanity. Those predictions came to a head days before the story became popular as Elon Musk and Mark Zuckerberg argued about the dangers of AI – with Mr Zuckerberg saying that the danger had been overstated, after Mr Musk has repeatedly suggested that artificial intelligence could take over the world if it is not properly regulated and restrained.

But artificial intelligence researchers including those involved in the project have looked to calm those worries.

The idea of a chatbot inventing its own language might sound terrifying, those behind the Facebook research say. But it is actually a long-running part of the way that AI works and is studied – sometimes being encouraged, and at other times happening by itself.

Similar things have been seen in AI work done by Google for its Translate tool and at OpenAI, for instance.

In the case of the recent Facebook study, it was entirely accidental. The agents were simply not told to ensure that they worked using language comprehensible to their human masters – and so didn't.

"While the idea of AI agents inventing their own language may sound alarming/unexpected to people outside the field, it is a well-established sub-field of AI, with publications dating back decades," Dhruv Batra, who worked on the project, wrote on Facebook.

In the case of Facebook's AI, the messages might be incomprehensible but their meaning can be worked out, at least a little. It has been compared to the kinds of shorthand that are developed in all communities of specialists – where words might come to mean specific things to people, but be completely mystifying to anyone who is outside of the group.

Mr Batra also took issue with the phrasing of "shutting down" the chatbots, and said that such a decision was commonplace. Many AI experts have become irritated because some stories said that researchers had panicked and pulled the plug – but in fact researchers just changed the AI, killing the job but simply altering some of the rules that it worked by.

"Analyzing the reward function and changing the parameters of an experiment is NOT the same as 'unplugging' or 'shutting down AI'," he wrote. "If that were the case, every AI researcher has been 'shutting down AI' every time they kill a job on a machine."

Register for free to continue reading

Registration is a free and easy way to support our truly independent journalism

By registering, you will also enjoy limited access to Premium articles, exclusive newsletters, commenting, and virtual events with our leading journalists

Already have an account? sign in

By clicking ‘Register’ you confirm that your data has been entered correctly and you have read and agree to our Terms of use, Cookie policy and Privacy notice.

This site is protected by reCAPTCHA and the Google Privacy policy and Terms of service apply.

Join our new commenting forum

Join thought-provoking conversations, follow other Independent readers and see their replies

Comments

Thank you for registering

Please refresh the page or navigate to another page on the site to be automatically logged inPlease refresh your browser to be logged in