Facebook using artificial intelligence to help suicidal users

The company has developed algorithms designed to flag up warning signs

Aatif Sulleyman
Thursday 02 March 2017 11:44
Comments
The AI tool would send concerning posts to a human review team, which would get in touch with the user
The AI tool would send concerning posts to a human review team, which would get in touch with the user

Facebook has started using artificial intelligence to identify users who are potentially at risk of taking their own lives.

The social network has developed algorithms capable of scanning posts and comments for warning signs.

These could be phrases such as “Are you okay?” or “I’m worried about you”, or more general talk of sadness and pain.

The AI tool would send such posts to a human review team, which would get in touch with the user thought to be at risk and offer help, in the form of contact details for support services or a chat with a member of staff through Facebook Messenger.

The site had previously relied on other users reporting worrying updates.

“The AI is actually more accurate than the reports that we get from people that are flagged as suicide and self injury,” Facebook product manager Vanessa Callison-Burch told BuzzFeed. “The people who have posted that content [that AI reports] are more likely to be sent resources of support versus people reporting to us.”

The system is currently being tested in the US.

The site has also announced new safety features for Facebook Live, which has been used to live stream several suicides.

Users can now flag up concerning Facebook Live behaviour with the site, which will display advice and highlight the video to staff for immediate review.

The goal is to provide help as quickly as possible, mid-broadcast rather than post-broadcast.

“Some might say we should cut off the stream of the video the moment there is a hint of somebody talking about suicide,” said Jennifer Guadagno, the project’s lead researcher.

“But what the experts emphasised was that cutting off the stream too early would remove the opportunity for people to reach out and offer support. So, this opens up the ability for friends and family to reach out to a person in distress at the time they may really need it the most.”

Facebook CEO Mark Zuckerberg described plans to use AI to identify worrying content in a recently published manifesto.

“Looking ahead, one of our greatest opportunities to keep people safe is building artificial intelligence to understand more quickly and accurately what is happening across our community,” it read.

An earlier version of the piece said that it would take “many years to develop” AI systems capable of identifying issues such as bullying and terrorism risks online, but the section was removed before the manifesto was publicly issued.

Register for free to continue reading

Registration is a free and easy way to support our truly independent journalism

By registering, you will also enjoy limited access to Premium articles, exclusive newsletters, commenting, and virtual events with our leading journalists

Please enter a valid email
Please enter a valid email
Must be at least 6 characters, include an upper and lower case character and a number
Must be at least 6 characters, include an upper and lower case character and a number
Must be at least 6 characters, include an upper and lower case character and a number
Please enter your first name
Special characters aren’t allowed
Please enter a name between 1 and 40 characters
Please enter your last name
Special characters aren’t allowed
Please enter a name between 1 and 40 characters
You must be over 18 years old to register
You must be over 18 years old to register
Opt-out-policy
You can opt-out at any time by signing in to your account to manage your preferences. Each email has a link to unsubscribe.

By clicking ‘Create my account’ you confirm that your data has been entered correctly and you have read and agree to our Terms of use, Cookie policy and Privacy notice.

This site is protected by reCAPTCHA and the Google Privacy policy and Terms of service apply.

Already have an account? sign in

By clicking ‘Register’ you confirm that your data has been entered correctly and you have read and agree to our Terms of use, Cookie policy and Privacy notice.

This site is protected by reCAPTCHA and the Google Privacy policy and Terms of service apply.

Register for free to continue reading

Registration is a free and easy way to support our truly independent journalism

By registering, you will also enjoy limited access to Premium articles, exclusive newsletters, commenting, and virtual events with our leading journalists

Already have an account? sign in

By clicking ‘Register’ you confirm that your data has been entered correctly and you have read and agree to our Terms of use, Cookie policy and Privacy notice.

This site is protected by reCAPTCHA and the Google Privacy policy and Terms of service apply.

Join our new commenting forum

Join thought-provoking conversations, follow other Independent readers and see their replies

Comments

Thank you for registering

Please refresh the page or navigate to another page on the site to be automatically logged inPlease refresh your browser to be logged in