New glove translates sign language to speech in real time

The glove uses machine learning and a smartphone app to translate signs to words

Adam Smith
Thursday 02 July 2020 17:31
Comments

Bioengineers have designed a glove that is able to translate American Sign Language to speech in real time.

The glove has thin, stretchable sensors inside which run to the fingertips. These sensors are able to pick up and motions and finger placement through electrically conducting yarns.

Those sensors are then connected to a small piece of circuit board – approximately the size of a coin – that users wear on their wrists.

As such, when people move their hands and fingers to sign words, the glove is able to translate the individual letters, numbers, words and phrases into spoken words.

Extra sensors can be added to the face, between the eyebrows and on the sides of the mouth, to capture facial expressions.

The actual translation is done through a smartphone app, which uses a custom custom machine-learning algorithm to turned gestures into the letters, numbers and words.

The system recognises 660 signs, which includes each letter of the alphabet and the numbers from zero to nine.

It can translate at a speed of one word per second.

A commercial version of this technology would require more vocabulary and a faster translation time. UCLA has filed a patent on the glove.

“Our hope is that this opens up an easy way for people who use sign language to communicate directly with non-signers without needing someone else to translate for them,” said Jun Chen, the principal investigator on the glove.

“In addition, we hope it can help more people learn sign language themselves.”

Zhihao Zhao, Kyle Chen, Songlin Zhang, Yihao Zhou and Weili Deng, from the University of California, Los Angeles (UCLA), authored the study that developed the glove, alongside Chen.

Jin Yang, of China’s Chongqing University, also authored the study.

The benefits of the glove are its portability and weight. Previous wearable devices offered similar capabilities, but were heavy and impractical to wear.

This new design is far lighter, and the polymers are inexpensive – as are the electronic components.

However, some deaf researchers have criticised the development.

"The tech is redundant because deaf signers already make extensive use of text-to-speech or text translation software on their phones, or simply write with pen and paper, or even gesture clearly," said Gabrielle Hodge, a deaf researcher from the Deafness Cognition and Language Research Centre at University College London, to CNN. "There is nothing wrong with these forms of communication."

"It would be so much easier if tech focused on user-driven and user-centered design in the first instance, rather than dreaming up 'solutions' they think will fix all the problems in the world."

Register for free to continue reading

Registration is a free and easy way to support our truly independent journalism

By registering, you will also enjoy limited access to Premium articles, exclusive newsletters, commenting, and virtual events with our leading journalists

Please enter a valid email
Please enter a valid email
Must be at least 6 characters, include an upper and lower case character and a number
Must be at least 6 characters, include an upper and lower case character and a number
Must be at least 6 characters, include an upper and lower case character and a number
Please enter your first name
Special characters aren’t allowed
Please enter a name between 1 and 40 characters
Please enter your last name
Special characters aren’t allowed
Please enter a name between 1 and 40 characters
You must be over 18 years old to register
You must be over 18 years old to register
Opt-out-policy
You can opt-out at any time by signing in to your account to manage your preferences. Each email has a link to unsubscribe.

By clicking ‘Create my account’ you confirm that your data has been entered correctly and you have read and agree to our Terms of use, Cookie policy and Privacy notice.

This site is protected by reCAPTCHA and the Google Privacy policy and Terms of service apply.

Already have an account? sign in

By clicking ‘Register’ you confirm that your data has been entered correctly and you have read and agree to our Terms of use, Cookie policy and Privacy notice.

This site is protected by reCAPTCHA and the Google Privacy policy and Terms of service apply.

Register for free to continue reading

Registration is a free and easy way to support our truly independent journalism

By registering, you will also enjoy limited access to Premium articles, exclusive newsletters, commenting, and virtual events with our leading journalists

Already have an account? sign in

By clicking ‘Register’ you confirm that your data has been entered correctly and you have read and agree to our Terms of use, Cookie policy and Privacy notice.

This site is protected by reCAPTCHA and the Google Privacy policy and Terms of service apply.

Join our new commenting forum

Join thought-provoking conversations, follow other Independent readers and see their replies

Comments

Thank you for registering

Please refresh the page or navigate to another page on the site to be automatically logged inPlease refresh your browser to be logged in