Scientists create algorithm that 'reads' audio from near-invisible vibrations in silent videos

Engineers from MIT were able to recove audio from video footage of everyday objects including a potted plant, glass of water and teapot

Sound travels as vibrations through the air and whatever it hits vibrates too. If this is your ear then specialized hair cells translate these movements into something you can hear but for pretty much anything else these vibrations are simply lost.

Now, however, computer scientists have created an algorithim that translates these visual cues into an audio signal, 'reading' the near-invisible vibrations caused by people speaking or music playing from a range of everyday objects from a glass of water to a crisp packet.

Using video footage recorded by high speed cameras researchers from the Massachusetts Institute of Technology were able to listen to test subjects reading nursery rhymes from behind sound proof glass using only video of a crisp packet in the same room.

Previous techniques that recover sound from vibration have all relied on some sort of active interaction with the vibrating object (such as an infrared beam bouncing off a window using a ‘laser microphone’) but this new research needs only a visual signal.

Computer science graduate Abe Davis and his team were even able to extract audio from footage recorded by an ordinary digital camera and although the quality was good enough to make out distinct speech it was enough to identify information such as the number of people in a room and their gender.

These new techniques could have applications in law enforcement and forensics, but Davis has said that he’s more interested in a “new kind of imaging” that would allow scientists to analyse the material and structural properties of an object just by recording a video of it.

“We’re recovering sounds from objects,” he said in a press release. “That gives us a lot of information about the sound that’s going on around the object, but it also gives us a lot of information about the object itself, because different objects are going to respond to sound in different ways.”

Life and Style
ebookNow available in paperback
ebooks
ebookA delicious collection of 50 meaty main courses
Latest stories from i100
Have you tried new the Independent Digital Edition apps?
SPONSORED FEATURES
Independent Dating
and  

By clicking 'Search' you
are agreeing to our
Terms of Use.

ES Rentals

    iJobs Job Widget
    iJobs Gadgets & Tech

    Recruitment Genius: 1st Line Technical Support Engineer

    £19000 - £23000 per annum: Recruitment Genius: This IT and Telecoms company ar...

    Recruitment Genius: Client Services Administrator - IT Industry

    £18000 - £24000 per annum: Recruitment Genius: A leading provider of ICT servi...

    Recruitment Genius: 1st / 2nd Line Technical Support Advisor - Up to £26K inc bonus

    £20230 - £26000 per annum: Recruitment Genius: Are you looking for a career in...

    Recruitment Genius: Marketing Assistant

    £13500 - £15000 per annum: Recruitment Genius: This IT and Telecoms company is...

    Day In a Page

    War with Isis: Is Turkey's buffer zone in Syria a matter of self-defence – or just anti-Kurd?

    Turkey's buffer zone in Syria: self-defence – or just anti-Kurd?

    Ankara accused of exacerbating racial division by allowing Turkmen minority to cross the border
    Doris Lessing: Acclaimed novelist was kept under MI5 observation for 18 years, newly released papers show

    'A subversive brothel keeper and Communist'

    Acclaimed novelist Doris Lessing was kept under MI5 observation for 18 years, newly released papers show
    Big Blue Live: BBC's Springwatch offshoot swaps back gardens for California's Monterey Bay

    BBC heads to the Californian coast

    The Big Blue Live crew is preparing for the first of three episodes on Sunday night, filming from boats, planes and an aquarium studio
    Austin Bidwell: The Victorian fraudster who shook the Bank of England with the most daring forgery the world had known

    Victorian fraudster who shook the Bank of England

    Conman Austin Bidwell. was a heartless cad who carried out the most daring forgery the world had known
    Car hacking scandal: Security designed to stop thieves hot-wiring almost every modern motor has been cracked

    Car hacking scandal

    Security designed to stop thieves hot-wiring almost every modern motor has been cracked
    10 best placemats

    Take your seat: 10 best placemats

    Protect your table and dine in style with a bold new accessory
    Ashes 2015: Alastair Cook not the only one to be caught in The Oval mindwarp

    Cook not the only one to be caught in The Oval mindwarp

    Aussie skipper Michael Clarke was lured into believing that what we witnessed at Edgbaston and Trent Bridge would continue in London, says Kevin Garside
    Can Rafael Benitez get the best out of Gareth Bale at Real Madrid?

    Can Benitez get the best out of Bale?

    Back at the club he watched as a boy, the pressure is on Benitez to find a winning blend from Real's multiple talents. As La Liga begins, Pete Jenson asks if it will be enough to stop Barcelona
    Athletics World Championships 2015: Beijing witnesses new stage in the Jessica Ennis-Hill and Katarina Johnson-Thompson heptathlon rivalry

    Beijing witnesses new stage in the Jess and Kat rivalry

    The last time the two British heptathletes competed, Ennis-Hill was on the way to Olympic gold and Johnson-Thompson was just a promising teenager. But a lot has happened in the following three years
    Jeremy Corbyn: Joining a shrewd operator desperate for power as he visits the North East

    Jeremy Corbyn interview: A shrewd operator desperate for power

    His radical anti-austerity agenda has caught the imagination of the left and politically disaffected and set a staid Labour leadership election alight
    Isis executes Palmyra antiquities chief: Defender of ancient city's past was killed for protecting its future

    Isis executes Palmyra antiquities chief

    Robert Fisk on the defender of the ancient city's past who was killed for protecting its future
    Berlusconi's world of sleaze: The astonishing lifestyle once enjoyed by Italy's former PM

    Berlusconi's world of sleaze

    The astonishing lifestyle once enjoyed by Italy's former PM
    Disney plans galactic domination with endless Star Wars spin-offs

    Disney plans galactic domination with endless Star Wars spin-offs

    Films and theme parks are just the beginning. Disney believes its control of the now decades-old franchise can bring in merchandise and marketing millions for years to come
    Could the golden age of the gaming arcade ever be revived in the era of the Xbox?

    Could gaming arcades be revived?

    The days when coin-ops were the only way to play the latest video games are gone. But a small band of enthusiasts are keeping the button-pushing dream alive
    Edinburgh Fringe 2015: The 'tampon tax' has inspired a new wave of female comedians to reclaim period jokes

    Heard the one about menstruation?

    Yes, if you have been at the Fringe, where period pieces are taking centre stage