Scientists create algorithm that 'reads' audio from near-invisible vibrations in silent videos

Engineers from MIT were able to recove audio from video footage of everyday objects including a potted plant, glass of water and teapot

Sound travels as vibrations through the air and whatever it hits vibrates too. If this is your ear then specialized hair cells translate these movements into something you can hear but for pretty much anything else these vibrations are simply lost.

Now, however, computer scientists have created an algorithim that translates these visual cues into an audio signal, 'reading' the near-invisible vibrations caused by people speaking or music playing from a range of everyday objects from a glass of water to a crisp packet.

Using video footage recorded by high speed cameras researchers from the Massachusetts Institute of Technology were able to listen to test subjects reading nursery rhymes from behind sound proof glass using only video of a crisp packet in the same room.

Previous techniques that recover sound from vibration have all relied on some sort of active interaction with the vibrating object (such as an infrared beam bouncing off a window using a ‘laser microphone’) but this new research needs only a visual signal.

Computer science graduate Abe Davis and his team were even able to extract audio from footage recorded by an ordinary digital camera and although the quality was good enough to make out distinct speech it was enough to identify information such as the number of people in a room and their gender.

These new techniques could have applications in law enforcement and forensics, but Davis has said that he’s more interested in a “new kind of imaging” that would allow scientists to analyse the material and structural properties of an object just by recording a video of it.

“We’re recovering sounds from objects,” he said in a press release. “That gives us a lot of information about the sound that’s going on around the object, but it also gives us a lot of information about the object itself, because different objects are going to respond to sound in different ways.”

PROMOTED VIDEO
Life and Style
ebookNow available in paperback
ebooks
ebookPart of The Independent’s new eBook series The Great Composers
Latest stories from i100
Have you tried new the Independent Digital Edition apps?
Independent Dating
and  

By clicking 'Search' you
are agreeing to our
Terms of Use.

ES Rentals

    iJobs Job Widget
    iJobs Gadgets & Tech

    Recruitment Genius: Fundraising Manager / Income Generation Coach

    £21000 - £25000 per annum: Recruitment Genius: A smart software company locate...

    Ashdown Group: IT Support Analyst - London - £45,000

    £35000 - £45000 per annum + benefits: Ashdown Group: IT Support Analyst - Wes...

    Ashdown Group: Junior Web Developer - Kent - £40,000

    £30000 - £40000 per annum + benefits: Ashdown Group: Junior Web Developer - ne...

    Recruitment Genius: Electronics Engineers / Senior Electronics Engineers

    £25000 - £55000 per annum: Recruitment Genius: Based in Henley-on-Thames, this...

    Day In a Page

    Syria crisis: Celebrities call on David Cameron to take more refugees as one young mother tells of torture by Assad regime

    Celebrities call on David Cameron to take more Syrian refugees

    One young mother tells of torture by Assad regime
    The enemy within: People who hear voices in their heads are being encouraged to talk back – with promising results

    The enemy within

    People who hear voices in their heads are being encouraged to talk back
    'In Auschwitz you got used to anything'

    'In Auschwitz you got used to anything'

    Survivors of the Nazi concentration camp remember its horror, 70 years on
    Autumn/winter menswear 2015: The uniforms that make up modern life come to the fore

    Autumn/winter menswear 2015

    The uniforms that make up modern life come to the fore
    'I'm gay, and plan to fight military homophobia'

    'I'm gay, and plan to fight military homophobia'

    Army general planning to come out
    Iraq invasion 2003: The bloody warnings six wise men gave to Tony Blair as he prepared to launch poorly planned campaign

    What the six wise men told Tony Blair

    Months before the invasion of Iraq in 2003, experts sought to warn the PM about his plans. Here, four of them recall that day
    25 years of The Independent on Sunday: The stories, the writers and the changes over the last quarter of a century

    25 years of The Independent on Sunday

    The stories, the writers and the changes over the last quarter of a century
    Homeless Veterans appeal: 'Really caring is a dangerous emotion in this kind of work'

    Homeless Veterans appeal

    As head of The Soldiers' Charity, Martin Rutledge has to temper compassion with realism. He tells Chris Green how his Army career prepared him
    Wu-Tang Clan and The Sexual Objects offer fans a chance to own the only copies of their latest albums

    Smash hit go under the hammer

    It's nice to pick up a new record once in a while, but the purchasers of two latest releases can go a step further - by buying the only copy
    Geeks who rocked the world: Documentary looks back at origins of the computer-games industry

    The geeks who rocked the world

    A new documentary looks back at origins of the computer-games industry
    Belle & Sebastian interview: Stuart Murdoch reveals how the band is taking a new direction

    Belle & Sebastian is taking a new direction

    Twenty years ago, Belle & Sebastian was a fey indie band from Glasgow. It still is – except today, as prime mover Stuart Murdoch admits, it has a global cult following, from Hollywood to South Korea
    America: Land of the free, home of the political dynasty

    America: Land of the free, home of the political dynasty

    These days in the US things are pretty much stuck where they are, both in politics and society at large, says Rupert Cornwell
    A graphic history of US civil rights – in comic book form

    A graphic history of US civil rights – in comic book form

    A veteran of the Fifties campaigns is inspiring a new generation of activists
    Winston Churchill: the enigma of a British hero

    Winston Churchill: the enigma of a British hero

    A C Benson called him 'a horrid little fellow', George Orwell would have shot him, but what a giant he seems now, says DJ Taylor
    Growing mussels: Precious freshwater shellfish are thriving in a unique green project

    Growing mussels

    Precious freshwater shellfish are thriving in a unique green project