Scientists create algorithm that 'reads' audio from near-invisible vibrations in silent videos

Engineers from MIT were able to recove audio from video footage of everyday objects including a potted plant, glass of water and teapot

Sound travels as vibrations through the air and whatever it hits vibrates too. If this is your ear then specialized hair cells translate these movements into something you can hear but for pretty much anything else these vibrations are simply lost.

Now, however, computer scientists have created an algorithim that translates these visual cues into an audio signal, 'reading' the near-invisible vibrations caused by people speaking or music playing from a range of everyday objects from a glass of water to a crisp packet.

Using video footage recorded by high speed cameras researchers from the Massachusetts Institute of Technology were able to listen to test subjects reading nursery rhymes from behind sound proof glass using only video of a crisp packet in the same room.

Previous techniques that recover sound from vibration have all relied on some sort of active interaction with the vibrating object (such as an infrared beam bouncing off a window using a ‘laser microphone’) but this new research needs only a visual signal.

Computer science graduate Abe Davis and his team were even able to extract audio from footage recorded by an ordinary digital camera and although the quality was good enough to make out distinct speech it was enough to identify information such as the number of people in a room and their gender.

These new techniques could have applications in law enforcement and forensics, but Davis has said that he’s more interested in a “new kind of imaging” that would allow scientists to analyse the material and structural properties of an object just by recording a video of it.

“We’re recovering sounds from objects,” he said in a press release. “That gives us a lot of information about the sound that’s going on around the object, but it also gives us a lot of information about the object itself, because different objects are going to respond to sound in different ways.”

News
i100 In this video, the late actor Leonard Nimoy explains how he decided to use the gesture for his character
Arts and Entertainment
Secrets of JK Rowling's Harry Potter workings have been revealed in a new bibliography
arts + ents
News
Down-to-earth: Winstone isn't one for considering his 'legacy'
people
News
news
Life and Style
ebookNow available in paperback
ebooks
ebookPart of The Independent’s new eBook series The Great Composers
Latest stories from i100
Have you tried new the Independent Digital Edition apps?
Independent Dating
and  

By clicking 'Search' you
are agreeing to our
Terms of Use.

ES Rentals

    iJobs Job Widget
    iJobs Gadgets & Tech

    Ashdown Group: Front-End Developer - London - up to £40,000

    £35000 - £40000 per annum: Ashdown Group: Creative Front-End Developer - Claph...

    Ashdown Group: QA Tester - London - £30,000

    £28000 - £30000 per annum: Ashdown Group: QA Tester - London - £30,000 QA Tes...

    Ashdown Group: Linux Administrator - London - £50,000

    £45000 - £50000 per annum + bonus: Ashdown Group: Linux Systems Administrator ...

    Ashdown Group: Business Intelligence Analyst - London - £45,000

    £40000 - £45000 per annum: Ashdown Group: SQL Server Reporting Analyst (Busine...

    Day In a Page

    HIV pill: Scientists hail discovery of 'game-changer' that cuts the risk of infection among gay men by 86%

    Scientists hail daily pill that protects against HIV infection

    Breakthrough in battle against global scourge – but will the NHS pay for it?
    How we must adjust our lifestyles to nature: Welcome to the 'Anthropocene', the human epoch

    Time to play God

    Welcome to the 'Anthropocene', the human epoch where we may need to redefine nature itself
    MacGyver returns, but with a difference: Handyman hero of classic 1980s TV series to be recast as a woman

    MacGyver returns, but with a difference

    Handyman hero of classic 1980s TV series to be recast as a woman
    Tunnel renaissance: Why cities are hiding roads down in the ground

    Tunnel renaissance

    Why cities are hiding roads underground
    'Backstreet Boys - Show 'Em What You're Made Of': An affectionate look at five middle-aged men

    Boys to men

    The Backstreet Boys might be middle-aged, married and have dodgy knees, but a heartfelt documentary reveals they’re not going gently into pop’s good night
    Crufts 2015: Should foreign dogs be allowed to compete?

    Crufts 2015

    Should foreign dogs be allowed to compete?
    10 best projectors

    How to make your home cinema more cinematic: 10 best projectors

    Want to recreate the big-screen experience in your sitting room? IndyBest sizes up gadgets to form your film-watching
    Manchester City 1 Barcelona 2 player ratings: Luis Suarez? Lionel Messi? Joe Hart? Who was the star man?

    Manchester City vs Barcelona player ratings

    Luis Suarez? Lionel Messi? Joe Hart? Who was the star man at the Etihad?
    Arsenal vs Monaco: Monaco - the making of Gunners' manager Arsene Wenger

    Monaco: the making of Wenger

    Jack Pitt-Brooke speaks to former players and learns the Frenchman’s man-management has always been one of his best skills
    Cricket World Cup 2015: Chris Gayle - the West Indies' enigma lives up to his reputation

    Chris Gayle: The West Indies' enigma

    Some said the game's eternal rebel was washed up. As ever, he proved he writes the scripts by producing a blistering World Cup innings
    In Ukraine a dark world of hybrid warfare and murky loyalties prevails

    In Ukraine a dark world of hybrid warfare

    This war in the shadows has been going on since the fall of Mr Yanukovych
    'Birdman' and 'Bullets Over Broadway': Homage or plagiarism?

    Homage or plagiarism?

    'Birdman' shares much DNA with Woody Allen's 'Bullets Over Broadway'
    Broadchurch ends as damp squib not even David Tennant can revive

    A damp squib not even David Tennant can revive

    Broadchurch, Series 2 finale, review
    A Koi carp breeding pond, wall-mounted iPads and a bathroom with a 'wellness' shower: inside the mansion of Germany's 'Bishop of Bling'

    Inside the mansion of Germany's 'Bishop of Bling'

    A Koi carp breeding pond, wall-mounted iPads and a bathroom with a 'wellness' shower