The future according to Microsoft: the Magic Window revolutionizes video chat

<b>David Phelan</b> has an exclusive peek behind the scenes at Microsoft’s research and development facilities – here’s his first report

Steve Bathiche demonstrating how to reach through the screen using Microsoft's Magic Window technology

It may sound like something out of Play School, but the Magic Window is amazing out-there technology. Imagine a wall-sized window with you on one side and a friend on the other. Scrawl on the window with a marker pen and your friend can see it from the other side.

That’s all the Magic Window does, except that it’s really a screen, so you and your friend can be on opposite sides of the world.

It sounds simple, but making it happen is fiendishly difficult. We have two eyes, so we see in stereo. Steve Bathiche, a fast-talking boffin whose Microsoft team is behind the Magic Window explained that the window had to see in stereo, too.

“If you look around the window and move your head, the view changes. So we track the viewer’s head so the view changes as you move. This is what’s called motion view parallax and is a powerful technique to give the user the illusion of looking through a window.”

But an illusion of depth only comes through mimicking our eyes – so you need two cameras working together – and without the glasses you need for 3D TVs. This means using a special lens called the Wedge in conjunction with a motion sensor that knows where your eyes are. And that motion sensor? Well, you may have come across one already. It’s the Kinect sensor that comes with the Xbox.

In fact, it was Microsoft who developed this when exploring 3D movement, and when the Xbox team needed a sensor, they found the company had already invented one.

The Kinect is an example of how research into one thing, like the Magic Window, can have powerful side-effects. So, for instance, when working on transparent screens, the company devised a great way to improve video calls. When you have a see-through screen, you can place a camera right in the middle directly behind it.

You’ll be familiar with video calling apps like FaceTime on the iPhone and iPad: ever have a lingering feeling that you weren’t quite connecting? Some of that’s down to the way you and your caller are looking at the face in the middle of the display, not the camera at the top edge.

With this technology you create a video call where you can look your caller straight in the eyes and lose that feeling that the person you’re talking to is looking somewhere else. The result is a much better experience. Maybe, if that technology reaches fruition, it’ll be enough to actually persuade us to make video calls. Maybe not.

Even so, the transparent screen also lets you interact in different ways, grabbing the air behind the screen to affect what’s on the display. In practice, this takes some getting used to, but it’s fun. And very Minority Report.

When Steve’s team worked on making the Magic Window work for more than one person, they found a way to create a split screen where two people can play a video game at the same time, but see completely different screens. This technology is already available for some TVs and means two people can watch different channels simultaneously on the same TV (though at least one needs to wear headphones and it could feel somewhat isolating).

Steve has many more problems to solve before the Window is ready for home use, and he wouldn’t put a time frame on it, but if and when it happens the results will be realistic… but with extra features.

One person will be able to write a word on the window but – unlike a glass window, the Microsoft one can translate the word so the person on the other side sees it in their language.

The Magic Window exemplifies how crucial research and development departments are and how important it is to stay with developing technologies even if there’s no immediate implementation in sight.