New controllers: The future is in your hands

Our TVs will soon no longer be controlled by traditional remotes, but by fully integrated apps and devices that can flash up captions and even let us play games. Andy Robertson reports

If an Englishman's home is his castle, perhaps the living room is his country getaway. Retreating nightly to the opulent pleasures of modern leisure sees him whiling away evenings in front of an impressive array of technology. However, an increasing number of boxes, screens, controllers and cables can sometimes make a night in front of the television more trouble than it's worth. Surely some of this technological might could be spent on making things a little easier to use?

With the passing in May of Eugene Polley, creator of the television remote control, we'd do well to remember the value of usability. His 1955 Flash-Matic controller used light to change the television channel. It may look rudimentary by today's standards but Polley's invention instantly saved couch potatoes from billions trips from the settee to the television set.

Television remotes have since proliferated in the living room as VCRs, DVDs and televisions each required a separate controller. The universal remote came to the rescue in the 1990s and offered the illusion that our different boxes were in some way co-operating with each other.

It's now the number of screens rather than the number of controllers that makes relaxing such hard work. Smartphones, tablet devices and the television screen each demand our attention. However, looking up information about navigating and searching Electronic Program Guides (EPGs) and switching between different operating systems is a confusing and frustrating experience.

In California last month at the Electronic Entertainment Expo (E3), Microsoft unveiled its vision of the modern-day universal remote. Xbox SmartGlass is a free app for mobile phones and tablet computers that enables you to browse and launch video, music and game content on your Xbox 360. Nothing hugely revolutionary there. But, because SmartGlass knows what you are watching, playing or listening to, it can then feed back related information to your phone or tablet. While you are watching a television show it provides details on the cast, other episodes and even the virtual location of the show, without the need to go looking for this manually.

It's this potential for simplifying how we use the different screens in our living room that makes SmartGlass so notable. "Xbox SmartGlass was developed within the Interactive Entertainment Business, where we believe that your devices should work together to enhance your entertainment experience" says Lisa Worthington, senior PR manager for Interactive Entertainment at Microsoft. "We set out to answer the following question: what if your tablet or phone knew what you were watching on TV and presented bonus features and additional relevant experiences without you having to lift a finger?"

Peter Orullian, product manager for SmartGlass, at E3 describes it as "the only companion app you need for the device you already own". Because SmartGlass is launching not only on Windows devices, but also on iOS and Android it can stitch together these operating systems and greatly simplifying the user experience.

There is a big incentive here for Microsoft. Whereas product manufacturers used to clamour to be the box under our television screens, now the desire is to be the device we pick up on entering the living room – and more recently, to be the software running on that device.

Create an application or tablet that becomes the go-to gadget when we enter the living room and you have the audience's eye, along with the opportunity to curate their viewing much like a traditional television channel. Xbox SmartGlass could be Microsoft's Trojan horse to circumvent much of its competition. It will be interesting to see how much resistance there is from other platform holders to publishing the app.

Although SmartGlass is unique for its two-way flow of information, each platform holder seems to have similar aspirations. Apple television, PlayStation 3's Play television service and Nintendo's upcoming tablet-controlled Wii U console each offer a way to browse and select content for the main television screen via a remote device.

For Apple it's the iPad or iPhone and for the PS3 it's the PSP or PS Vita handheld that offer the second screen. Nintendo takes this a step further by including infrared technology in their new Wii U controller that means you can turn the television on as well as navigating content.

Here the Wii U outmanoeuvres the plethora of Smartphone apps that promise to replace your television remote. Because most mobile phone and tablets don't include infrared technology, apps such as L5 Remote can only control a television by plugging in an infrared transmitting dongle. Other apps such as the Samsung Remote control Smart televisions via their built-in Wi-Fi function – but are still a long way from being a universal controller.

Worthington clarifies this point. "The intent for Xbox SmartGlass is to make your devices work together in an intelligent manner... It does not control your television independent of the Xbox 360 console." If SmartGlass has an Achilles heel this is it, picking up something else to turn on the television is a chance for another device to arrest the viewer's attention. Additionally, although the app itself is free, you will need to invest in an Xbox 360 (£129.99) to use it if you don't already have one.

When asked about the Wii U's television remote feature, Worthington responds "Xbox SmartGlass works with the devices you already own: your television, your tablet or PC and your phone, and works across music, sports, videos, movies, television shows and games on Xbox." Although understandably oblique, what she doesn't mention is that in terms of video-game controls SmartGlass offers similar interactions to the new Wii U tablet controller.

Orullian addressed this when introducing the SmartGlass. "Games we've got such as Homerun Stars use SmartGlass as an input device so you can articulate a pitch or batting." This offers the sort of experience that Nintendo are leveraging for the Wii U. But here, like Microsoft's Kinect, the cost is kept to a minimum as the controller is an expansion for your existing console – whereas Wii owners will require a completely new system to use the Wii U tablet.

Microsoft has promised to commit to SmartGlass, like it did with Kinect, and leverage its functionality across the line-up of Xbox 360 games. If it employs a similar level of investment to Kinect, which became the fastest selling device on record in 2011, it is hard not to see the app becoming similarly popular when it launches later this year.

With many expecting game consoles to move to the cloud in coming years, essentially removing the need for hardware in the home (something which gathered pace this week with Sony's proposed purchase of cloud gaming company Gaikai), second screen and controller functionality like this will help platform holders maintain their relevance. Regardless of where video-game processing is performed, how we access and interact with games will remain a matter for the controllers we are holding in our living rooms.

Beyond the games themselves, it seems that tablet apps such as SmartGlass are perfectly placed to "join up" the technology in our homes. This progress will certainly make it harder for Apple's television set, rumoured to arrive next year and link television, entertainment, games and the web all controlled by an iPad or iPhone, to dominate – although the Apple magic may still manage to overtake the competition as it has in other sectors.

As these new fronts open up in the battle for the living room, guessing the eventual winners is still no easy task. The most positive development with SmartGlass is a shift towards making existing equipment more accessible and useable rather than selling more boxes to go under our TV screens.

Never leave the sofa again: The new remotes

Xbox SmartGlass app

An app for Windows, Android and iOS devices that enables you to control your Xbox 360 with your smartphone. It uniquely provides secondary information about the content you are watching on your television, back on your smartphone or tablet device.

Wii U Tablet

The new tablet controller for the new Wii U console. It provides a screen that enables you to control games and watch content. It also provides an infrared function to turn on your television.

PlayStation 3 Remote Play

Control your PlayStation 3 console, watch content and play certain games via the PSP and PS Vita's Remote Play feature.

Apple TV Remote app

The app uses Apple's AirPlay technology to enable you to select content via iTunes on a computer or on Apple TV connected to your television. It enables you to use gestures to navigate and provides album and television show artwork back on your iPhone or iPad.

Samsung Remote app

This app turns on a Samsung Smart TV. It enables the smartphone to not only browse content, but also be used as a keyboard and game controller on these devices.

L5 Remote app

Use the iOS app to control your television DVD, Digi-box like a traditional universal remote. It requires the purchase of an infrared transmitter that plugs into the bottom of the iPhone, iPad or iPod Touch.