Filter failure: Too much information?

Thousands of TV channels. Millions of web pages. We're drowning in the stream of data that surrounds us. We shouldn't try to stem the flood – just to filter it, argues Rhodri Marsden

Barring a grisly accident or the invention of some kind of immortality serum, I've probably got another 30 years or so left on this planet. Ten of those will be spent unconscious under a duvet (not consecutively, I hasten to add), leaving me 20 that I should optimise for maximum return and do my best not to fritter away. Some will be spent trying to achieve some notion of success – you know, make my parents proud, earn a bit of cash. I'll use others to absorb stimulating prose by leading thinkers, or alleviate stress via mindless entertainment, or perform dull administrative tasks to make all the other stuff achievable.

But earlier today I wasted 30 seconds reading an email inviting me to attend a children's fascinator-making craft session. I've no idea why I received it. I have no children and even if I did I reckon that small, decorative hats would play no part in their personal development. But I was sent the thing, I read it – and then I deleted it with an irritable click because it dared to waste my precious time. Time that could have been spent watching some bloke perform the world's first BMX triple backflip on YouTube. You know, important stuff.

The internet presents us with a dizzying array of choices that we navigate badly and with bad temper. We're increasingly resentful of being digitally waved at by thousands of companies, friends, musicians, charities, film studios, writers, news organisations and children's fascinator-making craft-session organisers who all want our time, our money or our opinions. This is usually called "information overload", but perhaps a more appropriate term – coined by the American writer Clay Shirky – is "filter failure". When the filters fail, we're confronted by things we have no interest in. Spam ends up in our inbox. We get inappropriate gifts for Christmas because of eye-catching offers on websites stating "people who bought this item also bought...". We encounter comments on the internet from people we'd normally flee buildings to avoid. Technology is constantly being developed to help us cope: search engines, for example, gather information about our behaviour to try to propel the kind of stuff we like to the top and hide the dross on page three or four, while a range of apps help us to prioritise and re-establish some structure in our online adventures. But we become resentful of search engines for daring to presume anything about us and the apps can be so irritatingly complex that we end up slinging them into the digital skip. There's too much noise, not enough signal and we moan about it constantly.

Rarely is this a life-or-death issue. It's usually a shrill, middle-class whine borne out of privilege, luxury and boredom – perhaps a niggling worry that you've missed something, be it news from family, friends or war-torn Afghanistan. Or it's feeling oppressed by a stack of DVD box sets that you seem to have accumulated but don't have time to watch, a pile of magazines you haven't managed to read or irritation at several hundred television channels that all fail to offer something that matches your mood.

Filter failure gives rise to frustration and a nostalgia for a supposedly simpler time. It's always been like this. A decade ago people got misty-eyed about the internet of the mid-1990s, when you could read everything you wanted to read and still have time to argue with your partner before bedtime about excessive computer use. A fourth terrestrial television channel was once considered by many to be unnecessary. Gutenberg put the fear of God into 15th-century scholars when they were suddenly confronted with a choice of books to read. And at some point in history, multiple tones from an African drum will have felt like too much information.

Traditionally, we've coped with this by constructing our own filters. We have our favourite television channels to which we instinctively head and our preferred newspapers and radio stations. If an envelope drops on the doormat telling us that we "may already have won a cash prize", we can fling it nonchalantly into the bin without reading it because we know that we haven't won anything. Online tools help us construct yet more filters. We can choose to send any email containing the phrase "bigger penis" to the trash. RSS feeds alert us to our favourite blogs being updated without us having to repeatedly check them. A whole technological ecosystem has developed around Twitter to help us mute dullards and tap us on the shoulder when someone interesting turns up. But the more information there is, the more onerous filtering gets. We need some help.

There have always been people and organisations filtering on our behalf. Editorial filtering ensures that this newspaper doesn't bring you, say, palpably untrue scare stories about immigration. These decisions are made for you.

The people present at the weekly meeting to decide the contents of the Radio 1 playlist exercise huge power over the nation's listening habits, because millions of us are happy to let them make that choice, rather than spend eight hours a day listening to new releases. The food we consume is largely dictated by decisions made by the head buyers at the big supermarkets. But one of the reasons the internet feels so thrilling is that it liberates us from editorial filters; suddenly everything is out there for us to grab – and much of it for free.

Multiple news sources, endless entertainment possibilities, limitless communication with countless people. Our yearning for the bigger picture is quickly and dramatically fulfilled. But when we get the bigger picture, it's bloody confusing and we suddenly yearn for a smaller picture. Maybe a different smaller picture than the one we had before. Broader... but smaller? Is that even realistic? Our friends aren't helping us downsize. We've relied on word-of-mouth information since the dawn of time, but social media have taken this age-old concept and tried to thrash it into some kind of working business model.

Take social bookmarking, which works on the basis that if a whole bunch of people like a particular website, you'll probably like it too. But is sheer popularity the best filtering system? A cursory scan down this week's top 40 albums suggests that it isn't. Do I want my consumption to be driven by the online herd? And who are these people anyway?

Social-media services beg us to make connections with as many other people as possible, which has, in turn, created a fearsome social overload. I'm suddenly bonded with this bunch of people, many of them strangers, who now have a disproportionate effect on the data I consume – cultural, political, ephemeral. The services respond to our annoyance at excessive connection by offering us more filters to help us cope – Twitter and Facebook "lists", for example, which allow us to separate the people we like from the tedious numpties they've brought within our ambit. Thanks for that, chaps.

Of course, it's easy to rail against Facebook. But some of it is unquestionably its fault. I never used to give more than a dozen birthday cards a year, but every morning I'm now informed by Facebook that it's someone's birthday and I feel obliged to go and wish them many happy returns – along with about 200 other people they probably met once or twice, if at all.

It's impossible to process this quantity of cheery birthday greetings and most of them are probably ignored. But we're complicit. After all, we're not forced to add to the information pool, but we do it anyway. Oversharing, as it's quaintly termed, sees us generate huge quantities of text, audio, photos and video, which we expect other people to consume; we feel vaguely irritated when they choose not to do so.

Some services can't decide whether to filter information for us or encourage supply, so they do both; Newsvine launched a few years back to streamline news consumption and only bring you the stories in which you were interested – but it also urged you to pen your own columns. It offered a filter, then pushed so much through that filter that many people were, in the words of one site tester, "completely adrift in a sea of confusion". But hey, that's the 21st-century web. People will upload pictures and video and post comments and blogs. You can't ask them not to do so. The genie is out of that bottle, dragging umpteen terabytes of data behind it.

The magic solution to all this – at least, in theory – lies in computer algorithms. Google's search engine reportedly has 57 "signals" that shape search results in different ways for each individual user, tailoring them to stop us from getting annoyed at irrelevant stuff or, heaven forbid, switching to another search engine instead. New web services constantly reassure us that they won't overload us because their algorithms – their ability to second guess us – are masterpieces of software engineering. But there's a limit to what they can achieve. Facebook, for example, keeps track of the people with whom we interact most often and prioritises news from those people. Because of this, I know that my friend Tommy just had a bad experience on eBay with a bloke in Finland who won't pay up – but an old friend of mine I'm not in regular touch with had a baby last month and I didn't even know she was pregnant. Facebook could never have known that I wanted to know about this.

Similarly, Google recently launched a "Priority Inbox" service which intuitively learns which emails are important to you. But I get emails every week from a service called Songkick, telling me about gigs by bands that appear in my iTunes library; I delete almost all of them without opening them, but once in a blue moon the subject line will alert me that Todd Rundgren is coming to town and I'll open it eagerly. Google has no idea that I like Todd Rundgren, because I haven't told it. Maybe I should.

One school of thought believes that this kind of filter failure – the ignorance of the algorithm – is having a more insidious effect. In his book The Filter Bubble: What the Internet is Hiding From You, Eli Pariser suggests that this tailoring of internet content for personal consumption isolates us "in a web of one"; we're offered links that we might like, we reinforce that supposition by clicking on them, which ends up creating a very narrow, highly distilled and somewhat distorted information channel. The algorithms may be doing their job perfectly, but Pariser believes that relying on them to combat information overload could have grim consequences for media, community and democracy.

What's the alternative? If I knew, I'd probably be sipping a cool drink next to a swimming pool in Palo Alto while the phone rings off the hook. Perhaps the most interesting development is "personalised serendipity" or "unexpected relevance"; services such as My6Sense and TrapIt, for example, are filters that emulate a more natural discovery process, throwing up stuff that traditional algorithms might not. But they're still automated filters that are filtering a bunch of filters; to succeed, the filters have to gain our trust and some of us are running out of patience.

In lifestyle magazines you'll regularly read the tales of some writer or other who spent a week in "digital detox", a cold-turkey scenario that supposedly offers perspective on the information age – but it's only of anecdotal value.

Modern life just isn't like that. The internet has become indispensable; that's not simply the view of a bespectacled geek like myself, that's the view of many governments around the world who are trying to establish internet access as a human right.

Essentially, it's up to us. It's a mental issue. Our frustration at filter failure wouldn't exist if we didn't feel the pressure to keep up with the information flow. We don't need to read everything we're told to read or watch everything that we're told to watch. Completism is dead; it's a matter of dipping in and dipping out. Don't worry. Just let the information go.