The silent snoop on your computer

Designed to speed up and enhance the internet experience, web markers known as 'cookies' make a mockery of our privacy. Every web user needs to understand (and control) these insidious cyber-spies, says Stephen Foley

You are not paranoid. You are being followed. Go have a look. Somewhere in your browser - under "privacy" in "preferences"

in Firefox, for example - you can find a list of all the cookies that have been installed on your computer from the websites you once visited, cookies that in all likelihood are pinging themselves back to giant corporate data collectors that are building a profile of where you go, what you do and what you seem to like online. Here are a few of the companies that have put cookies on my computer: aCerno, AdBrite, Adconion, Adify Media, Adgear, Adinterax, AdMeld, Aggregate Knowledge, Akamai, AudienceScience... and that's not even all the As.

I don't know what information these cookies contain, when they were installed and when they might be reactivated or who the data is going to end up with. I do know one thing they all have in common, though: I have never heard ofany of these companies or visited their websites.

I don't doubt the cookies on your computer all come from law-abiding and reputable firms, with wonderful (and wonderfully long and heavily legalled) privacy policies. But wouldn't you like to know just a bit more about why they are there and what they are doing? And while we are at it, what the hell is a cookie, anyway?

This is the story of how a few humble lines of computer code changed the internet and changed the world. It is a story of possibilities unbounded, but it has a dark side, too. For it is also the tale of corporate forces benefiting from our ignorance and posing unprecedented threats to our privacy. It is about what a new generation of technologists are trying to do to protect us - and about what our governments might have to do if they fail. Above all, it is about what we have to learn to protect ourselves. In the beginning, there were just words. Maybe a few pictures, but mainly just words. Surfing the internet was like hopping from page to page inside a book. The largest-ever book, sure, but all just a bit - well, static.

None of the things we now take for granted - all that rich interactivity, the back and forth of engaging with a website, having sites automatically log you in and remember your preferences, being able to buy things with a stored credit card or even get customised, local news and weather - was possible. It was only 17 years ago, but that is prehistory in technology terms. Or if not prehistoric, at least BC. Before the Cookie. The problem was that websites couldn't remember, from page to page, who you were or what you had told them last about where you were and what you wanted to see.

"Websites were more like a Word document, without anything moving," says Lou Montulli, a software engineer who in 1994, just out of college, helped found Netscape, the first really popular browser and a company that ironed out many of those early questions about how websites and personal computers should talk to each other. "Thanks to Netscape the web became an interactive medium and cookies were a strong part of that. Cookies allowed websites to have some memory and that created the modern web as we know it."

Montulli says he ranks the cookie quite far down the list of all his inventions at Netscape. In those heady days, when the full potential of the web was being suddenly grasped, the atmosphere was "euphoric", he says. "We were a very serious engineering company. Maybe we weren't the best business people at times, but we did put forward a lot of the technology that we all take for granted today."

Still, when he calls The Independent to reminisce, Mr Montulli introduces himself as "the cookie guy" and it is his best-known invention. A cookie is just a few lines of text sent from the website to your computer with a unique identification code and an instruction to send the cookie back the next time you hit one of the pages on the site. It is the equivalent of saying, "Hello, it's me". It was Montulli's solution for Netscape business partners that wanted to build an e-commerce site. Without cookies, these proto-Amazons couldn't remember, from one page to the next, what users had put in their shopping cart.

Even at the moment of its invention it was clear that the cookie would be ubiquitous and controversial.Montulli knew its applications would go far beyond e-commerce (it was designed, he said, "to be a tool, like a Swiss Army knife"). There were always going to be privacy concerns, too, with anything that allowed tracking of users on the internet, even if it might initially have seemed to be only about tracking you from one page of a website to another.

Alternative ideas in 1994 for solving the problem of memory included creating a serial number for every Netscape browser, which would have allowed tracking of an individual wherever they went on the web and whatever they did. "Cookies were an attempt to very clearly define a smaller pool of information for sharing,"Montulli says. "I would very strongly argue that cookies did an incredible amount to protect us, versus other technologies."

Cookie use exploded, and not just by websites that wanted to keep track of their users within the site, but also by advertisers. Unlike an ad in a newspaper, which gets sent in by the advertiser and placed on the page by the newspaper's staff, the ads that appear on a website are put there directly by a company called an advertising network. Watch next time you are on a slow internet connection; as you wait for the various pieces of the page to load, you will see "transferring data from..." and "connecting to..." on the bottom of the browser and you can see where they are all coming from.

DoubleClick, now owned by Google, was an early pioneer, but scores and scores quickly sprung up. These networks, too, started sending cookies with the ads, partly as a technical solution (to make sure the ad is served as promised and to track whether a consumer acts on it, something that can be important for payments to the website), but also as a means of storing data as a user hopped from site to site within the network.

With that information, the networks were better able to target ads, adding things like an internet user's location or information that they had given the host site about their age or gender or interests, to the new data they had gained about their browsing habits.

This difference between websites, whose cookies a user might implicitly accept, and ad networks, which are largely invisible and unknown to the user, created instant controversy - but an attempt to ban third-party cookies, or at least to disable them unless a user changed the default settings in the browser, was derailed by an alliance of browser makers (by then including Microsoft) and the ad firms themselves.

Dave Kristol, who led an effort to create technical specifications for cookies and the browsers that sent and received them, recalls how he found himself embroiled in a public-policy debate with powerful factions on all sides. "The process of developing standards and letting all voices be heard is also messy, like most democratic processes," he wrote in 2001, after more than five years of work. "It gives me a new appreciation for how hard it must be to write legislation, even ignoring the distorting influences of lobbyists."

The cookie controversies of the late Nineties and early Noughties includedthunderous hearings in the United States Congress and threats of legislation. The headlines did bring cookies to the attention of some privacyconscious individuals, for whom toughening up their privacy settings or routinely deleting their cookies has become a housecleaning habit. And it was a shot across the bows of the web publishers and the ad networks, who vowed self-regulation. Websites began publishing "privacy policies" that set outwhat they would and would not do with cookie-collected data, while ad networks were careful to insist that all their profiles on individual users would be anonymous and impossible to trace back to a named person. As self-regulation goes, this is what you might call light touch. Most browsers continued to allow third-party cookies by default, as long as the cookies came from "certified " companies only.

Meanwhile, the ad industry and a new generation of digital marketresearch firms got on with the business of assembling massive amounts of data on web users. Developments on the web these past few years have provided these firms with riches they could not have dreamed of a decade ago. We spend more time, and share more, online, providing clues for the harvesters of data. Websites contain content, such as videos, games, social-networking features and multiple ads, served up from a variety of different sources, all usually with cookies or other tracking devices embedded. In breadth and depth, the scale of the data-collection industry is breathtaking. A 2009 report by the University of California, Berkeley found an average of 12 trackers on each of the most-popular 100 websites, with one having 100 different trackers in a month. When a user visits that website, potentially 100 different entities - nearly all unseen by the user - will learn about their visit. One advertising company was able to monitor activity on 91 of the top 100 sites, and 88 per cent of the 350,000 sites tracked by Berkeley.

Last month, the British advertising giant WPP boasted that it had just created the world's largest database of individuals' online behaviour: profiles of 500million people, covering, it says, almost 100 per cent of the people online in the countries in which it operates, including the UK, the US, Australia and eight others. The firm said it was pooling data from many of the world's major websites and networks of online advertisers and adding it to information purchased from traditional market-research firms that assemblereal-world data such as what people are buying in high street stores. That real-world information has been "anonymised", WPP says, before it is added to its database of cookies.

Ashkan Soltani, the privacy researcher who co-authored the Berkeley study, told a Congressional hearing earlier this year that online tracking has become so sophisticated that advertising networks can serve up ads for depression medicine to people who have searched for "depression" on a health site, and that recent computer- science research suggests it is often quite easy to identify the people whose profiles include so much supposedly anonymised user information. Worse, ad networks always seem to be one step ahead of even the most privacyminded internet users. Many now use "Flash cookies", served from sites that use Adobe's programming language Flash, which have the ability to restore other deleted cookies.

Here's the upside. Websites are expected to reap $86bn (£54bn) in revenues from online advertising this year, as marketers have switched their budgets to the net, lured by the opportunity to target adsmore closely to people based on their browsing history. This is money that helps pay for the vast amount of content on the web that is free to users, and without which the quality - if not the quantity - of that content wouldcrash. This is why a new generation of Dave Kristols is trying to square the continued use of third-party cookies with greater control for the user who is concerned about the use to which these cookies are being put.

It is also why web publishers, advertisers and ad networksare working to come up with a new self-regulatory regime, as Congressional hearings and European Union directives threaten them again. And it is why the makers of web browsers - Firefox, Microsoft and Google among them - have started launching new features such as incognito browsing and do-not-track buttons. The trouble now is getting all the firms involved to figure out the technology so that users' preferences can be translated into restrictions on cookie uses that don't curtail all those rich interactive features that cookies can allow.

In the UK, the implementation of an EU directive for internet users to give their consent before cookies are used has been delayed by a year to next May, to provide time to find a workable technology. "I find it most interesting that browser vendors are competing at providing privacy controls now, given how contentious the issue was in 1996," Kristol says. "One difference is that I think the various browser vendors - and there are more now - are working together in good faith. I didn't feel that was so in 1996."

The ad industry, too, is trying to coalesce around user-friendly solutions and has set up two websites - Network Advertising Initiative and Digital Advertising Alliance - where users can goto discover whose cookies might be on their computer. The industry has a few tricks up its sleeve to protect itself, though. Some of the opt-out options it purports to be offering allow the users to opt out only of receiving targeted ads - not to have their data deleted fromthese giant databases.

And the technology used to ensure that these ad networks remember you have opted out is none other than the cookie. People who habitually delete their cookies for privacy's sake may find that by doing so, they are opting to have their privacy invaded again. The cookie guy looks at all these new options, whether on these industry websites or in the new generation of privacy-conscious browsers, and expresses some pride. "It is difficult for the layman - it is difficult even for me - toknow how cookies are being used in all circumstances, and it is difficult to control their use,"Montulli says. "But you have a lot of power over cookies today, if you choose to use it. And that is because of the base design for the cookie."

News
Young Winstone: His ‘tough-guy’ image is a misconception
people
Sport
Adnan Januzaj and Gareth Bale
footballManchester United set to loan out Januzaj to make room for Bale - if a move for the Welshman firms up
Arts and Entertainment
Ellie Levenson’s The Election book demystifies politics for children
bookNew children's book primes the next generation for politics
News
Outspoken: Alexander Fury, John Rentoul, Ellen E Jones and Katy Guest
newsFrom the Scottish referendum to the Ice Bucket Challenge, our writers voice their opinions
PROMOTED VIDEO
Life and Style
ebookNow available in paperback
ebooks
ebookPart of The Independent’s new eBook series The Great Composers
News
i100
Sport
Yaya Sanogo, Mats Hummels, Troy Deeney and Adnan Januzaj
footballMost Premier League sides are after a striker, but here's a full run down of the ins and outs that could happen over the next month
Arts and Entertainment
L to R: Hawkeye (Jeremy Renner), Captain America (Chris Evans) & Black Widow (Scarlett Johansson) in Avengers Assemble
film
News
Nigel Farage celebrates with a pint after early local election results in the Hoy and Helmet pub in South Benfleet in Essex
peopleHe has shaped British politics 'for good or ill'
Arts and Entertainment
Pharrell Williams' “Happy” was the most searched-for song lyric of 2014
musicThe power of song never greater, according to our internet searches
Sport
Tim Sherwood raises his hand after the 1-0 victory over Stoke
footballFormer Tottenham boss leads list of candidates to replace Neil Warnock
Arts and Entertainment
Sink the Pink's 2013 New Year's Eve party
musicFour of Britain's top DJs give their verdict on how to party into 2015
Voices
Strictly Come Dancing was watched by 6.9m viewers
voicesIt has been hard to form generally accepted cultural standards since the middle of the 19th century – and the disintegration is only going to accelerate, says DJ Taylor
Arts and Entertainment
Roffey says: 'All of us carry shame and taboo around about our sexuality. But I was determined not to let shame stop me writing my memoir.'
books
News
i100
News
Caplan says of Jacobs: 'She is a very collaborative director, and gives actors a lot of freedom. She makes things happen.'
people
Latest stories from i100
Have you tried new the Independent Digital Edition apps?
Independent Dating
and  

By clicking 'Search' you
are agreeing to our
Terms of Use.

ES Rentals

    iJobs Job Widget
    iJobs Gadgets & Tech

    Ashdown Group: Moodle Developer (PHP ,Linux, Apache, MySQL, Moodle)

    £35000 - £45000 per annum: Ashdown Group: Moodle Developer (PHP ,Linux, Apache...

    Recruitment Genius: Web Developer

    £17000 - £30000 per annum: Recruitment Genius: This is a fantastic opportunity...

    Recruitment Genius: Junior .NET Web Developer - Winform / MVC

    £21000 - £26000 per annum: Recruitment Genius: This Award-winning pharma softw...

    Recruitment Genius: Senior Java Developer

    £30000 - £45000 per annum: Recruitment Genius: A Senior Java Developer is requ...

    Day In a Page

    War with Isis: The West needs more than a White Knight

    The West needs more than a White Knight

    Despite billions spent on weapons, the US has not been able to counter Isis's gruesome tactics, says Patrick Cockburn
    Return to Helmand: Private Davey Graham recalls the day he was shot by the Taliban

    'The day I was shot by the Taliban'

    Private Davey Graham was shot five times during an ambush in 2007 - it was the first, controversial photograph to show the dangers our soldiers faced in Helmand province
    Revealed: the best and worst airlines for delays

    Revealed: the best and worst airlines for delays

    Many flyers are failing to claim compensation to which they are entitled, a new survey has found
    The stories that defined 2014: From the Scottish independence referendum to the Ice Bucket Challenge, our writers voice their opinions

    The stories that defined 2014

    From the Scottish independence referendum to the Ice Bucket Challenge, our writers voice their opinions
    Stoke-on-Trent becomes first British city to be classified as 'disaster resilient' by the United Nations

    Disaster looming? Now you know where to head...

    Which British city has become the first to be awarded special 'resilience' status by the UN?
    Finally, a diet that works: Californian pastor's wildly popular Daniel Plan has seen his congregation greatly reduced

    Finally, a diet that works

    Californian pastor's wildly popular Daniel Plan has seen his congregation greatly reduced
    Say it with... lyrics: The power of song was never greater, according to our internet searches

    Say it with... lyrics

    The power of song was never greater, according to our internet searches
    Professor Danielle George: On a mission to bring back the art of 'thinkering'

    The joys of 'thinkering'

    Professor Danielle George on why we have to nurture tomorrow's scientists today
    Monique Roffey: The author on father figures, the nation's narcissism and New Year reflections

    Monique Roffey interview

    The author on father figures, the nation's narcissism and New Year reflections
    Introducing my anti-heroes of 2014

    Introducing my anti-heroes of 2014

    Their outrageousness and originality makes the world a bit more interesting, says Ellen E Jones
    DJ Taylor: Good taste? It's all a matter of timing...

    Good taste? It's all a matter of timing...

    It has been hard to form generally accepted cultural standards since the middle of the 19th century – and the disintegration is only going to accelerate, says DJ Taylor
    Olivia Jacobs & Ben Caplan: 'Ben thought the play was called 'Christian Love'. It was 'Christie in Love' - about a necrophiliac serial killer'

    How we met

    Olivia Jacobs and Ben Caplan
    Bill Granger recipes: Our chef's breakfasts will revitalise you in time for the New Year

    Bill Granger's healthy breakfasts

    Our chef's healthy recipes are perfect if you've overindulged during the festive season
    Transfer guide: From Arsenal to West Ham - what does your club need in the January transfer window?

    Who does your club need in the transfer window?

    Most Premier League sides are after a striker, but here's a full run down of the ins and outs that could happen over the next month
    The Last Word: From aliens at FA to yak’s milk in the Tour, here’s to 2015

    Michael Calvin's Last Word

    From aliens at FA to yak’s milk in the Tour, here’s to 2015