The silent snoop on your computer

Designed to speed up and enhance the internet experience, web markers known as 'cookies' make a mockery of our privacy. Every web user needs to understand (and control) these insidious cyber-spies, says Stephen Foley

You are not paranoid. You are being followed. Go have a look. Somewhere in your browser - under "privacy" in "preferences"

in Firefox, for example - you can find a list of all the cookies that have been installed on your computer from the websites you once visited, cookies that in all likelihood are pinging themselves back to giant corporate data collectors that are building a profile of where you go, what you do and what you seem to like online. Here are a few of the companies that have put cookies on my computer: aCerno, AdBrite, Adconion, Adify Media, Adgear, Adinterax, AdMeld, Aggregate Knowledge, Akamai, AudienceScience... and that's not even all the As.

I don't know what information these cookies contain, when they were installed and when they might be reactivated or who the data is going to end up with. I do know one thing they all have in common, though: I have never heard ofany of these companies or visited their websites.

I don't doubt the cookies on your computer all come from law-abiding and reputable firms, with wonderful (and wonderfully long and heavily legalled) privacy policies. But wouldn't you like to know just a bit more about why they are there and what they are doing? And while we are at it, what the hell is a cookie, anyway?

This is the story of how a few humble lines of computer code changed the internet and changed the world. It is a story of possibilities unbounded, but it has a dark side, too. For it is also the tale of corporate forces benefiting from our ignorance and posing unprecedented threats to our privacy. It is about what a new generation of technologists are trying to do to protect us - and about what our governments might have to do if they fail. Above all, it is about what we have to learn to protect ourselves. In the beginning, there were just words. Maybe a few pictures, but mainly just words. Surfing the internet was like hopping from page to page inside a book. The largest-ever book, sure, but all just a bit - well, static.

None of the things we now take for granted - all that rich interactivity, the back and forth of engaging with a website, having sites automatically log you in and remember your preferences, being able to buy things with a stored credit card or even get customised, local news and weather - was possible. It was only 17 years ago, but that is prehistory in technology terms. Or if not prehistoric, at least BC. Before the Cookie. The problem was that websites couldn't remember, from page to page, who you were or what you had told them last about where you were and what you wanted to see.

"Websites were more like a Word document, without anything moving," says Lou Montulli, a software engineer who in 1994, just out of college, helped found Netscape, the first really popular browser and a company that ironed out many of those early questions about how websites and personal computers should talk to each other. "Thanks to Netscape the web became an interactive medium and cookies were a strong part of that. Cookies allowed websites to have some memory and that created the modern web as we know it."

Montulli says he ranks the cookie quite far down the list of all his inventions at Netscape. In those heady days, when the full potential of the web was being suddenly grasped, the atmosphere was "euphoric", he says. "We were a very serious engineering company. Maybe we weren't the best business people at times, but we did put forward a lot of the technology that we all take for granted today."

Still, when he calls The Independent to reminisce, Mr Montulli introduces himself as "the cookie guy" and it is his best-known invention. A cookie is just a few lines of text sent from the website to your computer with a unique identification code and an instruction to send the cookie back the next time you hit one of the pages on the site. It is the equivalent of saying, "Hello, it's me". It was Montulli's solution for Netscape business partners that wanted to build an e-commerce site. Without cookies, these proto-Amazons couldn't remember, from one page to the next, what users had put in their shopping cart.

Even at the moment of its invention it was clear that the cookie would be ubiquitous and controversial.Montulli knew its applications would go far beyond e-commerce (it was designed, he said, "to be a tool, like a Swiss Army knife"). There were always going to be privacy concerns, too, with anything that allowed tracking of users on the internet, even if it might initially have seemed to be only about tracking you from one page of a website to another.

Alternative ideas in 1994 for solving the problem of memory included creating a serial number for every Netscape browser, which would have allowed tracking of an individual wherever they went on the web and whatever they did. "Cookies were an attempt to very clearly define a smaller pool of information for sharing,"Montulli says. "I would very strongly argue that cookies did an incredible amount to protect us, versus other technologies."

Cookie use exploded, and not just by websites that wanted to keep track of their users within the site, but also by advertisers. Unlike an ad in a newspaper, which gets sent in by the advertiser and placed on the page by the newspaper's staff, the ads that appear on a website are put there directly by a company called an advertising network. Watch next time you are on a slow internet connection; as you wait for the various pieces of the page to load, you will see "transferring data from..." and "connecting to..." on the bottom of the browser and you can see where they are all coming from.

DoubleClick, now owned by Google, was an early pioneer, but scores and scores quickly sprung up. These networks, too, started sending cookies with the ads, partly as a technical solution (to make sure the ad is served as promised and to track whether a consumer acts on it, something that can be important for payments to the website), but also as a means of storing data as a user hopped from site to site within the network.

With that information, the networks were better able to target ads, adding things like an internet user's location or information that they had given the host site about their age or gender or interests, to the new data they had gained about their browsing habits.

This difference between websites, whose cookies a user might implicitly accept, and ad networks, which are largely invisible and unknown to the user, created instant controversy - but an attempt to ban third-party cookies, or at least to disable them unless a user changed the default settings in the browser, was derailed by an alliance of browser makers (by then including Microsoft) and the ad firms themselves.

Dave Kristol, who led an effort to create technical specifications for cookies and the browsers that sent and received them, recalls how he found himself embroiled in a public-policy debate with powerful factions on all sides. "The process of developing standards and letting all voices be heard is also messy, like most democratic processes," he wrote in 2001, after more than five years of work. "It gives me a new appreciation for how hard it must be to write legislation, even ignoring the distorting influences of lobbyists."

The cookie controversies of the late Nineties and early Noughties includedthunderous hearings in the United States Congress and threats of legislation. The headlines did bring cookies to the attention of some privacyconscious individuals, for whom toughening up their privacy settings or routinely deleting their cookies has become a housecleaning habit. And it was a shot across the bows of the web publishers and the ad networks, who vowed self-regulation. Websites began publishing "privacy policies" that set outwhat they would and would not do with cookie-collected data, while ad networks were careful to insist that all their profiles on individual users would be anonymous and impossible to trace back to a named person. As self-regulation goes, this is what you might call light touch. Most browsers continued to allow third-party cookies by default, as long as the cookies came from "certified " companies only.

Meanwhile, the ad industry and a new generation of digital marketresearch firms got on with the business of assembling massive amounts of data on web users. Developments on the web these past few years have provided these firms with riches they could not have dreamed of a decade ago. We spend more time, and share more, online, providing clues for the harvesters of data. Websites contain content, such as videos, games, social-networking features and multiple ads, served up from a variety of different sources, all usually with cookies or other tracking devices embedded. In breadth and depth, the scale of the data-collection industry is breathtaking. A 2009 report by the University of California, Berkeley found an average of 12 trackers on each of the most-popular 100 websites, with one having 100 different trackers in a month. When a user visits that website, potentially 100 different entities - nearly all unseen by the user - will learn about their visit. One advertising company was able to monitor activity on 91 of the top 100 sites, and 88 per cent of the 350,000 sites tracked by Berkeley.

Last month, the British advertising giant WPP boasted that it had just created the world's largest database of individuals' online behaviour: profiles of 500million people, covering, it says, almost 100 per cent of the people online in the countries in which it operates, including the UK, the US, Australia and eight others. The firm said it was pooling data from many of the world's major websites and networks of online advertisers and adding it to information purchased from traditional market-research firms that assemblereal-world data such as what people are buying in high street stores. That real-world information has been "anonymised", WPP says, before it is added to its database of cookies.

Ashkan Soltani, the privacy researcher who co-authored the Berkeley study, told a Congressional hearing earlier this year that online tracking has become so sophisticated that advertising networks can serve up ads for depression medicine to people who have searched for "depression" on a health site, and that recent computer- science research suggests it is often quite easy to identify the people whose profiles include so much supposedly anonymised user information. Worse, ad networks always seem to be one step ahead of even the most privacyminded internet users. Many now use "Flash cookies", served from sites that use Adobe's programming language Flash, which have the ability to restore other deleted cookies.

Here's the upside. Websites are expected to reap $86bn (£54bn) in revenues from online advertising this year, as marketers have switched their budgets to the net, lured by the opportunity to target adsmore closely to people based on their browsing history. This is money that helps pay for the vast amount of content on the web that is free to users, and without which the quality - if not the quantity - of that content wouldcrash. This is why a new generation of Dave Kristols is trying to square the continued use of third-party cookies with greater control for the user who is concerned about the use to which these cookies are being put.

It is also why web publishers, advertisers and ad networksare working to come up with a new self-regulatory regime, as Congressional hearings and European Union directives threaten them again. And it is why the makers of web browsers - Firefox, Microsoft and Google among them - have started launching new features such as incognito browsing and do-not-track buttons. The trouble now is getting all the firms involved to figure out the technology so that users' preferences can be translated into restrictions on cookie uses that don't curtail all those rich interactive features that cookies can allow.

In the UK, the implementation of an EU directive for internet users to give their consent before cookies are used has been delayed by a year to next May, to provide time to find a workable technology. "I find it most interesting that browser vendors are competing at providing privacy controls now, given how contentious the issue was in 1996," Kristol says. "One difference is that I think the various browser vendors - and there are more now - are working together in good faith. I didn't feel that was so in 1996."

The ad industry, too, is trying to coalesce around user-friendly solutions and has set up two websites - Network Advertising Initiative and Digital Advertising Alliance - where users can goto discover whose cookies might be on their computer. The industry has a few tricks up its sleeve to protect itself, though. Some of the opt-out options it purports to be offering allow the users to opt out only of receiving targeted ads - not to have their data deleted fromthese giant databases.

And the technology used to ensure that these ad networks remember you have opted out is none other than the cookie. People who habitually delete their cookies for privacy's sake may find that by doing so, they are opting to have their privacy invaded again. The cookie guy looks at all these new options, whether on these industry websites or in the new generation of privacy-conscious browsers, and expresses some pride. "It is difficult for the layman - it is difficult even for me - toknow how cookies are being used in all circumstances, and it is difficult to control their use,"Montulli says. "But you have a lot of power over cookies today, if you choose to use it. And that is because of the base design for the cookie."

News
Alan Bennett has criticised the “repellent” reality shows which dominate our screens
tvBut he does like Stewart Lee
Life and Style
The Google Doodle celebrating the start of the first day of autumn, 2014.
tech
Arts and Entertainment
Sheridan Smith as Cilla Black and Ed Stoppard as her manager Brian Epstein
tvCilla Episode 2 review: Grit under the glamour in part two of biopic series starring Sheridan Smith
Sport
David Moyes and Louis van Gaal
football
PROMOTED VIDEO
Life and Style
ebooksA superb mix of recipes serving up the freshest of local produce in a delicious range of styles
Life and Style
ebooksFrom the lifespan of a slug to the distance to the Sun: answers to 500 questions from readers
News
i100
Life and Style
Vote with your wallet: the app can help shoppers feel more informed about items on sale
lifeNew app reveals political leanings of food companies
News
Former Governor of Alaska Sarah Palin, left, with her daughter, Bristol
newsShe's 'proud' of eldest daughter, who 'punched host in the face'
Sport
New Zealand fly-half Aaron Cruden pictured in The Zookeeper's Son on a late-night drinking session
rugby
Arts and Entertainment
Salmond told a Scottish television chat show in 2001that he would also sit in front of a mirror and say things like,
tvCelebrity Trekkies from Alex Salmond to Barack Obama
Life and Style
Carol O'Brien, whose son Rob suffered many years of depression
healthOne mother's story of how London charity Maytree helped her son with his depression
Arts and Entertainment
The cover of Dark Side of the Moon
musicCan 'The Endless River' carry on the tradition? See for yourself
Life and Style
food + drink
News
Rob Merrick's Lobby Journalists were playing Ed Balls' Labour Party MPs. The match is an annual event which takes place ahead of the opening of the party conference
newsRob Merrick insistes 'Ed will be hurting much more than me'
News
A cabin crew member photographed the devastation after one flight
news
Voices
A new app has been launched that enables people to have a cuddle from a stranger
voicesMaybe the new app will make it more normal to reach out to strangers
Latest stories from i100
Have you tried new the Independent Digital Edition apps?
Independent Dating
and  

By clicking 'Search' you
are agreeing to our
Terms of Use.

ES Rentals

    iJobs Job Widget
    iJobs Gadgets & Tech

    Trainee / Experienced Recruitment Consultants

    £20000 - £25000 per annum + OTE £40,000: SThree: SThree are a global FTSE 250 ...

    Trainee Recruitment Consultant - Soho

    £20000 - £25000 per annum + OTE £40000: SThree: As a Recruitment Consultant, y...

    Trainee Recruitment Consultants - Banking & Finance

    £20000 - £25000 per annum + OTE £40,000: SThree: SThree Group have been well e...

    Website Editor

    £15 - £17 Per Hour: Clearwater People Solutions Ltd: Our client is currently r...

    Day In a Page

    Secret politics of the weekly shop

    The politics of the weekly shop

    New app reveals political leanings of food companies
    Beam me up, Scottie!

    Beam me up, Scottie!

    Celebrity Trekkies from Alex Salmond to Barack Obama
    Beware Wet Paint: The ICA's latest ambitious exhibition

    Beware Wet Paint

    The ICA's latest ambitious exhibition
    Pink Floyd have produced some of rock's greatest ever album covers

    Pink Floyd have produced some of rock's greatest ever album covers

    Can 'The Endless River' carry on the tradition?
    Sanctuary for the suicidal

    Sanctuary for the suicidal

    One mother's story of how London charity Maytree helped her son with his depression
    A roller-coaster tale from the 'voice of a generation'

    Not That Kind of Girl:

    A roller-coaster tale from 'voice of a generation' Lena Dunham
    London is not bedlam or a cradle of vice. In fact it, as much as anywhere, deserves independence

    London is not bedlam or a cradle of vice

    In fact it, as much as anywhere, deserves independence
    Vivienne Westwood 'didn’t want' relationship with Malcolm McLaren

    Vivienne Westwood 'didn’t want' relationship with McLaren

    Designer 'felt pressured' into going out with Sex Pistols manager
    Jourdan Dunn: Model mother

    Model mother

    Jordan Dunn became one of the best-paid models in the world
    Apple still coolest brand – despite U2 PR disaster

    Apple still the coolest brand

    Despite PR disaster of free U2 album
    Scottish referendum: The Yes vote was the love that dared speak its name, but it was not to be

    Despite the result, this is the end of the status quo

    Boyd Tonkin on the fall-out from the Scottish referendum
    Manolo Blahnik: The high priest of heels talks flats, Englishness, and why he loves Mary Beard

    Manolo Blahnik: Flats, Englishness, and Mary Beard

    The shoe designer who has been dubbed 'the patron saint of the stiletto'
    The Beatles biographer reveals exclusive original manuscripts of some of the best pop songs ever written

    Scrambled eggs and LSD

    Behind The Beatles' lyrics - thanks to Hunter Davis's original manuscript copies
    'Normcore' fashion: Blending in is the new standing out in latest catwalk non-trend

    'Normcore': Blending in is the new standing out

    Just when fashion was in grave danger of running out of trends, it only went and invented the non-trend. Rebecca Gonsalves investigates
    Dance’s new leading ladies fight back: How female vocalists are now writing their own hits

    New leading ladies of dance fight back

    How female vocalists are now writing their own hits