'ICED coffee,' says the man on the bench by way of salutation as I park my bike on Main Street. He has red socks with his sneakers, a yachting cap and a pinky ring with a diamond and he is there most mornings at seven, drinking coffee on a bench outside the Paradise Soda Grill in Sag Harbor.

A couple of weeks into my summer in this village of 2,500 at the end of Long Island, and I figure I've got the hang of small-town life. I have mastered a sprinkler system more complicated than brain surgery for the lawn Assiduously, I recycle the garbage. And every morning, I eat breakfast at the Paradise Grill.

My first morning, the guy with the yachting hat was at the counter. I ordered iced coffee. He grinned. 'Hot things are hot, cold things cold,' he said, Zen master of the sensible, and ever since, when I disembark on Main Street, he is always there, nattily dressed and genial. 'Iced Coffee,' he says, as if it were my name; I feel like a visiting rap artist.

Early mornings, Main Street is clean as a whistle and almost empty, except for maybe a dozen people drinking coffee on benches or at the Paradise or eating Danish pastry at the Harbor Deli, where crab-cake hero sandwiches are also a specialty. The busy trade of the night before has gone. The moviegoers, the tourists shopping for things in the shape of whales, the kids dripping ice-cream in flavours like Wavy Gravy and Cherry Garcia, the literary wannabes posturing at the bar of the American Hotel . . . all seem to have simply drained away, as if someone pulled a plug. Only the scatter of breakfast eaters populate the street, the pristine American landscape empty, but sunny, as if Edward Hopper had suddenly cheered up before painting it.

By seven, when the Paradise Grill opens, the regulars are perched on brown leatherette stools at the counter up front. There are also booths, each with a juke-box on the wall beside it, and a lot of large green plants hanging from plastic baskets. Like a million all-American diners, the menu shows a Greek influence - avgolemono is often featured alongside two types of clam chowder.

Almost all the coffee drinkers are men. The older guys - the locals - wear socks with their shoes, by and large. The others - summer people, maybe, or the 'all-year summer people', or the writers who think Sag Harbor is their secret - do not. The sans socks peruse their New York Times and eat cantaloupe melon.

'Morning.' 'Morning to you.'

'You sit right there where your grandfather sat when he was a bridegroom,' says one regular to a younger man who joins him.

The regulars discuss boats and bikes and maybe the health of Herb's wife. Sometimes there is talk of the weather or the Second World War, of Lancasters and Spitfires.

'So we say good morning and you say good morning, but we don't know who you are,' says one of the men, after I show up a few mornings running. He wears work clothes and yellow boots and has the last third of a cigar in his mouth.

'Oh, I'm just visiting,' I say.

'Just being friendly, eh? Being friendly.' He has a sardonic expression and looks like Robert Mitchum.

His usual companion I have come to think of as the Philosopher. He has a foreign accent - Middle European? South African? - and, with his half glasses on a cord around his neck, he holds forth on a number of subjects: the French civil service, gout ('an inflammation of the joints'), skate ('a fish bigger than that newspaper'). Clearly this is an educated man, a courtly pedant. I think his name is Lou - but I feel it would be cheating to ask.

Anyhow, it could take weeks, years, decades, maybe, to tune into the nuance of caste and class in Sag Harbor, where nothing much ever happens and the tensions of a dozen tiny social strata can be invested in who wears socks and where they eat their breakfast.

Only a few miles south, for instance, towards the ocean, across rich farmland, is the Candy Kitchen in Bridgehampton. Here local farmer meets summer celebrity at breakfast, and indeed, if you go in early enough, you may spot a famous anchorperson sucking up to a famous television executive, or maybe Martha Stewart, America's doyenne of country living.

Come 9am though, come the 60-year-old women with 30-year-old bodies encased in spandex, sans socks in Gucci loafers; the exquisite little children, the accessory of choice. Tan, blonde, they are made of the flesh and bone that good American money can pull from the gene pool in a single generation. At the Candy Kitchen, Hollywood agents whose pinky rings are as big as meatballs talk about killings financial, while weekend studs eating pancakes talk about killings they made in bed.

'So did you sleep with Gail last night? Did you screw her? Is she bugging you?' I hear one of them say.

I retreat to Sag Harbor, where even on Sunday mornings, the Paradise Soda Grill regulars conduct their civilised conversations exactly at 7am and the only thing anyone talks about killing are the deer.