Summer in the Hamptons. The pristine acres of white beach and shingled oceanfront mansions shimmer in the Atlantic sun. P Diddy unloads his hamper from the back of his summer Hummer, while media billionaires eat lobster roll and compare suntans.

Summer in Littlehampton. The English Channel scuds under a lowering grey sky. A few dog walkers brave the wind on the promenade, and the odd family straggles into the flyblown funfair, or thrashes around the miniature golf course. Fruit machines blink from the dark interiors of beachfront cafés, while teenage girls eat chips and compare tattoos.

There's little danger of the south coast resort of Littlehampton becoming a summer haven for the rich and famous. Unless they happen to be sent down there as part of a Channel Four taskforce for urban regeneration. Despite its fine stretch of sandy beach, Georgian terraces and proximity to prosperous Arundel, Littlehampton is a traditional English seaside town that has fallen on hard times.

The town centre feels frozen in the Seventies, complete with Wimpy Bar. The beach area aspires to the Seventies, exuding an out-of-season desolation even in late June. But keep walking, past the funfair and empty car parks, and as the kiss-me-quick tat thins out, an extraordinary, Dali-esque building rises up. Approached from the side, it looks unlike anything you've ever seen; an organic, steel structure, striated like an oyster shell, with the rusty patina of sea-weathered iron.

From the front, the effect is more conventional, but the purpose no less shocking in this setting. This is East Beach Café, the deliriously optimistic collaboration between a local family of neophyte restaurateurs and the quicksilver design genius Thomas Heatherwick. Jane Wood, a Littlehampton resident who was horrified to see that the beachfront café had applied to rebuild on a huge and ugly scale, bought the site herself in order to protect it. Heatherwick signed up willingly – his iconic Rolling Bridge in Paddington Basin was built by a Littlehampton foundry, and he relished the idea of designing his first building.

Long before it threw open its state-of-the-art shutters, the new building was having a mini Bilbao effect on the town. Press coverage has been fervent, with Vogue going so far as to dub Littlehampton the UK's coolest seaside resort. (Probably safe to assume the author didn't actually pay a visit.) And the food side has been approached as seriously as the architecture, with a former Ritz chef in charge of the kitchen, and menu advice from Rick Stein's team.

Visiting on a midweek lunchtime a week after the opening, we found a queue snaking out of the door; weird, because the beach was pretty much deserted. The interior is small, seating just 45, with double that number accommodated on the decked area outside. However, this being June in England, that obviously wasn't an option.

The interior is bracingly clean-lined, with not a trace of Cath Kidston retro cutesiness about it. Who needs sail-sprigged oilcloth when one side of the room offers floor-to-ceiling views of the sea and beach? Otherwise, the room is windowless, its white walls undulating gently like the inside of a conch shell.

The lunch menu is obviously intended to keep the traditional seaside crowd happy, with fish and chips and bacon sandwiches appearing alongside more adventurous dishes. The emphasis is on fish and seafood, with highlights including potted shrimp, grilled sardines with beetroot and an English version of spaghetti alla vongole which uses cockles rather than clams. Specials included half a grilled lobster, which my companion, Marina from Brighton, almost snapped the waitress's hand off to bagsie.

Brighton Marina's moules marinières (and try saying that with a mouthful of granary bread soaked in gorgeously creamy, oniony, parsley-flecked broth) were a burstingly fresh rebuke to anyone who has read Kitchen Confidential and vowed never again to order mussels in a restaurant. My salt and pepper squid delivered an authentically spicy chilli-garlic kick, while the presence of flat-leaf parsley and whole radishes demonstrated that chef David Whiteside is no hide-bound traditionalist.

Fish and chips set a high standard for that devalued staple, the chips served in a zinc bucket, the beer-battered fish – a surprisingly fine piece of coley – supporting the menu's claim to use only fish from sustainable sources. Herb-sprinkled grilled lobster (£20) was well prepared, though the use of aromatic herbs like basil and coriander slightly fought against its subtle flavour.

With puddings – apricot trifle, and a chunky salad of watermelon, mango and pineapple in elderflower syrup – and a couple of glasses from a small winelist, our bill came to £78. Service is left to the customer's discretion.

As the lunchtime rush subsided around us, the skies darkened, and soon our numbers were swelled by rain-lashed holiday-makers in search of tea and cake. Heatherwick has described his building as "a place of prospect and refuge", which makes it sound like a church, and certainly it's destined to become a place of pilgrimage; the only potential contender for the Stirling Prize to boast a Mr Whippy machine. The target audience, though, isn't just Grand Designs disciples on an architectural Grand Tour; apart from a couple of young families, our fellow diners were mostly as white-capped as the choppy English Channel.

Jane Wood and her daughter Sophie, who manages the restaurant with a shy charm, have done something miraculous in raising this incredible Heatherwick building in such unlikely surroundings. And it's even more inspiring to find their commitment and passion running through the restaurant operation, right down to the smallest details. Unlike the New York Hamptons, it may not be smart. But it is very, very clever.

Food......... threestar

Architecture......... fivestar

Service......... fourstar

Around £40 each for three courses with wine

East Beach Café, Littlehampton, West Sussex (01903 731 903)