How we used to eat: A band prepares to entertain the diners at Hector's restaurant in Devonshire in 1930 / Getty Images

Gone are the days of violinists serenading diners. Now, it's all about carefully selected playlists to accompany your lunchtime burger or dinner with friends, says Tracey MacLeod.

When Britain's most famous chef, Gordon Ramsay, appeared on Desert Island Discs, his musical selection was remarkable. An eight-song horror show of the banal and bombastic, it included Tina Turner's "Simply the Best", "Everything I Do, I Do For You" by Bryan Adams and Tom Jones's "Sex Bomb". If you were genuinely stranded on a desert island with this selection on repeat, you'd be demonstrating your knife skills on your own throat within days.

But then it's hardly fair to expect chefs to have sophisticated musical taste. Chefs are like laboratory mice, raised in captivity, working long hours in airless rooms, with only the sound of Absolute or Magic on a tinny radio for company. Their senses are finely tuned when it comes to taste and smell, and many will have a heightened visual sense, too. Small wonder that their hearing, abused by years of shouting and crashing pans, may be less than sensitive.

Recently, though, a new generation of musically-minded chefs and restaurateurs has emerged, who give as much thought to their playlists as they do to their menus. Some of them are ex-DJs. Others are just music fans who see the sharing of their enormous record collections with their guests as a natural extension of feeding them.

Nicholas Balfe, chef/proprietor of Brixton's Salon, is one of the new breed. He used to DJ while working freelance as a chef, and once cooked and DJed at Glastonbury in the same weekend. So when he opened his first restaurant, above a deli in Brixton market, music was always going to be important. "I wanted to create an atmosphere beyond the food," he explains. "It's people's Saturday night, after all. They want to relax and have good tunes on. It's a leisure experience."

A typical Salon playlist, compiled by Balfe, might start with "a 14-minute cosmic remix of Four Tet, so everyone starts their shift in a bit of a trance. Then a bit of space rock. Maybe some Ebo Taylor or African music during the daytime. And I'll throw in a few more well-known pop songs. I had to ban Nina Simone when we started. All that plinky-plonky, twee stuff – it's just so horribly clichéd."

Salon's clientele are as likely to be seen holding up their phone to tag a song on Shazam as using it to Instagram a photo of their dinner. But even restaurants who don't have an in-house chef de party are now raising their musical game. It used to be enough for the manager just to bung on a tasteful CD. The Buena Vista Social Club, Air, The Gotan Project, maybe some dinner-party trip-hop. And of course, Zero 7. As a restaurant reviewer, I've eaten more meals to the sound of Zero 7 than I've had hot dinners.

That kind of stick-it-on-as-an-afterthought approach to background music just feels lazy now that we all carry our entire record collections around in our pocket, and are used to controlling our musical environments. Music has to do more than just create a pleasant, neutral wash. It's a statement of intent. "It's part of the ever-increasing competition in London restaurants," according to Daniel Willis, DJ turned co-founder of Shoreditch hot-spot The Clove Club. "You can't just open. You've got to think about why people would come to you, not to the place down the road."

It isn't only cool East End venues that are working on their musical offer, as I realised recently in a very smart, all-white hotel dining room with Michelin aspirations. On the sound system, playing fairly loud, was what sounded like someone's mix-tape. Gussied-up waiters poured sauces over immaculate towers of food while doomy alt-folk ballads gave way to some bleating angst from I Am Kloot. All great tunes, but not to play in an uptight Belgravia hotel dining room. Someone had obviously thought hard about the music, and come up with completely the wrong soundtrack.

That kind of mismatch is unusual, thanks to the rise of professional music consultants who offer streaming services and tailored playlists to restaurants and hotels. Using digital profiling techniques similar to those pioneered by Muzak, the daddy of the industry, specialist companies here in the UK are springing up to supply dining rooms with the appropriate mood music.

"We help owners get away from the problems caused by the staff choosing the music themselves," says Roy Court of Leeds-based Muzo. "What the staff want is not necessarily want the clientele want." A relatively new player, Muzo offers a constantly updated series of themed playlists to international hotel groups and restaurants. There are some 20 styles, available for restaurants to stream; they include 'vintage chic' (classic rock'n'roll and soul); 'nu-vibes' ("acid and nu-jazz flavours for a lazy Sunday sound") and 'laidback lounging' ("tropically tranquil tunes", including, inevitably, Nina Simone.).

Every song is analysed by a team of profilers, and playlists are adjusted according to the time of day. "If you go to a restaurant for afternoon tea, you don't want to listen to the same music as at lunchtime," says Roy Court. "Busy restaurants don't want to play slow music, while on slow days, they may play slow music because they want people to stay longer." Another big consultancy, C-Burn, has a full time staff of 16, most of them employed to listen to new music. They supply 10 million tracks a month to venues, with clients ranging from Nobu and the Soho House group to Prezzo.

"Everyone thinks they can choose good music, just like everyone thinks they can write a novel," says C-Burn's Adam Smith. "But you have to keep doing it day in, day out. You can't have bar staff who've been raving all weekend coming down on Monday by playing banging techno to ladies who lunch. We try and objectively tie in the music with the kind of customers the restaurant wants to attract."

That kind of analytical profiling is anathema to Russell Norman, the man who brought a downtown New York vibe to London's West End with restaurants like Polpo, Spuntino and Mishkin's. Norman chooses or approves all the music played in his restaurants.

"For me music isn't an afterthought – it's key," he says. "I think long and hard about it. Each of my restaurants has its own playlist. In Mishkin's, it's funk and soul. In Polpo it's eclectic. When Spuntino started it was all scratchy blues recordings from the Forties and Fifties, though lately the manager asked me if they could update it with some stuff from the Sixties and Seventies, like Dylan and Led Zeppelin." The result, together with the group's no-bookings policy, low lighting and hard seating, is almost tailor-made to send older, stuffier diners stampeding for the exits.

"If people complain that the music is too loud, we always turn it down," says Norman. "In fact I find the music in my restaurants too loud sometimes. But then I didn't make them for people my age – I'm 47. They're for people in their twenties and thirties, who want a party atmosphere."

Spuntino's head chef, Rachel O'Sullivan, another big music fan, is trusted by Norman to create playlists for the restaurant. "Sharing music with other people is a beautiful thing," she says. "It's the same impulse as cooking for people. It's a sensory thing – it triggers memories."

Rachel belongs to an informal network of music-minded chefs who create playlists for each other's restaurants. Shoreditch's Clove Club is a hub of the action. It grew out of a series of music-led supper clubs that Daniel Willis and his fellow DJ Johnny Smith held in their East End flat. "We used to be aspiring DJs putting on a supper club," laughs Willis. "Now we're aspiring restaurateurs."

Willis says he has "three to four thousand" records at home, and that a venue aimed at the playlist generation was always going to have music at its heart. "We wanted to capture that New York feel; cooler, buzzier, darker places where music is more of a factor."

Friends of the Clove Club collective, including Rachel O'Sullivan, create playlists for the bar. But unusually, there's no music in the dining room. "We struggled with it and in the end took the braver decision not to have any," explains Willis. "It worked when we were running supper clubs in our flat, but in a restaurant, it's about the table, about conversation. Plus, we're fed up with everyone going on about us being full of 'hipsters'. If we'd played music in the restaurant, it would just have been another stick to beat us with."

When restaurants are busy and bustling, of course, they don't need music. It's the quiet ones that cry out for it. If the couple two tables down are having a hissed argument, or the chef is having a minor nervous breakdown in the kitchen, you need something to mask the sound, even if it is Zero 7.

It's also crucial that the staff don't get bored. The right music helps keep service bustling along; having the same songs pop up on rotation is likely to cause a mutiny. "Sometimes you'll get a track with a really jarring loud guitar solo that just jumps out at you," says Rachel O'Sullivan. "That soon gets deleted from the playlist." Bar-owner and restaurateur Jonathan Downey, whose Rotary Bar and Diner recently opened in Clerkenwell, goes further. "We have a black list button," he laughs. "You can often see me storming through the restaurant shouting 'Black list!'." Most likely to cause an explosion? "Anything by Morcheeba or Nouvelle Vague."

Whoever is compiling the playlist, the key thing is to avoid causing a walk-out.

Muzo's Roy Court recalls being in a hotel when the original version of Cee Lo Green's "Forget You" came on, featuring the earthier F-word. "I could see parents looking at each other in horror. So I went over to the front of house manager to speak to him, and it turned out he didn't speak much English." It's the kind of cautionary tale Muzo's sales reps must surely use in their pitch; needless to say, all of the company's selections are carefully screened for profanity.

That wouldn't work in Beagle, another new Shoreditch opening from another set of moonlighting DJs. Brothers Danny and Kieran Clancy, aka party organisers Krankbrother, installed a custom-built audiophile system when they converted three railway arches into a bar and restaurant. As well as their own records, they asked friends, including Andrew Weatherall and house DJ Heidi, to compile playlists. Profanity is not an issue: when asked for his secret weapon track, Danny Clancy offers The Cramps' "Let's Get Fucked Up". "Scuzzy rockabilly meets garage punk. You hear the Cramps at volume, it's hard not to have some sort of reaction."

Indeed. Still, when you own your own restaurant, it's your prerogative to play whatever you like. But the next phase of the musical revolution is on its way – diner-controlled music. Consultancy C-Burn recently launched an app, Secret DJ, which allows customers to choose songs themselves, using their mobile phones. It sounds high-risk, exposing a whole dining room to a fellow diner's selection of nosebleed techno. But the system works more like a digital jukebox, with each venue supplied with a pre-selected list of suitable songs.

"The idea behind it is that people are more musically aware than ever and have greater expectation of music, and more space to find and explore it," explains C-Burn's Adam Smith. "People also expect to control their environment more. It's great for first dates, and for impressing your mates."

None of the music-mad restaurant folk I spoke to mentioned live music – the sound of the pianist noodling away in the corner seems to be a thing of the past. But several confessed a guilty secret; that they really like to eat in a music-free room, or one in which the music is faded down as the hubbub of conversation takes over. As Russell Norman says, "That's the best music of all. The sound of people talking and laughing and having a good time."

Sounds familiar? Playlists by C-Burn

The hipster burger joint

Whiskey Do Your Stuff by Louis Jordan
Blood on the Bluegrass by Legendary Shack Shakers
Born on the Bayou by Creedence Clearwater Revival
Rip This Joint by The Rolling Stones
Mama Don't by JJ Cale
Good Rockin' Daddy by Etta James
Stealin' All Day by CC Adcock
Misty Mountain Hop by Led Zeppelin
Break 'em on Down by Soledad Brothers
Keep on Movin' by King Tuff

Notes: Referencing classic Americana and rock for a retro feel. The antithesis of what would be played in McDonald's/Burger King, to stress that all beef patties are not the same.

The family-friendly pizza place

Human Spirit by Amy MacDonald
10/10 by Paolo Nutini
Falling by Haim
Where Did You Go? by Shea
Pack Up by Eliza Doolittle
Mirrors by Justin Timberlake
Best Thing I Never Had by Beyoncé
Rolling in the Deep by Adele
L.I.F.E.G.O.E.S.O.N by Noah and the Whale
Make a Little Room by Al Lewis

Notes: Music for mums, dads and 8-year-olds, quality chart, no cheese. School-run singalong.

The city-centre brasserie

Green Garden by Laura Mvula
The Fall by Rhye
Tell the Truth by Lady
The Boxer by Mumford & Sons feat Paul Simon
Son of a Preacher Man by Dusty Springfield
Look Around the Corner by Alice Russell feat Quantic
Always Waiting by Michael Kiwanuka
Linger by The Cranberries
Tangled Up by Caro Emerald
I Need a Dollar by Aloe Blacc

Notes: Warm contemporary and classics suited to a catch-up with friends over a bottle of wine or two.

The paragon of fine dining

Knee Deep in the North Sea by Portico Quartet
Why Did We Fire the Gun? by Waldeck
All in My Head by The Beauty Room
Icarus by Winter Hinterland
Life by Ludovico Einaudi
Avril 14th by Aphex Twin
Manifesto by Gonzales
Rebirth of Cool by DJ Cam Quartet
Perverted Undertone by Prefuse 73
Towers by Bonobo

Notes: Sophisticated tunes reflecting the status or aspiration of the clientele. Played just loudly enough to give the room some colour without being intrusive or detracting from critically-acclaimed cuisine.