Why 'random' shuffle feels far from random

If your shuffle setting really chose music completely randomly, it would actually feel like there were more coincidences

You’re listening to music on shuffle, and an artist appears twice in a row. It feels like more than a coincidence – perhaps your music player is playing tricks on you, or has chosen favoured artists – but is it?

In fact, music players tend not to shuffle randomly. But that’s because if it did, it would actually feel less random.

Human brains will start seeing patterns in even the smallest coincidences – and in a long shuffled playlist, it’s likely that certain songs or artists will come next to one another.

“The problem is that to humans, truly random does not feel random,” Mattias Petter Johansson, a Spotify developer, wrote in an internet post earlier this month. “So we got tons of complaints from users about it not being random.

“Last year, we updated it with a new algorithm, that is intended to feel more random to a human.”

The feeling that playlists are being played with comes partly from the Gambler’s Fallacy – a trick of the brain that leads us to think that if something hasn’t happened for a long time, it is more likely to happen again. If a coin lands on heads three times, for instance, it instinctively feels like it should – but there is an equal chance of it landing on heads again.

That means that if a playlist repeats artists, songs or genres then it feels wrong. If shuffle is truly random, the Gambler’s Fallacy leads us to think, then the songs should be more spaced out. But there is an equal chance that songs will come close together as there is that they will be far apart.

Those patterns led many users to complain, Babar Zafar, a lead developer at Spotify, told the BBC last week. Some angry users even proposed conspiracy theories, such as the idea that record labels had struck deals with Spotify so that their artists would be played more than others. But the company has been working hard to make Spotify feel more random.

When Spotify launched, the company explained in a blog post, it made its random playlists using the “Fisher-Yates shuffle”. That is an efficient algorithm that takes only three lines of code to make “the optimal amount of operations and optimal amount of randomness”, creating random playlists without using much computing power.

But users complained that the playlist wasn’t genuinely random. Artists or genres tended to appear next to each other, giving the sense of the playlist being unfair – even though it is likely that different artists will bunch up as that they will be evenly distributed throughout.

Instead, Spotify designed a new algorithm that distributes artists and genres more evenly. If there are four songs by the White Stripes in a playlist, for instance, the algorithm will aim to play them at roughly 25% intervals.

Despite that new algorithm, Johansson said that users still tell Spotify developers that the shuffle functionality is not random. And it isn’t – but it’s calculated to feel more random, not less.