Online sales put in their worst monthly performance in nearly five years last month as consumers watched the Olympics on television rather than shop on the internet.
Non-store retailing sales volumes, which also includes mail order, fell by 6.7% between July and August, the biggest drop since December 2007, the Office for National Statistics (ONS) said.
But the London 2012 games had a negligible overall effect, the ONS added, as total retail sales volumes dipped 0.2% in August, following a 0.3% rise in July.
The figures echo survey evidence from the British Retail Consortium and CBI earlier this month, which also pointed to a minimal impact from the Olympics.
Samuel Tombs, UK economist at Capital Economics, said the figures "add to the evidence that the Olympics did not provide the hoped-for boost to spending on the high street".
But Mr Tombs said that even after the drop in August, retail sales volumes are still 0.6% higher than three months ago and are likely to contribute to a brief return of GDP growth in the third quarter.
He added: "With consumer confidence still very weak and inflation set to outpace earnings growth for another six months or so, we would not be surprised to see further falls in retail sales in the coming months."
Suit specialist Moss Bros is among retailers that have blamed Britain's summer of major events for slow business.
The retailer said it had missed out on nearly £2 million of hire business due to events such as the Queen's diamond jubilee and the Olympic Games as more weddings were deferred.
There was some positive impact from the London 2012 games at sporting goods stores, which helped sales volumes in the "other stores" category jump 1.5% month on month.
But this was offset by a 0.7% fall in department stores and a 2.7% fell in household good stores between July and August.
Food stores saw a slight 0.2% rise in sales volumes in the period, which some retailers put down to improved weather and the Olympics.
Meanwhile, the proportion of internet sales also decreased between July and August by 0.9%, although the proportion was still up on a year ago. The average weekly spend in August online was £466.1 million.
Hopes that the squeeze on consumer spending will ease have faded as economists warn that inflation will fall more slowly than previously expected.
The rate of inflation edged down to 2.5% in August from 2.6% in July, while average earnings rose 1.4% in the year to July.
Rising petrol prices, forthcoming utility bill hikes and higher tuition fees will pile pressure on the cost of living in the months ahead.