The Sky Blues, with club captain and top scorer Dion Dublin left out of their line-up as he considers a move to Blackburn, crashed out in the third round 2-0 thanks to goals from Phil Gray and Steve Davis.
Afterwards, the Coventry manager, Gordon Strachan, tore into his team. "It's OK getting beaten but we were embarrassed," he said. "People were laughing at us. Luton were bigger in terms of passing the ball, heart, the will to win, and in terms of energy. The only way we were bigger was in terms of egos."
The Northern Ireland international Gray, enjoying his second spell at Kenilworth Road, brought back memories of the Hatters' finest hour when they beat Arsenal at Wembley 10 years ago to claim this trophy.
His 50th-minute strike rocked Coventry before Steve Davis snookered the Premiership club with the second 12 minutes from time.
Meanwhile, Coventry's fellow top-flight strugglers Nottingham Forest had their veteran goalkeeper Dave Beasant to thank after they had thrown away a three-goal lead and been forced into a penalty shoot-out with Third Division Cambridge United.
Forest, who also had Thierry Bonalair sent off for handball in the 80th minute, led 3-0 with 22 minutes to go but the underdogs, conquerors of Watford and Sheffield Wednesday in earlier rounds, pulled the game back to 3-3 to earn extra time.
Beasant then brought his side back from the brink with saves from Alex Russell, Paul Wanless and Trevor Benjamin, whose sudden-death miss sent Forest into the next round.
It was the cruellest of Cup exits for Cambridge, who had bounced back after Forest had gone streaking into the lead with goals from Dougie Freedman, Marlon Harewood and Craig Armstrong.
But the visitors, roared on by a 3,500-strong travelling army, clawed their way back with three goals in 12 minutes through Benjamin, Butler and a John Taylor penalty.
Bolton also needed penalties to beat their First Division rivals Norwich at Carrow Road. It was goalless after 90 minutes, but Robbie Elliott tipped the scales Wanderers' way with the opening goal six minutes later. Transfer- listed striker Keith O'Neill levelled the scores for City's manager, Bruce Rioch, against his old club, but Bolton took the shoot-out 3-1.
Niall Quinn sprung from the substitutes' bench to save Sunderland from a similar nerve-racking finish as they overcame Grimsby 2-1. Quinn's goal came five minutes from the end of extra time. Lee Nogan had shocked the home side by giving Grimsby the lead after 11 minutes but Michael Bridges equalised midway through the second half.