Now, though, the fame and fortune of the third round and perhaps beyond beckons once again. The Diadora League club from suburban Surrey went to Colchester United in the first round, and won 4-3. Tomorrow Sutton travel to another Third Division side, Torquay United, in search of a place in the third-round draw later in the evening.
Sutton have a score to settle with the Devon club. In the season after their defeat of Coventry, their first- round opponents were Torquay, who beat them 4-0 in a Plainmoor replay after a 1-1 draw in Surrey. Since that tie, though, Torquay have been beaten by two of the four non- League sides they have faced in the FA Cup: Farnborough two seasons ago followed by a 5-2 home thrashing by Yeovil last term.
When Coventry were being humiliated in front of the Match of the Day cameras at Gander Green Lane, the Sutton manager was the pipe- smoking, poetry-quoting Barrie Williams. Now the team are in the more orthodox hands of two former players: the manager, Alan Gane, and his coach, Larry Pritchard.
Gane made a mere 32 appearances for Sutton in the 1968-69 season, when he was a non-playing substitute at Wembley when they lost the FA Amateur Cup final to North Shields. Pritchard, however, is one of Sutton's most notable former players, with a record 786 appearances and 48 England amateur international caps to his name.
Gane, a 43-year-old computer salesman in his third season as manager at Sutton, was previously in charge at Wealdstone, Staines and Wycombe Wanderers, where he contributed to the club's success story by guiding them to promotion to the Conference in 1987.
'Wycombe and Sutton are very similar clubs,' Gane said. 'I want Sutton to do what they've done. I've been a player and a manager at both clubs, and I always felt that they were the two best non-League teams in the south.
'Sutton should be in the Conference. On our day, we've been the best side in this league for the past two years, but consistency has been lacking, particularly last season. I doubt we could sustain a Football League club here, though, we're too close to Wimbledon and Crystal Palace.'
Torquay fans expecting to see their team face a Wimbledon-style long-ball side will be in for a shock. Despite relegation from the Conference in 1991, Sutton have maintained a tradition of playing purists' football, with the ball on the ground and moves developed from the back. 'It suits the players we have,' Gane said. 'It's the only style I can sit and watch every week, to be honest.'
'We've always been a passing side,' Pritchard added. When it was put to the pair of them that Sutton might be described as the Nottingham Forest of the non-League world - a pretty side that has found it hard to win anything recently - Pritchard said: 'Many years ago that was one of the reasons I left, because that was a criticism I had of the club. Now, though, we are more competitive and more likely to win things.'
A key performer in their forward line is the 21-year-old Olisah Morah. Released by Tottenham Hotspur last term, he arrived at Sutton in the first week of the season, and will have to be watched closely by Torquay.
'During the summer we suffered a couple of injuries to our front players,' Gane explained, 'and the Spurs youth coach, Keith Walden (a former Sutton player), put us on to Morah. Potentially, with his pace, he could go all the way. I think he'll be a late developer, he's still very raw.'
While Morah is new to the club, another striker, Paul McKinnon, first joined Sutton in 1977 and is now in his eighth spell at Gander Green Lane. United's record goalscorer, with 279 goals in less than 550 games, he spent several summers in Sweden and also played League football for Blackburn Rovers. He once played for Malmo against Benfica in the Stadium of Light in a Cup-Winners' Cup tie.
McKinnon has been a regular substitute so far this term. 'He missed the pre-season period with injury, but he's almost fit now,' Gane said. 'He combines an exceptional level of skill with an excellent attitude.'
Along with the defender, Nigel Golley - 'A mainstay, an exceptional player, very underrated,' according to Gane - McKinnon is the only survivor of the team that beat Coventry.
That match, though, is history. Gane and his team are concentrating on Torquay: 'They're on to a loser,' he said. 'If they win, they're expected to, if they lose they get slaughtered. The second round is so frantic because of the potential rewards for winning. We would have preferred an easier draw, but we're not frightened of them.'