Brett Ormerod had given Southampton the lead, but a twice-taken Inigo Idiakez penalty and Andrew Davies' goal looked to have turned the game on its head.
Southampton were livelier at the start, Kamil Kosowski's whipped cross finding Matt Oakley who must have thought he had scored when he connected, but Lee Camp, the Rams goalkeeper, reacted superbly to keep the ball out.
That shocked the hosts into action and Stern John won a free-kick which Idiakez fired goalward, but Darren Powell headed clear.
Derby's manager Phil Brown introduced club captain Jeff Kenna at half-time but something was amiss in the Rams rearguard as Dennis Wise curled in a free-kick which hit the post. Again Brown responded, replacing Khalilou Fadiga and giving a debut to on-loan Tottenham striker Mounir El Hamdaoui.
The move backfired, as Wise again proved his dead-ball proficiency to set up the opener for Southampton. His free-kick found Fuller, who had space to feed a clever backheel to Ormerod, the striker taking his time to steer the ball past Camp.
Brown played his last card by introducing Paul Peschisolido and this time his substitution paid off, Derby winning a penalty as Danny Higginbotham was adjudged to have handled.
There was more drama to come, however, as Idiakez saw his spot-kick saved by Antti Niemi - only for referee Nigel Miller to order a retake. This time there was no mistake as the midfielder hammered the ball home.
Redknapp vented his fury and was promptly sent to the stand. Like their manager the players lost their composure and Derby won a free-kick on the left-edge of the box, Idiakez picking out Davies to head home.
Hajto was then sent off for a second booking and it seemed Southampton's day was to end in misery. But a Niemi clearance saw Camp advance out of his area to head clear and Fuller lobbed the ball straight back from 35 yards into the empty net.
Goals: Ormerod 58 (0-1), Idiakez 80 pen (1-1), Davies 85 (2-1), Fuller 90 (2-2).
Derby County (4-4-2): Camp; Edworthy (Kenna 45), Davies, M Johnson, S Johnson; Bisgaard, Fadiga (El Hamdaoui, 55), Idiakez, Whittingham; Smith, John (Peschisolido, 76). Substitutes Not Used: Poole (gk), Bolder.
Southampton (4-4-2): Niemi; Hajto, Lundekvam, Powell, Higginbotham; Kosowski (McCann, 90), Wise, Oakley, Quashie; Ormerod (Jones, 81), Fuller (Delap, 90). Substitutes Not Used: Smith (gk), Cranie.
Referee: N Miller (Co Durham).
Booked: Derby Idiakez, Davies, S Johnson; Southampton Fuller, Ormerod, Higginbotham, Kosowski, Lundekvam, Hajto, Wise.
Sent off: Southampton Hajto (86).
Man of the match: Wise.Reuse content