Extra time had seen the game remain level after the second round replay had finished 2-2 in normal time. Lloyd Owusu put the London club ahead early on, but the former Scottish international John McGinlay equalised in the 20th minute with a deftly taken lob.
Despite a red card for Jamie Bates, for bringing down McGinlay, Darren Freeman restored Brentford's lead, but a diving header from Lee Duxbury led to extra time and then the shoot-out.
Plymouth Argyle can look forward to a home tie with Derby County after seeing off Wycombe Wanderers 3-2 at Home Park. The Pilgrims, who beat the Rams on the way to the Cup semi-finals in 1984, went 2-0 up through Paul Wotton, from the penalty spot, and Mick Heathcote.
Wycombe's Paul Read pulled one back midway through the second half, but a first senior goal from Terry Sweeney made the game safe for Argyle before Dave Carroll hit a second for Wanderers from the spot in the last minute.
Rotherham United, who beat Rochdale 4-0, and Bristol Rovers, who thrashed Exeter City 5-0, will meet in the third round at Millmoor next month.
Graham Allner resigned as the manager of Kidderminster Harriers last night. In charge for more than 15 years, he was the longest serving manager in the Football Conference.