The British No 2's 6-1, 6-4 success followed that of Tim Henman, who ended the hopes of Malivai Washington, the No 3 seed, on Monday.
Rusedski, world ranked 76, won four games in a row to take the first set against the Spaniard, a clay-court specialist ranked 17 in the world.
Costa managed to save three match points in the 10th game of the second set before netting two consecutive volleys to give Rusedski victory.
Mark Woodforde joined the other major seeds on the sidelines when he lost 6-2, 7-6 to the American Vince Spadea. The defeat of the fourth seed from Australia leaves Jan Siemerink, the fifth seed from the Netherlands, as the highest-ranked player left. Siemerink managed a 6-3, 6-3 victory over Russia's Andrei Olhovskiy.
At the Wimbledon qualifying tournament in Roehampton, Andrew Richardson overpowered his fellow Briton Simon Pender to move to within one match of booking a place in next week's first round proper.
The tall 22-year-old left- hander from Bourne, Lincolnshire, proved too strong and experienced for his 18-year-old Cornish opponent and finished a 6-2, 6-4 winner.
Richardson, who at 6ft 7in towers above most of his opponents, said his game had improved with the help of Peter Fleming, the former doubles partner of John McEnroe. "He is a big guy like myself so he can understand the way I feel and the way I should play," he said.
Four of the seven British women on view won their first matches against much higher-ranked players.
Jane Wood celebrated her best win since she returned to the women's circuit after a break of nearly 10 years to study in America and to coach. She scored a surprise 6-4, 6-3 victory over the Beckenham finalist Maria Vento of Venezuela, the second seed who is nearly 300 places above her at 116 in the world.
Gloucestershire's Emily Bond, who languishes at 456 in the world, bridged an even wider gulf in the world rankings to put out the No 148, Sonya Jeyaseelan of Canada, 7-5, 6-1.
The other home winners were Essex's Mandy Wainwright who beat the Spaniard Paula Hermida in three sets and Julie Pullin, of Sussex, who routed the Hungarian Viraag Csurgo 6-1, 6-2.Reuse content