Mr Edmonds, general secretary of the General Municipal Boilermakers union, also told the tribunal in Croydon, south London, that Tahir Raffi Janjua "screwed up" his timetable, because he did not know routes properly, making him late for appointments.
Mr Janjua, 38, of Norbury, south London, who was Mr Edmonds's chauffeur from February 1993 to March last year when he was dismissed, is claiming racial discrimination against the union.
Mr Edmonds said: "There were occasions when I was greatly embarrassed when Mr Janjua failed to provide the courteousness I would expect from any employee."
He said Mr Janjua would take a quarter of an hour or more than normal on regular journeys between the GMB headquarters, in Wimbledon, south London, to central London.
Mr Edmonds said he told Mr Janjua many times that he did not have to drive through Clapham to the city centre. "It was screwing up a large part of my days," Mr Edmonds said.
"I was getting pretty fed up. I need time in my car as work time. The last thing I wanted to be was a back-seat driver."
Mr Edmonds added that he found it difficult to read in the back of the car and his wife complained that Mr Janjua's driving made her car sick.
Mr Janjua told him that a number of unions had better and newer cars than the Ford Granada he drove.
"I pointed out to him that the basic Granada could do 100,000 miles and then we would renew it. He said that would take many years and it was not good to keep a car that long."
Mr Edmonds said that Mr Janjua then tried to persuade his personal assistant to try and talk him into getting a new car.
On one occasion, Mr Edmonds said, he had to struggle with baggage in one hand and an umbrella in the pouring rain, but Mr Janjua did not open the car door for him. "It would have been helpful if Mr Janjua had taken the bag off me. I had to put things down before I could get in the car," he said.
Mr Edmonds said Mr Janjua was quiet and courteous until he was told to do something that he did not want to do.
Mr Edmonds said there had been no reference to racial discrimination in any of the disciplinary hearings and appeals which led to Mr Janjua's dismissal and that the union considered the claim to be "frivolous".
The hearing was adjourned until today.