The traditional elderly crossing sign perpetuates a "damaging stereotype" that deters employers from recruiting jobseekers over the age of 50, the government tsar for the elderly claimed.
Pensions expert and campaigner Dr Ros Altmann called for the sign to be banned from use after being displayed on British roads for 33 years as she said it assists in continuing a false perception of older people.
She hopes to bring the issue to the women and equalities minister Jo Swinson at a meeting next month to discuss the impact of the signs, which are meant to warn drivers of less-mobile pedestrians, and to potentially get them taken down.
Dr Altmann said: "I think we do not need a sign to warn people of older people. It is redundant.
"I don't think they are serving any useful purpose and they are damaging."
She added that, while she agrees that signs should warn of children or animals potentially running into the road, drivers do not need a sign stating that elderly people may be walking nearby.
Dr Altmann will also unveil proposals to encourage internships, or "returnships", for those looking for a career change at the age of 50 and over at a meeting on Tuesday with firms including Centrica, Legal & General and Barclays.
"There is almost a subliminal message that older people are in some way physically or mentally sub-par. However, as there are more and more seeking work and I do not want to see their skills and talents go to waste. I want them to be taken seriously," she added.
The road sign image of two silhouettes walking with stooped backs and a stick was a winning entry in a children's competition held in 1981 and the sign used to be accompanied by another saying 'elderly people' below it but the latter have been removed since 2003.
Campaigns by Age Concern and Help the Aged charities are reported to have failed to get the sign design changed in 2008 after they argued that it does not accurately depict the varying physical capabilities of older people.
However, Cumbria Tourism enlisted the help of a local artist who replaced 10 signs with a silhouette of a couple walking up an incline with straighter backs and hiking poles in 2009.
Artist Steve Messam said to the BBC: "Life doesn't stop at a certain age and there are a lot of active visitors regularly climbing and walking here that are a world away from the image of the elderly people depicted on the existing road sign."