Richard Bacon allegedly said, in rehearsals: "What do you get when you put two brothers and sisters in a band? A big fat melting pot of talent."
The band took such offence to the comments that they decided they were unable to perform their new single "Love Me Like You" from their self-titled album.
The Magic Numbers, two pairs of siblings, "brother and sister twice", mixing melody, harmony and rock, are one of the UK's most popular new pop bands.
A BBC spokesman said Bacon used the word "fat" as a "figure of speech referring to the band's status".
"It was absolutely in no way referring to the appearance of any of the band members," he said. "Richard was mortified about the misunderstanding and any offence caused, so he attempted to apologise personally to the band.
"However, their management denied him access to the band and they decided not to go ahead with their performance on the show."
But, ever-conscious of their fans, among them the former Beach Boys singer Brian Wilson who has described them as the best band of the past 10 years, The Magic Numbers put an apology on their website.
"We would like to apologise to anyone who was disappointed by our non-appearance on TOTP on Sunday (believe us we were disappointed too)."
It goes on to say: "Due to derogatory, unfunny remarks made during the guest presenter's introduction to our performance, we felt we had to make a stand and leave. It was an important day for us and should have been special. We didn't take this decision lightly but stand by it."
Their place on Top of the Pops was taken by the electro-pop group Goldfrapp, who were in the studio at the time to record a performance for a future programme.