O'Leary's alleged outburst came after he ran across the pitch at St Andrew's to salute Ellis after Villa's 1-0 win over Birmingham City on Sunday.
Ellis insists such behaviour would be out of character and is asking the FA to take into account the context of what happened minutes after Villa's first League win over the Blues in 18 years.
Ellis said: "I have never heard David O'Leary swear, absolutely never. It is usually a death knell when a chairman supports a manager, but my support in this case for the manager is not the 'normal' support.
"What he did was in the heat of the moment. If he has ever sworn, I cannot remember it. I have no doubt that he uses strong language in the dressing-room, but I hope that the FA view his actions in the context of the game. I liked the manager's gesture. It was emotional, spur of the moment and spontaneous."
O'Leary is to request a personal hearing with the FA to answer the charge.
His counterpart at West Bromwich Albion, Bryan Robson, has urged fans to get off Ronnie Wallwork's back after the midfielder was booed before Saturday's Premiership win over Arsenal.
Wallwork was held responsible by a small section of Albion fans as the club plummeted into the bottom three. Robson insists he remains one of the most valuable members of his squad.
Robson said: "The reaction from a small set of supporters to Ronnie was very much noticed by the management, staff and all the players - and we are all very disappointed."Reuse content