The retired late night TV icon discussed the playful moment when Tonight Show host Jimmy Fallon tousled Mr Trump's hair during an appearance on his show during the presidential campaign.
In an interview with Vulture, Letterman said Fallon got a "fantastic viral clip" out of the moment, but said he "would have gone to work on Trump."
He said: "I think I would be in the position to give him a bit of a scolding and he would have to sit there and take it."
"Trumpy," his pet name for the president, "broke" his heart during his feud with Representative John Lewis, he said.
"In addition to every other thing that’s wrong with the Trump, he’s ignorant in a way that’s insulting to the office, insulting to America, insulting to human rights, insulting to civil rights, insulting to John Lewis. Trump saying that broke my heart.
"I thought, You stupid son of a b***h. You ought to have known better than that."
Mr Lewis received a skull fracture after marching across the Edmund Pettus Bridge in 1965.
Letterman added: "John Lewis goes down there and gets a goddamned skull fracture. I mean, Trumpy will never have to worry about a skull fracture because of the hair.
"How do you know if Donald Trump is lying? His lips are moving."
David Letterman through the years
David Letterman through the years
1/10 David Letterman
Jay Leno, left, hosted by David Letterman on The Tonight Show in 1979
2/10 David Letterman
David Letterman sits at his desk on the television series 'Late Night with David Letterman,' New York, 1986
3/10 David Letterman
David Letterman holds an Emmy trophy after his win in 1987
4/10 David Letterman
David Letterman will retire after 31 years as America's longest-standing late night TV show host
5/10 David Letterman
Madonna holds hands with television talks show host David Letterman at the 11th Annual MTV Video Music Awards in New York in 1994
6/10 David Letterman
David Letterman and Harrison Ford ride horses named Chase and Shane outside the Ed Sullivan Theater in New York, 2011
7/10 David Letterman
David Letterman speaks onstage at the First Annual Comedy Awards at Hammerstein Ballroom in New York City, 2011
8/10 David Letterman
David Letterman and Oprah Winfrey attend 'A Conversation With David Letterman And Oprah Winfrey' at Ball State University in Muncie, 2012
9/10 David Letterman
David Cameron was welcomed on to the Late Show by host David Letterman to the tune of the house band playing Rule Britannia and dry ice pumping into the studio to replicate a London fog
10/10 David Letterman
US President Barack Obama tapes an appearance on the 'Late Show with David Letterman' in New York, 2015
During the interview, he said senior Trumpe aide Kellyanne Conway was his "favourite for a long time" and described White House press secretary Sean Spicer as "a boob who just got out of a cab and now here he is."
When asked about Stephen Miller, he said: "Wow, that guy is creepy. He fell out of a truck."
He also mocked Steve Bannon, Mr Trump's chief strategist.
“And he [Mr Trump] hires the Hunchback of Notre Dame, Steve Bannon, to be his little buddy.
"Bannon looks like a guy who goes to lunch, gets drunk, and comes back to the office: ‘Steve, could you have just one drink?’ ‘F**k you.’ How is a white supremacist the chief adviser to our president?”
Letterman retired in 2015 after 33 years as a late night host.