Walsh is one the show’s longest standing judges and has appeared on a panel for 11 seasons of the ITV competition.
The 62-year-old made the comments during an appearance on the Seven O’Clock Show on Thursday, telling his hosts: “I've done 11 seasons and I’m quite happy.”
When he was asked if he was done with the show, he responded: "I’ve done 11 seasons - that's a long time in television. I think they’re going to change the show and get new faces."
He said Cheryl Fernandez-Versini will return, but failed to mention any other judges. “I think I’m done at X-Factor,” he insisted.
Walsh also said he would have to carefully consider returning for a 12th season, explaining: “My day job is as a manager. I’ve kind of neglected that and I want to get back to being a manager. I’m working with Shane Filan - he’s got a new record coming out, and I’m working with Hometown.
“I’d like to get back to where I started off… I never even wanted to be on TV, ever, it just happened.”
His comments come after Dermot O’Leary, another enduring face on the programme, stepped down from presenting the show.
The 41-year-old has been an X-Factor host for eight years. Announcing his departure, he said it was time for him to “move on” after “eight wonderful years”.