In an interview with Daily Star, the Irish music manager said: "He's got my job. I've flat out left. I'm not going back to the show, absolutely gone."
Neither Nick nor X Factor have confirmed this though.
Radio 1 Breakfast Show host Nick said recently he would love to be an X Factor judge as long as it didn't interfere with his weekly show, saying: "I wouldn't risk my job at Radio 1 but I think I could do both. I don't see why I couldn't do it. I don't think it would contradict my show on Radio 1. I think they would complement each other."
The whole saga of whether Louis has quit/walked/will return has been going on for ages now to the point of nobody really knowing what is actually happening!
But this interview with Daily Star does seem to confirm that Louis will definitely NOT be returning to the show. Which will be weird. After all, he's been on the judging panel since day one!
Louis, however, doesn't seem too bothered by it, telling the newspaper he's happy to be getting back to his day job as a music manager. His new band is called HomeTown.
As always, bosses at X Factor are sticking with the usual line: no formal decisions have been made regarding the judging lineup for the new series.
It's been widely rumoured, however, that both Simon Cowell and Cheryl Fernandez-Versini will be back, while Mel B will be joining Louis in departing the show.
What has been confirmed is that Caroline Flack and Olly Murs will be the hosts, following Dermot O'Leary's decision to leave earlier this year. Also leaving is Sarah-Jane Crawford, who hosted the Xtra Factor. Her replacement has yet to be announced.
So much change!
X Factor's most shocking moments: