BBC Radio 1 Breakfast Show host Nick and his close friend, singer Rita Ora, were confirmed as the new judges this week.
Not everyone was so happy though: a recent poll by our friends over at Digital Spy found 75% of people did not want Nick on the panel.
However, the show's new host Caroline Flack has said Nick's not paying any attention, telling The Mirror: "He's all good – he's just said he can't wait to get started."
Oldham-raised Nick said this week: "I love X Factor and have watched the series religiously since it started, so obviously I'm hugely excited to be joining the show. Music is my job and my passion and I'm really looking forward to seeing what talent we can find and nurture this year. I can't wait for Judges Houses' in Oldham!"
He will join Rita on the panel alongside Cheryl Fernandez-Versini and Simon Cowell.
Caroline will present the main show alongside Olly Murs, while The Saturdays' Rochelle Humes and KISS FM DJ Melvin Odoom will take the reigns on Xtra Factor. It's all change this year!
X Factor returns to ITV later this year.
X Factor's most shocking moments: