Since Nick's place on the judging panel was confirmed, he has constantly been subject to criticism, with many believing the Radio 1 DJ was the wrong choice for The X Factor.
Head judge Simon, however, has always defended his decision to hire Nick and as talk arises of Louis Walsh's possible return to the singing competition, the 55-year-old has assured Nick's place is "100 percent safe."
Despite going all out for the new series and instigating several changes to shake up this year's competition, reports suggest The X Factor hasn't managed to pull in the ratings it had hoped.
As a result, it has been speculated that Irish music mogul Louis is being roped in to joining Simon, Nick, Rita Ora and Cheryl Fernandez-Versini on the judging panel for the live shows. He will act as a neutral fifth judge in a bid to boost the show's viewing figures and secure a lead against rival show Strictly Come Dancing.
While some have suggested Nick is to blame for the show's ratings drop and Louis is being drafted in to make up for it, Simon was quick to shrug off such claims.
He told the Daily Mirror: "I think [Nick] is doing good. He is tough and he gets it. Is he being sidelined for Louis Walsh? No chance. Forget it. He is 100 percent safe."
Simon added: "It just takes a while for people to get used to new people. When you do the live shows that is when it really counts. Nick is fun to work with and loves being on the show."
Simon's comments come after Louis hinted his return to The X Factor is definitely a "possibility".
Addressing the rumours during a recent interview with ITV's Loose Women, Louis said: " Simon said the door was open to me months ago... There is a possibility I might end up on the show...But I don't know for definite. I'd like to. I'm available."
Louis then teased: "I love doing X Factor, I love working with Simon. He is a loose cannon but he makes great TV. He is going to have the best TV show this year. But he has four judges - does he need five?"
Nick, meanwhile, has hit back at his haters, insisting he doesn't take any notice of their negative comments.
"That's kind of the gig," he said. "You sign up for it and that sort of thing happens. I'm still having so much fun, it's a ball! I don't really pay attention to stuff really."
For exclusive competitions and amazing prizes visit Reveal's Facebook page NOW!