Sharon appeared as a guest panelist on ITV1's Loose Women earlier today (4 June) and confirmed that she will not be joining Simon Cowell, Cheryl Cole and Louis Walsh when the show returns this year.
During last year's series, Sharon was also hosting US chat show The Talk, meaning she had to jet backwards and forwards from Los Angeles to the UK, and it seems she - and her bosses - are not up for her doing it again!
When asked if she would be returning to the judging panel, Sharon replied: "No, no, no, you can't go back again. No. Love the show, it's become a part of British culture, everybody loves it, you can't help but get pulled into it but no, it's too much."
Referring to The Talk, Sharon added: "I'd lose my job in America because they let me out for one season. I'd wake up in hotel rooms and not know where I was. Also I've got a family so I can't do it."
Earlier this week, Louis Walsh finally confirmed that he is returning to The X Factor. Speaking to 2fm, the show veteran said: "Do you know what I did today? Exclusive, because I've told nobody. I signed for X Factor, like fifteen minutes ago."
Days later, Simon Cowell revealed that we will be finding out who the fourth judge is this week - and he's since admitted that Alesha Dixon is in talks and that it'll definitely be a female.
With Sharon out of the running, we wonder who Simon has up his sleeve?!
Who do you think the fourth X Factor judge will be? Tell us below, as the best comment from across the website each week wins £50.