The former Pussycat Doll has yet to comment on the speculation herself, but a newspaper report today claims Nicole has made the "heartbreaking decision" to step away from the show in order to focus on music.
A source told The Sun: "She put her music career on hold last year but now she wants to focus on her music."
Nicole, 35, had promised fans during the final Xtra Factor episode of last series that they would hear new music from her very, very soon. Her last single was 'Boomerang', released in January 2013.
While we're all for new Nicole music we'll be rather sad if this is one X Factor rumour that turns out to be true. As we said, though, there's been no official word from Nicole or X Factor. They do like to keep us all guessing, don't they?!
One person we know who definitely will be on the panel this year is Simon Cowell, who announced last week he's returning to the UK show.
The 55-year-old initially left the UK series to launch the show in America in 2011, but last week revealed he would return to the UK, signaling the end of the US version after three series.
Simon tweeted at the time: "Today was a day of mixed emotions. Excited to confirm me returning to the UK X Factor and sad to leave America. You realise how much you owe to the fans who have supported all of the shows we made in both countries.
"I have had a blast. Really the best time of my life. Thank you for the support and the fun we have had together. It's never forgotten. We found some amazing talent. And now the search begins to find the next star. That's why we make these shows."
So far, Simon is the only confirmed name for X Factor's next series in the UK, although that's not surprising given that the show generally keeps rather tight-lipped on the judging lineup until nearer the time of auditions.
One name that has been thrown into the mix this year is another former X Factor judge, Cheryl Cole, who, like Simon, left the UK version to judge on the US version only to be fired a few days into her role.
Simon recently told Reveal that he's hoping to get the former Girls Aloud star on board for the show this year, adding: "I think she'd be crazy not to come back, but you never know with her.
"I'm hopeful that we can do something, but people get cold feet and it's a big big commitment, so we'll have to wait and see."
He added: "Cheryl hasn't been on TV for about three years. The person I met recently, she looks better and is in a much better space than I've seen her before. She's a really good judge and that's the most important thing."
The previous series of X Factor, won by Sam Bailey, featured Gary Barlow, Sharon Osbourne, Louis Walsh and Nicole as the four judges. Both Gary and Sharon have left to focus on other projects, while Louis has been a bit confusing about his future, first saying he would leave, then suggesting he'd come back if Simon came back, so we're not really sure what he's doing this year.
Meanwhile, the X Factor live tour kicks off this weekend at the Odyssey Arena in Belfast, and continues through to the end of March taking in dates across the UK and Ireland.
The top eight acts from the last series will be performing including winner Sam and runner-up Nicholas Bailey.
Wow! Try REVEAL Mag for FREE on Android tablets!!!
Got an iPad or iPhone? Reveal Mag is specially designed to fit!
Relive the X Factor 2013 finale in our gallery below:
Copyright: Rex Features Tom Dymond / Thames