Just days after it was announced that former judge Cheryl Cole will be returning this year, new reports say Simon is lining up Fergie to complete the panel.
The Mirror reveals that show bosses are keen to complete the line-up with the American singer.
Whoever joins the talent show will sit alongside Louis Walsh and returning judges Cheryl and Simon Cowell - and pocket a rumoured £1.5million.
The newspaper says that talks between Fergie, 38, and producers have been successful.
The source said: "It's not a done deal just yet but things have been positive. Both sides are keen to move things forward and people at the show and ITV are excited at the prospect of having Fergie."
As well as a successful musical career alongside will.i.am in the Black Eyed Peas and as a soloist, Fergie is also a mummy to baby son Axl.
Her first child with actor Josh Duhamel was born in August 2013.
Do you think Fergie would be a good X Factor judge? Tell us what you think below!
Watch Fergie's video to her track 'A Little Party Never Killed Nobody (All We Got)':