Ever since Louis Walsh stepped down as judge, the rumour mill has been churning out name after name, with everyone from Louis Tomlinson to Nick Grimshaw offered up as potential replacements.
Speaking at Capital's Summertime Ball, Olly, who will be hosting this year's X Factor alongside Caroline Flack, said: "When you're picking judges you've got to remember that Louis, Sharon and Simon back in the day when they were on the panel, they weren't celebrities at the time."
He added: "If you've got a really strong personality and got a good appeal you'd be amazing on it, so it could be someone completely unknown."
The 'Troublemaker' singer did seem pretty keen on rumoured judge Rita Ora taking on the role, saying: "I think she's great, she's a really amazing great fun person, and she'd be great on the panel. She's very opinionated and obviously she was great on The Voice. It would be good for Simon if he got her, but it's all rumours."
Still sounds promising though, right?!
Olly has even taken matters into his own hands when it comes to filling the sought after position, admitting: "I was trying to convince Robbie (Williams) the other week to do it. I thought he'd be amazing but it was a straight no from him."
Well at least that's one person we can cross off the list!
Olly revealed he's excited to kick off his new presenting role, saying: "I can't wait, I've been working really hard this year focusing on my music and the touring, so its going to be a nice end of the year to go back on to the hosting and presenting."
X Factor's most shocking moments:
For exclusive competitions and amazing prizes visit Reveal's Facebook page NOW!