And it's something fans - who would LOVE them to get together - will no doubt be thrilled to hear.
The X Factor presenter reveals that amid undeniable chemistry, the pair *almost* had sex after filming the spin-off show four years ago.
In an interview with The Sun, Caroline confesses: "There was a time where we nearly did have sex.
"It happened at the first Judges Houses, four years ago.
"We had a lot of fun after filming and Olly was like, 'I'll come see you in your room' and I thought, 'Oh my God, is it happening?'"
But Caroline, who implies she ran to her room to change into some sexy underwear, says that ultimately nothing happened!
She added: "We didn't even snog. I don't know whether we were just waiting for each other to make the first move. Maybe. But nothing happened.
"Once you've gone through that 'almost' phase you know it's never going to happen."
Although the pair both insist they are now firmly in the friend zone, it doesn't stop their fans hoping they'll change their minds!
We have to admit, we think they'd be amazing together...
This Sunday (13 December), Caroline and Olly will announce the X Factor winner between finalists Louisa Johnson and Reggie 'N' Bollie.
Ché Chesterman received the fewest votes on the Saturday night show and left the competition.
Louisa is the firm favourite to win with the bookies; William Hill believe that Louisa has it in the bag and is the red hot favourite at 1/7.
Adele, Coldplay and One Direction will all perform on the night.
For exclusive competitions and amazing prizes visit Reveal's Facebook page NOW!