A newspaper today claimed Caroline and Olly's friendship is on the rocks after Caroline found out Olly knew before her the pair would be axed from the show after one series.
But Caroline, 36, took to Twitter to deny any falling out and reiterate that she chose to leave X Factor on her own accord.
Caroline said: "I hate being made to make a comment on these things but I feel forced .....
"I'm not bitter about X Factor ...I have not fallen out with Olly, I wanted to leave, I had the best time. I'm happy. Rested. Feel blessed and am looking forward to the rest of the year .... Everything evolves and changes. Time to move on xx."
Caroline and Olly, 31, became the show's first-ever co-presenters when they took over from long-time host Dermot O'Leary last year. They'd previously formed a successful partnership on spinoff show Xtra Factor before their promotion to the big stage.
But they both announced this year they would be stepping down.
Olly was first to break the news, saying: "This was an incredibly hard decision to make and one I didn't take lightly as I've really enjoyed co-hosting The X Factor.
"It's been one big adventure and I loved every minute of it. But my heart belongs to music and I want to concentrate 100 per cent on making my next album and getting back to performing."
He added: "Last November I was doing the show, promoting a new single and I felt increasingly like the music side of things had to be shoehorned into the TV schedule."
Caroline later followed, saying: "I have had a brilliant time working on The X Factor over the last few years, and hosting the main show was just fantastic, I made some amazing friends. I'm looking forward to carrying on working with the team over at ITV."
It was recently confirmed Dermot will return to host X Factor this year.