On Sunday night (15 November), Olly came under fire for telling Monica Michael she had left the competition, before the public vote had been read out.
At the time, the 'Kiss Me' singer apologised profusely and said he hopes it "never happens again", but that didn't stop Olly from being heavily criticised on Twitter. Shrugging off such criticism, though, Caroline showed her support for Olly and insisted "everybody makes mistakes" from time to time.
Reveal caught up with the 36-year-old on the red carpet at the ITV Gala last night and speaking about Olly's mistake, Caroline said: "The online critics are a tough crowd to please. He made a mistake, I make loads of mistakes, but you cover it up and you move on."
Insisting there is no point dwelling on the blunder, the TV presenter added: "You have to leave it behind you. We might mention it this weekend as a joke... but he'll be fine! And I'm like, 'It's fine Olly, just don't do it again!'"
Many had questioned whether Olly's slip up would cost him his job co-hosting The X Factor, but Caroline seemed to hint that wasn't case.
Recalling Simon's reactions, she told us the music mogul was nothing but lovely to Olly about it.
"[Simon's] a good boss, he's not the guy to be like, 'You shouldn't have done that!' He was just like, 'You live and learn. It happens.'"
Olly, meanwhile, has opened up about the incident, claiming his slip up has been "blown out of proportion".
He told The Mirror: "At the end of the day, I made a single mistake at a critical point in X Factor which was massively blown out of proportion."
"I was expecting a telling off if I'm honest," he admitted. "But I didn't get one which was nice."
As well as the backlash towards his mistake, Olly also received plenty of support. Axed X Factor contestant Monica said she "felt sorry" for Olly over the whole thing, while Xtra Factor presenter Rochelle Humes claimed her "heart broke" for him.
And even former X Factor host Dermot O'Leary made sure to show Olly his support, telling him it "happens to us all".
For exclusive competitions and amazing prizes visit Reveal's Facebook page NOW!
Olly Murs jokes about X Factor hosting job and says he'd love to return to TV again
Love Island's Caroline Flack 'has found love' with artist Blue Logan
Caroline Flack escapes cold UK for sunny LA - and shows off hot body!