Andrea was in the bottom two on Sunday night and was saved from the chop by mentor Mel B, Cheryl Fernandez-Versini and Louis Walsh.
Naturally, Simon voted to save his act Stevi Ritchie - who ended up going home - and he told Andrea he hadn't performed as passionately as his rival.
Simon has since been quoted in The Sun saying Andrea won't win the show, adding: "[It's] because I think he's peaked. It's the same thing week after week."
Appearing on ITV1's Loose Women today (26 November), the Italian hopeful told the panel that he is trying not to think about Simon's comments.
"He's having some doubts," Andrea said. "I want to enjoy myself so much and I don't want to think about anything and I want to sing my heart out.
"I do hope that Simon will change his mind. Obviously it's not good but it's fine because I have to learn everything about this job and improve myself."
Andrea first caught the attention of the judges when he rocked up to the auditions in a jumper featuring a picture of a pug dog. Since then, he has continued to impress with his vocals, which is why many viewers were shocked he ended up in the bottom two on the weekend.
But despite this hurdle, Andrea is still embracing his X Factor journey with both hands.
"It's like living in a bubble and I'm not realising what's happening," he explained. "People stop me in the street and it's like, 'Wow - are you sure? It's me!'.
"But it's so beautiful and I can feel the love from this country. It's amazing."
Asked who he thought was his biggest rival in the competition, Andrea added: "I think we are at a point where everyone can win or everyone could go.
"I do like Lauren [Platt], I think she's incredible."
Since Mel B picked Andrea for her top three at Judges' Houses in the summer, the pair have grown increasingly close. Unsurprisngly, he was full of praise for the former Spice Girl today!
"I think she's great," Andrea gushed. "She's quite crazy in a good way. She's always present and constantly in touch with me during the week. She's giving me a lot of advice. She's a support more than a judge."
Aww, bless him!
Who do you want to win The X Factor? Tell us below, as the best comment from across the website each week wins £50.