Jay was voted off the singing competition during last night's results show (16 November) after ending up in the bottom two with Stevi Ritchie.
Dermot O'Leary revealed that Jay would be going home after receiving the least amount of public votes, and soon enough, his mentor Simon Cowell joined him on stage to praise and defend the singer.
Simon said the claims about Jay were "completely not true", and he praised the 30-year-old for being a "gentleman" to work with.
Speaking to Reveal this morning, Jay tells us: "I was gutted about that. I was made aware of it yesterday morning and it crushed me. I'm a massive people person, I really am, and I hate seeing people upset.
"Throughout this competition there have been people in that position and I've been the first one to comfort them, so to find myself in that position… I was like, 'What the f**k?'
"That was horrible and at that moment I thought that if I was to go I'd be happy because that's not what I'm about at all. It's the polar opposite of what I am."
Jay, who says he took on a "father figure role" in the house, continues: "I didn't expect Simon to bring it up but I'm really grateful that he did. It's total b****cks and I was really upset by it. I genuinely hope that people have seen enough of me to know that I'm not that type of person and I wouldn't condone that bull***t."
Reveal is now on Google Plus! Join us here for the latest pictures, fashion and celebrity news!
Watch Jay's sing-off performance below: