Matt, who will make his X Factor debut tomorrow night (31 August), has also described meeting Sharon Osbourne as "surreal" and says his partner in crime, Caroline Flack, is like a "big sister".
Stand-up comedian Matt said: "It is surreal, exciting and terrifying, all in one.
"I love it because I get to work with Caroline and everyone on the Xtra Factor team, but it's still really odd being part of the biggest show on television."
Speaking in more detail about Caroline, Matt added: "She feels like my older sister and I can't believe I haven't known her longer than I have.
"I feel like we've been friends forever. I spend most of my time trying to make her laugh."
Matt says his most memorable moment so far is when he met the judging panel, including returning judge Sharon.
"It was such a surreal moment," he said. "My first day was Sharon's first day back on the panel, and she's just been brilliant to me."
He continued: "She's so unique, compassionate and you can tell she's a great manager."
Our eyes and ears of backstage, Matt has already seen some of the first auditions, now held in smaller rooms with only the judging panel as an audience.
He thinks with this format, we get the best of both worlds: "I loved how intense and how different the rooms were, because I've been used to watching the arena auditions," Matt admitted.
"But then being in the arenas themselves, it's so exciting and it feels like going to a Bon Jovi concert everyday!"
It seems Matt is quickly fitting into the show, and already knows what he likes to see in a contestant's first meeting with the panel.
He explained: "When they sing really obscure songs, I always think that's very brave. And I always love it when a contestant belts out a big rock number!"
The X Factor returns on ITV1 on Saturday 31 August at 8pm. The next episode will be broadcast on Sunday 1 September, also at 8pm.
By Rebecca King
Are you looking forward to The X Factor's return? Let us know below, as the best comment each week wins £50!