The presenter told an audience this week that he wants to exit his BBC late night programme on a high… which seems to mean no reality stars allowed!
Graham said: "The thing no one wants is the inching backwards where you're kind of going, 'Yes, the cast of TOWIE can come on… That's about retiring when you're at the top."
Hmm… we wonder what the TOWIE lot will think?!
"You want to get out when people are still watching your show," he explained. "You don't want it to be when people say, 'That is still on, that is still a thing?'
"This is the 18th year of me doing a chat show," he continued. "We've been working up to this moment of being on BBC1 on that prime Friday night slot so we've always been inching forward in those 18 years.
"I'm not jumping tomorrow but it is in my head. You want to get out when you can still enjoy your life."
Fingers crossed he doesn't want to quit just yet though!
Graham was speaking at a panel event in London ahead of the launch of his memoir The Life and Loves of a He Devil.
For exclusive competitions and amazing prizes visit Reveal's Facebook page NOW!