After Kavana made the comment earlier that evening, Keith was unable to let it lie telling the 'I Can Make You Feel Good' singer he has never been spoken to like that before.
As they headed to bed, Keith confronted him, but their conversation soon escalated leading Kavana to launch into a long rant about the TV presenter.
Explaining why he was left hurt after Kavana's remark, Keith told him: "It was just weird that you said 'f*** you Cheggers', I've never had that said to me in my life."
Blaming the house for leading him to come out with comments like that, Kavana tried to bury the hatchet, asking Keith if they were friends.
But not willing to forgive and forget just yet, Keith told his housemate he will speak to him again in the morning once he is sober - leading a drunk Kavana to hit out at his new pal.
"No it's not about being sober," he said. "It's about being real.
"You're stone cold sober and I'm drunk and you're the better person for it. You're perfect, aren't you? I've apologised to you hand on heart, but you keep making little digs at me.
"You've done it the entire time I've been here - 'What about you last night?', 'What did you do last night?'. You do it more than anybody."
Not one to get involved with the house drama, Keith has been criticised before about "sitting on the fence". Making another dig at the 58-year-old, Kavana again accused Keith of deliberately avoiding confrontation.
"Don't get me f****** started Cheggers," he continued. "I could say a million things about you sitting on the fence. You sit on the f****** fence every single day."
Defending himself again, Keith responded: "I'm allowed to do as I wish, as you are right now. I've never been talked to like that in the whole of my life. No one has said to me 'f*** you Cheggers' and that's why I'm speaking my mind."
Meanwhile, the house were told earlier that evening by Big Brother that the second eviction had been cancelled due to Alexander O'Neal's departure over the weekend.
In a nominations twist, Big Brother then revealed that instead Katie Price had to save one of the nominated housemates and replace them with someone else.
After having 24 hours to make her decision, Katie saved Katie Hopkins from the public vote and nominated Calum Best.
Calum will now face eviction alongside Perez Hilton, Alicia Douvall and Nadia Sawalha on Friday.
Celebrity Big Brother airs nightly on Channel 5.
Reveal is now on Google Plus! Join us here for the latest pictures, fashion and celebrity news!