The Celebrity Big Brother star, 34, has written down his memories in new autobiography Second Best: My Dad And Me.
In extracts published in the Daily Mail (14 March), Calum reveals the difficulties of growing up with an alcoholic father.
Footballing legend George famously battled alcoholism during his adult years and eventually died from alcohol related issues in 2005.
In the first person account, Calum shares a number of shocking stories, including one alleged incident when his father left him to fend for himself overnight - when he was just 11 years old.
Calum writes: "[The next day] he comes up to the table where I'm eating on my own and, in ways I don't comprehend at the time, he tells me he left me last night to go and see a hooker."
The former model also claims his father tried to kiss him once while drunk, mistaking him for a woman.
The most shocking story, though, is perhaps when Calum describes his dad's alleged violent rage when he claims his father grabbed him by the throat, hitting his head against the wall, and tightening his hand as his "feet lift up off the ground".
Calum eventually ended up living with his mother in the US, after she won sole custody for their son.
He recently returned to the spotlight in CBB, where he opened up to his fellow housemates about his famous dad.
He told housemate Perez Hilton he was "devastated that I lost my old man so young".
He also described how he himself tried to blot out the pain by turning to drink, saying: "I went thought that for a good three or four years, like a complete blur, and then I came out of it.
"[Alcohol] was never something I was worried about. I never woke up and drank all day.
"The only time I partied too hard was when I lost him and that was because I was traumatised."
Calum is dating model and former Made In Chelsea star Ianthe Rose.
For exclusive competitions and amazing prizes visit Reveal's Facebook page NOW!