Katie, 35, also insists that she is still unclear on the reason behind their divorce, meaning she is unable to get "closure" on their failed marriage.
The mum-of-four has spilled the beans on her relationship with Peter in her new autobiography, Love, Lipstick and Lies, which is being serialised in The Sun.
Today (14 October), the newspaper quotes Katie as saying in her book: "I thought we had a secure, happy family, something which I have always wanted.
"But that happy family was destroyed when Peter walked out - telling me it was over in a phone call.
"To this day he has never properly spoken to me about why he left so I could never get any closure on our marriage."
Katie and Peter split in May 2009, three-and-a-half years after tying the knot. The couple already had two children together before they married, Junior, now eight, and six-year-old Princess.
They met while in the Australian jungle together on I'm A Celebrity...Get Me Out Of Here! and, at first, it appeared they had a fairytale romance.
However, following their very bitter and public break-up, the couple stopped speaking. They still do not talk to each other to this day.
Katie, whose tell-all book is released on 24 October, has recently become a mum to her fourth child Jett Riveria, who she has had with third husband Kieran Hayler.
Peter, meanwhile, is expecting a baby with his girlfriend Emily MacDonagh in the new year.
Junior and Princess still appear on Peter's TV show, Peter Andre: My Life, however Katie refuses to watch it and wishes her middle two children weren't in front of the camera.
"One day he'll see what I'm trying to do in removing them from the public eye is the right thing," she adds. "Constant filming was one of the reasons out marriage broke up."
Reveal has contacted Peter's rep for a comment.