The Keeping Up With The Kardashians star, 34, says it stems from her first pregnancy, which "changed her personality alot".
In more extracts from her interview with C Magazine, the reality TV star admits she has never forgotten harsh comments about her weight after 'bloating' when she was pregnant with North West in 2013.
Because of pre-eclampsia, Kim says she swelled during pregnancy and was hurt by people's body-shaming.
She tells the magazine: "Last time [contributed to] not only the weight, but also the swelling that everyone would make fun of, not understanding that I had this condition."
Speaking to Amanda de Cadenet in the magazine's 10th anniversary cover story, she continues: "I couldn't help it, and everyone would say, 'She can't stop eating'.
"I delivered at 180, and they were like, 'She's 210 pounds. She's getting dumped because she's too fat' and all these ridiculous stories. It really took a toll after the fact, when I was losing weight. I gained 50 pounds, and it's tough to get it off."
She goes on to say that the abuse "changed the way I viewed wanting my picture taken".
She decided she no longer wanted to smile for the "same people that made fun of me, and posted the stories that were so awful, calling me fat for something I couldn't control".
However, Kim - who is currently pregnant with her second child - seems to be happier with her pregnancy body this time around.
She has even posed naked in a selfie!
Kim and her rapper husband Kanye West, 38, are expecting a baby boy in December.
There has been mounting speculation about what they plan to call their child, with some reports saying they have settled on Easton.
All you need to know about Kim Kardashian:
For exclusive competitions and amazing prizes visit Reveal's Facebook page NOW!