The 43-year-old discusses the split in her new book True Love and admits that she was blindsided by the actor's decision to call off their wedding.
"Ben and I split up at the moment when I thought we were committing to each other forever, it was my first real heartbreak, it felt like my heart had been torn out of my chest," she writes.
The actress and singer fell for the Good Will Hunting star when they appeared in the movie Gigli together in 2002.
They quickly became the hottest showbiz couple around and got engaged later that year, but the wedding was halted just days before they were due to walk up the aisle.
The pair initially stayed together and insisted that the nuptials had simply been postponed, but split for good in January 2004.
"I sought out comfort in another person, tried to find someone who could make me feel loved and wanted in my loneliest hour," explains the American Idol judge, who began dating singer Marc Anthony shortly after breaking up with Ben.
The American Idol judge says that Marc, who had been a long time friend, rescued her from her pain and they married in June 2004.
But she admits that her decision to marry Marc so quickly may have been an attempt to "put a Band-Aid on the cut".
"I pushed all that to the back of my mind," she says. "Because life takes unpredictable twists and turns, right? And you've just got to go with what you feel is best in the moment."
Jennifer filed for divorce from Marc, who is father to her six-year-old twins Max and Emme, in 2012 after nearly seven years of marriage.
She was previously wed to waiter Ojani Noa and dancer Cris Judd.
The mother-of-two is currently enjoying being single for the first time in years, but has publicly congratulated former husband Marc, who recently got engaged to model Shannon De Lima.