The reality TV star confesses he has been taking steroid pills, and describes it as a "serious error of judgment".
The Mirror reports how Made In Chelsea's Spencer revealed all in a "seemingly honest statement".
After being pictured leaving Australia via Brisbane airport, Spencer said: "Why did I leave I'm A Celebrity? The simple answer is that I screwed up, and have some explaining to do.
"Shortly after I arrived in Australia I told the production team that I was taking a steroid-based medication that had begun some weeks ago.
"When I started taking this programme of pills it was in preparation for a charity boxing match which, ironically, never took place."
Spencer goes on to blame "vanity" for the reason he wanted to bulk up for the fight.
He added: "I had intended to come off the pills gradually, but there was no time to do this before I reached the jungle.
"My admission to the production team that I was still taking these pills led to the conclusion that it would be unsafe for me to stay in the jungle - a decision which I found extremely hard to stomach.
"I pushed back but ultimately the show's duty of care prevailed and bags had to be packed."
Spencer went on to say he "hugely regretted" taking the pills and apologised to his family, friends and fans.
I have disappointed my family, friends and fans and need to apologise. I would like to thank ITV for the opportunity and their support..1/2— Spencer Matthews (@SpencerMatthews) November 21, 2015
... I loved every minute of being in the jungle and wish the remaining contestants the best of luck. 2/2— Spencer Matthews (@SpencerMatthews) November 21, 2015
Taking to Twitter on Saturday (21 November), he said: "I have disappointed my family, friends and fans and need to apologise. I would like to thank ITV for the opportunity and their support.
"I loved every minute of being in the jungle and wish the remaining contestants the best of luck."
He is said to have flown to Abu Dhabi until the storm settles.
Spencer was notably absent from Friday night's show (20 November) after ITV confirmed he had left the jungle.
For exclusive competitions and amazing prizes visit Reveal's Facebook page NOW!