The music mogul publicly apologised to Gifty on Sunday night when she was eliminated from the competition following a sing off with girl group Four of Diamonds.
He said picking Fifth Harmony's 'In Love With A Monster' was the wrong choice.
Speaking afterwards on Xtra Factor, he elaborated: "You know what, it's one of those things. I think it was a mistake the song I chose. It happens and I feel bad for her but you can't take it back.
"When I heard her sing tonight at the end [in the sing off] and realised what a great singer she is, I think the song was probably too gimmicky. You can't beat yourself up and like I said on the show, you have to use this opportunity in a positive way."
Gifty sang 'A Song For You' in the sing off.
Simon had said on stage after Gifty's elimination: "I'll tell you what happened, it was my mistake, Gifty. I should never have given you this song and you shouldn't be out of the competition at this point. It is not you, it's me."
But Gifty later replied: "Simon has been absolutely fantastic, he's always had my best interests at heart and has always done what made me feel comfortable. With him saying he picked the wrong song, I don't blame him, I feel he honestly felt it was the right song for me and I would smash it.
"I tried my absolute best. I knew it was a risk doing Fifth Harmony song, like third time. I personally feel people are going to think she's too predictable, but you know what, I did it and I have complete faith in Simon; he knows what he's doing."
Gifty added on Good Morning Britain on Monday: "Personally, I feel the song choice I had been given was a bit… I personally didn't feel too great doing it, another Fifth Harmony song, a third time. I honestly thought people would think, 'Oh, she's a bit too predictable.'"
"I had a chat with [Simon] after my performance on the Saturday and told him I was going home, I knew it," she added. "I didn't get a chance to speak to him on [Sunday] as he had to rush off.]
"Obviously I'm grateful for the opportunity and the fact he put me through as one of his final three girls and had faith in me. So many people auditioned for the show – thousands – and I made it to the final three years, and that's something to be proud of. I'm chuffed, I'm so happy."