Model Oliver, 25, and salesman Troy, 23, had just one day to secure their place in the villa before the girls voted to dump one.
Unfortunately for Oliver, that was him, as the girls decided Troy had connected with Tina Stinnes and there could be the chance of romance there, as Tina is single.
Just before leaving, he muttered: "I'm not going to put myself out there if I don't feel it. He will."
He was overheard by some of the islanders, who wondered if he'd been accusing Troy of being fake. "I don't know if he meant it like that or said it off the cuff or was pissed off," mused Olivia Buckland. "I was a bit confused."
Kady McDermott decided to investigate and approached Oliver, who told her there was "no connection" between Troy and Tina, despite what Troy said.
"It's not like it's faked," he added, saying: "Time will tell."
"The second another girl comes in he's attracted to, done," said Oliver. "And you'll be like for f**k's sake."
His parting words were: "I think the islanders need to keep a close watch on Troy and his games because he's smart playing them and one by one, he'll play the girls and cause a big problem."
It was enough for Kady to put her guard up as she told the Beach Hut: "Troy knew he had to stay in and getting close to a girl is the easiest way to stay in. If people look at him differently, I don't know. Maybe we'll be a but more wary of him."
Love Island continues at 9pm on ITV2