The Loose Women star voiced her opinion after the last round of nominations, where James surprised housemates by nominating house BFF Austin Armacost for eviction as he didn't approve of his behaviour.
Sherrie told James: "It's not up to you to be the ruler of the house or the conscience of the house. We do what we do but you aren't the person in this house..."
James, however, didn't want to talk about it anymore, saying: "You know what? It's done. We don't have to keep revisiting it, do we? I don't want to talk about it anymore. I've spoken about it more than enough."
Austin piped up to say Sherrie was just saying what he had said: that by nominating Austin, James had wanted him to leave. James had said he nominated Austin to show he didn't appreciate the behaviour.
James defended himself, saying: "For me it's not about the consequence [of possible eviction]. It's about right and wrong. The kid can't do anything wrong in your eyes..."
"That's not true," said Sherrie. "I would tell Austin if I thought he was out of line. But he's never been out of line with me. Why are you the conscience of this house?"
Jenna Jameson and Janice Dickinson were evicted last night.