That being said, his hardships - and relationship with Brienne - did allow viewers to see a different side to the Kingslayer, played by actor Nikolaj Coster-Waldau.
"Jaime is a great character to play. I like the fact he's so conflicted and has all these secrets," said Nikolaj at a recent cast junket in London.
"It's good writing. He seems like a whole human so there are always things you can identity with. I think most of us dislike when people jump to conclusions about who you are. Sometimes it happens when you're an actor, as they see you in a part and presume you're that role. [I sometimes get], 'Hi Jaime, how's your sister?!' That happens for Jaime on a huge scale as he is known as the Kingslayer."
Here's what else Nikolaj said at the junket - warning, spoilers ahead!
On that scene where he lost his hand
"Noah Taylor, who played Locke, was so scary. The night we shot that scene I was sick which was actually very perfect. I felt really bad and he was acting very violent, as he should be. They built this huge knife, a version that he could whack down on me. Of course, he had to be very accurate – and he wasn't always! That was good for my reaction. It did actually hurt because he went for it. There was another day we shot a scene after that where I fall off the horse and he beats me up. It was so muddy and he had to kick me in the stomach. That isn't dangerous but you have to be specific about the kick. Because it was so muddy and we were rolling around, he missed and hit my rib, which took a month to get over. They had to reshoot it three times as Gwen (who plays Brienne) was laughing."
On the obstacles he faces now his character is without a hand
"As an actor obstacles are gifts. For a guy who is defined in many ways by his ability with a sword, to lose that part of him is a huge obstacle. The writing was so strong and so good that the scenes after that were a lot of fun to shoot."
On whether Jaime has evolved since series one
"It seems like a huge change and there are obviously some changes in him, but I think it's all about point of view. If you're told before the scene in episode one where he's having sex with Cersei and pushes Bran out of the window, that, yes, it's his sister but it's true love and they want to be together – it's horrible but it's a romantic love – and there's a horrible, evil boy climbing the wall. If that horrible boy tells anyone what he saw they'll have to kill Cersei's children and she would die and he would probably die, it would be a different scenario. You might think differently.
"He does something because he's about loyalty and family and protecting the people he loves. That goes through to what happens with Brienne, I think. The fact he meets this woman is huge for him as he's forced to spend so much time with her. For the first time, I think, he does something for someone outside his family and risks everything for this woman.
"I think the biggest change is the fact he loses his identify in many ways and has to reevaluate and rediscover himself. That goes back to the scene with Brienne in the bath where he reveals this huge secret that he's carried since he was sixteen. He tells, of all people, this woman because I think they have connected and he sees himself in her. The big question, of course, is what happens when he gets back to King's Landing? That's where we'll find out what has changed. At the end of season three something has changed but he doesn't know what it is. I think he surprised himself and doesn't really know why he went back and rescued Brienne from the bear pit."
On the show's directors
"They work differently and that's what's very inspiring both for us as the actors but also for the crew. I think what the producers have been very good at is finding the directors that really love the show and have a huge passion for it. Sometimes actors think we work hard but on a show like this it's really directors as they are booked for six months solid because of how they film the show in different locations. One day you shoot episode four and seven; in the morning four and the afternoon seven. The director is there the whole time."
"It was amazing. I was astounded by it. It was a very difficult episode because it's such a huge part of the story. Of course, there are variations from the book to the show, but for me the execution [was great]. Even when you're on the show you say, 'How the hell did they do this?' It is a television show and even if you have a huge budget it's still limited. The scripts are brilliant but the directing is amazing. David Nutter did that one and it was just brilliant.
On whether he worries he will die before the last episode
"It's great to be part of this series. I'm so grateful and it's wonderful but it's also a television show and another job - a fantastic job but it will end one day. I'd love to be part of the show until the end, I love working with these people – they're amazing – but you know it's still a television show."
On what season four has in store
"I think for Jaime it will be the fact the Lannister family is together for the first time on the show. That will be very interesting to explore. It's a very dramatic season and where before you'd have one or two episodes that peak, here there will be a lot of peaks."
Game of Thrones: The Complete Third Season is out on Blu-ray and DVD from HBO Home Entertainment on Monday 17 February
With thanks to Julie Bundy