Guy playing with my hands?

What is going on? This guy that I like (and who I am almost sure likes me) keeps playing with my hands - like casually but not.

This is the second time he's done this. The first was a high five that turned into a casual sliding his hand across mine as if he wasn't paying attention - but then he deliberately started to (tentatively) wrap his fingers around my hand and I responded - once I responded he sort of held onto my hand until we let go. It was only for a few seconds but it felt like forever.

The second was tonight. I was talking to him about something and he reached out his hand to me casually so I reached mine toward him, not really knowing what he was doing. He just sort of took hold of my hand and - I don't know what - it was like a high five but he kind of gently played with my hand for a minute. I apologized because my hands were freezing from washing them. He didn't really respond and just sort of folded his hand around mine a little and then pulled back and started to - I think - interlace his fingers with mine when his brother walked in and he acted all casual and we just dropped hands. This was during a normal conversation - nothing romantic or anything. I mean, joking around, but nothing mushy or whatever.

Am I wrong in thinking that he really cares for me? Background: I am 99.99999% positive that he really likes me and has for a while, and (long story) fears he will lose me.
Updates
+1 y
Answers from guys particularly appreciated (and girls - I just want to know what guys think about what's probably going through his head). Thanks everyone :)
Guy playing with my hands?
Post Opinion