Why did this guy push my hand away?

I have a guy friend that I am attracted to. He is a conservative christian and I am not.

He shows signs that he likes me.
But today I held his hand to read the lines on his palm jokingly. Because we were having a conversation about fortune tellers.

When I grabbed his hand and opened his palms he pushed my hand away abruptly.
But then he put his hand around my shoulders and rubbed my arm.
Why did this guy push my hand away?
Post Opinion