Don't know if this will help but here goes: I have OCD, actually diagnosed and everything so not just made up. One of my quirks is watching every physical thing people do around me, kind of like people watching.
This guy could have a tendency to do that, not saying he is or has OCD, but has the itch to watch your every move. He may not even notice he does it, just a nervous tick. I wouldn't think anything of it.
My advice, so long as he is paying attention to you and not completely disregarding or ignoring you, I don't think it would be a problem.
He is very, very nervous. He doesn't mean to look at your hands, but when people become anxious, scared, or nervous they look directly at movement. It just so happens that your hands are moving the most. He doesn't mean anything negative by doing this. It's just human nature to look at movement when feeling anxious. I look around the room when talking to women, because I'm nervous.
Most girls have very graceful and beautiful hands. I think a girl's hands are very underrated as a body part. I personally love to watch a girl do things with her hands. It sort of fascinates me. So this guy might have some of that same fascination going on, especially if he likes you.