My guy friend cannot look me in the eye.. He talks to me just fine, but if I look him in the eye, he'll look away and down.. The only times he's been able to hold eye contact, it was held for a REALLY long time.
Either he has a crush on you and has trouble looking you in the eyes, or he has a thing about looking people in the eyes. I know some people can't hold eye contact very easily because it's too intimate. Not romantic, it's just a personal gesture towards another person which some people feel uncomfortable with.
It could be lots of things. Maybe he's shy, has low self esteem, or maybe it's a culture thing. Usually shift eyes are associated with lying, but it's more than likely one of the first diagnosis mentioned.
Low self esteem, being self conscious, and what not causes people to have trouble looking people in the eyes. My cousin is like that too. He'll look down, pretend to focus on something else, or play with something else to avoid eye contact.