What do you think about eye contact? (not 'does she like me' Q)

I've always been confused when it comes to eye contact, both with other men and with women. I'm entirely capable of maintain eye contact, it's just that I'm unsure how others feel about it.

With other men, I tend to feel as if keeping eye contact implies the message that I'm threatening them, trying to start something. I'll give an example. I'd be sitting in the train, and just casually checking out my surroundings to be aware, and I'd notice some guy already turn to see me or already be looking at me. Either case, I usually just look at them for a couple seconds and turn away. I'll admit that this feels like I'm somehow weaker, I know in the end it's stupid but still, breaking eye contact first always has felt more 'defeat'-like than just being nonchalant about things. Guys, what do you think about eye contact with other men?

Similarly with women, my natural instinct is to look away to not give the impression of being a creep. This may have to do with confidence issues, but I just imagine the girl seeing me keep eye contact as some weird dude trying to creep on her or look incredibly menacing. Sort of get the feeling that girls would overall get annoyed so I just diverge my attention elsewhere first. I should say that I can't bring myself to smile, always more of a just indifferent expression, but that's because of a whole other issue. Girls, what do you think about this?
What do you think about eye contact? (not 'does she like me' Q)
Post Opinion