Two of my best friends are really tall. One is 6"6', the other is 6"4'. I have a friend who is 4"11', and she dated the first one for 2 years and is currently dating the other going on one year. Its completely acceptable. Sure, some people may harmlessly joke about it sometimes, but at the end of the day it really doesn't matter.
I think because he's a guy and he's tall most people want care, even if it's visa versa, and he was the short one, people would notice, but the world is full of different couples, I don't think people judge so much anymore.