Number has meaning. and height would be height weather we could associate it with a number or not. So its not "just" a number. But that has nothing to do with your actual question so: Height doesn't matter if both you and your partner don't think it matters. I for one would have a really really hard time being with a girl taller than me. Especially considering girls wear heels now and then so hell no. If I was totally in love though; I wouldn't give a crap about height, My grandfather married a taller women. My grandmother is way taller than him. But they seem not to care about it at all. Of course they have spent decades together. And he doesn't seem to feel a bit less manly at all. I on the other hand would have a problem with it now, but maybe in a few years when I'm even more confident in myself and more mature and stuff I won't care?
I just think "Wow, she's tall!" If she's tall, if she's not particularly talI, I just take in the height difference like I do any other piece of information. I think most people are interested in couples to see what attracts each one to the other one, whether they seem suited to each other, whether theoretically they are on equal footing, but everyone is different and in a relationship someone's got to be taller, friendlier, lighter or darker skinned, longer-haired, thinner, more muscular, longer-legged. Nah, it's not weird, its just another characteristic.
I personally have never seen a girl that is taller than 6'3"... It doesn't mean they don't exist but I've never encountered that problem. As far as height being a problem, girls that are way shorter than me are fine with me. It makes kissing kind of clumsy sometimes and certain sexual positions kind of unworkable but that's fine. I can always pick her up to kiss her, sit her on my lap, or tilt her head back and bend to meet. So to me, yes I guess height is just a number. =)
With other people, no I don't find it weird at all.
I think it depends on the guy. I am on the shorter side, 5' 10", and I hate it when a girl is taller than me. It makes me feel insecure, probably because I'm already a bit insecure about my height. So I tend to go after shorter girls.
My best friend, on the other hand, is like 6'2", and loves tall ladies. He doesn't care if they are taller than him at all, in fact he kinda likes it because he just likes his girls to be tall in general.
So yea, there ya go.
To most guys height doesn't matter much, to girls its the complete opposite.