it doesn't! most girls wouldn't even notice, and if they did they probably wouldn't care. Facial hair isn't really in fashion now anyways, and I heard that guys with less facial hair growth have less propensity to balding as they get older! It's all in the genes.
Not at all man. I just turned 20 and I can't really grow a beard either. I have some facial hair if I don't shave, but nothing major. My girlfriend likes me clean shaven anyway. I think that's how most women feel about it.