Honestly, for the vast majority of people, they can take steps and action to be more attractive. If guys eat right, work out, get a cool haircut and just generally put some effort into their appearance, it's not all that hard to be at least minimally attractive. So women don't have much respect for guys who don't look like they put any effort into their appearance at all. If a guy is walking around with a ton of extra weight on, or he looks like he's never worked out a day in his life, or he dresses like he found all his clothes in a dumpster, then there's really no reason for him to be saying sexual things to attractive women.
Lack of attention to looks shows lack of self respect and discipline, and that's just not attractive. No one wants to get hit on by someone who can't even manage their own looks to a basic extent.
It's what you say and how you say it, not what you look like. Most guys who are considered attractive are considered to be so because of a variety of things, not just looks. Good social skills gain a guy hella points. Knowing how to joke and flirt. I've known tons of good looking guys who are ultra creepy and don't do well with women. Its because there is a hint of anger and "I wanna hold you down and hurt you." to their sexuality, a lot of times.