I would take it as a compliment that so many people are attracted to you. You have to keep trying. Don't write off everybody quickly that is my only advice.
Like attractive guys are judged as players. Some dudes are shy, so they might stumble with words when you first meet them. Are these creepers just ugly people that make the good try of asking somebody out?
Get to know some of the stereotypes. There are no normal people around and there are no perfect men out there. lol
The only reason why a girl would attract pervs is because you're looking for them.
What I mean is, that you're falling for the good looking guys, who you assume to be an amazing prince-charming. In reality, these good looking folks are exactly the ones who want to get into people's pants.
You have to look elsewhere, outside your comfort zone. I'm sure a hard working nerds aren't s*xual, as they're lucky enough girls talk to them.