pr3ttybr0wn has a point but it also depends on the person. Hot will win over average but rarely you'd take hot seriously (more than sex) which is a curse for them hotties (both boys and girls) since everyone wants a relationship at some point and since everyone deserves it. But in generally I agree with your statement.
Most, like 80%, of guys and girls are average looking. Who you are attracted to out of the 80% is largely dependent on personal preferences, but most people don't look noticeably better or worse than most people their age.
If you are in the 10% of people who look like models, you will attract lots of attention and if you are in the bottom 10%, you will have a harder (but still possible) road ahead.