Honestly, they are always a factor. If there's no physical attraction at all then a romantic relationship probably won't work. If I really loved someone's personality, then they don' t have to be exceptionally attractive to me but there has to be at least some degree of attraction. If they're initially average looking it's possible that I might find them attractive if I really like them as a person, but it depends. Also, what constitutes "average looking" or "attractive" definitely varies by person.
All people are the same. When you first meet them, you want their respect and they wan yours. If you are obese, it is obvious to the other party that you do not respect yourself so why should they respect you. Stop waddling and start walking, you might find life a new and better package.