My family is part Irish, and I have grown up with a lot of my family members wearing caps like this, but now that I have moved out to go to college I notice that a lot of people don't wear these. Are they alright, or do they (as my friend says) yell "giant douche"?
I don't think you should be concerned with what people think. If they assume you're a douche because of the way you dress they're probably a waste of your time anyways. Dress in a way that empowers you not in a way that pleases others