Perhaps your problem is not your interests, but how you potray yourself to people, (lacking in confidence, being ashamed of your interests, etc..), or you're going for the wrong type of girls?
I have a wide palette of friends(jocks, models, nerds, metal heads, etc..) and the guy I'm most attracted to is a friend of mine whose a stereotypical nerd. He plays a substancial amount of starcraft and W.O.W, he is very academic, doesn't work on his appearance, is not athletic (I was very close to beating him in an arm-wrestle, haha), and he is not physically, exceptionally attractive.
His appeal to me is his intelligence, and how confident he is in it. He is very opinionated, and slightly egotistical and he is always ready to have a big intellectual discussion and assert his opinions.
Even when he talks about the "nerdy" interests of his (Starcraft), he is very confident, open, and not self-concious discussing them, and to me, that level of confidence, in himself, is very sexy.
However, we do share very similar interests. Without being too conceited, I am also an academic(not that academia always equates to inteligence), assertive, and so I am always up for intellectual discussions. That's why, I find my nerdy friends much more attractive than the "jocks", even if they are more conventionally, aesthetically appealing. A man who is able to provoke thought, and even sometimes make me feel intellectualy inferior, intrigue and fascinate me.
So maybe you:
1) Have to be a little more confident in your "nerdy-ness" and your interests
2) Pick the right girls!
Hope that helps ;)