No, I don't consider it slutty or wrong. If you both want to have sex, why not? Just use your judgment. If you're looking for a relationship and you're having sex with a guy you just met at a bar, you're going to be disappointed. If you're just looking for sex, however, you have the right guy! If the guy you're about to have sex with is interested in a relationship as well, sex on the first date won't stop that from happening. The widely-mentioned stereotype of a guy leaving a girl shocked and dismayed after he screws her is hardly as widespread as people seem to think. I'd say most of that happens on what is obviously a one-night-stand. Or High School/College. You can see them coming from a long way off.
'Wrong' is a point of view, don't let anyone else define it for you! ;) If you want to have sex with a guy, go for it. You don't have to wait a set period of time to avoid being seen a certain way..