I've noticed this is a frequent question on GaG and quite evident in our society. Before I answe your question I have to ask you something. What is the worst thing you can call a girl? A lot of names are probably coming into your head at the moment none of which I shall state in this message due to younger audiences. Now think of the worst thing you can call a guy. In most cases the names overlap with some exceptions. A girl is shamed for being a girl, and the worst thing you can call a guy is a girl. Now if society treats being a girl a bad thing, miraculously, people will see being a girl/feminine a bad thing. Of course this wouldn't be the case if there weren't stereotypes assigned to each gender such as "girls clothes" and "guys clothes" but it is what has been for centuries. Now a girl is not going to be as severely harassed about wearing pants as a guy will for wearing a dress. Because the worst thing a girl can be is a girl and the worst thing a guy can be is well, a girl. In France, women couldn't wear pants until 2013, in fact it was illegal to do so there. Lately quite a few feminist movements have been active and are trying to end this gap that involves these crude gender stereotypes. If you want a bit more information of feminism I'd suggest checking out this website. www.feministsforall.weebly.com Hope this helps.
Social perception. People have been conditioned over generations to think that a man wearing women's close SHOULD look ridiculous, therefore that's what they think. In time that attitude will eventually change, but we aren't there yet.
What exactly do you mean by "girl clothes"? Like summer dresses and skirts, or tight-fitting jeans?
I think it has something to do with society's strict rules on who should wear what. Guys are "supposed" to wear manly clothes. Feminism has certainly gone a long way in reducing the standards placed on women to always appear ladylike and innocent (no longer are women only allowed to wear dresses and skirts). However, men are now in the same position, which is what most feminists are hoping to curb.
The beauty - pardon the pun - in male clothes is that both can look alright in them, it's one of those things where maleness can look fine on anyone. It's not the same perk with women's clothes. However, some male wear looks ridiculous on women.
Many guys wear women's jeans, certain shirts, and even sneakers. It's more common than people realize, that's because (surprise!) not every article of women's clothing is girly some can be gender neutral.