Well lets boil it down past the sexes issue. People are generally afraid of rejection, plain and simple. Who really likes to get their ego smashed to pieces, and regularly if they're trying all the time? It will be very detrimental to your self-esteem, and probably start causing other mental issues.
I personally hate it with a passion. That's why I don't ask certain girls out, because of this rejection factor. I do ask girls out, and I usually do get rejected, but by that time, I've already mentally prepared myself to accept that there's nothing going to happen between us; i.e. ready to accept to be in the friend zone. But I will not go up to a girl I don't know, and ask her out. The chances of rejection are so high, and at these times, you can never mentally prepare for this, because like an anonymous poster wrote, there are so many ways a girl can turn a guy down these days.
Now I think it was easier back in the hey day, because their was inequality (I'm not pro in-equality, but here this out), and we had roles we all played. Guys would ask girls out, and that was that. A girl could accept or deny. But given that the gender roles are nowadays ever changing, anything can happen, and in different method. I've always thought that back in the hey day, that if a girl was going to reject your advances, although they'd be flattering, she would do it gracefully, and the gentleman (I'm assuming) would also accept the rejection gratefully.
But these days, some girls are given so much power in this dynamic, that they can flame you till your chargrilled, and would reject you not so gracefully. Actually some make it a public spectacle. A girl will speak her mind (which I'm all for), but sometimes, when she does speak her mind, it comes out more hurtful, than being a nice person; some want to take you to task, for even contemplating asking them out.
Also, it's been stated below, that some would see you as a creep. No longer is it flattering if a guy were to ask you out, and you'd reject him. Now, it seems that you're a creep for even thinking about it. Imagined if you asked around work / school / uni, about her - the creep factor has multiplied exponentially. Tell me that this isn't a smash to someone's confidence, and self-esteem. I don't think I've seen a guy reject a girl humiliatingly before (but that's probably because the girl isn't asking the guy out in the first place...haha)
Although some things have changed since equality has been raised as an issue, that doesn't mean everything has progressed in the same fashion - if we were equal, guys and girls would be asking each other out.
And you're right, guys should man up, and just ask girls out. But there will always be the fear factor which will hold them back, and sometimes, the risks don't justify the benefits.