According to my own experience: yes, it's possible that being too nervous can stop men from hanging out with girls. I was too nervous myself to get to this point years ago... until I saw it's no big deal, that it made me regret for not doing anything and today nothing can stop me if I want to.
So, today the answer is no. Have in mind though, that it can happen and it happens.
If he is young and in experienced like a highschool student or is new to looking for women to date then yes he can get to nervous, panic, and chicken out. The more times that your put into a situation like that the more you get used to it and will be less nervous about it. However that feeling of butterflies never goes away.