Is he avoiding me?

It doesn't sound like it to me. Maybe he's a little forgetful, but that's not a good thing. Him inviting other friends is probably because he thinks that that's all there is to the friendship. Be honest with yourself here: is that what he wants? What do you really want? Would you rather have him as a friend, or boyfriend?

he might just be unaware of your crush. Guys generally live their lives in the open. Usually, not always, mind, it is possible to see how a guy feels about you even without them opening their mouths. Don't, whatever you do, read too much into what you see, though, without asking directly.

Hey sweety,
Maybe he feels that you are rushing on him and he doesn't feel the same way therefor he wants to avoid being alone with you because he feels awkward but at the same time he enjoys spending time with you and maybe doesn't want you to take a step forward not to ruin the friendship you are building. I think you should act strictly as a friend with him , try bringing up the subject of other guys. For example if he says what are you doing this Saturday , tell him you're going for supper with this guy you know , make it sound like a date but talk very briefly about it ,not for it to sound you're trying to hard, you will get a reaction whether its indifference , or he changes in the way he behaves with you and if he does then it means he likes you , even if his reaction is withdrawal from you it means it affects him and that points to romantic feelings he has towards you. DON'T misunderstand this , don't start talking and bragging about other guys just slip a small comment like that and change subject to show , you're not showing off Okay? Let me know what happens with you!

Good Luck!