I think my boyfriend's annoyed with me. Help!!!

Alright, even though my boyfriend is older than me (by a year and plus he's in 9th grade) he's never had a girlfriend before me and I was his first kiss. But on the other hand, I've had a lot of boyfriends before him.

Thing is...A lot of my ex-boyfriends and I are still good friends. So whenever one of my exes comes to hang out with me, my boyfriend goes quiet or says "I'll see you later" and then walks off. I think he's annoyed with the fact that I hang out with my exes and if he is; he's a shy guy so he doesn't really express much feelings.

(I don't like my exes anymore). But I still want to hang out with them because they're still good friends. Plus, I used to have a girlfriend (I'm bisexual) and I think he thinks he has to compete with her to prove he's a better partner. Someone help me! I don't wanna' stop hanging out with my friends!