I want to be closer to my siblings, but their group of friends are toxic. I'm struggling with setting boundaries?

I have two brothers who are close in age. As individuals, I love them so much, but their social lives make it so difficult to keep a close relationship. They both were very popular In High School and hung out with super toxic people that treated others terribly and gossiped. 10 years later, somehow that same group of people has managed to stay together. They do literally everything together. My brothers will try to include me when they plan cookouts, dinners, trips, and going out. I realize they do it to be kind, and I greatly appreciate it, but I honestly can't be around their friends anymore because I'm tired of how toxic they are. They've been rude to me and my friends several times to the point where i feel harassed honestly. I've tried to talk to my brothers about it, but they get super defensive and protective over their friends because they are like "family" to them and they say I'm just being dramatic and need to give them another chance. I don't know what to do anymore, and I've started to distance myself from them and turn down invitations to things. All I wanted was to be close to them but it's hard. Does anybody have any advice? Am I crazy for being upset over this?
I want to be closer to my siblings, but their group of friends are toxic. I'm struggling with setting boundaries?
Post Opinion