Why doesn't my boyfriend's brother like me?

My boyfriend and I have been dating for a year and a half...he is 3 years older than me, but one of his brothers Joey is in my grade (senior). We have a couple of mutual friends so I hear A LOT about him and I'm pretty sure he hears a lot about me. But I'm finding out that what he hears about me is either a lie or exaggerated to the point where it's basically a lie. And what's worse is that it's usually about me doing something behind my boyfriend's back. However I am completely honest with him and he doesn't believe what others say, but his brother does and even wants him to dump me. It seems like he takes any opportunity he gets to badmouth me. He is extremely judgmental particularly towards me even though he is the one guilty of going behind his "girlfriend's" back. I've tried to be nice, apologized for messing up, but obviously it's failed. I'm p*ssed and I don't know if I should let it go or talk to him...I don't trust him and sometimes he lies...but I feel like I should stand up. What is going on with him and what should I do?