Why are special education students so "looked down" upon by some students and teachers?

I work in a high school as a custodian. We have one special needs kid who the football players act all "buddy" with, but when he has his back turned they'd talk about him and mock him because he is special needs. Is it just that people are that mean, or is it more group mentality thinking? Because many special Education and IEP students are actually brilliant and capable.
Why are special education students so "looked down" upon by some students and teachers?
Post Opinion