What is it about being a "weird guy" that makes guys want to test you or fight you?

I'm a very serious/apathetic type of guy but I'm not the most attractive

That leads to a lot of people feeling uncomfortable around me but specifically guys who want to passively "test" me

They drop the act when they realize I had SOME boxing experience & that I can "Rep Out" in many exercises

From then on I'm just ignored

What is it about being a "weird guy" that makes guys want to test you or fight you?
Post Opinion