Recently I started a new part time job in an area he also happens to work in. That area is very congested during rush hour and I don't want to fight traffic on my way home, so I started going to the gym near my workplace instead of my home gym. At our home gym, We started talking about how awful traffic is when we get off work, and I told him im going to they gym near my workplace, and casually told him that he's welcome to join me after work if he wants to. He said he prefers going to our home gym, so he doesn't want to go to that other gym, which is not a problem.
Now that I think about it, I'm wondering if it sounded like I unintentionally asked him out. Yes, I'm interested in him but I said it casually since we both work over there. I don't want him to think I asked him out and start acting weird with me. Thoughts?