Guy friend found out I wouldn't have sex with him. Is this why he's become so distant?

This guy and I used to work together for years & it evolved into a friendship that's now lasted 3.5 years. He would often ask me to hang out, run errands, grab lunch at work & he would reach out to me almost every single day. It was effortless enjoyment & lots of laughs/jokes. I could make weird noises & ridiculous faces around him and he'd like the humor. We were very close, yet he is married (even b4 meeting me)-- No, we're not having an affair, we are just friends. Never kissed, nothing.
(FYI, because his wife isn't very sexual, he finds other women to satisfy him so that he doesn't grow resentment for his wife.) I think he knows I have feelings for him. ( I will never act on them though. )

After 3.5 years of friendship, out of nowhere, he'll pulled away from me. I now only hear from him every few weeks. This has been going on for the past 3 months. Two weeks before he started being distant, he asked if I would ever have sex with him if he was single. (He's really confident, so he doesn't care to be blunt). I said that although I do like him, I would not, because I wouldn't want to ruin our friendship.
--When I asked why he's been distant, he claims his wife has nothing to do with this & instead, gave two excuses as to why he's been distant. One was: "Being busy." 2nd: He "gets blue balls" around me.

Does he not value our friendship anymore? What's happened? How could someone who's enjoyed your friendship everyday for years & years suddenly be so distant all of a sudden? Was this friendship nothing to him all along?
I dont want to lose my friend. Please no unhelpful answers like "Why are you talking to a married man?" I don't need that. I need actual, helpful and valuable responses that actually answer the question.]

Thanks guys!
Guy friend found out I wouldn't have sex with him. Is this why he's become so distant?
Add Opinion