Maybe he likes being married for selfish reasons that have nothing to do with you. It could be he likes the fact that he has a wife, he doesn't want to live alone, he's too lazy to move out - there are dozens of reasons that he might want to stay married, but none of them have anything to do with loving you.
I think you should leave. I know that's drastic, but after being with someone for 20 years - you would think that he would have a little more compassion towards you by being affectionate because it's something that you need in your relationship. I'm aware that men express themselves differently, and the key to a happy marriage is compromise and acceptance. But if he shows no interest in you (strike one) and cheated on you (strike two, three, four, five) what's the point in staying married?
I know we all just want to be loved - but a pseudo marriage can't even come close to love. It's just going to tear you down. Leave him so you can have an opportunity to find some happiness in your life - even if it's just from yourself. I know more than anyone it's easier said than done - I've overstayed in an abusive relationship myself. I just hope you have to courage to take the first step in saving yourself.