He messed me around?

I deleted a man I had been seeing on a casual basis he done done thing nasty to me so I unfriended him but week later he contacted me again and we met up chatted and I stayed the night. next day he was cold towards me but I thought nothing of it I went home messaged him and readded him to Facebook he accepted days later and but I got no message from him so I decided to contact him and he completely ignored my message about how unwell I was so I got the hint and deleted him again and didn't contact him. On Friday after a month of silence from him I got a message from him asking "have you fell out with me" I replied no he then messaged again with "deleting me, adding me, deleting me again" I apologised and explained why but he mentioned it again in our chat. Next day I met him and he brought it up again and I told him again why, we went to bed and next morning he went cold on me. I went home and messaged him next day saying I really enjoyed it and he replied back "me too"

I made a post on Facebook and he messaged with "I hope your not talking about me" I said I haven't said anything about you and he said "just kidding lol x" I've just blocked him on Facebook and Snapchat as I can't be bothered with that paranoid nonsense anymore for something that's supposed to be about fun I'm not having fun because when he goes silent on me it crushes my self esteem. Casual or not it's the disrespect I get from him as well.

Am I right to block him so I don't get messaged again or am I being harsh?

He messed me around?
Post Opinion