Search Results: hang out with me and not have sex

I met him half year ago from a dating app.
We hanged out Several times (mainly go hiking /dinner ) and then he confessed on our third date but I refused. I thought it’s too early , the feeling wasn’t there yet and he’s only 2years older than me I felt like I need someone mature and stable.

After refusing him , he started distance himself from me. I blocked him. ( I know it’s weird but that’s what I did to respond)

I unblocked him last month and we began to chat but yea it’s unlike how we used to.
Then recently we had sex.
The feeling has became stronger and I really like him.
I know I’m also not ready for a relationship (so does he) I’m okay to be his friends with benefits.
The thing is he distanced himself from me again.
The last time we met he told me it’s hard to like and love me because There’s so much responsibilities and things he can’t control (as I planned to work abroad after 1year ) he asked me why did I blocked him and how bad he felt when things didn’t go the way he thought.
he said he was optimistic half year ago when he first met me now he realized there’s a lot of things he can’t do because he’s poor he can’t offer. I told him not to worry too much I didn’t ask for anything , we’re ok to keep relationship like this. He said he doesn’t worth spending time on he said he’s not handsome , poor and so.

What really hurted me is he didn’t show up on my birthday as we promised. He said he has to work and even tried to act like he forgot my birthday.

I want him back but I feel like everything he did is pushing me away.
We stopped talking to each other for a week and I recently posted somethings that might make him thought I was seeing someone. I don’t want him to leave. What should I do?