Do you think I should text him back?

Wow your both pretty messed up. So to answer your question an emphatic NO.

Okay...so... what is it that you want from this guy? Just sex? Cause it sounds like he wants a relationship and you just want sex. I'd say time to find a new FWB. And please be careful and use protection.

Oh give it up, any right minded idiot knows he is using you for sex, putting the blame on you for his own shallow behaviour (even though your both to blame) then acting like you aren't needed then he gets lonely, needs sex and crawls back to you to repeat the process.
Stop degrading yourself with that guy, if you want to be a free ride and happy that way then at least find some guy who is honest about it and doesn't mess you around.
This is the type of guy who cheats on his girl then blames the girl and the girl he cheats with, basically a finger pointer who can't take responsibility for anything, god help whoever he gets pregnant with because I guarandamntee he will be washing his hands of that too.
If you allow it and do the chasing then your just playing into it and you need as much a slap around the head as he does.

I know everyone is going to tell you he wants sex and he is using you for sex but that isn't what's happening. He wants the relationship and you want sex. Trust me. I've been there.

As a woman, we are told we should want relationships. And men are told they should only want sex. You two are flipped. He desperately wants a woman to love him and commit to him. You desperately want to avoid commitment at any cost. You want sex so you seek it out with him. You guys struggle so bad cause you both have the voices in your heads telling you that you should either do what all women do and settle down. Or in his case, do what men do and play the field and not get to worried about relationships. But you also have individual drives pushing you in the opposite way of how you "should" behave.

He uses sex as a tool, the way a woman would. He knows you want it but he really hates just having it without a relationship. I know that goes against EVERYTHING we are taught about men but there are men who truly act that way. He gets mad when you only come over for sex and don't want intimacy like cuddling or laying in bed all day. Sex ties his heart to women so he is hoping the same thing will happen to you. That if you guys had sex a few times you would stick around. Plus, he probably was worried you were sleeping with someone else and thought if you guys had sex a few times he might win you over. He doesn't want meanless sex. But he wants the relationship so he will have unattached sex to get it. Or at least he says he will. When it comes time for it, he is so afraid of getting hurt he will pull away.

I know you think this is crazy but it really happens.

Let me take a guess, he is a very romantic guy. Sweetest guy you ever met. Didn't know a man could treat a woman the way he wants to treat you. But there isn't that fire in him that other guys that just want you for sex have. He is kind of dull. A little boring. But there is just something about him.

That something about him is his desire to commit to a woman. You are attracted to it cause you don't have it.

I would say until you are ready to commit to him I would leave him be. He may find someone else. But if one day you realize your care for him is stronger than you realized and he is with no one, he will take you back. He is strongly attached to you. HOWEVER, you need to understand guys like him are to be held gently in your hands. You can't play the "I'm commitment wait no I'm not" game. They do not bounce back like other guys. He needs a very gentle woman cause he probably has some SERIOUS baggage that has left him the way he is. Don't give him anymore.