Was he cheating?

If you explicitly discussed having an open relationship, then no, I don't think he was cheating. I'm sure that he would have the good sense not to brag to you about the women he was seeing. However, if you never really talked about it, then yes, I think he was being less than honest with you.

Did you guys ever have a talk about it JUST being the two of you? I mean, 3 years is a long FWB relationship not to turn into anything serious.

One one hand yes. I think its wrong that he was sleeping with another woman and NOT telling you. On the other hand if you guys never had THAT talk about it just being you two then he is open to live his life and you live yours so you can't truly get to to mad at him.

Maybe a talk about defining what you guys are would help out to avoid any future problems

Let's clarify something----when you started the friends with benefits relationship, did you make it clear that you wanted an exclusive FWB? Was it made clear that you expected him not to see anybody else? Did you make it clear that you wanted him to tell you if he found another sexual partner to inform you? If you said any of these to him before you started the friends with benefits relationship, then sure, it was cheating as he wasn't being honest. However, if you didn't make any of that clear, then he was in all his rights to do what he did.
Something else---if you want something more than a friends with benefits from a guy, then stop having friends with benefits relationships, and concentrate on finding meaningful relationships. I have had some of these friends with benefits relationships, and every time the girl came out with. 'so where is this going. I thought there was something more. ' No darling. There is nothing more, but only a sexual relationship. Which part of friends with benefits don't you understand?

If he purposely tried to give you the impression that there was no one else, then you can reasonably call it cheating.

If he never did anything to give you that impression, then its tougher to make the argument that he cheated.

Well, it's not really cheating if you both discussed having an open relationship. You said "I would think that after 3 years that there was something more than just a FWB thing", but did you talk about that with him? My boyfriend now used to be just a FWB for about a year with me, but we discussed being exclusive with each other pretty often, then it somehow turned into a full-fledged loving relationship for over a year now (go figure haha). I don't think you should be asking if it's considered cheating, I think you should be asking where this guy saw the whole FWB thing going from the time it started, because once you get that answer from him, then you'll know if it really is more than just a FWB thing. You can't really call it cheating if you never discussed exclusivity at all. Maybe in your mind it was an exclusive thing, but if you didn't talk to him about it, then I'm sure he just assumed that it wasn't.