Should he have to pay me back?

My boyfriend wants to borrow some money from me just $50. I agreed to pay him and told him he will have to pay me back and he got offended by me telling him that. I know $50 isn’t that much but im not comfortable giving him especially since he hasn’t done much for me.

Should he have to pay me back?
Post Opinion