Most Helpful Guy
If there was an agreement to pay it back, then yes, pay him back. Otherwise, he really doesn't have much of a legal standing to collect it.
I chalk up any money that was exchanged between me and previous GFs as money lost. Unrecoverable. It is what it is, take that to the next relationship and learn from it.