Was my reply rude or fair?

I just lost a chance with a possible good match. He texted back saying he's not going to call me again after the way I answered and it was a call wishing me to have a good afternoon. I replied ''Ok bye''. This is what happened:

I get a sudden call during my working hours, I don't even recognize the number and when I said ''hello'', he replied ''hey my love''. I replied ''Who's this, who's this I'm working''. In addition, I've been lovebombed before, had to discard guys that were after sex, had been harassed by a couple guys that wouldn't stop calling and saying sweet names and come from having bad luck.

He got upset I replied in that manner. I didn't even know it was him calling. Can you blame me? Was I really rude or fair?

Updates
+1 y
Luckily he's not the only one. I've already set up a date with someone else tomorrow. I'm trying all my options at this moment.
I got discarded for my rude answer but after getting lovebombed so much, getting harrassed and not recognizing the number can you blame me?
Plus I don't like getting called suddenly.. inform me at least. I was working.
Was my reply rude or fair?
Post Opinion