Why was he harping about me taking leave?

Yesterday my crush and I were to have dinner together, and I thought I'll just take a day's leave to have my hair cut to look good.

But when I went to find him early, he asked if I was working. I said I am and I took leave. He replied I am not working then.

Then he brought up how I just took leave no long ago on a Wednesday to visit him (it was at least 1-2 months ago).

And he continued to ask in a hinting way why I seem to be taking leave so often (in reality it was just 2 times in 6 months).

He harped on it until I exploded and told him that he just wants to say I'm a bad employee, and that he is judging me.

He laughed and said it's not his intention, but went on even after knowing I'm angry. What's his issue?

Why was he harping about me taking leave?
Post Opinion