He called me out on not looking him in the eye?

So, there's this guy...we used to be super sexual over text then we just kinda stopped...and so we happened to go to the same event...We talk a bit and he notices I struggle with keeping eye contact with him. He calls me out on it. Is it just me or is keeping eye contact with people who you were sexual with hard? or a person you like...idk I felt really bad. Should I ?
He called me out on not looking him in the eye?
Post Opinion