I don't think there's any way to know for certain.
There's guy in my class that keeps looking in my direction everyday.
It's bothersome because I can't tell if he's looking at me or something in my direction. Though somehow I have a feeling he really is looking at me and want to make eye contact with me but I won't do it.
OR a sly trick is to look at the clock or the ceiling and make a face like something super interesting or weird is there and see if the look. if they look at the ceiling then they were looking at you the whole time.