For one, there isn't necessarily an indication that they're interested in you. Most likely, they are just being observant of their surroundings. I find myself at least glancing at people all the time (and believe me, it isn't because 60 year old men are attractive). Alternatively, they could just be checking you out. Maybe they like the shirt you're wearing. Maybe they noticed a feature of your body that they liked (or didn't like, which is also possible). Maybe you had toilet paper on your shoe. Maybe he does think you're super hot, but his girlfriend would flip out if she saw him checking you out.
In short, somebody looking at you doesn't mean they think you're super hot and want to date you immediately. Most of the time, just looking at someone doesn't mean they need to initiate conversation.