This has happened to me, and even when I do not look back at them. Why?
Most Helpful Opinion
Probably because you're attractive, but they don't want to look like they're staring.
They look at you, then when you look, they break the glance so as to not seem creepy by staring or something, then when you look (or walk) away, they look back again because they still want to check you out.
I am very guilty of this.