Most Helpful Guy
Part of it is that you are getting older, and have a desire to explore. Part of you sees him as tying you to your past, and you might be thinking that you are not quite ready yet to settle down into a perfect little life. Perhaps a little stifled? Well, that might be what's causing this. So what do you want? Think about it, long and hard, and then decide if it is fair to keep him around.
(It might be that it is, don't think that I am telling you to dump him. Just don't keep this pushing/pulling routine up it's hurting you both)