Most Helpful Girl
I would carry on with it for a while if you still like him, BUT keep your distance, and keep up your guard in case. This is very quick to have moved from one girl to the next and not to be harsh to you because I am sure he is with you for many a reason, but it is likely he has to filled the void (gap) his girlfriend has left and he has to do it quickly, because he isn't used to being on his own. Now your relationship could be great, however I know from experience, I started dating someone quickly who got out of a relationship 2 months before and at first it was great. He was charming, sweet, loving, romantic. 4 weeks past and BAM that all disappeared, he began to lose interest and started treating me like a friend, it ended painfully after a year of just dragging it on.
This may well not happen to you, but be careful, because this could really hurt you and you don't want to go through that pain.
Ask him outright if you have to, you have every right. Is he over his girlfriend completely, if he hesitates you know he isn't and you don't have to break up with him, but you can ask him to take it slowly.
I wish you well and I hope he is doing the best by you :)