A: Two things: 1) You've only been on one date so don't harass the hell outta him or you'll push him away guaranteed. 2) What do you mean 'stupid you' for ending the night early?? You'll have to explain that a bit further before I can give you advice there....
A: To a large degree I agree with the people above, however I have two girlfriends (not literally) that I have known for over 20 years that I might say that to. Sit down and ask him about it, but be prepared for the "I can't believe you didn't trust me" blah blah blah.