We had sex too soon and now I think he's pulling away. How do I handle this?

Anonymous
Long story short, me and this guy I'd been speaking too online, slept together on the 2nd date. We spent the day together, driving around the county and visiting towns and places to eat, and then he treated me to an expensive meal in the evening which was far from where I lived. (By the way, I did offer to go halves for the meal and made sure drinks were paid equally). Anyway, after the meal it was quite late and there were no trains back to my town. I said I felt bad for him driving the hour journey back to mine, and so we ended up back at his (which was closer).

We were watching a film, and one thing lead to another and we slept together. Afterwards, it was the early hours and he even said do you want me to drive you home (I was a bit bemused why he asked this), to which I replied I can stay over if you want me too, and he said that would be nice. So I did. Anyway, he ended up sleeping away from me... not even spooning which made me feel a bit hurt, when he'd been affectionate before. In the morning, he got out of bed, and made me a cup of coffee.

Things didn't feel too awkward when he dropped me off today, and we kissed and he said we should do another day date again. He text me later when he got home, saying he was tired, and we swapped a few one-worded messages but that's it so far. I know in my gut that this may well be the last time I see him, but I don't know if that's just negative thinking.

I don't believe having sex early on damages a potential relationship, as my ex and I had sex on our third date and we were together for 5 years, but this scenario I'm in, is usually the common outcome.

Is there any way of saving this? Sensible opinions appreciated.
We had sex too soon and now I think he's pulling away. How do I handle this?
8 Opinion