Most Helpful Guy
I know how you feel, but this may not be your fault. He may be right that you two are not compatible. Generally, I think one of the most important things in a relationship is being able to understand each other. He probably feels that you don't understand him, but it's also clear that he doesn't understand you. You might be better off with a guy who can understand your shyness.
That said, you are also right that he is throwing in the towel too soon. After the initial chemistry, building a relationship takes time. It's very rare for two people to be perfectly compatible when they first meet, even if they have great chemistry, and so it takes time to get to know each other.
He sounds kind of impatient and immature.
That in itself is also a red flag for you to consider. You can try to change whatever in yourself, but if he can't also change to become more patient, your relationship will have problems.
So basically, he sees red flags. And if you are looking at this correctly, you should also see red flags. Do those red flags mean to abandon ship? Not necessarily. There are some red flags where you would abandon ship early on, like strong differences in morals, and lifestyle differences with drugs, alcohol, sex, etc.
But other red flags are worth the risk in the short-term, so long as you are both willing to concede that the relationship may not work in the long run. He is throwing in the towel now because he is either afraid of getting hurt, hurting you, or because he was never that into you to begin with.
But keep in mind that it is a guy's responsibility to pursue a girl, and he knows this. For that reason alone, if I were you, I would stand firm on the fact that you've only been dating a few weeks. Thus it is unreasonable for him to expect you to completely bare your soul and become so intimate with him so quickly. If he is nt just looking for an excuse to break up with you, he should concede that he was moving too quickly.
Most Helpful Girl
It would take a lot from your part to change :) but its possible