Most Helpful Girl
Try to pick and choose your battles more wisely. Don't complain to him about every worry you have, whether it be in your own life or regarding him. Try to keep it to yourself unless it becomes a recurring issue.
One thing I learned when in a similar situation where it got to me complaining about petty things than anything else with my boyfriend was also that, when bringing up something you'er not happy about with him, try to make it more constructive than negative. So rather than just explaining in detail what you think is wrong, tell him why you feel like that. Like in your case, you can tell him you feel sidelined by everyone and essentially alone, so as a result you love the security and outlet for frustration he provides.
Hopefully once he understands your reasoning behind it, he'll be more tolerant of it.