Ask yourself this: Can you keep doing this and being treated this way for the rest of your life? If the answer is an honest no, then get out of this. You don't have a boyfriend, you have a third parent!
My problem is this, you can't do anything about this because it's out of your hands. Your boyfriend needs to know that this is YOUR relationship too, and being treated like a child who needs to be babysat is not the kind of relationship you want to be in. So he either cuts the crap or you walk. I have been IN this situation and I have had friends in this situation and I can tell you that sometimes it means the guy is cheating or has cheating tendencies too and wants to fault-find anything he can in you to justify his actions, OR he is simply an insecure louse who has no respect for you or this relationship. (Myself, I had one of each situation. Both cases, the guys were insecure no matter what.)
Right now you are showing signs of enabling his problem. You avoid his friend and you let him go through your phone. Time to stop this. He is stunting your social life and treating you as if you are already guilty of something. He's trying so hard to prevent it that it's killing anything good you might have together. Ask him simply why he's with you if he's that suspicious. What is it about a cheating woman that he is trying to hang on to so hard for? And if he comes back with something like he's trying to prevent it from happening, tell him that YOUR business is to prevent it from happening, not his.
Honestly, some people should NOT get in to relationships if they can't figure this stuff out - but that's a whole other issue. My concern is your happiness because I really do feel for you.
I would tell him that things are going to change as of today and he isn't going to like it. But given that you have been accommodating in every way to enable his problem to continue, you're going to stop it cold turkey by putting a passcode on your phone, and as for your computer, if you must, reformat it and put a user password on it. Tell him that you are no longer going to live your life under the presumptions that make you out to be a cheating girlfriend and it stops today.
I'd even suggest some couples therapy since you two have invested four years together, it's somewhat like a marriage now. HE needs counselling, and you would be there as support. Ask him if he'll go. If he starts denying all of these suggestions, ask him if he's got better ones and see if you agree..