I guess I will try to tackle the "I don't care" thing first. I'm the same way as you, I really don't care what I'm doing most of the time. Sometime I would try to find the time to explain that to him. Probably NOT at a time when he is already annoyed with you for saying "I dunno". Try talking about it at a time when you are already together or on the phone or whatever. Just tell him you know it frustrates him and you're not trying to be difficult but that most of the time you really just don't care about what you are doing because all that matters to you is being together. Hopefully that will reassure him a little that you do want to be with him and be together, etc.
My husband (and some of my friends) get REALLY annoyed at the "I don't care" response too. The main reason is because then they feel like they have to do ALL the planning ALL the time. All the success/failure of a date together or whatever, lies with them. Nobody likes that kind of pressure. So now usually what I will try to do, is come up with 3 or 4 options that I would enjoy and then let him make the final decision. He still doesn't like it but it's better than just "I dunno" because it narrows downs his options for him. When you give them nothing then they have to choose from EVERY possible variable and that can be daunting. So maybe you can try that also? Tell him "these are the sorts of things I like to do." so that he can be sure to pick something he is sure you like.
Plus with this boyfriend of yours, I'm sure he is probably looking at this a little differently. You're really busy and can't see him when he wants to half the time. Then when you can see him, you don't seem to care about it at all, you don't help plan what you'll be doing, etc. Rather than seeing it as "Oh she is really busy and when she finally makes time for me in her schedule she is so excited to see me that she doesn't care what we do together, she just wants to see me!" No no. He is probably thinking, "She never wants to see me. Even when we do get to see each other, she doesn't even care." People tend to take "I don't care" very negatively. Like the situation is so unimportant to you that you haven't even bothered to think about it. My understanding and my experience, is that people like you and me are fairly uncommon. Most people do care. So they also tend to think that we're not being honest when we say that. Like we do care and just won't tell them and then when they "screw up" by making the wrong choice then we'll be annoyed. Seriously I have been dealing with this for at least a decade longer than you and people in general seem to have "issues" with this, lol.
I also have problems with planning. I'm very spontaneous and usually can't decide what I want to do until the moment is upon me, lol. Which drives some people nuts too. Luckily my hubby is the same way on that one. If we go out, we usually decide that same day, just a couple hours beforehand, lol. Do you ever have that problem?