Does he not care about, or is it just because of things that he's going through?

Several months ago, I started a new job. This coworker and I hit it off, and generally get along great. Most of the time at work, it is just the two of us, and I love it. The thing is he really struggles with anxiety, and I think depression too. His good days at work are the best. We laugh, tease each other, he pulls pranks on me. He encourages me to come to stuff outside of work. Our bad days though, bring each other down. Because of some stuff from my past, I really struggle with people and feeling like they don't care. I know not everyone will like me, but what I cannot handle more than anything is when I don't know where I stand. If it's important to me, and he knows it, I get his full attention. He opens up to me sometimes and we have the best heart to hearts. But there are some days where I feel like he simply doesn't care beyond not wanting to hurt me. Always on his phone when we're talking. Not telling me bye. Lots of little things. And even on normal days, he still hasn't accepted my friend requests, and he doesn't ask how I am. He doesn't seem to overly value my opinions. He doesn't ask about the little things (i. e, likes/dislikes with colors food and stuff) but he definitely listens and remembers it. He parroted back something on Friday that I told a kid a month ago and didn't know he heard. We have the same favorite food, and a running joke is that he has me craving it from a month ago. It's starting to annoy him, but he still remembers it and joked about it the other day. We teach together, and I know our class is really, really stressing him out. I need to know how far to go with him though: how much does he care about me? Is he just extremely overwhelmed/ stressed/anxious, or does he simply not care beyond not wanting to hurt me? I can't explain why it's so important to me besides deep psychological stuff, but it is. Thanks!
Does he not care about, or is it just because of things that he's going through?
Post Opinion