Most Helpful Girl
You were right to say no and do not feel guilty. He didn't say anything about talking outside of work in the meantime so I would assume he has his eyes on another girl. Gaining people's trust back means that he is either a liar, a gambler, or irresponsible with his promises/duties ( which would be bad for you if you became his girlfriend). Having a lot of family issues does not take up all of his free time. He probably still spends a few hours a week watching his favorite shows or out with friends since you did not mentions the extra strains from going to school in addition to working. It's unlikely that he would turn down being with you because he's probably still at the age where sex is extremely important for a guy. Therefore, it's likely that he trying to get another girl while having you as a back-up and he's the type to look for the next best thing.
Hopefully, you can learn from my situation. I dated a guy who was happy to use his (real) busy life as a reason to 'take a break' (although he meant break up) . He just felt bad because he was my first and I was his fifth (he's 29). I convinced him to stay in the relationship, which you'll see why it was bad to do so in the 4th paragraph.
Later on after we got back together, I brought up theoretical situations, and I got him to admit that he would never try to break up with 'The One' over a busy life and financial stress. At that moment, he didn't realize that he had just confessed that I am only around until he sees the next best thing.
If he still has a lot of "stressors" if you contact him again, please do NOT ask to still be in a relationship like I did. I convinced him to go back into a relationship by saying that I would make most of the effort by changing my schedule to fit around his. I also mentioned that I always have brought food over (and would continue to do so after taking me back) so it would lessen his financial and time strains. After getting back together, I was more explicit in my affections. He got used to that and ended up not trying as hard to make me feel loved until the times when he felt too guilty. However he was very consistent in his affection(more than to me) towards his roommate's cute cat. This means that there is room in heart to love an actual girl more than me.
Please do not get back with him because he will ask for another "break-up" when things get rough to get what he wants out of you (or keep you in line like in my case) or if he sees an opportunity with another girl.