How can I ask the guy out without making it sound too much like a date?

Phone call best way to do it. I recommend though that you try to find out what his schedule is, that way you will increase your chances of success. . For example, you mentioned he was late to class; does he have a class after that? And are you also free after his class? Lets assume that he has no class after that class he was late to. So matter of fact you ask, 'what time is your class over? ' Whatever he responds, you reply with, "perfect. I"m gonna be going to get some coffee/tea/snack/whatever at around that time. Why don't you join me. '
By the way, keep in mind that if you just say "we should hang out" and this guy is not confident, he will most likely say, 'yea, we should", but he is not going to say, "we should, when are you free to hang out? " That's why if you're going to say that, you might want to also specify a time. Example, 'we should hang out--are you free to hang out this weekend? "
Or you can put some light humor and say, we should hang out; I can make some time for you this weekend----what bout you, mr? Can you make some time for me? And on and on. :)

Get some gift certificates to a movie, or concert tickets and tell him you got them, but have no one to go with you.

Prime for one-on-one casual time together. He'll think he's doing you a favor and might actually realize at the end of the night he likes to hang out alone with you

I would say a phone call would be better just tell him that you been wanting to see a movie lately and if he wold like to go then after the movie maybe y'all could have dinner and just converse and that's where you actually get to know him better.

Just say "Hey we should hang out. " Do it on the phone or in person when the conversation is going well.