Pretty sure it means he's just not that into you. Any guy who was even the tiniest bit into you would never hang up on you. Or think that you are annoying. Think about why he thinks you are annoying. Do you have Facebook? If so add the Honesty Box app and ask the question "Do you think I'm annoying?" and see what happens. Just a suggestion.