Guys/Girls: Is this a bad sign?

My boyfriend says he loves me, but at times it seems like he isn't as into me as he could be. We started dating about a month after he and his ex broke up. They had dated for a year and a half. He has broken up with me briefly a few times throughout our relationship. But he has always come back within a week. I'm worried but I don't know what to think?