Am I thinking too far in it?

My boyfriend has a telloym and someone asked if he missed his ex he said, "I dont know if i necessarily miss her, or if I miss some of the banter. At the end of the day, she made her choice and I made mine"****

He told me he was over her, and they dont talk. So that was a good sign. He doesn't talk ill about her and they were together for 3 years.
Im not looking for someone perfect. I broke it off with him for a few days after reading this and my friend reached out to him and talked to him and he said it was a misunderstanding. And said that he would like to work it out with me to her she told him to. So he messaged me saying we needed to talk.
We are back together, but in the back of my head it's always gonna be there. What if he ex came back? Would he leave me for her? Probably?
We don't really have any banter either, I've tried with him but he doesn't give it back so I stopped doing it.

I dont understand why you would pursue another relationship? That puts me in a sticky spot. What if she were to come back? Would he take her back? Would he talk to her behind my back? All concerning questions that caused unnecessary worry.

His friend on the game we play brought her up once, he was drunk and said she's a fun person. And it kinda makes me feel some type of way cause I'm not a fun person. Im actually boring. And they know that.

I do really like him, and he's a very good man, but maybe I'm being too hard on it? He told me he was over her and said that it's hard for him to love on when people bring her up.

I am at a loss for words here. Am I thinking hard into it?
Am I thinking too far in it?
Post Opinion