How to stop worrying that I'm infertile?

I have a wife and child. Recently I've been having irrational fears that I'm actually infertile. I know I could go and get tested and "be done with it," but doing so would show outright that I don't trust my wife and risk ruining our relationship if it turns out I really can't trust her. Plus I know my brain and even if the test says I'm fertile I could still have lingering suspicions about the baby. Getting tested would NOT fix the problem, in fact it would probably just make it worse.
How to stop worrying that Im infertile?
I love her and our baby and there is NO reason for me to have this fear except my fucking brother putting the idea in my head this past Easter. My wife and I have talked about it and she said she understands if I do get tested but hopes I don't feel like I need to. She's never given me reasons to doubt her.

I feel horrible I can't get the thoughts out of my head and they just keep getting more intrusive since the baby has been born. Even if it wasn't mine I don't think I could leave them. Getting tested would do absolutely nothing but hurt. I need to stop thinking about it I just don't know how.
How to stop worrying that I'm infertile?
Post Opinion