Religion & Spirituality

Am I A Jerk For Not Excepting The Invite To Church?

So, I live in this little town and close to our house is a church. The minister is someone who has done a while lot for me and my family (he helped up get by. Bought the kids school clothes, helped pay for my drivers education a while back) and he is a very religious man. Every time I see him he remind me that church is this day or that day, but I never really go. I usually say things like, "we'll see what happens " or "if ik not busy." But I never really go.

I am not really a relious person and church makes me uncomfortable including things that had happened before. (I used to go to church before I moved where I am now, and this was years ago. I would get bullied at church and beat when followed home. One of the times they recorded me while going at it and posted it online) I just am acared something would happen on my say to or from church, and I also don't like being around many people and a lot of the people are rude at this church. (I went once or twice when I first moved here). I'm just kinda stuck and don't know what to do. Am I being rude for bit going? And if not, how do I explain to him that I don't want to go, but without being rude, having a long story or lying?
Am I A Jerk For Not Excepting The Invite To Church?
Post Opinion