Most Helpful Guy
Logically speaking it's not worth it. Not at all.
Let me tell you that you are in a very toxic relationship from both ends, it's fully of negativity and it's very unhealthy, this type of relationship is not going to do you any good. Plus your children are also growing up in this environment which is not good either. You said that you don't want to lose him, you don't want to lose his family and you don't want to lose your kids either.
Well that's not going to be possible, if you care about your happiness, if you want things to be right. If you care about your children's future and well being, then this thinking of yours is not going to help you at all. You can't have your partner, his family and your kids as well. Yes, I know if you can have all then that would be the best situation.
However reality isn't always like that. Life is not so simple. Sometimes, or many times in life you will have to take some tough decisions.
You need to take a decision as to what you want to do.
1. You should be ready to lose him, lose his family and maybe your kids also and then you will be alone.
2. You can continue being in this unhealthy relationship with his family and your kids.
However just think of this what will happen if your kids grow up in such an unhealthy environment? ask yourself that question. Hence you will have to decide one way or the other, you can't have both.
I think one of the reasons he wants to take the kids away is because he believes that you will be a bad influence, a bad example to the kids. You yourself accept that you have been a terrible partner and the things you have done in your relationship. Hence I would say he has a point there.
Yes, you do seem confused and you don't know what you want, hence you need to make up your mind.
That's all I can say.
Most Helpful Girl
This is a toxic environment for the kids to grow up in, and his threats are just that - THREATS. If he cannot prove that you are an unfit mother, there is no court that will grant him full custody and ban you from seeing them. The default in custody battles is to give joint custody to the parents unless one doesn't want it or there is solid evidence they are negligent or abusive or unable to provide for the children.
The fact that he would actually make that type of threat tells me more than enough about what kind of person he is to say this is not a healthy relationship or a healthy home and all of you - kids included - will be better of with it ending.