Am I wrong for not wanting my family at my wedding?

My fiancee and I are getting married next year. Before we even got engaged I made it clear I didn't want my family at my wedding cause they ruined my first one for me. My mom and my sisters are very self centered especially my mother. My first wedding I asked my mom to French braid my hair-she didn't feel like it and put it up in a half pony tail. I bought matching earrings for all of us to wear, they didn't wear them. I gave my mom her mother of the bride corsage she didn't wear it. At the reception my sister was talking over me and yelling at me to ' calm down' and I got cut off by them most of the reception. When my fiancee proposed my mom didn't seem excited and her and my sister asked why I would still want to get married. The original plan was my fiancee and I were just going to elope and find a pastor to marry us and go on a trip. Today my fiancee now mentioned he wants our families there and to get married at a restaurant. I made it clear I didn't like that idea because I don't want my mom and sisters ruining it. He got upset and said ' fine forget I said anything ' now I feel like an ass because now he isn't getting what he wants. But I felt I made it very clear my stance on having my family there. Am I in the wrong tho for not wanting my second wedding ruined? How can we compromise to where we are both happy?

Am I wrong for not wanting my family at my wedding?
Post Opinion