Not sure what's going on

I have known and worked with guy for about 4 years now. In the past 2 years we have gotten super close. He dated my ex best friend. I always had to be there when they went out I thought it was weird anyway. We have hooked up a few times. We have talked about us being together in the past couple months, I asked him if he ever saw us together and he said yes. But he has at that time he didn't want a relationship with anyone right now cause he;s not ready to settle down. I asked him if I ever made him like that and he said no. Everyone that knows us and meets us think we are together but weren't. We just started looking for apartments. His family loves me and I love them. I just don't know what to do and I'm sure what to do ..