When was the right time to tell my boyfriend I've never done it before and still want to remain a virgin?

We've been dating for 4 months at this moment but neither of us talked about past relationships. That conversation somehow never showed up and he's someone that applies ''the past is the past'' equally to both genders.

The times he wanted to make further moves, even though I would've wanted to, I rejected his advances. He believes I'm just taking some time. Well yes, that's part of the reason but not the real reason. I don't have a past relationship. I'm 22, just graduated from college and have never done it before. This might sound stupid but I just find that too special (I'm not religious though) to just give it up easily. What if we break up and I already gave it up and I can have it back ever again, and I end up regret it for years? I

I would like the relationship to continue but I still want to remain a virgin. I don't feel secure giving it up. Honestly, I would just like to give it up to one person. I know. A stupid fantasy that doesn't work anymore. I'm too late.

Updates
24 d
I would just like to have sex with only one man in my life.
Updates
22 d
We've had this discussion by now. This is how he feels and thinks about it:
While my feelings and views are valid, this was something I should've mentioned from the beginning because he's not too much into the ''waiting till marriage'' part but can make an exception for me. He will stay for now but can't guarantee 100%. If it's too much for him to handle, he will have to break up.
When was the right time to tell my boyfriend I've never done it before and still want to remain a virgin?
Post Opinion