Why do we both feel like we have to keep a are friendship secret just because he's a man and Im a woman?

I have a male friend at work and over time we've become really close, I wouldn't say like brother and sister close but I'd say besties who can drink a bottle of wine together and talk shit all night close.

We are both straight and are I'm separate relationships. We've suggested getting are partners to come out with us too but they just don't gel well and there work lifes don't match up

His girlfriend is very much a gamer girl, loves reading and is super quiet and shy and my boyfriend is a mans man if you get my drift.

His girlfriend works away sometimes and my boyfriend works nights. I don't like being in the house by myself especially as I newish to the area and I'm a young woman in a big city in a house by herself it can be scary and lonely.

He suggested he come over and keep me company when my boyfriend is working nights. My boyfriend is totally fine with him coming over and actually thinks it's really nice of him. It's just his girlfriend can't know and my neighbours love to gossip.

We pretty much drink wine, talk shit, watch TV and order take out. We have a fun night together and that's that.

It's just his other gets jelouse so she can't know he keeps telling he's visiting his mum and my next door neighbor told my boyfriend that she thinks I'm having a affair as there's always a young man over at night. He's explained to her but that's not what's going on but she's massive Karen and gives me dirty looks in the street so now I have to sneak him in from the back garden but she still always watching me and sneaking around makes us both feel as if we are the bad guys.

Updates
1 y
We've had the talk to about sex and what not and he's amitted but he's a man and that of course he's attracted to me but he's attracted to most women doesn't mean he's gonna jump into bed with them and he respects his girlfriend and me to much to do something like that.
Why do we both feel like we have to keep a are friendship secret just because he's a man and Im a woman?
Post Opinion