That's not too much distance for sound to travel quickly and clearly, so there's no problem there. I suspect that if you're having sex, you do it while lying down, so once again no problem. I suggest you tell your friends to shut up, preferably while standing on a chair to illustrate the irrelevance of height difference.
the height difference is quite a big one but honestly it all depends on you guys' relationship. if you love each other enough you will make it work - just like they say, "love always finds a way.' and as for what your friends say, who cares? its not their relationship. its between you and the guy, so do your thing and do what you want. If he makes you happy what else matters?