Even if they know the word it still makes them sound childish. It might just be me, but to just use a random word that most people won't even understand doesn't seem like something you should do unless everyone gets it. Why can't they just say "cute" instead. It sounds better and everyone understands it.
Here's the thing though. People do understand it depending on who you are, where you are, and what you're into. There are people out there from other countries who don't know what the word "cute" is. There's also "borrowed" English words in other countries that people say, but wouldn't understand. As for it sounding childish the word "kawaii" is normally used to describe a more innocent doll-like almost over the top cute. So using the "normal" version of cute holds no meaning to some because you can slap it on anything, even a porn star with a more sexy mature look. I personally like kawaii better when it comes to describing things that are more innocently cute and not over sexualized. The word also doesn't sound better to quite a few people out there, if it did they'd use "cute" more than they would "kawaii".
I don't know how it is over where you live but I've never heard of people actually use "kawaii" in any context outside of using it as a fun word to sounds a bit more silly with. The thing is also that yes there are words that are borrowed from other languages, but from my understanding that is because they don't have any word for it in their own language and have been influenced by the internet to use the English words because you know English being the main language used across the whole world. The word "kawaii" has yet to actually be used outside of those somehow related to anything Japanese like anime or manga so even if you say that it is used to describe something doll-like, that's only inside that circle. Though I'm sure it will be used more and more with the recent popularity in using Japanese for the reason of only sounding silly and special. The word originally doesn't have only that use, so how it's used right now is only in the western countries.
I just think it is annoying that people start to use a word because of it's "silly" sound or whatever only to say it has a different meaning. People should in my opinion use it if they had a better idea of what it meant and not because they heard someone else say it.
I'm trying to speak for people who actually know what is means, who use it in context, and not because it sounds different or silly. Like what I said before about using it to describe something more cute than just plan old "cute", or maybe something over the top cute with an exaggeration of an innocent or doll-like look.
But if you're speaking for those who understand then shouldn't you understand that it should also only be used in situations where everyone understands it completely? Otherwise it's just silly and might people might as well have used "cute". Also the original word has something more closely related to cute than what you're describing, so why should it be used as something else?
Most Helpful Opinions