Why do women not like to be called ma’am?

if i’m addressing women that are strangers in public i usually try to say “ms.” but if i say ma’am to a lady under 50 years old I sometimes get an upset or annoyed look back from them. Is it really that big of deal?
Why do women not like to be called ma’am?
Post Opinion