Should RPG "Weapon Swapping" be penalized to avoid a character being "perfect at everything"? What do you think?

So right now, I think I'll limit weapon swapping to only the Short Bow and only swords as big as Long Sword or smaller. They could also allow Weapon Swapping between Short Bow and Quarterstaff, but not stronger staves. Characters would also need to do a Dexterity Check against an average difficulty +2 (still needs to be calculated by me,) plus any penalties from circumstance. So for a 3 second round, if you want to sheath your sword and unsling your short bow, you'd need to do a Dexterity Check to see whether you character can accomplish that in 3 seconds or less.

What do you think about this idea? This would prevent a Ranger or Fighter from being "Good at everything".

Should RPG "Weapon Swapping" be penalized to avoid a character being "perfect at everything"? What do you think?
Post Opinion