As long as you pay the taxes, is it really "bad" to make an early withdrawal from a rollover IRA?

If you have a SMALL amount of money in a rollover IRA from a previous employer (was a 401. K), is it really "bad" in the long run to just withdrawal that money? As long as you pay the taxes on it, does it really "hurt" anything if it is not a huge amount?

Updates
+1 y
And what percent of taxes should you go ahead and withhold upfront when making the withdrawal for your state and your federal? I'd want to withhold SOMETHING now to avoid paying more later. Should I just do the basic minimum they offer, or should I do more? I know obviously my overall income will be a factor as it always is for tax season. And yes, I do have another job I've been at for awhile now. Will I get a separate form from the IRA withdrawal in addition to my usual W2s next tax season?
As long as you pay the taxes, is it really "bad" to make an early withdrawal from a rollover IRA?
Post Opinion