If you now or if you have ever had hiring and/or firing authority, what is or was the worst part about it?

It is hard to fire someone. 

We had to close one of our locations once. The worker understood, she seen the sales there daily as well. So she said she knew it had to be coming. 

The only other was just a temporary fill in worker. I figured out she was skimming money. So we just do not call her anymore.  She has never put me down as a reference so I guess she knows why.

I never "officially" had the authority, but have often been asked for input since they would be a coworker.

I hate the new(?) trend of having a group of potential coworkers interview the candidate.
It seems to me that means the manager is too lazy or indecisive to make the decision themselves.  And that is part of their job.
I already don't have enough time to do all my job.  I don't have a "spare" hour to do what is the manager's job, as well.

I only fired one person. It was easy...the punk didn't want to work anyway. In the navy, we could only recommend transfer to another shop if he was deemed incompetent.