Prior to the new three hitter minimum (2020), a relief pitcher had to complete one PA, or record one out, whichever came first, before he could be replaced.
The manager on offense always has had the last substitution opportunity.
The manager on defense waits until the pinch hitter has been announced, because the player isn't officially in the game until then.