Who Sets the Tone?

You do.
If the conversation’s tone isn’t to your liking, it is because you have forgotten that you set the tone.
When resetting the tone of a conversation, the first response you will most likely get is more of the old tone, or an outright challenge to your changing the tone (‘what made you suddenly so high and mighty?’). This is a test. The other person is testing you to see if you really mean it. They want to know if it is ‘safe’. Will you stand your ground with the new tone in the face of the safeness test? Because being mean feels a lot more safe than being vulnerable, and the other party won’t let down their guard in return until they know you mean it. So you may be in for a challenge to ride out the test.