Annotative Text works like this: you set the text style to be annotative, you set the final printed-on-paper size only, nothing else, and you let the program scale based on your annotative scale settings or viewport scale settings.

So it appears you don't know about CANNOSCALE command in modelspace https://knowledge.autodesk.com/support/autocad-for-mac/learn-explore/caas/CloudHelp/cloudhelp/2018/E...

If you want to jump around between model and paperspace, you have to set modelspace to be the same scale as the viewport in paperspace.

It's better if you get in the habit of only working through your viewport in paperspace to avoid jumping around, but that's up to you.

Now you can reset your annotative text size back to 5mm.