This is obliviously a reoccurring issue over the years. I've had it for a few years now, but just ignored it, and just dealt with it. Now, it seemingly has got worse with any AutoCAD 2022 product. That being paperSpace viewports being sluggish when touching them, pan, stretch, etc. all are set to wireframe for best performance.
I had Printer Support Paths set to a Network location that is a little slow connecting. When I set these to a local Path the problem went away. Doesn't help with Group Standards at all. But now I just sync all the support files from server to local and Path to the Local storage. seems to work better.
Somehow PaperSpace is constantly reading you Page Setup when refreshing.
Go to Page Setup Manager on the Layout Tab. Uncheck "Display Plot Styles". This stops AutoCad from checking the .ctb /.stb files for settings when viewing PS. I did this to keep my network Pathed support files location.