Such a little problem, so much headache...
Ok, at least I've found a way to get rid of my own toolbars. Here is how:
First, go to Preferences.../General and uncheck Save UI Configuration on Exit (you never know...)
Now, make your changes of the ui. E.g, to remove a toolbar, open Customize User Interface, Toolbar tab. In the right list, select the toolbar name and click Delete...
Save your new configuration using Save Custom UI Scheme... under a new name.
Now, open the Workspaces menu (far right of the Menu row), select Manage Workspaces and save the active workspace as New with a different name.
Using Save Custom UI Scheme... again, save the ui as MaxStartUI.ui to
C:\Users\yournamehere\AppData\Local\Autodesk\3dsMax\2024 - 64bit\ENU\en-US\UI\
Quit and restart Max and activate Save UI Configuration on Exit.
Maybe one or more of these steps is optional - I don't know...
Now if someone could try removing the Arnold Toolbox that way, please. I don't want to loose it...