By nested do you mean "Bound", "Inserted", "Attached", or "Overlayed" ?
We have sheet files with attached xrefs inside xrefs. Its causing problems when we are trying to bind. I'm wondering if there is a way to programmatically step through each sheet file detach all xrefs and reinsert them unnested
You would need to modify *all* of the nested XREFs to either remove those XREFs or covert them to overlay first. But you might want to generate a list of loaded XREFs in the top-most drawing first before you get there. By the time you get all this done it might have been faster to manually update the source XREFs.
I like to keep a tight ship when it comes to x-refs, so you can preset the XREFTYPE variable to 1 on load. This sets overlay as default, but if someone changes it to attachement during xref creation, then it will set it back to 0. That's why it's important to keep it as a "Load on start". If you don't know how to do this (I don't but it's got something to do with acaddoc.lsp), someone here should be able to help. This variable is stored in the users profile in the registry and in the .ARG file.
As for changing the existing x-res you can use the CLASSICXREF command and double click the x-ref to change it from attachment to overlay and vice-versa.
Here is what you really need for updating existing DWG files. Look under your program folder for the REFERENCE MANAGER.
You will be able to edit the type of xref in a file from here.