I think I found the problem, when I go to level 5 there are patches of random sections that are subdivided. If I delete the patches which are selected in mudbox, the entire mesh gets deleted I don't know how the heck this happened as I didn't do anything unusual.
If anyone has any recommendations on how to fix besides retopping which I'm going to go ahead with and hopefully it fixes both problems in one shot, first I'm going to sculpt some more detail 🙂