Hi Drew,
Inactive Groups are automatically cleared when the file is closed, so they only hang around during your current session. Once you open the file back up, you should see no Inactive Groups listed until you start making edits that result in deleted Groups again.
These primarily assist our Undo system, but since you can't do much with the list of Inactive Groups, we may eliminate this list in the future and display only Active Groups.
Regards,