Your commentary on my word selection notwithstanding, I had hoped to gain some insight on what to do about the 'infection'. I chose that word because of the way that we use XREFs around here. Even if the XREF is detached from the drawing, we are seeing the proxy object warnings, and partial open is disabled. If one drawing file has these objects in it, it will pass them on to any drawing that it is referenced into. Then that one will pass them on to other drawings, hence the 'infection'. I know how to get the AEC objects out, I'm just wondering why they stay there when the offending XREF is gone. I'm not sure if this 'infection' scenario is normal, either.
There is a performance hit, even on our CAD PCs. We use Dual-core Xeons, 4GB RAM, Quadro FX4600. I primarily see the change during file open and switching layouts. Cleaning up the AEC objects does help with these times, and cuts file sizes, sometimes dramatically.
I have noticed that the problem seems to be reduced by removing the object enablers. Some of our disciplines have requested that the OEs not be installed in order to combat this problem.
Perhaps your file sizes are so small that you consider the problem negligible? With very large files and large numbers of XREFs, this is more than an annoyance, it is a time-waster.