Sounds like a nice wish list feature....
Would be even more cool if it also worked on entities within C3D objects.
The only way I know > open up layer mangager, right-click then choose select all, turn off all layers then turn the xref layer back on.
ACADuser
Civil 3D 2018, Raster Design 2018
Windows 7 Enterprise
Dell Precision 5810 Workstation
Intel Xeon E5-1630 v3 @ 3.70GHz
32GB RAM, NVIDIA Quadro K2200 4 GB GDDR5
DUAL 27" Dell UltraSharp U2713HM