We have scale and filter to hide view reference. I understand the point, but to most of architecture firm I worked for, reference are only showed in a single view.
So we dont have to manage their visibility in other views.
I would suggest to have reference visible only where it is created and by default hidden elsewhere.