(edit, as somehow I wound up with a completely blank post)
Try deleting it from a DOS prompt? Make sure you right click and select "run as administrator" when you start the DOS prompt.
You may have deleted the IPJ short cut, but did you delete the actual IPJ file it'self?
I had indeed deleted the actual IPJ file.
I had another user in my dept (same level of full permissions as me) try to delete the workspace (folder structure) and he was able to.
Even though it was created through IV on my system and I was the owner, IV must have put some bizzare permission on it that exclusively blocked me from doing anything with it.
I am not even going to try and contemplate why.
Thanks for the replies nonetheless