Hi;
Ok now i understand your problem.
As a rule of refraction in 3D world, your object which has refractive material (your glass here, in this case) is processing refraction with the illumination of your native background when you hit your render button. Which leads us to: it actually is not the background you see there, it's the illumination that your background gives to your refractive object to calculate its refraction, even if you try to mask it with alpha channels, and / or render elements, it will be visible so what you are getting is perfectly normal.
Please do the following:
-Under the Refraction section of your glass material, change Affect Channels: Color Only to Color + Alpha
-You can do this for Reflection too depending on your scene setup.
And you will be able to remove / blend your refraction as an alpha channel in Photoshop or similar, just save this as another bitmap image (jpg, png, tiff or any format you want, and use it as a blending layer on your final render.
Hope this solves it for you.
Regards;