Welcome to the community.
This problem could be caused by a couple of things.
1. You are running out of heap space in Windows. You can correct this by using the Fix it option in this Microsoft KB article:
2. You are running into a video driver anamoly. What you might try doing is upgrading your video card driver to a WHQL certified driver as a clean install and reboot. Programs like Maya are graphic intensive, so you need to have a stable video driver. Never run a Microsoft compatible or factory installed driver.