Render Maya Viewport in Separate OpenGL Application

[MikeSSSS]

Hello,

I'm working on a C++ plugin where I would like to render what is currently in the Maya viewport in a separate OpenGL Window that has been created through the C++ plugin interface. I am reviewing this documentation (https://help.autodesk.com/view/MAYAUL/2016/ENU/?guid=__files_GUID_9B9CDD21_F3AD_4B79_B610_AAA42D4C65... , and I am wondering if it is a matter of just getting an MFrame and then maybe an MTexture and then I can draw that on a quad in my OpenGL application? Does anyone have any guidance on doing something similar?

Thanks for any help!

[MikeSSSS]

I did a little more research, and it looks like I can inherit from pretty much any class that has MDrawContext and then try to get the raw internal texture bytes from that. I think I still need a little more guidance however on if I could retrieve this information once a frame, or perhaps every time a user renders? I will try to post my findings to help others as well.

[zewt]

You can use MRenderer::addNotification with kEndRenderSemantic to get a callback at the end of rendering, which also gives you the MDrawContext.  That'll be called for all viewports, you can use MDrawContext::renderingDestination to check if it's the viewport (modelPanel) you're interested in.

You should be able to get the viewport contents with copyCurrentColorRenderTargetToTexture.  You'd need to lock the texture and read it out, it'd be hard to avoid reading the texture out and recreating the texture in the other application.

[MikeSSSS]

Hi this was very useful!

Is there a callback available for every time the user updates the viewport? I want to capture that, and then render that as a texture to my OpenGL window. I'm looking at hwRenderHelper.cpp to see if I can renderTextureToScreen, but instead redirect that MTexture to a pure OpenGL texture.

As it is now, I am getting sporadic crashes if I try to reclaim any memory (I suspect I can only do this in Maya).

I am going to try another experiment to see if I can just copy the raw pixels from MDrawContext and create an OpenGL texture. I imagine it to be slow, but perhaps my application will crash less if I own all of the memory?

Thanks again for your help!

[zewt]

kEndRenderSemantic should do that, it's called at the end of each viewport render.

copyCurrentColorRenderTargetToTexture will give you an MTexture, I think that's roughly the same as that renderTextureToTarget sample, but doing the low-level work for you.  You can get the underlying API texture from an MTexture with MTexture::resourceHandle.  I haven't used that with OpenGL, but I assume that just returns the texture ID cast to a pointer.

But if you want to render the texture in a different application, you need the pixel data, since I don't think there's any way to share a texture object between applications (at least in Windows with OpenGL).  MTexture::rawData will give you that.  It'll be a little slow (flushes the render to read back, transfer the texture to another application--shared memory would make that cheap--and updating a texture on the other side), but probably not too bad.

[MikeSSSS]

Got it, that's very helpful! I will proceed by trying to work with the raw pixels (and potentially the handle in OpenGL--which is just an integer) and try to make some progress.

I think you are correct that passing data will be slow. Perhaps the solution is to launch a window from Maya and draw a raw texture to that window? I will look into the devkit and see if there are such samples that avoid launching a separate application.

Thanks again for your help!

[MikeSSSS]

I still cannot for the life of me get a copy of the pixels.

I have tried running hwApiTextureTest and working from that, but I get an assertion (On Windows Visual Studio 2019) every time I try to delete targetData (heap_scalar.cpp line 904 is the aserrtion that fails).

	if (targetData != NULL)
	{
		MHWRender::MTextureDescription textureDesc;
		textureDesc.fWidth = targetDesc.width();
		textureDesc.fHeight = targetDesc.height();
		textureDesc.fDepth = 1;
		textureDesc.fBytesPerRow = rowPitch;
		textureDesc.fBytesPerSlice = slicePitch;
		textureDesc.fMipmaps = 1;
		textureDesc.fArraySlices = targetDesc.arraySliceCount();
		textureDesc.fFormat = targetDesc.rasterFormat();
		textureDesc.fTextureType = MHWRender::kImage2D;
		textureDesc.fEnvMapType = MHWRender::kEnvNone;

		// 'oneTextureToRuleThem' is where I am trying to capture' 
                // a texture, and then render that raw pixels to another
                // window. 
		oneTextureToRuleThem = textureManager->acquireTexture(MString("hwApiTextureTest"), textureDesc, targetData);
		//delete targetData;  // Code fails here if I uncomment. If I do not uncomment, then I get a memory leak.
	}

I'm thinking there must be some better way to capture the viewport window than to try to use the texture manager.  Perhaps there is something with M3dView where I can get the camera, then get the device context and do a copy of the pixels from there?

Any thoughts are appreciated.

Again, the goal is just to capture what is shown in the viewport, and then display that in a separate glfw window (which I have instantiated as a Maya plugin).

[zewt]

This is code for creating a new texture, not reading back the screen, so I can't really tell what you're doing.  If "targetData" is coming from MTexture::rawData, you need to free it with MTexture::freeRawData, you can't delete it directly.

Also, if you're rendering into a window in the same application, you probably don't need to do any of this.  The title said rendering in a separate application, but if it's another window in the *same* application you should be able to copy the texture on the GPU (glCopyImageSubData?) and not have a readback, as long as Maya is set to OpenGL.

[MikeSSSS]

Ah thank you!

I am now able to get the pixels.

Now however, I believe I am running into an issue, in that some time the issues are black. Meaning, my external window is perhaps updating at a different rate than the m3dview?

Is there a callback for every time something changes in the activeView that I can add, and only refresh my window when that occurs? Or perhaps there is a better way to synchronize updates between my external window and the maya viewport.