Object sorting of transparent objects not working with MRT MSceneRender

[drsanty]

Hello Autodesk dev support and everyone else interested in render overrides,

We've been encountering problems with MSceneRender when rendering transparent objects to multiple render targets.

• When rendering to two targets, object depth sorting of both opaque and transparent objects work.
• When rendering to three or more targets, object depth sorting works for opaque objects, but transparent objects' depth sorting stops working.

I understand this is a complex behavior to reproduce, so we've made the effort and developed a test plugin (following devkit conventions) to make it easier. You may find the plugin source and instructions in this repo: https://github.com/artineering-io/viewOverride.

The MSceneRender in the plugin renders to 3 targets (color, depth, normals) showing the object sorting issues with transparent objects [see attached screencast] (open TransparencyTest.ma file to observe the problem on different transparent materials using the View Override plugin).

When you change line 51 in viewRenderOperations.cpp to render to 2 targets instead of 3, recompile and reload the plugin, object sorting works well again.

We need to render to multiple render targets in our NPR engine and our clients are beginning to experience transparency issues when having multiple transparent objects in the scene. We've tried solving this issue on our own with the existing API, but it seems to be out of our control. Is there something that we missed out that we can fix?

We could create our own scene renderer for transparent objects, but we are hoping for a more straightforward solution.

We'd appreciate your ideas and support and are happy to help troubleshoot and test this in any way we can.

[zewt · Accepted solution]

I dug into this a bit.  It looks like what's happening is the sortTransparent in the mayaTransparentDrawPass script fragment is ending up false: if you edit that script and comment out the if around the two sortRenderListByZ calls, you get sorting again.

That might point to a solution (at least one that's better than writing your own renderer): compile a script fragment that explicitly forces that on.  If you want the full Maya renderer to run then that's a bit tricky, I think you'd need to replace MayaShadedBeauty, which is a nontrivial block of XML and can't just be copied and edited due to licensing.  I suppose you could read the file, programmatically search and replace the sortTransparent connection with a setter, and write that out locally.  Pretty ugly but manageable, maybe there's a nicer way.

(Also, I've only looked at basic object sorting, not depth peeling.)

[cheng_xi_li]

Hi,

I tried to find a workaround but I didn't find an easy one. I also checked with our engineers, they'll do a research. I've logged a defect in our system on your behalf.

Yours,

Li

[drsanty]

Woah that works, thanks @zewt!

At least we can offer a workaround for older versions of Maya and for new ones until Maya gets patched..

Out of curiosity, how do you go about troubleshooting script fragments and their values/arguments?

[drsanty]

Thanks @cheng_xi_li, hopefully this can get patched soon in a future update.
When bugs like these are fixed, up to how many Maya versions usually get the update? This will help us plan ahead with different customers that may be using different Maya versions.

[zewt]

I've always found them a headache to troubleshoot, as far as I know there's no debugging or visibility into them and it's really easy to crash Maya with a bad fragment.  It's just making edits and testing.  It's not a fun system to use, there's not much visibility into it or diagnostics when something fails...

[cheng_xi_li]

Hi,

It depends on the patching schedule and how complex the patch could be. If there is a planned update for older versions, it might be integrated into those patches.

Yours,

Li