While trying to figure out why the 3D Paint Tool (Rendering Menu -> Texturing -> 3D Paint Tool) did not work with my custom materials (I’m not the first user encountering this issue), I seem to have found a nasty bug within the art3dPaintCtx command.
It turns out that no matter which custom material I take (e.g dx11Shader, hwPhongShader [devkit], lambertShader [devkit], etc.) the 3D Paint Tool doesn’t work (as of 2016.5 and 2017). After digging the mel and python files concerning the art3dPaint tool, I found out that the art3dPaintCtx command was the culprit of this issue.
How to recreate this?
- Assign a custom material to your shaded object (dx11Shader, cgFX, any devkit shader)
- Select the custom shaded object and go to Rendering Menu -> Texturing -> 3D Paint Tool [ ]
Without tinkering any further, you will see that no brush projection will appear on objects with custom materials and you may also find in the tool settings under File Textures that the Attribute to paint combo box will be empty. With most Maya shaders (lambert, blinn, etc), the painting projection will work fine and the combo box will contain the different attributes to paint.
The python file that populates the attributes to paint can be found at:
C:\Program Files\Autodesk\Maya2016.5\Python\Lib\site-packages\maya\app\general\art3dPaintGetPaintableAttr.py
At around line 51, affected shader names try to be retrieved by calling the art3dPaintCtx command:
shaders = cmds.art3dPaintCtx(cmds.currentCtx(), q=True, shadernames=True).split()
Running this line of code with custom shaders will not return the material names, so the attributes to paint cannot be parsed. However, with normal Maya shaders, it will return the material names normally.
I thought I could work around this issue and created a custom function to query the materials of selected objects to populate the attributes to paint. It worked, but it didn’t solve the painting problem.
The art3dPaintCtx is widely used in the 3D Paint Tool and other queries were not returning any values with custom materials within the Maya scripts. Another example is on:
C:\Program Files\Autodesk\Maya2016.5\scripts\others\art3dPaintAssignFileTextures.mel
At around line 733, the names of the shape nodes are retrieved:
string $srfNames = eval("art3dPaintCtx -q -shapenames `currentCtx`");Again, a similar problem with custom shading nodes, but this time no shape names are retrieved (with normal Maya shading nodes, it works). I again had to code a workaround which allowed me to get the shape node names. It worked again, but the problem persisted somewhere else though, where no workaround seems to be possible.
C:\Program Files\Autodesk\Maya2016.5\scripts\others\art3dPaintCallback.mel
At around line 2052, the created texture is assigned to the attribute in the current context:
art3dPaintCtx -e -assigntxt `currentCtx`;
This edit command will error out with custom material nodes with the following message:
“No paintable surfaces found. Have you set their shape nodes to be visible? //”
The problem seems to be again with the shape nodes, which can’t be found in custom materials within the art3dPaingCtx command and I have no way of setting it myself or creating a workaround in this case (everything handled within Maya)…
I believe this problem propagates throughout all the art3dPaint scripts, as they all call the art3dPaintCtx command. This is also the likely reason why there is no brush projection and the tool and in general, is unusable with custom material nodes.
This seems to be a serious bug for anyone creating/using custom material that would like to take advantage of the 3D Paint Tool.
Has anyone found a solution to using the 3D Paint Tool with custom material nodes before? There might be two ways to go about this:
- Maybe there is something we can add in our custom material nodes, so that the art3dPaintCtx command can work and gather all its necessary data.
- Consider this a bug issue to be addressed in a next Maya update. This could potentially be quite straightforward to fix within the art3dPaintCtx command source code itself. I would also do it, if the 3D Paint Tool were to be available in the devkit.
I appreciate any input you guys can give me. It would be amazing to get this working.
Cheers,
Santiago.
