Mesh changes color after using Sculpting Tools

[user20202022]

Sorry, I don't know if this is more of a modeling or surfacing question, but hopefully somebody knows the answer 🙂

A lot of times, after I use Maya's sculpting brushes on a character, the preview color of the mesh turns black when I turn on preview subdivisions (occasionally it's white or a light gray instead). Does anybody know how to stop this from happening? 

Here's a normal situation to illustrate what I'm talking about:

- I have a character model with various colors applied to each of the separate meshes (usually just lamberts of different colors, no texture maps yet)

- We're showing our director and interactively tumbling around the mesh and he'll say something like, "Can we see what it looks like with a little longer nose?" 

- So I turn off preview subdivision (i.e. hit "1" on my keyboard), use the Grab tool and pull the nose out

- When I subdivide the mesh again (i.e. hit "3" on my keyboard), the model turns black (sometimes it's white or a light shade of gray instead)

I figured out that I can get back the colors I expect to see by going in to the Color Set Editor and deleting anything that might be in there, but it's really annoying to have to do that every time. Is there a way to turn off whatever's happening so that it doesn't happen in the first place? 

[Christoph_Schaedl]

Are you on the latest Maya Version?

Could you show a little video? I would forward it to the dev team.

[user20202022]

Thanks so much for the reply @Christoph_Schaedl ! I actually wasn't on the newest version. I upgraded and the problem hasn't reappeared yet, but I'll send an update if I come across it again 🙂

[user20202022]

Unfortunately I started coming across the problem again... Here's a little video illustrating what's happening. I don't know much about the Color Set Editor (I just found the fix by complete accident), so I'm not sure if that's the correct way to fix this issue and whether or not there's a way to stop it from happening so I don't have to take these steps every time. If anyone has any insights about this, I'd greatly appreciate it. Thanks! 🙂

[damaggio]

Try switching to Direct X 11 for the viewport render to see if it makes a difference.

[user20202022]

Hey @damaggio , thanks for the reply! Unfortunately setting to Direct X 11 didn't make a difference... Still seeing the same issue after using the sculpt brushes 😞

[Christoph_Schaedl]

I assume its caused by vertexcolors.

Delete the colorset.

[user20202022]

Yeah that's what I've been doing, but since I work as a modeler/vis dev artist, it's very cumbersome to have to open up the color set editor every time I use any sort of sculpting brush on the model. Especially for things like interactive sessions with our director, production designer or art director. Like I said though, I found that fix entirely by accident and don't actually know much about the color set editor. Is there a way to prevent the colors from being affected in the first place so that I don't have to delete the colorset every time I adjust something on the model?

I don't really remember this happening in previous versions of Maya. It's really only something I've noticed within maybe the last year or two... 

[Christoph_Schaedl]

I might be wrong but i think its fixed in maya 2023. I double check tomorrow.

[user20202022]

Thanks! Appreciate the help! We actually already updated to 2023.2 though and I'm still seeing the issue...

[mcginnc]

Hi there - thanks for video. I'm trying to reproduce what you're experiencing, but on a simple sphere, lambert, etc. I can not.

Can you tell me more about that model. Is it a combined mesh, etc.  Is it happening on all models?

What kinds of Smooth Mesh settings are you using (Default OpenSubdiv Catmull-Clark)?

[user20202022]

Hi @mcginnc 

It's one, unified mesh. There aren't any separate pieces that have been combined or anything like that. And I just use the default smoothing on it (OpenSubdiv Catmull-Clark). 

[user20202022]

After months of banging my head against a wall trying to figure this out, I have a small find that might be useful insight. In the Shape tab for my mesh, under "Mesh Component Display", I noticed that it appears to be the Display Color Channel that causes the issue. 

Here's what I mean:

- My mesh looks as expected. It's just a normal model that has some texture map or basic lambert applied to it, and it looks fine whether or not I have preview subdivisions turned on in the viewport. 

- I use a sculpting brush like the Grab tool on it to adjust the model. Then, all of a sudden, when I turn on preview subdivisions, the mesh will turn either all black or all white. But if I turn off preview subdivisions, it looks correct again (see previously posted video). Sometimes I can just open the Color Set Editor and delete what's there to fix the issue, but when nothing shows up in the Color Set Editor I just have to export the model as an obj and reimport it.

- What I recently discovered is that if I go to the "Display Color Channel" in the shape tab and don't have it set to either "Ambient+Diffuse" or "Diffuse", the mesh looks correct again when subdivided (ie having it set to "Ambient" makes the model look correct). It also works if I leave it as "Ambient+Diffuse" but change the "Material Blend" mode to "add" instead of "overwrite".

- I can also just turn off "Display Colors" altogether and everything looks fine too. This is a very tedious process to do this on every mesh every time I use a sculpting brush though... 

I don't know if my sculpting brushes are somehow affecting the diffuse channel of my display color channels, but I thought I'd mention it in case anybody more knowledgeable about this can tell me if my assessment sounds accurate (and how to stop this from happening so I don't always have to go in and adjust this and/or export and reimport my models all the time). 

[apphelliuxs]

Perhaps i can help you, because i got same problem. So I dont Know Where is My problem exact location. Accidently i got it fix, After I compare my real and backup Model. After I try anything setting from My backup model i got it. You merely go to : Color Sets -> Color Set Editor -> Display -> All -> Color In Shade Display Off. After doing that my model can back to original material. Hope that's help.   

[louannordanez]

This fixed the issue for me! THANK YOU! It was so annoying xD