Xgen - import photoshop mask?

[Anonymous]

Hi there,

Is it possible to paint a map outside of Maya, to mask things like clumping within xGen? I'm having a bunch of issues with the 3d paint tool and figure it'd be simpler to just knock one up in photoshop and import it. I have a horrible feeling that it's just 3d paint tool or nothing though?

Thanks!

Ricky

[pshwayka]

It's a little tricky because xGen uses ptex maps, but it can be done. You may want to try a test scene with just a density mask:

1. Create a sphere, and apply a new xGen description. For your test, just use the default options in the "create xGen description" window.
2. For the mask field right under density, click the down arrow and "create map" like you normally would. Leave the start color at white, and then just click the save button. Get out of painting mode.
3. Save your scene. (this step is important.)
4. Open the Hypershade and go to the textures tab. There should be a file1, and in the attribute editor for that node you will see that Maya has created an iff file. Note the name and location of that file.
5. Open the iff file with fCheck, and save it as a .psd file.
6. Open the psd file in Photoshop, and edit as desired.
7. Save the file in an image format. (I like png.)
8. Back in Maya, replace the file1 iff image with your new png file.
9. Go back into xGen, and next to the mask field, click on the paintbrush icon as if you are going to paint the mask again. Note that your edited mask has replaced the original. IMPORTANT: Click the save icon next to the mask field, even though you have not made any new edits. I believe this step turns your new image file into a ptex map for xGen.
10. Your new density map should now be in effect when you click the preview button.

Another way you could probably do this (instead of converting the iff file that xGen creates) would be to make a temporary material assignment and a psd network using "color" as the selected attribute, then make a mask image out of that in photoshop. That method would allow you to have the UVs as a layer in Photoshop so you could orient yourself...but don't include that layer when you save your mask image. This method would still require steps 1-3 and 6-10 above.

[Anonymous]

Incredible reply pshwayka! I will be giving this a go first thing tomorrow 🙂

[Anonymous]

Sadly I got a host of errors and warnings. Am I doomed?

// Error: line 1: XGen:  Failed to evaluate subd. description8,pSphere1(896:0.264782,0.319337)  // 

// Error: XGen: Runtime error while setting up paint tool (C:\Program Files\Autodesk\Maya2016\plug-ins\xgen\scripts\xgenm\ui\widgets\xgExpressionUI.py:939) //

// Warning: file: C:/Program Files/Autodesk/Maya2016/scripts/others/art3dPaintSetPaintAttr.mel line 45: Some surfaces have no file texture assigned to the current attribute. //

// Error: XGen: Trouble closing ptx file.

The only difference encounted following your steps was that fCheck didn't allow me to save as a psd. So I saved as a targa instead. I don't suspect that's what's causing all these messages though?

[Anonymous]

Got it working!

It didn't like the png format for me. I tried tga and iff and both seem to be working!

Thank you so much! Going to try on my cat now...

[pshwayka]

Cool! Keep me posted.

[Anonymous]

I am getting hit with these errors each time I update the preview though. Any idea what that's about? It's just a default sphere  :

// Error: line 1: XGen: Failed to evaluate subd. description8,pSphere1(854:0.405213,0.473110) //
// Error: line 1: XGen: Failed to evaluate subd. description8,pSphere1(854:0.405213,0.473110) //
// Error: line 1: XGen: Failed to evaluate subd. description8,pSphere1(842:0.280164,0.467221) //
// Error: line 1: XGen: Failed to evaluate subd. description8,pSphere1(845:0.379402,0.242790) //
// Error: line 1: XGen: Failed to evaluate subd. description8,pSphere1(851:0.264782,0.319337) //
// Error: line 1: XGen: Failed to evaluate subd. description8,pSphere1(944:0.379402,0.242790) //
// Error: line 1: XGen: Failed to evaluate subd. description8,pSphere1(959:0.280164,0.467221) //
// Error: line 1: XGen: Failed to evaluate subd. description8,pSphere1(1013:0.442642,0.341042) //
// Error: line 1: XGen: Failed to evaluate subd. description8,pSphere1(1019:0.280164,0.467221) //

Thanks,

Ricky

[Anonymous]

And now the usual process is working (after countless failed attempts previously)!

I've no clue why, but perhaps the order of steps that you outlined previously is what enabled it to work using the 3d Paint tool. 

Thanks again pshwyka! It's clumping time.

[pshwayka]

Sorry....I got sidetracked for a while. I just tried the process again and didn't get those errors, so I'm not sure why you are. Hmmm.

[Anonymous]

Mysteriously the errors aren't there on my cat mesh, however I have a different issue now...

I'm trying to clump some hairs on the body with the face masked out. However I'm getting a small patch above the eyes that's getting shorter. I've tried flooding the mask black and setting clump to 0 to debug, but the influence remains. It's the same on both sides. Actually all the hairs on the cat move ever so slightly when I preview (still set to 0) but not enough to be a problem. All "Effects" are also set to 0.

The only way I've found to get rid of this is to uncheck clumping entirely, which is obviously not want I want - although manually painting the clumps might actually be quicker in the end!

Do you have any ideas pshwayka?

[Anonymous]

Ok, so simply adding a clump mod, setting the amount to 0 and hitting preview causes numerous changes throughout the character, including some unwanted short spots. The worst is after I add the mask, but currently looking like I'll have to manually groom the effect 😕

[Michael_Todd]

Are you using region mapping?

If you are then this can cause some issues if you are using maps painted outside of Maya. When generating clump maps, if you have a region amp, then the clump map uses the region map as a control. What this does is to make sure that the clump cell borders match with the region borders to prevent clumping region misalaignment with the region map. If this were to happen, clumps that should be on one side of a region could be pulled to the wrong area as the cell borders do not align.

Is there any chance you could psot the scene for me to take a look at? It's hard to assess what the true issue is without seeing the setup..

Cheers

[pshwayka]

Hi Michael,

Speaking of region maps...I went to edit one region on a model the other day, and every time I applied a brush stroke, all of the regions' colors changed...but only during the brush stroke. This was on a 2013 Mac Pro. I tried the same operation on the same model under Windows, and it worked fine.

Have you seen this happen before?

BTW...every time I see your avatar, I think of Rutger Hauer in Blade Runner.

[Michael_Todd]

Good, that's who its supposed to be!

Sounds like a video card/driver issue....

[pshwayka]

Actually, my memory of the issue was incorrect...the region map stays the wrong color after it is painted...and I can recreate it at will. This is on 2016 Ext2 (El Capitan.) It seems to be an issue with the new OpenGL core profile-strict mode, because it doesn't happen under regular OpenGL, and it also does not happen in Ext1 SP6 under the old OpenGL core profile.

Might be something to look at if Maya 2017 is going to be supported on El Capitan...

[Michael_Todd]

That must be an Artisan/drivers issue specific to OSX. I'm not sure if it's a known issue though, in terms of being looked into.

It looks like the color value being applied is the inverse of the selected color.

[pshwayka]

Iterestingly, it doesn't seem to occur during texture painting using the 3d Paint tool. (Sorry...didn't mean to hijack this thread.)

[Michael_Todd]

if you hold cmnd down while painting, do you get the correct color?

[pshwayka]

Command+LMB (or tablet pen drag) is actually like alt+LMB (it rotates the view) but ctrl+LMB causes the painted area to be a different incorrect color...so painting with green as the selected color ends up as yellow instead of purple as I showed previously.