topic Re: how to bake a normal map from a mesh that has rounded edges? in Arnold for Maya Forum

how to bake a normal map from a mesh that has rounded edges?

Mike_Isaakidis — Tue, 13 Aug 2019 21:47:53 GMT

New to Arnold so please bare with me.

Trying out some new workflows, to see if I can quickly get a normal map from an asset (in Maya).

So what I've done so far was:

1. Apply a roundcorners node on a aiutilty node (set the shade more to flat and color mode to Normal)

Result in render

2. Use render to texture an AOV (diffuse) to get the World Space normals.( i assume they are World Space?)

3. Open the exr file convert to 16bit in photoshop and save as a png.(cant save as another file format)

4. Use Handplane to convert the png file from Object space to tangent space (Handplane does not convert World space to tangent space).

5. Take the new converted file and apply it back in Maya as a normal map.

Result (normals are wrong)

I assume there is no way of getting a normal map directly baked on a texture from an AOV?

Is there a way to get the object space normals instead?

If you can render out a normal map, how should I have my smoothing groups?

How do I control padding?

How do I fix the graininess on the render to texture settings? (Camera settings (AA) were set to 4

Any help would be greatly appreciated.

Cheers

-Mike

Re: how to bake a normal map from a mesh that has rounded edges?

askinner6CDVL — Sun, 29 Sep 2019 17:32:37 GMT

I've done this. Add an aov for normals - n. Render that to texture. You will get a world space normal map. You can convert that to tangent space in substance designer in a few steps.

Re: how to bake a normal map from a mesh that has rounded edges?

madsd — Mon, 30 Sep 2019 11:49:09 GMT

You can convert world space normals to tangent space with the Space Transform node. You also need to move it from -1 to 1 into 0-1 by adding up 1 and multiply with 0.5 afterwards.
Solution fabricated by Arnold devs, I just forward info, its also what happens under the hood.

Re: how to bake a normal map from a mesh that has rounded edges?

askinner6CDVL — Sat, 12 Oct 2019 18:03:53 GMT

That's interesting to know. It might actually be less hassle than the method i tried. I will give this a shot. Cheers !

Re: how to bake a normal map from a mesh that has rounded edges?

askinner6CDVL — Mon, 14 Oct 2019 20:16:06 GMT

@Mads DrÃ¸schler Sorry I can't get this to work at all 😞 Could you elaborate on the steps you took to make this work? I have the shader set up like this - then I do render to texture and still get -ve values in the texture, also I can't get render to texture to render anything other than an exr. I'd like to automate this eventually so it would be good to get this working!

01.png

Re: how to bake a normal map from a mesh that has rounded edges?

askinner6CDVL — Mon, 14 Oct 2019 22:04:53 GMT

Here's the viewport and the texture bake in substance, see how there's still black areas that are negative.

02.png 03.png

Re: how to bake a normal map from a mesh that has rounded edges?

Anonymous — Tue, 15 Oct 2019 02:27:23 GMT

@Aten Skinner add 1 and multiply by .5 now to remap it into 0-1 space

Re: how to bake a normal map from a mesh that has rounded edges?

askinner6CDVL — Tue, 15 Oct 2019 08:33:46 GMT

I'm already doing that in the shader though, see pic01 ^^

Re: how to bake a normal map from a mesh that has rounded edges?

askinner6CDVL — Tue, 15 Oct 2019 08:37:52 GMT

Oh hang on have I got the wrong end of the stick here? You mean +1 and *0.5 as a post process, i.e. in Substance I was thinking you could somehow bake it out like that

Re: how to bake a normal map from a mesh that has rounded edges?

madsd — Tue, 15 Oct 2019 11:22:06 GMT

I know this ain't a maya screenshot.
However.

Source the bumped Normal from AOV's back into a shade chain with READ AOV, the most left top.
Then source the unbumped normal with a state vector N, bottom, and source them to the space transform. See settings on space transform node.
Then move the entire thing into the correct space 0-1 with the constantbias scale +1 *0.5 Then exit to a WRITE AOV, I call mine test. then flip to it, and this is what I see.

Also, you need to setup the material, its the 2 nodes seperate from the transformation kit, its basically just an Arnold Material with the rounded corner plugined to Normal ( and we read this back into chain which is the first thing I described, top left node )

This is basically also what you do if you toy with volumetric displacements using OSL shaders or whatever if they sit in tangent space from the get go, then we can move it into the right space so for example a cloud wont start to shift in a 45 degree angle when displaced, but instead take middle as starting point so volume gets displaced properly in all directions.

I also do this on the OSL flowmap, we paint in 0-1 space and move the data to -1 to 1 so that a paint strokes has 4 directions, eg, we can make circular strokes and the uv offsetting is following the brush, spotless perfection around in a circle.

Re: how to bake a normal map from a mesh that has rounded edges?

Mike_Isaakidis — Mon, 21 Oct 2019 00:17:22 GMT

Does the round corners work for angles that are not 90 degrees?

(like 60 or 105, 125)

Re: how to bake a normal map from a mesh that has rounded edges?

3dbenjarn — Wed, 16 Jun 2021 21:15:10 GMT

Hi did anyone get a final Maya shader that worked with this? The answers are great but still can't get it to work. If anyone could share an image of their setup that would be great thank you

Re: how to bake a normal map from a mesh that has rounded edges?

skx555 — Mon, 21 Mar 2022 23:58:01 GMT

I'm bumping this - @Ben Moseley - over at the Arnold Support Blog is an ASS Material of the tangent calculation, you can copy the code lines and import the node-setup https://arnoldsupport.com/2021/02/19/tangent-space-normal-maps/

its in the same way as Mads did it, but also the actual part how to visualize the Normal AOV is not clarified. I am uncertain, but could be that the part of setting up the (Normal) AOV as Material even differs between DCCs. I couldn't make it work in C4D. It's not clear how to feed / read / loop N from the scene geometry back to the custom Tangent Normal-AOV. Either by a second AOVwrite assigned globally as material or by object mask or aov tags locally? Or by a read/write from a Material on the normal nodes itself? And does the main AOVwrite Node need a Material assigned to the pass-through to actually draw anything at all?

Can anyone share a scene with the complete AOV setup set in place?