Problem Baking Normal Maps with Transfer Maps

[Anonymous]

Hey all!, I'm new to Maya but have been modelling with Blender for a while. So far I enjoy how powerful Maya can be! However, there are many aspects that were fairly straightforward in Blender that are downright infuriating to get through in Maya. Anyway enough ranting.

I am wanting to do the ol' normals bake from a higher poly mesh to a lower poly mesh. For the past couple of days, I have been unable to muscle through this issue. I know there are a bunch of threads on this topic, however, none that I've followed have resolved my issue. I have gone through and tweaked each option in transfer maps, only getting marginally better results in some configs but for the most part, this is what I keep getting:

Stacked NormalsNormals UnstackedId prefer the uvs of the axe head be mirrored for my texturing workflow, many threads suggested that any stacking of uvs would interfere with the baking process so I did both for funsies (not sure to what extent "Ignore Mirrored Faces" effects this since I have yet achieve desired results to test with).

I'm not sure what I have missed. I have attached my axe.mb project file, if you have a spare minute I'd really appreciate yall takin' a gander and point me in the right direction. 

Thanks for your time

 - B.

Also, I realize my current topology will leave some unwanted artifacts but I will cross that bridge when I get there.

[Anonymous]

Update: Still haven't resolved the issue. Heres what I've done since my last post and its only unearthed more questions.

I figured that since changing the "search envelope (%)" size did not update on screen (which was a known bug in Maya 2018 I believe) I thought that updating maya may fix it. After some issues with Autodesk Desktop App I ended up just completely reinstalling Maya 2018 ( now running version 2018.3 ). Made some test cubes and the search envelope updated as expected. Great! so I opened the older axeUVcopy.mb file. exported both high and low poly meshes to .obj files and imported them into a new scene. Unfortunately, I am now getting the same search envelope display update bug. Here are some screencaps of the envelope issue. Envelope 0%Envelope 100%

This was how it was in the old scene where the only visual update of the search envelope was to perform a bake, then reopen Transfer Maps.

Now I'm 100% certain these issues are stemming from these "things" that also transferred with them.

Outliner

I did a test export to .obj with only normals selected and did not get these "things"

Will update later with how this affects the bake (out of time for now). But until then I will pivot my questions to these:

1. What are these "things" that have imported with the .obj file, and how would they affect mesh operations such as baking.

2. I was unable to find specific information for these "file Type Specific Options" for exporting as .obj specifically the first two: Groups and point groups. if someone could shed some light on what these two are and in what situations you may want this data saved within a .obj

[mspeer]

Hi!

For me this looks more like "bad" geometry and UVs.

Try optimizing mesh and UVs for your lowpoly object.

[Anonymous]

Is there another method for pin-pointing / removing bad geo & uvs other than Mesh ►Cleanup? I've used that and is still getting the envelope error (in the old scene).

As for the new scene I'm still not getting the normal bake I'm looking for... will update again once I mess with different configs within transfer maps.

Any other input and/or ideas are greatly appreciated.

[mspeer]

Hi!

No, you need to correct the mesh and the UVs manual.

There is no one button solution for this.

[Anonymous]

After a long session of fiddling with the most of the Mesh Display tools and manually tuning some vertex normals, I'm getting closer to my desired output. Still getting some unwanted artifacts:

I'm pretty sure the warping on the axe head is from poorly cut UV shells but the ones on the handle and base have me stumped for now. I will update again when I make more progress but I'll leave you with one more question if ya don't mind :).

What are some operations during the modelling process that would alter the vertex/face normals that drastically?

Really appreciate the input so far @mspeer.

[mspeer]

Hi!

1. Look at your UVs, zero area, where should the detail-information be stored? Correct the UVs.

2. Why these round corners at the lowpoly object? Remove those extra edges and adjust the overall shape, so it better fits the high-poly object.

[Anonymous]

After MUCH trial and error (the "preview" sampling quality is a godsend) I've achieved something close enough that I'm happy with.

To get this I found the "Soften/Harden Edge" tool in mesh display very useful, Just had to tweak the angle cutoff.

Something I don't understand, however, why did I have to tweak the vertex/face normals on the Low Poly model? Isn't the purpose of the Transfer Maps tool (in the context of Normal Map outputs) to transfer the vertex/face normals of the higher poly mesh to the UV set defined by the low poly mesh?

I would expect awkward vertex/face normals on the low poly model to produce artifacts during a render or in scene view, not during the baking process. Any clarification would be helpful.

Thanks again - B

[Anonymous]

apologies @mspeer, didn't see your reply while I was typing my update. Strangely your suggestions were exactly what I had been fixing most of this morning

Related → the original file upload is obsolete.

[Anonymous]

---

Something I don't understand, however, why did I have to tweak the vertex/face normals on the Low Poly model? Isn't the purpose of the Transfer Maps tool (in the context of Normal Map outputs) to transfer the vertex/face normals of the higher poly mesh to the UV set defined by the low poly mesh?

Yes, but it looks to me like your problem originated with the modeling of your low poly mesh to closely match the SILHOUETTE of your hi poly mesh, and this affected the creation of your UV map, which of course would then not match your hi poly mesh silhouette when you transferred the UV map from low poly to hi poly.

I'm still learning myself so I could be wrong, but when I see undefined edges like in the handle of axe in the normal map you displayed, that starts indicating to me that there is a mismatch of distance in that area between the high poly and low poly versions of a model in those areas.

All maps start from the UV map.
If the UV map shared between two meshes don't reflect both meshes accurately enough, all further map baking will reflect or amplify the disparities.