Scaling issue with FBX's

[BvC_Prod]

I'm encountering a related issue, much appreciate your help, Alfred. If I may begin by describing a workflow that works, then some intervention in 3DS that causes me problems with scale and other instabilities. 

I export many dozens of meshes from a photogrammetry app which lives in meters, run a script in 3DS (thanks to a coder's help) to add a Smoothing Group to each fbx in a folder, creating a new Output folder with these new fbx's. These import nicely into UE4, all good. I'm then needing to edit a selection of this group of fbx's to kill some unwanted polys (large triangles), then bring them back into UE4. If I make those edits to this fbx in the Output folder, it having passed through 3DS once before returning to 3DS, not changing any settings on import of fbx and only checking Smoothing Group under Geometry on export, the mesh won't import to UE4, just hangs up Editor. If instead I first edit the fbx in 3DS to kill large triangle polys after export from original app, then run the script in 3DS to the folder, those fbx's drop into UE4 at the wrong scale.

The last scenario came along when I went hunting for a way to preserve scale on import of the fbx, found under Advanced> Units> unchecking Automatic, the list for various metrics, and tried setting this to meters. I used this same setting on export, and when I then ran the script to batch adding the Smoothing Group to both the edited and unedited fbx's in the same folder, 3DS stops the script with a Warning (see attached) when it reaches the first instance of those fbx's that had the units set to meters.

What I'm not understanding is how this script doesn't appear to have anything in it that speaks to scale, and the scale is fine in UE4. It's only when I import an fbx onsey twosey to edit them, that I lose scale. Thanks for explaining scale and how to preserve it running fbx through 3DS.

EDIT: Title edited slightly for clarity - Alfred

[Alfred.DeFlaminis]

Hello @BvC_Prod,

Thank you for the detailed write up.  My first hunch is that the system units in Max are not in Meters when you are importing these objects and so the unit conversion is getting a bit weird.  There is a difference between display units and system units and usually that is the cause of problems like this.  Is there a simple object that you can attach to a message so I can verify this?  

Or, if you cannot share the file publicly I could create a private folder for you and investigate privately on my end.  I am happy to do this if you'd like.  Then I can write a detailed description that is tailored specifically for you.  Otherwise it's a lot of guess work which I am willing to do but isn't very efficient. 

Whichever you choose, I would try the following in the meanwhile:

1) Open Max, go to Customize->Unit Setup->System Units

2) Change System Units to Meters

3) Import your model

4) Make changes as needed, then export.  Try both Automatic and Meters as the export settings.  

I would also want to test this script out to see if I can repro the behavior.  Which version of 3ds Max is this on which OS?  This reads like a 2016 on W10 issue based on the error message, but confirmation would be great.  If it is, I have a link here that may help.  If it's not, thanks for the information!  I would like to get to the bottom of this and thanks for any extra info or a file to test.  Just let me know if you want me to make you a private folder.  

Best Regards,

[BvC_Prod]

Thank you, Alfred, much appreciate your assistance. I'm at the
airport, back in a few days, happily share sample fbx and such. I'll
be back asap. Again, many thanks for your time.

Benjy

Sent from my iPhone

[Alfred.DeFlaminis]

No problem at all @BvC_Prod, I'm here when you get back in.  I hope your trip is pleasant!

Best Regards,

[BvC_Prod]

Hello Alfred,

Back behind the wheel, have worked about every permutation and combination in trying to detect the pattern, learning some things, but no cigars, so I'm bringing in the heavy guns, which is where you fit in :^)

To simplify things, I'm sending you a OneDrive link to a couple of nested meshes in the state they're in after export from 3DS with that batch script run that I told you about to add the Smoothing Group, these dropping into UE4 in a way that's working for me regarding the lighting as it relates to scale. Just a word about that. My UE4 scene is set up in centimeters, and I've not had a chance to test how the lighting behaves if I change that to meters, but what I know is that setting my scene to meters in RealityCapture, adding Smoothing Group to 3DS without changing any units settings, using Automatic for units on export, and not checking Convert Units in UE4 on import, using the defaults there, that these meshes, two of which I'll share with you, behave well with regard to the lighting. By well, I mean if I scale the mesh too big I can't make my light sources strong enough to do their job, and too small a mesh I lose control at the other end, everything is too bright, I get out of the sweet spot range. So the scale is a bit arbitrary because of how the lighting works, but regardless, since I have these two meshes behaving well, let's say we make that our standard we wish to work to when editing one of these meshes and bring it back, this second mesh helping you more easily determine if a step introduced a change in scale, a loss of position, or both, which is much of what I'm seeing and can't seem to figure out.

Without parsing out every combination and permutation I tried, I should at least mention some of what I was doing. First thing was to generate the files that work to this standard I described above working for me in terms of the lighting in UE4 and which you'll be downloading from OneDrive. Under Customize> Unit Setup I tried Meters and Centimeters on import or after import. I tried Automatic under Units under Advanced and then unchecking to then try Meters and Centimeters on export. In UE4 I tried Convert Units checked and unchecked and I tried various combinations of each of these.

I'm seeing changes in scale, but seems like I'm also losing World Space as position is changing. If I scale up in UE4 to try to roughly match the scale there, position is off, which you'd think if World Space were preserved, that shouldn't be the case. Maybe, I'm too noob to see that it doesn't work that way. I much appreciate you efforts to help me figure this out, how to edit Pt-1a_pitbase_0000005, maybe export various versions as 5b, to have it match the original in scale and coordinate system. I'd love it if you had UE4 loaded and could being this in, hoping that isn't introducing tough variables.

Oh, and I'm attaching the macro sript along with a folder of the meshes that export from RealityCapture (RC) from a scene scaled in meters.

Thanks!

Benjy

[Alfred.DeFlaminis]

Thanks for the files and information @BvC_Prod, I appreciate it.  I have a couple questions, and I am seeing some of the scale differences you are describing.  

1) Which version of Max are you using?  I want to be sure I'm on the same one for consistencies sake.

2) How much do you need to scale these up these objects in UDK to be the right size?  

3) When they scale, do they scale from the pivot (which is out a ways from the object) or do they scale in place?  I'm trying to discern if the scaling is part of the reason that the location is changing on you.

4) Just need to make sure, when you were in Units Setup in Max, were you changing system units or display units? 

Thanks for the information. 

Best Regards,

[BvC_Prod]

Hi Alfred, big thanks for your time. I'm using Max 2017. When I export from RC a scene set in meters, run the Macro in Max, and import to UE4 which defaults to centimeters, I don't have to scale at all. The range of control over the lighting hits the sweet spot. The pivot definitely in World Space, not Object Space, and I see that preserved in whatever I do in Max after trying to edit the meshes. And when I set up units in Max I see the window says System Unit Setup. It stands to reason that if I change the units there, I will see the change reflected on import to UE4, versus if it were limited to display units, I'd only see this in Max, not on export, yes?

If you want to replicate what I'm doing, run the macro on the folder with the originals, which will be identical to the one I also provided with Smoothing Group applied. Then import ...0005, delete those flat polys on the top left of the cave wall object, export at whatever settings, then bring that in next to either ...0004 to see if it nests, or scales and aligns against ...0005, in UE4 ideally.

Thank you!

Benjy

[Alfred.DeFlaminis]

Hello @BvC_Prod,

Happy to help, I love detective work.    

Yes the display units should have no effect on importing and exporting, it's just to make measurements easier within the interface.  

I think I'm starting to understand the problem a bit better here.  I have just one last question, when you are making these parts in RC, what would be the correct height of the part named Pt-1a_pitbase_0000005?  (roughly) I want to make sure that I have the proper import and export settings but to do that I do need to know roughly how big this part should be so I can verify the scales.  Thank you so much for taking the time to keep posting!

Best Regards,

[BvC_Prod]

I'm struggling with that, no finding a measurement tool in RC. It has way to scale the model using control points on photographs, which mine feature a measurement tape, so that's how I created the scale in RC. But once reconstruction is done, not seeing a way to measure. I'd guess that 0005 part is about 5 meters from top to bottom. Possibly close enough. Thanks, I'll keep hunting a real way to measure it.

Benjy

[Alfred.DeFlaminis]

Thanks very much @BvC_Prod, I am seeing imports at around that size.  

To do a proper job here, I should test myself in UDK.  I am signing up and downloading now. 

Best Regards,

[Alfred.DeFlaminis · Accepted solution]

Hello @BvC_Prod,

This is a weird one.  I'm working mostly with pitbase_00005.fbx as my test object.  

I think the key is to keep system units in Inches since the FBX importer in Max is properly converting units.  According to a few pages I've read, 1 inch in Max converts properly to 1 system unit in UDK.  My tests seem to indicate that inches are the correct system unit for this script, which is counterintuitive since it's in meters.  

I'm a bit rusty with UDK and I really only used UDK3 in production, so 4 is a bit outside my comfort zone but that won't stop me.  The process that seems to work for me is this:

1) Open Max, set system units to Inches

2) Run Script

3) Import into UDK with conversion checkbox on.  (Screenshot below.  I left convert scene checked as well as Import Uniform Scale on 1.)  

Can you try that?  Thanks very much!  I did make a box 6 feet tall and export it to test scale in there and it seemed correct.  It took 3 six foot boxes to make up the height of 1 of your cave walls, which checks out when I do the math to meters. I could be wrong though since all the units seem normalized in UDK and I don't know how to check measurements any other way, yet.  

If this doesn't work for you, that's ok it's just round 1 of testing on my end.  Just let me know roughly how 'too big' it is and I'll revise.  Thanks!

Best Regards,

[Alfred.DeFlaminis]

Hello @BvC_Prod,

I just wanted to follow up here, any progress on this issue?

Best Regards,

[BvC_Prod]

Hi Alfred,

I was about to ask you the same  I've not tried anything else because I'm out of ideas. I've tried so many permutations and combinations, but seems like whatever I try, Max is changing something besides those things I want changed, namely my edits to meshes and deleting those large tris.

You indicated originally that you also saw something going on with changes in scale (and possibly position) on your end. This seems so straightforward a thing, right?, fbx imports in World Space, settings in Max preserve that World Space, only other changes to the mesh are then exported back to fbx, then imported to UE4 this edited fbx should import with the same scale and coordinate system (in World Space) as applied to the original, unedited fbx. Are we seeing all that the same?

I greatly appreciate you're time here. I'm heading to your headquarters in SF actually within a week, also presenting this capture tech at NASA Ames, big presentation there, so if I could clean up these meshes and have those align/scale to the unedited fbx's, I'd be golden.

Let me know if there's anything else I can try or do. 

Big thanks,

Benjy

[Alfred.DeFlaminis]

Hello @BvC_Prod,

For me, using inches caused the script to export parts that were correctly scaled into UDK.  Does that not happen on your end?  The problem I think is that the script assumes the system units will be in inches and scales the model weirdly if not. 

If I used CM or Meters as my system unit and ran the script, when I imported it back into Max with automatic scaling I did see scale differences.  But not with inches.  I know you'd tried some of the metric system units, but did you get a chance to try inches?  (Just want to compare notes.)  Thanks!

Best Regards,

[BvC_Prod]

I've not tried inches. Interesting, will do this in the am. Aren't inches the default? If so, then that would explain why the batch run using that script I provided, which doesn't set any kind of scale, defaults to something, inches in this case, thought that's not what's leaving RealityCapture and not what UE4 assumes, defaults to something that nonetheless works in UE4. If inches is the default and you got this edit to work manually setting to inches, then I get it that inches is what I should use. What doesn't make sense, not that I should slow down to care but so much, if inches is the default, then why wouldn't leaving Max to it's default with Automatic not yield something that also works in UE4? Why on this second editing/exporting of fbx would one need to manually set to inches?

In any event, you're saying you were able to edit like 0005 and have it drop into UE4 same scale/coordinate system as unedited 0005?  

Big, bigger thanks.

Benjy

[BvC_Prod]

Hello Alfred,

You solved it, Kudos! I set to inches coming in and leaving, drops right
in, I'm set.

Those questions I posed in the previous email I think are still valid
questions, might be something to look into. I can see the ambiguity around
the question of whether this is simply a potentially confusing issue that a
user can avoid by learning from the ground up, if the functionality and
documentation is counter-intuitive, or if this is outright a bug. The only
reason I include this last thought is because it seems Max is changing to
inches in Automatic, such that if an imported fbx is already in inches,
maybe nothing is changed, versus if it's in meters, as in my case, it is
changed. Furthermore, if it's changed once through Max and is still made to
work in UE4, then fine, but if you never go to UE4, but import/edit a
second time in Max, it's clearly changing or converting a second time,
which is what my workflow exposes, that would appear to be a bug trap for
others. Your thoughts.

In any event, had I continued with permutations and combinations I'd have
had tripped upon trying inches, so thanks for getting further out of the
box in your thinking and absolving me of trying harder. As so often is the
case, these problems are most often solved alone just by trying harder, one
gives up, goes crying to mommy, served a cookie, then regrets having given
up just then.

Now, how can I make my kudos more meaningful to you? You so deserve to be
recognized!

Benjy

--
***********************
Photo2Topo, LLC

Benjamin von Cramon - Founder, CTO
678-283-2557

[BvC_Prod]

I may have spoken too soon, not your fix, but now something odd and related. I'm just not getting it how to set things up to maintain scale and coordinate system, actually it's just scale I believe. 

If edited a whole folder worth of fbx's, these all behaved nicely in UE4 reimporting as aligned. I then ran the batch Smoothing Group on a second folder, brought these in to UE4, and now these don't align to the first folder worth of fbx's. I'm sure it's because of settings in Max because I tried importing two folders worth of exports straight from RC, one being the originals before we changed settings in Customize, Units Setup, System Unit Setup, which is currently in Metric, Centimeters, BTW, the other folder being this second folder from above that came in about 2.5 too small. These two imports align, both too small, so we know RC didn't throw a curve. I'm thinking the change we made under Customize may have come between my batch Smoothing Group routine to folder 1 using the default setting (how to get back to that setting?, because that's correct for UE4's lighting range issue) and the script run for folder 2. I'll try changing the System Unit Setup to folder 1 again, see if this works in UE4 for lighting, then apply the same setting to folder 2, affirm alignment, then go back an use the workflow using inches coming in and leaving for editing the parts. I'll let you know. 

I'm kind of doing work arounds here, thanks if you can help me make better sense of how best to understand Max System Units coming in and going out. 

Sorry, and thanks!

Benjy

[BvC_Prod · Accepted solution]

More testing and yes, that change we made to System Unit Setup wasn't carried through to the second folder. I switched System Units to Inches and back in alignment. Whew.

Again, many thanks!!

Benjy

[Alfred.DeFlaminis]

Hello @BvC_Prod,

I'm so glad it's working!    Yeah it looks like inches are the unit it's expecting, I'm glad for the confirmation.  Just to be sure, are you all set now?  I did read the other posts but I wanted to be sure that if you had any questions I could get an answer for you.  I'm going to change the solution to my post with images to make it easier for future users with this problem, unlikely as that might be.  I just wanted to let you know!  If you have any questions I am happy to help.

Best Regards,

[Alfred.DeFlaminis]

"What doesn't make sense, not that I should slow down to care but so much, if inches is the default, then why wouldn't leaving Max to it's default with Automatic not yield something that also works in UE4? Why on this second editing/exporting of fbx would one need to manually set to inches?"

I forgot to answer this question.  The script was likely written with the default settings in 3ds Max, which is generic.  Generic units translate to inches 1:1.  I'm not a programmer, but I don't think the script took into account the system units when it did what it does.  It all worked fine for me before the script, so I think something about that script doesn't look at the system unit differences.  The only place I think it could be is where this might happen is where it loads the FBX exporter.  There are three lines here:

pluginManager.loadClass FbxExporter --ensure the FBX exporter plugin is loaded
FbxExporterSetParam "SmoothingGroups" true --set the FBX exporter settings
FbxExporterSetParam "EmbedTextures" true

What is missing there is any parameters to enable the FBX scaling on export.  I found the parameters here.  The programming forum might be more able to help here, but my guess is that the script would need to look at the system units if they are not inches and find the right scale factor for the ScaleFactor parameter. This would be a bit of a pain to make since the system unit options are many, so it makes sense to me for it to assume the default 3ds Max settings. No biggie since most people that use UDK work with system units in inches (default.)  

I do want to stress one more time that I am not a great programmer or scripter, this may be incorrect on some level.  But we know it's all fine until the script runs, and we know that it works in inches.  So this is my best guess.  

Best Regards,