units precision preference vs. data entry precision

[mbostonsprint]

I am working on a design currently in mm units, with a display precision level (general and angular) of 0.12345. I have noted several occasions where I have entered numerical data for a positional command, such as Move or Plane Offset, at an exact integer value with only (3) trailing zeros after the decimal point, and come back later to find that the actual measured distance using the Measure function is not showing as exact; e.g., entry of 6.000 mm might yield a value of 5.99995 mm when measured later.

My question is basically whether making an "incomplete" command data entry without all the decimal places filled with zeros allows the software to artifically round off the value in executing the command, or if there are other contributing variables in this (small) degree of fuzziness. I have noticed this before, basically wrote it off to sloppiness in my point selection, but I had a couple of cases in the last day or so that seemed pretty clear-cut.

Obviously, quibbling over 50 nanometers is "gettin' down there", but sometimes it's important.

[Phil.E]

Hi,

Thanks for posting. I cannot reproduce this effect based on your description.

Do you have, or can you make, a simple model that shows this? It sounds like you can reproduce it with your workflows, which are likely key. Another big help would be a video made with Autodesk Screencast so I can see the exact steps and clicks and results.

The reason there is rounding at all is because CM are the working units inside Fusion. So any use of inches, for instance, requires a conversion that would eventually have to be rounded. I just don't recall seeing anything like this inside of 6 or 7 places precision. Normal precision used in manufacturing should not show this at all. And as I said, I cannot reproduce it so your help is key.

BTW Inventor and other CAD programs do the same thing.

Thanks again,

[mbostonsprint]

Hi, Phil --
Thanks for the response. If I have time sometime soon, I'll study this in more detail and see if I can reproduce it consistently. I'm still not absolutely certain of the correlation yet.