Floating point error in simple expression

mikee2185 · ‎11-22-2023

Hi,

why subtracting 3 mm here is a problem? I understand floating-point arithmetic and its limitations, but that most definitely should not affect values with one decimal point under 30. Also, problem is only when working with mm. Unitless math works just fine.

Thanks

jhackney1972 · ‎11-22-2023

The Logical Operator "==" is used along with an "if" Function. I think you are using it in the wrong context. Below is the statement from the Fusion 360 Help page and how I think your parameter table should read.

John Hackney, Retired
Did you find this post helpful? Feel free to Like this post.
Did your question get successfully answered? Then click on the ACCEPT SOLUTION button.

mikee2185 · ‎11-23-2023

I used the operator only to visualize the problem. But still, I don't get why in some cases it works, and in others don't.
But I encountered it while trying to use modulo, which also returns invalid value.

I'm pretty sure this is a bug. It should either work, or not work. In both cases.