Is this a bug . strange values

[StephenCim-001]

Hi All

My Add-in has a couple of Value inputs, one for Shank Width (ShankWidth) and one for Stone Diameter (StoneDia), both are in mm

The values inputed are stored has User Parameters .

here are a few screen shots  both set to 4mm, and you can see the user parameters showing 4mm

now with settings set to  2mm

and user parameters

But if I use any number 3.x number I get this 

If I entre 3.5 , it will get stored has 

3.5000000000000004

3.708 and above is store has it's entered 

3.707 becomes 3.7070000000000003 mm

2.999 is fine  if I enter 2.9999 it then becomes 2.9999000000000002 mm

Which is does not look so good when I read the value back into the input

Has anyone come across this before?

Cheers, Stephen

[StephenCim-001]

Am I right in saying no one has seen this sort of error before?

Cheers, Stephen

[BrianEkins · Accepted solution]

Welcome to the world of floating point numbers and computers.  It's not a bug but is just a reflection of how floating point numbers are stored in a computer.  I wrote a post about this a few years ago illustrating it using Visual Basic.  I also just found a good write-up in the Python documentation.  It's important to note that it's not a programming language issue but just a reflection of the internals of how computers work with floating point numbers.

[StephenCim-001]

Thanks Brian I've added some code to truncate figures to 2 decimal places.... 🙂 Cheers, Stephen

[JeromeBriot]

Hello,

As @BrianEkins  already said, this is related to how floating-point numbers are encoded into memory. See IEEE 754 for more information. Most of the time, these numbers are encoded with 64 bits and are represented with 16 decimals.

The only problem here is that the last digit of the numbers are incorrect. It's obvious for 3.5 that should be exactly represented as 3.5000000000000000

Here is a small python code:

X = [2.99, 3.5, 3.707, 3.708]
['{:.16f}'.format(x) for x in X]

And the results:

['2.9900000000000002',
'3.5000000000000000',
'3.7069999999999999',
'3.7080000000000002']

As you can see, these values are different from the ones you reported in the first post.