Message 1 of 8
[iLogic] Why Don't User Parameters Work With Arithmetic?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report
length is a UserParameter with "in" units and expression "55 in + 5/16"".
length Mod 55.3125
returns 55.3125 instead of 0. Why?
I did some testing and doing
Dim number As Double = 55.3125
number Mod 55.3125
returns 0, as exptected.
Trying
(length).Value Mod 55.3125
and
Dim middleMan As Double = length
middleMan Mod 55.3125
Do not work either.
But
Math.Round(length, 9) Mod 55.3125
works ok, so I think it has something to do with the number of decimal places of doubles. But, according to Microsoft VB.NET documentation, Double can handle many more than only 9 decimal places. So what's the issue? This is code-breaking but I haven't seen it documented anywhere. Also, is rounding Parameter values to 9 decimal places guaranteed to work every time or is there a better method? Maybe accessing the Parameter.Value property through the API and converting units is best.