Bug in Val() with Floats and fix for it

[Stefan Weber (AH)]

Hi everyone, esp Allan.

I just have fixed a rare bug that occurs when using Val() to an Extended type with 30+ digits.
This bug occured only when truncating this value to a 64-bit integer.
NDN's internal 64-bit calculator has showed this, and thanks to the help of a helper named CandyMan
I was able to fix the bug in the VP RTL.

I have written a test program to show the problem and also have added a bad and fixed W32
executable to that test archive (testval).

Here's the shortened output to show the actual problem:
Comparing values before and after Trunc64()
-1 -> -1.000000000000000E+0000 Passed!
-1.00000000000000000000000000000 -> -1.000000000000000E+0000 Passed!
-1.000000000000000000000000000000 -> -0.000000000000000E+0000 Failed!
Please note the number of input zeroes.

Trunc64() is a simple frndint with rounding towards zero Please see mathx.pas for
that function and several other 64-bit Comp/TQuad/QuadWord functions.

Please look into system.pas for FLOAT_VAL_FIX to find all changed code.
I don't want to remove the old code since it may be helpful to see the differences.

I compared the binary output of the actual values with the old and new implementation and
in the error case they look totally different, I guess this is some sort of precission error or
something.

Anyway, I hope I have supplied all necessary code with this post.

All the best,
Stefan / AH

[Allan Mertner]

Hi Stefan,

Thanks for that - that is a tricky corner case! I have tested and included the fix in system.pas.

I also looked at your mathx unit, and I think it makes sense to include much of that code in VP's int64 unit. Do you have any objection to me doing this? Anything you think I should know or be aware of if I do?

Thanks,
Allan

[Marco van de Voort]

Is it a bug?

I made the following mods trunc64->trunc and commenting the uses line. I also printed RSA and r.

Delphi 2006

Comparing values before and after Trunc64()
rsa:-1.00000000000000E+0000 r: 0
-1 -> -1 Failed!
rsa:-1.00000000000000E+0000 r: 0
-1.00000000000000000000000000000 -> -1 Failed!
rsa:-1.00000000000000E+0000 r: 0
-1.000000000000000000000000000000 -> 0 Failed!
rsa:-2.00000000000000E+0000 r: 0
-2 -> -2 Failed!
rsa:-2.00000000000000E+0000 r: 0
-2.00000000000000000000000000000 -> -2 Failed!
rsa:-2.00000000000000E+0000 r: 0
-2.000000000000000000000000000000 -> -1 Failed!
rsa:-3.00000000000000E+0000 r: 0
-3 -> -3 Failed!
rsa:-3.00000000000000E+0000 r: 0
-3.00000000000000000000000000000 -> -3 Failed!
rsa:-3.00000000000000E+0000 r: 0
-3.000000000000000000000000000000 -> -3 Failed!

FPC:

Comparing values before and after Trunc64()
rsa:-1.0000000000000000E+0000 r: 0
-1 -> -1 Failed!
rsa:-1.0000000000000000E+0000 r: 0
-1.00000000000000000000000000000 -> -1 Failed!
rsa:-1.0000000000000000E+0000 r: 0
-1.000000000000000000000000000000 -> -1 Failed!
rsa:-2.0000000000000000E+0000 r: 0
-2 -> -2 Failed!
rsa:-2.0000000000000000E+0000 r: 0
-2.00000000000000000000000000000 -> -2 Failed!
rsa:-2.0000000000000000E+0000 r: 0
-2.000000000000000000000000000000 -> -2 Failed!
rsa:-3.0000000000000000E+0000 r: 0
-3 -> -3 Failed!
rsa:-3.0000000000000000E+0000 r: 0
-3.00000000000000000000000000000 -> -3 Failed!
rsa:-3.0000000000000000E+0000 r: 0
-3.000000000000000000000000000000 -> -3 Failed!

[Stefan Weber (AH)]

Hi Marco,

now you are forcing me to look at my fix again. :P

Ok, first of all your comparisons all fail because RSa is different from RSb :P
Obviously the result of Str() in VP on numbers without digits after the decimal point
is different from FP/Delphi. That makes the reading quite hard. ;)

Your results show that Delphi has the same bug with -1 and -2 having 30+ digits.
FP doesn't have the bug.
Here it also helps to look at the binary result, I found a nice site which showed
the correct binary results.
http://babbage.cs.qc.edu/courses/cs341/IEEE-754references.html

I assume that FP uses a different algorithm, maybe the same as VP has now.
I also ask Allan to keep the DEFINE in system.pas which makes it possible to switch
back to the old code for whatever reason.

Please also note that the VP system._Trunc() actually truncates to a
32-bit integer, while Trunc64() handles a 64-Bit Integer.
I assume that Delphi and FP have 64-Bit versions already built in.

So, yes, I think it is a bug. :)

Allan, I forgot to add that, if a (const x:TQuad) ever changes to a pointer
to a TQuad, that all code has ot be rewritten for that.
At the moment VP always stores the value on stack.

All the best,
Stefan / AH

[Marco van de Voort]

I could retest this with COMP logic. Maybe that changes, since comp is FPU, and int64 not, it would stress difference part of VAL maybe. I'll pester some Delphi forums with the difference tomorrow, see if they come up with something, and will report back here.

[Stefan Weber (AH)]

JFYI:
TQuad (the VP 64-Bit integer type) is a Comp.
VP doesn't know a *real* int64 type.

[Stefan Weber (AH)]

Hi Allan,

I have no objections at all and hope that someone may find it useful.
All I can say is that most of the code in mathx is actually used in NDNs calculator,
so bugs should be rare.

Please read my reply to Marco for some mroe information on the Val() bug.