Hi everyone, esp Allan.
I just have fixed a rare bug that occurs when using Val() to an Extended type with 30+ digits.
This bug occured only when truncating this value to a 64-bit integer.
NDN's internal 64-bit calculator has showed this, and thanks to the help of a helper named CandyMan
I was able to fix the bug in the VP RTL.
I have written a test program to show the problem and also have added a bad and fixed W32
executable to that test archive (testval).
Here's the shortened output to show the actual problem:
Comparing values before and after Trunc64()
-1 -> -1.000000000000000E+0000 Passed!
-1.00000000000000000000000000000 -> -1.000000000000000E+0000 Passed!
-1.000000000000000000000000000000 -> -0.000000000000000E+0000 Failed!
Please note the number of input zeroes.
Trunc64() is a simple frndint with rounding towards zero Please see mathx.pas for
that function and several other 64-bit Comp/TQuad/QuadWord functions.
Please look into system.pas for FLOAT_VAL_FIX to find all changed code.
I don't want to remove the old code since it may be helpful to see the differences.
I compared the binary output of the actual values with the old and new implementation and
in the error case they look totally different, I guess this is some sort of precission error or
something.
Anyway, I hope I have supplied all necessary code with this post.
All the best,
Stefan / AH