Floating point arithmetic 1

[kshewitt]

Hi,

I'm having a problem with comparisons and floating point arithmetic in awk. Some very simple comparisons don't give the correct answer.

I've created a simple script to attempt to find the cause of the problem. The script loops through a range of floating point numbers, and performs a comparison. The results of the script are as follows:

0.02 = 0.01 + 0.01
0.03 = 0.01 + 0.02
0.04 = 0.01 + 0.03
0.05 = 0.01 + 0.04
0.06 != 0.01 + 0.05 (FAIL)
0.07 != 0.01 + 0.06 (FAIL)
0.08 = 0.01 + 0.07
0.09 = 0.01 + 0.08
0.1 != 0.01 + 0.09 (FAIL)
0.11 = 0.01 + 0.1
0.12 = 0.01 + 0.11
0.13 = 0.01 + 0.12
0.14 = 0.01 + 0.13
0.15 != 0.01 + 0.14 (FAIL)
0.16 = 0.01 + 0.15
0.17 = 0.01 + 0.16
0.18 != 0.01 + 0.17 (FAIL)
0.19 = 0.01 + 0.18
0.2 = 0.01 + 0.19
0.21 != 0.01 + 0.2 (FAIL)
0.22 = 0.01 + 0.21
0.23 = 0.01 + 0.22
0.24 != 0.01 + 0.23 (FAIL)
0.25 = 0.01 + 0.24
0.26 = 0.01 + 0.25
0.27 = 0.01 + 0.26
0.28 = 0.01 + 0.27
0.29 != 0.01 + 0.28 (FAIL)

.. and this is the script..

#! /bin/ksh93
vat=0
net=0.01

while [ $vat -lt 11 ]
do
(( vat = $vat + 0.01 ))
(( total = $net + $vat ))

printf "$total\t$net\t$vat\t" > /tmp/x.tmp

result=`awk -F" " '
{
v_total = $1
v_net = $2
v_vat = $3
if ( v_total == v_net + v_vat ) {
output=v_total " = " v_net " + " v_vat }
else {
output=v_total " != " v_net " + " v_vat " (FAIL)" }
}
END { print output }
' /tmp/x.tmp`

echo $result

done

done


I'd really appreciate any advice anyone could give. This problem is driving me crazy.

Thanks,

Kevin

 

[PHV]

In my opinion this is standard behaviour for floating point arithmetic as the value of a double real is only an approximation ...

Hope This Helps, PH.
Want to get great answers to your Tek-Tips questions? Have a look at FAQ219-2884 or FAQ181-2886

 

[kshewitt]

Hi,

Thanks, I forgot about that issue. I remember reading somewhere that even 0.1 is stored as an approximation, as it's impossible to store it exactly.

I'll do the comparison in Unix - it seems to work there (will need to check this some more) :

#! /bin/ksh93
a=0.01
b=0.05
(( c = a + b ))
echo "c=$c"
if [[ $c = 0.06 ]]
then
echo "Match"
else
echo "No match"
fi

Kevin

 

[Mike042]

I agree with PHV. The problem is that a floating point number displayed as 0.02 could in reality be 0.020001 or even 0.019998 ! (Only when it is made readable by humans as a 2 decimal place floating point number is it 0.02) This makes it difficult to use comparisons which are probably better for testing integers. However, some of the techniques I use to test floating point numbers are:
1. Convert to integer (or scaled integer) before comparison.
2. Test against a range of values, eg: 'less than or equal to' (for not greater than) and 'greater than or equal to' (for not less than).
3. For testing equality, try this. If the floating point number is 'greater than, but not equal to' 0.01 and 'less than, but not equal to' 0.03 then it must be 'equal to' 0.02

I hope that helps.

Mike

 

[kshewitt]

Thanks Mike, good methods.

As I can guarantee that all the numbers that I receive will contain no more than 2 decimal places, I'd just started changing mu program to multiply each number by 100 before comparing them.

Thanks,

Kevin

 

[Ogzilal]

Hi,

Even if you multiply by 100, you will still have some FAILed comparaison because you only shift the decimal point 2 positions but the inifinite number of decimals is still present because of the floating point representation in binary digits.
One way to eliminate all the FAILed is to limit the number of digits to a fixed value replacing your thre instructions by this ones :

              v_total = sprintf("%.6s", $1 * 100 )
              v_net   = sprintf("%.6s",$2 * 100 )
              v_vat   = sprintf("%.6s",$3 * 100 )

Then, you will have v_total, v_net and v_vat fixed length of 6 digits max and decimal point is discarded by multiplying by 100

You can fix the length as you like depending on the knowledge you have about your values. "%.6s" is an example here