working with floating points in AWK

[snogales]

Hi!

I am trying to make a program, for which I need high precision. My problem is that I do not understand how AWK is working, I mean:

echo 1.00001E-6 | awk -v FS=" " -v D=0 'D=D+$1 {printf("%.12g\n",D)}'

1.00001e-06

(I obtain this result from %.06g to %.17g)
but:

echo 1.00001E-6 | awk -v FS=" " -v D=0 'D=D+$1 {printf("%.18g\n",D)}'

1.00001000000000002e-06

??????

and these numbers can change my results.

thankx in advance

sergio

 

[PHV]

Anyway a float (double real) has a max precision of less than 16 digits and is only an approximation.
For arbitrary precision computing have a look at dc and/or bc (in unix).

Hope This Helps, PH.
Want to get great answers to your Tek-Tips questions? Have a look at FAQ219-2884 or FAQ181-2886