Tek-Tips is the largest IT community on the Internet today!
Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!
-
Congratulations gkittelson on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!
-
Floating point arithmetic
Thanks Mike, good methods. As I can guarantee that all the numbers that I receive will contain no more than 2 decimal places, I'd just started changing mu program to multiply each number by 100 before comparing them. Thanks, Kevin -
Floating point arithmetic
Hi, Thanks, I forgot about that issue. I remember reading somewhere that even 0.1 is stored as an approximation, as it's impossible to store it exactly. I'll do the comparison in Unix - it seems to work there (will need to check this some more) : #! /bin/ksh93 a=0.01 b=0.05 (( c = a + b ))... -
Floating point arithmetic
Hi, I'm having a problem with comparisons and floating point arithmetic in awk. Some very simple comparisons don't give the correct answer. I've created a simple script to attempt to find the cause of the problem. The script loops through a range of floating point numbers, and performs a...
