Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations Chriss Miller on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

How to avoid floating-point rounding error in CF? 1

Status
Not open for further replies.

philcha

Programmer
May 10, 2000
109
GB
The answer to this could become FAQ of the month!<br><br>I'm developing a book-keeping app in CF and am currently&nbsp;&nbsp;working on the Trial Balance.<br><br>Everything's fine until I check whether the accounts balance.&nbsp;&nbsp;Then, on my test database:<br>a)&nbsp;&nbsp;&nbsp;(A - B - C + D) gave a non-zero result, i.e the accounts don't balance although the sub-totals A, B, C, and D tell me they do!<br>b)&nbsp;&nbsp;&nbsp;(A - B - (C - D)) gave a zero result<br>although the 2 formulae are logically the same and A, B, C and D display correctly when formatted to 2 decimal places by the DecimalFormat function.<br><br>When I displayed the result of formula (a) without any formatting I saw -9.09494701773E-013, so it's a floating-point rounding error, i.e. A, B, C and D (sub-totals) are themselves not accurate to 2 decimal places.&nbsp;&nbsp;In that case with different data formula (a) might give the right answer and formula (b) the wrong one.<br><br>Unfortunately CF's Round function rounds to the nearest integer and does not have an option to round to 2 decimal places.<br><br>Shopping cart applications must have a way of getting this kind of totalling and sub-totalling right.<br><br>Does anyone know how to avoid the problem?<br><br>Background info:<br>*&nbsp;&nbsp;&nbsp;A, B, C and D are themselves the sums of various sets of monetary values.<br>*&nbsp;&nbsp;&nbsp;All monetary values are from an Access DB and are of data type &quot;currency&quot;.<br>*&nbsp;&nbsp;&nbsp;I'm using CF 4.01 on Win 98 with Personal Web Server for testing.<br>
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top