Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations IamaSherpa on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

Rounding Error Problem Due to Different Grouping

Status
Not open for further replies.

earthling16

Programmer
Sep 25, 2001
1
SG
I have this set of summarized data in place :

Group Unit of Measure(UOM)
------------------------------------
A 16.5852 (1)
B 16.8321 (2)
Total 33.4173 (3)

(1) and (2) are derived from the formula :
SUM ({@UOMCalculation},{@Group})
(3) is derived from the formula :
{@GlobalVarUOMCal}- SUM ({@UOMCalculation},{@Group})
where @GlobalVarUOMCal =
global numberVar UOM := UOM + Round(Sum({@UOMCalculation}, {@Group}), 4)


There's no problem in the above. However If there's only one group, then the result will be different due to rounding error as shown below

Group Unit of Measure(UOM)
------------------------------------
A 33.41736
Total 33.4174

I'm not sure if anyone of you understand where the problem lies. The end result that I would like to achieve is to be able to have consistency in the figure irregardless of how the data are being grouped.

Any suggestion or assistance in solving this problem would be greatly appreciated.

N.B. I'm using Crystal 8.


Thanks in advance
earthling16
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top