I'm pretty sure this math is wrong....

[rtho782]

I'm scraping the screen to obtain some monetary values, annoyingly listed with commas, so I'm reading each "section" and appending them, skipping the commas.

I'm then adding them together and displaying them in a message box.

It seems that Extra basic can't add up!?

This is the code:

                        net=Csng(trim((Sess0.Screen.GetString(6,66,3)+Sess0.Screen.GetString(6,70,3)+Sess0.Screen.GetString(6,74,6))))  
                        pen=Csng(trim((Sess0.Screen.GetString(7,66,3)+Sess0.Screen.GetString(7,70,3)+Sess0.Screen.GetString(7,74,6))))
                        fee=Csng(trim((Sess0.Screen.GetString(8,66,3)+Sess0.Screen.GetString(8,70,3)+Sess0.Screen.GetString(8,74,6))))
                        arr=Csng(trim((Sess0.Screen.GetString(14,66,3)+Sess0.Screen.GetString(14,70,3)+Sess0.Screen.GetString(14,74,6))))
                        arrexvat=(arr/1.2)
                       
                        cust=(net+pen+fee+arrexvat)
   
                        message$=(str$(net)+" "+str$(pen)+" "+str$(fee)+" "+str$(arr)+" "+str$(arrexvat)+" "+str$(cust))  
               
                        MsgBox message$

The output values in the message box on a test run are:

3487.97
0
100
0
0
3587.96997070313

How is 3487.97+100 resulting in a load of decimal places?!

Thanks in advance!

 

[SkipVought]

hi,

When you are performaing FLOATING POINT MATH, there is a known loss of precision. It's a fact that you have to 1) recognize and 2) deal with.

If you want precision to a certain number of decimal places, then make appropriate conversion and use INTEGER MATH.

Skip,
[sub]
Just traded in my old subtlety...
for a NUANCE![/sub]

 

[Lakare]

Hi,

As an alternative to the commas, you can do this

data = Sess0.Screen.GetString(6,66,14)
net=trim(Getfield(data,1,",") + Getfield(data,2,",") + Getfield(data,3,","))

data = Sess0.Screen.GetString(7,66,14)
pen=trim(Getfield(data,1,",") + Getfield(data,2,",") + Getfield(data,3,","))

data = Sess0.Screen.GetString(8,66,14)
fee=trim(Getfield(data,1,",") + Getfield(data,2,",") + Getfield(data,3,","))

data = Sess0.Screen.GetString(14,66,14)
arr=trim(Getfield(data,1,",") + Getfield(data,2,",") + Getfield(data,3,","))

don't know if it is better or worse, just another way to do it. Getfield seperates a string at the character given and without including it.

If you use format, Extra will round for you

cust=format((net+pen+fee+arrexvat),"#.##")

And Skip is right. Be careful when using floating point. You can get some nasty surprises if you don't handle it.

Lakare