Comparing values

[AOLBoy]

I have a strange situation and I do not what to do about it.

I have two fields

Dim A As Double
Dim B As Double

In my code I am accumulating values from Excel cells and then I am doing a comparison of A and B.

With the 'Add Watch' facility in the debugger I have
A = -3263.18
B = -3263.18

In my code the following check returns true:

If A <> B

I added a new field x = A - B
and this contains the value

4.54747350886464E-13

This makes no sense to me

Can anybody please help me out here? The code works OK in a VB6 application.

 

[royalcheese]

if you format "a" and "b" (look at the format function) then compare this issue should disappear , the number shown is very small

 

[royalcheese]

attually the round function is more applicable , (sorry it is early in the morning , not woken up yet)

 

[Bong]

I think it depends on how the numbers are generated and why you're comparing them. Do they really have to be rigorously equal, or is there a tolerance on your comparison. That is, you can compare using: if A-B<tolerance ....

If the numbers are generated from calculations, and gathered in different paths, they can, actually, be different due to truncating and "machine noise".

_________________
Bob Rashkin

 

[SkipVought]

The PRECISION of Double is not precise as you have experienced.

Best to use integer arithmetic OR establish a tolerance in your expression...

Const TOLER = .0001

if Abs(a - b) < TOLER then

Skip,
[sub]
[red][/red]
[/sub]

 

[AOLBoy]

Thanks very much for all your comments and setting me straight on this issue.