Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations SkipVought on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

Comparing numbers

Status
Not open for further replies.

AOLBoy

IS-IT--Management
May 7, 2002
68
0
0
GB
Hi,
I have a frustrating problem concerning the comparison of numeric values.

I have two values held in:

Dim Val1 As Double
Dim Val2 As Double

On the debugger the values are both the same but when I action the code:-

If Val1 <> Val2 Then
MsgBox (&quot;Error&quot;)
End If

display &quot;Error&quot;
The difference is 7.27595761418343E-12

The values have been derived from an Access database.

Has anybody got any ideas as to where I should start looking in order to resolve this?


 
Do you to use a Double?
Try using a Currency or a Decimal:

Dim Val1 As Decimal
 
Sorry, I meant with the decimal:

Dim Val1 As Double

Val1=CDec(SomeValue)
 
I am a c/c++ programmer and not a huge fan of VB but I think I know whats wrong.
In c/c++ we can’t compare floats or doubles for likeness because of the inaccuracies in the conversion process. Even though you may type in the value 4.5 and store it in a double it will not be stored as 4.5, most likely it will be stored as 4.49999999 OR 4.50000001 or some very small difference.

Also are you sure you have the field that you are reading the data from in your db set to double.

Possible solution: use the “round”(not sure of its name) function on both values before the comparison and Im sure that will work.
----------------------------------------
Friends are generally of the same sex, for when men and women agree,it is only in the conclusions; their reasons are always different.
 
Rounding errors will always cause problems when comparing Singles and Doubles

Try this:
If Val1 - Val2 < .000000001 Then
MsgBox (&quot;Error&quot;)
End If

Choose the value .000000001 size to suit your application
________________________________________________________________
If you want to get the best response to a question, please check out FAQ222-2244 first

'People who live in windowed environments shouldn't cast pointers.'
 
johnwm - you might want to consider using the ABS function since you may not know for sure which of the two values is greater.

If ABS(Val1 - Val2) < .000000001 Then

:) Good Luck
--------------
As a circle of light increases so does the circumference of darkness around it. - Albert Einstein
 
Thank you CC

[smile]
________________________________________________________________
If you want to get the best response to a question, please check out FAQ222-2244 first

'People who live in windowed environments shouldn't cast pointers.'
 
Jonnwm is correct but you may need to tweak his solution for instances where Val2 > Val1 IE

(4 - 5) = -1 < 0.0000001 = True

Regards

 
Thanks very much for all your replies. They all seem to work fine. This is not a case of WYSIWYG.

Regards AOLBoy.
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top