Number comparison problem 1

[MePenguin]

Can anybody explain the following for me:

CDbl(Int(CDec(87.0290060) * 10000000) / 10000000)<87.0290060 = false

 BUT

CDbl(Int(CDec(87.0290070) * 10000000) / 10000000)<87.0290070 = true

I find this totally bizarre, and it's giving me a headache... %-(

(Oh, and it seems to be crippling my application, as the numbers are taxonomic codes for insect species, and so fixed. The above comparison is used in part of a routine to verify new taxonomic codes when users add species)


Phil

---------------
Pass me the ether.

 

[MePenguin]

... (not only,) but also

Val(CDbl(Int(CDec(87.0290060) * 10000000) / 10000000))<87.0290060 = false

Val(CDbl(Int(CDec(87.0290070) * 10000000) / 10000000))<87.0290070 = false

something to do with MSKB 242933 ?

Phil

---------------
Pass me the ether.

 

[PHV]

Floating point values (ie Single or Double) are approximation ...
For "fixed" number use Decimal and stay away from "real numbers".

Hope This Helps, PH.
Want to get great answers to your Tek-Tips questions? Have a look at FAQ219-2884 or FAQ181-2886

 

[MePenguin]

Thank's PHV. I think I was just starting to realise that.

Could you give me some advice?
I've (stupidly) used double data type fields as primary keys in various places - how much trouble am I going to get into by changing these to decimal?
I know the scale & precision needed for all my values. I also know I refer to these fields in SQL, queries and DAO...

I've used double variables to refer to the fields in code - will this still be OK if I change the data type?

Regards,



Phil

---------------
Pass me the ether.

 

[PHV]

Backup your DB (as usual I guess)
Change the fields definition in table design view
Review all relevant queries SQL code to ensure coercion to decimal
Review all VBA code ...

Hope This Helps, PH.
Want to get great answers to your Tek-Tips questions? Have a look at FAQ219-2884 or FAQ181-2886

 

[MePenguin]

Sound advice. Some long nights ahead I suppose.
But this is not to be one of them.

Thanks, and good night (at least Central European Time)


Phil

---------------
Pass me the ether.