Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations strongm on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

Ha, wierd. (2.3 - 2) <> (.3)

Status
Not open for further replies.

thesleepylizard

Programmer
Mar 18, 2002
41
AU
debug.print Format(DateAdd("s", (120) + ((.3) * 100), Date), "h:nn:ss") outputs:

0:02:30


debug.print Format(DateAdd("s", (120) + ((2.3 - 2) * 100), Date), "h:nn:ss") outputs:

0:02:29



El-wierd-o. I can get around it, it's just a curiosity. Probably a Double type rounding error at the 40th decimal point etc or something.
 
Not odd at all. Just a another example of the fact that a computer cannot hold an exact representation of a floating point number

Another example might be:

Dim lp As Double
lp = 1.06
Debug.Print lp - 1
 
Figured that.

What was wierd was that 2.3 - 2 was only reproducable with the code as above, ie .if i removed dateadd() it seemed to work OK.

Anyway, no biggie.
 

I try to use CDec() on numbers before converting then to dates.

CDec(120 + ((2.3 - 2) * 100))

(CSng() will not always work either, and CCur() is not enough accuracy)
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top