Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations SkipVought on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

hi there! this does not work wel 1

Status
Not open for further replies.

bibicu

Programmer
Mar 13, 2002
3
0
0
RO
hi there!

this does not work well at all... (at least in IE6)

alert(12347 - 12345.01)

instead of returning 1.99 (like any well-behaved parser!!!!) it returns some crap (I forgot wich one)

anybody got any clue??

thanks
[thumbsup2]

 
thanks for the tip! :) this one I knew already. the question is:


WHY IS THIS???
 
I find the same kind of problem:

Try this:
var A = Math.floor(8.70*100);
var B = Math.floor(870);
alert("A=" +A+ "\nB=" +B);

Return:
A= 869
B= 870

We meet this problem in C language.
This is seems to be a problem between Float and Integer….
But we didn't find a good solution…

Good luck
 
Hi all,
Nothing strange here, it should be this way. The program returns some values of calculation, and now it's your turn to handle them.

The solution is very simple: round the result to some decimal point. I wrote such a script where you can set the precision. It was posted here:
thread216-205191

good luck
 
It's returning that weird value because the calculation is being done in BINARY rather than decimal, and certain decimal numbers do not have a finite representation in binary. Tracy Dryden
tracy@bydisn.com

Meddle not in the affairs of dragons,
For you are crunchy, and good with mustard. [dragon]
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top