Does anyone know how to overcome the problem javascript and vbscript have with digit precision in long numbers (not long as in data type)? I am trying to work with numbers on the order of 1EE38 and am losing precision because the interpreter only works with 4 byte numbers.
Tek-Tips is the largest IT community on the Internet today!
Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!
-
Congratulations IamaSherpa on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!
Floating Point Precision
- Thread starter o517375
- Start date
To my understanding, numbers are stored as 8 byte numbers.<br><br>But that aside, I have had similar problems but seldom work with anything below four decimals. The rounding problems in JavaScript are essentially inherent. JavaScript is not the best choice of languages for high precision.<br><br>However, I remember reading about the JavaScript rounding problems. I can't find it anymore, but I believe the reference directed the reader to Netscapes site for a script to handle this problem.<br><br>I know Netscapes site is difficult to navigate, but perhaps you can search their for a solution.<br><br>Russ
I found the script to handle precision. It's at:<br><br><A HREF=" TARGET="_new">
- Status
Similar threads
