Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations Chris Miller on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

zero values show as large negative exponents??

Status
Not open for further replies.

techseek

MIS
Nov 5, 2010
97
US
Hello
I have a macro/sub that prepares a report. when done "manually"
some columns (which have cells in "general" format) will end with zeros in them - I expect that and they show as such.
However, when using the macro, some of those zero values show as 0 and some show as large negative exponents. I found that if i convert the format to number, then the negative exponents go away.
I'd like to know why this happens and if in fact, changing the format to number is the solution.

Thank you
 
Usually, this behavior is attributable to the way numerical calculations are done. Due to numerical "slag", what would analytically be "0" is numerically like 0.000000000000000001.

_________________
Bob Rashkin
 


Do some research regarding floating point arithmetic, as opposed to integer arithmetic.

Please describe your data and the variables you have declared in your procedure. And please post your code.

If you have not declared variables, your variables are VARIANTS, which would result in the floating point precision behavior you have described.

Skip,

[glasses]Just traded in my old subtlety...
for a NUANCE![tongue]
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top