Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations strongm on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

Scientific Notation 1

Status
Not open for further replies.

wonk65

MIS
Apr 24, 2002
2
US
I am a novice
qbasic programmer and have run into a problem with a legacy system which I inherited. I was hoping someone might be able to give me some direction.

result# = VAL(result$)
result# = result# / ps#
result$ = LTRIM$(STR$(result#))
result$ = LEFT$(result$, 8)

ps# is a value between (.01) and (.99).
I am having trouble with the result# not converting out of scientific notation when the VAL(result$) is at or below the (ps# / 10). The ouput for the final "result$" has the literal value and not the value converted back to a standard decimal number. The error does not occur on a consistent basis....sometimes the conversion works and sometimes it does not....But it always works when the VAL(result) is > the ps#. I cannot see a specific pattern.

I greatly appreciate any input anyone might have.
Thanks in advance.
 
I can only reproduce your result for extremely small values of result# (less than 10^-15). If you frequently need to turn such values into text representations, you may need to scale the numbers up to fit them into the desired representation. The following code may help:
[tt]
FUNCTION lstr$(n#)
IF n# < 0# THEN
lstr$ = &quot;-&quot; + lstr$(-n#)
ELSEIF n# > 1# THEN
lstr$ = LTRIM$(STR$(n#))
ELSE
a$ = &quot;.&quot;
m# = n#
WHILE m# < .1#
a$ = a$ + &quot;0&quot;
m# = m# * 10#
WEND
lstr$ = a$ + MID$(STR$(m#), 3)
END IF
END FUNCTION

[/tt]
This function makes no attempt to correct extremely large values (greater than 10^15) going into scientific notation, but from your description this shouldn't be a problem.
 
agual, as far as I understand it, he doesn't want the scientific notation. He wants to see 0.00000000000000001 instead of 1D-16. However, your suggestion of using [tt]PRINT USING[/tt] can still be applied, if he's using [tt]PRINT[/tt] to display the values. I provided a more generic solution since he was simply placing the text representation into a string variable and did not indicate what he was doing with it.
 
Logiclrd,
Thanks for the assistance. I implemented the code you suggested and it worked. I have been beating my head against a wall off and on for the last few days. Thanks for the aspirin! Thanks also to agual for your suggestion.
I am thoroughly impressed with the speed of response and the help with this thread.
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top