Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations biv343 on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

trunc causing an error

Status
Not open for further replies.

csbdeady

Programmer
May 18, 2002
119
GB
Hi

When using the following we get the output of 37.75 when we expect 36.76. Does anyone know why? I've noticed the same in VB 5 (maybe 6?)

trunc(36.76*100)/100

Adding extra brackets or using the result of a function such as abs has the same result.

Ta
-Colin
 
Thanks

In Delphi that doesn't work. In VB when I save before running it , the 100.0 gets replaced with 100# .

Oh well!
 
The only alternative I could find (quickly) is to use round() instead of trunc(), also, putting 36.761 in the formula gives the correct result.

Well.....
After fiddling a bit with trunc and round, I looked up the trunc function in the sourcecode. It is mentioned there, in the system unit, that 'compiler magic' is to be performed on some routines, including trunc, sin and some more. Then, after seeing that trunc is implemented in quite simple asm code, (I can't really read it, wrote some long tome ago, but have seen the hang of it) I concluded that it might be that by the way of handling parameters there, it is possible that sometimes a bit falls off of a value. Replacing the requested function with:
Code:
var d : double; h : integer;
...
h := 100;
d := 36.76 * h;
result := trunc(d) / h);
gives the correct result, and changing d from double to extended (the default parametertype for trunc) gives me back the *wrong* result again! s-) So I conclude that my conclusion on trunc is correct, passing specific values as type extended to trunc can chop off the wrong bits...

Best workaround would be to first place the value in another 'real' type than extended, then apply the math.

HTH, TonHu
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top