Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations Mike Lewis on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

CALCULATE STD(MyField) gives wrong result

Status
Not open for further replies.

papadopo

Programmer
Mar 11, 2003
10
0
0
CY
Hello everybody,

I use VFP 7

I have a table MyTable.DBF with a field MyField N(15,4) with values

23334.9000
23334.9000
23334.9000

I go:

Code:
SET DECIMALS TO 4

SELECT MyTable
CALCULATE STD(MyField) TO X

VFP gives as result x = *******************

instead of 0

the really weird thing is that when i change the values to

23334.9100
23334.9100
23334.9100

I get the correct result

Another thing is that if i go

SET DECIMALS TO 2

the STD works fine with these specific values (23334.9000)

Have you heard of anything about this sort of thing? Is this a bug?

I really need to understand what other case there might be that give wrong results, and what if

SET DECIMALS TO 2

gives wrong results to some other values

thanks alot

papadopo

----------
"Never Send a Human to do a Machine's Job"
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top