Problem with number and decimal sizes

[Brak]

I've noticed half of this problem for sometime, but now it is getting in the way - with the new problem.

The first problem is that Fox seems to ignore any decimal place past 15. So if I do the following:
? 0.1234567890123456
I get:
0.1234567890123460

If I do:
? 0.123456789012345678
I get:
? 0.123456789012346000

And yes I have decimals set to 18.

Here's another problem, If I do:
? 1234567890.123456789
I get:
? 1234567890.123457000

Also if I do:
? 1234567890123456789
I get:
? 1234567890123457000

So the proble is that Fox is ignoring anthing more that 16 numbers, and is rounding the 17 number and replacing the rest with 0s.
So why if Fox doing this, what can be done to fix it?

I am using FPW2.6a with the fast machine patch installed.


Thanks in advance.

 

[mmerlinn]

[ ]
There is a lot of discussion about this in Tek-Tips. Do a search.

Basically no one has yet to invent a 5-way switch, so ALL decimal computations REQUIRE rounding. FoxPro does math out to about 18 significant digits, then rounds back to about 15 significant digits for math precision.

Nothing can be done about this directly. It is a built-in problem with ALL computer computations on all computer and all programs. The only difference is that some computers and programs do the math to more significant digits and some to fewer significant digits. Regardless, ALL decimal math in ALL programs REQUIRES rounding.

When you need precision beyond the capabilities of the program, which you obviously need, this rounding produces what you think are 'errors'. These are not errors. The program is computing correctly. It is just that you are expecting the program to do what it is NOT capable of doing and can NEVER be capable of doing.

Since you need greater precision than 15 digits, you will need to use a program that splits your number into smaller chunks, does the math on those small chuncks, and then reassembles the small chunks back into a long number. There are programs available that will do that. A number of TT listings will point you in that direction. You just need to find them by searching TT.


mmerlinn

"Political correctness is the BADGE of a COWARD!"

 

[Brak]

I did search the Fox forum on this before I posted and didn't find a thing. If you know of these threads please let me know so I can check them out.

Heere's the thing that don't make sense, Fox has a SET DECIMAL command that allows you go control the number of decimals displayed (18 being the max) but yet only uses 15, making the other 3 digits 0s. What is the point!?!? Either Fox should use all 18, or only allow the SET DECIMAL command to go up to 15.

But outside of the decimal issue - what about the whole numbers? As I mentioned if I type this:

? 1234567890123456789
I get:
1234567890123457000

So why is Fox messing up my whole numbers???

It seems to be a number of digits problem - not a decimal one. IT seems Fox only shows properly 16 digits.
0.123456789012345
and
1234567890123456

If you try using any more digits they become 0s.

Very weird.
I get
123456789012346000

 

[alvechurchdata]

1234567890123456789 is beyond the range for integers. The 32-bit environment limits you to the range -2147483647 to 2147483647. As mmerlin said, the computer just cannot store a number beyond this degree of precision. All that it can do is round it to the nearest floating point value. If you really do need to do arithmetic with these very large numbers then you might do better looking at other languages which have special routines to manipulate very long strings of digits.

Geoff Franklin

http://www.alvechurchdata.co.uk

 

[Brak]

I understand what you are saying - but I don't think that is what is going on. Let me explain why.

You state that 32-bit intergers have a range of -2147483647 to 2147483647. I understand but Fox has no problem dealing with
1234567890123456! So if I type:
? 1234567890123456+1234567890123456
I get 2469135780246912 - which is right. Do of the issue was the -2147483647 to 2147483647 limitation then that calc wouldn't work - but yet it does.

Again the problem seems to be Fox can't display numbers with more than 16 digits for all the following variations work:
1234567890123456+1234567890123456
1.234567890123456+1.234567890123456
12.34567890123456+12.34567890123456
123.4567890123456+123.4567890123456
1234.567890123456+1234.567890123456
12345.67890123456+12345.67890123456
123456.7890123456+123456.7890123456
etc..... all the way to:
123456789012345.6+123456789012345.6

Also all this does answer the questions as to why Fox gives the option of 18 decimal places when it can't handle that many.

 

[Brak]

Oh, just as a point of clairification:
When I type 1234567890123456, it is just of the sake of seeing easily the number of digits. In actuality the number can be 9999999999999999 (16 9s).

Also the problem also exists with stored numbers in DBFs. Fox allows you to have a 20 digit number - but it only stores the first 16 of the them right - replacing the other 4 with 0s.

I hope that its clear the my main problem is with Fox giving options that don't exist. If Fox is only supposed to handle 16 digits WHY give the option for 20 in the DBF and give the option of 18 decimal places. If Fox is supposed to only handle 16 digits than the SET DECIMAL should only give the option of up to 15, and the DBF should only give the option of 16 digits.
Since these options are given it would only make sense for those options to be available.

Befuddling!

 

[mmerlinn]

[ ]
Number of decimal places has no relation whatsoever to the number of significant digits nor to the precision of those digits. I don't know how FP handles the decimal position, but theorically they could allow thousands of zeroes AFTER the decimal point and BEFORE the FIRST significant digit without affecting the precision of the number in any way. Or, for that matter, allow thousands of zeroes AFTER the LAST significant digit and BEFORE the decimal point with no bearing whatsoever on the precision of the number.

To get a better handle on this you might want to do a google search for 'significant digits'. I didn't do one, but there should be lots of information on what they are, why they are important, and how they work. This is an issue that scientists have had to grapple with for centuries, long before computers ever existed.

Some threads about this issue: thread184-1290417 thread184-1278669 thread184-531793.


mmerlinn

"Political correctness is the BADGE of a COWARD!"