real number decimal prob.? 1

[Dragonsoft]

In our postscript program we have prices that sum up in a total value. Each price is also printed.

When the total and the price is summed we get a automatic round if the value has more than 9999 and two decimals.


/print_adprice
{
/adprice exch def
adprice 10 string cvs dup
/total_price total_price adprice add def
adtext.cfont fontchange
stringwidth pop adprice.right exch sub adtext.curtop m
show
} bind def


31220.01 print_adprice
this call produces in our form: 31220.0

If we change the value to this:
3122.01 print_adprice
this call produces in our form: 3122.01
and that is what we was expecting whith the value 31220.01


The problem is when the values are shown as strings there is only one decimal and when the values are summed the decimal is rounded to ceil, so .01 == .1?

the total_price has incorrect value because of the rounding, i dont know postscript that well and dont know if the error comes when the values are passed and converted to strings or if threre is a limit in the real datatype or mabye ghostscript cant handle the values????

Weve tried to use whole integers and when we sould print them divide with 100 to shift the decimal, but we get the same result as above.

Any remark is welcome.
Best regards
/Olof Lager.




"Some thoughts has a certain sound..."

 

[tgreer]

I hate this about PostScript. While you can have numbers accurate to about 8 decimal places, each interpreter can do what it wants with regard to precision when performing calcuations with those numbers.

Meaning, you can start with accurate numbers, but as soon as you want to DO ANYTHING WITH THEM, including "add" and "cvs" or even "==", the accuracy defenestrates (that's Latin for "flies out the window").

Keep in mind that PostScript usually uses numbers to specify tints of color, or positions on a page, where such precision as "0.01" doesn't come into play, or if it did, would be visually insignificant.

The bottom line: do your calculations in the program that produces your PostScript, and only output the final, computed numbers as strings in the PostScript.

Final Note: precision does indeed vary from one PostScript interpreter to another. For example, Acrobat Distiller is horrible and doesn't retain much precision in calculations. PSAlter (a PostScript debugger) does a very good job in retaining precision. GhostScript does not do well.



Thomas D. Greer

http://www.tgreer.com

Providing PostScript & PDF
Training, Development & Consulting

 

[tgreer]

Through testing, it appears that decimals are retained in proportion to the whole number component.

For example, compare:

500.001 499.001 add ==

to:

500.001 500.001 add ==



Thomas D. Greer

http://www.tgreer.com

Providing PostScript & PDF
Training, Development & Consulting