Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations strongm on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

compute with decimal number

Status
Not open for further replies.

cbmstr

Programmer
Feb 9, 2009
4
Hello,

i have some problem with compute with decimal places. I have an amount without a decimal places and a field as minor unit code, where are the number of decimal places.
The compute below doesn´t work, if the minor unit code are greater than 6.
decimal = PIC 9(39V(9)6 COMP-4.

COMPUTE decimal = amount * (10 ** (dcml * -1))

With a floating-point i get the proper value

COMPUTE decimal = amount * (10 ** (dcml * -1)) + 0.0E+00

Can somebody explain that or rather make proposal for a better way ?

Regards

Bernd
 
COMPUTE uses its target variable for computing its immediate results. Therefore, COMPUTE will produce incorrect results if its target value is not defined to properly receive all intermediate values. COMPUTE is a bad idea anyway for this reason (function that is otherwise not immediately evident).

So split this calculation up and use the proper sizes of variables for receiving each intermediate result. Then you should be fine.

We could help you further in doing this, if necessary, but you do not give how you defined "amount" and "dcml".

Measurement is not management.
 
Hello,

thanks.
Below the data definition:

decimal PIC 9(3)V(9)6 COMP-4
amount PIC 9(18) COMP-3
Dcml PIC 9(9).
 
First point: It's usually standard to define values with sign storage (S9(4) BINARY as opposed to 9(4) binary ). This would be the first thing I would do here.

COMPUTE decimal = amount * (10 ** (dcml * -1))

Order of ops - write your code so you split each step.

1) dcml * -1 (this might be the start of a problem) --> INT01
2) 10 ** Int01 --> INT02
3) amount * INT02 --> final result.

To follow the formula as written:
INT01 should be the same size as dcml but signed.
INT02 should be a decimal the size of the maximum of dcml.

I would try to rework the formula as well, but odds are that much can be simplified. If DCML truly has no sign and is always positive, you can apply algebra (n^-3 = 1 / n^3). Furthermore, since you use 10 as a base power (10^-3 = 1 / 10^3 = 0.001), and dcml would have to be severely limited in size for this formula to work *well* (maximum of 18), and is an integer, I would consider other methods than using the power functionality, such as redefinitions.

In other words, in the long run it would be more profitable to rethink your approach.

Measurement is not management.
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top