Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations Chris Miller on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

Webfocus - need to truncate a decimal

Status
Not open for further replies.

albeaty

Programmer
Apr 29, 2009
2
US
I'm computing a GPA and with 3 decimals it comes to 3.536 and I want it to display 3.53. Everything I do rounds it to 3.54.

COMPUTE SEM_GPA/D12.3 =
IF SEM_EARN EQ 0 AND SEM_COMP EQ 0 THEN 0 ELSE
IF SEM_COMP GT 0 THEN SEM_GP / SEM_COMP;
 
Normally, when displaying less precision than the data has, you'd WANT to round, but that's not the case here.

Using 'D' format actually holds 'as much precision' as possible, rounding on display. So, you don't WANT a 'D' format. You only want to hold 2 decimal places. So, I'd recommend using 'P' format, which holds digits, instead of 'D'.

To round DOWN, you could do the following:
Code:
COMPUTE SEM_GPA/P12.2 =    
IF SEM_EARN EQ 0 AND SEM_COMP EQ 0 THEN 0 ELSE    
IF SEM_COMP GT 0 THEN SEM_GP / SEM_COMP - .005;
If the result of the division was 3.536, the subtraction would give 3.531, which would result in 3.53.

If the result was 3.531, the subtraction would give 3.526, which would round to 3.53.
Storing the result as 'P' format would eliminate any further decimal precision.
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top