Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations strongm on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

ROUNDING INSTEAD OF PRECISION 2?

Status
Not open for further replies.

itprsn

MIS
Jan 6, 2004
40
US
Having no luck trying to get precision 2 data in a stored procedure. I've tried variations of the following logic with no success. Can someone help? Thanks...

SELECT
CONVERT(DECIMAL(10,2),SUM(PRD.Pages*GrossFolder)/1000) MGP
FROM...
 
If you want to return 2 digits of precision, you need to use Decimal(x,2). You can round the number before converting it though.

Ex:

Code:
[COLOR=blue]declare[/color] @Blah [COLOR=blue]Float[/color]
[COLOR=blue]Set[/color] @Blah = 1234.5678

[COLOR=blue]Select[/color] [COLOR=#FF00FF]Round[/color](@Blah, 2)
[COLOR=blue]Select[/color] [COLOR=#FF00FF]Convert[/color]([COLOR=blue]Decimal[/color](10,2), [COLOR=#FF00FF]Round[/color](@Blah,2))

Returns....

[tt][blue]

-----------------------------------------------------
1234.5699999999999

(1 row(s) affected)


------------
1234.57

(1 row(s) affected)

[/blue][/tt]

Make sense?

-George

"The great things about standards is that there are so many to choose from." - Fortune Cookie Wisdom
 
SUM(PRD.Pages*GrossFolder)/1000

are Pages and grossfolder fields also integer? YOu may be suffering from integer math issues. Try 1000.000 instead of 1000

"NOTHING is more important in a database than integrity." ESquared
 
Apparently so - 1000.0000 worked. Thanks!
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top