Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations Mike Lewis on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

double * double = imprecise result

Status
Not open for further replies.

rnrn

Programmer
Aug 23, 2002
8
0
0
GB
I have a java class that multiplies one double with another. The result is sometimes imprecise, for example
7.2 * 21 gives 151.20000000002 when it should be 151.2
and
1.2 * 9 gives 10.799999999 when it should be 10.8.

Can anyone tell me why this is so and what datatype I should use to ensure precision in arithmetic?
 
Have a look in the java API documentation at
Code:
BigDecimal
for
"Immutable, arbitrary-precision signed decimal numbers. A BigDecimal consists of an arbitrary precision integer unscaled value
and a non-negative 32-bit integer scale, which represents the number of digits to the right of the decimal point. The number
represented by the BigDecimal is (unscaledValue/10scale). BigDecimal provides operations for basic arithmetic, scale
manipulation, comparison, hashing, and format conversion"

This may help. -------------------------------------------
There are no onions, only magic
-------------------------------------------
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top