Rounding numbers in stylesheets

[fawkes]

Does anyone now why the following code rounds erratically?

<xsl:value-of select="format-number(round($originalValue*25.4*100)*0.01,'#.00')"/>

To give you an example,

$originalValue=2.375 unformatted result=60.325 result=60.33
$originalValue=2.875 unformatted result=73.025 result=73.02

What i'm attempting to do is provide a value rounded to the nearest second decimal place.

 

[JontyMC]

It's not the rounding function at fault, its the maths engine. Try:

<xsl:value-of select=".*25.4*100"/>

You get a similar problem with javascript:

http://www.bugnet.com/alerts/bugalert9298.html

Jon

"I don't regret this, but I both rue and lament it.

 

[fawkes]

So , following the link it's a reported problem, but is there a reported solution?

The reason i'm using a paremeter in the equation is that I'm using a template to perform unit conversion so I need to pass different values to the template each time.

 

[fawkes]

After thinking about the answer the bug provides I came up with the following:

<xsl:value-of select="format-number(round($originalValue*25.4*1000) div 1000,'#.00')"/>

Essentially what I need are two decimal places. The bug can provide an error where the correct result of 0.05 can be represented as either 0.05 or 0.04999999. If I muliply by 10 raised to the power of decimal places plus 1 (e.g. *(10^(x+1)) where x=number decimal places), round to the nearest integer and then divide by the same value I enforce rounding of all the nines which creates the solution I'm after.

 

[JontyMC]

Its not strictly a problem, because it behaves as it should. To overcome your problem, you'll need to move the required number of decimal places before you do the rounding. Something like:

<xsl:value-of select="format-number(round($originalValue*25.4*100*100)*0.01*0.01,'#.00')"/>

Jon

"I don't regret this, but I both rue and lament it.

 

[JontyMC]

Lol. Nice.

Jon

"I don't regret this, but I both rue and lament it.

 

[fawkes]

There's a theory about blue tits (birds) perfecting solutions to problems at exactly the same time, morphic resonance or something like that it's called. Seems fitting.