Spurious decimals 3

[simonkue]

I have a problem with maths!

I am inputting a numeric into a text control on a form (eg "12.34&quot.
I can take the integer portion via:

dim xnInt as Integer
xnInt=CInt(text1.text)

To get the decimal portion I do this:

dim xnWrk as Double
dim xnDec as Double
xnWrk=CDbl(text1.text)
xnDec=xnWrk-xnInt

When I use 12.34 as the value, I correctly get 12 for Int and 34 for Decimal.
When I use 12.345, I get 12 for Int, but 345000000000001 as the decimal.

As a further test, if I add the decimal portion to the integer portion, it always comes back to the starting figure - even though I don't see how!
eg
Int 12 + dec .34 = 12.34
Int 12 + dec .345000000000001 = 12.345

Anybody have any ideas as to why?

 

[MrPtt]

Hi

Try:

s_Numbers = Split(text1.text, ".&quot
s_Integer = s_Numbers(0)
s_Decimal = "0." & s_Numbers(1)

Good luck!
;-)

 

[zemp]

Try declaring your decimal variable as type Single.

dim xnInt as Integer
dim xnWrk as Double
dim xnDec as Single

xnInt=CInt(text1.text)
xnWrk=CDbl(text1.text)
xnDec=xnWrk-xnInt

Take Care,

zemp

"Show me someone with both feet on the ground and I will show you someone who can't put their pants on."

 

[ClickHere]

Looks like a good place to use the Decimal data type.
Take a look at thread222-537500

Have a great day!

 

[CCLINT]

Also:

CInt() will round!
This may not be in your best interest.
So, CInt(12.545) with return 13.
Use the Int() function instead.

 

[CCLINT]

Using the Split() function is not usually the recommended method for something like this.
To use this method, you would also need to query the system to find out what decimal separator is being used.

Imagine if the user sets the decimal seperator in the region settings to something different than a dot, which in very many other countries the case.

 

[simonkue]

Thank you all for your responses; however, I still don't know why my code doesn't work.

I have workout out a solution using some of the above, but I would like to know why the theoretically correct code above doesn't work in all cases.

Anyone have any ideas?

 

[zemp]

One reason could be that single and double data types are floating point numbers. These are estimations, by definition, and are not exact values.

To be exact you may need to use a decimal data type. However, you will find this in the help files,

"At this time the Decimal data type can only be used within a Variant, that is, you cannot declare a variable to be of type Decimal. You can, however, create a Variant whose subtype is Decimal using the CDec function."

With that in mind you can try this,

Dim xnInt As Integer
Dim xnWrk As Double
Dim xnDec As Variant

xnInt = Int(Text5.Text)
xnWrk = CDbl(Text5.Text)
xnDec = CDec(xnWrk - xnInt)

At least that is what I understood, and it seems to work.



Take Care,

zemp

"Show me someone with both feet on the ground and I will show you someone who can't put their pants on."

 

[AndyWatt]

Ultimately it is because decimal (real) numbers cannot be represented accurately in binary. Then there are rounding errors because a fixed number of decimal places cannot represent an arbitrarily long decimal. (e.g. Pi or e)

Andy
"Logic is invincible because in order to combat logic it is necessary to use logic." -- Pierre Boutroux
"Why does my program keep showing error messages every time something goes wrong?"

 

[CCLINT]

>xnDec = CDec(xnWrk - xnInt)

You will need to go a step further, otherwise you will have expected results only part of the time (try Text5.Text = 12.645)


xnDec = CDec(xnWrk) - CDec(xnInt)

 

[zemp]

CCLINT, using 12.645 and my method got the results I was after

xnInt = 12
xnDec = 0.645

But if I tried 12.899, I got

xnInt = 12
xnDec = 0.89899999

Switching to your suggestion with 12.899 returned

xnInt = 12
xnDec = 0.899

Thanks for the helpful pointer, you have earned another star. What I have now is

   Dim xnInt As Integer
   Dim xnWrk As Double
   Dim xnDec As Variant
   
   xnInt = Int(Text5.Text)
   xnWrk = CDbl(Text5.Text)
   xnDec = CDec(xnWrk) - CDec(xnInt)

Take Care,

zemp

"Show me someone with both feet on the ground and I will show you someone who can't put their pants on."

 

[CCLINT]

Thank you zemp!

(I had just pulled a number out of my head w/o testing it...I thought this would be one to fail...12.655)

 

[simonkue]

Thank you all again!

AndyWatt, forgive my ignorance, but I thought that all numbers could be represented in binary terms - provided you had enough digits?

I didn't think that 12.345 was a difficult number to represent. Certainly, Pi as a recurring number presents a challenge, but whenever I've used it, I set it as a constant and defined the required decimals myself. In this way, VB can create the number with the required accuracy - or so I thought!
What concerns me is whether I can rely on a single or double to contain what I put into it - every time.
The calculations I'm doing often use 5 or 6 decimal places and I'm concerned as to the reliability of the results because I don't understand why this is happening.

 

[AndyWatt]

simonkue

As an attempt to illustrate:

Create a new project, add a multiline textbox and then paste this in:

Private Sub Form_Load()
  Dim sngX As Single
  Dim lngC As Long
  
  sngX = 0#
  For lngC = 1 To 1000
    sngX = sngX + 0.001
    Text1.Text = Text1.Text & sngX & vbCrLf
  Next lngC  
End Sub

You will note that it has a real (no pun intended) hard time with some values.

If you're dealing with money the best thing to use is the Currency data type which only pretends to be a four decimal place value - it is actually a 64-bit integer with a decimal point "inserted".

Hope this helps, or at least doesn't obfuscate even further.

Andy
"Logic is invincible because in order to combat logic it is necessary to use logic." -- Pierre Boutroux
"Why does my program keep showing error messages every time something goes wrong?"

 

[simonkue]

Andy,

Hmmmmm. I tried this and am even more concerned by what you're now showing me!!!
I need accuracy in decimals as I am calculating numbers (velocities, weights, forces etc with no sign of currency) with lots of significant decimal places.

Why is this happening? Am I missing a Service Pack (VB6 SP5 on Win2K) or does VB do this for free?!

Thank you for your persistence.

 

[AndyWatt]

I think it's something we all have to live with. It's the price we pay for discrete rather than continous data. If you can get your values into the Currency range (by multiplying by powers of ten, which if course you'll need to keep track of...) then you should be okay.

I tried the above example using a double instead of a single and thought it was working, but it too packs up and gets inaccurate, so if Currency doesn't work I really don't know what to suggest.

Andy
"Logic is invincible because in order to combat logic it is necessary to use logic." -- Pierre Boutroux
"Why does my program keep showing error messages every time something goes wrong?"

 

[simonkue]

Thanks for all your help.

 

[AndyWatt]

You're welcome.

Andy
"Logic is invincible because in order to combat logic it is necessary to use logic." -- Pierre Boutroux
"A computer program does what you tell it to do, not what you want it to do." -- Greer's Third Law