Overflow Error using Integers and Doubles

[EagleTempest]

VB6

I get an overflow error when calculating dblArea:

Private Sub Form_Load()
  Dim dblArea As Double
  Dim intWidth As Integer
  Dim intHeight As Integer
         
  intWidth = 559
  intHeight = 1219
  [b]dblArea = CDbl(intWidth * intHeight / 645.16)[/b]
End Sub

Shouldn't the Double datatype handle the calculated value of 1056.20466?

 

[EagleTempest]

Small change in code to:

dblArea = CDbl(intWidth) * CDbl(intHeight) / 645.16

and VB is happy.

 

[johnwm]

The Double datatype is fine: The problem is that intWidth * intHeight has an implied Integer type and will overflow. You're probably always better to use Longs rather than Integers (they are the native storage size for integers in VB6)

________________________________________________________________
If you want to get the best response to a question, please check out FAQ222-2244 first

'If we're supposed to work in Hex, why have we only got A fingers?'

 

[chiph]

Because multiplication and division can be performed in any order, try grouping your formula to keep the intermediate results from overflowing:

dblArea = CDbl(intWidth) * (CDbl(intHeight) / 645.16)

Chip H.


____________________________________________________________________
If you want to get the best response to a question, please read FAQ222-2244 first

 

[strongm]

If you do a search in this forum you should find my explanations of what VB does with calculations like this. And from those explanations you should find that the only necessary change to prevent overflow and give the correct result in this particular calculation should be:

CDbl(intWidth) * intHeight / 645.16