Overflow error in MD5 script

[waiterm]

Hi I'm integrating an MD5 script within a login page and for some reason when I moved the site from the testing environment to the live server I started getting an Overflow Error whenever someone tries to login:

Microsoft VBScript runtime error '800a0006' 
Overflow: 'lResult' 
/***/***.asp, line 163

The function which this is within is as follows (line 163 is below my debug line, lResult = lResult Xor &H80000000 Xor lX8 Xor lY8):

Private Function AddUnsigned(lX, lY)
    Dim lX4
    Dim lY4
    Dim lX8
    Dim lY8
    Dim lResult
 
    lX8 = lX And &H80000000
    lY8 = lY And &H80000000
    lX4 = lX And &H40000000
    lY4 = lY And &H40000000
 
    lResult = (lX And &h4FFFFFFF) + (lY And &h4FFFFFFF)
   	If lX4 And lY4 Then
		debug(lResult & "*" & &H80000000 & "*" & lX8 & "*" & lY8)
		lResult = lResult Xor &H80000000 Xor lX8 Xor lY8
    ElseIf lX4 Or lY4 Then
        If lResult And &H40000000 Then
            lResult = lResult Xor &HC0000000 Xor lX8 Xor lY8
        Else
            lResult = lResult Xor &H40000000 Xor lX8 Xor lY8
        End If
    Else
        lResult = lResult Xor lX8 Xor lY8
    End If
 
    AddUnsigned = lResult
End Function

Please let me know if you need any more code or have any suggestions as to what might be causing the problem, as I said before, it has only stopped working since moving to the live server environment.

Many thanks in advance.


Rob Waite

http://www.hostpipe.co.uk?src=TT

 

[Tarwn]

I think the variant nature of VBScript is getting you. Since Hex is limited to 4 Bytes (signed) in VBScript your having problems any time your lX4 and lY4 evaluate to &H40000000. Basically I think what is happening is when you go to add lX and lY together VBScript is internally converting to signed values, adding the values, then returning them as a signed 8B value to your variable. Later when you try to Xor them I believe it is trying to treat it as a 4B int and, of course, overflowing.

Another example of this behaviour is adding anything to &H7FFFFFFF (except 0 obviously). Instead of going negative as a signed 4B int it becomes a signed 8B value.

Most of this was digging around in VBScript by throwing values at it and seeing how it reacted. I did find some odd things (I think 8B+ integers become types as doubles). Binary logic has always given me issues in VBScript, a lot of times I have had to resort to silliness like if statements.

Best MS KB Ever:

http://support.microsoft.com/kb/555375

 

[waiterm]

Many thanks for that, in the debug &H80000000 was actually throwing out a 0 value, whilst the lX8 and lXY were throwing our corresponding positive and negative numbers, both greater than 4B values, hence the overflow like you said, however it wasn't even possible to convert them to 8B.

I didn't actually find out what caused the problem, I simply reverted back to a version I used in another project and it worked fine without any overflow errors...I think it's the typical, late night scenario, inadvertantly deleted a line of code and sent the whole thing out of sync.

Thanks again.

Rob Waite

http://www.hostpipe.co.uk?src=TT