Winsock - receiving multiple files in one session

[yvrMarc]

Hi There,

I am using VB6 and Winsock to send and receive encrypted files to and from a remote server. Winsock works fine for this except when the remote server sends multiple files to me.

My program will receive the first file just fine, however all the subsequent files have seem to have two extra bytes added to the file. If I open the file up in a text editor and remove the bytes, the file will open up fine. It is just a pain to have to open all files and remove these bytes.

Here is the DataArrival code:

Private Sub Winsock_DataArrival(ByVal bytesTotal As Long)
    Dim strData As String
    Dim hFile         As Long
    
    Winsock.GetData strData, vbString, bytesTotal
    
   ' If we're not in a file transfer operation, this could well be a "BOF".
   If (Not blnConnected) Then
          '
          If (Mid$(strData, 1, 3) = "BOF") Then
                 
                 ' We have an incoming file - save the filename, and set the operation
                 ' flag so that next time data arrives (file data) it will be saved.
                 blnConnected = True
                 m_strFilename = Mid$(strData, 4)
              
                 ' Send back a "NEXT" to get some of the file data.
                 Call Winsock.SendData("NEXT")
                 DoEvents
                 
          End If
          
   Else
          
          ' If we're already in a transfer, this data could either be file data, or
          ' an EOF marker.
          If (Mid$(strData, 1, 3) = "EOF") Then
                 '
                 ' The transfer is complete.
                 blnConnected = False
                 DoEvents
                 '
          Else
                 ' Open a temporary file in the current directory - this is where all the
                 ' file data is saved.
                 hFile = FreeFile
                 Open outputFileName For Binary As #hFile
                 
                 ' Move to the end of the file, and write the data.
                 Seek #hFile, LOF(hFile) + 1
                 Put #hFile, , strData
                 '
                 Close #hFile
                 '
                 ' Send the server another "NEXT" command to get the next piece of data.
               
                 Call Winsock.SendData("NEXT")
                 DoEvents
                 '
          End If
          '
   End If
End Sub

Any thoughts on why this is happening? Thanks in advance for any insight you can offer.

 

[dilettante]

This is one mistake:

[tt]Winsock.GetData strData, vbString, bytesTotal[/tt]

You are saying "only take bytesTotal and discard any other received input after that." The bytesTotal parameter to this event handler means "there are at least bytesTotal bytes of data received so far" but more might be there before you call GetData. Passing vbString (type) is only used when strData is a Variant.

Reopening and reclosing the file is an expensive operation. I have no idea why you are doing this.

Stream sockets (TCP) do not preserve message boundaries. There is absolutely no guarantee that your program (or the other end) will receive your BOF, EOF, or other SendData's in exactly one GetData call.

Each GetData will only guarantee at least one byte of data for each DataArrival event.


If this code seems to be working at all it is sheer luck. Often people try to develop using internal (LocalHost to LocalHost) connections or fast LAN connections. This can lull you into thinking such code is viable, because it can seem to work much of the time. When you try to use the program in a real network it breaks.

You're also using a very chatty protocol, sending "go aheads" (NEXT). This can also help mask the flaws mentioned above, but even worse it can slow file transfers to a crawl. TCP goes to a lot of trouble to provide a smooth full duplex data channel using windowed ACKs just to avoid this sort of slowdown in throughput. Using a chatty protocol on top of TCP invalidates all of this effort.

You do not want to call DoEvents from within DataArrival. The results can be catastrophic, anywhere from stack overflow exceptions to data corruption (loss, mis-sequencing, or duplication of data). Again, use of a chatty protocol may seem to mask these problems most of the time - until it doesn't.


Then you have the problem that you seem to be using String variables with binary data. The implicit conversions involved (sending a String via SendData implies Unicode to ANSI conversion, GetData implies ANSI to Unicode conversion) might seem to be canceling out right now. All it takes is for a server or client to be running with different locale settings than the other end to scramble everything though. If the data truly is text (e.g. Base64 encoded binary) you're better off bcause the characters involved should map the same for nearly every locale.


I didn't pick up where these "extra 2 characters" (they can't be bytes, you're doing String operations and Strings use 16-bit characters in VB6) end up in your files. The beginning? The end?

Do they always have the same values or do they seem random?

 

[yvrMarc]

Thanks Dilettante for your informative response. The extra two characters (I think it is LF and CR) appear after the closing --Boundary caption which encapulates the mime message.

The transmission protocol that must be used to communicate with the remote server is HTTP - it will not accept any other protocol. I found that I had to use a String as the data is Base64 encoded binary, as binary inserts characters that are confused with LF and CR so the file will never opens.

I accept what you say that this might be a working fluke but I have not had many problems to date. The system would only transmit to the remote server about once an hour and no receiving file will ever be more than 500kb.

Can you point me to a better winsock example that will be more stable?

Thanks again

 

[strongm]

>The transmission protocol that must be used to communicate with the remote server is HTTP


I'm confused. The protocol in your example looks nothing like HTTP.

 

[yvrMarc]

Because it's an encrypted environment; I must send an encrypted file (authenicating file) to activate the download.

The HTTP header is created and the encryted file (authenicating file) is appended to it. The file is then sent using:

Winsock.SendData strHttpDownload

Then, if there is a file(s) there for me, winsock will then download it.

Does that help?

 

[strongm]

>Does that help?

Nope. Not really. That still doesn't sound (or look) like HTTP. Maybe I'm just being dense.

 

[tedsmith]

The problem is the data arrival can fire a few times for a long file even in a fast network and when the winsock transmit buffer is exceeded.

If you can change the sending software why not simply send the length of the file N (in 6 bytes of HEX so this is always the same length) before you send each message.
As soon as the first 6 bytes are received this tells the receiver to look for exactly N more bytes next.
This should be able to easily handle continuous sending of any character without the need to be looking for eofs or bofs or worrying about LF or CR.


Something like
Transmit.
Send Right("000000"& Hex(Len(MyData),6) & MyData

Receive Declarations
Dim N as long, Buffer as string

Sub DataArrival(ByVal bytesTotal As Long)
'This will keep receiving in a a few loops until all N+6 bytes have been received and no more
Winsock.GetData Buffer
If N=0 then
N = Val(("&h" & left(Buffer),6) 'No of original data bytes
Timer1.enabled=true
else
if Len(Buffer=(N+6)) then ProcessData
End if
End sub

Sub Processdata ()
'This will fire once for receive each transmission in turn
MyData=mid(Buffer,6) 'Trim off first 6 bytes (N)
N=0
Buffer=""
timer1.enabled=false
'You now have MyData exactly the same as it was originally sent. You must process it before the next arrives or put it into another FIFO buffer

End sub

Sub Timer1_Timer()
'I'd also have a one shot short timer set off by any DataArrival to cancel N and Strdata in the rare event the signal was corrupted and the N bytes expected never got there.
N=0
Buffer=""
Timer1.enabled=false
End sub