FTP of large files 1

[chelseatech]

I'm a software developer with a 12MB install of one of my applications. I put all the setup files into a big zip file which is around 12MB.

That is where the problem occurs. If I use FTP or Frontpage to transfer the file to my web site, the file is damaged around the 10MB point. It is still the same size but ZIP complains that it has a CRC error.

If I take the file around on a CD to my ISP, it can be downloaded fine. If I check the uploaded file, it is damaged at the web site.

I'm running WindowsXP, and using WsFTP-LE or FrontPage 2002 to upload. I'm connected to the Net using a DSLINK 500 ADSL router. I've tried different PC's, removing the firewall, trying a different brand ADSL router, but the damage still happens on upload. If I split the large Setup file into smaller ones, and zip them, the damage still appears around the 10MB mark.

I thought FTP was a reliable protocol, and would retransmit if there was an error. But that isn't happening for me.

Any suggestions welcome. Editor and Publisher of Crystal Clear

http://www.chelseatech.co.nz/pubs.htm

 

[Accessdabbler]

Here's a wild stab in the dark.

I once had a problem sending a file and I was told to change the name to no more than 11 characters (like the 8.3 DOS format) and it worked! If your file has a long name, try shortening it.

 

[chelseatech]

Thanks for your suggestion, but the name is already in 8.3 format.

I've tried two web sites - one is running Linux/Apache and the other is running Windows 2000 Server/IIS

Editor and Publisher of Crystal Clear

http://www.chelseatech.co.nz/pubs.htm

 

[Accessdabbler]

Have you tried sending another 10MB file? Maybe the ISP is having problems accepting large files?

Can you FTP the large file to another FTP site? If you can, the problem my be at your ISP. If you can't, your system may be the problem.

 

[chelseatech]

I can FTP the file between machines on my local network and it is okay.

I have tried two ISP sites, and the problem happens on both. They will both accept the file when it it there. it is just that the file is damaged in the transfer.

It is a ZIP file with a large compressed CAB file and some other Installshield setup files inside. It is almost as if the compression contains some flow control characters or something. Editor and Publisher of Crystal Clear

http://www.chelseatech.co.nz/pubs.htm

 

[Accessdabbler]

The last thing I can think of is changing from Binary to ASCII mode for the transfer (or vice versa). I don't know why, but ASCII mode seems to be more reliable than Binary mode.

Well, one more last thing. FTP connections can time out. I wonder if the session is taking too long and the FTP session is timing out?

Beyond this, my thinker is all thunked out.

 

[chelseatech]

I've tried both BINARY mode and ASCII mode.

I also noticed that the Transmissions buffer size in FTP was set to 512 while the receive was at 4096 bytes. So I increased the transmission to 4096 and then to 1600000 and it still corrups on upload.

Any other sugegstions welcome. Is there such a thing as FTP flow control chars that mess up transmission. Sort of the TCP/IP equivalent of XON/XOFF Editor and Publisher of Crystal Clear

http://www.chelseatech.co.nz/pubs.htm

 

[bcastner]

I am almost positive this is an MTU setting problem. Set the MTU of the client to 1492 to start, as you are using ADSL. The default of 1500 will certainly show this problem with FTP.

You might have to reduce it further to 1492.

You might try testing MTU settings at

http://www.dlsreports.com,

and use their Tweak Test.

 

[chelseatech]

I followed the WinXP instructions for changing the MTU, rebooted and uploaded my test file.

The corruption has still happened. Is there a smaller MTU I should try?

Editor and Publisher of Crystal Clear

http://www.chelseatech.co.nz/pubs.htm

 

[bcastner]

See this link for more information. Essentially you need to make certain both your NIC and router settings are correct.

http://www.broadbandreports.com/forum/remark,1700907~root=equip,16~mode=flat

 

[TheeAngel]

Chelseatech,

I'm having the same problem as you. Did you find a solution? I'm uploading over Comcast cable -- large real audio files are corupted after the upload.

I'll be working until I get a solution, but I'm hoping you someone will give me a shortcut.

 

[chelseatech]

I still haven't found an answer to the problem. I take my zip file around on a CD to the web server.

Interestingly, a ZIP file created by PKZIP doesn't seem to have the problem that a ZIP file created by Windows XP compressed folder does. And it is slightly smaller.

WinRAR created ZIP files are damaged. WinRAR RAR files seem to be okay. If I use WinRAR to create an EXE (self extracting EXE) then the ZIP version is damaged, while the RAR version seems okay.

It is almost as if some bytes in the zip file are interpreted by FTP in some way.

Changing the file extenstion didn't help.

All the FTP experts I've consulted, say the problem shouldn't happen. It does.

I've even changed my ADSL router and it still has a problem.


Editor and Publisher of Crystal Clear

http://www.chelseatech.co.nz/pubs.htm

 

[bcastner]

chelseatech,

Please read carefully the follow Link info, as you really should not be having the problem you describe.

http://www.broadbandreports.com/forum/remark,1700907~root=equip,16~mode=flat

http://www.broadbandreports.com/forum/remark,1381832;root=equip,16;mode=flat

 

[TheeAngel]

After further investigation...

I'm able to upload the same files to my Comcast server, via the Comcast ftp integrated into their homepage. Files upload perfectly! Unfortunately, Comcast only provides 25MB worth of storage and no personal domain names, so I can't use them as a permanent solution. That said, this new discovery probably indicates my ftp software is the culprit! I'm using FTP_Editor ver_2.5 to upload large files. Comcast uses Unlimited_FTP ver_2.5.2.

You might want to change your ftp software as well. Let me know...

 

[TheeAngel]

Solved my problem! Switch to a new FTP client!!!

 

[pansophic]

TheeAngel,

Have you contacted the vendor of the original client? It would be nice if you gave them a "heads up!"


pansophic

 

[TheeAngel]

Good idea Pansophic! Doing my part to make the web a better place to surf.

 

[TheeAngel]

Chelseatech - Did you try another FTP s/w client? If so, did it solve your problem?

 

[chelseatech]

Hi everyone,

I finally solved the problem. It looks like the 3COM Home Connect gateway/Firewall I was using was the cause of my problem. I've upgraded the firmware on that box and I've successfully FTP'd files up and back to my web site.

We suspect that a large file filled up a buffer somewhere in the firewall, before the packets got passed on.

Anyway, thanks everyone for all your suggestions. It helped to point us in the right direction.

Editor and Publisher of Crystal Clear

http://www.chelseatech.co.nz/pubs.htm