maximum file size?

[nychris]

I have a 4.2GB file, and a 5GB file in my htdocs folder that I want to download via a web browser. While running Apache 1.3.29, the files show as 165MB and 787MB, respectively. As a test, I compiled Apache 2.2.0 (on a SLES9 server). The file sizes look okay (in a web browser) under v2.2.0, but when you try to download them they both fail once you download the first 4GB of the file.

Does anyone know of this limitation? Is there a way around it?

Thanks

 

[danomac]

Some filesystems do not support a file over 4GB in size. It's probably a client issue (filesystem won't go over 4GB per files), not a server issue.

 

[nychris]

The client that I'm using for testing is a WinXP client with the NTFS filesystem. I just read that NTFS supports 16TB files.

 

[danomac]

Have you tried a different browser? It's the only other thing I can think of.

 

[nychris]

Yes, I've tested with multiple browsers.

 

[danomac]

Hmm, it seems that apache does have a limit on the size you can transfer over http. I accidentally came across the directive LimitRequestBody and it states a 2GB limit on file transfers. Who knows when it was last updated. It could have very well been changed to 4GB and this manual not updated.

If you look in your error_log after the transfer fails, do you get a message like "[notice] child pid 26832 exit signal File size limit exceeded (25)"? I'm sure if you check there it will tell you what is going on. It sounds like apache has hit it's limit.

 

[nychris]

It seems that Apache 2.2.x doesn't have the 4G limit anymore, but the problem is that modern web browsers still do not support files larger than 4G. The combination of Apache 2.2.0 with a text-based utility such as wget and/or curl will allow you to download files > 4G.

--
Chris
RHCE, SCSA, AIX5L, LPIC, CNE, CCNA, MCSE

 

[danomac]

That would make sense. Another thing though: I don't think http is such a stable protocol to download files of that size anyway. While wget may be able to resume, most browsers can't, so if there is an interruption it would start from scratch.