Multiple Zip file creation and download with one button 1

[Katerine]

Hi,
I have a client who wants their clients to be able to download their entire library of large pdf files from the website. So I've written code that is supposed to iterate through a query, add each file in the query to a temp zip file, and when the total number of files reaches 200, it's supposed to download the zip file and start a new zip file, downloading as many zip files as necessary to get the entire library.

The code does successfully get the first 200-file zip file, but it stops there. How do I get it to keep generating and downloading zip files?

Here's the relevant code:

if ($rowcount>0) {
	$maxnumfiles = 200;
	$i = 0;
	$zipnum = 1;
	foreach($rows as $row) { //loop through the result array.
		if ($i==0) {
			$zipname = $rootdirprefix . 'downloads/' . sprintf('%04d', $clientid) . '_' . date('Y-m-d-His',time()) . '_' . $zipnum . '.zip';
			$zip = new ZipArchive;
			if ($zip->open($zipname, ZIPARCHIVE::CREATE )!==TRUE) {
				exit("cannot open <$zipname>\n");
			}
			$debug_msg .= 'new zipfile created: ' . $zipname . '<br />';
		}
		$url = $rootdirprefix . 'docs/sds' . $row['URL'];
		if (file_exists($url)) { //file exists... go ahead and add it.
			$zip->addFile($url);
			$i += 1;
			$debug_msg .= 'file added: ' . $url . '<br />';
		}
		
		if ($i>=$maxnumfiles) {
			$zip->close();
			$debug_msg .= 'streaming zip file: ' . $zipname . '<br />';
			stream_zip_file($zipname);
			$i = 0;
			$zipnum += 1;
		}
	}
	
	if ($i>0) {
		$zip->close();
		$debug_msg .= 'streaming zip file: ' . $zipname . '<br />';
		stream_zip_file($zipname);
	}
}

Edit: (forgot) - this is the stream_zip_file function:

function stream_zip_file($zipname) {
	header('Content-Type: application/zip');
	header('Content-disposition: attachment; filename=' . $zipname);
	header('Content-Length: ' . filesize($zipname));
	header("Pragma: no-cache");
	header("Expires: 0");
	readfile($zipname);
}

Thanks!

Katie

 

[spamjim]

I can't speak to the request of PHP function but I have to mention that this seems unusual and dangerous. This would be a good way to cripple a server with DoS. (EDIT: counting 200 files does not consider the size of those files [does the server have the resources to build it?] or the time it takes to ZIP and serve [will the client request time out before the zipping completes?])

Instead of building ZIP archives with each user request, couldn't you assemble a single archive (via cron) as a standard resource that can be later called on demand?

Also, the ZIP format is probably not reducing filesize. It's only used for bundling. If the documents could be merged into a single PDF file, you would save a lot of filesize. Each PDF file may have a good portion of it devoted to embedded fonts and common graphics (logos). If you can consolidate PDF files, you don't have to pack and serve all that redundant data in each PDF file.

 

[vacunita]

Short answer is, you can't.

You can only output headers once. Once Headers have been output once, that's it. And if you try to output again, you will get a PHP error:

"Cannot modify header information - headers already sent by (output started at..."

Unfortunately, you also cannot send more than one file at a time for download. This is the point of zipping files. So you only send one.

If you need people to download the entire library of PDFs. You may need to offer a list of files to download, instead of automatically zipping them. Have a pre-zipped set of files, and a page with a list that can be downloaded instead.








----------------------------------
Phil AKA Vacunita
----------------------------------
OS-ception: Running Linux on a Virtual Machine in Windows which itself is running in a Virtual Machine on Mac OSx.

Web & Tech

 

[Katerine]

Unfortunately, the library of PDFs is customized for each client, and can be limited by searches (or not) as well, so having a standard PDF library won't work. It's also always changing. I'm not too concerned about it crippling the server, because the number of client logins who even have access to this functionality is very small, and it's a pretty rare request (maybe it happens once a month, that somebody might actually use this).

The reason we can't just do a single zip file is because of memory limits. 200 seems like a safe option... after all, in order for the PDFs to be on the website in the first place, they have to be small enough to upload without running into the limit, which means 200 PDF files per zip file should be under the limit to create and download as well.

Thanks for the info... it's useful to know that PHP can't do it alone. I'll look into PHP+javascript options (maybe have each of the zip files created first, then redirect to another page that uses javascript to change the headers, or something like that).

Thanks again!

Katie

 

[vacunita]

Yes, but if they are downloading the entire library, then nothing to customize. Not saying you replace the entire system, just have some more generic packages that contain everything they may want to download. Predefined ones, that are just generic enough to fit with "download entire library" filter.

Another option, would be for PHP to create the files as you mention in a dynamically allocated directory. Then have the same PHP page read that directory and display the created files with links for manual download. You can have a server side Job clear the folder after say 24 hours of being created or so.

Javascript has no way of altering PHPs behavior, or sending PHP headers. They run entirely separately and independently, and by the time Javascript is running, PHP has already finished. There is no direct connection between the two. The best you could do is, have JS through AJAX request the files, and again have PHP create them and store them in a folder, and return this list of files to JS so it can then display them as a list for manual download.

----------------------------------
Phil AKA Vacunita
----------------------------------
OS-ception: Running Linux on a Virtual Machine in Windows which itself is running in a Virtual Machine on Mac OSx.

Web & Tech

 

[spamjim]

it's a pretty rare request (maybe it happens once a month, that somebody might actually use this)

If it is so rare, you might just display a HTML list of linked PDF files for the end user and suggest a download helper to scrape the page. There are too many variables on the client end to make this type of file push work well.



Firefox:

https://addons.mozilla.org/en-US/firefox/addon/downthemall/

Chrome:

https://chrome.google.com/webstore/...der/aiahkbnnpafepcgnhhecilboebmmolnn?hl=en-US

 

[Unscruffed]

If the size of the files is the issue, then I would recommend generating the files and then notifying the client by email when they are ready for download:

1. Client makes request.
2. Respond by informing client that they will receive an email when the files are ready for download.
3. Server generates zip files.
4. Server sets cron job to delete zips after 7 days or whatever.
5. Server sends email to client with download links, login page, whatever, and notifies client of the time limit to download.
6. Files are automatically deleted by cron job after 7 days or whatever time limit you apply.

If security is an issue, then make sure that the zips are located in an area of the site where the client needs to be "logged in" to access.


Heaven doesn't want me, and Hell's afraid I'll take over!