Progress Bar 1

[audiopro]

I am trying to make sense of CGI:rogressBar and tk:rogressBar.
I have found a few partial examples but the explanations just go right over my head.
I require a progress bar in an existing file upload routine, what are my best options?
Do the tk modules work with a standard installation of Perl?

Keith

http://www.studiosoft.co.uk

 

[audiopro]

The required process appears to be simple and probably is.
I have a loop which uploads 1024 byte chunks of data to a server.
How hard can it be to update a jquery script after every chunk is uploaded?
Suggestions please of how this can be achieved.

Keith

http://www.studiosoft.co.uk

 

[audiopro]

I have tracked down a basic uploader and progress bar but I have encountered an odd problem.

The following code snippet is invoked from a HTML form containing an input type='file' field.
The first line of code runs the sub routine but there is a delay of over 30 seconds before the sub routine is called when loading a 3Mb file.

my $q = new CGI (\&hook);

sub hook {
print "Hook<br>";

Is there an obvious reason for this behaviour?

Keith

http://www.studiosoft.co.uk

 

[1DMF]

I believe CGI needs a hook to report the progress via the temp file it uses during upload. It's been a while since I looked at this and in the end I decided to move to HTML5 instead.

Is there a reason you cannot use HTML5 for this? It is so much nicer and easier!



"In complete darkness we are all the same, it is only our knowledge and wisdom that separates us, don't let your eyes deceive you."

"If a shortcut was meant to be easy, it wouldn't be a shortcut, it would be the way!"
Free Electronic Dance Music

 

[audiopro]

Thanks for the reply.
I am just revisiting an old problem because some of the users need to upload 8Mb files and many of them are using older browsers without HTML5 support.

The delay is occurring before the upload actually starts and seems to be linked with the <input type='file' command.
I am wondering what actually happens as it appears to do something with the file ID before the script is processed.

Keith

http://www.studiosoft.co.uk

 

[1DMF]

It may be possible to shim/polyfill the missing HTML5 behavior, it was rather flaky and I had a lot of issues with Webshim when I tried to support IE9 with it and eventually gave up, but they may have fixed it by now?

Alternatively can they run Flash?

I use this for my music site uploads currently before I port to HTML5, and it works very well uploading 100mb+ files no problem!

http://www.uploadify.com/about/

Though I must admit the provider seems to have gone a bit sucky as it appears the new version for HTML5 is no longer free and the free version is Flash only with no HTML5 support / Flash fallback.

If you want I can send you the older version that was HTML5 with Flash fallback, see my sig and pop me an email!

All my solutions use a client side progress mechanism with Perl back-end file upload script, I gave up trying to work out the weirdness of CGI, file uploads and the progress hook!


"In complete darkness we are all the same, it is only our knowledge and wisdom that separates us, don't let your eyes deceive you."

"If a shortcut was meant to be easy, it wouldn't be a shortcut, it would be the way!"
Free Electronic Dance Music

 

[audiopro]

The actual upload works fine and I have introduced a CSS driven progress bar which works very well except for the delay in starting.
It is possible that this delay has always been present but has not been noticeable when uploading small files.
The user selects the file in the normal way, clicks upload and then there is a delay of 30 seconds or so.
After that the upload starts and the progress bar reflects the percentage uploaded until completion.
The length of start delay appears to reflect the size of the file so the key is there somewhere.

I have avoided Flash for quite a while now as it's use upsets the mobile users.

The internet seems to be awash with incompatible technologies making a developer's life a bit of a functional lottery.

Keith

http://www.studiosoft.co.uk

 

[1DMF]

Hmm, how do you get CSS to display feedback from the progress of the upload, that intrigues me!

Is it possible that the file is uploaded first via CGI and then the feedback / progress reports retrospectively.

So you don't actually have a real-time upload mechanism and possibly the upload is taking twice as long as it should, firstly to upload the file, then secondly to pretend it's uploading and do the progress thing?

The internet seems to be awash with incompatible technologies making a developer's life a bit of a functional lottery.

Yup, I have used an all Perl / PHP tool before:

http://uber-uploader.sourceforge.net/

, so this might help.

I spent many years trying different solutions before HTML5, since HTML5 I have ditched anything < HTML5 and only support those browsers that have

window.FileReader 
window.FormData 
'upload' in ($.ajaxSettings.xhr())

Now coding form submissions and file uploads with a progress GUI is so simple. The client side doing all the work and it matters not what back-end is actually receiving the data.

I know not very PC, but as my users are in Financial Services, they should not be running OSes that are not patched / supported, so XP is out of the picture, all the software providers in the industry only put out Windows compatible software so mobiles / Apple is out the picture also, leaving only Vista+ / IE10+ or any of the other free browsers that all support the new HTML5 stuff. Has made my life so much easier!

Plus our members' extra-net is cross browser compatible (Perl/JQuery/HTML5/CSS3/IIS7), and works in browsers running on Linux / Windows, the only thing I cannot test is Safari as I don't have a Mac, but as there is Firefox for Mac's, it's all good








"In complete darkness we are all the same, it is only our knowledge and wisdom that separates us, don't let your eyes deceive you."

"If a shortcut was meant to be easy, it wouldn't be a shortcut, it would be the way!"
Free Electronic Dance Music

 

[audiopro]

The delay occurs before the upload routine is even started.

I am using the simplest of HTML/Perl to test it.
Try uploading an 8Mb image and see how long the script takes to run.

<!DOCTYPE html>
<html>
<head>
<title>Upload Test</title>
</head>
<body>
<h1>File Upload Test</h1>
<FORM METHOD="POST" ACTION="cgi-bin/test2.pl" ENCTYPE="multipart/form-data">
<INPUT TYPE="file" NAME="FILEID" VALUE="" SIZE=50 MAXLENGTH=80><BR>
<INPUT TYPE="hidden" NAME="call" VALUE="uploadimage">
<INPUT TYPE="submit" NAME="submit" VALUE="Upload File To Server"></FORM>
</body>
</html>

#!/usr/bin/perl
use strict;
use warnings;
print "Content-Type: text/html\n\n";
print "This simple script only displays this text and performs no other function.";

Keith

http://www.studiosoft.co.uk

 

[1DMF]

Bit odd, I just ran your test with a 12MB ZIP file and it uploaded in < 5 seconds!

"In complete darkness we are all the same, it is only our knowledge and wisdom that separates us, don't let your eyes deceive you."

"If a shortcut was meant to be easy, it wouldn't be a shortcut, it would be the way!"
Free Electronic Dance Music

 

[audiopro]

I assume you mean that it took 5 secs for the text to appear.
The connection speed is quite slow here so I can only think that has something to do with the difference.
It appears to load the file before the upload starts as I can see transfer activity on the router, if that is the case, where does it upload it to?

Keith

http://www.studiosoft.co.uk

 

[1DMF]

I assume you mean that it took 5 secs for the text to appear.

Yes.

We have a 40mb/40mb fibre lease line so uploads as pretty nifty.

It appears to load the file before the upload starts as I can see transfer activity on the router, if that is the case, where does it upload it to?

The browser shouldn't be caching / loading it anywhere first that I'm aware of, it simply reads the file from the UNC location.

Is it perhaps a setting on the web server? Are you running (LAMP), could a modperl configuration or other config be causing this behavior?


"In complete darkness we are all the same, it is only our knowledge and wisdom that separates us, don't let your eyes deceive you."

"If a shortcut was meant to be easy, it wouldn't be a shortcut, it would be the way!"
Free Electronic Dance Music

 

[audiopro]

Your questions are the same I have been asking but nobody seems to have a definitive answer.
My ISP are saying that they don't see much of a delay either but that is not surprising as they do have a very fast connection.
When I say ours is slow, it is currently 6-8Mb so although it is slow by modern standards, a 3Mb file should be processed in much less than 30 seconds.


Keith

http://www.studiosoft.co.uk

 

[1DMF]

it is currently 6-8Mb

I assume you are talking upstream. if so , yes it should upload in no time.

It seems as though it is being uploaded first then processed?

"In complete darkness we are all the same, it is only our knowledge and wisdom that separates us, don't let your eyes deceive you."

"If a shortcut was meant to be easy, it wouldn't be a shortcut, it would be the way!"
Free Electronic Dance Music

 

[audiopro]

That is the conclusion I have arrived at but is this behaviour something which can be set up on a server?


Keith

http://www.studiosoft.co.uk

 

[1DMF]

Out of my knowledge-base, I use Catalyst MVC framework for all my Perl stuff now and it takes the request and packages it up in an easy to use object, I've not suffered from this phenomenon.

If the server has mod_perl installed it might be related to the Apache request setup, could it be taking the request then handing it over to your script so in theory processing it twice?

http://www.informit.com/articles/article.aspx?p=27110&seqNum=10

What's your environment?

"In complete darkness we are all the same, it is only our knowledge and wisdom that separates us, don't let your eyes deceive you."

"If a shortcut was meant to be easy, it wouldn't be a shortcut, it would be the way!"
Free Electronic Dance Music

 

[audiopro]

I use Perl a lot and this is the first real illogical issue I have had with it.
There have been many things I have had to learn but this issue defies all reason and like you I believe that the file is being loaded before control is handed to the script.

My ISP are struggling to understand the actual problem and their most recent contribution was telling me that the HTML file which calls the uploading script loads in under a second - not sure what relevance that has and until they actually understand the problem, I feel a solution could be a long time coming.

Perl 5.8.8 is installed on the shared hosting server.

Keith

http://www.studiosoft.co.uk

 

[1DMF]

hmm that's an older version of perl.

Is it on FreeBSD / Linux, is it Apache / NGINX , is Mod_Perl installed?

Could they have a proxy in-between?

Like you, grasping a straws here!

"In complete darkness we are all the same, it is only our knowledge and wisdom that separates us, don't let your eyes deceive you."

"If a shortcut was meant to be easy, it wouldn't be a shortcut, it would be the way!"
Free Electronic Dance Music

 

[audiopro]

It is on Linux and I don't think Mod_Perl is installed, although it could be for all I know.
If I can get them to understand the actual problem - I think that would go a long way towards solving it.

Keith

http://www.studiosoft.co.uk

 

[audiopro]

It would appear that the file is cached into the server's memory before being saved to the server itself.
Upload times are comparable to the time taken to FTP the same files.

The whole point of this exercise was to add a progress bar to an existing website to keep the visitor informed about upload progress.
As it doesn't work reliably, there is no point in having it so I have replaced it with an animated gif.



Keith

http://www.studiosoft.co.uk