Upload a file

[lorenzodv]

Hi.

I'm desperately searching for someone able to tell me what Perl code to use to upload a file from a web browser form. I have tried understainding pre-made scripts, but they were all to complex: I just need the code to upload a file from the browser and save it into a subdirectory on my server. Please help me!

--
Lorenzo

 

[hmerrill]

Hopefully I didn't forget anything
=============================

Need something like this in your screen html:
--------------------------------
File to upload:
<input type=&quot;file&quot; size=&quot;40&quot; name=&quot;upload_file&quot;>

Wherever you POST to, use this to get the filename:
-------------------------------------
$upload_file = $q->param(&quot;upload_file&quot;

Decide what you will call the new file on your system(including the full absolute path/filename,
and open that:
----------------------------------
open(NEW_FILE,&quot;>$abs_file_to_create&quot || die &quot;Can't open $abs_file_to_create for output: $!&quot;;

Now read from the upload file, and write to the local file:
---------------------------------------
$total_size = 0;
my $print_error = 0;
while ($size = read($upload_abs_filename, $data, 8192)) {
$print_error = 1 unless print NEW_FILE $data;
$total_size += $size;
}
if (! defined($size)) {
print STDERR &quot;Error reading upload file $upload_abs_filename&quot;;
}
if ($print_error) {
print STDERR &quot;Error writing upload file <upload_abs_filename> to server - possible space problem on server&quot;;
}
close NEW_FILE;
============================================= HTH.

Hardy Merrill
Mission Critical Linux, Inc.

http://www.missioncriticallinux.com

 

[hmerrill]

Slight but significant correction - the

read($upload_abs_filename...

should be

read($upload_file... HTH.

Hardy Merrill
Mission Critical Linux, Inc.

http://www.missioncriticallinux.com

 

[hmerrill]

Another update:
------------
1. I forgot to mention that you need to make your form that includes the file upload field a &quot;multipart/form-data&quot;, like this:

<form action=&quot;some_script.cgi&quot; method=&quot;post&quot; enctype=&quot;multipart/form-data&quot;>

2. You can take an easier(?) approach by using the Perl CGI.pm module - CGI.pm is included in the standard perl distribution(I'm pretty sure), so all you need to do to see the docs is &quot;perldoc CGI&quot;, and search for &quot;upload&quot; - you'll see a whole section on &quot;CREATING A FILE UPLOAD FIELD&quot;. HTH.

Hardy Merrill
Mission Critical Linux, Inc.

http://www.missioncriticallinux.com

 

[goBoating]

Hi Lorenzo,
a previous post can be found at thread219-28183




keep the rudder amid ship and beware the odd typo

 

[lorenzodv]

I have tried the code given in the <a href=&quot;javascript:gothread(219,28183)&quot;>Thread219-28183</a>, but it generates the following error (returned by the Carp module):

CGI open of tmpfile: No such file or directory

I used the upsolute path for the directory and I tried both 666 and 777 permissions. The fact is that I don't like using CGI.pm: is there a way to upload a file with just Perl code (i.e. get bytes from POST query and write them to a file)?

Thanks again.

--
Lorenzo

 

[hmerrill]

Lorenzo, the 3 previous posts by me in this thread are &quot;almost&quot; straight perl code. The only exception is that I use the CGI.pm &quot;param&quot; method to retrieve the value of the &quot;upload_file&quot; field. HTH.

Hardy Merrill
Mission Critical Linux, Inc.

http://www.missioncriticallinux.com

 

[lorenzodv]

I'm using exactly the following code:

======================================
#!/usr/local/bin/perl
use CGI::Carp qw(fatalsToBrowser);

&GetFormInput;

$upload_file = $field{'upload_file'};

$abs_file_to_create = '/home/userid/cgi-bin/';
$abs_file_to_create .= 'tempfile';

open(NEW_FILE,&quot;>$abs_file_to_create&quot || die &quot;Can't open $abs_file_to_create for output: $!&quot;;
$total_size = 0;
my $print_error = 0;
while ($size = read($upload_file, $data, 8192)) {
print NEW_FILE $data;
$total_size += $size;
}
if (! defined($size)) {
print STDERR &quot;Error reading upload file $upload_abs_filename&quot;;
}
if ($print_error) {
print STDERR &quot;Error writing upload file <upload_abs_filename> to server - possible space problem on server&quot;;
}
close NEW_FILE;

print &quot;Content-type: text/html\n\nSuccessfull!&quot;;

exit;


sub GetFormInput {

(*fval) = @_ if @_ ;

local ($buf);
if ($ENV{'REQUEST_METHOD'} eq 'POST') {
read(STDIN,$buf,$ENV{'CONTENT_LENGTH'});
}
else {
$buf=$ENV{'QUERY_STRING'};
}
if ($buf eq &quot;&quot {
return 0 ;
}
else {
@fval=split(/&/,$buf);
foreach $fval (@fval){
($name,$val)=split (/=/,$fval);
$val=~tr/+/ /;
$val=~ s/%(..)/pack(&quot;c&quot;,hex($1))/ge;
$name=~tr/+/ /;
$name=~ s/%(..)/pack(&quot;c&quot;,hex($1))/ge;

if (!defined($field{$name})) {
$field{$name}=$val;
$fieldcount++;
@names[$fieldcount] = $name;
} else {
$field{$name} .= &quot;,$val&quot;;
$fieldcount++;
@names[$fieldcount] = $name;
}
}
}
return 1;
};
======================================

The script completes without errors, but produces a zero-length binary file. I CHMODed cgi-bin to 777, used POST method and multipart/form-data enctype for the HTML form.
What's the problem?
Please help me, I'm going MAD with this.

Thank you very much.

--
Lorenzo

 

[hmerrill]

Can you do some debugging? With my setup, I can

print STDERR &quot;I am here: value of \$var1 = $var1\n&quot;;

and that output goes to my webserver error log. This helps me figure out what's going on. Can you do this? Try to figure out exactly what is happening. After the while loop, try inserting this print to make sure you've actually been able to read and write:

print STDERR &quot;Total size read is $total_size\n&quot;;

What platform are you on? I'm on Redhat 6.1 Linux, with an Apache web server, and Perl 5.005_03.

One difference between your code and mine is in the read/write loop - change your

print NEW_FILE $data;

to

$print_error = 1 unless print NEW_FILE $data;

Another thing you can do is create a directory that will hold the uploaded files, and make that directory owned by the user that the webserver runs as(&quot;nobody&quot;?). Try creating the directory under /tmp(/tmp should be 777) - maybe something like /tmp/uploads - and make the &quot;uploads&quot; directory owned by the user that the webserver runs as.

HTH.

Hardy Merrill
Mission Critical Linux, Inc.

http://www.missioncriticallinux.com

 

[lorenzodv]

This is the code I am currently using:

==========================
#!/usr/local/bin/perl -w
use CGI::Carp qw(fatalsToBrowser);

&GetFormInput;

$upload_file = $field{'resfile'};

$abs_file_to_create = '/home/username/cgi-bin/';
$abs_file_to_create .= 'tempfile';

print &quot;Content-type: text/html\n\n&quot;;

open(NEW_FILE,&quot;>$abs_file_to_create&quot || die &quot;Can't open $abs_file_to_create for output: $!&quot;;
$total_size = 0;
my $print_error = 0;
while ($size = read($upload_file, $data, 8192)) {
print NEW_FILE $data;
$total_size += $size;
}
if (! defined($size)) {
print STDERR &quot;Error reading upload file $upload_file&quot;;
}
if ($print_error) {
print STDERR &quot;Error writing upload file <upload_file> to server - possible space problem on server&quot;;
}
close NEW_FILE;

exit;
==========================

When I run the script, I get the error &quot;Error reading upload file&quot;, and that's mean $size is not defined. Also, when I check the error log, I found two warnings from Perl: it says I'm using an uninizialized variable at lines 16 and 20, and that can only be $size.

Maybe this can help you: after running the script, if I try to chmod the zero-lenght tempfile, it does not allow me, while I am usually allowed. Couldn't this be related with script problems?

The server is an &quot;Apache/1.3.14 (Unix) PHP/4.0.0&quot; running &quot;Perl 5.00503&quot; with &quot;CGI/1.1&quot;.

Thanks a lot.

--
Lorenzo

 

[hmerrill]

I'm about out of ideas. One final thing to check which I'm sure you've done already, is make sure the file on the client being uploaded is readable by world. If that's not it, then I don't know what else to suggest.

Since no-one else in this forum has really chimed in with suggestions, I would suggest asking your question in maybe comp.lang.perl.misc or comp.infosystems.

http://www.authoring.cgi

newsgroups.

Good luck! HTH.

Hardy Merrill
Mission Critical Linux, Inc.

http://www.missioncriticallinux.com

 

[goBoating]

hello lorenzo,
the O'Reilly 'Programming Perl' book says, &quot;The read function returns the number of bytes acturally read, 0 at end-of-file.&quot; So, when you have read in the entire file, $size gets set to 0.

I'm willing to keep working on this with you, if you are willing to use CGI.pm. I do not know how to do it otherwise. I guess we could dis-assemble CGI.pm to see how it is done, but that really seems to be doing things the hard way. Once you play with the object syntax a little and play with CGI.pm a little, it is a lot faster than writing most CGI apps long hand. If you can post (cut and paste) the error you are getting from running my example in the earlier thread, then maybe we can get some more forward movement.

You should be able to copy and paste the code from that earlier thread into a file in your cgi-bin dir, fix this line to where you want the new file to go,

$newFile = '/path/to/where/you/want/to/put/newFile/';

and, run the page via a browser. It works. I promise. Once that is working, it is a simple matter of cutting the portions you want and dropping them into you code.

'hope this helps.




keep the rudder amid ship and beware the odd typo

 

[lorenzodv]

Dear goBoating, THANKS!

When running your example I was getting the &quot;Dangerous file type&quot; error and, because I had already failed many times with CGI.pm, I gave up immediately. Today I carefully read the code and I found that it was only an error check coded by you. I just commented it out and it worked perfectly! How could I be so stupid?

If you don't mind, I would be grateful if you help me trimming the code to fit my needs. For example, I would like to use a form of mine, with other form fields, and not the one automatically generated by the CGI module. I would also like to limit the size of the files that can be uploaded before uploading them.
I would make this adjustments myself, but I have just started studying CGI.pm syntax and object syntax in particular, so it would take me ages, while I'm sure it would be easy for you.

Thanks again for your help.

--
Lorenzo

 

[hmerrill]

goBoating,

I've copied your code that uses CGI.pm down and saved it for future reference - thanks. But I don't see that it is very different at all from the code I submitted - the code that does the &quot;while...read...print&quot; is for-all-intents-and-purposes identical, except that you read a buffer of 1024 and I read one of 8192. If you have a few minutes, I'd be interested in you looking over the code I submitted and comparing that to your code - do you see any significant differences? Any idea why Lorenzo was having problems with mine, but not yours?

One thing that your code lacks is a check on the &quot;print&quot; to the new file - if the upload file is big and the server runs out of space trying to write to it, a check on the print will at least notify you of that problem. HTH.

Hardy Merrill
Mission Critical Linux, Inc.

http://www.missioncriticallinux.com

 

[goBoating]

Hardy,

I don't see any significant differences in our versions. However, the code I posted was a complete piece of CGI that just needed a couple of paths fixed and it would run anywhere the CGI.pm was available. Your code, while accurate, was delivered in pieces that had to be assembled into a program which allowed for some typos and such. That is the only thing I see that is significantly different between the two. Admittedly, my version left a few things out that would probably good to do, like watching file sizes. It was trimmed down to the version that was posted to make the basic function as obvious as possible.

BTW, in case readers think the buffer size limits the file size, it does not. The buffer will fill and flush repeatedly while the read is working. Larger buffers work faster and require more RAM. Buffer size is a matter of choice for a given situation.

Lorenzo,
I'm a little backed up at the moment. It may be a little while, maybe tomorrow, before I can post a new version of the code. It should not take long. You should take a crack at it. The code in the previous thread did one of two things based on whether or not the 'fileID' param was populated. If the page was a first launch of the code, the 'fileID' param was null and the input page is built. If the 'fileID' param is populated, then that is a result of the input page having been submitted the file is read and written to local disk. By watching what is done in either case, you should be able to grab the upload chunks that you need and leave the parts that write the input page. I'll check back to see if you have beaten me to a solution. Sorry, I can't be of more help immediately.

'hope this helps.




keep the rudder amid ship and beware the odd typo

 

[lorenzodv]

Hardy,

I'm very grateful for your help. I thought that maybe the string parsed from the form (remote file path) and the $fileID returned by the CGI.pm param function are not both suitable for use as file handles, and maybe that's why your code didn't work. Thanks anyway!

goBoating,

I corrected the code to work with any form. The only thing O can't figure out is how to limit upload file size. As you correctly said it is not controlled by the buffer size.

Thanks for help.

--
Lorenzo

 

[hmerrill]

Lorenzo, by doing &quot;perldoc CGI&quot; and searching for &quot;limit&quot;, I found this info on how to limit the size of upload files:
---------------------------------------
CGI.pm also has some simple built-in protections against
denial of service attacks, but you must activate them
before you can use them. These take the form of two
global variables in the CGI name space:

$CGI:OST_MAX
If set to a non-negative integer, this variable puts a
ceiling on the size of POSTings, in bytes. If CGI.pm
detects a POST that is greater than the ceiling, it
will immediately exit with an error message. This
value will affect both ordinary POSTs and multipart
POSTs, meaning that it limits the maximum size of file
uploads as well. You should set this to a reasonably
high value, such as 1 megabyte.

$CGI:ISABLE_UPLOADS
If set to a non-zero value, this will disable file
uploads completely. Other fill-out form values will
work as usual.

You can use these variables in either of two ways.

1. On a script-by-script basis
Set the variable at the top of the script, right after
the &quot;use&quot; statement:

use CGI qw/:standard/;
use CGI::Carp 'fatalsToBrowser';
$CGI:OST_MAX=1024 * 100; # max 100K posts
$CGI:ISABLE_UPLOADS = 1; # no uploads


2. Globally for all scripts
Open up CGI.pm, find the definitions for $POST_MAX and
$DISABLE_UPLOADS, and set them to the desired values.
You'll find them towards the top of the file in a
subroutine named initialize_globals().

Since an attempt to send a POST larger than $POST_MAX
bytes will cause a fatal error, you might want to use
CGI::Carp to echo the fatal error message to the browser
window as shown in the example above. Otherwise the
remote user will see only a generic &quot;Internal Server&quot;
error message. See the the CGI::Carp manpage manual page
for more details.
---------------------------------------
I also wanted to point out that &quot;the string parsed from the form&quot; is *NOT* suitable for use as a file handle, and that's why in both my example, and &quot;goBoating&quot;s example we don't open the upload file for reading by using the &quot;open&quot; command(to get a filehandle to the file) - instead we use the direct &quot;read&quot; command to read a specified number of bytes directly from the upload filename(which if I remember right is an absolute path/filename when received by the CGI &quot;param&quot; method). Maybe that's why you had trouble with my code - you did your own parse of the POST variables instead of letting CGI.pm &quot;param&quot; method do it for you - maybe your parse did not properly receive the name(including absolute path?) of the upload file - so maybe the &quot;read&quot; couldn't find the file on the client system. Just a guess.

One more thing - when you finish this file upload code, it would make a great &quot;file upload faq&quot; for this Perl forum. HTH.

Hardy Merrill
Mission Critical Linux, Inc.

http://www.missioncriticallinux.com

 

[there]

Hi,

I went over the the posting in this thread and TRIED and TRIED to make the coding work. I must be doing something wrong. PLEASE HELP.

in the html coding
------------------
<form action=&quot;script_name.cgi&quot; method=&quot;post&quot; enctype=&quot;multipart/form-data&quot;>
<input type=&quot;file&quot; name=&quot;logo&quot; size=&quot;20&quot;>

in perl coding
--------------
##--------
# uploadImage()
##--------
sub uploadImage
{

$upload_logo = $field(&quot;logo&quot;

$abs_file_to_create .= &quot;/location_on_the_web_server/&quot;;
$abs_file_to_create .= &quot;tempfile&quot;;

open(NEW_FILE,&quot;>$abs_file_to_create&quot || die &quot;Can't open $abs_file_to_create for output: $!&quot;;
$total_size = 0;
while ($size = read($upload_logo, $data, 8192))
{
print NEW_FILE $data;
$total_size += $size;
}
close NEW_FILE;

}

Problem I am having
-------------------
it saves the file in the right location (location_on_the_web_server) but calls the file &quot;tempfile&quot;. I would like each entry to have their own file name, example: logo.gif. and logged in each database.

PLEASE HELP. I would love to know where I went wrong

Thank you for your knowledge, wisdom and time!

There