Tek-Tips is the largest IT community on the Internet today!
Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!
-
Congratulations Mike Lewis on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!
I got a server error... How do I debug a Perl CGI script?
Debugging
There are many ways that a Perl script can trip up. And when your output is to a web browser, it is often even harder to figure out what is going wrong.
First thing you should do is read through faq452-1816, which describes some of the more common problems that programmers often face when first getting into CGI with Perl. They include uploading from a workstation of one OS type to a server of another without using ASCII mode which converts newlines to the right format, not setting the correct permissions (executable), and using the wrong location for the perl interpreter in the shebang (#!) line. I'd also like to add that you should make certain that your web server allows you to run executables in the directory you want to... sometimes you can only run them in cgi-bin, or some similar restricted zone. This directory should generally be owned by the "nobody" user or whichever user your webserver runs scripts as. Additionally, you should make sure that any modules you want to use are actually installed. It's also important to "[tt]use strict[/tt]" at the top of your code, as this will flag unsafe (error-prone) code as errors so that you can correct them before they become a real headache. One of the important things that the strict pragma will enforce is declaring all of your variables (eg. [tt]my $var = 0;[/tt]). And finally, make sure you have output the correct HTTP header followed by two newlines (eg. [tt]print "Content-type: text/html\n\n"[/tt]) before printing anything else. If you still have problems, then read on...
Your first clue is to check your error log. If you have physical or ssh access to your server, you can usually find it in its default location of "logs/error_log" under linux/unix or "logs/error.log" under Windows if you are using Apache. With Cobalt servers, you may find it under "/home/logs/httpd/error". If you can't find it, you can ask your administrator where it is, or you can "locate httpd.conf" (again, for Apache) and read through that file until you find a line that says something like "ErrorLog logs/error_log". ErrorLog is the directive, and the string that follows it is the location of the error log relative to the server root, or relative to absolute root if preceded by a "/".
Now, assuming you've found it, do "tail -20 /path/to/error_log" in a text terminal on a *nix OS, or open it with a graphical text editor under Windows or X, where "/path/to/error_log" is the location of the error log. The tail command will give you the last few lines, specified by the integer following it. Since the error log is cumulative, you want to see only the last few lines to determine where your script is having problems.
If you do not have physical or ssh access to your server, or if you just want an easier way of viewing errors, you can add the following line to the top of your script after the shebang line:
[tt]use CGI:
ebug (report=>'everything',on=>'anything');[/tt]
"CGI:ebug will catch (almost) all compilation and runtime errors and warnings and will display them in the browser," according to the documentation which you can find at http://cpan.org. It will report to the browser: "bad HTTP-headers, empty HTTP-body, warnings and errors, elapsed time, query parameters, cookies, and environment variables (max 60 chars in value)." If you do not have this module installed, then you can install it from CPAN if you have root access or ask your administrator to install it for you.
Now, once you can see the errors that are returned using one of the above methods, you are much closer to learning where the bug is. There are two kinds of information returned in the error log... errors (of course) and warnings. Warnings, usually designated by [warn], are "non-fatal" in that they don't crash your program, or at least not right away anyway. However, they can be telltale signs that errors may result if they are not fixed, or may in fact be occurring later in the script because of them. In most cases, unless you know for a fact that the warning is frivolous, you should fix it right away even if it isn't causing any immediate problems, if for nothing else than because it is filling up your error log. You can use the "-w" switch after your interpreter location on the shebang line (eg. [tt]#!/usr/bin/perl -w[/tt]) to turn on "optional" warnings and increase your likelihood of identifying problems in your code. Fatal errors, usually designated by [error], will be the direct cause of your script failing to execute. Sometimes the only thing that will differentiate an error and a warning is a date and time of failure in front of the error, with no such data for warnings. Sometimes both errors and warnings have this data depending on your web server.
There should be a line number associated with each warning and error, so that's where you should start looking. Usually, the error will be within the syntax on the line specified. However, sometimes the error is actually several lines away and the interpreter didn't realize there was an error until it found something that didn't make sense several lines later. For instance, you may have failed to use an opening or closing paren or quote, or perhaps you forgot a semicolon at the end of a line. In this case, look back a few lines and see if you can spot the problem. Other times, you'll have a reference to another piece of code, either in your script or a module that you are using. In this case, you'll have to examine more closely how to call the subroutine or module and how it should appropriately interact.
Whenever possible, catch errors with error routines supplied with modules or with "die". For instance, when opening a file, you should catch a possible error with "or die" (eg. [tt]open (FILE, "$file") or die "Couldn't open $file: $!";[/tt]). If "open" does not return true, the "or" phrase is executed instead. The $! (bang) is a special variable which contains the error string returned by functions such as "open". This will make tracking down an error much easier, as it will tell you what failed and why.
Sometimes an error will still elude you though, perhaps because the error message is ambiguous, or because it is specific but doesn't make sense according to the code you wrote. It may complain about an undefined reference (for example) when accessing an array that you know exists. In this case, you should first see if anything useful is returned on STDOUT. You can do this by running your script on the command line. If it ran correctly, then it would output your HTML. Sometimes it will print a portion of the correct output and then stop, and other times it will return an error message that wasn't included in the error log. Either way, these are extra clues to help you track down the bug. Another way to get more information than your web browser provides (especially if you don't have command line access) is to pretend to be a web browser. You can do this by telnetting to port 80 of your server and issuing HTTP commands (see RFC1945, RFC2616). Eg:
[tt]telnet your.server.com 80
Trying xxx.xxx.xxx.xxx...
Connected to your.server.com (xxx.xxx.xxx.xxx).
Escape character is '^]'.
GET /path/to/your_script.pl HTTP/1.0
HTTP/1.1 302 Found
Date: Day, DD Mon YYYY HH:MM:SS GMT
Server: YourWebServerVersion
Location: http://your.server.com/path/to/your_script.pl
Connection: close
Content-Type: text/html
output of script...
Connection closed by foreign host.[/tt]
As a last resort for syntax errors, and also to solve logical errors which produce the wrong output but don't crash your program, you may have to step through it. Other programming languages have Integrated Development Environments which execute one line at a time and show you the value of your variables at that time. While you may be able to find something similar for Perl or not, there is a simple time-tested method for stepping through your program... simply print out a string at various points in your code, including important variables if necessary. Sometimes a simple "I got this far" is enough to show you where an error is or isn't. This is especially true in loops... print out the loop counter and see if it goes on forever. If so, you've got an infinite loop and you need to make sure that the loop is able to exit. The other way is to disable blocks of code with comments (#) and see if the rest of it works without that block. This way you can narrow down the area where the bug is living. If commenting out one line is the difference between errors being returned and smooth sailing, then you know that the bug lives on that line. If necessary, comment everything until all you've got is a "hello world" program, then start adding lines back one at a time until the error pops up its ugly little head. If "hello world" does not work, then you didn't appropriately heed the warning about "common problems".
If you've totally exhausted all possible clues and still can't find the error, it may be that you are just in the wrong mindset. Get up, stretch, eat a snack, put on some music, and try again a little later when you can take a fresh look. Research some documentation (eg. http://perldoc.com), tutorials, and FAQs for a new perspective. This is also the point that you may consider coming to Tek-Tips and posting your problem for others to review as well. Then at least you will be able to form an intelligent, well-thought-out question with appropriate error messages. The experts here will be happy to help you at this point.
Good luck, and happy hunting!
First thing you should do is read through faq452-1816, which describes some of the more common problems that programmers often face when first getting into CGI with Perl. They include uploading from a workstation of one OS type to a server of another without using ASCII mode which converts newlines to the right format, not setting the correct permissions (executable), and using the wrong location for the perl interpreter in the shebang (#!) line. I'd also like to add that you should make certain that your web server allows you to run executables in the directory you want to... sometimes you can only run them in cgi-bin, or some similar restricted zone. This directory should generally be owned by the "nobody" user or whichever user your webserver runs scripts as. Additionally, you should make sure that any modules you want to use are actually installed. It's also important to "[tt]use strict[/tt]" at the top of your code, as this will flag unsafe (error-prone) code as errors so that you can correct them before they become a real headache. One of the important things that the strict pragma will enforce is declaring all of your variables (eg. [tt]my $var = 0;[/tt]). And finally, make sure you have output the correct HTTP header followed by two newlines (eg. [tt]print "Content-type: text/html\n\n"[/tt]) before printing anything else. If you still have problems, then read on...
Your first clue is to check your error log. If you have physical or ssh access to your server, you can usually find it in its default location of "logs/error_log" under linux/unix or "logs/error.log" under Windows if you are using Apache. With Cobalt servers, you may find it under "/home/logs/httpd/error". If you can't find it, you can ask your administrator where it is, or you can "locate httpd.conf" (again, for Apache) and read through that file until you find a line that says something like "ErrorLog logs/error_log". ErrorLog is the directive, and the string that follows it is the location of the error log relative to the server root, or relative to absolute root if preceded by a "/".
Now, assuming you've found it, do "tail -20 /path/to/error_log" in a text terminal on a *nix OS, or open it with a graphical text editor under Windows or X, where "/path/to/error_log" is the location of the error log. The tail command will give you the last few lines, specified by the integer following it. Since the error log is cumulative, you want to see only the last few lines to determine where your script is having problems.
If you do not have physical or ssh access to your server, or if you just want an easier way of viewing errors, you can add the following line to the top of your script after the shebang line:
[tt]use CGI:
ebug (report=>'everything',on=>'anything');[/tt]"CGI:
ebug will catch (almost) all compilation and runtime errors and warnings and will display them in the browser," according to the documentation which you can find at http://cpan.org. It will report to the browser: "bad HTTP-headers, empty HTTP-body, warnings and errors, elapsed time, query parameters, cookies, and environment variables (max 60 chars in value)." If you do not have this module installed, then you can install it from CPAN if you have root access or ask your administrator to install it for you.Now, once you can see the errors that are returned using one of the above methods, you are much closer to learning where the bug is. There are two kinds of information returned in the error log... errors (of course) and warnings. Warnings, usually designated by [warn], are "non-fatal" in that they don't crash your program, or at least not right away anyway. However, they can be telltale signs that errors may result if they are not fixed, or may in fact be occurring later in the script because of them. In most cases, unless you know for a fact that the warning is frivolous, you should fix it right away even if it isn't causing any immediate problems, if for nothing else than because it is filling up your error log. You can use the "-w" switch after your interpreter location on the shebang line (eg. [tt]#!/usr/bin/perl -w[/tt]) to turn on "optional" warnings and increase your likelihood of identifying problems in your code. Fatal errors, usually designated by [error], will be the direct cause of your script failing to execute. Sometimes the only thing that will differentiate an error and a warning is a date and time of failure in front of the error, with no such data for warnings. Sometimes both errors and warnings have this data depending on your web server.
There should be a line number associated with each warning and error, so that's where you should start looking. Usually, the error will be within the syntax on the line specified. However, sometimes the error is actually several lines away and the interpreter didn't realize there was an error until it found something that didn't make sense several lines later. For instance, you may have failed to use an opening or closing paren or quote, or perhaps you forgot a semicolon at the end of a line. In this case, look back a few lines and see if you can spot the problem. Other times, you'll have a reference to another piece of code, either in your script or a module that you are using. In this case, you'll have to examine more closely how to call the subroutine or module and how it should appropriately interact.
Whenever possible, catch errors with error routines supplied with modules or with "die". For instance, when opening a file, you should catch a possible error with "or die" (eg. [tt]open (FILE, "$file") or die "Couldn't open $file: $!";[/tt]). If "open" does not return true, the "or" phrase is executed instead. The $! (bang) is a special variable which contains the error string returned by functions such as "open". This will make tracking down an error much easier, as it will tell you what failed and why.
Sometimes an error will still elude you though, perhaps because the error message is ambiguous, or because it is specific but doesn't make sense according to the code you wrote. It may complain about an undefined reference (for example) when accessing an array that you know exists. In this case, you should first see if anything useful is returned on STDOUT. You can do this by running your script on the command line. If it ran correctly, then it would output your HTML. Sometimes it will print a portion of the correct output and then stop, and other times it will return an error message that wasn't included in the error log. Either way, these are extra clues to help you track down the bug. Another way to get more information than your web browser provides (especially if you don't have command line access) is to pretend to be a web browser. You can do this by telnetting to port 80 of your server and issuing HTTP commands (see RFC1945, RFC2616). Eg:
[tt]telnet your.server.com 80
Trying xxx.xxx.xxx.xxx...
Connected to your.server.com (xxx.xxx.xxx.xxx).
Escape character is '^]'.
GET /path/to/your_script.pl HTTP/1.0
HTTP/1.1 302 Found
Date: Day, DD Mon YYYY HH:MM:SS GMT
Server: YourWebServerVersion
Location: http://your.server.com/path/to/your_script.pl
Connection: close
Content-Type: text/html
output of script...
Connection closed by foreign host.[/tt]
As a last resort for syntax errors, and also to solve logical errors which produce the wrong output but don't crash your program, you may have to step through it. Other programming languages have Integrated Development Environments which execute one line at a time and show you the value of your variables at that time. While you may be able to find something similar for Perl or not, there is a simple time-tested method for stepping through your program... simply print out a string at various points in your code, including important variables if necessary. Sometimes a simple "I got this far" is enough to show you where an error is or isn't. This is especially true in loops... print out the loop counter and see if it goes on forever. If so, you've got an infinite loop and you need to make sure that the loop is able to exit. The other way is to disable blocks of code with comments (#) and see if the rest of it works without that block. This way you can narrow down the area where the bug is living. If commenting out one line is the difference between errors being returned and smooth sailing, then you know that the bug lives on that line. If necessary, comment everything until all you've got is a "hello world" program, then start adding lines back one at a time until the error pops up its ugly little head. If "hello world" does not work, then you didn't appropriately heed the warning about "common problems".
If you've totally exhausted all possible clues and still can't find the error, it may be that you are just in the wrong mindset. Get up, stretch, eat a snack, put on some music, and try again a little later when you can take a fresh look. Research some documentation (eg. http://perldoc.com), tutorials, and FAQs for a new perspective. This is also the point that you may consider coming to Tek-Tips and posting your problem for others to review as well. Then at least you will be able to form an intelligent, well-thought-out question with appropriate error messages. The experts here will be happy to help you at this point.
Good luck, and happy hunting!