Hello Tek-Tippers,
I am having a problem accessing an asp.net site and get a 403 Forbidden error with my script. This does not happen to any html type website so I am asking you for help to scrape asp.net site.
Thank you in advance for all the help.
use LWP:
ebug qw(+);
use LWP::UserAgent;
use HTTP::Cookies;
my $ua = new LWP::UserAgent;
my $url = ' my $cookie_jar = HTTP::Cookies->new;
$ua->proxy("http", "proxysite:80");
my $req = new HTTP::Request GET => $url;
my $res = $ua->request($req);
$req = $res->request();
$cookie_jar->extract_cookies($res);
print "\nREQUEST-HEADERS\n";
print $req->headers_as_string();
print "\nREQUEST-CONTENT\n";
print $req->content;
if ($res->is_success) {
print "\nRESPONSE-HEADERS\n";
print $res->headers_as_string();
print "\nRESPONSE-CONTENT\n";
print $res->content;
} else {
print "\nRESPONSE-ERROR\n";
print $res->error_as_HTML();
}
Output:
REQUEST-HEADERS
User-Agent: lib
REQUEST-CONTENT
RESPONSE-ERROR
<html>
<head><title>An Error Occurred</title></head>
<body>
<h1>An Error Occurred</h1>
<p>403 Forbidden</p>
</body>
</html>
I am having a problem accessing an asp.net site and get a 403 Forbidden error with my script. This does not happen to any html type website so I am asking you for help to scrape asp.net site.
Thank you in advance for all the help.
use LWP:
ebug qw(+);use LWP::UserAgent;
use HTTP::Cookies;
my $ua = new LWP::UserAgent;
my $url = ' my $cookie_jar = HTTP::Cookies->new;
$ua->proxy("http", "proxysite:80");
my $req = new HTTP::Request GET => $url;
my $res = $ua->request($req);
$req = $res->request();
$cookie_jar->extract_cookies($res);
print "\nREQUEST-HEADERS\n";
print $req->headers_as_string();
print "\nREQUEST-CONTENT\n";
print $req->content;
if ($res->is_success) {
print "\nRESPONSE-HEADERS\n";
print $res->headers_as_string();
print "\nRESPONSE-CONTENT\n";
print $res->content;
} else {
print "\nRESPONSE-ERROR\n";
print $res->error_as_HTML();
}
Output:
REQUEST-HEADERS
User-Agent: lib
REQUEST-CONTENT
RESPONSE-ERROR
<html>
<head><title>An Error Occurred</title></head>
<body>
<h1>An Error Occurred</h1>
<p>403 Forbidden</p>
</body>
</html>
