can't get cookies FROM web page

[gorgor]

I'm trying to use HTTP::Cookies and LWP::UserAgent to fake a real HTTP request. When executing this script from a browser, I want to set the cookies that the website normally would and load the page that the website normally would also. I can't get the cookie part to work. I'm able to send a 'fake' referer, but I'm not able to get the cookie information. Could someone take a look at my script and tell me what's wrong? I'm sure it's all messed up, but I'm so frustrated with it that I don't know what to do. The examples I found in CPAN are terrible.


My script:

#!c:\perl -w

use strict;
use CGI::Carp qw(fatalsToBrowser);

use HTTP::Request::Common qw(GET);
use HTTP::Cookies;
use LWP::UserAgent;

my $cookieFile = "lwpcookies.txt";
my $ua = new LWP::UserAgent;

$ua->agent('Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; DigExt)');
my $cookie_jar = HTTP::Cookies->new( File => "$cookieFile", AutoSave => 1, );

# the test script we are GETting prints out the referer and query string and sets a cookie
my $req = GET '

http://www.globalnamespace.com/sample/test.pl';

#this is my own test script
$req->push_header(REFERER => '

http://www.somesite.com/');

my $response = $ua->request($req)->as_string;
$cookie_jar->extract_cookies($response);

#print "Content-type: text/html\n\n";

print $response , &quot;<br>\n&quot;;

exit;



Here is my error I'm getting when running the script:

Software error:
Can't locate object method &quot;_header&quot; via package &quot;HTTP/1.0 200 OK
Date: Wed, 04 Sep 2002 15:58:18 GMT
Server: Microsoft-IIS/5.0
Content-Type: text/html
Client-Date: Wed, 04 Sep 2002 15:58:18 GMT
Client-Peer: xx.xx.xxx.xxx:80
Set-Cookie: mycookie=set; domain=.globalnamespace.com; path=/;

referer =

http://www.somesite.com/<br>

query string = <br>

&quot; (perhaps you forgot to load &quot;HTTP/1.0 200 OK
Date: Wed, 04 Sep 2002 15:58:18 GMT
Server: Microsoft-IIS/5.0
Content-Type: text/html
Client-Date: Wed, 04 Sep 2002 15:58:18 GMT
Client-Peer: xx.xx.xxx.xxx:80
Set-Cookie: mycookie=set; domain=.globalnamespace.com; path=/;

referer =

http://www.somesite.com/<br>

query string = <br>

&quot;?) at C:/Perl/site/lib/HTTP/Cookies.pm line 244.

For help, please send mail to this site's webmaster, giving this error message and the time and date of the error.

 

[trepur]

I am a little confused about what you want the exact outcome to be, but.....

If you are trying to return a cookie with your HTML content then this is what you need to do (as documented in perldoc for CGI module):

#################################################
$title = &quot;Example&quot;;

$cookie = cookie(-name => 'Example',
-value => 'A:B',
-domain => '.yourdomain.com',
-expires => '+15m');

print header(-cookie => $cookie),
start_html(-title => $title,
-BgColor => 'White');
print &quot;HTML content here!\n&quot;;
print end_html();
#################################################

However, if you want to retrive a page from another location, parse it and add your own stuff you would need to use the following example:

#################################################

#!/usr/bin/perl

use LWP;
use CGI qwstandard);
use CGI::Carp qw(fatalsToBrowser);

#*** Global Variables

$html_doc_address = &quot;

http://www.yourdomain.com&quot;;

$proxy_ip_address = &quot;

http://168.192.0.1:8080/&quot;;

#$DEBUG = 1;


#*** Subroutines

sub retrieve_file {

local($url,$proxyaddress,$user_agent,$request,$response,$content,@content);
($url,$proxyaddress) = @_;
print &quot;$proxyaddress\n&quot; if ($DEBUG == 1);

#Create a user agent object
$user_agent = LWP::UserAgent->new;

#Set proxy information if there is one
$user_agent->proxy('http',$proxyaddress) if ($proxyaddress);

#Create a Request
$request = HTTP::Request->new(GET => $url);
$request->content_type('text/html');

#Pass request to user agent and get response back
$response = $user_agent->request($request);

#Get content from response
$content = $response->content;

return($content);

}



#*** Main Code Block

$html_file = &retrieve_file($html_doc_address,$proxy_ip_address);

print header(),
start_html(-BgColor => 'White'),
&quot;$file\n&quot;,
end_html();

#################################################

Hope this helps.....

 

[gorgor]

The SECOND code looks more like what I'm trying to do. However, I'm not familiar with the syntax. What I want to do is:

1) retrieve the cookies set by that location
2) find out the names and values of those cookies (for testing purposes)
3) make sure that, if my script (your sample script) is loaded from a web browser, that the cookies are SET just like I was going to the actual location without going through my script.
4) Also, I MUST fake a referer to the page. That is why I used push_header.

I'm going to use this for some automated tasks such as online banking, etc.

 

[trepur]

gorgor,

You can use the HTTP::Cookies module to grab the cookies as follows:

$cookie_jar = HTTP::Cookies->new;
$cookie_jar->extract_cookies($content);

Once that information is retrieved into $cookie_jar you can then manipulate it, and/or put it back into the content....

$cookie_jar->add_cookie_header($content);

But if you are just passing it on through your script there is no need to do anything as it should just get passed through.

to add a referer you should just have to put the following in the start_html tag:

print header(),
start_html(-xbase => '

http://www.yourdomain.com/',

-BgColor => 'White'),
&quot;$file\n&quot;,
end_html();

The argument xbase allows you to provide an HREF for the <base> tag different from the current location. All relative links will be interpreted relative to this tag.

 

[gorgor]

thanks! I'll give it a shot