Get sequential response from a http server ...

[Guest_imported]

Hi !

I've been looking for threads that could solve my problem but I didn't find ... Maybe some of you could help me !

I want a script that could connect to a http server and modifying à part of the url in a sequential way ...

http://www.toto.fr/1/index.htm

http://www.toto.fr/2/index.htm

http://www.toto.fr/3/index.htm

...
...

http://www.toto.fr/X/index.htm

I supposed I have to use system("telnet

http://www.toto.fr

80&quot ... combined with the GET method ...
Thanx for your help !
SpBoy10 ;-)

 

[vgersh99]

I suggest postong to the "UNIX Scripting" forum.....

vlad

 

[marsd]

No need for that vlad


function basever() {
"gawk -W -version" | getline base
return base
}

BEGIN {
#get version
if (a = basever()) {
m = split(a,arr,".&quot
if (m && arr[2] < 1 && arr[3] < 95) {
print &quot;Incompatible awk version for networking.&quot;
exit
}
} else {
print &quot;Problem determining awk version.
exit
}

#start action
RS = ORS = &quot;\r\n\r\n&quot;

if (!ARGV[1]) {
printf &quot;Need number of connects.&quot;
exit
} else {
#open connection
sock = &quot;/inet/tcp/0/

http://www.toto.fr/80&quot;

for (x=1 ; x <= ARGV[1] ; x++) {
#format request
url = sprintf(&quot;%s%s%s&quot;, &quot;

http://www.toto.fr/&quot;,

x ,&quot;/index.htm&quot
#send to sock buffer
print &quot;GET&quot; &quot; &quot; url |& sock

#send/read from sock::bidirectional pipe

while ((sock |& getline) > 0) {
print $0
#alternately build a global array with all the different pages
#array[x,&quot;PAGE&quot;] = $0
#good for mirroring maybe....
}
printf &quot;Ending%2d&quot;, x
}
}
close(sock)
}

called like awk -f scriptname 10
Thanks go to J.Kahr for his work on this extension.

 

[vgersh99]

Nice one, marsd!

Wouldn't
called like awk -f scriptname 10
be

called like gawk -f scriptname 10

Do other 'awk-s' [tawk, mawk etc] have the networking capabilities? Just curious.........

vlad

 

[marsd]

I can't totally take the credit for this one: it is heavily adapted from Juergen Kahr's GETURL.

Yes, you are probably right, I run SuSE linux with the later awk versions and they just symlink awk and gawk to the same binary,so I got into that habit wherever I go..
Not good when people are using solaris and tru-64 and whatnot
Also the basever function needs help: if someone is running gawk 2.(? > 1).95+ (if there is such a thing) it fails.

Lastly:
I think it is just gawk > 3.0.95 that has this capability.

Bye.