using PHP to grab the source of a remote webpage

[monkle]

I am working on a script that requires grabbing the source html of a remote webpage. I have poured a good bit of time into this already, both in searching for information and in experimentation.

I've already written scripts that access remote files successfully, but I'm running into trouble with dynamically generated web pages. For instance, I can use fread to get the information in test.txt, but I don't know how to get the source html when the url is "

http://www.test.com/test.php?value=1"

I know this must be possible, but I'm not having any luck finding any information on anything even remotely related. Please point me in the right direction.

 

[Olavxxx]

Hi,

http://no.php.net/manual/en/function.fopen.php

look at the examples there, as it might give you some clues.

Olav Alexander Mjelde

http://www.volvo-power.net

Admin & Webmaster

 

[monkle]

Thank you. I believe I'm on the right track now, and making good progress.

 

[Olavxxx]

np

Olav Alexander Mjelde

http://www.volvo-power.net

Admin & Webmaster

 

[Itshim]

Greetings,

I don't mean to highjack this thread, but...

I would like to create a script which spins through the links in my database and checks for any that are broken. I know this can be achieved by using fopen(), but it there a better php function for this?

Thanks,
Itshim

 

[Olavxxx]

If the open fails, the function returns FALSE and an error of level E_WARNING is generated.

uno momento, and I'll post an example code for you

Olav Alexander Mjelde

http://www.volvo-power.net

Admin & Webmaster

 

[Olavxxx]

ok, now I made it for you..
ps. it's not very fast!! I think there might be better ways of doing this, but it can be used, let's say.. if you allow your users to post theire homepages to your user database?

or, maybe if someone reports an url "not working", you can double-check it, before the admin gets notified??

anyways, here is the code..

<?php

$webpage['url'] = "[URL unfurl="true"]http://asdasdiajsdasjd.com";[/URL]
$webpage['title'] = "test-url";

echo check_url($webpage);

function check_url($url, $mode="r") {
	if (!@fopen($url['url'], $mode)) {
	    $res = "Sorry, the page {$url['url']} is offline!<br /> Please check back later!";
	  }
	else {
	    $res = "<a href=\"{$url['url']}\" target=\"_blank\">{$url['title']}</a>";
	  }
  return $res;
} // end function check_url
?>

working and tested!
ps. you might want to control the variables, check if the array is set, etc.

good luck!

Olav Alexander Mjelde

http://www.volvo-power.net

Admin & Webmaster