download multiple pages

[mwpc]

I am seeking a browser or plugin that will download multiple pages, as follows:

Many websites have database-driven content, powered by php or asp pages which require a specific referer page. They usually only display 10 or 20 results per page. If I am trying to grab the whole database, it takes too long.

If the referer requirement is not present, I can use a backend perl script to download all the pages server-to-server. However, if the php or asp prevents a download when the correct referer page is not present, it doesn't work. I have had no luck spoofing the referer. Likewise, I have had no luck using web-grabber programs when this referer requirement is blocking the road.

However, if a browser had a built in feature like "save all the links to file", then the correct referer would be present..........

Anybody know of such a program?

Thanks, Mike

 

[bcastner]

Other than copyright issues, and a concern for how much hard disk space you have:

Tons. Google "offline browsing utilities"