iluvperl
Programmer
- Jan 22, 2006
- 107
I am a programmer, not a techie, so bare with me if this seems like a a n00b question.
I built a site scraper that collects 6000+ proxies live once a day from 3 proxy web sites.
I then use the proxies to test my server with a second script I wrote using Perl. The problem is, most proxies I can't connect to and it takes forever for my Perl script to determine whether or not it can connect to it to scrape my web site. That's all it does. It loads the IP address as a proxy and then loads a single web site.
Is there a quick way I can determine which proxies are good/bad for what I need? Since mine is strictly for web scraping my server, is there a specific port number to look for?
I built a site scraper that collects 6000+ proxies live once a day from 3 proxy web sites.
I then use the proxies to test my server with a second script I wrote using Perl. The problem is, most proxies I can't connect to and it takes forever for my Perl script to determine whether or not it can connect to it to scrape my web site. That's all it does. It loads the IP address as a proxy and then loads a single web site.
Is there a quick way I can determine which proxies are good/bad for what I need? Since mine is strictly for web scraping my server, is there a specific port number to look for?