Global Data Extraction Project

[rmahawaii]

Hello All,
I just got an assignment and it's a tough one. This assignment ask me to develop a solution to have download fix amount of data from different IP address around the world every month. I was wondering how can I do this. Should I write a DOS batch file and install in the client computers so that it will execute every month? How can I make the batch file runs once every month? I also need to take in the security issue. How can I accomplished this task? Please give me any ideas that may work. Thank you all.

Ronald

 

[conklinw]

Don't know if this will help...

Use wget. Search for it on Google. You can create a batch file to run it. Just use windows scheduler to run it once a month. I haven't used all the options within wget but maybe there is a way to get a fixed amount of data. Or if you download the whole site you are mining and then study it you can narrow it down to certain directories or files that can then be scheduled using wget command lines in a batch file.

http://wget.sunsite.dk/