Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations Chris Miller on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

unix script for downloading the files from website

Status
Not open for further replies.

nynin9

Programmer
Jul 9, 2015
3
AU
Hi,

I need to download the files from the website and put in to the UNIX server, Can you please suggest.

The website look like this

Thanks in Advance
Ratna Kumar
 
The best way is to have a logon/signon on their server, and then have a sftp job to get (or mget) the files. man sftp for help with secure file transfer protocol.

Some considerations:
[ul]
[li]How often will the files be downloaded?[/li]
[li]Once the files are downloaded, how do you make sure you don't download them again?[/li]
[li]If the files aren't present when your process runs, how do you requeue your process? How long do you wait between attempts? How many attempts before you create an error? How and to whom do you report the error?[/li]
[li]If your server is down for a few days, what happens to the files on the host server? Do they append? Will they overwrite the current files? One file for each date?[/li]
[/ul]

==================================
adaptive uber info galaxies (bigger, better, faster, and more adept than agile big data clouds)


 
You can use wget or curl
[3eyes]


----------------------------------------------------------------------------
The person who says it can't be done should not interrupt the person doing it. -- Chinese proverb
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top