ftp mass dl

[Igaduma]

Hi all,

How could one use a ftp command to get everything from within the working directory down, including subfolders & files.
And possibly,retain the tree structure.

Possible ?

Thanks.

Iga-Duma

 

[KenCunningham]

Quick and dirty method would be to tar up the directory tree, ftp the tar file and untar it on the destination machine. HTH.

 

[Igaduma]

Hi Ken,

Yeah, that would be quick & dirty fix, but I don't have a tar capable login on that machine, only ftp access.
It's a remote machine that stores "geebee" captured streams of cellphone communications, it listens between base stations & base station controller. Capture all your favorite friends calls in just one go!
But, I need to write a little script that goes checking every day and downloads the files, but files have random names & are stored in subfolders.
FTP can't use wildcards huh ?

Iga-Duma

 

[KenCunningham]

Iga_Duma, sorry, didn't appreciate you only had ftp access. ftp can use wildcards in terms of mgetting and mputting files (multiple gets and multiple puts), but you'd probably need to traverse each of the source directories using cd within ftp. If you're using mput and mget within a script, it would be adviseable to use ftp -i <IP address> to turn off interactive prompting for each filename (as long as you're sure you won't be overwriting things you want on the destination machine!).

I'm almost sure there's an ftp (gtar possibly?) variant which allows tarring on the fly as files are transferred, but as I've never used it, others may be able to advise better. Best wishes.

 

[Igaduma]

Hi Kencunningham,

Thanks for the info,
I also seem to vaguely remember something with FTP and tarring-on-the-spot.
Maybe someone on the forum knows it a lil bit more....I'm hoping!

Ta!

Iga-Duma

 

[KenCunningham]

Iga-Duma, take a look at:

http://archive.ncsa.uiuc.edu/General/Training/InterUnix/files/pipes.html

there's a portion of the document (towards the bottom) dealing with what we were discussing above. HTH.

 

[KenCunningham]

Ah well, perhaps not (at least I haven't been able to get it working with an mget). I'll keep looking. Cheers.

 

[Igaduma]

Hi Ken,

I've got it to tar up a working directory and send the tarball to the ftp server .
Next test with subfolders!

Looks like a fine solution.

Already thinking of doing my 1:1 backups like that, beats nfs file copying & creates an archive on-the-spot.

Ta!

Iga-Duma

 

[KenCunningham]

Good stuff, glad it's working for you. Good luck with the sub-directories. Cheers.

 

[mikeclark]

You can use something like put &quot;| tar cf - .&quot; <filename.tar> inside ftp to create a tarfile on the fly for fast backup.