Recursive scp - w/ wildcard - Maintain file structure 1

[skicamel]

The goal is to scp into another server and pull all files matching a format, say 20060408.gz, from all folders in a certain directory, say /home/tmp.

scp -r user@server:/home/tmp/ del/
keeps the file structure intact, but obviously, pulls all files, not just 20060408.gz.

scp -r user@server:/home/tmp/*/20060408.gz del/
-r is ignored in this case and files are copied w/out file structure. Having the same filename, not all of the data is pulled. Also I'd need two scripts in this fashion since /home/tmp contains hundreds of folders containing .gz files, but also one other sub-folder with dozens of folders also containing .gz files. So if the above example worked, I'd also need
scp -r user@server:/home/tmp/*/*/20060408.gz del/

Is there any way to either keep the directory structure and only pull files matching a certain pattern? Or is there a way to force a rename on all copied files (hundreds of files w/ the same name. Doesn't matter naming convention on pulled files, as they will all be parsed out & loaded into a db)?

Let me know if I'm not being clear on the goal. Still on the first cup of coffee. Thanks, all.

 

[thedaver]

Have you considered running rsync over ssh?

D.E.R. Management - IT Project Management Consulting

http://www.dermanagement.com/

 

[skicamel]

Actually, I hadn't. Ran into rsync while googling and passed over it thinking it was basically a bulk-type mirroring tool w/o much in the way of options. Didn't realize how powerful it could be. Ended up w/ something like this:

rsync -av --include="*/" --include="*20060408.gz" --exclude="*" user@server:/home/tmp/ del/

Works beautifully. Thanks for the tip.

 

[WhiteVolg]

for I in `ssh usr@desthost find /home/tmp -name 20060408.gz `; do FNAME=`echo $I|sed 's/\/home\/tmp\///'`;scp srchost:$I /del/$FNAME;done

sick and twisted but would maintain the dir structure and maintain the encryption of scp. Downside find is a resource hog.

 

[WhiteVolg]

ooh missed a detail:

for I in `ssh usr@desthost find /home/tmp -name 20060408.gz `; do FNAME=`echo $I|sed 's/\/home\/tmp\///'`; mkdir -p `echo $I|sed 's/\/[^/]*$//'`;scp srchost:$I /del/$FNAME;done

 

[thedaver]

WhiteVolg;
I have a ton of respect for your rewrite of rsync, man!
;-)

D.E.R. Management - IT Project Management Consulting

http://www.dermanagement.com/