tar a over an ssh tunnel. 1

[Eldaria]

I have a folder on a Sun Unix machine, that I want to copy to a Linux machine over an SSH tunnel.
It is the only way I can get out through the firewall.
Also I need to tunnel through a tunnel, since I need to stepstone out.
I have already created the stepstone tunnel by using the command:

ssh username@stepstone -2 -N -f -C -L 54322:remoteserver:22

I'm able to connect to the remote server now by using the command.

ssh -2 -C -p 54322 user@localhost

So the connection is working.
I have also been able to move a single file over the tunnel by using scp, however I need to move a folder that has thousands of files.
I have heard that tar is able to send it's output to almost anything, so can I send it over an SSH tunnel to the remote directory?

I can not tar it to one file and then scp it, since there is simply not enough space on the local drive to hold the directory in one file.
So what I need is to directly stream it over the tunnel, to the other end.

Is it possible?

Regards.
Brian


Eldaria

http://www.eldaria.net

That was my 25cent** of opinion.

** Inclusive Intrest, tax on interest, Genral tax, Enviromental tax, Tax, and tax on intrest, tax on fees, tax on tax, and other Various taxes and fees.

 

[thedaver]

rsync!

http://www.surfinbox.com/hosting/

 

[Eldaria]

A person of short words..

How does rsync do it?
Is it compressed, before sent through the tunnel?
How would I do it?

Regards.
Brian


Eldaria

http://www.eldaria.net

That was my 25cent** of opinion.

** Inclusive Intrest, tax on interest, Genral tax, Enviromental tax, Tax, and tax on intrest, tax on fees, tax on tax, and other Various taxes and fees.

 

[thedaver]

Rsync is capable of doing a lot of things, with lots of parameters. There is a lot of good googling to be had on rsync.

Essentially rsync's value is to say that one box is the source and one box is the destination. You then tell rsync to "copy/move files from source to destination with certain criteria (possibly compressing them on the way) using an SSH tunnel if necessary and optionally delete from the source once the copy is done".

I recommend googling for recipes. Rsync's palette of command line options can be daunting.

http://www.surfinbox.com/hosting/

 

[ericbrunson]

( cd sourcedir ; tar cf - . ) | ( ssh remotemachine " cd destdir ; tar xvf - " )

 

[thedaver]

Showoff! ;-)

http://www.surfinbox.com/hosting/

 

[Eldaria]

Nice one.
Would it be possible that you could explain to me as being a newbie to linux, how this works?

Regards.
Brian


Eldaria

http://www.eldaria.net

That was my 25cent** of opinion.

** Inclusive Intrest, tax on interest, Genral tax, Enviromental tax, Tax, and tax on intrest, tax on fees, tax on tax, and other Various taxes and fees.

 

[ericbrunson]

tar cf - . ": runs tar to create ("c") current directory, (".") and write it to the archive file specified by "f", but the specified is "-" which says write to standard out (stdout) instead of an actual file (in unix *everything* is a file). More on stdout later.

The parentheses causes the shell to execute a subshell, which is just a way of making sure your cd doesn't affect the shell you are running in.

The pipe ("|") is the shell construct to say: take the output (stdout) of the command in front of the pipe and make it the input (stdin) of the command following it.

So, tar up the directory and pipe it to another subshell that uses ssh to the remote machine and run a command. The quotes are important, because it makes ssh run the entire contents of the quotes on the remote machine, i.e. cd to the destination directory and run tar this time with "x" to extract, "v" for verbose and finally "f -" for use standard in as the input file.

 

[Eldaria]

Thank you for that, always nice to learn something new.

Eldaria

http://www.eldaria.net

That was my 25cent** of opinion.

** Inclusive Intrest, tax on interest, Genral tax, Enviromental tax, Tax, and tax on intrest, tax on fees, tax on tax, and other Various taxes and fees.