scp files between hosts through intermediate

Question

I have access to 3 machines, A, B, and C. The only possible (ssh) connections are:

A -> B

A -> C

I need to get files from B to C, I can only scp the files from A to B (on A: scp remote local), and then from A to C (on A: scp local remote). However, A does not have enough disk space for some rather large files. Firewall between B and C as well as reverse path from B -> A and C -> A. and I'm not able to put keys on either B or C for my connection from A to B and C. (long story, another issue)

Is there a way to scp the file on B to C by piping the scp output between A and B directly into an scp from A to C? such the actual file is not stored locally?

Looking at verbose output, the -3 option seems to send a remote scp command to the remote host B. that fails because of the intermediate firewall blocking the connection.

Answer 1

You can use a reverse SSH tunnel for this:

On host A, run:

ssh -R 3000:ip.of.host.c:22 ip.of.host.b

On host B, run

scp -P 3000 /source/file username@localhost:/destination_file

This will create a tunnel, listening on host B, which forwards through host A to host C. So you can use localhost:3000 on host B to make connections to hostc:22.

Answer 2

You can use a version of the tar-ssh-tar technique to do this. From server A, run this command:

ssh B 'cd /src && tar cf - file1 file2...' | ssh C 'cd /target && tar xvf -'

This creates a tar-format data stream on B containing the files, pipes that stream back to A through SSH, then to C through SSH, then into the tar program on C which unpacks the files. No files have to be stored on A.

You should set up SSH keys for both B and C, so that you won't be prompted for a password. Two ssh instances prompting for a password at the same time will be confusing.

If this is a long-distance transfer, you could try compressing the tar stream to speed things up:

ssh B 'cd /src && tar zcf - file1 file2...' | ssh C 'cd /target && tar zxvf -'

Answer 3

I think the easiest solution is to make a named pipe on pc A. A named pipe is a file that behaves like a standard pipe, i.e. it passes along its content as soon as it becomes available, without storing anything per se. It is a standard means of implementing IPC, Inter Process Communication, on *Nix systems.

  mkfifo my-pipe

creates a named pipe with name my-pipe. The only problem is that you cannot scp a named pipe (the OS will complain about my-pipe not being a regular file). So the way to do it is: in one terminal on pcA, issue the following command:

  cat my-pipe | ssh me@pcC 'cat > /path/to/destination/file'

This command streams the named pipe's content to the pcC, and this is redirected to the destination file.

In another terminal, again on pcA, issue:

  ssh me@pcB 'cat /path/of/source/file' > my-pipe

This command puts to standard output the source file located on the remote pcB, and redirects it to the named pipe.

The named pipe is emptied as soon as material comes in. The end-of-file from pcB closes both connections, to B and to C. Nothing is stored on A. At this point, you may remove the pipe,

 rm my-pipe. 

It seems to me quite simple.

Answer 4

If you do this frequently, I would suggest adding a proxy command in your ssh config file on server A located in ~/.ssh/config like this:

host serverB
User <user on B>
HostName <hostname of B>

host serverC
User <user on C>
ProxyCommand ssh -q serverB nc -q0 <C hostname> 22

-q is for quiet mode, meaning less output.

Now when you want to copy files from machine A to machine C all you have to do is type scp file1 file2 .. serverC:/some/path You can do this on server C as well if you want to do it in reverse. This is especially helpful if you have different usernames on the different machines, and it can of course also be combined with ssh-copy-id if you don't want to enter your password every time.