Can I Use SCP using data piped from an unzip -p cmd?

问题

We're using:

Linux version 2.6.32-696.20.1.el6.x86_64 (gcc version 4.4.7 20120313 (Red Hat 4.4.7-18) (GCC) ) #1 SMP Fri Jan 12 15:07:59 EST 2018 x86_64 x86_64 x86_64 GNU/Linux

I want to send data from one server to another using scp ( secure copy procedure).

We're unzipping the data first. Also we're only unzipping the data and piping it (hence the unzip -p) to the server as opposed to unzipping to a file on the local server and then sending it. This is because the server is a producion server and they don't want us storing any additional files on the production server that might accumulate.

The initial concept command looks like this;

unzip -p car.zip car1 | scp eagle@svamn14glbsalh:/pacelog/AUTOSYS/upload_event_logs/data/car1

What I'm trying to tell Linux to do is:

1. Unzip ONLY the data from the file car1 from the zipfile car.zip.
2. pipe that infomation to the scp command.

3. Using scp, copy the data to the following directory on the following server:

   eagle@svamn14glbsalh:/pacelog/AUTOSYS/upload_event_logs/data/

I'm getting the usage error:

usage: scp [-1246BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file]
           [-l limit] [-o ssh_option] [-P port] [-S program]
           [[user@]host1:]file1 ... [[user@]host2:]file2

So I tried doing it differently using the following command:

unzip -p car.zip car1 >  scp eagle@svamn14glbsalh:/pacelog/AUTOSYS/upload_event_logs/data/car1

I was thinking take the data stream from the unzip commmand and send the output to the scp command... It didn't work. I got the following error:

caution: filename not matched:  eagle@svamn14glbsalh:/pacelog/AUTOSYS/upload_event_logs/data/car1

Does anybody have any ideas? What am I missing?

回答1:

SCP isn't well-suited for your purpose. The SCP protocol doesn't support sending an unknown-sized stream of bytes to the remote system to be saved as a file. The SCP protocol message for sending a file requires the size of the file to be sent first, followed by the bytes that make up the file. With a stream of bytes read from a pipe, you typically wouldn't know how many bytes the pipe is going to produce so there's no way to send an SCP protocol message including the correct size.

(The best online description of the SCP protocol that I know of is here. Pay attention to the "C" message.)

The SFTP protocol can be used for this sort of thing. As far as I know, the normal sftp command-line utility doesn't support reading a pipe and storing it as a remote file. But there are SSH/SFTP libraries for most modern programming languages (perl, python, ruby, C#, Java, C, etc). If you know how to use one of these languages, it should be straightforward to write a utility that does what you need.

If you're stuck with shell scripting, it's possible to spoof enough of the SCP protocol to transfer a file. Here's an example:

#!/bin/bash
cmd='cat /etc/group'

size=$($cmd | wc -c)    
{
        echo C0644 $size some-file
        $cmd
        echo -n -e '\000'
} | ssh user@host scp -v -p -t /some/directory

This will create some-file in /some/directory on the remote system with permissions 644. The file contents will be whatever $cmd writes to its standard output. Note that you are running the command twice, with whatever resource consumption and side effects that implies. And the command must output the same number of bytes each time.

回答2:

You can do this with ssh provided you have the ability to run arbitrary commands on the server:

unzip -p  car.zip car1 | 
  ssh eagle@svamn14glbsalh 'cat > /pacelog/AUTOSYS/upload_event_logs/data/car1'

If you do not have the ability to run arbitrary commands, only the server-side mode of scp (scp -t with OpenSSH), it's still theoretically possible but I don't know how, because that mode is not documented see Kenster's answer.

来源：https://stackoverflow.com/questions/49884079/can-i-use-scp-using-data-piped-from-an-unzip-p-cmd