Zip file corrupted after SSH file transfer

Question

I'm trying to zip up some files on my Windows laptop, then transfer them to my Ubuntu server, and unzip them there, but I always get an error while unzipping the archive telling me that the archive is corrupt.

I've done some checks, and though the zip files have the same filesize, the SHA256 checksum is different after transferring the files.

I used the following command (on my Windows laptop with using PowerShell):

cat .\archive.zip | ssh user@address "cat > ~/archive.zip"

I'm guessing it has something to do with the difference in operating systems, maybe the different line endings. Does anyone know how I can transfer the archive properly?

Answer 1

One of the comments under the question:

It could be that cat is changing individual occurrences of LF and CR to the sequence CRLF because it is intended for text.

Real cats are too lazy for this sh… shenanigan. In PowerShell cat is an alias for Get-Content. You need few parameters to make it behave like a nice cat.

Example 6: Get raw content

The commands in this example get the contents of a file as one string, instead of an array of strings. By default, without the Raw dynamic parameter, content is returned as an array of newline-delimited strings. […]

$raw = Get-Content -Path .\LineNumbers.txt -Raw

I guess in your case this "array of newline-delimited strings" was concatenated and some monkey business with line endings occurred. But even with Raw you will still get a string. You don't want a string.

Example 8: Get file contents as a byte array

This example demonstrates how to get the contents of a file as a [byte[]] as a single object.

$byteArray = Get-Content -Path C:\temp\test.txt -AsByteStream -Raw

[…]

The first command uses the AsByteStream parameter to get the stream of bytes from the file. The Raw parameter ensures that the bytes are returned as a [System.Byte[]]. If the Raw parameter was absent, the return value is a stream of bytes, which is interpreted by PowerShell as [System.Object[]].

^(source)

I don't know PowerShell at all. I don't know if you should want [System.Byte[]] or [System.Object[]], or if it matters at all in your case. I found this statement:

Combining -Raw with -AsByteStream / -Encoding Byte is actually the simplest way to get the file's content as a raw byte array ([byte[]]), output as a single object (as opposed to the bytes streaming one by one, in the absence of -Raw).

I guess probably you should use Raw. I'm certain you should use AsByteStream.

Answer 2

I found out how to use SFTP, and that did transfer the file correctly. I'd still like to know why cat over ssh doesn't work and if this is possible without SFTP, though.