Skip to main content

You are not logged in. Your edit will be placed in a queue until it is peer reviewed.

We welcome edits that make the post easier to understand and more valuable for readers. Because community members review edits, please try to make the post substantially better than how you found it, for example, by fixing grammar or adding additional resources and hyperlinks.

4
  • is it possible to install aws cli in google cloud shell? if so can you tell me how Commented Feb 21, 2020 at 16:54
  • You can just execute curl "awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" unzip awscliv2.zip sudo ./aws/install
    – Chris
    Commented Oct 21, 2022 at 2:07
  • 1
    This works ok, although I got a load of 'connection refused' and 'broken pipe' errors doing this from OS X so many retries have been necessary with rsync -i to ignore files that already copied ok (OS X single process mode is more reliable but slooow). Also I got an MD5 mismatch error that stopped many files from copying over. Error was 'md5 signature for source object doesn't match destination object digest'. This can be resolved by specifying the encryption type in the command: gsutil -h "x-amz-server-side-encryption: AES256" -m rsync -rdi gs://storagename s3://bucketname
    – urchino
    Commented Oct 24, 2022 at 16:38
  • My current estimate is that it will take approximately a week to transfer 250Gb of data using this mechanism. It produces a lot of errors. Random 'Connection reset by peer' is the latest. Note that all the errors are on the gsync to google side - you get the same behaviour if you try to take a local copy of the files without using S3 at all. The GSUtil cmd tool is very flaky and should be avoided if at all possible.
    – urchino
    Commented Oct 24, 2022 at 18:16