1

I tried to migrate data from Google Cloud Platform, I tried this method by Mike Exporting data from Google Cloud Storage to Amazon S3, but got the complaint, anyone knows what should I do next?

enter image description here

Improve this question

2 Answers 2

Reset to default
0

Make sure your internet connection is stable, this happens when you don't have a stable network and as a result it will return a broken pipe.

Improve this answer
0

Note that when you're using gsutil rsync method, the transferred data will actually go through the machine you run the command from. If it has a poor or unstable internet connection you may face this kind of issues.

To avoid it, one option is to create a Compute Engine VM in your Google Cloud project and run the gsutil rsync command in there, so you can take advantage of Google network. Once the transfer is finished you can delete the VM.

Another option is to use Google Storage Transfer Service.

Improve this answer
1
  • 1
    Note for anyone looking for GCS to S3 transfers. The Transfer Service only writes to GCS destinations.
    – Devon_C_Miller
    Commented Jan 11, 2022 at 20:48

Not the answer you're looking for? Browse other questions tagged or ask your own question.