2

Uploading a file over 10 GB to s3 fails with:

aws s3 cp 'foo.log.gz' s3://mybucket/2018/12/ --storage-class REDUCED_REDUNDANCY
upload failed: ./foo.log.gz to s3://mybucket/2018/12/foo.log.gz 

An error occurred (BadDigest) when calling the UploadPart operation (reached max retries: 4): The Content-MD5 you specified did not match what we received.

How to work around that error?

Improve this question

1 Answer 1

Reset to default
1

This worked in part: md5 is wrong is should use sha256, setting this helps:

aws configure set default payload_signing_enabled = true

Second Solution
Issue more in depth is related to: http://tracker.ceph.com/issues/20447

Even deeper: https://github.com/ceph/ceph/pull/15965/files

TLDR
Some regions use ONLY "v4 signatures" (so does eu-central-1) there is a bug in the sha2 signing in awscli.

The issue only strikes on multi part uploads. If the minimal junk size ia greater than the file, the bug in part singing is not fatal:

aws configure set  multipart_chunksize = 20GB
Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .