Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mismatch between MD5 hash and ETag for multipart upload #8919

Closed
cyberduck opened this issue Jul 9, 2015 · 3 comments
Closed

Mismatch between MD5 hash and ETag for multipart upload #8919

cyberduck opened this issue Jul 9, 2015 · 3 comments
Assignees
Labels
bug high priority s3 AWS S3 Protocol Implementation worksforme
Milestone

Comments

@cyberduck
Copy link
Collaborator

db7f651 created the issue

For several months since buying CyberDuck I've been unable to upload large files to Amazon S3 (greater than 4 or 5 GB) about 95% of the time. The error returned is

Mismatch between MD5 hash (long string) of uploaded data and ETAG (long string) returned by the server.

I've uploaded with and without a VPN and from different locations and different computers (All Mac). Attached is the log file from yesterday (folder names obscured for security). The first upload of 10.45GB failed about 8GB into it. I retried it and it worked (VERY rare). The second file failed 17.7GB into a 26.1GB file.

GET /?max-keys=1000&prefix=XXXX.XX.-converted.videotapes%2F&delimiter=%2F HTTP/1.1
Date: Wed, 08 Jul 2015 18:58:28 GMT
x-amz-request-payer: requester
Authorization: AWS AKIAICYF5VD7M5F7J5JQ:Dcza6UzeiOyuSSUzCRFcuW2BGzI=
Host: XXXX-XXXXXX-videos.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2: y4am2JuQYXUQnnPrdhgja1qpAixHb6jgDE/KGd6e1u/aM/RujV2n6pMk8pdMBWNo
x-amz-request-id: 17127AE0D31F8BA9
Date: Wed, 08 Jul 2015 18:58:29 GMT
x-amz-bucket-region: us-east-1
Content-Type: application/xml
Transfer-Encoding: chunked
Server: AmazonS3
GET /?versioning HTTP/1.1
Date: Wed, 08 Jul 2015 18:58:28 GMT
x-amz-request-payer: requester
Authorization: AWS AKIAICYF5VD7M5F7J5JQ:beq57viq5PVQL1DiDQj8fwCf0Sw=
Host: XXX-XXXXXX-videos.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2: it75FkKMTGARsIevWq1VuUA4sStEWQoakCDe7TtyAfQoQaq6dVsj2GkksqB350Ne
x-amz-request-id: C4DEC4D20395AF2A
Date: Wed, 08 Jul 2015 18:58:29 GMT
Transfer-Encoding: chunked
Server: AmazonS3
GET /?max-keys=1000&prefix=XXXX.XX.-converted.videotapes%2F&delimiter=%2F HTTP/1.1
Date: Wed, 08 Jul 2015 18:58:29 GMT
x-amz-request-payer: requester
Authorization: AWS AKIAICYF5VD7M5F7J5JQ:fGhb848vvpg0OKN45HM+cZR1Km0=
Host: XXXX-XXXXXX-videos.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2: Fwz+JjYrmOV9fsKlLJTNnI/I+ZbulOUdv0Fzdzoeb7bEJ4BdX6Y2SPuzV8n17k7o
x-amz-request-id: 4CD54467246DE112
Date: Wed, 08 Jul 2015 18:58:29 GMT
x-amz-bucket-region: us-east-1
Content-Type: application/xml
Transfer-Encoding: chunked
Server: AmazonS3
GET /?max-keys=1000&prefix=XXXX.XX.-converted.videotapes%2F&delimiter=%2F HTTP/1.1
Date: Wed, 08 Jul 2015 20:27:52 GMT
x-amz-request-payer: requester
Authorization: AWS AKIAICYF5VD7M5F7J5JQ:NU0ekJA7ykAcmt5QzPB16oS7OmM=
Host: XXXX-XXXXXX-videos.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7.1.17920 (Mac OS X/10.10.3) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2: 1kLo6DGW4sLS2iaowUSql91seqA+m1fAymLUbj7xJqv6s24QrquuVo3A6WwEeMgc
x-amz-request-id: 9C05ED42504A1C68
Date: Wed, 08 Jul 2015 20:27:53 GMT
x-amz-bucket-region: us-east-1
Content-Type: application/xml
Transfer-Encoding: chunked
Server: AmazonS3

Attachments

@cyberduck
Copy link
Collaborator Author

db7f651 commented

Pasted text didn't keep line breaks. Attached log file

@cyberduck
Copy link
Collaborator Author

@dkocher commented

#8948 closed as duplicate.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Please update to the latest snapshot build available. We have made a change in 18136 that possibly affects write performance to S3.

@iterate-ch iterate-ch locked as resolved and limited conversation to collaborators Nov 26, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug high priority s3 AWS S3 Protocol Implementation worksforme
Projects
None yet
Development

No branches or pull requests

2 participants