Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Multipart uploads fail if key begins or ends with whitespace character #10628

Closed
cyberduck opened this issue Feb 22, 2019 · 3 comments
Closed

Multipart uploads fail if key begins or ends with whitespace character #10628

cyberduck opened this issue Feb 22, 2019 · 3 comments
Assignees
@dkocher
Labels
bug fixed s3 AWS S3 Protocol Implementation
Milestone
6.9.4

Comments

@cyberduck
Copy link
Collaborator

b2d87cd created the issue

When uploading a multi-part file to S3 (>100MB), the upload fails if the destination path begins with a space.

Looking at the log from the transfer window, it appears that the actual upload works correctly, up until the point where a POST request is sent that excludes the leading space, which gives a 404 error from S3, and the transfer fails.

If you re-attempt to upload the same file to the same location, it fails immediately without uploading anything, presumably because it attempts to resume the existing multi-part upload. In this case it is a GET request that excludes the leading space, leading to the problem.

I've included (redacted) log files demonstrating a successful single-part upload to a path with a leading space, and first and second attempts to perform a multi-part upload.

Attachments
@cyberduck
Copy link
Collaborator Author

@dkocher commented

POST /%20Test%20Folder/Multi-Part.wav?uploads HTTP/1.1
Date: Fri, 22 Feb 2019 17:53:32 GMT
x-amz-request-payer: requester
Content-Type: audio/x-wav
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Host: [bucket-name-redacted].s3.amazonaws.com
x-amz-date: 20190222T175332Z
Authorization: ********
Content-Length: 0
Connection: Keep-Alive
User-Agent: Cyberduck/6.9.3.30061 (Mac OS X/10.14.2) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2: ++MOHaYD4MERXuE4ZjxFj8V7S2IaiEE1Hwgrr3NXBy0m5kKfekqPzrcGvp3VU1Jcmzo1HnfQ8+4=
x-amz-request-id: 3ECFCD9F9C40C6FA
Date: Fri, 22 Feb 2019 17:53:33 GMT
Transfer-Encoding: chunked
Server: AmazonS3
…
POST /Test%20Folder/Multi-Part.wav?uploadId=4ttBwSJewknBpvQw2BBQWYpX_X7B3UgkKWyazqMZ9cDZ4wKVKBi7vF5x4FlNfUTL4X3BVv6qNrsxMg_76Hp0yWE4QCIX_WFs_D1dpg7Ofcw3LnyJqayfIppndD7yKijs HTTP/1.1
Date: Fri, 22 Feb 2019 17:54:41 GMT
x-amz-request-payer: requester
Content-Type: text/plain
x-amz-content-sha256: bc11dafbf73c87aa4658637823ac9ac754f4709e75e979d312d1edb14184511b
Host: [bucket-name-redacted].s3.amazonaws.com
x-amz-date: 20190222T175441Z
Authorization: ********
Content-Length: 1079
Connection: Keep-Alive
User-Agent: Cyberduck/6.9.3.30061 (Mac OS X/10.14.2) (x86_64)
HTTP/1.1 404 Not Found

@cyberduck
Copy link
Collaborator Author

@dkocher commented

GET /Test%20Folder/Multi-Part.wav?uploadId=4ttBwSJewknBpvQw2BBQWYpX_X7B3UgkKWyazqMZ9cDZ4wKVKBi7vF5x4FlNfUTL4X3BVv6qNrsxMg_76Hp0yWE4QCIX_WFs_D1dpg7Ofcw3LnyJqayfIppndD7yKijs&max-parts=1000 HTTP/1.1
Date: Fri, 22 Feb 2019 17:55:45 GMT
x-amz-request-payer: requester
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Host: [bucket-name-redacted].s3.amazonaws.com
x-amz-date: 20190222T175545Z
Authorization: ********
Connection: Keep-Alive
User-Agent: Cyberduck/6.9.3.30061 (Mac OS X/10.14.2) (x86_64)
HTTP/1.1 404 Not Found

@cyberduck
Copy link
Collaborator Author

@ylangisc commented

In fd28391.

@iterate-ch iterate-ch locked as resolved and limited conversation to collaborators Nov 26, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@dkocher dkocher

Labels
bug fixed s3 AWS S3 Protocol Implementation
Projects
None yet
Milestone
6.9.4
Development

No branches or pull requests
2 participants
@dkocher @cyberduck