Cyberduck Mountain Duck CLI

Opened on Feb 22, 2019 at 6:04:19 PM

Closed on Feb 25, 2019 at 7:14:16 AM

#10628 closed defect (fixed)

Multipart uploads fail if key begins or ends with whitespace character

Reported by: ian_weir Owned by: dkocher
Priority: normal Milestone: 6.9.4
Component: s3 Version: 6.9.3
Severity: normal Keywords:
Cc: Architecture:
Platform:

Description

When uploading a multi-part file to S3 (>100MB), the upload fails if the destination path begins with a space.

Looking at the log from the transfer window, it appears that the actual upload works correctly, up until the point where a POST request is sent that excludes the leading space, which gives a 404 error from S3, and the transfer fails.

If you re-attempt to upload the same file to the same location, it fails immediately without uploading anything, presumably because it attempts to resume the existing multi-part upload. In this case it is a GET request that excludes the leading space, leading to the problem.

I've included (redacted) log files demonstrating a successful single-part upload to a path with a leading space, and first and second attempts to perform a multi-part upload.

Attachments (3)

Single-Part Success.txt​ (4.9 KB) - added by ian_weir on Feb 22, 2019 at 6:04:55 PM.
Successful upload of single-part file
Multi-Part Initial Failure.txt​ (15.8 KB) - added by ian_weir on Feb 22, 2019 at 6:05:21 PM.
Failure of a multi-part upload to a destination with a leading space
Multi-Part Second Attempt Failure.txt​ (3.3 KB) - added by ian_weir on Feb 22, 2019 at 6:05:45 PM.
Failure when re-attempting a multi-part upload

Download all attachments as: .zip

Change History (8)

Changed on Feb 22, 2019 at 6:04:55 PM by ian_weir

Successful upload of single-part file

Changed on Feb 22, 2019 at 6:05:21 PM by ian_weir

Failure of a multi-part upload to a destination with a leading space

Changed on Feb 22, 2019 at 6:05:45 PM by ian_weir

Failure when re-attempting a multi-part upload

comment:1 Changed on Feb 23, 2019 at 2:48:27 PM by dkocher

  • Component changed from core to s3
  • Milestone set to 6.9.4
  • Owner set to dkocher
  • Status changed from new to assigned 
POST /%20Test%20Folder/Multi-Part.wav?uploads HTTP/1.1
Date: Fri, 22 Feb 2019 17:53:32 GMT
x-amz-request-payer: requester
Content-Type: audio/x-wav
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Host: [bucket-name-redacted].s3.amazonaws.com
x-amz-date: 20190222T175332Z
Authorization: ********
Content-Length: 0
Connection: Keep-Alive
User-Agent: Cyberduck/6.9.3.30061 (Mac OS X/10.14.2) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2: ++MOHaYD4MERXuE4ZjxFj8V7S2IaiEE1Hwgrr3NXBy0m5kKfekqPzrcGvp3VU1Jcmzo1HnfQ8+4=
x-amz-request-id: 3ECFCD9F9C40C6FA
Date: Fri, 22 Feb 2019 17:53:33 GMT
Transfer-Encoding: chunked
Server: AmazonS3
…
POST /Test%20Folder/Multi-Part.wav?uploadId=4ttBwSJewknBpvQw2BBQWYpX_X7B3UgkKWyazqMZ9cDZ4wKVKBi7vF5x4FlNfUTL4X3BVv6qNrsxMg_76Hp0yWE4QCIX_WFs_D1dpg7Ofcw3LnyJqayfIppndD7yKijs HTTP/1.1
Date: Fri, 22 Feb 2019 17:54:41 GMT
x-amz-request-payer: requester
Content-Type: text/plain
x-amz-content-sha256: bc11dafbf73c87aa4658637823ac9ac754f4709e75e979d312d1edb14184511b
Host: [bucket-name-redacted].s3.amazonaws.com
x-amz-date: 20190222T175441Z
Authorization: ********
Content-Length: 1079
Connection: Keep-Alive
User-Agent: Cyberduck/6.9.3.30061 (Mac OS X/10.14.2) (x86_64)
HTTP/1.1 404 Not Found

comment:2 Changed on Feb 23, 2019 at 2:49:55 PM by dkocher 

GET /Test%20Folder/Multi-Part.wav?uploadId=4ttBwSJewknBpvQw2BBQWYpX_X7B3UgkKWyazqMZ9cDZ4wKVKBi7vF5x4FlNfUTL4X3BVv6qNrsxMg_76Hp0yWE4QCIX_WFs_D1dpg7Ofcw3LnyJqayfIppndD7yKijs&max-parts=1000 HTTP/1.1
Date: Fri, 22 Feb 2019 17:55:45 GMT
x-amz-request-payer: requester
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Host: [bucket-name-redacted].s3.amazonaws.com
x-amz-date: 20190222T175545Z
Authorization: ********
Connection: Keep-Alive
User-Agent: Cyberduck/6.9.3.30061 (Mac OS X/10.14.2) (x86_64)
HTTP/1.1 404 Not Found

comment:3 Changed on Feb 23, 2019 at 3:07:35 PM by dkocher

  • Summary changed from Multipart uploads to S3 fail if the destination path begins with a space to Multipart uploadsfail if key begins or ends with whitespace character

comment:4 Changed on Feb 23, 2019 at 5:25:57 PM by dkocher

  • Summary changed from Multipart uploadsfail if key begins or ends with whitespace character to Multipart uploads fail if key begins or ends with whitespace character

comment:5 Changed on Feb 25, 2019 at 7:14:16 AM by yla

  • Resolution set to fixed
  • Status changed from assigned to closed

In r46426.

Note: See TracTickets for help on using tickets.