Listing directory failure on folder within bucket with ~ in path

[katdev]

I have tried repeatedly to upload about 2100 files to a particular folder within a bucket, both all at once and in smaller chunks. At some point (which seems to vary), the upload fails, and when I try to list the current contents of the folder, I get the dreaded error:

Listing directory xxxx failed. The request signature we calculated does not match the signature you provided. Check your key and signing method. Please contact your web hosting service provider for assistance.

Note that this problem occurs on a folder within a bucket -- I can log in and see the contents of the bucket OK and can list other folder contents without a problem, so the issue is not my credentials. I can log in via the web interface or AWS cli (same credentials) and list the folder contents without a problem. I've had the issue on a few folders, while others have uploaded OK. The files in the folder have some "~" characters in them, but these don't seem to cause issues in the other folders. No equals or @ signs in the filenames, as reported in previous tickets with the same error. I have not run into this problem until yesterday (6/28/16), and tried with the latest snapshot build as well as the current release. Any help would be much appreciated!

[dkocher]

What is the name of the folder that gives the error when attempting to browse?

[katdev]

Hi -- It's 1404. 1403 works OK and is in the same enclosing folder, so I don't think the folder name . Would logs be helpful, and if so, where would I find them?

[dkocher]

Please post the transcript from the log drawer (⌘-L) from both the Browser and the Transfers window.

[katdev]

Thank you! Here's the transcript from the browser -- it includes listing of a directory that does work (1403) and another that does not (1404). Bucket name and credentials have been changed. I'll send the transfer log as soon as the current transfer fails.

GET /?max-keys=1000&prefix=MSD%20Archive%20%28Ikaros%20images%29%2F2014%2F1403%2F&delimiter=%2F HTTP/1.1
Date: Fri, 01 Jul 2016 18:51:02 GMT
x-amz-request-payer: requester
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Host: ms-archives.s3-us-west-2.amazonaws.com
x-amz-date: 20160701T185102Z
Authorization: AWS4-HMAC-SHA256 Credential=KIA97DDH2L4BF21FJBEB/20160701/us-west-2/s3/aws4_request,SignedHeaders=date;host;x-amz-content-sha256;x-amz-date;x-amz-request-payer,Signature=0a4c5a5f1cd91a83ca263ba2244d0f6d9be423fa24c2718dbd1123f9962633e1
Connection: Keep-Alive
User-Agent: Cyberduck/5.0.3.20504 (Mac OS X/10.11.5) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2: XnAlWVsgVAAOlOq/3ZHKvfpKVaMDQ6g4NS4rKJZRN6Iqvfh4TCasPs8WbwYUn/d4CVGao0NUOg4=
x-amz-request-id: B89EAD6EC64B4199
Date: Fri, 01 Jul 2016 18:51:03 GMT
x-amz-bucket-region: us-west-2
Content-Type: application/xml
Transfer-Encoding: chunked
Server: AmazonS3
GET /?max-keys=1000&prefix=MSD%20Archive%20%28Ikaros%20images%29%2F2014%2F1404%2F&delimiter=%2F HTTP/1.1
Date: Fri, 01 Jul 2016 18:51:05 GMT
x-amz-request-payer: requester
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Host: ms-archives.s3-us-west-2.amazonaws.com
x-amz-date: 20160701T185105Z
Authorization: AWS4-HMAC-SHA256 Credential=KIA97DDH2L4BF21FJBEB/20160701/us-west-2/s3/aws4_request,SignedHeaders=date;host;x-amz-content-sha256;x-amz-date;x-amz-request-payer,Signature=99ad1e14868b1d8ec163ff21d17f83211f33c13943aab253e60a3dd93e02acec
Connection: Keep-Alive
User-Agent: Cyberduck/5.0.3.20504 (Mac OS X/10.11.5) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2: 9GRzJzDAycnqnoiZA1vAlQLJEwm++BfJgKKN+rnLHsltLy5YbUp61XRoDBykRvuywtUMTzCHAUI=
x-amz-request-id: 29AA67FB0C0730ED
Date: Fri, 01 Jul 2016 18:51:06 GMT
x-amz-bucket-region: us-west-2
Content-Type: application/xml
Transfer-Encoding: chunked
Server: AmazonS3
GET /?max-keys=1000&prefix=MSD%20Archive%20%28Ikaros%20images%29%2F2014%2F1404%2F&delimiter=%2F&marker=MSD%20Archive%20%28Ikaros%20images%29%2F2014%2F1404%2FM2-14-05845%7EB1%7EA.0021.MMI HTTP/1.1
Date: Fri, 01 Jul 2016 18:51:06 GMT
x-amz-request-payer: requester
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Host: ms-archives.s3-us-west-2.amazonaws.com
x-amz-date: 20160701T185106Z
Authorization: AWS4-HMAC-SHA256 Credential=KIA97DDH2L4BF21FJBEB/20160701/us-west-2/s3/aws4_request,SignedHeaders=date;host;x-amz-content-sha256;x-amz-date;x-amz-request-payer,Signature=3b6018859d7d07546aafec867a1db49864b66f6b4630373887cbf5978a098be3
Connection: Keep-Alive
User-Agent: Cyberduck/5.0.3.20504 (Mac OS X/10.11.5) (x86_64)
HTTP/1.1 403 Forbidden
x-amz-bucket-region: us-west-2
x-amz-request-id: 48AB81AAE01D77F9
x-amz-id-2: 117gjWgy+Hx1H2q+opqzA6QNrAKvFnnFkyorAmBmgnDPsxfpSj+3YZXxFE4FLFQF7rUxq/Q6JEE=
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Fri, 01 Jul 2016 18:51:05 GMT
Server: AmazonS3
GET /?max-keys=1000&prefix=MSD%20Archive%20%28Ikaros%20images%29%2F2014%2F1404%2F&delimiter=%2F HTTP/1.1
Date: Fri, 01 Jul 2016 18:51:08 GMT
x-amz-request-payer: requester
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Host: ms-archives.s3-us-west-2.amazonaws.com
x-amz-date: 20160701T185108Z
Authorization: AWS4-HMAC-SHA256 Credential=KIA97DDH2L4BF21FJBEB/20160701/us-west-2/s3/aws4_request,SignedHeaders=date;host;x-amz-content-sha256;x-amz-date;x-amz-request-payer,Signature=5a1eb105421cb645e2f035aa3d9bd2943cadecb0c37f87a0770200ec870e2551
Connection: Keep-Alive
User-Agent: Cyberduck/5.0.3.20504 (Mac OS X/10.11.5) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2: crJOiP4uJIuXJ2uMWEl9+9pxoYVRyoan1VNlrRukgXB6q33CbOEvpAAjtJzMPqyF+zv3iWiDUto=
x-amz-request-id: BF2C53B66963C69E
Date: Fri, 01 Jul 2016 18:51:09 GMT
x-amz-bucket-region: us-west-2
Content-Type: application/xml
Transfer-Encoding: chunked
Server: AmazonS3
GET /?max-keys=1000&prefix=MSD%20Archive%20%28Ikaros%20images%29%2F2014%2F1404%2F&delimiter=%2F&marker=MSD%20Archive%20%28Ikaros%20images%29%2F2014%2F1404%2FM2-14-05845%7EB1%7EA.0021.MMI HTTP/1.1
Date: Fri, 01 Jul 2016 18:51:08 GMT
x-amz-request-payer: requester
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Host: ms-archives.s3-us-west-2.amazonaws.com
x-amz-date: 20160701T185108Z
Authorization: AWS4-HMAC-SHA256 Credential=KIA97DDH2L4BF21FJBEB/20160701/us-west-2/s3/aws4_request,SignedHeaders=date;host;x-amz-content-sha256;x-amz-date;x-amz-request-payer,Signature=d928f27e58282dd90c26dcbc89cbf2afe54f8b89c5110e894003cfc064ec5b16
Connection: Keep-Alive
User-Agent: Cyberduck/5.0.3.20504 (Mac OS X/10.11.5) (x86_64)
HTTP/1.1 403 Forbidden
x-amz-bucket-region: us-west-2
x-amz-request-id: CCE24D83E2952FDA
x-amz-id-2: kfF2vyzuJ8NGr7vObNjQ7HgkFqNuzlROpyHr2MMcGxyjz1fNTYmmkMc561n8x7BpSvfQYOWfTLA=
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Fri, 01 Jul 2016 18:51:07 GMT
Server: AmazonS3

[dkocher]

In r21066. ​Upstream changeset.

[katdev]

Thank you!! Just downloaded the latest snapshot build and everything seems to be working great. Much appreciated.