Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

S3 Empty prefix causing "Failure to read attributes of <prefix>" #11752

Closed
cyberduck opened this issue Aug 4, 2021 · 3 comments
Closed

S3 Empty prefix causing "Failure to read attributes of <prefix>" #11752

cyberduck opened this issue Aug 4, 2021 · 3 comments
Assignees
@dkocher
Labels
bug high priority s3 AWS S3 Protocol Implementation wontfix

Comments

@cyberduck
Copy link
Collaborator

a11fab0 created the issue

USe Case
S3 credentials are restricted to a specific prefix (not root) "XYZ/".
Cyberduck is configured as landing path in the bookmark under "More Options/Path"

-Expected*: Cyberduck connects and displays the content of an empty folder

-Issue*
Attempt to login when there's no file with that prefix (empty prefix "folder") causes error "Failure to read attributes of ".
When there is no file with the prefix, the "folder" itself is eventually deleted by S3 (there are no actual s3 folders) and files under the prefix will eventually expire according to the lifecycle policy.

"Details"
Logs tailed via:

log stream --predicate '(process ## "Cyberduck") && (category "transcript")' --level info

Show a failed HEAD call to the prefix (as expected)

HEAD /orders/**********/ HTTP/1.1
HTTP/1.1 404 Not Found

Followed by a succeessful GET

GET /?encoding-type=url&max-keys=1&prefix=orders%2F**********%2F&delimiter=orders%2F**********%2F HTTP/1.1
HTTP/1.1 200 OK

-Notes*

  • Version 6.7.0 connects to an empty path successfully
  • aws s3 ls XXXXXX returns a successful response with empty payload

Attachments
@cyberduck
Copy link
Collaborator Author

@dkocher commented

We explicitly fail when attempting to list a prefix with a non existent placeholder file for the parent path and no key found under this prefix. This is by design to make this work as a filesystem in Mountain Duck when we want to fail when attempting to read directory contents for arbitrary prefixes.

You can workaround this by making sure you always have at least one object under the prefix specified as the default path. As of 7c7b95c.

@cyberduck
Copy link
Collaborator Author

a11fab0 commented

You can workaround this by making sure you always have at least one object under the prefix specified as the default path."''
Sounds a bit hacky, especially since lifecycle rules will cause the files to be deleted eventually.

This is by design to make this work as a filesystem in ​Mountain Duck when we want to fail when attempting to read directory contents for arbitrary prefixes."''
I'm not aware of Mountain Duck, but shouldn't Cyberduck keep returning the empty list as it used to, letting Mountain Duck fail on empty payload if it needs to?
It's odd that a feature on a new product introduces a breaking change on an existing product that should be unrelated.

Anyway, this issue has been quickly dismissed as "wontfix", so we can only tell the users who reported it to stick to aws cli or aws sdk.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Another workaround might be to make sure the prefix placeholder always exists and is excluded from the lifecycle policy. The implementation is shared among Cyberduck and Mountain Duck and we also want the Go → Go to Folder… command in Cyberduck to fail for prefixes with no keys.

@iterate-ch iterate-ch locked as resolved and limited conversation to collaborators Nov 27, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@dkocher dkocher

Labels
bug high priority s3 AWS S3 Protocol Implementation wontfix
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@dkocher @cyberduck