Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Listing directory is slow with versioned bucket when looking for delete markers #10426

Closed
cyberduck opened this issue Aug 8, 2018 · 9 comments
Assignees
Labels
bug fixed s3 AWS S3 Protocol Implementation
Milestone

Comments

@cyberduck
Copy link
Collaborator

b4af655 created the issue

Just updated to the latest version and it takes forever to login to my S3 bucket. When trying to upload files it is also very slow.

Before the update today everything worked super fast, as it should be.

@cyberduck
Copy link
Collaborator Author

@ylangisc commented

Do you have a default path set to a bucket in your bookmark? If yes, is there versioning enabled for this bucket? This might be a consequence of our improved versioning support (#10357).

@cyberduck
Copy link
Collaborator Author

b4af655 commented

Hi yla,

Yes, I have a default path set and yes, my bucked has versioning enabled.

@cyberduck
Copy link
Collaborator Author

@ylangisc commented

Is there any chance to post a transcript (Cmd-L) from the authentication process?

@cyberduck
Copy link
Collaborator Author

b4af655 commented

yla,

Unfortunately I don't think I can share that, however, I did notice that in the authentication process it makes HTTP requests for all folders I have there. And I have a lot of folders:

Here's an example request:

GET /?max-keys=1&versions&prefix=apps%2Fthirdparty%2Fumarket%2F&delimiter=%2F HTTP/1.1

And there are dozens of not hundreds of such requests in that log. I assume that's what started causing the delay.
Thank you.

@cyberduck
Copy link
Collaborator Author

@ylangisc commented

Thanks, that's enough information. There reason for this behaviour is that in the latest version we try to properly decide if a folder is hidden or not for versioned buckets. For this decision we have to issue a list operation for each folder in order to see if there is at least one object without delete marker in the subtree.

@cyberduck
Copy link
Collaborator Author

b4af655 commented

Any updates here?

@cyberduck
Copy link
Collaborator Author

@ylangisc commented

In 0b255d2.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Milestone renamed

@cyberduck
Copy link
Collaborator Author

18f6d8a commented

This seems to be back - a folder with nothing but a subfolder (and a ton of deleted/glaciar'd files) takes many seconds to load. (Version 7.3.0 (32621))

@iterate-ch iterate-ch locked as resolved and limited conversation to collaborators Nov 26, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug fixed s3 AWS S3 Protocol Implementation
Projects
None yet
Development

No branches or pull requests

2 participants