Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Selectable multipart upload max thread count #6368

Closed
cyberduck opened this issue Nov 15, 2011 · 7 comments
Closed

Selectable multipart upload max thread count #6368

cyberduck opened this issue Nov 15, 2011 · 7 comments
Assignees
Labels
enhancement s3 AWS S3 Protocol Implementation worksforme

Comments

@cyberduck
Copy link
Collaborator

38a28fd created the issue

Hi!

I see in firewall, this app uses max 5 (multipart) upload thread at the same time (with one file).
This is not enough always, example, if one thread speed is 100KiB/s, then 5 thread speed is only 500KiB/s.

I think good idea, if max miltipart upload thread count is selectable in S3 options GUI/ or in hidden options. I don't know what is the Amazon S3 limit, but max upload thread count is 10 or 15 I think good idea.

Thank you.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

The thread count is currently set to 5 indeed. You can configure this with the hidden option s3.upload.multipart.concurency.

@cyberduck
Copy link
Collaborator Author

38a28fd commented

Replying to [comment:1 dkocher]:

The thread count is currently set to 5 indeed. You can configure this with the hidden option s3.upload.multipart.concurency.

Thank you :)

@cyberduck
Copy link
Collaborator Author

38a28fd commented

Hi!

This settings not works correctly in 9241 version. If I set value to 5 or lower, this works, but if I set higher than 5, this is don't works, I see in firewall.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Replying to [comment:3 mikk]:

Hi!

This settings not works correctly in 9241 version. If I set value to 5 or lower, this works, but if I set higher than 5, this is don't works, I see in firewall.

It will not create more threads than the file size divided by the part size.

@cyberduck
Copy link
Collaborator Author

38a28fd commented

Hi!

I know this this is normal. Test file size is about 1GB.
I try delete user.config file. Start app, after exit, apply only this in config file:

<setting name="s3.upload.multipart.concurency" value="10" />

Start app, upload start, and I see, no more active thread than 5, I see in firewall.

ps.: I cant build newer Cberduck version on windows, I get errors. Last workable version is 9263. Last tested version is 9290.

Thanks.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Replying to [comment:5 mikk]:

ps.: I cant build newer Cberduck version on windows, I get errors. Last workable version is 9263. Last tested version is 9290.

Sorry for the current state of the trunk. Will get fixed soon.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

We will leave that with a hidden configuration option and optimize this in future builds. I don' think this is something we put in the graphical Preferences the user has to bother about.

@iterate-ch iterate-ch locked as resolved and limited conversation to collaborators Nov 26, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement s3 AWS S3 Protocol Implementation worksforme
Projects
None yet
Development

No branches or pull requests

2 participants