Opened on Nov 15, 2011 at 1:21:17 PM
Closed on Mar 22, 2013 at 9:53:47 AM
Last modified on Mar 22, 2013 at 10:14:24 AM
#6368 closed enhancement (worksforme)
Selectable multipart upload max thread count
Reported by: | mikk | Owned by: | dkocher |
---|---|---|---|
Priority: | normal | Milestone: | |
Component: | s3 | Version: | 4.1.3 |
Severity: | normal | Keywords: | |
Cc: | Architecture: | Intel | |
Platform: | Windows 7 |
Description
Hi!
I see in firewall, this app uses max 5 (multipart) upload thread at the same time (with one file). This is not enough always, example, if one thread speed is 100KiB/s, then 5 thread speed is only 500KiB/s.
I think good idea, if max miltipart upload thread count is selectable in S3 options GUI/ or in hidden options. I don't know what is the Amazon S3 limit, but max upload thread count is 10 or 15 I think good idea.
Thank you.
Change History (11)
comment:1 follow-up: ↓ 2 Changed on Nov 15, 2011 at 1:38:33 PM by dkocher
- Milestone set to 4.1.4
- Resolution set to worksforme
- Status changed from new to closed
comment:2 in reply to: ↑ 1 Changed on Nov 15, 2011 at 3:32:00 PM by mikk
Replying to dkocher:
The thread count is currently set to 5 indeed. You can configure this with the hidden option s3.upload.multipart.concurency.
Thank you :)
comment:3 follow-up: ↓ 4 Changed on Nov 16, 2011 at 5:10:59 AM by mikk
Hi!
This settings not works correctly in 9241 version. If I set value to 5 or lower, this works, but if I set higher than 5, this is don't works, I see in firewall.
comment:4 in reply to: ↑ 3 Changed on Nov 18, 2011 at 10:06:28 AM by dkocher
Replying to mikk:
Hi!
This settings not works correctly in 9241 version. If I set value to 5 or lower, this works, but if I set higher than 5, this is don't works, I see in firewall.
It will not create more threads than the file size divided by the part size.
comment:5 follow-up: ↓ 6 Changed on Nov 18, 2011 at 2:57:12 PM by mikk
Hi!
I know this this is normal. Test file size is about 1GB. I try delete user.config file. Start app, after exit, apply only this in config file:
<setting name="s3.upload.multipart.concurency" value="10" />
Start app, upload start, and I see, no more active thread than 5, I see in firewall.
ps.: I cant build newer Cberduck version on windows, I get errors. Last workable version is 9263. Last tested version is 9290.
Thanks.
comment:6 in reply to: ↑ 5 Changed on Nov 18, 2011 at 3:13:39 PM by dkocher
Replying to mikk:
ps.: I cant build newer Cberduck version on windows, I get errors. Last workable version is 9263. Last tested version is 9290.
Sorry for the current state of the trunk. Will get fixed soon.
comment:7 Changed on Nov 18, 2011 at 3:13:54 PM by dkocher
- Resolution worksforme deleted
- Status changed from closed to reopened
comment:8 Changed on Nov 18, 2011 at 3:14:02 PM by dkocher
- Summary changed from Amazon S3 - selectable multipart upload max thread count to Selectable multipart upload max thread count
comment:9 Changed on Nov 24, 2011 at 9:21:50 PM by dkocher
- Milestone changed from 4.2 to 4.3
comment:10 Changed on Mar 22, 2013 at 9:53:47 AM by dkocher
- Resolution set to worksforme
- Status changed from reopened to closed
We will leave that with a hidden configuration option and optimize this in future builds. I don' think this is something we put in the graphical Preferences the user has to bother about.
comment:11 Changed on Mar 22, 2013 at 10:14:24 AM by dkocher
- Milestone 4.4 deleted
The thread count is currently set to 5 indeed. You can configure this with the hidden option s3.upload.multipart.concurency.