Cyberduck Mountain Duck CLI

Opened on Nov 15, 2011 at 1:21:17 PM

Closed on Mar 22, 2013 at 9:53:47 AM

Last modified on Mar 22, 2013 at 10:14:24 AM

#6368 closed enhancement (worksforme)

Selectable multipart upload max thread count

Reported by: mikk Owned by: dkocher
Priority: normal Milestone:
Component: s3 Version: 4.1.3
Severity: normal Keywords:
Cc: Architecture: Intel
Platform: Windows 7

Description

Hi!

I see in firewall, this app uses max 5 (multipart) upload thread at the same time (with one file). This is not enough always, example, if one thread speed is 100KiB/s, then 5 thread speed is only 500KiB/s.

I think good idea, if max miltipart upload thread count is selectable in S3 options GUI/ or in hidden options. I don't know what is the Amazon S3 limit, but max upload thread count is 10 or 15 I think good idea.

Thank you.

Change History (11)

comment:1 follow-up: Changed on Nov 15, 2011 at 1:38:33 PM by dkocher

  • Milestone set to 4.1.4
  • Resolution set to worksforme
  • Status changed from new to closed

The thread count is currently set to 5 indeed. You can configure this with the hidden option s3.upload.multipart.concurency.

comment:2 in reply to: ↑ 1 Changed on Nov 15, 2011 at 3:32:00 PM by mikk

Replying to dkocher:

The thread count is currently set to 5 indeed. You can configure this with the hidden option s3.upload.multipart.concurency.

Thank you :)

comment:3 follow-up: Changed on Nov 16, 2011 at 5:10:59 AM by mikk

Hi!

This settings not works correctly in 9241 version. If I set value to 5 or lower, this works, but if I set higher than 5, this is don't works, I see in firewall.

comment:4 in reply to: ↑ 3 Changed on Nov 18, 2011 at 10:06:28 AM by dkocher

Replying to mikk:

Hi!

This settings not works correctly in 9241 version. If I set value to 5 or lower, this works, but if I set higher than 5, this is don't works, I see in firewall.

It will not create more threads than the file size divided by the part size.

comment:5 follow-up: Changed on Nov 18, 2011 at 2:57:12 PM by mikk

Hi!

I know this this is normal. Test file size is about 1GB. I try delete user.config file. Start app, after exit, apply only this in config file: 

<setting name="s3.upload.multipart.concurency" value="10" />

Start app, upload start, and I see, no more active thread than 5, I see in firewall.

ps.: I cant build newer Cberduck version on windows, I get errors. Last workable version is 9263. Last tested version is 9290.

Thanks.

Last edited on Nov 18, 2011 at 3:13:04 PM by dkocher (previous) (diff)

comment:6 in reply to: ↑ 5 Changed on Nov 18, 2011 at 3:13:39 PM by dkocher

Replying to mikk:

ps.: I cant build newer Cberduck version on windows, I get errors. Last workable version is 9263. Last tested version is 9290.

Sorry for the current state of the trunk. Will get fixed soon.

comment:7 Changed on Nov 18, 2011 at 3:13:54 PM by dkocher

  • Resolution worksforme deleted
  • Status changed from closed to reopened

comment:8 Changed on Nov 18, 2011 at 3:14:02 PM by dkocher

  • Summary changed from Amazon S3 - selectable multipart upload max thread count to Selectable multipart upload max thread count

comment:9 Changed on Nov 24, 2011 at 9:21:50 PM by dkocher

  • Milestone changed from 4.2 to 4.3

comment:10 Changed on Mar 22, 2013 at 9:53:47 AM by dkocher

  • Resolution set to worksforme
  • Status changed from reopened to closed

We will leave that with a hidden configuration option and optimize this in future builds. I don' think this is something we put in the graphical Preferences the user has to bother about.

comment:11 Changed on Mar 22, 2013 at 10:14:24 AM by dkocher

  • Milestone 4.4 deleted
Note: See TracTickets for help on using tickets.