Cyberduck Mountain Duck CLI

#4498 closed feature (fixed)

Segmented downloads with multiple connections per file

Reported by: taylor.steven@… Owned by: dkocher
Priority: normal Milestone: 7.0
Component: core Version: 4.5.2
Severity: normal Keywords:
Cc: aj@… Architecture: Intel
Platform:

Description

Cyberduck does not seem capable of multi-segmenting files in FTP transfer. Consequently, for servers using this method of transfer, files being transferred via Cyberduck do not make full use of bandwidth and transfer far slower than they should. Although I have donated to Cyberduck development because I believe it is a very nice product, I found that for my needs, a product called Speed Download by Yazsoft does provide for multi-segmented transfers at far faster speed.

Change History (30)

comment:1 Changed on May 30, 2010 at 2:22:46 AM by dkocher

  • Component changed from ftp to core
  • Milestone 3.6 deleted
  • Priority changed from highest to normal

comment:2 Changed on Jul 27, 2010 at 5:05:10 PM by dkocher

  • Architecture set to Intel
  • Resolution set to duplicate
  • Status changed from new to closed

Actually a duplicate for #1135.

comment:3 Changed on Nov 15, 2014 at 8:56:42 AM by ajostergaard

  • Platform Mac OS X 10.6 deleted
  • Resolution duplicate deleted
  • Status changed from closed to reopened
  • Version 3.4.2 deleted

Segmented download is not the same as multi-connections (#1135) though it does depend on it... With segmented downloads a single large file is downloaded in several parts each using a separate connection. Multiple other download clients now support this - will CyberDuck support this in future?

comment:4 Changed on Nov 15, 2014 at 8:57:31 AM by ajostergaard

  • Cc aj@… added

comment:5 Changed on Nov 15, 2014 at 9:01:29 AM by dkocher

  • Milestone set to 4.6
  • Resolution set to duplicate
  • Status changed from reopened to closed
  • Version set to 4.5.2

Duplicate for #1135. We have implemented multiple connections per transfer but it is not currently enabled by default.

comment:6 follow-up: Changed on Nov 15, 2014 at 10:39:41 AM by ajostergaard

  • Resolution duplicate deleted
  • Status changed from closed to reopened

lol - this is explicitly NOT the same as #1135! If you don't want to do it that's your prerogative but at least take the time to understand the request!

Last edited on Nov 15, 2014 at 11:28:16 AM by ajostergaard (previous) (diff)

comment:7 Changed on Nov 15, 2014 at 11:28:00 AM by ajostergaard

https://whatbox.ca/wiki/Multi-threaded_and_Segmented_FTP is a good description of the difference.

comment:8 Changed on Nov 15, 2014 at 1:27:25 PM by dkocher

  • Milestone 4.6 deleted
  • Status changed from reopened to new
  • Summary changed from Multi-Segmented Transfers to Segmented transfers with multiple concurrent connections per file

comment:9 in reply to: ↑ 6 Changed on Nov 15, 2014 at 1:27:56 PM by dkocher

Replying to ajostergaard:

lol - this is explicitly NOT the same as #1135! If you don't want to do it that's your prerogative but at least take the time to understand the request!

Sorry I was just skimming over and not reading properly.

comment:10 Changed on Jan 26, 2015 at 8:59:56 AM by bobthebuilder

A thumbs up for implementing segmented downloads. -Parallel threads downloading different parts of the same files which are assembled when the download has completed - enabling a 100mbit connection (becoming more frequent) to max out the transfer speed rather than be limited by latency.

comment:11 Changed on Feb 3, 2015 at 3:58:32 PM by dkocher

  • Milestone set to 4.7
  • Status changed from new to assigned

comment:12 Changed on Feb 19, 2015 at 3:21:38 PM by dkocher

  • Resolution set to fixed
  • Status changed from assigned to closed

In r16861. The feature is not currently enabled by default. To enable, set the hidden settings queue.download.segments.threshold=0 and queue.session.pool.size=2 for two concurrent connections. When using the CLI run with env "queue.download.segments.threshold=0" env "queue.session.pool.size=1" duck ….

comment:13 Changed on Feb 20, 2015 at 4:14:33 AM by ajostergaard

Thank you! :)

comment:14 follow-up: Changed on Feb 20, 2015 at 6:36:10 AM by bobthebuilder

Fantastic that the feature is now in the code, thank you so so much. Would it be possible to add access to it via the GUI ?

comment:15 in reply to: ↑ 14 Changed on Feb 20, 2015 at 8:32:43 AM by dkocher

Replying to bobthebuilder:

Fantastic that the feature is now in the code, thank you so so much. Would it be possible to add access to it via the GUI ?

We will have to let the new feature bake for 4.7 until we will make it available through the user interface.

comment:16 Changed on Feb 20, 2015 at 10:35:47 AM by dkocher

Make sure to use r16887 which includes an important fix.

comment:17 Changed on Feb 20, 2015 at 1:53:17 PM by dkocher

As of r16903 use the setting queue.transfer.type.enabled to enable the option for concurrent connections in the Preferences → Transfers → Transfer Files… and the bookmark settings.

defaults write ~/Library/Preferences/ch.sudo.cyberduck.plist queue.transfer.type.enabled "newconnection browser concurrent" 

This will add the option Open multiple connections. Use the setting Number of concurrent transfers to control the number of parallel connections.

Last edited on Feb 20, 2015 at 1:54:43 PM by dkocher (previous) (diff)

comment:18 Changed on Feb 24, 2015 at 10:31:36 AM by dkocher

Enabled in r16977.

comment:19 Changed on Apr 17, 2015 at 2:10:09 PM by dkocher

  • Milestone changed from 4.7 to 4.8

Disabled by default in r17316. Set the hidden configuration option queue.download.segments to true to enable.

Last edited on Jan 13, 2019 at 8:58:05 PM by dkocher (previous) (diff)

comment:20 Changed on Aug 9, 2015 at 7:22:23 PM by dkocher

  • Milestone changed from 4.7.1 to 4.8

comment:21 Changed on Oct 25, 2015 at 9:07:00 PM by dkocher

  • Milestone changed from 4.7.3 to 4.8

comment:22 Changed on Mar 28, 2016 at 12:13:53 PM by dkocher

  • Milestone changed from 4.8 to 6.0

comment:23 Changed on Apr 19, 2016 at 12:24:29 PM by dkocher

  • Milestone changed from 6.0 to 5.0
  • Summary changed from Segmented transfers with multiple concurrent connections per file to Segmented downloads with multiple connections per file

comment:24 Changed on May 17, 2016 at 1:00:01 PM by dkocher

  • Milestone changed from 5.0 to 5.1

comment:25 Changed on Jan 11, 2017 at 8:33:00 AM by dkocher

  • Milestone changed from 5.1 to 6.0
  • Resolution fixed deleted
  • Status changed from closed to reopened

Reopen this issue to enable by default and add configuration option.

comment:26 Changed on Jan 11, 2017 at 8:33:37 AM by dkocher

  • Milestone changed from 6.0 to 5.3

Milestone renamed

comment:27 Changed on Jan 11, 2017 at 8:35:03 AM by dkocher

  • Milestone changed from 5.3 to 6.0

comment:28 Changed on Feb 25, 2017 at 7:14:00 PM by dkocher

  • Milestone changed from 6.0 to 7.0

comment:29 Changed on Jan 17, 2019 at 12:06:24 PM by dkocher

  • Type changed from enhancement to feature

comment:30 Changed on Feb 15, 2019 at 3:04:02 PM by dkocher

  • Resolution set to fixed
  • Status changed from reopened to closed

In r46381.

Note: See TracTickets for help on using tickets.
swiss made software