Cyberduck Mountain Duck CLI

Opened 8 years ago

Last modified 17 months ago

#4498 reopened enhancement

Segmented downloads with multiple connections per file

Reported by: taylor.steven@… Owned by: dkocher
Priority: normal Milestone: 7.0
Component: core Version: 4.5.2
Severity: normal Keywords:
Cc: aj@… Architecture: Intel
Platform:

Description

Cyberduck does not seem capable of multi-segmenting files in FTP transfer. Consequently, for servers using this method of transfer, files being transferred via Cyberduck do not make full use of bandwidth and transfer far slower than they should. Although I have donated to Cyberduck development because I believe it is a very nice product, I found that for my needs, a product called Speed Download by Yazsoft does provide for multi-segmented transfers at far faster speed.

Change History (28)

comment:1 Changed 8 years ago by dkocher

  • Component changed from ftp to core
  • Milestone 3.6 deleted
  • Priority changed from highest to normal

comment:2 Changed 8 years ago by dkocher

  • Architecture set to Intel
  • Resolution set to duplicate
  • Status changed from new to closed

Actually a duplicate for #1135.

comment:3 Changed 4 years ago by ajostergaard

  • Platform Mac OS X 10.6 deleted
  • Resolution duplicate deleted
  • Status changed from closed to reopened
  • Version 3.4.2 deleted

Segmented download is not the same as multi-connections (#1135) though it does depend on it... With segmented downloads a single large file is downloaded in several parts each using a separate connection. Multiple other download clients now support this - will CyberDuck support this in future?

comment:4 Changed 4 years ago by ajostergaard

  • Cc aj@… added

comment:5 Changed 4 years ago by dkocher

  • Milestone set to 4.6
  • Resolution set to duplicate
  • Status changed from reopened to closed
  • Version set to 4.5.2

Duplicate for #1135. We have implemented multiple connections per transfer but it is not currently enabled by default.

comment:6 follow-up: Changed 4 years ago by ajostergaard

  • Resolution duplicate deleted
  • Status changed from closed to reopened

lol - this is explicitly NOT the same as #1135! If you don't want to do it that's your prerogative but at least take the time to understand the request!

Last edited 4 years ago by ajostergaard (previous) (diff)

comment:7 Changed 4 years ago by ajostergaard

https://whatbox.ca/wiki/Multi-threaded_and_Segmented_FTP is a good description of the difference.

comment:8 Changed 4 years ago by dkocher

  • Milestone 4.6 deleted
  • Status changed from reopened to new
  • Summary changed from Multi-Segmented Transfers to Segmented transfers with multiple concurrent connections per file

comment:9 in reply to: ↑ 6 Changed 4 years ago by dkocher

Replying to ajostergaard:

lol - this is explicitly NOT the same as #1135! If you don't want to do it that's your prerogative but at least take the time to understand the request!

Sorry I was just skimming over and not reading properly.

comment:10 Changed 3 years ago by bobthebuilder

A thumbs up for implementing segmented downloads. -Parallel threads downloading different parts of the same files which are assembled when the download has completed - enabling a 100mbit connection (becoming more frequent) to max out the transfer speed rather than be limited by latency.

comment:11 Changed 3 years ago by dkocher

  • Milestone set to 4.7
  • Status changed from new to assigned

comment:12 Changed 3 years ago by dkocher

  • Resolution set to fixed
  • Status changed from assigned to closed

In r16861. The feature is not currently enabled by default. To enable, set the hidden settings queue.download.segments.threshold=0 and queue.session.pool.size=2 for two concurrent connections. When using the CLI run with env "queue.download.segments.threshold=0" env "queue.session.pool.size=1" duck ….

comment:13 Changed 3 years ago by ajostergaard

Thank you! :)

comment:14 follow-up: Changed 3 years ago by bobthebuilder

Fantastic that the feature is now in the code, thank you so so much. Would it be possible to add access to it via the GUI ?

comment:15 in reply to: ↑ 14 Changed 3 years ago by dkocher

Replying to bobthebuilder:

Fantastic that the feature is now in the code, thank you so so much. Would it be possible to add access to it via the GUI ?

We will have to let the new feature bake for 4.7 until we will make it available through the user interface.

comment:16 Changed 3 years ago by dkocher

Make sure to use r16887 which includes an important fix.

comment:17 Changed 3 years ago by dkocher

As of r16903 use the setting queue.transfer.type.enabled to enable the option for concurrent connections in the Preferences → Transfers → Transfer Files… and the bookmark settings.

defaults write ~/Library/Preferences/ch.sudo.cyberduck.plist queue.transfer.type.enabled "newconnection browser concurrent" 

This will add the option Open multiple connections. Use the setting Number of concurrent transfers to control the number of parallel connections.

Last edited 3 years ago by dkocher (previous) (diff)

comment:18 Changed 3 years ago by dkocher

Enabled in r16977.

comment:19 Changed 3 years ago by dkocher

  • Milestone changed from 4.7 to 4.8

Disabled by default in r17316. Set the hidden configuration option queue.download.segments.threshold to 0 to enable.

comment:20 Changed 3 years ago by dkocher

  • Milestone changed from 4.7.1 to 4.8

comment:21 Changed 3 years ago by dkocher

  • Milestone changed from 4.7.3 to 4.8

comment:22 Changed 2 years ago by dkocher

  • Milestone changed from 4.8 to 6.0

comment:23 Changed 2 years ago by dkocher

  • Milestone changed from 6.0 to 5.0
  • Summary changed from Segmented transfers with multiple concurrent connections per file to Segmented downloads with multiple connections per file

comment:24 Changed 2 years ago by dkocher

  • Milestone changed from 5.0 to 5.1

comment:25 Changed 18 months ago by dkocher

  • Milestone changed from 5.1 to 6.0
  • Resolution fixed deleted
  • Status changed from closed to reopened

Reopen this issue to enable by default and add configuration option.

comment:26 Changed 18 months ago by dkocher

  • Milestone changed from 6.0 to 5.3

Milestone renamed

comment:27 Changed 18 months ago by dkocher

  • Milestone changed from 5.3 to 6.0

comment:28 Changed 17 months ago by dkocher

  • Milestone changed from 6.0 to 7.0
Note: See TracTickets for help on using tickets.
swiss made software