Cyberduck Mountain Duck CLI

#9871 closed defect (fixed)

Opens and closes too many connections when up- or downloading several files.

Reported by: preolon Owned by: dkocher
Priority: normal Milestone: 5.4
Component: core Version: 5.3.9
Severity: normal Keywords:
Cc: Architecture:
Platform:

Description (last modified by preolon)

Since a few months I have problems with some hosting providers, specially when downloading or uploading "large" file structures, more than 5 files can be enough...

It seems that when doing bigger transactions with lots of files and archives, that cyberduck opens and closes too much connections, thus provoking on some hostings an overflow in allowed max-user-connections...

Here a log from cyberduck when I tried to upload 7 files... the log ends when the server gave me an 421 server closed connection error... The hosting is on mediatemple.net and I made the same experience also on a bluehost.com hosting...

What I did was take 7 local files (from the same folder) and drag them to a cyberduck connection browser with an opened connection...

If I see it right Cyberduck first checks if he can upload 7 files, then quits 7 times and then begins to upload, every file with a new connection, which (from a parallelized perspective) is great, but that kills on hostings where no more than 8 parallel ftp connections are allowed...

It seems to me that there's a connection overflow for such simple transactions. Wouldn't it be better to divide the Task-Queue into max 5 threads (for example) and then each thread is working thrugh the file uploads or downloads?

CWD /gppro.drilo.cl/html
250 OK. Current directory is /gppro.drilo.cl/html
TYPE A
200 TYPE is now ASCII
PASV
227 Entering Passive Mode (64,207,139,145,210,216)
MLSD
150 Accepted data connection
type=cdir;sizd=8;modify=20170306183221;UNIX.mode=0750;UNIX.uid=1526717;UNIX.gid=33;unique=1dg1326065; .
type=pdir;sizd=5;modify=20170306182102;UNIX.mode=0755;UNIX.uid=1526717;UNIX.gid=33;unique=1dg7ebdffe; ..
type=file;size=418;modify=20170306183210;UNIX.mode=0755;UNIX.uid=1526717;UNIX.gid=148336;unique=1dg4d3b8e9; index.php
type=file;size=43;modify=20170306182400;UNIX.mode=0755;UNIX.uid=1526717;UNIX.gid=148336;unique=1dg48ace94; info.php
type=file;size=17935;modify=20170306183210;UNIX.mode=0755;UNIX.uid=1526717;UNIX.gid=148336;unique=1dg4d3b8ea; licencia.txt
type=file;size=19935;modify=20170306183213;UNIX.mode=0755;UNIX.uid=1526717;UNIX.gid=148336;unique=1dg4d3b8fa; license.txt
type=file;size=7636;modify=20170306183213;UNIX.mode=0755;UNIX.uid=1526717;UNIX.gid=148336;unique=1dg4d3b900; readme.html
type=file;size=5447;modify=20170306183213;UNIX.mode=0755;UNIX.uid=1526717;UNIX.gid=148336;unique=1dg4d3b8ff; wp-activate.php
226-Options: -a -l 
226 8 matches total
QUIT
QUIT
221-Goodbye. You uploaded 0 and downloaded 0 kbytes.
221 Logout.
221-Goodbye. You uploaded 0 and downloaded 0 kbytes.
221 Logout.
QUIT
221-Goodbye. You uploaded 0 and downloaded 0 kbytes.
221 Logout.
QUIT
QUIT
221-Goodbye. You uploaded 0 and downloaded 0 kbytes.
221 Logout.
QUIT
221-Goodbye. You uploaded 0 and downloaded 0 kbytes.
221 Logout.
221-Goodbye. You uploaded 0 and downloaded 0 kbytes.
221 Logout.
QUIT
221-Goodbye. You uploaded 0 and downloaded 0 kbytes.
221 Logout.
TYPE I
200 TYPE is now 8-bit binary
PASV
227 Entering Passive Mode (64,207,139,145,229,31)
STOR /gppro.drilo.cl/html/wp-blog-header.php
150 Accepted data connection
226-File successfully transferred
226 0.001 seconds (measured here), 317.70 Kbytes per second
QUIT
TYPE I
221-Goodbye. You uploaded 0 and downloaded 0 kbytes.
221 Logout.
200 TYPE is now 8-bit binary
PASV
227 Entering Passive Mode (64,207,139,145,205,27)
STOR /gppro.drilo.cl/html/wp-links-opml.php
150 Accepted data connection
226-File successfully transferred
226 0.007 seconds (measured here), 343.67 Kbytes per second
QUIT
TYPE I
221-Goodbye. You uploaded 1 and downloaded 0 kbytes.
221 Logout.
200 TYPE is now 8-bit binary
PASV
227 Entering Passive Mode (64,207,139,145,226,175)
STOR /gppro.drilo.cl/html/wp-comments-post.php
150 Accepted data connection
226-File successfully transferred
226 0.008 seconds (measured here), 198.33 Kbytes per second
QUIT
TYPE I
221-Goodbye. You uploaded 3 and downloaded 0 kbytes.
221 Logout.
200 TYPE is now 8-bit binary
PASV
227 Entering Passive Mode (64,207,139,145,230,136)
STOR /gppro.drilo.cl/html/wp-load.php
TYPE I
150 Accepted data connection
200 TYPE is now 8-bit binary
PASV
227 Entering Passive Mode (64,207,139,145,211,4)
226-File successfully transferred
226 0.001 seconds (measured here), 2.84 Mbytes per second
QUIT
STOR /gppro.drilo.cl/html/wp-login.php
221-Goodbye. You uploaded 2 and downloaded 0 kbytes.
221 Logout.
150 Accepted data connection
TYPE I
200 TYPE is now 8-bit binary
PASV
227 Entering Passive Mode (64,207,139,145,222,116)
STOR /gppro.drilo.cl/html/wp-cron.php
226-File successfully transferred
226 0.577 seconds (measured here), 57.48 Kbytes per second
QUIT
150 Accepted data connection
221-Goodbye. You uploaded 34 and downloaded 0 kbytes.
221 Logout.
QUIT
226-File successfully transferred
226 0.008 seconds (measured here), 404.92 Kbytes per second
QUIT
221-Goodbye. You uploaded 0 and downloaded 0 kbytes.
221 Logout.
221-Goodbye. You uploaded 4 and downloaded 0 kbytes.
221 Logout.
QUIT
221-Goodbye. You uploaded 0 and downloaded 0 kbytes.
221 Logout.

Change History (8)

comment:1 Changed on Mar 6, 2017 at 7:16:44 PM by preolon

  • Description modified (diff)

comment:2 Changed on Mar 6, 2017 at 8:14:11 PM by preolon

  • Description modified (diff)

comment:3 Changed on Mar 6, 2017 at 8:16:30 PM by dkocher

  • Milestone set to 5.3.10
  • Owner set to dkocher
  • Status changed from new to assigned

comment:4 Changed on Mar 6, 2017 at 8:17:05 PM by dkocher

What is your setting in the lower right of the Transfers window?

https://trac.cyberduck.io/wiki/help/en/howto/transfer#NumberofconcurrentTransfers

comment:5 Changed on Mar 6, 2017 at 10:07:09 PM by dkocher

  • Summary changed from Cyberduck opens and closes too many connections when up- or downloading several files. to Opens and closes too many connections when up- or downloading several files.

comment:6 Changed on Mar 7, 2017 at 7:58:07 AM by dkocher

  • Resolution set to fixed
  • Status changed from assigned to closed

In r38348 and r38349. You can now choose again Open single connection for file transfers.

comment:7 Changed on Mar 7, 2017 at 5:39:32 PM by preolon

Great thanks... I had chosen "use connection in broswer"... I assume this is the same as "open single connection"? The same problem appeared when choosing "open several connections"

comment:8 Changed on Mar 8, 2017 at 9:53:50 AM by dkocher

  • Milestone changed from 5.3.10 to 5.4

Milestone renamed

Note: See TracTickets for help on using tickets.
swiss made software