Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Opens and closes too many connections when up- or downloading several files. #9871

Closed
cyberduck opened this issue Mar 6, 2017 · 4 comments
Closed

Opens and closes too many connections when up- or downloading several files. #9871

cyberduck opened this issue Mar 6, 2017 · 4 comments
Assignees
@dkocher
Labels
bug core fixed
Milestone
5.4

Comments

@cyberduck
Copy link
Collaborator

202bb6d created the issue

Since a few months I have problems with some hosting providers, specially when downloading or uploading "large" file structures, more than 5 files can be enough...

It seems that when doing bigger transactions with lots of files and archives, that cyberduck opens and closes too much connections, thus provoking on some hostings an overflow in allowed max-user-connections...

Here a log from cyberduck when I tried to upload 7 files... the log ends when the server gave me an 421 server closed connection error...
The hosting is on mediatemple.net and I made the same experience also on a bluehost.com hosting...

What I did was take 7 local files (from the same folder) and drag them to a cyberduck connection browser with an opened connection...

If I see it right Cyberduck first checks if he can upload 7 files, then quits 7 times and then begins to upload, every file with a new connection, which (from a parallelized perspective) is great, but that kills on hostings where no more than 8 parallel ftp connections are allowed...

It seems to me that there's a connection overflow for such simple transactions. Wouldn't it be better to divide the Task-Queue into max 5 threads (for example) and then each thread is working thrugh the file uploads or downloads?

CWD /gppro.drilo.cl/html
250 OK. Current directory is /gppro.drilo.cl/html
TYPE A
200 TYPE is now ASCII
PASV
227 Entering Passive Mode (64,207,139,145,210,216)
MLSD
150 Accepted data connection
type=cdir;sizd=8;modify=20170306183221;UNIX.mode=0750;UNIX.uid=1526717;UNIX.gid=33;unique=1dg1326065; .
type=pdir;sizd=5;modify=20170306182102;UNIX.mode=0755;UNIX.uid=1526717;UNIX.gid=33;unique=1dg7ebdffe; ..
type=file;size=418;modify=20170306183210;UNIX.mode=0755;UNIX.uid=1526717;UNIX.gid=148336;unique=1dg4d3b8e9; index.php
type=file;size=43;modify=20170306182400;UNIX.mode=0755;UNIX.uid=1526717;UNIX.gid=148336;unique=1dg48ace94; info.php
type=file;size=17935;modify=20170306183210;UNIX.mode=0755;UNIX.uid=1526717;UNIX.gid=148336;unique=1dg4d3b8ea; licencia.txt
type=file;size=19935;modify=20170306183213;UNIX.mode=0755;UNIX.uid=1526717;UNIX.gid=148336;unique=1dg4d3b8fa; license.txt
type=file;size=7636;modify=20170306183213;UNIX.mode=0755;UNIX.uid=1526717;UNIX.gid=148336;unique=1dg4d3b900; readme.html
type=file;size=5447;modify=20170306183213;UNIX.mode=0755;UNIX.uid=1526717;UNIX.gid=148336;unique=1dg4d3b8ff; wp-activate.php
226-Options: -a -l 
226 8 matches total
QUIT
QUIT
221-Goodbye. You uploaded 0 and downloaded 0 kbytes.
221 Logout.
221-Goodbye. You uploaded 0 and downloaded 0 kbytes.
221 Logout.
QUIT
221-Goodbye. You uploaded 0 and downloaded 0 kbytes.
221 Logout.
QUIT
QUIT
221-Goodbye. You uploaded 0 and downloaded 0 kbytes.
221 Logout.
QUIT
221-Goodbye. You uploaded 0 and downloaded 0 kbytes.
221 Logout.
221-Goodbye. You uploaded 0 and downloaded 0 kbytes.
221 Logout.
QUIT
221-Goodbye. You uploaded 0 and downloaded 0 kbytes.
221 Logout.
TYPE I
200 TYPE is now 8-bit binary
PASV
227 Entering Passive Mode (64,207,139,145,229,31)
STOR /gppro.drilo.cl/html/wp-blog-header.php
150 Accepted data connection
226-File successfully transferred
226 0.001 seconds (measured here), 317.70 Kbytes per second
QUIT
TYPE I
221-Goodbye. You uploaded 0 and downloaded 0 kbytes.
221 Logout.
200 TYPE is now 8-bit binary
PASV
227 Entering Passive Mode (64,207,139,145,205,27)
STOR /gppro.drilo.cl/html/wp-links-opml.php
150 Accepted data connection
226-File successfully transferred
226 0.007 seconds (measured here), 343.67 Kbytes per second
QUIT
TYPE I
221-Goodbye. You uploaded 1 and downloaded 0 kbytes.
221 Logout.
200 TYPE is now 8-bit binary
PASV
227 Entering Passive Mode (64,207,139,145,226,175)
STOR /gppro.drilo.cl/html/wp-comments-post.php
150 Accepted data connection
226-File successfully transferred
226 0.008 seconds (measured here), 198.33 Kbytes per second
QUIT
TYPE I
221-Goodbye. You uploaded 3 and downloaded 0 kbytes.
221 Logout.
200 TYPE is now 8-bit binary
PASV
227 Entering Passive Mode (64,207,139,145,230,136)
STOR /gppro.drilo.cl/html/wp-load.php
TYPE I
150 Accepted data connection
200 TYPE is now 8-bit binary
PASV
227 Entering Passive Mode (64,207,139,145,211,4)
226-File successfully transferred
226 0.001 seconds (measured here), 2.84 Mbytes per second
QUIT
STOR /gppro.drilo.cl/html/wp-login.php
221-Goodbye. You uploaded 2 and downloaded 0 kbytes.
221 Logout.
150 Accepted data connection
TYPE I
200 TYPE is now 8-bit binary
PASV
227 Entering Passive Mode (64,207,139,145,222,116)
STOR /gppro.drilo.cl/html/wp-cron.php
226-File successfully transferred
226 0.577 seconds (measured here), 57.48 Kbytes per second
QUIT
150 Accepted data connection
221-Goodbye. You uploaded 34 and downloaded 0 kbytes.
221 Logout.
QUIT
226-File successfully transferred
226 0.008 seconds (measured here), 404.92 Kbytes per second
QUIT
221-Goodbye. You uploaded 0 and downloaded 0 kbytes.
221 Logout.
221-Goodbye. You uploaded 4 and downloaded 0 kbytes.
221 Logout.
QUIT
221-Goodbye. You uploaded 0 and downloaded 0 kbytes.
221 Logout.
@cyberduck
Copy link
Collaborator Author

@dkocher commented

What is your setting in the lower right of the Transfers window?

https://trac.cyberduck.io/wiki/help/en/howto/transfer#NumberofconcurrentTransfers

@cyberduck
Copy link
Collaborator Author

@dkocher commented

In 409d915 and b1b82b1. You can now choose again Open single connection for file transfers.

@cyberduck
Copy link
Collaborator Author

202bb6d commented

Great thanks... I had chosen "use connection in broswer"... I assume this is the same as "open single connection"?
The same problem appeared when choosing "open several connections"

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Milestone renamed

@iterate-ch iterate-ch locked as resolved and limited conversation to collaborators Nov 26, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@dkocher dkocher

Labels
bug core fixed
Projects
None yet
Milestone
5.4
Development

No branches or pull requests
2 participants
@dkocher @cyberduck