Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Very slow to list directory containing multiple symbolic links #6791

Closed
cyberduck opened this issue Aug 2, 2012 · 3 comments
Closed

Very slow to list directory containing multiple symbolic links #6791

cyberduck opened this issue Aug 2, 2012 · 3 comments
Labels
bug ftp FTP Protocol Implementation wontfix

Comments

@cyberduck
Copy link
Collaborator

22a98a5 created the issue

I've noticed that Cyberduck can take quite some time listing directories that contain links over FTP/FTPS connections. It seems that the more there are, the slower the process gets:

NOOP
200 Zzz...
STAT /test
213-STAT
drwxr-xr-x    2 firewing   firewing         4096 Aug  2 14:44 .
drwxr-x---   13 firewing   99               4096 Aug  2 14:44 ..
lrwxrwxrwx    1 firewing   firewing           11 Aug  2 14:44 linked_file_1 -> normal_file_1
lrwxrwxrwx    1 firewing   firewing           12 Aug  2 14:44 linked_file_2 -> normal_file_2
-rw-r--r--    1 firewing   firewing            0 Aug  2 14:44 normal_file_1
-rw-r--r--    1 firewing   firewing            0 Aug  2 14:44 normal_file_2
213 End.
CWD /test/linked_file_1
550 Can't change directory to /test/linked_file_1: Not a directory
CWD /test/normal_file_1
550 Can't change directory to /test/normal_file_1: Not a directory
CWD /test/linked_file_2
550 Can't change directory to /test/linked_file_2: Not a directory
CWD /test/normal_file_2
550 Can't change directory to /test/normal_file_2: Not a directory

2 symlinks in 4.7s

NOOP
200 Zzz...
STAT /test
213-STAT
drwxr-xr-x    2 firewing   firewing         4096 Aug  2 14:47 .
drwxr-x---   13 firewing   99               4096 Aug  2 14:44 ..
lrwxrwxrwx    1 firewing   firewing           11 Aug  2 14:44 linked_file_1 -> normal_file_1
lrwxrwxrwx    1 firewing   firewing           12 Aug  2 14:44 linked_file_2 -> normal_file_2
lrwxrwxrwx    1 firewing   firewing           13 Aug  2 14:47 linked_file_3 -> normal_file_3
lrwxrwxrwx    1 firewing   firewing           13 Aug  2 14:47 linked_file_4 -> normal_file_4
lrwxrwxrwx    1 firewing   firewing           13 Aug  2 14:47 linked_file_5 -> normal_file_5
lrwxrwxrwx    1 firewing   firewing           13 Aug  2 14:47 linked_file_6 -> normal_file_6
lrwxrwxrwx    1 firewing   firewing           13 Aug  2 14:47 linked_file_7 -> normal_file_7
lrwxrwxrwx    1 firewing   firewing           13 Aug  2 14:47 linked_file_8 -> normal_file_8
lrwxrwxrwx    1 firewing   firewing           13 Aug  2 14:47 linked_file_9 -> normal_file_9
-rw-r--r--    1 firewing   firewing            0 Aug  2 14:44 normal_file_1
-rw-r--r--    1 firewing   firewing            0 Aug  2 14:44 normal_file_2
-rw-r--r--    1 firewing   firewing            0 Aug  2 14:47 normal_file_3
-rw-r--r--    1 firewing   firewing            0 Aug  2 14:47 normal_file_4
-rw-r--r--    1 firewing   firewing            0 Aug  2 14:47 normal_file_5
-rw-r--r--    1 firewing   firewing            0 Aug  2 14:47 normal_file_6
-rw-r--r--    1 firewing   firewing            0 Aug  2 14:47 normal_file_7
-rw-r--r--    1 firewing   firewing            0 Aug  2 14:47 normal_file_8
-rw-r--r--    1 firewing   firewing            0 Aug  2 14:47 normal_file_9
213 End.
CWD /test/linked_file_1
550 Can't change directory to /test/linked_file_1: Not a directory
CWD /test/normal_file_1
550 Can't change directory to /test/normal_file_1: Not a directory
CWD /test/linked_file_2
550 Can't change directory to /test/linked_file_2: Not a directory
CWD /test/normal_file_2
550 Can't change directory to /test/normal_file_2: Not a directory
CWD /test/linked_file_3
550 Can't change directory to /test/linked_file_3: Not a directory
CWD /test/normal_file_3
550 Can't change directory to /test/normal_file_3: Not a directory
CWD /test/linked_file_4
550 Can't change directory to /test/linked_file_4: Not a directory
CWD /test/normal_file_4
550 Can't change directory to /test/normal_file_4: Not a directory
CWD /test/linked_file_5
550 Can't change directory to /test/linked_file_5: Not a directory
CWD /test/normal_file_5
550 Can't change directory to /test/normal_file_5: Not a directory
CWD /test/linked_file_6
550 Can't change directory to /test/linked_file_6: Not a directory
CWD /test/normal_file_6
550 Can't change directory to /test/normal_file_6: Not a directory
CWD /test/linked_file_7
550 Can't change directory to /test/linked_file_7: Not a directory
CWD /test/normal_file_7
550 Can't change directory to /test/normal_file_7: Not a directory
CWD /test/linked_file_8
550 Can't change directory to /test/linked_file_8: Not a directory
CWD /test/normal_file_8
550 Can't change directory to /test/normal_file_8: Not a directory
CWD /test/linked_file_9
550 Can't change directory to /test/linked_file_9: Not a directory
CWD /test/normal_file_9
550 Can't change directory to /test/normal_file_9: Not a directory

9 symlinks in 39.4s

4.7/2*9=21.2 seconds so this isn't a linear progression, which explains why things get much slower the more symlinks there are.

I've confirmed this on two different servers, although both were running Pure-FTPd (one cPanel, one not).
@cyberduck
Copy link
Collaborator Author

@dkocher commented

Unfortunately we have no better way than to CWD to determine if the symbolic link target is a directory or not.

@cyberduck
Copy link
Collaborator Author

Stewart Adam commented

(FYI: The login system seems broken, I couldn't login with the account I created and the password reset didn't help. OpenID worked OK)

Why not display the directory contents and fill in the file types / change the icon as the type is discovered? Also, if it's just probing files shouldn't we work towards making this a linear instead of exponential cost?

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Replying to [comment:2 Stewart Adam]:

(FYI: The login system seems broken, I couldn't login with the account I created and the password reset didn't help. OpenID worked OK)

Yes we are having issues with the account management. Thanks for letting me know.

Why not display the directory contents and fill in the file types / change the icon as the type is discovered?

To determine this with a deferred operation would be much better I agree.

Also, if it's just probing files shouldn't we work towards making this a linear instead of exponential cost?

It should be linear unless the server is throttling requests.

@iterate-ch iterate-ch locked as resolved and limited conversation to collaborators Nov 26, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
bug ftp FTP Protocol Implementation wontfix
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@cyberduck