Cyberduck Mountain Duck CLI

#6791 closed defect (wontfix)

Very slow to list directory containing multiple symbolic links

Reported by: firewing1 Owned by:
Priority: normal Milestone:
Component: ftp Version: 4.2.1
Severity: normal Keywords:
Cc: Architecture: Intel
Platform: Mac OS X 10.7

Description

I've noticed that Cyberduck can take quite some time listing directories that contain links over FTP/FTPS connections. It seems that the more there are, the slower the process gets:

NOOP
200 Zzz...
STAT /test
213-STAT
drwxr-xr-x    2 firewing   firewing         4096 Aug  2 14:44 .
drwxr-x---   13 firewing   99               4096 Aug  2 14:44 ..
lrwxrwxrwx    1 firewing   firewing           11 Aug  2 14:44 linked_file_1 -> normal_file_1
lrwxrwxrwx    1 firewing   firewing           12 Aug  2 14:44 linked_file_2 -> normal_file_2
-rw-r--r--    1 firewing   firewing            0 Aug  2 14:44 normal_file_1
-rw-r--r--    1 firewing   firewing            0 Aug  2 14:44 normal_file_2
213 End.
CWD /test/linked_file_1
550 Can't change directory to /test/linked_file_1: Not a directory
CWD /test/normal_file_1
550 Can't change directory to /test/normal_file_1: Not a directory
CWD /test/linked_file_2
550 Can't change directory to /test/linked_file_2: Not a directory
CWD /test/normal_file_2
550 Can't change directory to /test/normal_file_2: Not a directory

2 symlinks in 4.7s

NOOP
200 Zzz...
STAT /test
213-STAT
drwxr-xr-x    2 firewing   firewing         4096 Aug  2 14:47 .
drwxr-x---   13 firewing   99               4096 Aug  2 14:44 ..
lrwxrwxrwx    1 firewing   firewing           11 Aug  2 14:44 linked_file_1 -> normal_file_1
lrwxrwxrwx    1 firewing   firewing           12 Aug  2 14:44 linked_file_2 -> normal_file_2
lrwxrwxrwx    1 firewing   firewing           13 Aug  2 14:47 linked_file_3 -> normal_file_3
lrwxrwxrwx    1 firewing   firewing           13 Aug  2 14:47 linked_file_4 -> normal_file_4
lrwxrwxrwx    1 firewing   firewing           13 Aug  2 14:47 linked_file_5 -> normal_file_5
lrwxrwxrwx    1 firewing   firewing           13 Aug  2 14:47 linked_file_6 -> normal_file_6
lrwxrwxrwx    1 firewing   firewing           13 Aug  2 14:47 linked_file_7 -> normal_file_7
lrwxrwxrwx    1 firewing   firewing           13 Aug  2 14:47 linked_file_8 -> normal_file_8
lrwxrwxrwx    1 firewing   firewing           13 Aug  2 14:47 linked_file_9 -> normal_file_9
-rw-r--r--    1 firewing   firewing            0 Aug  2 14:44 normal_file_1
-rw-r--r--    1 firewing   firewing            0 Aug  2 14:44 normal_file_2
-rw-r--r--    1 firewing   firewing            0 Aug  2 14:47 normal_file_3
-rw-r--r--    1 firewing   firewing            0 Aug  2 14:47 normal_file_4
-rw-r--r--    1 firewing   firewing            0 Aug  2 14:47 normal_file_5
-rw-r--r--    1 firewing   firewing            0 Aug  2 14:47 normal_file_6
-rw-r--r--    1 firewing   firewing            0 Aug  2 14:47 normal_file_7
-rw-r--r--    1 firewing   firewing            0 Aug  2 14:47 normal_file_8
-rw-r--r--    1 firewing   firewing            0 Aug  2 14:47 normal_file_9
213 End.
CWD /test/linked_file_1
550 Can't change directory to /test/linked_file_1: Not a directory
CWD /test/normal_file_1
550 Can't change directory to /test/normal_file_1: Not a directory
CWD /test/linked_file_2
550 Can't change directory to /test/linked_file_2: Not a directory
CWD /test/normal_file_2
550 Can't change directory to /test/normal_file_2: Not a directory
CWD /test/linked_file_3
550 Can't change directory to /test/linked_file_3: Not a directory
CWD /test/normal_file_3
550 Can't change directory to /test/normal_file_3: Not a directory
CWD /test/linked_file_4
550 Can't change directory to /test/linked_file_4: Not a directory
CWD /test/normal_file_4
550 Can't change directory to /test/normal_file_4: Not a directory
CWD /test/linked_file_5
550 Can't change directory to /test/linked_file_5: Not a directory
CWD /test/normal_file_5
550 Can't change directory to /test/normal_file_5: Not a directory
CWD /test/linked_file_6
550 Can't change directory to /test/linked_file_6: Not a directory
CWD /test/normal_file_6
550 Can't change directory to /test/normal_file_6: Not a directory
CWD /test/linked_file_7
550 Can't change directory to /test/linked_file_7: Not a directory
CWD /test/normal_file_7
550 Can't change directory to /test/normal_file_7: Not a directory
CWD /test/linked_file_8
550 Can't change directory to /test/linked_file_8: Not a directory
CWD /test/normal_file_8
550 Can't change directory to /test/normal_file_8: Not a directory
CWD /test/linked_file_9
550 Can't change directory to /test/linked_file_9: Not a directory
CWD /test/normal_file_9
550 Can't change directory to /test/normal_file_9: Not a directory

9 symlinks in 39.4s

4.7/2*9=21.2 seconds so this isn't a linear progression, which explains why things get much slower the more symlinks there are.

I've confirmed this on two different servers, although both were running Pure-FTPd (one cPanel, one not).

Change History (4)

comment:1 Changed on Aug 17, 2012 at 7:15:29 PM by dkocher

  • Component changed from core to ftp
  • Resolution set to wontfix
  • Status changed from new to closed

Unfortunately we have no better way than to CWD to determine if the symbolic link target is a directory or not.

comment:2 follow-up: Changed on Aug 17, 2012 at 8:20:41 PM by Stewart Adam

(FYI: The login system seems broken, I couldn't login with the account I created and the password reset didn't help. OpenID worked OK)

Why not display the directory contents and fill in the file types / change the icon as the type is discovered? Also, if it's just probing files shouldn't we work towards making this a linear instead of exponential cost?

comment:3 in reply to: ↑ 2 Changed on Aug 17, 2012 at 11:11:18 PM by dkocher

Replying to Stewart Adam:

(FYI: The login system seems broken, I couldn't login with the account I created and the password reset didn't help. OpenID worked OK)

Yes we are having issues with the account management. Thanks for letting me know.

Why not display the directory contents and fill in the file types / change the icon as the type is discovered?

To determine this with a deferred operation would be much better I agree.

Also, if it's just probing files shouldn't we work towards making this a linear instead of exponential cost?

It should be linear unless the server is throttling requests.

comment:4 Changed on Aug 17, 2012 at 11:11:34 PM by dkocher

  • Summary changed from Cyberduck very slow to list directory containing multiple symbolic links over FTP to Very slow to list directory containing multiple symbolic links
Note: See TracTickets for help on using tickets.
swiss made software