Cyberduck Mountain Duck CLI

#10834 closed defect (thirdparty)

Read timed out failure for large file upload

Reported by: binba Owned by:
Priority: normal Milestone:
Component: ftp Version: 7.1
Severity: normal Keywords:
Cc: Architecture: Intel
Platform: macOS 10.13

Description (last modified by binba)

This one has me astonished... Steps to reproduce:

  1. Upload a large file.
  2. Upload reaches 100.0%.
  3. Cyberduck deletes the remote file it just uploaded, and restarts the upload (this time at a speed that is several times slower than the available bandwidth and original speed, for some reason).

My guess is that the threshold is 2GB: I uploaded a 1.1GB file that did not exhibit that behavior, followed by a 3.2GB file that did.

I have experienced this behavior in two different systems working with different files and different FTP servers.

  • Cyberduck 6.9.4 on OS X 10.10.5, via FTP.
  • Cyberduck 7.1.0 on OS X 10.13.6, via FTPS.

Same behavior.

This has been reproducible, happens every time to me.

Tried with "upload with temporary filename" both enabled and disabled - same behavior. "Always overwrite when reloading" has always been disabled, and "Existing files" is set to "Prompt".

Only workaround I found is... to use FileZilla. It doesn't exhibit this problem at all, with the same files and FTP server. Needless to say, this is pretty insane/maddening. But I've been loyal to Cyberduck for years and am willing to help troubleshooting this.

Log attached as file.

Attachments (2)

Cyberduck_log_10834.txt (5.4 KB) - added by binba on Oct 7, 2019 at 3:31:27 AM.
Log
Cyberduck debug log 20191108.txt (806.4 KB) - added by binba on Nov 9, 2019 at 3:23:52 AM.
Debug log (system.log) 2019-11-08

Download all attachments as: .zip

Change History (12)

Changed on Oct 7, 2019 at 3:31:27 AM by binba

Log

comment:1 Changed on Oct 7, 2019 at 3:33:02 AM by binba

  • Description modified (diff)

comment:2 Changed on Oct 23, 2019 at 7:38:58 PM by dkocher

DELE /public_html/clients/nevenka/Complete performance.mp4-nC9toDgL
250 Deleted /public_html/clients/nevenka/Complete performance.mp4-nC9toDgL

comment:3 Changed on Oct 23, 2019 at 7:40:50 PM by dkocher

The log indicates that Upload with temporary filename is enabled.

comment:4 follow-up: Changed on Oct 23, 2019 at 7:42:25 PM by dkocher

I cannot seem to reproduce the issue

STOR /sandbox/TRAC-10834/10-3.png-OCrI0FZC
150 Ok to send data.
226 Transfer complete.
RNFR /sandbox/TRAC-10834/10-3.png-OCrI0FZC
350 Ready for RNTO.
RNTO /sandbox/TRAC-10834/10-3.png
250 Rename successful.

comment:5 Changed on Oct 23, 2019 at 7:47:06 PM by dkocher

Please post the transcript from the log drawer of the Transfers window. Choose ⌘-L on Mac or right-click the toolbar from the Transfers window and choose Log on Windows.

comment:6 Changed on Nov 9, 2019 at 1:38:20 AM by binba

Updated to latest 7.1.2, still happens. Here is the transfer log:

CWD /public_html/clients/nevenka
250 OK. Current directory is /public_html/clients/nevenka
TYPE A
200 TYPE is now ASCII
PASV
227 Entering Passive Mode (198,252,102,197,124,106)
MLSD
150 Accepted data connection
type=cdir;sizd=4096;modify=20191026195506;UNIX.mode=0755;UNIX.uid=1121;UNIX.gid=1126;unique=821g6724c1c; .
type=pdir;sizd=4096;modify=20191006061520;UNIX.mode=0750;UNIX.uid=1121;UNIX.gid=99;unique=821g6722e9e; ..
type=file;size=129;modify=20191005182755;UNIX.mode=0644;UNIX.uid=1121;UNIX.gid=1126;unique=821g6724c1f; .htaccess
type=file;size=337;modify=20190202202357;UNIX.mode=0644;UNIX.uid=1121;UNIX.gid=1126;unique=821g6724c1d; index.php
226-Options: -a -l 
226 4 matches total
TYPE I
200 TYPE is now 8-bit binary
PASV
227 Entering Passive Mode (198,252,102,197,127,233)
STOR /public_html/(.....).mp4
150 Accepted data connection
TYPE I
200 TYPE is now 8-bit binary
PASV
227 Entering Passive Mode (198,252,102,197,122,225)
STOR /public_html/(.....).mp4
150 Accepted data connection

comment:7 in reply to: ↑ 4 Changed on Nov 9, 2019 at 1:41:22 AM by binba

Replying to dkocher:

I cannot seem to reproduce the issue

.

dkocher, as stated, the issue is with large files. Please try with uploads >2GB, not a PNG.

Last edited on Nov 9, 2019 at 2:59:47 AM by binba (previous) (diff)

comment:8 Changed on Nov 9, 2019 at 3:21:43 AM by binba

I enabled debugging defaults write ~/Library/Preferences/ch.sudo.cyberduck.plist logging debug

The debug log is chock full of pretty-private data - client folders, files, usernames, anything transferred that's in the history... basically everything except passwords. Not great at all for sharing online. But I was able to scrub things though, and attached.

The test file name is "Complete performance.mp4". I believe the "good stuff" is between 18:57:27 and 18:57:57.

Nov  8 18:57:57 MacPro Cyberduck[4362]: [QRLltKrD-transfer-1] WARN  ch.cyberduck.core.worker.AbstractTransferWorker - Failure transferring TransferItem{remote=Path{path='/public_html/xxxxxx/xxxxxx/Complete performance.mp4', type=[file]}, local=Local{path='/Volumes/Barnyard 2 /xxxxxx 2019-09-21/Outputs/Complete performance.mp4'}, lockId='null'}. BackgroundException{file=Path{path='/', type=[directory]}, message='Connection timed out', detail='Read timed out.', class='ch.cyberduck.core.exception.ConnectionTimeoutException', cause='java.net.SocketTimeoutException: Read timed out'}

This would seem to suggest that the issue is not about file size, but lengthy transfers, that get identified as failed & timed out even though they didn't and were successful.

If I'm not mistaken, I canceled the operation at 18:58:36.

Last edited on Nov 9, 2019 at 3:31:26 AM by binba (previous) (diff)

Changed on Nov 9, 2019 at 3:23:52 AM by binba

Debug log (system.log) 2019-11-08

comment:9 Changed on Dec 3, 2019 at 10:17:34 AM by dkocher

  • Priority changed from high to normal
  • Severity changed from critical to normal
  • Summary changed from Large files are immediately deleted the moment uploading finishes, in an infinite loop to Read timed out failure for large file upload

comment:10 Changed on Dec 12, 2019 at 9:18:37 AM by dkocher

  • Resolution set to thirdparty
  • Status changed from new to closed

Please check the server logs for any reason why it stops responding.

Note: See TracTickets for help on using tickets.
swiss made software