Cyberduck Mountain Duck CLI

#10861 closed defect (thirdparty)

Files with same name are concatenated, not over-written

Reported by: dwinyard Owned by: dkocher
Priority: normal Milestone:
Component: sftp Version: 6.7.0
Severity: normal Keywords:
Cc: Architecture: Intel
Platform: macOS 10.13

Description

On MacOS 10.13.6, using Cyberduck Version 6.7.0 (28613), when I upload a file with the same name via SFTP to our SFTP server, the uploaded file is concatenated onto the existing file, it does not overwrite it.

The 'Existing Files' setting on the 'General' tab of the 'Transfers' preferences page is set to 'Overwrite'.

For some files types of course, this breaks the file. I have tested this with Excel (files get corrupted), txt (files are concatenated and legible with the content of the existing and new file both visible), gzip archives (the archive is concatenated and this *survives* gzip decompression so that the resulting decompressed file is also concatenated - pretty magical!).

Attached video shows this - you can see that each time I drag and drop a file into the Cyberduck browser window, the size of the remote file increases by the size of the uploaded file.

Attachments (1)

IMG_4098 (1).m4v (13.1 MB) - added by dwinyard on Nov 1, 2019 at 12:52:56 PM.
Video illustrating the issue

Change History (6)

Changed on Nov 1, 2019 at 12:52:56 PM by dwinyard

Video illustrating the issue

comment:1 Changed on Nov 3, 2019 at 9:14:36 PM by dkocher

  • Component changed from core to sftp
  • Owner set to dkocher

I cannot reproduce this against OpenSSH_7.4. What server software are you running?

comment:2 Changed on Nov 4, 2019 at 12:02:10 PM by dwinyard

Okay - so if login via command-line sftp from the same computer I'm running Cyberduck on, and use verbose, I get:

"OpenSSH_7.8p1, LibreSSL 2.6.2"

Once logged in to the SFTP server in question, if I type 'version' I get:

"SFTP protocol version 3"

I've emailed the operators of the SFTP server to find out more details of the software they're running. I could create a test account on that server for you if that would help? If so - please tell me how to securely send you account details.

comment:3 Changed on Nov 4, 2019 at 2:49:34 PM by dwinyard

Further to this - logging in with a command-line client and issuing a 'put' to the server in question over-writes an existing file. Using Cyberduck, concatenates the file. I've spoken now with the server provider, and they've checked the logs and said that Cyberduck is opening the file for writing without deleting an existing file with the same name in the directory first. Whereas, when we use the command-line client, the existing file is deleted before opening the new one for writing.

I have also changed the preferences in Cyberduck to 'Prompt' instead of 'Overwrite' and in this case CyberDuck does indeed prompt what to do - so it is checking for existing files and finding them. But, when I then click 'Overwrite' - it then still appends data instead of deleting first.

Finally - I turned on 'Upload with a temporary filename' - and that fixes the issue - in that of course CyberDuck now uploads to a temp file name, then issues a REMOVE on the existing file, before renaming the temporary file. The CyberDuck log shows this. Lines 160-185 below are with 'Overwrite' selected, and no temporary filename. Then lines 186+ are for with 'Use temporary filename' switched on.

160 OPENDIR 161 READDIR 162 READDIR 163 CLOSE 164 OPEN 165 WRITE 166 WRITE 167 WRITE 168 WRITE 169 WRITE 170 WRITE 171 WRITE 172 WRITE 173 WRITE 174 WRITE 175 WRITE 176 WRITE 177 WRITE 178 WRITE 179 WRITE 180 CLOSE 181 SETSTAT 182 OPENDIR 183 READDIR 184 READDIR 185 CLOSE 186 OPENDIR 187 READDIR 188 READDIR 189 CLOSE 190 OPEN 191 WRITE 192 WRITE 193 WRITE 194 WRITE 195 WRITE 196 WRITE 197 WRITE 198 WRITE 199 WRITE 200 WRITE 201 WRITE 202 WRITE 203 WRITE 204 WRITE 205 WRITE 206 CLOSE 207 SETSTAT 208 REMOVE 209 RENAME 210 OPENDIR 211 READDIR 212 READDIR 213 CLOSE

So 'Use temporary filename' at least provides us with a workaround at the moment, but I still feel the 'Overwrite' preference (whether prompted or set prejudicially) is not working correctly.

comment:4 Changed on Nov 8, 2019 at 9:20:21 AM by dkocher

When a file already exists, we open the file in truncate mode specified in the SFTP protocol. This forces an existing file with the same name to be truncated to zero length.

comment:5 Changed on Nov 21, 2019 at 9:34:29 PM by dkocher

  • Resolution set to thirdparty
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.
swiss made software