Cyberduck Mountain Duck CLI

#11581 closed defect (duplicate)

Downloads of 850MB or larger files fail on S3 with "access denied" error when "Segmented downloads with multiple connections per file" is checked

Reported by: martin_w Owned by:
Priority: normal Milestone: 7.8.5
Component: core Version: 7.8.2
Severity: normal Keywords: segmented download
Cc: Architecture: Intel
Platform: Windows 10

Description

With the default setting (under Preferences... Transfers... General tab, Downloads section) "Segmented downloads with multiple connections per file" is checked, attempting to download a file larger than about 850MB fails after just a few seconds with an "Access Denied" error. Files 800MB or smaller are able to download successfully.

When I uncheck the "Segmented downloads with multiple connections per file" setting, I am able to download large files (850MB to 3GB range).

My guess is that this error is occurring when trying to open multiple local files for the different download file pieces under Windows. Too many files open, or too many connections? It is not an issue with the S3 access permissions. All object permissions in the S3 bucket are the same.

Side notes: I tried to turn on debug logging by adding logging=debug in a newly created default.properties file in the %APPDATA%\Cyberduck folder, and although a cyberduck.log file was created, the file remained empty. Also, under Windows the "Access Denied" dialog does not allow you to copy and paste the message details, so I am attaching screenshots of the details as a PDF.

Attachments (1)

Cyberduck_download_error_20210208.pdf (866.6 KB) - added by martin_w on Feb 8, 2021 at 10:14:24 PM.

Download all attachments as: .zip

Change History (5)

comment:1 Changed on Feb 9, 2021 at 10:35:09 AM by dkocher

This is possibly a duplicate of #10726. What is the length of the filename?

comment:2 Changed on Feb 10, 2021 at 8:29:37 AM by martin_w

Interesting! The total length of the full path and filename is 129 characters. The filename itself is long: 103 characters. Still, it's hard to understand why this would exceed NTFS limits, which are 255 characters for an individual file or folder name, and 260 characters by default for the total path length. (But per this article, the limitation can be removed to support up to 32,767 characters).

Ah, I see that CyberDuck's temporary download path effectively more than doubles the filename length. With a download folder of "C:\Users\Martin\Downloads\" and a 103 character long filename of "20200827_this-is-an-extremely-very-very-very-very-very-long-big-lengthy-filename-here_1of1_1080p_en.mp4", CyberDuck first creates a download folder named:

C:\Users\Martin\Downloads\20200827_this-is-an-extremely-very-very-very-very-very-long-big-lengthy-filename-here_1of1_1080p_en.mp4.cyberducksegment\

but then cannot create the first segment filename, which would be 270 characters:

C:\Users\Martin\Downloads\20200827_this-is-an-extremely-very-very-very-very-very-long-big-lengthy-filename-here_1of1_1080p_en.mp4.cyberducksegment\20200827_this-is-an-extremely-very-very-very-very-very-long-big-lengthy-filename-here_1of1_1080p_en.mp4-1.cyberducksegment

To further test this, I tried "Download as..." and gave the downloaded file a very short name. Indeed, with a short name, the file download continued without error even when "Segmented downloads with multiple connections per file" is checked.

I've configured my Windows registry (Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem\LongPathsEnabled set to 1) to allow along paths. However, I was still not able to download the long filename after restarting CyberDuck. It appears that CyberDuck does not support long Windows paths via the longPathAware element in its application manifest (per this article?

Or, alternatively, since most Windows users will not make the necessary Windows registry change to support long paths, could CyberDuck use a different naming convention for its download segments to avoid the otherwise inevitable more than doubling of the filename length?

Last edited on Feb 10, 2021 at 8:55:54 AM by martin_w (previous) (diff)

comment:3 Changed on Mar 27, 2021 at 9:59:12 AM by dkocher

  • Milestone set to 8.0
  • Resolution set to duplicate
  • Status changed from new to closed

Thanks for the profound reply. I have reopened #10726.

comment:4 Changed on Apr 8, 2021 at 7:44:27 AM by dkocher

  • Milestone changed from 8.0 to 7.8.5
Note: See TracTickets for help on using tickets.