You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Downloads of 850MB or larger files fail on S3 with "access denied" error when "Segmented downloads with multiple connections per file" is checked
#11581
Closed
cyberduck opened this issue
Feb 8, 2021
· 3 comments
With the default setting (under Preferences... Transfers... General tab, Downloads section) "Segmented downloads with multiple connections per file" is checked, attempting to download a file larger than about 850MB fails after just a few seconds with an "Access Denied" error. Files 800MB or smaller are able to download successfully.
When I uncheck the "Segmented downloads with multiple connections per file" setting, I am able to download large files (850MB to 3GB range).
My guess is that this error is occurring when trying to open multiple local files for the different download file pieces under Windows. Too many files open, or too many connections? It is not an issue with the S3 access permissions. All object permissions in the S3 bucket are the same.
Side notes: I tried to turn on debug logging by adding logging=debug in a newly created default.properties file in the %APPDATA%\Cyberduck folder, and although a cyberduck.log file was created, the file remained empty. Also, under Windows the "Access Denied" dialog does not allow you to copy and paste the message details, so I am attaching screenshots of the details as a PDF.
Interesting! The total length of the full path and filename is 129 characters. The filename itself is long: 103 characters. Still, it's hard to understand why this would exceed NTFS limits, which are 255 characters for an individual file or folder name, and 260 characters by default for the total path length. (But per this article, the limitation can be removed to support up to 32,767 characters).
Ah, I see that CyberDuck's temporary download path effectively more than doubles the filename length. With a download folder of "C:\Users\Martin\Downloads\" and a 103 character long filename of "20200827_this-is-an-extremely-very-very-very-very-very-long-big-lengthy-filename-here_1of1_1080p_en.mp4", CyberDuck first creates a download folder named:
To further test this, I tried "Download as..." and gave the downloaded file a very short name. Indeed, with a short name, the file download continued without error even when "Segmented downloads with multiple connections per file" is checked.
I've configured my Windows registry (Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem\LongPathsEnabled set to 1) to allow along paths. However, I was still not able to download the long filename after restarting CyberDuck. It appears that CyberDuck does not support long Windows paths via the longPathAware element in its application manifest (per this article?
Or, alternatively, since most Windows users will not make the necessary Windows registry change to support long paths, could CyberDuck use a different naming convention for its download segments to avoid the otherwise inevitable more than doubling of the filename length?
With the default setting (under Preferences... Transfers... General tab, Downloads section)
"Segmented downloads with multiple connections per file"
is checked, attempting to download a file larger than about 850MB fails after just a few seconds with an "Access Denied" error. Files 800MB or smaller are able to download successfully.When I uncheck the
"Segmented downloads with multiple connections per file"
setting, I am able to download large files (850MB to 3GB range).My guess is that this error is occurring when trying to open multiple local files for the different download file pieces under Windows. Too many files open, or too many connections? It is not an issue with the S3 access permissions. All object permissions in the S3 bucket are the same.
Side notes: I tried to turn on debug logging by adding logging=debug in a newly created default.properties file in the %APPDATA%\Cyberduck folder, and although a cyberduck.log file was created, the file remained empty. Also, under Windows the "Access Denied" dialog does not allow you to copy and paste the message details, so I am attaching screenshots of the details as a PDF.
Attachments
Cyberduck_download_error_20210208.pdf
(866.6 KiB)The text was updated successfully, but these errors were encountered: