Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow transfers skipping pre-scanning of folders and files to download #11614

Open
cyberduck opened this issue Mar 17, 2021 · 4 comments
Open

Allow transfers skipping pre-scanning of folders and files to download #11614

cyberduck opened this issue Mar 17, 2021 · 4 comments
Assignees
@dkocher
Labels
core enhancement

Comments

@cyberduck
Copy link
Collaborator

21f9301 created the issue

When starting to download a S3 directory full of nested subdirectories and files, the download starts with a long hang. For a directory with size 1G, the hanging can be as long as 10 minutes. Downloading the directory with aws_cli or other file transfer tools didn't encounter the same issue.

We tried both Windows 10 and macOS 11 and experienced the same issue.

Let me know if more information is needed to reproduce the issue.

external image
external image

Attachments
@cyberduck
Copy link
Collaborator Author

@dkocher commented

We are reading remote file attributes prior a file transfer which causes the delay in the Prepare… status.

@cyberduck
Copy link
Collaborator Author

21f9301 commented

Is there a way to turn off the "pre-scanning" somewhere? It would be nice if it can start the download recursively just like aws-cli by options.

If this option is not available and you think this is a good feature to add in, let me know. I can look over and push an MR about this.

@cyberduck
Copy link
Collaborator Author

3cbfc3c commented

There is currently no way to turn off or work around the pre-scanning process.

@lukechable
Copy link

If this option is not available and you think this is a good feature to add in, let me know. I can look over and push an MR about this.

Well, i have just enjoyed waiting nearly 2 days while Cyberduck is 'preparing' nearly 4tb of files. So yes, skipping the prepare phase would definitely be a time saver. Please think about adding this feature. For the love of time!

@dkocher dkocher self-assigned this Mar 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dkocher dkocher

Labels
core enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@dkocher @lukechable @cyberduck and others