Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zero bytes uploaded when uploading multiple folders at once #6597

Closed
cyberduck opened this issue Mar 22, 2012 · 8 comments
Closed

Zero bytes uploaded when uploading multiple folders at once #6597

cyberduck opened this issue Mar 22, 2012 · 8 comments
Assignees
Labels
bug fixed high priority s3 AWS S3 Protocol Implementation
Milestone

Comments

@cyberduck
Copy link
Collaborator

649e914 created the issue

I have found an issue with uploading large data sets to GreenQloud (S3 compatible) and found a way to repeat the problem.

I have a hard drive that I wanted to upload to GreenQloud's StorageQloud. The root of the hard drive has about 10 folders and within those folders are more nested folders (with utf-8 names and deep nesting) and thousands of images, video files etc. roughly 300GB. If I select all of them and drag to upload in CyberDuck then the app will prepare for a very long time and then chug along and even get better and better upload speeds. However when I checked the folders on the remote service - all or most files are 0 bytes. CyberDuck silently failed to upload the contents for each object and even reported that it had transfered all of the data (in terms of transmitted data).

At first I thought this must be GreenQloud's service failing but after watching the GQ logs and finding nothing suspicious there I did a lot of testing within CyberDuck. I realised that individually uploading each root folder worked perfectly with no 0 bytes files. There could possibly be something failing in the staging/preparing phase in Cyberduck. I did see some string to date errors in the Console.app but I'm not sure if that was related to the issue.

cheers
Eiki, CEO GreenQloud

@cyberduck
Copy link
Collaborator Author

649e914 commented

btw. email me (eiki@greenqloud.com) for a test account if you need it.
-Eiki

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Replying to [6597 greenqloud]:

I did see some string to date errors in the Console.app but I'm not sure if that was related to the issue.

Please post these if they are related to Cyberduck.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

I haven't been able to reproduce the issue yet but have tested with the latest snapshot build available. Let me know if updating makes any difference.

@cyberduck
Copy link
Collaborator Author

649e914 commented

Replying to [comment:5 dkocher]:

I haven't been able to reproduce the issue yet but have tested with the latest snapshot build available. Let me know if updating makes any difference.

Thanks I will try a snapshot build

@cyberduck
Copy link
Collaborator Author

649e914 commented

Haven't been able to test it yet but immediately found a bug in the latest snapshot. I can't delete files any more. Did the your delete function for the S3 protocol change?

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Replying to [comment:7 greenqloud]:

Haven't been able to test it yet but immediately found a bug in the latest snapshot. I can't delete files any more. Did the your delete function for the S3 protocol change?

That is most possibly related to #6438 implementing the new Multiple File Delete from AWS.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Similar issue reported in #6690 for WebDAV.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Possibly fixed with a3820fc and previous changes. Can you please test with the latest snapshot build available and report back.

@iterate-ch iterate-ch locked as resolved and limited conversation to collaborators Nov 26, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug fixed high priority s3 AWS S3 Protocol Implementation
Projects
None yet
Development

No branches or pull requests

2 participants