Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[S3 compatible - OpenStack Swift] Multipart upload chunks are not deleted after overwrite operation #11690

Closed
cyberduck opened this issue May 30, 2021 · 4 comments
Closed

[S3 compatible - OpenStack Swift] Multipart upload chunks are not deleted after overwrite operation #11690

cyberduck opened this issue May 30, 2021 · 4 comments
Labels
bug openstack OpenStack Swift Protocol Implementation thirdparty Issue caused by third party

Comments

@cyberduck
Copy link
Collaborator

28592ff created the issue

I have noticed that when we overwrite a file with s3 protocol, the data chunks are still present for both the upload id in +segments container.

Storage: Openstack Swift - S3 Compatible.
Steps to reproduce the bug.

  1. Upload a file - SLO/DLO
  2. Reupload the file with overwrite option.
  3. Check the +segments containers in openstack swift. It should have both the upload ID with data chunks.

Expected behavior: Data chunks in older upload ID folder should have been deleted.

Cyberduck Version: 7.9.2
Platform: Mac, Windows

Screenshots:
Two folders with different upload ID after successful overwrite: https://imgur.com/S4hY9zY
Screenshot from Cyberduck showing successful overwrite: https://imgur.com/b1zTVlZ

Attachments
@cyberduck
Copy link
Collaborator Author

@dkocher commented

We do delete segments of large file uploads when connected to OpenStack Swift. My understanding is that when connecting with S3 interoperability this would be the responsibility of the server as S3 does not have such a concept.

@cyberduck
Copy link
Collaborator Author

28592ff commented

Here is another aspect I observed: When I issue a normal delete, the chunks in +segments folder is deleted properly. It's only with overwrites that I hit the issue.

@cyberduck
Copy link
Collaborator Author

28592ff commented

Closing this. Seems to be bug in openstack swift.

Future Reference: https://bugs.launchpad.net/swift/+bug/1813202

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Replying to [comment:2 highlow]:

Here is another aspect I observed: When I issue a normal delete, the chunks in +segments folder is deleted properly. It's only with overwrites that I hit the issue.

Overwriting files from a transfer will not explicitly delete files and thus we do not handle this use case.

@iterate-ch iterate-ch locked as resolved and limited conversation to collaborators Nov 27, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
bug openstack OpenStack Swift Protocol Implementation thirdparty Issue caused by third party
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@cyberduck