Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chunked downloads result in zero sized file #7933

Closed
cyberduck opened this issue May 4, 2014 · 11 comments
Closed

Chunked downloads result in zero sized file #7933

cyberduck opened this issue May 4, 2014 · 11 comments
Assignees
Labels
bug fixed s3 AWS S3 Protocol Implementation
Milestone

Comments

@cyberduck
Copy link
Collaborator

023e870 created the issue

Hello,

I discover that when a .txt file is uploaded to a S3 bucket, CyberDuck cannot download it back, but it workds fine with s3cmd.

Headers defined on the .txt files at the upload seems to be :

  • Content-Encoding: gzip
  • Content-Type: text/plain

For info "file" cmd on Mac determines it as " UTF-8 Unicode text".

A workaround to download back the file is to :

  • modify Content-Type to "application/octet-stream"
  • delete the Content-Encoding header

Except if I miss something, it seems there is a glitch with some files type and associated headers ;-)

On the other hand, why Content-Encoding is defined to "gzip" ?

Cheers,

Cédric


Attachments

@cyberduck
Copy link
Collaborator Author

@dkocher commented

I don not understand and cannot reproduce the issue.

  • The Content-Encoding is only about the compressed content over the wire.
  • The Content-Type does not change the file transfer in any way.
  • The Finder.app will determine UTF-8 Unicode text because of the .txtextension in the filename.

It might be a server issue if the content is advertised to be compressed over the wire but is in fact sent as plain text.

@cyberduck
Copy link
Collaborator Author

023e870 commented

Replying to [comment:1 dkocher]:

I don not understand and cannot reproduce the issue.

Could you please clarify which point you don't understand ?

  • The Content-Encoding is only about the compressed content over the wire.
    Did know that, ok.
  • The Content-Type does not change the file transfer in any way.
    It seams it does, because when I change it I can download the file.
  • The Finder.app will determine UTF-8 Unicode text because of the .txtextension in the filename.
    I do not agree, "file" cmd goes read the meta-data and head of files, the extension does play any role here. An example with 1 text and one pdf :
$ file file.txt test.pdf 
file.txt: UTF-8 Unicode text
test.pdf: PDF document, version 1.5
$ mv file.txt file
$ mv test.pdf test
$ file file test 
file: UTF-8 Unicode text
test: PDF document, version 1.5

It might be a server issue if the content is advertised to be compressed over the wire but is in fact sent as plain text.

Note that it works well with other clients like s3cmd, DragonFly, so this tend to exclude the "server side" issue, don't you agree ?

@cyberduck
Copy link
Collaborator Author

023e870 commented

Hello Dkocher,

I forget to give this info when the download fail : the file is created localy, but is empty.

Still looking for a possible resolution of this issue, and I would like insist on facts that the download with other largely used s3 clients (s3cmd and DragonDisk) works well, so everything makes think that there is a problem with CyberDuck on at least my context at the download or file creation on my disk.

I plane to check my drive and do the same test on a Windows platform, will let you inform.

It would be great if you can try to reproduce it or at least give me some pointers that could help ?

Cheers,

Cédric

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Replying to [comment:3 dafresh]:

Hello Dkocher,

I forget to give this info when the download fail : the file is created localy, but is empty.

Thanks for the additional information. There is an issue with transfers not taking place when the Content-Length header is missing (the case for chunked transfers).

@cyberduck
Copy link
Collaborator Author

@dkocher commented

In 9564993.

@cyberduck
Copy link
Collaborator Author

023e870 commented

Good news.

It seems it is related with my previous opened case #7906, I think I need to improve a bit my "reporting skill".

Thanks for the great works,

Cédric

@cyberduck
Copy link
Collaborator Author

023e870 commented

Hello,

Issue seems still present with 394e8c1.

Thanks !

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Additional fix in 2b103f9.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Added test in 27e153a.

@cyberduck
Copy link
Collaborator Author

023e870 commented

I confirm, works with 1eaacb5 .

Thanks !

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Replying to [comment:12 dafresh]:

I confirm, works with 1eaacb5 .

Thanks !

Thanks for testing.

@iterate-ch iterate-ch locked as resolved and limited conversation to collaborators Nov 26, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug fixed s3 AWS S3 Protocol Implementation
Projects
None yet
Development

No branches or pull requests

2 participants