Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Amazon S3 gives a "Request Error.. Not Found" message on some 'folders' #3347

Closed
cyberduck opened this issue Jul 17, 2009 · 7 comments
Closed
Assignees
Labels
bug fixed s3 AWS S3 Protocol Implementation
Milestone

Comments

@cyberduck
Copy link
Collaborator

078fc51 created the issue

We use Cyberduck for our Amazon S3 buckets. Since upgrading to 3.2.1 we are getting a message Request Error.. Not Found. trying to open some folders within the bucket. The transcript says...

x-amz-id-2: i8P2jW5UuoHZ5iqm++iCS1TyVvyHI5pPZazXkRGDRsmbF+K7BtlaBuv+zXSKfO63[\r][\n]
Content-Type: application/xml[\r][\n]
Transfer-Encoding: chunked[\r][\n]
Date: Fri, 17 Jul 2009 16:36:56 GMT[\r][\n]
Server: AmazonS3[\r][\n]
[\r][\n]

I switched back to 3.2 and these folders open fine.

Some folders are still ok in 3.2.1. The only thing I can see is that the ones that give the error have lots of files inside them.


Attachments

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Is there a specific naming pattern you can see for these folders or other unique characteristics that may help me to reproduce the problem.

@cyberduck
Copy link
Collaborator Author

078fc51 commented

Here is a table of the folder names, a Y if they open in 3.2.1 an N if they don't and the number of files they contain...

||css||N||38
||dynamic||N||98
||files||Y||1
||flash||Y||1
||img||N||87
||js||N||26

The problem seems to be somehow connected to zero size files.

I looked in each of the above folders...

The 'files' folder which works in 3.2.1 contains one sub folder and nothing else. There are subfolders within that folder.

Folder 'flash' which also works on 3.2.1 contains a single .swf file.

All the folders that do not work with 3.2.1 contain a mixture of files AND subfolders. There all also contain some files with names like ajax_$folder$ which are 0B in size. I believe these are 'folder aliases' created by applications other than Cyberduck. We have used Interarchy and S3Fox previously. I've attached a screen shot of the js folder contents called js_folder.

To try and pin down the problem I thought I would download the js folder (being the smallest non working example), upload it with a different name and then delete the odd azero byte files to see if that made any difference in 3.2.1 but I'm actually having some trouble with 3.2

I received some errors, it doesn't seem to like the sub-folders. I got this message...

I/O Error: Cannot read file attributes
/mybucket.name.edited/js/ajax
S3 GET failed for '/js%2Fajax'

The forward slash between the js and the ajax is being escaped?

I got similar messages for 'ticker' and 'js'. All 3 are zero B files.

However despite the error messages the folder did download (see second screenshot js_folder_desktop)

I uploaded a copy of the folder, calling it js_copy with no errors.

I then opened this newly uploaded js_copy folder in 3.2.1 with no problem, I am still unable to open the original js folder in 3.2.1

@cyberduck
Copy link
Collaborator Author

0180f64 commented

I've pinned this down to a zero byte file inside folders created with Interarchy. I've tested with the latest version Interarchy 9.0.1/5484 and this is 100% reproducible.

Create a folder in your S3 bucket using Interarchy. Put a normal file inside. In this case 'test.txt'

Launch Cyberduck 3.2.1 open the S3 bucket, try to open the folder. "Request Error.. Not Found."

Using Cyberduck 3.2 open the S3 bucket, open the folder, see the 'test.txt' file. No problems. Notice also a zero byte file with the same name as the parent folder.

Delete the zero byte file, you will need to use S3Fox (or maybe some other S3 apps but not Cyberduck that doesn't seem to be able to delete it).

Now use Cyberduck 3.2.1 again, open the S3 bucket, open the folder, see the 'test.txt' file. No problems.

While I suspect Interarchy is to blame here, this did work in Cyberduck 3.2 but is now broken in Cyberduck 3.2.1

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Thanks very much for the detailed report. I will look into this as soon as possible.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Replying to [comment:4 Paul Willis <info@…>]:

I've pinned this down to a zero byte file inside folders created with Interarchy.

What Interarchy does is create keys named /bucket/folder/ where the full key is folder/ but if displayed hierarchically the last part of the key is an empty string. Cyberduck always normalizes path references internally therefore this actually points back to the parent key.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

In 4981.

@cyberduck
Copy link
Collaborator Author

kiddailey commented

Just FYI for those that stumble on this as I did:

-Be warned that you should not use 3.2 to delete the 0byte files causing this issue, or you may loose files!*

I reverted to 3.2 to try and work-around this until the next update and discovered that when I tried to delete the 0byte file out of a folder, Cyberduck 3.2 immediately started deleting ALL files within that folder -- and it's pretty obvious why:

1.) /bucket/myfolder contains a file called "myfolder" that is 0bytes and 3GB of other files ...

2.) Cyberduck issues the command "Delete /buckey/myfolder" in an attempt to delete the 0byte file ...

3.) Poof!

4.) Spend the next few hours re-uploading 1GB worth of files that got wiped before you realized what was happening :)

@iterate-ch iterate-ch locked as resolved and limited conversation to collaborators Nov 26, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug fixed s3 AWS S3 Protocol Implementation
Projects
None yet
Development

No branches or pull requests

2 participants