Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Interoperability with leading / in key name #8064

Closed
cyberduck opened this issue Jun 26, 2014 · 15 comments
Closed

Interoperability with leading / in key name #8064

cyberduck opened this issue Jun 26, 2014 · 15 comments
Assignees
Labels
enhancement low priority openstack OpenStack Swift Protocol Implementation wontfix

Comments

@cyberduck
Copy link
Collaborator

bccfca5 created the issue

I have a container with a file called "/x/y.pdf"
Cyberduck finds the container and lists the contents, but it shows as y.pdf
When I try to download, sync, view, get status of the file it always fails with a 404 because https://identity.api.rackspacecloud.com//y.pdf

This turns out to be inconsistent handling of the leading /. Rackspace don't treat /x/ as a placeholder folder, but cyberduck does.

Have tagged it as openstack as a guess.

Log drawer not interesting as it doesn't show the error, just successfully getting the files list.

GET /v1/MossoClouxxxxxxxxxxxxxxxxxxxxxxx/xxxxx4742?format=xml&prefix=&limit=10000&delimiter=%2F HTTP/1.1
X-Auth-Token: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Host: storage101.lon3.clouddrive.com
Connection: Keep-Alive
User-Agent: Cyberduck/4.4.5 (14721) (Windows 7/6.1) (x86)
HTTP/1.1 200 OK
Content-Length: 5069
X-Container-Object-Count: 22
Accept-Ranges: bytes
X-Timestamp: 1401738255.23634
X-Container-Bytes-Used: 21453741
Content-Type: application/xml; charset=utf-8
X-Trans-Id: txcf8bxxxxxxxxxxxxxxb7-0053ac31celon3
Date: Thu, 26 Jun 2014 14:44:30 GMT
@cyberduck
Copy link
Collaborator Author

@dkocher commented

In cab6e79 and 59b0901.

@cyberduck
Copy link
Collaborator Author

bccfca5 commented

Hi

Many thanks for the speedy investigation, but it still doesn't work for me.

Not sure what the differences are between your successful test and my failure - so here are some hopefully salient bits..

I'm on the beta build 14906 - hopefully that includes revision 14866 (snapshot is the same build number)

Not sure if the placeholder folder is supposed to be shown in the main tree (it doesn't, it's just a flat view of all files), but on the transfers dialog the URL us shown as clientfiles-4742/x.pdf, ie no placeholder.

I'm using / as the separator in my cloud files, and the files often begin with a leading / but am on Windows (you use path.DELIMETER i think - my java is rusty (>12 years ago) so I don't remember if this is the defined platform value or fixed)

My connection uses the Rackspace (UK) profile - but I get the same with Swift (Openstack) and Rackspace (US). I created a new bookmark in case there was a cached value.

Turned on debug logging. the first ref to my file is this - there is no reference to the placeholder/path.. (may be irrelevant)

2014-07-22 12:43:25,275 [Thread-0] DEBUG RegistryApplicationFinder - GetRegisteredDefaultApplication for filename Address proof xxxxx.pdf

and when I start the transfer.. still no path.

2014-07-22 12:43:25,987 [background-1] DEBUG ch.cyberduck.core.transfer.Queue - Add transfer Transfer{transferred=null, size=null, roots=[TransferItem{local=Local{path='C:\Users\paul\Downloads\Address proof xxxxx.pdf'}, remote=Path{path='/container-nnnnnn/Address proof xxxx.pdf', type=[file]}}], state=stopped, host=Host{credentials=Credentials{user='xxxxxxx'}, hostname='lon.identity.api.rackspacecloud.com', port=443, protocol=Profile{parent=swift, image=Local{path='C:\Users\paul\AppData\Local\Temp\\xxxxxxxxxxxxxico'```}

}}}

..and not surprisingly I get 404.

Any further assistance let me know. 

Paul

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Please update to the latest snapshot build available. Should be fixed with 5fea16e.

@cyberduck
Copy link
Collaborator Author

bccfca5 commented

Really sorry, still not working - same symptoms as before as far as I can see - on snapshot 4.5.2 (14943)

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Reading the ticket description again I now understand that this issue describes that objects in placeholders are actually displayed in the root of the container instead in the browser and the placeholder path is omitted.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Please post again a current transcript from the log drawer (⌘-L). You should see requests with the format

GET /v1/MossoCloudFS_59113590-c679-46c3-bf62-9d7c3d5176ee/test.cyberduck.ch?format=xml&prefix=<x>%2F&limit=10000&delimiter=%2F HTTP/1.1

@cyberduck
Copy link
Collaborator Author

bccfca5 commented

Arghh! This is frustrating! I'm on 15069. Still NOT working for me.

Certainly the files are displayed flat - with no placeholder dir - I'd say that was a strong(ish) hint that the placeholder dir has been lost processing the file list from rackspace

One oddity is that I can't connect using openswift profile- I use Rackspace US profile (even though the DC is in london). however the log implies the swift helper is being used

It looks to me like the placeholder is not extracted...

2014-09-09 22:44:34,043 [Thread-0] DEBUG ch.cyberduck.ui.AbstractController - Run action WorkerBackgroundAction{worker=SessionListWorker{directory=Path{path='/clientfiles-xxxx', type=[directory, volume]}}, result=null} in background
2014-09-09 22:44:34,044 [Thread-0] DEBUG ch.cyberduck.ui.AbstractController - Synchronize on lock Session{host=Host{credentials=Credentials{user='username'}, hostname='identity.api.rackspacecloud.com', port=443, protocol=Profile{parent=swift, image=Local{path='C:\Users\paul\AppData\Local\Temp\\4bfc20b0-9fb1-4bb6-afd0-56421f079db2.ico'```, state=closed} for action WorkerBackgroundAction{worker=SessionListWorker{directory=Path{path='/clientfiles-xxxx', type=[directory, volume]}}, result=null}
}}}



2014-09-09 22:44:46,308 [background-1] DEBUG ch.cyberduck.core.Profile - No value for key:Scheme ??


I see this..


2014-09-09 22:44:50,667 [background-1] DEBUG ch.cyberduck.core.threading.AbstractBackgroundAction - Finish background task WorkerBackgroundAction{worker=SessionListWorker{directory=Path{path='/clientfiles-xxxx', type=[directory, volume]}}, result=[Path{path='/clientfiles-xxxx/Address proof xxxx.pdf', type=[file]}, ...
}]}


So by this point already the placeholders have been lost/removed

My code uses the .net library openstack, and calling listfiles on the container I get the first file as /Admin/Address proof xxxx.pdf (is the leading / a problem??)

and then on the transfer..


2014-09-09 22:34:02,944 [background-1] DEBUG ch.cyberduck.ui.threading.TransferCollectionBackgroundAction - Finish background action for transfer Transfer{transferred=null, size=null, roots=[TransferItem{local=Local{path='C:\Users\paul\Downloads\Address proof xxxxx.pdf'}, remote=Path{path='/clientfiles-xxxx/Address proof xxxx.pdf', type=[file]}}], state=stopped, host=Host{credentials=Credentials{user='username'}, hostname='identity.api.rackspacecloud.com', port=443, protocol=Profile{parent=swift, image=Local{path='C:\Users\paul\AppData\Local\Temp\b660f76b-bb93-4b9f-a0f5-fe795a51ab8b.ico'```}
}}}

What's the difference with my setup and yours?

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Can you please contact Rackspace support and confirm your layout in the container adheres to the recommendation in Pseudo hierarchical folders and directories.

@cyberduck
Copy link
Collaborator Author

bccfca5 commented

It is in fact the leading / I have on my files that is breaking it.
Removed the leading / and their web interface now presents a hierarchy (never seen it before so I didn't realise it would do that) and it's replicated in Cyberduck.

Well we got there in the end.

Thanks for your help

@cyberduck
Copy link
Collaborator Author

bccfca5 commented

..so there is still a bug, and that is that files beginning with a / can't be downloaded - there's a discrepancy in the handling - you think it's a placeholder folder, but Rackspace don't treat it as such.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Path names are always normalized. See also #8238.

@cyberduck
Copy link
Collaborator Author

bccfca5 commented

Hi David -
A bit more background - I've been attempting to rename all my files from /x/y/z.pdf to x/y/z.pdf
It looks like the leading / is causing rackspace a problem too - as all my object renames are failing.. although with a non-obvious error.
I've raised a ticket with them and if anything salient comes out of it I'll chuck a link on here.

Paul

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Replying to [comment:20 paulnicklin]:

Hi David -
A bit more background - I've been attempting to rename all my files from /x/y/z.pdf to x/y/z.pdf
It looks like the leading / is causing rackspace a problem too - as all my object renames are failing.. although with a non-obvious error.
I've raised a ticket with them and if anything salient comes out of it I'll chuck a link on here.

Paul

Thanks for the update.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Replying to [comment:19 dkocher]:

Path names are always normalized. See also #8238.

No longer true as of f029f4c.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Because we use / as the delimiter for listing keys this cannot be fixed.

@iterate-ch iterate-ch locked as resolved and limited conversation to collaborators Nov 26, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement low priority openstack OpenStack Swift Protocol Implementation wontfix
Projects
None yet
Development

No branches or pull requests

2 participants