Cyberduck Mountain Duck CLI

Opened 4 years ago

Last modified 4 years ago

#8064 reopened enhancement

Interoperability with leading / in key name

Reported by: paulnicklin Owned by: dkocher
Priority: low Milestone:
Component: openstack Version: 4.4.5
Severity: normal Keywords:
Cc: Architecture: Intel
Platform: Windows 7

Description (last modified by paulnicklin)

I have a container with a file called "/x/y.pdf" Cyberduck finds the container and lists the contents, but it shows as y.pdf When I try to download, sync, view, get status of the file it always fails with a 404 because https://identity.api.rackspacecloud.com/<container>/y.pdf

This turns out to be inconsistent handling of the leading /. Rackspace don't treat /x/ as a placeholder folder, but cyberduck does.

Have tagged it as openstack as a guess.

Log drawer not interesting as it doesn't show the error, just successfully getting the files list.

GET /v1/MossoClouxxxxxxxxxxxxxxxxxxxxxxx/xxxxx4742?format=xml&prefix=&limit=10000&delimiter=%2F HTTP/1.1
X-Auth-Token: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Host: storage101.lon3.clouddrive.com
Connection: Keep-Alive
User-Agent: Cyberduck/4.4.5 (14721) (Windows 7/6.1) (x86)
HTTP/1.1 200 OK
Content-Length: 5069
X-Container-Object-Count: 22
Accept-Ranges: bytes
X-Timestamp: 1401738255.23634
X-Container-Bytes-Used: 21453741
Content-Type: application/xml; charset=utf-8
X-Trans-Id: txcf8bxxxxxxxxxxxxxxb7-0053ac31celon3
Date: Thu, 26 Jun 2014 14:44:30 GMT

Change History (21)

comment:1 Changed 4 years ago by dkocher

  • Description modified (diff)

comment:2 Changed 4 years ago by dkocher

  • Summary changed from Cyberduck does not handle Rackspace UK files with / in the name. to 404 error response when downloading file in placeholder folder

comment:3 Changed 4 years ago by dkocher

  • Milestone set to 4.5

comment:4 Changed 4 years ago by dkocher

  • Resolution set to fixed
  • Status changed from new to closed

In r14866 and r14881.

Last edited 4 years ago by dkocher (previous) (diff)

comment:5 Changed 4 years ago by paulnicklin

  • Resolution fixed deleted
  • Status changed from closed to reopened

Hi

Many thanks for the speedy investigation, but it still doesn't work for me.

Not sure what the differences are between your successful test and my failure - so here are some hopefully salient bits..

I'm on the beta build 14906 - hopefully that includes revision 14866 (snapshot is the same build number)

Not sure if the placeholder folder is supposed to be shown in the main tree (it doesn't, it's just a flat view of all files), but on the transfers dialog the URL us shown as clientfiles-4742/x.pdf, ie no placeholder.

I'm using / as the separator in my cloud files, and the files often begin with a leading / but am on Windows (you use path.DELIMETER i think - my java is rusty (>12 years ago) so I don't remember if this is the defined platform value or fixed)

My connection uses the Rackspace (UK) profile - but I get the same with Swift (Openstack) and Rackspace (US). I created a new bookmark in case there was a cached value.

Turned on debug logging. the first ref to my file is this - there is no reference to the placeholder/path.. (may be irrelevant)

2014-07-22 12:43:25,275 [Thread-0] DEBUG RegistryApplicationFinder - GetRegisteredDefaultApplication for filename Address proof xxxxx.pdf

and when I start the transfer.. still no path.

2014-07-22 12:43:25,987 [background-1] DEBUG ch.cyberduck.core.transfer.Queue - Add transfer Transfer{transferred=null, size=null, roots=[TransferItem{local=Local{path='C:\Users\paul\Downloads\Address proof xxxxx.pdf'}, remote=Path{path='/container-nnnnnn/Address proof xxxx.pdf', type=[file]}}], state=stopped, host=Host{credentials=Credentials{user='xxxxxxx'}, hostname='lon.identity.api.rackspacecloud.com', port=443, protocol=Profile{parent=swift, image=Local{path='C:\Users\paul\AppData\Local\Temp\\xxxxxxxxxxxxxico'}}}}

..and not surprisingly I get 404.

Any further assistance let me know.

Paul

Last edited 4 years ago by dkocher (previous) (diff)

comment:6 Changed 4 years ago by dkocher

  • Milestone changed from 4.5 to 4.5.2

comment:7 Changed 4 years ago by dkocher

  • Resolution set to fixed
  • Status changed from reopened to closed

Please update to the latest snapshot build available. Should be fixed with r14917.

comment:8 Changed 4 years ago by paulnicklin

Really sorry, still not working - same symptoms as before as far as I can see - on snapshot 4.5.2 (14943)

comment:9 Changed 4 years ago by paulnicklin

  • Resolution fixed deleted
  • Status changed from closed to reopened

comment:10 Changed 4 years ago by dkocher

Reading the ticket description again I now understand that this issue describes that objects in placeholders are actually displayed in the root of the container instead in the browser and the placeholder path is omitted.

comment:11 Changed 4 years ago by dkocher

Please post again a current transcript from the log drawer (⌘-L). You should see requests with the format

GET /v1/MossoCloudFS_59113590-c679-46c3-bf62-9d7c3d5176ee/test.cyberduck.ch?format=xml&prefix=<x>%2F&limit=10000&delimiter=%2F HTTP/1.1

comment:12 Changed 4 years ago by dkocher

  • Milestone 4.5.2 deleted

comment:13 Changed 4 years ago by dkocher

  • Milestone set to 4.5.2
  • Resolution set to worksforme
  • Status changed from reopened to closed

comment:14 Changed 4 years ago by paulnicklin

  • Resolution worksforme deleted
  • Status changed from closed to reopened

Arghh! This is frustrating! I'm on 15069. Still NOT working for me.

Certainly the files are displayed flat - with no placeholder dir - I'd say that was a strong(ish) hint that the placeholder dir has been lost processing the file list from rackspace

One oddity is that I can't connect using openswift profile- I use Rackspace US profile (even though the DC is in london). however the log implies the swift helper is being used

It looks to me like the placeholder is not extracted...

2014-09-09 22:44:34,043 [Thread-0] DEBUG ch.cyberduck.ui.AbstractController - Run action WorkerBackgroundAction{worker=SessionListWorker{directory=Path{path='/clientfiles-xxxx', type=[directory, volume]}}, result=null} in background
2014-09-09 22:44:34,044 [Thread-0] DEBUG ch.cyberduck.ui.AbstractController - Synchronize on lock Session{host=Host{credentials=Credentials{user='username'}, hostname='identity.api.rackspacecloud.com', port=443, protocol=Profile{parent=swift, image=Local{path='C:\Users\paul\AppData\Local\Temp\\4bfc20b0-9fb1-4bb6-afd0-56421f079db2.ico'}}}, state=closed} for action WorkerBackgroundAction{worker=SessionListWorker{directory=Path{path='/clientfiles-xxxx', type=[directory, volume]}}, result=null}
2014-09-09 22:44:46,308 [background-1] DEBUG ch.cyberduck.core.Profile - No value for key:Scheme ??

I see this..

2014-09-09 22:44:50,667 [background-1] DEBUG ch.cyberduck.core.threading.AbstractBackgroundAction - Finish background task WorkerBackgroundAction{worker=SessionListWorker{directory=Path{path='/clientfiles-xxxx', type=[directory, volume]}}, result=[Path{path='/clientfiles-xxxx/Address proof xxxx.pdf', type=[file]}, ... 
}]}

So by this point already the placeholders have been lost/removed

My code uses the .net library openstack, and calling listfiles on the container I get the first file as /Admin/Address proof xxxx.pdf (is the leading / a problem??)

and then on the transfer..

2014-09-09 22:34:02,944 [background-1] DEBUG ch.cyberduck.ui.threading.TransferCollectionBackgroundAction - Finish background action for transfer Transfer{transferred=null, size=null, roots=[TransferItem{local=Local{path='C:\Users\paul\Downloads\Address proof xxxxx.pdf'}, remote=Path{path='/clientfiles-xxxx/Address proof xxxx.pdf', type=[file]}}], state=stopped, host=Host{credentials=Credentials{user='username'}, hostname='identity.api.rackspacecloud.com', port=443, protocol=Profile{parent=swift, image=Local{path='C:\Users\paul\AppData\Local\Temp\\b660f76b-bb93-4b9f-a0f5-fe795a51ab8b.ico'}}}}

What's the difference with my setup and yours?

Last edited 4 years ago by dkocher (previous) (diff)

comment:15 Changed 4 years ago by dkocher

  • Resolution set to worksforme
  • Status changed from reopened to closed

Can you please contact Rackspace support and confirm your layout in the container adheres to the recommendation in Pseudo hierarchical folders and directories.

comment:16 Changed 4 years ago by paulnicklin

It is in fact the leading / I have on my files that is breaking it. Removed the leading / and their web interface now presents a hierarchy (never seen it before so I didn't realise it would do that) and it's replicated in Cyberduck.

Well we got there in the end.

Thanks for your help

comment:17 Changed 4 years ago by paulnicklin

  • Description modified (diff)
  • Resolution worksforme deleted
  • Status changed from closed to reopened
  • Summary changed from 404 error response when downloading file in placeholder folder to 404 error response when downloading file that begins with a /

..so there is still a bug, and that is that files beginning with a / can't be downloaded - there's a discrepancy in the handling - you think it's a placeholder folder, but Rackspace don't treat it as such.

comment:18 Changed 4 years ago by dkocher

  • Milestone 4.5.2 deleted
  • Priority changed from normal to low
  • Summary changed from 404 error response when downloading file that begins with a / to Interoperability with leading / in key name
  • Type changed from defect to enhancement

comment:19 Changed 4 years ago by dkocher

Path names are always normalized. See also #8238.

comment:20 follow-up: Changed 4 years ago by paulnicklin

Hi David - A bit more background - I've been attempting to rename all my files from /x/y/z.pdf to x/y/z.pdf It looks like the leading / is causing rackspace a problem too - as all my object renames are failing.. although with a non-obvious error. I've raised a ticket with them and if anything salient comes out of it I'll chuck a link on here.

Paul

comment:21 in reply to: ↑ 20 Changed 4 years ago by dkocher

Replying to paulnicklin:

Hi David - A bit more background - I've been attempting to rename all my files from /x/y/z.pdf to x/y/z.pdf It looks like the leading / is causing rackspace a problem too - as all my object renames are failing.. although with a non-obvious error. I've raised a ticket with them and if anything salient comes out of it I'll chuck a link on here.

Paul

Thanks for the update.

Note: See TracTickets for help on using tickets.
swiss made software