Cyberduck Mountain Duck CLI

#8064 reopened enhancement

Interoperability with leading / in key name

Reported by: paulnicklin Owned by: dkocher
Priority: low Milestone:
Component: openstack Version: 4.4.5
Severity: normal Keywords:
Cc: Architecture: Intel
Platform: Windows 7

Description (last modified by paulnicklin)

I have a container with a file called "/x/y.pdf" Cyberduck finds the container and lists the contents, but it shows as y.pdf When I try to download, sync, view, get status of the file it always fails with a 404 because https://identity.api.rackspacecloud.com/<container>/y.pdf

This turns out to be inconsistent handling of the leading /. Rackspace don't treat /x/ as a placeholder folder, but cyberduck does.

Have tagged it as openstack as a guess.

Log drawer not interesting as it doesn't show the error, just successfully getting the files list.

GET /v1/MossoClouxxxxxxxxxxxxxxxxxxxxxxx/xxxxx4742?format=xml&prefix=&limit=10000&delimiter=%2F HTTP/1.1
X-Auth-Token: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Host: storage101.lon3.clouddrive.com
Connection: Keep-Alive
User-Agent: Cyberduck/4.4.5 (14721) (Windows 7/6.1) (x86)
HTTP/1.1 200 OK
Content-Length: 5069
X-Container-Object-Count: 22
Accept-Ranges: bytes
X-Timestamp: 1401738255.23634
X-Container-Bytes-Used: 21453741
Content-Type: application/xml; charset=utf-8
X-Trans-Id: txcf8bxxxxxxxxxxxxxxb7-0053ac31celon3
Date: Thu, 26 Jun 2014 14:44:30 GMT

Change History (21)

comment:1 Changed on Jun 26, 2014 at 3:11:53 PM by dkocher

  • Description modified (diff)

comment:2 Changed on Jun 26, 2014 at 3:19:52 PM by dkocher

  • Summary changed from Cyberduck does not handle Rackspace UK files with / in the name. to 404 error response when downloading file in placeholder folder

comment:3 Changed on Jun 28, 2014 at 6:29:38 AM by dkocher

  • Milestone set to 4.5

comment:4 Changed on Jul 2, 2014 at 9:15:29 AM by dkocher

  • Resolution set to fixed
  • Status changed from new to closed

In r14866 and r14881.

Last edited on Jul 4, 2014 at 1:45:03 PM by dkocher (previous) (diff)

comment:5 Changed on Jul 22, 2014 at 11:50:51 AM by paulnicklin

  • Resolution fixed deleted
  • Status changed from closed to reopened

Hi

Many thanks for the speedy investigation, but it still doesn't work for me.

Not sure what the differences are between your successful test and my failure - so here are some hopefully salient bits..

I'm on the beta build 14906 - hopefully that includes revision 14866 (snapshot is the same build number)

Not sure if the placeholder folder is supposed to be shown in the main tree (it doesn't, it's just a flat view of all files), but on the transfers dialog the URL us shown as clientfiles-4742/x.pdf, ie no placeholder.

I'm using / as the separator in my cloud files, and the files often begin with a leading / but am on Windows (you use path.DELIMETER i think - my java is rusty (>12 years ago) so I don't remember if this is the defined platform value or fixed)

My connection uses the Rackspace (UK) profile - but I get the same with Swift (Openstack) and Rackspace (US). I created a new bookmark in case there was a cached value.

Turned on debug logging. the first ref to my file is this - there is no reference to the placeholder/path.. (may be irrelevant)

2014-07-22 12:43:25,275 [Thread-0] DEBUG RegistryApplicationFinder - GetRegisteredDefaultApplication for filename Address proof xxxxx.pdf

and when I start the transfer.. still no path.

2014-07-22 12:43:25,987 [background-1] DEBUG ch.cyberduck.core.transfer.Queue - Add transfer Transfer{transferred=null, size=null, roots=[TransferItem{local=Local{path='C:\Users\paul\Downloads\Address proof xxxxx.pdf'}, remote=Path{path='/container-nnnnnn/Address proof xxxx.pdf', type=[file]}}], state=stopped, host=Host{credentials=Credentials{user='xxxxxxx'}, hostname='lon.identity.api.rackspacecloud.com', port=443, protocol=Profile{parent=swift, image=Local{path='C:\Users\paul\AppData\Local\Temp\\xxxxxxxxxxxxxico'}}}}

..and not surprisingly I get 404.

Any further assistance let me know.

Paul

Last edited on Jul 22, 2014 at 12:55:48 PM by dkocher (previous) (diff)

comment:6 Changed on Jul 25, 2014 at 12:40:34 PM by dkocher

  • Milestone changed from 4.5 to 4.5.2

comment:7 Changed on Jul 29, 2014 at 2:22:46 PM by dkocher

  • Resolution set to fixed
  • Status changed from reopened to closed

Please update to the latest snapshot build available. Should be fixed with r14917.

comment:8 Changed on Aug 4, 2014 at 4:06:29 PM by paulnicklin

Really sorry, still not working - same symptoms as before as far as I can see - on snapshot 4.5.2 (14943)

comment:9 Changed on Aug 4, 2014 at 4:06:50 PM by paulnicklin

  • Resolution fixed deleted
  • Status changed from closed to reopened

comment:10 Changed on Aug 11, 2014 at 8:23:10 PM by dkocher

Reading the ticket description again I now understand that this issue describes that objects in placeholders are actually displayed in the root of the container instead in the browser and the placeholder path is omitted.

comment:11 Changed on Aug 11, 2014 at 8:25:22 PM by dkocher

Please post again a current transcript from the log drawer (⌘-L). You should see requests with the format

GET /v1/MossoCloudFS_59113590-c679-46c3-bf62-9d7c3d5176ee/test.cyberduck.ch?format=xml&prefix=<x>%2F&limit=10000&delimiter=%2F HTTP/1.1

comment:12 Changed on Aug 12, 2014 at 10:14:53 AM by dkocher

  • Milestone 4.5.2 deleted

comment:13 Changed on Aug 14, 2014 at 8:08:55 PM by dkocher

  • Milestone set to 4.5.2
  • Resolution set to worksforme
  • Status changed from reopened to closed

comment:14 Changed on Sep 9, 2014 at 10:08:05 PM by paulnicklin

  • Resolution worksforme deleted
  • Status changed from closed to reopened

Arghh! This is frustrating! I'm on 15069. Still NOT working for me.

Certainly the files are displayed flat - with no placeholder dir - I'd say that was a strong(ish) hint that the placeholder dir has been lost processing the file list from rackspace

One oddity is that I can't connect using openswift profile- I use Rackspace US profile (even though the DC is in london). however the log implies the swift helper is being used

It looks to me like the placeholder is not extracted...

2014-09-09 22:44:34,043 [Thread-0] DEBUG ch.cyberduck.ui.AbstractController - Run action WorkerBackgroundAction{worker=SessionListWorker{directory=Path{path='/clientfiles-xxxx', type=[directory, volume]}}, result=null} in background
2014-09-09 22:44:34,044 [Thread-0] DEBUG ch.cyberduck.ui.AbstractController - Synchronize on lock Session{host=Host{credentials=Credentials{user='username'}, hostname='identity.api.rackspacecloud.com', port=443, protocol=Profile{parent=swift, image=Local{path='C:\Users\paul\AppData\Local\Temp\\4bfc20b0-9fb1-4bb6-afd0-56421f079db2.ico'}}}, state=closed} for action WorkerBackgroundAction{worker=SessionListWorker{directory=Path{path='/clientfiles-xxxx', type=[directory, volume]}}, result=null}
2014-09-09 22:44:46,308 [background-1] DEBUG ch.cyberduck.core.Profile - No value for key:Scheme ??

I see this..

2014-09-09 22:44:50,667 [background-1] DEBUG ch.cyberduck.core.threading.AbstractBackgroundAction - Finish background task WorkerBackgroundAction{worker=SessionListWorker{directory=Path{path='/clientfiles-xxxx', type=[directory, volume]}}, result=[Path{path='/clientfiles-xxxx/Address proof xxxx.pdf', type=[file]}, ... 
}]}

So by this point already the placeholders have been lost/removed

My code uses the .net library openstack, and calling listfiles on the container I get the first file as /Admin/Address proof xxxx.pdf (is the leading / a problem??)

and then on the transfer..

2014-09-09 22:34:02,944 [background-1] DEBUG ch.cyberduck.ui.threading.TransferCollectionBackgroundAction - Finish background action for transfer Transfer{transferred=null, size=null, roots=[TransferItem{local=Local{path='C:\Users\paul\Downloads\Address proof xxxxx.pdf'}, remote=Path{path='/clientfiles-xxxx/Address proof xxxx.pdf', type=[file]}}], state=stopped, host=Host{credentials=Credentials{user='username'}, hostname='identity.api.rackspacecloud.com', port=443, protocol=Profile{parent=swift, image=Local{path='C:\Users\paul\AppData\Local\Temp\\b660f76b-bb93-4b9f-a0f5-fe795a51ab8b.ico'}}}}

What's the difference with my setup and yours?

Last edited on Sep 17, 2014 at 4:33:07 PM by dkocher (previous) (diff)

comment:15 Changed on Oct 8, 2014 at 7:53:36 PM by dkocher

  • Resolution set to worksforme
  • Status changed from reopened to closed

Can you please contact Rackspace support and confirm your layout in the container adheres to the recommendation in Pseudo hierarchical folders and directories.

comment:16 Changed on Oct 9, 2014 at 10:37:34 AM by paulnicklin

It is in fact the leading / I have on my files that is breaking it. Removed the leading / and their web interface now presents a hierarchy (never seen it before so I didn't realise it would do that) and it's replicated in Cyberduck.

Well we got there in the end.

Thanks for your help

comment:17 Changed on Oct 9, 2014 at 11:42:18 AM by paulnicklin

  • Description modified (diff)
  • Resolution worksforme deleted
  • Status changed from closed to reopened
  • Summary changed from 404 error response when downloading file in placeholder folder to 404 error response when downloading file that begins with a /

..so there is still a bug, and that is that files beginning with a / can't be downloaded - there's a discrepancy in the handling - you think it's a placeholder folder, but Rackspace don't treat it as such.

comment:18 Changed on Oct 9, 2014 at 12:37:45 PM by dkocher

  • Milestone 4.5.2 deleted
  • Priority changed from normal to low
  • Summary changed from 404 error response when downloading file that begins with a / to Interoperability with leading / in key name
  • Type changed from defect to enhancement

comment:19 Changed on Oct 9, 2014 at 12:39:24 PM by dkocher

Path names are always normalized. See also #8238.

comment:20 follow-up: Changed on Oct 16, 2014 at 8:03:58 AM by paulnicklin

Hi David - A bit more background - I've been attempting to rename all my files from /x/y/z.pdf to x/y/z.pdf It looks like the leading / is causing rackspace a problem too - as all my object renames are failing.. although with a non-obvious error. I've raised a ticket with them and if anything salient comes out of it I'll chuck a link on here.

Paul

comment:21 in reply to: ↑ 20 Changed on Oct 16, 2014 at 8:25:35 AM by dkocher

Replying to paulnicklin:

Hi David - A bit more background - I've been attempting to rename all my files from /x/y/z.pdf to x/y/z.pdf It looks like the leading / is causing rackspace a problem too - as all my object renames are failing.. although with a non-obvious error. I've raised a ticket with them and if anything salient comes out of it I'll chuck a link on here.

Paul

Thanks for the update.

Note: See TracTickets for help on using tickets.
swiss made software