Interoperability with Minio

[si458]

7.1.2 works without issues

7.2.0 onwards keeps getting an error on just simply listing a bucket contents

Request Error: null. Please contact your web hosting service provider for assistance.

server: minio 2019-10-12T01:39:57Z with --compat option

[si458]

THIS IS THE CONSOLE DEBUG OUTPUT IF I CONNECT TO THE MINIO SERVER IP AND PORT AND THEN ASK TO LIST CONTENTS TO MYBUCKET123

default	11:56:20.321525+0000	Cyberduck	Not supported to read versioning configuration for Path{path='/mybucket123', type=[directory, volume]} Request Error: null

default	11:56:20.323044+0000	Cyberduck	No retry for failure BackgroundException{file=Path{path='/mybucket123', type=[directory, volume]}, message='Listing directory mybucket123 failed.', detail='Request Error: null.', class='ch.cyberduck.core.exception.InteroperabilityException', cause='org.jets3t.service.ServiceException: Request Error: null'}

default	11:56:20.323921+0000	Cyberduck	Failure BackgroundException{file=Path{path='/mybucket123', type=[directory, volume]}, message='Listing directory mybucket123 failed.', detail='Request Error: null.', class='ch.cyberduck.core.exception.InteroperabilityException', cause='org.jets3t.service.ServiceException: Request Error: null'} running background task
java.lang.Exception
	at ch.cyberduck.core.threading.BackgroundCallable.<init>(BackgroundCallable.java:36)
	at ch.cyberduck.core.threading.DefaultBackgroundExecutor.execute(DefaultBackgroundExecutor.java:67)
	at ch.cyberduck.core.AbstractController.background(AbstractController.java:71)
	at ch.cyberduck.ui.cocoa.controller.BrowserController.reload(BrowserController.java:511)
	at ch.cyberduck.ui.cocoa.controller.BrowserController.setWorkdir(BrowserController.java:3041)
	at ch.cyberduck.ui.cocoa.controller.BrowserController.setWorkdir(BrowserController.java:3016)
	at ch.cyberduck.ui.cocoa.controller.BrowserController.insideButtonClicked(BrowserController.java:2682)
	at ch.cyberduck.ui.cocoa

default	11:56:21.481405+0000	Cyberduck	Ignore return code -1

default	11:56:21.512844+0000	Cyberduck	Missing result for worker SessionListWorker{directory=Path{path='/mybucket123', type=[directory, volume]}}. Use default value.

[si458]

THIS IS THE CONSOLE DEBUG OUTPUT IF I USE THE VIRTUAL HOST METHOD IN MINIO e.g ​http://mybucket123.s3.myserver.com:9000

i have checked the XML output being returned from the minio server using Charles Proxy and its a VALID XML that gets returned with Contents listed for files

default	12:40:18.754559+0000	Cyberduck	No retry for failure BackgroundException{file=Path{path='/', type=[directory, volume]}, message='Listing directory / failed.', detail='Failed to parse XML document with handler class org.jets3t.service.impl.rest.XmlResponsesSaxParser$ListAllMyBucketsHandler.', class='ch.cyberduck.core.exception.InteroperabilityException', cause='org.jets3t.service.S3ServiceException: Failed to parse XML document with handler class org.jets3t.service.impl.rest.XmlResponsesSaxParser$ListAllMyBucketsHandler'}

default	12:40:18.755397+0000	Cyberduck	Failure BackgroundException{file=Path{path='/', type=[directory, volume]}, message='Listing directory / failed.', detail='Failed to parse XML document with handler class org.jets3t.service.impl.rest.XmlResponsesSaxParser$ListAllMyBucketsHandler.', class='ch.cyberduck.core.exception.InteroperabilityException', cause='org.jets3t.service.S3ServiceException: Failed to parse XML document with handler class org.jets3t.service.impl.rest.XmlResponsesSaxParser$ListAllMyBucketsHandler'} running background task
java.lang.Exception
	at ch.cyberduck.core.threading.BackgroundCallable.<init>(BackgroundCallable.java:36)
	at ch.cyberduck.core.threading.DefaultBackgroundExecutor.execute(DefaultBackgroundExecutor.java:67)
	at ch.cyberduck.core.AbstractController.background(AbstractController.java:71)
	at ch.cyberduck.ui.cocoa.controller.BrowserController$29.run(BrowserController.java:3070)
	at ch.cyberduck.ui.cocoa.controller.BrowserController$32.run(BrowserController.java:3189)
	at ch.cyberduck.ui.cocoa.controller.Bro
default	12:40:20.375029+0000	Cyberduck	Ignore return code -1
default	12:40:20.392688+0000	Cyberduck	Missing result for worker MountWorker{cache=ch.cyberduck.core.ReverseLookupCache@3eee3e2b}. Use default value.

[dkocher]

Please make sure to enable ​virtual-host-style requests in their Minio configuration.

[si458]

Replying to dkocher:

Please make sure to enable ​virtual-host-style requests in their Minio configuration.

this is already enabled, as i couldnt access it first time round without setting the MINIO_DOMAIN in docker as it generated a different error

but after enabling it, im now getting this XML error as listed above

[si458]

ive just tried 7.1.2 with virtual-host-style and that doesn't work either

just keeps getting same error

Failed to parse XML document with handler class org.jets3t.service.impl.rest.XmlResponsesSaxParser$ListAllMyBucketsHandler. Please contact your web hosting service provider for assistance

but the log drawer show it returns HTTP/1.1 200 OK with a valid XML too

[dkocher]

Regression from r48152.

[dkocher]

Can you provide the captured XML response?

[si458]

Replying to dkocher:

Can you provide the captured XML response?

​https://pastebin.com/7knB6DcW

[si458]

ok i think i might have found the problem and maybe a bug but will try explain

1. if you use an IP address instead of a hostname, it looks for ​http://wtf123.192.168.1.100:9000/ which is invalid,

so maybe a checker if i use an IP address instead of a hostname then ALWAYS USE PATH-STYLE !!!

2. if i use a hostname with minio and have virtual-host-style DISABLED, cyberduck ALWAYS trys accessing the bucket using virtual-host-style regardless e.g: ​http://wtf123.s3.myserver.com:12345/

so i think it would be nice to have an option inside the connection settings to specify weather to use virtual-host-style or path-style (maybe per connection?)

3. if i use the full virtual-host-style as the hostname e.g: ​http://wtf123.s3.myserver.com:12345/

i get an XML errror, even tho the XML is valid and returned correctly? ​https://pastebin.com/7knB6DcW

[dkocher]

Thanks for summarizing the issues. Accessing via the hostname set with MINIO_DOMAIN this cannot be reproduced.

[dkocher]

The error when accessing using the bucket name in Host configuration setting of the bookmark is that we expect the server to return a bucket list and not an object list.

[dkocher]

We have requested a documentation update for Minio in ​https://github.com/minio/cookbook/issues/280.

[si458]

surely it would be a better option to give the user control over path-style or virtual-host-style than forcing the user to use virtual-host-style?

[dkocher]

Replying to si458:

surely it would be a better option to give the user control over path-style or virtual-host-style than forcing the user to use virtual-host-style?

We will see if we can support this via an option in the connection profiles.

[si458]

Replying to dkocher:

Replying to si458:

surely it would be a better option to give the user control over path-style or virtual-host-style than forcing the user to use virtual-host-style?

We will see if we can support this via an option in the connection profiles.

thank you that would be helpful!

or even a tick box saying saying 'USE PATH-STYLE' in the edit bookmarks window/open connection window

[dkocher]

Replying to si458:

ok i think i might have found the problem and maybe a bug but will try explain

1. if you use an IP address instead of a hostname, it looks for ​http://wtf123.192.168.1.100:9000/ which is invalid,

so maybe a checker if i use an IP address instead of a hostname then ALWAYS USE PATH-STYLE !!!

In r48257.

[dkocher]

Pending change for Failed to parse XML document error to detect if we are connecting to a specific bucket and should list objects instead of all buckets.

[dkocher]

Replying to si458:

3. if i use the full virtual-host-style as the hostname e.g: ​http://wtf123.s3.myserver.com:12345/

i get an XML errror, even tho the XML is valid and returned correctly? ​https://pastebin.com/7knB6DcW

Set the bucket name in the Path setting in the bookmark instead.

[yla]

In r48333.

[dimm00]

Trying to make cyberduck work with ceph S3, which only supports path style Tried using IP, which broke the cert and didn't work either Is there a way to reenable path style S3?

[dkocher]

Replying to dimm00:

Trying to make cyberduck work with ceph S3, which only supports path style Tried using IP, which broke the cert and didn't work either Is there a way to reenable path style S3?

Is this Ceph instance publicly reachable over the Internet?

[dimm00]

Replying to dkocher:

Replying to dimm00:

Trying to make cyberduck work with ceph S3, which only supports path style Tried using IP, which broke the cert and didn't work either Is there a way to reenable path style S3?

Is this Ceph instance publicly reachable over the Internet?

Yes. How can I send you credentials to access it?

[dkocher]

Replying to dimm00:

Replying to dkocher:

Replying to dimm00:

Trying to make cyberduck work with ceph S3, which only supports path style Tried using IP, which broke the cert and didn't work either Is there a way to reenable path style S3?

Is this Ceph instance publicly reachable over the Internet?

Yes. How can I send you credentials to access it?

Awesome. Please write to support@….

[dkocher]

Duplicate for #10956.