Interoperability with Minio

[si458]

7.1.2 works without issues

7.2.0 onwards keeps getting an error on just simply listing a bucket contents

Request Error: null. Please contact your web hosting service provider for assistance.

server: minio 2019-10-12T01:39:57Z with --compat option

[si458]

THIS IS THE CONSOLE DEBUG OUTPUT IF I CONNECT TO THE MINIO SERVER IP AND PORT AND THEN ASK TO LIST CONTENTS TO MYBUCKET123

default	11:56:20.321525+0000	Cyberduck	Not supported to read versioning configuration for Path{path='/mybucket123', type=[directory, volume]} Request Error: null

default	11:56:20.323044+0000	Cyberduck	No retry for failure BackgroundException{file=Path{path='/mybucket123', type=[directory, volume]}, message='Listing directory mybucket123 failed.', detail='Request Error: null.', class='ch.cyberduck.core.exception.InteroperabilityException', cause='org.jets3t.service.ServiceException: Request Error: null'}

default	11:56:20.323921+0000	Cyberduck	Failure BackgroundException{file=Path{path='/mybucket123', type=[directory, volume]}, message='Listing directory mybucket123 failed.', detail='Request Error: null.', class='ch.cyberduck.core.exception.InteroperabilityException', cause='org.jets3t.service.ServiceException: Request Error: null'} running background task
java.lang.Exception
	at ch.cyberduck.core.threading.BackgroundCallable.<init>(BackgroundCallable.java:36)
	at ch.cyberduck.core.threading.DefaultBackgroundExecutor.execute(DefaultBackgroundExecutor.java:67)
	at ch.cyberduck.core.AbstractController.background(AbstractController.java:71)
	at ch.cyberduck.ui.cocoa.controller.BrowserController.reload(BrowserController.java:511)
	at ch.cyberduck.ui.cocoa.controller.BrowserController.setWorkdir(BrowserController.java:3041)
	at ch.cyberduck.ui.cocoa.controller.BrowserController.setWorkdir(BrowserController.java:3016)
	at ch.cyberduck.ui.cocoa.controller.BrowserController.insideButtonClicked(BrowserController.java:2682)
	at ch.cyberduck.ui.cocoa

default	11:56:21.481405+0000	Cyberduck	Ignore return code -1

default	11:56:21.512844+0000	Cyberduck	Missing result for worker SessionListWorker{directory=Path{path='/mybucket123', type=[directory, volume]}}. Use default value.

[si458]

THIS IS THE CONSOLE DEBUG OUTPUT IF I USE THE VIRTUAL HOST METHOD IN MINIO e.g ​http://mybucket123.s3.myserver.com:9000

i have checked the XML output being returned from the minio server using Charles Proxy and its a VALID XML that gets returned with Contents listed for files

default	12:40:18.754559+0000	Cyberduck	No retry for failure BackgroundException{file=Path{path='/', type=[directory, volume]}, message='Listing directory / failed.', detail='Failed to parse XML document with handler class org.jets3t.service.impl.rest.XmlResponsesSaxParser$ListAllMyBucketsHandler.', class='ch.cyberduck.core.exception.InteroperabilityException', cause='org.jets3t.service.S3ServiceException: Failed to parse XML document with handler class org.jets3t.service.impl.rest.XmlResponsesSaxParser$ListAllMyBucketsHandler'}

default	12:40:18.755397+0000	Cyberduck	Failure BackgroundException{file=Path{path='/', type=[directory, volume]}, message='Listing directory / failed.', detail='Failed to parse XML document with handler class org.jets3t.service.impl.rest.XmlResponsesSaxParser$ListAllMyBucketsHandler.', class='ch.cyberduck.core.exception.InteroperabilityException', cause='org.jets3t.service.S3ServiceException: Failed to parse XML document with handler class org.jets3t.service.impl.rest.XmlResponsesSaxParser$ListAllMyBucketsHandler'} running background task
java.lang.Exception
	at ch.cyberduck.core.threading.BackgroundCallable.<init>(BackgroundCallable.java:36)
	at ch.cyberduck.core.threading.DefaultBackgroundExecutor.execute(DefaultBackgroundExecutor.java:67)
	at ch.cyberduck.core.AbstractController.background(AbstractController.java:71)
	at ch.cyberduck.ui.cocoa.controller.BrowserController$29.run(BrowserController.java:3070)
	at ch.cyberduck.ui.cocoa.controller.BrowserController$32.run(BrowserController.java:3189)
	at ch.cyberduck.ui.cocoa.controller.Bro
default	12:40:20.375029+0000	Cyberduck	Ignore return code -1
default	12:40:20.392688+0000	Cyberduck	Missing result for worker MountWorker{cache=ch.cyberduck.core.ReverseLookupCache@3eee3e2b}. Use default value.

[dkocher]

Please make sure to enable ​virtual-host-style requests in their Minio configuration.

[si458]

Replying to dkocher:

Please make sure to enable ​virtual-host-style requests in their Minio configuration.

this is already enabled, as i couldnt access it first time round without setting the MINIO_DOMAIN in docker as it generated a different error

but after enabling it, im now getting this XML error as listed above

[si458]

ive just tried 7.1.2 with virtual-host-style and that doesn't work either

just keeps getting same error

Failed to parse XML document with handler class org.jets3t.service.impl.rest.XmlResponsesSaxParser$ListAllMyBucketsHandler. Please contact your web hosting service provider for assistance

but the log drawer show it returns HTTP/1.1 200 OK with a valid XML too

[dkocher]

Regression from r48152.

[dkocher]

Can you provide the captured XML response?

[si458]

Replying to dkocher:

Can you provide the captured XML response?

​https://pastebin.com/7knB6DcW

[si458]

ok i think i might have found the problem and maybe a bug but will try explain

1. if you use an IP address instead of a hostname, it looks for ​http://wtf123.192.168.1.100:9000/ which is invalid,

so maybe a checker if i use an IP address instead of a hostname then ALWAYS USE PATH-STYLE !!!

2. if i use a hostname with minio and have virtual-host-style DISABLED, cyberduck ALWAYS trys accessing the bucket using virtual-host-style regardless e.g: ​http://wtf123.s3.myserver.com:12345/

so i think it would be nice to have an option inside the connection settings to specify weather to use virtual-host-style or path-style (maybe per connection?)

3. if i use the full virtual-host-style as the hostname e.g: ​http://wtf123.s3.myserver.com:12345/

i get an XML errror, even tho the XML is valid and returned correctly? ​https://pastebin.com/7knB6DcW

[dkocher]

Thanks for summarizing the issues. Accessing via the hostname set with MINIO_DOMAIN this cannot be reproduced.

[dkocher]

The error when accessing using the bucket name in Host configuration setting of the bookmark is that we expect the server to return a bucket list and not an object list.

[dkocher]

We have requested a documentation update for Minio in ​https://github.com/minio/cookbook/issues/280.

[si458]

surely it would be a better option to give the user control over path-style or virtual-host-style than forcing the user to use virtual-host-style?

[dkocher]

Replying to si458:

surely it would be a better option to give the user control over path-style or virtual-host-style than forcing the user to use virtual-host-style?

We will see if we can support this via an option in the connection profiles.

[si458]

Replying to dkocher:

Replying to si458:

surely it would be a better option to give the user control over path-style or virtual-host-style than forcing the user to use virtual-host-style?

We will see if we can support this via an option in the connection profiles.

thank you that would be helpful!

or even a tick box saying saying 'USE PATH-STYLE' in the edit bookmarks window/open connection window

[dkocher]

Replying to si458:

ok i think i might have found the problem and maybe a bug but will try explain

1. if you use an IP address instead of a hostname, it looks for ​http://wtf123.192.168.1.100:9000/ which is invalid,

so maybe a checker if i use an IP address instead of a hostname then ALWAYS USE PATH-STYLE !!!

In r48257.

[dkocher]

Pending change for Failed to parse XML document error to detect if we are connecting to a specific bucket and should list objects instead of all buckets.

[dkocher]

Replying to si458:

3. if i use the full virtual-host-style as the hostname e.g: ​http://wtf123.s3.myserver.com:12345/

i get an XML errror, even tho the XML is valid and returned correctly? ​https://pastebin.com/7knB6DcW

Set the bucket name in the Path setting in the bookmark instead.

[yla]

In r48333.

[dimm00]

Trying to make cyberduck work with ceph S3, which only supports path style Tried using IP, which broke the cert and didn't work either Is there a way to reenable path style S3?

[dkocher]

Replying to dimm00:

Trying to make cyberduck work with ceph S3, which only supports path style Tried using IP, which broke the cert and didn't work either Is there a way to reenable path style S3?

Is this Ceph instance publicly reachable over the Internet?

[dimm00]

Replying to dkocher:

Replying to dimm00:

Trying to make cyberduck work with ceph S3, which only supports path style Tried using IP, which broke the cert and didn't work either Is there a way to reenable path style S3?

Is this Ceph instance publicly reachable over the Internet?

Yes. How can I send you credentials to access it?

[dkocher]

Replying to dimm00:

Replying to dkocher:

Replying to dimm00:

Trying to make cyberduck work with ceph S3, which only supports path style Tried using IP, which broke the cert and didn't work either Is there a way to reenable path style S3?

Is this Ceph instance publicly reachable over the Internet?

Yes. How can I send you credentials to access it?

Awesome. Please write to support@….

[dkocher]

Duplicate for #10956.

[zagy__]

Replying to dimm00:

Trying to make cyberduck work with ceph S3, which only supports path style Tried using IP, which broke the cert and didn't work either Is there a way to reenable path style S3?

Was that solved? I'm seeing the same problem with Ceph here.

[si458]

i dont have this issue anymore as its been fixed in latest cyberduck version using the S3 (HTTPS) profile from the minio help page

the only issue i found was when using my home broadband 'virgin media' they take over my dns records,

so when cyberduck was looking for 'mybucket.s3.myserver.com' is should of returned blank A record as i only have an A record for 's3.myserver.com' which ment cyberduck would then reverted back to using the path format,

but 'virgin media' kept returning an IP address of theres for there stupid advanced search error page, so cyberduck kept trying to connect to them and just timing out instead of an error

once i disabled the 'advanced search error page' and rebooted our router and cleared my dns cache on laptop,

cyberduck started working without any issues

[zagy__]

Aha. What worked for me now, was to create a bookmark and add the bucket-url to the PATH there. So, worksforme now. Thanks.

[dkocher]

We have another fix for Minio (#11018) that will be included in version 7.3.1.