Opened on Nov 27, 2019 at 11:42:46 AM
Closed on Dec 9, 2019 at 7:45:19 AM
Last modified on Jan 18, 2021 at 10:23:10 AM
#10888 closed defect (fixed)
Interoperability with Minio
Reported by: | si458 | Owned by: | dkocher |
---|---|---|---|
Priority: | normal | Milestone: | 7.2.2 |
Component: | s3 | Version: | 7.1.2 |
Severity: | normal | Keywords: | |
Cc: | cz@… | Architecture: | Intel |
Platform: | macOS 10.15 |
Description
7.1.2 works without issues
7.2.0 onwards keeps getting an error on just simply listing a bucket contents
Request Error: null. Please contact your web hosting service provider for assistance.
server: minio 2019-10-12T01:39:57Z with --compat option
Change History (34)
comment:1 Changed on Nov 27, 2019 at 11:57:16 AM by si458
- Priority changed from normal to high
comment:2 Changed on Nov 27, 2019 at 1:01:42 PM by si458
THIS IS THE CONSOLE DEBUG OUTPUT IF I USE THE VIRTUAL HOST METHOD IN MINIO e.g http://mybucket123.s3.myserver.com:9000
i have checked the XML output being returned from the minio server using Charles Proxy and its a VALID XML that gets returned with Contents listed for files
default 12:40:18.754559+0000 Cyberduck No retry for failure BackgroundException{file=Path{path='/', type=[directory, volume]}, message='Listing directory / failed.', detail='Failed to parse XML document with handler class org.jets3t.service.impl.rest.XmlResponsesSaxParser$ListAllMyBucketsHandler.', class='ch.cyberduck.core.exception.InteroperabilityException', cause='org.jets3t.service.S3ServiceException: Failed to parse XML document with handler class org.jets3t.service.impl.rest.XmlResponsesSaxParser$ListAllMyBucketsHandler'} default 12:40:18.755397+0000 Cyberduck Failure BackgroundException{file=Path{path='/', type=[directory, volume]}, message='Listing directory / failed.', detail='Failed to parse XML document with handler class org.jets3t.service.impl.rest.XmlResponsesSaxParser$ListAllMyBucketsHandler.', class='ch.cyberduck.core.exception.InteroperabilityException', cause='org.jets3t.service.S3ServiceException: Failed to parse XML document with handler class org.jets3t.service.impl.rest.XmlResponsesSaxParser$ListAllMyBucketsHandler'} running background task java.lang.Exception at ch.cyberduck.core.threading.BackgroundCallable.<init>(BackgroundCallable.java:36) at ch.cyberduck.core.threading.DefaultBackgroundExecutor.execute(DefaultBackgroundExecutor.java:67) at ch.cyberduck.core.AbstractController.background(AbstractController.java:71) at ch.cyberduck.ui.cocoa.controller.BrowserController$29.run(BrowserController.java:3070) at ch.cyberduck.ui.cocoa.controller.BrowserController$32.run(BrowserController.java:3189) at ch.cyberduck.ui.cocoa.controller.Bro default 12:40:20.375029+0000 Cyberduck Ignore return code -1 default 12:40:20.392688+0000 Cyberduck Missing result for worker MountWorker{cache=ch.cyberduck.core.ReverseLookupCache@3eee3e2b}. Use default value.
comment:3 follow-up: ↓ 5 Changed on Nov 27, 2019 at 1:04:50 PM by dkocher
Please make sure to enable virtual-host-style requests in their Minio configuration.
comment:4 Changed on Nov 27, 2019 at 1:05:55 PM by dkocher
- Component changed from core to s3
- Owner set to dkocher
- Priority changed from high to normal
- Severity changed from critical to normal
comment:5 in reply to: ↑ 3 Changed on Nov 27, 2019 at 1:12:15 PM by si458
Replying to dkocher:
Please make sure to enable virtual-host-style requests in their Minio configuration.
this is already enabled, as i couldnt access it first time round without setting the MINIO_DOMAIN in docker as it generated a different error
but after enabling it, im now getting this XML error as listed above
comment:6 Changed on Nov 27, 2019 at 1:27:01 PM by dkocher
- Milestone set to 8.0
- Summary changed from update 7.2.0 onwards seems to kill minio s3 support to Failed to parse XML document (Minio)
comment:7 Changed on Nov 27, 2019 at 1:32:55 PM by si458
- Version changed from 7.2 to 7.1.2
ive just tried 7.1.2 with virtual-host-style and that doesn't work either
just keeps getting same error
Failed to parse XML document with handler class org.jets3t.service.impl.rest.XmlResponsesSaxParser$ListAllMyBucketsHandler. Please contact your web hosting service provider for assistance
but the log drawer show it returns HTTP/1.1 200 OK with a valid XML too
comment:8 Changed on Nov 27, 2019 at 1:34:25 PM by dkocher
Regression from r48152.
comment:9 follow-up: ↓ 10 Changed on Nov 27, 2019 at 1:35:10 PM by dkocher
Can you provide the captured XML response?
comment:10 in reply to: ↑ 9 Changed on Nov 27, 2019 at 1:49:24 PM by si458
comment:11 follow-ups: ↓ 18 ↓ 20 Changed on Nov 27, 2019 at 2:04:15 PM by si458
ok i think i might have found the problem and maybe a bug but will try explain
- if you use an IP address instead of a hostname, it looks for http://wtf123.192.168.1.100:9000/ which is invalid,
so maybe a checker if i use an IP address instead of a hostname then ALWAYS USE PATH-STYLE !!!
- if i use a hostname with minio and have virtual-host-style DISABLED, cyberduck ALWAYS trys accessing the bucket using virtual-host-style regardless e.g: http://wtf123.s3.myserver.com:12345/
so i think it would be nice to have an option inside the connection settings to specify weather to use virtual-host-style or path-style (maybe per connection?)
- if i use the full virtual-host-style as the hostname e.g: http://wtf123.s3.myserver.com:12345/
i get an XML errror, even tho the XML is valid and returned correctly? https://pastebin.com/7knB6DcW
comment:12 Changed on Nov 27, 2019 at 2:43:13 PM by dkocher
Thanks for summarizing the issues. Accessing via the hostname set with MINIO_DOMAIN this cannot be reproduced.
comment:13 Changed on Nov 27, 2019 at 2:44:04 PM by dkocher
The error when accessing using the bucket name in Host configuration setting of the bookmark is that we expect the server to return a bucket list and not an object list.
comment:14 Changed on Nov 27, 2019 at 2:52:04 PM by dkocher
We have requested a documentation update for Minio in https://github.com/minio/cookbook/issues/280.
comment:15 follow-up: ↓ 16 Changed on Nov 27, 2019 at 2:55:13 PM by si458
surely it would be a better option to give the user control over path-style or virtual-host-style than forcing the user to use virtual-host-style?
comment:16 in reply to: ↑ 15 ; follow-up: ↓ 17 Changed on Nov 27, 2019 at 3:07:52 PM by dkocher
Replying to si458:
surely it would be a better option to give the user control over path-style or virtual-host-style than forcing the user to use virtual-host-style?
We will see if we can support this via an option in the connection profiles.
comment:17 in reply to: ↑ 16 Changed on Nov 27, 2019 at 3:40:21 PM by si458
Replying to dkocher:
Replying to si458:
surely it would be a better option to give the user control over path-style or virtual-host-style than forcing the user to use virtual-host-style?
We will see if we can support this via an option in the connection profiles.
thank you that would be helpful!
or even a tick box saying saying 'USE PATH-STYLE' in the edit bookmarks window/open connection window
comment:18 in reply to: ↑ 11 Changed on Nov 28, 2019 at 9:47:57 AM by dkocher
- Resolution set to fixed
- Status changed from new to closed
Replying to si458:
ok i think i might have found the problem and maybe a bug but will try explain
- if you use an IP address instead of a hostname, it looks for http://wtf123.192.168.1.100:9000/ which is invalid,
so maybe a checker if i use an IP address instead of a hostname then ALWAYS USE PATH-STYLE !!!
In r48257.
comment:19 Changed on Nov 28, 2019 at 9:55:04 AM by dkocher
- Resolution fixed deleted
- Status changed from closed to reopened
Pending change for Failed to parse XML document error to detect if we are connecting to a specific bucket and should list objects instead of all buckets.
comment:20 in reply to: ↑ 11 Changed on Nov 28, 2019 at 2:03:07 PM by dkocher
Replying to si458:
- if i use the full virtual-host-style as the hostname e.g: http://wtf123.s3.myserver.com:12345/
i get an XML errror, even tho the XML is valid and returned correctly? https://pastebin.com/7knB6DcW
Set the bucket name in the Path setting in the bookmark instead.
comment:21 Changed on Nov 28, 2019 at 8:14:56 PM by dkocher
- Summary changed from Failed to parse XML document (Minio) to Interoperability with Minio
comment:22 Changed on Dec 6, 2019 at 2:25:43 PM by dkocher
- Milestone changed from 8.0 to 7.2.2
comment:23 Changed on Dec 9, 2019 at 7:45:19 AM by yla
- Resolution set to fixed
- Status changed from reopened to closed
In r48333.
comment:24 follow-ups: ↓ 25 ↓ 29 Changed on Jan 7, 2020 at 6:43:16 PM by dimm00
Trying to make cyberduck work with ceph S3, which only supports path style Tried using IP, which broke the cert and didn't work either Is there a way to reenable path style S3?
comment:25 in reply to: ↑ 24 ; follow-up: ↓ 26 Changed on Jan 10, 2020 at 7:54:50 PM by dkocher
Replying to dimm00:
Trying to make cyberduck work with ceph S3, which only supports path style Tried using IP, which broke the cert and didn't work either Is there a way to reenable path style S3?
Is this Ceph instance publicly reachable over the Internet?
comment:26 in reply to: ↑ 25 ; follow-up: ↓ 27 Changed on Jan 11, 2020 at 7:06:46 AM by dimm00
Replying to dkocher:
Replying to dimm00:
Trying to make cyberduck work with ceph S3, which only supports path style Tried using IP, which broke the cert and didn't work either Is there a way to reenable path style S3?
Is this Ceph instance publicly reachable over the Internet?
Yes. How can I send you credentials to access it?
comment:27 in reply to: ↑ 26 Changed on Jan 22, 2020 at 1:55:45 PM by dkocher
Replying to dimm00:
Replying to dkocher:
Replying to dimm00:
Trying to make cyberduck work with ceph S3, which only supports path style Tried using IP, which broke the cert and didn't work either Is there a way to reenable path style S3?
Is this Ceph instance publicly reachable over the Internet?
Yes. How can I send you credentials to access it?
Awesome. Please write to support@….
comment:28 Changed on Jan 22, 2020 at 1:56:52 PM by dkocher
Duplicate for #10956.
comment:29 in reply to: ↑ 24 ; follow-up: ↓ 31 Changed on Apr 24, 2020 at 11:30:37 AM by zagy__
Replying to dimm00:
Trying to make cyberduck work with ceph S3, which only supports path style Tried using IP, which broke the cert and didn't work either Is there a way to reenable path style S3?
Was that solved? I'm seeing the same problem with Ceph here.
comment:30 Changed on Apr 24, 2020 at 11:30:52 AM by zagy__
- Cc cz@… added
comment:31 in reply to: ↑ 29 Changed on Apr 24, 2020 at 11:41:14 AM by si458
i dont have this issue anymore as its been fixed in latest cyberduck version using the S3 (HTTPS) profile from the minio help page
the only issue i found was when using my home broadband 'virgin media' they take over my dns records,
so when cyberduck was looking for 'mybucket.s3.myserver.com' is should of returned blank A record as i only have an A record for 's3.myserver.com' which ment cyberduck would then reverted back to using the path format,
but 'virgin media' kept returning an IP address of theres for there stupid advanced search error page, so cyberduck kept trying to connect to them and just timing out instead of an error
once i disabled the 'advanced search error page' and rebooted our router and cleared my dns cache on laptop,
cyberduck started working without any issues
comment:32 Changed on Apr 24, 2020 at 1:22:24 PM by zagy__
Aha. What worked for me now, was to create a bookmark and add the bucket-url to the PATH there. So, worksforme now. Thanks.
comment:33 Changed on Apr 24, 2020 at 3:52:13 PM by dkocher
comment:34 Changed on Jan 18, 2021 at 10:23:10 AM by dkocher
Documentation for Minio in Minio Cloud Storage.
THIS IS THE CONSOLE DEBUG OUTPUT IF I CONNECT TO THE MINIO SERVER IP AND PORT AND THEN ASK TO LIST CONTENTS TO MYBUCKET123