Cyberduck Mountain Duck CLI

#10471 closed defect (fixed)

Interoperability with OpenStack Swift S3 middleware

Reported by: pawan Owned by: dkocher
Priority: high Milestone: 6.8.1
Component: s3 Version: 6.8.0
Severity: normal Keywords:
Cc: Architecture:
Platform: Windows 10

Description (last modified by pawan)

Hi, I have recently deployed an openstack cluster (Rocky version) using swift and keystone. I am testing my cluster with ec2 credentials for s3 support. Somehow, I am not able to see the objects inside containers, while using cyberduck, whereas, If I use other clients e.g. s3 browser etc., I am able to see all those objects. Cyberduck lists all containers and folder(s) inside those, but not objects.

Another thing, which I have noticed, If I connect via keystone using cyberduck, everything works great.

I have tried looking into code of s3api, and it seems to be returning correct object. I am not sure, why object listing is not happening with cyberduck client only....

Change History (12)

comment:1 Changed on Sep 25, 2018 at 7:29:28 AM by pawan

  • Description modified (diff)

I was exploring more on this and noticed this in debug log

2018-09-25 12:23:54,073 [background-14] DEBUG org.jets3t.service.impl.rest.XmlResponsesSaxParser - '''Parsing XML response document with handler: class org.jets3t.service.impl.rest.XmlResponsesSaxParser$ListBucketHandler'''

if the function above gets called, it lists objects otherwise not.

It so happened, that cyberduck listed objects only once, and, I was able to capture it in debug logs, when I refreshed, objects were not getting listed again

Last edited on Sep 25, 2018 at 1:36:53 PM by dkocher (previous) (diff)

comment:2 follow-up: Changed on Sep 25, 2018 at 1:53:05 PM by dkocher

It looks like we are expecting buckets whereas objects are returned. What is the URL you connect to?

Last edited on Sep 25, 2018 at 7:27:15 PM by dkocher (previous) (diff)

comment:3 in reply to: ↑ 2 Changed on Sep 25, 2018 at 2:26:11 PM by pawan

Replying to dkocher:

It looks like we are expecting buckets whereas objects are returned. What is the URL you connect to?

Hi David, I am connecting to openstack cluster swift proxy URL. I have created ec2 credentials of user and trying to connect via those. I am able to login and could see containers list, but when I try to go inside container and see objects, it doesn't list anything. Interestingly, if folders are present, those get displayed. Below is the sample response from s3api ( openstack swift middleware) for container's object and folders.

<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <Name>testing-swapi-1</Name>
    <Prefix/>
    <Marker/>
    <MaxKeys>1000</MaxKeys>
    <Delimiter>/</Delimiter>
    <IsTruncated>false</IsTruncated>
    <Contents>
        <Key>Tuning Your Swift Installation _ Software tuning.pdf</Key>
        <LastModified>2018-09-22T08:52:57.994Z</LastModified>
        <ETag>"0fa07e197c81907fdd9a20720ab82e51"</ETag>
        <Size>205721</Size>
        <Owner>
            <ID>swift3-test-creds-13840:swift3-test-creds-13840</ID>
            <DisplayName>swift3-test-creds-13840:swift3-test-creds-13840</DisplayName>
        </Owner>
        <StorageClass>STANDARD</StorageClass>
    </Contents>
    <Contents>
        <Key>cosbench_icpe13.pdf</Key>
        <LastModified>2018-09-22T07:25:37.525Z</LastModified>
        <ETag>"0b2d57b34ef2120cfb5bbf1a827af292"</ETag>
        <Size>761002</Size>
        <Owner>
            <ID>swift3-test-creds-13840:swift3-test-creds-13840</ID>
            <DisplayName>swift3-test-creds-13840:swift3-test-creds-13840</DisplayName>
        </Owner>
        <StorageClass>STANDARD</StorageClass>
    </Contents>
    <Contents>
        <Key>workload-config.xml</Key>
        <LastModified>2018-09-24T07:37:25.246Z</LastModified>
        <ETag>"f0581f6217d303d0504e96aa64c6f1c7"</ETag>
        <Size>7778</Size>
        <Owner>
            <ID>swift3-test-creds-13840:swift3-test-creds-13840</ID>
            <DisplayName>swift3-test-creds-13840:swift3-test-creds-13840</DisplayName>
        </Owner>
        <StorageClass>STANDARD</StorageClass>
    </Contents>
    <CommonPrefixes>
        <Prefix>folder1/</Prefix>
    </CommonPrefixes>
</ListBucketResult>

I do apologize for not formatting the contents, as I am very new to this.

Last edited on Sep 25, 2018 at 2:43:47 PM by dkocher (previous) (diff)

comment:4 follow-up: Changed on Sep 25, 2018 at 5:12:45 PM by pawan

This is the url, I am trying to connect to or.object-store.api.idrivecloud.io . At present, user creation is restricted, therefore, please let me know, where could I share ec2 credentials? e.g. please provide me your email address, where I could share those with you.

Last edited on Sep 25, 2018 at 5:13:35 PM by pawan (previous) (diff)

comment:5 in reply to: ↑ 4 ; follow-up: Changed on Sep 25, 2018 at 6:19:15 PM by dkocher

Replying to pawan:

This is the url, I am trying to connect to or.object-store.api.idrivecloud.io . At present, user creation is restricted, therefore, please let me know, where could I share ec2 credentials? e.g. please provide me your email address, where I could share those with you.

Please share the credentials with support@cyberduck.io.

comment:6 in reply to: ↑ 5 Changed on Sep 25, 2018 at 6:46:43 PM by pawan

Replying to dkocher:

Replying to pawan:

This is the url, I am trying to connect to or.object-store.api.idrivecloud.io . At present, user creation is restricted, therefore, please let me know, where could I share ec2 credentials? e.g. please provide me your email address, where I could share those with you.

Please share the credentials with support@cyberduck.io.

I have sent the same to given email address from my email address.

comment:7 Changed on Sep 25, 2018 at 7:18:08 PM by dkocher

The reply does not include the Version element we expect as we request the versions subresource in the GET request.

comment:8 Changed on Sep 25, 2018 at 7:28:43 PM by dkocher

  • Summary changed from Unable to see list of objects, while using ec2 credentials for openstack cluster to Interoperability with OpenStack Swift S3 middleware

comment:9 Changed on Sep 28, 2018 at 6:25:06 AM by pawan

as per this URL https://docs.openstack.org/swift/latest/s3_compat.html, openstack s3 api doesn't support versioning. It would be great, cyberduck could provide an option and it starts working without versions subresource in GET request.

Version 0, edited on Sep 28, 2018 at 6:25:06 AM by pawan (next)

comment:10 follow-up: Changed on Sep 30, 2018 at 11:31:54 AM by dkocher

  • Milestone set to 6.8.1
  • Owner set to dkocher
  • Status changed from new to assigned

comment:11 in reply to: ↑ 10 Changed on Oct 1, 2018 at 1:55:38 AM by pawan

Replying to dkocher: Thanks for updating the issue and setting the milestone.

comment:12 Changed on Oct 1, 2018 at 8:08:21 PM by dkocher

  • Resolution set to fixed
  • Status changed from assigned to closed

In r45172.

Note: See TracTickets for help on using tickets.
swiss made software