Cyberduck Mountain Duck CLI

#10471 closed defect (fixed)

Interoperability with OpenStack Swift S3 middleware

Reported by: pawan Owned by: dkocher
Priority: high Milestone: 6.8.1
Component: s3 Version: 6.8.0
Severity: normal Keywords:
Cc: Architecture:
Platform: Windows 10

Description (last modified by pawan)

Hi, I have recently deployed an openstack cluster (Rocky version) using swift and keystone. I am testing my cluster with ec2 credentials for s3 support. Somehow, I am not able to see the objects inside containers, while using cyberduck, whereas, If I use other clients e.g. s3 browser etc., I am able to see all those objects. Cyberduck lists all containers and folder(s) inside those, but not objects.

Another thing, which I have noticed, If I connect via keystone using cyberduck, everything works great.

I have tried looking into code of s3api, and it seems to be returning correct object. I am not sure, why object listing is not happening with cyberduck client only....

Change History (12)

comment:1 Changed on Sep 25, 2018 at 7:29:28 AM by pawan

  • Description modified (diff)

I was exploring more on this and noticed this in debug log

2018-09-25 12:23:54,073 [background-14] DEBUG org.jets3t.service.impl.rest.XmlResponsesSaxParser - '''Parsing XML response document with handler: class org.jets3t.service.impl.rest.XmlResponsesSaxParser$ListBucketHandler'''

if the function above gets called, it lists objects otherwise not.

It so happened, that cyberduck listed objects only once, and, I was able to capture it in debug logs, when I refreshed, objects were not getting listed again

Last edited on Sep 25, 2018 at 1:36:53 PM by dkocher (previous) (diff)

comment:2 follow-up: Changed on Sep 25, 2018 at 1:53:05 PM by dkocher

It looks like we are expecting buckets whereas objects are returned. What is the URL you connect to?

Last edited on Sep 25, 2018 at 7:27:15 PM by dkocher (previous) (diff)

comment:3 in reply to: ↑ 2 Changed on Sep 25, 2018 at 2:26:11 PM by pawan

Replying to dkocher:

It looks like we are expecting buckets whereas objects are returned. What is the URL you connect to?

Hi David, I am connecting to openstack cluster swift proxy URL. I have created ec2 credentials of user and trying to connect via those. I am able to login and could see containers list, but when I try to go inside container and see objects, it doesn't list anything. Interestingly, if folders are present, those get displayed. Below is the sample response from s3api ( openstack swift middleware) for container's object and folders.

<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <Name>testing-swapi-1</Name>
    <Prefix/>
    <Marker/>
    <MaxKeys>1000</MaxKeys>
    <Delimiter>/</Delimiter>
    <IsTruncated>false</IsTruncated>
    <Contents>
        <Key>Tuning Your Swift Installation _ Software tuning.pdf</Key>
        <LastModified>2018-09-22T08:52:57.994Z</LastModified>
        <ETag>"0fa07e197c81907fdd9a20720ab82e51"</ETag>
        <Size>205721</Size>
        <Owner>
            <ID>swift3-test-creds-13840:swift3-test-creds-13840</ID>
            <DisplayName>swift3-test-creds-13840:swift3-test-creds-13840</DisplayName>
        </Owner>
        <StorageClass>STANDARD</StorageClass>
    </Contents>
    <Contents>
        <Key>cosbench_icpe13.pdf</Key>
        <LastModified>2018-09-22T07:25:37.525Z</LastModified>
        <ETag>"0b2d57b34ef2120cfb5bbf1a827af292"</ETag>
        <Size>761002</Size>
        <Owner>
            <ID>swift3-test-creds-13840:swift3-test-creds-13840</ID>
            <DisplayName>swift3-test-creds-13840:swift3-test-creds-13840</DisplayName>
        </Owner>
        <StorageClass>STANDARD</StorageClass>
    </Contents>
    <Contents>
        <Key>workload-config.xml</Key>
        <LastModified>2018-09-24T07:37:25.246Z</LastModified>
        <ETag>"f0581f6217d303d0504e96aa64c6f1c7"</ETag>
        <Size>7778</Size>
        <Owner>
            <ID>swift3-test-creds-13840:swift3-test-creds-13840</ID>
            <DisplayName>swift3-test-creds-13840:swift3-test-creds-13840</DisplayName>
        </Owner>
        <StorageClass>STANDARD</StorageClass>
    </Contents>
    <CommonPrefixes>
        <Prefix>folder1/</Prefix>
    </CommonPrefixes>
</ListBucketResult>

I do apologize for not formatting the contents, as I am very new to this.

Last edited on Sep 25, 2018 at 2:43:47 PM by dkocher (previous) (diff)

comment:4 follow-up: Changed on Sep 25, 2018 at 5:12:45 PM by pawan

This is the url, I am trying to connect to or.object-store.api.idrivecloud.io . At present, user creation is restricted, therefore, please let me know, where could I share ec2 credentials? e.g. please provide me your email address, where I could share those with you.

Last edited on Sep 25, 2018 at 5:13:35 PM by pawan (previous) (diff)

comment:5 in reply to: ↑ 4 ; follow-up: Changed on Sep 25, 2018 at 6:19:15 PM by dkocher

Replying to pawan:

This is the url, I am trying to connect to or.object-store.api.idrivecloud.io . At present, user creation is restricted, therefore, please let me know, where could I share ec2 credentials? e.g. please provide me your email address, where I could share those with you.

Please share the credentials with support@cyberduck.io.

comment:6 in reply to: ↑ 5 Changed on Sep 25, 2018 at 6:46:43 PM by pawan

Replying to dkocher:

Replying to pawan:

This is the url, I am trying to connect to or.object-store.api.idrivecloud.io . At present, user creation is restricted, therefore, please let me know, where could I share ec2 credentials? e.g. please provide me your email address, where I could share those with you.

Please share the credentials with support@cyberduck.io.

I have sent the same to given email address from my email address.

comment:7 Changed on Sep 25, 2018 at 7:18:08 PM by dkocher

The reply does not include the Version element we expect as we request the versions subresource in the GET request.

comment:8 Changed on Sep 25, 2018 at 7:28:43 PM by dkocher

  • Summary changed from Unable to see list of objects, while using ec2 credentials for openstack cluster to Interoperability with OpenStack Swift S3 middleware

comment:9 Changed on Sep 28, 2018 at 6:25:06 AM by pawan

as per this URL https://docs.openstack.org/swift/latest/s3_compat.html, openstack s3 api doesn't support versioning. It would be great, cyberduck could provide an option and it starts working without versions subresource in GET request. Also, I would appreciate, if I would have an idea about timelines for this issue.

Last edited on Sep 28, 2018 at 6:26:34 AM by pawan (previous) (diff)

comment:10 follow-up: Changed on Sep 30, 2018 at 11:31:54 AM by dkocher

  • Milestone set to 6.8.1
  • Owner set to dkocher
  • Status changed from new to assigned

comment:11 in reply to: ↑ 10 Changed on Oct 1, 2018 at 1:55:38 AM by pawan

Replying to dkocher: Thanks for updating the issue and setting the milestone.

comment:12 Changed on Oct 1, 2018 at 8:08:21 PM by dkocher

  • Resolution set to fixed
  • Status changed from assigned to closed

In r45172.

Note: See TracTickets for help on using tickets.
swiss made software