Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Openstack Swift (Amazon S3) - Listing directory failed #13383

Closed
tdvgit opened this issue May 12, 2022 · 13 comments · Fixed by #13405
Closed

Openstack Swift (Amazon S3) - Listing directory failed #13383

tdvgit opened this issue May 12, 2022 · 13 comments · Fixed by #13405
Assignees
Labels
openstack OpenStack Swift Protocol Implementation s3 AWS S3 Protocol Implementation
Milestone

Comments

@tdvgit
Copy link

tdvgit commented May 12, 2022

Describe the bug
Hello, when i try to use Openstack swift over Amazon S3, i get an error when try to listing "bucket".
The behavior is absolutely similar on several providers.
Have been tested: OVH, Oracle, SC

When trying to list the contents of the container, an error occurs and nothing else happens.
"Listing directory XXX failed. Failed to parse XML document with handler class org.jets3t.service.impl.rest.XmlResponsesSaxParser$ListBucketHandler. Please contact your web hosting service provider for assistance."
To Reproduce
To repeat, you can use the object storage service of any provider that provides Openstack Swift through S3.

Expected behavior
I expected to see a listing of files in a container

Cyberduck Version 8.3.3 (37544) (MacOS)

@tdvgit
Copy link
Author

tdvgit commented May 13, 2022

Very similar to a bug that may have been previously fixed, but is now back again.
#10471

Now i have next reply with s3cmd:

DEBUG: Response:
{'data': b"<?xml version='1.0' encoding='UTF-8'?>\n<ListBucketResult xml"
         b'ns="http://s3.amazonaws.com/doc/2006-03-01/"><Name>bucketname</Name><Prefix'
         b'/><Marker/><MaxKeys>1000</MaxKeys><Delimiter>/</Delimiter><IsTruncat'
         b'ed>false</IsTruncated><Contents><Key>Screenshot 2022-04-14 at 23.50.'
         b'21.png</Key><LastModified>2022-05-12T13:40:17.541Z</LastModified><ET'
         b'ag>"3fcf1189802cc75006100190372048ad"</ETag><Size>187976</Size><Owne'
         b'r><ID>testtenant:testuser</ID><DisplayName>testtenant:testuser</DisplayName></'
         b'Owner><StorageClass>STANDARD</StorageClass></Contents><Contents><Key'
         b'>Screenshot 2022-04-14 at 23.50.36.png</Key><LastModified>2022-05-12'
         b'T13:40:17.541Z</LastModified><ETag>"5e3fedc3bf0fd8e3aead5cecaa285fdb'
         b'"</ETag><Size>137389</Size><Owner><ID>testtenant:testuser</ID><DisplayNam'
         b'e>testtenant:testuser</DisplayName></Owner><StorageClass>STANDARD</Storag'
         b'eClass></Contents></ListBucketResult>',
 'headers': {'connection': 'keep-alive',
             'content-length': '845',
             'content-type': 'application/xml',
             'date': 'Fri, 13 May 2022 10:22:51 GMT',
             'server': 'nginx',
             'x-amz-id-2': 'txd6d39ecaa69c4078b7074-00627e317b',
             'x-amz-request-id': 'txd6d39ecaa69c4078b7074-00627e317b',
             'x-openstack-request-id': 'txd6d39ecaa69c4078b7074-00627e317b',
             'x-trans-id': 'txd6d39ecaa69c4078b7074-00627e317b'},
 'reason': 'OK',
 'status': 200}

Clean XML

<?xml version='1.0' encoding='UTF-8'?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <Name>bucketname</Name>
    <Prefix/>
    <Marker/>
    <MaxKeys>1000</MaxKeys>
    <Delimiter>/</Delimiter>
    <IsTruncated>false</IsTruncated>
    <Contents>
        <Key>Screenshot 2022-04-14 at 23.50.21.png</Key>
        <LastModified>2022-05-12T13:40:17.541Z</LastModified>
        <ETag>"3fcf1189802cc75006100190372048ad"</ETag>
        <Size>187976</Size>
        <Owner>
            <ID>testtenant:testuser</ID>
            <DisplayName>testtenant:testuser</DisplayName>
        </Owner>
        <StorageClass>STANDARD</StorageClass>
    </Contents>
    <Contents>
        <Key>Screenshot 2022-04-14 at 23.50.36.png</Key>
        <LastModified>2022-05-12T13:40:17.541Z</LastModified>
        <ETag>"5e3fedc3bf0fd8e3aead5cecaa285fdb"</ETag>
        <Size>137389</Size>
        <Owner>
            <ID>testtenant:testuser</ID>
            <DisplayName>testtenant:testuser</DisplayName>
        </Owner>
        <StorageClass>STANDARD</StorageClass>
    </Contents>
</ListBucketResult>

Just like that time, the element Version is not available.
Also, listing working with any other client, like s3cmd, except Cyberduck.

@clementnuss
Copy link

just for info, I'm reporting the same behaviour with MacOS version 8.3.3 and trying to connect to Infomaniak SwissBackup (also an Openstack Swift S3 mode):
image

@dkocher dkocher added the s3 AWS S3 Protocol Implementation label May 19, 2022
@dkocher dkocher self-assigned this May 19, 2022
@dkocher
Copy link
Contributor

dkocher commented May 19, 2022

just for info, I'm reporting the same behaviour with MacOS version 8.3.3 and trying to connect to Infomaniak SwissBackup (also an Openstack Swift S3 mode):

As a workaround please try to connect using the OpenStack Swift connection profile from Preferences → Profiles.

@dkocher
Copy link
Contributor

dkocher commented May 19, 2022

Very similar to a bug that may have been previously fixed, but is now back again. #10471

I cannot seem to get a test failure in e84e78f

@dkocher
Copy link
Contributor

dkocher commented May 19, 2022

The behavior is absolutely similar on several providers. Have been tested: OVH, Oracle, SC

As a workaround please try to connect using the OpenStack Swift connection profiles for OVH and S3 connection profiles for OCI Object Storage from Preferences → Profiles.

@dkocher dkocher added the openstack OpenStack Swift Protocol Implementation label May 19, 2022
@tdvgit
Copy link
Author

tdvgit commented May 20, 2022

Hello,

i am try with "OCI Object Storage" and get next error when connecting to storage:

Listing directory / failed.
The request signature we calculated does not match the signature you provided. Check your key and signing method. Please contact your web hosting service provider for assistance.

Screenshot 2022-05-20 at 12 52 00

But, to Oracle connecting successful and listing also.

@tdvgit
Copy link
Author

tdvgit commented May 20, 2022

Later i am test some profiles and found working profile with Openstack Swift over S3 - "S3 (Deprecated path style requests)".

Please create one more profile with name "Openstack Swift over S3", or something similar. Or add comment to "S3 (Deprecated path style requests)".

@clementnuss
Copy link

hi,

in my case, with OCI Object Storage (Oracle logo), I changed the server name and it works perfectly for infomaniak SwissBackup S3, thanks for the Workaround!

it would be nice to have a more generic profile for problematic S3 shares, or a tickbox "compatibility mode" for the "Amazon S3" to permit using 3rd party S3 providers.

thanks for the workaround already, really helping!

@dkocher
Copy link
Contributor

dkocher commented May 23, 2022

Fix error reporting to get exact parser failure in #13405.

dkocher added a commit that referenced this issue May 23, 2022
Fix error reporting to get parser failure.
@dkocher dkocher linked a pull request May 23, 2022 that will close this issue
@dkocher dkocher closed this as completed Jun 13, 2022
@dkocher dkocher added this to the 8.4.0 milestone Jun 13, 2022
@clementnuss
Copy link

@dkocher I think we need to reopen this issue, with 8.4.2 the error is still there, and the parser doesn't show much more info:

image

also, for whatever reason, the workaround using OCI Object Storage also produces the same error now.

If you prefer, I could also reopen another issue

@dkocher
Copy link
Contributor

dkocher commented Jul 31, 2022

@dkocher I think we need to reopen this issue, with 8.4.2 the error is still there, and the parser doesn't show much more info:

Can you confirm you are using one of the Infomaniak Swiss Backup connection profiles?

@dkocher
Copy link
Contributor

dkocher commented Jul 31, 2022

@dkocher I think we need to reopen this issue, with 8.4.2 the error is still there, and the parser doesn't show much more info:

Can you confirm you are using one of the Infomaniak Swiss Backup connection profiles?

Connection Profiles for Infomaniak Swiss Backup have been made available in e4bc1eb and #13468.

@clementnuss
Copy link

I'm not using SwissBackup profiles, as those are made for the Swift API of Infomaniak, and expect the following credentials:
image

In my case however, I'm using SwissBackup in S3 mode, for which the endpoint is s3.swiss-backup02.infomaniak.com and it uses the S3 protocol.

with the previous version of Cyberduck and the Oracle OCI S3 profile, I was able to work with SwissBackup S3 mode.
With the current version however, the error is again there.

I can open a new issue if you prefer?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
openstack OpenStack Swift Protocol Implementation s3 AWS S3 Protocol Implementation
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants