Cyberduck Mountain Duck CLI

#11041 closed defect (fixed)

Cannot use AWS regions other than us-east-1 for S3

Reported by: Shu99 Owned by: dkocher
Priority: normal Milestone: 7.4
Component: s3 Version: 7.3.1
Severity: normal Keywords:
Cc: Architecture:
Platform:

Description

I am unable to use s3.us-west-2.amazonaws.com as the server.

The error I get is:

"Listing directory / failed. The specified bucket is not valid. Please contact your web hosting provider for assistance.

I am able to connect fine to s3.amazonaws.com.

Looking at the tcpdump for an unencrypted session showed me that S3 endpoint is responding with the following error:

<?xml version="1.0" encoding="UTF-8"?> <Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'us-west-2'</Message><Region>us-west-2</Region><RequestId>0E88FE65A230B8D5</RequestId><HostId>5O27uP0oDq/v2S0r9Wi4TwnNRbinyIQBZ1361AKHzQBaSrWnK/9flEI+ntI0WG0grGutfH0TQ=</HostId></Error> 0

The "Authorization" header contains the 'us-east-1' region even when 's3.us-west-2.amazonaws.com' is used as the server.

Looks like a default value is being picked up in jets3t here: https://github.com/mondain/jets3t/blob/master/jets3t/src/main/java/org/jets3t/service/impl/rest/httpclient/RestStorageService.java#L788-L790

Please investigate this issue.

Cyberduck Version = 7.3.1 (32784)

OS Version = macOS 10.15.3 (19D76)

Change History (6)

comment:1 Changed on May 3, 2020 at 9:01:50 AM by dkocher

  • Component changed from core to s3
  • Owner set to dkocher

We only have region specific connection profiles for China at https://cyberduck.io/s3/. For other regions, supplying a region specific endpoint is not supported. However you should be able to set the endpoint to a specific bucket using the old virtual host style endpoint pattern using bucketname.s3.amazonaws.com. We should derive the correct location automatically.

comment:2 Changed on May 3, 2020 at 9:02:33 AM by dkocher

Relates to #11036.

comment:3 Changed on May 3, 2020 at 4:40:57 PM by Shu99

Thanks for the quick response! I was able to connect using the bucketname.s3.amazonaws.com server. Then I realized that even using s3.amazonaws.com as server results in connections being opened to the bucket's region (us-west-2 in my case).

I have two recommendations:

  • Endpoint locality behavior is worth noting on the Cyberduck/S3 page, as folks who're trying to optimize the performance of their transfers may be misled and think that their data is flowing to the endpoint specified in the server field.
  • The error message was very confusing when I used s3.us-west-2.amazonaws.com. Improving this error message will greatly improve customer experience in this scenario.

comment:4 Changed on May 5, 2020 at 6:54:53 PM by dkocher

  • Milestone set to 8.0
  • Resolution set to fixed
  • Status changed from new to closed

You will no longer see this error message with our latest changes from #11036.

comment:5 Changed on May 6, 2020 at 7:48:08 AM by Shu99

Thank you!

comment:6 Changed on May 22, 2020 at 9:13:36 AM by dkocher

  • Milestone changed from 8.0 to 7.4
Note: See TracTickets for help on using tickets.
swiss made software