Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cannot use AWS regions other than us-east-1 for S3 #11041

Closed
cyberduck opened this issue May 3, 2020 · 6 comments
Closed

Cannot use AWS regions other than us-east-1 for S3 #11041

cyberduck opened this issue May 3, 2020 · 6 comments
Assignees
@dkocher
Labels
bug fixed s3 AWS S3 Protocol Implementation
Milestone
7.4.0

Comments

@cyberduck
Copy link
Collaborator

8ae4c3e created the issue

I am unable to use s3.us-west-2.amazonaws.com as the server.

The error I get is:

"Listing directory / failed.
The specified bucket is not valid. Please contact your web hosting provider for assistance.

I am able to connect fine to s3.amazonaws.com.

Looking at the tcpdump for an unencrypted session showed me that S3 endpoint is responding with the following error:

AuthorizationHeaderMalformedThe authorization header is malformed; the region 'us-east-1' is wrong; expecting 'us-west-2'us-west-20E88FE65A230B8D55O27uP0oDq/v2S0r9Wi4TwnNRbinyIQBZ1361AKHzQBaSrWnK/9//flEI+ntI0WG0grGutfH0TQ=
0

The "Authorization" header contains the 'us-east-1' region even when 's3.us-west-2.amazonaws.com' is used as the server.

Looks like a default value is being picked up in jets3t here:
https://github.com/mondain/jets3t/blob/master/jets3t/src/main/java/org/jets3t/service/impl/rest/httpclient/RestStorageService.java#L788-L790

Please investigate this issue.

Cyberduck Version = 7.3.1 (32784)

OS Version = macOS 10.15.3 (19D76)
@cyberduck
Copy link
Collaborator Author

@dkocher commented

We only have region specific connection profiles for China at https://cyberduck.io/s3/. For other regions, supplying a region specific endpoint is not supported. However you should be able to set the endpoint to a specific bucket using the old virtual host style endpoint pattern using bucketname.s3.amazonaws.com. We should derive the correct location automatically.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Relates to #11036.

@cyberduck
Copy link
Collaborator Author

8ae4c3e commented

Thanks for the quick response! I was able to connect using the bucketname.s3.amazonaws.com server. Then I realized that even using s3.amazonaws.com as server results in connections being opened to the bucket's region (us-west-2 in my case).

I have two recommendations:

  • Endpoint locality behavior is worth noting on the Cyberduck/S3 page, as folks who're trying to optimize the performance of their transfers may be misled and think that their data is flowing to the endpoint specified in the server field.
  • The error message was very confusing when I used s3.us-west-2.amazonaws.com. Improving this error message will greatly improve customer experience in this scenario.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

You will no longer see this error message with our latest changes from #11036.

@cyberduck
Copy link
Collaborator Author

8ae4c3e commented

Thank you!

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Milestone renamed

@iterate-ch iterate-ch locked as resolved and limited conversation to collaborators Nov 27, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@dkocher dkocher

Labels
bug fixed s3 AWS S3 Protocol Implementation
Projects
None yet
Milestone
7.4.0
Development

No branches or pull requests
2 participants
@dkocher @cyberduck