Cyberduck Mountain Duck CLI

#10594 closed defect (fixed)

Temporary tokens (credentials from AWS STS) do not work with AWS GovCloud S3

Reported by: cduser Owned by: dkocher
Priority: high Milestone: 6.9.3
Component: s3 Version: 6.9.2
Severity: major Keywords: s3 iam sts mfa
Cc: Architecture:
Platform: macOS 10.13

Description

The S3(Credentials from AWS Security Token Service) profile (​https://svn.cyberduck.io/trunk/profiles/S3%20(Credentials%20from%20AWS%20Security%20Token%20Service).cyberduckprofile) does not work with AWS GovCloud accounts. Cyberduck gets into a loop where it says "Authenticating as publish_profile" followed by "Login failed". I also tried using the AWS GovCloud profile (https://svn.cyberduck.io/trunk/profiles/S3%20Gov%20Cloud.cyberduckprofile), but it doesn't support temporary tokens. I also tried creating my own profile merging 'S3(Credentials from AWS Security Token Service) and AWS GovCloud but that didn't work either. Here is the custom profile I tried out:

<?xml version="1.0" encoding="UTF-8"?>
<!--
  ~ Copyright (c) 2002-2018 iterate GmbH. All rights reserved.
  ~ https://cyberduck.io/
  ~
  ~ This program is free software; you can redistribute it and/or modify
  ~ it under the terms of the GNU General Public License as published by
  ~ the Free Software Foundation, either version 3 of the License, or
  ~ (at your option) any later version.
  ~
  ~ This program is distributed in the hope that it will be useful,
  ~ but WITHOUT ANY WARRANTY; without even the implied warranty of
  ~ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  ~ GNU General Public License for more details.
  -->

<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
    <dict>
        <key>Protocol</key>
        <string>s3</string>
        <key>Vendor</key>
        <string>s3-token</string>
        <key>Scheme</key>
        <string>https</string>
        <key>Description</key>
        <string>AWS GovCloud S3</string>
        <key>Default Port</key>
        <string>443</string>
        <key>Default Nickname</key>
        <string>AWS GovCloud S3</string>
        <key>Default Hostname</key>
        <string>s3-us-gov-west-1.amazonaws.com</string>
        <key>Username Placeholder</key>
        <string>Profile Name</string>
        <key>Password Configurable</key>
        <false/>
        <key>Token Configurable</key>
        <false/>
        <key>Anonymous Configurable</key>
        <false/>
        <key>Region</key>
        <string>us-gov-west-1</string>
    </dict>
</plist>

This is the AWS credentials file I'm using:

[publish_profile]
output = json
region = us-gov-west-1
aws_access_key_id = AAAAAAAAAAAAAAAAAAAA
aws_secret_access_key = KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK
aws_session_token = SSSSSSSSSSS//////////SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS

Is there a way to support both AWS GovCloud and S3 (Credentials from AWS Security Token Service) at the same time?

Thanks!

Change History (12)

comment:1 Changed on Feb 3, 2019 at 4:51:28 PM by dkocher

  • Milestone set to 6.9.3
  • Owner set to dkocher
  • Status changed from new to assigned

comment:2 Changed on Feb 3, 2019 at 6:33:35 PM by dkocher

The auto configuration from ~/aws/credentials is currently only triggered when the hostname defaults to s3.amazonaws.com.

comment:3 follow-up: Changed on Feb 4, 2019 at 12:35:51 AM by cduser

Thanks for checking! I also want to mention that there are now 2 GovCloud regions with different endpoints:

s3-us-gov-west-1.amazonaws.com
s3.us-gov-east-1.amazonaws.com
Last edited on Feb 4, 2019 at 12:36:18 AM by cduser (previous) (diff)

comment:4 follow-up: Changed on Feb 4, 2019 at 7:00:18 AM by yla

  • Resolution set to fixed
  • Status changed from assigned to closed

Fixed in r46275.

comment:5 in reply to: ↑ 4 Changed on Feb 4, 2019 at 8:29:49 AM by cduser

Replying to yla:

Fixed in r46275.

Thanks a lot! Two questions:

  1. When will the update make it to a snapshot release?
  2. Is there a way to configure a default value for the Profile Name in ~/.aws/credentials in the .cyberduckprofile file?

Thanks!

comment:6 in reply to: ↑ 3 Changed on Feb 4, 2019 at 4:40:18 PM by dkocher

Replying to cduser:

Thanks for checking! I also want to mention that there are now 2 GovCloud regions with different endpoints:

s3-us-gov-west-1.amazonaws.com
s3.us-gov-east-1.amazonaws.com

Added profile in r46283.

comment:7 follow-up: Changed on Feb 4, 2019 at 5:21:41 PM by dkocher

A new snapshot build has now been published. Please comment on this ticket if the issue is resolved (or reopen) as we cannot fully test this ourselves.

comment:8 in reply to: ↑ 7 Changed on Feb 4, 2019 at 6:41:32 PM by cduser

Replying to dkocher:

A new snapshot build has now been published. Please comment on this ticket if the issue is resolved (or reopen) as we cannot fully test this ourselves.

Hi dkocher,

I can help with the testing. Unfortunately I'm still having issues, but its looking better.

This is what I'm doing:

I'm using the following profile:

<?xml version="1.0" encoding="UTF-8"?>
<!--
  ~ Copyright (c) 2002-2018 iterate GmbH. All rights reserved.
  ~ https://cyberduck.io/
  ~
  ~ This program is free software; you can redistribute it and/or modify
  ~ it under the terms of the GNU General Public License as published by
  ~ the Free Software Foundation, either version 3 of the License, or
  ~ (at your option) any later version.
  ~
  ~ This program is distributed in the hope that it will be useful,
  ~ but WITHOUT ANY WARRANTY; without even the implied warranty of
  ~ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  ~ GNU General Public License for more details.
  -->

<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
    <dict>
        <key>Protocol</key>
        <string>s3</string>
        <key>Vendor</key>
        <string>s3-token</string>
        <key>Description</key>
        <string>S3 (Credentials from AWS Security Token Service)</string>
        <key>Default Nickname</key>
        <string>S3 (Credentials from AWS Security Token Service)</string>
        <key>Username Placeholder</key>
        <string>Profile Name in ~/.aws/credentials</string>
        <key>Password Configurable</key>
        <false/>
        <key>Token Configurable</key>
        <false/>
        <key>Anonymous Configurable</key>
        <false/>
        <key>Region</key>
        <string>us-gov-west-1</string>
    </dict>
</plist>

When adding the profile to cyberduck I setServer to s3-us-gov-west-1.amazonaws.com and Profile Name in ~/.aws/credentials to cyberduck. I then get new temporary credentials from AWS and put them in my ~/.aws/credentials file like this:

[cyberduck]
output = json
region = us-gov-west-1
aws_access_key_id = AAAAAAAAAAAAAAAAAAAA
aws_secret_access_key = KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK
aws_session_token = SSSSSSSSSSS//////////SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS

When I double click on the the shortcut in cyberduck I see Authenticating as cyberduck in the lower left corner with a spinning icon, but it never connects. If I try to close cyberduck it gets locked up and I have to force quit. Is there a way to enable debug logs?

Thank you for all your help and prompt response!

comment:9 Changed on Feb 4, 2019 at 6:41:41 PM by cduser

  • Resolution fixed deleted
  • Status changed from closed to reopened

comment:10 Changed on Feb 4, 2019 at 8:34:37 PM by dkocher

The error I can reproduce here is

<?xml version="1.0" encoding="UTF-8"?><Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization header is malformed; the region 'us-gov-west-1' is wrong; expecting 'us-east-1'</Message><Region>us-east-1</Region><RequestId>6DC92B83F187052E</RequestId><HostId>/i/KpB+I0jY/luCGm6wHoW5YJjdxMYTknIMe9rYtCMInebV+rJBtiI8b9sK7NXfOzS+n0wuMUTQ=</HostId></Error>

comment:11 follow-up: Changed on Feb 4, 2019 at 9:38:06 PM by dkocher

  • Resolution set to fixed
  • Status changed from reopened to closed

Fix regression in r46287.

comment:12 in reply to: ↑ 11 Changed on Feb 4, 2019 at 11:29:13 PM by cduser

Replying to dkocher:

Fix regression in r46287.

Great! The AWS GovCloud S3 login issues are fixed. I can now log in (using the configuration in the credentials file) and see the directory listing. Unfortunately I run into an error when trying to download files or directories. Since I get the same error using the normal access keys, I created a new ticket #10596.

Thanks!

Last edited on Feb 5, 2019 at 4:22:42 PM by dkocher (previous) (diff)
Note: See TracTickets for help on using tickets.
swiss made software