Cyberduck Mountain Duck CLI

Opened 3 years ago

Last modified 8 weeks ago

#8880 assigned feature

Authentication using AWS AssumeRole and GetSessionToken with AWS STS

Reported by: tigris Owned by: dkocher
Priority: high Milestone: 7.0
Component: s3 Version: 4.7
Severity: normal Keywords: s3 iam sts mfa
Cc: Architecture: Intel
Platform: Mac OS X 10.10

Description (last modified by dkocher)

I am using amazon AssumeRole function to assume a role that can access an S3 bucket. This means to connect to S3, it needs more than just SecretKey and AccessKey, it also need SecurityToken or SessionToken which is an extremely large string.

If you set these 3 things in your environment, you can use tools like awscli etc from command line. I would like to use cyberduck instead as it can thread nicely. But unfortunately cyberduck only supports IAM users and not roles.

It does support roles from an EC2 instance, so I think it should be very easy to support from my own OSX laptop? I was thinking of just running a local proxy for to fake the fact I am not running on EC2, but it seemed like overkill.

I notice a few people are suggesting entry of the security token - but isn't that short-lived? Don't see how that's a stable configuration solution. When configuring AWS CLI for this, I'd have an entry for the master account, and then one entry for each assumed role, such as:

[profile master]
region = us-east-1
output = json

[profile security]
role_arn = arn:aws:iam::999999999999:role/MyAccessRole
source_profile = master

I think you need a way to collect and use this information, mainly the role_arn and reference to the source_profile.

Change History (41)

comment:1 Changed 3 years ago by dkocher

Referencing Support S3 authentication via IAM Role credentials #8610.

comment:2 Changed 3 years ago by dkocher

  • Component changed from core to s3
  • Owner set to dkocher
  • Summary changed from S3 via an amazon assumed role to Authentication using AWS AssumeRole and GetSessionToken

comment:3 follow-up: Changed 3 years ago by limitusus

  • Keywords sts added

I also would like this fixed. STS provides safer access privileges.

Does it take time to fix?

comment:4 in reply to: ↑ 3 Changed 3 years ago by dkocher

Replying to limitusus:

I also would like this fixed. STS provides safer access privileges.

AWS Security Token Service (STS)

comment:5 Changed 3 years ago by dkocher

Can you post the transcript from the log drawer (⌘-L) for the authentication failure that we get when trying to authenticate with the AccessKeyId and SecretAccessKey only with the token missing.

comment:6 Changed 3 years ago by dkocher

  • Milestone set to 4.8
  • Status changed from new to assigned

comment:7 Changed 3 years ago by limitusus

Sure. I set ACCESS_KEY_ID as user name and SECRET_ACCESS_KEY as password, without setting SESSION_TOKEN, and got the below transcript log. Please check it out.

GET / HTTP/1.1
Date: Thu, 10 Sep 2015 15:34:51 GMT
x-amz-request-payer: requester
Authorization: AWS ASIAJMHJ3CDT6TZFE5VQ:n6tV+L8WNQJPzE8U9bLJupZwEgg=
Connection: Keep-Alive
User-Agent: Cyberduck/ (Mac OS X/10.10.5) (x86_64)
HTTP/1.1 403 Forbidden
x-amz-request-id: 18855270603EBB20
x-amz-id-2: MLFktIVbM4nJRyzpHZhEdMTBODjIdcFGbPfzJXIPbavXlyzFfYDbW75Hx7fNk5wFMC/RU3PXHig=
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Thu, 10 Sep 2015 15:34:51 GMT
Server: AmazonS3

comment:8 Changed 2 years ago by e-man-calle

I also need this!

comment:9 Changed 2 years ago by dkocher

  • Milestone changed from 5.0 to 5.1

comment:10 Changed 2 years ago by dkocher

  • Summary changed from Authentication using AWS AssumeRole and GetSessionToken to Authentication using AWS AssumeRole and GetSessionToken with AWS STS

comment:11 Changed 2 years ago by dkocher

If you are running Cyberduck within EC2, please refer to Connecting with temporary access credentials from EC2.

comment:12 Changed 2 years ago by dkocher

Can you clarify the use case and if we should use the AWS STS API to obtain the session token from

comment:13 Changed 2 years ago by dkocher


Returns a set of temporary security credentials (consisting of an access key ID, a secret access key, and a security token) that you can use to access AWS resources that you might not normally have access to. Typically, you use AssumeRole for cross-account access or federation.

comment:14 Changed 2 years ago by dkocher

Do you require multi-factor authentication (MFA) information for AssumeRole?

comment:15 Changed 23 months ago by dkocher

  • Milestone changed from 5.1 to 5.2

comment:16 Changed 22 months ago by dkocher

  • Milestone changed from 5.2 to 5.1.3

Milestone renamed

comment:17 Changed 22 months ago by dkocher

  • Milestone changed from 5.1.3 to 6.0

Ticket retargeted after milestone closed

comment:18 Changed 21 months ago by dkocher

  • Milestone 6.0 deleted

comment:19 Changed 13 months ago by dkocher

#9992 closed as duplicate.

comment:20 Changed 12 months ago by dkocher

#10013 closed as duplicate.

comment:21 Changed 12 months ago by dkocher

We possibly just need to add support to allow the input of a SessionToken in the authentication prompt to set the X-Amz-Security-Token header.

comment:22 Changed 8 months ago by dkocher

  • Milestone set to 6.3.1

Related ticket #10009.

comment:23 Changed 8 months ago by dkocher

  • Milestone 6.3.1 deleted

Ticket retargeted after milestone closed

comment:24 Changed 7 months ago by kuhnboy

Last edited 7 months ago by kuhnboy (previous) (diff)

comment:25 Changed 7 months ago by mjcsb

  • Description modified (diff)
  • Type changed from enhancement to feature

I also need this.

What would be AWESOME: instead of collecting the access_key_id and secret_key, you instead would collect the $AWS_DEFAULT_PROFILE or "--profile" as used in aws s3 CLI commands, so if we have configured awscli to use roles via lines in ~/.aws/config, this would simply work without having to double-enter the data in two locations.

Happy if you provide an option to both store inside Cyberduck AND if not stored internally, attempt to lookup the profile inside ~/.aws/* too.

But, AWS best practice for the last few years has been to use role assumption in any multi-account scenario, and they've been pushing multi-account at the enterprise level also for a few years, so I think you need to prioritize this - it seems to have been a request for years now.

comment:26 Changed 4 months ago by jibi-waba

What is the status of this feature request? It has been delayed and/or removed from the schedule multiple times over the past two years. As mentioned by someone else a few months ago, please make this a priority and add support for session tokens soon.

comment:28 Changed 4 months ago by jibi-waba

If Cyberduck supported the use of aws_session_token from the credentials file, then this would definitely be the route to take. However, using only the aws_access_key_id and aws_secret_access_key from that file does not allow authentication to the service. It needs a combination of all three values. You may want to add support for the legacy variable name - aws_security_token - which shares the same value as aws_session_token (at least in our environment). The token information is generated via the STS service when authenticating via SAML-based identity provider (whether that is Okta or ADFS or Auth0 or other provider).

Here's a truncated profile in my credentials file:

aws_access_key_id = ASIA...
aws_secret_access_key = ++LQV7...
aws_session_token = FQoDY...
aws_security_token = FQoDY...
last_updated = 2018-03-26T18:14:35Z

More information:

Last edited 4 months ago by jibi-waba (previous) (diff)

comment:29 Changed 4 months ago by mjcsb

aws_session_token and aws_security_token are, I think, the wrong way to fix this. That's not how AWS recommends you configure cross-account roles in AWS CLI.

The original ticket description remains the correct approach, IMHO. The IAM access code should look up a profile in ~/.aws/config - NOT - specify secret/access keys explicitly. This profile may contain either the secret/access keys needed, or it may contain a role_arn combined with a reference to a source_profile. It is the combination of the source_profile to get the secret/access key, with the role_arn to assume that role in another account, which is needed to access the S3 bucket in the other account.

I'm pretty sure all the code necessary to make this work is open source and visible in the AWS CLI GitHub project, someone just needs to refactor it to work here. Not sure why this is taking so long...

comment:30 Changed 4 months ago by jibi-waba

Both scenarios could/should be supported. Cyberduck should not define best practice or method. If the SDK or boto support a specific credentials configuration (or cross-account access configuration) for authentication, then clients like Cyberduck should also strive to support this authentication method(s). We have no issues with other clients, such as the AWSCLI or Terraform, supporting aws_session_token, so the expectation is that Cyberduck would also support it.

As you said, this should be a rather simple change since this is standard support within the SDK. Perhaps this needs to be split into two separate requests, but they’re closely related.

comment:31 Changed 4 months ago by dkocher

  • Milestone set to 7.0

comment:32 Changed 3 months ago by mcnicr

  • Keywords mfa added

I would also suggest that allowing for MFA is/should be closely tied to this feature. S3 security is at the forefront of many highlevel breaches in the news recently and permitting temporary credentials via sts:AssumeRole coupled with sts:GetSessionToken and allowing a client to use MFA significantly improves safety of data.

We have 100's of Cyberduck users who can no longer use the product due to this missing feature. I am very much looking forward to seeing this implemented so my user community can return to using CyberDuck

comment:33 Changed 3 months ago by ekent

  • Priority changed from low to high

Are there any updates on when this will be available? I've increased the priority to high as mentioned above, hundreds of Cyberduck users are no longer able to use the product due to this missing feature. This means that potential customers will also look elsewhere. Many companies are starting to use the AssumeRole function, and so the need for this is increasing by the day. Please add the feature in soon!

My suggestion is to pull all the credentials from the .aws/credentials file, upon every connect. The contents of the file will change, and so having the ability to pull the information every time will be useful and enable users not to have to re-enter the details (for if their credentials have a timeout/expiry of 60 mins).

comment:34 Changed 3 months ago by dkocher

Thanks everyone for the input provided!

As we are not accustomed ourselves to using session tokens, we would love to get some feedback if requirements are met when we implement this with the following constraints:

  • Read AWS credential profiles file from the default location (~/.aws/credentials). We already do this – note the AWS access key being set when you add a new bookmark and set the protocol to S3.
  • If the AWS Access Key set in the bookmark matches, read the session token.
  • We use the default profile "default" unless a custom profile name is set in the connection profile using the key Context. Refer to Profiles.
  • When Contextis set to an URL, the session token is retrieved from the EC2 instance metadata service (already supported).

comment:36 Changed 2 months ago by jibi-waba

@dkocher, I will be more than willing to test our use case when the above constraints are accounted for. For us, the big one is the "read the session token" portion. I believe what you have stated in the above criteria will address our scenario, which is passing the session token for authentication when a named profile is referenced. Thanks for your work and research on this ticket.

comment:37 Changed 2 months ago by mcnicr

A typical use case we have is switching roles between accounts that require MFA for the assume role to succeed. A sample of the type of config file most users are using is adding the mfa_serial to the config default profile and then referencing this in other profiles. This setup is using a single sign-on account '00000000000' for user management for passwords/access keys and MFA. Then the users will assume role into a different account to access S3.

When accessing S3 the UI should allow the user to input the MFA token to retrieve an sts:SessionToken which will carry the MFA characteristics along to be used to get sts:AssumeRole credentials.

User Credentials -> Session Credentials with MFA -> Assume Role into accounts with S3 data.


[profile assumerole]
[profile assumerole2]

comment:38 Changed 2 months ago by jibi-waba

@mcnicr MFA support with session tokens might be better served as a different ticket/feature request that is dependent on this one for session token support. Just my opinion.

comment:39 Changed 2 months ago by dkocher

  • Description modified (diff)

comment:40 Changed 2 months ago by dt1001

I already have a script that gets me this far in ~/.aws/credentials:

output = json
region = us-west-1
aws_access_key_id = AAAAAAAAAAAAAAAAAAAA

I want to configure Cyberduck with just profile = "publish_profile" and have it use those three values. It would also be nice to catch any expiration error so you could remind the user that their session has expired and they need to run through their external SSO tool again to refresh the aws_session_token.

comment:41 Changed 8 weeks ago by dkocher

Related issue #10222 asking for generic support of MFA.

Note: See TracTickets for help on using tickets.
swiss made software