Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

S3 connection with credentials file on Windows #11719

Closed
cyberduck opened this issue Jul 1, 2021 · 3 comments
Closed

S3 connection with credentials file on Windows #11719

cyberduck opened this issue Jul 1, 2021 · 3 comments
Labels
bug fixed s3 AWS S3 Protocol Implementation

Comments

@cyberduck
Copy link
Collaborator

9956eda created the issue

Hello there

I am trying to connect from Windows to AWS S3 using temporary credentials that are in %USERPROFILE%.aws\credentials. These credentials work with the AWS CLI and another third party tool but with CyberDuck I get the following error:

The AWS Access Key Id you provided does not exist in our records

I am using the following Cyberduck profile https://trac.cyberduck.io/wiki/help/en/howto/s3#Connectingusingcredentialsin.awscredentials and for Profile Name in ~/.aws/credentials I specify the profile that I configured (test_s3_profile). My .aws/credentials file looks like this:

[profile test_s3_profile]
aws_access_key_id=EXAMPLEKEYID
aws_secret_access_key=EXAMPLESECRETKEY
aws_session_token=EXAMPLETOKEN

The errors I see in the logs are:

2021-07-01 08:50:10,668 [background-7] DEBUG ch.cyberduck.core.sts.STSCredentialsConfigurator - Look for profile name test_s3_profile in Local{path='C:\Users\ImageBuilderAdmin\.aws\credentials'}
2021-07-01 08:50:10,676 [background-7] WARN  ch.cyberduck.core.sts.STSCredentialsConfigurator - Failure reading Local{path='C:\Users\ImageBuilderAdmin\.aws\credentials'}
java.lang.IllegalArgumentException: Invalid property format: no '=' character is found on line 1
	at com.amazonaws.auth.profile.internal.AbstractProfilesConfigFileScanner.parsePropertyLine(AbstractProfilesConfigFileScanner.java:160)
	at com.amazonaws.auth.profile.internal.AbstractProfilesConfigFileScanner.run(AbstractProfilesConfigFileScanner.java:119)
	at ch.cyberduck.core.sts.STSCredentialsConfigurator$ProfilesConfigFileLoaderHelper.parseProfileProperties(STSCredentialsConfigurator.java:302)
	at ch.cyberduck.core.sts.STSCredentialsConfigurator.configure(STSCredentialsConfigurator.java:91)
	at ch.cyberduck.core.s3.S3Session.login(S3Session.java:175)
	at ch.cyberduck.core.KeychainLoginService.authenticate(KeychainLoginService.java:175)
	at ch.cyberduck.core.LoginConnectionService.authenticate(LoginConnectionService.java:180)
	at ch.cyberduck.core.LoginConnectionService.connect(LoginConnectionService.java:171)
	at ch.cyberduck.core.LoginConnectionService.check(LoginConnectionService.java:110)
	at ch.cyberduck.core.pool.StatelessSessionPool.borrow(StatelessSessionPool.java:59)
	at ch.cyberduck.core.threading.SessionBackgroundAction.run(SessionBackgroundAction.java:118)
	at ch.cyberduck.core.threading.SessionBackgroundAction$1.call(SessionBackgroundAction.java:103)
	at ch.cyberduck.core.threading.DefaultRetryCallable.call(DefaultRetryCallable.java:50)
	at ch.cyberduck.core.threading.SessionBackgroundAction.call(SessionBackgroundAction.java:105)
	at ch.cyberduck.core.threading.BackgroundCallable.run(BackgroundCallable.java:94)
	at ch.cyberduck.core.threading.BackgroundCallable.call(BackgroundCallable.java:58)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:618)
	at ch.cyberduck.core.threading.NamedThreadFactory$1.run(NamedThreadFactory.java:59)
	at java.lang.Thread.run(Thread.java:955)
2021-07-01 08:50:10,678 [background-7] DEBUG ch.cyberduck.core.shared.WorkdirHomeFeature - No workdir set for bookmark Host{protocol=Profile{parent=s3, vendor=s3-cli, description=S3 (Credentials from AWS Command Line Interface), image=null}, port=443, hostname='s3.amazonaws.com', credentials=Credentials{user='test_s3_profile', oauth='Tokens{accessToken='null', refreshToken='null'}', token='', identity=null}, uuid='b6754e17-a0df-48f2-8921-55eda692de6c', nickname='null', defaultpath='null', workdir=null, labels=null}

I also see the following in the logs which suggests that the profile name is being sent as the access_key_id.

2021-07-01 08:50:11,276 [background-7] DEBUG ch.cyberduck.core.threading.DefaultFailureDiagnostics - Determine cause for failure BackgroundException{class=class ch.cyberduck.core.exception.LoginFailureException, file=Path{path='/', type=[directory, volume]}, message='Listing directory / failed.', detail='The AWS Access Key Id you provided does not exist in our records.', cause='org.jets3t.service.S3ServiceException: Service Error Message. -- ResponseCode: 403, ResponseStatus: Forbidden, XML Error Message: <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidAccessKeyId</Code><Message>The AWS Access Key Id you provided does not exist in our records.</Message><AWSAccessKeyId>test_s3_profile</AWSAccessKeyId><RequestId>0HGDVF9PG4H3Z1EY</RequestId><HostId>lHoHceCje4e4GMBFFRY3gsVmxhiacEydfMk41eQFe1gO0uPGdE+NHKUvC3cdQs2c+YZXRvD6D1U=</HostId></Error>'}

I also tried to rename the profile to default in the credentials file and not specify the profile in Cyberduck but the logs then show this (note the blank profile name):

2021-07-01 09:15:11,419 [background-10] DEBUG ch.cyberduck.core.sts.STSCredentialsConfigurator - Look for profile name  in Local{path='C:\Users\ImageBuilderAdmin\.aws\credentials'}
2021-07-01 09:15:11,420 [background-10] WARN  ch.cyberduck.core.sts.STSCredentialsConfigurator - Failure reading Local{path='C:\Users\ImageBuilderAdmin\.aws\credentials'}
java.lang.IllegalArgumentException: Invalid property format: no '=' character is found on line 1
	at com.amazonaws.auth.profile.internal.AbstractProfilesConfigFileScanner.parsePropertyLine(AbstractProfilesConfigFileScanner.java:160)

...

2021-07-01 09:15:12,654 [background-10] WARN  ch.cyberduck.core.threading.BackgroundCallable - Failure BackgroundException{class=class ch.cyberduck.core.exception.InteroperabilityException, file=Path{path='/', type=[directory, volume]}, message='Listing directory / failed.', detail='The authorization header is malformed; a non-empty Access Key (AKID) must be provided in the credential.', cause='org.apache.http.client.HttpResponseException: status code: 400, reason phrase: The authorization header is malformed; a non-empty Access Key (AKID) must be provided in the credential.'} running background task
java.lang.Exception
	at ch.cyberduck.core.threading.BackgroundCallable.<init>(BackgroundCallable.java:36)
	at ch.cyberduck.core.threading.DefaultBackgroundExecutor.execute(DefaultBackgroundExecutor.java:67)
	at ch.cyberduck.core.AbstractController.background(AbstractController.java:71)
	at ch.cyberduck.core.threading.BackgroundCallable$1.run(BackgroundCallable.java:74)
	at cli.System.Delegate.DynamicInvokeImpl(Unknown Source)
	at cli.System.Windows.Forms.Control.InvokeMarshaledCallbackDo(Unknown Source)
	at cli.System.Windows.Forms.Control.InvokeMarshaledCallbackHelper(Unknown Source)
	at cli.System.Threading.ExecutionContext.RunInternal(Unknown Source)
	at cli.System.Threading.ExecutionContext.Run(Unknown Source)
	at cli.System.Threading.ExecutionContext.Run(Unknown Source)
	at cli.System.Windows.Forms.Control.InvokeMarshaledCallback(Unknown Source)
	at cli.System.Windows.Forms.Control.InvokeMarshaledCallbacks(Unknown Source)
	at cli.System.Windows.Forms.Control.WndProc(Unknown Source)
	at cli.System.Windows.Forms.Form.WndProc(Unknown Source)
	at cli.System.Windows.Forms.NativeWindow.Callback(Unknown Source)
	at cli.System.Windows.Forms.UnsafeNativeMethods.DispatchMessageW(Unknown Source)
	at cli.System.Windows.Forms.UnsafeNativeMethods.DispatchMessageW(Unknown Source)
	at cli.System.Windows.Forms.Application$ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop(Unknown Source)
	at cli.System.Windows.Forms.Application$ThreadContext.RunMessageLoopInner(Unknown Source)
	at cli.System.Windows.Forms.Application$ThreadContext.RunMessageLoop(Unknown Source)
	at cli.Ch.Cyberduck.Ui.Program.Main(Unknown Source)

Am I doing something incorrectly or is there a bug in how the credentials file is being parsed or the credentials sent to S3?

Many thanks,

Thanh
@cyberduck
Copy link
Collaborator Author

@dkocher commented

Documentation for Configuration and credential file settings

@cyberduck
Copy link
Collaborator Author

72efc17 commented

This error occurs when there is an invalid profile entry line in the .aws/credentials. Forexample in the first line of the below credentials, the square brackets [ ] are missing for the profile entry.

-INVALID*

profile test_s3_profile
aws_access_key_id=EXAMPLEKEYID
aws_secret_access_key=EXAMPLESECRETKEY
aws_session_token=EXAMPLETOKEN

-VALID*

[profile test_s3_profile]
aws_access_key_id=EXAMPLEKEYID
aws_secret_access_key=EXAMPLESECRETKEY
aws_session_token=EXAMPLETOKEN

There is a possibility to validate the credentials from (https://awscli.amazonaws.com/v2/documentation/api/latest/reference/appconfig/validate-configuration.html).

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Test in 02da443.

@iterate-ch iterate-ch locked as resolved and limited conversation to collaborators Nov 27, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
bug fixed s3 AWS S3 Protocol Implementation
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@cyberduck