Cyberduck Mountain Duck CLI

#9322 closed defect (fixed)

S3 ACLs can't be changed in third-party buckets (due to incorrect Owner specification?)

Reported by: bretmartin Owned by: dkocher
Priority: normal Milestone: 4.8.4
Component: s3 Version: Nightly Build
Severity: normal Keywords:
Cc: Architecture:
Platform:

Description

Hello/Grüezi,

Thank you for your work on Cyberduck. We have found it useful at my workplace as an S3 transfer client for external collaborators.

When working with a bucket that we own, providing access to a third party using an IAM user in their account, we've found that the third party IAM user is unable to change ACLs on objects in our bucket, yielding this error:

Cannot change permissions of Creating an AWS IAM user to share data with H3 Biomedicine via Amazon S3.pdf. Access Denied. Please contact your web hosting service provider for assistance.

even though their IAM policy and our bucket policy both permit the ACL change. With the same third party IAM credentials, these ACL changes are possible using the AWS CLI.

By turning on Cyberduck debug logging, I found that the ACL change request included the canonical ID of the third party account in the <Owner> element of the access control policy. However, the owner of the object is our account, not the third party account. I believe this is the reason for the "Access Denied" error from S3 and the difference in behavior from the AWS CLI.

I found this behavior to be the same under 4.6.5, 4.8.2, and 5.0 (19065).

Please let me know if I can provide any additional information or facilitate testing (for example, if you need a third party S3 bucket to test with).

Thanks,

--Bret

Change History (16)

comment:1 Changed on Mar 1, 2016 at 10:24:39 AM by dkocher

  • Component changed from core to s3
  • Milestone set to 5.0
  • Owner set to dkocher
  • Status changed from new to assigned

comment:2 follow-up: Changed on Mar 1, 2016 at 10:36:52 AM by dkocher

If you can setup a test bucket with the same ACL configuration that would be great. Let me know if I should provide an IAM user you can reference.

comment:3 Changed on Mar 1, 2016 at 2:47:48 PM by bretmartin

Absolutely -- I'll provide a bucket and steps to reproduce. Please do provide an IAM user and I'll let you know when the bucket and policy are in place.

comment:4 in reply to: ↑ 2 Changed on Mar 1, 2016 at 3:14:06 PM by dkocher

Replying to dkocher:

If you can setup a test bucket with the same ACL configuration that would be great. Let me know if I should provide an IAM user you can reference.

The IAM user is AKIAJWP2C4QGJZ4JD6KA with ARN

arn:aws:iam::189584543480:user/TRAC-9322

comment:5 Changed on Mar 1, 2016 at 4:48:34 PM by bretmartin

I have created bucket bretmartin-cyberduck-trac-9322 with the following bucket policy:

{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Sid": "https://trac.cyberduck.io/ticket/9322",
			"Effect": "Allow",
			"Principal": {
				"AWS": [
					"arn:aws:iam::189584543480:user/TRAC-9322",
					"arn:aws:iam::597082535337:user/bam"
				]
			},
			"Action": "s3:*",
			"Resource": [
				"arn:aws:s3:::bretmartin-cyberduck-trac-9322",
				"arn:aws:s3:::bretmartin-cyberduck-trac-9322/*"
			]
		}
	]
}

...:user/bam is also a third party IAM user from outside the account that owns this bucket. Using that user in Cyberduck 5.0 (19065), I did the following:

  • connected to S3 specifying path /bretmartin-cyberduck-trac-9322 (since it is outside the account of the connecting IAM user)
  • uploaded test.txt successfully
    • this object had a single ACL entry granting FULL_CONTROL to the third party account (not the bucket owner) -- this is expected
  • Command-I > Permissions on object test.txt, try to add FULL_CONTROL ACL entry by Amazon Customer Email Address, supply bam@miranda.org (email address for the bucket owner account)
    • yields error: Cannot change permissions of test.txt. Access Denied. Please contact your web hosting service provider for assistance. [ Cancel ] [ Try Again ]

I will run this test again in a moment with debug logging on and supply relevant excerpts.

comment:6 Changed on Mar 1, 2016 at 5:05:01 PM by bretmartin

Here is the original ACL on test.txt as retrieved using aws s3api get-object-acl .... Note in particular that the Owner is the third party account with ID starting e455d7...:

{
    "Owner": {
        "DisplayName": "aws",
        "ID": "e455d7150e7518eef8ed181a177463ef25e98fa91ba7b18ecdcb4181626ee607"
    },
    "Grants": [
        {
            "Grantee": {
                "DisplayName": "aws",
                "ID": "e455d7150e7518eef8ed181a177463ef25e98fa91ba7b18ecdcb4181626ee607"
            },
            "Permission": "FULL_CONTROL"
        }
    ]
}

Here is the XML of the ACL Cyberduck tries to set per my test above (extracted from Cyberduck debug log entries):

<?xml version="1.0"?>
<AccessControlPolicy xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Owner>
    <ID>ea495239f890028c88eb505804ee843a3c4b6eaa0f7033702baa21ddcebe7e00</ID>
    <DisplayName>bam</DisplayName>
  </Owner>
  <AccessControlList>
    <Grant>
      <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser">
        <ID>ea495239f890028c88eb505804ee843a3c4b6eaa0f7033702baa21ddcebe7e00</ID>
      </Grantee>
      <Permission>FULL_CONTROL</Permission>
    </Grant>
    <Grant>
      <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser">
        <ID>e455d7150e7518eef8ed181a177463ef25e98fa91ba7b18ecdcb4181626ee607</ID>
      </Grantee>
      <Permission>FULL_CONTROL</Permission>
    </Grant>
    <Grant>
      <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="AmazonCustomerByEmail">
        <EmailAddress>bam@miranda.org</EmailAddress>
      </Grantee>
      <Permission>FULL_CONTROL</Permission>
    </Grant>
  </AccessControlList>
</AccessControlPolicy>

Note that the Owner has changed to the bucket owner account with ID starting ea4952.... Unless I am mistaken, it is not possible to change the ownership of an S3 object (normally the recommended solution to change actual ownership is for the new owner to copy the object, and then delete the original one). I believe this is causing the error.

comment:7 Changed on Mar 1, 2016 at 5:07:43 PM by bretmartin

Last update for now. I realized I incorrectly stated the problem in my original description:

By turning on Cyberduck debug logging, I found that the ACL change request included the canonical ID of the third party account in the <Owner> element of the access control policy. However, the owner of the object is our account, not the third party account. I believe this is the reason for the "Access Denied" error from S3 and the difference in behavior from the AWS CLI.

I had things inverted in this paragraph. Sorry for being confusing. It should have read

By turning on Cyberduck debug logging, I found that the ACL change request included the canonical ID of the bucket owner account in the <Owner> element of the access control policy. However, the owner of the object is the third party account, not the bucket owner account. I believe this is the reason for the "Access Denied" error from S3 and the difference in behavior from the AWS CLI.

comment:8 follow-ups: Changed on Mar 4, 2016 at 8:05:28 PM by dkocher

When querying the ACL for the bucket bretmartin-cyberduck-trac-9322.s3.amazonaws.com with GET /?acl HTTP/1.1 I get

<?xml version="1.0"?>
<AccessControlPolicy xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <Owner>
        <ID>ea495239f890028c88eb505804ee843a3c4b6eaa0f7033702baa21ddcebe7e00</ID>
        <DisplayName>bam</DisplayName>
    </Owner>
    <AccessControlList>
        <Grant>
            <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser">
                <ID>ea495239f890028c88eb505804ee843a3c4b6eaa0f7033702baa21ddcebe7e00</ID>
            </Grantee>
            <Permission>FULL_CONTROL</Permission>
        </Grant>
    </AccessControlList>
</AccessControlPolicy>

When trying to read the existing ACL on the file test.txt in the bucket we get a 403 permission failure. Therefore when updating the ACL where the Owner element must always be set the best thing to do is to use the known owner from the bucket.

comment:9 in reply to: ↑ 8 ; follow-up: Changed on Mar 4, 2016 at 8:11:38 PM by dkocher

Replying to dkocher:

Regardless of this we always defer the owner for the ACL we apply from the bucket and not from any existing owner of the ACL on the file. This is possibly the bug you describe.

comment:10 Changed on Mar 7, 2016 at 9:02:47 AM by dkocher

  • Resolution set to fixed
  • Status changed from assigned to closed

In r19522.

comment:11 in reply to: ↑ 8 Changed on Mar 7, 2016 at 4:30:06 PM by bretmartin

Replying to dkocher:

When trying to read the existing ACL on the file test.txt in the bucket we get a 403 permission failure.

This is expected -- I left the ACL on that object as set after upload by the third party IAM user. Because of the bug, that user couldn't change it afterward. Sorry for the confusion; I wasn't intending that you'd be able to change that object's ACL.

Therefore when updating the ACL where the Owner element must always be set the best thing to do is to use the known owner from the bucket.

I agree that is a reasonable default. However, my proposal for you to reproduce my problem would have been for you to upload another object yourself, say test2.txt, and try to change its ACL. Since you would own it, you'd be able to retrieve its ACL as well.

comment:12 in reply to: ↑ 9 ; follow-up: Changed on Mar 7, 2016 at 4:30:49 PM by bretmartin

Replying to dkocher:

Regardless of this we always defer the owner for the ACL we apply from the bucket and not from any existing owner of the ACL on the file. This is possibly the bug you describe.

Yes, this sounds like it.

Thanks for the change. I will test as soon as it appears in a snapshot build.

comment:13 in reply to: ↑ 12 ; follow-up: Changed on Mar 10, 2016 at 4:04:20 PM by dkocher

Replying to bretmartin:

Replying to dkocher:

Regardless of this we always defer the owner for the ACL we apply from the bucket and not from any existing owner of the ACL on the file. This is possibly the bug you describe.

Yes, this sounds like it.

Thanks for the change. I will test as soon as it appears in a snapshot build.

The latest snapshot builds include this changeset. Can you confirm this is resolved?

comment:14 in reply to: ↑ 13 Changed on Mar 10, 2016 at 4:33:54 PM by bretmartin

Replying to dkocher:

The latest snapshot builds include this changeset. Can you confirm this is resolved?

I'm still seeing Cyberduck trying to change the owner to the bucket owner (when it should remain the third party account) when trying to modify the ACL as the third party IAM user -- looks the same as my test above.

Are the build numbers related to changeset numbers? I note that I have build 19182 but this was fixed in r19522. Does that mean I have a build not including that changeset?

comment:15 Changed on Mar 10, 2016 at 9:01:32 PM by dkocher

  • Milestone changed from 5.0 to 4.8.4
  • Resolution fixed deleted
  • Status changed from closed to reopened

comment:16 Changed on Mar 14, 2016 at 1:32:55 PM by dkocher

  • Resolution set to fixed
  • Status changed from reopened to closed

Additional fix in r19656.

Note: See TracTickets for help on using tickets.
swiss made software