Cyberduck Mountain Duck CLI

Opened on Jun 20, 2018 at 7:22:55 PM

Closed on Jun 26, 2018 at 9:08:16 AM

Last modified on Jul 6, 2018 at 9:31:13 AM

#10371 closed defect (fixed)

Incorrectly assumes ETag and MD5 are equivalent

Reported by: rpkelly Owned by: dkocher
Priority: normal Milestone: 6.7.0
Component: s3 Version: 6.6.1
Severity: normal Keywords:
Cc: Architecture:
Platform:

Description

Cyberduck assumes that the ETag returned by S3 is equivalent to the MD5 checksum of the file's content. This is not a valid assumption. The ETag will only be the MD5 of the object data when the object is stored as plaintext or encrypted using SSE-S3. If the object is encrypted using another method (such as SSE-C or SSE-KMS) the ETag is not the MD5 of the object data. If the object was created via a multipart upload, the ETag is not the MD5 of the object data.

Attachments (3)

Screen Shot 2018-06-25 at 11.25.58 AM.png​ (101.2 KB) - added by rpkelly on Jun 25, 2018 at 6:30:17 PM.
Warning message
Screen Shot 2018-06-25 at 11.26.54 AM.png​ (26.0 KB) - added by rpkelly on Jun 25, 2018 at 6:30:37 PM.
MD5 via command line
Screen Shot 2018-06-25 at 11.27.23 AM.png​ (16.8 KB) - added by rpkelly on Jun 25, 2018 at 6:30:52 PM.
ETag from S3 console

Download all attachments as: .zip

Change History (10)

comment:1 Changed on Jun 25, 2018 at 2:28:44 PM by dkocher

  • Component changed from core to s3
  • Owner set to dkocher
  • Summary changed from Cyberduck incorrectly assumes ETag and MD5 are equivalent for S3 to Incorrectly assumes ETag and MD5 are equivalent

comment:2 Changed on Jun 25, 2018 at 2:29:27 PM by dkocher

What makes you think that we make this false assumption?

comment:3 Changed on Jun 25, 2018 at 6:29:50 PM by rpkelly

See the attached screenshots. Cyberduck displays a warning saying there is a mismatch. I also calculate the MD5 of the file via the command line and am attaching a screenshot of the ETag shown in the S3 console.

Changed on Jun 25, 2018 at 6:30:17 PM by rpkelly

Warning message

Changed on Jun 25, 2018 at 6:30:37 PM by rpkelly

MD5 via command line

Changed on Jun 25, 2018 at 6:30:52 PM by rpkelly

ETag from S3 console

comment:4 Changed on Jun 25, 2018 at 6:48:06 PM by dkocher

  • Milestone set to 7.0
  • Status changed from new to assigned

comment:5 Changed on Jun 26, 2018 at 9:08:16 AM by dkocher

  • Resolution set to fixed
  • Status changed from assigned to closed

In r44541. Revised fix in r44569 retaining checksum check for files not encrypted with SSE-KMS. For multipart uploads we will fail parsing the ETag as it has a suffix appended to the MD5.

comment:6 Changed on Jul 5, 2018 at 7:38:04 PM by dkocher

  • Milestone changed from 7.0 to 6.6.3

comment:7 Changed on Jul 6, 2018 at 9:31:13 AM by dkocher

  • Milestone changed from 6.6.3 to 6.7.0

Milestone renamed

Note: See TracTickets for help on using tickets.