Cyberduck Mountain Duck CLI

Opened 6 months ago

Closed 6 months ago

Last modified 5 months ago

#10371 closed defect (fixed)

Incorrectly assumes ETag and MD5 are equivalent

Reported by: rpkelly Owned by: dkocher
Priority: normal Milestone: 6.7.0
Component: s3 Version: 6.6.1
Severity: normal Keywords:
Cc: Architecture:
Platform:

Description

Cyberduck assumes that the ETag returned by S3 is equivalent to the MD5 checksum of the file's content. This is not a valid assumption. The ETag will only be the MD5 of the object data when the object is stored as plaintext or encrypted using SSE-S3. If the object is encrypted using another method (such as SSE-C or SSE-KMS) the ETag is not the MD5 of the object data. If the object was created via a multipart upload, the ETag is not the MD5 of the object data.

Attachments (3)

Screen Shot 2018-06-25 at 11.25.58 AM.png​ (101.2 KB) - added by rpkelly 6 months ago.
Warning message
Screen Shot 2018-06-25 at 11.26.54 AM.png​ (26.0 KB) - added by rpkelly 6 months ago.
MD5 via command line
Screen Shot 2018-06-25 at 11.27.23 AM.png​ (16.8 KB) - added by rpkelly 6 months ago.
ETag from S3 console

Download all attachments as: .zip

Change History (10)

comment:1 Changed 6 months ago by dkocher

  • Component changed from core to s3
  • Owner set to dkocher
  • Summary changed from Cyberduck incorrectly assumes ETag and MD5 are equivalent for S3 to Incorrectly assumes ETag and MD5 are equivalent

comment:2 Changed 6 months ago by dkocher

What makes you think that we make this false assumption?

comment:3 Changed 6 months ago by rpkelly

See the attached screenshots. Cyberduck displays a warning saying there is a mismatch. I also calculate the MD5 of the file via the command line and am attaching a screenshot of the ETag shown in the S3 console.

Changed 6 months ago by rpkelly

Warning message

Changed 6 months ago by rpkelly

MD5 via command line

Changed 6 months ago by rpkelly

ETag from S3 console

comment:4 Changed 6 months ago by dkocher

  • Milestone set to 7.0
  • Status changed from new to assigned

comment:5 Changed 6 months ago by dkocher

  • Resolution set to fixed
  • Status changed from assigned to closed

In r44541. Revised fix in r44569 retaining checksum check for files not encrypted with SSE-KMS. For multipart uploads we will fail parsing the ETag as it has a suffix appended to the MD5.

comment:6 Changed 5 months ago by dkocher

  • Milestone changed from 7.0 to 6.6.3

comment:7 Changed 5 months ago by dkocher

  • Milestone changed from 6.6.3 to 6.7.0

Milestone renamed

Note: See TracTickets for help on using tickets.
swiss made software