Cyberduck Mountain Duck CLI

Opened on Feb 2, 2014 at 11:31:26 PM

Closed on Feb 3, 2014 at 9:47:21 AM

Last modified on Feb 3, 2014 at 9:48:38 AM

#7772 closed defect (fixed)

Proposed upload exceeds the maximum allowed size

Reported by: vitovito Owned by: dkocher
Priority: normal Milestone: 4.4.4
Component: s3 Version: 4.4.3
Severity: normal Keywords: amazon, aws, s3, multipart, upload
Cc: Architecture:
Platform: Mac OS X 10.8

Description

Hello, I've been trying to upload a 75GB file to Amazon S3 without success.

I've tried twice with the current version. All upload parts appear to succeed, and I am able to stop and resume the connection, and also resume after a network connection failure, but on the last part, it reports this error. It never "completes" the upload, and the file is never visible in the bucket.

My transfer log is attached. It has all of the second upload attempt, and I think at least the end of the first upload attempt.

Attachments (1)

cyberduck-75gb-upload-failure.txt​ (7.0 MB) - added by vitovito on Feb 2, 2014 at 11:31:45 PM.

Change History (4)

Changed on Feb 2, 2014 at 11:31:45 PM by vitovito

comment:1 Changed on Feb 3, 2014 at 9:20:09 AM by dkocher

  • Component changed from core to s3
  • Owner set to dkocher
  • Summary changed from Amazon S3: Proposed upload exceeds the maximum allowed size to Proposed upload exceeds the maximum allowed size

comment:2 Changed on Feb 3, 2014 at 9:47:21 AM by dkocher

  • Milestone set to 4.4.4
  • Resolution set to fixed
  • Status changed from new to closed

We changed the part size for multipart uploads to 10MB in r14143 to allow multipart uploads up to 100GB in total size due to the maximum number of parts restriction of 10'000 by S3. Please try with the latest ​snapshot build available and reopen this ticket if you are still having this issue.

comment:3 Changed on Feb 3, 2014 at 9:48:38 AM by dkocher

The first failing request with part number 10001 that exceeds the maximum allowed part numbers. 

PUT /vitorio-macbookair-jan2014.dmg?uploadId=YKsMV3EBieZyS8YjFqyPMlJE7jkYQrziWbIDE6ZjEIy3Er7F7zzz251PfEfloPXjSne5PNEpzN1SeVziwdCT7A--&partNumber=10001 HTTP/1.1
Date: Sun, 02 Feb 2014 23:11:44 GMT
Content-Type: application/octet-stream
Authorization: AWS AKIAJP37KLPYNJNFL42Q:suo+wbyuQ+D5yWCFzYrlkM7MHQI=
Content-Length: 2031
Host: vitorio.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.4.3 (Mac OS X/10.8.5) (x86_64)
Expect: 100-continue
HTTP/1.1 400 Bad Request
x-amz-request-id: F62AD349F7EBE3F7
x-amz-id-2: Pnpi5g53N4wNOzTreCUKSTO4wutmZKSAntTBiWhvpqTL5OFXJ6SbIIWEDu4mKkiM
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Sun, 02 Feb 2014 23:11:43 GMT
Connection: close
Server: AmazonS3
Note: See TracTickets for help on using tickets.