Context Navigation

← Previous Ticket
Next Ticket →

Opened on Mar 25, 2015 at 5:23:16 PM

Closed on May 13, 2015 at 1:36:09 PM

#8697 closed defect (duplicate)

Error: S3 Server "Failed to Respond" Following 26 GB Upload

Reported by:	dag02005	Owned by:	dkocher
Priority:	normal	Milestone:	4.7
Component:	s3	Version:	4.6.5
Severity:	major	Keywords:
Cc:		Architecture:
Platform:

Description (last modified by dkocher)

Hello,

Big fans of Cyberduck! For work, it's very important that we be able to upload files >10GB to our S3 server. Cyberduck worked great for this on our old system, but on a new MacPro with Yosemite, we keep getting a perplexing error message.

Every time the upload is almost finished (99% or more) it gives us an error message that server "Failed to respond". This happens every time. Oddly, if I hit refresh the entire file does appear to be on the server, but I'm not sure if it's complete and functional. I've included the transcript below...

GET /?delimiter=%2F&max-keys=1000&prefix=31831%2F HTTP/1.1
Date: Wed, 25 Mar 2015 00:53:48 GMT
Authorization: AWS 0ERCJ8PGP9FCHANHFD82:mXvqNTbiOLAgzj7xCsMXfwKcZcE=
Host: mubi-encoder-source.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.6.5 (Mac OS X/10.10.1) (x86_64)
GET /?delimiter=%2F&max-keys=1000&prefix=31831%2F HTTP/1.1
Date: Wed, 25 Mar 2015 00:53:48 GMT
Authorization: AWS 0ERCJ8PGP9FCHANHFD82:mXvqNTbiOLAgzj7xCsMXfwKcZcE=
Host: mubi-encoder-source.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.6.5 (Mac OS X/10.10.1) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2: oUi1p7FQm/yeSipnS1tIhzQsUhsjlLcMfT5OnT1WVCCXC8zK0KA0n+sxXfVVFO+taqGcm/WTkLk=
x-amz-request-id: 401AC64CD027DD33
Date: Wed, 25 Mar 2015 00:53:50 GMT
Content-Type: application/xml
Transfer-Encoding: chunked
Server: AmazonS3
GET /?delimiter=%2F&max-keys=1000&prefix=31831%2F HTTP/1.1
Date: Wed, 25 Mar 2015 17:14:29 GMT
Authorization: AWS 0ERCJ8PGP9FCHANHFD82:v1RjsA2+YoOSFNf+3af+ad9gc3o=
Host: mubi-encoder-source.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.6.5 (Mac OS X/10.10.1) (x86_64)
GET /?delimiter=%2F&max-keys=1000&prefix=31831%2F HTTP/1.1
Date: Wed, 25 Mar 2015 17:14:29 GMT
Authorization: AWS 0ERCJ8PGP9FCHANHFD82:v1RjsA2+YoOSFNf+3af+ad9gc3o=
Host: mubi-encoder-source.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.6.5 (Mac OS X/10.10.1) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2: PBPuVh0OKLkbpKpT/fcFeb3094Dh5HqiI4sqv3UOIlgl/LPI1qpq4lvAKSpsQjh6k2TBe6JBIWI=
x-amz-request-id: 9325FEAADABE0892
Date: Wed, 25 Mar 2015 17:14:31 GMT
Content-Type: application/xml
Transfer-Encoding: chunked
Server: AmazonS3

Thank you. Keep up the great work, and let us know if there's any info you need from us!

Change History (15)

comment:1 Changed on Mar 26, 2015 at 7:57:39 PM by dkocher

Component changed from core to s3
Description modified (diff)
Milestone set to 4.7
Owner set to dkocher

comment:2 follow-up: ↓ 3 Changed on Mar 26, 2015 at 7:58:59 PM by dkocher

Do you get the error message in the Transfer or in the Browser window?

comment:3 in reply to: ↑ 2 Changed on Mar 26, 2015 at 8:20:36 PM by dag02005

Replying to dkocher:

Do you get the error message in the Transfer or in the Browser window?

The Transfer window.

Thanks!

comment:4 Changed on Mar 31, 2015 at 2:45:05 PM by dkocher

Resolution set to worksforme
Status changed from new to closed

Please try r17204 or later.

comment:5 Changed on Apr 7, 2015 at 8:04:42 AM by dkocher

Resolution worksforme deleted
Status changed from closed to reopened

Reopening in response to #8731.

comment:6 Changed on Apr 7, 2015 at 8:10:17 AM by dkocher

We reverted changeset r17204 with r17224 defaulting to use defaulting to enable connection reuse with HTTP Keep-Alive. The issue that connections were not validated prior reuse was supposed to be fixed with r17222 instead.

comment:7 Changed on Apr 7, 2015 at 8:12:57 AM by dkocher

Set the hidden setting http.connections.reuse to false to disable the reuse of HTTP connections with Keep-Alive headers and let us know if this makes a difference. This would also apply if you use a build prior to r17224.

comment:8 in reply to: ↑ description Changed on Apr 7, 2015 at 1:59:51 PM by dkocher

Replying to dag02005:

Every time the upload is almost finished (99% or more) it gives us an error message that server "Failed to respond". This happens every time. Oddly, if I hit refresh the entire file does appear to be on the server, but I'm not sure if it's complete and functional.

If the file does appear in the file listing the multipart upload was successful and checksums verified.

comment:9 Changed on Apr 7, 2015 at 7:28:13 PM by dag02005

Thanks for the fast reply! I quit out of Cyberduck, and in the Terminal, I entered...

defaults write ch.sudo.cyberduck http.connections.reuse false

After relaunching Cyberduck, I tried the upload again and the error still happened.

Last edited on Apr 7, 2015 at 8:07:32 PM by dkocher (previous) (diff)

comment:10 Changed on Apr 8, 2015 at 7:36:08 PM by dkocher

Please post the transcript from the log drawer of the Transfers window. Choose ⌘-L on Mac or right-click the toolbar from the Transfers window and choose Log on Windows

comment:11 Changed on Apr 9, 2015 at 7:13:04 PM by dag02005

Sure thing. I tried to upload it again to reproduce the problem, and it gave a more complex, drawn-out version of the bug. Here's the transcript of the end...

Date: Thu, 09 Apr 2015 19:05:30 GMT
Expect: 100-continue
Content-Type: application/octet-stream
Authorization: AWS 0ERCJ8PGP9FCHANHFD82:+qVMfVguAofNl3CODyZT8HVb5S4=
Content-Length: 10485760
Host: mubi-encoder-source.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64)
PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1610 HTTP/1.1
Date: Thu, 09 Apr 2015 19:05:30 GMT
Expect: 100-continue
Content-Type: application/octet-stream
Authorization: AWS 0ERCJ8PGP9FCHANHFD82:iwlFb14KcZKf7C6q3SjPapZbX5k=
Content-Length: 10485760
Host: mubi-encoder-source.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64)
PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1611 HTTP/1.1
Date: Thu, 09 Apr 2015 19:05:31 GMT
Expect: 100-continue
Content-Type: application/octet-stream
Authorization: AWS 0ERCJ8PGP9FCHANHFD82:Jw4qYr/SrAMd3kspZPmv+hP/Biw=
Content-Length: 10485760
Host: mubi-encoder-source.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64)
PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1613 HTTP/1.1
Date: Thu, 09 Apr 2015 19:05:31 GMT
Expect: 100-continue
Content-Type: application/octet-stream
Authorization: AWS 0ERCJ8PGP9FCHANHFD82:ULBrj0jzSGvNXZL+r0z/JAAEwh4=
Content-Length: 10485760
Host: mubi-encoder-source.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64)
PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1614 HTTP/1.1
Date: Thu, 09 Apr 2015 19:05:31 GMT
Expect: 100-continue
Content-Type: application/octet-stream
Authorization: AWS 0ERCJ8PGP9FCHANHFD82:pjkXB46nzb+u2KXw00I/KAZZ9G4=
Content-Length: 10485760
Host: mubi-encoder-source.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64)
PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1612 HTTP/1.1
Date: Thu, 09 Apr 2015 19:05:31 GMT
Expect: 100-continue
Content-Type: application/octet-stream
Authorization: AWS 0ERCJ8PGP9FCHANHFD82:qL2Hz71cvP8fddJvh4PFox8cWNo=
Content-Length: 10485760
Host: mubi-encoder-source.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2: BO3Mtwq40EaRc2V6n16m8A22fHTLJKrYlvLlLuzBM1+kOHjN9gyG8smhBKG1VMq3tL65r0hEiRI=
x-amz-request-id: 026DF13E9B97DB92
Date: Thu, 09 Apr 2015 19:05:33 GMT
ETag: "1f0600ec3570148574781806e0d23fb7"
Content-Length: 0
Server: AmazonS3
PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1615 HTTP/1.1
Date: Thu, 09 Apr 2015 19:05:31 GMT
Expect: 100-continue
Content-Type: application/octet-stream
Authorization: AWS 0ERCJ8PGP9FCHANHFD82:waViQEDCJHIc9KgoVCnC9+3paew=
Content-Length: 10485760
Host: mubi-encoder-source.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2: 1rv0US1iT0soBN5gY0caKWhIEelIpT2BOioXt+uFWgW09k6wY7CkAIyt3P6bhrvYVAn0h+lbJdY=
x-amz-request-id: A906A17AC629536F
Date: Thu, 09 Apr 2015 19:05:27 GMT
ETag: "621dc2b1bc89e6b2b10f5c440b578669"
Content-Length: 0
Server: AmazonS3
PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1616 HTTP/1.1
Date: Thu, 09 Apr 2015 19:05:31 GMT
Expect: 100-continue
Content-Type: application/octet-stream
Authorization: AWS 0ERCJ8PGP9FCHANHFD82:Pq+6yfg2UlZxJsFXx1Qu7PsGfkY=
Content-Length: 10485760
Host: mubi-encoder-source.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64)
PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1613 HTTP/1.1
Date: Thu, 09 Apr 2015 19:05:31 GMT
Expect: 100-continue
Content-Type: application/octet-stream
Authorization: AWS 0ERCJ8PGP9FCHANHFD82:ULBrj0jzSGvNXZL+r0z/JAAEwh4=
Content-Length: 10485760
Host: mubi-encoder-source.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2: pY2ju8xfLG/r2Tt84swvI0FM+Zwm5FIkSpbmfvb6nBxRx4zr9ZiNUSFMpavM23WbgSvlP0FQ9BE=
x-amz-request-id: 1CDF868276781344
Date: Thu, 09 Apr 2015 19:05:33 GMT
ETag: "2f3fe830138656e3fa4a92f8ea1fc99d"
Content-Length: 0
Server: AmazonS3
PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1617 HTTP/1.1
Date: Thu, 09 Apr 2015 19:05:31 GMT
Expect: 100-continue
Content-Type: application/octet-stream
Authorization: AWS 0ERCJ8PGP9FCHANHFD82:7rO3sqezSbwwj22E2/fD1vZwmMg=
Content-Length: 10485760
Host: mubi-encoder-source.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2: 2+3EPSb9Ast4wWhDQrzPR5US34fLB4Fj3j4IXGK+8XRKkibW7FGCi3OM3IXUfRMNkk0vZr2z0G4=
x-amz-request-id: AB5B94E375B58B04
Date: Thu, 09 Apr 2015 19:05:50 GMT
ETag: "3ff605522c04a0b71c94f4454b3786c7"
Content-Length: 0
Server: AmazonS3
PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1618 HTTP/1.1
Date: Thu, 09 Apr 2015 19:05:31 GMT
Expect: 100-continue
Content-Type: application/octet-stream
Authorization: AWS 0ERCJ8PGP9FCHANHFD82:cJlAFR2zgG+1thFV4PNSdk0d7Zc=
Content-Length: 10485760
Host: mubi-encoder-source.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2: O7BWpQ7UaSOVeHYwfcf0Mi4Obr2kR4aRIBRmkIE+1YEya/sQ6bw+t+irQZdhAIXihA+l+CHbryA=
x-amz-request-id: E86EFC1925AD87A1
Date: Thu, 09 Apr 2015 19:05:48 GMT
ETag: "6edfc7bb6f85b5a5355d5871bb5d5792"
Content-Length: 0
Server: AmazonS3
PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1619 HTTP/1.1
Date: Thu, 09 Apr 2015 19:05:31 GMT
Expect: 100-continue
Content-Type: application/octet-stream
Authorization: AWS 0ERCJ8PGP9FCHANHFD82:38vXO8dKiRfYWyeJkdAeWghtnqM=
Content-Length: 10485760
Host: mubi-encoder-source.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64)
PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1620 HTTP/1.1
Date: Thu, 09 Apr 2015 19:05:32 GMT
Expect: 100-continue
Content-Type: application/octet-stream
Authorization: AWS 0ERCJ8PGP9FCHANHFD82:Pj68mjunbhHGQgm0XUihK1FASlk=
Content-Length: 10485760
Host: mubi-encoder-source.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64)
PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1614 HTTP/1.1
Date: Thu, 09 Apr 2015 19:05:32 GMT
Expect: 100-continue
Content-Type: application/octet-stream
Authorization: AWS 0ERCJ8PGP9FCHANHFD82:ms3ShLDTdaiJDzNUBZeYBTuEJaY=
Content-Length: 10485760
Host: mubi-encoder-source.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64)
PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1615 HTTP/1.1
Date: Thu, 09 Apr 2015 19:05:32 GMT
Expect: 100-continue
Content-Type: application/octet-stream
Authorization: AWS 0ERCJ8PGP9FCHANHFD82:6VaZvZODjILgIukOAOtR10r+//k=
Content-Length: 10485760
Host: mubi-encoder-source.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64)
PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1621 HTTP/1.1
Date: Thu, 09 Apr 2015 19:05:32 GMT
Expect: 100-continue
Content-Type: application/octet-stream
Authorization: AWS 0ERCJ8PGP9FCHANHFD82:yreJF3FXGUuhmMpmoyNqYMRVmX4=
Content-Length: 10485760
Host: mubi-encoder-source.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64)
PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1616 HTTP/1.1
Date: Thu, 09 Apr 2015 19:05:32 GMT
Expect: 100-continue
Content-Type: application/octet-stream
Authorization: AWS 0ERCJ8PGP9FCHANHFD82:S7Y9PF0VjH0Qm8Zm2v+/i362Huk=
Content-Length: 10485760
Host: mubi-encoder-source.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2: TGJVWKWKUZYINtxdlx0mwthClQj8Hr4Rn9e9OwhVJPxNJfCBqZxCl6FCqi73qhypHstbuRNSH3U=
x-amz-request-id: D56FEE768AE769B0
Date: Thu, 09 Apr 2015 19:05:53 GMT
ETag: "5ea3cc27c5d35bed0cbed8ad2f5093c1"
Content-Length: 0
Server: AmazonS3
GET /?max-keys=1000&prefix=31339%2F&delimiter=%2F HTTP/1.1
Date: Thu, 09 Apr 2015 19:05:32 GMT
Authorization: AWS 0ERCJ8PGP9FCHANHFD82:IARiG3GwAB+OHzGnxVvH77tRLHo=
Host: mubi-encoder-source.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2: 0mSXU4Cp72ef6kAkEmMljbvLK4O3GDVeZ4dHXftoC/Lv9nGVXRpqQW2Qz+eQZIKnTfqqIZdm8Kk=
x-amz-request-id: 722EA6BAD3991EA2
Date: Thu, 09 Apr 2015 19:06:09 GMT
Content-Type: application/xml
Transfer-Encoding: chunked
Server: AmazonS3

Last edited on Apr 9, 2015 at 9:09:45 PM by dkocher (previous) (diff)

comment:12 Changed on Apr 12, 2015 at 7:10:08 PM by dkocher

Looks like the Connection: Keep-Alive header is still sent. Can you try again to disable with

defaults write ~/Library/Preferences/ch.sudo.cyberduck.plist http.connections.reuse false

and also update to the latest snaphot build.

comment:13 Changed on Apr 15, 2015 at 12:51:03 PM by dkocher

Resolution set to worksforme
Status changed from reopened to closed

comment:14 Changed on May 13, 2015 at 1:35:51 PM by dkocher

Resolution worksforme deleted
Status changed from closed to reopened

comment:15 Changed on May 13, 2015 at 1:36:09 PM by dkocher

Resolution set to duplicate
Status changed from reopened to closed

Duplicate for #8634.

Note: See TracTickets for help on using tickets.

Download in other formats: