#8697 closed defect (duplicate)
Error: S3 Server "Failed to Respond" Following 26 GB Upload
Reported by: | dag02005 | Owned by: | dkocher |
---|---|---|---|
Priority: | normal | Milestone: | 4.7 |
Component: | s3 | Version: | 4.6.5 |
Severity: | major | Keywords: | |
Cc: | Architecture: | ||
Platform: |
Description (last modified by dkocher)
Hello,
Big fans of Cyberduck! For work, it's very important that we be able to upload files >10GB to our S3 server. Cyberduck worked great for this on our old system, but on a new MacPro with Yosemite, we keep getting a perplexing error message.
Every time the upload is almost finished (99% or more) it gives us an error message that server "Failed to respond". This happens every time. Oddly, if I hit refresh the entire file does appear to be on the server, but I'm not sure if it's complete and functional. I've included the transcript below...
GET /?delimiter=%2F&max-keys=1000&prefix=31831%2F HTTP/1.1 Date: Wed, 25 Mar 2015 00:53:48 GMT Authorization: AWS 0ERCJ8PGP9FCHANHFD82:mXvqNTbiOLAgzj7xCsMXfwKcZcE= Host: mubi-encoder-source.s3.amazonaws.com:443 Connection: Keep-Alive User-Agent: Cyberduck/4.6.5 (Mac OS X/10.10.1) (x86_64) GET /?delimiter=%2F&max-keys=1000&prefix=31831%2F HTTP/1.1 Date: Wed, 25 Mar 2015 00:53:48 GMT Authorization: AWS 0ERCJ8PGP9FCHANHFD82:mXvqNTbiOLAgzj7xCsMXfwKcZcE= Host: mubi-encoder-source.s3.amazonaws.com:443 Connection: Keep-Alive User-Agent: Cyberduck/4.6.5 (Mac OS X/10.10.1) (x86_64) HTTP/1.1 200 OK x-amz-id-2: oUi1p7FQm/yeSipnS1tIhzQsUhsjlLcMfT5OnT1WVCCXC8zK0KA0n+sxXfVVFO+taqGcm/WTkLk= x-amz-request-id: 401AC64CD027DD33 Date: Wed, 25 Mar 2015 00:53:50 GMT Content-Type: application/xml Transfer-Encoding: chunked Server: AmazonS3 GET /?delimiter=%2F&max-keys=1000&prefix=31831%2F HTTP/1.1 Date: Wed, 25 Mar 2015 17:14:29 GMT Authorization: AWS 0ERCJ8PGP9FCHANHFD82:v1RjsA2+YoOSFNf+3af+ad9gc3o= Host: mubi-encoder-source.s3.amazonaws.com:443 Connection: Keep-Alive User-Agent: Cyberduck/4.6.5 (Mac OS X/10.10.1) (x86_64) GET /?delimiter=%2F&max-keys=1000&prefix=31831%2F HTTP/1.1 Date: Wed, 25 Mar 2015 17:14:29 GMT Authorization: AWS 0ERCJ8PGP9FCHANHFD82:v1RjsA2+YoOSFNf+3af+ad9gc3o= Host: mubi-encoder-source.s3.amazonaws.com:443 Connection: Keep-Alive User-Agent: Cyberduck/4.6.5 (Mac OS X/10.10.1) (x86_64) HTTP/1.1 200 OK x-amz-id-2: PBPuVh0OKLkbpKpT/fcFeb3094Dh5HqiI4sqv3UOIlgl/LPI1qpq4lvAKSpsQjh6k2TBe6JBIWI= x-amz-request-id: 9325FEAADABE0892 Date: Wed, 25 Mar 2015 17:14:31 GMT Content-Type: application/xml Transfer-Encoding: chunked Server: AmazonS3
Thank you. Keep up the great work, and let us know if there's any info you need from us!
Change History (15)
comment:1 Changed on Mar 26, 2015 at 7:57:39 PM by dkocher
- Component changed from core to s3
- Description modified (diff)
- Milestone set to 4.7
- Owner set to dkocher
comment:2 follow-up: ↓ 3 Changed on Mar 26, 2015 at 7:58:59 PM by dkocher
comment:3 in reply to: ↑ 2 Changed on Mar 26, 2015 at 8:20:36 PM by dag02005
Replying to dkocher:
Do you get the error message in the Transfer or in the Browser window?
The Transfer window.
Thanks!
comment:4 Changed on Mar 31, 2015 at 2:45:05 PM by dkocher
- Resolution set to worksforme
- Status changed from new to closed
Please try r17204 or later.
comment:5 Changed on Apr 7, 2015 at 8:04:42 AM by dkocher
- Resolution worksforme deleted
- Status changed from closed to reopened
Reopening in response to #8731.
comment:6 Changed on Apr 7, 2015 at 8:10:17 AM by dkocher
comment:7 Changed on Apr 7, 2015 at 8:12:57 AM by dkocher
Set the hidden setting http.connections.reuse to false to disable the reuse of HTTP connections with Keep-Alive headers and let us know if this makes a difference. This would also apply if you use a build prior to r17224.
comment:8 in reply to: ↑ description Changed on Apr 7, 2015 at 1:59:51 PM by dkocher
Replying to dag02005:
Every time the upload is almost finished (99% or more) it gives us an error message that server "Failed to respond". This happens every time. Oddly, if I hit refresh the entire file does appear to be on the server, but I'm not sure if it's complete and functional.
If the file does appear in the file listing the multipart upload was successful and checksums verified.
comment:9 Changed on Apr 7, 2015 at 7:28:13 PM by dag02005
Thanks for the fast reply! I quit out of Cyberduck, and in the Terminal, I entered...
defaults write ch.sudo.cyberduck http.connections.reuse false
After relaunching Cyberduck, I tried the upload again and the error still happened.
comment:10 Changed on Apr 8, 2015 at 7:36:08 PM by dkocher
Please post the transcript from the log drawer of the Transfers window. Choose ⌘-L on Mac or right-click the toolbar from the Transfers window and choose Log on Windows
comment:11 Changed on Apr 9, 2015 at 7:13:04 PM by dag02005
Sure thing. I tried to upload it again to reproduce the problem, and it gave a more complex, drawn-out version of the bug. Here's the transcript of the end...
Date: Thu, 09 Apr 2015 19:05:30 GMT Expect: 100-continue Content-Type: application/octet-stream Authorization: AWS 0ERCJ8PGP9FCHANHFD82:+qVMfVguAofNl3CODyZT8HVb5S4= Content-Length: 10485760 Host: mubi-encoder-source.s3.amazonaws.com:443 Connection: Keep-Alive User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64) PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1610 HTTP/1.1 Date: Thu, 09 Apr 2015 19:05:30 GMT Expect: 100-continue Content-Type: application/octet-stream Authorization: AWS 0ERCJ8PGP9FCHANHFD82:iwlFb14KcZKf7C6q3SjPapZbX5k= Content-Length: 10485760 Host: mubi-encoder-source.s3.amazonaws.com:443 Connection: Keep-Alive User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64) PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1611 HTTP/1.1 Date: Thu, 09 Apr 2015 19:05:31 GMT Expect: 100-continue Content-Type: application/octet-stream Authorization: AWS 0ERCJ8PGP9FCHANHFD82:Jw4qYr/SrAMd3kspZPmv+hP/Biw= Content-Length: 10485760 Host: mubi-encoder-source.s3.amazonaws.com:443 Connection: Keep-Alive User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64) PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1613 HTTP/1.1 Date: Thu, 09 Apr 2015 19:05:31 GMT Expect: 100-continue Content-Type: application/octet-stream Authorization: AWS 0ERCJ8PGP9FCHANHFD82:ULBrj0jzSGvNXZL+r0z/JAAEwh4= Content-Length: 10485760 Host: mubi-encoder-source.s3.amazonaws.com:443 Connection: Keep-Alive User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64) PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1614 HTTP/1.1 Date: Thu, 09 Apr 2015 19:05:31 GMT Expect: 100-continue Content-Type: application/octet-stream Authorization: AWS 0ERCJ8PGP9FCHANHFD82:pjkXB46nzb+u2KXw00I/KAZZ9G4= Content-Length: 10485760 Host: mubi-encoder-source.s3.amazonaws.com:443 Connection: Keep-Alive User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64) PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1612 HTTP/1.1 Date: Thu, 09 Apr 2015 19:05:31 GMT Expect: 100-continue Content-Type: application/octet-stream Authorization: AWS 0ERCJ8PGP9FCHANHFD82:qL2Hz71cvP8fddJvh4PFox8cWNo= Content-Length: 10485760 Host: mubi-encoder-source.s3.amazonaws.com:443 Connection: Keep-Alive User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64) HTTP/1.1 200 OK x-amz-id-2: BO3Mtwq40EaRc2V6n16m8A22fHTLJKrYlvLlLuzBM1+kOHjN9gyG8smhBKG1VMq3tL65r0hEiRI= x-amz-request-id: 026DF13E9B97DB92 Date: Thu, 09 Apr 2015 19:05:33 GMT ETag: "1f0600ec3570148574781806e0d23fb7" Content-Length: 0 Server: AmazonS3 PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1615 HTTP/1.1 Date: Thu, 09 Apr 2015 19:05:31 GMT Expect: 100-continue Content-Type: application/octet-stream Authorization: AWS 0ERCJ8PGP9FCHANHFD82:waViQEDCJHIc9KgoVCnC9+3paew= Content-Length: 10485760 Host: mubi-encoder-source.s3.amazonaws.com:443 Connection: Keep-Alive User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64) HTTP/1.1 200 OK x-amz-id-2: 1rv0US1iT0soBN5gY0caKWhIEelIpT2BOioXt+uFWgW09k6wY7CkAIyt3P6bhrvYVAn0h+lbJdY= x-amz-request-id: A906A17AC629536F Date: Thu, 09 Apr 2015 19:05:27 GMT ETag: "621dc2b1bc89e6b2b10f5c440b578669" Content-Length: 0 Server: AmazonS3 PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1616 HTTP/1.1 Date: Thu, 09 Apr 2015 19:05:31 GMT Expect: 100-continue Content-Type: application/octet-stream Authorization: AWS 0ERCJ8PGP9FCHANHFD82:Pq+6yfg2UlZxJsFXx1Qu7PsGfkY= Content-Length: 10485760 Host: mubi-encoder-source.s3.amazonaws.com:443 Connection: Keep-Alive User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64) PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1613 HTTP/1.1 Date: Thu, 09 Apr 2015 19:05:31 GMT Expect: 100-continue Content-Type: application/octet-stream Authorization: AWS 0ERCJ8PGP9FCHANHFD82:ULBrj0jzSGvNXZL+r0z/JAAEwh4= Content-Length: 10485760 Host: mubi-encoder-source.s3.amazonaws.com:443 Connection: Keep-Alive User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64) HTTP/1.1 200 OK x-amz-id-2: pY2ju8xfLG/r2Tt84swvI0FM+Zwm5FIkSpbmfvb6nBxRx4zr9ZiNUSFMpavM23WbgSvlP0FQ9BE= x-amz-request-id: 1CDF868276781344 Date: Thu, 09 Apr 2015 19:05:33 GMT ETag: "2f3fe830138656e3fa4a92f8ea1fc99d" Content-Length: 0 Server: AmazonS3 PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1617 HTTP/1.1 Date: Thu, 09 Apr 2015 19:05:31 GMT Expect: 100-continue Content-Type: application/octet-stream Authorization: AWS 0ERCJ8PGP9FCHANHFD82:7rO3sqezSbwwj22E2/fD1vZwmMg= Content-Length: 10485760 Host: mubi-encoder-source.s3.amazonaws.com:443 Connection: Keep-Alive User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64) HTTP/1.1 200 OK x-amz-id-2: 2+3EPSb9Ast4wWhDQrzPR5US34fLB4Fj3j4IXGK+8XRKkibW7FGCi3OM3IXUfRMNkk0vZr2z0G4= x-amz-request-id: AB5B94E375B58B04 Date: Thu, 09 Apr 2015 19:05:50 GMT ETag: "3ff605522c04a0b71c94f4454b3786c7" Content-Length: 0 Server: AmazonS3 PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1618 HTTP/1.1 Date: Thu, 09 Apr 2015 19:05:31 GMT Expect: 100-continue Content-Type: application/octet-stream Authorization: AWS 0ERCJ8PGP9FCHANHFD82:cJlAFR2zgG+1thFV4PNSdk0d7Zc= Content-Length: 10485760 Host: mubi-encoder-source.s3.amazonaws.com:443 Connection: Keep-Alive User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64) HTTP/1.1 200 OK x-amz-id-2: O7BWpQ7UaSOVeHYwfcf0Mi4Obr2kR4aRIBRmkIE+1YEya/sQ6bw+t+irQZdhAIXihA+l+CHbryA= x-amz-request-id: E86EFC1925AD87A1 Date: Thu, 09 Apr 2015 19:05:48 GMT ETag: "6edfc7bb6f85b5a5355d5871bb5d5792" Content-Length: 0 Server: AmazonS3 PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1619 HTTP/1.1 Date: Thu, 09 Apr 2015 19:05:31 GMT Expect: 100-continue Content-Type: application/octet-stream Authorization: AWS 0ERCJ8PGP9FCHANHFD82:38vXO8dKiRfYWyeJkdAeWghtnqM= Content-Length: 10485760 Host: mubi-encoder-source.s3.amazonaws.com:443 Connection: Keep-Alive User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64) PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1620 HTTP/1.1 Date: Thu, 09 Apr 2015 19:05:32 GMT Expect: 100-continue Content-Type: application/octet-stream Authorization: AWS 0ERCJ8PGP9FCHANHFD82:Pj68mjunbhHGQgm0XUihK1FASlk= Content-Length: 10485760 Host: mubi-encoder-source.s3.amazonaws.com:443 Connection: Keep-Alive User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64) PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1614 HTTP/1.1 Date: Thu, 09 Apr 2015 19:05:32 GMT Expect: 100-continue Content-Type: application/octet-stream Authorization: AWS 0ERCJ8PGP9FCHANHFD82:ms3ShLDTdaiJDzNUBZeYBTuEJaY= Content-Length: 10485760 Host: mubi-encoder-source.s3.amazonaws.com:443 Connection: Keep-Alive User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64) PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1615 HTTP/1.1 Date: Thu, 09 Apr 2015 19:05:32 GMT Expect: 100-continue Content-Type: application/octet-stream Authorization: AWS 0ERCJ8PGP9FCHANHFD82:6VaZvZODjILgIukOAOtR10r+//k= Content-Length: 10485760 Host: mubi-encoder-source.s3.amazonaws.com:443 Connection: Keep-Alive User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64) PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1621 HTTP/1.1 Date: Thu, 09 Apr 2015 19:05:32 GMT Expect: 100-continue Content-Type: application/octet-stream Authorization: AWS 0ERCJ8PGP9FCHANHFD82:yreJF3FXGUuhmMpmoyNqYMRVmX4= Content-Length: 10485760 Host: mubi-encoder-source.s3.amazonaws.com:443 Connection: Keep-Alive User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64) PUT /31339%2Fim-still-here_en_xx_1920x1080_30723.mp4?uploadId=lg_Lht5_WOHUIWMjRkU.JKwjyI_SUBqN8Zf2TW3O4uACPRaqd5qsyHbzZkL4JEdH5xTWAhcKYTmF_sLzvfKfnw--&partNumber=1616 HTTP/1.1 Date: Thu, 09 Apr 2015 19:05:32 GMT Expect: 100-continue Content-Type: application/octet-stream Authorization: AWS 0ERCJ8PGP9FCHANHFD82:S7Y9PF0VjH0Qm8Zm2v+/i362Huk= Content-Length: 10485760 Host: mubi-encoder-source.s3.amazonaws.com:443 Connection: Keep-Alive User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64) HTTP/1.1 200 OK x-amz-id-2: TGJVWKWKUZYINtxdlx0mwthClQj8Hr4Rn9e9OwhVJPxNJfCBqZxCl6FCqi73qhypHstbuRNSH3U= x-amz-request-id: D56FEE768AE769B0 Date: Thu, 09 Apr 2015 19:05:53 GMT ETag: "5ea3cc27c5d35bed0cbed8ad2f5093c1" Content-Length: 0 Server: AmazonS3 GET /?max-keys=1000&prefix=31339%2F&delimiter=%2F HTTP/1.1 Date: Thu, 09 Apr 2015 19:05:32 GMT Authorization: AWS 0ERCJ8PGP9FCHANHFD82:IARiG3GwAB+OHzGnxVvH77tRLHo= Host: mubi-encoder-source.s3.amazonaws.com:443 Connection: Keep-Alive User-Agent: Cyberduck/4.7 (Mac OS X/10.10.1) (x86_64) HTTP/1.1 200 OK x-amz-id-2: 0mSXU4Cp72ef6kAkEmMljbvLK4O3GDVeZ4dHXftoC/Lv9nGVXRpqQW2Qz+eQZIKnTfqqIZdm8Kk= x-amz-request-id: 722EA6BAD3991EA2 Date: Thu, 09 Apr 2015 19:06:09 GMT Content-Type: application/xml Transfer-Encoding: chunked Server: AmazonS3
comment:12 Changed on Apr 12, 2015 at 7:10:08 PM by dkocher
Looks like the Connection: Keep-Alive header is still sent. Can you try again to disable with
defaults write ~/Library/Preferences/ch.sudo.cyberduck.plist http.connections.reuse false
and also update to the latest snaphot build.
comment:13 Changed on Apr 15, 2015 at 12:51:03 PM by dkocher
- Resolution set to worksforme
- Status changed from reopened to closed
comment:14 Changed on May 13, 2015 at 1:35:51 PM by dkocher
- Resolution worksforme deleted
- Status changed from closed to reopened
comment:15 Changed on May 13, 2015 at 1:36:09 PM by dkocher
- Resolution set to duplicate
- Status changed from reopened to closed
Duplicate for #8634.
Do you get the error message in the Transfer or in the Browser window?