Cyberduck Mountain Duck CLI

#8975 closed defect (worksforme)

400 Bad Request error for idle connection

Reported by: rb808 Owned by: dkocher
Priority: normal Milestone: 4.7.3
Component: s3 Version: 4.7.2
Severity: normal Keywords:
Cc: Architecture: Intel
Platform: Windows 10

Description

Hi

I'm trying up upload a directory of photos to S3, I'm using CyberDuck for the first time to do this. It starts well, but after a few hours it has a problem, then starts again from the very first file. This means the upload never completes, eg a estimated 4 hour upload has been going for a few days.

I have a copy of the debug log file. The log drawer just shows successfully uploaded files.

I'm running Windows 10, but had same problem last week with Windows 8.

Thanks

Richard

Attachments (1)

cyberduck.log (513.0 KB) - added by rb808 on Aug 16, 2015 at 3:23:53 AM.

Download all attachments as: .zip

Change History (9)

Changed on Aug 16, 2015 at 3:23:53 AM by rb808

comment:1 Changed on Aug 16, 2015 at 3:24:19 AM by rb808

I had to change the URLs to avoid spam filter

comment:2 Changed on Aug 21, 2015 at 9:43:41 AM by dkocher

  • Component changed from core to s3
  • Milestone set to 4.8
  • Owner set to dkocher

comment:3 Changed on Aug 21, 2015 at 1:13:33 PM by dkocher

  • Summary changed from S3 upload fails, starts from beginning to 400 Bad Request error for idle connection
PUT /iPhone/2014/IMG_0041.JPG HTTP/1.1
Date: Sun, 16 Aug 2015 02:07:48 GMT
Expect: 100-continue
Content-Type: image/jpeg
Authorization: AWS IVECHANGEDTHIS:u1IVqOz0mrA4Hm0Qonf7LSBKfqU=
Content-Length: 2295546
Host: mys3backup.s3.amazonaws.com:443
Connection: Keep-Alive
User-Agent: Cyberduck/4.7.2.18004 (Windows 8/6.2) (x86)
HTTP/1.1 100 Continue
HTTP/1.1 400 Bad Request
x-amz-request-id: 3D597225908074BA
x-amz-id-2: wr8D5gUMVpja0b8ffgMWhoVYAs80kHL8g1zg0RgRXJsJqHvTBCGnXm/sHmMxUXGzE868FoJJdtI=
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Sun, 16 Aug 2015 02:07:32 GMT
Connection: close
Server: AmazonS3

<?xml version="1.0" encoding="UTF-8"?><Error><Code>RequestTimeout</Code><Message>Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed.</Message><RequestId>3D597225908074BA</RequestId><HostId>wr8D5gUMVpja0b8ffgMWhoVYAs80kHL8g1zg0RgRXJsJqHvTBCGnXm/sHmMxUXGzE868FoJJdtI=</HostId></Error>

comment:4 Changed on Sep 3, 2015 at 1:27:26 PM by dkocher

From https://forums.aws.amazon.com/message.jspa?messageID=116727

Since you are reusing connections, you should also take note of the guidance provided in the Amazon S3 best practices article regarding the maximum number of requests Amazon S3 will allow over a single connection (you should close the connection after 80-90 requests).

comment:5 Changed on Sep 3, 2015 at 1:42:14 PM by dkocher

From https://aws.amazon.com/items/1904?externalID=1904

Amazon S3 will accept up to 100 requests before it closes a connection (resulting in 'connection reset'). Rather than having this happen, use a connection for 80-90 requests before closing and re-opening a new connection.

comment:6 Changed on Sep 3, 2015 at 2:23:20 PM by dkocher

S3 responds with a Connection: Close response header after re-using the same connection 100 times. Therefore we handle this gracefully and this should not be an issue.

comment:7 Changed on Sep 17, 2015 at 8:11:55 AM by dkocher

Possible fix in r18144 with fix HTTPCLIENT-1655 included.

Sends RST instead of proper FIN ACK sequence when using non-persistant connections

comment:8 Changed on Sep 17, 2015 at 8:15:45 AM by dkocher

  • Resolution set to worksforme
  • Status changed from new to closed

General performance fix in r18136 which could also affect this. Please update to the latest snapshot build available.

Note: See TracTickets for help on using tickets.
swiss made software