Cyberduck Mountain Duck CLI

#7642 closed defect (thirdparty)

Interoperability with Gallery2

Reported by: wwwpixime Owned by: dkocher
Priority: lowest Milestone: 4.4.4
Component: webdav Version: Nightly Build
Severity: minor Keywords:
Cc: Architecture:
Platform:

Description (last modified by wwwpixime)

hi David,

When uploading files to https://g2.pixi.me/w/webdav/ - Cyberduck only render 0-byte files. Finder's built-in WebDAV client and Forklift 2 does the same thing.

I asked a number of the developers who's WebDAV-apps we tested about this issue and was told they use a fallback method when a server fails to respond to an "Expect: 100-continue" request and when chunked uploads are not supported by the server. The fallback method ensures files are properly uploaded, not ending up 0-byte filenames.

If you can shed some light on this, it'd be much appreciated. Even a fix is not an option for Cyberduck, at least I can lay this issue to rest. And will make a donation for pushing to support better, more secure TLS protocols.

Thanks very much!

Change History (23)

comment:1 Changed on Nov 27, 2013 at 5:13:02 AM by wwwpixime

  • Description modified (diff)
  • Summary changed from fallback method when a server fails to respond to an "Expect: 100-continue" request and when chunked uploads are not supported by the server. to fallback to "Expect: 100-continue" and Chunked Uploads

comment:2 Changed on Nov 27, 2013 at 5:39:23 AM by wwwpixime

  • Description modified (diff)

comment:3 Changed on Nov 27, 2013 at 6:11:21 AM by wwwpixime

maybe an option under Preferences > Transfers to enable/disable (Expect: 100-continue and Chunked Uploads) .

I' ve seen these options when I was testing SmartFTP & WebDrive for Windows.

comment:4 Changed on Nov 27, 2013 at 4:31:01 PM by dkocher

Interoperability for 401 replies to HEAD in r14114.

comment:5 Changed on Nov 27, 2013 at 4:31:22 PM by dkocher

Fix to actually retry failed request in r14115.

comment:6 Changed on Nov 27, 2013 at 6:17:03 PM by dkocher

Failing test case in r14128.

comment:7 Changed on Nov 27, 2013 at 6:17:54 PM by dkocher

  • Summary changed from fallback to "Expect: 100-continue" and Chunked Uploads to Interoperability with Gallery2

comment:8 Changed on Nov 28, 2013 at 9:35:54 AM by dkocher

The request body is not sent because the server replies with status 200 to the request with Expect: 100-Continue header.

comment:9 Changed on Nov 28, 2013 at 9:39:00 AM by dkocher

  • Milestone set to 4.4.3
  • Resolution set to fixed
  • Status changed from new to closed

Fix in r14135.

comment:10 follow-up: Changed on Nov 29, 2013 at 1:20:13 AM by wwwpixime

hi David,

Thank you for adding a workaround but it's now ending up with duplicate 0-byte files, which can result in 500 Internal Server errors when deleted from within the client itself - I'd have to use a web browser to bypass the errors.

I think the ideal solution lies within "headers.remove(HTTP.EXPECT_DIRECTIVE);" that should be controlled in the UI (Preferences > Transfers > General > Uploads, or under Open Connection > More Options) so the user has full control to disable/enable it whenever needed, on demand.

When enabled (default) - Cyberduck sends PUT requests with HTTP.EXPECT_DIRECTIVE When disabled - Cyberduck sends PUT request (without HTTP.EXPECT_DIRECTIVE eliminating 0 byte files altogether.

I am seriously desperate to pay for this option because it will benefit so many (of my) users and resolve this particular PUT request issue.

Please advise, and thank you for really taking the time to look into this.

--- here's some excerpt from logs files:

framework: Apache/mod_dav (responds to HTTP.EXPECT_DIRECTIVE with 100-Continue)

 -  dav.pixi.me 2525 [12:23:52 -0500] webdav "PROPFIND / HTTP/1.1" 207 652 1306 "-" "Cyberduck/4.4.3 "
 -  dav.pixi.me 2532 [12:24:02 -0500] webdav "HEAD / HTTP/1.1" 403 523 2808 "-" "Cyberduck/4.4.3 "
 -  dav.pixi.me 2533 [12:24:02 -0500] webdav "PROPFIND / HTTP/1.1" 207 652 1306 "-" "Cyberduck/4.4.3 "
 -  dav.pixi.me 2533 [12:24:02 -0500] webdav "HEAD /images/ HTTP/1.1" 404 224 0 "-" "Cyberduck/4.4.3 "
 -  dav.pixi.me 2534 [12:24:02 -0500] webdav "HEAD /BZBaN3aCIAAF2N8.jpg_large.jpg HTTP/1.1" 404 509 133 "-" "Cyberduck/4.4.3 "
 -  dav.pixi.me 2535 [12:24:03 -0500] webdav "HEAD /BZSmwh9CIAAtK7h.jpg_large.jpg HTTP/1.1" 404 509 133 "-" "Cyberduck/4.4.3 "
 -  dav.pixi.me 2537 [12:24:03 -0500] webdav "HEAD /BZzu0fyIgAAKpfv.jpg_large.jpg HTTP/1.1" 404 509 133 "-" "Cyberduck/4.4.3 "
 -  dav.pixi.me 2536 [12:24:03 -0500] webdav "HEAD /webdav-logo.jpg HTTP/1.1" 404 495 133 "-" "Cyberduck/4.4.3 "
 -  dav.pixi.me 2538 [12:24:03 -0500] webdav "MKCOL /images/ HTTP/1.1" 201 507 577 "-" "Cyberduck/4.4.3 "
 -  dav.pixi.me 2538 [12:24:03 -0500] webdav "PUT /images/photojojo.jpg HTTP/1.1" 201 3575 507 "-" "Cyberduck/4.4.3 "
 -  dav.pixi.me 2538 [12:24:03 -0500] webdav "PUT /images/smartbackup.settings HTTP/1.1" 201 989 514 "-" "Cyberduck/4.4.3 "
 -  dav.pixi.me 2538 [12:24:03 -0500] webdav "PUT /images/webdav-logo.jpg HTTP/1.1" 201 4334 509 "-" "Cyberduck/4.4.3 "
 -  dav.pixi.me 2538 [12:24:03 -0500] webdav "PUT /BZBaN3aCIAAF2N8.jpg_large.jpg HTTP/1.1" 201 117977 516 "-" "Cyberduck/4.4.3 "
 -  dav.pixi.me 2538 [12:24:03 -0500] webdav "PUT /BZSmwh9CIAAtK7h.jpg_large.jpg HTTP/1.1" 201 20228 516 "-" "Cyberduck/4.4.3 "
 -  dav.pixi.me 2538 [12:24:03 -0500] webdav "PUT /BZzu0fyIgAAKpfv.jpg_large.jpg HTTP/1.1" 201 69186 516 "-" "Cyberduck/4.4.3 "
 -  dav.pixi.me 2538 [12:24:03 -0500] webdav "PUT /webdav-logo.jpg HTTP/1.1" 201 4327 502 "-" "Cyberduck/4.4.3 "
 -  dav.pixi.me 2539 [12:24:03 -0500] webdav "PROPFIND / HTTP/1.1" 207 652 5298 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2544 [12:24:36 -0500] webdav "HEAD /w/webdav/ HTTP/1.1" 200 531 2808 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2545 [12:24:37 -0500] webdav "PROPFIND /w/webdav/ HTTP/1.1" 207 734 2344 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2546 [12:24:44 -0500] webdav "HEAD /w/webdav/ HTTP/1.1" 200 531 2808 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2547 [12:24:44 -0500] webdav "HEAD /w/webdav/images/ HTTP/1.1" 401 569 133 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2548 [12:24:44 -0500] - "HEAD /w/webdav/BZBaN3aCIAAF2N8.jpg_large.jpg HTTP/1.1" 401 548 133 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2549 [12:24:45 -0500] webdav "HEAD /w/webdav/BZBaN3aCIAAF2N8.jpg_large.jpg HTTP/1.1" 401 591 133 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2550 [12:24:45 -0500] - "HEAD /w/webdav/BZSmwh9CIAAtK7h.jpg_large.jpg HTTP/1.1" 401 548 133 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2551 [12:24:45 -0500] webdav "HEAD /w/webdav/BZSmwh9CIAAtK7h.jpg_large.jpg HTTP/1.1" 401 591 133 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2552 [12:24:45 -0500] - "HEAD /w/webdav/BZzu0fyIgAAKpfv.jpg_large.jpg HTTP/1.1" 401 548 133 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2553 [12:24:45 -0500] webdav "HEAD /w/webdav/BZzu0fyIgAAKpfv.jpg_large.jpg HTTP/1.1" 401 591 133 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2554 [12:24:46 -0500] - "HEAD /w/webdav/webdav-logo.jpg HTTP/1.1" 401 534 133 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2555 [12:24:46 -0500] webdav "HEAD /w/webdav/webdav-logo.jpg HTTP/1.1" 401 577 133 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2556 [12:24:46 -0500] - "HEAD /w/webdav/images/ HTTP/1.1" 401 526 133 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2557 [12:24:46 -0500] webdav "HEAD /w/webdav/images/ HTTP/1.1" 401 569 133 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2558 [12:24:46 -0500] - "HEAD /w/webdav/BZBaN3aCIAAF2N8.jpg_large.jpg HTTP/1.1" 401 548 133 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2559 [12:24:47 -0500] webdav "HEAD /w/webdav/BZBaN3aCIAAF2N8.jpg_large.jpg HTTP/1.1" 401 591 133 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2560 [12:24:47 -0500] - "HEAD /w/webdav/BZSmwh9CIAAtK7h.jpg_large.jpg HTTP/1.1" 401 548 133 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2561 [12:24:47 -0500] webdav "HEAD /w/webdav/BZSmwh9CIAAtK7h.jpg_large.jpg HTTP/1.1" 401 591 133 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2562 [12:24:47 -0500] - "HEAD /w/webdav/BZzu0fyIgAAKpfv.jpg_large.jpg HTTP/1.1" 401 548 133 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2563 [12:24:47 -0500] webdav "HEAD /w/webdav/BZzu0fyIgAAKpfv.jpg_large.jpg HTTP/1.1" 401 591 133 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2564 [12:24:48 -0500] - "HEAD /w/webdav/webdav-logo.jpg HTTP/1.1" 401 534 133 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2565 [12:24:48 -0500] webdav "HEAD /w/webdav/webdav-logo.jpg HTTP/1.1" 401 577 133 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2566 [12:24:48 -0500] - "MKCOL /w/webdav/images/ HTTP/1.1" 201 546 574 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2567 [12:24:49 -0500] - "PUT /w/webdav/images/photojojo.jpg HTTP/1.1" 200 635 517 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2568 [12:24:49 -0500] - "PUT /w/webdav/images/photojojo.jpg HTTP/1.1" 200 3830 553 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2568 [12:24:49 -0500] - "PUT /w/webdav/images/smartbackup.settings HTTP/1.1" 200 392 384 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2569 [12:24:50 -0500] - "PUT /w/webdav/images/smartbackup.settings HTTP/1.1" 200 1244 553 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2569 [12:24:50 -0500] - "PUT /w/webdav/images/webdav-logo.jpg HTTP/1.1" 200 374 384 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2570 [12:24:50 -0500] - "PUT /w/webdav/images/webdav-logo.jpg HTTP/1.1" 200 4589 553 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2570 [12:24:51 -0500] - "PUT /w/webdav/BZBaN3aCIAAF2N8.jpg_large.jpg HTTP/1.1" 200 383 384 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2571 [12:24:51 -0500] - "PUT /w/webdav/BZBaN3aCIAAF2N8.jpg_large.jpg HTTP/1.1" 200 118232 553 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2571 [12:24:52 -0500] - "PUT /w/webdav/BZSmwh9CIAAtK7h.jpg_large.jpg HTTP/1.1" 200 382 384 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2572 [12:24:52 -0500] - "PUT /w/webdav/BZSmwh9CIAAtK7h.jpg_large.jpg HTTP/1.1" 200 20483 553 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2572 [12:24:53 -0500] - "PUT /w/webdav/BZzu0fyIgAAKpfv.jpg_large.jpg HTTP/1.1" 200 382 384 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2573 [12:24:53 -0500] - "PUT /w/webdav/BZzu0fyIgAAKpfv.jpg_large.jpg HTTP/1.1" 200 69441 553 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2573 [12:24:54 -0500] - "PUT /w/webdav/webdav-logo.jpg HTTP/1.1" 200 367 384 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2574 [12:24:54 -0500] - "PUT /w/webdav/webdav-logo.jpg HTTP/1.1" 200 4582 553 "-" "Cyberduck/4.4.3 "
 -  g2.pixi.me 2575 [12:24:54 -0500] webdav "PROPFIND /w/webdav/ HTTP/1.1" 207 734 8781 "-" "Cyberduck/4.4.3 "

Last edited on Nov 29, 2013 at 8:44:23 AM by dkocher (previous) (diff)

comment:11 Changed on Nov 29, 2013 at 1:45:40 PM by dkocher

  • Milestone changed from 4.4.3 to 4.5
  • Resolution fixed deleted
  • Status changed from closed to reopened

comment:12 Changed on Jan 13, 2014 at 10:32:20 PM by dkocher

Removed workaround in r14204.

comment:13 Changed on Jan 13, 2014 at 10:34:27 PM by dkocher

You can disable the use of the Expect header with setting the hidden propertywebdav.expect-continue to false.

comment:14 follow-ups: Changed on Jan 13, 2014 at 11:59:21 PM by wwwpixime

David,

Thank you for adding a workaround, even if its not in the UI - at least upload woes are now resolved. Tested on Cyberduck 4.4.x nightly on Mac OS X 10.7.5 - haven't tested the Windows build yet because of some odd handshake failures.

As promised, my donation will be coming mid-next week on payday. Will be sending an email to my mailing list so Pixi.me can finally use Cyberduck to upload boatloads of pictures into their Gallery 2 sites. Thanks again!

comment:15 in reply to: ↑ 14 Changed on Jan 14, 2014 at 7:50:54 AM by dkocher

Replying to wwwpixime:

David,

Thank you for adding a workaround, even if its not in the UI - at least upload woes are now resolved. Tested on Cyberduck 4.4.x nightly on Mac OS X 10.7.5 - haven't tested the Windows build yet because of some odd handshake failures.

As promised, my donation will be coming mid-next week on payday. Will be sending an email to my mailing list so Pixi.me can finally use Cyberduck to upload boatloads of pictures into their Gallery 2 sites. Thanks again!

Please note the issue is only resolved when setting the preference. The previously added workaround has been removed as it leads to duplicate files in G2 leading to 500 server errors.

comment:16 in reply to: ↑ 14 Changed on Jan 14, 2014 at 7:51:52 AM by dkocher

Replying to wwwpixime:

David,

Thank you for adding a workaround, even if its not in the UI - at least upload woes are now resolved. Tested on Cyberduck 4.4.x nightly on Mac OS X 10.7.5 - haven't tested the Windows build yet because of some odd handshake failures.

I suppose these are issues with TLSv1.2 when running on Windows. See #7637.

comment:17 follow-up: Changed on Jan 17, 2014 at 2:20:05 PM by wwwpixime

looks like 100-continue is appearing where it's not suppose to - PROPFIND requests, the old behavior was only during PUT requests. But even after setting "defaults write ch.sudo.cyberduck webdav.expect-continue false" I'm now seeing this latest snapshot running on Mac OS X 10.7.5 (have "Java 7 update 51" installed)

208.185.83.154 g2.pixi.me TLSv1.2 ECDHE-RSA-AES128-SHA - 25988 [17/Jan/2014:09:13:39 -0500] webdav "HEAD /w/webdav/ HTTP/1.1" 200 598 2840 "-" "Cyberduck/4.4.4 (Mac OS X/10.7.5) (x86_64)"

208.185.83.154 g2.pixi.me TLSv1.2 ECDHE-RSA-AES128-SHA - 26011 [17/Jan/2014:09:13:39 -0500] webdav "HEAD /w/webdav/BdgmuUKCIAANeru.jpg HTTP/1.1" 401 651 161 "-" "Cyberduck/4.4.4 (Mac OS X/10.7.5) (x86_64)"

208.185.83.154 g2.pixi.me TLSv1.2 ECDHE-RSA-AES128-SHA - 26012 [17/Jan/2014:09:13:39 -0500] webdav "HEAD /w/webdav/BdgmuUKCIAANeru.jpg HTTP/1.1" 401 651 161 "-" "Cyberduck/4.4.4 (Mac OS X/10.7.5) (x86_64)"

208.185.83.154 g2.pixi.me TLSv1.2 ECDHE-RSA-AES128-SHA - 26013 [17/Jan/2014:09:13:39 -0500] - "HEAD /w/webdav/BdgmuUKCIAANeru.jpg HTTP/1.1" 401 603 161 "-" "Cyberduck/4.4.4 (Mac OS X/10.7.5) (x86_64)"

208.185.83.154 g2.pixi.me TLSv1.2 ECDHE-RSA-AES128-SHA 100-continue 26018 [17/Jan/2014:09:13:40 -0500] - "PUT /w/webdav/BdgmuUKCIAANeru.jpg HTTP/1.1" 200 736 491 "-" "Cyberduck/4.4.4 (Mac OS X/10.7.5) (x86_64)"

208.185.83.154 g2.pixi.me TLSv1.2 ECDHE-RSA-AES128-SHA 100-continue 25987 [17/Jan/2014:09:13:40 -0500] webdav "PROPFIND /w/webdav/ HTTP/1.1" 207 880 3328 "-" "Cyberduck/4.4.4 (Mac OS X/10.7.5) (x86_64)"


comment:18 Changed on Jan 17, 2014 at 2:33:01 PM by wwwpixime

yep, just installed 4.4.3 and defaults write ch.sudo.cyberduck webdav.expect-continue false works as expected.

comment:19 in reply to: ↑ 17 Changed on Jan 17, 2014 at 10:42:19 PM by dkocher

Replying to wwwpixime:

I'm now seeing this latest snapshot running on Mac OS X 10.7.5 (have "Java 7 update 51" installed)

Cyberduck requires no Java installation.

comment:20 Changed on Jan 17, 2014 at 10:42:34 PM by dkocher

Reverted default use of Expect header in r14218.

comment:21 in reply to: ↑ 10 Changed on Jan 18, 2014 at 10:01:03 AM by dkocher

Replying to wwwpixime:

hi David,

Thank you for adding a workaround but it's now ending up with duplicate 0-byte files, which can result in 500 Internal Server errors when deleted from within the client itself - I'd have to use a web browser to bypass the errors.

Is there a bug filed with Gallery2 to add support for Expect: 100-continue?

comment:22 Changed on Jan 18, 2014 at 12:21:18 PM by dkocher

  • Milestone changed from 4.5 to 4.4.4
  • Resolution set to thirdparty
  • Status changed from reopened to closed

comment:23 Changed on Jan 18, 2014 at 3:50:40 PM by wwwpixime

hi David,

Unfortunately, Gallery2 is no longer in development. It's a solid web based photo management platform (have been using it since 2003), but with a not-so-modern WebDAV implementation.

It looks like Cyberduck 4.4.3 with the manual "defaults write ch.sudo.cyberduck webdav.expect-continue false " workaround will do for my users. I'll just let them know not to upgrade to latter versions or their uploads will fail.

Thanks again for providing the manual workaround. Sorry to keep pestering about adding a UI option! I'll let this issue lay rest now because it's been too long and you probably have other features to implement far more in demand and important than this.

Best regards,

Note: See TracTickets for help on using tickets.
swiss made software