Cyberduck Mountain Duck CLI

Version 11 (modified by dkocher, on Jan 24, 2018 at 1:52:53 PM) (diff)


Zero byte file truncate issue with Nextcloud and ownCloud deployed with FastCGI


Using a client to upload files with HTTP chunked transfer encoding to a server with fastcgi/php-fpm enabled can lead to zero byte files. Chunked transfer encoding is used when the content length is unknown at the time the transfer is started and no Content-Length header can be set.


There are basically two options to encode a PUT request. You either know the length of the data you want to transfer at the time of starting the request then you can set the HTTP Content-Length header appropriately or you don't know the length and thus have to choose a streaming approach using the chunked transfer encoding which does not need a Content-Length header at all. A client is free to choose either of the two request types as both are completely fine with the HTTP specification.

Most clients out there, including web browsers, use the former method and thus do not hit this issue. There are use cases though which make it necessary to transfer data chunked. For example Mountain Duck that implements a virtual file system for accessing your cloud storage online. The write callbacks we get from the OS just include an offset, a buffer length and the buffer itself. Mountain Duck does not know the final file size in advance. From a virtual filesystem perspective the callbacks Mountain Duck get for an upload are as follows.

1 - CreateFile myFile.txt
2 - SetAllocationSize myFile.txt, 1024
3 - WriteFile myFile.txt, offset 0, length 1024, buffer
4 - SetAllocationSize myFile.txt, 2048
5 - WriteFile myFile.txt, offset 1024, length 1024, buffer
6 - CloseFile myFile.txt

In step 3 Mountain Duck opens a connection to the remote server, sends a PUT request with the http header Transfer-Encoding: chunked and streams through all subsequent write callbacks. Finally in step 6 the connection is closed.

Related bug reports

Since the expected length is missing in the header the streaming characteristic of such requests makes them more difficult to be handled in the components being passed. From our research the issue only exists in environments that use FastCGI to speak to their PHP application. See related bug reports

Recommended configuration


The nginx developers try to work around this issue by simply buffering the entire incoming stream and forward it through the FastCGI interface as a request with a well-known length. Make sure request buffering setting fastcgi_request_buffering is enabled (Nginx does request buffering by default). There are several options to adjust the buffering behaviour.


We are not aware of mitigations for Apache HTTP Server such as disabling any buffering strategies in mod_fcgid or mod_proxy_fcgi / php-fpm. Thus you should avoid using FastCGIand instead use use mod_php which is known to handle chunked transfers correctly.


sabre/dav WebDAV library used in NextCloud and ownCloud

swiss made software