Zero byte file truncate issue with Nextcloud and ownCloud deployed with FastCGI

Summary

Using a client to upload files with HTTP chunked transfer encoding to a server with fastcgi/php-fmp enabled can lead to zero byte files. Chunked transfer encoding is used when the content length is unknown at the time the transfer is started and no Content-length header can be set.

Background

There are basically two options to encode a PUT request. You either know the length of the data you want to transfer at the time of starting the request then you can set the HTTP Content-Length header appropriately or you don’t know the length and thus have to choose a streaming approach using the chunked transfer encoding which does not need a Content-Length header at all. A client is free to choose either of the two request types as both are completely fine with the HTTP specification.

Most clients out there, including web browsers, use the former method and thus do not hit this issue. There are use cases though which make it necessary to transfer data chunked. For example Mountain Duck that implements a virtual file system for accessing your cloud storage online. The write callbacks we get from the OS just include an offset, a buffer length and the buffer itself. Mountain Duck does not know the final size in advance. From a virtual file system perspective the call backs Mountain Duck get for an upload are as follows

1 - CreateFile myFile.txt
2 - SetAllocationSize myFile.txt, 1024
3 - WriteFile myFile.txt, offset 0, length 1024, buffer
4 - SetAllocationSize myFile.txt, 2048
5 - WriteFile myFile.txt, offset 1024, length 1024, buffer
6 - CloseFile myFile.txt

In step 3 Mountain Duck opens a connection to the remote server, send a PUT request with the HTTP header Transfer-Encoding: chunked and streams through all subsequent write callbacks. Finally, in step 6, the connection is closed.

References