Commit Graph

11 Commits

Author SHA1 Message Date
Glenn Strauss a65c57a548 [core] open fd when appending file to cq (fixes #2655)
http_chunk_append_file() opens fd when appending file to chunkqueue.
Defers calculation of content length until response is finished.

This reduces race conditions pertaining to stat() and then (later)
open(), when the result of the stat() was used for Content-Length
or to generate chunked headers.

Note: this does not change how lighttpd handles files that are modified
in-place by another process after having been opened by lighttpd --
don't do that.  This *does* improve handling of files that are
frequently modified via a temporary file and then atomically renamed
into place.

mod_fastcgi has been modified to use http_chunk_append_file_range() with
X-Sendfile2 and will open the target file multiple times if there are
multiple ranges.

Note: (future todo) not implemented for chunk.[ch] interfaces used by
range requests in mod_staticfile or by mod_ssi.  Those uses could lead
to too many open fds.  For mod_staticfile, limits should be put in place
for max number of ranges accepted by mod_staticfile.  For mod_ssi,
limits would need to be placed on the maximum number of includes, and
the primary SSI file split across lots of SSI directives should either
copy the pieces or perhaps chunk.h could be extended to allow for an
open fd to be shared across multiple chunks.  Doing either of these
would improve the performance of SSI since they would replace many file
opens on the pieces of the SSI file around the SSI directives.

  "Serving a file that is getting updated can cause an empty response or incorrect content-length error"

Closes #49
2016-04-18 04:27:08 -04:00
Glenn Strauss 8abd06a7ff consistent inclusion of config.h at top of files (fixes #2073)
From: Glenn Strauss <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-03-19 15:14:35 +00:00
Stefan Bühler 91a9a6b391 rename buffer_append_long_hex to buffer_append_uint_hex
* takes uintmax_t now
* use in http_chunk_append_len

From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2015-02-08 19:10:46 +00:00
Stefan Bühler ad3e93ea96 Use buffer API to read and modify "used" member
- a lot of code tried to handle manually adding terminating zeroes and
  keeping track of the correct "used" count.
  Replaced all "external" usages with simple wrapper functions:
  * buffer_string_is_empty (used <= 1), buffer_is_empty (used == 0);
    prefer buffer_string_is_empty
  * buffer_string_set_length
  * buffer_string_length
  * CONST_BUF_LEN() macro
- removed "static" buffer hacks (buffers pointing to constant/stack
  memory instead of malloc()ed data)
- buffer_append_strftime(): refactor buffer+strftime uses
- li_tohex(): no need for a buffer for binary-to-hex conversion:
  the output data length is easy to predict
- remove "-Winline" from extra warnings: the "inline" keyword just
  supresses the warning about unused but defined (static) functions;
  don't care whether it actually gets inlined or not.

From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2015-02-08 19:10:44 +00:00
Stefan Bühler 4365bdbebe Remove buffer_prepare_copy() and buffer_prepare_append()
* removed almost all usages of buffer as "memory" (without terminating
* refactored cgi variable name encoding

From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2015-02-08 19:10:39 +00:00
Stefan Bühler 6afad87d2e fix buffer, chunk and http_chunk API
* remove unused structs and functions
    (buffer_array, read_buffer)
  * change return type from int to void for many functions,
    as the return value (indicating error/success) was never checked,
    and the function would only fail on programming errors and not on
    invalid input; changed functions to use force_assert instead of
    returning an error.
  * all "len" parameters now are the real size of the memory to be read.
    the length of strings is given always without the terminating 0.
  * the "buffer" struct still counts the terminating 0 in ->used,
    provide buffer_string_length() to get the length of a string in a
    unset config "strings" have used == 0, which is used in some places
    to distinguish unset values from "" (empty string) values.
  * most buffer usages should now use it as string container.
  * optimise some buffer copying by "moving" data to other buffers
  * use (u)intmax_t for generic int-to-string functions
  * remove unused enum values: UNUSED_CHUNK, ENCODING_UNSET
  * converted BUFFER_APPEND_SLASH to inline function (no macro feature
  * refactor: create chunkqueue_steal: moving (partial) chunks into another
  * http_chunk: added separate function to terminate chunked body instead of
    magic handling in http_chunk_append_mem().
    http_chunk_append_* now handle empty chunks, and never terminate the
    chunked body.

From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2015-02-08 12:37:10 +00:00
Stefan Bühler 22e8b456a9 Fix header inclusion order, always include "config.h" before any system header
git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2009-10-11 14:31:42 +00:00
Stefan Bühler 52861d77df Replace buffer_{append,copy}_string with the _len variant where possible (#1732, thx crypt)

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2008-07-30 19:38:32 +00:00
Marcus Rückert 8cd1471cb3 - white space cleanup part 2 this time 1.4 ;)
i hope it helps with merging stuff back to 1.5

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2006-10-04 13:26:23 +00:00
Jan Kneschke 2721197ff6 workaround a hanging ssl-connections in Opera 9.01 and 8.54 (and earlier) if chunked-encoding is used
git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2006-09-18 16:41:43 +00:00
Jan Kneschke bcdc6a3bbc moved everything below trunk/ and added branches/ and tags/
git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2005-02-20 14:27:00 +00:00