Commit Graph

88 Commits

Author SHA1 Message Date
Glenn Strauss c752d4696e [multiple] correct misspellings in comments
  "Script for fixing spelling errors with codespell"
2020-07-08 19:54:30 -04:00
Glenn Strauss 7c7f8c467c [multiple] split con, request (very large change)
NB: r->tmp_buf == srv->tmp_buf (pointer is copied for quicker access)

NB: request read and write chunkqueues currently point to connection
    chunkqueues; per-request and per-connection chunkqueues are
    not distinct from one another
      con->read_queue  == r->read_queue
      con->write_queue == r->write_queue

NB: in the future, a separate connection config may be needed for
    connection-level module hooks.  Similarly, might need to have
    per-request chunkqueues separate from per-connection chunkqueues.
    Should probably also have a request_reset() which is distinct from
2020-07-08 19:54:29 -04:00
Glenn Strauss 31d9495330 [core] store subrequest_handler instead of mode
store pointer to module in handler_module instead of con->mode id
2020-07-08 19:54:29 -04:00
Glenn Strauss 1474be7859 [core] move addtl request-specific struct members 2020-07-08 19:54:29 -04:00
Glenn Strauss 27e27e5e40 [core] static buffers for mtime_cache 2020-07-08 19:54:29 -04:00
Glenn Strauss 0ff60d8218 [multiple] rename r to rc rv rd wr to be different
variable rename
2020-07-08 19:54:28 -04:00
Glenn Strauss 03b4c993d2 [multiple] generic config array type checking 2020-07-08 19:54:28 -04:00
Glenn Strauss 68d8d4c532 [multiple] stat_cache singleton 2020-07-08 19:54:28 -04:00
Glenn Strauss f24e6d696a [multiple] plugin_stats array
use global rather than passing around (server *) just for that

li_itostrn() and li_utostrn() return string length
(rather than requiring subsequent strlen() to find length)
2020-07-08 19:54:28 -04:00
Glenn Strauss 50bdb55de8 [multiple] connection hooks no longer get (srv *)
(explicit (server *) not passed; available in con->srv)
2020-07-08 19:54:28 -04:00
Glenn Strauss 010c28949c [multiple] prefer (connection *) to (srv *)
convert all log_error_write() to log_error() and pass (log_error_st *)

use con->errh in preference to srv->errh (even though currently same)

avoid passing (server *) when previously used only for logging (errh)
2020-07-08 19:54:28 -04:00
Glenn Strauss 2ec70f234a [core] stat_cache_path_contains_symlink use errh
use log_error() with con->errh
2020-07-08 18:08:52 -04:00
Glenn Strauss b73949e03f [multiple] plugin.c handles common FREE_FUNC code
(simpler for modules; less boilerplate to cut-n-paste)
2020-07-08 18:08:51 -04:00
Glenn Strauss ed62e354ff [core] use config_plugin_values_init() 2020-07-08 18:08:51 -04:00
Glenn Strauss 16665ae5f4 [mod_compress] use config_plugin_values_init() 2020-07-08 18:08:51 -04:00
Glenn Strauss d7a6a7a263 [core] (const buffer *) from strftime_cache_get() 2020-05-23 17:59:29 -04:00
Glenn Strauss e2de4e581e [core] const char *name in struct plugin
put void *data (always used) as first member of struct plugin

add int nconfig member to PLUGIN_DATA

calloc() inits p->data to NULL
2020-05-23 17:59:29 -04:00
Glenn Strauss 36f64b26a1 [core] simpler config_check_cond()
optimize for common case where condition has been evaluated for
the request and a cached result exists

(also: begin isolating data_config)
2020-05-23 17:59:29 -04:00
Glenn Strauss 6eb34ef5ab [core] add const to callers of http_header_*_get()
(The few places where value is modified in-place were not made const)
2020-02-24 11:15:32 -05:00
Glenn Strauss 601c572c39 [core] inline buffer as part of data_string value
(instead of value being (buffer *))
2020-02-24 11:15:32 -05:00
Glenn Strauss 47a758f959 [core] inline buffer key for *_patch_connection()
handle buffer key as part of DATA_UNSET in *_patch_connection()
(instead of key being (buffer *))
2020-02-24 11:15:32 -05:00
Glenn Strauss aaccb1bc5e [multiple] address coverity warnings 2020-01-26 00:41:05 -05:00
Glenn Strauss 1300815688 [core] use buffer_eq_icase_ssn func
specialized buffer_eq_icase_ssn func replace strncasecmp()
in cases where string lengths are not known to be at least
as large as the len being compared case-insensitively.
(Separate commit in case any future changes modify the
implementation to be unsafe for shorter strings, where
strncasecmp() would stop at '\0' in either string)
2019-06-06 02:48:43 -04:00
Glenn Strauss 44156bbe81 [multiple] open target file earlier in some cases
open target file earlier in some cases to validate readability
2019-05-04 13:48:22 -04:00
Glenn Strauss 73bfee6308 [stat_cache] separate func for symlink policy chk
Note: historical ToC-ToU race condition still exists in implementation
server.follow-symlink = "disable" is not recommended (default: "enable")
2019-04-29 18:11:15 -04:00
Glenn Strauss a15f40a569 [core] replace open() with fdevent_open_cloexec()
fdevent_open_cloexec() opens files O_BINARY for consistency,
and with O_NONBLOCK, so that the server will not block if the
target file to open is a fifo.
2019-04-20 02:10:28 -04:00
Mohammed Sadiq 6a988bb0d0 [multiple] cleaner calloc use in SETDEFAULTS_FUNC
github: closes #99

  "cleaner calloc use in SETDEFAULTS_FUNC"
2019-04-20 02:09:04 -04:00
Glenn Strauss f69bd9cdb8 [core] perf: simple, quick buffer_clear()
quickly clear buffer instead of buffer_string_set_length(b, 0) or
buffer_reset(b).  Avoids free() of large buffers about to be reused,
or buffers that are module-scoped, persistent, and reused.

(buffer_reset() should still be used with buffers in connection *con
 when the data in the buffers is supplied by external, untrusted source)
2018-11-23 00:37:38 -05:00
Glenn Strauss 3dd3cde902 [core] abstraction layer for HTTP header manip
convert existing calls to manip request/response headers
convert existing calls to manip environment array (often header-related)
2018-09-23 18:01:58 -04:00
Glenn Strauss 04d76e7afd [core] some header cleanup
provide standard types in first.h instead of base.h
provide lighttpd types in base_decls.h instead of settings.h
reduce headers exposed by headers for core data structures
  do not expose <pcre.h> or <stdlib.h> in headers
move stat_cache_entry to stat_cache.h
reduce use of "server.h" and "base.h" in headers
2018-04-08 22:22:23 -04:00
Glenn Strauss b1df38ab6a [core] increase stat_cache abstraction
reduce dependency on struct connection
routines for getting/caching content_type and etag separate from stat
2018-02-02 23:28:38 -05:00
Glenn Strauss e4ed2ed4ae [mod_compress,mod_deflate] try mmap MAP_PRIVATE
try mmap MAP_PRIVATE if mmap MAP_SHARED fails with errno == EINVAL
Some file systems such as jffs2 and btrfs might not support MAP_SHARED
2017-12-09 20:22:29 -05:00
Glenn Strauss 2986221cab [core] sys-strings.h abstraction for strings.h 2017-04-24 10:14:17 -04:00
Glenn Strauss a53f662a30 [core] remove some unused header includes
remove exposure of stdio.h in buffer.h for print_backtrace(), now static
2017-03-28 02:17:33 -04:00
Glenn Strauss bd77abe0f8 [config] more specific checks for array lists
More specific checks on contents of array lists.  Each module using
lists now does better checking on the types of values in the list
(strings, integers, arrays/lists)

This helps prevent misconfiguration of things like cgi.assign,
fastcgi.server, and scgi.server, where source code might be
served as static files if parenthesis are misplaced.

2017-03-08 11:42:59 -05:00
Glenn Strauss 1f3ad401ba [mod_deflate] skip deflate if loadavg too high (fixes #1505)
[mod_deflate] skip deflate if 1 min loadavg too high
deflate.max-loadavg  = "3.50"  # express value as string of float num

[mod_compress] skip compression if 1 min loadavg too high
compress.max-loadavg = "3.50"  # express value as string of float num

Feature available on BSD-like systems which have getloadavg() in libc

Note: load average calculations are different on different operating
systems and different types of system loads, so there is no value that
can be recommended for one-size-fits-all.

  "Enable mod_compress to abandon compression when load average is too high"
2016-10-19 16:38:47 -04:00
Glenn Strauss 72a5ff1f21 [mod_accesslog] %{ratio}n logs compression ratio (fixes #2133)
mod_deflate and mod_compress now provide data for mod_accesslog
"%{ratio}n%%" log format to log compression ratio

Implementation detail: compression ratio is stored in con->environment
since lighttpd does not currently have concept of module notes, which is
from where %{VARNAME}n originates.  In the future, this might change in
lighttpd, so be sure to use %{ratio}n%% and not %{...}e for this info.

  "accesslog support "%n" (compress ratio)"
2016-10-19 16:37:22 -04:00
Glenn Strauss 1f4874cb9c build w/o compiler warnings if no zlib or bz2lib 2016-10-16 01:58:36 -04:00
Glenn Strauss a9568d0ad6 check close() return code after writing to file 2016-05-10 22:33:44 -04:00
Glenn Strauss c9b56735b8 [mod_compress] use mmap and trap SIGBUS (#2666, fixes #1879)
use mmap and trap SIGBUS in mod_compress
(if lighttpd build with --enable-mmap)

mod_compress has not used mmap since Feb 2012 (see #2391)

  "Lighttpd 1.4.20 Crash (SIGBUS in mod_compress)"
  "Crash SIGBUS"
  "handle filesystems without mmap() support"

github: closes #56
2016-04-28 13:11:15 -04:00
Glenn Strauss 8abd06a7ff consistent inclusion of config.h at top of files (fixes #2073)
From: Glenn Strauss <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-03-19 15:14:35 +00:00
Glenn Strauss 75e4859a1b [mod_compress] case-insensitive content-codings (fixes #2645)
From: Glenn Strauss <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-02-28 11:59:09 +00:00
Stefan Bühler c512345fa2 [config] check config option scope; warn if server option is given in conditional
From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2015-11-07 12:51:11 +00:00
Stefan Bühler 33cebeb0f7 fix segfaults in many plugins if they failed configuration
From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2015-05-14 09:38:33 +00:00
Stefan Bühler ad3e93ea96 Use buffer API to read and modify "used" member
- a lot of code tried to handle manually adding terminating zeroes and
  keeping track of the correct "used" count.
  Replaced all "external" usages with simple wrapper functions:
  * buffer_string_is_empty (used <= 1), buffer_is_empty (used == 0);
    prefer buffer_string_is_empty
  * buffer_string_set_length
  * buffer_string_length
  * CONST_BUF_LEN() macro
- removed "static" buffer hacks (buffers pointing to constant/stack
  memory instead of malloc()ed data)
- buffer_append_strftime(): refactor buffer+strftime uses
- li_tohex(): no need for a buffer for binary-to-hex conversion:
  the output data length is easy to predict
- remove "-Winline" from extra warnings: the "inline" keyword just
  supresses the warning about unused but defined (static) functions;
  don't care whether it actually gets inlined or not.

From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2015-02-08 19:10:44 +00:00
Stefan Bühler 4365bdbebe Remove buffer_prepare_copy() and buffer_prepare_append()
* removed almost all usages of buffer as "memory" (without terminating
* refactored cgi variable name encoding

From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2015-02-08 19:10:39 +00:00
Stefan Bühler 1be163b44a Remove chunkqueue_get_{append,prepend}* API
Although those were "easy" to use, they violated the abstraction:
  content of the chunkqueue should only be modified via the API.
  Replace with chunkqueue_get_memory() and chunkqueue_use_memory() for
  functions that read data from network (reusing large buffers),
  chunkqueue_steal_with_tempfiles() to store request bodies on disk
  Modules that were generating content and need a buffer maintain the
  buffer manually (have to be careful to free the buffer on errors, as
  it isn't part of the chunkqueue yet).

From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2015-02-08 19:10:36 +00:00
Stefan Bühler 6afad87d2e fix buffer, chunk and http_chunk API
* remove unused structs and functions
    (buffer_array, read_buffer)
  * change return type from int to void for many functions,
    as the return value (indicating error/success) was never checked,
    and the function would only fail on programming errors and not on
    invalid input; changed functions to use force_assert instead of
    returning an error.
  * all "len" parameters now are the real size of the memory to be read.
    the length of strings is given always without the terminating 0.
  * the "buffer" struct still counts the terminating 0 in ->used,
    provide buffer_string_length() to get the length of a string in a
    unset config "strings" have used == 0, which is used in some places
    to distinguish unset values from "" (empty string) values.
  * most buffer usages should now use it as string container.
  * optimise some buffer copying by "moving" data to other buffers
  * use (u)intmax_t for generic int-to-string functions
  * remove unused enum values: UNUSED_CHUNK, ENCODING_UNSET
  * converted BUFFER_APPEND_SLASH to inline function (no macro feature
  * refactor: create chunkqueue_steal: moving (partial) chunks into another
  * http_chunk: added separate function to terminate chunked body instead of
    magic handling in http_chunk_append_mem().
    http_chunk_append_* now handle empty chunks, and never terminate the
    chunked body.

From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2015-02-08 12:37:10 +00:00
Stefan Bühler 07dd0bd0a5 add force_assert() to enforce assertions as simple assert()s are disabled by -DNDEBUG (fixes #2546)
From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2014-02-16 13:08:20 +00:00
Stefan Bühler bcd35cc264 remove logical dead code
From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2014-02-14 21:06:14 +00:00