Commit Graph

84 Commits

Author SHA1 Message Date
Glenn Strauss c9f1b612f1 [core] keep a->data[] sorted; remove a->sorted[] 2020-02-24 11:15:32 -05:00
Glenn Strauss b2991c686d [core] perf: array.c performance enhancements
mark array_get_index() as hot, rewrite to be pure and return sorted pos

mark routines as pure, as appropriate

mark routines as cold if used only at startup for config processing

mark params const, as appropriate

array_get_buf_ptr() for modifiable value buffer after insert into array

uint32_t used and size members instead of size_t

remove a->unique_ndx member; simply add to end of array for value lists
remove du->is_index_key member; simply check buffer_is_empty(du->key)

array_insert_key_value() used to be a hint that lookup could be skipped,
but the state from array_get_index() is now saved and reused internally,
so the distinction is no longer needed.  Use array_set_key_value().
2020-02-24 11:15:32 -05:00
Glenn Strauss aaccb1bc5e [multiple] address coverity warnings 2020-01-26 00:41:05 -05:00
Glenn Strauss e20b5318d5 [core] use buffer_eq_icase_ssn func
specialized buffer_eq_icase_ssn func replace strncasecmp()
in cases where string lengths are known to be at least as
large as the len being compared case-insensitively
2019-06-06 02:48:43 -04:00
Glenn Strauss 44156bbe81 [multiple] open target file earlier in some cases
open target file earlier in some cases to validate readability
2019-05-04 13:48:22 -04:00
Glenn Strauss a15f40a569 [core] replace open() with fdevent_open_cloexec()
fdevent_open_cloexec() opens files O_BINARY for consistency,
and with O_NONBLOCK, so that the server will not block if the
target file to open is a fifo.
2019-04-20 02:10:28 -04:00
Mohammed Sadiq 6a988bb0d0 [multiple] cleaner calloc use in SETDEFAULTS_FUNC
github: closes #99

  "cleaner calloc use in SETDEFAULTS_FUNC"
2019-04-20 02:09:04 -04:00
Glenn Strauss b31e7840d5 [multiple] quiet clang --analyze where trivial
quiet clang --analyze warnings where trivial to do so
2019-01-21 18:05:10 -05:00
Glenn Strauss 80638252dc [multiple] validate UTF-8 in url-decoded paths
validate UTF-8 in url-decoded paths obtained elsewhere than from request

(burl_normalize(), if enabled with server.http-parseopts, checks url for
 overlong encodings of ASCII chars in the HTTP request-line)
2018-11-25 19:52:08 -05:00
Glenn Strauss f69bd9cdb8 [core] perf: simple, quick buffer_clear()
quickly clear buffer instead of buffer_string_set_length(b, 0) or
buffer_reset(b).  Avoids free() of large buffers about to be reused,
or buffers that are module-scoped, persistent, and reused.

(buffer_reset() should still be used with buffers in connection *con
 when the data in the buffers is supplied by external, untrusted source)
2018-11-23 00:37:38 -05:00
Glenn Strauss e7c840502a [core] perf: copy small strings; better buf reuse
copy small strings to write queue for better buffer reuse
(instead of swapping with larger buffers in write chunkqueue)
2018-10-27 14:00:08 -04:00
Glenn Strauss 062089ff14 [core] perf: array_reset_data_strings()
array_reset_data_strings() specialization
2018-10-22 20:28:53 -04:00
Glenn Strauss d61f33817c [multiple] code reuse: employ array_match_*() 2018-09-23 18:01:58 -04:00
Glenn Strauss 3dd3cde902 [core] abstraction layer for HTTP header manip
convert existing calls to manip request/response headers
convert existing calls to manip environment array (often header-related)
2018-09-23 18:01:58 -04:00
Glenn Strauss a7c27c9f99 [core] code reuse with array_insert_key_value()
code reuse with array_insert_key_value() and related array manipulation
2018-09-16 05:18:05 -04:00
Glenn Strauss 04d76e7afd [core] some header cleanup
provide standard types in first.h instead of base.h
provide lighttpd types in base_decls.h instead of settings.h
reduce headers exposed by headers for core data structures
  do not expose <pcre.h> or <stdlib.h> in headers
move stat_cache_entry to stat_cache.h
reduce use of "server.h" and "base.h" in headers
2018-04-08 22:22:23 -04:00
Glenn Strauss 7bda136e5f [mod_webdav] PROPFIND getetag attr must match GET
PROPFIND getetag attr must match Etag response header from GET request

For consistency, make similar change in mod_ssi.

(thx ethoms)

  "mod_webdav: Etag in response differs between PROPFIND and GET"
2017-07-23 19:02:31 -04:00
Glenn Strauss a9970fec23 [core] consolidate fork()/execve() code (#1393)
(refactoring work to address issue #1393)

  "access log pipe writer should restart child process if it exits"
2017-07-15 22:42:15 -04:00
Glenn Strauss 86bb8be2c8 [core] perf: skip redundant strlen() if len known
performance: skip redundant strlen() if length is already known

introduce array_get_element_klen() to take key and klen params
2017-05-15 22:02:33 -04:00
Glenn Strauss 2986221cab [core] sys-strings.h abstraction for strings.h 2017-04-24 10:14:17 -04:00
Glenn Strauss a53f662a30 [core] remove some unused header includes
remove exposure of stdio.h in buffer.h for print_backtrace(), now static
2017-03-28 02:17:33 -04:00
Glenn Strauss bd77abe0f8 [config] more specific checks for array lists
More specific checks on contents of array lists.  Each module using
lists now does better checking on the types of values in the list
(strings, integers, arrays/lists)

This helps prevent misconfiguration of things like cgi.assign,
fastcgi.server, and scgi.server, where source code might be
served as static files if parenthesis are misplaced.

2017-03-08 11:42:59 -05:00
Glenn Strauss b1405360fe apply debian/patches/spelling.patch
description: fix upstream spelling errors
author: Michael Gilbert <>
2017-01-31 14:36:15 -05:00
Glenn Strauss d246656f5b [mod_ssi] send #exec cmd="..." output to temp file
prior code could leak pipe fd if ioctl() failed
prior code could leak pid (zombie) if waitpid() interrupted 4x
prior code could deadlock if child produced too much output and
  blocked in writing output while parent waited for child to exit

NOTE: mod_ssi #exec cmd="..." is still executed synchronously
      and *blocks* entire lighttpd server while executing.
2017-01-31 14:36:15 -05:00
Glenn Strauss c367b1ca80 [mod_ssi] implement, ignore <!--#comment ... --> 2016-12-11 13:25:07 -05:00
Glenn Strauss 185e262bf5 [mod_ssi] basic recursive SSI include virtual (fixes #536)
EXPERIMENTAL: basic recursive SSI <!--#include virtual="..." -->
Marked experimental since behavior may change in future.

Prior behavior was simpler and treated them all as files included as-is.

New behavior treats all #include virtual="..." targets as SSI files.

In the future, this may change to be a full recursive subrequest and the
virtual path may be treated as a new subrequest and might be something
other than SSI (e.g. might be CGI).  This has not been implemented.

Current behavior processes <!--#include virtual="..." --> as static file
Enable new behavior by setting ssi.recursion-max to value other than 0.
ssi.recursion-max = X to set maximum recusion depth

  "add recursion to the SSI #include directive"
2016-12-11 02:40:58 -05:00
Glenn Strauss 83ec97a054 [mod_ssi] produce content in subrequest hook
(prerequisite for future mod_ssi enhancements)

This commit also addresses the concern that mod_geoip would
(previously) need to be listed in modules prior to mod_ssi.
2016-12-06 13:32:29 -05:00
Glenn Strauss 431bc346df remove #include "stream.h" where not used 2016-12-05 17:43:59 -05:00
Glenn Strauss 7fa5bfc938 consistent, shared code to create CGI env
consolidated from CGI, FastCGI, SCGI, SSI

Note: due to prior inconsistencies between the code in mod_cgi,
mod_fastcgi, mod_scgi, and mod_ssi, there are some minor behavior

CONTENT_LENGTH is now always set, even if 0
  (though CONTENT_LENGTH is never set for FASTCGI_AUTHORIZER)
PATH_INFO is created only if present, not if empty.
  (mod_fastcgi and mod_ssi previously set PATH_INFO="" (blank value))
PATH_TRANSLATED is now set if PATH_INFO is present
  (previously missing from mod_cgi and mod_ssi)

mod_ssi now sets DOCUMENT_ROOT to con->physical.basedir, like others
  (previously, mod_ssi set DOCUMENT_ROOT to con->physical.doc_root,
   which matched con->physical.basedir unless mod_alias changed basedir)
mod_ssi now sets REQUEST_URI to con->request.orig_uri, like others
  (previously, mod_ssi set REQUEST_URI to con->request.uri, which
   matched con->request.orig_uri except after redirects, error docs)
2016-10-10 13:37:36 -04:00
Glenn Strauss 779c133c16 [security] do not emit HTTP_PROXY to CGI env
Strip bogus "Proxy" header before creating subprocess environment.
(mod_cgi, mod_fastcgi, mod_scgi, mod_ssi, mod_proxy)

Do not emit HTTP_PROXY to subprocess environment.
Some executables use HTTP_PROXY to configure outgoing proxy.

This is not a lighttpd security issue per se, but this change to
lighttpd adds a layer of defense to protect backend processes which
might be vulnerable due to blindly using this untrusted environment
variable.  The HTTP_PROXY environment variable should not be trusted
by a program running in a CGI-like environment.

Mitigation in lighttpd <= 1.4.40 is to reject requests w/ Proxy header:

* Create "/path/to/deny-proxy.lua", read-only to lighttpd, with content:
  if (lighty.request["Proxy"] == nil) then return 0 else return 403 end
* Modify lighttpd.conf to load mod_magnet and run lua code
    server.modules += ( "mod_magnet" )
    magnet.attract-raw-url-to = ( "/path/to/deny-proxy.lua" )

CGI web servers assign Proxy header values from client requests to
internal HTTP_PROXY environment variables
httpoxy: A CGI application vulnerability
2016-07-19 01:22:33 -04:00
Glenn Strauss 393dfd8cb9 [mod_ssi] fix #config sizefmt="bytes" 2016-07-17 14:54:03 -04:00
Glenn Strauss a714f4f720 fix gcc 6.1.1 compiler warn misleading-indentation 2016-07-12 20:19:32 -04:00
Glenn Strauss 69ec5728f9 use con->conf.server_tag in modules
do not expose server info server_tag is configured by admin

default con->conf.server_tag is still PACKAGE_DESC, those
admin can configure server.tag otherwise.

(these changes reduce recompilation and relinking when switching
 brances in source control)
2016-06-25 18:52:06 -04:00
Glenn Strauss fde843f63e [mod_ssi] fix parse of tag across buf boundary (fixes #2732)
thx fbrosson

  "char copied to wrong place in SSI output"
2016-05-17 15:33:27 -04:00
Glenn Strauss a5fcfee6fc [mod_ssi] more flexible quoting (fixes #1768)
allow double-quotes, single-quotes or no quote on SSI param values

remove use of PCRE from mod_ssi

fix misspelling of 'unknow' to be 'unknown'

  "mod_ssi doesn't accept single quotes"
2016-05-14 05:50:37 -04:00
Glenn Strauss abfb9d9e2d [mod_ssi] add PCRE_* options to constrain regex 2016-05-14 01:17:52 -04:00
Glenn Strauss ede4d17913 [mod_ssi] include relative to alias,userdir (fixes #222)
adjust paths relative to changes made by mod_alias and mod_userdir

Note: this still works only for direct file inclusion.
lighttpd mod_ssi does not perform an "internal subrequest" for the
virtual path, so things like virtual include of CGI are not supported

  "ssi virtual include uses wrong path"
2016-05-14 01:17:52 -04:00
Glenn Strauss df146a7724 [mod_ssi] fix SSI statement parser
bug introduced in 8e3c6bf7 when statement parser was replaced
2016-05-14 01:17:52 -04:00
Glenn Strauss a0a7b9fbf5 [mod_ssi, mod_cml] set DOCUMENT_ROOT to basedir (fixes #2383)
fixes inconsistency w/ mod_fastcgi, mod_scgi, mod_cgi change in adc97e5b

  "mod_alias: use alias directory as doc-root too"
  "mod_userdir doesn't set environment variable DOCUMENT_ROOT"
2016-05-02 14:31:36 -04:00
Glenn Strauss 8e3c6bf754 fallback to lseek()/read() if mmap() fails (#fixes 2666)
fallback to lseek()/read() if mmap() fails (#fixes 2666)
e.g. when mmap() is used on lighttpd-controlled temporary files
used POST request body (mod_cgi) and PUT file upload (mod_webdav)

replace use of stream_open() on potentially untrusted files
(protect against SIGBUS if a file is modified while map is read)
Note: stream.[ch] may be removed in a future release
For now, stream.[ch] will read entire file into memory if mmap fails
and so it should only be used on trusted files, e.g. config files.

http_auth basic and digest files are typically small and so buffered
stdio fopen(), fgets(), fclose() will likely be approximately as fast
as mmap.

mod_dirlisting header and readme files are typically small and so
open(), read(), close() will typically be approximately as fast as mmap

mod_ssi will likely be much faster, now buffering SSI page construction
rather than a potentially huge number of file open() calls, one for each
tiny chunk of text between SSI directives.

mod_webdav COPY and MOVE may be slower due to removal of mmap, but are
now more resilient to partial writes.

  "handle filesystems without mmap() support"
  "WebDAV upload-> mmap failed: operation not permitted"
  "Lighttpd 1.4.20 Crash (SIGBUS in mod_compress)"
  "Crash SIGBUS"

github: closes #57
2016-04-28 13:13:36 -04:00
Glenn Strauss 5492063f35 [core] set REDIRECT_STATUS to error_handler_saved_status (fixes #1828)
set REDIRECT_STATUS to con->error_handler_saved_status in dynamic
handlers for PHP compiled with --force-redirect.  Set to "200"
if (0 == con->error_handler_saved_status)
(mod_cgi, mod_fastcgi, mod_scgi, mod_ssi)

FYI: setting REDIRECT_STATUS in con->environment allows access and
manipulation by mod_magnet.

  "REDIRECT_STATUS == 200 on 404 redirect"

github: closes #35
2016-04-25 01:00:06 -04:00
Glenn Strauss fa3a36ffd4 [mod_ssi] config ssi.exec (fixes #2051)
(thx benbrown)

  "mod_ssi Add configuration item to disable SSI exec."
2016-04-14 08:59:07 -04:00
fbrosson 6982b1930e [mod_ssi] config ssi.conditional-requests
A new SSI directive, "ssi.conditional-requests", allows to inform
lighttpd which SSI pages should be considered as cacheable and which
should not. In particular, the "ETag" & "Last-Modified" headers will
only be sent for those SSI pages for which the directive is enabled.

Long description:
"ETag" and "Last-Modified" headers were being sent for all SSI pages,
regardless of whether they were cacheable or not. And yet, there was
no cache validation at all for any SSI page.
This commit fixes these two minor issues by adding a new directive,
"ssi.conditional-requests", which allows to specify which SSI pages
are cacheable and which are not, and by adding cache validation to
those SSI pages which are cacheable. And since sending ETags for
non-cacheable documents is not appropriate, they are no longuer
computed nor sent for those SSI pages which are not cacheable.
Regarding the "Last-Modified" header for non-cacheable documents,
the standards allow to either send the current date and time for
that header or to simply skip it. The approach chosen is to not send
it for non-cacheable SSI pages. "ETag" and "Last-Modified" headers
are therefore only sent for an SSI page if ssi.conditional-requests
is enabled for that page.

The ssi.conditional-requests directive can be enabled or disabled
globally and/or in any context. It is disabled by default.

An index.shtml which only includes deterministic SSI commands such as:
<!--#echo var="LAST_MODIFIED"-->
is a trivial example of a dynamic SSI page that is cacheable.
2016-04-14 12:35:10 +00:00
Glenn Strauss 47f3dbebe4 use li_[iu]tostrn() instead of li_[iu]tostr()
From: Glenn Strauss <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-04-01 17:38:26 +00:00
fbrosson a579e7ffc0 [mod_ssi] Add SSI vars SCRIPT_{URI,URL} and REQUEST_SCHEME (fixes #2721)
This is a proposal to add to lighttpd the famous SSI variables
SCRIPT_URI and SCRIPT_URL (known to Apache users), as well as a bonus
ENV variable called REQUEST_SCHEME.

SCRIPT_URI and SCRIPT_URL will be available as SSI variables from
within documents handled by mod_ssi.
They can be used like any other SSI var with the "#echo var" command:
<!--#echo var="SCRIPT_URI"-->
<!--#echo var="SCRIPT_URL"-->
Webmasters willing to display links to the W3C Validator will be able
to use:
<a href="<!--#echo var="SCRIPT_URI"-->">…</a>
instead of the generic link
which does not work on some (most?) browsers which do not send
referers when the link itself resides in a document sent through

REQUEST_SCHEME will be available both as an environment variable. It
is defined as "http" or "https", depending on the scheme of the
connection. It is safe to use this name as it does not conflict with
any existing variable on Apache or Nginx. This is slightly different
from the HTTPS var which is often added by webadmins on their server's
configuration. EDIT: Some Apache modules also define REQUEST_SCHEME
with the same possible values as this proposal.

From: fbrosson <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-03-26 11:14:21 +00:00
Glenn Strauss 8abd06a7ff consistent inclusion of config.h at top of files (fixes #2073)
From: Glenn Strauss <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-03-19 15:14:35 +00:00
fbrosson d8e028e069 [mod_ssi] enhance support for ssi vars
Try ssi_vars if ssi_cgi_env does not have a matching var name.
Allow var names to also include digits after the initial letter or underscore.

From: fbrosson <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-01-03 14:48:07 +00:00
Stefan Bühler c512345fa2 [config] check config option scope; warn if server option is given in conditional
From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2015-11-07 12:51:11 +00:00
Stefan Bühler 33cebeb0f7 fix segfaults in many plugins if they failed configuration
From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2015-05-14 09:38:33 +00:00
Stefan Bühler ad3e93ea96 Use buffer API to read and modify "used" member
- a lot of code tried to handle manually adding terminating zeroes and
  keeping track of the correct "used" count.
  Replaced all "external" usages with simple wrapper functions:
  * buffer_string_is_empty (used <= 1), buffer_is_empty (used == 0);
    prefer buffer_string_is_empty
  * buffer_string_set_length
  * buffer_string_length
  * CONST_BUF_LEN() macro
- removed "static" buffer hacks (buffers pointing to constant/stack
  memory instead of malloc()ed data)
- buffer_append_strftime(): refactor buffer+strftime uses
- li_tohex(): no need for a buffer for binary-to-hex conversion:
  the output data length is easy to predict
- remove "-Winline" from extra warnings: the "inline" keyword just
  supresses the warning about unused but defined (static) functions;
  don't care whether it actually gets inlined or not.

From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2015-02-08 19:10:44 +00:00