decode Transfer-Encoding: chunked from gw (gateway backends)
Transfer-Encoding: chunked is a hop-by-hop header.
Handling chunked encoding remove a hurdle for mod_proxy to send HTTP/1.1
requests to backends and be able to handle HTTP/1.1 responses.
Other backends ought not to send Transfer-Encoding: chunked, but in
practice, some implementations do.
Using BIO_new_mem_buf() allows lighttpd to control file reads and
securely wiping memory, avoiding the use of BIO_s_file(), which
employs stdio and BIO internal copying without securely wiping memory.
BIO_new_mem_buf() is also much more performant than BIO_s_file()
or BIO_new_fd() without a buffering BIO, which might make many
syscalls reading a single character at a time.
BIO_new_mem_buf() appears to be supported by older OpenSSL versions
as well as by numerous OpenSSL-compatible APIs, e.g. in BoringSSL,
LibreSSL, WolfSSL
translate config server.modules "mod_compress" to "mod_deflate"
accept compress.* directives, but issue DEPRECATED warning trace
mod_deflate differences from mod_compress:
- mod_compress compress.filetype was exact match; deflate.mimetypes is
prefix match (behavior change might compress longer mimetype matches,
which are likely of similar type and compressability)
- mod_compress always sent entire (compressed) file for Range request
mod_deflate will stream compress range result (not stored in cache)
- mod_compress would short-circuit request with 403 Forbidden error
if request file did not exist (stat() failed) (This behavior was
unfriendly to other handlers)
- mod_compress compress.cache-dir layout differs from deflate.cache-dir
layout; file cache should be cleared (or renamed) when migrating from
mod_compress to mod_deflate
- mod_deflate does not issue Vary: Accept-Encoding if request does not
contain Accept-Encoding. The identity response can be cache by
proxies and served to clients. Historically, some proxies disabled
caching if any Vary: response was seen. If the Vary header is
desirable, mod_deflate code which checks for Accept-Encoding and
compression type can be moved down a few lines to be below the
setting of the Vary response header.
use crypt() instead of crypt_r() to save stack space,
as struct crypt_data might be very large.
While crypt() is not thread-safe, lighttpd is single-threaded
auth.backend.ldap.timeout = "2000000" # quoted-string; microseconds
vhostdb.ldap += ("timeout" => "2000000") # quoted-string; microseconds
Default is 2000000 microseconds (2 secs)
These values are converted to struct timeval and passed to
ldap_set_option(ld, LDAP_OPT_NETWORK_TIMEOUT, );
ldap_set_option(ld, LDAP_OPT_TIMEOUT, ...);
if those LDAP_OPT_* values are available (both are OpenLDAP-specific).
x-ref:
"mod_auth caching"
https://redmine.lighttpd.net/issues/2805
auth.cache = ("max-age" => "600")
vhostdb.cache = ("max-age" => "600")
If specified with an empty array, default max-age is 600 secs (10 mins)
auth.cache = ()
vhostdb.cache = ()
(Note: cache expiration occurs every 8 seconds, so maximum cache time
might be up to max-age + 8 seconds)
x-ref:
"mod_auth caching"
https://redmine.lighttpd.net/issues/2805
Fixes the following error when building with -Dwith_maxminddb=true:
meson.build:916:1: ERROR: Unknown variable "libmaxminddb".
A full log can be found at meson-logs/meson-log.txt
Signed-off-by: Rosen Penev <rosenp@gmail.com>
libmariadb is what should be used as only the library portion is used.
Fixes compilation under OpenWrt.
Note that mariadb.pc is a superset that links to libmariadb.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
fix theoretical NULL dereference identified by Coverity Scan
possible for PROPFIND with specific atypical choices in lighttpd.conf:
- possible for getcontenttype if no content type matches resource
and no default type configured in lighttpd.conf
- possible for getetag if etag disabled in lighttpd.conf
(pedantic; no impact)
upon error, server will exit, so the impact of momentarily leaking fd
has no impact. This commit holds the fd in srv->stdin_fd to address
Coverity warning about leaking fd when using server.bind = "/dev/stdin"
add warning at server startup when mod_mysql_vhost is loaded
mod_vhostdb_mysql subsumes mod_mysql_vhost. Individual mod_mysql_vhost
directives map one-to-one to keywords in vhostdb.mysql = (...) directive
(expansion of buffer_string_lenth() inline function and CONST_BUF_LEN()
macro, which always check for NULL, appears to cause the analyzer to
believe that a pointer might be NULL in cases where it otherwise can
not be NULL)
x-ref:
http://clang-analyzer.llvm.org/faq.html
Prefer some WolfSSL native APIs when building with WolfSSL.
However, some functionality in WolfSSL is available only through the
WolfSSL compatibility layer for OpenSSL, so the effort to create a
native mod_wolfssl halted here.
webdav.opts = ("unsafe-propfind-follow-symlink" => "enable")
This option is unsafe and unsupported. This option enables non-standard
behavior. If it works for you, great. If it does not work for you,
then too bad.
WebDAV resource and collection concepts do not have an equivalence
to unix symlinks. If "unsafe-propfind-follow-symlink" is "enable",
then lighttpd mod_webdav PROPFIND handling will follow symlinks
if and only if webdav.is_readonly = "enable" is also set.
Allowing symlinks is unsafe in the general case. Using WebDAV methods
to MOVE a relative symlink does not update the symlink relative target.
LOCK is on the resource (e.g. a symlink), not the target of the symlink.
COPY replaces the resource (e.g. a symlink), not the target of the
symlink. There are only a few examples of possibly many more reasons
why using symlinks in a WebDAV-writable collection is unsafe.
provide option to override GnuTLS priority string
ssl.openssl.ssl-conf-cmd = ("gnutls-override" => "...")
will *entirely* replace the priority string constructed by mod_gnutls.
Admin is responsible to ensure that the override is complete.
debugging:
A non-zero value for debug.log-ssl-noise = x *in the global scope*
will cause mod_gnutls to print the GnuTLS priority string to the
error log at startup. debug.log-ssl-noise = 0 in $SERVER["socket"]
scopes can then be used to disable runtime ssl log noise, or
debug.log-ssl-noise can be removed from the global scope after debugging
difference from mod_openssl:
Admin should schedule an independent job to periodically
generate a new STEK before prior STEK lifetime expires.
Only one STEK is active at a time in mod_gnutls.
(more details in prior commit message for mod_openssl)
difference from mod_openssl:
Admin should schedule an independent job to periodically
generate a new STEK up to 2 times during key lifetime
(mbedtls internals store up to 2 keys)
(more details in prior commit message for mod_openssl)
ssl.stek-file to specify session ticket encryption key (STEK)
If ssl.stek-file is specified, it overrides builtin STEK rotation.
STEK file is checked for changes (stat()) once every 64 seconds.
STEK file should be stored in non-persistent storage,
e.g. /dev/shm/lighttpd/stek-file (in memory)
with appropriate permissions set to keep stek-file from being
read by other users. Where possible, systems should also be
configured without swap.
Admin should schedule an independent job to periodically
generate a new STEK up to 3 times during key lifetime
(lighttpd stores up to 3 keys)
format of binary file is:
4-byte - format version (always 0; for use if format changes)
4-byte - activation timestamp
4-byte - expiration timestamp
16-byte - session ticket key name
32-byte - session ticket HMAC encrpytion key
32-byte - session ticket AES encrpytion key
STEK file can be created with a command such as:
dd if=/dev/random bs=1 count=80 status=none | \
perl -e 'print pack("iii",0,time()+300,time()+86400),<>' \
> STEK-file.$$ && mv STEK-file.$$ STEK-file
The above delays activation time by 5 mins (+300 sec) to allow file to
be propagated to other machines. (admin must handle this independently)
If STEK generation is performed immediately prior to starting lighttpd,
admin should activate keys immediately (without +300).
server ticket encryption key (STEK) rotation occurs every 28800 seconds
(8 hours) and upon lighttpd server restart. While lighttpd is running,
(3) encryption keys are preserved, so tickets expire after 1 day.
If using lighttpd with multiple lighttpd workers, then restarting
lighttpd keeps the STEK in sync between lighttpd workers, though
restarting lighttpd three times a day might not be palatable.
Work is in progress to allow admin to supply new encryption keys.
stricter parse of numerical digits for http status code, port num,
and a few other places. (stricter parse than that of strtol())
content ranges are still parsed more loosely at points of use
augment simple strtoll() which allowed number to begin with '+'
This is not exploitable for HTTP Request Smuggling since lighttpd
mod_proxy sends "Connection: close" to backends, and other CGI-based
backends reconstitute CONTENT_LENGTH in the environment without '+'.
(thx Amit Klein, Safebreach)
mod_openssl leverages cert callback in openssl 1.0.2 and later
(SSL_CTX_set_cert_cb())
server certificate chain
is now set in the cert callback
verify_store (for client certificate verification) (ssl.ca-file)
is now set in the cert callback
more carefully load sensitive files and clear temporary storage
(with openssl 1.1.1 and later)
x-ref:
"Lighttpd Returns Wrong Cert In Multi-cert Set-up"
https://redmine.lighttpd.net/issues/2842
set server certificate from callback in openssl 1.0.2 and later
(SSL_CTX_set_cert_cb())
For existing versions of lighttpd, certificate selection influenced by
ssl.cipher-list which can be used to set server cipher order preference
(along with ssl.honor-cipher-order = "enable", which is the default)
x-ref:
"Lighttpd Returns Wrong Cert In Multi-cert Set-up"
https://redmine.lighttpd.net/issues/2842
"lighttpd uses wrong pem-file"
https://redmine.lighttpd.net/issues/3009
server ticket encryption key (STEK) rotation occurs every 86400 seconds
and upon lighttpd server restart. If using lighttpd with multiple
lighttpd workers, then restarting lighttpd keeps the STEK in sync
between lighttpd workers.
(experimental)
mod_gnutls supports most ssl.* config options supported by mod_openssl
x-ref:
"GnuTLS support for the mod_ssl"
https://redmine.lighttpd.net/issues/109
report SSL_R_UNEXPECTED_EOF_WHILE_READING if debug.log-ssl-noise enabled
SSL_R_UNEXPECTED_EOF_WHILE_READING
added in openssl 1.1.1e and reverted in 1.1.1f, but kept in 3.0 branch
(experimental)
mod_mbedtls supports most ssl.* config options supported by mod_openssl
thx Ward Willats for the initial discussion and attempt in the comments
https://redmine.lighttpd.net/boards/3/topics/7029
Process basic backslash-escapes in format string from lighttpd.conf
Supported sequences: \a \b \f \n \r \t \v
Other backslash-sequences are replaces with the char following backslash
(Apache mod_log_config supports \n and \t as special-cases)
./configure --with-nettle to use Nettle crypto lib for algorithms,
instead of OpenSSL or wolfSSL. Note: Nettle does not provide TLS.
x-ref:
"How to use SHA-256 without OpenSSL?"
https://redmine.lighttpd.net/boards/2/topics/8903
stream request body using HTTP/1.1 Transfer-Encoding: chunked
(Note: if backend proxy target does not support HTTP/1.1,
then do not use server.stream-request-body = 1 or 2)
If not streaming to backend, collect request body
(now supporting Transfer-Encoding: chunked from client
and then sending with Content-Length to backend)
x-ref:
"Lighty returns HTTP 411 Length Required with proxy and streaming requests/reponses body"
https://redmine.lighttpd.net/issues/3006
"nonce_secret" option to validate nonce was generated by the server
Marginally hardens HTTP Digest Auth. Necessary piece, but not
sufficient, to restrict re-use of nonce (mitigations for replay
or limiting nonce count reuse via nc=... are not implemented)
x-ref:
"Digest auth nonces are not validated"
https://redmine.lighttpd.net/issues/2976
group HANDLER_COMEBACK logic in http_response_comeback() and call it
from places that reset state in order to (sometimes partially) reprocess
a request. This includes error handler (server.error-handler),
r->handler_module when cgi.local-redir, and looping in
http_response_prepare() when modules make changes to the request and
return HANDLER_COMEBACK (e.g. mod_rewrite, mod_magnet, mod_cml)
Also, set r->conditional_is_valid closer to where elements are set
(and become valid for use in condition checks), and parse target
in http_request_parse() instead of http_response_prepare()
NB: r->tmp_buf == srv->tmp_buf (pointer is copied for quicker access)
NB: request read and write chunkqueues currently point to connection
chunkqueues; per-request and per-connection chunkqueues are
not distinct from one another
con->read_queue == r->read_queue
con->write_queue == r->write_queue
NB: in the future, a separate connection config may be needed for
connection-level module hooks. Similarly, might need to have
per-request chunkqueues separate from per-connection chunkqueues.
Should probably also have a request_reset() which is distinct from
connection_reset().
fdevent.c no longer directly uses struct server *srv
srv->srvconf.max_fds (if set) is used to set rlimits
set max_conns in server.c after fdevent_init(), which sets srv->max_fds
using srv->srvconf.max_fds (if set) as input hint
e.g. different server.errorlog for different virtual hosts
Also, support different server.breakagelog to have separate script
error logs, applicable to mod_cgi and mod_ssi exec.
use global rather than passing around (server *) just for that
li_itostrn() and li_utostrn() return string length
(rather than requiring subsequent strlen() to find length)
convert all log_error_write() to log_error() and pass (log_error_st *)
use con->errh in preference to srv->errh (even though currently same)
avoid passing (server *) when previously used only for logging (errh)
Glenn Strauss
Rosen Penev