Empty field-name is already ignored when generating response headers,
but this is an explicit skip of empty field-name sent from backends,
including field-names that look like HTTP/2 pseudo-headers, as those
begin with ':'.
Location response header is permitted to use relative-path in
RFC 7231 Section 7.1.2. Location
Prefer relative path in redirection for the benefit of reverse proxies
and CDNs. Doing so also avoids potentially disclosing internal schemes
and server names which client might not be able to directly reach.
To restore prior behavior of sending a fully-qualified absolute URI:
server.feature-flags += ("absolute-dir-redirect" => "enable")
x-ref:
https://bz.apache.org/bugzilla/show_bug.cgi?id=63357
http_request_parse_header() specialized for HTTP/2 request headers
to be parsed as each field-name and value is HPACK-decoded; send headers
directly from HPACK decoder, rather than double-buffering in chunkqueue
http_request_headers_process_h2() for post-processing
LiteSpeed ls-hpack v2.2.1
XXX: might be better to include this as a git submodule
but minor code changes were made here for portability:
- C99 flexible array members defined as a[] instead of a[0])
- pedantic compiler warnings (excess ';' and missing declarations)
- deletion of large tables from ls-hpack/huff-tables.h (code size)
(subsequently incrementally updated using git rebase)
huge props and many thank yous to writers of testing tools used while
developing HTTP/2 support in lighttpd:
h2spec - conformance testing tool for HTTP/2 implementation
https://github.com/summerwind/h2spec
h2load - HTTP/2 benchmarking tool
https://nghttp2.org/documentation/h2load-howto.html
curl - command line tool and library for transferring data with URLs
https://curl.haxx.se/
move code from connections-glue.c back into connections.c
move code from connections-glue.c to http-header-glue.c
rename connection_response_reset()
to http_response_reset()
rename connection_handle_read_post_error()
to http_response_reqbody_read_error()
r->con->reqbody_read() replaces connection_handle_read_post_state()
future: might provide different callbacks for request body with
Content-Length versus request body sent via Transfer-Encoding: chunked
Note: rrdtool counts do not include HTTP/2 protocol overhead.
Continue to count mod_rrdtool per request rather than per connection
so that data is updated after each request, rather than aggregated
to the end of a potentially long-lived connection with many keep-alives.
reset connection counters per connection, not per request
adjust mod_accesslog and mod_rrdtool usage
continue to count mod_rrdtool per request rather than per connection
so that data is updated after each request, rather than aggregated
to the end of a potentially long-lived connection with many keep-alives.
Even though request headers are per-request, update the proto on the
connection level for trusted (proxy) clients. Note: the proxy must use
each connection only for a single connection from a single client
(typically true in practice), and not for multiple clients.
proto was previously saved on connection level for mod_extforward with
HAProxy PROXY protocol, but did not occur with X-Forwarded-Proto or the
Forwarded request header with proto=...
Before this change, modules which returned HANDLER_COMEBACK could lose
HTTPS=on if the resulting request was for a backend such as mod_fastcgi.
This was reported in mod_rewrite, but could also affect mod_magnet if
MAGNET_RESTART_REQUEST, or mod_cgi with cgi.local-redir = "enable"
x-ref:
"FastCGI behavior different when using rewrite?"
https://redmine.lighttpd.net/boards/2/topics/9293
gamin should be used instead of fam; fam is no longer maintained
This patch makes it safe to build lighttpd with gamin, but run on
systems with the (deprecated) fam installed, which can happen due
to historical package dependency declarations on some platforms
(Debian, Ubuntu).
gamin and fam are not 100% binary compatible.
(Among other things, fam does not provide FAMNoExists())
x-ref:
"llibgamin vs libfam conflict solving"
https://salsa.debian.org/debian/lighttpd/-/merge_requests/18
"libgamin0: libfam shlib dependency wrongly set to libfam0"
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=510368
"undefined symbol: FAMNoExists"
https://bugs.launchpad.net/bugs/1453463
fix fallback if linkat() fails
check at startup if /proc/self/fd is present on systems with O_TMPFILE
(containers might not mount /proc)
x-ref:
"mod_webdav - PUT files with < 64kb Content-Length reults in zero length file"
https://redmine.lighttpd.net/boards/2/topics/9273
decode Transfer-Encoding: chunked from gw (gateway backends)
Transfer-Encoding: chunked is a hop-by-hop header.
Handling chunked encoding remove a hurdle for mod_proxy to send HTTP/1.1
requests to backends and be able to handle HTTP/1.1 responses.
Other backends ought not to send Transfer-Encoding: chunked, but in
practice, some implementations do.
Using BIO_new_mem_buf() allows lighttpd to control file reads and
securely wiping memory, avoiding the use of BIO_s_file(), which
employs stdio and BIO internal copying without securely wiping memory.
BIO_new_mem_buf() is also much more performant than BIO_s_file()
or BIO_new_fd() without a buffering BIO, which might make many
syscalls reading a single character at a time.
BIO_new_mem_buf() appears to be supported by older OpenSSL versions
as well as by numerous OpenSSL-compatible APIs, e.g. in BoringSSL,
LibreSSL, WolfSSL
translate config server.modules "mod_compress" to "mod_deflate"
accept compress.* directives, but issue DEPRECATED warning trace
mod_deflate differences from mod_compress:
- mod_compress compress.filetype was exact match; deflate.mimetypes is
prefix match (behavior change might compress longer mimetype matches,
which are likely of similar type and compressability)
- mod_compress always sent entire (compressed) file for Range request
mod_deflate will stream compress range result (not stored in cache)
- mod_compress would short-circuit request with 403 Forbidden error
if request file did not exist (stat() failed) (This behavior was
unfriendly to other handlers)
- mod_compress compress.cache-dir layout differs from deflate.cache-dir
layout; file cache should be cleared (or renamed) when migrating from
mod_compress to mod_deflate
- mod_deflate does not issue Vary: Accept-Encoding if request does not
contain Accept-Encoding. The identity response can be cache by
proxies and served to clients. Historically, some proxies disabled
caching if any Vary: response was seen. If the Vary header is
desirable, mod_deflate code which checks for Accept-Encoding and
compression type can be moved down a few lines to be below the
setting of the Vary response header.
use crypt() instead of crypt_r() to save stack space,
as struct crypt_data might be very large.
While crypt() is not thread-safe, lighttpd is single-threaded
auth.backend.ldap.timeout = "2000000" # quoted-string; microseconds
vhostdb.ldap += ("timeout" => "2000000") # quoted-string; microseconds
Default is 2000000 microseconds (2 secs)
These values are converted to struct timeval and passed to
ldap_set_option(ld, LDAP_OPT_NETWORK_TIMEOUT, );
ldap_set_option(ld, LDAP_OPT_TIMEOUT, ...);
if those LDAP_OPT_* values are available (both are OpenLDAP-specific).
x-ref:
"mod_auth caching"
https://redmine.lighttpd.net/issues/2805
auth.cache = ("max-age" => "600")
vhostdb.cache = ("max-age" => "600")
If specified with an empty array, default max-age is 600 secs (10 mins)
auth.cache = ()
vhostdb.cache = ()
(Note: cache expiration occurs every 8 seconds, so maximum cache time
might be up to max-age + 8 seconds)
x-ref:
"mod_auth caching"
https://redmine.lighttpd.net/issues/2805
Fixes the following error when building with -Dwith_maxminddb=true:
meson.build:916:1: ERROR: Unknown variable "libmaxminddb".
A full log can be found at meson-logs/meson-log.txt
Signed-off-by: Rosen Penev <rosenp@gmail.com>
libmariadb is what should be used as only the library portion is used.
Fixes compilation under OpenWrt.
Note that mariadb.pc is a superset that links to libmariadb.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
fix theoretical NULL dereference identified by Coverity Scan
possible for PROPFIND with specific atypical choices in lighttpd.conf:
- possible for getcontenttype if no content type matches resource
and no default type configured in lighttpd.conf
- possible for getetag if etag disabled in lighttpd.conf
(pedantic; no impact)
upon error, server will exit, so the impact of momentarily leaking fd
has no impact. This commit holds the fd in srv->stdin_fd to address
Coverity warning about leaking fd when using server.bind = "/dev/stdin"
add warning at server startup when mod_mysql_vhost is loaded
mod_vhostdb_mysql subsumes mod_mysql_vhost. Individual mod_mysql_vhost
directives map one-to-one to keywords in vhostdb.mysql = (...) directive
(expansion of buffer_string_lenth() inline function and CONST_BUF_LEN()
macro, which always check for NULL, appears to cause the analyzer to
believe that a pointer might be NULL in cases where it otherwise can
not be NULL)
x-ref:
http://clang-analyzer.llvm.org/faq.html
Prefer some WolfSSL native APIs when building with WolfSSL.
However, some functionality in WolfSSL is available only through the
WolfSSL compatibility layer for OpenSSL, so the effort to create a
native mod_wolfssl halted here.
webdav.opts = ("unsafe-propfind-follow-symlink" => "enable")
This option is unsafe and unsupported. This option enables non-standard
behavior. If it works for you, great. If it does not work for you,
then too bad.
WebDAV resource and collection concepts do not have an equivalence
to unix symlinks. If "unsafe-propfind-follow-symlink" is "enable",
then lighttpd mod_webdav PROPFIND handling will follow symlinks
if and only if webdav.is_readonly = "enable" is also set.
Allowing symlinks is unsafe in the general case. Using WebDAV methods
to MOVE a relative symlink does not update the symlink relative target.
LOCK is on the resource (e.g. a symlink), not the target of the symlink.
COPY replaces the resource (e.g. a symlink), not the target of the
symlink. There are only a few examples of possibly many more reasons
why using symlinks in a WebDAV-writable collection is unsafe.
provide option to override GnuTLS priority string
ssl.openssl.ssl-conf-cmd = ("gnutls-override" => "...")
will *entirely* replace the priority string constructed by mod_gnutls.
Admin is responsible to ensure that the override is complete.
debugging:
A non-zero value for debug.log-ssl-noise = x *in the global scope*
will cause mod_gnutls to print the GnuTLS priority string to the
error log at startup. debug.log-ssl-noise = 0 in $SERVER["socket"]
scopes can then be used to disable runtime ssl log noise, or
debug.log-ssl-noise can be removed from the global scope after debugging
difference from mod_openssl:
Admin should schedule an independent job to periodically
generate a new STEK before prior STEK lifetime expires.
Only one STEK is active at a time in mod_gnutls.
(more details in prior commit message for mod_openssl)
difference from mod_openssl:
Admin should schedule an independent job to periodically
generate a new STEK up to 2 times during key lifetime
(mbedtls internals store up to 2 keys)
(more details in prior commit message for mod_openssl)
ssl.stek-file to specify session ticket encryption key (STEK)
If ssl.stek-file is specified, it overrides builtin STEK rotation.
STEK file is checked for changes (stat()) once every 64 seconds.
STEK file should be stored in non-persistent storage,
e.g. /dev/shm/lighttpd/stek-file (in memory)
with appropriate permissions set to keep stek-file from being
read by other users. Where possible, systems should also be
configured without swap.
Admin should schedule an independent job to periodically
generate a new STEK up to 3 times during key lifetime
(lighttpd stores up to 3 keys)
format of binary file is:
4-byte - format version (always 0; for use if format changes)
4-byte - activation timestamp
4-byte - expiration timestamp
16-byte - session ticket key name
32-byte - session ticket HMAC encrpytion key
32-byte - session ticket AES encrpytion key
STEK file can be created with a command such as:
dd if=/dev/random bs=1 count=80 status=none | \
perl -e 'print pack("iii",0,time()+300,time()+86400),<>' \
> STEK-file.$$ && mv STEK-file.$$ STEK-file
The above delays activation time by 5 mins (+300 sec) to allow file to
be propagated to other machines. (admin must handle this independently)
If STEK generation is performed immediately prior to starting lighttpd,
admin should activate keys immediately (without +300).
server ticket encryption key (STEK) rotation occurs every 28800 seconds
(8 hours) and upon lighttpd server restart. While lighttpd is running,
(3) encryption keys are preserved, so tickets expire after 1 day.
If using lighttpd with multiple lighttpd workers, then restarting
lighttpd keeps the STEK in sync between lighttpd workers, though
restarting lighttpd three times a day might not be palatable.
Work is in progress to allow admin to supply new encryption keys.
stricter parse of numerical digits for http status code, port num,
and a few other places. (stricter parse than that of strtol())
content ranges are still parsed more loosely at points of use
augment simple strtoll() which allowed number to begin with '+'
This is not exploitable for HTTP Request Smuggling since lighttpd
mod_proxy sends "Connection: close" to backends, and other CGI-based
backends reconstitute CONTENT_LENGTH in the environment without '+'.
(thx Amit Klein, Safebreach)
mod_openssl leverages cert callback in openssl 1.0.2 and later
(SSL_CTX_set_cert_cb())
server certificate chain
is now set in the cert callback
verify_store (for client certificate verification) (ssl.ca-file)
is now set in the cert callback
more carefully load sensitive files and clear temporary storage
(with openssl 1.1.1 and later)
x-ref:
"Lighttpd Returns Wrong Cert In Multi-cert Set-up"
https://redmine.lighttpd.net/issues/2842
set server certificate from callback in openssl 1.0.2 and later
(SSL_CTX_set_cert_cb())
For existing versions of lighttpd, certificate selection influenced by
ssl.cipher-list which can be used to set server cipher order preference
(along with ssl.honor-cipher-order = "enable", which is the default)
x-ref:
"Lighttpd Returns Wrong Cert In Multi-cert Set-up"
https://redmine.lighttpd.net/issues/2842
"lighttpd uses wrong pem-file"
https://redmine.lighttpd.net/issues/3009
server ticket encryption key (STEK) rotation occurs every 86400 seconds
and upon lighttpd server restart. If using lighttpd with multiple
lighttpd workers, then restarting lighttpd keeps the STEK in sync
between lighttpd workers.
(experimental)
mod_gnutls supports most ssl.* config options supported by mod_openssl
x-ref:
"GnuTLS support for the mod_ssl"
https://redmine.lighttpd.net/issues/109
report SSL_R_UNEXPECTED_EOF_WHILE_READING if debug.log-ssl-noise enabled
SSL_R_UNEXPECTED_EOF_WHILE_READING
added in openssl 1.1.1e and reverted in 1.1.1f, but kept in 3.0 branch
(experimental)
mod_mbedtls supports most ssl.* config options supported by mod_openssl
thx Ward Willats for the initial discussion and attempt in the comments
https://redmine.lighttpd.net/boards/3/topics/7029
Process basic backslash-escapes in format string from lighttpd.conf
Supported sequences: \a \b \f \n \r \t \v
Other backslash-sequences are replaces with the char following backslash
(Apache mod_log_config supports \n and \t as special-cases)
./configure --with-nettle to use Nettle crypto lib for algorithms,
instead of OpenSSL or wolfSSL. Note: Nettle does not provide TLS.
x-ref:
"How to use SHA-256 without OpenSSL?"
https://redmine.lighttpd.net/boards/2/topics/8903
stream request body using HTTP/1.1 Transfer-Encoding: chunked
(Note: if backend proxy target does not support HTTP/1.1,
then do not use server.stream-request-body = 1 or 2)
If not streaming to backend, collect request body
(now supporting Transfer-Encoding: chunked from client
and then sending with Content-Length to backend)
x-ref:
"Lighty returns HTTP 411 Length Required with proxy and streaming requests/reponses body"
https://redmine.lighttpd.net/issues/3006
"nonce_secret" option to validate nonce was generated by the server
Marginally hardens HTTP Digest Auth. Necessary piece, but not
sufficient, to restrict re-use of nonce (mitigations for replay
or limiting nonce count reuse via nc=... are not implemented)
x-ref:
"Digest auth nonces are not validated"
https://redmine.lighttpd.net/issues/2976
group HANDLER_COMEBACK logic in http_response_comeback() and call it
from places that reset state in order to (sometimes partially) reprocess
a request. This includes error handler (server.error-handler),
r->handler_module when cgi.local-redir, and looping in
http_response_prepare() when modules make changes to the request and
return HANDLER_COMEBACK (e.g. mod_rewrite, mod_magnet, mod_cml)
Also, set r->conditional_is_valid closer to where elements are set
(and become valid for use in condition checks), and parse target
in http_request_parse() instead of http_response_prepare()
NB: r->tmp_buf == srv->tmp_buf (pointer is copied for quicker access)
NB: request read and write chunkqueues currently point to connection
chunkqueues; per-request and per-connection chunkqueues are
not distinct from one another
con->read_queue == r->read_queue
con->write_queue == r->write_queue
NB: in the future, a separate connection config may be needed for
connection-level module hooks. Similarly, might need to have
per-request chunkqueues separate from per-connection chunkqueues.
Should probably also have a request_reset() which is distinct from
connection_reset().
Glenn Strauss
Rosen Penev