Note: http_auth_backend_t digest interface returns result as a
binary MD5 (16-bytes) so that caller consistently converts to
lowercase before using it in further digest calculation.
(Alternatively, the http_auth_backend_t digest interface could have
taken a 33-char buffer and returned an explicitly lowercased hex str)
create new, extensible interface for (additional) auth backends
attempt to handle HANDLER_WAIT_FOR_EVENT returned by auth backends
to allow for async auth backends (e.g. to mysql database)
separate auth backends from mod_auth and http_auth
mod_authn_file.c htdigest, htpasswd, plain auth backends
mod_authn_ldap.c ldap auth backend
add http_auth.c to common_sources for auth backend registration
(mod_authn_file could be three separate modules, but no need for now)
copied javascript from mod_status and from lighttpd2 mod_dirlist
modified and specialized for stable dirlist sorting by name
Partial implementation of Apache autoindex request query arguments
(https://httpd.apache.org/docs/current/en/mod/mod_autoindex.html)
If query string is supplied, allow specifying initial column to sort
?C=N name (default)
?C=M last-modified, then by name
?C=S size, then by name
?C=T type, then by name
?C=D type, then by name
and O=[AD] can be added for descending or ascending order, e.g.
?C=N&O=D descending (default)
?C=N&O=A ascending
(While functional, no effort was made on js performance.
Patches welcome)
New directive dir-listing.external-js for user to replace sorting js
Note: dir-listing.external-js or default js sorting is enabled only
if dir-listing.auto-layout = "enable" (which is the default)
x-ref:
"client-selectable directory list sorting"
https://redmine.lighttpd.net/issues/613
"dir-listing.external-js"
https://redmine.lighttpd.net/issues/2315
Due to the POLLHUP behavior triggered on *BSD/Darwin, the shutdown()
had previously been limited to local connections. If interested in
squeezing the last bits of performance out of a machine, an admin
should configure local connections to be AF_UNIX instead of AF_INET
or AF_INET6 to localhost. The reason the shutdown() was originally
added in mod_proxy and mod_scgi was to aggressively reduce the number
of potential sockets in TIME_WAIT held by lighttpd.
(See commit:923688d2 "drain backend socket/pipe bufs upon FDEVENT_HUP",
done for reliability given the aforementioned *BSD/Darwin behavior.)
When using AF_UNIX, the TIME_WAIT issue does not exist, ergo, the
recommendation is to use AF_UNIX for local sockets, when available.
Using AF_UNIX sockets is a better solution to eliminate TIME_WAIT
than is TCP shutdown() half-close which, as we have seen, might not
be handled well by frameworks which are more complex than basic read
request, send response, and close.
x-ref:
"1.4.40/41 mod_proxy, mod_scgi may trigger POLLHUP on *BSD,Darwin"
https://redmine.lighttpd.net/issues/2743
do not enter handler a second time in connection_state_machine()
after read body completes if dynamic handler is still waiting for event
and CON_STATE_READ_POST transitioned to CON_STATE_HANDLE_REQUEST
mod_fastcgi, mod_scgi, and mod_proxy must enforce wait for POLLWR
after EINPROGRESS or else getsockopt(fd, SOL_SOCKET, SO_ERROR, ...)
may succeed even though socket connection is not yet established,
and subsequent writev() will fail ENOTCONN.
(thx pkubaj)
x-ref:
"1.4.40/41 writev failed: Socket is not connected (fastcgi,scgi,proxy)"
https://redmine.lighttpd.net/issues/2744
Check if client half-closed TCP connection if POLLHUP is received.
This more robustly handles if client called shutdown(fd, SHUT_WR).
This patch reverts commit:ab05eb7c which should now be handled properly.
(Time will tell.)
x-ref:
"1.4.40/41 mod_proxy, mod_scgi may trigger POLLHUP on *BSD,Darwin"
https://redmine.lighttpd.net/issues/2743
shutdown(fd, SHUT_WR) after sending request to proxy or SCGI
only if remote is local and platform is not *BSD or Darwin.
The reason this fix is special-casing *BSD and Darwin is that the Single
Unix Specification and POSIX.1-2013 clearly specify that POLLHUP event
should be returned by poll only when the stream is no longer writable.
A half-closed socket that is still writable clearly does not match that
condition, yet that is what I am seeing on Darwin (El Capitan), and
presumably what others are seeing on *BSD, from which Apple originally
inherited the Darwin TCP stack.
Single Unix Specification (SUSv2) from 1997
(yes, that is nearly 20 years ago):
http://pubs.opengroup.org/onlinepubs/007908799/xsh/poll.html
POLLHUP
The device has been disconnected. This event and POLLOUT are
mutually exclusive; a stream can never be writable if a hangup has
occurred. However, this event and POLLIN, POLLRDNORM, POLLRDBAND or
POLLPRI are not mutually exclusive. This flag is only valid in the
revents bitmask; it is ignored in the events member.
Updated version of The Open Group Base Specifications Issue 7
(published in 2013):
http://pubs.opengroup.org/onlinepubs/9699919799/
POLLHUP
A device has been disconnected, or a pipe or FIFO has been closed
by the last process that had it open for writing. Once set, the
hangup state of a FIFO shall persist until some process opens the
FIFO for writing or until all read-only file descriptors for the
FIFO are closed. This event and POLLOUT are mutually-exclusive;
a stream can never be writable if a hangup has occurred. However,
this event and POLLIN, POLLRDNORM, POLLRDBAND, or POLLPRI are not
mutually-exclusive. This flag is only valid in the revents bitmask;
it shall be ignored in the events member.
x-ref:
"1.4.40/41 mod_proxy, mod_scgi may trigger POLLHUP on *BSD,Darwin"
https://redmine.lighttpd.net/issues/2743
disable stat_cache if server.follow-symlink = "disable"
if server.stat-cache-engine = "simple". Caching is still enabled
for server.stat-cache-engine = "fam" since the FAM notification is
almost immediate, however there is still a small race condition.
NOTE: server.follow-symlink = "disable" implementation still has
time-of-check versus time-of-use (ToC-ToU) race conditions and
its use is *not recommended* except to discourage symlinking.
It *does not* prevent symlinking by a determined attacker with
the ability to create files on the server.
server.stat-cache-engine = "disable" can also be used to discourage
symlinking, and also does not eliminate ToC-ToU race conditions.
While more modern systems might use openat() and other *at() routines
to eliminate the ToC-ToU race conditions, this is not currently
implemented in lighttpd. Besides, for systems needing such
protections against actors able to modify local files, it would be
better to set up multiple lighttpd servers running in separate user
contexts with filesystem permissions preventing access, rather than
giving a single lighttpd server running under a single lighttpd user
access to files across security boundaries, and trying to prevent
access by lighttpd user if a file is a symlink.
Note that there are performance implications to setting either of
server.follow-symlink = "disable"
server.stat-cache-engine = "disable"
since stat cache normally reduces filesystem overhead for
frequently-accessed files.
x-ref:
"security: stat cache *very large* race condition if caching when
follow_symlink disabled"
https://redmine.lighttpd.net/issues/2724
server.username can not be root or 0.
server.groupname can not be root or 0.
If server.username is set, previous behavior might retain gid 0
if server.groupname was not set.
New behavior calls setgid() on server.username primary gid, and
then initgroups on server.username if server.username is set but
server.groupname is not set.
x-ref:
"server.groupname not required with server.username"
https://redmine.lighttpd.net/issues/2725
If server.upload-dirs is not configured, then attempt to use TMPDIR
from the environment, if set, or else use /var/tmp which is not often
a tmpfs, unlike /tmp. Warn at startup if tempdirs are not present.
If server.upload-dirs is not configured, then attempt to use TMPDIR
from the environment, if set, or else use /tmp. Warn at startup if
tempdirs are not present.
try AF_INET after AF_INET6 if server.use-ipv6 = "enable" and
getaddrinfo() fails EAI_ADDRFAMILY when hints.ai_family is AF_INET6.
(Prefer IPv6 instead of setting hinst.ai_family to AF_UNSPEC since
lighttpd only uses the first address returned)
read all available records from SSL_read(), even if larger than
MAX_READ_LIMIT, since the data is already in memory. openssl is
configured with SSL_MODE_RELEASE_BUFFERS and will release openssl
buffers once records have been read.
Without reading available data, there was a chance that the connection
would hang waiting for a read event on the fd, even though all the
data had already been read from kernel socket buffers and was in openssl
memory waiting to be read with SSL_read().
(thx glen and avij)
reverts part of commit:dbdab5db which swapped REQUEST_URI, REDIRECT_URI
x-ref:
"mediawiki redirect loop if REQUEST_URI not orig req in 1.4.40"
https://redmine.lighttpd.net/issues/2738
Explanation:
REQUEST_URI and REDIRECT_URI are not part of CGI standard environment.
The reason for their existence is that PATH_INFO in CGI environment may
be different from the path in the current request. The main reason for
this potential difference is that the URI path is normalized to a path
in the filesystem and tested against the filesystem to determine which
part is SCRIPT_NAME and which part is PATH_INFO. In case-insensitive
filesystems, the URI might be lowercased before testing against the
filesystem, leading to loss of case-sensitive submission in any
resulting PATH_INFO. Also, duplicated slashes "///" and directory
references "/." and "/.." are removed, including prior path component in
the case of "/..". This might be undesirable when the information after
the SCRIPT_NAME is virtual information and there target script needs the
virtual path preserved as-is. In that case, the target script can
re-parse REQUEST_URI (or REDIRECT_URI, as appropriate) to obtain the
unmodified information from the URI.
con->request.uri is equivalent to con->request.orig_uri unless the
request has been internally rewritten (e.g. by mod_rewrite, mod_magnet,
others), in which case con->request.orig_uri is the request made by the
client, and con->request.uri is the current URI being processed.
Historical REQUEST_URI (environment variable) lighttpd inconsistencies
- mod_cml set REQUEST_URI to con->request.orig_uri
- mod_cgi set REQUEST_URI to con->request.orig_uri
- mod_fastcgi set REQUEST_URI to con->request.orig_uri
- mod_scgi set REQUEST_URI to con->request.orig_uri
- mod_ssi set REQUEST_URI to current con->request.uri
- mod_magnet set MAGNET_ENV_REQUEST_URI to current con->request.uri
and MAGNET_ENV_REQUEST_ORIG_URI to con->request.orig_uri
Historical REDIRECT_URI (environment variable) previously set only in
mod_fastcgi and mod_scgi, and set to con->request.uri
Since lighttpd 1.4.40 provides REDIRECT_URI with con->request.orig_uri,
changes were made to REQUEST_URI for consistency, with the hope that
there would be little impact to existing configurations since the
request uri and original request uri are the same unless there has been
an internal redirect. It turns out that various PHP frameworks use
REQUEST_URI and require that it be the original URI requested by client.
Therefore, this change is being reverted, and lighttpd will set
REQUEST_URI to con->request.orig_uri in mod_cgi, mod_fastcgi, mod_scgi
as was done in lighttpd 1.4.39 and earlier. Similarly, REDIRECT_URI
also has the prior behavior in mod_fastcgi and mod_scgi, and added to
mod_cgi.
A future release of lighttpd might change mod_ssi to be consistent with
the other modules in setting REQUEST_URI to con->request.orig_uri and to
add REDIRECT_URI, when an internal redirect has occurred.
[core] $HTTP["remoteip"] must handle IPv6 w/o [] (existing behavior)
This was inadvertently broken in lighttpd 1.4.40 when IP address
normalization was added.
In $HTTP["remoteip"], IPv6 is now accepted with or without '[]'.
http_request_host_normalize() expects IPv6 with '[]', and config
processing at runtime expects COMP_HTTP_REMOTE_IP compared without '[]',
so '[]' is stripped (internally) after normalization
Do not switch to CON_STATE_ERROR upon idle timeout if already in
CON_STATE_CLOSE. Changing to CON_STATE_ERROR might keep resetting
con->close_timeout_ts if repeated calls to shutdown() succeed.
clock_gettime() needs -lrt with glibc < 2.17,
and possibly other platforms
This commit contains fixes for CMake and SCONS
See also commit:4d920466 which updated configure.ac for same
x-ref:
"1.4.40 compiling issuses on Debian Wheezy"
https://redmine.lighttpd.net/issues/2737
Strip bogus "Proxy" header before creating subprocess environment.
(mod_cgi, mod_fastcgi, mod_scgi, mod_ssi, mod_proxy)
Do not emit HTTP_PROXY to subprocess environment.
Some executables use HTTP_PROXY to configure outgoing proxy.
This is not a lighttpd security issue per se, but this change to
lighttpd adds a layer of defense to protect backend processes which
might be vulnerable due to blindly using this untrusted environment
variable. The HTTP_PROXY environment variable should not be trusted
by a program running in a CGI-like environment.
Mitigation in lighttpd <= 1.4.40 is to reject requests w/ Proxy header:
* Create "/path/to/deny-proxy.lua", read-only to lighttpd, with content:
if (lighty.request["Proxy"] == nil) then return 0 else return 403 end
* Modify lighttpd.conf to load mod_magnet and run lua code
server.modules += ( "mod_magnet" )
magnet.attract-raw-url-to = ( "/path/to/deny-proxy.lua" )
References:
https://www.kb.cert.org/vuls/id/797896
CGI web servers assign Proxy header values from client requests to
internal HTTP_PROXY environment variables
https://httpoxy.org/
httpoxy: A CGI application vulnerability
Glenn Strauss
Stefan Bühler