Commit Graph

2235 Commits (82501d24f23719e69c85479595eea28c992bae62)

Author SHA1 Message Date
Glenn Strauss 20a2a0d2e3 remove unused sys-mmap.h from stat_cache.c 2016-09-22 13:36:31 -04:00
Glenn Strauss 2af88b2357 [autobuild] test_configfile might need vector.c (fixes #2752)
needed to build on Solaris using Oracle Solaris Studio (thx petrs)

  "vector_realloc missing when linking test_configfile"
2016-09-22 13:36:31 -04:00
Glenn Strauss cb1a3c6299 backport mod_deflate to lighttpd 1.4 (fixes #1824, fixes #2753)
lots of fixes and improvements

limitations: see comments at top of mod_deflate.c

missing functionality: encode streaming response
  (module currently requires response be collected before being sent)

potential functionality: addition of compressed file cache would
  allow mod_deflate to fully supplant mod_compress in lighttpd 1.4.x

  "Adding mod_deflate to 1.4.xx"
  "mod_deflate backport compile error if ENABLE_MMAP not defined"

github: closes #67
2016-09-22 13:36:04 -04:00
Glenn Strauss 7b7350ee19 [mod_fastcgi] allow authorizer, responder for same path/ext (#321)
allow authorizer and responder to be configured for same path or ext

  "mod_fastcgi authorizers cannot protect fastcgi responders"
2016-09-19 20:12:28 -04:00
Glenn Strauss dc91e40657 dynamic handlers store debug flag in handler_ctx
(for persistence across multiple re-entries into routines upon
 receiving fdevent)

(setting module debug flag in global scope is still recommended
 since there are places where p->conf.debug is checked since
 handler_ctx might not be available at all points)
2016-09-19 20:02:02 -04:00
Christoph Kreutzer 7ef569b204 [tests] test coverage for issues (#321, #322)
FastCGI Authorizer support with FastCGI Responders

  "mod_fastcgi authorizers cannot protect fastcgi responders"

  "FastCGI Authorizer support for Variable-name variable passing"
2016-09-19 20:02:02 -04:00
Glenn Strauss 2dcfe1733e [mod_fastcgi] Authorizer support with Responder (fixes #321, fixes #322)
import Variable-* from FastCGI authorizer response into con->environment
restart request after FastCGI authorizer if no fastcgi.server docroot

(thx Christoph Kreutzer for initial patch attempt)

  "mod_fastcgi authorizers cannot protect fastcgi responders"

  "FastCGI Authorizer support for Variable-name variable passing"

github: closes #70
2016-09-19 20:02:02 -04:00
Glenn Strauss 5dfe21acc9 [mod_geoip] add to default build (fixes #2705, fixes #2101, fixes #2092, fixes #2025, fixes #1962, fixes #1938)
(add to default build to reduce distributor package maintenance)

  "broken module API since 1.4.38"
  "lighttpd-1.4.24 fails to compile with mod_geoip.c"
  "unsafe sprintfs mod_geoip"
  "mod_geoip crashes lighttpd 1.5.x on FreeBSD 7.2 AMD64"
  "lighttpd 1.4 crashes on FreeBSD 7.0 AMD64 when mod_geoip compiled in"
2016-09-13 02:49:00 -04:00
Glenn Strauss ab935a2b96 [mod_uploadprogress] add to default build
(module is distributed in Gentoo)
(add to default build to reduce distributor package maintenance)
2016-09-12 02:53:11 -04:00
Glenn Strauss b9f245f263 [mod_cgi] permit CGI exec of unreadable files (fixes #2374)
CGI target might be executable (+x), but not readable (-r)

  "lighttpd-1.4.29 cannot execute unreadable CGIs"
2016-09-11 11:44:02 -04:00
Glenn Strauss ae91578895 [mod_auth] mod_authn_mysql.c MySQL auth backend (fixes #752, fixes #1845)
(automatically load mod_authn_mysql with mod_auth for compatibility with
 existing config usage via patches in various distros, e.g. FreeBSD)

  "mySQL auth"
  "MySQL Digest Authentication"
2016-09-11 10:59:05 -04:00
Glenn Strauss 17b2a38721 [mod_auth] remove empty mod_auth.h 2016-09-09 22:31:46 -04:00
Glenn Strauss cde68b7b23 [mod_auth] http_auth_md5_hex2bin()
Note: http_auth_backend_t digest interface returns result as a
binary MD5 (16-bytes) so that caller consistently converts to
lowercase before using it in further digest calculation.

(Alternatively, the http_auth_backend_t digest interface could have
 taken a 33-char buffer and returned an explicitly lowercased hex str)
2016-09-09 22:28:01 -04:00
Glenn Strauss ede5ea2d83 fix mis-cast in unused code
For correctness, fix cast to (fdnode *) in #ifdef __WIN32 code.
(lighttpd compiles under cygwin, but not under native _WIN32)

(thx ToApolytoXaos)
2016-08-25 02:01:33 -04:00
Glenn Strauss 40f16d52db [core] fix crash if ready events on abandoned fd (fixes #2748)
  "1.4.40/1.4.41 uploads to CGI may cause crash (SIGABRT)"
2016-08-24 15:30:11 -04:00
Glenn Strauss 3c24ec7393 [mod_auth] terminate salt for CRYPT-MD5-NTLM 2016-08-21 16:00:46 -04:00
Glenn Strauss 65efc2eda8 [mod_auth] support CRYPT-MD5-NTLM algorithm (fixes #1743)
(based on patch submitted in #1743)
(minimally tested using example in #1743 with password 'test')

  "[PATCH] Add support for CRYPT-MD5-NTLM"
2016-08-21 01:49:09 -04:00
Glenn Strauss 9e7083582d [mod_auth] include base.h for USE_OPENSSL def 2016-08-20 20:38:47 -04:00
Glenn Strauss 6ec66c4dce [core] better DragonFlyBSD support (fixes #2746)
(thx xenu)

  "[PATCH] better DragonFlyBSD support; fix crash"
2016-08-20 14:19:10 -04:00
Glenn Strauss b22269c2f3 [mod_auth] extensible interface for auth backends
Merge branch 'feature-auth-reorg' into gmaster
2016-08-20 13:43:27 -04:00
Glenn Strauss 4b3a91e64b [mod_auth] extensible interface for auth backends
create new, extensible interface for (additional) auth backends

attempt to handle HANDLER_WAIT_FOR_EVENT returned by auth backends
to allow for async auth backends (e.g. to mysql database)

separate auth backends from mod_auth and http_auth
  mod_authn_file.c htdigest, htpasswd, plain auth backends
  mod_authn_ldap.c ldap auth backend
add http_auth.c to common_sources for auth backend registration

(mod_authn_file could be three separate modules, but no need for now)
2016-08-20 13:42:08 -04:00
Glenn Strauss 3dcca966f4 [mod_auth] refactor out auth backend code
separate routines for each auth backend in http_auth.c,
move ldap backend init from mod_auth.c to http_auth.c
2016-08-18 10:18:14 -04:00
Glenn Strauss 81b2d1f020 [mod_auth] refactor out auth backend code
move basic and digest code into mod_auth.c,
and leave auth backend code in http_auth.c
2016-08-18 10:16:01 -04:00
Glenn Strauss 31250a9af8 [mod_auth] refactor out auth backend code
separate subroutines in http_auth.c
2016-08-14 13:15:08 -04:00
Glenn Strauss cb24958c01 [mod_auth] Digest auth fails after rewrite (fixes #2745)
(affects lighttpd 1.4.41)

  "HTTP digest + rewrite fails with: digest: auth failed: uri mismatch (1.4.41)"
2016-08-13 14:07:36 -04:00
Glenn Strauss cfa3d27fc3 [mod_dirlisting] js column sort for dirlist table (fixes #613, fixes #2315)
copied javascript from mod_status and from lighttpd2 mod_dirlist

modified and specialized for stable dirlist sorting by name

Partial implementation of Apache autoindex request query arguments
If query string is supplied, allow specifying initial column to sort
  ?C=N name (default)
  ?C=M last-modified, then by name
  ?C=S size, then by name
  ?C=T type, then by name
  ?C=D type, then by name
and O=[AD] can be added for descending or ascending order, e.g.
  ?C=N&O=D descending (default)
  ?C=N&O=A ascending

(While functional, no effort was made on js performance.
 Patches welcome)

New directive dir-listing.external-js for user to replace sorting js

Note: dir-listing.external-js or default js sorting is enabled only
if = "enable" (which is the default)

  "client-selectable directory list sorting"
2016-08-11 00:27:11 -04:00
Glenn Strauss 09a663b95b [mod_dirlisting] dirlist does not handle POST 2016-08-10 04:13:10 -04:00
Glenn Strauss 27f85dbdf4 [core] proxy,scgi omit shutdown() to backend (fixes #2743)
Due to the POLLHUP behavior triggered on *BSD/Darwin, the shutdown()
had previously been limited to local connections.  If interested in
squeezing the last bits of performance out of a machine, an admin
should configure local connections to be AF_UNIX instead of AF_INET
or AF_INET6 to localhost.  The reason the shutdown() was originally
added in mod_proxy and mod_scgi was to aggressively reduce the number
of potential sockets in TIME_WAIT held by lighttpd.
(See commit:923688d2 "drain backend socket/pipe bufs upon FDEVENT_HUP",
 done for reliability given the aforementioned *BSD/Darwin behavior.)
When using AF_UNIX, the TIME_WAIT issue does not exist, ergo, the
recommendation is to use AF_UNIX for local sockets, when available.
Using AF_UNIX sockets is a better solution to eliminate TIME_WAIT
than is TCP shutdown() half-close which, as we have seen, might not
be handled well by frameworks which are more complex than basic read
request, send response, and close.

  "1.4.40/41 mod_proxy, mod_scgi may trigger POLLHUP on *BSD,Darwin"
2016-08-07 13:09:21 -04:00
Glenn Strauss 7e2090b96b [core] do not enter handler twice after read body
do not enter handler a second time in connection_state_machine()
after read body completes if dynamic handler is still waiting for event
2016-08-07 07:35:47 -04:00
Glenn Strauss 666b9fd726 [core] enforce wait for POLLWR after EINPROGRESS (fixes #2744)
mod_fastcgi, mod_scgi, and mod_proxy must enforce wait for POLLWR
after EINPROGRESS or else getsockopt(fd, SOL_SOCKET, SO_ERROR, ...)
may succeed even though socket connection is not yet established,
and subsequent writev() will fail ENOTCONN.

(thx pkubaj)

 "1.4.40/41 writev failed: Socket is not connected (fastcgi,scgi,proxy)"
2016-08-07 00:42:58 -04:00
Glenn Strauss 4bc06bfc0b [core] check if client half-closed TCP if POLLHUP (#2743)
Check if client half-closed TCP connection if POLLHUP is received.
This more robustly handles if client called shutdown(fd, SHUT_WR).

This patch reverts commit:ab05eb7c which should now be handled properly.
(Time will tell.)

  "1.4.40/41 mod_proxy, mod_scgi may trigger POLLHUP on *BSD,Darwin"
2016-08-06 04:28:45 -04:00
Glenn Strauss 1de652f40b [mod_proxy,mod_scgi] shutdown remote only if local (#2743)
shutdown(fd, SHUT_WR) after sending request to proxy or SCGI
only if remote is local and platform is not *BSD or Darwin.

The reason this fix is special-casing *BSD and Darwin is that the Single
Unix Specification and POSIX.1-2013 clearly specify that POLLHUP event
should be returned by poll only when the stream is no longer writable.
A half-closed socket that is still writable clearly does not match that
condition, yet that is what I am seeing on Darwin (El Capitan), and
presumably what others are seeing on *BSD, from which Apple originally
inherited the Darwin TCP stack.

Single Unix Specification (SUSv2) from 1997
(yes, that is nearly 20 years ago):

    The device has been disconnected. This event and POLLOUT are
    mutually exclusive; a stream can never be writable if a hangup has
    occurred. However, this event and POLLIN, POLLRDNORM, POLLRDBAND or
    POLLPRI are not mutually exclusive. This flag is only valid in the
    revents bitmask; it is ignored in the events member.

Updated version of The Open Group Base Specifications Issue 7
(published in 2013):

    A device has been disconnected, or a pipe or FIFO has been closed
    by the last process that had it open for writing. Once set, the
    hangup state of a FIFO shall persist until some process opens the
    FIFO for writing or until all read-only file descriptors for the
    FIFO are closed.  This event and POLLOUT are mutually-exclusive;
    a stream can never be writable if a hangup has occurred. However,
    this event and POLLIN, POLLRDNORM, POLLRDBAND, or POLLPRI are not
    mutually-exclusive. This flag is only valid in the revents bitmask;
    it shall be ignored in the events member.

  "1.4.40/41 mod_proxy, mod_scgi may trigger POLLHUP on *BSD,Darwin"
2016-08-06 02:24:54 -04:00
Glenn Strauss 156bea3859 [TLS] SSL_shutdown() only if handshake finished
avoid noise in logs due to calling SSL_shutdown() on a connection
that has not yet completed TLS handshake
2016-08-02 22:32:28 -04:00
Glenn Strauss ccd817d3c9 - next is 1.4.42 2016-07-31 08:40:41 -04:00
Glenn Strauss 29fa805695 [doc] NEWS 2016-07-31 02:41:20 -04:00
Glenn Strauss fbae795dfa [cmake] set cmake_minimum_required to 2.8.2
CHECK_SYMBOL_EXISTS() is available in CMake >= 2.8.0
Clang is supported in CMake >= 2.8.2
2016-07-31 02:28:58 -04:00
Stefan Bühler 46b0e01217 [cmake] enable warnings for GCC and Clang
Also set -Wno-cast-align for lemon; lemon is only the parser generator, either
it crashes or it works.
2016-07-30 23:42:57 -04:00
Stefan Bühler f7b3745552 [cmake] always define _GNU_SOURCE
first.h only defines _GNU_SOURCE if no config.h is present.
2016-07-30 14:20:52 +02:00
Glenn Strauss 5863d05ec1 [security] encode quoting chars in HTML and XML
(affects mod_dirlisting, mod_ssi, mod_status)
2016-07-30 04:11:21 -04:00
Glenn Strauss 375022a1d1 fix buffer.c comments to match encoded_chars_*
fix buffer.c comments to match encoded_chars_* changes made in 3943de28
2016-07-30 04:02:21 -04:00
Glenn Strauss ebf3af8b12 [core] fix buffer_copy_string_hex() assert (fixes #2742)
fix buffer_copy_string_hex() passing incorrect length to li_tohex()

(thx Isibaar)

  "Assert wrongly triggered in buffer_copy_string_hex()"
2016-07-30 02:48:20 -04:00
Glenn Strauss acd5e450b5 [security] disable stat_cache if !follow-symlink (fixes #2724)
disable stat_cache if server.follow-symlink = "disable"
if server.stat-cache-engine = "simple".  Caching is still enabled
for server.stat-cache-engine = "fam" since the FAM notification is
almost immediate, however there is still a small race condition.

NOTE: server.follow-symlink = "disable" implementation still has
time-of-check versus time-of-use (ToC-ToU) race conditions and
its use is *not recommended* except to discourage symlinking.
It *does not* prevent symlinking by a determined attacker with
the ability to create files on the server.

server.stat-cache-engine = "disable" can also be used to discourage
symlinking, and also does not eliminate ToC-ToU race conditions.

While more modern systems might use openat() and other *at() routines
to eliminate the ToC-ToU race conditions, this is not currently
implemented in lighttpd.  Besides, for systems needing such
protections against actors able to modify local files, it would be
better to set up multiple lighttpd servers running in separate user
contexts with filesystem permissions preventing access, rather than
giving a single lighttpd server running under a single lighttpd user
access to files across security boundaries, and trying to prevent
access by lighttpd user if a file is a symlink.

Note that there are performance implications to setting either of
  server.follow-symlink = "disable"
  server.stat-cache-engine = "disable"
since stat cache normally reduces filesystem overhead for
frequently-accessed files.

  "security: stat cache *very large* race condition if caching when
follow_symlink disabled"
2016-07-30 02:10:44 -04:00
Glenn Strauss 558bfc4e1e [security] ensure gid != 0 if server.username set (fixes #2725)
server.username can not be root or 0.
server.groupname can not be root or 0.

If server.username is set, previous behavior might retain gid 0
if server.groupname was not set.

New behavior calls setgid() on server.username primary gid, and
then initgroups on server.username if server.username is set but
server.groupname is not set.

  "server.groupname not required with server.username"
2016-07-30 02:10:01 -04:00
Glenn Strauss f7410da5d2 [core] set chunkqueue tempdirs at startup /var/tmp
If server.upload-dirs is not configured, then attempt to use TMPDIR
from the environment, if set, or else use /var/tmp which is not often
a tmpfs, unlike /tmp.  Warn at startup if tempdirs are not present.
2016-07-29 15:01:46 -04:00
Glenn Strauss ad6d41896e [core] check if EAI_ADDRFAMILY is defined
(EAI_ADDRFAMILY is not available on FreeBSD)
2016-07-29 12:48:00 -04:00
Glenn Strauss c8e647ad31 [core] set chunkqueue tempdirs at startup
If server.upload-dirs is not configured, then attempt to use TMPDIR
from the environment, if set, or else use /tmp.  Warn at startup if
tempdirs are not present.
2016-07-28 03:57:52 -04:00
Glenn Strauss a62bff9866 [core] fix result copy from getaddrinfo()
(thx avij)
2016-07-27 22:26:32 -04:00
Glenn Strauss a69a803e35 [core] try AF_INET after AF_INET6 if use-ipv6
try AF_INET after AF_INET6 if server.use-ipv6 = "enable" and
getaddrinfo() fails EAI_ADDRFAMILY when hints.ai_family is AF_INET6.
(Prefer IPv6 instead of setting hinst.ai_family to AF_UNSPEC since
lighttpd only uses the first address returned)
2016-07-27 15:37:46 -04:00
Glenn Strauss a95aaa9de9 [TLS] read all available records from SSL_read()
read all available records from SSL_read(), even if larger than
MAX_READ_LIMIT, since the data is already in memory.  openssl is
configured with SSL_MODE_RELEASE_BUFFERS and will release openssl
buffers once records have been read.

Without reading available data, there was a chance that the connection
would hang waiting for a read event on the fd, even though all the
data had already been read from kernel socket buffers and was in openssl
memory waiting to be read with SSL_read().

(thx glen and avij)
2016-07-27 06:00:44 -04:00
Glenn Strauss bce293e4a7 [TLS] better handling of SSL_ERROR_WANT_READ/WRITE
2016-07-27 02:24:53 -04:00