Commit Graph

254 Commits (31011adda99d938a28f78944f34bd85eb6701b6e)

Author SHA1 Message Date
Glenn Strauss 371e1bf723 [mod_extforward] support Forwarded HTTP Extension (#2703)
enable with, e.g.:
extforward.headers = ( "Forwarded" )
extforward.headers = ( "Forwarded", "X-Forwarded-For" )
extforward.headers = ( "Forwarded", "X-Forwarded-For", "Forwarded-For" )

The default remains:
extforward.headers = ( "X-Forwarded-For", "Forwarded-For" )

Support for "Forwarded" is not enabled by default since intermediate
proxies might not be aware of Forwarded, and might therefore pass
spoofed Forwarded header received from client.

extforward.params = ( # overwrite "Host" with Forwarded value
                      #"host" => 1
                      # set REMOTE_USER with Forwarded value
                      #"remote_user" => 1
Note: be cautious configuring trusted proxies if enabling these options
since Forwarded header may be spoofed and passed along indescriminantly
by proxies which do not handle Forwarded.

To remove "Forwarded" from incoming requests, do not enable these
options and instead use mod_setenv to clear the request header:
  setenv.set-request-header = ( "Forwarded" => "" )

Other proxy-related headers which admin might evaluate to keep or clear:
  setenv.set-request-header = ( "X-Forwarded-For" => "",
                                "X-Forwarded-By" => "",
                                "X-Forwarded-Server" => "",
                                "X-Origin-IP" => "",
                                "Via" => "",

  "Forwarded HTTP Extension"
  "Forward authenticated user to proxied requests"
2017-04-06 00:22:41 -04:00
Glenn Strauss aa14493e47 [tests] reduce time waiting for backends to start
reduce time spent waiting for backends to start

tests check for active listening port before proceeding

test runs now complete in about 2/3 the time
2017-03-19 23:48:50 -04:00
Glenn Strauss 0a635fc8be [core] consolidate dynamic handler response parse
- consolidate dynamic handler HTTP response parsing code
- reduce string copies for CGI, FastCGI, SCGI, proxy response headers
- let read() signal EOF or EAGAIN instead of ioctl FIONREAD 0-data-ready
2017-03-19 23:48:50 -04:00
Glenn Strauss 57ab20ace5 [mod_cgi] cgi.local-redir = [enable|disable] (#2108, #2793)
new directive cgi.local-redir = [enable|disable]

*disable* RFC3875 6.2.2 local-redir by default.
(behavior change from when local-redir support added in lighttpd 1.4.40)

The reason for this behavior change is that CGI local-redir support
(RFC3875 6.2.2) is an optimization.  Absence of support may result in
additional latency in servicing a request due the additional round-trip
to the client, but that was the prior behavior (before lighttpd 1.4.40)
and is the behavior of web servers which do not support CGI local-redir.

However, enabling CGI local-redir by default may result in broken links
in the case where a user config (unaware of CGI local-redir behavior)
returns HTML pages containing *relative* paths (not root-relative paths)
which are relative to the location of the local-redir target document,
and the local-redir target document is located at a different URL-path
from the original CGI request.

  RFC3875 CGI 1.1 specification section 6.2.2 Local Redirect Response
  "CGI local redirect not implemented correctly"
  "1.4.40 regression: broken redirect (using Location) between url.rewrite-once URLs"
2017-02-26 18:03:10 -05:00
Glenn Strauss ab077790f2 [tests] correct skip count for mod-scgi.t 2017-02-25 16:02:50 -05:00
Glenn Strauss 739ccb5de0 [tests] remove unused file depending on
lighttpd tests do not depend on

remove *unused* file tests/docroot/www/404.fcgi
which used CGI::Fast, which depends on
2017-01-31 14:36:16 -05:00
Glenn Strauss 4a68780e1b [mod_scgi] tests/mod-scgi.t unit tests
(copied from tests/mod-fastcgi.t fcgi-responder tests)
2017-01-31 14:36:15 -05:00
Glenn Strauss 37dac9a23c [core] support Expect: 100-continue with HTTP/1.1 (fixes #377, #1017, #1953, #2438)
support Expect: 100-continue with HTTP/1.1 requests

Ignore config option server.reject-expect-100-with-417;
server.reject-expect-100-with-417 will be removed in a future release.

  "Incorrect handling of the 100 (Continue) Status"
  "'Expect' header gives HTTP error 417"
  "Improve DAV support to be able to handle git as a client"
  "Change server.reject-expect-100-with-417 from flag to regular expression matching the URL"
2017-01-31 14:36:15 -05:00
Glenn Strauss afce434e0b [mod_secdownload] new directives modify hash path (fixes #646, fixes #1904)
secdownload.path-segments = <number>
  include only given number of path segments in hash digest calculation

secdownload.hash-querystr = "enable" | "disable"
  include the query string in the hash digest calculation

  "secdownload.path_elements support"
  "mod_secdownload option to include url GET parameters in md5"
2017-01-31 14:36:15 -05:00
Glenn Strauss 4d92366ab2 [mod_setenv] directives to overwrite/remove hdrs (fixes #650, fixes #2295)
directives to set value, rather than append values to headers, env
These directives take precedence over the setenv.add-* counterparts

Set a blank value for request or response header to remove the header
(blank value in environment will be set as the value; not removed)

setenv.*-environment is now deferred to handle_request_env hook.
setenv.*-response-header is now processed in handle_response_start hook.

  "set-request-header or remove-request-header support for mod_setenv"
2017-01-31 14:36:15 -05:00
Glenn Strauss 83b2b71527 [tests] give time for periodic jobs to detect exit
give time for periodic jobs to detect backend exit
2017-01-10 19:18:36 -05:00
Glenn Strauss 16f171588f [tests] FCGI_Finish() final request before exit 2017-01-10 18:16:35 -05:00
Glenn Strauss 32443ea1c7 [tests] update test skip count for !fcgi-responder 2017-01-10 17:04:11 -05:00
Glenn Strauss 64df38aad8 [mod_fastcgi] detect child exit, restart proactively
(instead of detecting upon a subsequent HTTP request)

(for backends spawned by mod_fastcgi)
2017-01-10 07:54:47 -05:00
Glenn Strauss f57d8c54b4 [mod_cgi] skip local-redir handling if to self (fixes #2779, #2108)
Loosen local redirect handling in mod_cgi to skip handling as local
redirect if the Location matches con->uri.path, since if the request
is intended to redirect back to the same CGI using the same request
method, path info, and query string, the CGI would logically just
return the final intended response.  Loosening this handling avoids a
problem with applications (potentially) accessible through multiple
gateways, where the application is not aware of this specific handling
of Location in the Common Gateway Interface (CGI/1.1), the application
sends abs-path in the Location response header instead of absoluteURI,
and the application expects the client to receive this Location response
header instead of the server to process as a CGI local redirect.

One example of such an application is LuCI,
which sends Set-Cookie with Location: /abs-path

(Note that this loose check for matching con->uri.path is not perfect
 and might not match if the CGI returned a path with a different case
 and the server is on a case-insensitive filesystem, or if the path
 returned by the CGI is rewritten elsewhere to a different con->uri.path
 before getting to mod_cgi.)

RFC3875 CGI 1.1 specification section 6.2.2 Local Redirect Response

  "CGI local-redir handling conflicts with LuCI redirect w/ Set-Cookie"
  "CGI local redirect not implemented correctly"
2017-01-09 22:27:53 -05:00
Glenn Strauss 15bc2313a4 [tests] mark tests/docroot/www/*.pl scripts a+x 2016-12-22 05:40:37 -05:00
Glenn Strauss 0f4cc0d697 [mod_evhost] fix an incorrect error trace 2016-12-19 02:50:46 -05:00
Glenn Strauss 4d7f5737f1 [core] support Transfer-Encoding: chunked req body (fixes #2156)
support Transfer-Encoding: chunked request body in conjunction with = 0

dynamic handlers will still return 411 Length Required if = 1 or 2 (!= 0)
  since CGI-like env requires CONTENT_LENGTH be set
  (and mod_proxy currently sends HTTP/1.0 requests to backends,
   and Content-Length recommended for robust interaction with backend)

  "request: support Chunked Transfer Coding for HTTP PUT"
2016-12-16 16:58:04 -05:00
Glenn Strauss 7d339e21db comment out auth.backend.ldap.* in tests/*.conf
(mod_authn_ldap is not loaded in these test confs,
 so mod_authn_ldap directives are not available)
2016-12-03 03:56:41 -05:00
Glenn Strauss f664e77909 load mod_auth & mod_authn_file in sample/test.conf 2016-11-29 22:32:28 -05:00
Glenn Strauss 75040e9988 [mod_evhost] mod-evhost.t tests (#1194)
(thx Daniel-Brandt)

  "Partial matching in mod_evhost patterns"
2016-10-20 14:22:46 -04:00
Glenn Strauss b11d059843 [cmake] build fcgi-auth, fcgi-responder for tests
Aside: must have cmake enable building openssl for tests to pass
due to tests/lighttpd.conf including config options requiring openssl
algorithms in mod_secdownload.c:
  (secdownload.algorithm       = "hmac-sha1")
  (secdownload.algorithm       = "hmac-sha256")

$ cmake -L .
$ make -j 4 -k
$ make test

2016-10-18 17:47:33 -04:00
Glenn Strauss e9ee22c204 [autobuild] skip two new tests if no fcgi-auth 2016-09-23 00:58:41 -04:00
Glenn Strauss 7ba06c71a6 [mod_auth] structured data, register auth schemes
- parse auth.* directives into structured data during config processing
- register auth schemes (basic, digest, extern, ...) for extensibility
- remove auth.debug directive
2016-09-22 19:54:57 -04:00
Glenn Strauss 7b7350ee19 [mod_fastcgi] allow authorizer, responder for same path/ext (#321)
allow authorizer and responder to be configured for same path or ext

  "mod_fastcgi authorizers cannot protect fastcgi responders"
2016-09-19 20:12:28 -04:00
Christoph Kreutzer 7ef569b204 [tests] test coverage for issues (#321, #322)
FastCGI Authorizer support with FastCGI Responders

  "mod_fastcgi authorizers cannot protect fastcgi responders"

  "FastCGI Authorizer support for Variable-name variable passing"
2016-09-19 20:02:02 -04:00
Glenn Strauss 6ec66c4dce [core] better DragonFlyBSD support (fixes #2746)
(thx xenu)

  "[PATCH] better DragonFlyBSD support; fix crash"
2016-08-20 14:19:10 -04:00
Glenn Strauss 4bc06bfc0b [core] check if client half-closed TCP if POLLHUP (#2743)
Check if client half-closed TCP connection if POLLHUP is received.
This more robustly handles if client called shutdown(fd, SHUT_WR).

This patch reverts commit:ab05eb7c which should now be handled properly.
(Time will tell.)

  "1.4.40/41 mod_proxy, mod_scgi may trigger POLLHUP on *BSD,Darwin"
2016-08-06 04:28:45 -04:00
Glenn Strauss 9af58a9716 revert 1.4.40 swap of REQUEST_URI, REDIRECT_URI (fixes #2738)
reverts part of commit:dbdab5db which swapped REQUEST_URI, REDIRECT_URI

  "mediawiki redirect loop if REQUEST_URI not orig req in 1.4.40"


REQUEST_URI and REDIRECT_URI are not part of CGI standard environment.
The reason for their existence is that PATH_INFO in CGI environment may
be different from the path in the current request.  The main reason for
this potential difference is that the URI path is normalized to a path
in the filesystem and tested against the filesystem to determine which
part is SCRIPT_NAME and which part is PATH_INFO.  In case-insensitive
filesystems, the URI might be lowercased before testing against the
filesystem, leading to loss of case-sensitive submission in any
resulting PATH_INFO.  Also, duplicated slashes "///" and directory
references "/." and "/.." are removed, including prior path component in
the case of "/..".  This might be undesirable when the information after
the SCRIPT_NAME is virtual information and there target script needs the
virtual path preserved as-is.  In that case, the target script can
re-parse REQUEST_URI (or REDIRECT_URI, as appropriate) to obtain the
unmodified information from the URI.

con->request.uri is equivalent to con->request.orig_uri unless the
request has been internally rewritten (e.g. by mod_rewrite, mod_magnet,
others), in which case con->request.orig_uri is the request made by the
client, and con->request.uri is the current URI being processed.

Historical REQUEST_URI (environment variable) lighttpd inconsistencies
- mod_cml     set REQUEST_URI to con->request.orig_uri
- mod_cgi     set REQUEST_URI to con->request.orig_uri
- mod_fastcgi set REQUEST_URI to con->request.orig_uri
- mod_scgi    set REQUEST_URI to con->request.orig_uri

- mod_ssi     set            REQUEST_URI to current con->request.uri
- mod_magnet  set MAGNET_ENV_REQUEST_URI to current con->request.uri
              and MAGNET_ENV_REQUEST_ORIG_URI to con->request.orig_uri

Historical REDIRECT_URI (environment variable) previously set only in
mod_fastcgi and mod_scgi, and set to con->request.uri

Since lighttpd 1.4.40 provides REDIRECT_URI with con->request.orig_uri,
changes were made to REQUEST_URI for consistency, with the hope that
there would be little impact to existing configurations since the
request uri and original request uri are the same unless there has been
an internal redirect.  It turns out that various PHP frameworks use
REQUEST_URI and require that it be the original URI requested by client.

Therefore, this change is being reverted, and lighttpd will set
REQUEST_URI to con->request.orig_uri in mod_cgi, mod_fastcgi, mod_scgi
as was done in lighttpd 1.4.39 and earlier.  Similarly, REDIRECT_URI
also has the prior behavior in mod_fastcgi and mod_scgi, and added to

A future release of lighttpd might change mod_ssi to be consistent with
the other modules in setting REQUEST_URI to con->request.orig_uri and to
add REDIRECT_URI, when an internal redirect has occurred.
2016-07-23 02:13:41 -04:00
Glenn Strauss 00cc4d7c0e [mod_auth] fix Digest auth to be better than Basic (fixes #1844)
Make Digest authentication more compliant with RFC.

Excerpt from Section 5.13:
    The bottom line is that any compliant implementation will be
    relatively weak by cryptographic standards, but any compliant
    implementation will be far superior to Basic Authentication.

  "Serious security problem in Digest Authentication"
2016-07-16 23:25:53 -04:00
Glenn Strauss 8861c2bb54 [mod_cgi] handle local redirect response (fixes #2108)
RFC3875 CGI 1.1 specification section 6.2.2 Local Redirect Response

  "CGI local redirect not implemented correctly"
2016-07-14 16:31:08 -04:00
Glenn Strauss bcddcf8b0e [tests] remove some tests duplicated in mod-cgi.t 2016-07-14 13:21:43 -04:00
Glenn Strauss ce7d040bf3 [mod_access] new directive url.access-allow (fixes #1421)
url.access-allow is list of allowed url suffixes (e.g. file extensions)
If url.access-allow has been set, then deny any URL that does not match
the explicitly listed suffixes.

(thx japc)

  "access_allow directive for lighttpd"
2016-07-13 04:12:08 -04:00
Glenn Strauss d147673d40 [tests] remove dependency on is no longer shipped as part of Perl core distribution
(and is easily replaced)
2016-07-02 13:47:22 -04:00
Glenn Strauss 8dcbd61a45 [cygwin] fix mod_proxy and mod_fastcgi ioctl use
cygwin does not support ioctl on sockets, returning EOPTNOTSUPP
(would be better if cygwin used Windows ioctlsocket() instead)

Windows uses signed (socklen_t), so add some casts to quiet warnings

Windows path handling is convoluted, so disable one tests in mod_fastcgi
since trailing spaces are removed from URL for _WIN32 and __CYGWIN__ in
2016-06-23 15:46:44 -04:00
Glenn Strauss 16a3f3b6fc [build] update EXTRA_DIST w/ new files 2016-06-21 04:46:04 -04:00
Glenn Strauss a5fcfee6fc [mod_ssi] more flexible quoting (fixes #1768)
allow double-quotes, single-quotes or no quote on SSI param values

remove use of PCRE from mod_ssi

fix misspelling of 'unknow' to be 'unknown'

  "mod_ssi doesn't accept single quotes"
2016-05-14 05:50:37 -04:00
Glenn Strauss df146a7724 [mod_ssi] fix SSI statement parser
bug introduced in 8e3c6bf7 when statement parser was replaced
2016-05-14 01:17:52 -04:00
Glenn Strauss b47c393e26 [mod_auth] skip blank lines and comment lines (fixes #2327)
  "Ignore comments and empty lines in ht{digest,passwd} files"
2016-04-28 13:13:43 -04:00
Glenn Strauss dbdab5dbc9 [core] server.error-handler new directive for error pages (fixes #2702)
server.error-handler preserves HTTP status error code when error page
is static, and allows dynamic handlers to change HTTP status code
when error page is provided by dynamic handler.  server.error-handler
intercepts all HTTP status codes >= 400 except when the content is
generated by a dynamic handler (cgi, ssi, fastcgi, scgi, proxy, lua).
The request method is unconditionally changed to GET for the request
to service the error handler, and the original request method is
later restored (for logging purposes).  request body from the
original request, if present, is discarded.

server.error-handler is somewhat similar to server.error-handler-404,
but server.error-handler-404 is now deprecated, intercepts only 404
and 403 HTTP status codes, and returns 200 OK for static error pages,
a source of confusion for some admins.  On the other hand, the new
server.error-handler, when set, will intercept all HTTP status error
codes >= 400.  server.error-handler takes precedence over
server.error-handler-404 when both are set.

NOTE: a major difference between server.error-handler and the
now-deprecated server.error-handler-404 is that the values of the
non-standard CGI environment variables REQUEST_URI and REDIRECT_URI
have been swapped.  Since REDIRECT_STATUS is the original HTTP
status code, REDIRECT_URI is now the original request, and REQUEST_URI
is the current request (e.g. the URI/URL to the error handler).
The prior behavior -- which reversed REQUEST_URI and REDIRECT_URI values
from those described above -- is preserved for server.error-handler-404.

Additionally, REDIRECT_STATUS is now available to mod_magnet, which
continues to have access to request.uri and request.orig_uri.

See further discussion at

github: closes #36
2016-04-25 01:01:08 -04:00
Glenn Strauss 1c01a42aa3 [core] never evaluate else branches until the previous branches are aready (fixes #2598)
The first condition which evaluates true in any if-else... condition
chain short-circuits the chain, and any remaining conditions in the
chain are marked false.

Previous conditions in if-else condition chaining must be evaluatable
(to true or false) -- must not remain in unset (not yet evaluatable)
state -- prior to evaluating later conditions.  Since any true
condition short-circuits remaining conditions, all prev conditions
must be false prior to evaluating later conditions.

From: Glenn Strauss <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-02-21 17:56:24 +00:00
Stefan Bühler ab05eb7cec [tests] do not half-close socket before having received the response (fixes #2688)
From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2015-12-04 20:22:38 +00:00
Stefan Bühler 1566748b1a [tests] test apr-md5 in mod-auth.t
From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2015-12-04 20:11:35 +00:00
Kyle J. McKay 2bcb73cdb8 mod-auth.t: no crypt md5 for darwin
Darwin's crypt does not support the '$...' extensions.

Signed-off-by: Kyle J. McKay

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2015-12-04 20:11:33 +00:00
Stefan Bühler bfaa48260a [mod_secdownload] add required algorithm option; old behaviour available as "md5", new options "hmac-sha1" and "hmac-sha256"
Differential Revision:

From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2015-11-22 22:22:22 +00:00
Stefan Bühler 85d8a17575 [core] encode path with ENCODING_REL_URI in redirect to directory (fixes #2661, thx gstrauss)
From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2015-11-07 15:00:18 +00:00
Stefan Bühler 68284bd7f1 [scons] various improvements
- don't generate files in src/
- move all build stuff to sconsbuild/
- have different output directories for static/ and fullstatic/,
  so we can use that directory for the test suite
- each build type (dynamic, static, fullstatic) has its own check target
- read CFLAGS, LDFLAGS and LIBS from environment
  (LIBS are appended after all other dependencies)

From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2015-08-29 09:28:04 +00:00
Stefan Bühler 9290e923d9 [tests] fix warning about newline in filename
From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2015-08-22 21:39:22 +00:00
Stefan Bühler 87c5ec9651 [tests] search for perl in PATH instead of /usr/bin; whitespace + test config cleanups
From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2015-08-22 20:51:08 +00:00
Stefan Bühler 403afcd251 [cmake] lowercase commands, whitespace cleanup, remove clutter in else(...), endif(...), endforeach(...)
From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2015-08-22 16:01:02 +00:00