reverts part of commit:dbdab5db which swapped REQUEST_URI, REDIRECT_URI
x-ref:
"mediawiki redirect loop if REQUEST_URI not orig req in 1.4.40"
https://redmine.lighttpd.net/issues/2738
Explanation:
REQUEST_URI and REDIRECT_URI are not part of CGI standard environment.
The reason for their existence is that PATH_INFO in CGI environment may
be different from the path in the current request. The main reason for
this potential difference is that the URI path is normalized to a path
in the filesystem and tested against the filesystem to determine which
part is SCRIPT_NAME and which part is PATH_INFO. In case-insensitive
filesystems, the URI might be lowercased before testing against the
filesystem, leading to loss of case-sensitive submission in any
resulting PATH_INFO. Also, duplicated slashes "///" and directory
references "/." and "/.." are removed, including prior path component in
the case of "/..". This might be undesirable when the information after
the SCRIPT_NAME is virtual information and there target script needs the
virtual path preserved as-is. In that case, the target script can
re-parse REQUEST_URI (or REDIRECT_URI, as appropriate) to obtain the
unmodified information from the URI.
con->request.uri is equivalent to con->request.orig_uri unless the
request has been internally rewritten (e.g. by mod_rewrite, mod_magnet,
others), in which case con->request.orig_uri is the request made by the
client, and con->request.uri is the current URI being processed.
Historical REQUEST_URI (environment variable) lighttpd inconsistencies
- mod_cml set REQUEST_URI to con->request.orig_uri
- mod_cgi set REQUEST_URI to con->request.orig_uri
- mod_fastcgi set REQUEST_URI to con->request.orig_uri
- mod_scgi set REQUEST_URI to con->request.orig_uri
- mod_ssi set REQUEST_URI to current con->request.uri
- mod_magnet set MAGNET_ENV_REQUEST_URI to current con->request.uri
and MAGNET_ENV_REQUEST_ORIG_URI to con->request.orig_uri
Historical REDIRECT_URI (environment variable) previously set only in
mod_fastcgi and mod_scgi, and set to con->request.uri
Since lighttpd 1.4.40 provides REDIRECT_URI with con->request.orig_uri,
changes were made to REQUEST_URI for consistency, with the hope that
there would be little impact to existing configurations since the
request uri and original request uri are the same unless there has been
an internal redirect. It turns out that various PHP frameworks use
REQUEST_URI and require that it be the original URI requested by client.
Therefore, this change is being reverted, and lighttpd will set
REQUEST_URI to con->request.orig_uri in mod_cgi, mod_fastcgi, mod_scgi
as was done in lighttpd 1.4.39 and earlier. Similarly, REDIRECT_URI
also has the prior behavior in mod_fastcgi and mod_scgi, and added to
mod_cgi.
A future release of lighttpd might change mod_ssi to be consistent with
the other modules in setting REQUEST_URI to con->request.orig_uri and to
add REDIRECT_URI, when an internal redirect has occurred.
Strip bogus "Proxy" header before creating subprocess environment.
(mod_cgi, mod_fastcgi, mod_scgi, mod_ssi, mod_proxy)
Do not emit HTTP_PROXY to subprocess environment.
Some executables use HTTP_PROXY to configure outgoing proxy.
This is not a lighttpd security issue per se, but this change to
lighttpd adds a layer of defense to protect backend processes which
might be vulnerable due to blindly using this untrusted environment
variable. The HTTP_PROXY environment variable should not be trusted
by a program running in a CGI-like environment.
Mitigation in lighttpd <= 1.4.40 is to reject requests w/ Proxy header:
* Create "/path/to/deny-proxy.lua", read-only to lighttpd, with content:
if (lighty.request["Proxy"] == nil) then return 0 else return 403 end
* Modify lighttpd.conf to load mod_magnet and run lua code
server.modules += ( "mod_magnet" )
magnet.attract-raw-url-to = ( "/path/to/deny-proxy.lua" )
References:
https://www.kb.cert.org/vuls/id/797896
CGI web servers assign Proxy header values from client requests to
internal HTTP_PROXY environment variables
https://httpoxy.org/
httpoxy: A CGI application vulnerability
do not expose server info server_tag is configured by admin
default con->conf.server_tag is still PACKAGE_DESC, those
admin can configure server.tag otherwise.
(these changes reduce recompilation and relinking when switching
brances in source control)
Set server.stream-response-body = 1 or server.stream-response-body = 2
to have lighttpd stream response body to client as it arrives from the
backend (CGI, FastCGI, SCGI, proxy).
default: buffer entire response body before sending response to client.
(This preserves existing behavior for now, but may in the future be
changed to stream response to client, which is the behavior more
commonly expected.)
x-ref:
"fastcgi, cgi, flush, php5 problem."
https://redmine.lighttpd.net/issues/949
"Random crashing on FreeBSD 6.1"
https://redmine.lighttpd.net/issues/760
"Memory usage increases when proxy+ssl+large file"
https://redmine.lighttpd.net/issues/1283
"lighttpd+fastcgi memory problem"
https://redmine.lighttpd.net/issues/1387
move code in dynamic handlers (CGI, FastCGI, SCGI, proxy)
from *_handle_fdevent() to *_recv_response() for reuse
outside the *_handle_fdevent() routine
Set server.stream-request-body = 1 or server.stream-request-body = 2
to have lighttpd connect to backend (CGI, FastCGI, SCGI, proxy)
immediately after parsing request headers, and to stream request body
as it arrives.
default: buffer entire request body before connecting to backend,
in order to avoid tying up (limited) backend resources which are often
implemented using libraries which wait for entire request body before
proceeding.
x-ref:
"Reimplement upload (POST) handling to match apache/zeus/thttpd/boa functionality"
https://redmine.lighttpd.net/issues/376
fallback to lseek()/read() if mmap() fails (#fixes 2666)
e.g. when mmap() is used on lighttpd-controlled temporary files
used POST request body (mod_cgi) and PUT file upload (mod_webdav)
replace use of stream_open() on potentially untrusted files
(protect against SIGBUS if a file is modified while map is read)
Note: stream.[ch] may be removed in a future release
For now, stream.[ch] will read entire file into memory if mmap fails
and so it should only be used on trusted files, e.g. config files.
http_auth basic and digest files are typically small and so buffered
stdio fopen(), fgets(), fclose() will likely be approximately as fast
as mmap.
mod_dirlisting header and readme files are typically small and so
open(), read(), close() will typically be approximately as fast as mmap
mod_ssi will likely be much faster, now buffering SSI page construction
rather than a potentially huge number of file open() calls, one for each
tiny chunk of text between SSI directives.
mod_webdav COPY and MOVE may be slower due to removal of mmap, but are
now more resilient to partial writes.
x-ref:
"handle filesystems without mmap() support"
https://redmine.lighttpd.net/issues/2666
"WebDAV upload-> mmap failed: operation not permitted"
https://redmine.lighttpd.net/issues/962
"Lighttpd 1.4.20 Crash (SIGBUS in mod_compress)"
https://redmine.lighttpd.net/issues/1879
"Crash SIGBUS"
https://redmine.lighttpd.net/issues/2391
github: closes #57
server.error-handler preserves HTTP status error code when error page
is static, and allows dynamic handlers to change HTTP status code
when error page is provided by dynamic handler. server.error-handler
intercepts all HTTP status codes >= 400 except when the content is
generated by a dynamic handler (cgi, ssi, fastcgi, scgi, proxy, lua).
The request method is unconditionally changed to GET for the request
to service the error handler, and the original request method is
later restored (for logging purposes). request body from the
original request, if present, is discarded.
server.error-handler is somewhat similar to server.error-handler-404,
but server.error-handler-404 is now deprecated, intercepts only 404
and 403 HTTP status codes, and returns 200 OK for static error pages,
a source of confusion for some admins. On the other hand, the new
server.error-handler, when set, will intercept all HTTP status error
codes >= 400. server.error-handler takes precedence over
server.error-handler-404 when both are set.
NOTE: a major difference between server.error-handler and the
now-deprecated server.error-handler-404 is that the values of the
non-standard CGI environment variables REQUEST_URI and REDIRECT_URI
have been swapped. Since REDIRECT_STATUS is the original HTTP
status code, REDIRECT_URI is now the original request, and REQUEST_URI
is the current request (e.g. the URI/URL to the error handler).
The prior behavior -- which reversed REQUEST_URI and REDIRECT_URI values
from those described above -- is preserved for server.error-handler-404.
Additionally, REDIRECT_STATUS is now available to mod_magnet, which
continues to have access to request.uri and request.orig_uri.
See further discussion at https://redmine.lighttpd.net/issues/2702
and https://redmine.lighttpd.net/issues/1828
github: closes #36
set REDIRECT_STATUS to con->error_handler_saved_status in dynamic
handlers for PHP compiled with --force-redirect. Set to "200"
if (0 == con->error_handler_saved_status)
(mod_cgi, mod_fastcgi, mod_scgi, mod_ssi)
FYI: setting REDIRECT_STATUS in con->environment allows access and
manipulation by mod_magnet.
x-ref:
"REDIRECT_STATUS == 200 on 404 redirect"
https://redmine.lighttpd.net/issues/1828
github: closes #35
read request body in dynamic handlers supporting request body
(mod_cgi, mod_fastcgi, mod_proxy, mod_scgi, mod_webdav)
(In the future, each dynamic handler might choose whether or not to
buffer request body or to stream request body to backend as request
body is received.)
modify mod_webdav to mark request in handle_physical hook, and move
the main logic to handle_subrequest hook, where the main logic is
for other dynamic handlers.
read request body right before calling subrequest handler,
allowing request to be handled prior to reading request body,
e.g. to send 401 Unauthorized response when authentication is required
(In the future, this might move into each dynamic handler which supports
request body (mod_cgi, mod_fastcgi, mod_proxy, mod_scgi, mod_webdav) so
that each dynamic handler can choose whether or not to buffer request
body or to stream request body to backend as request body is received.)
keep-alive is disabled if request body has not been completely read
prior to sending response
x-ref:
"HTTP 401 Unauthorized only sent back after full POST request is read"
https://redmine.lighttpd.net/issues/2541
for dynamic handlers mod_cgi, mod_fastcgi, mod_scgi, mod_proxy
(mod_cgi control flow logic simplification began in a prior commit)
- connection state machine calls the subrequest handler
- subrequest handler sets up the connection to the backend
and registers fdevent handler to handle backend events
- fdevent handler handles backend events and then schedules
a call to connection state machine
- when retrying an alternate backend, backend state is reset
and then response state is reset so that the connection state
machine will call back into the subrequest handler to retry
Dynamic handlers no longer directly modify connection state
(calls to connection_set_state() from dynamic handlers were removed)
Dynamic handlers no longer reset con->physical.path, and they
preserve con->mode when retrying alternate backends. This is done
to skip repeated processing in response.c:http_response_prepare()
While this patch increases consistency in control flow handling,
there is more work to be done that can further improve upon this.
x-ref:
"handle-req time too long"
https://redmine.lighttpd.net/issues/1149
wait for CGI to close stdout, so we read EOF on pipe to end CGI response
remove extra call to waitpid() which will occur after process exits
if it has not already been explicitly closed by CGI (and has not been
inherited by CGI forked children) (If CGI forks, then it should close
its stdout response pipe when response is done, especially if it intends
to perform lengthy post-processing in the background.)
From: Glenn Strauss <gstrauss@gluelogic.com>
git-svn-id: svn://svn.lighttpd.net/lighttpd/branches/lighttpd-1.4.x@3091 152afb58-edef-0310-8abb-c4023f1b3aa9
(Thx, anomie, who identified and explained problem in above ticket)
From: Glenn Strauss <gstrauss@gluelogic.com>
git-svn-id: svn://svn.lighttpd.net/lighttpd/branches/lighttpd-1.4.x@3089 152afb58-edef-0310-8abb-c4023f1b3aa9
Flash filesystem JFFS2 does not support mmap PROT_READ MAP_SHARED,
though it does support mmap PROT_READ MAP_PRIVATE
Although MAP_SHARED is preferred, CGI input body is fully collected
prior to handler invoking the CGI, so the temporary file is never
modified after being mapped. Since the request input body is specific
to request and is temporary file, mmap PROT_READ MAP_PRIVATE works fine.
From: Glenn Strauss <gstrauss@gluelogic.com>
git-svn-id: svn://svn.lighttpd.net/lighttpd/branches/lighttpd-1.4.x@3075 152afb58-edef-0310-8abb-c4023f1b3aa9
- a lot of code tried to handle manually adding terminating zeroes and
keeping track of the correct "used" count.
Replaced all "external" usages with simple wrapper functions:
* buffer_string_is_empty (used <= 1), buffer_is_empty (used == 0);
prefer buffer_string_is_empty
* buffer_string_set_length
* buffer_string_length
* CONST_BUF_LEN() macro
- removed "static" buffer hacks (buffers pointing to constant/stack
memory instead of malloc()ed data)
- buffer_append_strftime(): refactor buffer+strftime uses
- li_tohex(): no need for a buffer for binary-to-hex conversion:
the output data length is easy to predict
- remove "-Winline" from extra warnings: the "inline" keyword just
supresses the warning about unused but defined (static) functions;
don't care whether it actually gets inlined or not.
From: Stefan Bühler <stbuehler@web.de>
git-svn-id: svn://svn.lighttpd.net/lighttpd/branches/lighttpd-1.4.x@2979 152afb58-edef-0310-8abb-c4023f1b3aa9
* removed almost all usages of buffer as "memory" (without terminating
zero)
* refactored cgi variable name encoding
From: Stefan Bühler <stbuehler@web.de>
git-svn-id: svn://svn.lighttpd.net/lighttpd/branches/lighttpd-1.4.x@2977 152afb58-edef-0310-8abb-c4023f1b3aa9
* remove unused structs and functions
(buffer_array, read_buffer)
* change return type from int to void for many functions,
as the return value (indicating error/success) was never checked,
and the function would only fail on programming errors and not on
invalid input; changed functions to use force_assert instead of
returning an error.
* all "len" parameters now are the real size of the memory to be read.
the length of strings is given always without the terminating 0.
* the "buffer" struct still counts the terminating 0 in ->used,
provide buffer_string_length() to get the length of a string in a
buffer.
unset config "strings" have used == 0, which is used in some places
to distinguish unset values from "" (empty string) values.
* most buffer usages should now use it as string container.
* optimise some buffer copying by "moving" data to other buffers
* use (u)intmax_t for generic int-to-string functions
* remove unused enum values: UNUSED_CHUNK, ENCODING_UNSET
* converted BUFFER_APPEND_SLASH to inline function (no macro feature
needed)
* refactor: create chunkqueue_steal: moving (partial) chunks into another
queue
* http_chunk: added separate function to terminate chunked body instead of
magic handling in http_chunk_append_mem().
http_chunk_append_* now handle empty chunks, and never terminate the
chunked body.
From: Stefan Bühler <stbuehler@web.de>
git-svn-id: svn://svn.lighttpd.net/lighttpd/branches/lighttpd-1.4.x@2975 152afb58-edef-0310-8abb-c4023f1b3aa9
These should all be non critical:
* memory leaks on startup in error cases (which lead to
immediate shutdowns anyway)
* http_auth/ldap: passing uninitialized "ret" to ldap_err2string
* sizeof(T) not matching the target pointer in malloc/calloc calls;
those cases were either:
* T being the wrong pointer type - shouldn't matter as long as all
pointers have same size
* T being larger than the type needed
* mod_accesslog: direct use after free in cleanup (server shutdown);
could crash before "clean" shutdown
* some false positives (mod_compress, mod_expire)
* assert(srv->config_context->used > 0); - this is always the case,
as there is always a global config block
From: Stefan Bühler <stbuehler@web.de>
git-svn-id: svn://svn.lighttpd.net/lighttpd/branches/lighttpd-1.4.x@2920 152afb58-edef-0310-8abb-c4023f1b3aa9
con->conf.is_ssl got removed and replaced by:
* con->conf.ssl_enabled for the config var "ssl.engine" - it is only
used to determine which server-sockets should use ssl. (usually not
needed as it is mandatory and enough to set ssl.pemfile anyway)
* con->srv_socket->is_ssl to detect the actual ssl status of the
bound socket, which is the same as the ssl status of the connection
* con->uri.scheme for the actual $HTTP["scheme"] value, also used for
the CGI "HTTPS=ON" variable. This defaults to "https" if the
connection uses ssl, but can be changed for example by mod_extforward
if X-Forwarded-Proto: is set to either "http" or "https" (other values
are ignored right now)
Also removed the broken srv_socket->is_proxy_ssl as it was a connection
value in a server_socket struct...
git-svn-id: svn://svn.lighttpd.net/lighttpd/branches/lighttpd-1.4.x@2887 152afb58-edef-0310-8abb-c4023f1b3aa9
There are 6 remaining occurrences after this commit, in mod_compress,
due to zlib's Z_NULL being defined as 0 instead of NULL.
Signed-off-by: Cyril Brulebois <kibi@debian.org>
git-svn-id: svn://svn.lighttpd.net/lighttpd/branches/lighttpd-1.4.x@2845 152afb58-edef-0310-8abb-c4023f1b3aa9
Gaurav