handle CGI partial write of first response header
e.g. gSoap stdsoap2.c might inefficiently write "Status" to response
pipe and lighttpd might read that prior to the backend writing the
subsequent ": " which marks "Status:" as a response header.
x-ref:
https://redmine.lighttpd.net/boards/2/topics/8028
provide standard types in first.h instead of base.h
provide lighttpd types in base_decls.h instead of settings.h
reduce headers exposed by headers for core data structures
do not expose <pcre.h> or <stdlib.h> in headers
move stat_cache_entry to stat_cache.h
reduce use of "server.h" and "base.h" in headers
fix rare race condition from backends with server.stream-response-body=2
(thx abelbeck)
x-ref:
"fastcgi and stream-response-body=2 hangs on last chunk"
https://redmine.lighttpd.net/issues/2878
When backend returns an invalid response header which is exactly a
blank line (\n or \r\n), adjust the offset so as not to discard the
first character following, which is probably intended to be the
beginning of the response body.
Prior code was effectively a 1-element cache after the initial fill
of the array since only the first element was replaced after the
initial fill. New code does round-robin replacement.
(whether or not #define FILE_CACHE_MAX 16 is appropriately sized here
is a question for another day)
common codebase for socket backends, based off mod_fastcgi with
some features added for mod_proxy
(mostly intended to reduce code duplication and enhance code isolation)
mod_fastcgi and mod_scgi can now use fastcgi.balance and scgi.balance
for similar behavior as proxy.balance, but the balancing is per-host
and not per-proc. proxy.balance is also per-host and not per-proc.
mod_proxy and mod_scgi can now use proxy.map-extensions and
scgi.map-extensions, similar to fastcgi.map-extensions.
mod_fastcgi behavior change (affects only mod_status):
- statistics tags have been renamed from "fastcgi.*" to "gw.*"
"fastcgi.backend.*" -> "gw.backend.*"
"fastcgi.active-requests" -> "gw.active-requests"
("fastcgi.requests" remains "fastcgi.requests")
("proxy.requests" is new)
("scgi.requests" is new)
mod_scgi behavior change (likely minor):
- removed scgi_proclist_sort_down() and scgi_proclist_sort_up().
procs now chosen based on load as measured by num socket connnections
Note:
modules using gw_backend.[ch] are currently still independent modules.
If it had been written as a single module with fastcgi, scgi, proxy
implementations, then there would have been a chance of breaking some
existing user configurations where module ordering made a difference
for which module handled a given request, though for most people, this
would have made no difference.
Details about mod_fastcgi code transformations:
unsigned int debug -> int debug
fastcgi_env member removed from plugin_config
renamed "fcgi" and "fastcgi" to "gw", and "FCGI" to "GW"
reorganize routines for high-level and lower-level interfaces
some lower-level internal interfaces changed to use host,proc,debug
args rather than knowing about higher-level (app) hctx and plugin_data
tabs->spaces and reformatting
Better handling if server.error_handler_404 is a dynamic handler which
returns X-Sendfile pointing to a file which does not exist
(server.error_handler_404 historically did not reset con->file_started,
and for mod_fastcgi, an X-Sendfile failure in the error handler would
result in an empty response body.)
x-ref:
"Option to map send-file file-not-found error to normal 404"
https://redmine.lighttpd.net/issues/2474
Use same funcs as other dynamic handlers to recv data from backend.
Add hook for fastcgi to process FastCGI packets (and other future
dynamic handlers may hook this in order to handle custom data framing)
Provide a simple mechanism for mapping host and urlpath header strings
in proxied request and response well-known headers. This *is not*
intended as a one-size-fits-all, infinitely extensible, regex rewriting
engine. Instead, the proxy.header directive aims to provide built-in
functionality in mod_proxy for a few common use cases by performing
simple host matching or urlpath prefix matching, and using the
mapping of the first match. More complex use cases could possibly be
handled by a custom lighttpd module (which does not currently exist).
Note: the contents of the HTTP request-line and HTTP headers may or
may not be in normalized canonical forms, which may or may not influence
the simple matching performed. Admins should take care to provide safe
defaults (fail closed) if mapping is expected to occur and blindly
passing non-mapped requests is undesirable.
proxy.header = (
#"map-host-request" => (
#"-" => "...",#replace provided given Host request authority
#"..." => "-",#preserve existing authority (no further matching)
#"..." => "", #preserve existing authority (no further matching)
# #(equivalent to "xxx" => "xxx")
#"xxx" => "yyy", #map one string ("xxx") to another ("yyy")
#),
#"map-host-response" => (
#"-" => "...",#replace authority used in backend request
#"..." => "-",#replace with original authority
#"..." => "", #preserve existing authority (no further matching)
# #(equivalent to "xxx" => "xxx")
#"xxx" => "yyy", #map one string ("xxx") to another ("yyy")
#),
#"map-urlpath" => (
#"/xxx" => "/yyy",#map one urlpath prefix to another
#"/xxx/" => "/", #map one urlpath prefix to another
#"/xxx" => "", #map one urlpath prefix to another
#"/key" => "/value",
# Note: request headers have matching "key" prefix replaced with
# "value", and response headers have matching "value" prefix
# replaced with "key", with a pre-test of the "value" from the
# first-matched "key" in request headers (if there was a match)
#),
#"https-remap" => "enable",
# For https requests from client, map https:// to http://
# when map-host-request matches URI in request, and map http://
# to https:// when map-host-response matches URI in response.
# (mod_proxy currently sends all backend requests as http)
)
x-ref:
"feature to remove part of the URI when passing along requests..."
https://redmine.lighttpd.net/issues/152
This patch fixes the build issue introduced with when code has been
shared in commit a448886485.
http-header-glue.c: In function 'http_cgi_headers':
http-header-glue.c:1555:39: error: 'b2' undeclared (first use in this function)
http-header-glue.c:1555:39: note: each undeclared identifier is reported only once for each function it appears in
github: closes #79
large code move, but minimal changes made to code (besides whitespace),
so that code builds
next: need to isolate openssl data structures and config parsing
handle_request_env (called on demand by handlers to populate env)
handle_connection_accept
handle_connection_shut_wr (was handle_connection_close)
handle_connection_close (now occurs at socket close())
move http_cgi_ssl_env() from response.c to http-header-glue.c
for symbol visibility on Mac OS X.
x-ref:
"Undefined symbols: _http_cgi_ssl_env"
https://redmine.lighttpd.net/issues/2757
initialized for mod_magnet and dynamic CGI-like handlers
(mod_cgi, mod_fastcgi, mod_scgi, mod_ssi) (*not* mod_proxy)
Note: in the future a config flag (does not yet exist) might be required
to activate initialization of these SSL_* env variables. This might
occur if there are requests to access these variables in mod_accesslog,
and/or if more SSL_* varables are created, which would be more work.
x-ref:
"pass protocol and cipher details to fcgi env"
https://redmine.lighttpd.net/issues/2511
consolidated from CGI, FastCGI, SCGI, SSI
Note: due to prior inconsistencies between the code in mod_cgi,
mod_fastcgi, mod_scgi, and mod_ssi, there are some minor behavior
changes.
CONTENT_LENGTH is now always set, even if 0
(though CONTENT_LENGTH is never set for FASTCGI_AUTHORIZER)
PATH_INFO is created only if present, not if empty.
(mod_fastcgi and mod_ssi previously set PATH_INFO="" (blank value))
PATH_TRANSLATED is now set if PATH_INFO is present
(previously missing from mod_cgi and mod_ssi)
mod_ssi now sets DOCUMENT_ROOT to con->physical.basedir, like others
(previously, mod_ssi set DOCUMENT_ROOT to con->physical.doc_root,
which matched con->physical.basedir unless mod_alias changed basedir)
mod_ssi now sets REQUEST_URI to con->request.orig_uri, like others
(previously, mod_ssi set REQUEST_URI to con->request.uri, which
matched con->request.orig_uri except after redirects, error docs)
cygwin does not support ioctl on sockets, returning EOPTNOTSUPP
(would be better if cygwin used Windows ioctlsocket() instead)
Windows uses signed (socklen_t), so add some casts to quiet warnings
Windows path handling is convoluted, so disable one tests in mod_fastcgi
since trailing spaces are removed from URL for _WIN32 and __CYGWIN__ in
response.c
handle X-Sendfile and X-LIGHTTPD-send-file w/ http_response_xsendfile()
if host is configured ( "x-sendfile" = "enable" )
Note: X-Sendfile path is url-decoded for consistency, like X-Sendfile2
(response headers should be url-encoded to avoid tripping over
chars allowed in filesystem but which might change response
header parsing semantics)
Note: deprecated: "allow-x-send-file"; use "x-sendfile"
Note: deprecated: X-LIGHTTPD-send-file header; use X-Sendfile header
Note: deprecated: X-Sendfile2 header; use X-Sendfile header
For now, X-Sendfile2 is still handled internally by mod_fastcgi.
Since http_response_send_file() supports HTTP Range requests,
X-Sendfile2 is effectively obsolete. However, any code, e.g. PHP,
currently using X-Sendfile2 is probably manually generating 206 Partial
Content status and Range response headers. A future version of lighttpd
might *remove* X-Sendfile2. Existing code should be converted to use
X-Sendfile, which is easily done by removing all the special logic
around using X-Sendfile2, since the 206 Partial Content status and Range
response headers are handled in http_response_send_file().
x-ref:
"mod_fastcgi + X-Sendfile -> mod_staticfile"
https://redmine.lighttpd.net/issues/799
"Feature Request: New option "x-send-file-docroot""
https://redmine.lighttpd.net/issues/851
"X-Sendfile handoff to mod-static-file in 1.4.x"
https://redmine.lighttpd.net/issues/2017
"X-sendfile should be able to set content-type"
https://redmine.lighttpd.net/issues/2076
move code from mod_staticfile.c to http-header-glue.c to allow reuse
(includes ETag, Last-Modified headers, Range requests for static files)
operate on path arg instead of con->physical.path
skip Range requests if con->http_status already set >= 300
remove redundant calls to stat_cache_get_entry() handling Range requests
x-ref:
"X-Sendfile handoff to mod-static-file in 1.4.x"
https://redmine.lighttpd.net/issues/2017
iterate over environ via array-index notation with char **ptr on stack
(instead of repeatedly re-accessing global 'environ')
check getsockname() return values including addrlen
[mod_dirlisting] pass buf size into http_list_directory_sizefmt()
github: resolves #48
- a lot of code tried to handle manually adding terminating zeroes and
keeping track of the correct "used" count.
Replaced all "external" usages with simple wrapper functions:
* buffer_string_is_empty (used <= 1), buffer_is_empty (used == 0);
prefer buffer_string_is_empty
* buffer_string_set_length
* buffer_string_length
* CONST_BUF_LEN() macro
- removed "static" buffer hacks (buffers pointing to constant/stack
memory instead of malloc()ed data)
- buffer_append_strftime(): refactor buffer+strftime uses
- li_tohex(): no need for a buffer for binary-to-hex conversion:
the output data length is easy to predict
- remove "-Winline" from extra warnings: the "inline" keyword just
supresses the warning about unused but defined (static) functions;
don't care whether it actually gets inlined or not.
From: Stefan Bühler <stbuehler@web.de>
git-svn-id: svn://svn.lighttpd.net/lighttpd/branches/lighttpd-1.4.x@2979 152afb58-edef-0310-8abb-c4023f1b3aa9
* removed almost all usages of buffer as "memory" (without terminating
zero)
* refactored cgi variable name encoding
From: Stefan Bühler <stbuehler@web.de>
git-svn-id: svn://svn.lighttpd.net/lighttpd/branches/lighttpd-1.4.x@2977 152afb58-edef-0310-8abb-c4023f1b3aa9
* remove unused structs and functions
(buffer_array, read_buffer)
* change return type from int to void for many functions,
as the return value (indicating error/success) was never checked,
and the function would only fail on programming errors and not on
invalid input; changed functions to use force_assert instead of
returning an error.
* all "len" parameters now are the real size of the memory to be read.
the length of strings is given always without the terminating 0.
* the "buffer" struct still counts the terminating 0 in ->used,
provide buffer_string_length() to get the length of a string in a
buffer.
unset config "strings" have used == 0, which is used in some places
to distinguish unset values from "" (empty string) values.
* most buffer usages should now use it as string container.
* optimise some buffer copying by "moving" data to other buffers
* use (u)intmax_t for generic int-to-string functions
* remove unused enum values: UNUSED_CHUNK, ENCODING_UNSET
* converted BUFFER_APPEND_SLASH to inline function (no macro feature
needed)
* refactor: create chunkqueue_steal: moving (partial) chunks into another
queue
* http_chunk: added separate function to terminate chunked body instead of
magic handling in http_chunk_append_mem().
http_chunk_append_* now handle empty chunks, and never terminate the
chunked body.
From: Stefan Bühler <stbuehler@web.de>
git-svn-id: svn://svn.lighttpd.net/lighttpd/branches/lighttpd-1.4.x@2975 152afb58-edef-0310-8abb-c4023f1b3aa9
con->conf.is_ssl got removed and replaced by:
* con->conf.ssl_enabled for the config var "ssl.engine" - it is only
used to determine which server-sockets should use ssl. (usually not
needed as it is mandatory and enough to set ssl.pemfile anyway)
* con->srv_socket->is_ssl to detect the actual ssl status of the
bound socket, which is the same as the ssl status of the connection
* con->uri.scheme for the actual $HTTP["scheme"] value, also used for
the CGI "HTTPS=ON" variable. This defaults to "https" if the
connection uses ssl, but can be changed for example by mod_extforward
if X-Forwarded-Proto: is set to either "http" or "https" (other values
are ignored right now)
Also removed the broken srv_socket->is_proxy_ssl as it was a connection
value in a server_socket struct...
git-svn-id: svn://svn.lighttpd.net/lighttpd/branches/lighttpd-1.4.x@2887 152afb58-edef-0310-8abb-c4023f1b3aa9
fbrosson
