also remove some now-redundant tests from request.t
and reduce scripts and directories under tests
(because automake is sloooow and the fewer dirs, the better)
rename log_error_multiline_buffer() to log_error_multiline()
and take (char *)ptr and (size_t)len instead of (buffer *)b
When debug printing request and response headers,
print each header on separate line for readability
and omit '\r' if "\r\n" ends line
improve CGI offloading when not streaming request body
If not streaming request body, collect request body into single tempfile
and use fd to tempfile as stdin to CGI (in lieu of input pipe()).
For prior behavior, set the following config option, enabled by default:
server.feature-flags += ("cgi.tempfile-accum" => "disable")
If there is no request body, open /dev/null instead of creating pipe().
RFC4648 base64url pad char is '='; change from '.' in lighttpd
The base64url encoding was introduced in lighttpd 1.4.38 in 2015,
but at the time, does not appear to have been used in base64url
decoding where padding might be present.
In lighttpd 1.4.50, base64url decoding was possible with %{decb64u:...}
potentially used in mod_redirect and mod_rewrite rules. However, this
is not believed to be widely used, and even if used, it is strongly
recommended that the URLs contain a base64url-encoded string WITHOUT
padding, since padding of '=' might be %-encoded and require decoding
before base64url-decoding.
Note: this change may affect %{decb64u:...} substitution in
mod_redirect and mod_rewrite, but *ONLY* in case where URL contains
a base64url-encoded string *WITH* padding, and that padding had been
created specifically for use with lighttpd, using '.' as padding char
x-ref:
RFC4648 https://datatracker.ietf.org/doc/html/rfc4648#section-5
ignore 1xx return code from lua in response start phase.
Since response is about to start, send any added/modified headers
along with final response. (If we did not ignore, then 1xx return
code from lua would incorrectly overwrite the final response status.)
reset config cache for changes to
lighty.r.req_attr["request.remote-addr"]
Changes might affect subsequent modules when those modules patch config,
but does not affect the core config options which have already been
processed.
If lighttpd config needs to be reprocessed after making changes, then
return lighty.RESTART_REQUEST from lua script to tell lighttpd to
restart the request, including reprocessing the core config options.
reset config cache after modifications to uri components
lighty.r.req_attr["uri.scheme"]
lighty.r.req_attr["uri.authority"]
lighty.r.req_attr["uri.path"]
lighty.r.req_attr["uri.query"]
Changes might affect subsequent modules when those modules patch config,
but does not affect the core config options which have already been
processed.
If lighttpd config needs to be reprocessed after making changes, then
return lighty.RESTART_REQUEST from lua script to tell lighttpd to
restart the request, including reprocessing the core config options.
rename magnet_cgi_* to magnet_envvar_* to be less confusing
The (array *)r->env is populated with connection-level env (e.g TLS)
but is not populated with CGI/1.1 environment variables.
Values similar to most CGI/1.1 environment variables are already available
from lighty.r.req_attr[] (or lighty.env[]) which are prepared on demand.
lighty.c.readdir dir walk
Note: the "lighty.c.*" namespace is EXPERIMENTAL / UNSTABLE
In the future, these may be removed, altered, or moved to a different
namespace.
lighty.c.cookie_tokens parse cookie header into table
Note: the "lighty.c.*" namespace is EXPERIMENTAL / UNSTABLE
In the future, these may be removed, altered, or moved to a different
namespace.
refuse excess h2 streams up to stream id 199 within 1-2 seconds of
establishing connection. This attempts to more gracefully handle
excessively agressive browsers sending too many new requests *before*
processing the initial SETTINGS frame sent by the server.
lighttpd HTTP/2 support limits SETTINGS_MAX_CONCURRENT_STREAMS to 8
and communicates this in the initial SETTINGS sent by the server.
x-ref:
"Chrome 92, HTTP/2, fcgi, mutiple puts no response"
https://redmine.lighttpd.net/issues/3093
"POST request DATA part for non-existing URI closes HTTP/2 connection prematurely"
https://redmine.lighttpd.net/issues/3078
lighty.r.req_header[] HTTP request headers
lighty.r.req_attr[] HTTP request attributes / components
lighty.r.req_env[] HTTP request environment variables
lighty.r.resp_header[] HTTP response headers
lighty.r.resp_body.* HTTP response body accessors
lighty.r.resp_body.len HTTP response body length
lighty.r.resp_body.add() HTTP response body add (string or table)
lighty.r.resp_body.set() HTTP response body set (string or table)
lighty.r.req_header[] allows get/set of request headers
If modifications would affect config processing, script should return
lighty.MAGNET_RESTART to have lighttpd restart the modified request
This differs from the older lighty.env[] table, which (previously) did
not permit modification of request headers.
lighty.r.resp_header[] allows get/set of response headers
(certain connection-level headers such as Connection and
Transfer-Encoding are restricted from modification)
This differs from the older lighty.header[] table, which is
collected and deferred, being applied after the script exits.
lighry.r.resp_body.* adds/sets response body content.
This differs from the older lighty.content[] table, which is
collected and deferred, being applied after the script exits.
e.g.
r = lighty.r
resp_header = r.resp_header
resp_header["Content-Type"] = "text/html"
or
lighty.r.resp_header["Content-Type"] = "text/html"
older syntax (less clear)
lighty.header["Content-Type"] = "text/html"
lighty.c.b64urldec validate and decode base64url
lighty.c.b64urlenc base64url encode, no padding
lighty.c.b64dec validate and decode base64
lighty.c.b64enc base64 encode, no padding
lighty.c.hexdec validate and decode hex str
lighty.c.hexenc uc; lc w/ lua s = s:lower()
lighty.c.xmlenc xml-encode/html-encode: <>&'\"`
lighty.c.urldec url-decode
lighty.c.urlenc url-encode
lighty.c.urldec_query url-decode query-string
lighty.c.urlenc_query url-encode query-string
lighty.c.urlenc_normalize url-encode normalization
lighty.c.fspath_simplify simplify fspath
Note: the "lighty.c.*" namespace is EXPERIMENTAL / UNSTABLE
In the future, these may be removed, altered, or moved to a different
namespace.
protect lighty table against mistaken modification
(lighty table is resused between script invocations)
detect if lighty.content[] is set by script (or if not set)
lighty.stat now returns a userdata object instead of a populated table.
The userdata object provides methods to access the (stat_cache_entry *)
stored in the userdata object. (This approach is often much faster than
populating the table of stat entries, as the fields get copied on demand
(upon use) into lua types.)
allow modification of request headers, with some limitations:
- lighttpd config conditions are not reset;
lua script must return lighty.RESTART_REQUEST to reprocess request
(if applicable to the running lighttpd config)
- lighttpd config request header policy is not applied;
lua script must not set unvalidated, untrusted, or non-normalized vals
- Host may not be unset
- Content-Length may not be modified
- some hop-by-hop (connection) level headers may not be modified
(e.g. Connection, Transfer-Encoding)
lighty.c.md("algo", "data")
lighty.c.hmac("algo", "secret", "data")
"algo" can be one of: "md5", "sha1", "sha256", "sha512"
(as long as lighttpd compiled w/ crypto lib supporting those algorithms)
lighty.c.digest_eq("digest1", "digest2")
- performs a timing-safe, case-insensitive comparison of two hex digests
- "digest1" and "digest2" are hex strings (of binary digests)
- returns boolean true or false
lighty.c.secret_eq("data1", "data2")
- performs a timing-safe comparison of two strings
(and attempts to hides differences in string lengths)
- "data1" and "data2" are strings
- returns boolean true or false
lighty.c.time()
- cached time(); seconds since 1 Jan 1970 00:00:00 GMT
(faster than os.time())
lighty.c.rand()
- generate pseudo-random number
Note: the "lighty.c.*" namespace is EXPERIMENTAL / UNSTABLE
In the future, these may be removed, altered, or moved to a different
namespace.
If a system call returns EMFILE, then admin should increase
server.max-fds and check/increase rlimits for num files (ulimit -Hn)
Alternatively, the admin might decrease server.max-connections to limit
the number of connections served in parallel.
merge connection_list_append() into connection_fdwaitqueue_append()
(not converted to singly-linked-list since fdwaitqueue is not expected
to be used during normal operation (except extreme overload condition),
so use allocated list of pointers (allocated when needed) instead of
adding ptr member to (every) allocated struct connection)
remove connections-glue.c
remove inclusion of connections.h by non-base files
avoids separate memory allocation for list of pointers
adds ability to check if con is already in joblist,
so do not re-add con if already in joblist
since con is checked if in joblist before being added to joblist,
there is no longer need for two lists and jobs can be processed
before poll() for to process new events
srv->lim_conns tracks remaining conns until limit is reached,
replacing (srv->max_conns - srv->conns.used)
srv->srvconf.max_conns is now updated at startup, so
srv->srvconf.max_conns serves as srv->max_conns
keep conns_pool of struct connection separate from conns list
and allocate conns list to srv->srvconf.max_conns size at startup
x-ref:
"Memory fragmentation with HTTP/2 enabled"
https://redmine.lighttpd.net/issues/3084
Glenn Strauss
Juliusz Sosinowicz