(experimental)
add option to run lua scripts in lighttpd response start hook
allows for response header manipulation
new params provide read-only access:
lighty.env["response.http-status"]
lighty.env["response.body-length"]
lighty.env["response.body"]
allows for content manipulation if the response body is complete
The HTTP response status can be accessed in lua via
lighty.env["response.http-status"] and should be checked, as
appropriate, prior to body manipulation. The value is non-zero
in response start hook (magnet.attract-response-start-to), but is
likely to be 0 in scripts run from other lighttpd hooks earlier in
request processing,
e.g. magnet.attract-raw-url-to or magnet.attract-physical-path-to
Caller should check lighty.env["response.body-length"]
is a smaller and sane amount to read into memory and copy
a second time into lua data structures. The value is lua nil
if the response body is not yet complete (or if it is >= 2GB-1)
Loading the response body (and all mod_magnet lua scripts) are
executed serially (blocking) in lighttpd, so its use is highly
discouraged on large files. The body can be accessed in lua via
lighty.env["response.body"] if the response body is complete.
(recommended config option: server.stream-response-body = 0 (default)
if mod_magnet scripts must process the response body)
Modifying HTTP response status and response body has not changed
and is achieved by setting lua script return value and modifying
the lighty.content lua table.
(note: mod_magnet, mod_setenv, mod_deflate, mod_expire have their
response start hooks run in the order listed in server.modules)
relay 1xx from backend over HTTP/1.1, e.g. 103 Early Hints
(if client is connected using HTTP/1.1)
enabled by default unless disabled in lighttpd.conf with:
server.feature-flags += ( "server.h1-discard-backend-1xx" = "enable" )
Warning: backends which send 103 Early Hints should check User-Agent
before doing so since naive clients might not handle unexpected 1xx.
Some clients may take the 1xx response as the final response, expecting
only one response. Some clients might not properly handle 100 Continue
if the client did not send Expect: 100-continue with the request.
https://tools.ietf.org/html/rfc8297#section-3 Security Considerations
x-ref:
An HTTP Status Code for Indicating Hints (103 Early Hints)
https://tools.ietf.org/html/rfc8297
relay 1xx from backend over HTTP/2, e.g. 103 Early Hints
(if client is connected using HTTP/2)
enabled by default unless disabled in lighttpd.conf with:
server.feature-flags += ( "server.h2-discard-backend-1xx" = "enable" )
Warning: backends which send 103 Early Hints should check User-Agent
before doing so since naive clients might not handle unexpected 1xx.
Some clients may take the 1xx response as the final response, expecting
only one response. Some clients might not properly handle 100 Continue
if the client did not send Expect: 100-continue with the request.
https://tools.ietf.org/html/rfc8297#section-3 Security Considerations
x-ref:
An HTTP Status Code for Indicating Hints (103 Early Hints)
https://tools.ietf.org/html/rfc8297
support multiple 1xx intermediate responses from backends
Currently, all 1xx responses from backends are discarded.
In the future, these 1xx responses may be forwarded to the client
(when lighttpd also configured server.stream-response-body = 1 or = 2)
(optional addition to (data_string *), used by http_header.[ch])
extend (data_string *) instead of creating another data_* TYPE_*
(new data type would probably have (data_string *) as base class)
(might revisit choice in the future)
HTTP_HEADER_UNSPECIFIED has been removed. It was used in select
locations as an optimization to avoid looking up enum header_header_e
before checking the array, but the ordering in the array now relies
on having the id. Having the id allows for a quick check if a common
header is present or not in the htags bitmask, before checking the
array, and allows for integer comparison in the log(n) search of the
array, instead of strncasecmp().
With HTTP_HEADER_UNSPECIFIED removed, add optimization to set bit
in htags for HTTP_HEADER_OTHER when an "other" header is added,
but do not clear the bit, as there might be addtl "other" headers
header field-names are required to be lowercase for HTTP/2
so specialize http_header_hkey_get() as http_header_hkey_get_lc()
for simpler comparison
lowercase field-names in http_headers[], as it does not matter for
buffer_eq_icase_ssn(), which is used with http_headers[] for HTTP/1.x
separate http_header_e index from r->{rqst,resp}_htags bitmask
(allows http_header_e remain 32-bit and to be used in array indexes
while also allowing r->{rqst,resp}_htags to grow to 64-bits wide
in the future, without requiring invasive changes)
allow h2.c layer to modify r->state in addition to r->h2state
Do not retire/release r on error if trailers (as opposed to headers)
(stream r was just allocated in the same scope for headers,
so ok to release)
log stream id with debug.log-state-handling in
connection_state_machine_loop()
(id will be 0 for HTTP/2 connection, but not for streams on connection)
(id will be 0 for HTTP/1.x requests)
(thx avij)
clients should send a single Cookie header with multiple cookie values
separated with ';'.
https://tools.ietf.org/html/rfc6265#section-4.2.1
However, HTTP/2 loosens this requirement for Cookie.
https://tools.ietf.org/html/rfc7540#section-8.1.2
Section 8.1.2.5 Compressing the Cookie Header Field
and some HTTP/2 clients (Chrome, Firefox) send multiple
'cookie:' headers in a HEADERS frame.
only XXH32() is used by ls-hpack, so disable XXH64() and XXH3()
to reduce code size. (This maybe be changed in the future if
XXH64() or XXH3() are utilitized for their performance)
(bug on master branch; never released)
(thx avij)
update doc that dir-listing.hide-dotfiles = "enable" by default
since lighttpd 1.4.40 https://redmine.lighttpd.net/issues/1081
(bug on master branch; never released)
(thx avij)
fix crash on master if blank line precedes HTTP/1.1 keep-alive request
header parsing code previously made assumptions that request was
HTTP/1.0 or HTTP/1.1, where a request-line was required, and which
would error out elsewhere if request-line was missing. The parsing
code also previously looked for "\r\n\r\n" to end headers.
The header offset parsing code was modified and invalidated the above
assumptions, now looking only for blank line "\r\n", but the calling
code had not properly been updated. (until this patch)
define LSHPACK_DEC_HTTP1X_OUTPUT 0
lighttpd does not require HTTP/1.1 output compat from HPACK decoder
("field-name: value\r\n")
define NDEBUG (in ls-hpack/lshpack.c)
lighttpd spends upwards of 20% total lighttpd CPU time in HPACK
encode/decode in h2load test on static file over cleartext (not TLS)
Defining NDEBUG eliminates some asserts() and results in a small
but measurable reduction in CPU usage
Glenn Strauss