(thx flynn)
fix large memory usage for large file downloads from dynamic backends
reuse or release large memory chunks
x-ref:
"Memory Growth with PUT and full buffered streams"
https://redmine.lighttpd.net/issues/3033
replace /* fall through */ comment with __attribute_fallthrough__ macro
Note: not adding attribute to code with external origins:
xxhash.h (algo_xxhash.h)
ls-hpack/lshpack.c
so to avoid warnings, may need to compile with -Wno-implicit-fallthrough
use http_chunk_append_file_ref() and http_chunk_append_file_ref_range()
reduce resource usage (number of fds open) by reference counting open
fds to files served, and sharing the fd among FILE_CHUNKs in responses
relay 1xx from backend over HTTP/2, e.g. 103 Early Hints
(if client is connected using HTTP/2)
enabled by default unless disabled in lighttpd.conf with:
server.feature-flags += ( "server.h2-discard-backend-1xx" = "enable" )
Warning: backends which send 103 Early Hints should check User-Agent
before doing so since naive clients might not handle unexpected 1xx.
Some clients may take the 1xx response as the final response, expecting
only one response. Some clients might not properly handle 100 Continue
if the client did not send Expect: 100-continue with the request.
https://tools.ietf.org/html/rfc8297#section-3 Security Considerations
x-ref:
An HTTP Status Code for Indicating Hints (103 Early Hints)
https://tools.ietf.org/html/rfc8297
support multiple 1xx intermediate responses from backends
Currently, all 1xx responses from backends are discarded.
In the future, these 1xx responses may be forwarded to the client
(when lighttpd also configured server.stream-response-body = 1 or = 2)
Empty field-name is already ignored when generating response headers,
but this is an explicit skip of empty field-name sent from backends,
including field-names that look like HTTP/2 pseudo-headers, as those
begin with ':'.
Location response header is permitted to use relative-path in
RFC 7231 Section 7.1.2. Location
Prefer relative path in redirection for the benefit of reverse proxies
and CDNs. Doing so also avoids potentially disclosing internal schemes
and server names which client might not be able to directly reach.
To restore prior behavior of sending a fully-qualified absolute URI:
server.feature-flags += ("absolute-dir-redirect" => "enable")
x-ref:
https://bz.apache.org/bugzilla/show_bug.cgi?id=63357
move code from connections-glue.c back into connections.c
move code from connections-glue.c to http-header-glue.c
rename connection_response_reset()
to http_response_reset()
rename connection_handle_read_post_error()
to http_response_reqbody_read_error()
decode Transfer-Encoding: chunked from gw (gateway backends)
Transfer-Encoding: chunked is a hop-by-hop header.
Handling chunked encoding remove a hurdle for mod_proxy to send HTTP/1.1
requests to backends and be able to handle HTTP/1.1 responses.
Other backends ought not to send Transfer-Encoding: chunked, but in
practice, some implementations do.
translate config server.modules "mod_compress" to "mod_deflate"
accept compress.* directives, but issue DEPRECATED warning trace
mod_deflate differences from mod_compress:
- mod_compress compress.filetype was exact match; deflate.mimetypes is
prefix match (behavior change might compress longer mimetype matches,
which are likely of similar type and compressability)
- mod_compress always sent entire (compressed) file for Range request
mod_deflate will stream compress range result (not stored in cache)
- mod_compress would short-circuit request with 403 Forbidden error
if request file did not exist (stat() failed) (This behavior was
unfriendly to other handlers)
- mod_compress compress.cache-dir layout differs from deflate.cache-dir
layout; file cache should be cleared (or renamed) when migrating from
mod_compress to mod_deflate
- mod_deflate does not issue Vary: Accept-Encoding if request does not
contain Accept-Encoding. The identity response can be cache by
proxies and served to clients. Historically, some proxies disabled
caching if any Vary: response was seen. If the Vary header is
desirable, mod_deflate code which checks for Accept-Encoding and
compression type can be moved down a few lines to be below the
setting of the Vary response header.
stricter parse of numerical digits for http status code, port num,
and a few other places. (stricter parse than that of strtol())
content ranges are still parsed more loosely at points of use
stream request body using HTTP/1.1 Transfer-Encoding: chunked
(Note: if backend proxy target does not support HTTP/1.1,
then do not use server.stream-request-body = 1 or 2)
If not streaming to backend, collect request body
(now supporting Transfer-Encoding: chunked from client
and then sending with Content-Length to backend)
x-ref:
"Lighty returns HTTP 411 Length Required with proxy and streaming requests/reponses body"
https://redmine.lighttpd.net/issues/3006
NB: r->tmp_buf == srv->tmp_buf (pointer is copied for quicker access)
NB: request read and write chunkqueues currently point to connection
chunkqueues; per-request and per-connection chunkqueues are
not distinct from one another
con->read_queue == r->read_queue
con->write_queue == r->write_queue
NB: in the future, a separate connection config may be needed for
connection-level module hooks. Similarly, might need to have
per-request chunkqueues separate from per-connection chunkqueues.
Should probably also have a request_reset() which is distinct from
connection_reset().
Glenn Strauss