http_request_parse_header() specialized for HTTP/2 request headers
to be parsed as each field-name and value is HPACK-decoded; send headers
directly from HPACK decoder, rather than double-buffering in chunkqueue
http_request_headers_process_h2() for post-processing
(subsequently incrementally updated using git rebase)
huge props and many thank yous to writers of testing tools used while
developing HTTP/2 support in lighttpd:
h2spec - conformance testing tool for HTTP/2 implementation
https://github.com/summerwind/h2spec
h2load - HTTP/2 benchmarking tool
https://nghttp2.org/documentation/h2load-howto.html
curl - command line tool and library for transferring data with URLs
https://curl.haxx.se/
reset connection counters per connection, not per request
adjust mod_accesslog and mod_rrdtool usage
continue to count mod_rrdtool per request rather than per connection
so that data is updated after each request, rather than aggregated
to the end of a potentially long-lived connection with many keep-alives.
decode Transfer-Encoding: chunked from gw (gateway backends)
Transfer-Encoding: chunked is a hop-by-hop header.
Handling chunked encoding remove a hurdle for mod_proxy to send HTTP/1.1
requests to backends and be able to handle HTTP/1.1 responses.
Other backends ought not to send Transfer-Encoding: chunked, but in
practice, some implementations do.
NB: r->tmp_buf == srv->tmp_buf (pointer is copied for quicker access)
NB: request read and write chunkqueues currently point to connection
chunkqueues; per-request and per-connection chunkqueues are
not distinct from one another
con->read_queue == r->read_queue
con->write_queue == r->write_queue
NB: in the future, a separate connection config may be needed for
connection-level module hooks. Similarly, might need to have
per-request chunkqueues separate from per-connection chunkqueues.
Should probably also have a request_reset() which is distinct from
connection_reset().
server.http-parseopts = ( ... ) URL normalization options
Note: *not applied* to CONNECT method
Note: In a future release, URL normalization likely enabled by default
(normalize URL, reject control chars, remove . and .. path segments)
To prepare for this change, lighttpd.conf configurations should
explicitly select desired behavior by enabling or disabling:
server.http-parseopts = ( "url-normalize" => "enable", ... )
server.http-parseopts = ( "url-normalize" => "disable" )
x-ref:
"lighttpd ... compares URIs to patterns in the (1) url.redirect and (2) url.rewrite configuration settings before performing URL decoding, which might allow remote attackers to bypass intended access restrictions, and obtain sensitive information or possibly modify data."
https://www.cvedetails.com/cve/CVE-2008-4359/
"Rewrite/redirect rules and URL encoding"
https://redmine.lighttpd.net/issues/1720
provide standard types in first.h instead of base.h
provide lighttpd types in base_decls.h instead of settings.h
reduce headers exposed by headers for core data structures
do not expose <pcre.h> or <stdlib.h> in headers
move stat_cache_entry to stat_cache.h
reduce use of "server.h" and "base.h" in headers
server.http-parseopt-header-strict = "enable"
server.http-parseopt-host-strict = "enable" (implies host-normalize)
server.http-parseopt-host-normalize = "disable"
defaults retain current behavior, which is strict header parsing
and strict host parsing, with enhancement to normalize IPv4 address
and port number strings.
For lighttpd tests, these need to be enabled (and are by default)
For marginally faster HTTP header parsing for benchmarks, disable these.
To allow
- underscores in hostname
- hypen ('-') at beginning of hostname
- all-numeric TLDs
server.http-parseopt-host-strict = "disable"
x-ref:
"lighttpd doesn't allow underscores in host names"
https://redmine.lighttpd.net/issues/551
"hyphen in hostname"
https://redmine.lighttpd.net/issues/1086
"a numeric tld"
https://redmine.lighttpd.net/issues/1184
"Numeric tld's"
https://redmine.lighttpd.net/issues/2143
"Bad Request"
https://redmine.lighttpd.net/issues/2258
"400 Bad Request when using Numeric TLDs"
https://redmine.lighttpd.net/issues/2281
To allow a variety of numerical formats to be converted to IP addresses
server.http-parseopt-host-strict = "disable"
server.http-parseopt-host-normalize = "enable"
x-ref:
"URL encoding leads to "400 - Bad Request""
https://redmine.lighttpd.net/issues/946
"400 Bad Request when using IP's numeric value ("ip2long()")"
https://redmine.lighttpd.net/issues/1330
To allow most 8-bit and 7-bit chars in headers
server.http-parseopt-header-strict = "disable" (not recommended)
x-ref:
"Russian letters not alowed?"
https://redmine.lighttpd.net/issues/602
"header Content-Disposition with russian '?' (CP1251, ascii code 255) causes error"
https://redmine.lighttpd.net/issues/1016
Glenn Strauss
Jan Kneschke