Commit Graph

155 Commits

Author SHA1 Message Date
Glenn Strauss 21f0dabef4 [multiple] replace buffer_is_equal_caseless_string
buffer_is_equal_caseless_string() -> buffer_eq_icase_slen()
2021-01-07 09:08:53 -05:00
Glenn Strauss e9309ae6e6 [core] li_restricted_strtoint64()
(make public for reuse)
2020-12-15 22:31:08 -05:00
Glenn Strauss 35fa47d802 [core] quiet more request parse errs unless debug
quiet more request parse errors unless debug enabled with
  debug.log-request-header-on-error = "enable"

  "invalid character in URI -> 400 config?"
2020-11-27 08:12:21 -05:00
Glenn Strauss 3fbb5773e9 [core] http_header_e <=> lshpack_static_hdr_idx
map enum http_header_e to enum lshpack_static_hdr_idx
map enum lshpack_static_hdr_idx to enum http_header_e
2020-10-11 12:19:26 -04:00
Glenn Strauss 92e5a021d2 [core] http_header_hkey_get_lc() for HTTP/2
header field-names are required to be lowercase for HTTP/2
so specialize http_header_hkey_get() as http_header_hkey_get_lc()
for simpler comparison

lowercase field-names in http_headers[], as it does not matter for
buffer_eq_icase_ssn(), which is used with http_headers[] for HTTP/1.x
2020-10-11 12:19:26 -04:00
Glenn Strauss 9c8981a7d1 [core] tst,set,clr macros for r->{rqst,resp}_htags 2020-10-11 12:19:26 -04:00
Glenn Strauss c58b95f297 [core] light_isupper(), light_islower()
more efficient char checks
(replace one comparision and one branch with one subtraction)
2020-10-11 12:19:26 -04:00
Glenn Strauss 1d9709b8ab [core] update HTTP status codes list from IANA 2020-10-11 12:19:26 -04:00
Glenn Strauss 5547530a01 [core] do not require '\0' term for k,v hdr parse
no longer require '\0' terminated z-string for key,value header parsing
2020-10-11 12:19:26 -04:00
Glenn Strauss 8fc8ab891a [core] http_request_parse_header() specialized
http_request_parse_header() specialized for HTTP/2 request headers
to be parsed as each field-name and value is HPACK-decoded; send headers
directly from HPACK decoder, rather than double-buffering in chunkqueue

http_request_headers_process_h2() for post-processing
2020-10-11 11:43:06 -04:00
Glenn Strauss 1bf4555800 [core] update comment 2020-10-11 11:43:06 -04:00
Glenn Strauss 8d7e9439b5 [core] connection transition to HTTP/2; incomplete

(basic framework with some stub functions; incomplete)
(subsequently incrementally updated using git rebase)

HTTP/2 via TLS ALPN extension  (TLS)
HTTP/2 via Upgrade: h2c        (cleartext)
HTTP/2 via Prior Knowledge     (cleartext)

server.feature-flags += ("server.h2proto" => "enable")
to enable HTTP/2
server.feature-flags += ("server.h2c" => "enable")
to enable upgrade to HTTP/2 over cleartext http
2020-10-11 11:43:06 -04:00
Glenn Strauss 6971c6c811 [core] move http_request_headers_process()
move http_request_headers_process() to request.[ch]
2020-10-03 09:05:38 -04:00
Glenn Strauss 517e32785e [core] HTTP2-Settings 2020-10-03 09:05:38 -04:00
Glenn Strauss f37c90bccc [core] parse h2 pseudo-headers 2020-10-03 09:05:38 -04:00
Glenn Strauss c18f442a63 [multiple] add summaries to top of some modules 2020-07-08 22:51:31 -04:00
Glenn Strauss 6876c16be0 [core] RFC-strict parse of Content-Length
augment simple strtoll() which allowed number to begin with '+'

This is not exploitable for HTTP Request Smuggling since lighttpd
mod_proxy sends "Connection: close" to backends, and other CGI-based
backends reconstitute CONTENT_LENGTH in the environment without '+'.

(thx Amit Klein, Safebreach)
2020-07-08 22:51:31 -04:00
Glenn Strauss c752d4696e [multiple] correct misspellings in comments
  "Script for fixing spelling errors with codespell"
2020-07-08 19:54:30 -04:00
Glenn Strauss 41a268b805 [core] parse header label before end of line
parse header label for colon before checking end of line for whitespace
2020-07-08 19:54:29 -04:00
Glenn Strauss 3f4f934485 [core] http_response_comeback()
group HANDLER_COMEBACK logic in http_response_comeback() and call it
from places that reset state in order to (sometimes partially) reprocess
a request.  This includes error handler (server.error-handler),
r->handler_module when cgi.local-redir, and looping in
http_response_prepare() when modules make changes to the request and
return HANDLER_COMEBACK (e.g. mod_rewrite, mod_magnet, mod_cml)

Also, set r->conditional_is_valid closer to where elements are set
(and become valid for use in condition checks), and parse target
in http_request_parse() instead of http_response_prepare()
2020-07-08 19:54:29 -04:00
Glenn Strauss a0029b21a1 [core] remove r->uri.path_raw; generate as needed
(r->uri.path_raw previously duplicated from r->target, minus query-part)
2020-07-08 19:54:29 -04:00
Glenn Strauss d013d0abd3 [core] http_request_parse_target()
http_request_parse_target() split from http_response_prepare()
2020-07-08 19:54:29 -04:00
Glenn Strauss 9914bb297b [core] C99 restrict on some base funcs
buffer.[ch], chunk.[ch], request.[ch], sock_addr.[ch]
2020-07-08 19:54:29 -04:00
Glenn Strauss 7c7f8c467c [multiple] split con, request (very large change)
NB: r->tmp_buf == srv->tmp_buf (pointer is copied for quicker access)

NB: request read and write chunkqueues currently point to connection
    chunkqueues; per-request and per-connection chunkqueues are
    not distinct from one another
      con->read_queue  == r->read_queue
      con->write_queue == r->write_queue

NB: in the future, a separate connection config may be needed for
    connection-level module hooks.  Similarly, might need to have
    per-request chunkqueues separate from per-connection chunkqueues.
    Should probably also have a request_reset() which is distinct from
2020-07-08 19:54:29 -04:00
Glenn Strauss 8131e4396d [core] move addtl request-specific struct members 2020-07-08 19:54:29 -04:00
Glenn Strauss af5df35275 [core] rename content_length to reqbody_length
rename content_length to reqbody_length in request,
to more easily differentiate request body length
from response content_length
2020-07-08 19:54:29 -04:00
Glenn Strauss 6870b0f55b [core] pass (request_st *) to request.c funcs
instead of (struct connection *)
2020-07-08 19:54:29 -04:00
Glenn Strauss af3313bfa5 [core] pass http_parseopts around request.c 2020-07-08 19:54:29 -04:00
Glenn Strauss 6748a58cca [core] pass scheme port to http_request_parse()
con->proto_default_port is a property of the connection,
which influences the default port used in host normalization
2020-07-08 19:54:29 -04:00
Glenn Strauss 057d83c50b [core] move keep_alive flag into request_st 2020-07-08 19:54:29 -04:00
Glenn Strauss 33430ce2b0 [core] copy some srv->srvconf into con->conf
(for memory locality and to reduce pointer chasing)
2020-07-08 19:54:29 -04:00
Glenn Strauss 19985261b2 [core] convenience macros to check req methods 2020-07-08 19:54:29 -04:00
Glenn Strauss 1dd58c5ad8 [multiple] con->proto_default_port 2020-07-08 19:54:28 -04:00
Glenn Strauss 034d7d6734 [core] http_request_host_policy w/ http_parseopts
pass http_parseopts arg (instead of con) to http_request_host_policy()
2020-07-08 19:54:28 -04:00
Glenn Strauss 0ff60d8218 [multiple] rename r to rc rv rd wr to be different
variable rename
2020-07-08 19:54:28 -04:00
Glenn Strauss 010c28949c [multiple] prefer (connection *) to (srv *)
convert all log_error_write() to log_error() and pass (log_error_st *)

use con->errh in preference to srv->errh (even though currently same)

avoid passing (server *) when previously used only for logging (errh)
2020-07-08 19:54:28 -04:00
Glenn Strauss 6eb34ef5ab [core] add const to callers of http_header_*_get()
(The few places where value is modified in-place were not made const)
2020-02-24 11:15:32 -05:00
Glenn Strauss d58787189a [core] handle common case of alnum or - field-name
handle common case of alnum or - field-name before calling
2020-02-24 11:15:32 -05:00
Glenn Strauss 71a7b54947 [core] more 'const' in request.c prototypes 2020-02-24 11:15:32 -05:00
Glenn Strauss 36f3206a4c [core] pass ptr to http_request_parse() 2020-02-24 11:15:32 -05:00
Glenn Strauss 63e32e8100 [core] perf: HTTP header parsing using \n offsets 2020-02-24 11:15:32 -05:00
Glenn Strauss fa4ab19275 [core] reduce use of struct parse_header_state 2020-02-24 11:15:32 -05:00
Glenn Strauss cdf653f8ce [core] pass con around request, not srv and con
In the cold cases where srv is needed, obtain from con->srv
2020-02-24 11:15:27 -05:00
Glenn Strauss c22ec74770 [core] do not pass srv to http header parsing func
srv is retrieved from con->srv in the few cases where needed
2020-02-24 11:14:45 -05:00
Glenn Strauss e2b4c309f6 [core] http_header_str_contains_token() 2020-02-24 11:14:45 -05:00
Glenn Strauss 6f803af03c [core] perf: request processing 2020-02-24 11:14:43 -05:00
Glenn Strauss f37c16aadd [core] fix one-byte OOB read (underflow)
In some circumstances, if the character on the heap prior to the
beginning of the request is '\r', then it would be overwritten with '\0'

With default compiler flags, this does not appear to occur in practice
and we therefore believe it to be a low-probability vulnerability.

(thx Antonio Morales)

This issue was discovered and reported by GSL team member @
<>  (Antonio Morales)
2020-01-31 16:54:59 -05:00
Glenn Strauss 66624b375b [core] reject Transfer-Encoding + Content-Length (#2985)
reject requests with both Transfer-Encoding and Content-Length
as recommended in RFC 7230 Section 3.3.3.

strict header parsing is enabled by default in lighttpd.  However,
if explicitly disabled in lighttpd.conf, lighttpd will continue to
accept Transfer-Encoding and Content-Length in the same request,
and will ignore (and remove) Content-Length before passing to backend.
  UNSAFE: server.http-parseopts = ( "header-strict" => "disable" )
  This is NOT RECOMMENDED since doing so disables other protections
  provided by lighttpd strict http header parsing.

RFC7230 Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing
  3.3.3.  Message Body Length
  If a message is received with both a Transfer-Encoding and a
  Content-Length header field, the Transfer-Encoding overrides the
  Content-Length.  Such a message might indicate an attempt to
  perform request smuggling (Section 9.5) or response splitting
  (Section 9.4) and ought to be handled as an error.  A sender MUST
  remove the received Content-Length field prior to forwarding such
  a message downstream.

  stricter request header parsing
2020-01-26 00:41:04 -05:00
Glenn Strauss 61f85d14ee [core] reject WS following header field-name (fixes #2985)
reject whitespace following request header field-name and before colon
Such whitespace is forbidden in RFC 7230 Section 3.2.4.

strict header parsing is enabled by default in lighttpd.  However,
if explicitly disabled in lighttpd.conf, lighttpd will continue to
accept (and re-format) such field-names before passing to any backend.
  UNSAFE: server.http-parseopts = ( "header-strict" => "disable" )
  This is NOT RECOMMENDED since doing so disables other protections
  provided by lighttpd strict http header parsing.

(thx fedormixalich)

  stricter request header parsing
2020-01-26 00:40:20 -05:00
Glenn Strauss e3874a20ba [core] use buffer_eq_icase* funcs
specialized buffer_eq_icase* funcs replace buffer_caseless_compare()
2019-06-06 02:48:43 -04:00