Commit Graph

205 Commits

Author SHA1 Message Date
Glenn Strauss 18ed51f06d [mod_proxy] Length Req if proxy forcing HTTP/1.0
return 411 Length Required if mod_proxy configured to force HTTP/1.0
to backend and configured to stream request body, and client request
has a request body but did not provide Content-Length.
2021-10-27 04:16:38 -04:00
Glenn Strauss e8a6ed6e35 [core] thwart h2c smuggling when Upgrade enabled
Existing behavior: mod_proxy *does not* forward Upgrade header
unless explicitly enabled in lighttpd.conf (default: not enabled)
  (proxy.header += ("upgrade" => "enable"))

mod_cgi previously used to forward Upgrade request header, but would
remove Upgrade response header if cgi.upgrade was not explicitly enabled
  (cgi.upgrade = "enable")

This patch thwarts h2c smuggling when lighttpd.conf has also been
explicitly configured to pass "Upgrade" request header

  "h2c Smuggling: Request Smuggling Via HTTP/2 Cleartext (h2c)"
2021-10-27 04:16:38 -04:00
Glenn Strauss 4d99d9b78a [multiple] check feature flags funcs; code reuse
2021-09-30 17:34:03 -04:00
Glenn Strauss f19f71625c [multiple] internal control for backend read bytes
separate internal control for backend max_per_read

When not streaming, large reads will be flushed to temp files on disk.
When streaming, use a smaller buffer to help reduce memory usage.

When not streaming, attempt to read and empty kernel socket bufs.
  (e.g. MAX_READ_LIMIT 256k)

When writing to sockets (or pipes) attempt to fill kernel socket bufs.
  (e.g. MAX_WRITE_LIMIT 256k)
2021-09-28 11:05:55 -04:00
Glenn Strauss 9fe8fbaa72 [multiple] http_method_buf()
- http_method_buf() returns (const buffer *)
- comment out unused get_http_status_name()
- inline func for http_append_method()

config processing requires a persistent buffer for method on the
off-chance that the config performed a capturing regex match in
$HTTP["method"] condition and used it later (e.g. in mod_rewrite)
(Prior behavior using r->tmp_buf was undefined in this case)
2021-08-27 02:16:54 -04:00
Glenn Strauss 18e96334d6 [core] proxy_create_env() tweaks
reorder some code for better asm

proxy_set_Forwarded() sets multiple request headers, and does so prior
to walking all request headers to create request to backend.  This is
done so that specific already-existing request headers from client are
overwritten (intentionally) in proxy_set_Forwarded().

Expect header is handled, but not expected since client-sent Expect
header is handled (and unset) in connection_handle_read_post_state()
2021-08-27 02:16:54 -04:00
Glenn Strauss 9f82ba8fab [mod_proxy] proxy_response_headers load v earlier
proxy_response_headers() issue variable load slightly earlier
2021-08-27 02:16:54 -04:00
Glenn Strauss f1e8a82f1a [multiple] inline struct in con->dst_addr_buf
(mod_extforward recently changed to use buffer_move() to save addr
 instead of swapping pointers)
2021-08-27 02:16:54 -04:00
Glenn Strauss af3df29ae8 [multiple] reduce redundant NULL buffer checks
This commit is a large set of code changes and results in removal of
hundreds, perhaps thousands, of CPU instructions, a portion of which
are on hot code paths.

Most (buffer *) used by lighttpd are not NULL, especially since buffers
were inlined into numerous larger structs such as request_st and chunk.

In the small number of instances where that is not the case, a NULL
check is often performed earlier in a function where that buffer is
later used with a buffer_* func.  In the handful of cases that remained,
a NULL check was added, e.g. with r->http_host and r->conf.server_tag.

- check for empty strings at config time and set value to NULL if blank
  string will be ignored at runtime; at runtime, simple pointer check
  for NULL can be used to check for a value that has been set and is not
  blank ("")
- use buffer_is_blank() instead of buffer_string_is_empty(),
  and use buffer_is_unset() instead of buffer_is_empty(),
  where buffer is known not to be NULL so that NULL check can be skipped
- use buffer_clen() instead of buffer_string_length() when buffer is
  known not to be NULL (to avoid NULL check at runtime)
- use buffer_truncate() instead of buffer_string_set_length() to
  truncate string, and use buffer_extend() to extend

Examples where buffer known not to be NULL:
  - cpv->v.b from config_plugin_values_init is not NULL if T_CONFIG_BOOL
    (though we might set it to NULL if buffer_is_blank(cpv->v.b))
  - address of buffer is arg (&foo)
    (compiler optimizer detects this in most, but not all, cases)
  - buffer is checked for NULL earlier in func
  - buffer is accessed in same scope without a NULL check (e.g. b->ptr)

internal behavior change:
  callers must not pass a NULL buffer to some funcs.
  - buffer_init_buffer() requires non-null args
  - buffer_copy_buffer() requires non-null args
  - buffer_append_string_buffer() requires non-null args
  - buffer_string_space() requires non-null arg
2021-08-27 02:16:53 -04:00
Glenn Strauss 82abd16dd7 [mod_proxy] send HTTP/1.0 to backend if no Host
send HTTP/1.0 request to backend if no Host header sent with request

(If Host header is present with an HTTP/1.0 request from client, then
 lighttpd can still make an HTTP/1.1 request to backends)
2021-04-05 13:24:51 -04:00
Glenn Strauss dc01487ea6 [multiple] use buffer_append_* aggregates
reduces the number of round-trips into some frequently-called routines
2021-04-02 01:16:40 -04:00
Glenn Strauss 26f354cb37 [multiple] http_header APIs to reduce str copies 2021-03-26 22:38:36 -04:00
Glenn Strauss 38c8735850 [multiple] optimize primitives, buffer_extend()
optimize buffer_* primitives

Other than buffer_string_set_length(), reallocate with one power-2 step
in size (or use the requested size, if larger).  This replaces the fixed
BUFFER_PIECE_SIZE round-up of only 64 bytes extension each reallocation,
which could lead to excessive reallocations in some scenarios.

buffer_extend() convenience routine to prep for batch append
(combines buffer_string_prepare_append() and buffer_commit())

mod_fastcgi, mod_scgi, mod_proxy and others now leverage buffer_extend()

mod_scgi directly performs little-endian encoding of short ints

http_response_write_header() optimizes writing response header,
leveraging buffer_extend()

modify mod_proxy to append line ends
similar to how it is done in http_response_write_header()
(removes one call to buffer_append_string_len())
2021-03-26 07:33:42 -04:00
Glenn Strauss 891007fb6a [multiple] use HTTP_HEADER_* enum before strcmp
When known, use HTTP_HEADER_* enum before string comparisons
2021-01-07 08:58:30 -05:00
Glenn Strauss 2ecbe5948d [mod_proxy] fix sending of initial reqbody chunked
fix sending of initial reqbody chunked to backend
2020-12-24 01:15:55 -05:00
Glenn Strauss fe5740d5e5 [mod_proxy] proxy.header = ("force-http10" => ...)
compatibility option to force HTTP/1.0 requests to mod_proxy backend
proxy.header += ("force-http10" => "disable")  (default)

If proxy.header is set (for any options), it overrides the global
server.feature-flags += ("proxy.force-http10" => "disable")
2020-12-16 02:00:17 -05:00
Glenn Strauss 730c932e3c [multiple] more forgiving config str to boolean (fixes #3036)
more consistent use of shared code config_plugin_value_tobool()

(thx tow-conf)

  "The on/off keywords in boolean configuration options is inconsistent, which might be misleading and error-prone."
2020-11-16 01:39:14 -05:00
Glenn Strauss 81029b8b51 [multiple] inline chunkqueue where always alloc'd
inline struct chunkqueue where always allocated in other structs

(memory locality)
2020-10-11 12:19:27 -04:00
Glenn Strauss 9c8981a7d1 [core] tst,set,clr macros for r->{rqst,resp}_htags 2020-10-11 12:19:26 -04:00
Glenn Strauss 97e314fc9e [multiple] inline chunkqueue_length() 2020-10-11 12:19:26 -04:00
Glenn Strauss 346280fed7 [mod_proxy] do not forward Expect: 100-continue
do not forward Expect: 100-continue to backend
since we do not handle HTTP/1.1 100 Continue response
2020-08-10 12:54:33 -04:00
Glenn Strauss f7919c1ae3 [mod_proxy] send HTTP/1.1 requests to backends
For prior behavior (HTTP/1.0 requests to backend), force HTTP/1.0 with:
  server.feature-flags = ("proxy.force-http10" => "enable")
2020-08-02 07:47:42 -04:00
Glenn Strauss 33c8cf41db [multiple] rename connection_reset hook to request
rename connection_reset to handle_request_reset
2020-08-02 07:47:41 -04:00
Glenn Strauss 28f1867c11 quiet clang analyzer scan-build warnings
(expansion of buffer_string_lenth() inline function and CONST_BUF_LEN()
 macro, which always check for NULL, appears to cause the analyzer to
 believe that a pointer might be NULL in cases where it otherwise can
 not be NULL)

2020-07-08 22:51:32 -04:00
Glenn Strauss 6d62a498a2 [core] more precise check for request stream flags 2020-07-08 22:51:31 -04:00
Glenn Strauss bcddbe186f [mod_proxy] stream request using HTTP/1.1 chunked (fixes #3006)
stream request body using HTTP/1.1 Transfer-Encoding: chunked

(Note: if backend proxy target does not support HTTP/1.1,
 then do not use = 1 or 2)

If not streaming to backend, collect request body
(now supporting Transfer-Encoding: chunked from client
 and then sending with Content-Length to backend)

  "Lighty returns HTTP 411 Length Required with proxy and streaming requests/reponses body"
2020-07-08 19:54:29 -04:00
Glenn Strauss 7c7f8c467c [multiple] split con, request (very large change)
NB: r->tmp_buf == srv->tmp_buf (pointer is copied for quicker access)

NB: request read and write chunkqueues currently point to connection
    chunkqueues; per-request and per-connection chunkqueues are
    not distinct from one another
      con->read_queue  == r->read_queue
      con->write_queue == r->write_queue

NB: in the future, a separate connection config may be needed for
    connection-level module hooks.  Similarly, might need to have
    per-request chunkqueues separate from per-connection chunkqueues.
    Should probably also have a request_reset() which is distinct from
2020-07-08 19:54:29 -04:00
Glenn Strauss 31d9495330 [core] store subrequest_handler instead of mode
store pointer to module in handler_module instead of con->mode id
2020-07-08 19:54:29 -04:00
Glenn Strauss 100dfaa3f3 [core] move plugin_ctx into (request_st *)
NB: in the future, a separate plugin_ctx may be needed for
    connection-level plugins to keep state across multiple requests
2020-07-08 19:54:29 -04:00
Glenn Strauss 8131e4396d [core] move addtl request-specific struct members 2020-07-08 19:54:29 -04:00
Glenn Strauss 1474be7859 [core] move addtl request-specific struct members 2020-07-08 19:54:29 -04:00
Glenn Strauss af5df35275 [core] rename content_length to reqbody_length
rename content_length to reqbody_length in request,
to more easily differentiate request body length
from response content_length
2020-07-08 19:54:29 -04:00
Glenn Strauss 19985261b2 [core] convenience macros to check req methods 2020-07-08 19:54:29 -04:00
Glenn Strauss 03b4c993d2 [multiple] generic config array type checking 2020-07-08 19:54:28 -04:00
Glenn Strauss f24e6d696a [multiple] plugin_stats array
use global rather than passing around (server *) just for that

li_itostrn() and li_utostrn() return string length
(rather than requiring subsequent strlen() to find length)
2020-07-08 19:54:28 -04:00
Glenn Strauss 50bdb55de8 [multiple] connection hooks no longer get (srv *)
(explicit (server *) not passed; available in con->srv)
2020-07-08 19:54:28 -04:00
Glenn Strauss 010c28949c [multiple] prefer (connection *) to (srv *)
convert all log_error_write() to log_error() and pass (log_error_st *)

use con->errh in preference to srv->errh (even though currently same)

avoid passing (server *) when previously used only for logging (errh)
2020-07-08 19:54:28 -04:00
Glenn Strauss cec18f4381 [core] gw_exts_clear_check_local() 2020-07-08 18:08:52 -04:00
Glenn Strauss b73949e03f [multiple] plugin.c handles common FREE_FUNC code
(simpler for modules; less boilerplate to cut-n-paste)
2020-07-08 18:08:51 -04:00
Glenn Strauss 4a6fe83837 [multiple] gw_backends config_plugin_values_init() 2020-07-08 18:08:51 -04:00
Glenn Strauss e2de4e581e [core] const char *name in struct plugin
put void *data (always used) as first member of struct plugin

add int nconfig member to PLUGIN_DATA

calloc() inits p->data to NULL
2020-05-23 17:59:29 -04:00
Glenn Strauss 36f64b26a1 [core] simpler config_check_cond()
optimize for common case where condition has been evaluated for
the request and a cached result exists

(also: begin isolating data_config)
2020-05-23 17:59:29 -04:00
Glenn Strauss feb21b3da2 [core] inline header and env arrays into con 2020-05-23 17:59:29 -04:00
Glenn Strauss c2238256e2 [core] inline array as part of data_array value
(instead of value being (array *))
2020-02-24 11:15:32 -05:00
Glenn Strauss 6eb34ef5ab [core] add const to callers of http_header_*_get()
(The few places where value is modified in-place were not made const)
2020-02-24 11:15:32 -05:00
Glenn Strauss 601c572c39 [core] inline buffer as part of data_string value
(instead of value being (buffer *))
2020-02-24 11:15:32 -05:00
Glenn Strauss 47a758f959 [core] inline buffer key for *_patch_connection()
handle buffer key as part of DATA_UNSET in *_patch_connection()
(instead of key being (buffer *))
2020-02-24 11:15:32 -05:00
Glenn Strauss ad9b7e009b [core] inline buffer as part of DATA_UNSET key
(instead of key being (buffer *))
2020-02-24 11:15:32 -05:00
Glenn Strauss 83535bbef3 [core] differentiate array_get_* for ro and rw
array_get_element_klen() is now intended for read-only access
array_get_data_unset() is used by config processing for r/w access
array_get_buf_ptr() is used for r/w access to ds->value (string buffer)
2020-02-24 11:15:32 -05:00
Glenn Strauss e20b5318d5 [core] use buffer_eq_icase_ssn func
specialized buffer_eq_icase_ssn func replace strncasecmp()
in cases where string lengths are known to be at least as
large as the len being compared case-insensitively
2019-06-06 02:48:43 -04:00