Commit Graph

181 Commits

Author SHA1 Message Date
Glenn Strauss 6d62a498a2 [core] more precise check for request stream flags 2020-07-08 22:51:31 -04:00
Glenn Strauss bcddbe186f [mod_proxy] stream request using HTTP/1.1 chunked (fixes #3006)
stream request body using HTTP/1.1 Transfer-Encoding: chunked

(Note: if backend proxy target does not support HTTP/1.1,
 then do not use server.stream-request-body = 1 or 2)

If not streaming to backend, collect request body
(now supporting Transfer-Encoding: chunked from client
 and then sending with Content-Length to backend)

x-ref:
  "Lighty returns HTTP 411 Length Required with proxy and streaming requests/reponses body"
  https://redmine.lighttpd.net/issues/3006
2020-07-08 19:54:29 -04:00
Glenn Strauss 7c7f8c467c [multiple] split con, request (very large change)
NB: r->tmp_buf == srv->tmp_buf (pointer is copied for quicker access)

NB: request read and write chunkqueues currently point to connection
    chunkqueues; per-request and per-connection chunkqueues are
    not distinct from one another
      con->read_queue  == r->read_queue
      con->write_queue == r->write_queue

NB: in the future, a separate connection config may be needed for
    connection-level module hooks.  Similarly, might need to have
    per-request chunkqueues separate from per-connection chunkqueues.
    Should probably also have a request_reset() which is distinct from
    connection_reset().
2020-07-08 19:54:29 -04:00
Glenn Strauss 31d9495330 [core] store subrequest_handler instead of mode
store pointer to module in handler_module instead of con->mode id
2020-07-08 19:54:29 -04:00
Glenn Strauss 100dfaa3f3 [core] move plugin_ctx into (request_st *)
NB: in the future, a separate plugin_ctx may be needed for
    connection-level plugins to keep state across multiple requests
2020-07-08 19:54:29 -04:00
Glenn Strauss 8131e4396d [core] move addtl request-specific struct members 2020-07-08 19:54:29 -04:00
Glenn Strauss 1474be7859 [core] move addtl request-specific struct members 2020-07-08 19:54:29 -04:00
Glenn Strauss af5df35275 [core] rename content_length to reqbody_length
rename content_length to reqbody_length in request,
to more easily differentiate request body length
from response content_length
2020-07-08 19:54:29 -04:00
Glenn Strauss 19985261b2 [core] convenience macros to check req methods 2020-07-08 19:54:29 -04:00
Glenn Strauss 03b4c993d2 [multiple] generic config array type checking 2020-07-08 19:54:28 -04:00
Glenn Strauss f24e6d696a [multiple] plugin_stats array
use global rather than passing around (server *) just for that

li_itostrn() and li_utostrn() return string length
(rather than requiring subsequent strlen() to find length)
2020-07-08 19:54:28 -04:00
Glenn Strauss 50bdb55de8 [multiple] connection hooks no longer get (srv *)
(explicit (server *) not passed; available in con->srv)
2020-07-08 19:54:28 -04:00
Glenn Strauss 010c28949c [multiple] prefer (connection *) to (srv *)
convert all log_error_write() to log_error() and pass (log_error_st *)

use con->errh in preference to srv->errh (even though currently same)

avoid passing (server *) when previously used only for logging (errh)
2020-07-08 19:54:28 -04:00
Glenn Strauss cec18f4381 [core] gw_exts_clear_check_local() 2020-07-08 18:08:52 -04:00
Glenn Strauss b73949e03f [multiple] plugin.c handles common FREE_FUNC code
(simpler for modules; less boilerplate to cut-n-paste)
2020-07-08 18:08:51 -04:00
Glenn Strauss 4a6fe83837 [multiple] gw_backends config_plugin_values_init() 2020-07-08 18:08:51 -04:00
Glenn Strauss e2de4e581e [core] const char *name in struct plugin
put void *data (always used) as first member of struct plugin

add int nconfig member to PLUGIN_DATA

calloc() inits p->data to NULL
2020-05-23 17:59:29 -04:00
Glenn Strauss 36f64b26a1 [core] simpler config_check_cond()
optimize for common case where condition has been evaluated for
the request and a cached result exists

(also: begin isolating data_config)
2020-05-23 17:59:29 -04:00
Glenn Strauss feb21b3da2 [core] inline header and env arrays into con 2020-05-23 17:59:29 -04:00
Glenn Strauss c2238256e2 [core] inline array as part of data_array value
(instead of value being (array *))
2020-02-24 11:15:32 -05:00
Glenn Strauss 6eb34ef5ab [core] add const to callers of http_header_*_get()
(The few places where value is modified in-place were not made const)
2020-02-24 11:15:32 -05:00
Glenn Strauss 601c572c39 [core] inline buffer as part of data_string value
(instead of value being (buffer *))
2020-02-24 11:15:32 -05:00
Glenn Strauss 47a758f959 [core] inline buffer key for *_patch_connection()
handle buffer key as part of DATA_UNSET in *_patch_connection()
(instead of key being (buffer *))
2020-02-24 11:15:32 -05:00
Glenn Strauss ad9b7e009b [core] inline buffer as part of DATA_UNSET key
(instead of key being (buffer *))
2020-02-24 11:15:32 -05:00
Glenn Strauss 83535bbef3 [core] differentiate array_get_* for ro and rw
array_get_element_klen() is now intended for read-only access
array_get_data_unset() is used by config processing for r/w access
array_get_buf_ptr() is used for r/w access to ds->value (string buffer)
2020-02-24 11:15:32 -05:00
Glenn Strauss e20b5318d5 [core] use buffer_eq_icase_ssn func
specialized buffer_eq_icase_ssn func replace strncasecmp()
in cases where string lengths are known to be at least as
large as the len being compared case-insensitively
2019-06-06 02:48:43 -04:00
Mohammed Sadiq 6a988bb0d0 [multiple] cleaner calloc use in SETDEFAULTS_FUNC
github: closes #99

x-ref:
  "cleaner calloc use in SETDEFAULTS_FUNC"
  https://github.com/lighttpd/lighttpd1.4/pull/99
2019-04-20 02:09:04 -04:00
Glenn Strauss 8167497899 [mod_proxy] pass Content-Length to backend if > 0
pass Content-Length to backend if > 0, even if GET or HEAD method
(and pass Content-Length: 0 for other methods if no request body)
2019-03-07 00:32:17 -05:00
Glenn Strauss a1b527e473 [multiple] reduce initial buffer sz if large POST (fixes #2922)
reduce initial buffer size if large POST to backend stored in temp files

regression in lighttpd 1.4.52

(thx rgenoud)

x-ref:
  "[regression] lighttpd gets killed after uploading a big file"
  https://redmine.lighttpd.net/issues/2922
2018-12-12 20:00:07 -05:00
Glenn Strauss fed4573fad [mod_proxy] silence coverity false positive 2018-11-25 23:33:13 -05:00
Glenn Strauss f69bd9cdb8 [core] perf: simple, quick buffer_clear()
quickly clear buffer instead of buffer_string_set_length(b, 0) or
buffer_reset(b).  Avoids free() of large buffers about to be reused,
or buffers that are module-scoped, persistent, and reused.

(buffer_reset() should still be used with buffers in connection *con
 when the data in the buffers is supplied by external, untrusted source)
2018-11-23 00:37:38 -05:00
Glenn Strauss f13db69012 [core] fix setting of headers previously reset (fixes #2919)
bug may result in long delays when using mod_deflate on connections
with keep-alive, as the result is sent without Content-Length or
Transfer-Encoding

(regression in lighttpd 1.4.51)

(thx GilGalaad)

x-ref:
  "high latency on 1.4.51 + proxy + deflate"
  https://redmine.lighttpd.net/boards/2/topics/8365
  https://redmine.lighttpd.net/issues/2919
2018-11-17 01:24:26 -05:00
Glenn Strauss 9f6a46738d [multiple] dynamic handlers hint backend header sz 2018-11-12 21:56:05 -05:00
Glenn Strauss 3d8d56d8de [multiple] perf: reuse large buffers w/ backend 2018-11-12 21:56:05 -05:00
Glenn Strauss f410431b58 [multiple] perf: use larger initial backend buffer
use larger initial backend buffer if the client request is large
2018-11-12 08:25:01 -05:00
Glenn Strauss 7e8beee577 [multiple] perf: power-2 alloc large headers
allocate large headers by power-2 as headers to backend is generated
(mod_proxy, mod_fastcgi, mod_scgi)
2018-10-29 09:00:46 -04:00
Glenn Strauss 610d5c03ab [multiple] perf: write headers to backend write cq
write headers directly to dynamic backend write chunkqueue
(mod_proxy, mod_fastcgi, mod_scgi)
2018-10-29 09:00:46 -04:00
Glenn Strauss b61ed6da2a [core] http_method_append() 2018-09-23 19:18:47 -04:00
Glenn Strauss a458c2e731 [mod_proxy,mod_wstunnel] copy full plugin_config (fixes #2903)
x-ref:
  "gw backend redesign"
  https://redmine.lighttpd.net/issues/2903
2018-09-23 18:01:58 -04:00
Glenn Strauss eebc1b0eec [mod_proxy] fix proxy.forwarded and proxy.replace-http-host (fixes #2902)
config settings were not being copied into proxy request context

x-ref:
  "mod_proxy's “proxy.forwarded” option seems ignored when used with mod_auth."
  https://redmine.lighttpd.net/issues/2902
2018-09-23 18:01:58 -04:00
Glenn Strauss 3dd3cde902 [core] abstraction layer for HTTP header manip
http_header.[ch]
convert existing calls to manip request/response headers
convert existing calls to manip environment array (often header-related)
2018-09-23 18:01:58 -04:00
Glenn Strauss a7c27c9f99 [core] code reuse with array_insert_key_value()
code reuse with array_insert_key_value() and related array manipulation
2018-09-16 05:18:05 -04:00
Glenn Strauss c56b21084e [core] http_kv.[ch] method, status, version str
move method, status, version strings from keyvalue.[ch] to http_kv.[ch]
2018-08-05 03:44:15 -04:00
Glenn Strauss 04d76e7afd [core] some header cleanup
provide standard types in first.h instead of base.h
provide lighttpd types in base_decls.h instead of settings.h
reduce headers exposed by headers for core data structures
  do not expose <pcre.h> or <stdlib.h> in headers
move stat_cache_entry to stat_cache.h
reduce use of "server.h" and "base.h" in headers
2018-04-08 22:22:23 -04:00
Glenn Strauss 26fb8d3ee6 [mod_proxy] fix segfault in Set-Cookie reverse map (fixes #2879)
fix segfault in reverse url-path mapping of Set-Cookie sent from backend
when proxy.header = ( "map-urlpath" => ( ... ) ) is used and there are
multiple Set-Cookie response headers with path= attributes which need to
be reverse mapped.

(thx ganto)

x-ref:
  "Segfault with proxy-header map-urlpath"
  https://redmine.lighttpd.net/issues/2879
2018-03-22 00:25:06 -04:00
Glenn Strauss 3770df2387 [mod_proxy] basic support for HTTP CONNECT method (#2060)
For security reasons, this supports only specific, pre-configured
target backends and not arbitrary CONNECT targets.

x-ref:
  "mod_connect"
  https://redmine.lighttpd.net/issues/2060
  "ssh over https tunnel"
  https://redmine.lighttpd.net/boards/2/topics/7805

  https://en.wikipedia.org/wiki/HTTP_tunnel
  https://nurdletech.com/linux-notes/ssh/via-http.html
2017-11-25 19:01:16 -05:00
Glenn Strauss 1367f60626 [core] isolate sock_addr manipulation 2017-10-29 01:23:19 -04:00
Glenn Strauss 9030cfaecf [core] SIGCHLD handle_waitpid hook for modules
centralize most waitpid() handling in core server, with hooks for
modules to be informed of pid and status when a process exits.

This enables faster discovery (and restart) of exited processes,
and also allows for lighttpd to manage backend processes in the
parent (master) process when server.max-worker > 0.
2017-09-10 16:24:34 -04:00
Glenn Strauss 45b970e69b [core] shared code for socket backends
common codebase for socket backends, based off mod_fastcgi with
some features added for mod_proxy

(mostly intended to reduce code duplication and enhance code isolation)

mod_fastcgi and mod_scgi can now use fastcgi.balance and scgi.balance
  for similar behavior as proxy.balance, but the balancing is per-host
  and not per-proc.  proxy.balance is also per-host and not per-proc.

mod_proxy and mod_scgi can now use proxy.map-extensions and
scgi.map-extensions, similar to fastcgi.map-extensions.

mod_fastcgi behavior change (affects only mod_status):
- statistics tags have been renamed from "fastcgi.*" to "gw.*"
  "fastcgi.backend.*"       -> "gw.backend.*"
  "fastcgi.active-requests" -> "gw.active-requests"
  ("fastcgi.requests" remains "fastcgi.requests")
  ("proxy.requests" is new)
  ("scgi.requests" is new)

mod_scgi behavior change (likely minor):
- removed scgi_proclist_sort_down() and scgi_proclist_sort_up().
  procs now chosen based on load as measured by num socket connnections

Note:
modules using gw_backend.[ch] are currently still independent modules.
If it had been written as a single module with fastcgi, scgi, proxy
implementations, then there would have been a chance of breaking some
existing user configurations where module ordering made a difference
for which module handled a given request, though for most people, this
would have made no difference.

Details about mod_fastcgi code transformations:
unsigned int debug -> int debug
fastcgi_env member removed from plugin_config
renamed "fcgi" and "fastcgi" to "gw", and "FCGI" to "GW"
reorganize routines for high-level and lower-level interfaces
some lower-level internal interfaces changed to use host,proc,debug
  args rather than knowing about higher-level (app) hctx and plugin_data
tabs->spaces and reformatting
2017-07-23 19:02:00 -04:00
Glenn Strauss c7492b6e0a [mod_proxy] store address family at config time 2017-07-15 22:42:15 -04:00