Commit Graph

191 Commits

Author SHA1 Message Date
Glenn Strauss f7fc8d8d75 [core] connection_joblist global
con->srv->joblist is expensive to traverse when cache is cold
and called from connection_handle_fdevent(); var made a global
2020-10-11 12:19:27 -04:00
Glenn Strauss a8398e4596 [core] defer handling FDEVENT_HUP and FDEVENT_ERR
defer handling FDEVENT_HUP and FDEVENT_ERR to after processing
(connection *) in order to have a chance to read data in kernel
socket buffers
2020-10-11 12:19:27 -04:00
Glenn Strauss 94c4c63773 [core] graceful shutdown timeout option
server.feature-flags += ("server.graceful-shutdown-timeout" => 10)

After receiving SIGINT or SIGUSR1, lighttpd will gracefully shutdown,
waiting for existing connections to complete.  In the case of SIGUSR1,
this wait occurs before restarting lighttpd.  The default timeout is
none (unlimited).

When "server.graceful-shutdown-timeout" option is set, it defines the
number of seconds that lighttpd will wait for existing connections to
complete before shutting down the connection.

Sites which expect large uploads or downloads, or those with very slow
clients, might want to set a much longer timeout, e.g 60 seconds

For more immediate graceful restarts, while still allowing existing
connections time to complete, sites should additionally consider
whether or not
  server.feature-flags += ("server.graceful-restart-bg" => "enable")
is appropriate and compatible with their lighttpd.conf settings
2020-10-11 12:19:27 -04:00
Glenn Strauss 352d5d776d [core] graceful and immediate restart option
graceful and (nearly) immediate lighttpd restart option

For *some* configurations, it *may* be safe to background the current
lighttpd server (or workers) to continue processing active requests
and, in parallel, to start up a new lighttpd server with a new
configuration.  For other configurations, doing so might not be safe!

Therefore, this option must be explicitly configured to enable:
  server.feature-flags += ("server.graceful-restart-bg" => "enable")
  server.systemd-socket-activation = "enable"

Along with enabling server.feature-flags "server.graceful-restart-bg",
enabling server.systemd-socket-activation allows transfer of open
listening sockets to the new lighttpd server instance, and occurs
without closing the listening sockets and without destroying the
kernel listen backlog queue on the socket.

Safe configurations may include lighttpd.conf which connect to
standalone backend daemons, e.g. proxying to other servers,
including PHP-FPM backends.

Unsafe configurations include lighttpd.conf which use "bin-path" option
in *.server configs, instructing lighttpd to execute the backends.
Using the graceful-and-immediate-restart option is likely *unsafe* if
the backend daemon expects only one instance of itself to run at a time.

Current implementation of graceful and immediate restart option keeps
the backgrounded lighttpd in the same process group, so that subsequent
SIGINT or SIGTERM will shut down both the new and the backgrounded
servers.  (An alternative option (commented out in the code) is to
background and detach from the new lighttpd process.)  Regardless,
existing subprocesses, such as CGI, remain in original process group.
As a result, the new lighttpd server may receive SIGCHLD for unknown
processes inherited from the old server, which the new lighttpd server
will reap and discard.  The original lighttpd server, now a child, will
be unable to detect exit or reap and report status on those pre-existing
subprocesses.

Graceful restart is triggered in lighttpd by sending lighttpd SIGUSR1.
If lighttpd is configured with workers, then SIGINT (not SIGUSR1) is
sent to the process group, including other processes started by
lighttpd, e.g. CGI.  To work well with graceful restart, CGI scripts and
other processes should trap SIGINT (and SIGUSR1 for good measure).
Long-running scripts may want to checkpoint and close, e.g. a CGI script
implementing a long-running websocket connection.
2020-10-11 12:19:27 -04:00
Glenn Strauss 8e2cf55250 [core] redirect to dir using relative-path
Location response header is permitted to use relative-path in
RFC 7231 Section 7.1.2. Location

Prefer relative path in redirection for the benefit of reverse proxies
and CDNs.  Doing so also avoids potentially disclosing internal schemes
and server names which client might not be able to directly reach.

To restore prior behavior of sending a fully-qualified absolute URI:
server.feature-flags += ("absolute-dir-redirect" => "enable")

x-ref:
  https://bz.apache.org/bugzilla/show_bug.cgi?id=63357
2020-10-11 11:43:06 -04:00
Glenn Strauss 784f1ac1da [core] alternate between two joblists
process one joblist and queue work up on second joblist

loop to poll, but poll immediate if work was queued on second joblist
2020-10-11 11:43:06 -04:00
Glenn Strauss 6eca864c5e [core] h2.[ch] with stub funcs (incomplete)
(subsequently incrementally updated using git rebase)

huge props and many thank yous to writers of testing tools used while
developing HTTP/2 support in lighttpd:

  h2spec - conformance testing tool for HTTP/2 implementation
           https://github.com/summerwind/h2spec
  h2load - HTTP/2 benchmarking tool
           https://nghttp2.org/documentation/h2load-howto.html
  curl   - command line tool and library for transferring data with URLs
           https://curl.haxx.se/
2020-10-11 11:43:02 -04:00
Glenn Strauss 8940fec894 [multiple] modules read reqbody via fn ptr
r->con->reqbody_read() replaces connection_handle_read_post_state()

future: might provide different callbacks for request body with
Content-Length versus request body sent via Transfer-Encoding: chunked
2020-10-03 09:05:38 -04:00
Glenn Strauss 1041a56dd9 [core] server.feature-flags to enable h2
server.feature-flags = ("server.h2proto" => "enable") to enable h2
2020-08-13 15:05:25 -04:00
Glenn Strauss fea5bdc873 [core] con layer plugin_ctx separate from request 2020-08-02 07:47:41 -04:00
Glenn Strauss 5f7bf472c7 [core] server.feature-flags extensible config
server.feature-flags = ("server.xxxx" => "enable")
2020-08-02 07:46:27 -04:00
Glenn Strauss ea58cc6f3f [core] store fd in srv->stdin_fd during setup
(pedantic; no impact)

upon error, server will exit, so the impact of momentarily leaking fd
has no impact.  This commit holds the fd in srv->stdin_fd to address
Coverity warning about leaking fd when using server.bind = "/dev/stdin"
2020-07-10 19:30:24 -04:00
Glenn Strauss bc89b98acb [build] fix build on MacOS X Tiger
See: https://trac.macports.org/ticket/58550
fixes for Tiger build, but can be applied generally to all builds
also requires legacysupport PortGroup for *at functions
kencu@macports.org

x-ref:
  https://trac.macports.org/ticket/58550
  https://github.com/macports/macports-ports/pull/4760
2020-07-08 19:54:29 -04:00
Glenn Strauss 7c7f8c467c [multiple] split con, request (very large change)
NB: r->tmp_buf == srv->tmp_buf (pointer is copied for quicker access)

NB: request read and write chunkqueues currently point to connection
    chunkqueues; per-request and per-connection chunkqueues are
    not distinct from one another
      con->read_queue  == r->read_queue
      con->write_queue == r->write_queue

NB: in the future, a separate connection config may be needed for
    connection-level module hooks.  Similarly, might need to have
    per-request chunkqueues separate from per-connection chunkqueues.
    Should probably also have a request_reset() which is distinct from
    connection_reset().
2020-07-08 19:54:29 -04:00
Glenn Strauss 31d9495330 [core] store subrequest_handler instead of mode
store pointer to module in handler_module instead of con->mode id
2020-07-08 19:54:29 -04:00
Glenn Strauss aca9d45adf [core] move request state into (request_st *)
NB: in the future, a separate connection state may be needed for
    connection-level state (different from request state)
2020-07-08 19:54:29 -04:00
Glenn Strauss a22cdca1cb [core] move addtl request-specific struct members 2020-07-08 19:54:29 -04:00
Glenn Strauss 100dfaa3f3 [core] move plugin_ctx into (request_st *)
NB: in the future, a separate plugin_ctx may be needed for
    connection-level plugins to keep state across multiple requests
2020-07-08 19:54:29 -04:00
Glenn Strauss 8131e4396d [core] move addtl request-specific struct members 2020-07-08 19:54:29 -04:00
Glenn Strauss 1474be7859 [core] move addtl request-specific struct members 2020-07-08 19:54:29 -04:00
Glenn Strauss 6fe031ef37 [core] move request start ts into (request_st *)
move request start timestamps into (request_st *)
2020-07-08 19:54:29 -04:00
Glenn Strauss 6870b0f55b [core] pass (request_st *) to request.c funcs
instead of (struct connection *)
2020-07-08 19:54:29 -04:00
Glenn Strauss 668cb6418f [core] move request_st,request_config to request.h 2020-07-08 19:54:29 -04:00
Glenn Strauss 550faf988c [core] rename specific_config to request_config 2020-07-08 19:54:29 -04:00
Glenn Strauss 057d83c50b [core] move keep_alive flag into request_st 2020-07-08 19:54:29 -04:00
Glenn Strauss 33430ce2b0 [core] copy some srv->srvconf into con->conf
(for memory locality and to reduce pointer chasing)
2020-07-08 19:54:29 -04:00
Glenn Strauss 8588772caa [core] isolate fdevent subsystem
fdevent.c no longer directly uses struct server *srv
srv->srvconf.max_fds (if set) is used to set rlimits
set max_conns in server.c after fdevent_init(), which sets srv->max_fds
  using srv->srvconf.max_fds (if set) as input hint
2020-07-08 19:54:29 -04:00
Glenn Strauss 05cc88ddb4 [multiple] omit passing srv to fdevent_handler 2020-07-08 19:54:29 -04:00
Glenn Strauss fda01e3305 [core] support multiple error logs
e.g. different server.errorlog for different virtual hosts

Also, support different server.breakagelog to have separate script
error logs, applicable to mod_cgi and mod_ssi exec.
2020-07-08 19:54:29 -04:00
Glenn Strauss 27e27e5e40 [core] static buffers for mtime_cache 2020-07-08 19:54:29 -04:00
Glenn Strauss 1dd58c5ad8 [multiple] con->proto_default_port 2020-07-08 19:54:28 -04:00
Glenn Strauss 68d8d4c532 [multiple] stat_cache singleton 2020-07-08 19:54:28 -04:00
Glenn Strauss c8cd7cf49b [multiple] extern log_epoch_secs
replace srv->cur_ts
2020-07-08 19:54:28 -04:00
Glenn Strauss 409bba80b1 [multiple] reduce direct use of srv->cur_ts 2020-07-08 19:54:28 -04:00
Glenn Strauss f24e6d696a [multiple] plugin_stats array
use global rather than passing around (server *) just for that

li_itostrn() and li_utostrn() return string length
(rather than requiring subsequent strlen() to find length)
2020-07-08 19:54:28 -04:00
Glenn Strauss 50bdb55de8 [multiple] connection hooks no longer get (srv *)
(explicit (server *) not passed; available in con->srv)
2020-07-08 19:54:28 -04:00
Glenn Strauss 0fcd51438d [core] create http chunk header on the stack
streamline code in http_chunk.c
2020-07-08 19:54:28 -04:00
Glenn Strauss 010c28949c [multiple] prefer (connection *) to (srv *)
convert all log_error_write() to log_error() and pass (log_error_st *)

use con->errh in preference to srv->errh (even though currently same)

avoid passing (server *) when previously used only for logging (errh)
2020-07-08 19:54:28 -04:00
Glenn Strauss 24680a9142 [core] array_init() arg for initial size 2020-07-08 18:08:52 -04:00
Glenn Strauss c193da3404 [core] split cond cache from cond matches 2020-07-08 18:08:52 -04:00
Glenn Strauss 65ef0a3d69 [core] collect more config logic into configfile.c 2020-07-08 18:08:51 -04:00
Glenn Strauss ed62e354ff [core] use config_plugin_values_init() 2020-07-08 18:08:51 -04:00
Glenn Strauss d7a6a7a263 [core] (const buffer *) from strftime_cache_get() 2020-05-23 17:59:29 -04:00
Glenn Strauss e06ac39c81 [core] rearrange server_config, server members
rearrange server_config, server members to improved affinity
2020-05-23 17:59:29 -04:00
Glenn Strauss e2de4e581e [core] const char *name in struct plugin
put void *data (always used) as first member of struct plugin

add int nconfig member to PLUGIN_DATA

calloc() inits p->data to NULL
2020-05-23 17:59:29 -04:00
Glenn Strauss feb21b3da2 [core] inline header and env arrays into con 2020-05-23 17:59:29 -04:00
Glenn Strauss 78cf61b152 [core] streamline config_check_cond()
avoid passing srv; use con->srv when needed
2020-02-24 11:15:32 -05:00
Glenn Strauss 6eb34ef5ab [core] add const to callers of http_header_*_get()
(The few places where value is modified in-place were not made const)
2020-02-24 11:15:32 -05:00
Glenn Strauss a7a721ab43 [core] treat con->conditional_is_valid as bitfield 2020-02-24 11:15:32 -05:00
Glenn Strauss 78a6440492 [core] perf: initialize con->conf using memcpy()
initialize specific_config (con->conf) from top-level specific_config
using memcpy()
2020-02-24 11:15:32 -05:00