Commit Graph

3509 Commits (bdb5fb26a8ba433e853a5110a6cbbe9721d94ec1)
 

Author SHA1 Message Date
Glenn Strauss bdb5fb26a8 [wolfssl] wolfSSL_CTX_set_mode differs from others
wolfSSL_CTX_set_mode() differs from openssl SSL_CTX_set_mode().
wolfSSL_CTX_set_mode() takes a single flag at a time and has
sparse flag support (small number of recognized flags)
3 years ago
Glenn Strauss cf8cefceb6 [core] sys-crypto-md.h consistent interfaces
return values for sys-crypto-md.h interfaces

While some library implementations do not fail and have no return value,
others might fail on memory allocation or on failure to communicate with
an external or dedicated engine or device, e.g. which might store a
private key.

future: lighttpd callers of sys-crypto-md.h do not currently expect
        or check for errors from these digest functions, but should
        consider doing so.
3 years ago
Glenn Strauss a363383a92 [tests] detect multiple SSL/TLS/crypto providers 3 years ago
Glenn Strauss 7553ef7f44 [multiple] openssl 3.0.0 digest interface migrate
provide implementations for conventional digest interfaces
but use the newer openssl digest interfaces under the hood

<rant>
It is baffling that the openssl library -- with *thousands* of public
interfaces -- does not provide these, and suggests that openssl
developers do not frequently write apps which utilize these interfaces.
</rant>
3 years ago
Glenn Strauss e6a87a894d [mod_openssl] merge ssl_tlsext_ticket_key_cb impls 3 years ago
Glenn Strauss cc6a76be12 [mod_openssl] construct OSSL_PARAM on stack 3 years ago
Glenn Strauss ebafd914e7 [mod_openssl] migrate ticket cb to openssl 3.0.0
migrate ticket cb to openssl 3.0.0 interface
  SSL_CTX_set_tlsext_ticket_key_evp_cb()
3 years ago
Glenn Strauss e1bcdb5e9f [multiple] ignore openssl 3.0.0 deprecation warns
quiet openssl 3.0.0 deprecation warnings for
low-level digest algorithm interfaces

future: migrate to openssl interfaces:
  EVP_DigestInit_ex(3), EVP_DigestUpdate(3), and EVP_DigestFinal_ex(3)

x-ref:
  https://github.com/openssl/openssl/pull/10791
  https://www.openssl.org/docs/manmaster/man3/EVP_DigestInit_ex.html
  https://wiki.openssl.org/index.php/EVP_Message_Digests
3 years ago
Glenn Strauss a3ccc46be1 [mod_cgi] use splice() to send input to CGI
use splice(), if available (Linux), to send input from temp files to CGI
3 years ago
Glenn Strauss e18da93e44 [core] reuse chunkqueue_compact_mem*() 3 years ago
Glenn Strauss 9c25581d6f [core] alloc optim reading file, sending chunked
avoid potential double-copy due to not enough space for final '\0'
in http_chunk_append_read_fd_range() if read size is exactly multiple
of 8k and sending chunked response
3 years ago
Glenn Strauss 6d76f7ce49 [core] fix chunkqueue_compact_mem w/ partial chunk
(bug on master branch; never released)
3 years ago
Glenn Strauss 8abbf621d1 [mod_fastcgi] decode chunked is cold code path
decode chunked from FastCGI backend is cold code path
3 years ago
Glenn Strauss dc51e9d61c [tests] simulate slow, small packets more quickly
simulate slow, small packets more quickly
1ms between chars is plenty slow; 100ms is excessive

fixup
3 years ago
Glenn Strauss 96abd9cfb8 [core] coalesce nearby ranges in Range requests
Range requests must be HTTP/1.1 or later (not HTTP/1.0)
3 years ago
Glenn Strauss 2f7839e88f [doc] comment out config disabling Range for .pdf
comment out lighttpd.conf config disabling Range requests for .pdf

bug in Adobe Acrobat Reader browser plugin dates to over 10 years ago
https://redmine.lighttpd.net/issues/171
https://redmine.lighttpd.net/issues/541
https://jarrettbroder.blogspot.com/2010/11/adobe-acrobat-reader-plugin-stalls-when.html
one possible workaround was to disable "fast web view" in the
  Adobe Acrobat Reader

bug in Chrome fixed in 2015
https://bugs.chromium.org/p/chromium/issues/detail?id=442318

However, there still may be issues with PDF readers mishandling Range
requests, e.g. with coalesced ranges, so the section remains in the
sample lighttpd.conf, but is commented out.
https://github.com/mozilla/pdf.js/issues/9022
3 years ago
Glenn Strauss 66d1ec485c [core,mod_deflate] leverage cache of open fd
leverage simple cache of open file in stat_cache
(use stat_cache_get_entry_open())

future: reference count fd instead of dup()
  (requires extending chunkqueue interfaces)
3 years ago
Glenn Strauss d8e5e21eb7 [core] stat_cache_get_entry_open()
simple interface to cache open file by extending struct stat_cache_entry

future: should probably create fd cache separate from stat_cache,
        perhaps along w/ http-specific fields like etag and content_type
3 years ago
Glenn Strauss e99126074c [core] pass open fd to http_response_parse_range 3 years ago
Glenn Strauss 6219b861ce [core] http_response_parse_range() const file sz 3 years ago
Glenn Strauss cd2ff4202d [core] remove unused r->content_length
(was parsed Content-Length sent by backend)
(might add back in future along with stricter validation of field value)
3 years ago
Glenn Strauss 55e9f082a9 [core] limit num ranges in Range requests
parse Range header prior to constructing ranged response
3 years ago
Glenn Strauss 2dc3d5faac [mod_gnutls] quiet CLOSE_NOTIFY after conn reset
do not log error after connection reset
3 years ago
Glenn Strauss 496cd8ff44 [mod_mbedtls] quiet CLOSE_NOTIFY after conn reset
do not log error after connection reset
3 years ago
Glenn Strauss f846a392d5 [multiple] use stat_cache_path_isdir() 3 years ago
Glenn Strauss 7d368cd7a5 [core] stat_cache_path_isdir() 3 years ago
Glenn Strauss 14eb97f1b3 [core] dup FILE_CHUNK fd when splitting FILE_CHUNK 3 years ago
Glenn Strauss 2b4166fb74 [core] fdevent_dup_cloexec() 3 years ago
Glenn Strauss ad22eebd23 [mod_ssi] use stat_cache_open_rdonly_fstat() 3 years ago
Glenn Strauss b7370a6d46 [core] save errno around close() if fstat() fails
(should not happen, since file was just opened)
3 years ago
Glenn Strauss e2c832122b [mod_deflate] fix potential NULL deref in err case
(bug on master branch; never released)
3 years ago
Glenn Strauss 4d6d1e790a [multiple] use light_btst() for hdr existence chk 3 years ago
Glenn Strauss bd8edb51d0 [core] allow symlinks under /dev for rand devices
(fix code to match comment)
3 years ago
Glenn Strauss 874707cd66 [TLS] use fdevent_load_file_bytes() for STEK file
remove direct dependency on <unistd.h> from lighttpd TLS modules
3 years ago
Glenn Strauss 771ebd39a9 [core] fdevent_load_file_bytes()
wrapper to load defined number of bytes from file at given offset
3 years ago
Glenn Strauss ee1f7af24d [core] be more precise checking streaming flags 3 years ago
Glenn Strauss f7fc8d8d75 [core] connection_joblist global
con->srv->joblist is expensive to traverse when cache is cold
and called from connection_handle_fdevent(); var made a global
3 years ago
Glenn Strauss 01f90c58ef [core] walk queue in connection_write_chunkqueue()
walk chunkqueue up to first FILE_CHUNK (if present)
This may incur memory load misses for pointer chasing, but effectively
preloads part of the chunkqueue, something which used to be a side
effect of a previous (less efficient) version of chunkqueue_length()
which walked the entire chunkqueue (on each and every call).  The loads
here make a measurable difference in performance in underlying call to
con->network_write()
3 years ago
Glenn Strauss f1136f7897 [core] short-circuit connection_write_throttle()
short-circuit connection_write_throttle() when throttling not enabled
3 years ago
Glenn Strauss 7c1e81299f [core] handle unexpected EOF reading FILE_CHUNK
(replace existing check which suffered from ToC-ToU race condition)
enhances logic from 2015 commit 593599f1 and avoids repeated fstat()
checks when sending large files

For mmap(), lighttpd catches SIGBUS if file is (externally) truncated
and lighttpd attempts to access bytes in a read-only mapping more than
a memory page boundary following the end of the file.

For sendfile(), lighttpd returns an error if sendfile() reports no error
and that no bytes have been sent after lighttpd attempts to send a
non-zero number of bytes.
3 years ago
Glenn Strauss a8398e4596 [core] defer handling FDEVENT_HUP and FDEVENT_ERR
defer handling FDEVENT_HUP and FDEVENT_ERR to after processing
(connection *) in order to have a chance to read data in kernel
socket buffers
3 years ago
Glenn Strauss 52d9b0da88 [core] server_run_con_queue()
func to run queued (connection *) jobs
3 years ago
Glenn Strauss a3001b968f [core] propagate state after writing
(perf) avoid reload miss after partial write of chunkqueue data
3 years ago
Glenn Strauss 81029b8b51 [multiple] inline chunkqueue where always alloc'd
inline struct chunkqueue where always allocated in other structs

(memory locality)
3 years ago
Glenn Strauss 8b382a81c7 [multiple] use sock_addr_get_family in more places 3 years ago
Glenn Strauss 86e5f09062 [core] perf adjustments to avoid load miss 3 years ago
Glenn Strauss ed297e7e67 [core] lighttpd -1 supports pipes (e.g. netcat)
lighttpd -1 (one-shot mode) now supports pipes (e.g. with netcat)

x-ref:
  "Is there anyway to run lighttpd in the command line?"
  https://redmine.lighttpd.net/boards/2/topics/9348
3 years ago
Glenn Strauss 94c4c63773 [core] graceful shutdown timeout option
server.feature-flags += ("server.graceful-shutdown-timeout" => 10)

After receiving SIGINT or SIGUSR1, lighttpd will gracefully shutdown,
waiting for existing connections to complete.  In the case of SIGUSR1,
this wait occurs before restarting lighttpd.  The default timeout is
none (unlimited).

When "server.graceful-shutdown-timeout" option is set, it defines the
number of seconds that lighttpd will wait for existing connections to
complete before shutting down the connection.

Sites which expect large uploads or downloads, or those with very slow
clients, might want to set a much longer timeout, e.g 60 seconds

For more immediate graceful restarts, while still allowing existing
connections time to complete, sites should additionally consider
whether or not
  server.feature-flags += ("server.graceful-restart-bg" => "enable")
is appropriate and compatible with their lighttpd.conf settings
3 years ago
Glenn Strauss 76bd8bba9a [mod_ssi] init status var before waitpid() 3 years ago
Glenn Strauss 352d5d776d [core] graceful and immediate restart option
graceful and (nearly) immediate lighttpd restart option

For *some* configurations, it *may* be safe to background the current
lighttpd server (or workers) to continue processing active requests
and, in parallel, to start up a new lighttpd server with a new
configuration.  For other configurations, doing so might not be safe!

Therefore, this option must be explicitly configured to enable:
  server.feature-flags += ("server.graceful-restart-bg" => "enable")
  server.systemd-socket-activation = "enable"

Along with enabling server.feature-flags "server.graceful-restart-bg",
enabling server.systemd-socket-activation allows transfer of open
listening sockets to the new lighttpd server instance, and occurs
without closing the listening sockets and without destroying the
kernel listen backlog queue on the socket.

Safe configurations may include lighttpd.conf which connect to
standalone backend daemons, e.g. proxying to other servers,
including PHP-FPM backends.

Unsafe configurations include lighttpd.conf which use "bin-path" option
in *.server configs, instructing lighttpd to execute the backends.
Using the graceful-and-immediate-restart option is likely *unsafe* if
the backend daemon expects only one instance of itself to run at a time.

Current implementation of graceful and immediate restart option keeps
the backgrounded lighttpd in the same process group, so that subsequent
SIGINT or SIGTERM will shut down both the new and the backgrounded
servers.  (An alternative option (commented out in the code) is to
background and detach from the new lighttpd process.)  Regardless,
existing subprocesses, such as CGI, remain in original process group.
As a result, the new lighttpd server may receive SIGCHLD for unknown
processes inherited from the old server, which the new lighttpd server
will reap and discard.  The original lighttpd server, now a child, will
be unable to detect exit or reap and report status on those pre-existing
subprocesses.

Graceful restart is triggered in lighttpd by sending lighttpd SIGUSR1.
If lighttpd is configured with workers, then SIGINT (not SIGUSR1) is
sent to the process group, including other processes started by
lighttpd, e.g. CGI.  To work well with graceful restart, CGI scripts and
other processes should trap SIGINT (and SIGUSR1 for good measure).
Long-running scripts may want to checkpoint and close, e.g. a CGI script
implementing a long-running websocket connection.
3 years ago