Commit Graph

4283 Commits (be82f3d2e1e65faa54d335b53458294ef7ec8e8c)

Author SHA1 Message Date
Glenn Strauss 9d05b648ee [core] make ck_memeq_const_time() more generic (#3112)
make ck_memeq_const_time() more generically reusable

remove implementation requirements that strings be '\0' terminated,
or at least have an initialized byte following each string (s[len])

  "mod_auth cache password doesn't match"
2021-10-27 04:16:38 -04:00
Glenn Strauss 14b8d90288 [core] separate func for gw_authorizer_ok() 2021-10-27 04:16:38 -04:00
Glenn Strauss 47201bbb2e [core] resched HTTP/2 streams w/ pending data (#3111)
reschedule HTTP/2 streams w/ pending data if con->write_queue is empty

reschedule to read deferred (and pending) HTTP/2 frames when
MAX_CONCURRENT_STREAMS are active and a stream completes.

  "HTTP/2 requests sometimes take very long (missing last chunk)"
2021-10-27 04:16:38 -04:00
Glenn Strauss 0757d71e14 [core] short-circuit if response body recv w/ hdrs (fixes #3111)
short-circuit if response body completely received with response headers

  "HTTP/2 requests sometimes take very long (missing last chunk)"
2021-10-27 04:16:38 -04:00
Glenn Strauss 71544129f9 [core] expose chunkqueue_remove_empty_chunks()
expose chunkqueue_remove_empty_chunks() for use in error recovery
when aborting a chunkqueue_append_buffer_open*()
2021-10-27 04:16:14 -04:00
Glenn Strauss 44b47fc271 [core] accept up to 5 digit port num in host cond
(thx zebul66)

accept up to 5 digit port num in $HTTP["host"] condition match

  "Alls page and subdomain returns 404 with 1.4.60"
2021-10-16 19:56:27 -04:00
Glenn Strauss 69c2b2b207 [core] add comment to ck_memeq_const_time()
add comment to ck_memeq_const_time() with some implementation details
2021-10-16 02:05:19 -04:00
Glenn Strauss cf5644e0c2 [mod_webdav] ignore PROPFIND Depth for files
(thx meeb5)

ignore PROPFIND "Depth" request header for files (non-collections)

RFC4918 10.2. Depth Header
"If a resource does not have internal members, then the Depth header MUST be ignored."

  "Webdav + rclone backup"
2021-10-16 01:58:37 -04:00
Glenn Strauss 8d13233b69 [mod_ajp13,mod_fastcgi] comment: no response body
add comment for handling of streaming with no response body

add commented-out code to disable streaming to wait for backend protocol
to signal end of response (prevent http_response_write_prepare() from
short-circuiting and finishing responses without response body)

  "FastCGI premature socket close with response streaming and 204 status"
2021-10-16 01:58:37 -04:00
Glenn Strauss b1d1202af8 [mod_auth] fix Basic auth passwd cache (fixes #3112)
(thx manfred)

Basic auth passwd cache might fail to match when it should have
matched (false negative) when comparing an uninitialized byte.
That bug "fails closed" and does not use the cache when it could.
This patch allows for proper match in the cache when it should match.

  "mod_auth cache password doesn't match"
2021-10-16 01:57:50 -04:00
Glenn Strauss f5b5537ef1 [core] fix HTTP/2 upload > 64k w/ max-request-size (fixes #3108)
fix HTTP/2 upload > 64k with server.max-request-size > 0

(regression present only in lighttpd 1.4.60)

(thx SM)

  "File upload is broken after upgrade from 1.4.59 to 1.4.60"
2021-10-14 16:16:36 -04:00
Glenn Strauss 0e093d66ba [mod_extforward] keep remote IP thru request reset
preserve remote IP until request reset

(historical IP restore in request_done hook was obsoleted by
 commit fea5bdc8 in which request plugin context was split from
 connection plugin context, and by much older commits which
 ensured that request_reset hook was always run)

  "Remote address behind reverse proxy not logged"
2021-10-12 22:21:42 -04:00
Glenn Strauss 16f16dbfd5 [doc] update INSTALL 2021-10-12 16:13:51 -04:00
Glenn Strauss a5581b0319 [core] avoid repeated typedef for fdlog_st
  "Lighttpd 1.4.60 make error typedef fdlog_st redefinition"
2021-10-12 11:12:34 -04:00
Glenn Strauss a2bfccd1be [tests] force Fcntl::F_SETFD() autovivification (#3110)
  "RPM build fails at"
2021-10-10 18:28:01 -04:00
Glenn Strauss 1c41faaebc [mod_dirlisting] sort "../" to top (fixes #3109)
(thx otovalek)

  "mod_dirlisting javascript sort of ".." entries is broken"
2021-10-10 15:25:43 -04:00
Glenn Strauss 78eb0e3945 [tests] force POSIX::WNOHANG() autovivification (fixes #3110)
  "RPM build fails at"
2021-10-10 15:19:05 -04:00
Glenn Strauss 2a3cca7c56 [core] earlier macOS need define for errno_t (fixes #3107)
Earlier macOS need _DARWIN_C_SOURCE defined for errno_t from sys/types.h

Alternatively, define __STDC_WANT_LIB_EXT1__ >= 1 before include errno.h

  "error: unknown type name 'errno_t' (macOS 10.13.6)"
2021-10-10 14:09:12 -04:00
Glenn Strauss 124543bbe1 [mod_magnet] prefer lua_newuserdatauv() w/ lua 5.4
lua_newuserdata() -> lua_newuserdatauv() w/ lua 5.4
2021-10-07 19:10:26 -04:00
Glenn Strauss 07c8a6f056 [core] bounds check while url-decoding
(thx helmut)

do not read-ahead past '\0' while url-decoding

lighttpd 1.4.60 could previously have read one byte of potentially
uninitialized data.  lighttpd detects the '\0' so there is no exposure
of data.  This also can not cause a crash in lighttpd 1.4.60 due to how
lighttpd 1.4.60 allocates memory for buffers in sizes (power-2 + 1),
and typical system malloc alignment of 4- or 8- bytes.
2021-10-06 15:15:20 -04:00
Glenn Strauss 575665ad88 [multiple] __attribute_nonnull__ now takes params
define __attribute_nonnull__(params) with params to match
recent changes in glibc development (targetting glibc 2.35 in Feb 2022)

  new __attribute_nonnull__(params) conflicts with third-party
2021-10-05 19:12:23 -04:00
Glenn Strauss 47b10991fb [core] Y2038: error log high-precision timestamps
fix struct type used for error log high-precision timestamps
(difference is only for 32-bit)
2021-10-04 15:39:40 -04:00
Glenn Strauss cf4dfbe15e [core] define __BEGIN_DECLS, __END_DECLS if needed 2021-10-04 09:51:22 -04:00
Glenn Strauss d0f176ff6f - next is 1.4.61 2021-10-03 22:46:50 -04:00
Glenn Strauss 1a6fd58e37 [doc] NEWS 2021-10-03 14:34:53 -04:00
Glenn Strauss e0d7eafd6f [core] adjust indent for clarity 2021-10-03 12:54:20 -04:00
Glenn Strauss b17c37c5e5 [mod_openssl] boringssl compat 2021-10-03 11:01:40 -04:00
Istiak Ferdous 94edf7e844 [doc] update config files
github: closes #108
2021-10-02 23:06:09 -04:00
Glenn Strauss dd22dfa74c [core] fix chunked decoding from backend (#3044, #3046)
(fix typo missed in merge of 8e6dc4fe)
2021-10-02 12:35:18 -04:00
Glenn Strauss 3720908167 [core] portability tweaks for various platforms/cc 2021-10-01 22:02:22 -04:00
Glenn Strauss 434fab4929 [core] quiet coverity warnings 2021-10-01 20:59:08 -04:00
Glenn Strauss 23fe1312e5 [core] chunkqueue_peek_data() mmap experiment
chunkqueue_peek_data() experiment to mmap temporary files

(currently disabled in code due to not making measurable difference
 in throughput (during a specific microbenchmark load test),
 though it does reduce CPU use by ~10% in the same microbenchmark)

enabling this may cause large spikes in RSS mem usage reported by the
system, due to the read-only memory maps of the temporary files,
but this is nothing to be alarmed about, as the memory maps are
file-backed and read-only, so minimally add to memory pressure
2021-10-01 20:32:11 -04:00
Glenn Strauss faf76be187 [core] chunkqueue_mmap_chunk_len() for code reuse
(pull some code into smaller routines for potential code reuse)
2021-10-01 16:11:26 -04:00
Glenn Strauss 6e4cc3b0bb [core] read files into mem when framed for HTTP/2 2021-10-01 06:39:47 -04:00
Glenn Strauss e4cf6998a1 [core] limit initial response header backend read 2021-10-01 06:39:47 -04:00
Glenn Strauss b5d4169156 [core] allow up to 32k of data frames per stream
allow up to 32k of data frames per stream per round
(previously limited to single max_frame_size (default 16k))

For 8 streams, 32k*8 is 256k, which is current lighttpd MAX_WRITE_LIMIT,
so each stream still gets a chance to write data (unless write queue
not emptied on previous attempt, reducing add limit this round)
2021-10-01 06:39:47 -04:00
Glenn Strauss 62dc817caf [core] h2_send_cqdata() returns how much data sent 2021-10-01 06:39:47 -04:00
Glenn Strauss 042622c8c1 [core] use pread() to skip lseek() 2021-10-01 06:39:47 -04:00
Glenn Strauss 60a1224cd0 [core] fix typo in h2_send_cqdata()
fix typo in h2_send_cqdata(); no impact in practice since only caller,
connection_state_machine_h2() already limits dlen to <= max frame size
2021-10-01 06:39:47 -04:00
Glenn Strauss 39c0c2c3ed [core] cap size of data framed for HTTP/2 response
cap size of data framed for HTTP/2 response until more data sent to

make sure to reschedule connection in job queue if max_bytes reached
and then the entire con->write_queue was flushed to network, or else
there is a chance the request may not get rescheduled (and then will
timeout) if the request is completed from the backend and there is
no other traffic or streams to trigger connection processing.

(check con->write_queue > 8k rather than empty from last round,
 since small frames such as connection preface may have been added
 this round while processing con->read_queue)
2021-10-01 06:39:47 -04:00
Glenn Strauss ed2c698300 [tests] ignore SIGINT, SIGUSR1 in fcgi-responder
ignore SIGINT, SIGUSR1 in fcgi-responder if HAVE_SIGNAL is defined
(must be defined separately since config.h is not included)

Not required for test framework.  Added as an example in the code,
e.g. if code is reused with lighttpd and graceful shutdown or restart.
(backend will be sent SIGTERM when server is ready to restart)
2021-10-01 06:39:47 -04:00
Glenn Strauss 41c3cc0965 [core] disable server.graceful-restart-bg if spawn
disable server.graceful-restart-bg if spawning gateway backends
via *.server "bin-path"
2021-10-01 06:39:47 -04:00
Glenn Strauss ff45d5865e [multiple] clarify error msg when no cert avail 2021-10-01 06:39:47 -04:00
Glenn Strauss 6e45cff05a [core] disable streaming response with authorizer (fixes #3106)
disable streaming response while processing "authorizer" mode
until "authorizer" response 200 OK from the backend is complete

(thx jefftharris)

  "FastCGI authorizer hang with"
  "FastCGI authorizer hang with"
2021-10-01 06:39:23 -04:00
Glenn Strauss dd23fcb2b2 [mod_cgi] improve chunk buffer reuse from backends
mark and temporarily defer event handling of backend events in order
to handle at same time as the associated front-end connection events
2021-09-30 17:34:03 -04:00
Glenn Strauss bee35b4877 [core] improve chunk buffer reuse from backends
mark and temporarily defer event handling of backend events in order
to handle at same time as the associated front-end connection events
2021-09-30 17:34:03 -04:00
Glenn Strauss cb7deb493c [core] remove obsolete comment about r->gw_dechunk
r->gw_dechunk->b is not a candidate for using generic chunk buffers.
chunked headers are generally smaller and fit in default 64 byte alloc.
Also, lighttpd limits chunked header to 1k.

Avoid unneeded optimization since HTTP/1.1 use is likely to diminish
over time in favor of HTTP/2 or HTTP/3 or later.
2021-09-30 17:34:03 -04:00
Glenn Strauss 8e6dc4fe94 [core] fix chunked decoding from backend (#3044, #3046)
fix edge case for initial chunked data
(bug introduced in lighttpd 1.4.56)

If chunked header received without data before response headers sent,
then initial chunked data might be sent to client without chunked header
if client made an HTTP/1.1 request and the response is Transfer-Encoding
chunked and lighttpd is configured to stream the response (non-zero
value for  This might occur if lighttpd
backend is connected via a unix domain socket and the initial chunk is
large and coming from a temporary file.  It may be sent in a separate
packet since lighttpd does not use TCP_CORK on unix domain sockets.

  "Failure on second request in http proxy backend"
  "Socket errors after update to version 1.4.56"
2021-09-30 17:34:03 -04:00
Glenn Strauss 6e62b84258 [core] splice() data from backends to tempfiles
splice() data from backends to tempfiles (where splice() is available);
reduce copying data to userspace when writing data to tempfiles

Note: splice() on Linux returns EINVAL if target file has O_APPEND set
so lighttpd uses pwrite() (where available) when writing to tempfiles
(instead of lseek() + write(), or O_APPEND and write())
2021-09-30 17:34:03 -04:00
Glenn Strauss 6bd6226e90 [multiple] avoid empty chunks in chunkqueue
(This might render some calls to chunkqueue_remove_finished_chunks()
 no longer necessary)
2021-09-30 17:34:03 -04:00