Commit Graph

4605 Commits

Author SHA1 Message Date
Glenn Strauss 11864d2999 [autoconf] update ax_prog_cc_for_build.m4
update scripts/m4/ax_prog_cc_for_build.m4 from serial 20 to serial 21
(This should fix build errors occuring with './configure -C' caching);a=blob_plain;f=m4/ax_prog_cc_for_build.m4
2022-06-10 17:25:06 -04:00
Glenn Strauss 91ce3b0857 [core] buffer_append_bs_escaped_json()
separate func from buffer_append_bs_escaped() so that both
buffer_append_bs_escaped() and buffer_append_bs_escaped_json()
can be slightly more specialized and optimized
2022-06-10 13:15:18 -04:00
Glenn Strauss ca407dca5d [multiple] fix json encoding
(thx helmut)

fix json encoding to operate on unicode code points
2022-06-10 12:13:11 -04:00
Glenn Strauss 9ac5da2720 [mod_magnet] reset for http-response-send-file
reset r->http_status and clear response body for http-response-send-file
called from magnet.attract-response-start-to in order to match
http_response_send_file() behavior to send ETag, replace response body
2022-06-09 17:21:57 -04:00
Glenn Strauss f86b448799 [core] h2: optim: send window update in 16k units
h2: optim: send window update in 16k units for both sessions and streams
2022-06-09 14:33:53 -04:00
Glenn Strauss 40f289b6c3 - next is 1.4.66 2022-06-07 23:36:32 -04:00
Glenn Strauss 388aad082c [doc] NEWS 2022-06-07 19:23:53 -04:00
Glenn Strauss 71317bc997 [tests] test stubs for http_header.c and http_kv.c 2022-06-07 19:23:53 -04:00
Glenn Strauss 5a32a6dc51 [core] update cached tables with Priority header
(thx jens-maus)
2022-06-07 19:23:53 -04:00
Glenn Strauss f20061e482 [core] h2: avoid sending tiny DATA frames
h2: avoid sending tiny DATA frames when h2 window is tiny
and a larger amount of data is pending to be sent;
wait for slightly larger window to be available

note: must temporarily disable this when running h2spec since some
      h2spec tests expect 1-byte DATA frame, not a deferred response
2022-06-07 19:23:53 -04:00
Glenn Strauss ad4a790495 [core] h2: avoid sending small WINDOW_UPDATE frames
Note this may slow down uploads due to delay in sending WINDOW_UPDATE
smaller than the default max frame size (16384), but as a trade-off
this aims to reduce degenerative behavior from clients sending an
increasing number of tiny DATA frames.  The default 65535 value for
SETTINGS_INITIAL_WINDOW_SIZE (which lighttpd immediately increases to
65536) is larger than 16384, so deferring small updates should not
exhaust the window (from lighttpd's perspective).

  "Slow upload / Increase CPU/Memory usage with HTTP/2 enabled"
  "libnghttp2 degenerative behavior possible when HTTP/2 window size exhausted"
2022-06-07 19:23:53 -04:00
Glenn Strauss ef62ff9419 [core] increase session window size to 256k
(adjust the HTTP/2 frame value to match the comment;
 previous value was actually adjusting only to 128k)
2022-06-04 14:36:08 -04:00
Glenn Strauss 5d0f174feb [core] h2 SETTINGS_INITIAL_WINDOW_SIZE 64k (fixes #3089)
Increase stream window size to 64k (from default 64k-1)
Increase session window size to 256k (from default 64k-1)
  (multiple of SETTINGS_MAX_FRAME_SIZE (default 16k))
  (rely on TCP window scaling and TCP congestion control
   to manage client sending too much data)

Window size as multiple of SETTINGS_MAX_FRAME_SIZE (default 16k) may be
friendlier to client buffer management and more efficient for uploading.
Window sizes are not increased to arbitrarily large numbers as the
HTTP/2 flow control may be useful to help simplistic/naive clients
avoid symptoms of buffer bloat.

Bandwidth delay product on high bandwidth, high latency links may be
large, so increasing window sizes may increase performance.  However,
lighttpd code does not check actual per-connection RTT or kernel
socket buffer sizes.  256k chosen as session window size, and 192k as
stream window size.

Above changes avoid degenerative behavior from the widely deployed
libnghttp2 which may devolve to sending 1 byte at a time in some cases.

  "Slow upload / Increase CPU/Memory usage with HTTP/2 enabled"
  Delivering HTTP/2 upload speed improvements
2022-06-04 13:45:47 -04:00
Glenn Strauss 152596158a [mod_mbedtls] fix crt chain construction logic
fix crt chain construction logic inversion

(thx DamienT)
2022-06-04 13:45:47 -04:00
Glenn Strauss def1bfa976 [mod_webdav] alt handling PROPFIND on collection
alternative way of handling PROPFIND on collection where the request was
made without a trailing slash.  Instead of sending 308 redirect in some
cases, set Content-Location response header *and* treat as if request
had been made with trailing slash on the collection for PROPFIND and
  There is a standing convention that when a collection is referred to
  by its name without a trailing slash, the server MAY handle the
  request as if the trailing slash were present. In this case, it
  SHOULD return a Content-Location header in the response, pointing to
  the URL ending with the "/".

  "The previous workaround for GVFS is breaking the new version of GVFS"
  HTTP Extensions for Web Distributed Authoring and Versioning (WebDAV)
2022-06-04 13:45:47 -04:00
Glenn Strauss 5c4cc9f567 [core] avoid server.use-ipv6 warning after SIGUSR1 2022-06-04 13:45:47 -04:00
Glenn Strauss 833f6aa4c0 [TLS] inherit ssl.engine from global scope
Since lighttpd 1.4.56, an oversight in config processing missed
setting explicitly p->conf.ssl_enabled = 0 in network.c when
initializing conditions.  When ssl.engine = "enable" in lighttpd.conf
global scope, the missing reset in network.c required non-TLS ports
(e.g. $SERVER["socket"] == ":80") to contain ssl.engine = "disable"
in order for requests to those ports to be served rather than erroring.

(This error was discovered during collaboration with jens-maus in

There have been zero other instances of this error reported since the
release of lighttpd 1.4.56 in Nov 2020.

Therefore, having ssl.engine = "enable" inherited from the global scope
is unlikely to have any widespread impact in practice, and enabling
ssl.engine = "enable" (along with TLS certificate configuration) is now
recommended as default.  When ssl.engine = "enable" in the global scope,
ssl.engine = "disable" should be specified in those $SERVER["socket"]
conditions where clear-text is desired.
2022-06-04 13:45:47 -04:00
Glenn Strauss da8025fb30 [core] reset internal flags after graceful restart
reset internal socket flags after graceful restart:
re-init srv->socket->sidx and srv_socket->is_ssl
  after SIGUSR1 or systemd socket activation

(thx jens-maus)

2022-06-04 13:44:16 -04:00
Glenn Strauss 33f73b4d82 [multiple] application/javascript text/javascript
translate MIME type "application/javascript" to "text/javascript"

(if required, type may still be overwritten by mod_setenv or mod_magnet)

  "Updates to ECMAScript Media Types"
2022-05-26 01:34:38 -04:00
Glenn Strauss a6d4a98c07 [core] use current dir if context->basedir blank 2022-05-25 13:29:27 -04:00
Glenn Strauss ee6df0961b [build] update descriptions to remove old lua ver 2022-05-24 21:56:30 -04:00
Glenn Strauss 8bdea5db14 [mod_magnet] skip req body collect warn if modsec3
skip req body collect warning if handler is already mod_security3
(prototype mod_security3 uses similar technique to collect req body)
2022-05-24 21:56:30 -04:00
Glenn Strauss a01e62bb7d [core] "url-invalid-utf8-reject" normalization opt
server.http-parseopts "url-invalid-utf8-reject" url normalization option
default: "url-invalid-utf8-reject" => "enable"
2022-05-24 21:56:30 -04:00
Glenn Strauss 467fb2bfb7 [mod_cgi] fix detection of failing error handler (fixes #3157)
(thx sparlane)

failing error handler produced no output and POLLRDHUP received with

commit dd23fcb2 changed return value from HANDER_FINISHED to
HANDLER_GO_ON when introducing cgi_process_rd_revents(), and POLLRDHUP
case which previously fell through needed to continue to return
HANDLER_FINISHED after calling cgi_connection_close()

  "fall-back with cgi error handler no longer works"
2022-05-17 01:17:42 -04:00
Glenn Strauss f314d2cbb9 [mod_magnet] r.req_body .collect .get .set .add 2022-05-15 23:33:20 -04:00
Glenn Strauss 3cc1509015 [mod_webdav] If-None-Match: * on non-existent
fix logic for If-None-Match: * test on non-existent entity
(regression in lighttpd 1.4.64)

If-None-Match: * should not fail on a non-existent entity,
as it may be used to make the request conditional on the
origin server having no current representation of the entity.
(see If-None-Match in RFC2616 and RFC7232)

(This logic had been changed in lighttpd 1.4.64 in
 commit 8a535e7e06
 when allowing bogus, non-'*' If-None-Match etags
 for non-existent entities to not-match (and pass the check)
2022-05-14 11:36:42 -04:00
Glenn Strauss 669ae77fb8 [mod_wolfssl] compile fix for OpenWRT 2022-05-12 06:05:44 -04:00
Glenn Strauss 15e1a2d53a [mod_magnet] quiet coverity false positive 2022-05-12 05:18:27 -04:00
Glenn Strauss 58e1b61f5b [core] feature-flag server.metrics-high-precision
server.feature-flags += ("server.metrics-high-precision" => "enable")
(default: "disable")

enables high-resolution timestamps,
currently used for request start time

This is automatically enabled if mod_accesslog log record format uses
high-resolution format specifiers in the template, but this feature
needs to be enabled if that is not the case and a high-resolution
request start time is desired for use in lua scripts run by mod_magnet,
e.g. to measure time-to-first-byte
2022-05-11 23:51:48 -04:00
Glenn Strauss 2a7d3a2751 [multiple] rename status_counter -> plugin_stats 2022-05-11 23:51:48 -04:00
Glenn Strauss 6669846947 [doc] regenerate doc/config/conf.d/mime.conf
(matches preferred syntax for Content-Type from upcoming HTTP spec rev)
2022-05-11 23:51:48 -04:00
Glenn Strauss 85b5988dc7 [multiple] use preferred syntax for Content-Type
use preferred syntax for Content-Type: .../charset=...
(match preferred syntax specified in upcoming HTTP spec revision)
2022-05-11 23:51:48 -04:00
Glenn Strauss dbfee2a299 [mod_status] display HTTP/2 control stream w/ reqs
display HTTP/2 control stream as pseudo-request in request list

HTTP/2 control stream has URI "*" from "PRI *" HTTP/2 connection preface
2022-05-11 23:51:48 -04:00
Glenn Strauss a1c664a5c5 [mod_magnet] backspace-escape encode/decode
2022-05-11 23:51:48 -04:00
Glenn Strauss fc32d4c9d3 [core] buffer_append_string_c_escaped ASCII optim 2022-05-11 23:51:48 -04:00
Glenn Strauss d22e88b7d9 [core] buffer_append_bs_escaped()
move accesslog_append_escaped() to buffer_append_bs_escaped()
replace buffer_append_string_encoded_json()
2022-05-11 23:51:48 -04:00
Glenn Strauss 04f2fcf8f0 [mod_magnet] read-only access to r->server_name
read-only access to r->server_name for coverage

setting r.req_attr["uri.authority"] in lua sets both r->uri.authority
and points r->server_name to r->uri.authority, and there is not an
obvious (to me) need to be able to set r->server_name separately.

(r->server_name might be set in lighttpd.conf, or else is the same as
 r->uri.authority, unless modified by mod_simple_vhost default_host)
2022-05-11 23:51:47 -04:00
Glenn Strauss a6d40ce213 [core] remove unused srv->con_* counters 2022-05-11 23:51:47 -04:00
Glenn Strauss 2ad335f0ee [mod_accesslog] reorder fields in switch()
sort enum and switch to popular use, and have switch order match enum

split less popular format options to separate cold func
2022-05-11 23:51:47 -04:00
Glenn Strauss 2adc62e9bb [multiple] simplify bytes_in/bytes_out accounting
encapsulate accounting calculations in

more accurate accounting for HTTP/1.1 bytes_in on keep-alive requests
(affects case where client pipelines HTTP/1.1 requests)

remove con->bytes_read and con->bytes_written
(no longer needed since request_st was split from connection struct
 and request bytes_read_ckpt and bytes_written_ckpt are maintained
 for HTTP/1.x bytes_in and bytes_out accounting.  Also, further back,
 chunkqueue internal accounting was simplified to maintain bytes_in
 and bytes_out to always match chunkqueue length)
2022-05-11 23:51:47 -04:00
Glenn Strauss 36e6431764 [core] fix configparser_simplify_regex() comment 2022-05-11 23:51:47 -04:00
Glenn Strauss b229d028ae [mod_accesslog] remove begin/end tags from %{}t
remove begin/end tags from %{begin:...}t and %{end:...}t
as those should not be part of the strftime format string
2022-05-11 23:51:47 -04:00
Glenn Strauss f2610d23e2 [multiple] use buffer_append_char() 2022-05-11 23:51:47 -04:00
Glenn Strauss 1194e6d24b [mod_accesslog] accesslog.escaping = "json"
option to escape strings using json-compliant \uXXXX escape sequences
  accesslog.escaping = "json"
(default is accesslog.escaping = "default" or anything else for default)

additional mod_accesslog optimizations, code reuse, smaller funcs
2022-05-11 23:51:47 -04:00
Glenn Strauss f70409d2f8 [core] buffer_append_char() convenience func 2022-05-11 23:51:47 -04:00
Glenn Strauss fbefd65197 [mod_magnet] further isolate legacy API result tbl 2022-05-11 23:51:47 -04:00
Glenn Strauss 65b4997d7c [mod_magnet] adjust p->conf.stage checks 2022-05-11 23:51:47 -04:00
Glenn Strauss 404b3816a2 [mod_magnet] format multiline err traceback 2022-05-11 23:51:47 -04:00
Glenn Strauss b5490082fa [mod_magnet] lighty.c.stat high precision time
lighty.c.stat returns high precision timestamps, when available
2022-05-11 23:51:47 -04:00
Glenn Strauss c46aaa48a6 [mod_magnet] separate funcs for uri_path_raw
separate funcs to get/set uri_path_raw in target (r->target)
without modifying the query-string part of the target (r->target)
2022-05-11 23:51:47 -04:00