Commit Graph

226 Commits

Author SHA1 Message Date
Glenn Strauss cf5644e0c2 [mod_webdav] ignore PROPFIND Depth for files
(thx meeb5)

ignore PROPFIND "Depth" request header for files (non-collections)

RFC4918 10.2. Depth Header
"If a resource does not have internal members, then the Depth header MUST be ignored."

  "Webdav + rclone backup"
2021-10-16 01:58:37 -04:00
Glenn Strauss 9f62f1b196 [multiple] fdevent_mkostemp()
fdevent_mkostemp() with flags arg so that caller can pass O_APPEND

renamed from fdevent_mkstemp_append(), previously always O_APPEND
2021-09-30 17:34:03 -04:00
Glenn Strauss bd72d1927a [core] remove server.upload-temp-file-size limit
previously undocumented server.upload-temp-file-size in lighttpd 1.4.38
preceded introduction of lighttpd streaming options in lighttpd 1.4.40
( and
2021-09-08 15:06:07 -04:00
Glenn Strauss 2ef31a1b3f [core] chunkqueue_append_buffer always clears buf
chunkqueue_append_buffer() always clears buffer
(instead of relying on caller to do so after the call)
2021-09-08 15:06:06 -04:00
Glenn Strauss 309c1693ac [multiple] Y2038 32-bit signed time_t mitigations
Most OS platforms have already provided solutions to
Y2038 32-bit signed time_t 5 - 10 years ago (or more!)
Notable exceptions are Linux i686 and FreeBSD i386.

Since 32-bit systems tend to be embedded systems,
and since many distros take years to pick up new software,
this commit aims to provide Y2038 mitigations for lighttpd
running on 32-bit systems with Y2038-unsafe 32-bit signed time_t

* Y2038: lighttpd 1.4.60 and later report Y2038 safety
  $ lighttpd -V
    + Y2038 support                                    # Y2038-SAFE
  $ lighttpd -V
    - Y2038 support (unsafe 32-bit signed time_t)      # Y2038-UNSAFE

* Y2038: general platform info
  * Y2038-SAFE: lighttpd 64-bit builds on platforms using 64-bit time_t
      - all major 64-bit platforms (known to this author) use 64-bit time_t
  * Y2038-SAFE: lighttpd 32-bit builds on platforms using 64-bit time_t
      - Linux x32 ABI (different from i686)
      - FreeBSD all 32-bit and 64-bit architectures *except* 32-bit i386
      - NetBSD 6.0 (released Oct 2012) all 32-bit and 64-bit architectures
      - OpenBSD 5.5 (released May 2014) all 32-bit and 64-bit architectures
      - Microsoft Windows XP and Visual Studio 2005 (? unsure ?)
        Another reference suggests Visual Studio 2015 defaults to 64-bit time_t
      - MacOS 10.15 Catalina (released 2019) drops support for 32-bit apps
  * Y2038-SAFE: lighttpd 32-bit builds on platforms using 32-bit unsigned time_t
      - e.g. OpenVMS (unknown if lighttpd builds on this platform)
  * Y2038-UNSAFE: lighttpd 32-bit builds on platforms using 32-bit signed time_t
      - Linux 32-bit (including i686)
          - glibc 32-bit library support not yet available for 64-bit time_t
              - Linux kernel 5.6 on 32-bit platforms does support 64-bit time_t
                "Note: at this point, 64-bit time support in dual-time
                 configurations is work-in-progress, so for these
                 configurations, the public API only makes the 32-bit time
                 support available. In a later change, the public API will
                 allow user code to choose the time size for a given
                 compilation unit."
              - compiling with -D_TIME_BITS=64 currently has no effect
          - glibc recent (Jul 2021) mailing list discussion
      - FreeBSD i386
      - DragonFlyBSD 32-bit

* Y2038 mitigations attempted on Y2038-UNSAFE platforms (32-bit signed time_t)
  * lighttpd prefers system monotonic clock instead of realtime clock
    in places where realtime clock is not required
  * lighttpd treats negative time_t values as after 19 Jan 2038 03:14:07 GMT
  * (lighttpd presumes that lighttpd will not encounter dates before 1970
    during normal operation.)
  * lighttpd casts struct stat st.st_mtime (and st.st_*time) through uint64_t
    to convert negative timestamps for comparisions with 64-bit timestamps
    (treating negative timestamp values as after 19 Jan 2038 03:14:07 GMT)
  * lighttpd provides unix_time64_t (int64_t) and
  * lighttpd provides struct unix_timespec64 (unix_timespec64_t)
    (struct timespec equivalent using unix_time64_t tv_sec member)
  * lighttpd provides gmtime64_r() and localtime64_r() wrappers
    for platforms 32-bit platforms using 32-bit time_t and
    lighttpd temporarily shifts the year in order to use
    gmtime_r() and localtime_r() (or gmtime() and localtime())
    from standard libraries, before readjusting year and passing
    struct tm to formatting functions such as strftime()
  * lighttpd provides TIME64_CAST() macro to cast signed 32-bit time_t to
    unsigned 32-bit and then to unix_time64_t

* Note: while lighttpd tries handle times past 19 Jan 2038 03:14:07 GMT
  on 32-bit platforms using 32-bit signed time_t, underlying libraries and
  underlying filesystems might not behave properly after 32-bit signed time_t
  overflows (19 Jan 2038 03:14:08 GMT).  If a given 32-bit OS does not work
  properly using negative time_t values, then lighttpd likely will not work
  properly on that system.

* Other references and blogs
2021-09-04 08:08:26 -04:00
Glenn Strauss 3a568c639c [core] http_date_time_append() convenience macro
append IMF-fixdate time string to (buffer *)
2021-08-27 02:16:54 -04:00
Glenn Strauss e6f286ef5f [multiple] reduce use of BUFFER_INTLEN_PTR
reduce use of BUFFER_INTLEN_PTR where b->ptr known not to be NULL
2021-08-27 02:16:54 -04:00
Glenn Strauss af3df29ae8 [multiple] reduce redundant NULL buffer checks
This commit is a large set of code changes and results in removal of
hundreds, perhaps thousands, of CPU instructions, a portion of which
are on hot code paths.

Most (buffer *) used by lighttpd are not NULL, especially since buffers
were inlined into numerous larger structs such as request_st and chunk.

In the small number of instances where that is not the case, a NULL
check is often performed earlier in a function where that buffer is
later used with a buffer_* func.  In the handful of cases that remained,
a NULL check was added, e.g. with r->http_host and r->conf.server_tag.

- check for empty strings at config time and set value to NULL if blank
  string will be ignored at runtime; at runtime, simple pointer check
  for NULL can be used to check for a value that has been set and is not
  blank ("")
- use buffer_is_blank() instead of buffer_string_is_empty(),
  and use buffer_is_unset() instead of buffer_is_empty(),
  where buffer is known not to be NULL so that NULL check can be skipped
- use buffer_clen() instead of buffer_string_length() when buffer is
  known not to be NULL (to avoid NULL check at runtime)
- use buffer_truncate() instead of buffer_string_set_length() to
  truncate string, and use buffer_extend() to extend

Examples where buffer known not to be NULL:
  - cpv->v.b from config_plugin_values_init is not NULL if T_CONFIG_BOOL
    (though we might set it to NULL if buffer_is_blank(cpv->v.b))
  - address of buffer is arg (&foo)
    (compiler optimizer detects this in most, but not all, cases)
  - buffer is checked for NULL earlier in func
  - buffer is accessed in same scope without a NULL check (e.g. b->ptr)

internal behavior change:
  callers must not pass a NULL buffer to some funcs.
  - buffer_init_buffer() requires non-null args
  - buffer_copy_buffer() requires non-null args
  - buffer_append_string_buffer() requires non-null args
  - buffer_string_space() requires non-null arg
2021-08-27 02:16:53 -04:00
Glenn Strauss 980554bc70 [core] simplify buffer_path_simplify() 2021-05-08 14:34:05 -04:00
Glenn Strauss 3a845f7bec [mod_webdav] quiet pedantic compiler warning 2021-04-27 14:03:06 -04:00
Glenn Strauss e20e8e4686 [mod_webdav] accept alt syntax in webdav.opts 2021-04-25 15:46:59 -04:00
Glenn Strauss a7b95c5b6a [mod_expires,mod_webdav] fix truncated date string
(bug on master branch)
2021-04-24 04:26:07 -04:00
Glenn Strauss 8e14cdc5c5 [mod_webdav] always define webdav_mmap_file_chunk
(previously was defined only if mod_webdav was built with
PROPPATCH and/or LOCK support)
2021-04-07 00:20:46 -04:00
Glenn Strauss 325d89b99f [multiple] more reuse of http_date_time_to_str() 2021-04-05 13:24:51 -04:00
Glenn Strauss 7283c43566 [mod_webdav] limit mem use under extreme condition
limit memory use under extreme conditions (edge cases)
2021-04-05 13:24:51 -04:00
Glenn Strauss c6362e127f [mod_webdav] webdav_log_xml_response()
log XML response from chunkqueue instead of taking single buffer
(webdav.log-xml = "enable")

(prep for future changes)
2021-04-05 13:24:51 -04:00
Glenn Strauss 680e6b3bca [multiple] buffer_copy_path_len2() aggregate 2021-04-02 01:16:42 -04:00
Glenn Strauss dc01487ea6 [multiple] use buffer_append_* aggregates
reduces the number of round-trips into some frequently-called routines
2021-04-02 01:16:40 -04:00
Glenn Strauss a6d1dccad3 [multiple] strftime %F and %T
strftime %F for %Y-%m-%d, and %T for %H:%M:%S
2021-04-02 01:16:08 -04:00
Glenn Strauss 81ef66eaf0 [multiple] buffer_has_slash_suffix()
2021-03-26 07:33:42 -04:00
Glenn Strauss 86ede08134 [core] _WIN32 impl of read-only mmap(), munmap() 2021-03-07 04:38:34 -05:00
Glenn Strauss 0cb6466541 [mod_webdav] hide unused funcs depending on build
hide unused funcs depending on build flags
2020-12-27 22:51:40 -05:00
Glenn Strauss 2639e5ae43 [multiple] chunkqueue_write_chunk()
create API in chunk.[ch] for writing a chunk to an fd

(pull similar code from mod_cgi and mod_webdav)

This new API is intended for use on request body input, which is
written to size-limited temporary files controlled by lighttpd and
written to files or pipes.

(network_backend_write() is for writing chunkqueues to sockets)
2020-12-27 09:17:33 -05:00
Glenn Strauss 15bfe5ef0e [mod_webdav] typedef off_t loff_t for FreeBSD
FreeBSD provides an API similar to Linux copy_file_range()
but uses off_t instead of loff_t in the syscall.

off_t is equivalent to off64_t when lighttpd is built with LFS
(and lighttpd is built with LFS enabled by default)
2020-12-27 04:19:11 -05:00
Glenn Strauss b700a8ca09 [multiple] etag.[ch] -> http_etag.[ch]; better imp
more efficient implementation of HTTP ETag generation and comparison

modify dekhash() to take hash value to allow for incremental hashing
2020-12-25 14:41:16 -05:00
Glenn Strauss 1212f60991 buffer_append_path_len() to join paths
use buffer_append_path_len() to join path segments
2020-12-24 16:13:20 -05:00
Glenn Strauss 122094e3e3 [multiple] employ http_date.h, sys-time.h
- replace use of strptime() w/ implementation specialized for HTTP dates
- use thread-safe gmtime_r(), localtime_r() (replace localtime, gmtime)
2020-12-24 16:13:20 -05:00
Glenn Strauss 76faed9145 [multiple] replace fall through comment with attr
replace /* fall through */ comment with __attribute_fallthrough__ macro

Note: not adding attribute to code with external origins:
  xxhash.h (algo_xxhash.h)
so to avoid warnings, may need to compile with -Wno-implicit-fallthrough
2020-12-16 05:16:25 -05:00
Glenn Strauss b9bdca9b8a [mod_webdav] define _ATFILE_SOURCE if AT_FDCWD 2020-12-01 17:08:19 -05:00
Glenn Strauss ab5ae94509 [mod_webdav] compile fix for Mac OSX/11 2020-11-30 01:38:08 -05:00
Glenn Strauss ae228cadb5 [mod_webdav] workaround for gvfs dir redir bug
workaround for gvfs dir redir bug

fix for unhandled live properties

(thx montvid)

  "lighttpd webdav does not work with Nemo, Nautilus gvfs"
2020-11-29 12:20:00 -05:00
Glenn Strauss 88433270c6 [mod_webdav] fallbacks if _ATFILE_SOURCE not avail
Note: filesystem access race conditions exist without _ATFILE_SOURCE
2020-11-27 08:12:21 -05:00
Glenn Strauss 7a0d94cd6f [mod_webdav] pass full path to webdav_unlinkat()
so that path can be removed from stat_cache
2020-11-27 08:12:21 -05:00
Glenn Strauss 730c932e3c [multiple] more forgiving config str to boolean (fixes #3036)
more consistent use of shared code config_plugin_value_tobool()

(thx tow-conf)

  "The on/off keywords in boolean configuration options is inconsistent, which might be misleading and error-prone."
2020-11-16 01:39:14 -05:00
Glenn Strauss ad62991a5b [mod_webdav] define _NETBSD_SOURCE on NetBSD
NetBSD dirent.h improperly hides fdopendir() (POSIX.1-2008) declaration
which should be visible w/ _XOPEN_SOURCE 700 or _POSIX_C_SOURCE 200809L
2020-10-23 21:29:43 -04:00
Glenn Strauss 1cc81b703f [mod_webdav] update defaults after worker_init
update defaults after worker_init for config options in global scope

(bug on master branch; never released)
2020-10-19 21:40:14 -04:00
Glenn Strauss f973b87ca2 [mod_webdav] add missing update to cq accounting
(bug on master branch; never released)
2020-10-19 21:40:14 -04:00
Glenn Strauss 3f1a12e5fb [multiple] remove chunk file.start member
c->offset is now offset into file
c->file.length is end of octets to send (end pos + 1)
c->file.length - c->offset is num of octets to send
2020-10-19 21:40:14 -04:00
Glenn Strauss 4d6d1e790a [multiple] use light_btst() for hdr existence chk 2020-10-11 12:19:27 -04:00
Glenn Strauss ee1f7af24d [core] be more precise checking streaming flags 2020-10-11 12:19:27 -04:00
Glenn Strauss 81029b8b51 [multiple] inline chunkqueue where always alloc'd
inline struct chunkqueue where always allocated in other structs

(memory locality)
2020-10-11 12:19:27 -04:00
Glenn Strauss 367f30a645 [multiple] extend enum http_header_e list 2020-10-11 12:19:26 -04:00
Glenn Strauss 97e314fc9e [multiple] inline chunkqueue_length() 2020-10-11 12:19:26 -04:00
Glenn Strauss 3a766d3d02 [mod_webdav] preserve bytes_out when chunks merged 2020-10-11 12:19:26 -04:00
Glenn Strauss 8940fec894 [multiple] modules read reqbody via fn ptr
r->con->reqbody_read() replaces connection_handle_read_post_state()

future: might provide different callbacks for request body with
Content-Length versus request body sent via Transfer-Encoding: chunked
2020-10-03 09:05:38 -04:00
Glenn Strauss 8b4abaf498 [mod_webdav] fix fallback if linkat() fails
fix fallback if linkat() fails

check at startup if /proc/self/fd is present on systems with O_TMPFILE
(containers might not mount /proc)

  "mod_webdav - PUT files with < 64kb Content-Length reults in zero length file"
2020-08-10 12:54:33 -04:00
Glenn Strauss b2d7635801 [mod_webdav] fix inadvertent string truncation
(bug on master branch; never released)
2020-08-02 07:47:42 -04:00
Glenn Strauss 33c8cf41db [multiple] rename connection_reset hook to request
rename connection_reset to handle_request_reset
2020-08-02 07:47:41 -04:00
Glenn Strauss be8047d82f [build] build fixes for AIX
  "Compile on AIX conflicting types for 'etag_create'"
2020-07-21 12:08:27 -04:00
Glenn Strauss 3119d938cb [mod_webdav] build fix: ifdef live_properties 2020-07-10 21:54:27 -04:00