Commit Graph

137 Commits

Author SHA1 Message Date
Glenn Strauss 741513ecd1 [mod_ssi] merge mod_ssi_expr.c into mod_ssi.c
isolate this ancient relic
2021-11-12 17:56:38 -05:00
Glenn Strauss 6bd6226e90 [multiple] avoid empty chunks in chunkqueue
(This might render some calls to chunkqueue_remove_finished_chunks()
 no longer necessary)
2021-09-30 17:34:03 -04:00
Glenn Strauss 7b615d5d24 [multiple] de-dup file and piped loggers (fixes #3101)
de-dup file and piped loggers for error logs and access logs

x-ref:
  "RFE: de-dup file and piped loggers"
  https://redmine.lighttpd.net/issues/3101
2021-09-13 03:37:11 -04:00
Glenn Strauss 67c0b1498a [multiple] remove base.h include where not used
(substitute request.h if file only accesses request_st,
 and not connection or server structs)
2021-09-08 15:06:07 -04:00
Glenn Strauss 309c1693ac [multiple] Y2038 32-bit signed time_t mitigations
Most OS platforms have already provided solutions to
Y2038 32-bit signed time_t 5 - 10 years ago (or more!)
Notable exceptions are Linux i686 and FreeBSD i386.

Since 32-bit systems tend to be embedded systems,
and since many distros take years to pick up new software,
this commit aims to provide Y2038 mitigations for lighttpd
running on 32-bit systems with Y2038-unsafe 32-bit signed time_t

* Y2038: lighttpd 1.4.60 and later report Y2038 safety
  $ lighttpd -V
    + Y2038 support                                    # Y2038-SAFE
  $ lighttpd -V
    - Y2038 support (unsafe 32-bit signed time_t)      # Y2038-UNSAFE

* Y2038: general platform info
  * Y2038-SAFE: lighttpd 64-bit builds on platforms using 64-bit time_t
      - all major 64-bit platforms (known to this author) use 64-bit time_t
  * Y2038-SAFE: lighttpd 32-bit builds on platforms using 64-bit time_t
      - Linux x32 ABI (different from i686)
      - FreeBSD all 32-bit and 64-bit architectures *except* 32-bit i386
      - NetBSD 6.0 (released Oct 2012) all 32-bit and 64-bit architectures
      - OpenBSD 5.5 (released May 2014) all 32-bit and 64-bit architectures
      - Microsoft Windows XP and Visual Studio 2005 (? unsure ?)
        Another reference suggests Visual Studio 2015 defaults to 64-bit time_t
      - MacOS 10.15 Catalina (released 2019) drops support for 32-bit apps
  * Y2038-SAFE: lighttpd 32-bit builds on platforms using 32-bit unsigned time_t
      - e.g. OpenVMS (unknown if lighttpd builds on this platform)
  * Y2038-UNSAFE: lighttpd 32-bit builds on platforms using 32-bit signed time_t
      - Linux 32-bit (including i686)
          - glibc 32-bit library support not yet available for 64-bit time_t
              - https://sourceware.org/glibc/wiki/Y2038ProofnessDesign
              - Linux kernel 5.6 on 32-bit platforms does support 64-bit time_t
                https://itsubuntu.com/linux-kernel-5-6-to-fix-the-year-2038-issue-unix-y2k/
              - https://www.gnu.org/software/libc/manual/html_node/64_002dbit-time-symbol-handling.html
                "Note: at this point, 64-bit time support in dual-time
                 configurations is work-in-progress, so for these
                 configurations, the public API only makes the 32-bit time
                 support available. In a later change, the public API will
                 allow user code to choose the time size for a given
                 compilation unit."
              - compiling with -D_TIME_BITS=64 currently has no effect
          - glibc recent (Jul 2021) mailing list discussion
              - https://public-inbox.org/bug-gnulib/878s2ozq70.fsf@oldenburg.str.redhat.com/T/
      - FreeBSD i386
      - DragonFlyBSD 32-bit

* Y2038 mitigations attempted on Y2038-UNSAFE platforms (32-bit signed time_t)
  * lighttpd prefers system monotonic clock instead of realtime clock
    in places where realtime clock is not required
  * lighttpd treats negative time_t values as after 19 Jan 2038 03:14:07 GMT
  * (lighttpd presumes that lighttpd will not encounter dates before 1970
    during normal operation.)
  * lighttpd casts struct stat st.st_mtime (and st.st_*time) through uint64_t
    to convert negative timestamps for comparisions with 64-bit timestamps
    (treating negative timestamp values as after 19 Jan 2038 03:14:07 GMT)
  * lighttpd provides unix_time64_t (int64_t) and
  * lighttpd provides struct unix_timespec64 (unix_timespec64_t)
    (struct timespec equivalent using unix_time64_t tv_sec member)
  * lighttpd provides gmtime64_r() and localtime64_r() wrappers
    for platforms 32-bit platforms using 32-bit time_t and
    lighttpd temporarily shifts the year in order to use
    gmtime_r() and localtime_r() (or gmtime() and localtime())
    from standard libraries, before readjusting year and passing
    struct tm to formatting functions such as strftime()
  * lighttpd provides TIME64_CAST() macro to cast signed 32-bit time_t to
    unsigned 32-bit and then to unix_time64_t

* Note: while lighttpd tries handle times past 19 Jan 2038 03:14:07 GMT
  on 32-bit platforms using 32-bit signed time_t, underlying libraries and
  underlying filesystems might not behave properly after 32-bit signed time_t
  overflows (19 Jan 2038 03:14:08 GMT).  If a given 32-bit OS does not work
  properly using negative time_t values, then lighttpd likely will not work
  properly on that system.

* Other references and blogs
  - https://en.wikipedia.org/wiki/Year_2038_problem
  - https://en.wikipedia.org/wiki/Time_formatting_and_storage_bugs
  - http://www.lieberbiber.de/2017/03/14/a-look-at-the-year-20362038-problems-and-time-proofness-in-various-systems/
2021-09-04 08:08:26 -04:00
Glenn Strauss af3df29ae8 [multiple] reduce redundant NULL buffer checks
This commit is a large set of code changes and results in removal of
hundreds, perhaps thousands, of CPU instructions, a portion of which
are on hot code paths.

Most (buffer *) used by lighttpd are not NULL, especially since buffers
were inlined into numerous larger structs such as request_st and chunk.

In the small number of instances where that is not the case, a NULL
check is often performed earlier in a function where that buffer is
later used with a buffer_* func.  In the handful of cases that remained,
a NULL check was added, e.g. with r->http_host and r->conf.server_tag.

- check for empty strings at config time and set value to NULL if blank
  string will be ignored at runtime; at runtime, simple pointer check
  for NULL can be used to check for a value that has been set and is not
  blank ("")
- use buffer_is_blank() instead of buffer_string_is_empty(),
  and use buffer_is_unset() instead of buffer_is_empty(),
  where buffer is known not to be NULL so that NULL check can be skipped
- use buffer_clen() instead of buffer_string_length() when buffer is
  known not to be NULL (to avoid NULL check at runtime)
- use buffer_truncate() instead of buffer_string_set_length() to
  truncate string, and use buffer_extend() to extend

Examples where buffer known not to be NULL:
  - cpv->v.b from config_plugin_values_init is not NULL if T_CONFIG_BOOL
    (though we might set it to NULL if buffer_is_blank(cpv->v.b))
  - address of buffer is arg (&foo)
    (compiler optimizer detects this in most, but not all, cases)
  - buffer is checked for NULL earlier in func
  - buffer is accessed in same scope without a NULL check (e.g. b->ptr)

internal behavior change:
  callers must not pass a NULL buffer to some funcs.
  - buffer_init_buffer() requires non-null args
  - buffer_copy_buffer() requires non-null args
  - buffer_append_string_buffer() requires non-null args
  - buffer_string_space() requires non-null arg
2021-08-27 02:16:53 -04:00
Glenn Strauss 980554bc70 [core] simplify buffer_path_simplify() 2021-05-08 14:34:05 -04:00
Glenn Strauss 1aae63af62 [core] remove buffer_is_equal_right_len()
(not widely used or widely useful)
2021-05-07 16:39:52 -04:00
Glenn Strauss 9a5e1652be [multiple] static file optimization; reuse cache
reuse cache lookup in common case of serving a static file
rather than repeating the stat_cache_entry lookup
(which is more work than memcmp() to re-check stat_cache_entry match)
2021-05-06 17:35:00 -04:00
Glenn Strauss 680e6b3bca [multiple] buffer_copy_path_len2() aggregate 2021-04-02 01:16:42 -04:00
Glenn Strauss dc01487ea6 [multiple] use buffer_append_* aggregates
reduces the number of round-trips into some frequently-called routines
2021-04-02 01:16:40 -04:00
Glenn Strauss f608217955 [mod_ssi] use intermediate chunkqueue to aggregate
periodically flush intermediate chunkqueue to r->write_queue
to reduce occurrence of copying to reallocate larger chunk

use chunk_buffer to read ssi file
2021-04-02 01:16:12 -04:00
Glenn Strauss 7c8437a0ec [mod_ssi] shared code to wrap strftime() 2021-04-02 01:16:12 -04:00
Glenn Strauss a6d1dccad3 [multiple] strftime %F and %T
strftime %F for %Y-%m-%d, and %T for %H:%M:%S
2021-04-02 01:16:08 -04:00
Glenn Strauss 76b9cb8717 [mod_ssi] inline some buffers in ssi plugin_data 2021-03-26 07:33:42 -04:00
Glenn Strauss ea38287a4c [multiple] do not clear physical.path if finished
do not reset r->physical.path (unneeded) if returning HANDLER_FINISHED

error handler will reset r->physical.path, if necessary
2021-03-26 07:33:42 -04:00
Glenn Strauss 19bc88850e [multiple] add attrs from gcc -Wsuggest-attribute= 2021-03-26 07:33:41 -04:00
Glenn Strauss c95f832f99 [core] http_cgi.[ch] CGI interfaces (RFC 3875)
collect Common Gateway Interface (CGI) interfaces (RFC 3875)
2021-03-07 04:38:34 -05:00
Glenn Strauss b700a8ca09 [multiple] etag.[ch] -> http_etag.[ch]; better imp
more efficient implementation of HTTP ETag generation and comparison

modify dekhash() to take hash value to allow for incremental hashing
2020-12-25 14:41:16 -05:00
Glenn Strauss 1212f60991 buffer_append_path_len() to join paths
use buffer_append_path_len() to join path segments
2020-12-24 16:13:20 -05:00
Glenn Strauss 122094e3e3 [multiple] employ http_date.h, sys-time.h
- replace use of strptime() w/ implementation specialized for HTTP dates
- use thread-safe gmtime_r(), localtime_r() (replace localtime, gmtime)
2020-12-24 16:13:20 -05:00
Glenn Strauss 29f50bdeb0 [multiple] fdevent_waitpid() wrapper 2020-12-24 16:13:17 -05:00
Glenn Strauss ad22eebd23 [mod_ssi] use stat_cache_open_rdonly_fstat() 2020-10-11 12:19:27 -04:00
Glenn Strauss 81029b8b51 [multiple] inline chunkqueue where always alloc'd
inline struct chunkqueue where always allocated in other structs

(memory locality)
2020-10-11 12:19:27 -04:00
Glenn Strauss 76bd8bba9a [mod_ssi] init status var before waitpid() 2020-10-11 12:19:27 -04:00
Glenn Strauss 9f8a896891 [mod_ssi] update chunk after shell output redirect
chunkqueue_update_file() to update size of file chunk after
redirecting shell output to file fd
2020-10-11 12:19:26 -04:00
Glenn Strauss 33c8cf41db [multiple] rename connection_reset hook to request
rename connection_reset to handle_request_reset
2020-08-02 07:47:41 -04:00
Glenn Strauss 446866333f [multiple] use *(unsigned char *) with ctypes 2020-07-08 22:51:31 -04:00
Glenn Strauss 7c7f8c467c [multiple] split con, request (very large change)
NB: r->tmp_buf == srv->tmp_buf (pointer is copied for quicker access)

NB: request read and write chunkqueues currently point to connection
    chunkqueues; per-request and per-connection chunkqueues are
    not distinct from one another
      con->read_queue  == r->read_queue
      con->write_queue == r->write_queue

NB: in the future, a separate connection config may be needed for
    connection-level module hooks.  Similarly, might need to have
    per-request chunkqueues separate from per-connection chunkqueues.
    Should probably also have a request_reset() which is distinct from
    connection_reset().
2020-07-08 19:54:29 -04:00
Glenn Strauss 31d9495330 [core] store subrequest_handler instead of mode
store pointer to module in handler_module instead of con->mode id
2020-07-08 19:54:29 -04:00
Glenn Strauss 100dfaa3f3 [core] move plugin_ctx into (request_st *)
NB: in the future, a separate plugin_ctx may be needed for
    connection-level plugins to keep state across multiple requests
2020-07-08 19:54:29 -04:00
Glenn Strauss 8131e4396d [core] move addtl request-specific struct members 2020-07-08 19:54:29 -04:00
Glenn Strauss 1474be7859 [core] move addtl request-specific struct members 2020-07-08 19:54:29 -04:00
Glenn Strauss fda01e3305 [core] support multiple error logs
e.g. different server.errorlog for different virtual hosts

Also, support different server.breakagelog to have separate script
error logs, applicable to mod_cgi and mod_ssi exec.
2020-07-08 19:54:29 -04:00
Glenn Strauss 27e27e5e40 [core] static buffers for mtime_cache 2020-07-08 19:54:29 -04:00
Glenn Strauss 03b4c993d2 [multiple] generic config array type checking 2020-07-08 19:54:28 -04:00
Glenn Strauss 68d8d4c532 [multiple] stat_cache singleton 2020-07-08 19:54:28 -04:00
Glenn Strauss 50bdb55de8 [multiple] connection hooks no longer get (srv *)
(explicit (server *) not passed; available in con->srv)
2020-07-08 19:54:28 -04:00
Glenn Strauss 010c28949c [multiple] prefer (connection *) to (srv *)
convert all log_error_write() to log_error() and pass (log_error_st *)

use con->errh in preference to srv->errh (even though currently same)

avoid passing (server *) when previously used only for logging (errh)
2020-07-08 19:54:28 -04:00
Glenn Strauss 24680a9142 [core] array_init() arg for initial size 2020-07-08 18:08:52 -04:00
Glenn Strauss 2ec70f234a [core] stat_cache_path_contains_symlink use errh
use log_error() with con->errh
2020-07-08 18:08:52 -04:00
Glenn Strauss b73949e03f [multiple] plugin.c handles common FREE_FUNC code
(simpler for modules; less boilerplate to cut-n-paste)
2020-07-08 18:08:51 -04:00
Glenn Strauss ed62e354ff [core] use config_plugin_values_init() 2020-07-08 18:08:51 -04:00
Glenn Strauss 5b58e5e47e [mod_ssi] use config_plugin_values_init() 2020-05-23 17:59:30 -04:00
Glenn Strauss d7a6a7a263 [core] (const buffer *) from strftime_cache_get() 2020-05-23 17:59:29 -04:00
Glenn Strauss e2de4e581e [core] const char *name in struct plugin
put void *data (always used) as first member of struct plugin

add int nconfig member to PLUGIN_DATA

calloc() inits p->data to NULL
2020-05-23 17:59:29 -04:00
Glenn Strauss 36f64b26a1 [core] simpler config_check_cond()
optimize for common case where condition has been evaluated for
the request and a cached result exists

(also: begin isolating data_config)
2020-05-23 17:59:29 -04:00
Glenn Strauss e3dc34d142 [core] array a->sorted[] as ptrs rather than pos
While slightly more memory use in 64-bit (though same memory use as
prior versions of lighttpd), avoids bouncing through second array
when searching in sorted list.  Most use of arrays in lighttpd is to
build a list once, and elements are not removed from the list.
2020-05-23 17:59:29 -04:00
Glenn Strauss a762402da5 [core] keep a->data[] sorted (REVERT)
This reverts commit 2260a8062ee599ecf28d9b52b981603fd2084aff.

original ordering of array elements is significant
e.g. in lighttpd.conf lists where first match to request is applied
2020-02-24 11:15:32 -05:00
Glenn Strauss 601c572c39 [core] inline buffer as part of data_string value
(instead of value being (buffer *))
2020-02-24 11:15:32 -05:00