Commit Graph

57 Commits

Author SHA1 Message Date
Glenn Strauss 2e0676fd6d [core] extend (data_string *) to store header id
(optional addition to (data_string *), used by http_header.[ch])

extend (data_string *) instead of creating another data_* TYPE_*
  (new data type would probably have (data_string *) as base class)
  (might revisit choice in the future)

HTTP_HEADER_UNSPECIFIED has been removed.  It was used in select
locations as an optimization to avoid looking up enum header_header_e
before checking the array, but the ordering in the array now relies
on having the id.  Having the id allows for a quick check if a common
header is present or not in the htags bitmask, before checking the
array, and allows for integer comparison in the log(n) search of the
array, instead of strncasecmp().

With HTTP_HEADER_UNSPECIFIED removed, add optimization to set bit
in htags for HTTP_HEADER_OTHER when an "other" header is added,
but do not clear the bit, as there might be addtl "other" headers
2020-10-11 12:19:26 -04:00
Glenn Strauss 68ec5ad642 [core] array.[ch] using uint32_t instead of size_t 2020-10-11 12:19:26 -04:00
Glenn Strauss 24680a9142 [core] array_init() arg for initial size 2020-07-08 18:08:52 -04:00
Glenn Strauss b87e8783c4 [core] config_plugin_values_init() new interface
new data structures and interface for processing config directives
(towards more efficient approach to config merging)

continue work to isolate data_config
2020-05-23 17:59:29 -04:00
Glenn Strauss e3dc34d142 [core] array a->sorted[] as ptrs rather than pos
While slightly more memory use in 64-bit (though same memory use as
prior versions of lighttpd), avoids bouncing through second array
when searching in sorted list.  Most use of arrays in lighttpd is to
build a list once, and elements are not removed from the list.
2020-05-23 17:59:29 -04:00
Glenn Strauss a762402da5 [core] keep a->data[] sorted (REVERT)
This reverts commit 2260a8062ee599ecf28d9b52b981603fd2084aff.

original ordering of array elements is significant
e.g. in lighttpd.conf lists where first match to request is applied
2020-02-24 11:15:32 -05:00
Glenn Strauss c2238256e2 [core] inline array as part of data_array value
(instead of value being (array *))
2020-02-24 11:15:32 -05:00
Glenn Strauss 601c572c39 [core] inline buffer as part of data_string value
(instead of value being (buffer *))
2020-02-24 11:15:32 -05:00
Glenn Strauss b7942c58cc [core] (data_unset *) from array_get_element_klen
return (data_unset *) from array_get_element_klen() to prep for
putting buffer into data_string for value member, rather than as
(buffer *)

(allow in-place modification of these buffer values)
2020-02-24 11:15:32 -05:00
Glenn Strauss ad9b7e009b [core] inline buffer as part of DATA_UNSET key
(instead of key being (buffer *))
2020-02-24 11:15:32 -05:00
Glenn Strauss 83535bbef3 [core] differentiate array_get_* for ro and rw
array_get_element_klen() is now intended for read-only access
array_get_data_unset() is used by config processing for r/w access
array_get_buf_ptr() is used for r/w access to ds->value (string buffer)
2020-02-24 11:15:32 -05:00
Glenn Strauss 5c0c4936b4 [core] __attribute_returns_nonnull__ 2020-02-24 11:15:32 -05:00
Glenn Strauss c9f1b612f1 [core] keep a->data[] sorted; remove a->sorted[] 2020-02-24 11:15:32 -05:00
Glenn Strauss ddb78f75ee [core] remove unused array_reset() 2020-02-24 11:15:32 -05:00
Glenn Strauss 70b5d729ae [core] mark some data_* funcs cold
mark funcs cold if seldom used or used only at startup config processing

mark most data_config_* funcs cold


2020-02-24 11:15:32 -05:00
Glenn Strauss b2991c686d [core] perf: array.c performance enhancements
mark array_get_index() as hot, rewrite to be pure and return sorted pos

mark routines as pure, as appropriate

mark routines as cold if used only at startup for config processing

mark params const, as appropriate

array_get_buf_ptr() for modifiable value buffer after insert into array

uint32_t used and size members instead of size_t

remove a->unique_ndx member; simply add to end of array for value lists
remove du->is_index_key member; simply check buffer_is_empty(du->key)

array_insert_key_value() used to be a hint that lookup could be skipped,
but the state from array_get_index() is now saved and reused internally,
so the distinction is no longer needed.  Use array_set_key_value().
2020-02-24 11:15:32 -05:00
Glenn Strauss 062089ff14 [core] perf: array_reset_data_strings()
array_reset_data_strings() specialization
2018-10-22 20:28:53 -04:00
Glenn Strauss 8c7f1dfb03 [core] more memory-efficient fn table for data_*
save 40 bytes (64-bit), or 16 bytes (32-bit) per data_* element
at the cost of going through indirect function pointer to execute
methods.  At runtime, the reset() method is most used among them.
2018-09-23 18:01:58 -04:00
Glenn Strauss 002a4c524d [core] array_get_int_ptr() 2018-09-23 18:01:58 -04:00
Glenn Strauss e6741acd4e [core] code reuse array_match_*() routines 2018-09-23 18:01:58 -04:00
Glenn Strauss a7c27c9f99 [core] code reuse with array_insert_key_value()
code reuse with array_insert_key_value() and related array manipulation
2018-09-16 05:18:05 -04:00
Glenn Strauss a90526374f [core] abstraction to insert/modify response hdrs
consistent use of abstraction to insert/modify response headers
2018-09-16 05:18:05 -04:00
Glenn Strauss 04d76e7afd [core] some header cleanup
provide standard types in first.h instead of base.h
provide lighttpd types in base_decls.h instead of settings.h
reduce headers exposed by headers for core data structures
  do not expose <pcre.h> or <stdlib.h> in headers
move stat_cache_entry to stat_cache.h
reduce use of "server.h" and "base.h" in headers
2018-04-08 22:22:23 -04:00
Glenn Strauss 2496c1af4c [core] pass array_get_element_klen() const array * 2018-02-02 06:22:33 -05:00
Glenn Strauss 9b9f445a7b [mod_proxy] move data_fastcgi into mod_proxy.c
(data_fastcgi is used only by mod_proxy at this point)
2017-07-15 22:42:15 -04:00
Glenn Strauss 86bb8be2c8 [core] perf: skip redundant strlen() if len known
performance: skip redundant strlen() if length is already known

introduce array_get_element_klen() to take key and klen params
2017-05-15 22:02:33 -04:00
Glenn Strauss bd77abe0f8 [config] more specific checks for array lists
More specific checks on contents of array lists.  Each module using
lists now does better checking on the types of values in the list
(strings, integers, arrays/lists)

This helps prevent misconfiguration of things like cgi.assign,
fastcgi.server, and scgi.server, where source code might be
served as static files if parenthesis are misplaced.

2017-03-08 11:42:59 -05:00
Glenn Strauss 78047a3c1b [core] $REQUEST_HEADER[...] subsumes other config (#1556)
  "Allow matching against any arbitrary HTTP header in the configuration file regexps"
2017-02-27 02:05:56 -05:00
Glenn Strauss 2ac2911b9a [core] config match w/ arbitrary HTTP request hdrs (fixes #1556)
  "Allow matching against any arbitrary HTTP header in the configuration file regexps"
2017-01-31 14:36:16 -05:00
Glenn Strauss 79fb75709b [core] optional condition in config "else" clause (fixes #1268)
  "condition should be optional in "else" clause in configuration file"
2016-10-09 08:06:41 -04:00
Glenn Strauss 381aaae363 remove unused array type TYPE_COUNT data_count
(unused type, and very similar to TYPE_INTEGER data_integer,
 differing only in initial value and how dup inserts are merged)
2016-09-22 19:54:57 -04:00
Stefan Bühler 5c68caa6d7 [core] replace array weakref with vector
From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-03-19 15:27:38 +00:00
Glenn Strauss 8abd06a7ff consistent inclusion of config.h at top of files (fixes #2073)
From: Glenn Strauss <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-03-19 15:14:35 +00:00
Stefan Bühler 68e4a416cc [core] provide array_extract_element and use it
From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-03-15 18:56:02 +00:00
Stefan Bühler 224bf545c1 [core] refactor array search; raise array size limit to SSIZE_MAX
- raise array size limit from INT_MAX to SSIZE_MAX. INT_MAX already
  is way to high to be hit in any sane scenario, but SSIZE_MAX can
  *obviously* not be hit due to memory constraints.
- use size_t for array indices instead of int
- use binary search instead of next_power_of_2 hack; document invariants
  and check them in debug mode (asserts).
- return the actual insert position instead of something near

From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-03-15 18:41:57 +00:00
Stefan Bühler 8d8ae9cbc8 [core] improve array API to prevent theoretical memory leaks
- refactor insert into array_find_or_insert; if the element already
  exists the caller must resolve the conflict manually:
  - array_replace frees the old element
  - array_insert_unique calls "insert_dup"
  both have no return value anymore
- fix usages of array_replace; they now don't need to delete the old
  entry anymore; usage in configparser was probably broken, as it
  possibly deleted the old element before calling array_replace

This should fix a lot of the issues reported in "Fortify Open Review
Project - lighttpd 1.4.39" (usually hitting the array limit):
when the array size limit was hit "new" entries leaked instead of
getting added.

On 32-bit INT_MAX entries cannot actually be reached (each entry
requires at least 48 bytes, leading to a total of 96GB memory).

On 64-bit INT_MAX entries would require 224GB memory, so it would be
theoretically possible. But it would need 2^27 reallocations of two
C-arrays of up to 16GB size.

From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-03-15 18:26:57 +00:00
Stefan Bühler ad65603ec0 [core] fix conditional cache handling
- add new "skip" result to mark conditions that didn't actually get
  evaluated to false but just skipped because the preconditions failed.
- add "local_result" for each cache entry to remember whether the
  condition itself matched (not including the preconditions).
  this can be reused after a cache reset if the condition itself was not
  reset, but the preconditions were
- clear result of subtree (children and else-branches) when clearing a
  condition cache

From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-02-21 18:32:14 +00:00
Stefan Bühler 22e8b456a9 Fix header inclusion order, always include "config.h" before any system header
git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2009-10-11 14:31:42 +00:00
Stefan Bühler 4f4d40f3f7 Add "lighty.req_env" table to mod_magnet for setting/getting environment values for cgi (fixes #1967, thx presbrey)
git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2009-06-10 13:08:15 +00:00
Stefan Bühler 066b208bb6 Added Language conditional (fixes #1119); patch by petar
git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2009-02-05 22:36:58 +00:00
Xuefer 9ffe08a583 merged part of [1528] from trunk: added the new options and added alternative names for user-agent, remote-ip and query-string
git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2008-03-02 11:46:44 +00:00
Elan Ruusamäe d6b9709b54 - support for matching $HTTP["scheme"] in configs, patch from:
  apparently by swiergot/at/, adjusted for current svn.

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2007-11-04 16:53:17 +00:00
Jan Kneschke 3940c60e68 fixed aggressive caching of conditionals (#41)
$HTTP["url"] =~ "" { cgi.assign = ... } fails if there is a module
loaded which is called before uri_clean is set (mod_exforward,
mod_rewrite, ...) 

- merged [1792], [1798], [1807], [1810], [1811] from trunk

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2007-08-18 09:27:11 +00:00
Marcus Rückert 8cd1471cb3 - white space cleanup part 2 this time 1.4 ;)
i hope it helps with merging stuff back to 1.5

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2006-10-04 13:26:23 +00:00
Jan Kneschke 3b73da83a2 merged [1217]
fixed allow-http-11

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2006-09-07 11:05:41 +00:00
Jan Kneschke b4d9d061c8 either remove include config.h or covered it in HAVE_CONFIG_H
git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2005-09-24 12:06:04 +00:00
Xuefer f387d89870 fix merging of auto-indexing element, and make autoload modules hack to use array merging
git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2005-08-23 14:35:01 +00:00
Jan Kneschke c052589bb3 load mod_indexfile before, mod_fastcgi and friends and let array regenerated unique ids if necessary
git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2005-08-17 09:56:33 +00:00
Xuefer 6044dbc5b3 make output of array/config same format as lighttpd.conf, with auto indention
git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2005-08-17 06:47:07 +00:00
Jan Kneschke 75c3a8393a added include_shell option to configfiles (merged the rest of the trunk changesets)
git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2005-08-09 06:42:33 +00:00