Commit Graph

1924 Commits (4ef4baa59d5459ee5568dc3ca5db4b2df40f426a)
 

Author SHA1 Message Date
Glenn Strauss 4ef4baa59d http_response_backend_error()
consolidate repeated code in dynamic handlers which manipulates
con->file_finished.  Centralize calls to http_chunk_close().

(mod_cgi, mod_fastcgi, mod_scgi, mod_proxy)
7 years ago
Glenn Strauss 923688d2da drain backend socket/pipe bufs upon FDEVENT_HUP
(mod_cgi, mod_fastcgi, mod_scgi, mod_proxy)
7 years ago
Glenn Strauss 18a7b2be37 [core] option to stream response body to client (fixes #949, #760, #1283, #1387)
Set server.stream-response-body = 1 or server.stream-response-body = 2
to have lighttpd stream response body to client as it arrives from the
backend (CGI, FastCGI, SCGI, proxy).

default: buffer entire response body before sending response to client.
(This preserves existing behavior for now, but may in the future be
 changed to stream response to client, which is the behavior more
 commonly expected.)

x-ref:
  "fastcgi, cgi, flush, php5 problem."
  https://redmine.lighttpd.net/issues/949
  "Random crashing on FreeBSD 6.1"
  https://redmine.lighttpd.net/issues/760
  "Memory usage increases when proxy+ssl+large file"
  https://redmine.lighttpd.net/issues/1283
  "lighttpd+fastcgi memory problem"
  https://redmine.lighttpd.net/issues/1387
7 years ago
Glenn Strauss ddfae019cb separate routines for reading output from backends
move code in dynamic handlers (CGI, FastCGI, SCGI, proxy)
from *_handle_fdevent() to *_recv_response() for reuse
outside the *_handle_fdevent() routine
7 years ago
Glenn Strauss f69f209e6d [core] option to stream request body to backend (fixes #376)
Set server.stream-request-body = 1 or server.stream-request-body = 2
to have lighttpd connect to backend (CGI, FastCGI, SCGI, proxy)
immediately after parsing request headers, and to stream request body
as it arrives.

default: buffer entire request body before connecting to backend,
in order to avoid tying up (limited) backend resources which are often
implemented using libraries which wait for entire request body before
proceeding.

x-ref:
  "Reimplement upload (POST) handling to match apache/zeus/thttpd/boa functionality"
  https://redmine.lighttpd.net/issues/376
7 years ago
Glenn Strauss 695c8f4e07 [config] config options to stream request/response (#949, #376)
This allows admin to configure if response is collected in entirety
prior to sending data to client

For compatibility with existing configs, default is existing behavior:
  buffer entire response prior to sending data to client

The following are config options, though not all implemented yet

// default: buffer entire request body before connecting to backend
server.stream-request-body = 0

// stream request body to backend; buffer to temp files
server.stream-request-body = 1

// stream request body to backend; minimal buffering might block upload
server.stream-request-body = 2

// default: buffer entire response body before sending to client
server.stream-request-body = 0

// stream response body to client; buffer to temp files
server.stream-request-body = 1

// stream response body to client; minimal buffering might block backend
server.stream-request-body = 2

x-ref:
  "fastcgi, cgi, flush, php5 problem."
  https://redmine.lighttpd.net/issues/949
 "Reimplement upload (POST) handling to match apache/zeus/thttpd/boa functionality"
  https://redmine.lighttpd.net/issues/376
7 years ago
Glenn Strauss 5ab7944d34 [TLS] release openssl buffers as used (fixes #1265, fixes #1283, #881)
use SSL_MODE_RELEASE_BUFFERS (OpenSSL >= 1.0.0) to free buffers
as they are used, to potentially reduce memory footprint of
idle SSL connections

x-ref:
  "memory usage when ssl.engine used and large data uploaded through CGI"
  https://redmine.lighttpd.net/issues/881
  "SSL + file upload = lots of memory"
  https://redmine.lighttpd.net/issues/1265
  "Memory usage increases when proxy+ssl+large file"
  https://redmine.lighttpd.net/issues/1283
7 years ago
Glenn Strauss 53f550b290 [core] stream response to client (#949)
This replaces buffering entire response prior to sending data to client

x-ref:
  "fastcgi, cgi, flush, php5 problem."
  https://redmine.lighttpd.net/issues/949
7 years ago
Glenn Strauss 5a91fd4b90 [core] buffer large responses to tempfiles (fixes #758, fixes #760, fixes #933, fixes #1387, #1283, fixes #2083)
This replaces buffering entire response in memory which might lead to
huge memory footprint and possibly to memory exhaustion.

use tempfiles of fixed size so disk space is freed as each file sent

update callers of http_chunk_append_mem() and http_chunk_append_buffer()
to handle failures when writing to tempfile.

x-ref:
  "memory fragmentation leads to high memory usage after peaks"
  https://redmine.lighttpd.net/issues/758
  "Random crashing on FreeBSD 6.1"
  https://redmine.lighttpd.net/issues/760
  "lighty should buffer responses (after it grows above certain size) on disk"
  https://redmine.lighttpd.net/issues/933
  "Memory usage increases when proxy+ssl+large file"
  https://redmine.lighttpd.net/issues/1283
  "lighttpd+fastcgi memory problem"
  https://redmine.lighttpd.net/issues/1387
  "Excessive Memory usage with streamed files from PHP"
  https://redmine.lighttpd.net/issues/2083
7 years ago
Glenn Strauss 4f6bd42268 [mod_webdav] create file w/ LOCK request if ENOENT
(apply to wider scope)

Also, track creation of Destination to return proper
201 Created or 204 No Content (in order to pass more 'litmus' tests)
7 years ago
Glenn Strauss 34fbc868cf [mod_webdav] getetag and lockdiscovery live props
support getetag and lockdiscovery live properties in propfind requests
7 years ago
Glenn Strauss 488688301e [doc] NEWS 7 years ago
Glenn Strauss aee88d6916 [mod_webdav] create file w/ LOCK request if ENOENT
mod_webdav now works with davfs2 mounts on Linux
7 years ago
Glenn Strauss 4eeeb8fc76 [config] server.bsd-accept-filter option
BSD accept() filters

server.bsd-accept-filter = ""           (default)
server.bsd-accept-filter = "httpready"
server.bsd-accept-filter = "dataready"

Note: this is a behavior change from prior versions.
The default is now no additional accept() filter, whereas prior
versions unconditionally enabled "httpready" accept() filter

Additionally, server.defer-accept (Linux) is inherited from global scope
into $SERVER["socket"] blocks

github: closes #65
7 years ago
Glenn Strauss f3e36ccdbb use buffer_string_set_length() to truncate strings 7 years ago
Glenn Strauss df8032a7c8 use buffer_string_set_length() to truncate strings 7 years ago
Glenn Strauss c463860451 minor: quiet some compiler warnings 7 years ago
Glenn Strauss bb93414a23 [build_cmake] use MODULE on Mac OS X (fixes #1761)
add instructions in INSTALL to build using CMake and Xcode on Mac OS X
  along with MacPorts
update CMake to link test_configfile with pcre

x-ref:
  "loadable-module naming on macosx + cmake"
  https://redmine.lighttpd.net/issues/1761
7 years ago
Glenn Strauss e8dd8fdb53 [config] normalize IP strings in lighttpd.conf
normalize IP strings in lighttpd.conf conditionals:
  $SERVER["socket"], $HTTP["remoteip"], and $HTTP["host"]
7 years ago
Glenn Strauss b47494d4cd [config] opts for http header parsing strictness (fixes #551, fixes #1086, fixes #1184, fixes #2143, #2258, #2281, fixes #946, fixes #1330, fixes #602, #1016)
server.http-parseopt-header-strict  = "enable"
server.http-parseopt-host-strict    = "enable"  (implies host-normalize)
server.http-parseopt-host-normalize = "disable"

defaults retain current behavior, which is strict header parsing
and strict host parsing, with enhancement to normalize IPv4 address
and port number strings.

For lighttpd tests, these need to be enabled (and are by default)
For marginally faster HTTP header parsing for benchmarks, disable these.

To allow
  - underscores in hostname
  - hypen ('-') at beginning of hostname
  - all-numeric TLDs
  server.http-parseopt-host-strict    = "disable"

x-ref:
  "lighttpd doesn't allow underscores in host names"
  https://redmine.lighttpd.net/issues/551
  "hyphen in hostname"
  https://redmine.lighttpd.net/issues/1086
  "a numeric tld"
  https://redmine.lighttpd.net/issues/1184
  "Numeric tld's"
  https://redmine.lighttpd.net/issues/2143
  "Bad Request"
  https://redmine.lighttpd.net/issues/2258
  "400 Bad Request when using Numeric TLDs"
  https://redmine.lighttpd.net/issues/2281

To allow a variety of numerical formats to be converted to IP addresses
  server.http-parseopt-host-strict    = "disable"
  server.http-parseopt-host-normalize = "enable"

x-ref:
  "URL encoding leads to "400 - Bad Request""
  https://redmine.lighttpd.net/issues/946
  "400 Bad Request when using IP's numeric value ("ip2long()")"
  https://redmine.lighttpd.net/issues/1330

To allow most 8-bit and 7-bit chars in headers
  server.http-parseopt-header-strict  = "disable"  (not recommended)

x-ref:
  "Russian letters not alowed?"
  https://redmine.lighttpd.net/issues/602
  "header Content-Disposition with russian '?' (CP1251, ascii code 255) causes error"
  https://redmine.lighttpd.net/issues/1016
7 years ago
Glenn Strauss b29c8efcb9 [mod_fastcgi] no chunked response w/ X-Sendfile (fixes #2733)
x-ref:
  "X-LIGHTTPD-send-file return 0 bytes"
  https://redmine.lighttpd.net/issues/2733
7 years ago
Glenn Strauss c55cf3df10 [mod_cgi,mod_scgi] X-Sendfile sets file_started (fixes #2733)
x-ref:
  "X-LIGHTTPD-send-file return 0 bytes"
  https://redmine.lighttpd.net/issues/2733
7 years ago
Glenn Strauss fde843f63e [mod_ssi] fix parse of tag across buf boundary (fixes #2732)
thx fbrosson

x-ref:
  "char copied to wrong place in SSI output"
  https://redmine.lighttpd.net/issues/2732
7 years ago
Glenn Strauss a8c51b0cbe [core] wrap IPv6 literal in "[]" in redirect URL 7 years ago
Glenn Strauss a5fcfee6fc [mod_ssi] more flexible quoting (fixes #1768)
allow double-quotes, single-quotes or no quote on SSI param values

remove use of PCRE from mod_ssi

fix misspelling of 'unknow' to be 'unknown'

x-ref:
  "mod_ssi doesn't accept single quotes"
  https://redmine.lighttpd.net/issues/1768
7 years ago
Glenn Strauss abfb9d9e2d [mod_ssi] add PCRE_* options to constrain regex 7 years ago
Glenn Strauss ede4d17913 [mod_ssi] include relative to alias,userdir (fixes #222)
adjust paths relative to changes made by mod_alias and mod_userdir

Note: this still works only for direct file inclusion.
lighttpd mod_ssi does not perform an "internal subrequest" for the
virtual path, so things like virtual include of CGI are not supported

x-ref:
  "ssi virtual include uses wrong path"
  https://redmine.lighttpd.net/issues/222
7 years ago
Glenn Strauss df146a7724 [mod_ssi] fix SSI statement parser
bug introduced in 8e3c6bf7 when statement parser was replaced
7 years ago
Glenn Strauss 45021708bf [mod_accesslog] report aborted con state with %X (fixes #1890)
(compatibility with Apache
 https://httpd.apache.org/docs/trunk/mod/mod_log_config.html)

x-ref:
  "%X in accesslog.format not working as expected"
  https://redmine.lighttpd.net/issues/1890
7 years ago
Glenn Strauss 427acbf034 [build] enhancements for cross-compiling (fixes #2276)
x-ref:
  "libpcre - specify directory"
  https://redmine.lighttpd.net/issues/2276
7 years ago
Glenn Strauss 0cd7f2dda8 [mod_evasive] 302 redirect option if limit reached (fixes #2199)
x-ref:
  "[mod_evasive] redirect if maximum connections exceeded"
  https://redmine.lighttpd.net/issues/2199
7 years ago
Glenn Strauss bd66026fc2 [config] support include file glob (fixes #1221)
x-ref:
  "support *(wild card character) within include directive"
  https://redmine.lighttpd.net/issues/1221
7 years ago
Glenn Strauss 1cd31ae2cc adjustments for openssl 1.1.0 pre-release 7 years ago
Glenn Strauss 239c7e3026 [doc] NEWS 7 years ago
Glenn Strauss a9568d0ad6 check close() return code after writing to file 7 years ago
Glenn Strauss 29e7468d46 [mod_auth] preserve WWW-Authenticate for error docs (fixes #2730)
fixes auth broken by 9973fdb0:
  [core] reset response headers, write_queue for error docs

x-ref:
  "http auth does not send WWW-Authenticate header in git rev 598cdd0"
  https://redmine.lighttpd.net/issues/2730
7 years ago
Glenn Strauss 98acff0ea0 [core] add default modules while processing server config
(instead of doing separately, before processing server config)
7 years ago
Glenn Strauss edbe157245 [core] simplify config merge of array lists 7 years ago
Glenn Strauss 969456f3fa [core] fix config merge of array lists 7 years ago
Glenn Strauss 83d896d095 static build instructions using SCons or make 7 years ago
Glenn Strauss 598cdd0e5e [core] fix IPv6 address + port parsing (#2204) 7 years ago
Glenn Strauss 1ca52fdce3 build with libressl
libressl defines SSL_OP_NO_SSLv2 and SSL_OP_NO_SSLv3 as 0x0
  (thx Christian Heckendorf)

libressl matches ERR_remove_thread_state() signature from openssl 1.0.2
  (libressl pretends that libressl is openssl version 2.0.0,
   but openssl 1.1.0 changes signature of ERR_remove_thread_state())

libressl does not yet provide compatibility interfaces for the new
  prototypes introduced in openssl 1.1.0, including
  DH_set0_pqg() and DH_set_length()

remove OPENSSL_NO_KRB5 from build config (added in 5fab991b in 2005)
  (define USE_OPENSSL_KERBEROS if required)
  (Note: OPENSSL_NO_KRB5 removed in openssl 1.1.0)
7 years ago
Glenn Strauss 873eaf3f4a minor: fix compiler warning for extra ';' 7 years ago
Glenn Strauss ccb1f02b83 clean up oneshot_fd resource upon startup error 7 years ago
Glenn Strauss c6edb24803 fix typo in new cgi.x-sendfile directives 7 years ago
Glenn Strauss 299173a3b5 reset response headers, write_queue for error docs 7 years ago
Glenn Strauss 01ec2c458c [doc] NEWS 7 years ago
Glenn Strauss 9b14e017c8 [mod_webdav] improve PROPFIND,PROPPATCH; map COPY/MOVE Destination
improve PROPFIND,PROPPATCH
map COPY/MOVE Destination to aliases

Merge branch 'bug-1787-webdav-alias-destination' into master

github: closes #61
7 years ago
Glenn Strauss 80bb42266e [mod_webdav] improve PROPFIND,PROPPATCH (#1818, #1953)
fix "allprop" propfind request to report all 'live' properties
add "supportedlock" 'live' property, if ./configure --with-webdav-locks
report collections (directory) paths with trailing slash ('/') on path
redirect operations on collections without trailing slash ('/') to URI
  with trailing slash ('/')

fix PROPPATCH to work properly and eliminate PROPPATCH memory leak
fix property update after MOVE
move CREATE TABLE statements *before* any prepare statements to avoid
  invalidating the prepare statements when the tables are first created
  **thx Uranus Zhou for the explanation:
    https://zohead.com/archives/lighty-sqlite-err/?lang=en

x-ref:
  "Improve DAV support to be able to handle git as a client"
  https://redmine.lighttpd.net/issues/1953
  "add RFC-compliant LOCK support to mod_webdav"  (still not compliant)
  https://redmine.lighttpd.net/issues/1818

Note: this has not been tested whether or not mod_webdav works with git
The (highly) recommended method to support git via HTTP is to use
git-http-backend via CGI.  gitolite and gitosis provide other good
alternative ways to access git.

This patch does result in more WebDAV 'Litmus' tests passing, even
though mod_webdav still pretends to implement "If" conditional locking,
granting locks to all requestors and not strictly enforcing locks.
7 years ago
Glenn Strauss e0115208ec [mod_webdav] map COPY/MOVE Destination to aliases (fixes #1787)
attempt to remap COPY/MOVE Destination to aliased physical paths
by finding common URI prefix between request URI and Destination
and finding how that part of the request URI was mapped to a
physical path.

This will work if the aliased physical path is above the webdav root.
It is not a good idea to remap physical paths within a webdav root.

Note: webdav paths and webdav properties are managed by mod_webdav,
      so do not modify paths externally or else undefined behavior
      or corruption may occur

x-ref:
  "Bug in mod_webdav when using aliases and MOVE command"
  https://redmine.lighttpd.net/issues/1787
7 years ago