diff options
Diffstat (limited to '')
-rw-r--r-- | doc/CHANGES.txt | 1947 | ||||
-rw-r--r-- | doc/LICENSE.txt | 202 | ||||
-rw-r--r-- | doc/MESSAGES.txt | 79 | ||||
-rw-r--r-- | doc/favicon.ico | bin | 0 -> 1406 bytes | |||
-rw-r--r-- | doc/glossary.html | 603 | ||||
-rw-r--r-- | doc/headerfilterrules.txt | 97 | ||||
-rw-r--r-- | doc/images/ClientPrefer.png | bin | 0 -> 38948 bytes | |||
-rw-r--r-- | doc/images/ClosedLoop.png | bin | 0 -> 10852 bytes | |||
-rw-r--r-- | doc/images/Events.png | bin | 0 -> 35347 bytes | |||
-rw-r--r-- | doc/images/LimitCount.png | bin | 0 -> 27334 bytes | |||
-rw-r--r-- | doc/images/LimitCountExample.png | bin | 0 -> 17097 bytes | |||
-rw-r--r-- | doc/images/QS_ClientEventBlockCount.png | bin | 0 -> 35395 bytes | |||
-rw-r--r-- | doc/images/Rule.png | bin | 0 -> 15894 bytes | |||
-rw-r--r-- | doc/images/Serialization.png | bin | 0 -> 29805 bytes | |||
-rw-r--r-- | doc/images/SrvMinDataRate.png | bin | 0 -> 28390 bytes | |||
-rw-r--r-- | doc/images/UserTracking.png | bin | 0 -> 64473 bytes | |||
-rw-r--r-- | doc/images/directive_seq.gif | bin | 0 -> 186238 bytes | |||
-rw-r--r-- | doc/images/download.jpg | bin | 0 -> 3265 bytes | |||
-rw-r--r-- | doc/images/link.png | bin | 0 -> 208 bytes | |||
-rw-r--r-- | doc/images/mod_qos.gif | bin | 0 -> 7839 bytes | |||
-rw-r--r-- | doc/images/qsloc.png | bin | 0 -> 82611 bytes | |||
-rw-r--r-- | doc/images/qsloc1.png | bin | 0 -> 17772 bytes | |||
-rw-r--r-- | doc/images/qsloc2.png | bin | 0 -> 31104 bytes | |||
-rw-r--r-- | doc/images/qsloc3.png | bin | 0 -> 34926 bytes | |||
-rw-r--r-- | doc/images/qslogFormat.png | bin | 0 -> 45252 bytes | |||
-rw-r--r-- | doc/images/qslog_spreadsheet_example.png | bin | 0 -> 224579 bytes | |||
-rw-r--r-- | doc/index.html | 2751 | ||||
-rw-r--r-- | doc/qsdt.1.html | 85 | ||||
-rw-r--r-- | doc/qsexec.1.html | 72 | ||||
-rw-r--r-- | doc/qsfilter2.1.html | 127 | ||||
-rw-r--r-- | doc/qsfilter2_process.gif | bin | 0 -> 14909 bytes | |||
-rw-r--r-- | doc/qsgeo.1.html | 77 | ||||
-rw-r--r-- | doc/qsgrep.1.html | 67 | ||||
-rw-r--r-- | doc/qshead.1.html | 54 | ||||
-rw-r--r-- | doc/qslog.1.html | 146 | ||||
-rw-r--r-- | doc/qslogger.1.html | 75 | ||||
-rw-r--r-- | doc/qspng.1.html | 58 | ||||
-rw-r--r-- | doc/qsre.1.html | 56 | ||||
-rw-r--r-- | doc/qsrespeed.1.html | 54 | ||||
-rw-r--r-- | doc/qsrotate.1.html | 86 | ||||
-rw-r--r-- | doc/qssign.1.html | 79 | ||||
-rw-r--r-- | doc/qstail.1.html | 56 |
42 files changed, 6771 insertions, 0 deletions
diff --git a/doc/CHANGES.txt b/doc/CHANGES.txt new file mode 100644 index 0000000..5fd53bc --- /dev/null +++ b/doc/CHANGES.txt @@ -0,0 +1,1947 @@ +Version 11.74 + +- Fixed: Potential counter overflow for early event detection + (increment before block) or log only mode. + +Version 11.73 + + This release introduces support of the PCRE2 (10.x) library in place of + the now end-of-life PCRE version 1 (8.x) API. + + - Removes PCRE API dependency from mod_qos.c. The module no longer has an + explicit dependency to the PCRE library but uses ap_pregcomp(), + ap_regexec(), and ap_regexec_len() from ap_regex.h. + Wrapping the PCRE (v1) and PCRE2 interface by the Apache httpd allows you + to use either the old or the new API version (depends on locating + pcre2-config). PCRE2 compatibility requires Apache httpd 2.4.53 or newer. + + - Support utilities migrated to PCRE2 API (version 10.x). + Tested with PCRE2 version 10.41. + + - Removed compatibility to Apache 2.0 and 2.2. + +Version 11.72 + + - Improve the support of Apache "event" MPM by calculating + a higher QS_MaxClients default value based on the + AsyncRequestWorkerFactor setting. + +Version 11.71 + + - Removed directive QS_Chroot. + + - Minor code changes (improvements #39/#40 reported by + Rainer Jung - many thanks). + + - QS_LogOnly also disables QS_EventKBytesPerSecLimit and + QS_LocKBytesPerSecLimit (deactivates delay output filter). + + - Uses apr_temp_dir_get() to determine temp. directory for + semaphores/shared memory (default used to be /var/tmp). + Use QS_SemMemFile to override it. + +Version 11.70 + + - QS_ClientGeoCountryPriv skips clients whose IP address can't + be mapped to a country code if the argument 'excludeUnknown' + is specified. + +Version 11.69 + + - Internal: QS_SetEnvIf directives use an array to store all + rules (to ensure they are applied in the order they appear + in the configuration file). + + - Apache 2.4.49 compatibility fix introduced by mod_qos 11.68 is no + longer applied for Apache version 2.4.50 and newer. + 'QS_ForcedClose off' could be used to enable gentle connection + close handling manually. + +Version 11.68 + + - Compatibility with Apache 2.4.49 (avoid segfault when returning + error code in pre-connection hook / issue similar to CVE-2017-3169). + +Version 11.67 + + - The QS_LogEnv directive can be used to enable environment variable + logging. mod_qos writes all environment variables which are set when + entering a handler to the log. + +Version 11.66 + + - QS_ClientIpFromHeader supports pseudo IP by creating a hash + of a HTTP request header's value if the header name is prefixed + by '#', e.g. #Authorization to use the HTTP basic auth header. + It's also possible to use the client's SSL client certificate's + subject and issuer DN if you specify #SSL_CLIENT_S_DN instead + of a real HTTP header name. + Note: Does not work for IP geolocation. + +Version 11.65 + + - Fixed: QS_SrvMinDataRate did not enforce (log only) min data rate + in simple mode (only one arg). + Improved min. data rate calculation and updated documentation. + + - Fixed: Several typos in documentation. + +Version 11.64 + + - Updated request header filter rules (allows signed HTTP exchanges content + type in Accept header). + + - qsgeo: New pattern to detect "readable" format (no longer adding IP address + range twice for some file formats). + + - QS_Status: adds the QS_AllConn variable to the maxClients object. + +Version 11.63 + + - Adds the option 'jsredirect' to the QS_UserTrackingCookieName directive: + Client (browser) has to use Javascript within the cookie check page + to fetch the cookie and to redirect the client back to the initially + requested page (adding Javascript to the cookie challenge). + SSI variables to be used in the HTML page / Javascript: + - QS_UT_QUERY: query string to call (ajax) the cookie page again to + obtain the cookie. + - QS_UT_NAME: name of the cookie + - QS_UT_INITIAL_URI: initial page to redirect to + Sample page: http://mod-qos.sourceforge.net/cookie-ir.shtml + +Version 11.62 + + - Machine-readable version of the status viewer does no longer + show QS_SrvMaxConn* counters for virtual hosts using the base + server's configuration and counter. + + - New directive QS_MaxClients. + +Version 11.61 + + - QS_SrvMaxConnPerIP can handle more than MaxClient IP addresses + (if the server opens new connections faster than closing old) + and enables memory block distribution by default. + +Version 11.60 + + - Fixed: Wrong connection counter (total server connections stored in the + QS_AllConn variable) when using SrvMaxConn* directives globally only but + other QS_* directives within virtual hosts. Bug influenced + QS_SrvMinDataRate behavior as well (activating and increasing the min. + data rate too fast). + +Version 11.59 + + - QS_EventRequestLimit writes the current counter value to the + QS_EventRequestLimit_<env-variable>_Counter environment variable. + + - New directive QS_SetEnvIfCmp. + +Version 11.58 + + - Adds directive QS_UnsetReqHeader. + + - Removes version information in automake configure script (utilities). + +Version 11.57 + + - Adds qsrespeed and qsre (previous "regexspeed" and "regex" dev tool) + to the support utilities. + +Version 11.56 + + - Non-functional: Changed QS_Status invocation timer implementation. + + - Adds qsdt (previous "duration" dev tool) to the support utilities. + +Version 11.55 + + - qslog: "-l" supports files with huge gaps (missing lines). + + - QS_SetEnvIf and QS_SetEnvIfQuery directives can be used within Location + configuration. + +Version 11.54 + + - Introduces the QS_Block_Decrement variable. + +Version 11.53 + + - Adds CORS headers to the default QS_RequestHeaderFilter and + QS_ResponseHeaderFilter rule set. + + - qslog: shows also the distribution of response durations faster than + a second within the following intervals: + * 0-49ms + * 50-99ms + * 100-499ms + * 500-999ms + +Version 11.52 + + - Fixed: QS_LimitRequestBody did not work for chunked requests + when used in Apache 2.4. + + - QS_Status uses the server's base configuration settings while logging + allowing to configure a custom log format using ErrorLogFormat. + + - qssign's verification mode supports graceful restart (sigusr1) where + two qssign processes are running (one with the old sequence + counter and the new one starting at 1). + +Version 11.51 + + - Adds process-connection handler to close the connection in the case the + abort by the pre-connect hook was ignored (workaround for bug in + Apache 2.4.28 Event MPM ignoring the connection abort) (msg id 167). + + - Uses module name in log index. + +Version 11.50 + + - Changes pre-connection hook dependency (called later but still before + mod_ssl). + +Version 11.49 + + - Adapted connection handling to deal with master/slave connections + (introduced by Apache 2.4.18) avoiding inconsistent connection counters. + +Version 11.48 + + - Detects unexpected connection dispatching to old child process to avoid + invalid connection counter state for QS_ClientPrefer rules (msg id 166). + +Version 11.47 + + - Updates built-in filter pattern of QS_HeaderFilter (Transfer-Encoding). + + - qslog standard mode supports peak/max value counter (M). + + - Fixed: potential segfault if connection ctx is null when using h2. + +Version 11.46 + + - Updates built-in filter pattern of QS_HeaderFilter. + +Version 11.45 + + - QS_ClientEventRequestLimit supports the QS_ClientIpFromHeader directive. + +Version 11.44 + + - New directive QS_CondEventLimitCount. + + - QS_EventLimitCount (and QS_CondEventLimitCount) counter may be decremented + by environment variable suffixed by "_Decrement". + + - Slightly changed unique-id generator (shorter). + +Version 11.43 + + - Fixed: QS_IsVipRequest was not set if QS_ClientPrefer hasn't been + used. + +Version 11.42c + + - qslog supports QSCOUNTERPATH (-pc mode) environment variable which + defines a file containing a list of QS_ClientEventLimitCount rules. + The 'E' format character defines the event string in the log + to match (literal string) the event1 and event2 event names against. + + Rule syntax: <name>:<event1>-<n>*<event2>/<duration>=<limit> + + Example mod_qos config: + QS_ClientEventLimitCount 20 600 QS_LimitEv + QS_SetEnvRes Event AU04 QS_LimitEv + QS_SetEnvRes Event AU05 QS_LimitEv_Decrement=2 + Sample qslog rule: + QS_LimitEv:AU04-2*AU05/600=20 + + Special us case matching against the HTTP status code ('S' character) + is used if the rule 'name' starts with STATUS. + Example mod_qos configuration: + QS_ClientEventLimitCount 10 300 QS_LimisS + QS_SetEnvIfStatus 400 QS_LimitS=1 + QS_SetEnvIfStatus 405 QS_LimitS=1 + QS_SetEnvIfStatus 406 QS_LimitS=1 + QS_SetEnvIfStatus 408 QS_LimitS=1 + QS_SetEnvIfStatus 413 QS_LimitS=1 + QS_SetEnvIfStatus 414 QS_LimitS=1 + QS_SetEnvIfStatus 500 QS_LimitS=1 + Sample qslog rule: + STATUS:400_405_406_408_413_414_500-1*X/300=10 + +Version 11.42 + + - Message mod_qos(034) indicates (by "in:0") when the server not even + received a request line on a TCP connection (potentially a speculative + TCP pre-connection). + + - qslog -pc mode supports peak/max value counter (M). + + - qsrotate supports line-by-line data processing prefixing every line + by a timestamp when using the option "-d", e.g. for batch/script + output logging. + + - qsrotate supports option "-m" to define the file permissions. + +Version 11.41 + + - Fixed: wrong default file size used by qsrotate. + + - qsrotate supports signal USR1 to perform a file rotation manually + at any time. + +Version 11.40 + + - mod_qos and all utilities are now distributed under the Apache License. + +Version 11.39 + + - New directive QS_EventCount to enable an error message event counter + (counters are shown in the machine-readable status view). + + - Adds clientContentTypes statistics to the status view. + +Version 11.38 + + - New directive QS_SrvMinDataRateIgnoreVIP and QS_SrvMaxConnPerIPIgnoreVIP. + + - QS_ClientEventLimitCount may be decremented or cleared by environment + variable (suffixed by "_Decrement", e.g. QS_Limit_Decrement). + + - QS_MileStone sets cookie also within error filter supporting redirects + even not reaching the handler. + +Version 11.37 + + - Fixed: wrong include in qspng.c + + - Console: dump shows time of last db access (seconds since epoch). + + - Adds clientContentTypes to QS_Status. + +Version 11.36 + + - Fixed: moves code from "process connection" hook to the "pre + connection" hook for Apache 2.4 compatibility and higher efficiency. + + - Lower severity of message 036 to critical. + +Version 11.35 + + - Fixed: potential segfault by mod_ssl in ssl_io_filter_disable() + when closing a connection after a failed SSL handshake (CVE-2017-3169). + +Version 11.34 + + - New variable QS_SetEnvIfResBodyIgnore which might be used to + deactivate QS_SetEnvIfResBody. + + - mod_qos_ev: sets "v" if an IP is marked as VIP. + +Version 11.33 + + - QS_ClientLowPrio variable's value contains the status flag representing + the tracked attributes. + + - Sets QS_IsVipRequest variable for marked IP addresses at connection + processing handler and propagetes it to every request. + + - Fixed: message 045 (uri check ignores log-only mode). + +Version 11.32 + + - Header filter: accepts region specific language codes in + Accept-Language request header by default. + + - Compiles also against OpenSSL 1.1.0b. + +Version 11.31 + + - QS_SetReqHeader supports header removal (unset) by prefixing + the header name with "!". + + - QS_SetEnvIfResBody supports variable removal (unset) by + prefixing the variably with "!". + + - New cookie data format (user tracking, vip, milestones). + +Version 11.30 + + - qslogger supports option "-x" (adding a prefix to every message). + + - Fixed: calculation of MaxClients for MPM prefork binary (bug reported + by Fergus - thanks!). + +Version 11.29 + + - mod_qos_ev variable sets character "u" if server is accessed by a + client without a user tracking cookie (but QS_UserTrackingCookieName + has been configured). + + - Minor (non-functional) DSCP implementation code changes (incl. new log + messages). + + - Adds mod_qos events "qA" and "qu" to qslog and the output of event + counters (q*) has been made optional. + +Version 11.28 + + - New variable "QS_Set_DSCP" to set the IP differentiated services code + points (DiffServ / RFC 2474). + + - QS_RedirectIf also supports 301. + +Version 11.26 + + - Changed: QS_SrvMaxConn* directive counter's inheritance from the base + server to virtual hosts is no longer coupled to other QS_* directive + rules. This means, that a virtual host uses the very same counters + as the base server as long as neither QS_SrvMaxConn, QS_SrvMaxConnClose, + nor QS_SrvMaxConnPerIP has been configured within the virtual host. + +Version 11.25 + + - QS_ClientEntries max. value is now limited to 10000000. + + - New "special code" 'BrokenConnection' for the QS_SetEnvIfStatus + direcive. + +Version 11.24 + + - Fixed: Apache 2.4 had nested error page when using QS_ErrorPage + (bug introduced by version 11.22). + +Version 11.23 + + - Directive QS_SetEnvIf supports single variable match. + + - qslog -p: + * fallback to simple hour/minutes detection + * Fixed: writes now statistic line at the "next" minute (m+1:00) + +Version 11.22 + + - Disables client behavior (content type) measurement if + QS_ClientPrefer has been disabled. + + - Minor changes to the status viewer. + + - Fixed: segfault when using HTTP/2. + +Version 11.21 + + - Fixed: Implemented graceful restart detection for Apache 2.4 + to properly free shared memory and mutexes (prevents from + leaking while doing graceful restarts). + + - Disables keep-alive enforcement for MPM event binaries. + +Version 11.20 + + - Adds filter (option -f) to qssign. + + - Revised QS_ClientPrefer implementation (improved attribute weighting) + and the log messages 063/064 have been merged into message 066. + +Version 11.19 + + - Updates User-Agent header field pattern for the default + QS_RequestHeaderFilterRule rule set and adds the + Upgrade-Insecure-Requests header. + + - QS_ClientGeoCountryDB is able to read IP2LOCATION csv file without + prior transformation (country code '-'). + + - Fixed: QS_ClientIpFromHeader used to determine country code + (QS_ClientGeoCountryDB) even the country could not be determined + for the connection address (incomplete IP list). + +Version 11.18 + + - New directive QS_Status. + + - QS_SrvSampleRate must be greater then one second. + + - Avoids using RAND_bytes() and checks if the generator is seeded with + enough entropy (mod_qos(08x) messages). + + - Calculates MaxClients (instead of reading the directive). + +Version 11.17 + + - QS_SetEnvIfStatus supports QS_SrvMaxConnPerIP to increment the + QS_Block event variable. + + - qssign features the option "-a" to define which algorithm to use. + + - Enables QS_SrvMinDataRate, QS_SrvDataRateOff, and + QS_SrvMinDataRateOffEvent for Apache 2.4 as smoke test against + Apache 2.4.12 and 2.4.16 was now successful (MPM worker and event). + The QS_KeepAliveTimeout and QS_MaxKeepAliveRequests directives have + been disabled for the MPM event. Apache 2.4 is still not fully tested. + +Version 11.16 + + - Adds timeout option to the QS_SrvSerialize directive. + +Version 11.15 + + - Increased severity of message 035 and 036 from critical to alert. + + - Lowered severity of messages 100, 009, and 007 from emergency to + critical. + + - QS_RedirectIf directive supports 307 response code. + + - Set *_Counter variable of QS_EventLimitCount within post read request + handler. + + - QS_ClientSerialize honors the sequence of receiving requests (fifo) in + order to support requests from different clients accessing the server + via a proxy. + + - Fixed: potential deadlock (when reaching the 5min timeout) when using + the QS_SrvSerialize directive. + +Version 11.14 + + - New directive QS_SrvSerialize. + +Version 11.13 + + - Adds option "-u" to the qslogger and qssign utilities. + + - QS_MileStone features a "think time" attribute which defines how long + a client must wait between two milestones. + +Version 11.12 + + - Propagates the variables used by QS_ClientEventLimitCount to sub-requests + making them available within SSI pages. + + - New QS_ClientEventLimitCount variable suffixed by "_Remaining" containing + the remaining time in seconds a client is still blocked (to be used + within error pages to show a client how long he has to wait until he + might proceed). + + - New variable QS_MaxKeepAliveRequests. + +Version 11.11 + + - QS_Limit (resp. the event variable processed by the + QS_ClientEventLimitCount/QS_CondClientEventLimitCount directives) + event may specify a weighting of events defining by how many penalty + points the counter shall be increased. Default is 1. + +Version 11.10 + + - Adds qslog command to the module (dedicated statisic log facility on + a per Apache server instance basis). + +Version 11.9 + + - QS_Block event may specify a weighting of events defining by how many + penalty points the counter shall be increased. + + - Fixed: enables per client data store when using the QS_VipIPHeaderName + together with the QS_SrvMaxConn[PerIP] directive. + +Version 11.8 + + - New console command 'inclimit' increments the + QS_ClientEventLimitCount rule counter. + + - Adds the option "<domain>" to the QS_UserTrackingCookieName + directive. + +Version 11.7 + + - Man page for the module itself. + + - Adds option "session" to QS_UserTrackingCookieName. + +Version 11.6 + + - Adds "Public-Key-Pins" and "Public-Key-Pins-Report-Only" to the + pre-defined list of allowed HTTP response header fields. + + - Adds "Origin" to the pre-defined list of allowed HTTP response header + fields. + + - Fixed: qsrotate reads the size of an existing logfile at startup + (required by the "-b" option). + +Version 11.5 + + - qslog: improved performance. + + - Minor code refactoring. + + - Some log messages (010, 012, 013, 030, 031, 034, 040, 041, 042, 043, + 044, 046, 047, 048, 060, 063, 064, 065, 067, 101) indicate if + QS_LogOnly mode is active. + + - Apply QS_LogOnly to header filter (action drop). + + - qsgeo option "-l": + * adds the IP address if missing + * fixed: unintentional dropping of valid lines + * may be used to normalize "ip2location lite" DB1 files + * option "-v" to print all error messages + +Version 11.4 + + - Adds request ID to console log messages (07*). + + - qslog supports writing to stdout (if "-o <out_file>" is omitted). + + - qslog: improved performance. + +Version 11.3 + + - New directive QS_ClientEventBlockExcludeIP. + + - Minor changes to the status viewer. + + - Modified error messages 060 and 067 (adding the "age" parameter which + indicates the seconds since the event occurred the first time). + + - Fixed: Message 065 contained wrong directive name. + +Version 11.2 + + - Adds variable QS_ResponseDelayTime showing the delay time (us) + calculated for response throttling. + + - New variable QS_Timeout. + +Version 11.1 + + - Fixed: Shows "T" log marker only for requests which has really been + delayed by mod_qos. + + - Further improved bytes/sec limitation implementation. + +Version 11.0 + + - Highly improves bytes/sec limitation (response throttling) based on + the input I got from Jeff Trawick - many thanks! + * Calculates delay within filter (immediately(!) when reaching the + defined amount of bytes). + * Uses nanoseconds delay (instead of milliseconds). + * Splits large bucket brigades to 8k blocks (support for local files, + not using mod_proxy). + * Inserts filter late (after mod_deflate). + + - User tracking: set Cache-Control header when accessing the cookie + check page. + + - QS_UserTrackingCookieName: improved cookie header processing. + + - Fixed: 'qslog -pc' does no longer require 'S' nor a date. + +Version 10.30 + + - QS_SetReqHeader features the option 'late'. + + - New console output (without ':' suffix for the IP address). + + - Console 'search', 'limit', and 'unlimit' command support now the + 'event' parameter specifying which QS_ClientEventLimitCount event + variable to show/set/clear. + +Version 10.29 + + - Supports IPv6 clients. + +Version 10.28 + + - Fixed: QS_ClientEventLimit did overwrite counters of other clients if + multiple events have been configured. + +Version 10.27 + + - qslog features the option "-pu" and "-puc" used to gather request + information on a per URL basis. + + - Fixed: Wrong includes within the support utilities. + + - Extends QS_ClientSerialize max. timeout from 1 to 5 minutes. + +Version 10.26 + + - QS_ClientSerialize supports the QS_ClientIpFromHeader directive. + + - Refactor method used to determine redirect port (user tracking) + supporting servers not using virtual hosts. + + - Fixed: QS_UserTrackingCookieName uses correct server_rec to retrieve + configuration. + + - Hook implementing user tracking is now called after mod_unique_id. + + - Slightly changed unique-id generator. + + - Adds fflush() to qsgrep utility when writing data to stdout. + +Version 10.25 + + - QS_EventLimitCount writes the current value to the process environment + variables. + + - Fixed: QS_[Cond]ClientEventLimitCount logs request id and propagtes + message code (067) to the QS_ErrorNotes variable. + + - New variable QS_IPConn representing the number of connections opened + from the very same source IP (works in conjunction with + QS_SrvMaxConnPerIP only). + +Version 10.24 + + - New directive QS_CondClientEventLimitCount. + + - QS_SrvMinDataRate: limits the max. data rate to the configured value + (prevents invalid rate due to misconfiguration server or died child + process). + +Version 10.23 + + - Fixed: QS_ClientEventLimitCount log message 067 contains now the IP + address of the request header if QS_ClientIpFromHeader is used. + + - QS_SetEnvRes: supports multiple variables with the same name. + +Version 10.22 + + - Process QS_SetEnvResHeader(Match) and QS_SetEnvRes at error filter too. + +Version 10.21 + + - Fixed: qslogger may had detected the wrong message severity. + + - Adds debug message when detecting "NullConnection" events. + + - Built-in request header rules: adapt If-Match, If-None-Match, Cookie, + and Cookie2 HTTP header patterns. + +Version 10.20 + + - Fixed: QS_CondLocRequestLimitMatch did work only if other QS_Loc* + directive had been configured. + +Version 10.19 + + - New directive QS_RedirectIf. + +Version 10.18 + + - QS_ClientEventLimitCount may be cleared by environment variable + (suffixed by "_Clear", e.g. QS_Limit_Clear). + +Version 10.17 + + - QS_ClientEventLimitCount supports unlimited number of events. + + - Stores the value of the QS_ClientEventLimitCount variables as + environment variables suffixed by "_Counter", e.g. QS_Limit_Counter + for the default QS_Limit variable, in order to be processed by other + rules. + + - Add Content-Security-Policy to the default response header allow list. + + - qslog features enhanced "-pc" mode providing more information: + * Collects content type information (%{content-type}o). + * Duration between the first and the last request. + * Average response in ms. + * "ci" indicates if we have seen the client at the end or the + beginning of the file (maybe not all requests in the log due to + log rotation). + * Bytes downloaded. + * Writes status characters to stderr. + * HTTP request methods (GET/POST) + +- qsgeo features option "-l" and is able to process "qslog -pc" files. + +Version 10.16 + + - qslog adds 'E' (event identifiers) to the format string. QSEVENTPATH + environment variable specifies a file containing all known event + names (comma separated list). + + - qslog average counter (a/A) count only if a numeric value is available. + + - qssing does not try to execute invalid program name (space only). + +Version 10.15 + + - qsrotate supports DST and offset to UTC. + + - Add the "connections" argument to the QS_SrvMaxConnPerIP directive + to disable the rule enforcement on idle web servers. + +Version 10.14 + + - Minor changes to status viewer (color for QS_EventLimitCount counter). + + - Q3594444: adapted man page subject. + + - QS_ErrorResponseCode verifies that the defined error code is valid + resp. known by Apache. + + - Add option "-b" to the qsrotate utility. + +Version 10.13 + + - Add new directive QS_EventLimitCount. + +Version 10.12 + + - Fixed: Per-client status viewer did not show numbers correctly + (depending on the platform it has been compiled for). + +Version 10.11 + + - Don't write QS_ClientEventBlockCount event messages (060) every time + a client is blocked. + + - Adjust log message severity of permitted QS_SrvMinDataRate rule + violations from 'info' to 'debug'. + +Version 10.10 + + - Add DNT HTTP request header to the default request header allow list. + + - qslog "-pc" supports counting established connections. + + - Fixed: Endless loop when using option "-c" with only one rule. + + - New utility qshead. + +Version 10.9 + + - Q3535677: Don't use prce_info() any longer. + + - qslog option "-x" allows the specification how many files to keep. + Default are 14 days. + + - qslog counter 'a', 'A', and 's'. + + - Adapted log message mod_qos(069) + + - QS_ClientIpFromHeader@logger searches for the header in r->prev and + r->main too. + +Version 10.8 + + - Fixed: QS_SetEnvIfResBody did not properly detect pattern. + + - qslogger features severity filter (forward only messages with a + matching/higher severity) and adjustable default severity for those + log lines which do not contain the severity pattern. + +Version 10.7 + + - Writes notice message at server startup if the Apache version is not + supported (mod_qos has been implemented for Apache 2.2 worker + binaries only resp. Apache 2.0 is no longer supported). + + - Use pcre_study() API call only if QOS_EXTRA_USE_PCRE_STUDY has been + defined while compiling mod_qos. + + - Adds fflush() to qslogger/qsexec/qsgeo/qslogger utility when writing + data to stdout. + +Version 10.6 + + - qslog measures average response time in milliseconds (avms). + + - Fixed: Viewer shows number of per client ip connections if no server + limitations are set (query "option=ip"). + + - Fixed: qslogger did not compile on non-Linux platforms. + +Version 10.5 + + - New utility: qslogger. + + - JSON includes array index number (note: you need to adapt existing + JSON rules). + + - Experimental: mod_qos compiles with Apache 2.4 + * QS_SrvMinDataRate is not available (does not work, use mod_reqtimeout + instead) + * QS_Srv* directives shall not be used (connection cleanup takes + very long) + +Version 10.4 + + - Improved qs* utility performance. + +Version 10.3 + + - Fixed: ABR in QS_SetEnvIfResBody. + +Version 10.2 + + - Fixed: QS_Milestone uses now URL decoding before applying the + expression (pcre). + + - Add the qsgeo utility to the distribution archive file. + + - Fixed: Suppress warning message about missing mod_unique_id if + mod_navajo.cpp is available. + + - New connection correlation id QS_ConnectionId (available as + an event for logging purposes). + +Version 10.1 + + - QS_ClientIpFromHeader may be used to set QS_Country variable. + + - Viewer shows QS_AllConn variable. + +Version 10.0 + + - New directives QS_ClientGeoCountryDB and QS_ClientGeoCountryPriv. + + - New variables: QS_AllConn and QS_Country. + +Version 9.79 + + - Fixed: Wrong IP conversion (str2long) used by console and + QS_ClientIpFromHeader. + +Version 9.78 + + - Fixed: QS_UserTrackingCookieName enforcement did not work if server + creates internal redirect. + +Version 9.77 + + - Use pcre_study() and match_limit where applicable. + + - qslog features the option "-c" to collect separate statistics, + e.g., for different URLs. + + - qslog features the option "-pc" used to gather request information + per client. + + - New directive QS_SrvSampleRate (may be used to adjust the + QS_REQ_RATE_TM sample rate at runtime/post compilation). Not + documented. + + - Fixed: qslog line parsing bug (double backslash). + +Version 9.76 + + - New directive QS_ClientIpFromHeader (may be used in conjunction with + QS_ClientEventLimitCount only). + + - qslog measures new connections per minute (%k == 0). + + - Fixed: Don't show connections in the overview if not measured. + + - Internal: QS_EventRequestLimit are added (instead of set) to the event + table in order to prevent multiple increments by the very same request. + +Version 9.75 + + - New directive QS_SetEnvRes. + + - Viewer keeps value about the last measured kbytes/second result for + a longer time. + + - Update documentation (description of QS_LocKBytesPerSecLimit* + directives). + +Version 9.74 + + - Fixed header file in qsfilter2 (possible compile problems). + + - Fixed pre connection handling for outgoing (mod_proxy) connections. + +Version 9.73 + + - Q3429879: Format usage text of the mod_qos utilities to man page + format. Use "<utility> --man" to generate the man page. + + - Make "NullConnection" detection (known by QS_SetEnvIfStatus) more + aggressive. + +Version 9.72 + + - Module tries to detect a suitable default error document for + QS_ErrorPage automatically. + + - New status "NullConnection" known by QS_SetEnvIfStatus detecting + TCP connections which are not used to send a HTTP request (closed + without transmitting HTTP request line and header or denied by any + other module). + + - QS_ClientEventBlockCount is processed at pre_connection hook (more + aggressive, before mod_ssl). + + - Suppress warning message about missing mod_unique_id if mod_navajo is + available. + +Version 9.71 + + - QS_RequestHeaderFilterRule and QS_ResponseHeaderFilterRule may be + configured within a host (outside location). + + - QS_ResponseHeaderFilterRule features the action "silent" which drops + header silently without writing a log message. + + - Headers X-Content-Type-Options and X-XSS-Protection has been added to + the default response header rules. + + - Fixed: Bug in JSON parser. + + - Fixed: Propagation of Apache environment variables to sub-requests + (solves bug when using QS_ClientEventBlockCount and ErrorDocument). + +Version 9.70 + + - QS_EventPerSecLimit and QS_EventKBytesPerSecLimit counters are no + longer updated if a request has already been denied by a + QS_EventRequestLimit rule. + + - QS_LocRequestPerSecLimit* and QS_LocKBytesPerSecLimit* counter are + no longer updated if a request has already been denied by a + QS_LocRequestLimit* rule. + + - Adjust attributes/number of requests required to identify the client + behavior. + + - Update request header allow list rule for Content-Type. + +Version 9.69 + + - Client behavior (content type a client is downloading) is calculated + in a percent of the whole traffic type distribution. The directive + QS_ClientTolerance supports only values between 5 and 80. + + - Add directive QS_ClientContentTypes to define the normally downloaded + content types statically (instead of self learning). + + - Detection if module has been build for a different MPM implementation + than the server is using at runtime. + + - JSON parser processes request query (if starting with an array '[' of + object '{') if no body is available. + + - qssing supports additional log format detection. + + - qslog supports request time duration measurement in milli- and + microseconds too (t and D instead of T). + + - qslog isolates numeric values (B, i, T, t, D, S) even they are + surrounded or prefixed by other characters, e.g. time="<number>". + + - qslog treats single quoted string with (short) leading name and eaual + sign (e.g., agent='Mozilla 1') as single element (offline mode only). + + - qslog extracts additional time formats (offline mode). + + - Added "X-Do-Not-Track" to the built-in request header allow list. + + - Minor changes within the status viewer (machine-readable view). + +Version 9.68 + + - Change in order to support HP-UX. + +Version 9.67 + + - Fixed: QS_ClientSerialize has required other client level control + directive. + +Version 9.66 + + - Client data store updates entry time stamp every access. + +Version 9.65 + + - Fixed: Could not compile the support utility qscheck. + + - qsexec features option "-c" (pattern clearing the event counter). + +Version 9.64 + + - New utility: qsexec + + - Dynamic client data store partition (depending on the size of the + store as defined by QS_ClientEntries) for improved performance. + +Version 9.62 + + - Some code refactoring (performance improvements, no functional + changes). + +Version 9.61 + + - New directive QS_LogOnly may be used to disable rule enforcement + (permissive mode). + + - Minor changes within the status viewer. + + - "QS_SetEnvIfStatus QS_SrvMinDataRate QS_Block" limits the allowed + number of QS_SrvMinDataRate rule violations. + +Version 9.60 + + - Fixed: QS_ClientEventBlockCount/QS_ClientEventLimitCount get not reset + if client causes events continuously. + +Version 9.58 + + - Fixed: IP does not get marked as VIP if QS_ClientPrefer has not been + defined. + + - New variable QS_ErrorNotes. + + - Add "Transfer-Encoding" (very strict) to the built-in request header + allow list. + +Version 9.57 + + - Status viewer features query name "refresh" which causes the browser + to reload the page every 10 seconds. + +Version 9.56 + + - Clear per client data store counters at graceful restart to prevent + dead enties (counter grow) due unclear client shutdown. + + - qsfilter2 features url filter (-f). + + - QS_ClientSerialize does not block for more than 10 minutes. + +Version 9.55 + + - Minor changes in configure script (autotools) of the support utilities + (png library name). + + - Add allowed response header X-Content-Security-Policy. + + - Fixed: qslog cuts last character if parameter is at end of line. + + - Fixed: qsfilter2 handling of 0 byte characters. + +Version 9.54 + + - QS_SetEnvIf may unset a variable. + + - New variable QS_IsVipRequest. + +Version 9.53 + + - Re-introduce qscheck to the support utilities tarball. + +Version 9.52 + + - Double per client data store speed (insert new entries) by partitioning + of odd and even ip addresses. + + - Overview section in qos viewer (showing connections and load). + + - Remove packet-rate measurement. + +Version 9.51 + + - Set IP based VIP status to connection even before we receive the + HTTP request. + + - New argument "connections" for the QS_SrvMinDataRate directive allows + to disable the limitation if the server is idle/has only little + traffic. + + - Adapt built-in request header filter rules. + +Version 9.49 + + - Adapt built-in request header filter rules. + + - New utility: qsgrep. + + - Change process order: process QS_SetEnvResHeader after + QS_SetEnvResHeaderMatch. + + - New directive QS_UnsetResHeader. + + - New directive QS_ClientEventLimitCount (works similar as + QS_ClientEventBlockCount but enforces rule at request level only). + +Version 9.48 + + - qslog supports mod_logio (%I and %O). + + - Re-introduce deprecated QS_SetEnvStatus directive (for backwards + compatibility). + +Version 9.47 + + - QS_SetEnvIfStatus may be used within Locations. + + - Sequence: execute QS_SetEnvIfStatus earlier (before + QS_SetEnvResHeader). + + - Remove directive QS_SetEnvStatus (alias for QS_SetEnvIfStatus). + +Version 9.46 + + - QS_VipUser/QS_VipIpUser detects r->user earlier (@fixup). + + - QS_KeepAliveTimeout allows value "0" disabling keep-alive. + + - Process QS_KeepAliveTimeout variable at response too. + + - QS_SetEnvIfStatus may be specified multiple times for the same + response code. + + - QS_SetEnvIfStatus accepts the definition of a variable value. + +Version 9.45 + + - Add directive QS_ClientSerialize. + + - qslog used new parameter names for event message counts. + +Version 9.44 + + - Add directive QS_DisableHandler. + +Version 9.43 + + - QS_ClientEventBlockCount rule violation marks client to have low + priority. + +Version 9.42 + + - Console "action=search&address=*" returns a list of all clients. + + - Fixed: Removes the apr_shm_destroy() calls to avoid double-free + errors on Linux with old APR library versions. + +Version 9.41 + + - Fixed: Console action 'block' did not set event number. + +Version 9.40 + + - Fixed: Search IP in console + + - Fixed: User tracking set-cookie is set twice. + + - Process QS_SetEnvIfStatus on internal errors (protocol). + +Version 9.38 + + - Web console allows the modification of attributes of entries within + the client data store. + + - Status viewer supports query "ip" (showing the IP addresses of the + connected clients for all open TCP connections) in machine-readable + version. + + - Status viewer used new delimiter within rule names on machine-readable + version (query "auto"). + +Version 9.37 + + - Changed QS_ClientPrefer behavior: + - never block VIP IP + - step 1 denies slow marked clients only + + - Set the QS_ClientLowPrio variable for clients with low priority. + + - qssign: add option "-e" which ensures we don't lost any lines. + + - Update built-in header validation pattern. + +Version 9.36 + + - QS_SrvMinDataRateOffEvent processing at fixup (request). + + - Use apr_time_t instead of time_t. + +Version 9.34 + + - qslog counts response status codes per minute. + + - Use apr_time_t instead of time_t. + +Version 9.33 + + - User tracking cookie enforcement may be disabled by setting the + DISABLE_UTC_ENFORCEMENT environment variable, e.g. for certain + User-Agent headers. + +Version 9.32 + + - Status viewer returns "text/plain" for request query 'auto'. + +Version 9.31 + + - qsfilter2: encode double quotes and backslashes using their hex values + (no escaping within Apache configuration necessary). + + - Featuring JSON parser which may be used in conjunction with + QS_PermitUri. + +Version 9.30 + + - Fixed: qsfilter2 did not compile with OpenSSL 1.0.0. + +Version 9.29 + + - Add Strict-Transport-Security to the default response header rules. + + - Directive QS_UserTrackingCookieName features an optional "path" + attribute. This path specifies a local error page which is shown + to users not accepting the user tracking cookie (note: search engines + do probably not support this cookie enforcement and won't be able to + crawl the site). + + - Generates a simple request id (unique per pid/tid within a + millisecond) if mod_unique_id has not been loaded. + + - Fix: syntax check for QS_ErrorPage. + +Version 9.28 + + - QS_ErrorPage supports external HTTP redirect (302). + + - qsfilter2 features a rule id prefix (-k <prefix>). + + - qsfilter2 may process audit log using the sample log format + "%h %>s %{qos-loc}n %{qos-path}n%{qos-query}n" without pre-processing. + +Version 9.27 + + - Remove qscheck utility (don't compile it by default). + + - New variable %{qos-loc}n indicating the Location matching a request + (may be used to filter the audit log for dedicated locations in order + to generate QS_PermitUri rules). + + - qsfilter2 may process "standard" Apache access log (TransferLog) + files too (automatically detecting the request line). + + - Several adaptions/fixes to the machine-readable version of the status + viewer. + +Version 9.26 + + - Fix: no mutex destroy (called by register cleanup when destroying + pools). Should fix the restart issues with MPM prefork binaries. + + - Renew user tracking cookie once every month. + +Version 9.25 + + - Compile utilities using GNU autotools (hope this works at least on + some Linux platforms). + +Version 9.24 + + - QS_SrvMinConnPerIP: don't log every rule violation (consolidate log + messages and log only every 20th event, see QS_LOG_REPEAT). + + - Fixed: Removes thread_join for MPM prefork binaries. + +Version 9.23 + + - New directives: QS_MileStone*. + + - Q3032708: see http://www.openssl.org/support/faq.html#LEGAL2. + + - Add Access-Control-Allow-Origin to the default response header rules. + +Version 9.22 + + - New variable: QS_SrvConn + + - qslog shows total number of requests within a minute. + +Version 9.21 + + - New directive QS_UserTrackingCookieName. + +Version 9.20 + + - Fixed: Racing condition when using QS_SrvMinDataRate and + ThreadsPerChild > 64 may cause segfault. + +Version 9.19 + + - Fixed: Segfault at server start if no vhost has been defined. + + - QS_SrvMinDataRateOffEvent may be used at server and/or location level. + +Version 9.18 + + - QS_SrvMaxConnClose supports the definition of the number of + keep-alive connections as a percentage of MaxClients. + + - Updates built-in filter pattern of QS_HeaderFilter. + +Version 9.17 + + - Output filters are executed after mod_setenvifplus. + +Version 9.16 + + - New directive QS_SrvMinDataRateOffEvent. + + - Changes directive process order (QS_SetEnvIfStatus). + + - QS_SrvMinDataRate enforces keep-alive timeout (request line must be + received within the keep-alive timeout). + +Version 9.15 + + - New directives QS_ResponseHeaderFilter and QS_ResponseHeaderFilterRule. + +Version 9.14 + + - New directive QS_Decoding. + +Version 9.12 + + - New directive QS_SemMemFile. + + - Uses a checksum to represent IPV6 addresses. + +Version 9.10 + + - Fixed: ap_remove_input_filter(). + + - MaxClients overrides ServerLimit/Treads settings when calculating the + maximum number of possible client connections. + + - Log/debug message about used semaphore files. + +Version 9.9 + + - New implementation of the code for QS_SrvMaxConnPerIP to avoid + malfunction reported by mod_qos user. + + - Module dependency (execution order) to mod_setenvifplus. + +Version 9.8 + + - Internal code changes/maintenance (join thread). + +Version 9.7 + + - mod_qos may be compiled defining QS_NO_STATUS_HOOK which prevents + mod_qos from registering to mod_status. + +Version 9.6 + + - Environment variable QS_DeflateReqBody to deflate request body data + (update to mod_parp 0.8 in order to get a correct content-length + header after data deflating). + +Version 9.5 + + - New directives QS_SetReqHeader and QS_SetEnv. + +Version 9.4 + + - Fixed: Variable %{qos-query} is not set when using the + QS_DenyQueryBody directive (and neither QS_DenyBody nor + QS_PermitUriBody has been set). + + - Increased line buffer for qsfilter2 (2MB). + +Version 9.3 + + - New directive QS_SetEnvResBody. + +Version 9.2 + + - New syntax: + QS_VipHeaderName <header name>[=<regex>] [drop] + QS_VipIPHeaderName <header name>[=<regex>] [drop] + +Version 9.1 + + - QS_ClientEventRequestLimit limits the number of concurrent events on + a per client IP address basis (again increasing the per client memory + consumption). + +Version 9.0 + + - Client level control: request characteristics measuring adds content + type ration and number of 304 responses (requires now 64bytes instead + of 48bytes per client on a 32bit system). + + - Improved client level control (behavior detection, see above) is + processed by the QS_ClientPrefer directive. Directive + QS_ClientTolerance controls the allowed variation. + + - Directive QS_SrvPreferNet has been removed. It's recommended to use + QS_ClientPrefer instead. + +Version 8.18 + + - Q2841328: remove nasty pointer address cast to int. + +Version 8.16 + + - Q2834297: use a single mutex for all per virtual host ACT tables (too + many mutexes if a server uses many virtual hosts). + +Version 8.15 + + - New variable QS_Delay. + +Version 8.14 + + - New directive QS_SrvDataRateOff. + +Version 8.13 + + - New directives QS_DenyQueryBody and QS_PermitUriBody obsolete + QS_DenyBody. + + - Fixed: QS_Deny*/QS_Permit* directives can handle strings containing + 0 bytes (qsfilter2 still can't). + +Version 8.12 + + - New directive QS_InvalidUrlEncoding. + +Version 8.11 + + - Fixed: Change Apache 2.0 ifdef statements in order to compile with + any compiler. + +Version 8.10 + + - Fixed: Did not compile with Apache 2.0. + +Version 8.9 + + - QS_LimitRequestBody may be defined using mod_setenvif. + See new directive order in mod_qos_seq.gif + + - mod_qos uses anonymous shm by default. + + - Use constant semaphore/shared memory file names in order to reuse + resources after unclear server shutdown. + +Version 8.5 + + - New directive QS_EventKBytesPerSecLimit. + + - New structure of the source archive tarball, see index.html#build + for more information about building the binaries. + +Version 8.3 + + - QS_RequestHeaderFilterRule has new syntax. + + - QS_RequestHeaderFilter checks the header length too. It's possible + to use "QS_RequestHeaderFilter size" for header length checking only + (instead of using LimitRequestFieldsize). + +Version 8.2 + + - Fixed: Client prefer, don't mark connection timeout at keep alive + end (used in conjunction with QS_ClientPrefer). + + - Access log events (mod_qos_ev, mod_qos_cr, mod_qos_con) are stored as + variables (storing them in the out headers will be removed in one of + the next release). + +Version 8.1 + + - Fixed: Checks for enabled cc in input filter. + + - Don't allow requests without an URL. + +Version 8.0 + + - New server configuration merger: settings within virtual hosts are + merged with the settings from the base server (directives outside + virtual hosts). Virtual host settings do not overwrite base settings + any more. + + - New directive QS_LimitRequestBody. + +Version 7.20 + + - Fixed: Url decoding detecting %HH encoding (full range). + +Version 7.19 + + - QS_DenyEvent may be used to block requests which do NOT have the + specified event set. + + - QS_DenyEvent is applied after the QS_SetEnvIf* directives. + See mod_qos_seq.gif for more details. + +Version 7.18 + + - QS_Deny/Permit logs on severity warning if action is log only. + +Version 7.17 + + - QS_SetEnvIfBody recognizes the occurrence of $1 within the variable + value and replaces it by the subexpressions of the defined regex + pattern. + +Version 7.16 + + - Set audit log variables at header parser hook. + +Version 7.15 + + - Directive QS_EventRequestLimit may match variable values too. + + - New directive QS_SetEnvIfBody. + + - Audit log is enabled based on the defined log format variables. + +Version 7.14 + + - New directive QS_DenyBody implements generic request body filter + which can be used in conjunction with QS_DenyQuery, QS_PermitUri, + and body data audit log (to be processed my qsfilter2). + +Version 7.13 + + - Changed directive processing order, see mod_qos_seq.gif. + + - New directive QS_SetEnvIfParp (requires mod_parp, + see http://parp.sourceforge.net). + + Important: + mod_parp and the QS_SetEnvIfParp directive copies the whole HTTP + request message body into the servers memory (requires at least + twice the memory size of the posted data). It is very important + that you limit the messagy body size for requests processed my + mod_parp/QS_SetEnvIfParp using the Apache directive LimitRequestBody. + + - New directive QS_DenyEvent. + + - Chuck out mod_qos_control. + +Version 7.12 + + - Process event filter only if some rules have been defined. + + - Recovery rate (decrease limitation) for bandwidth and and request + limit has been increased from 16% to 25%. + +Version 7.11 + + - New directive QS_EventRequestLimit. + +Version 7.9 + + - Fixed: QS_SrvMinDataRate/QS_SrvRequestRate counts all server + connections (not only per child process). + +Version 7.8 + + - Directive QS_SrvMinDataRate/QS_SrvRequestRate supports min/max + limitation in order to increase the minimum upload/download bandwidth + on multiple simultaneously connections. + + - Fixed: Activation of QS_SrvMinDataRate did not work + (QS_SrvRequestRate only). + +Version 7.7 + + - New directive QS_SetEnvIfQuery. + +Version 7.6 + + - Use the HTTP response code defined by QS_ErrorResponseCode (default + is 500) settings for all denied requests expect for those requests + rejected to a QS_Deny*, QS_Permit*, or QS_RequestHeaderFilter rule. + +Version 7.5 + + - New directive QS_ErrorResponseCode + + - Multiple directives (QS_LocRequestLimit, QS_LocRequestLimitMatch, + QS_CondLocRequestLimitMatch, QS_ClientEventBlockCount, and + QS_ClientEventPerSecLimit) allow now a limitation set to "0". + + - QS_SrvMinDataRate replaces QS_SrvRequestRate. + +Version 7.4 + + - QS_SrvRequestRate supports chunked POST. + +Version 7.3 + + - Partial (not for chunked post) fixed error message for slow server + response when using QS_SrvRequestRate. + +Version 7.2 + + - New directive QS_SetEnvResHeaderMatch + +Version 7.1 + + - QS_SrvMaxConnExcludeIP works for QS_SrvRequestRate (may be used to + allow selected IP sources, e.g. slow spider). + +Version 7.0 + + - New directive QS_SrvRequestRate enforces minimum upload bandwidth + (used for TCP DoS prevention). Requires thread support. + + - QS_ClientPrefer allows definition of free connections in percent + in order to override the default of 80%. Available for Apache 2.2 + only. + + - QS_SrvConnTimeout is no longer available. + You may use QS_SrvRequestRate instead. + +Version 6.7 + + - Detects low priority clients (clients sending slow or using small + data packets get marked as low priority clients). + + - New directives QS_VipUser and QS_VipIpUser. + + - Status viewer shows information about client (IP) control status. + +Version 6.6 + + - mod_status handler hook supports short status flag. + +Version 6.5 + + - New directive QS_SetEnvResHeader. + + - mod_qos_control supports QS_SetEnvIf, QS_SetEnvStatus, and + QS_SetEnvIf directive editing. + +Version 6.4 + + - New directive QS_SetEnvStatus. + + - QS_SetEnvIf for response processing (log transaction). + + - QS_ClientEventBlockCount on response events (log transaction). + +Version 6.3 + + - New directive QS_VipIPHeaderName to mark clients (IP) without + providing them full VIP privileges. + + - Add details to log messages. + +Version 6.2 + + - New command: QS_ClientEventPerSecLimit. + +Version 6.1 + + - QS_SetEnvIf supports "NOT" operator. + + - Sets QS_VipRequest variable when receiving valid session cookie. + +Version 6.0 + + - mod_qos features per client (IP) control rules. + + - QS_ClientPrefer, prefers known VIP clients. + + - QS_ClientEventBlockCount, blocks clients on events. + +Version 5.17 + + - New directive QS_EventPerSecLimit allows req/sec limitation for + requests causing an event. + + - New directive QS_SetEnvIf allows combination of multiple environment + variables. + + - Fixed: sem/shm leak when using QS_SrvPreferNet. + +Version 5.16 + + - Mark QS_CondLocRequestLimitMatch in status viewer. + +Version 5.15 + + - New directive QS_CondLocRequestLimitMatch allows conditional request + level rules. + +Version 5.14 + + - Remove "nicetitles" from status viewer. + +Version 5.13 + + - Again, minor status viewer changes. + +Version 5.12 + + - Status viewer uses "nicetitles" to show long rule strings. + +Version 5.11 + + - Minor internal code changes. + +Version 5.10 + + - Rules do not use individual mutex any longer. This allows an unlimited + number of rules. + +Version 5.9 + + - mod_qos_control features additional qsfilter2 settings. + +Version 5.8 + + - Minor improvements in status viewer. + + - 5.7 did not compile with Apache 2.0 (ap_regex). + +Version 5.7 + + - Important: + QS_PermitUri, QS_Deny*, qsfilter2 apply filter rules against unescaped + URLs where %<hex>, \x<hex> and + (new!) is unescaped. + You should regenerate your QS_PermitUri rules using the updated + version of the qsfilter2 tool provided by this release. + + - Very first release of mod_qos_control. + +Version 5.6 + + - New status viewer implementation. + +Version 5.4 + + - Important: + QS_PermitUri, QS_Deny*, qsfilter2 apply filter rules against + unescaped URLs where %<hex> and \x<hex> (new!) is unescaped. + You should regenerate your QS_PermitUri rules using the updated + version of the qsfilter2 tool provided by this release. + +Version 5.2 + + - QS_VipHeaderName creates session cookie only once. + + - VIP has no QS_LocKBytesPerSecLimit/QS_LocKBytesPerSecLimitMatch + restrictions. + + - QS_SrvPreferNet triggers for VIP user on response header only. + +Version 5.1 + + - New directive QS_SrvPreferNet. + +Version 4.30 + + - Fixed: Segfault at server startup when no virtual host has been + configured. + +Version 4.29 + + - Debug log level lists available request header filter rules. + +Version 4.28 + + - Introduce request header filter. + +Version 4.18 + + - Introduce log message numbers and SSI support for error pages. + + - Add new directive QS_DenyInheritanceOff + + - Add qsfilter2, a tool to generate request URI allow list rules. + + - Use mod_unique_id to tag error messages. + +Version 4.13 + + - QS_PermitUri uses case sensitive pcre. + +Version 4.11 + + - Add new directive QS_PermitUri. + +Version 4.8 + + - Introduce generic request filtering (QS_Deny* directive). + +Version 4.3 + + - New handling of graceful server restart. + +Version 4.2 + + - QS_LocKBytesPerSecLimitMatch, QS_LocRequestPerSecLimitMatch + +Version 4.1 + + - QS_LocKBytesPerSecLimit + +Version 4.0 + + - Introduce request/response throttling. + +Version 3.12 + + - Update to mod_qos viewer (status handler). + +Version 3.10 + + - Dynamic error page definition using setenvif. + +Version 3.12 + + - Introduce mod_qos viewer (status handler). + +Version 3.5 + + - QS_KeepAliveTimeout + +Version 3.4 + + - QS_SrvConnTimeout + +Version 3.2 + + - QS_SrvMaxConnTimeout + +Version 3.1 + + - QS_SrvMaxConnExcludeIP + +Version 3.0 + + - Introduce connection level control (QS_SrvMaxConnClose QS_SrvMaxConn). + +Version 2.3 + + - VIP detection. + +Version 2.2 + + - qslog utility. + +Version 2.0 + + - New implementation of location based request limitation. + +Version 1.3 + + - Initial version (scoreboard based request limitation). diff --git a/doc/LICENSE.txt b/doc/LICENSE.txt new file mode 100644 index 0000000..57bc88a --- /dev/null +++ b/doc/LICENSE.txt @@ -0,0 +1,202 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + diff --git a/doc/MESSAGES.txt b/doc/MESSAGES.txt new file mode 100644 index 0000000..3afdb29 --- /dev/null +++ b/doc/MESSAGES.txt @@ -0,0 +1,79 @@ +mod_qos version 11.74 +mod_qos(001): QS_ClientEventLimitCount directives can't be added/removed by graceful restart. A server restart is required to apply the new configuration! +mod_qos(002): failed to create shared memory (ACT)(%s): %s (%lu bytes) +mod_qos(002): failed to create shared memory (client control)(%s): %s (%d bytes) +mod_qos(003): request level rule %s has no concurrent request limitations +mod_qos(004): failed to create mutex (ACT)(%s): %s +mod_qos(004): failed to create mutex (client control)(%s): %s +mod_qos(006): could not compile request header filter rules: %s +mod_qos(006): could not compile response header filter rules: %s +mod_qos(007): calculated MaxClients/MaxRequestWorkers (max connections): %d, applied limit: %d (QS_MaxClients) +mod_qos(008): could not create supervisor thread (%s), disable request rate enforcement +mod_qos(009): could not retrieve mod_ssl functions +mod_qos(009): failed to initialize the qslog facility '%s' +mod_qos(009): found default error document '%s'. Use the QS_ErrorPage directive to override this default page. +mod_qos(009): loaded MPM is '%s' but mod_qos should be used with MPM 'Worker' or 'Event' only. +mod_qos(009): mod_parp not available (required by some directives) +mod_qos(009): mod_unique_id not available (mod_qos generates simple request id if required) +mod_qos(009): running in 'log only' mode - rules are NOT enforced! +mod_qos(009): server version is %d.%d but mod_qos should be used with Apache 2.4 only. +mod_qos(010): access denied%s, QS_LocRequestLimit* rule: %s(%d), concurrent requests=%d, c=%s, id=%s +mod_qos(011): access denied, QS_CondLocRequestLimitMatch rule: %s(%d), concurrent requests=%d, c=%s, id=%s +mod_qos(012): access denied%s, QS_EventRequestLimit rule: %s(%d), concurrent requests=%d, c=%s, id=%s +mod_qos(013): access denied%s, QS_%sEventLimitCount rule: %s, max=%d, current=%d, c=%s, id=%s +mod_qos(021): session cookie verification failed, decoding failed, id=%s +mod_qos(023): session cookie verification failed, expired, id=%s +mod_qos(025): failed to create session cookie, id=%s +mod_qos(030): access denied%s, QS_SrvMaxConn rule: max=%d, concurrent connections=%d, c=%s +mod_qos(031): access denied (previously), QS_SrvMaxConnPerIP rule: max=%d, concurrent connections=%d, message repeated %d times, c=%s +mod_qos(031): access denied%s, QS_SrvMaxConnPerIP rule: max=%d, concurrent connections=%d, c=%s +mod_qos(031): access denied%s, QS_SrvMaxConnPerIP rule: max=%d, concurrent connections=%d, message repeated %d times, c=%s +mod_qos(034): %s, QS_SrvMinDataRate rule (enforce keep-alive), c=%s +mod_qos(034): %s, QS_SrvMinDataRate rule (%s%s): min=%d, this connection=%d, c=%s +mod_qos(035): QS_SrvMaxConn: no free IP slot available! Check log for unclean child exit and consider to do a graceful server restart if this condition persists. You might also increase the number of supported connections using the 'QS_MaxClients' directive. +mod_qos(036): QS_SrvMinDataRate: unexpected connection status! connections=%d, cal. request rate=%d, max. limit=%d. Check log for unclean child exit and consider to do a graceful server restart if this condition persists. You might also increase the number of supported connections using the 'QS_MaxClients' directive. +mod_qos(037): loaded MPM is 'event' and the QS_KeepAliveTimeout/QS_MaxKeepAliveRequests directives can't be used. +mod_qos(038): DSCP, failed to set socket options, QS_Set_DSCP=%s, socket=%s, rc=%d, id=%s +mod_qos(040): access denied, %s rule id: %s (%s), action=%s, c=%s, id=%s +mod_qos(041): access denied, no permit rule match, action=%s, c=%s, id=%s +mod_qos(042): drop %s header%s: '%s: %s', %s, c=%s, id=%s +mod_qos(043): access denied%s, %s header: '%s: %s', %s, c=%s, id=%s +mod_qos(044): access denied%s, QS_LimitRequestBody: invalid content-length header, c=%s, id=%s +mod_qos(044): access denied%s, QS_LimitRequestBody: max=%ld this=%ld, c=%s, id=%s +mod_qos(045): access denied, invalid request line: can't parse uri, c=%s, id=%s +mod_qos(046): access denied, invalid url encoding, action=%s, c=%s, id=%s +mod_qos(047): access denied, reached milestone '%d' (%s), user has already passed '%s', action=%s, c=%s, id=%s +mod_qos(048): access denied, invalid JSON syntax (%s), action=%s, c=%s, id=%s +mod_qos(049): redirect to %s, var=%s, action=%s, c=%s, id=%s +mod_qos(050): request rate limit, rule: %s(%ld), req/sec=%ld, delay=%dms%s +mod_qos(051): request rate limit, rule: %s(%ld), req/sec=%ld, delay=%dms +mod_qos(060): access denied (previously), QS_ClientEventBlockCount rule: max=%d, current=%hu, message repeated %d times, c=%s +mod_qos(060): access denied, QS_ClientEventBlockCount rule: max=%d, current=%hu, age=%ld, c=%s +mod_qos(060): access denied, QS_ClientEventBlockCount rule: max=%d, current=%hu, message repeated %d times, c=%s +mod_qos(060): access denied%s, QS_ClientEventBlockCount rule: max=%d, current=%hu, age=%ld, c=%s +mod_qos(061): request rate limit, rule: QS_Event(%d), req/sec=%ld, delay=%dms%s +mod_qos(062): request rate limit, rule: QS_Event(%d), req/sec=%ld, delay=%dms +mod_qos(065): access denied%s, QS_ClientEventRequestLimit rule: max=%d, current=%d, c=%s, id=%s +mod_qos(066): access denied%s, QS_ClientPrefer rule (penalty=%d 0x%02x): max=%d, concurrent connections=%d, c=%s +mod_qos(067): access denied%s, QS_%sClientEventLimitCount rule: event=%s, max=%hu, current=%hu, age=%ld, c=%s +mod_qos(068): QS_ClientSerialize exceeds limit of 5 minutes, c=%s, id=%s +mod_qos(068): QS_SrvSerialize exceeds limit of %d seconds, id=%s +mod_qos(069): no valid IP header found (@%s): header '%s' not available, fallback to connection's IP %s, id=%s +mod_qos(069): no valid IP header found (@%s): invalid header value '%s', fallback to connection's IP %s, id=%s +mod_qos(070): console, not acceptable, client data store has not been enabled, id=%s +mod_qos(070): console, not acceptable, invalid ip/wrong format, id=%s +mod_qos(070): console, not acceptable, missing request query (action/address), id=%s +mod_qos(070): console, not acceptable, qos client control has not been activated, id=%s +mod_qos(070): console, not acceptable, unknown action '%s', id=%s +mod_qos(071): console, action '%s' applied to client ip entry '%s', id=%s +mod_qos(071): console, add new client ip entry '%s', is=%s +mod_qos(072): handler has been disabled for this host, id=%s +mod_qos(080): Can't generate random data, id=%s +mod_qos(083): Can't generate random data. +mod_qos(100): QS_ClientGeoCountryDB has not been configured +mod_qos(101): access denied%s, QS_ClientGeoCountryPriv rule: max=%d, concurrent connections=%d, c=%s country=%s +mod_qos(147): access denied, reached milestone '%d' (%s), earlier than expected (right after %ld instead of %d seconds), action=%s, c=%s, id=%s +mod_qos(166): unexpected connection dispatching, skipping connection counter update for QS_ClientPrefer rule, c=%s +mod_qos(167): closing connection at process connection hook, c=%s +mod_qos(200): { "scoreboard": { "open": %d, "waiting": %d, "read": %d, "write": %d, "keepalive": %d, "start": %d, "log": %d, "dns": %d, "closing": %d, "finishing": %d, "idle": %d }, "maxclients": { "max": %d, "busy": %d%s }%s } +mod_qos(210): ENV %s %s %s diff --git a/doc/favicon.ico b/doc/favicon.ico Binary files differnew file mode 100644 index 0000000..717367b --- /dev/null +++ b/doc/favicon.ico diff --git a/doc/glossary.html b/doc/glossary.html new file mode 100644 index 0000000..66ddc49 --- /dev/null +++ b/doc/glossary.html @@ -0,0 +1,603 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html> +<head> + <title>mod_qos - Additional Information</title> +<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1" /> +<meta name="author" content="Pascal Buchbinder" /> +<meta name="KeyWords" content="mod_qos, Quality of Service, Apache Web Server" /> +<link rel="shortcut icon" href="favicon.ico" /> +<style TYPE="text/css"> +<!-- + body { + background-color: white; + color: black; + font-family: sans-serif, arial, verdana; + font-weight: normal; + text-align: left; + } + a:link { color:#00673F; text-decoration:none; } + a:visited { color:#00673F; text-decoration:none; } + a:focus { color:black; text-decoration:underline; } + a:hover { color:black; text-decoration:underline; } + a:active { color:black; text-decoration:underline; } + syntax { font-family: monospace; font-size: 14; line-height: 1.6; } + .btable { font-size:0.75em; } +--> +</style> +</head> +<body> +<!-- + + Quality of service module for Apache Web Server. + + See http://mod-qos.sourceforge.net/ for further details. + + Copyright (C) 2023 Pascal Buchbinder + + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + +--> +<table> +<tbody> +<tr><td><a href="index.html"><img src="images/mod_qos.gif" alt="mod_qos" title="mod_qos" /></a></td> + <td style="vertical-align: bottom;"><h1>Additional Information</h1></td></tr> +<tr><td> </td> + <td> + + +<p> +This page gives you additional information on certain aspects of +<a href="index.html">mod_qos</a> to get a better understanding about +how to use the module. +</p> +<hr> + +<p> + <ul> + <li><a href="#directives">Directives</a></li> + <li><a href="#rules">Rules</a></li> + <li><a href="#variables">Environment Variables</a></li> + <li><a href="#concurrency">Concurrency Counter</a></li> + <ul> + <li><a href="#QS_LocRequestLimit_Example">Sample Use Case</a></li> + </ul> + <li><a href="#repeat">Repeat Counter</a></li> + <li><a href="#throughput">Throughput Control</a></li> + <ul> + <li><a href="#requestPerSecond">Requests per Second</a></li> + </ul> + <li><a href="#serialization">Serialization</a></li> + <li><a href="#ssi">Error Pages and Server Side Includes (SSI)</a></li> + <li><a href="#UserTracking">User Tracking</a></li> + <li><a href="#RequestStatistics">Request Statistics Using <i>qslog</i></a></li> + </ul> +</p> + + +<!-- --------------------------------------------------------- --> +<hr> +<a name="directives"></a> +<h2>Directives</h2> +<p> +The module is configured by directives. All directives process the +connection, HTTP request, and response data in a pre-defined sequence. +The following graph shows the order in which the directives work. +</p> +<p> +<a href="images/directive_seq.gif"><img src="images/directive_seq.gif" height="435" width="558" title="directive sequence" alt="directive sequence"/></a> +</p> + + +<!-- --------------------------------------------------------- --> +<hr> +<a name="rules"></a> +<h2>Rules</h2> +<p> +<a href="index.html">mod_qos</a> allows you to configure different kind of +rules. The main component of a rule is its counter. A rule measures either +the <a href="#concurrency">concurrency</a> (how many times something happens +at the same time), the <a href="#repeat">occurrence </a> (how often does +something happen in a certain amount of time), or the +<a href="#throughput">throughput</a> (sent amount of data or number of +request) and stores this information within that counter.</p> +<p> +Every rule has it's own threshold and maintains its own counter. A rule +is identified by either an URL pattern/matching string or by an +<a href="#variables">environment variable name</a>. +You can configure as many rules as you want. </p> +<img src="images/Rule.png" height="164" width="514" alt="Rule" /> +<p> +<i>Note: Some counters are only available once. This applies to the counters of +the rules using the +<code><a href="index.html#QS_Block">QS_Block</a></code>, +<code><a href="index.html#QS_SrvSerialize_var">QS_SrvSerialize</a></code>, and +<code><a href="index.html#QS_Serialize">QS_Serialize</a></code> +<a href="#variables">environment variables</a>. +</i> +</p> + + +<!-- --------------------------------------------------------- --> +<hr> +<a name="variables"></a> +<h2>Environment Variables</h2> +<p> +The Apache web server provides a mechanism for storing information +in so called <i>environment variables</i>. <a href="index.html">mod_qos</a> uses these +variables to exchange data respectively signalize events between +<a href="#rules">different rules</a> defined by the +<a href="#directives">corresponding directive</a>. These +variables can also be written or read by other Apache modules, such as +<a href="http://modsetenvifplus.sourceforge.net/">mod_setenvifplus <img src="images/link.png"/></a> +or <a href="http://httpd.apache.org/docs/current/mod/mod_setenvif.html">mod_setenvif <img src="images/link.png"/></a>. +</p> +<p>Example:<br> +The <code><a href="http://httpd.apache.org/docs/current/mod/mod_setenvif.html#setenvif">SetEnvIf <img src="images/link.png"/></a></code> directive +is used to set the <i>LimitLogin</i> variable if the request line matches +the <code>^/wp-login.php</code> pattern while the +<a href="index.html#QS_ClientEventLimitCount"><code>QS_ClientEventLimitCount</code></a> +directives increments the <a href="#repeat">repeat counter</a> having the same name +if the variable is present. +<br> +<img src="images/Events.png" height="396" width="795" alt="Event set by SetEnvIf and processed by QS_ClientEventLimitCount" /> +</p> +<p>It is also possible to write the values of these variables to your log +file using the format string <code>%{VARNAME}e</code> within the +<code><a href="http://httpd.apache.org/docs/current/mod/mod_log_config.html">TransferLog/CustomLog <img src="images/link.png"/></a></code> +directives. Or you can use them within error pages using +<a href="#ssi">server-side includes (SSI)</a>. +</p> +<p> +<i>Note: Whenever you use a directive (such as +<code><a href="http://modsetenvifplus.sourceforge.net/#SetEnvIfPlus">SetEnvIfPlus <img src="images/link.png"/></a></code>) which can either process request attributes (such +as HTTP headers) or environment variables, you must make sure that +a client can not bypass your rules by sending a request header +with the same name as the environment variable used in your configuration. +Use either the <a href="index.html#QS_RequestHeaderFilter">request header filter</a> +or the <code><a href="index.html#QS_UnsetReqHeader">QS_UnsetReqHeader</a></code> +directive to prevent anyone from sending a request header with the same name as +the variable you have defined. +</i></p> + +<!-- --------------------------------------------------------- --> +<hr> +<a name="concurrency"></a> +<h2>Concurrency Counter</h2> +<p> +A "concurrency counter" is used to determine how many times something +happens at the same time (concurrent), e.g. HTTP requests accessing the same +resource/URL at the same time. The rules using this counter type are either +defined by an <a href="#variables">environment variable name</a> or an +URL pattern (regular expression or a string matching the request's URL). +Such a rule automatically increments the counter when the Apache web server +starts to process a matching request and decrements the counter when the +request processing is completed. +</p> +<p> +You have to configure a threshold and the rule's variable name resp. URL +pattern. Requests (or new connections) are denied as soon as the configured +threshold is reached. +</p> +<p> +Directives using this counter type are: +<ul> +<li><a href="index.html#QS_LocRequestLimitMatch"><syntax>QS_LocRequestLimitMatch</syntax></a></li> +<li><a href="index.html#QS_LocRequestLimit"><syntax>QS_LocRequestLimit</syntax></a></li> +<li><a href="index.html#QS_CondLocRequestLimitMatch"><syntax>QS_CondLocRequestLimitMatch</syntax></a></li> +<li><a href="index.html#QS_EventRequestLimit"><syntax>QS_EventRequestLimit</syntax></a></li> +<li><a href="index.html#QS_ClientEventRequestLimit"><syntax>QS_ClientEventRequestLimit</syntax></a></li> +</ul> +Also the +<a href="index.html#QS_SrvMaxConn"><code>QS_SrvMaxConn</code></a>, +<a href="index.html#QS_SrvMaxConnClose"><code>QS_SrvMaxConnClose</code></a>, and +<a href="index.html#QS_SrvMaxConnPerIP"><code>QS_SrvMaxConnPerIP</code></a> +directives use this counter type, although with fewer parameter options. +</p> +<p> +<a name="QS_LocRequestLimit_Example"></a> +<h4>Sample Use Case</h4> +Now let us look at an example to show where these rules can be used. +Let's assume that you have two applications. We call them +"<font color="green">A</font>" and "<font color="blue">B</font>". +Application "<font color="green">A</font>" has been deployed on +path <code>/app/a</code> and "<font color="blue">B</font>" +on <code>/app/b</code>. +<table width="780px"> + <tr> +<td> </td> +<td> +As long as the server has enough resources, users can access both +applications the same time without influencing each other.<br> +All requests are processed quickly and the workers (w) become free +again to serve new requests. +<br><br> +</td> +<td> +<small> </small> <img src="images/qsloc1.png" height="216" width="408" alt="trouble-free"/> +</td> + </tr> + <tr> +<td> </td> +<td> +But if the application "<font color="blue">B</font>" becomes slow, +the duration of request processing increases and all workers (w) become busy. +There are no free workers left and application +"<font color="green">A</font>" becomes unavailable because of that. +<br><br> +</td> +<td> + <img src="images/qsloc2.png" height="212" width="422" alt="disruption"/> +</td> + </tr> + <tr> +<td> </td> +<td> +A <a href="index.html#QS_LocRequestLimit"><code>QS_LocRequestLimit</code></a> or +<a href="index.html#QS_LocRequestLimitMatch"><code>QS_LocRequestLimitMatch</code></a> +rule can help in such a situation. mod_qos limits the +number of requests to application "<font color="blue">B</font>" +so that application "<font color="green">A</font>" remains +available even if application "<font color="blue">B</font>" +has performance problems. +</td> +<td> + <img src="images/qsloc3.png" height="244" width="422" alt="disruption"/> +</td> + </tr> +</table> +<br> +Such a scenario can occur due to various infrastructure problems, +e.g., by slow database queries.<br> A similar situation can also +arise through an external influence: if someone penetrates application +"<font color="blue">B</font>" with a <i>HTTP GET / POST flood DoS attack</i>, +then application "<font color="green">A</font>" could also become +unreachable. +A <a href="index.html#QS_LocRequestLimit"><code>QS_LocRequestLimit</code></a> +rule can prevent this. +</p> + +<!-- --------------------------------------------------------- --> +<hr> +<a name="repeat"></a> +<h2>Repeat Counter</h2> +<p> +"Repeat counters" limit the number how often (Cr) something is allowed to +happen in a certain amount of time (Td). These rules trigger a timer whenever +the defined event occurs the first time and start to count every subsequent event +until the timer expires. If the event counter reaches the defined limitation, +requests are blocked until the time is up. +</p> +<p> + <img src="images/LimitCount.png" height="432" width="576" alt="reapet counter"/> +</p> +<p> +All repeat counters allow you to define an event which shall +increment the counter if they occur. You also have to configure a +duration Td and the threshold Cr, defining how many events are +allowed within the time Td.</p> +<p>Directive parameter example:<br> +<img src="images/LimitCountExample.png" height="165" width="574" alt="QS_ClientEventLimitCount" /> +</p> +<p> +While the counter is automatically cleared (set to 0) when the +time Td is up, you might also configure additional events to +<a href="index.html#_Decrement">decrement</a> or +<a href="index.html#_Clear">clear</a> the counter earlier. +</p> +<p> +The directives using this counter type are: +<ul> +<li><a href="index.html#QS_EventLimitCount"><syntax>QS_EventLimitCount</syntax></a></li> +<li><a href="index.html#QS_CondEventLimitCount"><syntax>QS_CondEventLimitCount</syntax></a></li> +<li><a href="index.html#QS_ClientEventLimitCount"><syntax>QS_ClientEventLimitCount</syntax></a></li> +<li><a href="index.html#QS_CondClientEventLimitCount"><syntax>QS_CondClientEventLimitCount</syntax></a></li> +<li><a href="index.html#QS_ClientEventBlockCount"><syntax>QS_ClientEventBlockCount</syntax></a></li> +</ul> +<i>Note: Some directives support the increase of the counter by more than "1" +taking the variable's value into account.</i> +</p> + + +<!-- --------------------------------------------------------- --> +<hr> +<a name="throughput"></a> +<h2>Throughput Control</h2> +<p> +Throughput control is implemented by measuring the current usage and +calculating a necessary delay which needs to be applied to the +data processing in order to achieve the desired limitation (closed +loop control system). +</p> +<p> + <img src="images/ClosedLoop.png" height="216" width="478" alt="closed loop"/> +</p> +<p> +<a href="index.html">mod_qos</a> can limit the bandwidth when downloading +data from your web server to the client. This throughput control can +be configured by the following directives: +<ul> +<li><a href="index.html#QS_LocKBytesPerSecLimitMatch"><syntax>QS_LocKBytesPerSecLimitMatch</syntax></a></li> +<li><a href="index.html#QS_LocKBytesPerSecLimit"><syntax>QS_LocKBytesPerSecLimit</syntax></a></li> +<li><a href="index.html#QS_EventKBytesPerSecLimit"><syntax>QS_EventKBytesPerSecLimit</syntax></a></li> +</ul> +<i>Note: Throughput control should always be used with a <a href="#concurrency">concurrency counter</a> +because the added delay increases the number of simultaneous requests. +</i></p> +<a name="requestPerSecond"></a> +<h3>Requests per Second</h3> +<p> +It is also possible to limit the number or requests per second to a +resource. This control function is less accurate than the bandwidth +limitation, since the measurement of the request rate takes longer +(several seconds) and the request delay is more coarse-grained.<br> +The following directive can be used to limit the number of requests +per second: +<ul> +<li><a href="index.html#QS_LocRequestPerSecLimitMatch"><syntax>QS_LocRequestPerSecLimitMatch</syntax></a></li> +<li><a href="index.html#QS_LocRequestPerSecLimit"><syntax>QS_LocRequestPerSecLimit</syntax></a></li> +<li><a href="index.html#QS_EventPerSecLimit"><syntax>QS_EventPerSecLimit</syntax></a></li> +<li><a href="index.html#QS_ClientEventPerSecLimit"><syntax>QS_ClientEventPerSecLimit</syntax></a></li> +</ul> +</p> + + +<!-- --------------------------------------------------------- --> +<hr> +<a name="serialization"></a> +<h2>Serialization</h2> +<p> +<a href="index.html">mod_qos</a> offers you the option to serialize requests. +Serialization means, that requests are processed one after the other. Incoming +requests are queued if another request is in process and have to wait until +the previous request is finished. +</p> +<p> + <img src="images/Serialization.png" height="336" width="570" alt="serialization"/> +</p> +<p> +Requests, which shall be serialized, are tagged by one of the following +<a href="#variables">environment variables</a>: +<ul> +<li><syntax><a href="index.html#QS_SrvSerialize_var">QS_SrvSerialize</a></syntax></li> +<li><syntax><a href="index.html#QS_Serialize">QS_Serialize</a></syntax></li> +</ul> +</p> +<p> +Serialization might be applied on a per +<a href="index.html#QS_SrvSerialize">server level</a> (serializing all HTTP +requests) or on a <a href="index.html#QS_ClientSerialize">per client level</a> +(serializing multiple requests coming from the same client/IP address). +</p> + + +<!-- --------------------------------------------------------- --> +<hr> +<a name="ssi"></a> +<a name="SSI"></a> +<h2>Error Pages and Server Side Includes (SSI)</h2> +<p> +Custom error documents to be used by <a href="index.html">mod_qos</a> +are either configured using the +<code><a href="index.html#QS_ErrorPage">QS_ErrorPage</a></code> directive +or the <code><a href="index.html#QS_ErrorPage_Var">QS_ErrorPage</a></code> +variable.<br> +You may also use Apache's <a href="http://httpd.apache.org/docs/current/howto/ssi.html">server-side includes (SSI) <img src="images/link.png"/></a> to generate the content +of the error document dynamically. The <a href="index.html#errorlog">error codes</a> and +other <a href="index.html#variables">variables</a> set by +<a href="index.html">mod_qos</a> can be used. +</p> +<p> +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +Sample configuration:<br> +<pre> +AddType text/html .shtml +AddOutputFilter INCLUDES .shtml +QS_ErrorPage <a href="#qs_error.shtml">/errorpages/qs_error.shtml</a> +</pre> +</td></tr> +</table> +</p> + +<p> +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"><a name="qs_error.shtml"></a> +Sample page:<br> +<pre> +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html> + <head> + <!-- + -- mod_qos sample SSI error page (Apache 2.4) + --> + <meta http-equiv="Cache-Control" content="no-cache, no-store"/> + <meta http-equiv="expires" content="0" /> + <title>ERROR - <!--#echo var="REDIRECT_ERROR_NOTES" --></title> + </head> + <body> + <p> + <b><i>sorry - the server was unable to complete your request</i></b> + </p> + <p> + code: mod_qos(<!--#echo var="<a href="index.html#QS_ErrorNotes">QS_ErrorNotes</a>" -->)<br> + <!--#if expr="v('REDIRECT_ERROR_NOTES') =~ /00[0-9]/" --> + reason: initialisation failure + <!--#elif expr="v('REDIRECT_ERROR_NOTES') =~ /0[18][0-9]/" --> + reason: request rule + <!--#elif expr="v('REDIRECT_ERROR_NOTES') =~ /03[0-9]/" --> + reason: connection rule + <!--#elif expr="v('REDIRECT_ERROR_NOTES') =~ /[01]4[0-9]/" --> + reason: request filter + <!--#elif expr="v('REDIRECT_ERROR_NOTES') =~ /05[0-9]/" --> + reason: bandwidth limitation + <!--#elif expr="v('REDIRECT_ERROR_NOTES') =~ /[01]6[0-9]/" --> + reason: client limitation <br> + remaining time: <span id="remaining"><!--#echo var="<a href="index.html#QS_Limit_Remaining">QS_Limit_Remaining</a>" --></span> seconds + <script type="text/javascript"> + <!-- + setInterval(function () { + var msg = document.getElementById('remaining'); + if(msg) { + var value = msg.innerHTML; + var remainTime = value - 1; + if(remainTime < 0) { + window.location = window.location.pathname; + } else { + msg.innerHTML = remainTime; + } + } + }, 1000); + //--> + </script> + <!--#elif expr="v('REDIRECT_ERROR_NOTES') =~ /10[0-9]/" --> + reason: GEO location limitation + <!--#else --> + reason: generic failure + <!--#endif --> + </p> + </body> +</html> +</pre> +</td></tr> +</table> +</p> + + +<!-- --------------------------------------------------------- --> +<hr> +<a name="UserTracking"></a> +<a name="QS_UserTrackingCookieName"></a> +<h2>User Tracking</h2> +<p> +It might be necessary to identify individual users to define appropriate QoS +rules. For this reason, <a href="index.html">mod_qos</a> can set a cookie +containing a unique identifier. This identifier is then written to the +<code><a href="index.html#mod_qos_user_id">mod_qos_user_id</a></code> +environment variable and you can add it to your log files by the +format string <code>%{mod_qos_user_id}e</code>. This allows you to identify +all requests issued by a user.</p> +<p> +This feature is enabled by the following directive: +<ul> +<li><syntax>QS_UserTrackingCookieName <name> [<path>] [<domain>] ['session'] ['jsredirect']</syntax><br> +The parameter "name" defines the cookie's name and "domain" (optional) +the domain attribute for the Set-Cookie header. The string "session" can +be defined if the cookie should not be stored by the session (but can +be deleted when the user closes this browser). +</li> +</ul> +You can also enforce the browser to support cookies by specifying the +"path" parameter for the <code>QS_UserTrackingCookieName</code> directive. +This parameter defines an error document and mod_qos +answers the request with a redirect (302) to this document when setting +the cookie initially. The browser will follow the redirect and mod_qos +redirects the browser back to the initially requested page if the request +to this error document contains the tracking cookie. If the browser did not +send the cookie, the error document is shown. +</p> +<p> <img src="images/UserTracking.png" height="386" width="410" alt="user tracking cookie enforcement"/></p> +<p> +<i>Note: You can exclude certain clients from this enforcement by +setting the <code>DISABLE_UTC_ENFORCEMENT</code> +<a href="#variables">environment variable</a> +at server level (outside Location), e.g., to allow crawlers not +supporting cookies to access your site.</i> +</p> +<p> +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +Sample configuration:<br> +<pre> +<a href="index.html#QS_UserTrackingCookieName">QS_UserTrackingCookieName</a> qstrack <a href="#cookiecheck">/errorpages/cookiecheck.html</a> session +<a href="index.html#QS_SessionKey">QS_SessionKey</a> sB.F4_0%D700ahXT2 +</pre> +</td></tr> +</table> +</p> +<p> +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"><a name="cookiecheck"></a> +Sample page:<br> +<pre> +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html> + <head> + <meta http-equiv="Cache-Control" content="no-cache, no-store"/> + <meta http-equiv="expires" content="0" /> + <title>Cookie Check Page</title> + </head> + <body> + <h1>Cookie Check Failed</h1> + <p> + Please enable cookies in your browser.<br> + Bitte schalten Sie in Ihrem Browser Cookies ein.<br> + Activez les cookies dans votre navigateur, s'il vous pla&icirc;t.<br> + Por favor active las cookies en su navegador.<br> + Si prega di attivare i cookies nel proprio browser.<br> + </p> + </body> +</html> +</pre> +</td></tr> +</table> +</p> + +<!-- --------------------------------------------------------- --> +<hr> +<a name="qslog"></a> +<a name="RequestStatistics"></a> +<h2>Request Statistics Using <i>qslog</i></h2> +<p> +<code><a href="qslog.1.html">qslog</a></code> is a command line tool to +generate usage statistics data from log files. It can generate the data in +real time if defined within your Apache's server configuration. +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +Sample configuration:<br> +<pre> +CustomLog "|/usr/bin/qslog -o logs/qslog.csv -x -f ISBDk" "%h %>s %b %D %k" +</pre> +</td></tr> +</table> +</p><p> +Or you can use it to process any existing log file using the post processing +option <code>-p</code>. This does not only work for +<a href="http://httpd.apache.org/docs/current/mod/mod_log_config.html">Apache log <img src="images/link.png"/></a> files but for any other log file as well. You just have to know what's in the log +and configure the format string argument <code>-f</code> of the +<code><a href="qslog.1.html">qslog</a></code> command accordingly. +</p> +<p> +Example:<br> +<img src="images/qslogFormat.png" height="381" width="738" alt="log format definition"/> +</p> +<p> +The output shows the data as a function of time: a summary of what happened every minute. +Each line includes all measured values as semicolon spearated name/value pairs (CSV). +</p> +<p> +You can use the spreadsheet program of your choice to process the output:<br><br> +<img src="images/qslog_spreadsheet_example.png" height="413" width="850" alt="spreadsheet"/> +</p> + +<!-- --------------------------------------------------------- --> +</td></tr> +</tbody> +</table> +<br> +<hr> +<SMALL><SMALL>© 2023, Pascal Buchbinder</SMALL></SMALL> +</body> +</html> diff --git a/doc/headerfilterrules.txt b/doc/headerfilterrules.txt new file mode 100644 index 0000000..58d4d5c --- /dev/null +++ b/doc/headerfilterrules.txt @@ -0,0 +1,97 @@ + +QS_RequestHeaderFilter rules: + + name=Accept, action=drop, size=300, pattern=^([a-zA-Z0-9_*+-]+/[a-zA-Z0-9_*+.-]+(;[ ]?[a-zA-Z0-9]+=[0-9]+)?[ ]?(;[ ]?[qv]=[a-z0-9.]+)?){1}([ ]?,[ ]?([a-zA-Z0-9_*+-]+/[a-zA-Z0-9_*+.-]+(;[ ]?[a-zA-Z0-9]+=[0-9]+)?[ ]?(;[ ]?[qv]=[a-z0-9.]+)?))*$ + name=Accept-Charset, action=drop, size=300, pattern=^([a-zA-Z0-9*-]+(;[ ]?q=[0-9.]+)?){1}([ ]?,[ ]?([a-zA-Z0-9*-]+(;[ ]?q=[0-9.]+)?))*$ + name=Accept-Encoding, action=drop, size=500, pattern=^([a-zA-Z0-9*-]+(;[ ]?q=[0-9.]+)?){1}([ ]?,[ ]?([a-zA-Z0-9*-]+(;[ ]?q=[0-9.]+)?))*$ + name=Accept-Language, action=drop, size=200, pattern=^([a-zA-Z*-]+[0-9]{0,3}(;[ ]?q=[0-9.]+)?){1}([ ]?,[ ]?([a-zA-Z*-]+[0-9]{0,3}(;[ ]?q=[0-9.]+)?))*$ + name=Access-Control-Request-Method, action=drop, size=10, pattern=^[a-zA-Z]+$ + name=Access-Control-Request-Headers, action=drop, size=500, pattern=^([a-zA-Z0-9-]+){1}([ ]?,[ ]?([a-zA-Z0-9-]+))*$ + name=Authorization, action=drop, size=4000, pattern=^[a-zA-Z0-9 +/$=:]+$ + name=Cache-Control, action=drop, size=100, pattern=^(no-cache|no-store|max-age=[0-9]+|max-stale(=[0-9]+)?|min-fresh=[0-9]+|no-transform|only-if-chached){1}([ ]?,[ ]?(no-cache|no-store|max-age=[0-9]+|max-stale(=[0-9]+)?|min-fresh=[0-9]+|no-transform|only-if-chached))*$ + name=Connection, action=drop, size=100, pattern=^([teTE]+,[ ]?)?([a-zA-Z0-9-]+){1}([ ]?,[ ]?([teTE]+))?$ + name=Content-Encoding, action=deny, size=100, pattern=^[a-zA-Z0-9-]+(,[ ]*[a-zA-Z0-9-]+)*$ + name=Content-Language, action=drop, size=100, pattern=^([0-9a-zA-Z]{0,8}(-[0-9a-zA-Z]{0,8})*)(,[ ]*([0-9a-zA-Z]{0,8}(-[0-9a-zA-Z]{0,8})*))*$ + name=Content-Length, action=deny, size=10, pattern=^[0-9]+$ + name=Content-Location, action=deny, size=200, pattern=^[:/?#\[\]@!$&'()*+,;=a-zA-Z0-9._~% -]+$ + name=Content-md5, action=deny, size=50, pattern=^[a-zA-Z0-9 +/$=:]+$ + name=Content-Range, action=deny, size=50, pattern=^(bytes[ ]+([0-9]+-[0-9]+)/([0-9]+|\*))$ + name=Content-Type, action=deny, size=200, pattern=^(["a-zA-Z0-9*/; =-]+){1}([ ]?,[ ]?(["a-zA-Z0-9*/; =-]+))*$ + name=Cookie, action=drop, size=3000, pattern=^[:/?#\[\]@!$&'()*+,;="a-zA-Z0-9._~% -]+$ + name=Cookie2, action=drop, size=3000, pattern=^[:/?#\[\]@!$&'()*+,;="a-zA-Z0-9._~% -]+$ + name=DNT, action=drop, size=3, pattern=^[0-9]+$ + name=Expect, action=drop, size=200, pattern=^[a-zA-Z0-9= ;.,-]+$ + name=From, action=drop, size=100, pattern=^[a-zA-Z0-9=@;.,()-]+$ + name=Host, action=drop, size=100, pattern=^[a-zA-Z0-9.-]+(:[0-9]+)?$ + name=If-Invalid, action=drop, size=500, pattern=^[a-zA-Z0-9_.:;() /+!-]+$ + name=If-Match, action=drop, size=100, pattern=^(W/)?[a-zA-Z0-9=@;.,*"-]+$ + name=If-Modified-Since, action=drop, size=100, pattern=^[a-zA-Z0-9 :,]+$ + name=If-None-Match, action=drop, size=100, pattern=^(W/)?[a-zA-Z0-9=@;.,*"-]+$ + name=If-Range, action=drop, size=100, pattern=^[a-zA-Z0-9=@;.,*"-]+$ + name=If-Unmodified-Since, action=drop, size=100, pattern=^[a-zA-Z0-9 :,]+$ + name=If-Valid, action=drop, size=500, pattern=^[a-zA-Z0-9_.:;() /+!-]+$ + name=Keep-Alive, action=drop, size=20, pattern=^[0-9]+$ + name=Max-Forwards, action=drop, size=20, pattern=^[0-9]+$ + name=Origin, action=drop, size=2000, pattern=^[:/?#\[\]@!$&'()*+,;=a-zA-Z0-9._~% -]+$ + name=Proxy-Authorization, action=drop, size=400, pattern=^[a-zA-Z0-9 +/$=:]+$ + name=Pragma, action=drop, size=200, pattern=^[a-zA-Z0-9= ;.,-]+$ + name=Range, action=drop, size=200, pattern=^[a-zA-Z0-9=_.:;() /+!-]+$ + name=Referer, action=drop, size=2000, pattern=^[:/?#\[\]@!$&'()*+,;=a-zA-Z0-9._~% -]+$ + name=TE, action=drop, size=100, pattern=^([a-zA-Z0-9*-]+(;[ ]?q=[0-9.]+)?){1}([ ]?,[ ]?([a-zA-Z0-9*-]+(;[ ]?q=[0-9.]+)?))*$ + name=Transfer-Encoding, action=deny, size=100, pattern=^(chunked|Chunked|compress|Compress|deflate|Deflate|gzip|Gzip|identity|Identity)([ ]?,[ ]?(chunked|Chunked|compress|Compress|deflate|Deflate|gzip|Gzip|identity|Identity))*$ + name=Unless-Modified-Since, action=drop, size=100, pattern=^[a-zA-Z0-9 :,]+$ + name=User-Agent, action=drop, size=300, pattern=^[a-zA-Z0-9]+[a-zA-Z0-9_.:;()\[\]@ /+!=,-]+$ + name=Upgrade-Insecure-Requests, action=drop, size=1, pattern=^1$ + name=Via, action=drop, size=100, pattern=^[a-zA-Z0-9_.:;() /+!-]+$ + name=X-Forwarded-For, action=drop, size=100, pattern=^[a-zA-Z0-9_.:-]+(, [a-zA-Z0-9_.:-]+)*$ + name=X-Forwarded-Host, action=drop, size=100, pattern=^[a-zA-Z0-9_.:-]+$ + name=X-Forwarded-Server, action=drop, size=100, pattern=^[a-zA-Z0-9_.:-]+$ + name=X-lori-time-1, action=drop, size=20, pattern=^[0-9]+$ + name=X-Do-Not-Track, action=drop, size=20, pattern=^[0-9]+$ + +QS_ResponseHeaderFilter rules: + + name=Age, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Accept-Ranges, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Access-Control-Allow-Origin, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Access-Control-Allow-Methods, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Access-Control-Allow-Headers, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Access-Control-Max-Age, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Access-Control-Allow-Credentials, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Access-Control-Expose-Headers, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Allow, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Cache-Control, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Content-Disposition, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Content-Encoding, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Content-Language, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Content-Length, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Content-Location, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Content-MD5, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Content-Range, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Content-Security-Policy, action=drop, size=8000, pattern=^[\x20-\xFF]*$ + name=Content-Type, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Connection, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Date, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=ETag, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Expect, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Expires, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Keep-Alive, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Last-Modified, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Location, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Proxy-Authenticate, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Public-Key-Pins, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Public-Key-Pins-Report-Only, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Retry-After, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Pragma, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Server, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Set-Cookie, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Set-Cookie2, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Strict-Transport-Security, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=Vary, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=WWW-Authenticate, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=X-Content-Security-Policy, action=drop, size=8000, pattern=^[\x20-\xFF]*$ + name=X-Content-Type-Options, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=X-Frame-Options, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + name=X-XSS-Protection, action=drop, size=4000, pattern=^[\x20-\xFF]*$ + +mod_qos 11.74 diff --git a/doc/images/ClientPrefer.png b/doc/images/ClientPrefer.png Binary files differnew file mode 100644 index 0000000..ad1f759 --- /dev/null +++ b/doc/images/ClientPrefer.png diff --git a/doc/images/ClosedLoop.png b/doc/images/ClosedLoop.png Binary files differnew file mode 100644 index 0000000..8303b73 --- /dev/null +++ b/doc/images/ClosedLoop.png diff --git a/doc/images/Events.png b/doc/images/Events.png Binary files differnew file mode 100644 index 0000000..8ae21cc --- /dev/null +++ b/doc/images/Events.png diff --git a/doc/images/LimitCount.png b/doc/images/LimitCount.png Binary files differnew file mode 100644 index 0000000..7c8586d --- /dev/null +++ b/doc/images/LimitCount.png diff --git a/doc/images/LimitCountExample.png b/doc/images/LimitCountExample.png Binary files differnew file mode 100644 index 0000000..f210c36 --- /dev/null +++ b/doc/images/LimitCountExample.png diff --git a/doc/images/QS_ClientEventBlockCount.png b/doc/images/QS_ClientEventBlockCount.png Binary files differnew file mode 100644 index 0000000..452c887 --- /dev/null +++ b/doc/images/QS_ClientEventBlockCount.png diff --git a/doc/images/Rule.png b/doc/images/Rule.png Binary files differnew file mode 100644 index 0000000..9d3ff86 --- /dev/null +++ b/doc/images/Rule.png diff --git a/doc/images/Serialization.png b/doc/images/Serialization.png Binary files differnew file mode 100644 index 0000000..241a260 --- /dev/null +++ b/doc/images/Serialization.png diff --git a/doc/images/SrvMinDataRate.png b/doc/images/SrvMinDataRate.png Binary files differnew file mode 100644 index 0000000..e992a99 --- /dev/null +++ b/doc/images/SrvMinDataRate.png diff --git a/doc/images/UserTracking.png b/doc/images/UserTracking.png Binary files differnew file mode 100644 index 0000000..6ae02c6 --- /dev/null +++ b/doc/images/UserTracking.png diff --git a/doc/images/directive_seq.gif b/doc/images/directive_seq.gif Binary files differnew file mode 100644 index 0000000..0fcae8e --- /dev/null +++ b/doc/images/directive_seq.gif diff --git a/doc/images/download.jpg b/doc/images/download.jpg Binary files differnew file mode 100644 index 0000000..4c3ac5d --- /dev/null +++ b/doc/images/download.jpg diff --git a/doc/images/link.png b/doc/images/link.png Binary files differnew file mode 100644 index 0000000..d6a9ee2 --- /dev/null +++ b/doc/images/link.png diff --git a/doc/images/mod_qos.gif b/doc/images/mod_qos.gif Binary files differnew file mode 100644 index 0000000..fc4077c --- /dev/null +++ b/doc/images/mod_qos.gif diff --git a/doc/images/qsloc.png b/doc/images/qsloc.png Binary files differnew file mode 100644 index 0000000..3b8482f --- /dev/null +++ b/doc/images/qsloc.png diff --git a/doc/images/qsloc1.png b/doc/images/qsloc1.png Binary files differnew file mode 100644 index 0000000..6b710ff --- /dev/null +++ b/doc/images/qsloc1.png diff --git a/doc/images/qsloc2.png b/doc/images/qsloc2.png Binary files differnew file mode 100644 index 0000000..348e6bc --- /dev/null +++ b/doc/images/qsloc2.png diff --git a/doc/images/qsloc3.png b/doc/images/qsloc3.png Binary files differnew file mode 100644 index 0000000..2f48ef2 --- /dev/null +++ b/doc/images/qsloc3.png diff --git a/doc/images/qslogFormat.png b/doc/images/qslogFormat.png Binary files differnew file mode 100644 index 0000000..bc78bed --- /dev/null +++ b/doc/images/qslogFormat.png diff --git a/doc/images/qslog_spreadsheet_example.png b/doc/images/qslog_spreadsheet_example.png Binary files differnew file mode 100644 index 0000000..4b23801 --- /dev/null +++ b/doc/images/qslog_spreadsheet_example.png diff --git a/doc/index.html b/doc/index.html new file mode 100644 index 0000000..839d364 --- /dev/null +++ b/doc/index.html @@ -0,0 +1,2751 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html> +<head> + <title>mod_qos</title> +<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1" /> +<meta name="author" content="Pascal Buchbinder" /> +<meta name="KeyWords" content="mod_qos, Quality of Service, Apache Web Server, Throttling, Web application security, WAF, Open Source Software, Secure Reverse Proxy, Denial of Service Prevention, DoS, DDoS" /> +<link rel="shortcut icon" href="favicon.ico" /> +<style TYPE="text/css"> +<!-- + body { + background-color: white; + color: black; + font-family: sans-serif, arial, verdana; + font-weight: normal; + text-align: left; + } + a:link { color:#00673F; text-decoration:none; } + a:visited { color:#00673F; text-decoration:none; } + a:focus { color:black; text-decoration:underline; } + a:hover { color:black; text-decoration:underline; } + a:active { color:black; text-decoration:underline; } + li { margin: 4px 0; } + syntax { font-family: monospace; font-size: 14; line-height: 1.8; } + .btable { font-size:0.75em; } + .prept { font-size:0.75em; } +--> +</style> +</head> +<body> +<!-- + + Quality of service module for Apache Web Server. + + See http://mod-qos.sourceforge.net/ for further details. + + Copyright (C) 2023 Pascal Buchbinder + + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + +--> +<table> +<tbody> +<tr><td><img src="images/mod_qos.gif" alt="mod_qos" title="mod_qos" /></td> + <td style="vertical-align: bottom;"><h1>mod_qos</h1></td> +</tr> +<tr><td> </td> + <td> +<p> +In computer networking, the term quality of service (QoS) describes +resource management rather than the quality of a service. +Quality of service implements control mechanisms to provide +different priority to different users, applications, and data +connections. It is used to guarantee a certain level of performance +to data resources. The term quality of service is often used +in the field of wide area network protocols (e.g. ATM) and +telephony (e.g. VoIP), but rarely in conjunction with web applications. +<b>mod_qos is a quality of service module for the Apache web server</b> +implementing control mechanisms that can provide different levels of +priority to different HTTP requests. +</p> +<p> +But why do you need quality of service for a web application? Well, +web servers require threads and processes to serve HTTP requests. +Each TCP connection to the web server occupies one of these threads +respectively processes. Sometimes a server gets too busy to serve +every request due to the lack of free processes or threads. Another +parameter requiring control by mod_qos is the available bandwidth: +all clients communicate to the server over a network link with +limited bandwidth. Overfilling the link results in network +congestion and poor performance. +</p> +<p> +Example situations where web applications require QoS: +<ul> +<li> +More resources are consumed if request processing by an application +takes a long time, e.g. when request processing includes +time consuming database queries. +</li> +<li> +Oversubscription of link capabilities due to many concurrent +clients uploading or downloading data. +</li> +<li> +Penetration of the web server by attackers (DoS).<!-- line is a MARKER --> +</li> +</ul> +</p> +<p> +mod_qos may be used to determine which requests should be served and +which shouldn't in order to avoid resource oversubscription. The +module collects different attributes such as the request URL, +HTTP request and response headers, the IP source address, country codes, +the HTTP response code, history data (based on user session and source +IP address), the number of concurrent requests to the +server (total or requests having similar attributes), the number +of concurrent TCP connections (total or from a single source IP), +and so forth.</p> +<p> +<a name="rules"></a> +The <u><a href="glossary.html#rules">rules</a></u> you want to configure +are defined by the +<u><a href="glossary.html#directives">module's directives</a></u>. Every rule +reads attributes from different sources and using its own counters to +store their status. +</p> +<p> +Counteractive measures to enforce the defined rules are: request +blocking, dynamic timeout adjustment, request delay, response +throttling, and dropping of TCP connections. +</p> +<p> +The <a title="change log" href="CHANGES.txt">current release</a> of the mod_qos +module implements various control mechanisms: +<ul> +<li type=square> +The maximum number of <a href="glossary.html#concurrency">concurrent</a> +requests to a location/resource (URL) +or virtual host. +</li> +<li type=square> +Limitation of the <a href="glossary.html#throughput">bandwidth</a> such as the maximum +allowed number of requests per second to an URL or the maximum/minimum of downloaded +kbytes per second. +</li> +<li type=square> +Limits the number of request <a href="glossary.html#requestPerSecond">events per second</a> (special request conditions). +</li> +<li type=square> +Limits the number of request <a href="glossary.html#repeat">events within a defined +period of time</a>. +</li> +<li type=square> +It can also detect very important persons (VIP) which may access the +web server without or with fewer restrictions. +</li> +<li type=square> +Generic request line and header filter to deny unauthorized operations. +</li> +<li type=square> +Request body data limitation and filtering (requires +<a href="http://parp.sourceforge.net">mod_parp <img src="images/link.png"/></a>). +</li> +<li type=square> +Limits the number of request events for individual clients (IP). +</li> +<li type=square> +Limitations on the TCP connection level, e.g., the maximum number of +allowed connections from a single IP source address or dynamic +keep-alive control. +</li> +<li type=square> +Prefers known IP addresses when server runs out of free TCP connections. +</li> +<li type=square> +Serialization of requests. +</li> +</ul> +</p> +<hr> +</td></tr> +<tr><td> </td><td style="background-color: #E2EDE2"> +<p align="center"> +<br> +mod_qos is an open source software licensed under the +<a href="LICENSE.txt">Apache License</a>. You can download the latest release at +<a href="https://sourceforge.net/projects/mod-qos/files/">SourceForge.net</a>. +<br><br> +</p> +</td></tr> +<tr><td> </td><td> +<hr> +<p> +More information about mod_qos: +<ul> +<li><a href="#build">Build</a></li> +<!-- DIST START --> +<li><a href="#source">Source Code</a></li> +<li><a href="CHANGES.txt">Changes</a></li> +<!-- DIST END --> +<li><a href="#configuration">Configuration</a></li> +<ul> +<li><a href="#requestlevelcontrol">Request Level Control</a></li> +<li><a href="#statuscode">Status Code and Error Page</a></li> +<li><a href="#privilegedusers">Privileged Users</a></li> +<li><a href="#variables">Variables</a></li> +<li><a href="#conditionalrules">Conditional Rules</a></li> +<li><a href="#eventcontrol">Events</a></li> +<li><a href="#filter">Request Level, Generic Filter</a></li> +<li><a href="#connectionlevelcontrol">Connection Level Control</a></li> +<li><a href="#clientlevelcontrol">Client Level Control</a></li> +</ul> +<li><a href="#messages">Log Messages</a></li> +<ul> +<li><a href="#errorlog">Error Log</a></li> +<li><a href="#accesslog">Access Log</a></li> +<li><a href="#requeststatistics">Request Statistics</a></li> +<li><a href="#statusviewer">Status Viewer</a></li> +<li><a href="#webconsole">Web Console</a></li> +<li><a href="#utilities">Utilities</a></li> +</ul> +<li><a href="#usecases">Sample Use Cases</a></li> +</ul> +</p> + +<hr> +<a name="build"></a> +<h2>Build</h2> +<p> +mod_qos requires OpenSSL, PCRE, threading and shared memory support. +mod_qos is designed to be used with Apache's +<a href="http://httpd.apache.org/docs/current/mod/worker.html">MPM worker <img src="images/link.png"/></a> +binaries but works, with some restrictions, also with other Apache 2.4 multi-processing modules. +The module is optimized to be used in a +<a href="http://httpd.apache.org/docs/current/mod/mod_proxy.html">reverse proxy +<img src="images/link.png"/></a> server.<p> +<p> +<small><i>Notes:<br> You should choose the <a href="https://httpd.apache.org/docs/current/mod/worker.html">worker MPM <img src="images/link.png"/></a> +if you intend to use any <a href="#connectionlevelcontrol">connection level control</a> directive. <br> + If you decide to use <a href="https://httpd.apache.org/docs/current/howto/http2.html">HTTP/2 <img src="images/link.png"/></a>, +you should only use the <a href="#requestlevelcontrol">request level control</a> directives +as mod_qos works for the hypertext transfer protocol +version 1.0 and 1.1 (RFC1945/RFC2616) only. +</i></small> +</p> +<p> +You can compile the module using +<code><a href="http://httpd.apache.org/docs/current/programs/apxs.html">apxs <img src="images/link.png"/></a></code>. +Your httpd binary must support dynamically loaded objects +(DSO). Verify this by checking the availability of mod_so: The command +<code>httpd -l</code> must list the mod_so.c module. +The following command compiles the module and installs mod_qos into the +server's modules directory. +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +<pre> +cd mod_qos-11.74/apache2 +apxs -i -c mod_qos.c -lcrypto -lpcre2-8 +cd ../.. +</pre> +</td></tr> +</table> +If the necessary header files of OpenSSL, PCRE, etc. cannot be found, add +the <code>-I</code> option to the <code>apxs</code> command to specify +the directory where header files can be found and if any of the required +libraries cannot be found (may happen if you use mod_qos without mod_ssl), +add the <code>-L</code> option to specify the directory where libraries +can be found. +<br> +<small><i>Note: you may customize the code using the following preprocessor directives:</i></small> +<table class="prept" width="780px"> +<tr> + <td> </td> + <td bgcolor="#E2EDE2">Name</td> + <td bgcolor="#E2EDE2">Description</td> + <td bgcolor="#E2EDE2">Default</td> +</tr> +<tr> + <td> </td> + <td>QS_MOD_EXT_HOOKS</td> + <td>Enables the optional hooks defined in mod_qos.h</td> + <td><i>not set</i></td> +</tr> +<tr> + <td> </td> + <td>QSLOG_CLID</td> + <td>Defines the environment variable which shall be used for the "user tracking id" (U) +within the format string used by the <a href="#QSLog"><code>QSLog</code></a> directive.</td> + <td>mod_qos_user_id</td> +</tr> +<tr> + <td> </td> + <td>QSLOG_EVENT</td> + <td>Defines the environment variable which shall be used for the "event" (Q) +within the format string used by the <a href="#QSLog"><code>QSLog</code></a> directive.</td> + <td>Event</td> +</tr> +<tr> + <td> </td> + <td>QSLOG_AVERAGE</td> + <td>Defines the environment variable which shall be used for the "average" (a) +within the format string used by the <a href="#QSLog"><code>QSLog</code></a> directive.</td> + <td>QS_AllConn</td> +</tr> +<tr> + <td> </td> + <td>QS_LOG_REPEAT</td> + <td>Counter used to define how many repetitive messages are summarized.</td> + <td>20</td> +</tr> +<tr> + <td> </td> + <td>QS_REQ_RATE_TM</td> + <td>Default for the <a href="#QS_SrvSampleRate"><code>QS_SrvSampleRate</code></a> directive.</td> + <td>5</td> +</tr> +<tr> + <td> </td> + <td>QS_EXTRA_MATCH_LIMIT</td> + <td>Match limit field used for PCRE data processing.</td> + <td>1500</td> +</tr> +</table> +</p> +<p> +The <a href="#utilities">support tools</a> may be built (at least on some +Linux platforms) using the GNU autotools. Some of these +utilities require third-party libraries such as apr, apr-util, PCRE2, +libpng, and OpenSSL. +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +<pre> +cd mod_qos-11.74/tools +./configure +make +</pre> +</td></tr> +</table> +<!-- +<small><i>Note: +If you have a different version of <code>aclocal</code> or +<code>automake</code> on your system (you get a message like +"aclocal-x.y is missing on your system"), edit the configure +script and change the <code>am__api_version</code> variable +to match the version you have installed (<code>aclocal --versions</code> +shows you which version this is). +</i></small> +--> +</p> + +<!-- DIST START --> + +<a name="source"></a> +<h2>Source Code</h2> +<p> +<a href="../apache2">mod_qos</a> is available for Apache version 2.4. +</p> + +<!-- DIST END --> +<!-- CONFIGURATION --> + +<a name="configuration"></a> +<h2>Configuration</h2> +<p>Configuration is mostly done on a per-server basis (except the +<a href="#filter">generic request</a> filter and a few other directives). +<a href="glossary.html#directives">Directives</a> within a virtual +host are merged with the settings in the global configuration. +</p> +<p> +The <code><a href="#QS_SrvMinDataRate">QS_SrvMinDataRate</a></code>, +<code><a href="#QS_SrvRequestRate">QS_SrvRequestRate</a></code>, +<code><a href="#QS_RequestHeaderFilterRule">QS_RequestHeaderFilterRule</a></code>, +<code><a href="#QS_ResponseHeaderFilterRule">QS_ResponseHeaderFilterRule</a></code>, +and all <code><a href="#clientlevelcontrol">QS_Client*</a></code> +directives may be used outside of virtual host configurations only. +</p> +<p> +<a name="QS_LogOnly"></a> +The <code>QS_LogOnly on</code> directive may be used to put mod_qos +into a permissive mode where rule violations are logged only but +requests/connections are not blocked. This may be used for test purposes.<br> +Should not be activated if you are using any +<a href="glossary.html#throughput">throughput control</a> +directive (open loop). +</p> + +<p> +<a name="requestlevelcontrol"></a> +<h3>Request Level Control</h3> +The module features directives to control server access +on a per-URL level - basically the main function of mod_qos. <br> +Only one <code>QS_Loc*</code> rule (URL string or +regular expression) of each type is evaluated per request where +regular expression rules (*Match) have higher priority +than the rules using a literal URL-string. A +<code>QS_LocRequestLimit*</code> rule may be used in parallel to a +<code>QS_LocRequestPerSecLimit*</code> and/or +<code>QS_LocKBytesPerSecLimit*</code> rule if they use the very +same URL string or regular expression. +<ul> +<li> +<a name="QS_LocRequestLimitMatch"></a> +<syntax>QS_LocRequestLimitMatch <regex> <number></syntax><br> +Defines the number of <a href="glossary.html#concurrency">concurrent</a> +requests for the specified request pattern (path and query). +The rule with the lowest number of allowed concurrent connections has the +highest priority if multiple expressions match the request. +By default, no limitations are active. +</li> +<li> +<a name="QS_LocRequestPerSecLimitMatch"></a> +<syntax>QS_LocRequestPerSecLimitMatch <regex> <number></syntax><br> +Defines the allowed number of <a href="glossary.html#requestPerSecond">requests per second</a> +to the URL (path and query) pattern. Requests are limited by +adding a delay to each request (linear). The delay calculation is based on +an average request rate measurement using a sampling rate of 10 seconds. +By default, no limitation is active. This directive should be used in +conjunction with <code><a href="#QS_LocRequestLimitMatch">QS_LocRequestLimitMatch</a></code> +only (you must use the very same regex pattern with the +<code><a href="#QS_LocRequestPerSecLimitMatch">QS_LocRequestPerSecLimitMatch</a></code> +and <code><a href="#QS_LocRequestLimitMatch">QS_LocRequestLimitMatch</a></code> +directive) to avoid too many concurrent requests. +</li> +<li> +<a name="QS_LocKBytesPerSecLimitMatch"></a> +<syntax>QS_LocKBytesPerSecLimitMatch <regex> <number></syntax><br> +Defines the allowed download <a href="glossary.html#throughput">bandwidth</a> to the location +matching the defined URL (path and query) pattern. Responses are slowed down by +adding a delay to each response (every 8kbytes). Bandwidth calculation +is based on measuring the transferred data. +By default, no limitation is active. This directive should be used +in conjunction with <code><a href="#QS_LocRequestLimitMatch">QS_LocRequestLimitMatch</a></code> +only (you must use the very same regex pattern with the +<code><a href="#QS_LocKBytesPerSecLimitMatch">QS_LocKBytesPerSecLimitMatch</a></code> and +<code><a href="#QS_LocRequestLimitMatch">QS_LocRequestLimitMatch</a></code> directive) +to avoid too many concurrent requests. +</li> +<li> +<a name="QS_LocRequestLimit"></a> +<syntax>QS_LocRequestLimit <location> <number></syntax><br> +Defines the number of <a href="glossary.html#concurrency">concurrent</a> +requests for the specified location (applied to the parsed path). +By default, no limitations are active for locations. Has lower priority than +<code><a href="#QS_LocRequestLimitMatch">QS_LocRequestLimitMatch</a></code> +directives. +</li> +<li> +<a name="QS_LocRequestLimitDefault"></a> +<syntax>QS_LocRequestLimitDefault <number></syntax><br>Defines the +default limitation for the maximum of concurrent requests per-location +for those locations not defined by any +<code><a href="#QS_LocRequestLimit">QS_LocRequestLimit</a></code> +directive. It could also be used to limit the number of concurrent +requests to a virtual host. +</li> +<li> +<a name="QS_LocRequestPerSecLimit"></a> +<syntax>QS_LocRequestPerSecLimit <location> <number></syntax><br> +Defines the allowed number of <a href="glossary.html#requestPerSecond">requests per second</a> +to a location, similar to the +<a href="#QS_LocRequestPerSecLimitMatch"><code>QS_LocRequestPerSecLimitMatch</code></a> +directive. The maximum number of requests is limited by adding a delay to +each request (linear, each request gets the same delay). By default, +no limitation is active. +This directive should be used in conjunction with +<code><a href="#QS_LocRequestLimit">QS_LocRequestLimit</a></code> only (you +must use the same location for both directives) to avoid too many +concurrent requests.. Has lower priority than +<code><a href="#QS_LocRequestPerSecLimitMatch">QS_LocRequestPerSecLimitMatch</a></code>. +</li> +<li> +<a name="QS_LocKBytesPerSecLimit"></a> +<syntax>QS_LocKBytesPerSecLimit <location> <number></syntax><br> +Throttles the download <a href="glossary.html#throughput">bandwidth</a> to the defined +kbytes per second. Works similar as the +<a href="#QS_LocKBytesPerSecLimitMatch"><code>QS_LocKBytesPerSecLimitMatch</code></a> +directive slowing down HTTP responses by adding a delay to each response. +By default, no limitation is active. This directive should be used in +conjunction with <code><a href="#QS_LocRequestLimit">QS_LocRequestLimit</a></code> only +(you must use the same location for both directives) to avoid too many +concurrent requests.. Has lower priority than +<code><a href="#QS_LocKBytesPerSecLimitMatch">QS_LocKBytesPerSecLimitMatch</a></code>. +</li> +</ul> + +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +Sample configuration:<br> +<pre> +# maximum number of active TCP connections is limited to 512 +MaxClients 512 + +# limits concurrent requests to the locations: +# - /app/a max. 200 concurrent requests +# - /app/b and /app/c (together) max. 300 concurrent requests +# - /images max. 100 concurrent requests +<a href="#QS_LocRequestLimit">QS_LocRequestLimit</a> /app/a 200 +<a href="#QS_LocRequestLimitMatch">QS_LocRequestLimitMatch</a> ^(/app/b/|/app/c/).*$ 300 +<a href="#QS_LocRequestLimit">QS_LocRequestLimit</a> /images 100 +# limits download bandwidth to 5Mbit/sec (resp. 640kbytes/sec) +# for downloads from /app/a: +<a href="#QS_LocKBytesPerSecLimit">QS_LocKBytesPerSecLimit</a> /app/a 640 +</pre> +</td></tr> +</table> +<br> + + +<a name="statuscode"></a> +<h3>Status Code and Error Page</h3> +The <code>QS_Error*</code> directives are used to control the response +given to clients whose requests have been denied. +<ul> +<li> +<a name="QS_ErrorPage"></a> +<syntax>QS_ErrorPage <URL></syntax><br>Defines an error page to be +returned when a request is denied. The defined URL must be a (S)HTML +document accessible by the client. +You may enable <a href="glossary.html#ssi">server-side includes (SSI)</a> +in order to present detailed error messages based on the +<a href="#errorlog">error codes</a> provided by mod_qos.<br> +Alternatively, a HTTP redirect (302) to a dedicated error page may be +defined using an absolute URL defining schema, hostname, and path. +</li> +<li> +<a name="QS_ErrorResponseCode"></a> +<syntax>QS_ErrorResponseCode <code></syntax><br>Defines the HTTP +response code which is used when a request is denied. Requests denied +at connection level usually get a HTTP 500 response code (ignoring +the settings of the <code>QS_ErrorResponseCode</code> and +<code><a href="#QS_ErrorPage">QS_ErrorPage</a></code> directives).<br> +Default (no custom error code or page defined) codes are:<br> + 400: if a request has no valid URL.<br> + 403: for requests denied by a <code><a href="#filter">QS_Deny*</a></code>, +<code><a href="#filter">QS_Permit*</a></code> or +<code><a href="#QS_RequestHeaderFilter">QS_RequestHeaderFilter</a></code> +directive.<br> + 413: when limiting the max. body data length by the +<code><a href="#QS_LimitRequestBody">QS_LimitRequestBody</a></code> directive.<br> + 500: for requests denied by any other directive.<br> +</li> +</ul> + +<a name="privilegedusers"></a> +<h3>Privileged Users</h3> +Additional directives are used to identify VIPs (very important persons) +and to control the session life time and its cookie format. VIP users have +privileged access and less QoS restrictions than ordinary users. <br><br> +VIP information is stored and evaluated at different levels: +<ul> +<li> +Session: VIP identification is stored using a HTTP +session cookie. mod_qos starts a new session when detecting a HTTP +response header (the header name is defined by the +<code><a href="#QS_VipHeaderName">QS_VipHeaderName</a></code> +directive). Alternatively, a new session is started when detecting an +authenticated user, see <code><a href="#QS_VipUser">QS_VipUser</a></code>. +The <code><a href="#QS_Session">QS_Session*</a></code> +directives are used to set session attributes. +</li> +<li> +Request: The <code><a href="#QS_VipRequest">QS_VipRequest</a></code> +process environment may be evaluated by mod_qos rules. This +variable is set automatically when receiving a valid mod_qos +session cookie. The <code><a href="#QS_VipRequest">QS_VipRequest</a></code> +variable may also be set by configuration using a <code><a href="#QS_SetEnvIf">QS_SetEnvIf*</a></code> +or <code><a href="http://httpd.apache.org/docs/current/mod/mod_setenvif.html#setenvif">SetEnvIf <img src="images/link.png"/></a></code> +directive. VIP status lasts for the particular +request only. +</li> +<li> +Client IP address: VIP identification may be stored at the server side +on a per-client IP address basis. +The <code><a href="#QS_VipIPHeaderName">QS_VipIPHeaderName</a></code>, +<code><a href="#QS_VipHeaderName">QS_VipHeaderName</a></code>, +<code><a href="#QS_VipIPUser">QS_VipIPUser</a></code>, and +<code><a href="#QS_VipUser">QS_VipUser</a></code> directives are used +to define when an IP address should be marked as a VIP user. +</li> +</ul> + +Directives: + +<ul> +<li> +<a name="QS_VipHeaderName"></a> +<syntax>QS_VipHeaderName <header name>[=<regex>] [drop]</syntax><br> +Defines an HTTP response header which marks a user as a VIP. mod_qos creates +a session for this user by setting a cookie, e.g., after successful user +authentication. +Tests optionally its value against the provided regular expression. +Specify the action 'drop' if you want mod_qos to remove this +control header from the HTTP response. +</li> +<li> +<a name="QS_VipIPHeaderName"></a> +<syntax>QS_VipIPHeaderName <header name>[=<regex>] [drop]</syntax><br> +Defines an HTTP response header which marks a client source IP address as +a VIP. +Tests optionally its value against the provided regular expression. +Specify the action 'drop' if you want mod_qos to remove this +control header from the HTTP response. +</li> +<li> +<a name="QS_VipUser"></a> +<syntax>QS_VipUser</syntax><br> +Creates a VIP session for users which have been authenticated by the +Apache server, e.g., by the standard mod_auth* modules. +It works similar to the <code><a href="#QS_VipHeaderName">QS_VipHeaderName</a></code> directive. +</li> +<li> +<a name="QS_VipIPUser"></a> +<syntax>QS_VipIPUser</syntax><br> +Marks a source IP address as a VIP if the user has been authenticated by the +Apache server, e.g. by the standard mod_auth* modules. It works similar to +the <code><a href="#QS_VipIPHeaderName">QS_VipIPHeaderName</a></code> directive. +</li> +<li> +<a name="QS_Session"></a> +<a name="QS_SessionTimeout"></a> +<syntax>QS_SessionTimeout <seconds></syntax><br> +Defines the session life time for a VIP. It is only used for session +based (cookie) VIP identification (not for IP based). +Default is 3600 seconds. +</li> +<a name="QS_SessionCookieName"></a> +<li> +<syntax>QS_SessionCookieName <name></syntax><br> +A cookie is used to identify requests coming from a user which has +been identified as a VIP. This directive defines a custom cookie name +for the mod_qos session cookie. Default is MODQOS. +</li> +<li> +<a name="QS_SessionCookiePath"></a> +<syntax>QS_SessionCookiePath <path></syntax><br> +Defines the cookie path. Default is "/". +</li> +<li> +<a name="QS_SessionKey"></a> +<syntax>QS_SessionKey <string></syntax><br> +Secret key used for cookie encryption. This key must be defined +when using the same session cookie for multiple web servers +(load balancing) or the sessions should survive a server restart. +By default, a random key is used which changes every server restart. +</li> +</ul> + +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +Sample configuration:<br> +<pre> +<a href="#QS_ErrorPage">QS_ErrorPage</a> /error-docs/qs_error.html + +# restricts max concurrent requests for any location which has no +# individual rule: +<a href="#QS_LocRequestLimitDefault">QS_LocRequestLimitDefault</a> 200 + +# limits access to *.gif files to 100 concurrent requests: +<a href="#QS_LocRequestLimitMatch">QS_LocRequestLimitMatch</a> "^.*\.gif$" 100 + +# limits concurrent requests to the locations /images and /app/a: +<a href="#QS_LocRequestLimit">QS_LocRequestLimit</a> /images 100 +<a href="#QS_LocRequestLimit">QS_LocRequestLimit</a> /app/a 300 +# limits download bandwidth to 5Mbit/sec: +<a href="#QS_LocKBytesPerSecLimit">QS_LocKBytesPerSecLimit</a> /app/a 640 + +# two locations (/app/b and /app/c) representing a single application: +<a href="#QS_LocRequestLimitMatch">QS_LocRequestLimitMatch</a> "^(/app/b/|/app/c/).*$" 300 + + +# allows the application to nominate VIP users by sending a +# "mod-qos-vip" HTTP response header: +<a href="#QS_VipHeaderName">QS_VipHeaderName</a> mod-qos-vip +<a href="#QS_SessionKey">QS_SessionKey</a> na&5san-sB.F4_0a=%VBEBahXT1 +</pre> +</td></tr> +</table> +<br> + +<a name="privilegedusers_list"></a> +The following table shows if a rules may be deactivated for VIPs: +<table class="btable" width="280px"> +<tr><td>QS_ClientEventBlockCount</td><td>no</td></tr> +<tr><td>QS_ClientEventLimitCount</td><td>no</td></tr> +<tr><td>QS_ClientEventPerSecLimit</td><td>no</td></tr> +<tr><td>QS_ClientEventRequestLimit</td><td>no</td></tr> +<tr><td>QS_ClientPrefer</td><td>yes</td></tr> +<tr><td>QS_ClientSerialize</td><td>no</td></tr> +<tr><td>QS_ClientGeoCountryPriv</td><td>no</td></tr> +<tr><td>QS_CondLocRequestLimitMatch</td><td>yes</td></tr> +<tr><td>QS_CondEventLimitCount</td><td>no</td></tr> +<tr><td>QS_CondClientEventLimitCount</td><td>no</td></tr> +<tr><td>QS_DenyQueryBody</td><td>no</td></tr> +<tr><td>QS_PermitUriBody</td><td>no</td></tr> +<tr><td>QS_DenyEvent</td><td>no</td></tr> +<tr><td>QS_DenyPath</td><td>no</td></tr> +<tr><td>QS_DenyQuery</td><td>no</td></tr> +<tr><td>QS_DenyRequestLine</td><td>no</td></tr> +<tr><td>QS_EventKBytesPerSecLimit</td><td>yes</td></tr> +<tr><td>QS_EventPerSecLimit</td><td>yes</td></tr> +<tr><td>QS_EventRequestLimit</td><td>no</td></tr> +<tr><td>QS_EventLimitCount</td><td>no</td></tr> +<tr><td>QS_InvalidUrlEncoding</td><td>no</td></tr> +<tr><td>QS_LimitRequestBody</td><td>no</td></tr> +<tr><td>QS_LocKBytesPerSecLimit(Match)</td><td>yes</td></tr> +<tr><td>QS_LocRequestLimit(Match)</td><td>yes</td></tr> +<tr><td>QS_LocRequestPerSecLimit(Match)</td><td>yes</td></tr> +<tr><td>QS_MileStone</td><td>no</td></tr> +<tr><td>QS_RedirectIf</td><td>no</td></tr> +<tr><td>QS_PermitUri</td><td>no</td></tr> +<tr><td>QS_RequestHeaderFilter</td><td>no</td></tr> +<tr><td>QS_ResponseHeaderFilter</td><td>no</td></tr> +<tr><td>QS_SrvMaxConn</td><td>yes</td></tr> +<tr><td>QS_SrvMaxConnClose</td><td>no</td></tr> +<tr><td>QS_SrvMaxConnPerIP</td><td>yes*</td></tr> +<tr><td>QS_SrvMinDataRate</td><td>yes*</td></tr> +<tr><td>QS_SrvSerialize</td><td>no</td></tr> +<tr><td> </td><td> </td></tr> +</table> +<small><i>Notes:<br> + Directives marked by "*" allow you to disable VIP support.<br> + Event based or conditional rules may evaluate the +<a href="#QS_VipRequest">QS_VipRequest</a> and +<a href="#QS_IsVipRequest">QS_IsVipRequest</a> variables to decide +if the rule should be applied.</i></small> +<br> +<a name="variables"></a> +<h3>Variables</h3> +<a href="glossary.html#variables">Environment variables</a> are used on a per +request level and implement additional control mechanisms. Variables may be set +using the standard Apache module +<a href="http://httpd.apache.org/docs/current/mod/mod_setenvif.html">mod_setenvif <img src="images/link.png"/></a> or +<a href="http://modsetenvifplus.sourceforge.net/">mod_setenvifplus <img src="images/link.png"/></a>. +See also the <a href="#eventcontrol"> +<code><a href="#QS_SetEnvIf">QS_SetEnvIf*</a></code></a> directives in order to combine multiple +variables to form new variables interpreted by mod_qos rules. <br> +<br> +These are the variables recognized by mod_qos: +<ul> +<li> +<a name="QS_ErrorPage_Var"></a> +<syntax>QS_ErrorPage=<URL></syntax><br> +Defines the error page overriding the setting made by the +<code><a href="#QS_ErrorPage">QS_ErrorPage</a></code> directive. +</li> +<li> +<a name="QS_VipRequest"></a> +<syntax>QS_VipRequest=yes</syntax><br> +Disables some restrictions for this request (see <a href="#privilegedusers">privileged Users</a>). +Requires the definition of a VIP header using the +<code><a href="#QS_VipHeaderName">QS_VipHeaderName</a></code> directive +(this activates VIP verification). However, such an event does +not create a VIP session. The user has the VIP status only for +a single request. <br>The variable is set by mod_qos when +receiving a valid VIP <a href="#QS_SessionCookieName">session cookie</a>. +</li> +<li> +<a name="QS_KeepAliveTimeout"></a> +<syntax>QS_KeepAliveTimeout=<seconds></syntax><br> +Applies dynamic connection keep-alive settings overriding the Apache +<code><a href="http://httpd.apache.org/docs/current/mod/core.html#keepalivetimeout">KeepAliveTimeout <img src="images/link.png"/></a></code> directive settings. +</li> +<li> +<a name="QS_MaxKeepAliveRequests"></a> +<syntax>QS_MaxKeepAliveRequests=<number></syntax><br> +Applies dynamic connection keep-alive settings overriding the Apache +<code><a href="http://httpd.apache.org/docs/current/mod/core.html#maxkeepaliverequests">MaxKeepAliveRequests <img src="images/link.png"/></a></code> directive settings. +</li> +<li> +<a name="QS_Timeout"></a> +<syntax>QS_Timeout=<seconds></syntax><br> +Alters the I/O timeout (while reading the request body / writing the response) +of the current request overriding the Apache +<code><a href="http://httpd.apache.org/docs/current/mod/core.html#timeout">TimeOut <img src="images/link.png"/></a></code> +directive settings. +</li> +<li> +<a name="QS_Set_DSCP"></a> +<syntax>QS_Set_DSCP=<value></syntax><br> +Variable used to set the IP differentiated services code points +(DiffServ / RFC 2474). This allows you to classify the network +traffic when sending the response data to the client. "value" +represents the 6-bit DSCP field as a decimal number (0 to 63).<br> +Commonly used values: +<small> +<table class="btable"> +<tr><td>DSCP</td><td>Class</td><td> </td> <td>DSCP</td><td>Class</td></tr> +<tr><td>0</td><td>none</td><td> </td> <td>8</td><td>Class selector 1</td></tr> +<tr><td>10</td><td>Assured forwarding 11</td><td> </td> <td>12</td><td>Assured forwarding 12</td></tr> +<tr><td>14</td><td>Assured forwarding 13</td><td> </td> <td>16</td><td>Class selector 2</td></tr> +<tr><td>18</td><td>Assured forwarding 21</td><td> </td> <td>20</td><td>Assured forwarding 22</td></tr> +<tr><td>22</td><td>Assured forwarding 23</td><td> </td> <td>24</td><td>Class selector 3</td></tr> +<tr><td>26</td><td>Assured forwarding 31</td><td> </td> <td>28</td><td>Assured forwarding 32</td></tr> +<tr><td>30</td><td>Assured forwarding 33</td><td> </td> <td>32</td><td>Class selector 4</td></tr> +<tr><td>34</td><td>Assured forwarding 41</td><td> </td> <td>36</td><td>Assured forwarding 42</td></tr> +<tr><td>38</td><td>Assured forwarding 43</td><td> </td> <td>40</td><td>Class selector 5</td></tr> +<tr><td>44</td><td>Voice admit</td><td> </td> <td>46</td><td>Expedited forwarding</td></tr> +<tr><td>48</td><td>Class selector 6</td><td> </td> <td>56</td><td>Class selector 7</td></tr> +</table> +</small> +</li> +<li> +<a name="QS_Delay"></a> +<syntax>QS_Delay=<milliseconds></syntax><br> +Defines a number of milliseconds to delay the request processing. +</li> +<li> +<a name="QS_Event"></a> +<syntax>QS_Event</syntax><br> +The variable processed by the <code><a href="#QS_ClientEventPerSecLimit">QS_ClientEventPerSecLimit</a></code> directive. +</li> +<li> +<a name="QS_Block"></a> +<syntax>QS_Block[=<number>]</syntax><br> +Variable processed by the <code><a href="#QS_ClientEventBlockCount">QS_ClientEventBlockCount</a></code> +directive.<br> +The optional <code>number</code> value defines the penalty points to +increase the counter (default is 1). +</li> +<li> +<a name="QS_Limit"></a> +<syntax>QS_Limit[=<number>]</syntax><br> +(Default) variable processed by the +<code><a href="#QS_ClientEventLimitCount">QS_ClientEventLimitCount</a></code> +directive.<br> +The optional <code>number</code> value defines the penalty points to +increase the counter (default is 1). +</li> +<li> +<a name="_Clear"></a> +<syntax>*_Clear</syntax><br> +The counter of the variable processed by the +<code><a href="#QS_ClientEventLimitCount">QS_ClientEventLimitCount</a></code> +directive is reset if you set the same variable suffixed by <code>_Clear</code>, +e.g. <code>QS_Limit_Clear</code>. +</li> +<li> +<a name="_Decrement"></a> +<syntax>*_Decrement</syntax><br> +The counter of the variable processed by the +<code><a href="#QS_EventLimitCount">QS_EventLimitCount</a></code>, +<code><a href="#QS_CondEventLimitCount">QS_CondEventLimitCount</a></code>, +<code><a href="#QS_ClientEventLimitCount">QS_ClientEventLimitCount</a></code>, +<code><a href="#QS_CondClientEventLimitCount">QS_CondClientEventLimitCount</a></code>, and +<code><a href="#QS_ClientEventBlockCount">QS_ClientEventBlockCount</a></code> +directives +is decremented by the value set in the same variable suffixed by <code>_Decrement</code>, +e.g. <code>QS_Limit_Decrement=1</code> decrements the value of the <code>QS_Limit</code> +variable of the corresponding +<code><a href="#QS_ClientEventLimitCount">QS_ClientEventLimitCount</a></code> +rule by 1. The variable is evaluated at the end of the request processing. +</li> +<li> +<a name="QS_Serialize"></a> +<syntax>QS_Serialize</syntax><br> +Variable processed by the <code><a href="#QS_ClientSerialize">QS_ClientSerialize</a></code> +directive. +</li> +<li> +<a name="QS_SrvSerialize_var"></a> +<syntax>QS_SrvSerialize</syntax><br> +Variable processed by the <code><a href="#QS_SrvSerialize">QS_SrvSerialize</a></code> +directive. +</li> +<li> +<a name="QS_Cond"></a> +<syntax>QS_Cond</syntax><br> +Variable processed by the <code><a href="#QS_CondLocRequestLimitMatch">QS_CondLocRequestLimitMatch</a></code>, +<code><a href="#QS_CondEventLimitCount">QS_CondEventLimitCount</a></code>, and +<code><a href="#QS_CondClientEventLimitCount">QS_CondClientEventLimitCount</a></code> directives. +</li> +<li> +<a name="QS_EventRequest"></a> +<syntax>QS_EventRequest</syntax><br> +Variable processed by the <code><a href="#QS_ClientEventRequestLimit">QS_ClientEventRequestLimit</a></code> directive. +</li> +</ul> + +Variables set by mod_qos which may be processed by conditional or event based +rules, e.g., +<code><a href="#QS_CondLocRequestLimitMatch">QS_CondLocRequestLimitMatch</a></code>: +<ul> +<li> +<a name="QS_SrvConn"></a> +<syntax>QS_SrvConn</syntax><br> +Number of <a href="glossary.html#concurrency">concurrent</a> +connections for this server/virtual host. Value is set +when using either the <code><a href="#QS_SrvMaxConn">QS_SrvMaxConn</a></code>, +<code><a href="#QS_SrvMinDataRate">QS_SrvMinDataRate</a></code>, +<code><a href="#QS_SrvMaxConnClose">QS_SrvMaxConnClose</a></code>, or +<code><a href="#QS_ClientGeoCountryDB">QS_ClientGeoCountryDB</a></code> +directive. <br> +<small><i>Note: value is calulcated when the client establishes the connection +and remains the same for all HTTP requests performed on this connection.</i></small> +</li> +<li> +<a name="QS_AllConn"></a> +<syntax>QS_AllConn</syntax><br> +Number of all concurrent connections for this Apache instance. Value is set +when using either the <code><a href="#QS_SrvMaxConn">QS_SrvMaxConn</a></code>, +<code><a href="#QS_SrvMinDataRate">QS_SrvMinDataRate</a></code>, +<code><a href="#QS_SrvMaxConnClose">QS_SrvMaxConnClose</a></code>, or +<code><a href="#QS_ClientGeoCountryDB">QS_ClientGeoCountryDB</a></code> +directive. <br> +<small><i>Note: value is calulcated when the client establishes the connection +and remains the same for all HTTP requests performed on this connection.</i></small> +</li> +<li> +<a name="QS_IPConn"></a> +<syntax>QS_IPConn</syntax><br> +Number of IP connections open from the current IP address. Variable is +available when using the <code><a href="#QS_SrvMaxConnPerIP">QS_SrvMaxConnPerIP</a></code> +directive. <br> +<small><i>Note: value is calulcated when the client establishes the connection +and remains the same for all HTTP requests performed on this connection.</i></small> +</li> +<li> +<a name="QS_ClientLowPrio"></a> +<syntax>QS_ClientLowPrio</syntax><br> +The variable is set for connections by clients which have been marked to be +processed with low priority, see <code><a href="#QS_ClientPrefer">QS_ClientPrefer</a></code>. +The variable's value is determined when the client opens a new connection and +its value represents the status flag of the tracked client attributes +(hexadecimal). <a href="#privilegedusers">VIP</a> status is ignored and +the variable is always set even the IP has been marked as being VIP. +</li> +<li> +<a name="QS_IsVipRequest"></a> +<syntax>QS_IsVipRequest</syntax><br> +Variable is set when detecting a <a href="#privilegedusers">VIP</a> request +(either by cookie, IP address status, valid user, etc.). May be used by +various event based directives. +</li> +<li> +<a name="_Counter"></a> +<syntax>*_Counter</syntax><br> +The counter values of the variables used by the +<code><a href="#QS_ClientEventLimitCount">QS_ClientEventLimitCount</a></code> +and <code><a href="#QS_EventLimitCount">QS_EventLimitCount</a></code> +directive are stored within the variable whose name is suffixed by +<code>_Counter</code>, e.g. <code>QS_Limit_Counter</code> when limiting +<code><a href="#QS_Limit">QS_Limit</a></code> events. +</li> +<li> +<a name="QS_ErrorNotes"></a> +<syntax>QS_ErrorNotes</syntax><br> +The error code (number only) of a mod_qos <a href="#errorlog">log message</a> +that has occurred during a request. +</li> +<li> +<a name="QS_Country"></a> +<syntax>QS_Country</syntax><br> +ISO 3166 country code of client IPv4 address. Only available if the +<a href="#QS_ClientGeoCountryDB">geographical database</a> file has been loaded.<br> +<small><i>Note: You may use the <code>QS_ClientIpFromHeader <header></code> +directive to override the client's IP address based on the value within the defined +HTTP request header (e.g., X-Forwarded-For) instead of taking the IP address of +the client which has opened the TCP connection.</i></small> +</li> +<!--<li> +<syntax>QS_RuleId</syntax><br> +ID of the matching <code>QS_Deny*</code> rule. +</li>--> +</ul> + +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +Sample of variable usage:<br> +<pre> +# privileged access for curl clients: +BrowserMatch "curl" <a href="#QS_VipRequest">QS_VipRequest</a>=yes + +# allows privileged access to a single resource: +SetEnvIf Request_URI /app/start.html <a href="#QS_VipRequest">QS_VipRequest</a>=yes + +# allows privileged access from a specified source address +# or source address range: +SetEnvIf Remote_Addr 172.18.3.32 <a href="#QS_VipRequest">QS_VipRequest</a>=yes +SetEnvIf Remote_Addr 192.168.10. <a href="#QS_VipRequest">QS_VipRequest</a>=yes + +# set keep-alive timeout for MSIE version 5.x browser to 65 seconds: +BrowserMatch "(MSIE 5\.)" <a href="#QS_KeepAliveTimeout">QS_KeepAliveTimeout</a>=65 + +# dynamic error page URL (per host error page): +SetEnvIf Host ^([a-zA-Z0-9_\.\-]+) <a href="#QS_ErrorPage">QS_ErrorPage</a>=/error-docs/$1.html +# external redirect to a sever hosting the error page: +SetEnvIf Request_URI /app <a href="#QS_ErrorPage">QS_ErrorPage</a>=http://your.server.name/error.html +</pre> +</td></tr> +</table> +<a name="QS_LogEnv"></a> +<small><i>Note: The <code>QS_LogEnv</code> directive can be used to enable environment variable logging. mod_qos +writes all environment variables which are set when entering a <a href="glossary.html#directives">handler</a> +to the log.</i></small> +<br> + +<a name="conditionalrules"></a> +<h3>Conditional Rules</h3> +Conditional rules are only enforced if the <code><a href="#QS_Cond">QS_Cond</a></code> +variable matches the specified pattern. +<ul> +<li> +<a name="QS_CondLocRequestLimitMatch"></a> +<syntax>QS_CondLocRequestLimitMatch <regex> <number> <condition></syntax><br> +Rule works similar to <a href="#QS_LocRequestLimitMatch"><code>QS_LocRequestLimitMatch</code></a> +but it is only enforced for requests whose <code><a href="#QS_Cond">QS_Cond</a></code> +variable matches the specified condition (regular expression). Every request +matching the defined pattern is counted, but the defined limitation is only +enforced for those requests matching the specified condition. <br> +Only one <code>QS_CondLocRequestLimitMatch</code> rule is evaluated per request. +</li> +<li> +<a name="QS_CondEventLimitCount"></a> +<syntax>QS_CondEventLimitCount <env-variable> <number> <seconds> <pattern></syntax><br> +Same as <code><a href="#QS_EventLimitCount">QS_EventLimitCount</a></code> but +requests are only blocked if the value of the +<code><a href="#QS_Cond">QS_Cond</a></code> +variable matches the defined pattern (regex). +</li> +<li> +<a name="QS_CondClientEventLimitCount"></a> +<syntax>QS_CondClientEventLimitCount <number> <seconds> <variable> <pattern></syntax><br> +Defines the maximum number of the specified environment variable +allowed within the defined time. Directive works similar as +<a href="#QS_ClientEventLimitCount"><code>QS_ClientEventLimitCount</code></a> +but requests are only blocked if the value of the <code><a href="#QS_Cond">QS_Cond</a></code> +variable matches the defined pattern (regex). Directive is allowed +in global server context only. <br> +</li> +</ul> + +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +Sample of conditional rules:<br> +<pre> +# set the conditional variable to spider if detecting a +# "slurp" or "googlebot" search engine: +BrowserMatch "slurp" <a href="#QS_Cond">QS_Cond</a>=spider +BrowserMatch "googlebot" <a href="#QS_Cond">QS_Cond</a>=spider + +# limits the number of concurrent requests to two applications +# (/app/b and /app/c) to 300 but does not allow access by a "spider" +# if the number of concurrent requests exceeds the limit of 10: +<a href="#QS_LocRequestLimitMatch">QS_LocRequestLimitMatch</a> "^(/app/b/|/app/c/).*$" 300 +<a href="#QS_LocRequestLimitMatch">QS_CondLocRequestLimitMatch</a> "^(/app/b/|/app/c/).*$" 10 spider +</pre> +</td></tr> +</table> + +<a name="eventcontrol"></a> +<h3>Events</h3> +mod_qos may control the frequency of "events". An event may be any +request attribute which can be represented by an +<a href="glossary.html#variables">environment variable</a>. +Such variables may be set by +<a href="http://httpd.apache.org/docs/current/mod/mod_setenvif.html">mod_setenvif <img src="images/link.png"/></a>, +<a href="http://modsetenvifplus.sourceforge.net/">mod_setenvifplus <img src="images/link.png"/></a>, or +by other Apache modules. +Please consider the <a href="glossary.html#directives">order of command execution</a> +to ensure that the necessary variables are set. + +<ul> +<li> +<a name="QS_EventRequestLimit"></a> +<syntax>QS_EventRequestLimit <env-variable>[=<regex>] <number></syntax><br> +Defines the number of <a href="glossary.html#concurrency">concurrent</a> events. +Directive works similar to +<code><a href="#QS_LocRequestLimit">QS_LocRequestLimit</a></code>, but +counts the requests having the same environment variable (and optionally +matching its value, too) rather than those that have the same URL pattern. +<br><small><i>Note: The counter's value is stored in the environment variable +QS_EventRequestLimit_<env-variable>_Counter. +</i></small> +</li> +<li> +<a name="QS_EventPerSecLimit"></a> +<syntax>QS_EventPerSecLimit [!]<env-variable> <number></syntax><br> +Defines how often requests may have the defined environment variable +(literal string) set. It measures the occurrences of the defined +environment variable on a <a href="glossary.html#requestPerSecond">request per seconds</a> +level and tries to limit this occurrence to the defined number. It works similar as +<a href="#QS_LocRequestPerSecLimit"><code>QS_LocRequestPerSecLimit</code></a>, +but counts only the requests with the specified variable (or without it +if the variable name is prefixed by a "!"). If a request matches +multiple events, the rule with the lowest bandwidth is applied. +Events are limited by adding a delay to each request causing an +event. +</li> +<li> +<a name="QS_EventKBytesPerSecLimit"></a> +<syntax>QS_EventKBytesPerSecLimit [!]<env-variable> <number></syntax><br> +Throttles the download <a href="glossary.html#throughput">bandwidth</a> +of all requests having the defined variable set to the defined +kbytes per second. Responses are slowed by adding a delay to each +response (every 8kbytes). The delay calculation +is based on an average request rate measurement. +By default, no limitation is active. +This directive should be used in conjunction with +<code><a href="#QS_EventRequestLimit">QS_EventRequestLimit</a></code> +only (you must use the same variable name for both directives) to avoid too many +concurrent requests. +</li> +<li> +<a name="QS_EventLimitCount"></a> +<syntax>QS_EventLimitCount <env-variable> <number> <seconds></syntax><br> +Defines the maximum <a href="glossary.html#repeat">number of events allowed within the defined time</a>. +Requests causing the event are denied when reaching this limitation for the specified time +(blocked at request level).<br> +<small><i>Notes:<ul> +<li>The current counter value is propagated to the process environment within +the variable <code><env-variable><a href="#_Counter">_Counter</a></code>.</li> +<li>See also <a href="#QS_CondEventLimitCount"><code>QS_CondEventLimitCount</code></a> +if you want to enforce a rule under certain conditions only.</li> +<li>The event counter can be decremented by setting the environment +<code><env-variable><a href="#_Decrement">_Decrement</a></code>.</li> +</ul> +</i></small> +</li> +</ul> +Mulpiple built-in directives may be used to set or detect events (additional +event variable processing could be configured using +<a href="http://httpd.apache.org/docs/current/mod/mod_setenvif.html">mod_setenvif <img src="images/link.png"/></a> or +<a href="http://modsetenvifplus.sourceforge.net/">mod_setenvifplus <img src="images/link.png"/></a>). +<ul> +<li> +<a name="QS_SetEnvIf"></a> +<syntax>QS_SetEnvIf [!]<env-variable1> [!]<env-variable2> [!]<env-variable=value></syntax><br> +Sets (or unsets) the environment "variable=value" (literal string) if variable1 (literal +string) AND variable2 (literal string) are set in the request environment +variable list (not case sensitive). This is used to combine multiple +variables to a new event type.<br> +This directive may be used on a per-server or +<a href="http://httpd.apache.org/docs/current/mod/core.html.en#location">location <img src="images/link.png"/></a> +basis. +</li> +<li> +<a name="QS_SetEnvIfMatch"></a> +<syntax>QS_SetEnvIf <env-variable1>=<regex> [!]<env-variable>=<value></syntax><br> +Sets the environment variable if the environment variable1's value +matches the defined regular expression. <code>$1</code>..<code>$9</code> +within the value and are replaced by parenthesized subexpressions +of the regular expression.<br> +This directive may be used on a per-server or +<a href="http://httpd.apache.org/docs/current/mod/core.html.en#location">location <img src="images/link.png"/></a> +basis. +</li> +<li> +<a name="QS_SetEnv"></a> +<syntax>QS_SetEnv <env-variable> <value></syntax><br> +Sets the defined variable with the value where the value string may contain +other environment variables surrounded by "${" and "}". The variable is only +set if all defined variables within the value have been resolved. +</li> +<li> +<a name="QS_SetEnvIfQuery"></a> +<syntax>QS_SetEnvIfQuery <regex> [!]<env-variable>[=<value>]</syntax><br> +Directive works quite similar to the +<code><a href="http://httpd.apache.org/docs/current/mod/mod_setenvif.html#setenvif">SetEnvIf <img src="images/link.png"/></a></code> +directive of the Apache module +<a href="http://httpd.apache.org/docs/current/mod/mod_setenvif.html">mod_setenvif <img src="images/link.png"/></a>, +but the specified regex is applied against the query string +portion of the request line. The directive recognizes +the occurrences of $1..$9 within value and replaces them +by the sub-expressions of the defined regex pattern.<br> +This directive may be used on a per-server or +<a href="http://httpd.apache.org/docs/current/mod/core.html.en#location">location <img src="images/link.png"/></a> +basis. +</li> +<a name="QS_SetEnvIfCmp"></a> +<li> +<syntax>QS_SetEnvIfCmp <env-variable1> eq|ne|gt|lt <env-variable2> [!]<env-variable>[=<value>]</syntax><br> +Sets the defined environment variable if the specified env-variables[1|2] +are numerical or alphabetically (case insensitive) equal (<code>eq</code>), +not equal (<code>ne</code>) greater (<code>gt</code>), or less (<code>lt</code>).<br> +This directive may be used on a per-<a href="http://httpd.apache.org/docs/current/mod/core.html.en#location">location <img src="images/link.png"/></a> +basis only.</li> +<li> +<a name="QS_SetEnvIfParp"></a> +<syntax>QS_SetEnvIfParp <regex> [!]<env-variable>[=<value>]</syntax><br> +Directive parsing the request payload using the Apache module +<a href="http://parp.sourceforge.net">mod_parp <img src="images/link.png"/></a>. It matches +the request URL query and the HTTP request message body data as well +(<code>application/x-www-form-urlencoded</code>, +<code>multipart/form-data</code>, and <code>multipart/mixed</code>) +and sets the defined process variable (quite similar to the +<code><a href="#QS_SetEnvIfQuery">QS_SetEnvIfQuery</a></code> directive). +The directive recognizes the occurrences of $1..$9 within +value and replaces them by the sub-expressions +of the defined regex pattern. This directive activates +mod_parp for every request to the virtual host. +You may deactivate mod_parp for selected requests using the +<code><a href="http://httpd.apache.org/docs/current/mod/mod_setenvif.html#setenvif">SetEnvIf <img src="images/link.png"/></a></code> +or <code><a href="http://modsetenvifplus.sourceforge.net/#SetEnvIfPlus">SetEnvIfPlus <img src="images/link.png"/></a></code> +directive: unset the variable "parp" to do so. +Important: request message body processing requires that the server +loads the whole request into its memory (at least twice the length +of the message). You should limit the allowed size of the HTTP +request message body using the <code><a href="#QS_LimitRequestBody">QS_LimitRequestBody</a></code> directive +when using <code><a href="#QS_SetEnvIfParp">QS_SetEnvIfParp</a></code>! +</li> +<li> +<a name="QS_SetEnvIfBody"></a> +<syntax>QS_SetEnvIfBody <regex> [!]<env-variable>[=<value>]</syntax><br> +Directive parsing the request body using the Apache module +<a href="http://parp.sourceforge.net">mod_parp <img src="images/link.png"/></a>. Specify the content +types to process using the mod_parp directive +<code><a href="http://parp.sourceforge.net/#PARP_BodyData">PARP_BodyData <img src="images/link.png"/></a></code> +and ensure that mod_parp is enabled using the +<code><a href="http://httpd.apache.org/docs/current/mod/mod_setenvif.html#setenvif">SetEnvIf <img src="images/link.png"/></a></code> +or <code><a href="http://modsetenvifplus.sourceforge.net/#SetEnvIfPlus">SetEnvIfPlus <img src="images/link.png"/></a></code> +directive. +You should limit the allowed size of HTTP requests message body +using the <code><a href="#QS_LimitRequestBody">QS_LimitRequestBody</a></code> +directive when using mod_parp. The directive recognizes the occurrence of $1 +within the variable value and replaces it by the sub-expressions of the +defined regex pattern. The regular expressions is case insensitive. +</li> +<li> +<a name="QS_SetEnvIfStatus"></a> +<syntax>QS_SetEnvIfStatus <code> <env-variable>[=<value>]</syntax><br> +Sets the defined variable in the request environment if the HTTP +response status code matches the defined code. Default value is the status code, but +you might override this by any other value. +Directive may be used on a per-server or per-location basis. <br> +A possible use case for this directive is the prevention of +repetitive occurrence of unwanted response status codes in +conjunction with the +<code><a href="#QS_ClientEventBlockCount">QS_ClientEventBlockCount</a></code> or +<code><a href="#QS_ClientEventLimitCount">QS_ClientEventLimitCount</a></code> +directive. <br> +When using the special variable <code><a href="#QS_Block">QS_Block</a></code>, its +value is set to "1" by default. There are also four "special codes" available to +set the <code><a href="#QS_Block">QS_Block</a></code> event: +<ul> +<li><a name="QS_SrvMinDataRate_var"></a> +<code><a href="#QS_SrvMinDataRate">QS_SrvMinDataRate</a></code> may be used +to set <code><a href="#QS_Block">QS_Block</a></code> events in order to limit the +allowed number of <a href="#QS_SrvMinDataRate"><code>QS_SrvMinDataRate</code></a> +rule violations.</li> +<li><a name="QS_SrvMaxConnPerIP_var"></a> +<code><a href="#QS_SrvMaxConnPerIP">QS_SrvMaxConnPerIP</a></code> may be used +to increment the <code><a href="#QS_Block">QS_Block</a></code> +event when closing connections due to the reach of the limitation configured +by the <a href="#QS_SrvMaxConnPerIP"><code>QS_SrvMaxConnPerIP</code></a> directive.</li> +<li><a name="NullConnection"></a> +<code>NullConnection</code> detects connections +which are closed even no HTTP request has been received. <br> +<small><i>Note: The <code>NullConnection</code> event may happen silently (no log +message) expect when using <code><a href="http://httpd.apache.org/docs/current/mod/core.html#loglevel">LogLevel <img src="images/link.png"/></a></code> "debug". +The parameter may be used to defend against SSL DoS attacks. <!-- "defend against SSL DoS attacks" is a MARKER --> +Please pay attention to the fact that unused speculative TCP pre-connections of +browsers may unintentionally cause this event as well.</i></small></li> +<li><a name="BrokenConnection"></a> +<code>BrokenConnection</code> may be used +to mark clients aborting the TCP connection before reading the whole HTTP +response.<br> +<small><i>Note: Connections may also be aborted by mod_qos if client reads +the response too slow.</i></small></li> +</ul> +</li> +<li> +<a name="QS_SetEnvIfResBody"></a> +<syntax>QS_SetEnvIfResBody <string> [!]<env-variable></syntax><br> +Adds the defined environment variable (e.g., <code><a href="#QS_Block">QS_Block</a></code>) +if the response body contains the defined literal string. Used on a per- +<a href="http://httpd.apache.org/docs/current/mod/core.html.en#location">location <img src="images/link.png"/></a> +level. Only one directive may be defined per-location +(one search string per response). Prefixing the variably by a "!" +lets the variable being removed (unset). You may set the <code>QS_SetEnvIfResBodyIgnore</code> +environment variable if you want mod_qos to skip (not parsing) a request's response +body. +</li> +<li> +<a name="QS_SetEnvRes"></a> +<syntax>QS_SetEnvRes <env-variable> <regex> <env-variable2>[=<value>]</syntax><br> +Sets the environment variable (env-variable2) if the regular expression (regex) matches +against the value of the environment variable (env-variable). Occurrences of $1..$9 within +the value are replaced by parenthesized subexpressions of the regular expression. +</li> +<li> +<a name="QS_SetReqHeader"></a> +<syntax>QS_SetReqHeader [!]<header name> <env-variable> [late]</syntax><br> +Sets the defined HTTP request header with the value of the specified +environment variable if the variable is available.<br> +The header is unset (removed from the request) if the header name is prefixed by a "!". +</li> +<li> +<a name="QS_SetEnvResHeader"></a> +<syntax>QS_SetEnvResHeader <header name> [drop]</syntax><br> +Sets the defined HTTP response header (name and value) to the request environment variables. +Deletes the specified header if the action 'drop' has been specified. +</li> +<li> +<a name="QS_SetEnvResHeaderMatch"></a> +<syntax>QS_SetEnvResHeaderMatch <header name> <regex></syntax><br> +Sets the defined HTTP response header (name and value) to the request environment variables if +the specified regular expression (pcre, not case sensitive) matches +the header value. +</li> +<li> +<a name="QS_UnsetReqHeader"></a> +<syntax>QS_UnsetReqHeader <header name></syntax><br> +The request header of this name is removed. +</li> +<li> +<a name="QS_UnsetResHeader"></a> +<syntax>QS_UnsetResHeader <header name></syntax><br> +The response header of this name is removed. +</li> +<li> +<a name="QS_RedirectIf"></a> +<syntax>QS_RedirectIf <variable> <regex> [<code>:]<url></syntax><br> +Redirects the client to the configured url if the regular expression (case insensitive) +matches the value of the the environment variable. Occurrences of $1..$9 +within the url are replaced by parenthesized subexpressions of the +regular expression. The default status code used by this directive is 302 +but you may prefix the url parameter by <i>307:</i> or <i>301:</i> to change +it to a "307 Temporary Redirect" or "301 Moved Permanently" response. +Directive may be used on a per-server or per-location basis. +</li> +</ul> + +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +Sample of event rules:<br> +<pre> +# marks clients coming from the internal network: +SetEnvIf Remote_Addr ^192\.168\. QS_Intra + +# marks clients neither coming from the internal network +# nor are VIP clients as low priority clients: +<a href="#QS_SetEnvIf">QS_SetEnvIf</a> !<a href="#QS_VipRequest">QS_VipRequest</a> !QS_Intra QS_LowPrio=1 + +# limits the request rate for low priority (neither VIP nor internal) +# clients (and no more than 400 concurrent requests for them): +<a href="#QS_EventPerSecLimit">QS_EventPerSecLimit</a> QS_LowPrio 100 +<a href="#QS_EventRequestLimit">QS_EventRequestLimit</a> QS_LowPrio 400 + +# detects the variable "file" within the query portion of the URL: +<a href="#QS_SetEnvIfQuery">QS_SetEnvIfQuery</a> file=([a-zA-Z]*) QS_LowPrio=$1 + +# combine variables and propagate them to the application via HTTP header: +SetEnvIf Content-Length ([0-9]*) QS_Length=$1 +<a href="#QS_SetEnv">QS_SetEnv</a> QS_Type "length=${QS_Length}; file=${QS_LowPrio}" +<a href="#QS_SetReqHeader">QS_SetReqHeader</a> X-File QS_Type + +# limit the max. body size since mod_parp loads the whole message into +# the memory servers's: +<a href="#QS_LimitRequestBody">QS_LimitRequestBody</a> 131072 + +# body pattern detection, example limits the maximum number of concurrent +# requests posting "id=1234" to ten: +<a href="#QS_SetEnvIfParp">QS_SetEnvIfParp</a> id=([0-9]*) PARP_PATTERN=$1 +<a href="#QS_EventRequestLimit">QS_EventRequestLimit</a> PARP_PATTERN=1234 10 +# but ignore requests to the location /main/ (any sub-locations): +SetEnvIf Request_URI /main/.* !parp +</pre> +</td></tr> +</table> + +<a name="filter"></a> +<h3>Request Level, Generic Filter</h3> +These filters are defined on a per- +<a href="http://httpd.apache.org/docs/current/mod/core.html.en#location">location <img src="images/link.png"/></a> +level and are used to restrict access to resources in +general, independent of server resource availability. +New rules are added by defining a rule id prefixed by a '+'. Rules are merged +to sub-locations. If a rule should not be active for a sub-location, the +very same rule must be defined, but instead, the rule id must be prefixed with a '-'. The filter rules are implemented as Perl-compatible regular expressions +(pcre) and are applied to the decoded URL components (un-escaped characters, +e.g., %20 is a space). The generic request filter ignores the +<a href="#privilegedusers">VIP</a> status of a client.<br> +<small><i>Note: Compile mod_qos with the preprocessor definition +<code>-DQS_MOD_EXT_HOOKS</code> to enable the decoding hooks defined +in <code>mod_qos.h</code> if you intend to implement additional +decodings by other Apache modules.</i></small> +<ul> +<li> +<a name="QS_DenyRequestLine"></a> +<syntax>QS_DenyRequestLine '+'|'-'<id> 'log'|'deny' <pcre></syntax><br> +Generic request line (method, path, query, and protocol) filter used to +deny access for requests matching the defined expression (pcre, case insensitive). +The action taken for matching rules is either 'log' (access is granted but the rule +match is logged) or 'deny' (access is denied). +</li> +<li> +<a name="QS_DenyPath"></a> +<syntax>QS_DenyPath '+'|'-'<id> 'log'|'deny' <pcre></syntax><br> +Generic abs_path (see RFC 2616 section 3.2.2) filter used to deny access +for requests matching the defined expression (pcre, case insensitive). +The action taken for matching rules is either 'log' (access is granted +but the rule match is logged) or 'deny' (access is denied). +</li> +<li> +<a name="QS_DenyQuery"></a> +<syntax>QS_DenyQuery '+'|'-'<id> 'log'|'deny' <pcre></syntax><br> +Generic query (see RFC 2616 section 3.2.2) filter used to deny access for +requests matching the defined expression (pcre, case insensitive). +The action taken for matching rules is either 'log' (access is granted +but the rule match is logged) or 'deny' (access is denied). +</li> +<li> +<a name="QS_InvalidUrlEncoding"></a> +<syntax>QS_InvalidUrlEncoding 'log'|'deny'|'off'</syntax><br> +Enforces correct URL decoding in conjunction with the +<code><a href="#QS_DenyRequestLine">QS_DenyRequestLine</a></code>, +<code><a href="#QS_DenyPath">QS_DenyPath</a></code>, and +<code><a href="#QS_DenyQuery">QS_DenyQuery</a></code> directives. +Default is "off" which means that an incorrect encoding does stop +request processing. +</li> +<li> +<a name="QS_Decoding"></a> +<syntax>QS_Decoding 'uni'</syntax><br> +Enables additional string decoding functions which are applied before +matching <code>QS_Deny*</code> and <code>QS_Permit*</code> directives. +Default is URL decoding (%xx, \\xHH, '+'). <br>Available additional decodings: +<ul> +<li><code>uni</code>: unicode decoding for MS IIS (%uXXXX and \uXXXX) encoded characters. +</li> +</ul> +</li> +<li> +<a name="QS_DenyEvent"></a> +<syntax>QS_DenyEvent '+'|'-'<id> 'log'|'deny' [!]<env-variable></syntax><br> +Rule matching requests having the defined process environment variable set +(or NOT set if prefixed by a '!'). +The action taken for matching rules is either 'log' (access is granted +but the rule match is logged) or 'deny' (access is denied). +</li> +<li> +<a name="QS_PermitUri"></a> +<syntax>QS_PermitUri '+'|'-'<id> 'log'|'deny' <pcre></syntax><br> +Generic URL (path and query) filter implementing a request pattern +allow list. Only requests matching at least one <code>QS_PermitUri</code> +pattern are allowed. If a <code>QS_PermitUri</code> pattern has +been defined and the request does not match any rule, the request +is denied. +All rules must define the same action. pcre is case sensitive. +You may use the <code><a href="qsfilter2.1.html">qsfilter2</a></code> +utility to generate rules based on access log files. +</li> +<li> +<a name="QS_DenyInheritanceOff"></a> +<syntax>QS_DenyInheritanceOff</syntax><br> +Disables inheritance of <code>QS_Deny*</code> and <code>QS_Permit*</code> +directives (pattern definitions) to a location. +</li> +<li> +<a name="QS_RequestHeaderFilter"></a> +<syntax>QS_RequestHeaderFilter 'on'|'off'|'size'</syntax><br> +Filters request headers using validation rules <a href="headerfilterrules.txt">provided by mod_qos</a>. +Suspicious headers (not matching the pattern or those which are too long) are normally +dropped (removed from the request). Abnormal <code>content-*</code> headers cause +request blocking. Only the defined headers are allowed (allow list). Custom +rules (additional headers or different pattern/size definitions) may be +added using the +<code><a href="#QS_RequestHeaderFilterRule">QS_RequestHeaderFilterRule</a></code> +directive.<br> +This directive has three different operation modes: 'on' (activated), 'off' (disabled), +and 'size' (activated). The operation mode enabled by 'size' does not check the header +values against the patterns but limits the maximum length of request header +values only (similar to the Apache directive <code>LimitRequestFieldsize</code> +but with an individual rule for each header field). +This directive may be used on a per-server or per-location level.<br> +<small><i>Notes:<ul><li>Header validation is also useful to avoid bypassing of +<code><a href="http://httpd.apache.org/docs/current/mod/mod_setenvif.html#setenvif">SetEnvIf <img src="images/link.png"/></a></code> / +<code><a href="http://modsetenvifplus.sourceforge.net/#SetEnvIfPlus">SetEnvIfPlus <img src="images/link.png"/></a></code> +(if configured on a per-location level) directive settings as request headers have higher priority than +<a href="glossary.html#variables">environment variables</a> for those +directives and therefore a header sent a by client +can override an environment variable having the same name.</li> +<li> +You might also configure deny list rules (delete unwanted headers) using the +<code><a href="#QS_UnsetReqHeader">QS_UnsetReqHeader</a></code> or +<code><a href="#QS_UnsetResHeader">QS_UnsetResHeader</a></code> directive. +</li> +</ul></i></small> +</li> +<li> +<a name="QS_RequestHeaderFilterRule"></a> +<syntax>QS_RequestHeaderFilterRule <header name> 'drop'|'deny' <pcre> <size></syntax><br> +Used to add custom request header filter rules, e.g., to override the +<a href="headerfilterrules.txt">internal rules</a> (different pcre or size) +or to add additional headers which should be allowed. +Definitions are made globally (outside VirtualHost). The list of all loaded rules +is shown at server startup when using <code><a href="http://httpd.apache.org/docs/current/mod/core.html#loglevel">LogLevel <img src="images/link.png"/></a></code> "debug". pcre is +case sensitive. The size parameter defines the maximum length of a header value. +The action 'drop' removes a header not matching the pcre, the action 'deny' +rejects a request including such a header not matching the pcre. +</li> +<li> +<a name="QS_ResponseHeaderFilter"></a> +<syntax>QS_ResponseHeaderFilter 'on'|'silent'|'off'</syntax><br> +Filters response headers using validation rules <a href="headerfilterrules.txt">provided by mod_qos</a>. +Suspicious headers (not matching the pattern or those which are too long) are removed +from the response. Only the defined headers are allowed. Filter +is activated ('on' or 'silent') or deactivated ('off'). +</li> +<li> +<a name="QS_ResponseHeaderFilterRule"></a> +<syntax>QS_ResponseHeaderFilterRule <header name> <pcre> <size></syntax><br> +Used to add custom response header filter rules, e.g., to override the +<a href="headerfilterrules.txt">internal rules</a> +(different pcre or size) or to add additional headers which should be allowed. +Definitions are made globally (outside VirtualHost). A list of all loaded rules +is shown at server startup when using <code><a href="http://httpd.apache.org/docs/current/mod/core.html#loglevel">LogLevel <img src="images/link.png"/></a></code> "debug". pcre is +case sensitive. The size parameter defines the maximum length of a header value. +</li> +</ul> +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +Sample configuration:<br> +<pre> +<a href="#QS_ErrorPage">QS_ErrorPage</a> /error-docs/qs_error.html + +# add a custom request header rule: +<a href="#QS_RequestHeaderFilterRule">QS_RequestHeaderFilterRule</a> UA-CPU drop "^[a-zA-Z0-9]+$" 20 + +# enable header validation: +<a href="#QS_RequestHeaderFilter">QS_RequestHeaderFilter</a> on + +<Location /> + # don't allow access to the path /app/admin.jsp: + <a href="#QS_DenyPath">QS_DenyPath</a> +admin deny "^/app/admin.jsp$" + + # allow printable characters only within the request line: + <a href="#QS_DenyRequestLine">QS_DenyRequestLine</a> +printable deny ".*[\x00-\x19].*" +</Location> +</pre> +</td></tr> +</table> +</p> +<p> +Body data filtering requires <a href="http://parp.sourceforge.net">mod_parp <img src="images/link.png"/></a> +which processes the request's message body of the following HTTP request content types: +<code>application/x-www-form-urlencoded</code>, +<code>multipart/form-data</code>, and <code>multipart/mixed</code>. The content type +<code>application/json</code> may be processed by the built-in JSON parser of mod_qos. The body +data is transformed into a request query and may be filtered using the +<code><a href="#QS_DenyQuery">QS_DenyQuery</a></code> and +<code><a href="#QS_PermitUri">QS_PermitUri</a></code> directives. +<ul> +<li> +<a name="QS_DenyQueryBody"></a> +<syntax>QS_DenyQueryBody 'on|'off'</syntax><br> +Enables request body data filtering for the +<code><a href="#QS_DenyQuery">QS_DenyQuery</a></code> directive. +</li> +<li> +<a name="QS_PermitUriBody"></a> +<syntax>QS_PermitUriBody 'on|'off'</syntax><br> +Enables request body data filtering for the +<code><a href="#QS_PermitUri">QS_PermitUri</a></code> directive. +</li> +<li> +<a name="QS_LimitRequestBody"></a> +<syntax>QS_LimitRequestBody <bytes></syntax><br> +Limits the allowed size of an HTTP request message body. This directive may +be placed anywhere in the configuration. Alternatively, the limitation +may be set as an environment variable using +<a href="http://httpd.apache.org/docs/current/mod/mod_setenvif.html">mod_setenvif <img src="images/link.png"/></a> +(overriding the directive settings). +</li> +</ul> + +<a name="QS_DeflateReqBody"></a> +Set the <code>QS_DeflateReqBody</code> variable if the request body data has to +be deflated (compressed data) using +<a href="http://httpd.apache.org/docs/current/mod/mod_deflate.html">mod_deflate <img src="images/link.png"/></a>. + +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +Sample configuration:<br><a name="qsfiltersample"></a> +<pre> +# configure the audit log writing the request body data to a file +# (use this log to generate allow list rules using <a href="qsfilter2.1.html">qsfilter2</a> +# when <a href="#QS_PermitUriBody">QS_PermitUriBody</a> has been enabled) +# format: +# %h: +# The remote host (used to filter by IP address). +# %>s: +# The HTTP response status code. +# %{qos-loc}n +# The matching Location to generate the rules for. +# %{qos-path}n%{qos-query}n +# The request data required by qsfilter2 to generate rules. +CustomLog logs/qsaudit_log "%h %>s %{qos-loc}n %{qos-path}n%{qos-query}n" + +# enable json parser +PARP_BodyData application/json + +<a href="#QS_RequestHeaderFilter">QS_RequestHeaderFilter</a> on + +# limit the max. body size since mod_parp loads the whole message into the +# servers's memory: +SetEnvIfNoCase Content-Type application/x-www-form-urlencoded <a href="#QS_LimitRequestBody">QS_LimitRequestBody</a>=131072 +SetEnvIfNoCase Content-Type multipart/form-data <a href="#QS_LimitRequestBody">QS_LimitRequestBody</a>=131072 +SetEnvIfNoCase Content-Type multipart/mixed <a href="#QS_LimitRequestBody">QS_LimitRequestBody</a>=131072 +SetEnvIfNoCase Content-Type application/json <a href="#QS_LimitRequestBody">QS_LimitRequestBody</a>=65536 + +# enable mod_deflate input filter for compressed request body data: +SetEnvIfNoCase Content-Encoding (gzip|compress|deflate) <a href="#QS_DeflateReqBody">QS_DeflateReqBody</a> + +<Location /app> + # don't allow a certain string pattern within the request query or + # the request message body data: + <a href="#QS_DenyQueryBody">QS_DenyQueryBody</a> on + <a href="#QS_DenyQuery">QS_DenyQuery</a> +s01 deny "(EXEC|SELECT|INSERT|UPDATE|DELETE)" +</Location> +</pre> +</td></tr> +</table> +You may enable request body filtering for arbitrary content types: +<ul> +<li>Register the <a href="http://parp.sourceforge.net">mod_parp <img src="images/link.png"/></a> raw parser using the +<code><a href="http://parp.sourceforge.net/#PARP_BodyData">PARP_BodyData <img src="images/link.png"/></a></code> directive.</li> +<li>Enable mod_parp for the content type using the +<code><a href="http://httpd.apache.org/docs/current/mod/mod_setenvif.html#setenvifnocase">SetEnvIfNoCase <img src="images/link.png"/></a></code> directive.</li> +<li>Use <code><a href="#QS_SetEnvIfBody">QS_SetEnvIfBody</a></code> to detect patterns within the HTTP request body.</li> +<li>The <code><a href="#QS_DenyEvent">QS_DenyEvent</a></code> directive denies access for the request.</li> +</ul> +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +Sample configuration:<br> +<pre> +# sample (using the raw body parser of mod_parp) which denies XML documents +# containing the pattern "<code>delete</code>": +PARP_BodyData text/xml +SetEnvIfNoCase Content-Type text/xml.* parp +SetEnvIfNoCase Content-Type application/xml <a href="#QS_LimitRequestBody">QS_LimitRequestBody</a>=65536 +<a href="#QS_SetEnvIfBody">QS_SetEnvIfBody</a> <code>delete</code> DENYACTION +<Location /app/web> + <a href="#QS_DenyEvent">QS_DenyEvent</a> +BADCODE deny DENYACTION +</Location> +</pre> +</td></tr> +</table> +</p> +<p> +<a name="QS_MileStone"></a> +<h4>Milestones</h4> +You may define a number of resources (request line patterns) as milestones. A +client must access these resources in the correct order as they are defined within +the server configuration. A client is not allowed to skip these milestones (but may access +any other resource not covered by a milestone in between requests to milestones). +<ul> +<li> +<syntax>QS_MileStone 'log'|'deny' <pattern> [<thinktime>]</syntax><br> +Defines request line patterns a client must access in the defined order as +they are defined in the configuration file. The optional 'thinktime' parameter +defines the minimal elapse time (in seconds) between two milestones. +Milestones are defined on a per-server basis, outside +<a href="http://httpd.apache.org/docs/current/mod/core.html.en#location">location <img src="images/link.png"/></a>. +Access to milestones is tracked by a dedicated +<a href="#QS_SessionKey">session cookie</a> (QSSCD). +</li> +<li> +<a name="QS_MileStoneTimeout"></a> +<syntax>QS_MileStoneTimeout <seconds></syntax><br> +Defines the time in seconds within which a client must reach the next +milestone. Default are 3600 seconds. +</li> +</ul> +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +Sample configuration:<br> +<pre> +# four milestones: +# 1) client must start with /app/index.html +# 2) and then read some images (e.g. media used within the first page) +# 3) before posting data to /app/register +# 4) afterwards, the user may download zip files +<a href="#QS_MileStone">QS_MileStone</a> deny "^GET /app/index.html" +<a href="#QS_MileStone">QS_MileStone</a> deny "^GET /app/images/.*" +<a href="#QS_MileStone">QS_MileStone</a> deny "^POST /app/register*" +<a href="#QS_MileStone">QS_MileStone</a> deny "^GET /app/.*\.zip HTTP/..." +</pre> +</td></tr> +</table> +</p> + +<a name="connectionlevelcontrol"></a> +<h3>Connection Level Control</h3> +<p> +The module features the following directives to control server access on a per-server +(TCP connection) level. These directives must only be used in the global server context +and for port based virtual hosts. +Virtual hosts neither defining <code>QS_SrvMaxConn</code>, <code>QS_SrvMaxConnClose</code>, +nor <code>QS_SrvMaxConnPerIP</code> are using the base server's settings and counters. +And do not use these three directives for name based virtual hosts! +<ul> +<li> +<a name="QS_SrvMaxConn"></a> +<syntax>QS_SrvMaxConn <number></syntax><br> +Defines the maximum allowed number of <a href="glossary.html#concurrency">concurrent</a> +TCP connections for this server (virtual host). +</li> +<li> +<a name="QS_SrvMaxConnClose"></a> +<syntax>QS_SrvMaxConnClose <number>[%]</syntax><br> +Defines the maximum number of connections for this server (virtual host) supporting +HTTP keep-alive. If the number of <a href="glossary.html#concurrency">concurrent</a> +connections exceeds this threshold, the TCP connections +gets closed after each request. You may specify the number of +connections as a percentage of <a href="http://httpd.apache.org/docs/current/mod/mpm_common.html#maxrequestworkers"><code>MaxClients</code> <img src="images/link.png"/></a> if adding the suffix '%' to the +specified value.<br> +<small><i>Note: It's also possible to control the Keep-Alive settings dynamically using +the <code><a href="#QS_KeepAliveTimeout">QS_KeepAliveTimeout</a></code> +and <code><a href="#QS_MaxKeepAliveRequests">QS_MaxKeepAliveRequests</a></code> +environment <a href="glossary.html#variables">variables</a>.</i></small> +</li> +<li> +<a name="QS_SrvMaxConnPerIP"></a> +<syntax>QS_SrvMaxConnPerIP <number> [<connections>]</syntax><br> +Defines the maximum number of connections per source IP address for this server (virtual host). +The "connections" argument defines the number of busy connections of the server +(all virtual hosts) to enable this limitation, default is 0 (which means that the limitation +is always enabled, even the server is idle). +</li> +<li> +<a name="QS_SrvMaxConnExcludeIP"></a> +<syntax>QS_SrvMaxConnExcludeIP <address></syntax><br> +Defines an IP address or address range to be excluded from connection +level control restrictions (trusted proxy servers). An address range +must end with a "." or ":". +</li> +<li> +<a name="QS_SrvMaxConnPerIPIgnoreVIP"></a> +<syntax>QS_SrvMaxConnPerIPIgnoreVIP 'on'|'off'</syntax><br> +Tells the <a href="#QS_SrvMaxConnPerIP"><code>QS_SrvMaxConnPerIP</code></a> directive +to ignore (if set to "on") the <a href="#privilegedusers">VIP</a> status of +clients. Default is "off", which means that <code>QS_SrvMaxConnPerIP</code> gets +disabled for VIPs. +</li> +<li> +<a name="QS_SrvMinDataRate"></a> +<syntax>QS_SrvMinDataRate <bytes per second> [<max bytes per second> [<connections>]]</syntax><br> +Defines the minimum upload/download throughput a client must generate (the bytes +sent/received by the client per seconds). This bandwidth is measured while +receiving request data (<i>in</i>: request line, header fields, or body), sending response data +(<i>out</i>: header fields, body) and during keep-alive (<i>enforce keep-alive</i>). +The client connection is closed if the client does not fulfill this required minimal +data rate and the IP address of the causing client is marked in order to be handled +with low priority (see the <code><a href="#QS_ClientPrefer">QS_ClientPrefer</a></code> directive). +The "max bytes per second" activates <u><a href="images/SrvMinDataRate.png">dynamic minimum throughput control</a></u>: +The required minimal throughput is increased in parallel to the number of concurrent clients +sending/receiving data (starts increasing when reaching the "connections" threshold) +as a percentage of the "max bytes per second" which maximum is reached when the number of +sending/receiving clients is equal to the <code>MaxClients</code> setting. +The "connections" argument is used to specify the number of busy TCP connections a +server must have to enable this feature (used to disable the +<code>QS_SrvMinDataRate</code> rule enforcement on idle servers).<br> +This directives must only be used in the global server context. +</li> +<li> +<a name="QS_SrvRequestRate"></a> +<syntax>QS_SrvRequestRate <bytes per second> [<max bytes per second>]</syntax><br> +Same as <code><a href="#QS_SrvMinDataRate">QS_SrvMinDataRate</a></code> but enforcing a +minimal upload (reading request) throughput only. +</li> +<li> +<a name="QS_SrvDataRateOff"></a> +<syntax>QS_SrvDataRateOff</syntax><br> +Disables the <code><a href="#QS_SrvMinDataRate">QS_SrvMinDataRate</a></code> or +<code><a href="#QS_SrvRequestRate">QS_SrvRequestRate</a></code> enforcement +for a virtual host. +</li> +<li> +<a name="QS_SrvMinDataRateOffEvent"></a> +<syntax>QS_SrvMinDataRateOffEvent '+'|'-'<env-variable></syntax><br> +Disables the <code><a href="#QS_SrvMinDataRate">QS_SrvMinDataRate</a></code> or +<code><a href="#QS_SrvRequestRate">QS_SrvRequestRate</a></code> enforcement +for a connection when the defined process environment variable is set. +The '+' prefix is used to add a variable to the configuration while the '-' prefix +is used to remove a variable. Directive may be used on a per-server or a +per-<a href="http://httpd.apache.org/docs/current/mod/core.html.en#location">location <img src="images/link.png"/></a> basis. +</li> +<li> +<a name="QS_SrvSampleRate"></a> +<syntax>QS_SrvSampleRate <seconds></syntax><br> +Defines the sampling rate used to measure the data throughput. +Default is 5 seconds or the value you have used for +<code>QS_REQ_RATE_TM</code> while compiling the module. +Increase this value if you want to compensate bandwidth +variations.<br> +This directives must only be used in the global server context.<br> +<small><i>Note: It might also be increased to avoid too many error messages generated by a +<code>QS_SrvMinDataRate</code> rule for clients opening unused TCP pre-connections +which might happen if Apache's +<code><a href="http://httpd.apache.org/docs/current/mod/core.html#timeout">TimeOut <img src="images/link.png"/></a></code> directive is set to higher value than this sample rate. +</i></small> +</li> +<li> +<a name="QS_SrvMinDataRateIgnoreVIP"></a> +<syntax>QS_SrvMinDataRateIgnoreVIP 'on'|'off'</syntax><br> +Tells the <a href="#QS_SrvMinDataRate"><code>QS_SrvMinDataRate</code></a> directive +to ignore (if set to "on") the <a href="#privilegedusers">VIP</a> status of +clients. Default is "off", which means that <code>QS_SrvMinDataRate</code> gets +disabled for VIPs. +</li> +<li> +<a name="QS_SrvSerialize"></a> +<syntax>QS_SrvSerialize 'on'|'off' [<timeout>]</syntax><br> +Ensures that not more than one request having the +<a href="#QS_SrvSerialize_var"><code>QS_SrvSerialize</code></a> +variable set is processed at the same time by +<a href="glossary.html#serialization">serializing</a> them +(process one request after each other). Default is "off".<br> +<small><i>Note: Maximum wait time for a request is defined by the +optional timeout parameter (in seconds). The default is 300 seconds. +</i></small> +</li> +<li> +Throttling the download bandwidth: +mod_qos does not support bandwidth limitation on a per connection +basis but you might use the <code>RATE_LIMIT</code> filter +provided by the Apache module +<a href="https://httpd.apache.org/docs/2.4/mod/mod_ratelimit.html">mod_ratelimit <img src="images/link.png"/></a> +to implement a bandwidth rate limitation for connections. +</li> +</ul> + +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +Sample configuration:<br> +<pre> +# minimum data rate (bytes/sec) when the server +# has 150 or more open TCP connections: +<a href="#QS_SrvMinDataRate">QS_SrvMinDataRate</a> 64 256 150 + +# limits the connections for this virtual host: +<a href="#QS_SrvMaxConn">QS_SrvMaxConn</a> 800 + +# allows keep-alive support till the server reaches 600 connections: +<a href="#QS_SrvMaxConnClose">QS_SrvMaxConnClose</a> 600 + +# allows max 50 connections from a single ip address: +<a href="#QS_SrvMaxConnPerIP">QS_SrvMaxConnPerIP</a> 50 + +# disables connection restrictions for certain clients: +<a href="#QS_SrvMaxConnExcludeIP">QS_SrvMaxConnExcludeIP</a> 172.18.3.32 +<a href="#QS_SrvMaxConnExcludeIP">QS_SrvMaxConnExcludeIP</a> 192.168.10. +</pre> +</td></tr> +</table> +</p> + +<a name="clientlevelcontrol"></a> +<h3>Client Level Control</h3> +<p> +Client level control rules are applied per client (IP source address). +These directives must only be used in the global server context. +<ul> +<li> +<a name="QS_ClientEntries"></a> +<syntax>QS_ClientEntries <number></syntax><br> +Defines the number of individual clients managed by mod_qos. +Default is 50'000 concurrent IP addresses. Each client requires +about 150 bytes memory on a 64bit system (depending on how many +<code><a href="#QS_ClientEventLimitCount">QS_ClientEventLimitCount</a></code> +events you have configured). Client IP source address store +survives graceful server restart. The maximum value is 10'000'000. +</li> +<li> +<a name="QS_ClientEventRequestLimit"></a> +<syntax>QS_ClientEventRequestLimit <number></syntax><br> +Defines the allowed number of <a href="glossary.html#concurrency">concurrent</a> +requests coming from the same client source IP address having the +<code><a href="#QS_EventRequest">QS_EventRequest</a></code> variable set.<br> +<small><i>Note: You may use the <code>QS_ClientIpFromHeader <header></code> +directive to override the client's IP address based on the value within the defined +HTTP request header (e.g., X-Forwarded-For) instead of taking the IP address of +the client which has opened the TCP connection.</i></small> +</li> +<li> +<a name="QS_ClientEventPerSecLimit"></a> +<syntax>QS_ClientEventPerSecLimit <number></syntax><br> +Defines how often a client may cause a +<code><a href="#QS_Event">QS_Event</a></code> +<a href="glossary.html#requestPerSecond">per second</a>. Such events are requests having the +<code><a href="#QS_Event">QS_Event</a></code> variable set, e.g., defined by +using the <code><a href="http://httpd.apache.org/docs/current/mod/mod_setenvif.html#setenvif">SetEnvIf <img src="images/link.png"/></a></code> directive. +The rule is enforced by adding a delay to requests causing +the event (similar to the <code><a href="#QS_LocRequestPerSecLimit">QS_LocRequestPerSecLimit</a></code> +directive). +</li> +<li> +<a name="QS_ClientEventBlockCount"></a> +<a name="QS_ClientEventBlockExcludeIP"></a> +<syntax>QS_ClientEventBlockCount <number> [<seconds>]</syntax><br> +Defines the <a href="glossary.html#repeat">maximum number</a> of <code><a href="#QS_Block">QS_Block</a></code> +events allowed within the defined time (default is 600 seconds). +Client IP is blocked when reaching this counter for the specified +time (blocked at connection level: user might not always get a +user friendly error response).<br/> +<small><i>Notes: +<ul> +<li> +You may use <code>QS_ClientEventBlockExcludeIP <addr></code> +to exclude an IP address from being processed by this limitation +(e.g. for trusted clients connecting via a proxy server). An address +range must end with a "." or ":". +</li> +<li>The counter can be decremented by setting the environment +variable <code><a href="#_Decrement">QS_Block_Decrement</a></code>. +</li> +</ul></i></small> +</li> +<li> +<a name="QS_ClientEventLimitCount"></a> +<syntax>QS_ClientEventLimitCount <number> [<seconds> [<variable>]]</syntax><br> +Defines the <a href="glossary.html#repeat">maximum number</a> of requests +having the defined environment variables +(<code><a href="#QS_Limit">QS_Limit</a></code> by default) set allowed within +the defined time (default is 600 seconds). Requests from client IP's reaching +this limitation are denied for the specified time (blocked at request level). <br/> +<small><i>Notes: +<ul> +<li>The value of the variable defines the penalty points by which the counters +are increased. Default (empty or non-numeric value) is 1 (increment per request).</li> +<li><a name="QS_ClientIpFromHeader"></a> +You may use the <code>QS_ClientIpFromHeader <header></code> +directive to determine the client's IP address based on the defined HTTP +request header (e.g., X-Forwarded-For) instead of taking the IP address +of the client which has opened the TCP connection. The header must only +contain a single IP address.<br> +You might also use a pseudo IP address by creating a hash from the +header's value if you prefix the header name by a '#', +e.g. <code>#Authorization</code> to use the HTTP basic auth header. +as the pseudo IP address. The special name <code>#SSL_CLIENT_S_DN</code> +creates a pseudo IP from the SSL client certificate's subject and issuer DN. +</li> +<li>The current value of this counter is stored within the variable suffixed +by <code><a href="#_Counter">_Counter</a></code>, e.g. <code>QS_Limit_Counter</code> for further +processing by other rules. </li> +<a name="QS_Limit_Remaining"></a> +<li>The remaining time (in seconds) is stored within the variabled suffixed +by <code>_Remaining</code>, e.g. <code>QS_Limit_Remaining</code> to be +used within <a href="glossary.html#ssi">SSI</a> error pages.</li> +<li>The counter can be reset by setting the environment variable which name is +suffixed by <code><a href="#_Clear">_Clear</a></code>, e.g. <code>QS_Limit_Clear</code>. +<li>The counter can be decremented by setting the environment variable which name is +suffixed by <code><a href="#_Decrement">_Decrement</a></code>, e.g. <code>QS_Limit_Decrement</code>. +<li>Adding/removing events (configuration changes) require a server restart +(graceful restart is not supported).</li> +<li>Only the default rule (<code><a href="#QS_Limit">QS_Limit</a></code>) is accessibly by the +<a href="#statusviewer">status viewer</a> (you may use the +<a href="#webconsole">console</a> to view other variables alternatively).</li> +<li>See also <a href="#QS_CondClientEventLimitCount"><code>QS_CondClientEventLimitCount</code></a> +if you want to enforce a rule under certain conditions only.</li> +</ul></i></small> +</li> +<li> +<a name="QS_ClientSerialize"></a> +<syntax>QS_ClientSerialize</syntax><br> +<a href="glossary.html#serialization">Serializes</a> requests having the +<a href="#QS_Serialize"><code>QS_Serialize</code></a> +variable set if they are coming from the same IP address.<br> +<small><i>Notes: +<ul> +<li>You may use the <code>QS_ClientIpFromHeader <header></code> directive to +override the client's IP address based on the value within the defined HTTP request +header (e.g., X-Forwarded-For) instead of taking the IP address of the client which has opened +the TCP connection. +</li> +<li>Maximum wait time for a request is 5 minutes. +</li> +</ul></i></small> +</li> +<li> +<a name="QS_ClientPrefer"></a> +<syntax>QS_ClientPrefer [<percent>]</syntax><br> +Accepts only <a href="#privilegedusers">VIP</a> +and high priority clients when the server has less than 80% +(or the defined percentage) of free TCP connections. The server +<u><a href="images/ClientPrefer.png">continues dropping more and more clients</a></u> +(also those with few penalty points) the higher the number of connections +grows. +<br>Use the +<code><a href="#QS_VipHeaderName">QS_VipHeaderName</a></code> or +<code><a href="#QS_VipIPHeaderName">QS_VipIPHeaderName</a></code> +directive in order to identify <a href="#privilegedusers">VIP</a> clients.<br> +The distinction between high and low priority clients is made +based on penalty points which are calculated based of these attributes: +<ul> +<li>Data transfer behavior (clients sending data slowly / their transfer rate) (0x01).</li> +<li>Accessing "unusual" content types (see <code><a href="#QS_ClientTolerance">QS_ClientTolerance</a></code> +and <code><a href="#QS_ClientContentTypes">QS_ClientContentTypes</a></code>) +(0x00 unknown / 0x02 normal / 0x04 unusual).</li> +<li>Causing events <a href="#QS_ClientEventBlockCount">blocking</a> / +<a href="#QS_ClientEventLimitCount">limiting</a> them (0x08 block / 0x10 limit).</li> +<li>If their connections get closed due to timeouts (0x20).</li> +</ul> +HTTP requests causing a client to get marked as "low priority" have the +"r;" event within the <a href="#mod_qos_ev">mod_qos_ev</a> variable set. +You may use the <a href="#statusviewer">status viewer</a> to determine +which client addresses are identified as low priority clients. Feature is +disabled if directive is not set.<br> +A low priority flag is cleared after 24h hours. Clients identified by +<code><a href="#QS_SrvMaxConnExcludeIP">QS_SrvMaxConnExcludeIP</a></code> +are excluded from connection restrictions. Filter is applied on connection level +blocking clients even before the server starts reading the HTTP request data. +</li> +<li> +<a name="QS_ClientTolerance"></a> +<syntax>QS_ClientTolerance <percent></syntax><br> +Defines the allowed variation from a "normal" client (average) behavior when enabling +the <code><a href="#QS_ClientPrefer">QS_ClientPrefer</a></code> directive. Default is 20%. +</li> +<li> +<a name="QS_ClientContentTypes"></a> +<syntax>QS_ClientContentTypes <html> <css/js> <images> <other> <304></syntax><br> +Defines the distribution of HTTP response content types a client normally +receives when accessing the server. Can only be used in conjunction with the +<code><a href="#QS_ClientPrefer">QS_ClientPrefer</a></code> directive. +<code><a href="#QS_ClientTolerance">QS_ClientTolerance</a></code> defines +the allowed deviation from these values. mod_qos normally learns the average +behavior automatically by default (you can see the learned values within +the <a href="#statusviewer">status viewer</a> or by enabling the +<a href="#QS_Status"><code>QS_Status</code></a> log messages) but +you may specify a static configuration using this directive in order +to avoid influences by a high number of abnormal clients. Default is +automatic self-learning. +</li> +<li> +<a name="QS_ClientGeoCountryDB"></a> +<syntax>QS_ClientGeoCountryDB <path></syntax><br> +Defines the path to the geographical database file. +The file is a Comma Separated Value (CSV) format file +(<a href="https://sourceforge.net/p/mod-qos/source/HEAD/tree/trunk/test/conf/GeoIPCountryWhois.csv?format=raw">example</a>). +Each line contains the following fields: +<ul> +<li> +Double quoted beginning <i><id title="where w.x.y.z results in 16777216*w + 65536*x + 256*y + z">IPv4 number</id></i> of the address range, e.g. "1052272128" for 62.184.102.0 +</li> +<li> +Double quoted ending <i><id title="where w.x.y.z results in 16777216*w + 65536*x + 256*y + z">IPv4 number</id></i> of the address range, e.g. "1052272543" for 62.184.103.159. +</li> +<li> +Double quoted ISO 3166 country code, e.g. "FR" for France. +</li> +</ul> +The <a href="#QS_Country"><code>QS_Country</code></a> variable contains +the country code for the client's IP address. <br> +<small><i>Note: You may use the <code>QS_ClientIpFromHeader <header></code> directive to +override the client's IP address based on the value within the defined HTTP request +header (e.g., X-Forwarded-For) instead of taking the IP address of the client which has opened +the TCP connection to evaluate this variable.</i></small> +</li> +<li> +<a name="QS_ClientGeoCountryPriv"></a> +<syntax>QS_ClientGeoCountryPriv <list> <connections> ['excludeUnknown']</syntax><br> +Defines a comma separated list of country codes for origin client IPv4 address +which are allowed to access the server even if the number of busy TCP +connections reaches the defined number of connections.<br> +Uses the geographical database loaded by +<a href="#QS_ClientGeoCountryDB"><code>QS_ClientGeoCountryDB</code></a>. +<br>Clients whose IP can't be mapped to a country code can be excluded +from the limitation by configuring the 'excludeUnknown' argument. +</li> +</ul> + +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +Sample configuration:<br> +<pre> +# allows not more than 20 events/penalty points per 10 minutes: +<a href="#QS_ClientEventBlockCount">QS_ClientEventBlockCount</a> 20 + +# don't allow a client to access /app/start.html more than +# 20 times within 10 minutes: +SetEnvIf Request_URI /app/start.html <a href="#QS_Block">QS_Block</a>=1 + +# don't allow more than 4 "403" status code responses +# (forbidden) for a client within 10 minutes: +<a href="#QS_SetEnvIfStatus">QS_SetEnvIfStatus</a> 403 <a href="#QS_Block">QS_Block</a>=5 +</pre> +</td></tr> +</table> +</p> + +<a name="messages"></a> +<h2>Log Messages</h2> +<a name="errorlog"></a> +<h3>Error Log</h3> +<p> +mod_qos writes <a href="MESSAGES.txt">messages</a> to Apache's error log when +detecting a rule violation. Each error message is prefixed by an id: +<code>mod_qos(<number>)</code>. These error codes (number only) +are also written to the error notes (Apache's <code>error-notes</code> note +as well as the <a href="#QS_ErrorNotes"><code>QS_ErrorNotes</code></a> +variable) in order to be processed within error pages using +<a href="glossary.html#ssi">server-side includes (SSI)</a>. +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +<pre> +mod_qos(00x): initialisation event +mod_qos(01x): request level control event +mod_qos(08x): request level control event +mod_qos(02x): vip session event +mod_qos(03x): connection level event +mod_qos(04x): generic filter event +mod_qos(14x): generic filter event +mod_qos(05x): bandwidth limitation event +mod_qos(06x): client control event +mod_qos(16x): client control event +mod_qos(07x): console errors +mod_qos(08x): initialisation/resource errors +mod_qos(10x): geo errors +</pre> +</td></tr> +</table> +</p> + +<a name="accesslog"></a> +<h3>Access Log</h3> +<p> +mod_qos adds event variables to the request record which may be added +to access log messages. +<ul> +<li> +<a name="mod_qos_ev"></a> +<syntax>mod_qos_ev</syntax> <br> Status event message of mod_qos. It's a +single letter which is used to signalize an event: "D"=denied, "S"=pass +due to an available <a href="#privilegedusers">VIP</a> session, +"V"=create VIP session (cookie), "v"=marks an IP as VIP, +"K"=connection closed (no keep-alive), "T"=dynamic keep-alive, +"r"=IP is marked as a slow/bad client, "L"=means a request slowdown, +"u"=request without a <a href="#QS_UserTrackingCookieName">user tracking cookie</a>, +and "s" is used for serialized requests. The letter "A" for connection abort +is set if the status code detection +<a href="#BrokenConnection"><code>BrokenConnection</code></a> +has been configured. +</li> +<li> +<a name="mod_qos_cr"></a> +<syntax>mod_qos_cr</syntax> <br> The number of concurrent requests to a +location matching the <code><a href="#QS_LocRequestLimit">QS_LocRequestLimit</a></code>, +<code><a href="#QS_LocRequestLimitMatch">QS_LocRequestLimitMatch</a></code>, +<code><a href="#QS_LocRequestPerSecLimit">QS_LocRequestPerSecLimit</a></code>, +<code><a href="#QS_LocRequestPerSecLimitMatch">QS_LocRequestPerSecLimitMatch</a></code>, +<code><a href="#QS_LocKBytesPerSecLimit">QS_LocKBytesPerSecLimit</a></code>, +<code><a href="#QS_LocKBytesPerSecLimitMatch">QS_LocKBytesPerSecLimitMatch</a></code>, +<code><a href="#QS_CondLocRequestLimitMatch">QS_CondLocRequestLimitMatch</a></code>, +or <code><a href="#QS_EventRequestLimit">QS_EventRequestLimit</a></code> directive. +</li> +<li> +<a name="mod_qos_con"></a> +<syntax>mod_qos_con</syntax> <br> This event shows the number of +concurrent connections to this server. Only available if the directive +<a href="#QS_SrvMaxConn"><code>QS_SrvMaxConn</code></a> +is used. +</li> +<li> +<a name="mod_qos_user_id"></a> +<a name="QS_UserTrackingCookieName"></a> +<syntax>mod_qos_user_id</syntax> <br> The user id which is available when +enabling <a href="glossary.html#UserTracking">user tracking</a>.<br> +<a href="glossary.html#UserTracking">User tracking</a> is based on a unique identifier generated by +<a href="https://httpd.apache.org/docs/current/mod/mod_unique_id.html">mod_unique_id <img src="images/link.png"/></a>. +This unique identifier is stored as a cookie. The user tracking +feature is enabled by setting the +<code>QS_UserTrackingCookieName <name> [<path>] [<domain>] ['session'] ['jsredirect']</code> +directive.<br> +Options of the <code>QS_UserTrackingCookieName</code> directive are: +<ul> +<li>The <code>name</code> argument defining the name of the +user tracking cookie.</li> +<li>The <code>path</code> specifies a local error document +which is shown if a user does not accept the cookie (enforcement). +<br> +You may disable this enforcement for certain clients by setting the +<code>DISABLE_UTC_ENFORCEMENT</code> environment variable at server +level (outside Location), e.g., to allow crawlers not supporting +cookies to access your site.<br/> +This option can be used to ensure whether a client/browser accepts cookies +at all which might be a requirement of your application.</li> +<li><code>domain</code> defines optionally the domain attribute for +the Set-Cookie header.</li> +<li>The <code>session</code> flag indicates that a short lived (per +session) cookie shall be created which won't be stored by the browser +permanently.</li> +<li>When using the additional option <code>'jsredirect'</code>, +the client (browser) has to interpret Javascript used within the +<a href="http://mod-qos.sourceforge.net/cookie-ir.shtml">cookie check page</a> +to fetch the cookie and to execute the redirect back to the initially +requested page (adding Javascript to the cookie challenge).<br> +The following <a href="glossary.html#ssi">SSI variables</a> can be used:<ul> +<li><code>QS_UT_QUERY</code>: Query string to call (ajax) the cookie +page again to obtain the cookie.</li> +<li><code>QS_UT_NAME</code>: Name of the cookie.</li> +<li><code>QS_UT_INITIAL_URI</code>: Initial page to redirect to.</li> +</ul> +</ul> +<small><i> +Notes: +<ul> +<li><code>QS_UserTrackingCookieName</code> ignores the +<code><a href="#QS_LogOnly">QS_LogOnly</a></code> +directive.</li> +<li>The cookie is secured by the <code><a href="#QS_SessionKey">QS_SessionKey</a></code> +and you should set this directive to have a constant key.</li> +</ul> +</li> +</i></small> +<li> +<a name="UNIQUE_ID"></a> +<syntax>UNIQUE_ID</syntax> <br> This is a unique request id generated by +mod_unique_id. mod_qos uses this id to mark messages written to the +error log. So it might be useful to log the <code>UNIQUE_ID</code> +environment variable as well, in order to correlate errors +to access log messages. +</li> +<li> +<a name="QS_ConnectionId"></a> +<syntax>QS_ConnectionId</syntax> <br> Connection correlation id used to +mark all messages belonging to the same TCP connection. +</li> +</ul> +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +Sample configuration:<br> +<pre> +LogFormat "%h %u %t \"%r\" %>s %b %T \"%{content-length}i\" %k \"%{User-Agent}i\" \ + %{mod_qos_cr}e %{mod_qos_ev}e %{mod_qos_con}e %{QS_SrvConn}e %{QS_AllConn}e \ + id=%{UNIQUE_ID}e %{QS_ConnectionId}e %{mod_qos_user_id}e %{QS_Country}e #%P" +</pre> +</td></tr> +</table> +</p> + +<a name="requeststatistics"></a> +<h3>Request Statistics</h3> +<p> +The <code><a href="qslog.1.html">qslog</a></code> tool, which is part of +the support utilities of mod_qos, may be used to gather request +statistics from Apache's access log data. This includes data such +as the number of denied requests or new VIP session creations per +minute but also total requests per second and other data. Refer +to the usage text of the <code><a href="qslog.1.html">qslog</a></code> +utility and read <a href="glossary.html#RequestStatistics">"Request Statistics Using qslog"</a> +for further details. +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +<pre> +CustomLog "|/usr/bin/qslog -o logs/qslog.csv -x -f ISBDQkU" \ + "%h %>s %b %D %{mod_qos_ev}e %k %{mod_qos_user_id}e" +</pre> +</td></tr> +</table> +</p> +<p> +<a name="QSLog"></a> +Instead of using the standard Apache log <code>CustomLog</code> directive, +you may use the <code>QSLog</code> directive of mod_qos alternatively. This +allows you to configure a single log file for your Apache instance (globally, +not per virtual host) and you don't have to specify the format (-f) option. +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +<pre> +QSLog "|/usr/bin/qslog -o /var/log/apache/qslog.csv" +</pre> +</td></tr> +</table> +</p> + +<a name="statusviewer"></a> +<h3>Status Viewer</h3> +<p> +mod_qos features a handler showing the current connection and request status. +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +<pre> +<Location /qos> + SetHandler qos-viewer +</Location> +</pre> +</td></tr> +</table> +A machine-readable version of the status information is available when using +the request query string <code>auto</code>, e.g., +<code>http://your.server.name/qos?auto</code>. The page updates itself +automatically every 10 seconds if you add the request +query string <code>refresh</code>, e.g., +<code>http://your.server.name/qos?refresh</code>.<br> +<a name="QS_EventCount"></a> +<small><i>Note: This view also shows you the <a href="#errorlog">error log event</a> +counters if you enable event (errors and warnings) counting by +configuring <code>QS_EventCount on</code> and are using any +<a href="#clientlevelcontrol">client level limitation</a> using +<code>QS_Client*</code> directives.</i></small> +</p> +<p> +The status information is also provided on the server status page of +<a href="http://httpd.apache.org/docs/current/mod/mod_status.html">mod_status <img src="images/link.png"/></a> +(although in a reduced scope).<br> +<small><i>Note: Compile mod_qos with the preprocessor definition +<code>-DQS_NO_STATUS_HOOK</code> to disable its registration to +the status page rendered by mod_status.</i></small> +</p> +<p> +<a name="QS_DisableHandler"></a> +Use the directive <code>QS_DisableHandler on</code> to disable the qos-viewer and qos-console for +a virtual host in order to prevent accidental activation of these functions, including by configuration +settings of per-directory files (e.g., .htaccess). +</p> +<p> +<a name="QS_Status"></a> +The directive <code>QS_Status 'on'|'off'</code> may be used to enable a +status log message (<i>mod_qos(200)</i>) written to the Apache server's +<code>ErrorLog</code>. This message contains information about the server's +scoreboard. The message is written once every minute. +</p> + +<a name="webconsole"></a> +<h3>Web Console</h3> +<p> +mod_qos implements an Apache handler which acts as a web console for setting attributes via HTTP requests. +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +<pre> +<Location /qos/console> + SetHandler qos-console +</Location> +</pre> +</td></tr> +</table> +Access a location where you have enabled the <code>qos-console</code> handler +with a web client and use the following request query parameter to modify +the status of a client (may only be used if <a href="#clientlevelcontrol">client level control</a> +has been enabled). +<ul> +<li> +<syntax>address=<IP address></syntax><br>Specifies the IP address of the client to modify. +</li> +<li> +<syntax>action='block'|'unblock'|'limit'|'unlimit'|'inclimit'|'setvip'|'unsetvip'|'setlowprio'|'unsetlowprio'|'search'</syntax><br>Defines the command to be executed, or the attribute to be changed. +<ul> +<li><code>block</code>: <a href="#QS_ClientEventBlockCount">blocks</a> the client for the configured period of time, +see also <code><a href="#QS_ClientEventBlockCount">QS_ClientEventBlockCount</a></code>.</li> +<li><code>unblock</code>: clears the <a href="#QS_ClientEventBlockCount">block</a> attribute of the client.</li> +<li><code>limit</code>: <a href="#QS_ClientEventLimitCount">denies requests</a> +from the client IP for the configured period of time, see also +<code><a href="#QS_ClientEventLimitCount">QS_ClientEventLimitCount</a></code>.</li> +<li><code>unlimit</code>: clears the <a href="#QS_ClientEventLimitCount">limit</a> +attribute of the client.</li> +<li><code>inclimit</code>: increments the client's <a href="#QS_ClientEventLimitCount">limit</a> counter.</li> +<li><code>setvip</code>: sets the client status to <a href="#privilegedusers">VIP</a>.</li> +<li><code>unsetvip</code>: clears the <a href="#privilegedusers">VIP</a> status for a client.</li> +<li><code>setlowprio</code>: sets the client's <a href="#QS_ClientPrefer">priority</a> to 'low'.</li> +<li><code>unsetlowprio</code>: clears the 'low' <a href="#QS_ClientPrefer">priority</a> +attribute of the client.</li> +<li><code>search</code>: verifies the availability of a client IP address. <br/>Use the asterisk (*) +for the <code>address</code> parameter in order to get a list of all available clients (dump).</li> +</ul> +</li> +<li><syntax>event=<name></syntax><br>Specifies the event name of the +<code><a href="#QS_ClientEventLimitCount">QS_ClientEventLimitCount</a></code> +directive <a href="glossary.html#repeat">counter</a> which shall be +shown or modified (used in conjunction with the <code>limit</code>, +<code>unlimit</code>, <code>inclimit</code>, and <code>search</code> action). +Default is <code><a href="#QS_Limit">QS_Limit</a></code>.</li> +</ul> +</p> +<p> +The output (which is plain text) contains the following fields: +<ul> + <li>IP address</li> + <li><a href="#privilegedusers">VIP</a> status</li> + <li>low <a href="#QS_ClientPrefer">priority</a> status</li> + <li><code><a href="#QS_ClientEventBlockCount">QS_ClientEventBlockCount</a></code> + counter (<a href="glossary.html#repeat">Cr</a>) and + remaining time (<a href="glossary.html#repeat">Td</a>)</li> + <li><code><a href="#QS_ClientEventLimitCount">QS_ClientEventLimitCount</a></code> + counter (<a href="glossary.html#repeat">Cr</a>) and + remaining time (<a href="glossary.html#repeat">Td</a>)</li> +</ul> +The wildcard search (<code>address=*</code>) generates a by a newline separated list of +all client IP entries. Each line is prefixed by an index and terminated +by the time of the last entry update (seconds since epoch). +</p> +<p> +The console may be used to manually update the status of a client (IP) or +for automated actions. <br/>Examples: +<ul> +<li>To unlock a client which got blocked by mistake.</li> +<li>To synchronize events within multiple Apache instances. <br/> +An example using <code><a href="qsexec.1.html">qsexec</a></code> +is available within the +<a href="https://sourceforge.net/p/mod-qos/source/HEAD/tree/trunk/test/sync.sh?format=raw">source code repository</a>.</li> +<li><a href="https://sourceforge.net/p/mod-qos/source/HEAD/tree/trunk/tools/cc_sync_2k.sh?format=raw">Download/upload</a> client status from one +Apache instance to another (or to the same instance, +e.g., when restarting an instance).</li> +</ul> +</p> +<p> +Examples to access the console: +<ul> +<li>Sets VIP status for the IP 194.31.217.21:<br> +<code>http://your.server.name/qos/console?action=setvip&address=194.31.217.21</code></li> +<li>Clears the QS_Limit counter for the IP 194.31.217.21:<br> +<code>http://your.server.name/qos/console?action=unlimit&address=194.31.217.21&event=QS_Limit</code></li> +</ul> +</p> +<p> +The <a href="#statusviewer">status viewer</a> may be used as well to +verify the status of the client. Example: <br/> +<code>http://your.server.name/qos?action=search&address=194.31.217.21</code> +</p> + +<a name="utilities"></a> +<h3>Utilities</h3> +<p> +mod_qos provides optional tools for log data processing and analysis: +<ul> +<a name="qsdt"></a> +<li><syntax><a href="qsdt.1.html">qsdt</a></syntax><br>Simple tool +to measure the elapse time between related log messages.</li> +<a name="qsexec"></a> +<li><syntax><a href="qsexec.1.html">qsexec</a></syntax><br>Command execution +triggered by patterns within log files.</li> +<a name="qsfilter2"></a> +<li><syntax><a href="qsfilter2.1.html">qsfilter2</a></syntax><br> +Rule generator. Creates <code><a href="#filter">QS_Permit*</a></code> directives and rule patterns +from audit log files.</li> +<a name="qsgeo"></a> +<li><syntax><a href="qsgeo.1.html">qsgeo</a></syntax><br>Adds the country code +for the client IP address within a log file.</li> +<a name="qsgrep"></a> +<li><syntax><a href="qsgrep.1.html">qsgrep</a></syntax><br>Searches a file for a +pattern and prints the data in a new format.</li> +<a name="qslog"></a> +<li><syntax><a href="qslog.1.html">qslog</a></syntax><br>A real time +<code><a href="http://httpd.apache.org/docs/current/mod/mod_log_config.html">TransferLog/CustomLog <img src="images/link.png"/></a></code> +data analyzer. It reads the per request log data from stdin and generates +statistic records every minute.</li> +<a name="qslogger"></a> +<li><syntax><a href="qslogger.1.html">qslogger</a></syntax><br>Shell command +interface to the syslog(3) system log module.</li> +<a name="qspng"></a> +<li><syntax><a href="qspng.1.html">qspng</a></syntax><br>Creates graphics (png +images) from the output of <code>qslog</code>.</li> +<a name="qsre"></a> +<li><syntax><a href="qsre.1.html">qsre</a></syntax><br>Regular expression (pcre) +pattern match test tool.</li> +<a name="qsrespeed"></a> +<li><syntax><a href="qsrespeed.1.html">qsrespeed</a></syntax><br>Compares the +expected processing time per regular expression.</li> +<a name="qsrotate"></a> +<li><syntax><a href="qsrotate.1.html">qsrotate</a></syntax><br>Log rotation tool +similar to Apache's <code>rotatelogs</code>.</li> +<a name="qssign"></a> +<li><syntax><a href="qssign.1.html">qssign</a></syntax><br>A log data integrity +check tool. It reads log data from stdin (pipe) and writes the signed data +to stdout adding a sequence number and signature to ever log line.<br> +<a href="https://sourceforge.net/p/mod-qos/source/HEAD/tree/trunk/tools/logstash-filter-qssign/lib/logstash/filters/qssign.rb?format=raw"><code>qssign.rb</code></a> is a <a href="http://www.logstash.net/">Logstash <img src="images/link.png"/></a> filter +plugin which may be used to verify the signatures of log messages in real time.</li> +<a name="qstail"></a> +<li><syntax><a href="qstail.1.html">qstail</a></syntax><br>Shows the end of a log +file beginning at a defined pattern.</li> +</ul> + +</p> + + +<a name="usecases"></a> +<h2>Sample Use Cases</h2> + +The following use cases may give you an idea about how to use mod_qos. +<a name="Slow_Application"></a> +<h3>Slow Application</h3> +<p> +In case of a very slow application (e.g., at location /ccc), requests wait +until a timeout occurs. Due to many waiting requests, there are no free TCP +connections left and the web sever is not able to process other requests +to applications still working fine, e.g., to /aaa, /bbb /dd1, and /dd2. +mod_qos limits the number of <a href="glossary.html#concurrency">concurrent</a> +requests to an application in order to +assure the availability of other resources. +<br><br>Example:<br> +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +<pre> +# maximum number of active TCP connections is limited to 256 (limited +# by the available memory, adjust the settings according to the +# used hardware): +MaxClients 256 + +# limits the maximum of concurrent requests per application to 100: +<a href="#QS_LocRequestLimit">QS_LocRequestLimit</a> /aaa 100 +<a href="#QS_LocRequestLimit">QS_LocRequestLimit</a> /bbb 100 +<a href="#QS_LocRequestLimit">QS_LocRequestLimit</a> /ccc 100 +<a href="#QS_LocRequestLimitMatch">QS_LocRequestLimitMatch</a> "^(/dd1/|/dd2/).*$" 100 +</pre> +</td></tr> +</table> +The <code><a href="qslog.1.html">qslog</a></code> tool may be used +to analyze your log files in order to identify "slow" resources by +using the <code>-pu</code>, <code>-puc</code>, or <code>-c</code> option. +</p> + +<a name="HTTP_Keep-Alive"></a> +<h3>HTTP Keep-Alive</h3> +<p> +The keep-alive extension of HTTP 1.1 allows persistent TCP connections for +multiple requests/responses. This accelerates access to the web server due to less and optimized network traffic. The disadvantage of these persistent +connections is that server resources are blocked even when no data is exchanged +between client and server. mod_qos allows a server to support keep-alive +as long as sufficient connections are available, but stops the keep-alive +support when it reaches a defined connection threshold. +<br><br>Example:<br> +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +<pre> +# maximum number of active TCP connections is limited to 256 (limited +# by the available memory, adjust the settings according to the +# used hardware): +MaxClients 256 + +# disables keep-alive when 70% of the TCP connections are occupied: +<a href="#QS_SrvMaxConnClose">QS_SrvMaxConnClose</a> 70% +</pre> +</td></tr> +</table> +</p> + +<a name="Client_Opens_Many_Concurrent_Connections"></a> +<h3>Client Opens Many Concurrent Connections</h3> +<p> +A single client may open many TCP connections simultaneously in order to +download different content from the web server. So the client gets many +connections while other users may not be able to access the server because +no free connections remain for them. mod_qos can limit the number +of concurrent connections for a single IP source address. +<br><br>Example:<br> +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +<pre> +# maximum number of active TCP connections is limited to 896 +# (limited by the available memory, adjust the settings according to the +# used hardware): +MaxClients 896 + +# don't allow a single client to open more than 50 TCP connections if +# the server has not more than 196 free connections: +<a href="#QS_SrvMaxConnPerIP">QS_SrvMaxConnPerIP</a> 50 700 +</pre> +</td></tr> +</table> +</p> + +<a name="Many_Requests_to_a_Single_URL"></a> +<h3>Many Requests to a Single URL</h3> +<p> +If you have to limit the number of requests to an URL, mod_qos can help +with that, too. You may limit the number of requests per second to +an URL. mod_qos will then calculate the necessary delay time to be added +to each requests accessing this resource in order to achieve the defined +limitation. +<br><br>Example:<br> +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +<pre> +# does not allow more than 150 requests/sec: +<a href="#QS_LocRequestPerSecLimit">QS_LocRequestPerSecLimit</a> /download/mod_qos.so.gz 150 + +# but do not allow more than 600 concurrent requests: +<a href="#QS_LocRequestLimit">QS_LocRequestLimit</a> /download/mod_qos.so.gz 600 +</pre> +</td></tr> +</table> +</p> +<a name="Many_Requests_to_a_Single_URL_SLOW"></a> +<p> +Alternatively, if you need to reduce the number of processed requests +per time to a very low value, you might add a (predefined or +dynamically calculated) delay to each request and process only +one of them at the same time. However, this will delay every +request to the defined URI, even the server is idle. +<br><br>Example:<br> +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +<pre> +# does not allow more than 4 requests/sec by adding a wait time of 250ms +# to each request and process only one request at once: +SetEnvIf Request_URI ^/download/mod_qos.so.gz <a href="#QS_SrvSerialize">QS_SrvSerialize</a>=1 +SetEnvIf Request_URI ^/download/mod_qos.so.gz <a href="#QS_Delay">QS_Delay</a>=250 +<a href="#QS_SrvSerialize">QS_SrvSerialize</a> on + +# but do not allow more than 600 concurrent requests: +<a href="#QS_EventRequestLimit">QS_EventRequestLimit</a> <a href="#QS_SrvSerialize">QS_SrvSerialize</a> 600 +</pre> +</td></tr> +</table> +</p> + +<a name="Limit_per_IP"></a> +<p> +mod_qos can also restrict the access to an URL by limiting the number +of requests from a single IP address (<code>LimitDownloadCounter</code> is +the <a href="glossary.html#repeat">counter</a> to use while the +<code>LimitDownloadNow</code> pattern is used to limit access to this +specific resource only still allowing the IP address to access +other resources). +<br><br>Example:<br> +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +<pre> +# does not allow more than 4 downloads of mod_qos.so.gz per minute from a single IP address: +SetEnvIf Request_URI ^/download/mod_qos.so.gz LimitDownloadCounter +SetEnvIf Request_URI ^/download/mod_qos.so.gz <a href="#QS_Cond">QS_Cond</a>=LimitDownloadNow +<a href="#QS_CondClientEventLimitCount">QS_CondClientEventLimitCount</a> 4 60 LimitDownloadCounter LimitDownloadNow +</pre> +</td></tr> +</table> +</p> + +<a name="bandwidth_restriction"></a> +<h3>Bandwidth Restriction</h3> +<p> +It's sometimes necessary to restrict the bandwidth consumed by +clients downloading certain type of data in order to avoid +that the entire bandwidth of your Internet connection is +exploited by less important data traffic, e.g. if your web server +hosts large files to be downloaded.<br/> +mod_qos allows you to defined the bandwidth which may be +used when accessing a defined URL or when the server returns a +certain content-type. +<br><br>Example:<br> +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +<pre> +# limits the download bandwidth when accessing ISO images to 1 megabyte/sec +# and does not allow more then 300 clients to download such file type in +# parallel: +<a href="#QS_LocKBytesPerSecLimitMatch">QS_LocKBytesPerSecLimitMatch</a> \.iso 1024 +<a href="#QS_LocRequestLimitMatch">QS_LocRequestLimitMatch</a> \.iso 300 +</pre> +</td></tr> +</table> +</p> + +<a name="brute_force"></a> +<h3>Brute Force</h3> +<p> +Sometimes, you want to limit how often a resource may be accessed +within a certain amount of time, e.g., to defend against brute-force +respectively dictionary attacks or an account lockout DoS (someone +systematically locks user accounts by too many invalid sign-in +attempts). mod_qos allows you to limit this either server wide +(any request accessing the resource) by using the +<code><a href="#QS_EventLimitCount">QS_EventLimitCount</a></code> directive, +or on a per client IP basis using the +<code><a href="#QS_ClientEventLimitCount">QS_ClientEventLimitCount</a></code> +directive. +<br><br>Example:<br> +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +<pre> +# allows a single IP address to access the URI /wp-login.php not more +# than 10 times within an hour: +SetEnvIf Request_URI ^/wp-login.php LimitLogin +<a href="#QS_ClientEventLimitCount">QS_ClientEventLimitCount</a> 10 3600 LimitLogin +</pre> +</td></tr> +</table> +<small> +<i>Note: Multiple users may share an IP addresses which might cause false +positives. You might avoid this by decrementing the counter on successful +user authentication / login, e.g. by setting the variable +<code><a href="#_Decrement">LimitLogin_Decrement=1</a></code>. +</i></small> +</p> +<p> +A brute force attack might also be performed by many distributed +clients (thousands of clients, but every client performs a few +requests only). To add protection to your server, you might configure an +overall limitation for critical resources allowing only known clients +(<a href="#privilegedusers">VIPs</a>) +to access your server without any restrictions. The +<code><a href="#QS_CondEventLimitCount">QS_CondEventLimitCount</a></code> +directive might be used to achieve this. +</p> +<a name="ddos"></a> +<a name="Too_Many_Client_Connections"></a> +<h3>Too Many Client Connections</h3> +<p> +mod_qos may <a href="#QS_ClientPrefer">prefer</a> "known" client IP +addresses in the case that too many clients access the server. +"Known" clients are those which have once been identified by the +application by setting the corresponding +<a href="#QS_VipIPHeaderName ">HTTP response header</a>. +Such identification may happen at successful user login. +Connections from clients which are not known to mod_qos +(never marked by the corresponding response header) are denied +if the server runs on low TCP connection resources (20% or +fewer free connections in this example). mod_qos may +also prefer those clients which communicate with the server +instantaneously and fast, and denies access to slow clients +sending data irregularly, in case the server has not enough +resources.</p> +<p> +You may also set limitations defining how many resources +may be requested by a single IP address source, e.g., using +the <code><a href="#QS_SrvMaxConnPerIP">QS_SrvMaxConnPerIP</a></code> +directive and you can <a href="#HTTP_Keep-Alive">disable HTTP keep-alive</a> +dynamically.</p> +<p> +For more information about how mod_qos can help you in such situations, see the article +<a href="http://mod-qos.sourceforge.net/dos.html">"Denial of Service Defense"</a>. +<br><br>Example:<br> +<table border="0" cellspacing="5" cellpadding="10" width="100%"> +<tr><td bgcolor="#E2EDE2"> +<pre> +# maximum number of active TCP connections is limited to 896 (limited +# by the available memory, adjust the settings according to the used +# hardware): +MaxClients 896 + +# idle timeout: +Timeout 5 + +# keep alive (for up to 85% of all connections): +KeepAlive on +MaxKeepAliveRequests 40 +KeepAliveTimeout 2 +<a href="#QS_SrvMaxConnClose">QS_SrvMaxConnClose</a> 85% + +# name of the HTTP response header which marks preferred clients (this +# may be used to let the application decide which clients are "good" and +# have higher privileges, e.g. authenticated users. +# you may also use the <a href="#QS_VipIPUser">QS_VipIPUser</a> directive when using an Apache +# authentication module such as mod_auth_basic or <a href="http://auth-openid.sourceforge.net/">mod_auth_oid <img src="images/link.png"/></a>): +<a href="#QS_VipIPHeaderName">QS_VipIPHeaderName</a> mod-qos-login + +# enables the known client prefer mode (server allows new TCP connections +# from known/good clients only if there are more than 716 open TCP connections): +<a href="#QS_ClientPrefer">QS_ClientPrefer</a> 80% + +# don't allow more than 30 TCP connections per client source address being +# processed if the server has 500 or more open connections: +<a href="#QS_SrvMaxConnPerIP">QS_SrvMaxConnPerIP</a> 30 500 +</pre> +</td></tr> +</table> +</p> + + +</td></tr> +</tbody> +</table> +<br> +<hr> +<small><small>© 2007-2023, Pascal Buchbinder - mod_qos version 11.74</small></small> +</body> +</html> diff --git a/doc/qsdt.1.html b/doc/qsdt.1.html new file mode 100644 index 0000000..3dbafad --- /dev/null +++ b/doc/qsdt.1.html @@ -0,0 +1,85 @@ + +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<HTML><HEAD><TITLE>Man page of QSDT</TITLE> +<META name='KeyWords' content='Quality of Service, QoS, Apache Web Server, Web application security, WAF, Open Source Software, Security, Proxy'/> +<META name='author' content='Pascal Buchbinder' /> +</HEAD><BODY> +<H1>QSDT</H1> +Section: qsdt man page (1)<BR>Updated: May 2023<BR><A HREF="#index">Index</A> +<A HREF="index.html#utilities">Return to Main Contents</A><HR> + +<P> +<A NAME="lbAB"> </A> +<H2>NAME</H2> + +qsdt calculates the elapsed time between two related log messages. +<A NAME="lbAC"> </A> +<H2>SYNOPSIS</H2> + +qsdt [-t <regex>] -i <regex> -s <regex> -e <regex> [-v] [<path>] +<A NAME="lbAD"> </A> +<H2>DESCRIPTION</H2> + +qsdt is a simple tool to search two different messages in a log file and calculates the elapsed time between these lines. The two log messages need a common identifier such an unique request id (UNIQUE_ID), a thread id, or a transaction code. +<A NAME="lbAE"> </A> +<H2>OPTIONS</H2> + +<DL COMPACT> +<DT>-t <regex> <DD> +Defines a pattern (regular expression) matching the log line's timestamp. The pattern must include two sub-expressions, one matching hours, minutes and seconds the other matching the milliseconds. Default pattern is ([0-9]{2}:[0-9]{2}:[0-9]{2})[.,]([0-9]{3}) +<DT>-i <regex> <DD> +Pattern (regular expression) matching the identifier which the two messages have in common. The sub-expression defines the part which needs to be extracted from the matching string. Note: You can also use the start (-s) and end (-e) pattern to define the sub-expression matching this identifier. +<DT>-s <regex> <DD> +Defines the pattern (regular expression or literal string) identifying the first (start) of the two messages. +<DT>-e <regex> <DD> +Defines the pattern (regular expression or literal string) identifying the second (end) of the two messages. +<DT>-v <DD> +Verbose mode. +<DT><path> <DD> +Defines the input file to process. qsdt reads from from standard input if this parameter is omitted. +</DL> +<A NAME="lbAF"> </A> +<H2>EXAMPLE</H2> + +Sample command line arguments: +<P> +<BR> -i ' ([a-z0-9]+) [A-Z]+ ' -s 'Received Request' -e 'Received Response' +<P> +<BR> matching those sample log messages: +<BR> 2018-03-12 16:34:08.653 threadid23 INFO Received Request +<BR> 2018-03-13 16:35:09.891 threadid23 DEBUG MessageHandler Received Response +<P> +<A NAME="lbAG"> </A> +<H2>NOTE</H2> + +The four patterns (t,i,s,e) are concatenated into two search patterns: +<BR> first (start): [t (HH:MM:SS)(SSS) ].*[i (id) ].*[s ] +<BR> second (end): [t (HH:MM:SS)(SSS) ].*[i (id) ].*[e ] +<P> +And the three sub-expression are used to extract the timestamp and the unique identifier that the start and end message have in common. This means that you could specify the sub-expression for the unique identifier in the start (-s) or end (-e) pattern alternatively, e.g. in case the identifier is at the end of the log line. +<A NAME="lbAH"> </A> +<H2>SEE ALSO</H2> + +<A HREF="qsexec.1.html">qsexec</A>(1), <A HREF="qsfilter2.1.html">qsfilter2</A>(1), <A HREF="qsgeo.1.html">qsgeo</A>(1), <A HREF="qsgrep.1.html">qsgrep</A>(1), <A HREF="qshead.1.html">qshead</A>(1), <A HREF="qslog.1.html">qslog</A>(1), <A HREF="qslogger.1.html">qslogger</A>(1), <A HREF="qspng.1.html">qspng</A>(1), <A HREF="qsre.1.html">qsre</A>(1), <A HREF="qsrespeed.1.html">qsrespeed</A>(1), <A HREF="qsrotate.1.html">qsrotate</A>(1), <A HREF="qssign.1.html">qssign</A>(1), <A HREF="qstail.1.html">qstail</A>(1) +<A NAME="lbAI"> </A> +<H2>AUTHOR</H2> + +Pascal Buchbinder, <A HREF="http://mod-qos.sourceforge.net/">http://mod-qos.sourceforge.net/</A> +<P> + +<HR> +<A NAME="index"> </A><H2>Index</H2> +<DL> +<DT><A HREF="#lbAB">NAME</A><DD> +<DT><A HREF="#lbAC">SYNOPSIS</A><DD> +<DT><A HREF="#lbAD">DESCRIPTION</A><DD> +<DT><A HREF="#lbAE">OPTIONS</A><DD> +<DT><A HREF="#lbAF">EXAMPLE</A><DD> +<DT><A HREF="#lbAG">NOTE</A><DD> +<DT><A HREF="#lbAH">SEE ALSO</A><DD> +<DT><A HREF="#lbAI">AUTHOR</A><DD> +</DL> +<HR> + +</BODY> +</HTML> diff --git a/doc/qsexec.1.html b/doc/qsexec.1.html new file mode 100644 index 0000000..d9d0490 --- /dev/null +++ b/doc/qsexec.1.html @@ -0,0 +1,72 @@ + +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<HTML><HEAD><TITLE>Man page of QSEXEC</TITLE> +<META name='KeyWords' content='Quality of Service, QoS, Apache Web Server, Web application security, WAF, Open Source Software, Security, Proxy'/> +<META name='author' content='Pascal Buchbinder' /> +</HEAD><BODY> +<H1>QSEXEC</H1> +Section: qsexec man page (1)<BR>Updated: May 2023<BR><A HREF="#index">Index</A> +<A HREF="index.html#utilities">Return to Main Contents</A><HR> + +<P> +<A NAME="lbAB"> </A> +<H2>NAME</H2> + +qsexec - parses the data received via stdin and executes the defined command on a pattern match. +<P> +<A NAME="lbAC"> </A> +<H2>SYNOPSIS</H2> + +qsexec -e <pattern> [-t <number>:<sec>] [-c <pattern> [<command string>]] [-p] [-u <user>] <command string> +<A NAME="lbAD"> </A> +<H2>DESCRIPTION</H2> + +qsexec reads log lines from stdin and searches for the defined pattern. It executes the defined command string on pattern match. +<A NAME="lbAE"> </A> +<H2>OPTIONS</H2> + +<DL COMPACT> +<DT>-e <pattern> <DD> +Specifies the search pattern causing an event which shall trigger the command. +<DT>-t <number>:<sec> <DD> +Defines the number of pattern match within the the defined number of seconds in order to trigger the command execution. By default, every pattern match causes a command execution. +<DT>-c <pattern> [<command string>] <DD> +Pattern which clears the event counter. Executes optionally a command if an event command has been executed before. +<DT>-p <DD> +Writes data also to stdout (for piped logging). +<DT>-u <name> <DD> +Become another user, e.g. www-data. +<DT><command string> <DD> +Defines the event command string where $0-$9 are substituted by the submatches of the regular expression. +</DL> +<A NAME="lbAF"> </A> +<H2>EXAMPLE</H2> + +Executes the deny.sh script providing the IP address of the client causing a mod_qos(031) messages whenever the log message appears 10 times within at most one minute: +<BR> ErrorLog "|/usr/bin/qsexec -e \'mod_qos\(031\).*, c=([0-9a-zA-Z:.]*)\' -t 10:60 \'/usr/local/bin/deny.sh $1\'" +<P> +<A NAME="lbAG"> </A> +<H2>SEE ALSO</H2> + +<A HREF="qsdt.1.html">qsdt</A>(1), <A HREF="qsfilter2.1.html">qsfilter2</A>(1), <A HREF="qsgeo.1.html">qsgeo</A>(1), <A HREF="qsgrep.1.html">qsgrep</A>(1), <A HREF="qshead.1.html">qshead</A>(1), <A HREF="qslog.1.html">qslog</A>(1), <A HREF="qslogger.1.html">qslogger</A>(1), <A HREF="qspng.1.html">qspng</A>(1), <A HREF="qsre.1.html">qsre</A>(1), <A HREF="qsrespeed.1.html">qsrespeed</A>(1), <A HREF="qsrotate.1.html">qsrotate</A>(1), <A HREF="qssign.1.html">qssign</A>(1), <A HREF="qstail.1.html">qstail</A>(1) +<A NAME="lbAH"> </A> +<H2>AUTHOR</H2> + +Pascal Buchbinder, <A HREF="http://mod-qos.sourceforge.net/">http://mod-qos.sourceforge.net/</A> +<P> + +<HR> +<A NAME="index"> </A><H2>Index</H2> +<DL> +<DT><A HREF="#lbAB">NAME</A><DD> +<DT><A HREF="#lbAC">SYNOPSIS</A><DD> +<DT><A HREF="#lbAD">DESCRIPTION</A><DD> +<DT><A HREF="#lbAE">OPTIONS</A><DD> +<DT><A HREF="#lbAF">EXAMPLE</A><DD> +<DT><A HREF="#lbAG">SEE ALSO</A><DD> +<DT><A HREF="#lbAH">AUTHOR</A><DD> +</DL> +<HR> + +</BODY> +</HTML> diff --git a/doc/qsfilter2.1.html b/doc/qsfilter2.1.html new file mode 100644 index 0000000..dd4aa06 --- /dev/null +++ b/doc/qsfilter2.1.html @@ -0,0 +1,127 @@ + +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<HTML><HEAD><TITLE>Man page of QSFILTER2</TITLE> +<META name='KeyWords' content='Quality of Service, QoS, Apache Web Server, Web application security, WAF, Open Source Software, Security, Proxy'/> +<META name='author' content='Pascal Buchbinder' /> +</HEAD><BODY> +<H1>QSFILTER2</H1> +Section: qsfilter2 man page (1)<BR>Updated: May 2023<BR><A HREF="#index">Index</A> +<A HREF="index.html#utilities">Return to Main Contents</A><HR> + +<P> +<A NAME="lbAB"> </A> +<H2>NAME</H2> + +qsfilter2 - an utility to generate mod_qos request line rules out from existing access/audit log data. +<A NAME="lbAC"> </A> +<H2>SYNOPSIS</H2> + +qsfilter2 -i <path> [-c <path>] [-d <num>] [-h] [-b <num>] [-p|-s|-m|-o] [-l <len>] [-n] [-e] [-u 'uni'] [-k <prefix>] [-t] [-f <path>] [-v 0|1|2] +<A NAME="lbAD"> </A> +<H2>DESCRIPTION</H2><p><img src="qsfilter2_process.gif" alt="overview"></p> + +mod_qos implements a request filter which validates each request line. The module supports both, negative and positive security model. The QS_Deny* directives are used to specify request line patterns which are not allowed to access the server (negative security model / deny list). These rules are used to restrict access to certain resources which should not be available to users or to protect the server from malicious patterns. The QS_Permit* rules implement a positive security model (allow list). These directives are used to define allowed request line patterns. Request which do not match any of these patterns are not allowed to access the server. +<P> +qsfilter2 is an audit log analyzer used to generate filter rules (perl compatible regular expressions) which may be used by mod_qos to deny access for suspect requests (QS_PermitUri rules). It parses existing audit log files in order to generate request patterns covering all allowed requests. +<A NAME="lbAE"> </A> +<H2>OPTIONS</H2> + +<DL COMPACT> +<DT>-i <path> <DD> +Input file containing request URIs. The URIs for this file have to be extracted from the servers access logs. Each line of the input file contains a request URI consisting of a path and and query. +<BR> Example: +<BR> /aaa/index.do +<BR> /aaa/edit?image=1.jpg +<BR> /aaa/image/1.jpg +<BR> /aaa/view?page=1 +<BR> /aaa/edit?document=1 +<P> +These access log data must include current request URIs but also request lines from previous rule generation steps. It must also include request lines which cover manually generated rules. You may use the 'qos-path' and 'qos-query' variables to create an audit log containing all request data (path and query/body data). Example: 'CustomLog audit_log %{qos-path}n%{qos-query}n'. See also <A HREF="http://mod-qos.sourceforge.net#qsfiltersample">http://mod-qos.sourceforge.net#qsfiltersample</A> about the module settings. +<DT>-c <path> <DD> +mod_qos configuration file defining QS_DenyRequestLine and QS_PermitUri directives. qsfilter2 generates rules from access log data automatically. Manually generated rules (QS_PermitUri) may be provided from this file. Note: each manual rule must be represented by a request URI in the input data (-i) in order to make sure not to be deleted by the rule optimisation algorithm. QS_Deny* rules from this file are used to filter request lines which should not be used for allow list rule generation. +<BR> Example: +<BR> # manually defined allow list rule: +<BR> QS_PermitUri +view deny "^[/a-zA-Z0-9]+/view\?(page=[0-9]+)?$" +<BR> # filter unwanted request line patterns: +<BR> QS_DenyRequestLine +printable deny ".*[\x00-\x19].*" +<P> +<P> +<DT>-d <num> <DD> +Depth (sub locations) of the path string which is defined as a literal string. Default is 1. +<DT>-h <DD> +Always use a string representing the handler name in the path even the url does not have a query. See also -d option. +<DT>-b <num> <DD> +Replaces url pattern by the regular expression when detecting a base64/hex encoded string. Detecting sensibility is defined by a numeric value. You should use values higher than 5 (default) or 0 to disable this function. +<DT>-p <DD> +Represents query by pcre only (no literal strings). +<DT>-s <DD> +Uses one single pcre for the whole query string. +<DT>-m <DD> +Uses one pcre for multiple query values (recommended mode). +<DT>-o <DD> +Does not care the order of query parameters. +<DT>-l <len> <DD> +Outsizes the query length by the defined length ({0,size+len}), default is 10. +<DT>-n <DD> +Disables redundant rules elimination. +<DT>-e <DD> +Exit on error. +<DT>-u 'uni' <DD> +Enables additional decoding methods. Use the same settings as you have used for the QS_Decoding directive. +<DT>-k <prefix> <DD> +Prefix used to generate rule identifiers (QSF by default). +<DT>-t <DD> +Calculates the maximal latency per request (worst case) using the generated rules. +<DT>-f <path> <DD> +Filters the input by the provided path (prefix) only processing matching lines. +<DT>-v <level> <DD> +Verbose mode. (0=silent, 1=rule source, 2=detailed). Default is 1. Don't use rules you haven't checked the request data used to generate it! Level 1 is highly recommended (as long as you don't have created the log data using your own web crawler). +</DL> +<A NAME="lbAF"> </A> +<H2>OUTPUT</H2> + +The output of qsfilter2 is written to stdout. The output contains the generated QS_PermitUri directives but also information about the source which has been used to generate these rules. It is very important to check the validity of each request line which has been used to calculate the QS_PermitUri rules. Each request line which has been used to generate a new rule is shown in the output prefixed by "ADD line <line number>:". These request lines should be stored and reused at any later rule generation (add them to the URI input file). The subsequent line shows the generated rule. At the end of data processing a list of all generated QS_PermitUri rules is shown. These directives may be used withn the configuration file used by mod_qos. +<A NAME="lbAG"> </A> +<H2>EXAMPLE</H2> + +<BR> qsfilter2 -i loc.txt -c httpd.conf -m -e +<BR> ... +<BR> # ADD line 1: /aaa/index.do +<BR> # 003 ^(/[a-zA-Z0-9\-_]+)+[/]?\.?[a-zA-Z]{0,4}$ +<BR> # ADD line 3: /aaa/view?page=1 +<BR> # --- ^[/a-zA-Z0-9]+/view\?(page=[0-9]+)?$ +<BR> # ADD line 4: /aaa/edit?document=1 +<BR> # 004 ^[/a-zA-Z]+/edit\?((document)(=[0-9]*)*[&]?)*$ +<BR> # ADD line 5: /aaa/edit?image=1.jpg +<BR> # 005 ^[/a-zA-Z]+/edit\?((image)(=[0-9\.a-zA-Z]*)*[&]?)*$ +<BR> ... +<BR> QS_PermitUri +QSF001 deny "^[/a-zA-Z]+/edit\?((document|image)(=[0-9\.a-zA-Z]*)*[&]?)*$" +<BR> QS_PermitUri +QSF002 deny "^[/a-zA-Z0-9]+/view\?(page=[0-9]+)?$" +<BR> QS_PermitUri +QSF003 deny "^(/[a-zA-Z0-9\-_]+)+[/]?\.?[a-zA-Z]{0,4}$" +<P> +<A NAME="lbAH"> </A> +<H2>SEE ALSO</H2> + +<A HREF="qsdt.1.html">qsdt</A>(1), <A HREF="qsexec.1.html">qsexec</A>(1), <A HREF="qsgeo.1.html">qsgeo</A>(1), <A HREF="qsgrep.1.html">qsgrep</A>(1), <A HREF="qshead.1.html">qshead</A>(1), <A HREF="qslog.1.html">qslog</A>(1), <A HREF="qslogger.1.html">qslogger</A>(1), <A HREF="qspng.1.html">qspng</A>(1), <A HREF="qsre.1.html">qsre</A>(1), <A HREF="qsrespeed.1.html">qsrespeed</A>(1), <A HREF="qsrotate.1.html">qsrotate</A>(1), <A HREF="qssign.1.html">qssign</A>(1), <A HREF="qstail.1.html">qstail</A>(1) +<A NAME="lbAI"> </A> +<H2>AUTHOR</H2> + +Pascal Buchbinder, <A HREF="http://mod-qos.sourceforge.net/">http://mod-qos.sourceforge.net/</A> +<P> + +<HR> +<A NAME="index"> </A><H2>Index</H2> +<DL> +<DT><A HREF="#lbAB">NAME</A><DD> +<DT><A HREF="#lbAC">SYNOPSIS</A><DD> +<DT><A HREF="#lbAD">DESCRIPTION</A><DD> +<DT><A HREF="#lbAE">OPTIONS</A><DD> +<DT><A HREF="#lbAF">OUTPUT</A><DD> +<DT><A HREF="#lbAG">EXAMPLE</A><DD> +<DT><A HREF="#lbAH">SEE ALSO</A><DD> +<DT><A HREF="#lbAI">AUTHOR</A><DD> +</DL> +<HR> + +</BODY> +</HTML> diff --git a/doc/qsfilter2_process.gif b/doc/qsfilter2_process.gif Binary files differnew file mode 100644 index 0000000..907d466 --- /dev/null +++ b/doc/qsfilter2_process.gif diff --git a/doc/qsgeo.1.html b/doc/qsgeo.1.html new file mode 100644 index 0000000..33487ab --- /dev/null +++ b/doc/qsgeo.1.html @@ -0,0 +1,77 @@ + +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<HTML><HEAD><TITLE>Man page of QSGEO</TITLE> +<META name='KeyWords' content='Quality of Service, QoS, Apache Web Server, Web application security, WAF, Open Source Software, Security, Proxy'/> +<META name='author' content='Pascal Buchbinder' /> +</HEAD><BODY> +<H1>QSGEO</H1> +Section: qsgeo man page (1)<BR>Updated: May 2023<BR><A HREF="#index">Index</A> +<A HREF="index.html#utilities">Return to Main Contents</A><HR> + +<P> +<A NAME="lbAB"> </A> +<H2>NAME</H2> + +qsgeo - an utility to lookup a client's country code. +<A NAME="lbAC"> </A> +<H2>SYNOPSIS</H2> + +qsgeo -d <path> [-l] [-s] [-ip <ip>] +<A NAME="lbAD"> </A> +<H2>DESCRIPTION</H2> + +Use this utility to resolve the country codes of IP addresses within existing log files. The utility reads the log file data from stdin and writes them, with the injected country code, to stdout. +<A NAME="lbAE"> </A> +<H2>OPTIONS</H2> + +<P> +<DL COMPACT> +<DT>-d <path> <DD> +Specifies the path to the geographical database files (CSV file containing IP address ranges and country codes). +<DT>-s <DD> +Writes a summary of the requests per country only. +<DT>-l <DD> +Writes the database to stdout (ignoring stdin) inserting local (127.*) and private (10.*, 172.16*, 192.168.*) network addresses. +<DT>-ip <ip> <DD> +Resolves a single IP address instead of processing a log file. +</DL> +<A NAME="lbAF"> </A> +<H2>EXAMPLE</H2> + +Reading the file access.log and adding the country code to the IP address field: +<P> +<BR> cat access.log | qsgeo -d GeoIPCountryWhois.csv +<P> +Reading the file access.log and showing a summary only: +<P> +<BR> cat access.log | qsgeo -d GeoIPCountryWhois.csv -s +<P> +Resolving a single IP address: +<P> +<BR> qsgeo -d GeoIPCountryWhois.csv -ip 192.84.12.23 +<P> +<A NAME="lbAG"> </A> +<H2>SEE ALSO</H2> + +<A HREF="qsdt.1.html">qsdt</A>(1), <A HREF="qsexec.1.html">qsexec</A>(1), <A HREF="qsfilter2.1.html">qsfilter2</A>(1), <A HREF="qsgrep.1.html">qsgrep</A>(1), <A HREF="qshead.1.html">qshead</A>(1), <A HREF="qslog.1.html">qslog</A>(1), <A HREF="qslogger.1.html">qslogger</A>(1), <A HREF="qspng.1.html">qspng</A>(1), <A HREF="qsre.1.html">qsre</A>(1), <A HREF="qsrespeed.1.html">qsrespeed</A>(1), <A HREF="qsrotate.1.html">qsrotate</A>(1), <A HREF="qssign.1.html">qssign</A>(1), <A HREF="qstail.1.html">qstail</A>(1) +<A NAME="lbAH"> </A> +<H2>AUTHOR</H2> + +Pascal Buchbinder, <A HREF="http://mod-qos.sourceforge.net/">http://mod-qos.sourceforge.net/</A> +<P> + +<HR> +<A NAME="index"> </A><H2>Index</H2> +<DL> +<DT><A HREF="#lbAB">NAME</A><DD> +<DT><A HREF="#lbAC">SYNOPSIS</A><DD> +<DT><A HREF="#lbAD">DESCRIPTION</A><DD> +<DT><A HREF="#lbAE">OPTIONS</A><DD> +<DT><A HREF="#lbAF">EXAMPLE</A><DD> +<DT><A HREF="#lbAG">SEE ALSO</A><DD> +<DT><A HREF="#lbAH">AUTHOR</A><DD> +</DL> +<HR> + +</BODY> +</HTML> diff --git a/doc/qsgrep.1.html b/doc/qsgrep.1.html new file mode 100644 index 0000000..6dd5552 --- /dev/null +++ b/doc/qsgrep.1.html @@ -0,0 +1,67 @@ + +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<HTML><HEAD><TITLE>Man page of QSGREP</TITLE> +<META name='KeyWords' content='Quality of Service, QoS, Apache Web Server, Web application security, WAF, Open Source Software, Security, Proxy'/> +<META name='author' content='Pascal Buchbinder' /> +</HEAD><BODY> +<H1>QSGREP</H1> +Section: qsgrep man page (1)<BR>Updated: May 2023<BR><A HREF="#index">Index</A> +<A HREF="index.html#utilities">Return to Main Contents</A><HR> + +<P> +<A NAME="lbAB"> </A> +<H2>NAME</H2> + +qsgrep - prints matching patterns within a file. +<A NAME="lbAC"> </A> +<H2>SYNOPSIS</H2> + +qsgrep -e <pattern> -o <sub string> [<path>] +<A NAME="lbAD"> </A> +<H2>DESCRIPTION</H2> + +qsgrep is a simple tool to search patterns within files. It uses regular expressions to find patterns and prints the submatches within a pre-defined format string. +<A NAME="lbAE"> </A> +<H2>OPTIONS</H2> + +<DL COMPACT> +<DT>-e <pattern> <DD> +Specifies the search pattern. +<DT>-o <string> <DD> +Defines the output string where $0-$9 are substituted by the submatches of the regular expression. +<DT><path> <DD> +Defines the input file to process. qsgrep reads from from standard input if this parameter is omitted. +<P> +</DL> +<A NAME="lbAF"> </A> +<H2>EXAMPLE</H2> + +Shows the IP addresses of clients causing mod_qos(031) messages): +<P> +<BR> qsgrep -e 'mod_qos\(031\).*, c=([a-zA-Z0-9:.]*)' -o 'ip=$1' error_log +<P> +<A NAME="lbAG"> </A> +<H2>SEE ALSO</H2> + +<A HREF="qsdt.1.html">qsdt</A>(1), <A HREF="qsexec.1.html">qsexec</A>(1), <A HREF="qsfilter2.1.html">qsfilter2</A>(1), <A HREF="qsgeo.1.html">qsgeo</A>(1), <A HREF="qshead.1.html">qshead</A>(1), <A HREF="qslog.1.html">qslog</A>(1), <A HREF="qslogger.1.html">qslogger</A>(1), <A HREF="qspng.1.html">qspng</A>(1), <A HREF="qsre.1.html">qsre</A>(1), <A HREF="qsrespeed.1.html">qsrespeed</A>(1), <A HREF="qsrotate.1.html">qsrotate</A>(1), <A HREF="qssign.1.html">qssign</A>(1), <A HREF="qstail.1.html">qstail</A>(1) +<A NAME="lbAH"> </A> +<H2>AUTHOR</H2> + +Pascal Buchbinder, <A HREF="http://mod-qos.sourceforge.net/">http://mod-qos.sourceforge.net/</A> +<P> + +<HR> +<A NAME="index"> </A><H2>Index</H2> +<DL> +<DT><A HREF="#lbAB">NAME</A><DD> +<DT><A HREF="#lbAC">SYNOPSIS</A><DD> +<DT><A HREF="#lbAD">DESCRIPTION</A><DD> +<DT><A HREF="#lbAE">OPTIONS</A><DD> +<DT><A HREF="#lbAF">EXAMPLE</A><DD> +<DT><A HREF="#lbAG">SEE ALSO</A><DD> +<DT><A HREF="#lbAH">AUTHOR</A><DD> +</DL> +<HR> + +</BODY> +</HTML> diff --git a/doc/qshead.1.html b/doc/qshead.1.html new file mode 100644 index 0000000..5ef66d0 --- /dev/null +++ b/doc/qshead.1.html @@ -0,0 +1,54 @@ + +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<HTML><HEAD><TITLE>Man page of QSHEAD</TITLE> +<META name='KeyWords' content='Quality of Service, QoS, Apache Web Server, Web application security, WAF, Open Source Software, Security, Proxy'/> +<META name='author' content='Pascal Buchbinder' /> +</HEAD><BODY> +<H1>QSHEAD</H1> +Section: qshead man page (1)<BR>Updated: May 2023<BR><A HREF="#index">Index</A> +<A HREF="index.html#utilities">Return to Main Contents</A><HR> + +<P> +<A NAME="lbAB"> </A> +<H2>NAME</H2> + +qshead - an utility reading from stdin and printing all lines to stdout until reaching the defined pattern. +<A NAME="lbAC"> </A> +<H2>SYNOPSIS</H2> + +qshead -p <pattern> +<A NAME="lbAD"> </A> +<H2>DESCRIPTION</H2> + +qshead reads lines from stdin and prints them to stdout until a line contains the specified pattern (literal string). +<A NAME="lbAE"> </A> +<H2>OPTIONS</H2> + +<DL COMPACT> +<DT>-p <pattern> <DD> +Search pattern (literal string). +</DL> +<A NAME="lbAF"> </A> +<H2>SEE ALSO</H2> + +<A HREF="qsdt.1.html">qsdt</A>(1), <A HREF="qsexec.1.html">qsexec</A>(1), <A HREF="qsfilter2.1.html">qsfilter2</A>(1), <A HREF="qsgeo.1.html">qsgeo</A>(1), <A HREF="qsgrep.1.html">qsgrep</A>(1), <A HREF="qslog.1.html">qslog</A>(1), <A HREF="qslogger.1.html">qslogger</A>(1), <A HREF="qspng.1.html">qspng</A>(1), <A HREF="qsre.1.html">qsre</A>(1), <A HREF="qsrespeed.1.html">qsrespeed</A>(1), <A HREF="qsrotate.1.html">qsrotate</A>(1), <A HREF="qssign.1.html">qssign</A>(1) <A HREF="qstail.1.html">qstail</A>(1) +<A NAME="lbAG"> </A> +<H2>AUTHOR</H2> + +Pascal Buchbinder, <A HREF="http://mod-qos.sourceforge.net/">http://mod-qos.sourceforge.net/</A> +<P> + +<HR> +<A NAME="index"> </A><H2>Index</H2> +<DL> +<DT><A HREF="#lbAB">NAME</A><DD> +<DT><A HREF="#lbAC">SYNOPSIS</A><DD> +<DT><A HREF="#lbAD">DESCRIPTION</A><DD> +<DT><A HREF="#lbAE">OPTIONS</A><DD> +<DT><A HREF="#lbAF">SEE ALSO</A><DD> +<DT><A HREF="#lbAG">AUTHOR</A><DD> +</DL> +<HR> + +</BODY> +</HTML> diff --git a/doc/qslog.1.html b/doc/qslog.1.html new file mode 100644 index 0000000..349bf22 --- /dev/null +++ b/doc/qslog.1.html @@ -0,0 +1,146 @@ + +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<HTML><HEAD><TITLE>Man page of QSLOG</TITLE> +<META name='KeyWords' content='Quality of Service, QoS, Apache Web Server, Web application security, WAF, Open Source Software, Security, Proxy'/> +<META name='author' content='Pascal Buchbinder' /> +</HEAD><BODY> +<H1>QSLOG</H1> +Section: qslog man page (1)<BR>Updated: May 2023<BR><A HREF="#index">Index</A> +<A HREF="index.html#utilities">Return to Main Contents</A><HR> + +<P> +<A NAME="lbAB"> </A> +<H2>NAME</H2> + +qslog - collects request statistics from access log data. +<A NAME="lbAC"> </A> +<H2>SYNOPSIS</H2> + +qslog -f <format_string> -o <out_file> [-p[c|u[c]] [-v]] [-x [<num>]] [-u <name>] [-m] [-c <path>] +<A NAME="lbAD"> </A> +<H2>DESCRIPTION</H2> + +qslog is a real time access log analyzer. It collects the data from stdin. The output is written to the specified file every minute and includes the following entries: +<BR> - requests per second (r/s) +<BR> - number of requests within measured time (req) +<BR> - bytes sent to the client per second (b/s) +<BR> - bytes received from the client per second (ib/s) +<BR> - response status codes within the last minute (1xx,2xx,3xx,4xx,5xx) +<BR> - average response duration (av) +<BR> - average response duration in milliseconds (avms) +<BR> - distribution of response durations in seconds within the last minute +(<1s,1s,2s,3s,4s,5s,>5s) +<BR> - distribution of response durations faster than a second within the last minute +(0-49ms,50-99ms,100-499ms,500-999ms) +<BR> - number of established (new) connections within the measured time (esco) +<BR> - average system load (sl) +<BR> - free memory (m) (not available for all platforms) +<BR> - number of client ip addresses seen withn the last 600 seconds (ip) +<BR> - number of different users seen withn the last 600 seconds (usr) +<BR> - number of events identified by the 'E' format character +<BR> - number of mod_qos events within the last minute (qV=create session, +qv=VIP IP,qS=session pass, qD=access denied, qK=connection closed, qT=dynamic keep-alive, qL=request/response slow down, qs=serialized request, qA=connection abort, qU=new user tracking cookie) +<A NAME="lbAE"> </A> +<H2>OPTIONS</H2> + +<DL COMPACT> +<DT>-f <format_string> <DD> +Defines the log data format and the positions of data elements processed by this utility. See to the 'LogFormat' directive of the httpd.conf file to see the format definitions of the servers access log data. +<BR> qslog knows the following elements: +<BR> I defines the client ip address (%h) +<BR> R defines the request line (%r) +<BR> S defines HTTP response status code (%s) +<BR> B defines the transferred bytes (%b or %O) +<BR> i defines the received bytes (%I) +<BR> D defines the request duration in microseconds (%D) +<BR> t defines the request duration in milliseconds (may be used instead of D) +<BR> T defines the request duration in seconds (may be used instead of D or t) (%T) +<BR> k defines the number of keepalive requests on the connection (%k) +<BR> U defines the user tracking id (%{mod_qos_user_id}e) +<BR> Q defines the mod_qos_ev event message (%{mod_qos_ev}e) +<BR> C defines the element for the detailed log (-c option), e.g. "%U" +<BR> s arbitrary counter to add up (sum within a minute) +<BR> a arbitrary counter to build an average from (average per request) +<BR> A arbitrary counter to build an average from (average per request) +<BR> M arbitrary counter to measure the maximum value reached (peak) +<BR> E comma separated list of event strings +<BR> c content type (%{content-type}o), available in -pc mode only +<BR> m request method (GET/POST) (%m), available in -pc mode only +<BR> . defines an element to ignore (unknown string) +<P> +<DT>-o <out_file> <DD> +Specifies the file to store the output to. stdout is used if this option is not defined. +<DT>-p <DD> +Used for post processing when reading the log data from a file (cat/pipe). qslog is started using it's offline mode (extracting the time stamps from the log lines) in order to process existing log files. The option "-pc" may be used alternatively if you want to gather request information per client (identified by IP address (I) or user tracking id (U) showing how many request each client has performed within the captured period of time). "-pc" supports the format characters IURSBTtDkMEcm. The option "-pu" collects statistics on a per URL level (supports format characters RSTtD). "-puc" is very similar to "-pu" but cuts the end (handler) of each URL. +<DT>-v <DD> +Verbose mode. +<DT>-x [<num>] <DD> +Rotates the output file once a day (move). You may specify the number of rotated files to keep. Default are 14. +<DT>-u <name> <DD> +Becomes another user, e.g. www-data. +<DT>-m <DD> +Calculates free system memory every minute. +<DT>-c <path> <DD> +Enables the collection of log statistics for different request types. 'path' specifies the necessary rule file. Each rule consists of a rule identifier and a regular expression to identify a request seprarated by a colon, e.g., 01:^(/a)|(/c). The regular expressions are matched against the log data element which has been identified by the 'C' format character. +</DL> +<A NAME="lbAF"> </A> +<H2>VARIABLES</H2> + +The following environment variables are known to qslog: +<DL COMPACT> +<DT>QSEVENTPATH=<path> <DD> +Defines a file containing a comma or new line separated list of known event strings expected within the log filed identified by the 'E' format character. +<DT>QSCOUNTERPATH=<path> <DD> +Defines a file containing a by new line separated list of rules which reflect possible QS_ClientEventLimitCount directive settings (for simulation purpose / -pc option). The 'E' format character defines the event string in the log to match (literal string) the 'event1' and 'event2' event names against. +<P> +Rule syntax: <name>:<event1>-<n>*<event2>/<duration>=<limit> +<BR> 'name' defines the name you have given to the rule entry and is logged along with +with the number of times the 'limit' has been reached within the 'duration'. +<BR> 'event1' defines the variable name (if found in 'E') to increment the counter. +<BR> 'event2' defines the variable name (if found in 'E') to decrement the counter (and +the parameter 'n' defines by how much). +<BR> 'duration' defines the measure interval (in seconds) used for the +QS_ClientEventLimitCount directive. +<BR> 'limit' defines the threshold (number) defined for the QS_ClientEventLimitCount +directive. +<P> +Note: If the 'name' parameter is prefixed by 'STATUS', the rule is applied against the HTTP status code 'S' and the 'event1' string shall contain a list of relevant status codes separated by an underscore (while 'event2' is ignored). +</DL> +<A NAME="lbAG"> </A> +<H2>EXAMPLE</H2> + +Configuration using pipped logging: +<P> +<BR> CustomLog "|/usr/bin/qslog -f ISBDQ -x -o /var/log/apache/stat.csv" "%h %>s %b %D %{mod_qos_ev}e" +<P> +Post processing: +<P> +<BR> LogFormat "%t %h \"%r\" %>s %b \"%{User-Agent}i\" %T" +<BR> cat access.log | qslog -f ..IRSB.T -o stat.csv -p +<P> +<A NAME="lbAH"> </A> +<H2>SEE ALSO</H2> + +<A HREF="qsdt.1.html">qsdt</A>(1), <A HREF="qsexec.1.html">qsexec</A>(1), <A HREF="qsfilter2.1.html">qsfilter2</A>(1), <A HREF="qsgeo.1.html">qsgeo</A>(1), <A HREF="qsgrep.1.html">qsgrep</A>(1), <A HREF="qshead.1.html">qshead</A>(1), <A HREF="qslogger.1.html">qslogger</A>(1), <A HREF="qspng.1.html">qspng</A>(1), <A HREF="qsre.1.html">qsre</A>(1), <A HREF="qsrespeed.1.html">qsrespeed</A>(1), <A HREF="qsrotate.1.html">qsrotate</A>(1), <A HREF="qssign.1.html">qssign</A>(1), <A HREF="qstail.1.html">qstail</A>(1) +<A NAME="lbAI"> </A> +<H2>AUTHOR</H2> + +Pascal Buchbinder, <A HREF="http://mod-qos.sourceforge.net/">http://mod-qos.sourceforge.net/</A> +<P> + +<HR> +<A NAME="index"> </A><H2>Index</H2> +<DL> +<DT><A HREF="#lbAB">NAME</A><DD> +<DT><A HREF="#lbAC">SYNOPSIS</A><DD> +<DT><A HREF="#lbAD">DESCRIPTION</A><DD> +<DT><A HREF="#lbAE">OPTIONS</A><DD> +<DT><A HREF="#lbAF">VARIABLES</A><DD> +<DT><A HREF="#lbAG">EXAMPLE</A><DD> +<DT><A HREF="#lbAH">SEE ALSO</A><DD> +<DT><A HREF="#lbAI">AUTHOR</A><DD> +</DL> +<HR> + +</BODY> +</HTML> diff --git a/doc/qslogger.1.html b/doc/qslogger.1.html new file mode 100644 index 0000000..81029e9 --- /dev/null +++ b/doc/qslogger.1.html @@ -0,0 +1,75 @@ + +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<HTML><HEAD><TITLE>Man page of QSLOGGER</TITLE> +<META name='KeyWords' content='Quality of Service, QoS, Apache Web Server, Web application security, WAF, Open Source Software, Security, Proxy'/> +<META name='author' content='Pascal Buchbinder' /> +</HEAD><BODY> +<H1>QSLOGGER</H1> +Section: qslogger man page (1)<BR>Updated: May 2023<BR><A HREF="#index">Index</A> +<A HREF="index.html#utilities">Return to Main Contents</A><HR> + +<P> +<A NAME="lbAB"> </A> +<H2>NAME</H2> + +qslogger - another shell command interface to the system log module (syslog). +<A NAME="lbAC"> </A> +<H2>SYNOPSIS</H2> + +qslogger [-t <tag>] [-f <facility>] [-l <level>] [-x <prefix>] [-r <expression>] [-d <level>] [-u <name>] [-p] +<A NAME="lbAD"> </A> +<H2>DESCRIPTION</H2> + +Use this utility to forward log messages to the systems syslog facility, e.g., to forward the messages to a remote host. It reads data from stdin. +<A NAME="lbAE"> </A> +<H2>OPTIONS</H2> + +<P> +<DL COMPACT> +<DT>-t <tag> <DD> +Defines the tag name which shall be used to define the origin of the messages, e.g. 'httpd'. +<DT>-f <facility> <DD> +Defines the syslog facility. Default is 'daemon'. +<DT>-u <name> <DD> +Becomes another user, e.g. www-data. +<DT>-l <level> <DD> +Defines the minimal severity a message must have in order to be forwarded. Default is 'DEBUG' (forwarding everything). +<DT>-x <prefix> <DD> +Allows you to add a prefix (literal string) to every message. +<DT>-r <expression> <DD> +Specifies a regular expression which shall be used to determine the severity (syslog level) for each log line. The default pattern '^\[[0-9a-zA-Z :]+\] \[([a-z]+)\] ' can be used for Apache error log messages but you may configure your own pattern matching other log formats. Use brackets to define the pattern enclosing the severity string. Default level (if severity can't be determined) is defined by the option '-d' (see below). +<DT>-d <level> <DD> +The default severity if the specified pattern (-r) does not match and the message's severity can't be determined. Default is 'NOTICE'. +<DT>-p <DD> +Writes data also to stdout (for piped logging). +</DL> +<A NAME="lbAF"> </A> +<H2>EXAMPLE</H2> + +<BR> ErrorLog "|/usr/bin/qslogger -t apache -f local7" +<P> +<A NAME="lbAG"> </A> +<H2>SEE ALSO</H2> + +<A HREF="qsdt.1.html">qsdt</A>(1), <A HREF="qsexec.1.html">qsexec</A>(1), <A HREF="qsfilter2.1.html">qsfilter2</A>(1), <A HREF="qsgeo.1.html">qsgeo</A>(1), <A HREF="qsgrep.1.html">qsgrep</A>(1), <A HREF="qshead.1.html">qshead</A>(1), <A HREF="qslog.1.html">qslog</A>(1), <A HREF="qspng.1.html">qspng</A>(1), <A HREF="qsre.1.html">qsre</A>(1), <A HREF="qsrespeed.1.html">qsrespeed</A>(1), <A HREF="qsrotate.1.html">qsrotate</A>(1), <A HREF="qssign.1.html">qssign</A>(1), <A HREF="qstail.1.html">qstail</A>(1) +<A NAME="lbAH"> </A> +<H2>AUTHOR</H2> + +Pascal Buchbinder, <A HREF="http://mod-qos.sourceforge.net/">http://mod-qos.sourceforge.net/</A> +<P> + +<HR> +<A NAME="index"> </A><H2>Index</H2> +<DL> +<DT><A HREF="#lbAB">NAME</A><DD> +<DT><A HREF="#lbAC">SYNOPSIS</A><DD> +<DT><A HREF="#lbAD">DESCRIPTION</A><DD> +<DT><A HREF="#lbAE">OPTIONS</A><DD> +<DT><A HREF="#lbAF">EXAMPLE</A><DD> +<DT><A HREF="#lbAG">SEE ALSO</A><DD> +<DT><A HREF="#lbAH">AUTHOR</A><DD> +</DL> +<HR> + +</BODY> +</HTML> diff --git a/doc/qspng.1.html b/doc/qspng.1.html new file mode 100644 index 0000000..a27b74d --- /dev/null +++ b/doc/qspng.1.html @@ -0,0 +1,58 @@ + +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<HTML><HEAD><TITLE>Man page of QSPNG</TITLE> +<META name='KeyWords' content='Quality of Service, QoS, Apache Web Server, Web application security, WAF, Open Source Software, Security, Proxy'/> +<META name='author' content='Pascal Buchbinder' /> +</HEAD><BODY> +<H1>QSPNG</H1> +Section: qspng man page (1)<BR>Updated: May 2023<BR><A HREF="#index">Index</A> +<A HREF="index.html#utilities">Return to Main Contents</A><HR> + +<P> +<A NAME="lbAB"> </A> +<H2>NAME</H2> + +qspng - an utility to draw a png graph from <A HREF="qslog.1.html">qslog</A>(1) output data. +<A NAME="lbAC"> </A> +<H2>SYNOPSIS</H2> + +qspng -i <stat_log_file> -p <parameter> -o <out_file> [-10] +<A NAME="lbAD"> </A> +<H2>DESCRIPTION</H2> + +qspng is a tool to generate png (portable network graphics) raster images files from semicolon separated data generated by the qslog utility. It reads up to the first 1440 entries (24 hours) and prints a graph using the values defined by the 'parameter' name. +<A NAME="lbAE"> </A> +<H2>OPTIONS</H2> + +<DL COMPACT> +<DT>-i <stats_log_file> <DD> +Input file to read data from. +<DT>-p <parameter> <DD> +Parameter name, e.g. r/s or usr. +<DT>-o <out_file> <DD> +Output file name, e.g. stat.png. +</DL> +<A NAME="lbAF"> </A> +<H2>SEE ALSO</H2> + +<A HREF="qsdt.1.html">qsdt</A>(1), <A HREF="qsexec.1.html">qsexec</A>(1), <A HREF="qsfilter2.1.html">qsfilter2</A>(1), <A HREF="qsgeo.1.html">qsgeo</A>(1), <A HREF="qsgrep.1.html">qsgrep</A>(1), <A HREF="qshead.1.html">qshead</A>(1), <A HREF="qslogger.1.html">qslogger</A>(1), <A HREF="qslog.1.html">qslog</A>(1), <A HREF="qsre.1.html">qsre</A>(1), <A HREF="qsrespeed.1.html">qsrespeed</A>(1), <A HREF="qsrotate.1.html">qsrotate</A>(1), <A HREF="qssign.1.html">qssign</A>(1), <A HREF="qstail.1.html">qstail</A>(1) +<A NAME="lbAG"> </A> +<H2>AUTHOR</H2> + +Pascal Buchbinder, <A HREF="http://mod-qos.sourceforge.net/">http://mod-qos.sourceforge.net/</A> +<P> + +<HR> +<A NAME="index"> </A><H2>Index</H2> +<DL> +<DT><A HREF="#lbAB">NAME</A><DD> +<DT><A HREF="#lbAC">SYNOPSIS</A><DD> +<DT><A HREF="#lbAD">DESCRIPTION</A><DD> +<DT><A HREF="#lbAE">OPTIONS</A><DD> +<DT><A HREF="#lbAF">SEE ALSO</A><DD> +<DT><A HREF="#lbAG">AUTHOR</A><DD> +</DL> +<HR> + +</BODY> +</HTML> diff --git a/doc/qsre.1.html b/doc/qsre.1.html new file mode 100644 index 0000000..39753ba --- /dev/null +++ b/doc/qsre.1.html @@ -0,0 +1,56 @@ + +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<HTML><HEAD><TITLE>Man page of QSRE</TITLE> +<META name='KeyWords' content='Quality of Service, QoS, Apache Web Server, Web application security, WAF, Open Source Software, Security, Proxy'/> +<META name='author' content='Pascal Buchbinder' /> +</HEAD><BODY> +<H1>QSRE</H1> +Section: qsre man page (1)<BR>Updated: May 2023<BR><A HREF="#index">Index</A> +<A HREF="index.html#utilities">Return to Main Contents</A><HR> + +<P> +<A NAME="lbAB"> </A> +<H2>NAME</H2> + +qsre matches a regular expression against test strings. +<A NAME="lbAC"> </A> +<H2>SYNOPSIS</H2> + +qsre <string>|<path> <pcre>|<path> +<A NAME="lbAD"> </A> +<H2>DESCRIPTION</H2> + +Regular expression test tool. The provided regular expression (pcre, caseless matching, "." matches anything incl. newline) is appplied against the provided test strings to verify if the pattern matches. +<A NAME="lbAE"> </A> +<H2>OPTIONS</H2> + +<DL COMPACT> +<DT><string>|<path> <DD> +The first argument either defines a single test string of a path to a file containing either multiple test strings or a test pattern with newline characters (text). +<DT><pcre>|<path> <DD> +The second argument either defines a regular expression or a path to a file containing the expression. +</DL> +<A NAME="lbAF"> </A> +<H2>SEE ALSO</H2> + +<A HREF="qsdt.1.html">qsdt</A>(1), <A HREF="qsexec.1.html">qsexec</A>(1), <A HREF="qsfilter2.1.html">qsfilter2</A>(1), <A HREF="qsgeo.1.html">qsgeo</A>(1), <A HREF="qsgrep.1.html">qsgrep</A>(1), <A HREF="qshead.1.html">qshead</A>(1), <A HREF="qslog.1.html">qslog</A>(1), <A HREF="qslogger.1.html">qslogger</A>(1), <A HREF="qspng.1.html">qspng</A>(1), <A HREF="qsrespeed.1.html">qsrespeed</A>(1), <A HREF="qsrotate.1.html">qsrotate</A>(1), <A HREF="qssign.1.html">qssign</A>(1), <A HREF="qstail.1.html">qstail</A>(1) +<A NAME="lbAG"> </A> +<H2>AUTHOR</H2> + +Pascal Buchbinder, <A HREF="http://mod-qos.sourceforge.net/">http://mod-qos.sourceforge.net/</A> +<P> + +<HR> +<A NAME="index"> </A><H2>Index</H2> +<DL> +<DT><A HREF="#lbAB">NAME</A><DD> +<DT><A HREF="#lbAC">SYNOPSIS</A><DD> +<DT><A HREF="#lbAD">DESCRIPTION</A><DD> +<DT><A HREF="#lbAE">OPTIONS</A><DD> +<DT><A HREF="#lbAF">SEE ALSO</A><DD> +<DT><A HREF="#lbAG">AUTHOR</A><DD> +</DL> +<HR> + +</BODY> +</HTML> diff --git a/doc/qsrespeed.1.html b/doc/qsrespeed.1.html new file mode 100644 index 0000000..4a5b4d9 --- /dev/null +++ b/doc/qsrespeed.1.html @@ -0,0 +1,54 @@ + +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<HTML><HEAD><TITLE>Man page of QSRESPEED</TITLE> +<META name='KeyWords' content='Quality of Service, QoS, Apache Web Server, Web application security, WAF, Open Source Software, Security, Proxy'/> +<META name='author' content='Pascal Buchbinder' /> +</HEAD><BODY> +<H1>QSRESPEED</H1> +Section: qsrespeed man page (1)<BR>Updated: May 2023<BR><A HREF="#index">Index</A> +<A HREF="index.html#utilities">Return to Main Contents</A><HR> + +<P> +<A NAME="lbAB"> </A> +<H2>NAME</H2> + +Tool to compare / estimate the processing time for (Perl-compatible) regular expressions (PCRE). +<A NAME="lbAC"> </A> +<H2>SYNOPSIS</H2> + +qsrespeed <path> +<A NAME="lbAD"> </A> +<H2>DESCRIPTION</H2> + +qsrespeed loads regular expressions from the provided file and matches them against a built-in set of strings measuring the time needed to process them. It's a benchmark too to judge the expressions you have defined regarding the potential CPU consumption. +<A NAME="lbAE"> </A> +<H2>OPTIONS</H2> + +<DL COMPACT> +<DT><path> <DD> +Defines the input file to process. The file consists a list of (separated by a newline character) regular expressions to test +</DL> +<A NAME="lbAF"> </A> +<H2>SEE ALSO</H2> + +<A HREF="qsdt.1.html">qsdt</A>(1), <A HREF="qsexec.1.html">qsexec</A>(1), <A HREF="qsfilter2.1.html">qsfilter2</A>(1), <A HREF="qsgeo.1.html">qsgeo</A>(1), <A HREF="qsgrep.1.html">qsgrep</A>(1), <A HREF="qshead.1.html">qshead</A>(1), <A HREF="qslog.1.html">qslog</A>(1), <A HREF="qslogger.1.html">qslogger</A>(1), <A HREF="qspng.1.html">qspng</A>(1), <A HREF="qsre.1.html">qsre</A>(1), <A HREF="qsrotate.1.html">qsrotate</A>(1), <A HREF="qssign.1.html">qssign</A>(1), <A HREF="qstail.1.html">qstail</A>(1) +<A NAME="lbAG"> </A> +<H2>AUTHOR</H2> + +Pascal Buchbinder, <A HREF="http://mod-qos.sourceforge.net/">http://mod-qos.sourceforge.net/</A> +<P> + +<HR> +<A NAME="index"> </A><H2>Index</H2> +<DL> +<DT><A HREF="#lbAB">NAME</A><DD> +<DT><A HREF="#lbAC">SYNOPSIS</A><DD> +<DT><A HREF="#lbAD">DESCRIPTION</A><DD> +<DT><A HREF="#lbAE">OPTIONS</A><DD> +<DT><A HREF="#lbAF">SEE ALSO</A><DD> +<DT><A HREF="#lbAG">AUTHOR</A><DD> +</DL> +<HR> + +</BODY> +</HTML> diff --git a/doc/qsrotate.1.html b/doc/qsrotate.1.html new file mode 100644 index 0000000..faa787a --- /dev/null +++ b/doc/qsrotate.1.html @@ -0,0 +1,86 @@ + +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<HTML><HEAD><TITLE>Man page of QSROTATE</TITLE> +<META name='KeyWords' content='Quality of Service, QoS, Apache Web Server, Web application security, WAF, Open Source Software, Security, Proxy'/> +<META name='author' content='Pascal Buchbinder' /> +</HEAD><BODY> +<H1>QSROTATE</H1> +Section: qsrotate man page (1)<BR>Updated: May 2023<BR><A HREF="#index">Index</A> +<A HREF="index.html#utilities">Return to Main Contents</A><HR> + +<P> +<A NAME="lbAB"> </A> +<H2>NAME</H2> + +qsrotate - a log rotation tool (similar to Apache's rotatelogs). +<A NAME="lbAC"> </A> +<H2>SYNOPSIS</H2> + +qsrotate -o <file> [-s <sec> [-t <hours>]] [-b <bytes>] [-f] [-z] [-g <num>] [-u <name>] [-m <mask>] [-p] [-d] +<A NAME="lbAD"> </A> +<H2>DESCRIPTION</H2> + +qsrotate reads from stdin (piped log) and writes the data to the provided file rotating the file after the specified time. +<A NAME="lbAE"> </A> +<H2>OPTIONS</H2> + +<DL COMPACT> +<DT>-o <file> <DD> +Output log file to write the data to (use an absolute path). +<DT>-s <sec> <DD> +Rotation interval in seconds, default are 86400 seconds. +<DT>-t <hours> <DD> +Offset to UTC (enables also DST support), default is 0. +<DT>-b <bytes> <DD> +File size limitation (default/max. are 2147352576 bytes, min. are 1048576 bytes). +<DT>-f <DD> +Forced log rotation at the specified interval even no data is written. +<DT>-z <DD> +Compress (gzip) the rotated file. +<DT>-g <num> <DD> +Generations (number of files to keep). +<DT>-u <name> <DD> +Become another user, e.g. www-data. -m <mask> +File permission which is either 600, 640, 660 (default) or 664. +<DT>-p <DD> +Writes data also to stdout (for piped logging). -d +Line-by-line data reading prefixing every line with a timestamp. +</DL> +<A NAME="lbAF"> </A> +<H2>EXAMPLE</H2> + +<BR> TransferLog "|/usr/bin/qsrotate -f -z -g 3 -o /var/log/apache/access.log -s 86400" +<P> +The name of the rotated file will be /dest/filee.YYYYmmddHHMMSS where YYYYmmddHHMMSS is the system time at which the data has been rotated. +<A NAME="lbAG"> </A> +<H2>NOTE</H2> + +<BR> - Each qsrotate instance must use an individual file. +<BR> - You may trigger a file rotation manually by sending the signal USR1 +to the process. +<A NAME="lbAH"> </A> +<H2>SEE ALSO</H2> + +<A HREF="qsdt.1.html">qsdt</A>(1), <A HREF="qsexec.1.html">qsexec</A>(1), <A HREF="qsfilter2.1.html">qsfilter2</A>(1), <A HREF="qsgeo.1.html">qsgeo</A>(1), <A HREF="qsgrep.1.html">qsgrep</A>(1), <A HREF="qshead.1.html">qshead</A>(1), <A HREF="qslog.1.html">qslog</A>(1), <A HREF="qslogger.1.html">qslogger</A>(1), <A HREF="qsre.1.html">qsre</A>(1), <A HREF="qsrespeed.1.html">qsrespeed</A>(1), <A HREF="qspng.1.html">qspng</A>(1), <A HREF="qssign.1.html">qssign</A>(1), <A HREF="qstail.1.html">qstail</A>(1) +<A NAME="lbAI"> </A> +<H2>AUTHOR</H2> + +Pascal Buchbinder, <A HREF="http://mod-qos.sourceforge.net/">http://mod-qos.sourceforge.net/</A> +<P> + +<HR> +<A NAME="index"> </A><H2>Index</H2> +<DL> +<DT><A HREF="#lbAB">NAME</A><DD> +<DT><A HREF="#lbAC">SYNOPSIS</A><DD> +<DT><A HREF="#lbAD">DESCRIPTION</A><DD> +<DT><A HREF="#lbAE">OPTIONS</A><DD> +<DT><A HREF="#lbAF">EXAMPLE</A><DD> +<DT><A HREF="#lbAG">NOTE</A><DD> +<DT><A HREF="#lbAH">SEE ALSO</A><DD> +<DT><A HREF="#lbAI">AUTHOR</A><DD> +</DL> +<HR> + +</BODY> +</HTML> diff --git a/doc/qssign.1.html b/doc/qssign.1.html new file mode 100644 index 0000000..8c434ff --- /dev/null +++ b/doc/qssign.1.html @@ -0,0 +1,79 @@ + +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<HTML><HEAD><TITLE>Man page of QSSIGN</TITLE> +<META name='KeyWords' content='Quality of Service, QoS, Apache Web Server, Web application security, WAF, Open Source Software, Security, Proxy'/> +<META name='author' content='Pascal Buchbinder' /> +</HEAD><BODY> +<H1>QSSIGN</H1> +Section: qssign man page (1)<BR>Updated: May 2023<BR><A HREF="#index">Index</A> +<A HREF="index.html#utilities">Return to Main Contents</A><HR> + +<P> +<A NAME="lbAB"> </A> +<H2>NAME</H2> + +qssign - an utility to sign and verify the integrity of log data. +<A NAME="lbAC"> </A> +<H2>SYNOPSIS</H2> + +qssign -s|S <secret> [-e] [-v] [-u <name>] [-f <regex>] [-a 'sha1'|'sha256'] +<A NAME="lbAD"> </A> +<H2>DESCRIPTION</H2> + +qssign is a log data integrity check tool. It reads log data from stdin (pipe) and writes the data to stdout adding a sequence number and signature to ever log line. +<A NAME="lbAE"> </A> +<H2>OPTIONS</H2> + +<DL COMPACT> +<DT>-s <secret> <DD> +Passphrase used to calculate signature. +<DT>-S <program> <DD> +Specifies a program which writes the passphrase to stdout. +<DT>-e <DD> +Writes start/end marker when starting/stopping data signing. +<DT>-v <DD> +Verification mode checking the integrity of signed data. +<DT>-u <name> <DD> +Becomes another user, e.g. www-data. +<DT>-f <regex> <DD> +Filter pattern (case sensitive regular expression) for messages which do not need to be signed. +<DT>-a 'sha1'|'sha256' <DD> +Specifies the algorithm to use. Default is sha1. +</DL> +<A NAME="lbAF"> </A> +<H2>EXAMPLE</H2> + +Sign: +<P> +<BR> TransferLog "|/usr/bin/qssign -s password -e |/usr/bin/qsrotate -o /var/log/apache/access.log" +<P> +<P> +Verify: +<P> +<BR> cat access.log | qssign -s password -v +<P> +<A NAME="lbAG"> </A> +<H2>SEE ALSO</H2> + +<A HREF="qsdt.1.html">qsdt</A>(1), <A HREF="qsexec.1.html">qsexec</A>(1), <A HREF="qsfilter2.1.html">qsfilter2</A>(1), <A HREF="qsgeo.1.html">qsgeo</A>(1), <A HREF="qsgrep.1.html">qsgrep</A>(1), <A HREF="qshead.1.html">qshead</A>(1), <A HREF="qslog.1.html">qslog</A>(1), <A HREF="qslogger.1.html">qslogger</A>(1), <A HREF="qspng.1.html">qspng</A>(1), <A HREF="qsre.1.html">qsre</A>(1), <A HREF="qsrespeed.1.html">qsrespeed</A>(1), <A HREF="qsrotate.1.html">qsrotate</A>(1), <A HREF="qstail.1.html">qstail</A>(1) +<A NAME="lbAH"> </A> +<H2>AUTHOR</H2> + +Pascal Buchbinder, <A HREF="http://mod-qos.sourceforge.net/">http://mod-qos.sourceforge.net/</A> +<P> + +<HR> +<A NAME="index"> </A><H2>Index</H2> +<DL> +<DT><A HREF="#lbAB">NAME</A><DD> +<DT><A HREF="#lbAC">SYNOPSIS</A><DD> +<DT><A HREF="#lbAD">DESCRIPTION</A><DD> +<DT><A HREF="#lbAE">OPTIONS</A><DD> +<DT><A HREF="#lbAF">EXAMPLE</A><DD> +<DT><A HREF="#lbAG">SEE ALSO</A><DD> +<DT><A HREF="#lbAH">AUTHOR</A><DD> +</DL> +<HR> + +</BODY> +</HTML> diff --git a/doc/qstail.1.html b/doc/qstail.1.html new file mode 100644 index 0000000..cb2cbbf --- /dev/null +++ b/doc/qstail.1.html @@ -0,0 +1,56 @@ + +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<HTML><HEAD><TITLE>Man page of QSTAIL</TITLE> +<META name='KeyWords' content='Quality of Service, QoS, Apache Web Server, Web application security, WAF, Open Source Software, Security, Proxy'/> +<META name='author' content='Pascal Buchbinder' /> +</HEAD><BODY> +<H1>QSTAIL</H1> +Section: qstail man page (1)<BR>Updated: May 2023<BR><A HREF="#index">Index</A> +<A HREF="index.html#utilities">Return to Main Contents</A><HR> + +<P> +<A NAME="lbAB"> </A> +<H2>NAME</H2> + +qstail - an utility printing the end of a log file starting at the specified pattern. +<A NAME="lbAC"> </A> +<H2>SYNOPSIS</H2> + +qstail -i <path> -p <pattern> +<A NAME="lbAD"> </A> +<H2>DESCRIPTION</H2> + +qstail shows the end of a log file beginning with the line containing the specified pattern. This may be used to show all lines which has been written after a certain event (e.g., server restart) or time stamp. +<A NAME="lbAE"> </A> +<H2>OPTIONS</H2> + +<DL COMPACT> +<DT>-i <path> <DD> +Input file to read the data from. +<DT>-p <pattern> <DD> +Search pattern (literal string). +</DL> +<A NAME="lbAF"> </A> +<H2>SEE ALSO</H2> + +<A HREF="qsdt.1.html">qsdt</A>(1), <A HREF="qsexec.1.html">qsexec</A>(1), <A HREF="qsfilter2.1.html">qsfilter2</A>(1), <A HREF="qsgeo.1.html">qsgeo</A>(1), <A HREF="qsgrep.1.html">qsgrep</A>(1), <A HREF="qshead.1.html">qshead</A>(1), <A HREF="qslog.1.html">qslog</A>(1), <A HREF="qslogger.1.html">qslogger</A>(1), <A HREF="qspng.1.html">qspng</A>(1), <A HREF="qsre.1.html">qsre</A>(1), <A HREF="qsrespeed.1.html">qsrespeed</A>(1), <A HREF="qsrotate.1.html">qsrotate</A>(1), <A HREF="qssign.1.html">qssign</A>(1) +<A NAME="lbAG"> </A> +<H2>AUTHOR</H2> + +Pascal Buchbinder, <A HREF="http://mod-qos.sourceforge.net/">http://mod-qos.sourceforge.net/</A> +<P> + +<HR> +<A NAME="index"> </A><H2>Index</H2> +<DL> +<DT><A HREF="#lbAB">NAME</A><DD> +<DT><A HREF="#lbAC">SYNOPSIS</A><DD> +<DT><A HREF="#lbAD">DESCRIPTION</A><DD> +<DT><A HREF="#lbAE">OPTIONS</A><DD> +<DT><A HREF="#lbAF">SEE ALSO</A><DD> +<DT><A HREF="#lbAG">AUTHOR</A><DD> +</DL> +<HR> + +</BODY> +</HTML> |