summaryrefslogtreecommitdiffstats
path: root/doc
diff options
context:
space:
mode:
Diffstat (limited to 'doc')
-rw-r--r--doc/51Degrees-device-detection.txt174
-rw-r--r--doc/DeviceAtlas-device-detection.txt82
-rw-r--r--doc/SOCKS4.protocol.txt1
-rw-r--r--doc/SPOE.txt1255
-rw-r--r--doc/WURFL-device-detection.txt71
-rw-r--r--doc/acl.fig229
-rw-r--r--doc/architecture.txt1448
-rw-r--r--doc/coding-style.txt1566
-rw-r--r--doc/configuration.txt26732
-rw-r--r--doc/cookie-options.txt25
-rw-r--r--doc/design-thoughts/binding-possibilities.txt167
-rw-r--r--doc/design-thoughts/connection-reuse.txt224
-rw-r--r--doc/design-thoughts/http_load_time.url5
-rw-r--r--doc/design-thoughts/pool-debugging.txt243
-rw-r--r--doc/design-thoughts/thread-group.txt655
-rw-r--r--doc/gpl.txt340
-rw-r--r--doc/haproxy.1227
-rw-r--r--doc/internals/acl.txt82
-rw-r--r--doc/internals/api/appctx.txt142
-rw-r--r--doc/internals/api/buffer-api.txt653
-rw-r--r--doc/internals/api/event_hdl.txt1015
-rw-r--r--doc/internals/api/filters.txt1188
-rw-r--r--doc/internals/api/htx-api.txt570
-rw-r--r--doc/internals/api/initcalls.txt366
-rw-r--r--doc/internals/api/ist.txt167
-rw-r--r--doc/internals/api/layers.txt190
-rw-r--r--doc/internals/api/list.txt195
-rw-r--r--doc/internals/api/pools.txt585
-rw-r--r--doc/internals/api/scheduler.txt228
-rw-r--r--doc/internals/body-parsing.txt165
-rw-r--r--doc/internals/connect-status.txt28
-rw-r--r--doc/internals/connection-header.txt196
-rw-r--r--doc/internals/connection-scale.txt44
-rw-r--r--doc/internals/fd-migration.txt138
-rw-r--r--doc/internals/hashing.txt83
-rw-r--r--doc/internals/list.fig599
-rw-r--r--doc/internals/list.pngbin0 -> 33618 bytes
-rw-r--r--doc/internals/listener-states.fig150
-rw-r--r--doc/internals/listener-states.pngbin0 -> 36142 bytes
-rw-r--r--doc/internals/lua_socket.fig113
-rw-r--r--doc/internals/lua_socket.pdfbin0 -> 14969 bytes
-rw-r--r--doc/internals/muxes.fig401
-rw-r--r--doc/internals/muxes.pdfbin0 -> 12984 bytes
-rw-r--r--doc/internals/muxes.pngbin0 -> 32209 bytes
-rw-r--r--doc/internals/muxes.svg911
-rw-r--r--doc/internals/notes-layers.txt330
-rw-r--r--doc/internals/notes-poll-connect.txt93
-rw-r--r--doc/internals/notes-pollhup.txt281
-rw-r--r--doc/internals/notes-polling.txt192
-rw-r--r--doc/internals/pattern.diabin0 -> 5631 bytes
-rw-r--r--doc/internals/pattern.pdfbin0 -> 37269 bytes
-rw-r--r--doc/internals/polling-states.fig59
-rw-r--r--doc/internals/sched.fig748
-rw-r--r--doc/internals/sched.pdfbin0 -> 25334 bytes
-rw-r--r--doc/internals/sched.pngbin0 -> 150457 bytes
-rw-r--r--doc/internals/sched.svg1204
-rw-r--r--doc/internals/ssl_cert.diabin0 -> 6700 bytes
-rw-r--r--doc/internals/stats-v2.txt8
-rw-r--r--doc/internals/stconn-close.txt74
-rw-r--r--doc/internals/stream-sock-states.fig535
-rw-r--r--doc/intro.txt1700
-rw-r--r--doc/lgpl.txt504
-rw-r--r--doc/linux-syn-cookies.txt106
-rw-r--r--doc/lua-api/Makefile153
-rw-r--r--doc/lua-api/_static/channel.fig55
-rw-r--r--doc/lua-api/_static/channel.pngbin0 -> 18457 bytes
-rw-r--r--doc/lua-api/conf.py242
-rw-r--r--doc/lua-api/index.rst4491
-rw-r--r--doc/lua.txt972
-rw-r--r--doc/management.txt4521
-rw-r--r--doc/netscaler-client-ip-insertion-protocol.txt55
-rw-r--r--doc/network-namespaces.txt106
-rw-r--r--doc/peers-v2.0.txt294
-rw-r--r--doc/peers.txt491
-rw-r--r--doc/proxy-protocol.txt1051
-rw-r--r--doc/queuing.fig192
-rw-r--r--doc/regression-testing.txt706
-rw-r--r--doc/seamless_reload.txt62
78 files changed, 60878 insertions, 0 deletions
diff --git a/doc/51Degrees-device-detection.txt b/doc/51Degrees-device-detection.txt
new file mode 100644
index 0000000..2e31274
--- /dev/null
+++ b/doc/51Degrees-device-detection.txt
@@ -0,0 +1,174 @@
+51Degrees Device Detection
+--------------------------
+
+You can also include 51Degrees for inbuilt device detection enabling attributes
+such as screen size (physical & pixels), supported input methods, release date,
+hardware vendor and model, browser information, and device price among many
+others. Such information can be used to improve the user experience of a web
+site by tailoring the page content, layout and business processes to the
+precise characteristics of the device. Such customisations improve profit by
+making it easier for customers to get to the information or services they
+need. Attributes of the device making a web request can be added to HTTP
+headers as configurable parameters.
+
+In order to enable 51Degrees download the 51Degrees source code from the
+official git repository :
+
+ - either use the proven stable but frozen 3.2.10 version which
+ supports the Trie algorithm :
+
+ git clone https://github.com/51Degrees/Device-Detection.git -b v3.2.10
+
+ - use newer 3.2.12.12 version which continues to receive database
+ updates and supports a new Hash Trie algorithm, but which is not
+ compatible with older Trie databases :
+
+ git clone https://github.com/51Degrees/Device-Detection.git -b v3.2.12
+
+ - or use the latest 51Degrees version 4 with 51Degrees Hash algorithm,
+ not compatible with older databases :
+
+ git clone --recurse-submodules https://github.com/51Degrees/device-detection-cxx.git
+
+then run 'make' with USE_51DEGREES, optionally 51DEGREES_VER=4 (if using
+51Degrees version 4), and 51DEGREES_SRC set. Both 51DEGREES_INC and
+51DEGREES_LIB may additionally be used to force specific different paths for
+.o and .h, but will default to 51DEGREES_SRC. Make sure to replace
+'51D_REPO_PATH' with the path to the 51Degrees repository.
+
+51Degrees provide 4 different detection algorithms:
+
+ 1. Pattern - balances main memory usage and CPU.
+ 2. Trie - a very high performance detection solution which uses more main
+ memory than Pattern.
+ 3. Hash Trie - replaces Trie, 3x faster, 80% lower memory consumption and
+ tuning options.
+ 4. 51Degrees V4 Hash - only with 51Degrees Device Detection V4.
+
+To make with 51Degrees Pattern algorithm use the following command line.
+
+ $ make TARGET=<target> USE_51DEGREES=1 51DEGREES_SRC='51D_REPO_PATH'/src/pattern
+
+To use the 51Degrees Trie algorithm use the following command line.
+
+ $ make TARGET=<target> USE_51DEGREES=1 51DEGREES_SRC='51D_REPO_PATH'/src/trie
+
+To build with the 51Degrees Device Detection V4 use the following command line.
+
+ $ make TARGET=<target> USE_51DEGREES=1 51DEGREES_VER=4 51DEGREES_SRC='51D_REPO_PATH'/src
+
+A data file containing information about devices, browsers, operating systems
+and their associated signatures is then needed. 51Degrees provide a free
+database with Github repo for this purpose. These free data files are located
+in '51D_REPO_PATH'/data with the extensions .dat for Pattern data and .trie for
+Trie data. Free Hash Trie data file can be obtained by signing up for a licence
+key at https://51degrees.com/products/store/on-premise-device-detection.
+If using the 51degrees version 4, the free hash data file is located in
+'51D_REPO_PATH'/device-detection-data with the .hash extension.
+
+For HAProxy developers who need to verify that their changes didn't affect the
+51Degrees implementation, a dummy library is provided in the
+"addons/51degrees/dummy" directory. This does not function, but implements the
+API such that the 51Degrees module can be used (but not return any meaningful
+information). To test either Pattern or Hash Trie, or the 51Degrees version 4
+Hash algorithm, build with:
+
+ $ make TARGET=<target> USE_51DEGREES=1 51DEGREES_SRC=addons/51degrees/dummy/pattern
+or
+ $ make TARGET=<target> USE_51DEGREES=1 51DEGREES_SRC=addons/51degrees/dummy/trie
+or
+ $ make TARGET=<target> USE_51DEGREES=1 51DEGREES_VER=4 51DEGREES_SRC=addons/51degrees/dummy/v4hash
+
+respectively.
+
+The configuration file needs to set the following parameters:
+
+ global
+ 51degrees-data-file path to the Pattern, Trie or V4 Hash data file
+ 51degrees-property-name-list list of 51Degrees properties to detect
+ 51degrees-property-separator separator to use between values
+ 51degrees-cache-size LRU-based cache size (disabled by default)
+
+The following is an example of the settings for Pattern.
+
+ global
+ 51degrees-data-file '51D_REPO_PATH'/data/51Degrees-LiteV3.2.dat
+ 51degrees-property-name-list IsTablet DeviceType IsMobile
+ 51degrees-property-separator ,
+ 51degrees-cache-size 10000
+
+HAProxy needs a way to pass device information to the backend servers. This is
+done by using the 51d converter or fetch method, which intercepts the HTTP
+headers and creates some new headers. This is controlled in the frontend
+http-in section.
+
+The following is an example which adds two new HTTP headers prefixed X-51D-
+
+ frontend http-in
+ bind *:8081
+ default_backend servers
+ http-request set-header X-51D-DeviceTypeMobileTablet %[51d.all(DeviceType,IsMobile,IsTablet)]
+ http-request set-header X-51D-Tablet %[51d.all(IsTablet)]
+
+Here, two headers are created with 51Degrees data, X-51D-DeviceTypeMobileTablet
+and X-51D-Tablet. Any number of headers can be created this way and can be
+named anything. 51d.all( ) invokes the 51degrees fetch. It can be passed up to
+five property names of values to return. Values will be returned in the same
+order, separated by the 51-degrees-property-separator configured earlier. If a
+property name can't be found the value 'NoData' is returned instead.
+
+In addition to the device properties three additional properties related to the
+validity of the result can be returned when used with the Pattern method. The
+following example shows how Method, Difference and Rank could be included as one
+new HTTP header X-51D-Stats.
+
+ frontend http-in
+ ...
+ http-request set-header X-51D-Stats %[51d.all(Method,Difference,Rank)]
+
+These values indicate how confident 51Degrees is in the result that that was
+returned. More information is available on the 51Degrees web site at:
+
+ https://51degrees.com/support/documentation/pattern
+
+The above 51d.all fetch method uses all available HTTP headers for detection. A
+modest performance improvement can be obtained by only passing one HTTP header
+to the detection method with the 51d.single converter. The following example
+uses the User-Agent HTTP header only for detection.
+
+ frontend http-in
+ ...
+ http-request set-header X-51D-DeviceTypeMobileTablet %[req.fhdr(User-Agent),51d.single(DeviceType,IsMobile,IsTablet)]
+
+Any HTTP header could be used inplace of User-Agent by changing the parameter
+provided to req.fhdr.
+
+When compiled to use the Trie detection method the trie format data file needs
+to be provided. Changing the extension of the data file from dat to trie will
+use the correct data.
+
+ global
+ 51degrees-data-file '51D_REPO_PATH'/data/51Degrees-LiteV3.2.trie
+
+When used with Trie the Method, Difference and Rank properties are not
+available.
+
+When using the 51Degrees V4 Hash algorithm, the hash format data file needs
+to be provided as in the following example.
+
+ global
+ 51degrees-data-file '51D_REPO_PATH'/device-detection-data/51Degrees-LiteV4.1.hash
+
+The free Lite data file contains information about screen size in pixels and
+whether the device is a mobile. A full list of available properties is located
+on the 51Degrees web site at:
+
+ https://51degrees.com/resources/property-dictionary
+
+Some properties are only available in the paid for Premium and Enterprise
+versions of 51Degrees. These data sets not only contain more properties but
+are updated weekly and daily and contain signatures for 100,000s of different
+device combinations. For more information see the data options comparison web
+page:
+
+ https://51degrees.com/compare-data-options
diff --git a/doc/DeviceAtlas-device-detection.txt b/doc/DeviceAtlas-device-detection.txt
new file mode 100644
index 0000000..b600918
--- /dev/null
+++ b/doc/DeviceAtlas-device-detection.txt
@@ -0,0 +1,82 @@
+DeviceAtlas Device Detection
+----------------------------
+
+In order to add DeviceAtlas Device Detection support, you would need to download
+the API source code from https://deviceatlas.com/deviceatlas-haproxy-module.
+The build supports the USE_PCRE and USE_PCRE2 options. Once extracted :
+
+ $ make TARGET=<target> USE_PCRE=1 (or USE_PCRE2=1) USE_DEVICEATLAS=1 DEVICEATLAS_SRC=<path to the API root folder>
+
+Optionally DEVICEATLAS_INC and DEVICEATLAS_LIB may be set to override the path
+to the include files and libraries respectively if they're not in the source
+directory. However, if the API had been installed beforehand, DEVICEATLAS_SRC
+can be omitted. Note that the DeviceAtlas C API version supported is the 2.4.0
+at minimum.
+
+For HAProxy developers who need to verify that their changes didn't accidentally
+break the DeviceAtlas code, it is possible to build a dummy library provided in
+the addons/deviceatlas/dummy directory and to use it as an alternative for the
+full library. This will not provide the full functionalities, it will just allow
+haproxy to start with a deviceatlas configuration, which generally is enough to
+validate API changes :
+
+ $ make TARGET=<target> USE_PCRE=1 USE_DEVICEATLAS=1 DEVICEATLAS_SRC=$PWD/addons/deviceatlas/dummy
+
+These are supported DeviceAtlas directives (see doc/configuration.txt) :
+ - deviceatlas-json-file <path to the DeviceAtlas JSON data file>.
+ - deviceatlas-log-level <number> (0 to 3, level of information returned by
+ the API, 0 by default).
+ - deviceatlas-property-separator <character> (character used to separate the
+ properties produced by the API, | by default).
+
+Sample configuration :
+
+ global
+ deviceatlas-json-file <path to json file>
+
+ ...
+ frontend
+ bind *:8881
+ default_backend servers
+
+There are two distinct methods available, one which leverages all HTTP headers
+and one which uses only a single HTTP header for the detection. The former
+method is highly recommended and more accurate. There are several possible use
+cases.
+
+# To transmit the DeviceAtlas data downstream to the target application
+
+All HTTP headers via the sample / fetch
+
+ http-request set-header X-DeviceAtlas-Data %[da-csv-fetch(primaryHardwareType,osName,osVersion,browserName,browserVersion,browserRenderingEngine)]
+
+Single HTTP header (e.g. User-Agent) via the converter
+
+ http-request set-header X-DeviceAtlas-Data %[req.fhdr(User-Agent),da-csv-conv(primaryHardwareType,osName,osVersion,browserName,browserVersion,browserRenderingEngine)]
+
+# Mobile content switching with ACL
+
+All HTTP headers
+
+ acl is_mobile da-csv-fetch(mobileDevice) 1
+
+Single HTTP header
+
+ acl device_type_tablet req.fhdr(User-Agent),da-csv-conv(primaryHardwareType) "Tablet"
+
+Optionally a JSON download scheduler is provided to allow a data file being
+fetched automatically in a daily basis without restarting HAProxy :
+
+ $ cd addons/deviceatlas && make [DEVICEATLAS_SRC=<path to the API root folder>]
+
+Similarly, if the DeviceAtlas API is installed, DEVICEATLAS_SRC can be omitted.
+
+ $ ./dadwsch -u JSON data file URL e.g. "https://deviceatlas.com/getJSON?licencekey=<your licence key>&format=zip&data=my&index=web" \
+ [-p download directory path /tmp by default] \
+ [-d scheduled hour of download, hour when the service is launched by default]
+
+Noted it needs to be started before HAProxy.
+
+
+Please find more information about DeviceAtlas and the detection methods at
+https://deviceatlas.com/resources .
diff --git a/doc/SOCKS4.protocol.txt b/doc/SOCKS4.protocol.txt
new file mode 100644
index 0000000..06aee8a
--- /dev/null
+++ b/doc/SOCKS4.protocol.txt
@@ -0,0 +1 @@
+Please reference to "https://www.openssh.com/txt/socks4.protocol". \ No newline at end of file
diff --git a/doc/SPOE.txt b/doc/SPOE.txt
new file mode 100644
index 0000000..cc6d8dd
--- /dev/null
+++ b/doc/SPOE.txt
@@ -0,0 +1,1255 @@
+ -----------------------------------------------
+ Stream Processing Offload Engine (SPOE)
+ Version 1.2
+ ( Last update: 2020-06-13 )
+ -----------------------------------------------
+ Author : Christopher Faulet
+ Contact : cfaulet at haproxy dot com
+
+
+SUMMARY
+--------
+
+ 0. Terms
+ 1. Introduction
+ 2. SPOE configuration
+ 2.1. SPOE scope
+ 2.2. "spoe-agent" section
+ 2.3. "spoe-message" section
+ 2.4. "spoe-group" section
+ 2.5. Example
+ 3. SPOP specification
+ 3.1. Data types
+ 3.2. Frames
+ 3.2.1. Frame capabilities
+ 3.2.2. Frame types overview
+ 3.2.3. Workflow
+ 3.2.4. Frame: HAPROXY-HELLO
+ 3.2.5. Frame: AGENT-HELLO
+ 3.2.6. Frame: NOTIFY
+ 3.2.7. Frame: ACK
+ 3.2.8. Frame: HAPROXY-DISCONNECT
+ 3.2.9. Frame: AGENT-DISCONNECT
+ 3.3. Events & messages
+ 3.4. Actions
+ 3.5. Errors & timeouts
+ 4. Logging
+
+
+0. Terms
+---------
+
+* SPOE : Stream Processing Offload Engine.
+
+ A SPOE is a filter talking to servers managed by a SPOA to offload the
+ stream processing. An engine is attached to a proxy. A proxy can have
+ several engines. Each engine is linked to an agent and only one.
+
+* SPOA : Stream Processing Offload Agent.
+
+ A SPOA is a service that will receive info from a SPOE to offload the
+ stream processing. An agent manages several servers. It uses a backend to
+ reference all of them. By extension, these servers can also be called
+ agents.
+
+* SPOP : Stream Processing Offload Protocol, used by SPOEs to talk to SPOA
+ servers.
+
+ This protocol is used by engines to talk to agents. It is an in-house
+ binary protocol described in this documentation.
+
+
+1. Introduction
+----------------
+
+SPOE is a feature introduced in HAProxy 1.7. It makes possible the
+communication with external components to retrieve some info. The idea started
+with the problems caused by most ldap libs not working fine in event-driven
+systems (often at least the connect() is blocking). So, it is hard to properly
+implement Single Sign On solution (SSO) in HAProxy. The SPOE will ease this
+kind of processing, or we hope so.
+
+Now, the aim of SPOE is to allow any kind of offloading on the streams. First
+releases won't do lot of things. As we will see, there are few handled events
+and even less actions supported. Actually, for now, the SPOE can offload the
+processing before "tcp-request content", "tcp-response content", "http-request"
+and "http-response" rules. And it only supports variables definition. But, in
+spite of these limited features, we can easily imagine to implement SSO
+solution, ip reputation or ip geolocation services.
+
+Some example implementations in various languages are linked to from the
+HAProxy Wiki page dedicated to this mechanism:
+
+ https://github.com/haproxy/wiki/wiki/SPOE:-Stream-Processing-Offloading-Engine
+
+2. SPOE configuration
+----------------------
+
+Because SPOE is implemented as a filter, To use it, you must declare a "filter
+spoe" line in a proxy section (frontend/backend/listen) :
+
+ frontend my-front
+ ...
+ filter spoe [engine <name>] config <file>
+ ...
+
+The "config" parameter is mandatory. It specifies the SPOE configuration
+file. The engine name is optional. It can be set to declare the scope to use in
+the SPOE configuration. So it is possible to use the same SPOE configuration
+for several engines. If no name is provided, the SPOE configuration must not
+contain any scope directive.
+
+We use a separate configuration file on purpose. By commenting SPOE filter
+line, you completely disable the feature, including the parsing of sections
+reserved to SPOE. This is also a way to keep the HAProxy configuration clean.
+
+A SPOE configuration file must contains, at least, the SPOA configuration
+("spoe-agent" section) and SPOE messages/groups ("spoe-message" or "spoe-group"
+sections) attached to this agent.
+
+IMPORTANT : The configuration of a SPOE filter must be located in a dedicated
+file. But the backend used by a SPOA must be declared in HAProxy configuration
+file.
+
+2.1. SPOE scope
+-------------------------
+
+If you specify an engine name on the SPOE filter line, then you need to define
+scope in the SPOE configuration with the same name. You can have several SPOE
+scope in the same file. In each scope, you must define one and only one
+"spoe-agent" section to configure the SPOA linked to your SPOE and several
+"spoe-message" and "spoe-group" sections to describe, respectively, messages and
+group of messages sent to servers managed by your SPOA.
+
+A SPOE scope starts with this kind of line :
+
+ [<name>]
+
+where <name> is the same engine name specified on the SPOE filter line. The
+scope ends when the file ends or when another scope is found.
+
+ Example :
+ [my-first-engine]
+ spoe-agent my-agent
+ ...
+ spoe-message msg1
+ ...
+ spoe-message msg2
+ ...
+ spoe-group grp1
+ ...
+ spoe-group grp2
+ ...
+
+ [my-second-engine]
+ ...
+
+If no engine name is provided on the SPOE filter line, no SPOE scope must be
+found in the SPOE configuration file. All the file is considered to be in the
+same anonymous and implicit scope.
+
+The engine name must be uniq for a proxy. If no engine name is provided on the
+SPOE filter line, the SPOE agent name is used by default.
+
+2.2. "spoe-agent" section
+--------------------------
+
+For each engine, you must define one and only one "spoe-agent" section. In this
+section, you will declare SPOE messages and the backend you will use. You will
+also set timeouts and options to customize your agent's behaviour.
+
+
+spoe-agent <name>
+ Create a new SPOA with the name <name>. It must have one and only one
+ "spoe-agent" definition by SPOE scope.
+
+ Arguments :
+ <name> is the name of the agent section.
+
+ following keywords are supported :
+ - groups
+ - log
+ - maxconnrate
+ - maxerrrate
+ - max-frame-size
+ - max-waiting-frames
+ - messages
+ - [no] option async
+ - [no] option dontlog-normal
+ - [no] option pipelining
+ - [no] option send-frag-payload
+ - option continue-on-error
+ - option force-set-var
+ - option set-on-error
+ - option set-process-time
+ - option set-total-time
+ - option var-prefix
+ - register-var-names
+ - timeout hello|idle|processing
+ - use-backend
+
+
+groups <grp-name> ...
+ Declare the list of SPOE groups that an agent will handle.
+
+ Arguments :
+ <grp-name> is the name of a SPOE group.
+
+ Groups declared here must be found in the same engine scope, else an error is
+ triggered during the configuration parsing. You can have many "groups" lines.
+
+ See also: "spoe-group" section.
+
+
+log global
+log <address> [len <length>] [format <format>] <facility> [<level> [<minlevel>]]
+no log
+ Enable per-instance logging of events and traffic.
+
+ Prefix :
+ no should be used when the logger list must be flushed.
+
+ See the HAProxy Configuration Manual for details about this option.
+
+maxconnrate <number>
+ Set the maximum number of connections per second to <number>. The SPOE will
+ stop to open new connections if the maximum is reached and will wait to
+ acquire an existing one. So it is important to set "timeout hello" to a
+ relatively small value.
+
+
+maxerrrate <number>
+ Set the maximum number of errors per second to <number>. The SPOE will stop
+ its processing if the maximum is reached.
+
+
+max-frame-size <number>
+ Set the maximum allowed size for frames exchanged between HAProxy and SPOA.
+ It must be in the range [256, tune.bufsize-4] (4 bytes are reserved for the
+ frame length). By default, it is set to (tune.bufsize-4).
+
+max-waiting-frames <number>
+ Set the maximum number of frames waiting for an acknowledgement on the same
+ connection. This value is only used when the pipelinied or asynchronous
+ exchanges between HAProxy and SPOA are enabled. By default, it is set to 20.
+
+messages <msg-name> ...
+ Declare the list of SPOE messages that an agent will handle.
+
+ Arguments :
+ <msg-name> is the name of a SPOE message.
+
+ Messages declared here must be found in the same engine scope, else an error
+ is triggered during the configuration parsing. You can have many "messages"
+ lines.
+
+ See also: "spoe-message" section.
+
+
+option async
+no option async
+ Enable or disable the support of asynchronous exchanges between HAProxy and
+ SPOA. By default, this option is enabled.
+
+
+option continue-on-error
+ Do not stop the events processing when an error occurred on a stream.
+
+ By default, for a specific stream, when an abnormal/unexpected error occurs,
+ the SPOE is disabled for all the transaction. So if you have several events
+ configured, such error on an event will disabled all following. For TCP
+ streams, this will disable the SPOE for the whole session. For HTTP streams,
+ this will disable it for the transaction (request and response).
+
+ When set, this option bypass this behaviour and only the current event will
+ be ignored.
+
+
+option dontlog-normal
+no option dontlog-normal
+ Enable or disable logging of normal, successful processing.
+
+ Arguments : none
+
+ See also: "log" and section 4 about logging.
+
+
+option force-set-var
+ By default, SPOE filter only register already known variables (mainly from
+ parsing of the configuration), and process-wide variables (those of scope
+ "proc") cannot be created. If you want that haproxy trusts the agent and
+ registers all variables (ex: can be useful for LUA workload), activate this
+ option.
+
+ Caution : this option opens to a variety of attacks such as a rogue SPOA that
+ asks to register too many variables.
+
+
+option pipelining
+no option pipelining
+ Enable or disable the support of pipelined exchanges between HAProxy and
+ SPOA. By default, this option is enabled.
+
+
+option send-frag-payload
+no option send-frag-payload
+ Enable or disable the sending of fragmented payload to SPOA. By default, this
+ option is enabled.
+
+
+option set-on-error <var name>
+ Define the variable to set when an error occurred during an event processing.
+
+ Arguments :
+
+ <var name> is the variable name, without the scope. The name may only
+ contain characters 'a-z', 'A-Z', '0-9', '.' and '_'.
+
+ This variable will only be set when an error occurred in the scope of the
+ transaction. As for all other variables define by the SPOE, it will be
+ prefixed. So, if your variable name is "error" and your prefix is
+ "my_spoe_pfx", the variable will be "txn.my_spoe_pfx.error".
+
+ When set, the variable is an integer representing the error reason. For values
+ under 256, it represents an error coming from the engine. Below 256, it
+ reports a SPOP error. In this case, to retrieve the right SPOP status code,
+ you must remove 256 to this value. Here are possible values:
+
+ * 1 a timeout occurred during the event processing.
+
+ * 2 an error was triggered during the resources allocation.
+
+ * 3 the frame payload exceeds the frame size and it cannot be
+ fragmented.
+
+ * 4 the fragmentation of a payload is aborted.
+
+ * 5 The frame processing has been interrupted by HAProxy.
+
+ * 255 an unknown error occurred during the event processing.
+
+ * 256+N a SPOP error occurred during the event processing (see section
+ "Errors & timeouts").
+
+ Note that if "option continue-on-error" is set, the variable is not
+ automatically removed between events processing.
+
+ See also: "option continue-on-error", "option var-prefix".
+
+
+option set-process-time <var name>
+ Define the variable to set to report the processing time of the last event or
+ group.
+
+ Arguments :
+
+ <var name> is the variable name, without the scope. The name may only
+ contain characters 'a-z', 'A-Z', '0-9', '.' and '_'.
+
+ This variable will be set in the scope of the transaction. As for all other
+ variables define by the SPOE, it will be prefixed. So, if your variable name
+ is "process_time" and your prefix is "my_spoe_pfx", the variable will be
+ "txn.my_spoe_pfx.process_time".
+
+ When set, the variable is an integer representing the delay to process the
+ event or the group, in milliseconds. From the stream point of view, it is the
+ latency added by the SPOE processing for the last handled event or group.
+
+ If several events or groups are processed for the same stream, this value
+ will be overrideen.
+
+ See also: "option set-total-time".
+
+
+option set-total-time <var name>
+ Define the variable to set to report the total processing time SPOE for a
+ stream.
+
+ Arguments :
+
+ <var name> is the variable name, without the scope. The name may only
+ contain characters 'a-z', 'A-Z', '0-9', '.' and '_'.
+
+ This variable will be set in the scope of the transaction. As for all other
+ variables define by the SPOE, it will be prefixed. So, if your variable name
+ is "total_time" and your prefix is "my_spoe_pfx", the variable will be
+ "txn.my_spoe_pfx.total_time".
+
+ When set, the variable is an integer representing the sum of processing times
+ for a stream, in milliseconds. From the stream point of view, it is the
+ latency added by the SPOE processing.
+
+ If several events or groups are processed for the same stream, this value
+ will be updated.
+
+ See also: "option set-process-time".
+
+
+option var-prefix <prefix>
+ Define the prefix used when variables are set by an agent.
+
+ Arguments :
+
+ <prefix> is the prefix used to limit the scope of variables set by an
+ agent.
+
+ To avoid conflict with other variables defined by HAProxy, all variables
+ names will be prefixed. By default, the "spoe-agent" name is used. This
+ option can be used to customize it.
+
+ The prefix will be added between the variable scope and its name, separated
+ by a '.'. It may only contain characters 'a-z', 'A-Z', '0-9', '.' and '_', as
+ for variables name. In HAProxy configuration, you need to use this prefix as
+ a part of the variables name. For example, if an agent define the variable
+ "myvar" in the "txn" scope, with the prefix "my_spoe_pfx", then you should
+ use "txn.my_spoe_pfx.myvar" name in your HAProxy configuration.
+
+ By default, an agent will never set new variables at runtime: It can only set
+ new value for existing ones. If you want a different behaviour, see
+ force-set-var option and register-var-names directive.
+
+register-var-names <var name> ...
+ Register some variable names. By default, an agent will not be allowed to set
+ new variables at runtime. This rule can be totally relaxed by setting the
+ option "force-set-var". If you know all the variables you will need, this
+ directive is a good way to register them without letting an agent doing what
+ it want. This is only required if these variables are not referenced anywhere
+ in the HAProxy configuration or the SPOE one.
+
+ Arguments:
+ <var name> is a variable name without the scope. The name may only
+ contain characters 'a-z', 'A-Z', '0-9', '.' and '_'.
+
+ The prefix will be automatically added during the registration. You can have
+ many "register-var-names" lines.
+
+ See also: "option force-set-var", "option var-prefix".
+
+timeout hello <timeout>
+ Set the maximum time to wait for an agent to receive the AGENT-HELLO frame.
+ It is applied on the stream that handle the connection with the agent.
+
+ Arguments :
+ <timeout> is the timeout value specified in milliseconds by default, but
+ can be in any other unit if the number is suffixed by the unit,
+ as explained at the top of this document.
+
+ This timeout is an applicative timeout. It differ from "timeout connect"
+ defined on backends.
+
+
+timeout idle <timeout>
+ Set the maximum time to wait for an agent to close an idle connection. It is
+ applied on the stream that handle the connection with the agent.
+
+ Arguments :
+ <timeout> is the timeout value specified in milliseconds by default, but
+ can be in any other unit if the number is suffixed by the unit,
+ as explained at the top of this document.
+
+
+timeout processing <timeout>
+ Set the maximum time to wait for a stream to process an event, i.e to acquire
+ a stream to talk with an agent, to encode all messages, to send the NOTIFY
+ frame, to receive the corresponding acknowledgement and to process all
+ actions. It is applied on the stream that handle the client and the server
+ sessions.
+
+ Arguments :
+ <timeout> is the timeout value specified in milliseconds by default, but
+ can be in any other unit if the number is suffixed by the unit,
+ as explained at the top of this document.
+
+
+use-backend <backend>
+ Specify the backend to use. It must be defined.
+
+ Arguments :
+ <backend> is the name of a valid "backend" section.
+
+
+2.3. "spoe-message" section
+----------------------------
+
+To offload the stream processing, SPOE will send messages with specific
+information at a specific moment in the stream life and will wait for
+corresponding replies to know what to do.
+
+
+spoe-message <name>
+ Create a new SPOE message with the name <name>.
+
+ Arguments :
+ <name> is the name of the SPOE message.
+
+ Here you define a message that can be referenced in a "spoe-agent"
+ section. Following keywords are supported :
+ - acl
+ - args
+ - event
+
+ See also: "spoe-agent" section.
+
+
+acl <aclname> <criterion> [flags] [operator] <value> ...
+ Declare or complete an access list.
+
+ See section 7 about ACL usage in the HAProxy Configuration Manual.
+
+
+args [name=]<sample> ...
+ Define arguments passed into the SPOE message.
+
+ Arguments :
+ <sample> is a sample expression.
+
+ When the message is processed, if a sample expression is not available, it is
+ set to NULL. Arguments are processed in their declaration order and added in
+ the message in that order. It is possible to declare named arguments.
+
+ For example:
+ args frontend=fe_id src dst
+
+
+event <name> [ { if | unless } <condition> ]
+ Set the event that triggers sending of the message. It may optionally be
+ followed by an ACL-based condition, in which case it will only be evaluated
+ if the condition is true. A SPOE message can only be sent on one event. If
+ several events are defined, only the last one is considered.
+
+ ACL-based conditions are executed in the context of the stream that handle
+ the client and the server connections.
+
+ Arguments :
+ <name> is the event name.
+ <condition> is a standard ACL-based condition.
+
+ Supported events are:
+ - on-client-session
+ - on-server-session
+ - on-frontend-tcp-request
+ - on-backend-tcp-request
+ - on-tcp-response
+ - on-frontend-http-request
+ - on-backend-http-request
+ - on-http-response
+
+ See section "Events & Messages" for more details about supported events.
+ See section 7 about ACL usage in the HAProxy Configuration Manual.
+
+2.4. "spoe-group" section
+--------------------------
+
+This section can be used to declare a group of SPOE messages. Unlike messages
+referenced in a "spoe-agent" section, messages inside a group are not sent on a
+specific event. The sending must be triggered by TCP or HTTP rules, from the
+HAProxy configuration.
+
+
+spoe-group <name>
+ Create a new SPOE group with the name <name>.
+
+ Arguments :
+ <name> is the name of the SPOE group.
+
+ Here you define a group of SPOE messages that can be referenced in a
+ "spoe-agent" section. Following keywords are supported :
+ - messages
+
+ See also: "spoe-agent" and "spoe-message" sections.
+
+
+messages <msg-name> ...
+ Declare the list of SPOE messages belonging to the group.
+
+ Arguments :
+ <msg-name> is the name of a SPOE message.
+
+ Messages declared here must be found in the same engine scope, else an error
+ is triggered during the configuration parsing. Furthermore, a message belongs
+ at most to a group. You can have many "messages" lines.
+
+ See also: "spoe-message" section.
+
+
+2.5. Example
+-------------
+
+Here is a simple but complete example that sends client-ip address to a ip
+reputation service. This service can set the variable "ip_score" which is an
+integer between 0 and 100, indicating its reputation (100 means totally safe
+and 0 a blacklisted IP with no doubt).
+
+ ###
+ ### HAProxy configuration
+ frontend www
+ mode http
+ bind *:80
+
+ filter spoe engine ip-reputation config spoe-ip-reputation.conf
+
+ # Reject connection if the IP reputation is under 20
+ tcp-request content reject if { var(sess.iprep.ip_score) -m int lt 20 }
+
+ default_backend http-servers
+
+ backend http-servers
+ mode http
+ server http A.B.C.D:80
+
+ backend iprep-servers
+ mode tcp
+ balance roundrobin
+
+ timeout connect 5s # greater than hello timeout
+ timeout server 3m # greater than idle timeout
+
+ server iprep1 A1.B1.C1.D1:12345
+ server iprep2 A2.B2.C2.D2:12345
+
+ ####
+ ### spoe-ip-reputation.conf
+ [ip-reputation]
+
+ spoe-agent iprep-agent
+ messages get-ip-reputation
+
+ option var-prefix iprep
+
+ timeout hello 2s
+ timeout idle 2m
+ timeout processing 10ms
+
+ use-backend iprep-servers
+
+ spoe-message get-ip-reputation
+ args ip=src
+ event on-client-session if ! { src -f /etc/haproxy/whitelist.lst }
+
+
+3. SPOP specification
+----------------------
+
+3.1. Data types
+----------------
+
+Here is the bytewise representation of typed data:
+
+ TYPED-DATA : <TYPE:4 bits><FLAGS:4 bits><DATA>
+
+Supported types and their representation are:
+
+ TYPE | ID | DESCRIPTION
+ -----------------------------+-----+----------------------------------
+ NULL | 0 | NULL : <0>
+ Boolean | 1 | BOOL : <1+FLAG>
+ 32bits signed integer | 2 | INT32 : <2><VALUE:varint>
+ 32bits unsigned integer | 3 | UINT32 : <3><VALUE:varint>
+ 64bits signed integer | 4 | INT64 : <4><VALUE:varint>
+ 32bits unsigned integer | 5 | UNIT64 : <5><VALUE:varint>
+ IPV4 | 6 | IPV4 : <6><STRUCT IN_ADDR:4 bytes>
+ IPV6 | 7 | IPV6 : <7><STRUCT IN_ADDR6:16 bytes>
+ String | 8 | STRING : <8><LENGTH:varint><BYTES>
+ Binary | 9 | BINARY : <9><LENGTH:varint><BYTES>
+ 10 -> 15 unused/reserved | - | -
+ -----------------------------+-----+----------------------------------
+
+Variable-length integer (varint) are encoded using Peers encoding:
+
+
+ 0 <= X < 240 : 1 byte (7.875 bits) [ XXXX XXXX ]
+ 240 <= X < 2288 : 2 bytes (11 bits) [ 1111 XXXX ] [ 0XXX XXXX ]
+ 2288 <= X < 264432 : 3 bytes (18 bits) [ 1111 XXXX ] [ 1XXX XXXX ] [ 0XXX XXXX ]
+ 264432 <= X < 33818864 : 4 bytes (25 bits) [ 1111 XXXX ] [ 1XXX XXXX ]*2 [ 0XXX XXXX ]
+ 33818864 <= X < 4328786160 : 5 bytes (32 bits) [ 1111 XXXX ] [ 1XXX XXXX ]*3 [ 0XXX XXXX ]
+ ...
+
+For booleans, the value (true or false) is the first bit in the FLAGS
+bitfield. if this bit is set to 0, then the boolean is evaluated as false,
+otherwise, the boolean is evaluated as true.
+
+3.2. Frames
+------------
+
+Exchange between HAProxy and agents are made using FRAME packets. All frames
+must be prefixed with their size encoded on 4 bytes in network byte order:
+
+ <FRAME-LENGTH:4 bytes> <FRAME>
+
+A frame always starts with its type, on one byte, followed by metadata
+containing flags, on 4 bytes and a two variable-length integer representing the
+stream identifier and the frame identifier inside the stream:
+
+ FRAME : <FRAME-TYPE:1 byte> <METADATA> <FRAME-PAYLOAD>
+ METADATA : <FLAGS:4 bytes> <STREAM-ID:varint> <FRAME-ID:varint>
+
+Then comes the frame payload. Depending on the frame type, the payload can be
+of three types: a simple key/value list, a list of messages or a list of
+actions.
+
+ FRAME-PAYLOAD : <LIST-OF-MESSAGES> | <LIST-OF-ACTIONS> | <KV-LIST>
+
+ LIST-OF-MESSAGES : [ <MESSAGE-NAME> <NB-ARGS:1 byte> <KV-LIST> ... ]
+ MESSAGE-NAME : <STRING>
+
+ LIST-OF-ACTIONS : [ <ACTION-TYPE:1 byte> <NB-ARGS:1 byte> <ACTION-ARGS> ... ]
+ ACTION-ARGS : [ <TYPED-DATA>... ]
+
+ KV-LIST : [ <KV-NAME> <KV-VALUE> ... ]
+ KV-NAME : <STRING>
+ KV-VALUE : <TYPED-DATA>
+
+ FLAGS :
+
+ Flags are a 32 bits field. They are encoded on 4 bytes in network byte
+ order, where the bit 0 is the LSB.
+
+ 0 1 2-31
+ +---+---+----------+
+ | | A | |
+ | F | B | |
+ | I | O | RESERVED |
+ | N | R | |
+ | | T | |
+ +---+---+----------+
+
+ FIN: Indicates that this is the final payload fragment. The first fragment
+ may also be the final fragment.
+
+ ABORT: Indicates that the processing of the current frame must be
+ cancelled. This bit should be set on frames with a fragmented
+ payload. It can be ignore for frames with an unfragemnted
+ payload. When it is set, the FIN bit must also be set.
+
+
+Frames cannot exceed a maximum size negotiated between HAProxy and agents
+during the HELLO handshake. Most of time, payload will be small enough to send
+it in one frame. But when supported by the peer, it will be possible to
+fragment huge payload on many frames. This ability is announced during the
+HELLO handshake and it can be asynmetric (supported by agents but not by
+HAProxy or the opposite). The following rules apply to fragmentation:
+
+ * An unfragemnted payload consists of a single frame with the FIN bit set.
+
+ * A fragemented payload consists of several frames with the FIN bit clear and
+ terminated by a single frame with the FIN bit set. All these frames must
+ share the same STREAM-ID and FRAME-ID. The first frame must set the right
+ FRAME-TYPE (e.g, NOTIFY). The following frames must have an unset type (0).
+
+Beside the support of fragmented payload by a peer, some payload must not be
+fragmented. See below for details.
+
+IMPORTANT : The maximum size supported by peers for a frame must be greater
+than or equal to 256 bytes.
+
+3.2.1. Frame capabilities
+--------------------------
+
+Here are the list of official capabilities that HAProxy and agents can support:
+
+ * fragmentation: This is the ability for a peer to support fragmented
+ payload in received frames. This is an asymmectical
+ capability, it only concerns the peer that announces
+ it. This is the responsibility to the other peer to use it
+ or not.
+
+ * pipelining: This is the ability for a peer to decouple NOTIFY and ACK
+ frames. This is a symmectical capability. To be used, it must
+ be supported by HAProxy and agents. Unlike HTTP pipelining, the
+ ACK frames can be send in any order, but always on the same TCP
+ connection used for the corresponding NOTIFY frame.
+
+ * async: This ability is similar to the pipelining, but here any TCP
+ connection established between HAProxy and the agent can be used to
+ send ACK frames. if an agent accepts connections from multiple
+ HAProxy, it can use the "engine-id" value to group TCP
+ connections. See details about HAPROXY-HELLO frame.
+
+Unsupported or unknown capabilities are silently ignored, when possible.
+
+NOTE: HAProxy does not support the fragmentation for now. This means it is not
+ able to handle fragmented frames. However, if an agent announces the
+ fragmentation support, HAProxy may choose to send fragemented frames.
+
+3.2.2. Frame types overview
+----------------------------
+
+Here are types of frame supported by SPOE. Frames sent by HAProxy come first,
+then frames sent by agents :
+
+ TYPE | ID | DESCRIPTION
+ -----------------------------+-----+-------------------------------------
+ UNSET | 0 | Used for all frames but the first when a
+ | | payload is fragmented.
+ -----------------------------+-----+-------------------------------------
+ HAPROXY-HELLO | 1 | Sent by HAProxy when it opens a
+ | | connection on an agent.
+ | |
+ HAPROXY-DISCONNECT | 2 | Sent by HAProxy when it want to close
+ | | the connection or in reply to an
+ | | AGENT-DISCONNECT frame
+ | |
+ NOTIFY | 3 | Sent by HAProxy to pass information
+ | | to an agent
+ -----------------------------+-----+-------------------------------------
+ AGENT-HELLO | 101 | Reply to a HAPROXY-HELLO frame, when
+ | | the connection is established
+ | |
+ AGENT-DISCONNECT | 102 | Sent by an agent just before closing
+ | | the connection
+ | |
+ ACK | 103 | Sent to acknowledge a NOTIFY frame
+ -----------------------------+-----+-------------------------------------
+
+Unknown frames may be silently skipped.
+
+3.2.3. Workflow
+----------------
+
+ * Successful HELLO handshake:
+
+ HAPROXY AGENT SRV
+ | HAPROXY-HELLO |
+ | (healthcheck: false) |
+ | --------------------------> |
+ | |
+ | AGENT-HELLO |
+ | <-------------------------- |
+ | |
+
+ * Successful HELLO healthcheck:
+
+ HAPROXY AGENT SRV
+ | HAPROXY-HELLO |
+ | (healthcheck: true) |
+ | --------------------------> |
+ | |
+ | AGENT-HELLO + close() |
+ | <-------------------------- |
+ | |
+
+
+ * Error encountered by agent during the HELLO handshake:
+
+ HAPROXY AGENT SRV
+ | HAPROXY-HELLO |
+ | --------------------------> |
+ | |
+ | DISCONNECT + close() |
+ | <-------------------------- |
+ | |
+
+ * Error encountered by HAProxy during the HELLO handshake:
+
+ HAPROXY AGENT SRV
+ | HAPROXY-HELLO |
+ | --------------------------> |
+ | |
+ | AGENT-HELLO |
+ | <-------------------------- |
+ | |
+ | DISCONNECT |
+ | --------------------------> |
+ | |
+ | DISCONNECT + close() |
+ | <-------------------------- |
+ | |
+
+ * Notify / Ack exchange (unfragmented payload):
+
+ HAPROXY AGENT SRV
+ | NOTIFY |
+ | --------------------------> |
+ | |
+ | ACK |
+ | <-------------------------- |
+ | |
+
+ * Notify / Ack exchange (fragmented payload):
+
+ HAPROXY AGENT SRV
+ | NOTIFY (frag 1) |
+ | --------------------------> |
+ | |
+ | UNSET (frag 2) |
+ | --------------------------> |
+ | ... |
+ | UNSET (frag N) |
+ | --------------------------> |
+ | |
+ | ACK |
+ | <-------------------------- |
+ | |
+
+ * Aborted fragmentation of a NOTIFY frame:
+
+ HAPROXY AGENT SRV
+ | ... |
+ | UNSET (frag X) |
+ | --------------------------> |
+ | |
+ | ACK/ABORT |
+ | <-------------------------- |
+ | |
+ | UNSET (frag X+1) |
+ | -----------X |
+ | |
+ | |
+
+ * Connection closed by haproxy:
+
+ HAPROXY AGENT SRV
+ | DISCONNECT |
+ | --------------------------> |
+ | |
+ | DISCONNECT + close() |
+ | <-------------------------- |
+ | |
+
+ * Connection closed by agent:
+
+ HAPROXY AGENT SRV
+ | DISCONNECT + close() |
+ | <-------------------------- |
+ | |
+
+3.2.4. Frame: HAPROXY-HELLO
+----------------------------
+
+This frame is the first one exchanged between HAProxy and an agent, when the
+connection is established. The payload of this frame is a KV-LIST. It cannot be
+fragmented. STREAM-ID and FRAME-ID are must be set 0.
+
+Following items are mandatory in the KV-LIST:
+
+ * "supported-versions" <STRING>
+
+ Last SPOP major versions supported by HAProxy. It is a comma-separated list
+ of versions, following the format "Major.Minor". Spaces must be ignored, if
+ any. When a major version is announced by HAProxy, it means it also support
+ all previous minor versions.
+
+ Example: "2.0, 1.5" means HAProxy supports SPOP 2.0 and 1.0 to 1.5
+
+ * "max-frame-size" <UINT32>
+
+ This is the maximum size allowed for a frame. The HAPROXY-HELLO frame must
+ be lower or equal to this value.
+
+ * "capabilities" <STRING>
+
+ This a comma-separated list of capabilities supported by HAProxy. Spaces
+ must be ignored, if any.
+
+Following optional items can be added in the KV-LIST:
+
+ * "healthcheck" <BOOLEAN>
+
+ If this item is set to TRUE, then the HAPROXY-HELLO frame is sent during a
+ SPOE health check. When set to FALSE, this item can be ignored.
+
+ * "engine-id" <STRING>
+
+ This is a uniq string that identify a SPOE engine.
+
+To finish the HELLO handshake, the agent must return an AGENT-HELLO frame with
+its supported SPOP version, the lower value between its maximum size allowed
+for a frame and the HAProxy one and capabilities it supports. If an error
+occurs or if an incompatibility is detected with the agent configuration, an
+AGENT-DISCONNECT frame must be returned.
+
+3.2.5. Frame: AGENT-HELLO
+--------------------------
+
+This frame is sent in reply to a HAPROXY-HELLO frame to finish a HELLO
+handshake. As for HAPROXY-HELLO frame, STREAM-ID and FRAME-ID are also set
+0. The payload of this frame is a KV-LIST and it cannot be fragmented.
+
+Following items are mandatory in the KV-LIST:
+
+ * "version" <STRING>
+
+ This is the SPOP version the agent supports. It must follow the format
+ "Major.Minor" and it must be lower or equal than one of major versions
+ announced by HAProxy.
+
+ * "max-frame-size" <UINT32>
+
+ This is the maximum size allowed for a frame. It must be lower or equal to
+ the value in the HAPROXY-HELLO frame. This value will be used for all
+ subsequent frames.
+
+ * "capabilities" <STRING>
+
+ This a comma-separated list of capabilities supported by agent. Spaces must
+ be ignored, if any.
+
+At this time, if everything is ok for HAProxy (supported version and valid
+max-frame-size value), the HELLO handshake is successfully completed. Else,
+HAProxy sends a HAPROXY-DISCONNECT frame with the corresponding error.
+
+If "healthcheck" item was set to TRUE in the HAPROXY-HELLO frame, the agent can
+safely close the connection without DISCONNECT frame. In all cases, HAProxy
+will close the connection at the end of the health check.
+
+3.2.6. Frame: NOTIFY
+---------------------
+
+Information are sent to the agents inside NOTIFY frames. These frames are
+attached to a stream, so STREAM-ID and FRAME-ID must be set. The payload of
+NOTIFY frames is a LIST-OF-MESSAGES and, if supported by agents, it can be
+fragmented.
+
+NOTIFY frames must be acknowledge by agents sending an ACK frame, repeating
+right STREAM-ID and FRAME-ID.
+
+3.2.7. Frame: ACK
+------------------
+
+ACK frames must be sent by agents to reply to NOTIFY frames. STREAM-ID and
+FRAME-ID found in a NOTIFY frame must be reuse in the corresponding ACK
+frame. The payload of ACK frames is a LIST-OF-ACTIONS and, if supported by
+HAProxy, it can be fragmented.
+
+3.2.8. Frame: HAPROXY-DISCONNECT
+---------------------------------
+
+If an error occurs, at anytime, from the HAProxy side, a HAPROXY-DISCONNECT
+frame is sent with information describing the error. HAProxy will wait an
+AGENT-DISCONNECT frame in reply. All other frames will be ignored. The agent
+must then close the socket.
+
+The payload of this frame is a KV-LIST. It cannot be fragmented. STREAM-ID and
+FRAME-ID are must be set 0.
+
+Following items are mandatory in the KV-LIST:
+
+ * "status-code" <UINT32>
+
+ This is the code corresponding to the error.
+
+ * "message" <STRING>
+
+ This is a textual message describing the error.
+
+For more information about known errors, see section "Errors & timeouts"
+
+3.2.9. Frame: AGENT-DISCONNECT
+-------------------------------
+
+If an error occurs, at anytime, from the agent size, a AGENT-DISCONNECT frame
+is sent, with information describing the error. such frame is also sent in reply
+to a HAPROXY-DISCONNECT. The agent must close the socket just after sending
+this frame.
+
+The payload of this frame is a KV-LIST. It cannot be fragmented. STREAM-ID and
+FRAME-ID are must be set 0.
+
+Following items are mandatory in the KV-LIST:
+
+ * "status-code" <UINT32>
+
+ This is the code corresponding to the error.
+
+ * "message" <STRING>
+
+ This is a textual message describing the error.
+
+For more information about known errors, see section "Errors & timeouts"
+
+3.3. Events & Messages
+-----------------------
+
+Information about streams are sent in NOTIFY frames. You can specify which kind
+of information to send by defining "spoe-message" sections in your SPOE
+configuration file. for each "spoe-message" there will be a message in a NOTIFY
+frame when the right event is triggered.
+
+A NOTIFY frame is sent for an specific event when there is at least one
+"spoe-message" attached to this event. All messages for an event will be added
+in the same NOTIFY frame.
+
+Here is the list of supported events:
+
+ * on-client-session is triggered when a new client session is created.
+ This event is only available for SPOE filters
+ declared in a frontend or a listen section.
+
+ * on-frontend-tcp-request is triggered just before the evaluation of
+ "tcp-request content" rules on the frontend side.
+ This event is only available for SPOE filters
+ declared in a frontend or a listen section.
+
+ * on-backend-tcp-request is triggered just before the evaluation of
+ "tcp-request content" rules on the backend side.
+ This event is skipped for SPOE filters declared
+ in a listen section.
+
+ * on-frontend-http-request is triggered just before the evaluation of
+ "http-request" rules on the frontend side. This
+ event is only available for SPOE filters declared
+ in a frontend or a listen section.
+
+ * on-backend-http-request is triggered just before the evaluation of
+ "http-request" rules on the backend side. This
+ event is skipped for SPOE filters declared in a
+ listen section.
+
+ * on-server-session is triggered when the session with the server is
+ established.
+
+ * on-tcp-response is triggered just before the evaluation of
+ "tcp-response content" rules.
+
+ * on-http-response is triggered just before the evaluation of
+ "http-response" rules.
+
+
+The stream processing will loop on these events, when triggered, waiting the
+agent reply.
+
+3.4. Actions
+-------------
+
+An agent must acknowledge each NOTIFY frame by sending the corresponding ACK
+frame. Actions can be added in these frames to dynamically take action on the
+processing of a stream.
+
+Here is the list of supported actions:
+
+ * set-var set the value for an existing variable. 3 arguments must be
+ attached to this action: the variable scope (proc, sess, txn,
+ req or res), the variable name (a string) and its value.
+
+ ACTION-SET-VAR : <SET-VAR:1 byte><NB-ARGS:1 byte><VAR-SCOPE:1 byte><VAR-NAME><VAR-VALUE>
+
+ SET-VAR : <1>
+ NB-ARGS : <3>
+ VAR-SCOPE : <PROCESS> | <SESSION> | <TRANSACTION> | <REQUEST> | <RESPONSE>
+ VAR-NAME : <STRING>
+ VAR-VALUE : <TYPED-DATA>
+
+ PROCESS : <0>
+ SESSION : <1>
+ TRANSACTION : <2>
+ REQUEST : <3>
+ RESPONSE : <4>
+
+ * unset-var unset the value for an existing variable. 2 arguments must be
+ attached to this action: the variable scope (proc, sess, txn,
+ req or res) and the variable name (a string).
+
+ ACTION-UNSET-VAR : <UNSET-VAR:1 byte><NB-ARGS:1 byte><VAR-SCOPE:1 byte><VAR-NAME>
+
+ UNSET-VAR : <2>
+ NB-ARGS : <2>
+ VAR-SCOPE : <PROCESS> | <SESSION> | <TRANSACTION> | <REQUEST> | <RESPONSE>
+ VAR-NAME : <STRING>
+
+ PROCESS : <0>
+ SESSION : <1>
+ TRANSACTION : <2>
+ REQUEST : <3>
+ RESPONSE : <4>
+
+
+NOTE: Name of the variables will be automatically prefixed by HAProxy to avoid
+ name clashes with other variables used in HAProxy. Moreover, unknown
+ variable will be silently ignored.
+
+3.5. Errors & timeouts
+----------------------
+
+Here is the list of all known errors:
+
+ STATUS CODE | DESCRIPTION
+ ----------------+--------------------------------------------------------
+ 0 | normal (no error occurred)
+ 1 | I/O error
+ 2 | A timeout occurred
+ 3 | frame is too big
+ 4 | invalid frame received
+ 5 | version value not found
+ 6 | max-frame-size value not found
+ 7 | capabilities value not found
+ 8 | unsupported version
+ 9 | max-frame-size too big or too small
+ 10 | payload fragmentation is not supported
+ 11 | invalid interlaced frames
+ 12 | frame-id not found (it does not match any referenced frame)
+ 13 | resource allocation error
+ 99 | an unknown error occurrde
+ ----------------+--------------------------------------------------------
+
+An agent can define its own errors using a not yet assigned status code.
+
+IMPORTANT NOTE: By default, for a specific stream, when an abnormal/unexpected
+ error occurs, the SPOE is disabled for all the transaction. So
+ if you have several events configured, such error on an event
+ will disabled all following. For TCP streams, this will
+ disable the SPOE for the whole session. For HTTP streams, this
+ will disable it for the transaction (request and response).
+ See 'option continue-on-error' to bypass this limitation.
+
+To avoid a stream to wait undefinetly, you must carefully choose the
+acknowledgement timeout. In most of cases, it will be quiet low. But it depends
+on the responsivness of your service.
+
+You must also choose idle timeout carefully. Because connection with your
+service depends on the backend configuration used by the SPOA, it is important
+to use a lower value for idle timeout than the server timeout. Else the
+connection will be closed by HAProxy. The same is true for hello timeout. You
+should choose a lower value than the connect timeout.
+
+4. Logging
+-----------
+
+Activity of an SPOE is logged using HAProxy's logger. The messages are logged
+in the context of the streams that handle the client and the server
+connections. A message is emitted for each event or group handled by an
+SPOE. Depending on the status code, the log level will be different. In the
+normal case, when no error occurred, the message is logged with the level
+LOG_NOTICE. Otherwise, the message is logged with the level LOG_WARNING.
+
+The messages are logged using the agent's logger, if defined, and use the
+following format:
+
+ SPOE: [AGENT] <TYPE:NAME> sid=STREAM-ID st=STATUS-CODE reqT/qT/wT/resT/pT \
+ <idles>/<applets> <nb_sending>/<nb_waiting> <nb_error>/<nb_processed>
+
+ AGENT is the agent name
+ TYPE is EVENT of GROUP
+ NAME is the event or the group name
+ STREAM-ID is an integer, the unique id of the stream
+ STATUS_CODE is the processing's status code
+ reqT/qT/wT/resT/pT are the following time events:
+
+ * reqT : the encoding time. It includes ACLs processing, if any. For
+ fragmented frames, it is the sum of all fragments.
+ * qT : the delay before the request gets out the sending queue. For
+ fragmented frames, it is the sum of all fragments.
+ * wT : the delay before the response is received. No fragmentation
+ supported here.
+ * resT : the delay to process the response. No fragmentation supported
+ here.
+ * pT : the delay to process the event or the group. From the stream
+ point of view, it is the latency added by the SPOE processing.
+ It is more or less the sum of values above.
+
+ <idle> is the numbers of idle SPOE applets
+ <applets> is the numbers of SPOE applets
+ <nb_sending> is the numbers of streams waiting to send data
+ <nb_waiting> is the numbers of streams waiting for a ack
+ <nb_error> is the numbers of processing errors
+ <nb_processed> is the numbers of events/groups processed
+
+
+For all these time events, -1 means the processing was interrupted before the
+end. So -1 for the queue time means the request was never dequeued. For
+fragmented frames it is harder to know when the interruption happened.
+
+/*
+ * Local variables:
+ * fill-column: 79
+ * End:
+ */
diff --git a/doc/WURFL-device-detection.txt b/doc/WURFL-device-detection.txt
new file mode 100644
index 0000000..4786e22
--- /dev/null
+++ b/doc/WURFL-device-detection.txt
@@ -0,0 +1,71 @@
+Scientiamobile WURFL Device Detection
+-------------------------------------
+
+You can also include WURFL for inbuilt device detection enabling attributes.
+
+WURFL is a high-performance and low-memory footprint mobile device detection
+software component that can quickly and accurately detect over 500 capabilities
+of visiting devices. It can differentiate between portable mobile devices, desktop devices,
+SmartTVs and any other types of devices on which a web browser can be installed.
+
+In order to add WURFL device detection support, you would need to download Scientiamobile
+InFuze C API and install it on your system. Refer to www.scientiamobile.com to obtain a valid
+InFuze license.
+Compile haproxy as shown :
+
+ $ make TARGET=<target> USE_WURFL=1
+
+Optionally WURFL_DEBUG=1 may be set to increase logs verbosity
+
+For HAProxy developers who need to verify that their changes didn't accidentally
+break the WURFL code, it is possible to build a dummy library provided in the
+addons/wurfl/dummy directory and to use it as an alternative for the full library.
+This will not provide the full functionalities, it will just allow haproxy to
+start with a wurfl configuration, which generally is enough to validate API
+changes :
+
+ $ make -C addons/wurfl/dummy
+ $ make TARGET=<target> USE_WURFL=1 WURFL_INC=$PWD/addons/wurfl/dummy WURFL_LIB=$PWD/addons/wurfl/dummy
+
+These are the supported WURFL directives (see doc/configuration.txt) :
+- wurfl-data-file <path to WURFL data file>
+- wurfl-information-list [<string>] (list of WURFL capabilities,
+ virtual capabilities, property names we plan to use in injected headers)
+- wurfl-information-list-separator <char> (character that will be
+ used to separate values in a response header, ',' by default).
+- wurfl-cache-size <string> (Sets the WURFL caching strategy)
+- wurfl-patch-file [<file path>] (Sets the paths to custom WURFL patch files)
+
+Sample configuration :
+
+ global
+ wurfl-data-file /usr/share/wurfl/wurfl.zip
+
+ wurfl-information-list wurfl_id model_name
+
+ #wurfl-information-list-separator |
+
+ ## single LRU cache
+ #wurfl-cache-size 100000
+ ## no cache
+ #wurfl-cache-size 0
+
+ #wurfl-patch-file <paths to custom patch files>
+
+ ...
+ frontend
+ bind *:8888
+ default_backend servers
+
+There are two distinct methods available to transmit the WURFL data downstream
+to the target application:
+
+All data listed in wurfl-information-list
+
+ http-request set-header X-WURFL-All %[wurfl-get-all()]
+
+A subset of data listed in wurfl-information-list
+
+ http-request set-header X-WURFL-Properties %[wurfl-get(wurfl_id,is_tablet)]
+
+Please find more information about WURFL and the detection methods at https://www.scientiamobile.com
diff --git a/doc/acl.fig b/doc/acl.fig
new file mode 100644
index 0000000..253a053
--- /dev/null
+++ b/doc/acl.fig
@@ -0,0 +1,229 @@
+#FIG 3.2 Produced by xfig version 3.2.5-alpha5
+Portrait
+Center
+Metric
+A4
+100.00
+Single
+-2
+1200 2
+6 2430 1080 2700 2250
+1 2 0 1 0 11 52 -1 20 0.000 1 0.0000 2587 1687 113 563 2474 1687 2700 1687
+4 1 0 50 -1 16 8 1.5708 4 120 840 2610 1710 tcp-req inspect\001
+-6
+6 5805 1080 6255 2250
+1 2 0 1 0 29 52 -1 20 0.000 1 0.0000 6052 1687 203 563 5849 1687 6255 1687
+4 1 0 50 -1 16 8 1.5708 4 90 300 6030 1710 HTTP\001
+4 1 0 50 -1 16 8 1.5708 4 120 615 6165 1710 processing\001
+-6
+6 1575 3375 1800 4500
+2 2 0 1 0 29 52 -1 20 0.000 0 0 -1 0 0 5
+ 1575 3375 1800 3375 1800 4500 1575 4500 1575 3375
+4 1 0 50 -1 16 8 1.5708 4 120 735 1710 3960 http-resp out\001
+-6
+6 2025 3375 2250 4500
+2 2 0 1 0 29 52 -1 20 0.000 0 0 -1 0 0 5
+ 2025 3375 2250 3375 2250 4500 2025 4500 2025 3375
+4 1 0 50 -1 16 8 1.5708 4 120 735 2160 3960 http-resp out\001
+-6
+6 810 3600 1080 4230
+4 1 0 50 -1 16 8 1.5708 4 105 555 900 3915 Response\001
+4 1 0 50 -1 16 8 1.5708 4 105 450 1065 3915 to client\001
+-6
+6 720 1350 1035 2070
+4 1 0 50 -1 16 8 1.5708 4 120 540 855 1710 Requests \001
+4 1 0 50 -1 16 8 1.5708 4 105 645 1020 1710 from clients\001
+-6
+6 7695 1350 8010 1980
+4 1 0 50 -1 16 8 1.5708 4 120 510 7830 1665 Requests\001
+4 1 0 50 -1 16 8 1.5708 4 105 555 7995 1665 to servers\001
+-6
+6 7785 3600 8055 4230
+4 1 0 50 -1 16 8 1.5708 4 105 555 7875 3915 Response\001
+4 1 0 50 -1 16 8 1.5708 4 105 630 8055 3915 from server\001
+-6
+1 2 0 1 0 11 52 -1 20 0.000 1 0.0000 1687 1687 113 563 1574 1687 1800 1687
+1 2 0 1 0 11 52 -1 20 0.000 1 0.0000 7087 3937 113 563 6974 3937 7200 3937
+1 2 0 1 0 29 52 -1 20 0.000 1 0.0000 4072 3937 203 563 3869 3937 4275 3937
+1 2 0 1 0 29 52 -1 20 0.000 1 0.0000 2903 3937 203 563 2700 3937 3106 3937
+2 3 0 1 0 6 54 -1 20 0.000 0 0 -1 0 0 9
+ 1485 900 1485 2475 4140 2475 4140 1035 6390 1035 6390 2340
+ 6840 2340 6840 900 1485 900
+2 3 0 1 0 2 54 -1 20 0.000 0 0 -1 0 0 9
+ 4365 1035 4365 2475 7290 2475 7290 900 6840 900 6840 2340
+ 5715 2340 5715 1035 4365 1035
+2 2 0 1 0 29 52 -1 20 0.000 0 0 -1 0 0 5
+ 4950 1125 5175 1125 5175 2250 4950 2250 4950 1125
+2 2 0 1 0 29 52 -1 20 0.000 0 0 -1 0 0 5
+ 5400 1125 5625 1125 5625 2250 5400 2250 5400 1125
+2 2 0 1 0 11 52 -1 20 0.000 0 0 -1 0 0 5
+ 2025 1125 2250 1125 2250 2250 2025 2250 2025 1125
+2 2 0 1 0 11 52 -1 20 0.000 0 0 -1 0 0 5
+ 2925 1125 3150 1125 3150 2250 2925 2250 2925 1125
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 1125 1710 1575 1710
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 1125 1935 1575 1755
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 1125 1485 1575 1665
+2 2 0 1 0 29 52 -1 20 0.000 0 0 -1 0 0 5
+ 3825 1125 4050 1125 4050 2250 3825 2250 3825 1125
+2 2 0 1 0 6 50 -1 20 0.000 0 0 -1 0 0 5
+ 1575 450 2025 450 2025 540 1575 540 1575 450
+2 2 0 1 0 2 50 -1 20 0.000 0 0 -1 0 0 5
+ 1575 675 2025 675 2025 765 1575 765 1575 675
+2 2 0 1 0 11 50 -1 20 0.000 0 0 -1 0 0 5
+ 3150 450 3600 450 3600 540 3150 540 3150 450
+2 2 0 1 0 29 50 -1 20 0.000 0 0 -1 0 0 5
+ 3150 675 3600 675 3600 765 3150 765 3150 675
+2 2 0 1 0 29 52 -1 20 0.000 0 0 -1 0 0 5
+ 6525 1125 6750 1125 6750 2250 6525 2250 6525 1125
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 7200 1665 7650 1665
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 7200 1620 7650 1530
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 7200 1710 7650 1800
+2 2 0 1 0 29 52 -1 20 0.000 0 0 -1 0 0 5
+ 6975 1125 7200 1125 7200 2250 6975 2250 6975 1125
+2 2 0 1 0 29 52 -1 20 0.000 0 0 -1 0 0 5
+ 3375 1125 3600 1125 3600 2250 3375 2250 3375 1125
+2 2 0 1 0 11 52 -1 20 0.000 0 0 -1 0 0 5
+ 4500 1125 4725 1125 4725 2250 4500 2250 4500 1125
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 30.00 60.00
+ 1800 1665 2025 1665
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 30.00 60.00
+ 2250 1665 2475 1665
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 30.00 60.00
+ 2700 1665 2925 1665
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 30.00 60.00
+ 3150 1665 3375 1665
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 30.00 60.00
+ 3600 1665 3825 1665
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 30.00 60.00
+ 4725 1665 4950 1665
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 30.00 60.00
+ 5175 1665 5400 1665
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 30.00 60.00
+ 5625 1665 5850 1665
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 30.00 60.00
+ 6750 1665 6975 1665
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 30.00 60.00
+ 6255 1665 6525 1665
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 30.00 60.00
+ 4050 1665 4500 1665
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 30.00 60.00
+ 4050 1620 4500 1530
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 30.00 60.00
+ 4050 1710 4500 1800
+2 2 0 1 0 11 52 -1 20 0.000 0 0 -1 0 0 5
+ 6525 3375 6750 3375 6750 4500 6525 4500 6525 3375
+2 2 0 1 0 29 52 -1 20 0.000 0 0 -1 0 0 5
+ 6075 3375 6300 3375 6300 4500 6075 4500 6075 3375
+2 3 0 1 0 2 54 -1 20 0.000 0 0 -1 0 0 9
+ 7290 3150 7290 4725 5985 4725 5985 3285 2385 3285 2385 4590
+ 1935 4590 1935 3150 7290 3150
+2 3 0 1 0 6 54 -1 20 0.000 0 0 -1 0 0 9
+ 1935 3150 1485 3150 1485 4725 5985 4725 5985 3285 5085 3285
+ 5085 4590 1935 4590 1935 3150
+2 2 0 1 0 11 52 -1 20 0.000 0 0 -1 0 0 5
+ 5625 3375 5850 3375 5850 4500 5625 4500 5625 3375
+2 2 0 1 0 29 52 -1 20 0.000 0 0 -1 0 0 5
+ 5175 3375 5400 3375 5400 4500 5175 4500 5175 3375
+2 1 0 1 0 0 54 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 7650 3915 7200 3915
+2 1 0 1 0 0 54 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 1575 3915 1125 3915
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 30.00 60.00
+ 6975 3915 6750 3915
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 30.00 60.00
+ 6525 3915 6300 3915
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 30.00 60.00
+ 6075 3915 5850 3915
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 30.00 60.00
+ 5625 3915 5400 3915
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 30.00 60.00
+ 2025 3915 1800 3915
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 30.00 60.00
+ 5175 3915 4275 3915
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 30.00 60.00
+ 3870 3915 3105 3915
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 30.00 60.00
+ 2700 3915 2250 3915
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 3
+ 1 1 1.00 30.00 60.00
+ 3465 2250 3465 2880 2970 3465
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 4
+ 1 1 1.00 30.00 60.00
+ 5040 2250 5040 2655 3600 2880 3015 3510
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 4
+ 1 1 1.00 30.00 60.00
+ 6075 2250 6075 2565 3645 2925 3060 3555
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 4
+ 1 1 1.00 30.00 60.00
+ 6615 2250 6615 2610 3690 2970 3060 3645
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 4
+ 1 1 1.00 30.00 60.00
+ 7065 2250 7065 2655 3735 3015 3060 3690
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 4
+ 1 1 1.00 30.00 60.00
+ 5265 3375 5265 2970 3825 3105 3105 3780
+2 1 0 1 0 0 50 -1 -1 0.000 0 0 -1 1 0 4
+ 1 1 1.00 30.00 60.00
+ 6165 3375 6165 2835 3780 3060 3105 3735
+4 1 0 50 -1 16 8 1.5708 4 120 630 2160 1710 tcp-request\001
+4 1 0 50 -1 16 8 1.5708 4 120 870 3060 1710 tcp-req content\001
+4 1 0 50 -1 16 8 1.5708 4 120 600 5085 1710 http-req in\001
+4 1 0 50 -1 16 8 1.5708 4 105 690 3960 1710 use-backend\001
+4 1 0 50 -1 16 8 1.5708 4 75 570 5535 1710 use-server\001
+4 1 0 50 -1 16 8 1.5708 4 120 360 1710 1710 accept\001
+4 0 0 50 -1 18 6 0.0000 4 90 435 2115 540 frontend\001
+4 0 0 50 -1 18 6 0.0000 4 90 405 2115 765 backend\001
+4 0 0 50 -1 18 6 0.0000 4 105 150 3735 540 tcp\001
+4 0 0 50 -1 18 6 0.0000 4 105 450 3735 765 http only\001
+4 2 0 50 -1 18 6 0.0000 4 90 435 4050 2430 frontend\001
+4 0 0 50 -1 18 6 0.0000 4 90 405 4455 2430 backend\001
+4 1 0 50 -1 16 8 1.5708 4 120 675 6660 1710 http-req out\001
+4 1 0 50 -1 16 8 1.5708 4 120 675 7110 1710 http-req out\001
+4 1 0 50 -1 16 8 1.5708 4 120 600 3510 1710 http-req in\001
+4 1 0 50 -1 16 8 1.5708 4 120 870 4635 1710 tcp-req content\001
+4 1 0 50 -1 16 8 1.5708 4 120 660 6210 3960 http-resp in\001
+4 1 0 50 -1 16 8 1.5708 4 120 930 6660 3960 tcp-resp content\001
+4 1 0 50 -1 16 8 1.5708 4 120 900 7110 3960 tcp-resp inspect\001
+4 1 0 50 -1 16 8 1.5708 4 120 930 5760 3960 tcp-resp content\001
+4 1 0 50 -1 16 8 1.5708 4 120 660 5310 3960 http-resp in\001
+4 0 0 50 -1 18 6 0.0000 4 90 405 6075 4680 backend\001
+4 1 0 50 -1 16 8 1.5708 4 90 300 4050 3960 HTTP\001
+4 1 0 50 -1 16 8 1.5708 4 120 615 4185 3960 processing\001
+4 1 0 50 -1 16 8 1.5708 4 90 300 2835 3915 Error\001
+4 1 0 50 -1 16 8 1.5708 4 120 615 2970 3915 processing\001
+4 2 0 50 -1 18 6 0.0000 4 90 435 5895 4680 frontend\001
diff --git a/doc/architecture.txt b/doc/architecture.txt
new file mode 100644
index 0000000..c37632f
--- /dev/null
+++ b/doc/architecture.txt
@@ -0,0 +1,1448 @@
+ -------------------
+ HAProxy
+ Architecture Guide
+ -------------------
+ version 1.1.34
+ willy tarreau
+ 2006/01/29
+
+
+This document provides real world examples with working configurations.
+Please note that except stated otherwise, global configuration parameters
+such as logging, chrooting, limits and time-outs are not described here.
+
+===================================================
+1. Simple HTTP load-balancing with cookie insertion
+===================================================
+
+A web application often saturates the front-end server with high CPU loads,
+due to the scripting language involved. It also relies on a back-end database
+which is not much loaded. User contexts are stored on the server itself, and
+not in the database, so that simply adding another server with simple IP/TCP
+load-balancing would not work.
+
+ +-------+
+ |clients| clients and/or reverse-proxy
+ +---+---+
+ |
+ -+-----+--------+----
+ | _|_db
+ +--+--+ (___)
+ | web | (___)
+ +-----+ (___)
+ 192.168.1.1 192.168.1.2
+
+
+Replacing the web server with a bigger SMP system would cost much more than
+adding low-cost pizza boxes. The solution is to buy N cheap boxes and install
+the application on them. Install haproxy on the old one which will spread the
+load across the new boxes.
+
+ 192.168.1.1 192.168.1.11-192.168.1.14 192.168.1.2
+ -------+-----------+-----+-----+-----+--------+----
+ | | | | | _|_db
+ +--+--+ +-+-+ +-+-+ +-+-+ +-+-+ (___)
+ | LB1 | | A | | B | | C | | D | (___)
+ +-----+ +---+ +---+ +---+ +---+ (___)
+ haproxy 4 cheap web servers
+
+
+Config on haproxy (LB1) :
+-------------------------
+
+ listen webfarm 192.168.1.1:80
+ mode http
+ balance roundrobin
+ cookie SERVERID insert indirect
+ option httpchk HEAD /index.html HTTP/1.0
+ server webA 192.168.1.11:80 cookie A check
+ server webB 192.168.1.12:80 cookie B check
+ server webC 192.168.1.13:80 cookie C check
+ server webD 192.168.1.14:80 cookie D check
+
+
+Description :
+-------------
+ - LB1 will receive clients requests.
+ - if a request does not contain a cookie, it will be forwarded to a valid
+ server
+ - in return, a cookie "SERVERID" will be inserted in the response holding the
+ server name (eg: "A").
+ - when the client comes again with the cookie "SERVERID=A", LB1 will know that
+ it must be forwarded to server A. The cookie will be removed so that the
+ server does not see it.
+ - if server "webA" dies, the requests will be sent to another valid server
+ and a cookie will be reassigned.
+
+
+Flows :
+-------
+
+(client) (haproxy) (server A)
+ >-- GET /URI1 HTTP/1.0 ------------> |
+ ( no cookie, haproxy forwards in load-balancing mode. )
+ | >-- GET /URI1 HTTP/1.0 ---------->
+ | <-- HTTP/1.0 200 OK -------------<
+ ( the proxy now adds the server cookie in return )
+ <-- HTTP/1.0 200 OK ---------------< |
+ Set-Cookie: SERVERID=A |
+ >-- GET /URI2 HTTP/1.0 ------------> |
+ Cookie: SERVERID=A |
+ ( the proxy sees the cookie. it forwards to server A and deletes it )
+ | >-- GET /URI2 HTTP/1.0 ---------->
+ | <-- HTTP/1.0 200 OK -------------<
+ ( the proxy does not add the cookie in return because the client knows it )
+ <-- HTTP/1.0 200 OK ---------------< |
+ >-- GET /URI3 HTTP/1.0 ------------> |
+ Cookie: SERVERID=A |
+ ( ... )
+
+
+Limits :
+--------
+ - if clients use keep-alive (HTTP/1.1), only the first response will have
+ a cookie inserted, and only the first request of each session will be
+ analyzed. This does not cause trouble in insertion mode because the cookie
+ is put immediately in the first response, and the session is maintained to
+ the same server for all subsequent requests in the same session. However,
+ the cookie will not be removed from the requests forwarded to the servers,
+ so the server must not be sensitive to unknown cookies. If this causes
+ trouble, you can disable keep-alive by adding the following option :
+
+ option httpclose
+
+ - if for some reason the clients cannot learn more than one cookie (eg: the
+ clients are indeed some home-made applications or gateways), and the
+ application already produces a cookie, you can use the "prefix" mode (see
+ below).
+
+ - LB1 becomes a very sensible server. If LB1 dies, nothing works anymore.
+ => you can back it up using keepalived (see below)
+
+ - if the application needs to log the original client's IP, use the
+ "forwardfor" option which will add an "X-Forwarded-For" header with the
+ original client's IP address. You must also use "httpclose" to ensure
+ that you will rewrite every requests and not only the first one of each
+ session :
+
+ option httpclose
+ option forwardfor
+
+ - if the application needs to log the original destination IP, use the
+ "originalto" option which will add an "X-Original-To" header with the
+ original destination IP address. You must also use "httpclose" to ensure
+ that you will rewrite every requests and not only the first one of each
+ session :
+
+ option httpclose
+ option originalto
+
+ The web server will have to be configured to use this header instead.
+ For example, on apache, you can use LogFormat for this :
+
+ LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b " combined
+ CustomLog /var/log/httpd/access_log combined
+
+Hints :
+-------
+Sometimes on the internet, you will find a few percent of the clients which
+disable cookies on their browser. Obviously they have troubles everywhere on
+the web, but you can still help them access your site by using the "source"
+balancing algorithm instead of the "roundrobin". It ensures that a given IP
+address always reaches the same server as long as the number of servers remains
+unchanged. Never use this behind a proxy or in a small network, because the
+distribution will be unfair. However, in large internal networks, and on the
+internet, it works quite well. Clients which have a dynamic address will not
+be affected as long as they accept the cookie, because the cookie always has
+precedence over load balancing :
+
+ listen webfarm 192.168.1.1:80
+ mode http
+ balance source
+ cookie SERVERID insert indirect
+ option httpchk HEAD /index.html HTTP/1.0
+ server webA 192.168.1.11:80 cookie A check
+ server webB 192.168.1.12:80 cookie B check
+ server webC 192.168.1.13:80 cookie C check
+ server webD 192.168.1.14:80 cookie D check
+
+
+==================================================================
+2. HTTP load-balancing with cookie prefixing and high availability
+==================================================================
+
+Now you don't want to add more cookies, but rather use existing ones. The
+application already generates a "JSESSIONID" cookie which is enough to track
+sessions, so we'll prefix this cookie with the server name when we see it.
+Since the load-balancer becomes critical, it will be backed up with a second
+one in VRRP mode using keepalived under Linux.
+
+Download the latest version of keepalived from this site and install it
+on each load-balancer LB1 and LB2 :
+
+ http://www.keepalived.org/
+
+You then have a shared IP between the two load-balancers (we will still use the
+original IP). It is active only on one of them at any moment. To allow the
+proxy to bind to the shared IP on Linux 2.4, you must enable it in /proc :
+
+# echo 1 >/proc/sys/net/ipv4/ip_nonlocal_bind
+
+
+ shared IP=192.168.1.1
+ 192.168.1.3 192.168.1.4 192.168.1.11-192.168.1.14 192.168.1.2
+ -------+------------+-----------+-----+-----+-----+--------+----
+ | | | | | | _|_db
+ +--+--+ +--+--+ +-+-+ +-+-+ +-+-+ +-+-+ (___)
+ | LB1 | | LB2 | | A | | B | | C | | D | (___)
+ +-----+ +-----+ +---+ +---+ +---+ +---+ (___)
+ haproxy haproxy 4 cheap web servers
+ keepalived keepalived
+
+
+Config on both proxies (LB1 and LB2) :
+--------------------------------------
+
+ listen webfarm 192.168.1.1:80
+ mode http
+ balance roundrobin
+ cookie JSESSIONID prefix
+ option httpclose
+ option forwardfor
+ option httpchk HEAD /index.html HTTP/1.0
+ server webA 192.168.1.11:80 cookie A check
+ server webB 192.168.1.12:80 cookie B check
+ server webC 192.168.1.13:80 cookie C check
+ server webD 192.168.1.14:80 cookie D check
+
+
+Notes: the proxy will modify EVERY cookie sent by the client and the server,
+so it is important that it can access to ALL cookies in ALL requests for
+each session. This implies that there is no keep-alive (HTTP/1.1), thus the
+"httpclose" option. Only if you know for sure that the client(s) will never
+use keep-alive (eg: Apache 1.3 in reverse-proxy mode), you can remove this
+option.
+
+
+Configuration for keepalived on LB1/LB2 :
+-----------------------------------------
+
+ vrrp_script chk_haproxy { # Requires keepalived-1.1.13
+ script "killall -0 haproxy" # cheaper than pidof
+ interval 2 # check every 2 seconds
+ weight 2 # add 2 points of prio if OK
+ }
+
+ vrrp_instance VI_1 {
+ interface eth0
+ state MASTER
+ virtual_router_id 51
+ priority 101 # 101 on master, 100 on backup
+ virtual_ipaddress {
+ 192.168.1.1
+ }
+ track_script {
+ chk_haproxy
+ }
+ }
+
+
+Description :
+-------------
+ - LB1 is VRRP master (keepalived), LB2 is backup. Both monitor the haproxy
+ process, and lower their prio if it fails, leading to a failover to the
+ other node.
+ - LB1 will receive clients requests on IP 192.168.1.1.
+ - both load-balancers send their checks from their native IP.
+ - if a request does not contain a cookie, it will be forwarded to a valid
+ server
+ - in return, if a JESSIONID cookie is seen, the server name will be prefixed
+ into it, followed by a delimiter ('~')
+ - when the client comes again with the cookie "JSESSIONID=A~xxx", LB1 will
+ know that it must be forwarded to server A. The server name will then be
+ extracted from cookie before it is sent to the server.
+ - if server "webA" dies, the requests will be sent to another valid server
+ and a cookie will be reassigned.
+
+
+Flows :
+-------
+
+(client) (haproxy) (server A)
+ >-- GET /URI1 HTTP/1.0 ------------> |
+ ( no cookie, haproxy forwards in load-balancing mode. )
+ | >-- GET /URI1 HTTP/1.0 ---------->
+ | X-Forwarded-For: 10.1.2.3
+ | <-- HTTP/1.0 200 OK -------------<
+ ( no cookie, nothing changed )
+ <-- HTTP/1.0 200 OK ---------------< |
+ >-- GET /URI2 HTTP/1.0 ------------> |
+ ( no cookie, haproxy forwards in lb mode, possibly to another server. )
+ | >-- GET /URI2 HTTP/1.0 ---------->
+ | X-Forwarded-For: 10.1.2.3
+ | <-- HTTP/1.0 200 OK -------------<
+ | Set-Cookie: JSESSIONID=123
+ ( the cookie is identified, it will be prefixed with the server name )
+ <-- HTTP/1.0 200 OK ---------------< |
+ Set-Cookie: JSESSIONID=A~123 |
+ >-- GET /URI3 HTTP/1.0 ------------> |
+ Cookie: JSESSIONID=A~123 |
+ ( the proxy sees the cookie, removes the server name and forwards
+ to server A which sees the same cookie as it previously sent )
+ | >-- GET /URI3 HTTP/1.0 ---------->
+ | Cookie: JSESSIONID=123
+ | X-Forwarded-For: 10.1.2.3
+ | <-- HTTP/1.0 200 OK -------------<
+ ( no cookie, nothing changed )
+ <-- HTTP/1.0 200 OK ---------------< |
+ ( ... )
+
+Hints :
+-------
+Sometimes, there will be some powerful servers in the farm, and some smaller
+ones. In this situation, it may be desirable to tell haproxy to respect the
+difference in performance. Let's consider that WebA and WebB are two old
+P3-1.2 GHz while WebC and WebD are shiny new Opteron-2.6 GHz. If your
+application scales with CPU, you may assume a very rough 2.6/1.2 performance
+ratio between the servers. You can inform haproxy about this using the "weight"
+keyword, with values between 1 and 256. It will then spread the load the most
+smoothly possible respecting those ratios :
+
+ server webA 192.168.1.11:80 cookie A weight 12 check
+ server webB 192.168.1.12:80 cookie B weight 12 check
+ server webC 192.168.1.13:80 cookie C weight 26 check
+ server webD 192.168.1.14:80 cookie D weight 26 check
+
+
+========================================================
+2.1 Variations involving external layer 4 load-balancers
+========================================================
+
+Instead of using a VRRP-based active/backup solution for the proxies,
+they can also be load-balanced by a layer4 load-balancer (eg: Alteon)
+which will also check that the services run fine on both proxies :
+
+ | VIP=192.168.1.1
+ +----+----+
+ | Alteon |
+ +----+----+
+ |
+ 192.168.1.3 | 192.168.1.4 192.168.1.11-192.168.1.14 192.168.1.2
+ -------+-----+------+-----------+-----+-----+-----+--------+----
+ | | | | | | _|_db
+ +--+--+ +--+--+ +-+-+ +-+-+ +-+-+ +-+-+ (___)
+ | LB1 | | LB2 | | A | | B | | C | | D | (___)
+ +-----+ +-----+ +---+ +---+ +---+ +---+ (___)
+ haproxy haproxy 4 cheap web servers
+
+
+Config on both proxies (LB1 and LB2) :
+--------------------------------------
+
+ listen webfarm 0.0.0.0:80
+ mode http
+ balance roundrobin
+ cookie JSESSIONID prefix
+ option httpclose
+ option forwardfor
+ option httplog
+ option dontlognull
+ option httpchk HEAD /index.html HTTP/1.0
+ server webA 192.168.1.11:80 cookie A check
+ server webB 192.168.1.12:80 cookie B check
+ server webC 192.168.1.13:80 cookie C check
+ server webD 192.168.1.14:80 cookie D check
+
+The "dontlognull" option is used to prevent the proxy from logging the health
+checks from the Alteon. If a session exchanges no data, then it will not be
+logged.
+
+Config on the Alteon :
+----------------------
+
+ /c/slb/real 11
+ ena
+ name "LB1"
+ rip 192.168.1.3
+ /c/slb/real 12
+ ena
+ name "LB2"
+ rip 192.168.1.4
+ /c/slb/group 10
+ name "LB1-2"
+ metric roundrobin
+ health tcp
+ add 11
+ add 12
+ /c/slb/virt 10
+ ena
+ vip 192.168.1.1
+ /c/slb/virt 10/service http
+ group 10
+
+
+Note: the health-check on the Alteon is set to "tcp" to prevent the proxy from
+forwarding the connections. It can also be set to "http", but for this the
+proxy must specify a "monitor-net" with the Alteons' addresses, so that the
+Alteon can really check that the proxies can talk HTTP but without forwarding
+the connections to the end servers. Check next section for an example on how to
+use monitor-net.
+
+
+============================================================
+2.2 Generic TCP relaying and external layer 4 load-balancers
+============================================================
+
+Sometimes it's useful to be able to relay generic TCP protocols (SMTP, TSE,
+VNC, etc...), for example to interconnect private networks. The problem comes
+when you use external load-balancers which need to send periodic health-checks
+to the proxies, because these health-checks get forwarded to the end servers.
+The solution is to specify a network which will be dedicated to monitoring
+systems and must not lead to a forwarding connection nor to any log, using the
+"monitor-net" keyword. Note: this feature expects a version of haproxy greater
+than or equal to 1.1.32 or 1.2.6.
+
+
+ | VIP=172.16.1.1 |
+ +----+----+ +----+----+
+ | Alteon1 | | Alteon2 |
+ +----+----+ +----+----+
+ 192.168.1.252 | GW=192.168.1.254 | 192.168.1.253
+ | |
+ ------+---+------------+--+-----------------> TSE farm : 192.168.1.10
+ 192.168.1.1 | | 192.168.1.2
+ +--+--+ +--+--+
+ | LB1 | | LB2 |
+ +-----+ +-----+
+ haproxy haproxy
+
+
+Config on both proxies (LB1 and LB2) :
+--------------------------------------
+
+ listen tse-proxy
+ bind :3389,:1494,:5900 # TSE, ICA and VNC at once.
+ mode tcp
+ balance roundrobin
+ server tse-farm 192.168.1.10
+ monitor-net 192.168.1.252/31
+
+The "monitor-net" option instructs the proxies that any connection coming from
+192.168.1.252 or 192.168.1.253 will not be logged nor forwarded and will be
+closed immediately. The Alteon load-balancers will then see the proxies alive
+without perturbating the service.
+
+Config on the Alteon :
+----------------------
+
+ /c/l3/if 1
+ ena
+ addr 192.168.1.252
+ mask 255.255.255.0
+ /c/slb/real 11
+ ena
+ name "LB1"
+ rip 192.168.1.1
+ /c/slb/real 12
+ ena
+ name "LB2"
+ rip 192.168.1.2
+ /c/slb/group 10
+ name "LB1-2"
+ metric roundrobin
+ health tcp
+ add 11
+ add 12
+ /c/slb/virt 10
+ ena
+ vip 172.16.1.1
+ /c/slb/virt 10/service 1494
+ group 10
+ /c/slb/virt 10/service 3389
+ group 10
+ /c/slb/virt 10/service 5900
+ group 10
+
+
+Special handling of SSL :
+-------------------------
+Sometimes, you want to send health-checks to remote systems, even in TCP mode,
+in order to be able to failover to a backup server in case the first one is
+dead. Of course, you can simply enable TCP health-checks, but it sometimes
+happens that intermediate firewalls between the proxies and the remote servers
+acknowledge the TCP connection themselves, showing an always-up server. Since
+this is generally encountered on long-distance communications, which often
+involve SSL, an SSL health-check has been implemented to work around this issue.
+It sends SSL Hello messages to the remote server, which in turns replies with
+SSL Hello messages. Setting it up is very easy :
+
+ listen tcp-syslog-proxy
+ bind :1514 # listen to TCP syslog traffic on this port (SSL)
+ mode tcp
+ balance roundrobin
+ option ssl-hello-chk
+ server syslog-prod-site 192.168.1.10 check
+ server syslog-back-site 192.168.2.10 check backup
+
+
+=========================================================
+3. Simple HTTP/HTTPS load-balancing with cookie insertion
+=========================================================
+
+This is the same context as in example 1 above, but the web
+server uses HTTPS.
+
+ +-------+
+ |clients| clients
+ +---+---+
+ |
+ -+-----+--------+----
+ | _|_db
+ +--+--+ (___)
+ | SSL | (___)
+ | web | (___)
+ +-----+
+ 192.168.1.1 192.168.1.2
+
+
+Since haproxy does not handle SSL, this part will have to be extracted from the
+servers (freeing even more resources) and installed on the load-balancer
+itself. Install haproxy and apache+mod_ssl on the old box which will spread the
+load between the new boxes. Apache will work in SSL reverse-proxy-cache. If the
+application is correctly developed, it might even lower its load. However,
+since there now is a cache between the clients and haproxy, some security
+measures must be taken to ensure that inserted cookies will not be cached.
+
+
+ 192.168.1.1 192.168.1.11-192.168.1.14 192.168.1.2
+ -------+-----------+-----+-----+-----+--------+----
+ | | | | | _|_db
+ +--+--+ +-+-+ +-+-+ +-+-+ +-+-+ (___)
+ | LB1 | | A | | B | | C | | D | (___)
+ +-----+ +---+ +---+ +---+ +---+ (___)
+ apache 4 cheap web servers
+ mod_ssl
+ haproxy
+
+
+Config on haproxy (LB1) :
+-------------------------
+
+ listen 127.0.0.1:8000
+ mode http
+ balance roundrobin
+ cookie SERVERID insert indirect nocache
+ option httpchk HEAD /index.html HTTP/1.0
+ server webA 192.168.1.11:80 cookie A check
+ server webB 192.168.1.12:80 cookie B check
+ server webC 192.168.1.13:80 cookie C check
+ server webD 192.168.1.14:80 cookie D check
+
+
+Description :
+-------------
+ - apache on LB1 will receive clients requests on port 443
+ - it forwards it to haproxy bound to 127.0.0.1:8000
+ - if a request does not contain a cookie, it will be forwarded to a valid
+ server
+ - in return, a cookie "SERVERID" will be inserted in the response holding the
+ server name (eg: "A"), and a "Cache-control: private" header will be added
+ so that the apache does not cache any page containing such cookie.
+ - when the client comes again with the cookie "SERVERID=A", LB1 will know that
+ it must be forwarded to server A. The cookie will be removed so that the
+ server does not see it.
+ - if server "webA" dies, the requests will be sent to another valid server
+ and a cookie will be reassigned.
+
+Notes :
+-------
+ - if the cookie works in "prefix" mode, there is no need to add the "nocache"
+ option because it is an application cookie which will be modified, and the
+ application flags will be preserved.
+ - if apache 1.3 is used as a front-end before haproxy, it always disables
+ HTTP keep-alive on the back-end, so there is no need for the "httpclose"
+ option on haproxy.
+ - configure apache to set the X-Forwarded-For header itself, and do not do
+ it on haproxy if you need the application to know about the client's IP.
+
+
+Flows :
+-------
+
+(apache) (haproxy) (server A)
+ >-- GET /URI1 HTTP/1.0 ------------> |
+ ( no cookie, haproxy forwards in load-balancing mode. )
+ | >-- GET /URI1 HTTP/1.0 ---------->
+ | <-- HTTP/1.0 200 OK -------------<
+ ( the proxy now adds the server cookie in return )
+ <-- HTTP/1.0 200 OK ---------------< |
+ Set-Cookie: SERVERID=A |
+ Cache-Control: private |
+ >-- GET /URI2 HTTP/1.0 ------------> |
+ Cookie: SERVERID=A |
+ ( the proxy sees the cookie. it forwards to server A and deletes it )
+ | >-- GET /URI2 HTTP/1.0 ---------->
+ | <-- HTTP/1.0 200 OK -------------<
+ ( the proxy does not add the cookie in return because the client knows it )
+ <-- HTTP/1.0 200 OK ---------------< |
+ >-- GET /URI3 HTTP/1.0 ------------> |
+ Cookie: SERVERID=A |
+ ( ... )
+
+
+
+========================================
+3.1. Alternate solution using Stunnel
+========================================
+
+When only SSL is required and cache is not needed, stunnel is a cheaper
+solution than Apache+mod_ssl. By default, stunnel does not process HTTP and
+does not add any X-Forwarded-For header, but there is a patch on the official
+haproxy site to provide this feature to recent stunnel versions.
+
+This time, stunnel will only process HTTPS and not HTTP. This means that
+haproxy will get all HTTP traffic, so haproxy will have to add the
+X-Forwarded-For header for HTTP traffic, but not for HTTPS traffic since
+stunnel will already have done it. We will use the "except" keyword to tell
+haproxy that connections from local host already have a valid header.
+
+
+ 192.168.1.1 192.168.1.11-192.168.1.14 192.168.1.2
+ -------+-----------+-----+-----+-----+--------+----
+ | | | | | _|_db
+ +--+--+ +-+-+ +-+-+ +-+-+ +-+-+ (___)
+ | LB1 | | A | | B | | C | | D | (___)
+ +-----+ +---+ +---+ +---+ +---+ (___)
+ stunnel 4 cheap web servers
+ haproxy
+
+
+Config on stunnel (LB1) :
+-------------------------
+
+ cert=/etc/stunnel/stunnel.pem
+ setuid=stunnel
+ setgid=proxy
+
+ socket=l:TCP_NODELAY=1
+ socket=r:TCP_NODELAY=1
+
+ [https]
+ accept=192.168.1.1:443
+ connect=192.168.1.1:80
+ xforwardedfor=yes
+
+
+Config on haproxy (LB1) :
+-------------------------
+
+ listen 192.168.1.1:80
+ mode http
+ balance roundrobin
+ option forwardfor except 192.168.1.1
+ cookie SERVERID insert indirect nocache
+ option httpchk HEAD /index.html HTTP/1.0
+ server webA 192.168.1.11:80 cookie A check
+ server webB 192.168.1.12:80 cookie B check
+ server webC 192.168.1.13:80 cookie C check
+ server webD 192.168.1.14:80 cookie D check
+
+Description :
+-------------
+ - stunnel on LB1 will receive clients requests on port 443
+ - it forwards them to haproxy bound to port 80
+ - haproxy will receive HTTP client requests on port 80 and decrypted SSL
+ requests from Stunnel on the same port.
+ - stunnel will add the X-Forwarded-For header
+ - haproxy will add the X-Forwarded-For header for everyone except the local
+ address (stunnel).
+
+
+========================================
+4. Soft-stop for application maintenance
+========================================
+
+When an application is spread across several servers, the time to update all
+instances increases, so the application seems jerky for a longer period.
+
+HAProxy offers several solutions for this. Although it cannot be reconfigured
+without being stopped, nor does it offer any external command, there are other
+working solutions.
+
+
+=========================================
+4.1 Soft-stop using a file on the servers
+=========================================
+
+This trick is quite common and very simple: put a file on the server which will
+be checked by the proxy. When you want to stop the server, first remove this
+file. The proxy will see the server as failed, and will not send it any new
+session, only the old ones if the "persist" option is used. Wait a bit then
+stop the server when it does not receive anymore connections.
+
+
+ listen 192.168.1.1:80
+ mode http
+ balance roundrobin
+ cookie SERVERID insert indirect
+ option httpchk HEAD /running HTTP/1.0
+ server webA 192.168.1.11:80 cookie A check inter 2000 rise 2 fall 2
+ server webB 192.168.1.12:80 cookie B check inter 2000 rise 2 fall 2
+ server webC 192.168.1.13:80 cookie C check inter 2000 rise 2 fall 2
+ server webD 192.168.1.14:80 cookie D check inter 2000 rise 2 fall 2
+ option persist
+ redispatch
+ contimeout 5000
+
+
+Description :
+-------------
+ - every 2 seconds, haproxy will try to access the file "/running" on the
+ servers, and declare the server as down after 2 attempts (4 seconds).
+ - only the servers which respond with a 200 or 3XX response will be used.
+ - if a request does not contain a cookie, it will be forwarded to a valid
+ server
+ - if a request contains a cookie for a failed server, haproxy will insist
+ on trying to reach the server anyway, to let the user finish what they were
+ doing. ("persist" option)
+ - if the server is totally stopped, the connection will fail and the proxy
+ will rebalance the client to another server ("redispatch")
+
+Usage on the web servers :
+--------------------------
+- to start the server :
+ # /etc/init.d/httpd start
+ # touch /home/httpd/www/running
+
+- to soft-stop the server
+ # rm -f /home/httpd/www/running
+
+- to completely stop the server :
+ # /etc/init.d/httpd stop
+
+Limits
+------
+If the server is totally powered down, the proxy will still try to reach it
+for those clients who still have a cookie referencing it, and the connection
+attempt will expire after 5 seconds ("contimeout"), and only after that, the
+client will be redispatched to another server. So this mode is only useful
+for software updates where the server will suddenly refuse the connection
+because the process is stopped. The problem is the same if the server suddenly
+crashes. All of its users will be fairly perturbated.
+
+
+==================================
+4.2 Soft-stop using backup servers
+==================================
+
+A better solution which covers every situation is to use backup servers.
+Version 1.1.30 fixed a bug which prevented a backup server from sharing
+the same cookie as a standard server.
+
+
+ listen 192.168.1.1:80
+ mode http
+ balance roundrobin
+ redispatch
+ cookie SERVERID insert indirect
+ option httpchk HEAD / HTTP/1.0
+ server webA 192.168.1.11:80 cookie A check port 81 inter 2000
+ server webB 192.168.1.12:80 cookie B check port 81 inter 2000
+ server webC 192.168.1.13:80 cookie C check port 81 inter 2000
+ server webD 192.168.1.14:80 cookie D check port 81 inter 2000
+
+ server bkpA 192.168.1.11:80 cookie A check port 80 inter 2000 backup
+ server bkpB 192.168.1.12:80 cookie B check port 80 inter 2000 backup
+ server bkpC 192.168.1.13:80 cookie C check port 80 inter 2000 backup
+ server bkpD 192.168.1.14:80 cookie D check port 80 inter 2000 backup
+
+Description
+-----------
+Four servers webA..D are checked on their port 81 every 2 seconds. The same
+servers named bkpA..D are checked on the port 80, and share the exact same
+cookies. Those servers will only be used when no other server is available
+for the same cookie.
+
+When the web servers are started, only the backup servers are seen as
+available. On the web servers, you need to redirect port 81 to local
+port 80, either with a local proxy (eg: a simple haproxy tcp instance),
+or with iptables (linux) or pf (openbsd). This is because we want the
+real web server to reply on this port, and not a fake one. Eg, with
+iptables :
+
+ # /etc/init.d/httpd start
+ # iptables -t nat -A PREROUTING -p tcp --dport 81 -j REDIRECT --to-port 80
+
+A few seconds later, the standard server is seen up and haproxy starts to send
+it new requests on its real port 80 (only new users with no cookie, of course).
+
+If a server completely crashes (even if it does not respond at the IP level),
+both the standard and backup servers will fail, so clients associated to this
+server will be redispatched to other live servers and will lose their sessions.
+
+Now if you want to enter a server into maintenance, simply stop it from
+responding on port 81 so that its standard instance will be seen as failed,
+but the backup will still work. Users will not notice anything since the
+service is still operational :
+
+ # iptables -t nat -D PREROUTING -p tcp --dport 81 -j REDIRECT --to-port 80
+
+The health checks on port 81 for this server will quickly fail, and the
+standard server will be seen as failed. No new session will be sent to this
+server, and existing clients with a valid cookie will still reach it because
+the backup server will still be up.
+
+Now wait as long as you want for the old users to stop using the service, and
+once you see that the server does not receive any traffic, simply stop it :
+
+ # /etc/init.d/httpd stop
+
+The associated backup server will in turn fail, and if any client still tries
+to access this particular server, they will be redispatched to any other valid
+server because of the "redispatch" option.
+
+This method has an advantage : you never touch the proxy when doing server
+maintenance. The people managing the servers can make them disappear smoothly.
+
+
+4.2.1 Variations for operating systems without any firewall software
+--------------------------------------------------------------------
+
+The downside is that you need a redirection solution on the server just for
+the health-checks. If the server OS does not support any firewall software,
+this redirection can also be handled by a simple haproxy in tcp mode :
+
+ global
+ daemon
+ quiet
+ pidfile /var/run/haproxy-checks.pid
+ listen 0.0.0.0:81
+ mode tcp
+ dispatch 127.0.0.1:80
+ contimeout 1000
+ clitimeout 10000
+ srvtimeout 10000
+
+To start the web service :
+
+ # /etc/init.d/httpd start
+ # haproxy -f /etc/haproxy/haproxy-checks.cfg
+
+To soft-stop the service :
+
+ # kill $(</var/run/haproxy-checks.pid)
+
+The port 81 will stop responding and the load-balancer will notice the failure.
+
+
+4.2.2 Centralizing the server management
+----------------------------------------
+
+If one finds it preferable to manage the servers from the load-balancer itself,
+the port redirector can be installed on the load-balancer itself. See the
+example with iptables below.
+
+Make the servers appear as operational :
+ # iptables -t nat -A OUTPUT -d 192.168.1.11 -p tcp --dport 81 -j DNAT --to-dest :80
+ # iptables -t nat -A OUTPUT -d 192.168.1.12 -p tcp --dport 81 -j DNAT --to-dest :80
+ # iptables -t nat -A OUTPUT -d 192.168.1.13 -p tcp --dport 81 -j DNAT --to-dest :80
+ # iptables -t nat -A OUTPUT -d 192.168.1.14 -p tcp --dport 81 -j DNAT --to-dest :80
+
+Soft stop one server :
+ # iptables -t nat -D OUTPUT -d 192.168.1.12 -p tcp --dport 81 -j DNAT --to-dest :80
+
+Another solution is to use the "COMAFILE" patch provided by Alexander Lazic,
+which is available for download here :
+
+ http://w.ods.org/tools/haproxy/contrib/
+
+
+4.2.3 Notes :
+-------------
+ - Never, ever, start a fake service on port 81 for the health-checks, because
+ a real web service failure will not be detected as long as the fake service
+ runs. You must really forward the check port to the real application.
+
+ - health-checks will be sent twice as often, once for each standard server,
+ and once for each backup server. All this will be multiplicated by the
+ number of processes if you use multi-process mode. You will have to ensure
+ that all the checks sent to the server do not overload it.
+
+=======================
+4.3 Hot reconfiguration
+=======================
+
+There are two types of haproxy users :
+ - those who can never do anything in production out of maintenance periods ;
+ - those who can do anything at any time provided that the consequences are
+ limited.
+
+The first ones have no problem stopping the server to change configuration
+because they got some maintenance periods during which they can break anything.
+So they will even prefer doing a clean stop/start sequence to ensure everything
+will work fine upon next reload. Since those have represented the majority of
+haproxy uses, there has been little effort trying to improve this.
+
+However, the second category is a bit different. They like to be able to fix an
+error in a configuration file without anyone noticing. This can sometimes also
+be the case for the first category because humans are not failsafe.
+
+For this reason, a new hot reconfiguration mechanism has been introduced in
+version 1.1.34. Its usage is very simple and works even in chrooted
+environments with lowered privileges. The principle is very simple : upon
+reception of a SIGTTOU signal, the proxy will stop listening to all the ports.
+This will release the ports so that a new instance can be started. Existing
+connections will not be broken at all. If the new instance fails to start,
+then sending a SIGTTIN signal back to the original processes will restore
+the listening ports. This is possible without any special privileges because
+the sockets will not have been closed, so the bind() is still valid. Otherwise,
+if the new process starts successfully, then sending a SIGUSR1 signal to the
+old one ensures that it will exit as soon as its last session ends.
+
+A hot reconfiguration script would look like this :
+
+ # save previous state
+ mv /etc/haproxy/config /etc/haproxy/config.old
+ mv /var/run/haproxy.pid /var/run/haproxy.pid.old
+
+ mv /etc/haproxy/config.new /etc/haproxy/config
+ kill -TTOU $(cat /var/run/haproxy.pid.old)
+ if haproxy -p /var/run/haproxy.pid -f /etc/haproxy/config; then
+ echo "New instance successfully loaded, stopping previous one."
+ kill -USR1 $(cat /var/run/haproxy.pid.old)
+ rm -f /var/run/haproxy.pid.old
+ exit 1
+ else
+ echo "New instance failed to start, resuming previous one."
+ kill -TTIN $(cat /var/run/haproxy.pid.old)
+ rm -f /var/run/haproxy.pid
+ mv /var/run/haproxy.pid.old /var/run/haproxy.pid
+ mv /etc/haproxy/config /etc/haproxy/config.new
+ mv /etc/haproxy/config.old /etc/haproxy/config
+ exit 0
+ fi
+
+After this, you can still force old connections to end by sending
+a SIGTERM to the old process if it still exists :
+
+ kill $(cat /var/run/haproxy.pid.old)
+ rm -f /var/run/haproxy.pid.old
+
+Be careful with this as in multi-process mode, some pids might already
+have been reallocated to completely different processes.
+
+
+==================================================
+5. Multi-site load-balancing with local preference
+==================================================
+
+5.1 Description of the problem
+==============================
+
+Consider a world-wide company with sites on several continents. There are two
+production sites SITE1 and SITE2 which host identical applications. There are
+many offices around the world. For speed and communication cost reasons, each
+office uses the nearest site by default, but can switch to the backup site in
+the event of a site or application failure. There also are users on the
+production sites, which use their local sites by default, but can switch to the
+other site in case of a local application failure.
+
+The main constraints are :
+
+ - application persistence : although the application is the same on both
+ sites, there is no session synchronisation between the sites. A failure
+ of one server or one site can cause a user to switch to another server
+ or site, but when the server or site comes back, the user must not switch
+ again.
+
+ - communication costs : inter-site communication should be reduced to the
+ minimum. Specifically, in case of a local application failure, every
+ office should be able to switch to the other site without continuing to
+ use the default site.
+
+5.2 Solution
+============
+ - Each production site will have two haproxy load-balancers in front of its
+ application servers to balance the load across them and provide local HA.
+ We will call them "S1L1" and "S1L2" on site 1, and "S2L1" and "S2L2" on
+ site 2. These proxies will extend the application's JSESSIONID cookie to
+ put the server name as a prefix.
+
+ - Each production site will have one front-end haproxy director to provide
+ the service to local users and to remote offices. It will load-balance
+ across the two local load-balancers, and will use the other site's
+ load-balancers as backup servers. It will insert the local site identifier
+ in a SITE cookie for the local load-balancers, and the remote site
+ identifier for the remote load-balancers. These front-end directors will
+ be called "SD1" and "SD2" for "Site Director".
+
+ - Each office will have one haproxy near the border gateway which will direct
+ local users to their preference site by default, or to the backup site in
+ the event of a previous failure. It will also analyze the SITE cookie, and
+ direct the users to the site referenced in the cookie. Thus, the preferred
+ site will be declared as a normal server, and the backup site will be
+ declared as a backup server only, which will only be used when the primary
+ site is unreachable, or when the primary site's director has forwarded
+ traffic to the second site. These proxies will be called "OP1".."OPXX"
+ for "Office Proxy #XX".
+
+
+5.3 Network diagram
+===================
+
+Note : offices 1 and 2 are on the same continent as site 1, while
+ office 3 is on the same continent as site 3. Each production
+ site can reach the second one either through the WAN or through
+ a dedicated link.
+
+
+ Office1 Office2 Office3
+ users users users
+192.168 # # # 192.168 # # # # # #
+.1.0/24 | | | .2.0/24 | | | 192.168.3.0/24 | | |
+ --+----+-+-+- --+----+-+-+- ---+----+-+-+-
+ | | .1 | | .1 | | .1
+ | +-+-+ | +-+-+ | +-+-+
+ | |OP1| | |OP2| | |OP3| ...
+ ,-:-. +---+ ,-:-. +---+ ,-:-. +---+
+ ( X ) ( X ) ( X )
+ `-:-' `-:-' ,---. `-:-'
+ --+---------------+------+----~~~( X )~~~~-------+---------+-
+ | `---' |
+ | |
+ +---+ ,-:-. +---+ ,-:-.
+ |SD1| ( X ) |SD2| ( X )
+ ( SITE 1 ) +-+-+ `-:-' ( SITE 2 ) +-+-+ `-:-'
+ |.1 | |.1 |
+ 10.1.1.0/24 | | ,---. 10.2.1.0/24 | |
+ -+-+-+-+-+-+-+-----+-+--( X )------+-+-+-+-+-+-+-----+-+--
+ | | | | | | | `---' | | | | | | |
+ ...# # # # # |.11 |.12 ...# # # # # |.11 |.12
+ Site 1 +-+--+ +-+--+ Site 2 +-+--+ +-+--+
+ Local |S1L1| |S1L2| Local |S2L1| |S2L2|
+ users +-+--+ +--+-+ users +-+--+ +--+-+
+ | | | |
+ 10.1.2.0/24 -+-+-+--+--++-- 10.2.2.0/24 -+-+-+--+--++--
+ |.1 |.4 |.1 |.4
+ +-+-+ +-+-+ +-+-+ +-+-+
+ |W11| ~~~ |W14| |W21| ~~~ |W24|
+ +---+ +---+ +---+ +---+
+ 4 application servers 4 application servers
+ on site 1 on site 2
+
+
+
+5.4 Description
+===============
+
+5.4.1 Local users
+-----------------
+ - Office 1 users connect to OP1 = 192.168.1.1
+ - Office 2 users connect to OP2 = 192.168.2.1
+ - Office 3 users connect to OP3 = 192.168.3.1
+ - Site 1 users connect to SD1 = 10.1.1.1
+ - Site 2 users connect to SD2 = 10.2.1.1
+
+5.4.2 Office proxies
+--------------------
+ - Office 1 connects to site 1 by default and uses site 2 as a backup.
+ - Office 2 connects to site 1 by default and uses site 2 as a backup.
+ - Office 3 connects to site 2 by default and uses site 1 as a backup.
+
+The offices check the local site's SD proxy every 30 seconds, and the
+remote one every 60 seconds.
+
+
+Configuration for Office Proxy OP1
+----------------------------------
+
+ listen 192.168.1.1:80
+ mode http
+ balance roundrobin
+ redispatch
+ cookie SITE
+ option httpchk HEAD / HTTP/1.0
+ server SD1 10.1.1.1:80 cookie SITE1 check inter 30000
+ server SD2 10.2.1.1:80 cookie SITE2 check inter 60000 backup
+
+
+Configuration for Office Proxy OP2
+----------------------------------
+
+ listen 192.168.2.1:80
+ mode http
+ balance roundrobin
+ redispatch
+ cookie SITE
+ option httpchk HEAD / HTTP/1.0
+ server SD1 10.1.1.1:80 cookie SITE1 check inter 30000
+ server SD2 10.2.1.1:80 cookie SITE2 check inter 60000 backup
+
+
+Configuration for Office Proxy OP3
+----------------------------------
+
+ listen 192.168.3.1:80
+ mode http
+ balance roundrobin
+ redispatch
+ cookie SITE
+ option httpchk HEAD / HTTP/1.0
+ server SD2 10.2.1.1:80 cookie SITE2 check inter 30000
+ server SD1 10.1.1.1:80 cookie SITE1 check inter 60000 backup
+
+
+5.4.3 Site directors ( SD1 and SD2 )
+------------------------------------
+The site directors forward traffic to the local load-balancers, and set a
+cookie to identify the site. If no local load-balancer is available, or if
+the local application servers are all down, it will redirect traffic to the
+remote site, and report this in the SITE cookie. In order not to uselessly
+load each site's WAN link, each SD will check the other site at a lower
+rate. The site directors will also insert their client's address so that
+the application server knows which local user or remote site accesses it.
+
+The SITE cookie which is set by these directors will also be understood
+by the office proxies. This is important because if SD1 decides to forward
+traffic to site 2, it will write "SITE2" in the "SITE" cookie, and on next
+request, the office proxy will automatically and directly talk to SITE2 if
+it can reach it. If it cannot, it will still send the traffic to SITE1
+where SD1 will in turn try to reach SITE2.
+
+The load-balancers checks are performed on port 81. As we'll see further,
+the load-balancers provide a health monitoring port 81 which reroutes to
+port 80 but which allows them to tell the SD that they are going down soon
+and that the SD must not use them anymore.
+
+
+Configuration for SD1
+---------------------
+
+ listen 10.1.1.1:80
+ mode http
+ balance roundrobin
+ redispatch
+ cookie SITE insert indirect
+ option httpchk HEAD / HTTP/1.0
+ option forwardfor
+ server S1L1 10.1.1.11:80 cookie SITE1 check port 81 inter 4000
+ server S1L2 10.1.1.12:80 cookie SITE1 check port 81 inter 4000
+ server S2L1 10.2.1.11:80 cookie SITE2 check port 81 inter 8000 backup
+ server S2L2 10.2.1.12:80 cookie SITE2 check port 81 inter 8000 backup
+
+Configuration for SD2
+---------------------
+
+ listen 10.2.1.1:80
+ mode http
+ balance roundrobin
+ redispatch
+ cookie SITE insert indirect
+ option httpchk HEAD / HTTP/1.0
+ option forwardfor
+ server S2L1 10.2.1.11:80 cookie SITE2 check port 81 inter 4000
+ server S2L2 10.2.1.12:80 cookie SITE2 check port 81 inter 4000
+ server S1L1 10.1.1.11:80 cookie SITE1 check port 81 inter 8000 backup
+ server S1L2 10.1.1.12:80 cookie SITE1 check port 81 inter 8000 backup
+
+
+5.4.4 Local load-balancers S1L1, S1L2, S2L1, S2L2
+-------------------------------------------------
+Please first note that because SD1 and SD2 use the same cookie for both
+servers on a same site, the second load-balancer of each site will only
+receive load-balanced requests, but as soon as the SITE cookie will be
+set, only the first LB will receive the requests because it will be the
+first one to match the cookie.
+
+The load-balancers will spread the load across 4 local web servers, and
+use the JSESSIONID provided by the application to provide server persistence
+using the new 'prefix' method. Soft-stop will also be implemented as described
+in section 4 above. Moreover, these proxies will provide their own maintenance
+soft-stop. Port 80 will be used for application traffic, while port 81 will
+only be used for health-checks and locally rerouted to port 80. A grace time
+will be specified to service on port 80, but not on port 81. This way, a soft
+kill (kill -USR1) on the proxy will only kill the health-check forwarder so
+that the site director knows it must not use this load-balancer anymore. But
+the service will still work for 20 seconds and as long as there are established
+sessions.
+
+These proxies will also be the only ones to disable HTTP keep-alive in the
+chain, because it is enough to do it at one place, and it's necessary to do
+it with 'prefix' cookies.
+
+Configuration for S1L1/S1L2
+---------------------------
+
+ listen 10.1.1.11:80 # 10.1.1.12:80 for S1L2
+ grace 20000 # don't kill us until 20 seconds have elapsed
+ mode http
+ balance roundrobin
+ cookie JSESSIONID prefix
+ option httpclose
+ option forwardfor
+ option httpchk HEAD / HTTP/1.0
+ server W11 10.1.2.1:80 cookie W11 check port 81 inter 2000
+ server W12 10.1.2.2:80 cookie W12 check port 81 inter 2000
+ server W13 10.1.2.3:80 cookie W13 check port 81 inter 2000
+ server W14 10.1.2.4:80 cookie W14 check port 81 inter 2000
+
+ server B11 10.1.2.1:80 cookie W11 check port 80 inter 4000 backup
+ server B12 10.1.2.2:80 cookie W12 check port 80 inter 4000 backup
+ server B13 10.1.2.3:80 cookie W13 check port 80 inter 4000 backup
+ server B14 10.1.2.4:80 cookie W14 check port 80 inter 4000 backup
+
+ listen 10.1.1.11:81 # 10.1.1.12:81 for S1L2
+ mode tcp
+ dispatch 10.1.1.11:80 # 10.1.1.12:80 for S1L2
+
+
+Configuration for S2L1/S2L2
+---------------------------
+
+ listen 10.2.1.11:80 # 10.2.1.12:80 for S2L2
+ grace 20000 # don't kill us until 20 seconds have elapsed
+ mode http
+ balance roundrobin
+ cookie JSESSIONID prefix
+ option httpclose
+ option forwardfor
+ option httpchk HEAD / HTTP/1.0
+ server W21 10.2.2.1:80 cookie W21 check port 81 inter 2000
+ server W22 10.2.2.2:80 cookie W22 check port 81 inter 2000
+ server W23 10.2.2.3:80 cookie W23 check port 81 inter 2000
+ server W24 10.2.2.4:80 cookie W24 check port 81 inter 2000
+
+ server B21 10.2.2.1:80 cookie W21 check port 80 inter 4000 backup
+ server B22 10.2.2.2:80 cookie W22 check port 80 inter 4000 backup
+ server B23 10.2.2.3:80 cookie W23 check port 80 inter 4000 backup
+ server B24 10.2.2.4:80 cookie W24 check port 80 inter 4000 backup
+
+ listen 10.2.1.11:81 # 10.2.1.12:81 for S2L2
+ mode tcp
+ dispatch 10.2.1.11:80 # 10.2.1.12:80 for S2L2
+
+
+5.5 Comments
+------------
+Since each site director sets a cookie identifying the site, remote office
+users will have their office proxies direct them to the right site and stick
+to this site as long as the user still uses the application and the site is
+available. Users on production sites will be directed to the right site by the
+site directors depending on the SITE cookie.
+
+If the WAN link dies on a production site, the remote office users will not
+see their site anymore, so they will redirect the traffic to the second site.
+If there are dedicated inter-site links as on the diagram above, the second
+SD will see the cookie and still be able to reach the original site. For
+example :
+
+Office 1 user sends the following to OP1 :
+ GET / HTTP/1.0
+ Cookie: SITE=SITE1; JSESSIONID=W14~123;
+
+OP1 cannot reach site 1 because its external router is dead. So the SD1 server
+is seen as dead, and OP1 will then forward the request to SD2 on site 2,
+regardless of the SITE cookie.
+
+SD2 on site 2 receives a SITE cookie containing "SITE1". Fortunately, it
+can reach Site 1's load balancers S1L1 and S1L2. So it forwards the request
+so S1L1 (the first one with the same cookie).
+
+S1L1 (on site 1) finds "W14" in the JSESSIONID cookie, so it can forward the
+request to the right server, and the user session will continue to work. Once
+the Site 1's WAN link comes back, OP1 will see SD1 again, and will not route
+through SITE 2 anymore.
+
+However, when a new user on Office 1 connects to the application during a
+site 1 failure, it does not contain any cookie. Since OP1 does not see SD1
+because of the network failure, it will direct the request to SD2 on site 2,
+which will by default direct the traffic to the local load-balancers, S2L1 and
+S2L2. So only initial users will load the inter-site link, not the new ones.
+
+
+===================
+6. Source balancing
+===================
+
+Sometimes it may reveal useful to access servers from a pool of IP addresses
+instead of only one or two. Some equipment (NAT firewalls, load-balancers)
+are sensible to source address, and often need many sources to distribute the
+load evenly amongst their internal hash buckets.
+
+To do this, you simply have to use several times the same server with a
+different source. Example :
+
+ listen 0.0.0.0:80
+ mode tcp
+ balance roundrobin
+ server from1to1 10.1.1.1:80 source 10.1.2.1
+ server from2to1 10.1.1.1:80 source 10.1.2.2
+ server from3to1 10.1.1.1:80 source 10.1.2.3
+ server from4to1 10.1.1.1:80 source 10.1.2.4
+ server from5to1 10.1.1.1:80 source 10.1.2.5
+ server from6to1 10.1.1.1:80 source 10.1.2.6
+ server from7to1 10.1.1.1:80 source 10.1.2.7
+ server from8to1 10.1.1.1:80 source 10.1.2.8
+
+
+=============================================
+7. Managing high loads on application servers
+=============================================
+
+One of the roles often expected from a load balancer is to mitigate the load on
+the servers during traffic peaks. More and more often, we see heavy frameworks
+used to deliver flexible and evolutive web designs, at the cost of high loads
+on the servers, or very low concurrency. Sometimes, response times are also
+rather high. People developing web sites relying on such frameworks very often
+look for a load balancer which is able to distribute the load in the most
+evenly fashion and which will be nice with the servers.
+
+There is a powerful feature in haproxy which achieves exactly this : request
+queueing associated with concurrent connections limit.
+
+Let's say you have an application server which supports at most 20 concurrent
+requests. You have 3 servers, so you can accept up to 60 concurrent HTTP
+connections, which often means 30 concurrent users in case of keep-alive (2
+persistent connections per user).
+
+Even if you disable keep-alive, if the server takes a long time to respond,
+you still have a high risk of multiple users clicking at the same time and
+having their requests unserved because of server saturation. To work around
+the problem, you increase the concurrent connection limit on the servers,
+but their performance stalls under higher loads.
+
+The solution is to limit the number of connections between the clients and the
+servers. You set haproxy to limit the number of connections on a per-server
+basis, and you let all the users you want connect to it. It will then fill all
+the servers up to the configured connection limit, and will put the remaining
+connections in a queue, waiting for a connection to be released on a server.
+
+This ensures five essential principles :
+
+ - all clients can be served whatever their number without crashing the
+ servers, the only impact it that the response time can be delayed.
+
+ - the servers can be used at full throttle without the risk of stalling,
+ and fine tuning can lead to optimal performance.
+
+ - response times can be reduced by making the servers work below the
+ congestion point, effectively leading to shorter response times even
+ under moderate loads.
+
+ - no domino effect when a server goes down or starts up. Requests will be
+ queued more or less, always respecting servers limits.
+
+ - it's easy to achieve high performance even on memory-limited hardware.
+ Indeed, heavy frameworks often consume huge amounts of RAM and not always
+ all the CPU available. In case of wrong sizing, reducing the number of
+ concurrent connections will protect against memory shortages while still
+ ensuring optimal CPU usage.
+
+
+Example :
+---------
+
+HAProxy is installed in front of an application servers farm. It will limit
+the concurrent connections to 4 per server (one thread per CPU), thus ensuring
+very fast response times.
+
+
+ 192.168.1.1 192.168.1.11-192.168.1.13 192.168.1.2
+ -------+-------------+-----+-----+------------+----
+ | | | | _|_db
+ +--+--+ +-+-+ +-+-+ +-+-+ (___)
+ | LB1 | | A | | B | | C | (___)
+ +-----+ +---+ +---+ +---+ (___)
+ haproxy 3 application servers
+ with heavy frameworks
+
+
+Config on haproxy (LB1) :
+-------------------------
+
+ listen appfarm 192.168.1.1:80
+ mode http
+ maxconn 10000
+ option httpclose
+ option forwardfor
+ balance roundrobin
+ cookie SERVERID insert indirect
+ option httpchk HEAD /index.html HTTP/1.0
+ server railsA 192.168.1.11:80 cookie A maxconn 4 check
+ server railsB 192.168.1.12:80 cookie B maxconn 4 check
+ server railsC 192.168.1.13:80 cookie C maxconn 4 check
+ contimeout 60000
+
+
+Description :
+-------------
+The proxy listens on IP 192.168.1.1, port 80, and expects HTTP requests. It
+can accept up to 10000 concurrent connections on this socket. It follows the
+roundrobin algorithm to assign servers to connections as long as servers are
+not saturated.
+
+It allows up to 4 concurrent connections per server, and will queue the
+requests above this value. The "contimeout" parameter is used to set the
+maximum time a connection may take to establish on a server, but here it
+is also used to set the maximum time a connection may stay unserved in the
+queue (1 minute here).
+
+If the servers can each process 4 requests in 10 ms on average, then at 3000
+connections, response times will be delayed by at most :
+
+ 3000 / 3 servers / 4 conns * 10 ms = 2.5 seconds
+
+Which is not that dramatic considering the huge number of users for such a low
+number of servers.
+
+When connection queues fill up and application servers are starving, response
+times will grow and users might abort by clicking on the "Stop" button. It is
+very undesirable to send aborted requests to servers, because they will eat
+CPU cycles for nothing.
+
+An option has been added to handle this specific case : "option abortonclose".
+By specifying it, you tell haproxy that if an input channel is closed on the
+client side AND the request is still waiting in the queue, then it is highly
+likely that the user has stopped, so we remove the request from the queue
+before it will get served.
+
+
+Managing unfair response times
+------------------------------
+
+Sometimes, the application server will be very slow for some requests (eg:
+login page) and faster for other requests. This may cause excessive queueing
+of expectedly fast requests when all threads on the server are blocked on a
+request to the database. Then the only solution is to increase the number of
+concurrent connections, so that the server can handle a large average number
+of slow connections with threads left to handle faster connections.
+
+But as we have seen, increasing the number of connections on the servers can
+be detrimental to performance (eg: Apache processes fighting for the accept()
+lock). To improve this situation, the "minconn" parameter has been introduced.
+When it is set, the maximum connection concurrency on the server will be bound
+by this value, and the limit will increase with the number of clients waiting
+in queue, till the clients connected to haproxy reach the proxy's maxconn, in
+which case the connections per server will reach the server's maxconn. It means
+that during low-to-medium loads, the minconn will be applied, and during surges
+the maxconn will be applied. It ensures both optimal response times under
+normal loads, and availability under very high loads.
+
+Example :
+---------
+
+ listen appfarm 192.168.1.1:80
+ mode http
+ maxconn 10000
+ option httpclose
+ option abortonclose
+ option forwardfor
+ balance roundrobin
+ # The servers will get 4 concurrent connections under low
+ # loads, and 12 when there will be 10000 clients.
+ server railsA 192.168.1.11:80 minconn 4 maxconn 12 check
+ server railsB 192.168.1.12:80 minconn 4 maxconn 12 check
+ server railsC 192.168.1.13:80 minconn 4 maxconn 12 check
+ contimeout 60000
+
+
diff --git a/doc/coding-style.txt b/doc/coding-style.txt
new file mode 100644
index 0000000..02a55f5
--- /dev/null
+++ b/doc/coding-style.txt
@@ -0,0 +1,1566 @@
+2020/07/07 - HAProxy coding style - Willy Tarreau <w@1wt.eu>
+------------------------------------------------------------
+
+A number of contributors are often embarrassed with coding style issues, they
+don't always know if they're doing it right, especially since the coding style
+has elvoved along the years. What is explained here is not necessarily what is
+applied in the code, but new code should as much as possible conform to this
+style. Coding style fixes happen when code is replaced. It is useless to send
+patches to fix coding style only, they will be rejected, unless they belong to
+a patch series which needs these fixes prior to get code changes. Also, please
+avoid fixing coding style in the same patches as functional changes, they make
+code review harder.
+
+A good way to quickly validate your patch before submitting it is to pass it
+through the Linux kernel's checkpatch.pl utility which can be downloaded here :
+
+ http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/plain/scripts/checkpatch.pl
+
+Running it with the following options relaxes its checks to accommodate to the
+extra degree of freedom that is tolerated in HAProxy's coding style compared to
+the stricter style used in the kernel :
+
+ checkpatch.pl -q --max-line-length=160 --no-tree --no-signoff \
+ --ignore=LEADING_SPACE,CODE_INDENT,DEEP_INDENTATION \
+ --ignore=ELSE_AFTER_BRACE < patch
+
+You can take its output as hints instead of strict rules, but in general its
+output will be accurate and it may even spot some real bugs.
+
+When modifying a file, you must accept the terms of the license of this file
+which is recalled at the top of the file, or is explained in the LICENSE file,
+or if not stated, defaults to LGPL version 2.1 or later for files in the
+'include' directory, and GPL version 2 or later for all other files.
+
+When adding a new file, you must add a copyright banner at the top of the
+file with your real name, e-mail address and a reminder of the license.
+Contributions under incompatible licenses or too restrictive licenses might
+get rejected. If in doubt, please apply the principle above for existing files.
+
+All code examples below will intentionally be prefixed with " | " to mark
+where the code aligns with the first column, and tabs in this document will be
+represented as a series of 8 spaces so that it displays the same everywhere.
+
+
+1) Indentation and alignment
+----------------------------
+
+1.1) Indentation
+----------------
+
+Indentation and alignment are two completely different things that people often
+get wrong. Indentation is used to mark a sub-level in the code. A sub-level
+means that a block is executed in the context of another block (eg: a function
+or a condition) :
+
+ | main(int argc, char **argv)
+ | {
+ | int i;
+ |
+ | if (argc < 2)
+ | exit(1);
+ | }
+
+In the example above, the code belongs to the main() function and the exit()
+call belongs to the if statement. Indentation is made with tabs (\t, ASCII 9),
+which allows any developer to configure their preferred editor to use their
+own tab size and to still get the text properly indented. Exactly one tab is
+used per sub-level. Tabs may only appear at the beginning of a line or after
+another tab. It is illegal to put a tab after some text, as it mangles displays
+in a different manner for different users (particularly when used to align
+comments or values after a #define). If you're tempted to put a tab after some
+text, then you're doing it wrong and you need alignment instead (see below).
+
+Note that there are places where the code was not properly indented in the
+past. In order to view it correctly, you may have to set your tab size to 8
+characters.
+
+
+1.2) Alignment
+--------------
+
+Alignment is used to continue a line in a way to makes things easier to group
+together. By definition, alignment is character-based, so it uses spaces. Tabs
+would not work because for one tab there would not be as many characters on all
+displays. For instance, the arguments in a function declaration may be broken
+into multiple lines using alignment spaces :
+
+ | int http_header_match2(const char *hdr, const char *end,
+ | const char *name, int len)
+ | {
+ | ...
+ | }
+
+In this example, the "const char *name" part is aligned with the first
+character of the group it belongs to (list of function arguments). Placing it
+here makes it obvious that it's one of the function's arguments. Multiple lines
+are easy to handle this way. This is very common with long conditions too :
+
+ | if ((len < eol - sol) &&
+ | (sol[len] == ':') &&
+ | (strncasecmp(sol, name, len) == 0)) {
+ | ctx->del = len;
+ | }
+
+If we take again the example above marking tabs with "[-Tabs-]" and spaces
+with "#", we get this :
+
+ | [-Tabs-]if ((len < eol - sol) &&
+ | [-Tabs-]####(sol[len] == ':') &&
+ | [-Tabs-]####(strncasecmp(sol, name, len) == 0)) {
+ | [-Tabs-][-Tabs-]ctx->del = len;
+ | [-Tabs-]}
+
+It is worth noting that some editors tend to confuse indentations and alignment.
+Emacs is notoriously known for this brokenness, and is responsible for almost
+all of the alignment mess. The reason is that Emacs only counts spaces, tries
+to fill as many as possible with tabs and completes with spaces. Once you know
+it, you just have to be careful, as alignment is not used much, so generally it
+is just a matter of replacing the last tab with 8 spaces when this happens.
+
+Indentation should be used everywhere there is a block or an opening brace. It
+is not possible to have two consecutive closing braces on the same column, it
+means that the innermost was not indented.
+
+Right :
+
+ | main(int argc, char **argv)
+ | {
+ | if (argc > 1) {
+ | printf("Hello\n");
+ | }
+ | exit(0);
+ | }
+
+Wrong :
+
+ | main(int argc, char **argv)
+ | {
+ | if (argc > 1) {
+ | printf("Hello\n");
+ | }
+ | exit(0);
+ | }
+
+A special case applies to switch/case statements. Due to my editor's settings,
+I've been used to align "case" with "switch" and to find it somewhat logical
+since each of the "case" statements opens a sublevel belonging to the "switch"
+statement. But indenting "case" after "switch" is accepted too. However in any
+case, whatever follows the "case" statement must be indented, whether or not it
+contains braces :
+
+ | switch (*arg) {
+ | case 'A': {
+ | int i;
+ | for (i = 0; i < 10; i++)
+ | printf("Please stop pressing 'A'!\n");
+ | break;
+ | }
+ | case 'B':
+ | printf("You pressed 'B'\n");
+ | break;
+ | case 'C':
+ | case 'D':
+ | printf("You pressed 'C' or 'D'\n");
+ | break;
+ | default:
+ | printf("I don't know what you pressed\n");
+ | }
+
+
+2) Braces
+---------
+
+Braces are used to delimit multiple-instruction blocks. In general it is
+preferred to avoid braces around single-instruction blocks as it reduces the
+number of lines :
+
+Right :
+
+ | if (argc >= 2)
+ | exit(0);
+
+Wrong :
+
+ | if (argc >= 2) {
+ | exit(0);
+ | }
+
+But it is not that strict, it really depends on the context. It happens from
+time to time that single-instruction blocks are enclosed within braces because
+it makes the code more symmetrical, or more readable. Example :
+
+ | if (argc < 2) {
+ | printf("Missing argument\n");
+ | exit(1);
+ | } else {
+ | exit(0);
+ | }
+
+Braces are always needed to declare a function. A function's opening brace must
+be placed at the beginning of the next line :
+
+Right :
+
+ | int main(int argc, char **argv)
+ | {
+ | exit(0);
+ | }
+
+Wrong :
+
+ | int main(int argc, char **argv) {
+ | exit(0);
+ | }
+
+Note that a large portion of the code still does not conforms to this rule, as
+it took years to get all authors to adapt to this more common standard which
+is now preferred, as it avoids visual confusion when function declarations are
+broken on multiple lines :
+
+Right :
+
+ | int foo(const char *hdr, const char *end,
+ | const char *name, const char *err,
+ | int len)
+ | {
+ | int i;
+
+Wrong :
+
+ | int foo(const char *hdr, const char *end,
+ | const char *name, const char *err,
+ | int len) {
+ | int i;
+
+Braces should always be used where there might be an ambiguity with the code
+later. The most common example is the stacked "if" statement where an "else"
+may be added later at the wrong place breaking the code, but it also happens
+with comments or long arguments in function calls. In general, if a block is
+more than one line long, it should use braces.
+
+Dangerous code waiting of a victim :
+
+ | if (argc < 2)
+ | /* ret must not be negative here */
+ | if (ret < 0)
+ | return -1;
+
+Wrong change :
+
+ | if (argc < 2)
+ | /* ret must not be negative here */
+ | if (ret < 0)
+ | return -1;
+ | else
+ | return 0;
+
+It will do this instead of what your eye seems to tell you :
+
+ | if (argc < 2)
+ | /* ret must not be negative here */
+ | if (ret < 0)
+ | return -1;
+ | else
+ | return 0;
+
+Right :
+
+ | if (argc < 2) {
+ | /* ret must not be negative here */
+ | if (ret < 0)
+ | return -1;
+ | }
+ | else
+ | return 0;
+
+Similarly dangerous example :
+
+ | if (ret < 0)
+ | /* ret must not be negative here */
+ | complain();
+ | init();
+
+Wrong change to silent the annoying message :
+
+ | if (ret < 0)
+ | /* ret must not be negative here */
+ | //complain();
+ | init();
+
+... which in fact means :
+
+ | if (ret < 0)
+ | init();
+
+
+3) Breaking lines
+-----------------
+
+There is no strict rule for line breaking. Some files try to stick to the 80
+column limit, but given that various people use various tab sizes, it does not
+make much sense. Also, code is sometimes easier to read with less lines, as it
+represents less surface on the screen (since each new line adds its tabs and
+spaces). The rule is to stick to the average line length of other lines. If you
+are working in a file which fits in 80 columns, try to keep this goal in mind.
+If you're in a function with 120-chars lines, there is no reason to add many
+short lines, so you can make longer lines.
+
+In general, opening a new block should lead to a new line. Similarly, multiple
+instructions should be avoided on the same line. But some constructs make it
+more readable when those are perfectly aligned :
+
+A copy-paste bug in the following construct will be easier to spot :
+
+ | if (omult % idiv == 0) { omult /= idiv; idiv = 1; }
+ | if (idiv % omult == 0) { idiv /= omult; omult = 1; }
+ | if (imult % odiv == 0) { imult /= odiv; odiv = 1; }
+ | if (odiv % imult == 0) { odiv /= imult; imult = 1; }
+
+than in this one :
+
+ | if (omult % idiv == 0) {
+ | omult /= idiv;
+ | idiv = 1;
+ | }
+ | if (idiv % omult == 0) {
+ | idiv /= omult;
+ | omult = 1;
+ | }
+ | if (imult % odiv == 0) {
+ | imult /= odiv;
+ | odiv = 1;
+ | }
+ | if (odiv % imult == 0) {
+ | odiv /= imult;
+ | imult = 1;
+ | }
+
+What is important is not to mix styles. For instance there is nothing wrong
+with having many one-line "case" statements as long as most of them are this
+short like below :
+
+ | switch (*arg) {
+ | case 'A': ret = 1; break;
+ | case 'B': ret = 2; break;
+ | case 'C': ret = 4; break;
+ | case 'D': ret = 8; break;
+ | default : ret = 0; break;
+ | }
+
+Otherwise, prefer to have the "case" statement on its own line as in the
+example in section 1.2 about alignment. In any case, avoid to stack multiple
+control statements on the same line, so that it will never be the needed to
+add two tab levels at once :
+
+Right :
+
+ | switch (*arg) {
+ | case 'A':
+ | if (ret < 0)
+ | ret = 1;
+ | break;
+ | default : ret = 0; break;
+ | }
+
+Wrong :
+
+ | switch (*arg) {
+ | case 'A': if (ret < 0)
+ | ret = 1;
+ | break;
+ | default : ret = 0; break;
+ | }
+
+Right :
+
+ | if (argc < 2)
+ | if (ret < 0)
+ | return -1;
+
+or Right :
+
+ | if (argc < 2)
+ | if (ret < 0) return -1;
+
+but Wrong :
+
+ | if (argc < 2) if (ret < 0) return -1;
+
+
+When complex conditions or expressions are broken into multiple lines, please
+do ensure that alignment is perfectly appropriate, and group all main operators
+on the same side (which you're free to choose as long as it does not change for
+every block. Putting binary operators on the right side is preferred as it does
+not mangle with alignment but various people have their preferences.
+
+Right :
+
+ | if ((txn->flags & TX_NOT_FIRST) &&
+ | ((req->flags & BF_FULL) ||
+ | req->r < req->lr ||
+ | req->r > req->data + req->size - global.tune.maxrewrite)) {
+ | return 0;
+ | }
+
+Right :
+
+ | if ((txn->flags & TX_NOT_FIRST)
+ | && ((req->flags & BF_FULL)
+ | || req->r < req->lr
+ | || req->r > req->data + req->size - global.tune.maxrewrite)) {
+ | return 0;
+ | }
+
+Wrong :
+
+ | if ((txn->flags & TX_NOT_FIRST) &&
+ | ((req->flags & BF_FULL) ||
+ | req->r < req->lr
+ | || req->r > req->data + req->size - global.tune.maxrewrite)) {
+ | return 0;
+ | }
+
+If it makes the result more readable, parenthesis may even be closed on their
+own line in order to align with the opening one. Note that should normally not
+be needed because such code would be too complex to be digged into.
+
+The "else" statement may either be merged with the closing "if" brace or lie on
+its own line. The later is preferred but it adds one extra line to each control
+block which is annoying in short ones. However, if the "else" is followed by an
+"if", then it should really be on its own line and the rest of the if/else
+blocks must follow the same style.
+
+Right :
+
+ | if (a < b) {
+ | return a;
+ | }
+ | else {
+ | return b;
+ | }
+
+Right :
+
+ | if (a < b) {
+ | return a;
+ | } else {
+ | return b;
+ | }
+
+Right :
+
+ | if (a < b) {
+ | return a;
+ | }
+ | else if (a != b) {
+ | return b;
+ | }
+ | else {
+ | return 0;
+ | }
+
+Wrong :
+
+ | if (a < b) {
+ | return a;
+ | } else if (a != b) {
+ | return b;
+ | } else {
+ | return 0;
+ | }
+
+Wrong :
+
+ | if (a < b) {
+ | return a;
+ | }
+ | else if (a != b) {
+ | return b;
+ | } else {
+ | return 0;
+ | }
+
+
+4) Spacing
+----------
+
+Correctly spacing code is very important. When you have to spot a bug at 3am,
+you need it to be clear. When you expect other people to review your code, you
+want it to be clear and don't want them to get nervous when trying to find what
+you did.
+
+Always place spaces around all binary or ternary operators, commas, as well as
+after semi-colons and opening braces if the line continues :
+
+Right :
+
+ | int ret = 0;
+ | /* if (x >> 4) { x >>= 4; ret += 4; } */
+ | ret += (x >> 4) ? (x >>= 4, 4) : 0;
+ | val = ret + ((0xFFFFAA50U >> (x << 1)) & 3) + 1;
+
+Wrong :
+
+ | int ret=0;
+ | /* if (x>>4) {x>>=4;ret+=4;} */
+ | ret+=(x>>4)?(x>>=4,4):0;
+ | val=ret+((0xFFFFAA50U>>(x<<1))&3)+1;
+
+Never place spaces after unary operators (&, *, -, !, ~, ++, --) nor cast, as
+they might be confused with they binary counterpart, nor before commas or
+semicolons :
+
+Right :
+
+ | bit = !!(~len++ ^ -(unsigned char)*x);
+
+Wrong :
+
+ | bit = ! ! (~len++ ^ - (unsigned char) * x) ;
+
+Note that "sizeof" is a unary operator which is sometimes considered as a
+language keyword, but in no case it is a function. It does not require
+parenthesis so it is sometimes followed by spaces and sometimes not when
+there are no parenthesis. Most people do not really care as long as what
+is written is unambiguous.
+
+Braces opening a block must be preceded by one space unless the brace is
+placed on the first column :
+
+Right :
+
+ | if (argc < 2) {
+ | }
+
+Wrong :
+
+ | if (argc < 2){
+ | }
+
+Do not add unneeded spaces inside parenthesis, they just make the code less
+readable.
+
+Right :
+
+ | if (x < 4 && (!y || !z))
+ | break;
+
+Wrong :
+
+ | if ( x < 4 && ( !y || !z ) )
+ | break;
+
+Language keywords must all be followed by a space. This is true for control
+statements (do, for, while, if, else, return, switch, case), and for types
+(int, char, unsigned). As an exception, the last type in a cast does not take
+a space before the closing parenthesis). The "default" statement in a "switch"
+construct is generally just followed by the colon. However the colon after a
+"case" or "default" statement must be followed by a space.
+
+Right :
+
+ | if (nbargs < 2) {
+ | printf("Missing arg at %c\n", *(char *)ptr);
+ | for (i = 0; i < 10; i++) beep();
+ | return 0;
+ | }
+ | switch (*arg) {
+
+Wrong :
+
+ | if(nbargs < 2){
+ | printf("Missing arg at %c\n", *(char*)ptr);
+ | for(i = 0; i < 10; i++)beep();
+ | return 0;
+ | }
+ | switch(*arg) {
+
+Function calls are different, the opening parenthesis is always coupled to the
+function name without any space. But spaces are still needed after commas :
+
+Right :
+
+ | if (!init(argc, argv))
+ | exit(1);
+
+Wrong :
+
+ | if (!init (argc,argv))
+ | exit(1);
+
+
+5) Excess or lack of parenthesis
+--------------------------------
+
+Sometimes there are too many parenthesis in some formulas, sometimes there are
+too few. There are a few rules of thumb for this. The first one is to respect
+the compiler's advice. If it emits a warning and asks for more parenthesis to
+avoid confusion, follow the advice at least to shut the warning. For instance,
+the code below is quite ambiguous due to its alignment :
+
+ | if (var1 < 2 || var2 < 2 &&
+ | var3 != var4) {
+ | /* fail */
+ | return -3;
+ | }
+
+Note that this code does :
+
+ | if (var1 < 2 || (var2 < 2 && var3 != var4)) {
+ | /* fail */
+ | return -3;
+ | }
+
+But maybe the author meant :
+
+ | if ((var1 < 2 || var2 < 2) && var3 != var4) {
+ | /* fail */
+ | return -3;
+ | }
+
+A second rule to put parenthesis is that people don't always know operators
+precedence too well. Most often they have no issue with operators of the same
+category (eg: booleans, integers, bit manipulation, assignment) but once these
+operators are mixed, it causes them all sort of issues. In this case, it is
+wise to use parenthesis to avoid errors. One common error concerns the bit
+shift operators because they're used to replace multiplies and divides but
+don't have the same precedence :
+
+The expression :
+
+ | x = y * 16 + 5;
+
+becomes :
+
+ | x = y << 4 + 5;
+
+which is wrong because it is equivalent to :
+
+ | x = y << (4 + 5);
+
+while the following was desired instead :
+
+ | x = (y << 4) + 5;
+
+It is generally fine to write boolean expressions based on comparisons without
+any parenthesis. But on top of that, integer expressions and assignments should
+then be protected. For instance, there is an error in the expression below
+which should be safely rewritten :
+
+Wrong :
+
+ | if (var1 > 2 && var1 < 10 ||
+ | var1 > 2 + 256 && var2 < 10 + 256 ||
+ | var1 > 2 + 1 << 16 && var2 < 10 + 2 << 16)
+ | return 1;
+
+Right (may remove a few parenthesis depending on taste) :
+
+ | if ((var1 > 2 && var1 < 10) ||
+ | (var1 > (2 + 256) && var2 < (10 + 256)) ||
+ | (var1 > (2 + (1 << 16)) && var2 < (10 + (1 << 16))))
+ | return 1;
+
+The "return" statement is not a function, so it takes no argument. It is a
+control statement which is followed by the expression to be returned. It does
+not need to be followed by parenthesis :
+
+Wrong :
+
+ | int ret0()
+ | {
+ | return(0);
+ | }
+
+Right :
+
+ | int ret0()
+ | {
+ | return 0;
+ | }
+
+Parenthesisis are also found in type casts. Type casting should be avoided as
+much as possible, especially when it concerns pointer types. Casting a pointer
+disables the compiler's type checking and is the best way to get caught doing
+wrong things with data not the size you expect. If you need to manipulate
+multiple data types, you can use a union instead. If the union is really not
+convenient and casts are easier, then try to isolate them as much as possible,
+for instance when initializing function arguments or in another function. Not
+proceeding this way causes huge risks of not using the proper pointer without
+any notification, which is especially true during copy-pastes.
+
+Wrong :
+
+ | void *check_private_data(void *arg1, void *arg2)
+ | {
+ | char *area;
+ |
+ | if (*(int *)arg1 > 1000)
+ | return NULL;
+ | if (memcmp(*(const char *)arg2, "send(", 5) != 0))
+ | return NULL;
+ | area = malloc(*(int *)arg1);
+ | if (!area)
+ | return NULL;
+ | memcpy(area, *(const char *)arg2 + 5, *(int *)arg1);
+ | return area;
+ | }
+
+Right :
+
+ | void *check_private_data(void *arg1, void *arg2)
+ | {
+ | char *area;
+ | int len = *(int *)arg1;
+ | const char *msg = arg2;
+ |
+ | if (len > 1000)
+ | return NULL;
+ | if (memcmp(msg, "send(", 5) != 0)
+ | return NULL;
+ | area = malloc(len);
+ | if (!area)
+ | return NULL;
+ | memcpy(area, msg + 5, len);
+ | return area;
+ | }
+
+
+6) Ambiguous comparisons with zero or NULL
+------------------------------------------
+
+In C, '0' has no type, or it has the type of the variable it is assigned to.
+Comparing a variable or a return value with zero means comparing with the
+representation of zero for this variable's type. For a boolean, zero is false.
+For a pointer, zero is NULL. Very often, to make things shorter, it is fine to
+use the '!' unary operator to compare with zero, as it is shorter and easier to
+remind or understand than a plain '0'. Since the '!' operator is read "not", it
+helps read code faster when what follows it makes sense as a boolean, and it is
+often much more appropriate than a comparison with zero which makes an equal
+sign appear at an undesirable place. For instance :
+
+ | if (!isdigit(*c) && !isspace(*c))
+ | break;
+
+is easier to understand than :
+
+ | if (isdigit(*c) == 0 && isspace(*c) == 0)
+ | break;
+
+For a char this "not" operator can be reminded as "no remaining char", and the
+absence of comparison to zero implies existence of the tested entity, hence the
+simple strcpy() implementation below which automatically stops once the last
+zero is copied :
+
+ | void my_strcpy(char *d, const char *s)
+ | {
+ | while ((*d++ = *s++));
+ | }
+
+Note the double parenthesis in order to avoid the compiler telling us it looks
+like an equality test.
+
+For a string or more generally any pointer, this test may be understood as an
+existence test or a validity test, as the only pointer which will fail to
+validate equality is the NULL pointer :
+
+ | area = malloc(1000);
+ | if (!area)
+ | return -1;
+
+However sometimes it can fool the reader. For instance, strcmp() precisely is
+one of such functions whose return value can make one think the opposite due to
+its name which may be understood as "if strings compare...". Thus it is strongly
+recommended to perform an explicit comparison with zero in such a case, and it
+makes sense considering that the comparison's operator is the same that is
+wanted to compare the strings (note that current config parser lacks a lot in
+this regards) :
+
+ strcmp(a, b) == 0 <=> a == b
+ strcmp(a, b) != 0 <=> a != b
+ strcmp(a, b) < 0 <=> a < b
+ strcmp(a, b) > 0 <=> a > b
+
+Avoid this :
+
+ | if (strcmp(arg, "test"))
+ | printf("this is not a test\n");
+ |
+ | if (!strcmp(arg, "test"))
+ | printf("this is a test\n");
+
+Prefer this :
+
+ | if (strcmp(arg, "test") != 0)
+ | printf("this is not a test\n");
+ |
+ | if (strcmp(arg, "test") == 0)
+ | printf("this is a test\n");
+
+
+7) System call returns
+----------------------
+
+This is not directly a matter of coding style but more of bad habits. It is
+important to check for the correct value upon return of syscalls. The proper
+return code indicating an error is described in its man page. There is no
+reason to consider wider ranges than what is indicated. For instance, it is
+common to see such a thing :
+
+ | if ((fd = open(file, O_RDONLY)) < 0)
+ | return -1;
+
+This is wrong. The man page says that -1 is returned if an error occurred. It
+does not suggest that any other negative value will be an error. It is possible
+that a few such issues have been left in existing code. They are bugs for which
+fixes are accepted, even though they're currently harmless since open() is not
+known for returning negative values at the moment.
+
+
+8) Declaring new types, names and values
+----------------------------------------
+
+Please refrain from using "typedef" to declare new types, they only obfuscate
+the code. The reader never knows whether he's manipulating a scalar type or a
+struct. For instance it is not obvious why the following code fails to build :
+
+ | int delay_expired(timer_t exp, timer_us_t now)
+ | {
+ | return now >= exp;
+ | }
+
+With the types declared in another file this way :
+
+ | typedef unsigned int timer_t;
+ | typedef struct timeval timer_us_t;
+
+This cannot work because we're comparing a scalar with a struct, which does
+not make sense. Without a typedef, the function would have been written this
+way without any ambiguity and would not have failed :
+
+ | int delay_expired(unsigned int exp, struct timeval *now)
+ | {
+ | return now >= exp->tv_sec;
+ | }
+
+Declaring special values may be done using enums. Enums are a way to define
+structured integer values which are related to each other. They are perfectly
+suited for state machines. While the first element is always assigned the zero
+value, not everybody knows that, especially people working with multiple
+languages all the day. For this reason it is recommended to explicitly force
+the first value even if it's zero. The last element should be followed by a
+comma if it is planned that new elements might later be added, this will make
+later patches shorter. Conversely, if the last element is placed in order to
+get the number of possible values, it must not be followed by a comma and must
+be preceded by a comment :
+
+ | enum {
+ | first = 0,
+ | second,
+ | third,
+ | fourth,
+ | };
+
+
+ | enum {
+ | first = 0,
+ | second,
+ | third,
+ | fourth,
+ | /* nbvalues must always be placed last */
+ | nbvalues
+ | };
+
+Structure names should be short enough not to mangle function declarations,
+and explicit enough to avoid confusion (which is the most important thing).
+
+Wrong :
+
+ | struct request_args { /* arguments on the query string */
+ | char *name;
+ | char *value;
+ | struct misc_args *next;
+ | };
+
+Right :
+
+ | struct qs_args { /* arguments on the query string */
+ | char *name;
+ | char *value;
+ | struct qs_args *next;
+ | }
+
+
+When declaring new functions or structures, please do not use CamelCase, which
+is a style where upper and lower case are mixed in a single word. It causes a
+lot of confusion when words are composed from acronyms, because it's hard to
+stick to a rule. For instance, a function designed to generate an ISN (initial
+sequence number) for a TCP/IP connection could be called :
+
+ - generateTcpipIsn()
+ - generateTcpIpIsn()
+ - generateTcpIpISN()
+ - generateTCPIPISN()
+ etc...
+
+None is right, none is wrong, these are just preferences which might change
+along the code. Instead, please use an underscore to separate words. Lowercase
+is preferred for the words, but if acronyms are upcased it's not dramatic. The
+real advantage of this method is that it creates unambiguous levels even for
+short names.
+
+Valid examples :
+
+ - generate_tcpip_isn()
+ - generate_tcp_ip_isn()
+ - generate_TCPIP_ISN()
+ - generate_TCP_IP_ISN()
+
+Another example is easy to understand when 3 arguments are involved in naming
+the function :
+
+Wrong (naming conflict) :
+
+ | /* returns A + B * C */
+ | int mulABC(int a, int b, int c)
+ | {
+ | return a + b * c;
+ | }
+ |
+ | /* returns (A + B) * C */
+ | int mulABC(int a, int b, int c)
+ | {
+ | return (a + b) * c;
+ | }
+
+Right (unambiguous naming) :
+
+ | /* returns A + B * C */
+ | int mul_a_bc(int a, int b, int c)
+ | {
+ | return a + b * c;
+ | }
+ |
+ | /* returns (A + B) * C */
+ | int mul_ab_c(int a, int b, int c)
+ | {
+ | return (a + b) * c;
+ | }
+
+Whenever you manipulate pointers, try to declare them as "const", as it will
+save you from many accidental misuses and will only cause warnings to be
+emitted when there is a real risk. In the examples below, it is possible to
+call my_strcpy() with a const string only in the first declaration. Note that
+people who ignore "const" are often the ones who cast a lot and who complain
+from segfaults when using strtok() !
+
+Right :
+
+ | void my_strcpy(char *d, const char *s)
+ | {
+ | while ((*d++ = *s++));
+ | }
+ |
+ | void say_hello(char *dest)
+ | {
+ | my_strcpy(dest, "hello\n");
+ | }
+
+Wrong :
+
+ | void my_strcpy(char *d, char *s)
+ | {
+ | while ((*d++ = *s++));
+ | }
+ |
+ | void say_hello(char *dest)
+ | {
+ | my_strcpy(dest, "hello\n");
+ | }
+
+
+9) Getting macros right
+-----------------------
+
+It is very common for macros to do the wrong thing when used in a way their
+author did not have in mind. For this reason, macros must always be named with
+uppercase letters only. This is the only way to catch the developer's eye when
+using them, so that they double-check whether they are taking a risk or not. First,
+macros must never ever be terminated by a semi-colon, or they will close the
+wrong block once in a while. For instance, the following will cause a build
+error before the "else" due to the double semi-colon :
+
+Wrong :
+
+ | #define WARN printf("warning\n");
+ | ...
+ | if (a < 0)
+ | WARN;
+ | else
+ | a--;
+
+Right :
+
+ | #define WARN printf("warning\n")
+
+If multiple instructions are needed, then use a do { } while (0) block, which
+is the only construct which respects *exactly* the semantics of a single
+instruction :
+
+ | #define WARN do { printf("warning\n"); log("warning\n"); } while (0)
+ | ...
+ |
+ | if (a < 0)
+ | WARN;
+ | else
+ | a--;
+
+Second, do not put unprotected control statements in macros, they will
+definitely cause bugs :
+
+Wrong :
+
+ | #define WARN if (verbose) printf("warning\n")
+ | ...
+ | if (a < 0)
+ | WARN;
+ | else
+ | a--;
+
+Which is equivalent to the undesired form below :
+
+ | if (a < 0)
+ | if (verbose)
+ | printf("warning\n");
+ | else
+ | a--;
+
+Right way to do it :
+
+ | #define WARN do { if (verbose) printf("warning\n"); } while (0)
+ | ...
+ | if (a < 0)
+ | WARN;
+ | else
+ | a--;
+
+Which is equivalent to :
+
+ | if (a < 0)
+ | do { if (verbose) printf("warning\n"); } while (0);
+ | else
+ | a--;
+
+Macro parameters must always be surrounded by parenthesis, and must never be
+duplicated in the same macro unless explicitly stated. Also, macros must not be
+defined with operators without surrounding parenthesis. The MIN/MAX macros are
+a pretty common example of multiple misuses, but this happens as early as when
+using bit masks. Most often, in case of any doubt, try to use inline functions
+instead.
+
+Wrong :
+
+ | #define MIN(a, b) a < b ? a : b
+ |
+ | /* returns 2 * min(a,b) + 1 */
+ | int double_min_p1(int a, int b)
+ | {
+ | return 2 * MIN(a, b) + 1;
+ | }
+
+What this will do :
+
+ | int double_min_p1(int a, int b)
+ | {
+ | return 2 * a < b ? a : b + 1;
+ | }
+
+Which is equivalent to :
+
+ | int double_min_p1(int a, int b)
+ | {
+ | return (2 * a) < b ? a : (b + 1);
+ | }
+
+The first thing to fix is to surround the macro definition with parenthesis to
+avoid this mistake :
+
+ | #define MIN(a, b) (a < b ? a : b)
+
+But this is still not enough, as can be seen in this example :
+
+ | /* compares either a or b with c */
+ | int min_ab_c(int a, int b, int c)
+ | {
+ | return MIN(a ? a : b, c);
+ | }
+
+Which is equivalent to :
+
+ | int min_ab_c(int a, int b, int c)
+ | {
+ | return (a ? a : b < c ? a ? a : b : c);
+ | }
+
+Which in turn means a totally different thing due to precedence :
+
+ | int min_ab_c(int a, int b, int c)
+ | {
+ | return (a ? a : ((b < c) ? (a ? a : b) : c));
+ | }
+
+This can be fixed by surrounding *each* argument in the macro with parenthesis:
+
+ | #define MIN(a, b) ((a) < (b) ? (a) : (b))
+
+But this is still not enough, as can be seen in this example :
+
+ | int min_ap1_b(int a, int b)
+ | {
+ | return MIN(++a, b);
+ | }
+
+Which is equivalent to :
+
+ | int min_ap1_b(int a, int b)
+ | {
+ | return ((++a) < (b) ? (++a) : (b));
+ | }
+
+Again, this is wrong because "a" is incremented twice if below b. The only way
+to fix this is to use a compound statement and to assign each argument exactly
+once to a local variable of the same type :
+
+ | #define MIN(a, b) ({ typeof(a) __a = (a); typeof(b) __b = (b); \
+ | ((__a) < (__b) ? (__a) : (__b)); \
+ | })
+
+At this point, using static inline functions is much cleaner if a single type
+is to be used :
+
+ | static inline int min(int a, int b)
+ | {
+ | return a < b ? a : b;
+ | }
+
+
+10) Includes
+------------
+
+Includes are as much as possible listed in alphabetically ordered groups :
+ - the includes more or less system-specific (sys/*, netinet/*, ...)
+ - the libc-standard includes (those without any path component)
+ - includes from the local "import" subdirectory
+ - includes from the local "haproxy" subdirectory
+
+Each section is just visually delimited from the other ones using an empty
+line. The two first ones above may be merged into a single section depending on
+developer's preference. Please do not copy-paste include statements from other
+files. Having too many includes significantly increases build time and makes it
+hard to find which ones are needed later. Just include what you need and if
+possible in alphabetical order so that when something is missing, it becomes
+obvious where to look for it and where to add it.
+
+All files should include <haproxy/api.h> because this is where build options
+are prepared.
+
+HAProxy header files are split in two, those exporting the types only (named
+with a trailing "-t") and those exporting variables, functions and inline
+functions. Types, structures, enums and #defines must go into the types files
+which are the only ones that may be included by othertype files. Function
+prototypes and inlined functions must go into the main files. This split is
+because of inlined functions which cross-reference types from other files,
+which cause a chicken-and-egg problem if the functions and types are declared
+at the same place.
+
+Include files must be protected against multiple inclusion using the common
+#ifndef/#define/#endif trick with a tag derived from the include file and its
+location.
+
+
+11) Comments
+------------
+
+Comments are preferably of the standard 'C' form using /* */. The C++ form "//"
+are tolerated for very short comments (eg: a word or two) but should be avoided
+as much as possible. Multi-line comments are made with each intermediate line
+starting with a star aligned with the first one, as in this example :
+
+ | /*
+ | * This is a multi-line
+ | * comment.
+ | */
+
+If multiple code lines need a short comment, try to align them so that you can
+have multi-line sentences. This is rarely needed, only for really complex
+constructs.
+
+Do not tell what you're doing in comments, but explain why you're doing it if
+it seems not to be obvious. Also *do* indicate at the top of function what they
+accept and what they don't accept. For instance, strcpy() only accepts output
+buffers at least as large as the input buffer, and does not support any NULL
+pointer. There is nothing wrong with that if the caller knows it.
+
+Wrong use of comments :
+
+ | int flsnz8(unsigned int x)
+ | {
+ | int ret = 0; /* initialize ret */
+ | if (x >> 4) { x >>= 4; ret += 4; } /* add 4 to ret if needed */
+ | return ret + ((0xFFFFAA50U >> (x << 1)) & 3) + 1; /* add ??? */
+ | }
+ | ...
+ | bit = ~len + (skip << 3) + 9; /* update bit */
+
+Right use of comments :
+
+ | /* This function returns the position of the highest bit set in the lowest
+ | * byte of <x>, between 0 and 7. It only works if <x> is non-null. It uses
+ | * a 32-bit value as a lookup table to return one of 4 values for the
+ | * highest 16 possible 4-bit values.
+ | */
+ | int flsnz8(unsigned int x)
+ | {
+ | int ret = 0;
+ | if (x >> 4) { x >>= 4; ret += 4; }
+ | return ret + ((0xFFFFAA50U >> (x << 1)) & 3) + 1;
+ | }
+ | ...
+ | bit = ~len + (skip << 3) + 9; /* (skip << 3) + (8 - len), saves 1 cycle */
+
+
+12) Use of assembly
+-------------------
+
+There are many projects where use of assembly code is not welcome. There is no
+problem with use of assembly in haproxy, provided that :
+
+ a) an alternate C-form is provided for architectures not covered
+ b) the code is small enough and well commented enough to be maintained
+
+It is important to take care of various incompatibilities between compiler
+versions, for instance regarding output and cloberred registers. There are
+a number of documentations on the subject on the net. Anyway if you are
+fiddling with assembly, you probably know that already.
+
+Example :
+ | /* gcc does not know when it can safely divide 64 bits by 32 bits. Use this
+ | * function when you know for sure that the result fits in 32 bits, because
+ | * it is optimal on x86 and on 64bit processors.
+ | */
+ | static inline unsigned int div64_32(unsigned long long o1, unsigned int o2)
+ | {
+ | unsigned int result;
+ | #ifdef __i386__
+ | asm("divl %2"
+ | : "=a" (result)
+ | : "A"(o1), "rm"(o2));
+ | #else
+ | result = o1 / o2;
+ | #endif
+ | return result;
+ | }
+
+
+13) Pointers
+------------
+
+A lot could be said about pointers, there's enough to fill entire books. Misuse
+of pointers is one of the primary reasons for bugs in haproxy, and this rate
+has significantly increased with the use of threads. Moreover, bogus pointers
+cause the hardest to analyse bugs, because usually they result in modifications
+to reassigned areas or accesses to unmapped areas, and in each case, bugs that
+strike very far away from where they were located. Some bugs have already taken
+up to 3 weeks of full time analysis, which has a severe impact on the project's
+ability to make forward progress on important features. For this reason, code
+that doesn't look robust enough or that doesn't follow some of the rules below
+will be rejected, and may even be reverted after being merged if the trouble is
+detected late!
+
+
+13.1) No test before freeing
+----------------------------
+
+All platforms where haproxy is supported have a well-defined and documented
+behavior for free(NULL), which is to do nothing at all. In other words, free()
+does test for the pointer's nullity. As such, there is no point in testing
+if a pointer is NULL or not before calling free(). And further, you must not
+do it, because it adds some confusion to the reader during debugging sessions,
+making one think that the code's authors weren't very sure about what they
+were doing. This will not cause a bug but will result in your code to get
+rejected.
+
+Wrong call to free :
+
+ | static inline int blah_free(struct blah *blah)
+ | {
+ | if (blah->str1)
+ | free(blah->str1);
+ | if (blah->str2)
+ | free(blah->str2);
+ | free(blah);
+ | }
+
+Correct call to free :
+
+ | static inline int blah_free(struct blah *blah)
+ | {
+ | free(blah->str1);
+ | free(blah->str2);
+ | free(blah);
+ | }
+
+
+13.2) No dangling pointers
+--------------------------
+
+Pointers are very commonly used as booleans: if they're not NULL, then the
+area they point to is valid and may be used. This is convenient for many things
+and is even emphasized with threads where they can atomically be swapped with
+another value (even NULL), and as such provide guaranteed atomic resource
+allocation and sharing.
+
+The problem with this is when someone forgets to delete a pointer when an area
+is no longer valid, because this may result in the pointer being accessed later
+and pointing to a wrong location, one that was reallocated for something else
+and causing all sort of nastiness like crashes or memory corruption. Moreover,
+thanks to the memory pools, it is extremely likely that a just released pointer
+will be reassigned to a similar object with comparable values (flags etc) at
+the same positions, making tests apparently succeed for a while. Some such bugs
+have gone undetected for several years.
+
+The rule is pretty simple:
+
+ +-----------------------------------------------------------------+
+ | NO REACHABLE POINTER MAY EVER POINT TO AN UNREACHABLE LOCATION. |
+ +-----------------------------------------------------------------+
+
+By "reachable pointer", here we mean a pointer that is accessible from a
+reachable structure or a global variable. This means that any pointer found
+anywhere in any structure in the code may always be dereferenced. This can
+seem obvious but this is not always enforced.
+
+This means that when freeing an area, the pointer that was used to find that
+area must be overwritten with NULL, and all other such pointers must as well
+if any. It is one case where one can find more convenient to write the NULL
+on the same line as the call to free() to make things easier to check. Be
+careful about any potential "if" when doing this.
+
+Wrong use of free :
+
+ | static inline int blah_recycle(struct blah *blah)
+ | {
+ | free(blah->str1);
+ | free(blah->str2);
+ | }
+
+Correct use of free :
+
+ | static inline int blah_recycle(struct blah *blah)
+ | {
+ | free(blah->str1); blah->str1 = NULL;
+ | free(blah->str2); blah->str2 = NULL;
+ | }
+
+Sometimes the code doesn't permit this to be done. It is not a matter of code
+but a matter of architecture. Example:
+
+Initialization:
+
+ | static struct foo *foo_init()
+ | {
+ | struct foo *foo;
+ | struct bar *bar;
+ |
+ | foo = pool_alloc(foo_head);
+ | bar = pool_alloc(bar_head);
+ | if (!foo || !bar)
+ | goto fail;
+ | foo->bar = bar;
+ | ...
+ | }
+
+Scheduled task 1:
+
+ | static inline int foo_timeout(struct foo *foo)
+ | {
+ | free(foo->bar);
+ | free(foo);
+ | }
+
+Scheduled task 2:
+
+ | static inline int bar_timeout(struct bar *bar)
+ | {
+ | free(bar);
+ | }
+
+Here it's obvious that if "bar" times out, it will be freed but its pointer in
+"foo" will remain here, and if foo times out just after, it will lead to a
+double free. Or worse, if another instance allocates a pointer and receives bar
+again, when foo times out, it will release the old bar pointer which now points
+to a new object, and the code using that new object will crash much later, or
+even worse, will share the same area as yet another instance having inherited
+that pointer again.
+
+Here this simply means that the data model is wrong. If bar may be freed alone,
+it MUST have a pointer to foo so that bar->foo->bar is set to NULL to let foo
+finish its life peacefully. This also means that the code dealing with foo must
+be written in a way to support bar's leaving.
+
+
+13.3) Don't abuse pointers as booleans
+--------------------------------------
+
+Given the common use of a pointer to know if the area it points to is valid,
+there is a big incentive in using such pointers as booleans to describe
+something a bit higher level, like "is the user authenticated". This must not
+be done. The reason stems from the points above. Initially this perfectly
+matches and the code is simple. Then later some extra options need to be added,
+and more pointers are needed, all allocated together. At this point they all
+start to become their own booleans, supposedly always equivalent, but if that
+were true, they would be a single area with a single pointer. And things start
+to fall apart with some code areas relying on one pointer for the condition and
+other ones relying on other pointers. Pointers may be substituted with "flags"
+or "present in list" etc here. And from this point, things quickly degrade with
+pointers needing to remain set even if pointing to wrong areas, just for the
+sake of not being NULL and not breaking some assumptions. At this point the
+bugs are already there and the code is not trustable anymore.
+
+The only way to avoid this is to strictly respect this rule: pointers do not
+represent a functionality but a storage area. Of course it is very frequent to
+consider that if an optional string is not set, a feature is not enabled. This
+can be fine to some extents. But as soon as any slightest condition is added
+anywhere into the mux, the code relying on the pointer must be replaced with
+something else so that the pointer may live its own life and be released (and
+reset) earlier if needed.
+
+
+13.4) Mixing const and non-const
+--------------------------------
+
+Something often encountered, especially when assembling error messages, is
+functions that collect strings, assemble them into larger messages and free
+everything. The problem here is that if strings are defined as variables, there
+will rightfully be build warnings when reporting string constants such as bare
+keywords or messages, and if strings are defined as constants, it is not
+possible to free them. The temptation is sometimes huge to force some free()
+calls on casted strings. Do not do that! It will inevitably lead to someone
+getting caught passing a constant string that will make the process crash (if
+lucky). Document the expectations, indicate that all arguments must be freeable
+and that the caller must be capable of strdup(), and make your function support
+NULLs and document it (so that callers can deal with a failing strdup() on
+allocation error).
+
+One valid alternative is to use a secondary channel to indicate whether the
+message may be freed or not. A flag in a complex structure can be used for this
+purpose, for example. If you are certain that your strings are aligned to a
+certain number of bytes, it can be possible to instrument the code to use the
+lowest bit to indicate the need to free (e.g. by always adding one to every
+const string). But such a solution will require good enough instrumentation so
+that it doesn't constitute a new set of traps.
+
+
+13.5) No pointer casts
+----------------------
+
+Except in rare occasions caused by legacy APIs (e.g. sockaddr) or special cases
+which explicitly require a form of aliasing, there is no valid reason for
+casting pointers, and usually this is used to hide other problems that will
+strike later. The only suitable type of cast is the cast from the generic void*
+used to store a context for example. But in C, there is no need to cast to nor
+from void*, so this is not required. However those coming from C++ tend to be
+used to this practice, and others argue that it makes the intent more visible.
+
+As a corollary, do not abuse void*. Placing void* everywhere to avoid casting
+is a bad practice as well. The use of void* is only for generic functions or
+structures which do not have a limited set of types supported. When only a few
+types are supported, generally their type can be passed using a side channel,
+and the void* can be turned into a union that makes the code more readable and
+more verifiable.
+
+An alternative in haproxy is to use a pointer to an obj_type enum. Usually it
+is placed at the beginning of a structure. It works like a void* except that
+the type is read directly from the object. This is convenient when a small set
+of remote objects may be attached to another one because a single of them will
+match a non-null pointer (e.g. a connection or an applet).
+
+Example:
+
+ | static inline int blah_free(struct blah *blah)
+ | {
+ | /* only one of them (at most) will not be null */
+ | pool_free(pool_head_connection, objt_conn(blah->target));
+ | pool_free(pool_head_appctx, objt_appctx(blah->target));
+ | pool_free(pool_head_stream, objt_stream(blah->target));
+ | blah->target = NULL;
+ | }
+
+
+13.6) Extreme caution when using non-canonical pointers
+-------------------------------------------------------
+
+It can be particularly convenient to embed some logic in the unused bits or
+code points of a pointer. Indeed, when it is known by design that a given
+pointer will always follow a certain alignment, a few lower bits will always
+remain zero, and as such may be used as optional flags. For example, the ebtree
+code uses the lowest bit to differentiate left/right attachments to the parent
+and node/leaf in branches. It is also known that values very close to NULL will
+never represent a valid pointer, and the thread-safe MT_LIST code uses this to
+lock visited pointers.
+
+There are a few rules to respect in order to do this:
+ - the deviations from the canonical pointers must be exhaustively documented
+ where the pointer type is defined, and the whole control logic with its
+ implications and possible and impossible cases must be enumerated as well ;
+
+ - make sure that the operations will work on every supported platform, which
+ includes 32-bit platforms where structures may be aligned on as little as
+ 32-bit. 32-bit alignment leaves only two LSB available. When doing so, make
+ sure the target structures are not labelled with the "packed" attribute, or
+ that they're always perfectly aligned. All platforms where haproxy runs
+ have their NULL pointer mapped at address zero, and use page sizes at least
+ 4096 bytes large, leaving all values form 1 to 4095 unused. Anything
+ outside of this is unsafe. In particular, never use negative numbers to
+ represent a supposedly invalid address. On 32-bits platforms it will often
+ correspond to a system address or a special page. Always try a variety of
+ platforms when doing such a thing.
+
+ - the code must not use such pointers as booleans anymore even if it is known
+ that "it works" because that keeps a doubt open for the reviewer. Only the
+ canonical pointer may be tested. There can be a rare exception which is if
+ this is on a critical path where severe performance degradation may result
+ from this. In this case, *each* of the checks must be duly documented and
+ the equivalent BUG_ON() instances must be placed to prove the claim.
+
+ - some inline functions (or macros) must be used to turn the pointers to/from
+ their canonical form so that the regular code doesn't have to see the
+ operations, and so that the representation may be easily adjusted in the
+ future. A few comments indicating to a human how to turn a pointer back and
+ forth from inside a debugger will be appreciated, as macros often end up
+ not being trivially readable nor directly usable.
+
+ - do not use int types to cast the pointers, this will only work on 32-bit
+ platforms. While "long" is usually fine, it is not recommended anymore due
+ to the Windows platform being LLP64 and having it set to 32 bits. And
+ "long long" isn't good either for always being 64 bits. More suitable types
+ are ptrdiff_t or size_t. Note that while those were not available everywhere
+ in the early days of hparoxy, size_t is now heavily used and known to work
+ everywhere. And do not perform the operations on the pointers, only on the
+ integer types (and cast back again). Some compilers such as gcc are
+ extremely picky about this and will often emit wrong code when they see
+ equality conditions they believe is impossible and decide to optimize them
+ away.
+
+
+13.7) Pointers in unions
+------------------------
+
+Before placing multiple aliasing pointers inside a same union, there MUST be a
+SINGLE well-defined way to figure them out from each other. It may be thanks to
+a side-channel information (as done in the samples with a defined type), it may
+be based on in-area information (as done using obj_types), or any other trusted
+solution. In any case, if pointers are mixed with any other type (integer or
+float) in a union, there must be a very simple way to distinguish them, and not
+a platform-dependent nor compiler-dependent one.
diff --git a/doc/configuration.txt b/doc/configuration.txt
new file mode 100644
index 0000000..a1f15fc
--- /dev/null
+++ b/doc/configuration.txt
@@ -0,0 +1,26732 @@
+ ----------------------
+ HAProxy
+ Configuration Manual
+ ----------------------
+ version 2.9
+ 2024/02/15
+
+
+This document covers the configuration language as implemented in the version
+specified above. It does not provide any hints, examples, or advice. For such
+documentation, please refer to the Reference Manual or the Architecture Manual.
+The summary below is meant to help you find sections by name and navigate
+through the document.
+
+Note to documentation contributors :
+ This document is formatted with 80 columns per line, with even number of
+ spaces for indentation and without tabs. Please follow these rules strictly
+ so that it remains easily printable everywhere. If a line needs to be
+ printed verbatim and does not fit, please end each line with a backslash
+ ('\') and continue on next line, indented by two characters. It is also
+ sometimes useful to prefix all output lines (logs, console outputs) with 3
+ closing angle brackets ('>>>') in order to emphasize the difference between
+ inputs and outputs when they may be ambiguous. If you add sections,
+ please update the summary below for easier searching.
+
+
+Summary
+-------
+
+1. Quick reminder about HTTP
+1.1. The HTTP transaction model
+1.2. Terminology
+1.3. HTTP request
+1.3.1. The request line
+1.3.2. The request headers
+1.4. HTTP response
+1.4.1. The response line
+1.4.2. The response headers
+
+2. Configuring HAProxy
+2.1. Configuration file format
+2.2. Quoting and escaping
+2.3. Environment variables
+2.4. Conditional blocks
+2.5. Time format
+2.6. Size format
+2.7. Examples
+
+3. Global parameters
+3.1. Process management and security
+3.2. Performance tuning
+3.3. Debugging
+3.4. Userlists
+3.5. Peers
+3.6. Mailers
+3.7. Programs
+3.8. HTTP-errors
+3.9. Rings
+3.10. Log forwarding
+3.11. HTTPClient tuning
+
+4. Proxies
+4.1. Proxy keywords matrix
+4.2. Alphabetically sorted keywords reference
+4.3. Actions keywords matrix
+4.4. Alphabetically sorted actions reference
+
+5. Bind and server options
+5.1. Bind options
+5.2. Server and default-server options
+5.3. Server DNS resolution
+5.3.1. Global overview
+5.3.2. The resolvers section
+
+6. Cache
+6.1. Limitation
+6.2. Setup
+6.2.1. Cache section
+6.2.2. Proxy section
+
+7. Using ACLs and fetching samples
+7.1. ACL basics
+7.1.1. Matching booleans
+7.1.2. Matching integers
+7.1.3. Matching strings
+7.1.4. Matching regular expressions (regexes)
+7.1.5. Matching arbitrary data blocks
+7.1.6. Matching IPv4 and IPv6 addresses
+7.2. Using ACLs to form conditions
+7.3. Fetching samples
+7.3.1. Converters
+7.3.2. Fetching samples from internal states
+7.3.3. Fetching samples at Layer 4
+7.3.4. Fetching samples at Layer 5
+7.3.5. Fetching samples from buffer contents (Layer 6)
+7.3.6. Fetching HTTP samples (Layer 7)
+7.3.7. Fetching samples for developers
+7.4. Pre-defined ACLs
+
+8. Logging
+8.1. Log levels
+8.2. Log formats
+8.2.1. Default log format
+8.2.2. TCP log format
+8.2.3. HTTP log format
+8.2.4. HTTPS log format
+8.2.5. Error log format
+8.2.6. Custom log format
+8.3. Advanced logging options
+8.3.1. Disabling logging of external tests
+8.3.2. Logging before waiting for the stream to terminate
+8.3.3. Raising log level upon errors
+8.3.4. Disabling logging of successful connections
+8.4. Timing events
+8.5. Stream state at disconnection
+8.6. Non-printable characters
+8.7. Capturing HTTP cookies
+8.8. Capturing HTTP headers
+8.9. Examples of logs
+
+9. Supported filters
+9.1. Trace
+9.2. HTTP compression
+9.3. Stream Processing Offload Engine (SPOE)
+9.4. Cache
+9.5. fcgi-app
+9.6. OpenTracing
+9.7. Bandwidth limitation
+
+10. FastCGI applications
+10.1. Setup
+10.1.1. Fcgi-app section
+10.1.2. Proxy section
+10.1.3. Example
+10.2. Default parameters
+10.3. Limitations
+
+11. Address formats
+11.1. Address family prefixes
+11.2. Socket type prefixes
+11.3. Protocol prefixes
+
+
+1. Quick reminder about HTTP
+----------------------------
+
+When HAProxy is running in HTTP mode, both the request and the response are
+fully analyzed and indexed, thus it becomes possible to build matching criteria
+on almost anything found in the contents.
+
+However, it is important to understand how HTTP requests and responses are
+formed, and how HAProxy decomposes them. It will then become easier to write
+correct rules and to debug existing configurations.
+
+First, HTTP is standardized by a series of RFC that HAProxy follows as closely
+as possible:
+ - RFC 9110: HTTP Semantics (explains the meaning of protocol elements)
+ - RFC 9111: HTTP Caching (explains the rules to follow for an HTTP cache)
+ - RFC 9112: HTTP/1.1 (representation, interoperability rules, security)
+ - RFC 9113: HTTP/2 (representation, interoperability rules, security)
+ - RFC 9114: HTTP/3 (representation, interoperability rules, security)
+
+In addition to these, RFC 8999 to 9002 specify the QUIC transport layer used by
+the HTTP/3 protocol.
+
+
+1.1. The HTTP transaction model
+-------------------------------
+
+The HTTP protocol is transaction-driven. This means that each request will lead
+to one and only one response. Originally, with version 1.0 of the protocol,
+there was a single request per connection: a TCP connection is established from
+the client to the server, a request is sent by the client over the connection,
+the server responds, and the connection is closed. A new request then involves
+a new connection :
+
+ [CON1] [REQ1] ... [RESP1] [CLO1] [CON2] [REQ2] ... [RESP2] [CLO2] ...
+
+In this mode, often called the "HTTP close" mode, there are as many connection
+establishments as there are HTTP transactions. Since the connection is closed
+by the server after the response, the client does not need to know the content
+length, it considers that the response is complete when the connection closes.
+This also means that if some responses are truncated due to network errors, the
+client could mistakenly think a response was complete, and this used to cause
+truncated images to be rendered on screen sometimes.
+
+Due to the transactional nature of the protocol, it was possible to improve it
+to avoid closing a connection between two subsequent transactions. In this mode
+however, it is mandatory that the server indicates the content length for each
+response so that the client does not wait indefinitely. For this, a special
+header is used: "Content-length". This mode is called the "keep-alive" mode,
+and arrived with HTTP/1.1 (some HTTP/1.0 agents support it), and connections
+that are reused between requests are called "persistent connections":
+
+ [CON] [REQ1] ... [RESP1] [REQ2] ... [RESP2] [CLO] ...
+
+Its advantages are a reduced latency between transactions, less processing
+power required on the server side, and the ability to detect a truncated
+response. It is generally faster than the close mode, but not always because
+some clients often limit their concurrent connections to a smaller value, and
+this compensates less for poor network connectivity. Also, some servers have to
+keep the connection alive for a long time waiting for a possible new request
+and may experience a high memory usage due to the high number of connections,
+and closing too fast may break some requests that arrived at the moment the
+connection was closed.
+
+In this mode, the response size needs to be known upfront so that's not always
+possible with dynamically generated or compressed contents. For this reason
+another mode was implemented, the "chunked mode", where instead of announcing
+the size of the whole size at once, the sender only advertises the size of the
+next "chunk" of response it already has in a buffer, and can terminate at any
+moment with a zero-sized chunk. In this mode, the Content-Length header is not
+used.
+
+Another improvement in the communications is the pipelining mode. It still uses
+keep-alive, but the client does not wait for the first response to send the
+second request. This is useful for fetching large number of images composing a
+page :
+
+ [CON] [REQ1] [REQ2] ... [RESP1] [RESP2] [CLO] ...
+
+This can obviously have a tremendous benefit on performance because the network
+latency is eliminated between subsequent requests. Many HTTP agents do not
+correctly support pipelining since there is no way to associate a response with
+the corresponding request in HTTP. For this reason, it is mandatory for the
+server to reply in the exact same order as the requests were received. In
+practice, after several attempts by various clients to deploy it, it has been
+totally abandoned for its lack of reliability on certain servers. But it is
+mandatory for servers to support it.
+
+The next improvement is the multiplexed mode, as implemented in HTTP/2 and
+HTTP/3. In this mode, multiple transactions (i.e. request-response pairs) are
+transmitted in parallel over a single connection, and they all progress at
+their own speed, independent from each other. With multiplexed protocols, a new
+notion of "stream" was introduced, to represent these parallel communications
+happening over the same connection. Each stream is generally assigned a unique
+identifier for a given connection, that is used by both endpoints to know where
+to deliver the data. It is fairly common for clients to start many (up to 100,
+sometimes more) streams in parallel over a same connection, and let the server
+sort them out and respond in any order depending on what response is available.
+The main benefit of the multiplexed mode is that it significantly reduces the
+number of round trips, and speeds up page loading time over high latency
+networks. It is sometimes visibles on sites using many images, where all images
+appear to load in parallel.
+
+These protocols have also improved their efficiency by adopting some mechanisms
+to compress header fields in order to reduce the number of bytes on the wire,
+so that without the appropriate tools, they are not realistically manipulable
+by hand nor readable to the naked eye like HTTP/1 was. For this reason, various
+examples of HTTP messages continue to be represented in literature (including
+this document) using the HTTP/1 syntax even for newer versions of the protocol.
+
+HTTP/2 suffers from some design limitations, such as packet losses affecting
+all streams at once, and if a client takes too much time to retrieve an object
+(e.g. needs to store it on disk), it may slow down its retrieval and make it
+impossible during this time to access the data that is pending behind it. This
+is called "head of line blocking" or "HoL blocking" or sometimes just "HoL".
+
+HTTP/3 is implemented over QUIC, itself implemented over UDP. QUIC solves the
+head of line blocking at the transport level by means of independently handled
+streams. Indeed, when experiencing loss, an impacted stream does not affect the
+other streams, and all of them can be accessed in parallel.
+
+By default HAProxy operates in keep-alive mode with regards to persistent
+connections: for each connection it processes each request and response, and
+leaves the connection idle on both sides between the end of a response and the
+start of a new request. When it receives HTTP/2 connections from a client, it
+processes all the requests in parallel and leaves the connection idling,
+waiting for new requests, just as if it was a keep-alive HTTP connection.
+
+HAProxy essentially supports 3 connection modes :
+ - keep alive : all requests and responses are processed, and the client
+ facing and server facing connections are kept alive for new
+ requests. This is the default and suits the modern web and
+ modern protocols (HTTP/2 and HTTP/3).
+
+ - server close : the server-facing connection is closed after the response.
+
+ - close : the connection is actively closed after end of response on
+ both sides.
+
+In addition to this, by default, the server-facing connection is reusable by
+any request from any client, as mandated by the HTTP protocol specification, so
+any information pertaining to a specific client has to be passed along with
+each request if needed (e.g. client's source adress etc). When HTTP/2 is used
+with a server, by default HAProxy will dedicate this connection to the same
+client to avoid the risk of head of line blocking between clients.
+
+
+1.2. Terminology
+----------------
+
+Inside HAProxy, the terminology has evolved a bit over the ages to follow the
+evolutions of the HTTP protocol and its usages. While originally there was no
+significant difference between a connection, a session, a stream or a
+transaction, these ones clarified over time to match closely what exists in the
+modern versions of the HTTP protocol, though some terms remain visible in the
+configuration or the command line interface for the purpose of historical
+compatibility.
+
+Here are some definitions that apply to the current version of HAProxy:
+
+ - connection: a connection is a single, bidiractional communication channel
+ between a remote agent (client or server) and haproxy, at the lowest level
+ possible. Usually it corresponds to a TCP socket established between a pair
+ of IP and ports. On the client-facing side, connections are the very first
+ entities that are instantiated when a client connects to haproxy, and rules
+ applying at the connection level are the earliest ones that apply.
+
+ - session: a session adds some context information associated with a
+ connection. This includes and information specific to the transport layer
+ (e.g. TLS keys etc), or variables. This term has long been used inside
+ HAProxy to denote end-to-end HTTP/1.0 communications between two ends, and
+ as such it remains visible in the name of certain CLI commands or
+ statistics, despite representing streams nowadays, but the help messages
+ and descriptions try to make this unambiguous. It is still valid when it
+ comes to network-level terminology (e.g. TCP sessions inside the operating
+ systems, or TCP sessions across a firewall), or for non-HTTP user-level
+ applications (e.g. a telnet session or an SSH session). It must not be
+ confused with "application sessions" that are used to store a full user
+ context in a cookie and require to be sent to the same server.
+
+ - stream: a stream exactly corresponds to an end-to-end bidirectional
+ communication at the application level, where analysis and transformations
+ may be applied. In HTTP, it contains a single request and its associated
+ response, and is instantiated by the arrival of the request and is finished
+ with the end of delivery of the response. In this context there is a 1:1
+ relation between such a stream and the stream of a multiplexed protocol. In
+ TCP communications there is a single stream per connection.
+
+ - transaction: a transaction is only a pair of a request and the associated
+ response. The term was used in conjunction with sessions before the streams
+ but nowadays there is a 1:1 relation between a transaction and a stream. It
+ is essentially visible in the variables' scope "txn" which is valid during
+ the whole transaction, hence the stream.
+
+ - request: it designates the traffic flowing from the client to the server.
+ It is mainly used for HTTP to indicate where operations are performed. This
+ term also exists for TCP operations to indicate where data are processed.
+ Requests often appear in counters as a unit of traffic or activity. They do
+ not always imply a response (e.g. due to errors), but since there is no
+ spontaneous responses without requests, requests remain a relevant metric
+ of the overall activity. In TCP there are as many requests as connections.
+
+ - response: this designates the traffic flowing from the server to the
+ client, or sometimes from HAProxy to the client, when HAProxy produces the
+ response itself (e.g. an HTTP redirect).
+
+ - service: this generally indicates some internal processing in HAProxy that
+ does not require a server, such as the stats page, the cache, or some Lua
+ code to implement a small application. A service usually reads a request,
+ performs some operations and produces a response.
+
+
+1.3. HTTP request
+-----------------
+
+First, let's consider this HTTP request :
+
+ Line Contents
+ number
+ 1 GET /serv/login.php?lang=en&profile=2 HTTP/1.1
+ 2 Host: www.mydomain.com
+ 3 User-agent: my small browser
+ 4 Accept: image/jpeg, image/gif
+ 5 Accept: image/png
+
+
+1.3.1. The Request line
+-----------------------
+
+Line 1 is the "request line". It is always composed of 3 fields :
+
+ - a METHOD : GET
+ - a URI : /serv/login.php?lang=en&profile=2
+ - a version tag : HTTP/1.1
+
+All of them are delimited by what the standard calls LWS (linear white spaces),
+which are commonly spaces, but can also be tabs or line feeds/carriage returns
+followed by spaces/tabs. The method itself cannot contain any colon (':') and
+is limited to alphabetic letters. All those various combinations make it
+desirable that HAProxy performs the splitting itself rather than leaving it to
+the user to write a complex or inaccurate regular expression.
+
+The URI itself can have several forms :
+
+ - A "relative URI" :
+
+ /serv/login.php?lang=en&profile=2
+
+ It is a complete URL without the host part. This is generally what is
+ received by servers, reverse proxies and transparent proxies.
+
+ - An "absolute URI", also called a "URL" :
+
+ http://192.168.0.12:8080/serv/login.php?lang=en&profile=2
+
+ It is composed of a "scheme" (the protocol name followed by '://'), a host
+ name or address, optionally a colon (':') followed by a port number, then
+ a relative URI beginning at the first slash ('/') after the address part.
+ This is generally what proxies receive, but a server supporting HTTP/1.1
+ must accept this form too.
+
+ - a star ('*') : this form is only accepted in association with the OPTIONS
+ method and is not relayable. It is used to inquiry a next hop's
+ capabilities.
+
+ - an address:port combination : 192.168.0.12:80
+ This is used with the CONNECT method, which is used to establish TCP
+ tunnels through HTTP proxies, generally for HTTPS, but sometimes for
+ other protocols too.
+
+In a relative URI, two sub-parts are identified. The part before the question
+mark is called the "path". It is typically the relative path to static objects
+on the server. The part after the question mark is called the "query string".
+It is mostly used with GET requests sent to dynamic scripts and is very
+specific to the language, framework or application in use.
+
+HTTP/3 and HTTP/3 do not convey a version information with the request, so the
+version is assumed to be the same as the one of the underlying protocol (i.e.
+"HTTP/2"). In addition, these protocols do not send a request line as one part,
+but split it into individual fields called "pseudo-headers", whose name start
+with a colon, and which are conveniently reassembled by HAProxy into an
+equivalent request line. For this reason, request lines found in logs may
+slightly differ between HTTP/1.x and HTTP/2 or HTTP/3.
+
+
+1.3.2. The request headers
+--------------------------
+
+The headers start at the second line. They are composed of a name at the
+beginning of the line, immediately followed by a colon (':'). Traditionally,
+an LWS is added after the colon but that's not required. Then come the values.
+Multiple identical headers may be folded into one single line, delimiting the
+values with commas, provided that their order is respected. This is commonly
+encountered in the "Cookie:" field. A header may span over multiple lines if
+the subsequent lines begin with an LWS. In the example in 1.3, lines 4 and 5
+define a total of 3 values for the "Accept:" header. Finally, all LWS at the
+beginning or at the end of a header are ignored and are not part of the value,
+as per the specification.
+
+Contrary to a common misconception, header names are not case-sensitive, and
+their values are not either if they refer to other header names (such as the
+"Connection:" header). In HTTP/2 and HTTP/3, header names are always sent in
+lower case, as can be seen when running in debug mode. Internally, all header
+names are normalized to lower case so that HTTP/1.x and HTTP/2 or HTTP/3 use
+the exact same representation, and they are sent as-is on the other side. This
+explains why an HTTP/1.x request typed with camel case is delivered in lower
+case.
+
+The end of the headers is indicated by the first empty line. People often say
+that it's a double line feed, which is not exact, even if a double line feed
+is one valid form of empty line.
+
+Fortunately, HAProxy takes care of all these complex combinations when indexing
+headers, checking values and counting them, so there is no reason to worry
+about the way they could be written, but it is important not to accuse an
+application of being buggy if it does unusual, valid things.
+
+Important note:
+ As suggested by RFC7231, HAProxy normalizes headers by replacing line breaks
+ in the middle of headers by LWS in order to join multi-line headers. This
+ is necessary for proper analysis and helps less capable HTTP parsers to work
+ correctly and not to be fooled by such complex constructs.
+
+
+1.4. HTTP response
+------------------
+
+An HTTP response looks very much like an HTTP request. Both are called HTTP
+messages. Let's consider this HTTP response :
+
+ Line Contents
+ number
+ 1 HTTP/1.1 200 OK
+ 2 Content-length: 350
+ 3 Content-Type: text/html
+
+As a special case, HTTP supports so called "Informational responses" as status
+codes 1xx. These messages are special in that they don't convey any part of the
+response, they're just used as sort of a signaling message to ask a client to
+continue to post its request for instance. In the case of a status 100 response
+the requested information will be carried by the next non-100 response message
+following the informational one. This implies that multiple responses may be
+sent to a single request, and that this only works when keep-alive is enabled
+(1xx messages appeared in HTTP/1.1). HAProxy handles these messages and is able
+to correctly forward and skip them, and only process the next non-100 response.
+As such, these messages are neither logged nor transformed, unless explicitly
+state otherwise. Status 101 messages indicate that the protocol is changing
+over the same connection and that HAProxy must switch to tunnel mode, just as
+if a CONNECT had occurred. Then the Upgrade header would contain additional
+information about the type of protocol the connection is switching to.
+
+
+1.4.1. The response line
+------------------------
+
+Line 1 is the "response line". It is always composed of 3 fields :
+
+ - a version tag : HTTP/1.1
+ - a status code : 200
+ - a reason : OK
+
+The status code is always 3-digit. The first digit indicates a general status :
+ - 1xx = informational message to be skipped (e.g. 100, 101)
+ - 2xx = OK, content is following (e.g. 200, 206)
+ - 3xx = OK, no content following (e.g. 302, 304)
+ - 4xx = error caused by the client (e.g. 401, 403, 404)
+ - 5xx = error caused by the server (e.g. 500, 502, 503)
+
+Status codes greater than 599 must not be emitted in communications, though
+certain agents may produce them in logs to report their internal statuses.
+Please refer to RFC9110 for the detailed meaning of all such codes. HTTP/2 and
+above do not have a version tag and use the ":status" pseudo-header to report
+the status code.
+
+The "reason" field is just a hint, but is not parsed by clients. Anything can
+be found there, but it's a common practice to respect the well-established
+messages. It can be composed of one or multiple words, such as "OK", "Found",
+or "Authentication Required". It does not exist in HTTP/2 and above and is
+not emitted there. When a response from HTTP/2 or above is transmitted to an
+HTTP/1 client, HAProxy will produce such a common reason field that matches
+the status code.
+
+HAProxy may emit the following status codes by itself :
+
+ Code When / reason
+ 200 access to stats page, and when replying to monitoring requests
+ 301 when performing a redirection, depending on the configured code
+ 302 when performing a redirection, depending on the configured code
+ 303 when performing a redirection, depending on the configured code
+ 307 when performing a redirection, depending on the configured code
+ 308 when performing a redirection, depending on the configured code
+ 400 for an invalid or too large request
+ 401 when an authentication is required to perform the action (when
+ accessing the stats page)
+ 403 when a request is forbidden by a "http-request deny" rule
+ 404 when the requested resource could not be found
+ 408 when the request timeout strikes before the request is complete
+ 410 when the requested resource is no longer available and will not
+ be available again
+ 500 when HAProxy encounters an unrecoverable internal error, such as a
+ memory allocation failure, which should never happen
+ 501 when HAProxy is unable to satisfy a client request because of an
+ unsupported feature
+ 502 when the server returns an empty, invalid or incomplete response, or
+ when an "http-response deny" rule blocks the response.
+ 503 when no server was available to handle the request, or in response to
+ monitoring requests which match the "monitor fail" condition
+ 504 when the response timeout strikes before the server responds
+
+The error 4xx and 5xx codes above may be customized (see "errorloc" in section
+4.2). Other status codes can be emitted on purpose by specific actions (see the
+"deny", "return" and "redirect" actions in section 4.3 for example).
+
+
+1.4.2. The response headers
+---------------------------
+
+Response headers work exactly like request headers, and as such, HAProxy uses
+the same parsing function for both. Please refer to paragraph 1.3.2 for more
+details.
+
+
+2. Configuring HAProxy
+----------------------
+
+2.1. Configuration file format
+------------------------------
+
+HAProxy's configuration process involves 3 major sources of parameters :
+
+ - the arguments from the command-line, which always take precedence
+ - the configuration file(s), whose format is described here
+ - the running process's environment, in case some environment variables are
+ explicitly referenced
+
+The configuration file follows a fairly simple hierarchical format which obey
+a few basic rules:
+
+ 1. a configuration file is an ordered sequence of statements
+
+ 2. a statement is a single non-empty line before any unprotected "#" (hash)
+
+ 3. a line is a series of tokens or "words" delimited by unprotected spaces or
+ tab characters
+
+ 4. the first word or sequence of words of a line is one of the keywords or
+ keyword sequences listed in this document
+
+ 5. all other words are all arguments of the first one, some being well-known
+ keywords listed in this document, others being values, references to other
+ parts of the configuration, or expressions
+
+ 6. certain keywords delimit a section inside which only a subset of keywords
+ are supported
+
+ 7. a section ends at the end of a file or on a special keyword starting a new
+ section
+
+This is all that is needed to know to write a simple but reliable configuration
+generator, but this is not enough to reliably parse any configuration nor to
+figure how to deal with certain corner cases.
+
+First, there are a few consequences of the rules above. Rule 6 and 7 imply that
+the keywords used to define a new section are valid everywhere and cannot have
+a different meaning in a specific section. These keywords are always a single
+word (as opposed to a sequence of words), and traditionally the section that
+follows them is designated using the same name. For example when speaking about
+the "global section", it designates the section of configuration that follows
+the "global" keyword. This usage is used a lot in error messages to help locate
+the parts that need to be addressed.
+
+A number of sections create an internal object or configuration space, which
+requires to be distinguished from other ones. In this case they will take an
+extra word which will set the name of this particular section. For some of them
+the section name is mandatory. For example "frontend foo" will create a new
+section of type "frontend" named "foo". Usually a name is specific to its
+section and two sections of different types may use the same name, but this is
+not recommended as it tends to complexify configuration management.
+
+A direct consequence of rule 7 is that when multiple files are read at once,
+each of them must start with a new section, and the end of each file will end
+a section. A file cannot contain sub-sections nor end an existing section and
+start a new one.
+
+Rule 1 mentioned that ordering matters. Indeed, some keywords create directives
+that can be repeated multiple times to create ordered sequences of rules to be
+applied in a certain order. For example "tcp-request" can be used to alternate
+"accept" and "reject" rules on varying criteria. As such, a configuration file
+processor must always preserve a section's ordering when editing a file. The
+ordering of sections usually does not matter except for the global section
+which must be placed before other sections, but it may be repeated if needed.
+In addition, some automatic identifiers may automatically be assigned to some
+of the created objects (e.g. proxies), and by reordering sections, their
+identifiers will change. These ones appear in the statistics for example. As
+such, the configuration below will assign "foo" ID number 1 and "bar" ID number
+2, which will be swapped if the two sections are reversed:
+
+ listen foo
+ bind :80
+
+ listen bar
+ bind :81
+
+Another important point is that according to rules 2 and 3 above, empty lines,
+spaces, tabs, and comments following and unprotected "#" character are not part
+of the configuration as they are just used as delimiters. This implies that the
+following configurations are strictly equivalent:
+
+ global#this is the global section
+ daemon#daemonize
+ frontend foo
+ mode http # or tcp
+
+and:
+
+ global
+ daemon
+
+ # this is the public web frontend
+ frontend foo
+ mode http
+
+The common practice is to align to the left only the keyword that initiates a
+new section, and indent (i.e. prepend a tab character or a few spaces) all
+other keywords so that it's instantly visible that they belong to the same
+section (as done in the second example above). Placing comments before a new
+section helps the reader decide if it's the desired one. Leaving a blank line
+at the end of a section also visually helps spotting the end when editing it.
+
+Tabs are very convenient for indent but they do not copy-paste well. If spaces
+are used instead, it is recommended to avoid placing too many (2 to 4) so that
+editing in field doesn't become a burden with limited editors that do not
+support automatic indent.
+
+In the early days it used to be common to see arguments split at fixed tab
+positions because most keywords would not take more than two arguments. With
+modern versions featuring complex expressions this practice does not stand
+anymore, and is not recommended.
+
+
+2.2. Quoting and escaping
+-------------------------
+
+In modern configurations, some arguments require the use of some characters
+that were previously considered as pure delimiters. In order to make this
+possible, HAProxy supports character escaping by prepending a backslash ('\')
+in front of the character to be escaped, weak quoting within double quotes
+('"') and strong quoting within single quotes ("'").
+
+This is pretty similar to what is done in a number of programming languages and
+very close to what is commonly encountered in Bourne shell. The principle is
+the following: while the configuration parser cuts the lines into words, it
+also takes care of quotes and backslashes to decide whether a character is a
+delimiter or is the raw representation of this character within the current
+word. The escape character is then removed, the quotes are removed, and the
+remaining word is used as-is as a keyword or argument for example.
+
+If a backslash is needed in a word, it must either be escaped using itself
+(i.e. double backslash) or be strongly quoted.
+
+Escaping outside quotes is achieved by preceding a special character by a
+backslash ('\'):
+
+ \ to mark a space and differentiate it from a delimiter
+ \# to mark a hash and differentiate it from a comment
+ \\ to use a backslash
+ \' to use a single quote and differentiate it from strong quoting
+ \" to use a double quote and differentiate it from weak quoting
+
+In addition, a few non-printable characters may be emitted using their usual
+C-language representation:
+
+ \n to insert a line feed (LF, character \x0a or ASCII 10 decimal)
+ \r to insert a carriage return (CR, character \x0d or ASCII 13 decimal)
+ \t to insert a tab (character \x09 or ASCII 9 decimal)
+ \xNN to insert character having ASCII code hex NN (e.g \x0a for LF).
+
+Weak quoting is achieved by surrounding double quotes ("") around the character
+or sequence of characters to protect. Weak quoting prevents the interpretation
+of:
+
+ space or tab as a word separator
+ ' single quote as a strong quoting delimiter
+ # hash as a comment start
+
+Weak quoting permits the interpretation of environment variables (which are not
+evaluated outside of quotes) by preceding them with a dollar sign ('$'). If a
+dollar character is needed inside double quotes, it must be escaped using a
+backslash.
+
+Strong quoting is achieved by surrounding single quotes ('') around the
+character or sequence of characters to protect. Inside single quotes, nothing
+is interpreted, it's the efficient way to quote regular expressions.
+
+As a result, here is the matrix indicating how special characters can be
+entered in different contexts (unprintable characters are replaced with their
+name within angle brackets). Note that some characters that may only be
+represented escaped have no possible representation inside single quotes,
+hence its absence there:
+
+ Character | Unquoted | Weakly quoted | Strongly quoted
+ -----------+---------------+-----------------------------+-----------------
+ <TAB> | \<TAB>, \x09 | "<TAB>", "\<TAB>", "\x09" | '<TAB>'
+ -----------+---------------+-----------------------------+-----------------
+ <LF> | \n, \x0a | "\n", "\x0a" |
+ -----------+---------------+-----------------------------+-----------------
+ <CR> | \r, \x0d | "\r", "\x0d" |
+ -----------+---------------+-----------------------------+-----------------
+ <SPC> | \<SPC>, \x20 | "<SPC>", "\<SPC>", "\x20" | '<SPC>'
+ -----------+---------------+-----------------------------+-----------------
+ " | \", \x22 | "\"", "\x22" | '"'
+ -----------+---------------+-----------------------------+-----------------
+ # | \#, \x23 | "#", "\#", "\x23" | '#'
+ -----------+---------------+-----------------------------+-----------------
+ $ | $, \$, \x24 | "\$", "\x24" | '$'
+ -----------+---------------+-----------------------------+-----------------
+ ' | \', \x27 | "'", "\'", "\x27" |
+ -----------+---------------+-----------------------------+-----------------
+ \ | \\, \x5c | "\\", "\x5c" | '\'
+ -----------+---------------+-----------------------------+-----------------
+
+ Example:
+ # those are all strictly equivalent:
+ log-format %{+Q}o\ %t\ %s\ %{-Q}r
+ log-format "%{+Q}o %t %s %{-Q}r"
+ log-format '%{+Q}o %t %s %{-Q}r'
+ log-format "%{+Q}o %t"' %s %{-Q}r'
+ log-format "%{+Q}o %t"' %s'\ %{-Q}r
+
+There is one particular case where a second level of quoting or escaping may be
+necessary. Some keywords take arguments within parenthesis, sometimes delimited
+by commas. These arguments are commonly integers or predefined words, but when
+they are arbitrary strings, it may be required to perform a separate level of
+escaping to disambiguate the characters that belong to the argument from the
+characters that are used to delimit the arguments themselves. A pretty common
+case is the "regsub" converter. It takes a regular expression in argument, and
+if a closing parenthesis is needed inside, this one will require to have its
+own quotes.
+
+The keyword argument parser is exactly the same as the top-level one regarding
+quotes, except that the \#, \$, and \xNN escapes are not processed. But what is
+not always obvious is that the delimiters used inside must first be escaped or
+quoted so that they are not resolved at the top level.
+
+Let's take this example making use of the "regsub" converter which takes 3
+arguments, one regular expression, one replacement string and one set of flags:
+
+ # replace all occurrences of "foo" with "blah" in the path:
+ http-request set-path %[path,regsub(foo,blah,g)]
+
+Here no special quoting was necessary. But if now we want to replace either
+"foo" or "bar" with "blah", we'll need the regular expression "(foo|bar)". We
+cannot write:
+
+ http-request set-path %[path,regsub((foo|bar),blah,g)]
+
+because we would like the string to cut like this:
+
+ http-request set-path %[path,regsub((foo|bar),blah,g)]
+ |---------|----|-|
+ arg1 _/ / /
+ arg2 __________/ /
+ arg3 ______________/
+
+but actually what is passed is a string between the opening and closing
+parenthesis then garbage:
+
+ http-request set-path %[path,regsub((foo|bar),blah,g)]
+ |--------|--------|
+ arg1=(foo|bar _/ /
+ trailing garbage _________/
+
+The obvious solution here seems to be that the closing parenthesis needs to be
+quoted, but alone this will not work, because as mentioned above, quotes are
+processed by the top-level parser which will resolve them before processing
+this word:
+
+ http-request set-path %[path,regsub("(foo|bar)",blah,g)]
+ ------------ -------- ----------------------------------
+ word1 word2 word3=%[path,regsub((foo|bar),blah,g)]
+
+So we didn't change anything for the argument parser at the second level which
+still sees a truncated regular expression as the only argument, and garbage at
+the end of the string. By escaping the quotes they will be passed unmodified to
+the second level:
+
+ http-request set-path %[path,regsub(\"(foo|bar)\",blah,g)]
+ ------------ -------- ------------------------------------
+ word1 word2 word3=%[path,regsub("(foo|bar)",blah,g)]
+ |---------||----|-|
+ arg1=(foo|bar) _/ / /
+ arg2=blah ___________/ /
+ arg3=g _______________/
+
+Another approach consists in using single quotes outside the whole string and
+double quotes inside (so that the double quotes are not stripped again):
+
+ http-request set-path '%[path,regsub("(foo|bar)",blah,g)]'
+ ------------ -------- ----------------------------------
+ word1 word2 word3=%[path,regsub("(foo|bar)",blah,g)]
+ |---------||----|-|
+ arg1=(foo|bar) _/ / /
+ arg2 ___________/ /
+ arg3 _______________/
+
+When using regular expressions, it can happen that the dollar ('$') character
+appears in the expression or that a backslash ('\') is used in the replacement
+string. In this case these ones will also be processed inside the double quotes
+thus single quotes are preferred (or double escaping). Example:
+
+ http-request set-path '%[path,regsub("^/(here)(/|$)","my/\1",g)]'
+ ------------ -------- -----------------------------------------
+ word1 word2 word3=%[path,regsub("^/(here)(/|$)","my/\1",g)]
+ |-------------| |-----||-|
+ arg1=(here)(/|$) _/ / /
+ arg2=my/\1 ________________/ /
+ arg3 ______________________/
+
+Remember that backslashes are not escape characters within single quotes and
+that the whole word above is already protected against them using the single
+quotes. Conversely, if double quotes had been used around the whole expression,
+single the dollar character and the backslashes would have been resolved at top
+level, breaking the argument contents at the second level.
+
+Unfortunately, since single quotes can't be escaped inside of strong quoting,
+if you need to include single quotes in your argument, you will need to escape
+or quote them twice. There are a few ways to do this:
+
+ http-request set-var(txn.foo) str("\\'foo\\'")
+ http-request set-var(txn.foo) str(\"\'foo\'\")
+ http-request set-var(txn.foo) str(\\\'foo\\\')
+
+When in doubt, simply do not use quotes anywhere, and start to place single or
+double quotes around arguments that require a comma or a closing parenthesis,
+and think about escaping these quotes using a backslash if the string contains
+a dollar or a backslash. Again, this is pretty similar to what is used under
+a Bourne shell when double-escaping a command passed to "eval". For API writers
+the best is probably to place escaped quotes around each and every argument,
+regardless of their contents. Users will probably find that using single quotes
+around the whole expression and double quotes around each argument provides
+more readable configurations.
+
+
+2.3. Environment variables
+--------------------------
+
+HAProxy's configuration supports environment variables. Those variables are
+interpreted only within double quotes. Variables are expanded during the
+configuration parsing. Variable names must be preceded by a dollar ("$") and
+optionally enclosed with braces ("{}") similarly to what is done in Bourne
+shell. Variable names can contain alphanumerical characters or the character
+underscore ("_") but should not start with a digit. If the variable contains a
+list of several values separated by spaces, it can be expanded as individual
+arguments by enclosing the variable with braces and appending the suffix '[*]'
+before the closing brace. It is also possible to specify a default value to
+use when the variable is not set, by appending that value after a dash '-'
+next to the variable name. Note that the default value only replaces non
+existing variables, not empty ones.
+
+ Example:
+
+ bind "fd@${FD_APP1}"
+
+ log "${LOCAL_SYSLOG-127.0.0.1}:514" local0 notice # send to local server
+
+ user "$HAPROXY_USER"
+
+Some variables are defined by HAProxy, they can be used in the configuration
+file, or could be inherited by a program (See 3.7. Programs):
+
+* HAPROXY_LOCALPEER: defined at the startup of the process which contains the
+ name of the local peer. (See "-L" in the management guide.)
+
+* HAPROXY_CFGFILES: list of the configuration files loaded by HAProxy,
+ separated by semicolons. Can be useful in the case you specified a
+ directory.
+
+* HAPROXY_HTTP_LOG_FMT: contains the value of the default HTTP log format as
+ defined in section 8.2.3 "HTTP log format". It can be used to override the
+ default log format without having to copy the whole original definition.
+
+ Example:
+ # Add the rule that gave the final verdict to the log
+ log-format "${HAPROXY_TCP_LOG_FMT} lr=last_rule_file:last_rule_line"
+
+* HAPROXY_HTTPS_LOG_FMT: similar to HAPROXY_HTTP_LOG_FMT but for HTTPS log
+ format as defined in section 8.2.4 "HTTPS log format".
+
+* HAPROXY_TCP_LOG_FMT: similar to HAPROXY_HTTP_LOG_FMT but for TCP log format
+ as defined in section 8.2.2 "TCP log format".
+
+* HAPROXY_MWORKER: In master-worker mode, this variable is set to 1.
+
+* HAPROXY_CLI: configured listeners addresses of the stats socket for every
+ processes, separated by semicolons.
+
+* HAPROXY_MASTER_CLI: In master-worker mode, listeners addresses of the master
+ CLI, separated by semicolons.
+
+* HAPROXY_STARTUP_VERSION: contains the version used to start, in master-worker
+ mode this is the version which was used to start the master, even after
+ updating the binary and reloading.
+
+* HAPROXY_BRANCH: contains the HAProxy branch version (such as "2.8"). It does
+ not contain the full version number. It can be useful in case of migration
+ if resources (such as maps or certificates) are in a path containing the
+ branch number.
+
+In addition, some pseudo-variables are internally resolved and may be used as
+regular variables. Pseudo-variables always start with a dot ('.'), and are the
+only ones where the dot is permitted. The current list of pseudo-variables is:
+
+* .FILE: the name of the configuration file currently being parsed.
+
+* .LINE: the line number of the configuration file currently being parsed,
+ starting at one.
+
+* .SECTION: the name of the section currently being parsed, or its type if the
+ section doesn't have a name (e.g. "global"), or an empty string before the
+ first section.
+
+These variables are resolved at the location where they are parsed. For example
+if a ".LINE" variable is used in a "log-format" directive located in a defaults
+section, its line number will be resolved before parsing and compiling the
+"log-format" directive, so this same line number will be reused by subsequent
+proxies.
+
+This way it is possible to emit information to help locate a rule in variables,
+logs, error statuses, health checks, header values, or even to use line numbers
+to name some config objects like servers for example.
+
+See also "external-check command" for other variables.
+
+
+2.4. Conditional blocks
+-----------------------
+
+It may sometimes be convenient to be able to conditionally enable or disable
+some arbitrary parts of the configuration, for example to enable/disable SSL or
+ciphers, enable or disable some pre-production listeners without modifying the
+configuration, or adjust the configuration's syntax to support two distinct
+versions of HAProxy during a migration.. HAProxy brings a set of nestable
+preprocessor-like directives which allow to integrate or ignore some blocks of
+text. These directives must be placed on their own line and they act on the
+lines that follow them. Two of them support an expression, the other ones only
+switch to an alternate block or end a current level. The 4 following directives
+are defined to form conditional blocks:
+
+ - .if <condition>
+ - .elif <condition>
+ - .else
+ - .endif
+
+The ".if" directive nests a new level, ".elif" stays at the same level, ".else"
+as well, and ".endif" closes a level. Each ".if" must be terminated by a
+matching ".endif". The ".elif" may only be placed after ".if" or ".elif", and
+there is no limit to the number of ".elif" that may be chained. There may be
+only one ".else" per ".if" and it must always be after the ".if" or the last
+".elif" of a block.
+
+Comments may be placed on the same line if needed after a '#', they will be
+ignored. The directives are tokenized like other configuration directives, and
+as such it is possible to use environment variables in conditions.
+
+Conditions can also be evaluated on startup with the -cc parameter.
+See "3. Starting HAProxy" in the management doc.
+
+The conditions are either an empty string (which then returns false), or an
+expression made of any combination of:
+
+ - the integer zero ('0'), always returns "false"
+ - a non-nul integer (e.g. '1'), always returns "true".
+ - a predicate optionally followed by argument(s) in parenthesis.
+ - a condition placed between a pair of parenthesis '(' and ')'
+ - an exclamation mark ('!') preceding any of the non-empty elements above,
+ and which will negate its status.
+ - expressions combined with a logical AND ('&&'), which will be evaluated
+ from left to right until one returns false
+ - expressions combined with a logical OR ('||'), which will be evaluated
+ from right to left until one returns true
+
+Note that like in other languages, the AND operator has precedence over the OR
+operator, so that "A && B || C && D" evalues as "(A && B) || (C && D)".
+
+The list of currently supported predicates is the following:
+
+ - defined(<name>) : returns true if an environment variable <name>
+ exists, regardless of its contents
+
+ - feature(<name>) : returns true if feature <name> is listed as present
+ in the features list reported by "haproxy -vv"
+ (which means a <name> appears after a '+')
+
+ - streq(<str1>,<str2>) : returns true only if the two strings are equal
+ - strneq(<str1>,<str2>) : returns true only if the two strings differ
+ - strstr(<str1>,<str2>) : returns true only if the second string is found in the first one
+
+ - version_atleast(<ver>): returns true if the current haproxy version is
+ at least as recent as <ver> otherwise false. The
+ version syntax is the same as shown by "haproxy -v"
+ and missing components are assumed as being zero.
+
+ - version_before(<ver>) : returns true if the current haproxy version is
+ strictly older than <ver> otherwise false. The
+ version syntax is the same as shown by "haproxy -v"
+ and missing components are assumed as being zero.
+
+ - enabled(<opt>) : returns true if the option <opt> is enabled at
+ run-time. Only a subset of options are supported:
+ POLL, EPOLL, KQUEUE, EVPORTS, SPLICE,
+ GETADDRINFO, REUSEPORT, FAST-FORWARD,
+ SERVER-SSL-VERIFY-NONE
+
+Example:
+
+ .if defined(HAPROXY_MWORKER)
+ listen mwcli_px
+ bind :1111
+ ...
+ .endif
+
+ .if strneq("$SSL_ONLY",yes)
+ bind :80
+ .endif
+
+ .if streq("$WITH_SSL",yes)
+ .if feature(OPENSSL)
+ bind :443 ssl crt ...
+ .endif
+ .endif
+
+ .if feature(OPENSSL) && (streq("$WITH_SSL",yes) || streq("$SSL_ONLY",yes))
+ bind :443 ssl crt ...
+ .endif
+
+ .if version_atleast(2.4-dev19)
+ profiling.memory on
+ .endif
+
+ .if !feature(OPENSSL)
+ .alert "SSL support is mandatory"
+ .endif
+
+Four other directives are provided to report some status:
+
+ - .diag "message" : emit this message only when in diagnostic mode (-dD)
+ - .notice "message" : emit this message at level NOTICE
+ - .warning "message" : emit this message at level WARNING
+ - .alert "message" : emit this message at level ALERT
+
+Messages emitted at level WARNING may cause the process to fail to start if the
+"strict-mode" is enabled. Messages emitted at level ALERT will always cause a
+fatal error. These can be used to detect some inappropriate conditions and
+provide advice to the user.
+
+Example:
+
+ .if "${A}"
+ .if "${B}"
+ .notice "A=1, B=1"
+ .elif "${C}"
+ .notice "A=1, B=0, C=1"
+ .elif "${D}"
+ .warning "A=1, B=0, C=0, D=1"
+ .else
+ .alert "A=1, B=0, C=0, D=0"
+ .endif
+ .else
+ .notice "A=0"
+ .endif
+
+ .diag "WTA/2021-05-07: replace 'redirect' with 'return' after switch to 2.4"
+ http-request redirect location /goaway if ABUSE
+
+
+2.5. Time format
+----------------
+
+Some parameters involve values representing time, such as timeouts. These
+values are generally expressed in milliseconds (unless explicitly stated
+otherwise) but may be expressed in any other unit by suffixing the unit to the
+numeric value. It is important to consider this because it will not be repeated
+for every keyword. Supported units are :
+
+ - us : microseconds. 1 microsecond = 1/1000000 second
+ - ms : milliseconds. 1 millisecond = 1/1000 second. This is the default.
+ - s : seconds. 1s = 1000ms
+ - m : minutes. 1m = 60s = 60000ms
+ - h : hours. 1h = 60m = 3600s = 3600000ms
+ - d : days. 1d = 24h = 1440m = 86400s = 86400000ms
+
+
+2.6. Size format
+----------------
+
+Some parameters involve values representing size, such as bandwidth limits.
+These values are generally expressed in bytes (unless explicitly stated
+otherwise) but may be expressed in any other unit by suffixing the unit to the
+numeric value. It is important to consider this because it will not be repeated
+for every keyword. Supported units are case insensitive :
+
+ - k : kilobytes. 1 kilobyte = 1024 bytes
+ - m : megabytes. 1 megabyte = 1048576 bytes
+ - g : gigabytes. 1 gigabyte = 1073741824 bytes
+
+Both time and size formats require integers, decimal notation is not allowed.
+
+
+2.7. Examples
+-------------
+
+ # Simple configuration for an HTTP proxy listening on port 80 on all
+ # interfaces and forwarding requests to a single backend "servers" with a
+ # single server "server1" listening on 127.0.0.1:8000
+ global
+ daemon
+ maxconn 256
+
+ defaults
+ mode http
+ timeout connect 5000ms
+ timeout client 50000ms
+ timeout server 50000ms
+
+ frontend http-in
+ bind *:80
+ default_backend servers
+
+ backend servers
+ server server1 127.0.0.1:8000 maxconn 32
+
+
+ # The same configuration defined with a single listen block. Shorter but
+ # less expressive, especially in HTTP mode.
+ global
+ daemon
+ maxconn 256
+
+ defaults
+ mode http
+ timeout connect 5000ms
+ timeout client 50000ms
+ timeout server 50000ms
+
+ listen http-in
+ bind *:80
+ server server1 127.0.0.1:8000 maxconn 32
+
+
+Assuming haproxy is in $PATH, test these configurations in a shell with:
+
+ $ sudo haproxy -f configuration.conf -c
+
+
+3. Global parameters
+--------------------
+
+Parameters in the "global" section are process-wide and often OS-specific. They
+are generally set once for all and do not need being changed once correct. Some
+of them have command-line equivalents.
+
+The following keywords are supported in the "global" section :
+
+ * Process management and security
+ - 51degrees-allow-unmatched
+ - 51degrees-cache-size
+ - 51degrees-data-file
+ - 51degrees-difference
+ - 51degrees-drift
+ - 51degrees-property-name-list
+ - 51degrees-property-separator
+ - 51degrees-use-performance-graph
+ - 51degrees-use-predictive-graph
+ - ca-base
+ - chroot
+ - cluster-secret
+ - cpu-map
+ - crt-base
+ - daemon
+ - default-path
+ - description
+ - deviceatlas-json-file
+ - deviceatlas-log-level
+ - deviceatlas-properties-cookie
+ - deviceatlas-separator
+ - expose-experimental-directives
+ - external-check
+ - fd-hard-limit
+ - gid
+ - grace
+ - group
+ - h1-accept-payload-with-any-method
+ - h1-case-adjust
+ - h1-case-adjust-file
+ - h2-workaround-bogus-websocket-clients
+ - hard-stop-after
+ - insecure-fork-wanted
+ - insecure-setuid-wanted
+ - issuers-chain-path
+ - localpeer
+ - log
+ - log-send-hostname
+ - log-tag
+ - lua-load
+ - lua-load-per-thread
+ - lua-prepend-path
+ - mworker-max-reloads
+ - nbthread
+ - node
+ - numa-cpu-mapping
+ - pidfile
+ - pp2-never-send-local
+ - presetenv
+ - prealloc-fd
+ - resetenv
+ - set-dumpable
+ - set-var
+ - setenv
+ - ssl-default-bind-ciphers
+ - ssl-default-bind-ciphersuites
+ - ssl-default-bind-client-sigalgs
+ - ssl-default-bind-curves
+ - ssl-default-bind-options
+ - ssl-default-bind-sigalgs
+ - ssl-default-server-ciphers
+ - ssl-default-server-ciphersuites
+ - ssl-default-server-client-sigalgs
+ - ssl-default-server-curves
+ - ssl-default-server-options
+ - ssl-default-server-sigalgs
+ - ssl-dh-param-file
+ - ssl-propquery
+ - ssl-provider
+ - ssl-provider-path
+ - ssl-server-verify
+ - ssl-skip-self-issued-ca
+ - stats
+ - strict-limits
+ - uid
+ - ulimit-n
+ - unix-bind
+ - unsetenv
+ - user
+ - wurfl-cache-size
+ - wurfl-data-file
+ - wurfl-information-list
+ - wurfl-information-list-separator
+
+ * Performance tuning
+ - busy-polling
+ - max-spread-checks
+ - maxcompcpuusage
+ - maxcomprate
+ - maxconn
+ - maxconnrate
+ - maxpipes
+ - maxsessrate
+ - maxsslconn
+ - maxsslrate
+ - maxzlibmem
+ - no-memory-trimming
+ - noepoll
+ - noevports
+ - nogetaddrinfo
+ - nokqueue
+ - nopoll
+ - noreuseport
+ - nosplice
+ - profiling.tasks
+ - server-state-base
+ - server-state-file
+ - spread-checks
+ - ssl-engine
+ - ssl-mode-async
+ - tune.buffers.limit
+ - tune.buffers.reserve
+ - tune.bufsize
+ - tune.comp.maxlevel
+ - tune.disable-fast-forward
+ - tune.disable-zero-copy-forwarding
+ - tune.events.max-events-at-once
+ - tune.fail-alloc
+ - tune.fd.edge-triggered
+ - tune.h1.zero-copy-fwd-recv
+ - tune.h1.zero-copy-fwd-send
+ - tune.h2.be.initial-window-size
+ - tune.h2.be.max-concurrent-streams
+ - tune.h2.fe.initial-window-size
+ - tune.h2.fe.max-concurrent-streams
+ - tune.h2.fe.max-total-streams
+ - tune.h2.header-table-size
+ - tune.h2.initial-window-size
+ - tune.h2.max-concurrent-streams
+ - tune.h2.max-frame-size
+ - tune.h2.zero-copy-fwd-send
+ - tune.http.cookielen
+ - tune.http.logurilen
+ - tune.http.maxhdr
+ - tune.idle-pool.shared
+ - tune.idletimer
+ - tune.lua.forced-yield
+ - tune.lua.maxmem
+ - tune.lua.service-timeout
+ - tune.lua.session-timeout
+ - tune.lua.task-timeout
+ - tune.lua.log.loggers
+ - tune.lua.log.stderr
+ - tune.max-checks-per-thread
+ - tune.maxaccept
+ - tune.maxpollevents
+ - tune.maxrewrite
+ - tune.memory.hot-size
+ - tune.pattern.cache-size
+ - tune.peers.max-updates-at-once
+ - tune.pipesize
+ - tune.pool-high-fd-ratio
+ - tune.pool-low-fd-ratio
+ - tune.pt.zero-copy-forwarding
+ - tune.quic.frontend.conn-tx-buffers.limit
+ - tune.quic.frontend.max-idle-timeout
+ - tune.quic.frontend.max-streams-bidi
+ - tune.quic.max-frame-loss
+ - tune.quic.retry-threshold
+ - tune.quic.socket-owner
+ - tune.quic.zero-copy-fwd-send
+ - tune.rcvbuf.backend
+ - tune.rcvbuf.client
+ - tune.rcvbuf.frontend
+ - tune.rcvbuf.server
+ - tune.recv_enough
+ - tune.runqueue-depth
+ - tune.sched.low-latency
+ - tune.sndbuf.backend
+ - tune.sndbuf.client
+ - tune.sndbuf.frontend
+ - tune.sndbuf.server
+ - tune.stick-counters
+ - tune.ssl.cachesize
+ - tune.ssl.capture-buffer-size
+ - tune.ssl.capture-cipherlist-size (deprecated)
+ - tune.ssl.default-dh-param
+ - tune.ssl.force-private-cache
+ - tune.ssl.hard-maxrecord
+ - tune.ssl.keylog
+ - tune.ssl.lifetime
+ - tune.ssl.maxrecord
+ - tune.ssl.ssl-ctx-cache-size
+ - tune.ssl.ocsp-update.maxdelay
+ - tune.ssl.ocsp-update.mindelay
+ - tune.vars.global-max-size
+ - tune.vars.proc-max-size
+ - tune.vars.reqres-max-size
+ - tune.vars.sess-max-size
+ - tune.vars.txn-max-size
+ - tune.zlib.memlevel
+ - tune.zlib.windowsize
+
+ * Debugging
+ - anonkey
+ - quiet
+ - zero-warning
+
+ * HTTPClient
+ - httpclient.resolvers.disabled
+ - httpclient.resolvers.id
+ - httpclient.resolvers.prefer
+ - httpclient.retries
+ - httpclient.ssl.ca-file
+ - httpclient.ssl.verify
+ - httpclient.timeout.connect
+
+3.1. Process management and security
+------------------------------------
+
+51degrees-data-file <file path>
+ The path of the 51Degrees data file to provide device detection services. The
+ file should be unzipped and accessible by HAProxy with relevant permissions.
+
+ Please note that this option is only available when HAProxy has been
+ compiled with USE_51DEGREES.
+
+51degrees-property-name-list [<string> ...]
+ A list of 51Degrees property names to be load from the dataset. A full list
+ of names is available on the 51Degrees website:
+ https://51degrees.com/resources/property-dictionary
+
+ Please note that this option is only available when HAProxy has been
+ compiled with USE_51DEGREES.
+
+51degrees-property-separator <char>
+ A char that will be appended to every property value in a response header
+ containing 51Degrees results. If not set that will be set as ','.
+
+ Please note that this option is only available when HAProxy has been
+ compiled with USE_51DEGREES.
+
+51degrees-cache-size <number>
+ Sets the size of the 51Degrees converter cache to <number> entries. This
+ is an LRU cache which reminds previous device detections and their results.
+ By default, this cache is disabled.
+
+ Please note that this option is only available when HAProxy has been
+ compiled with USE_51DEGREES.
+
+51degrees-use-performance-graph { on | off }
+ Enables ('on') or disables ('off') the use of the performance graph in
+ the detection process. The default value depends on 51Degrees library.
+
+ Please note that this option is only available when HAProxy has been
+ compiled with USE_51DEGREES and 51DEGREES_VER=4.
+
+51degrees-use-predictive-graph { on | off }
+ Enables ('on') or disables ('off') the use of the predictive graph in
+ the detection process. The default value depends on 51Degrees library.
+
+ Please note that this option is only available when HAProxy has been
+ compiled with USE_51DEGREES and 51DEGREES_VER=4.
+
+51degrees-drift <number>
+ Sets the drift value that a detection can allow.
+
+ Please note that this option is only available when HAProxy has been
+ compiled with USE_51DEGREES and 51DEGREES_VER=4.
+
+51degrees-difference <number>
+ Sets the difference value that a detection can allow.
+
+ Please note that this option is only available when HAProxy has been
+ compiled with USE_51DEGREES and 51DEGREES_VER=4.
+
+51degrees-allow-unmatched { on | off }
+ Enables ('on') or disables ('off') the use of unmatched nodes in the
+ detection process. The default value depends on 51Degrees library.
+
+ Please note that this option is only available when HAProxy has been
+ compiled with USE_51DEGREES and 51DEGREES_VER=4.
+
+ca-base <dir>
+ Assigns a default directory to fetch SSL CA certificates and CRLs from when a
+ relative path is used with "ca-file", "ca-verify-file" or "crl-file"
+ directives. Absolute locations specified in "ca-file", "ca-verify-file" and
+ "crl-file" prevail and ignore "ca-base".
+
+chroot <jail dir>
+ Changes current directory to <jail dir> and performs a chroot() there before
+ dropping privileges. This increases the security level in case an unknown
+ vulnerability would be exploited, since it would make it very hard for the
+ attacker to exploit the system. This only works when the process is started
+ with superuser privileges. It is important to ensure that <jail_dir> is both
+ empty and non-writable to anyone.
+
+close-spread-time <time>
+ Define a time window during which idle connections and active connections
+ closing is spread in case of soft-stop. After a SIGUSR1 is received and the
+ grace period is over (if any), the idle connections will all be closed at
+ once if this option is not set, and active HTTP or HTTP2 connections will be
+ ended after the next request is received, either by appending a "Connection:
+ close" line to the HTTP response, or by sending a GOAWAY frame in case of
+ HTTP2. When this option is set, connection closing will be spread over this
+ set <time>.
+ If the close-spread-time is set to "infinite", active connection closing
+ during a soft-stop will be disabled. The "Connection: close" header will not
+ be added to HTTP responses (or GOAWAY for HTTP2) anymore and idle connections
+ will only be closed once their timeout is reached (based on the various
+ timeouts set in the configuration).
+
+ Arguments :
+ <time> is a time window (by default in milliseconds) during which
+ connection closing will be spread during a soft-stop operation, or
+ "infinite" if active connection closing should be disabled.
+
+ It is recommended to set this setting to a value lower than the one used in
+ the "hard-stop-after" option if this one is used, so that all connections
+ have a chance to gracefully close before the process stops.
+
+ See also: grace, hard-stop-after, idle-close-on-response
+
+cluster-secret <secret>
+ Define an ASCII string secret shared between several nodes belonging to the
+ same cluster. It could be used for different usages. It is at least used to
+ derive stateless reset tokens for all the QUIC connections instantiated by
+ this process. This is also the case to derive secrets used to encrypt Retry
+ tokens.
+
+ If this parameter is not set, a random value will be selected on process
+ startup. This allows to use features which rely on it, albeit with some
+ limitations.
+
+cpu-map [auto:]<thread-group>[/<thread-set>] <cpu-set>[,...] [...]
+ On some operating systems, it is possible to bind a thread group or a thread
+ to a specific CPU set. This means that the designated threads will never run
+ on other CPUs. The "cpu-map" directive specifies CPU sets for individual
+ threads or thread groups. The first argument is a thread group range,
+ optionally followed by a thread set. These ranges have the following format:
+
+ all | odd | even | number[-[number]]
+
+ <number> must be a number between 1 and 32 or 64, depending on the machine's
+ word size. Any group IDs above 'thread-groups' and any thread IDs above the
+ machine's word size are ignored. All thread numbers are relative to the group
+ they belong to. It is possible to specify a range with two such number
+ delimited by a dash ('-'). It also is possible to specify all threads at once
+ using "all", only odd numbers using "odd" or even numbers using "even", just
+ like with the "thread" bind directive. The second and forthcoming arguments
+ are CPU sets. Each CPU set is either a unique number starting at 0 for the
+ first CPU or a range with two such numbers delimited by a dash ('-'). These
+ CPU numbers and ranges may be repeated by delimiting them with commas or by
+ passing more ranges as new arguments on the same line. Outside of Linux and
+ BSD operating systems, there may be a limitation on the maximum CPU index to
+ either 31 or 63. Multiple "cpu-map" directives may be specified, but each
+ "cpu-map" directive will replace the previous ones when they overlap.
+
+ Ranges can be partially defined. The higher bound can be omitted. In such
+ case, it is replaced by the corresponding maximum value, 32 or 64 depending
+ on the machine's word size.
+
+ The prefix "auto:" can be added before the thread set to let HAProxy
+ automatically bind a set of threads to a CPU by incrementing threads and
+ CPU sets. To be valid, both sets must have the same size. No matter the
+ declaration order of the CPU sets, it will be bound from the lowest to the
+ highest bound. Having both a group and a thread range with the "auto:"
+ prefix is not supported. Only one range is supported, the other one must be
+ a fixed number.
+
+ Note that group ranges are supported for historical reasons. Nowadays, a lone
+ number designates a thread group and must be 1 if thread-groups are not used,
+ and specifying a thread range or number requires to prepend "1/" in front of
+ it if thread groups are not used. Finally, "1" is strictly equivalent to
+ "1/all" and designates all threads in the group.
+
+ Examples:
+ cpu-map 1/all 0-3 # bind all threads of the first group on the
+ # first 4 CPUs
+
+ cpu-map 1/1- 0- # will be replaced by "cpu-map 1/1-64 0-63"
+ # or "cpu-map 1/1-32 0-31" depending on the machine's
+ # word size.
+
+ # all these lines bind thread 1 to the cpu 0, the thread 2 to cpu 1
+ # and so on.
+ cpu-map auto:1/1-4 0-3
+ cpu-map auto:1/1-4 0-1 2-3
+ cpu-map auto:1/1-4 3 2 1 0
+ cpu-map auto:1/1-4 3,2,1,0
+
+ # bind each thread to exactly one CPU using all/odd/even keyword
+ cpu-map auto:1/all 0-63
+ cpu-map auto:1/even 0-31
+ cpu-map auto:1/odd 32-63
+
+ # invalid cpu-map because thread and CPU sets have different sizes.
+ cpu-map auto:1/1-4 0 # invalid
+ cpu-map auto:1/1 0-3 # invalid
+
+ # map 40 threads of those 4 groups to individual CPUs
+ cpu-map auto:1/1-10 0-9
+ cpu-map auto:2/1-10 10-19
+ cpu-map auto:3/1-10 20-29
+ cpu-map auto:4/1-10 30-39
+
+ # Map 80 threads to one physical socket and 80 others to another socket
+ # without forcing assignment. These are split into 4 groups since no
+ # group may have more than 64 threads.
+ cpu-map 1/1-40 0-39,80-119 # node0, siblings 0 & 1
+ cpu-map 2/1-40 0-39,80-119
+ cpu-map 3/1-40 40-79,120-159 # node1, siblings 0 & 1
+ cpu-map 4/1-40 40-79,120-159
+
+
+crt-base <dir>
+ Assigns a default directory to fetch SSL certificates from when a relative
+ path is used with "crtfile" or "crt" directives. Absolute locations specified
+ prevail and ignore "crt-base".
+
+daemon
+ Makes the process fork into background. This is the recommended mode of
+ operation. It is equivalent to the command line "-D" argument. It can be
+ disabled by the command line "-db" argument. This option is ignored in
+ systemd mode.
+
+default-path { current | config | parent | origin <path> }
+ By default HAProxy loads all files designated by a relative path from the
+ location the process is started in. In some circumstances it might be
+ desirable to force all relative paths to start from a different location
+ just as if the process was started from such locations. This is what this
+ directive is made for. Technically it will perform a temporary chdir() to
+ the designated location while processing each configuration file, and will
+ return to the original directory after processing each file. It takes an
+ argument indicating the policy to use when loading files whose path does
+ not start with a slash ('/'):
+ - "current" indicates that all relative files are to be loaded from the
+ directory the process is started in ; this is the default.
+
+ - "config" indicates that all relative files should be loaded from the
+ directory containing the configuration file. More specifically, if the
+ configuration file contains a slash ('/'), the longest part up to the
+ last slash is used as the directory to change to, otherwise the current
+ directory is used. This mode is convenient to bundle maps, errorfiles,
+ certificates and Lua scripts together as relocatable packages. When
+ multiple configuration files are loaded, the directory is updated for
+ each of them.
+
+ - "parent" indicates that all relative files should be loaded from the
+ parent of the directory containing the configuration file. More
+ specifically, if the configuration file contains a slash ('/'), ".."
+ is appended to the longest part up to the last slash is used as the
+ directory to change to, otherwise the directory is "..". This mode is
+ convenient to bundle maps, errorfiles, certificates and Lua scripts
+ together as relocatable packages, but where each part is located in a
+ different subdirectory (e.g. "config/", "certs/", "maps/", ...).
+
+ - "origin" indicates that all relative files should be loaded from the
+ designated (mandatory) path. This may be used to ease management of
+ different HAProxy instances running in parallel on a system, where each
+ instance uses a different prefix but where the rest of the sections are
+ made easily relocatable.
+
+ Each "default-path" directive instantly replaces any previous one and will
+ possibly result in switching to a different directory. While this should
+ always result in the desired behavior, it is really not a good practice to
+ use multiple default-path directives, and if used, the policy ought to remain
+ consistent across all configuration files.
+
+ Warning: some configuration elements such as maps or certificates are
+ uniquely identified by their configured path. By using a relocatable layout,
+ it becomes possible for several of them to end up with the same unique name,
+ making it difficult to update them at run time, especially when multiple
+ configuration files are loaded from different directories. It is essential to
+ observe a strict collision-free file naming scheme before adopting relative
+ paths. A robust approach could consist in prefixing all files names with
+ their respective site name, or in doing so at the directory level.
+
+description <text>
+ Add a text that describes the instance.
+
+ Please note that it is required to escape certain characters (# for example)
+ and this text is inserted into a html page so you should avoid using
+ "<" and ">" characters.
+
+deviceatlas-json-file <path>
+ Sets the path of the DeviceAtlas JSON data file to be loaded by the API.
+ The path must be a valid JSON data file and accessible by HAProxy process.
+
+deviceatlas-log-level <value>
+ Sets the level of information returned by the API. This directive is
+ optional and set to 0 by default if not set.
+
+deviceatlas-properties-cookie <name>
+ Sets the client cookie's name used for the detection if the DeviceAtlas
+ Client-side component was used during the request. This directive is optional
+ and set to DAPROPS by default if not set.
+
+deviceatlas-separator <char>
+ Sets the character separator for the API properties results. This directive
+ is optional and set to | by default if not set.
+
+expose-experimental-directives
+ This statement must appear before using directives tagged as experimental or
+ the config file will be rejected.
+
+external-check [preserve-env]
+ Allows the use of an external agent to perform health checks. This is
+ disabled by default as a security precaution, and even when enabled, checks
+ may still fail unless "insecure-fork-wanted" is enabled as well. If the
+ program launched makes use of a setuid executable (it should really not),
+ you may also need to set "insecure-setuid-wanted" in the global section.
+ By default, the checks start with a clean environment which only contains
+ variables defined in the "external-check" command in the backend section. It
+ may sometimes be desirable to preserve the environment though, for example
+ when complex scripts retrieve their extra paths or information there. This
+ can be done by appending the "preserve-env" keyword. In this case however it
+ is strongly advised not to run a setuid nor as a privileged user, as this
+ exposes the check program to potential attacks. See "option external-check",
+ and "insecure-fork-wanted", and "insecure-setuid-wanted" for extra details.
+
+fd-hard-limit <number>
+ Sets an upper bound to the maximum number of file descriptors that the
+ process will use, regardless of system limits. While "ulimit-n" and "maxconn"
+ may be used to enforce a value, when they are not set, the process will be
+ limited to the hard limit of the RLIMIT_NOFILE setting as reported by
+ "ulimit -n -H". But some modern operating systems are now allowing extremely
+ large values here (in the order of 1 billion), which will consume way too
+ much RAM for regular usage. The fd-hard-limit setting is provided to enforce
+ a possibly lower bound to this limit. This means that it will always respect
+ the system-imposed limits when they are below <number> but the specified
+ value will be used if system-imposed limits are higher. In the example below,
+ no other setting is specified and the maxconn value will automatically adapt
+ to the lower of "fd-hard-limit" and the system-imposed limit:
+
+ global
+ # use as many FDs as possible but no more than 50000
+ fd-hard-limit 50000
+
+ See also: ulimit-n, maxconn
+
+gid <number>
+ Changes the process's group ID to <number>. It is recommended that the group
+ ID is dedicated to HAProxy or to a small set of similar daemons. HAProxy must
+ be started with a user belonging to this group, or with superuser privileges.
+ Note that if HAProxy is started from a user having supplementary groups, it
+ will only be able to drop these groups if started with superuser privileges.
+ See also "group" and "uid".
+
+grace <time>
+ Defines a delay between SIGUSR1 and real soft-stop.
+
+ Arguments :
+ <time> is an extra delay (by default in milliseconds) after receipt of the
+ SIGUSR1 signal that will be waited for before proceeding with the
+ soft-stop operation.
+
+ This is used for compatibility with legacy environments where the haproxy
+ process needs to be stopped but some external components need to detect the
+ status before listeners are unbound. The principle is that the internal
+ "stopping" variable (which is reported by the "stopping" sample fetch
+ function) will be turned to true, but listeners will continue to accept
+ connections undisturbed, until the delay expires, after what the regular
+ soft-stop will proceed. This must not be used with processes that are
+ reloaded, or this will prevent the old process from unbinding, and may
+ prevent the new one from starting, or simply cause trouble.
+
+ Example:
+
+ global
+ grace 10s
+
+ # Returns 200 OK until stopping is set via SIGUSR1
+ frontend ext-check
+ bind :9999
+ monitor-uri /ext-check
+ monitor fail if { stopping }
+
+ Please note that a more flexible and durable approach would instead consist
+ for an orchestration system in setting a global variable from the CLI, use
+ that variable to respond to external checks, then after a delay send the
+ SIGUSR1 signal.
+
+ Example:
+
+ # Returns 200 OK until proc.stopping is set to non-zero. May be done
+ # from HTTP using set-var(proc.stopping) or from the CLI using:
+ # > set var proc.stopping int(1)
+ frontend ext-check
+ bind :9999
+ monitor-uri /ext-check
+ monitor fail if { var(proc.stopping) -m int gt 0 }
+
+ See also: hard-stop-after, monitor
+
+group <group name>
+ Similar to "gid" but uses the GID of group name <group name> from /etc/group.
+ See also "gid" and "user".
+
+h1-accept-payload-with-any-method
+ Does not reject HTTP/1.0 GET/HEAD/DELETE requests with a payload.
+
+ While It is explicitly allowed in HTTP/1.1, HTTP/1.0 is not clear on this
+ point and some old servers don't expect any payload and never look for body
+ length (via Content-Length or Transfer-Encoding headers). It means that some
+ intermediaries may properly handle the payload for HTTP/1.0 GET/HEAD/DELETE
+ requests, while some others may totally ignore it. That may lead to security
+ issues because a request smuggling attack is possible. Thus, by default,
+ HAProxy rejects HTTP/1.0 GET/HEAD/DELETE requests with a payload.
+
+ However, it may be an issue with some old clients. In this case, this global
+ option may be set.
+
+h1-case-adjust <from> <to>
+ Defines the case adjustment to apply, when enabled, to the header name
+ <from>, to change it to <to> before sending it to HTTP/1 clients or
+ servers. <from> must be in lower case, and <from> and <to> must not differ
+ except for their case. It may be repeated if several header names need to be
+ adjusted. Duplicate entries are not allowed. If a lot of header names have to
+ be adjusted, it might be more convenient to use "h1-case-adjust-file".
+ Please note that no transformation will be applied unless "option
+ h1-case-adjust-bogus-client" or "option h1-case-adjust-bogus-server" is
+ specified in a proxy.
+
+ There is no standard case for header names because, as stated in RFC7230,
+ they are case-insensitive. So applications must handle them in a case-
+ insensitive manner. But some bogus applications violate the standards and
+ erroneously rely on the cases most commonly used by browsers. This problem
+ becomes critical with HTTP/2 because all header names must be exchanged in
+ lower case, and HAProxy follows the same convention. All header names are
+ sent in lower case to clients and servers, regardless of the HTTP version.
+
+ Applications which fail to properly process requests or responses may require
+ to temporarily use such workarounds to adjust header names sent to them for
+ the time it takes the application to be fixed. Please note that an
+ application which requires such workarounds might be vulnerable to content
+ smuggling attacks and must absolutely be fixed.
+
+ Example:
+ global
+ h1-case-adjust content-length Content-Length
+
+ See "h1-case-adjust-file", "option h1-case-adjust-bogus-client" and
+ "option h1-case-adjust-bogus-server".
+
+h1-case-adjust-file <hdrs-file>
+ Defines a file containing a list of key/value pairs used to adjust the case
+ of some header names before sending them to HTTP/1 clients or servers. The
+ file <hdrs-file> must contain 2 header names per line. The first one must be
+ in lower case and both must not differ except for their case. Lines which
+ start with '#' are ignored, just like empty lines. Leading and trailing tabs
+ and spaces are stripped. Duplicate entries are not allowed. Please note that
+ no transformation will be applied unless "option h1-case-adjust-bogus-client"
+ or "option h1-case-adjust-bogus-server" is specified in a proxy.
+
+ If this directive is repeated, only the last one will be processed. It is an
+ alternative to the directive "h1-case-adjust" if a lot of header names need
+ to be adjusted. Please read the risks associated with using this.
+
+ See "h1-case-adjust", "option h1-case-adjust-bogus-client" and
+ "option h1-case-adjust-bogus-server".
+
+h2-workaround-bogus-websocket-clients
+ This disables the announcement of the support for h2 websockets to clients.
+ This can be use to overcome clients which have issues when implementing the
+ relatively fresh RFC8441, such as Firefox 88. To allow clients to
+ automatically downgrade to http/1.1 for the websocket tunnel, specify h2
+ support on the bind line using "alpn" without an explicit "proto" keyword. If
+ this statement was previously activated, this can be disabled by prefixing
+ the keyword with "no'.
+
+hard-stop-after <time>
+ Defines the maximum time allowed to perform a clean soft-stop.
+
+ Arguments :
+ <time> is the maximum time (by default in milliseconds) for which the
+ instance will remain alive when a soft-stop is received via the
+ SIGUSR1 signal.
+
+ This may be used to ensure that the instance will quit even if connections
+ remain opened during a soft-stop (for example with long timeouts for a proxy
+ in tcp mode). It applies both in TCP and HTTP mode.
+
+ Example:
+ global
+ hard-stop-after 30s
+
+ See also: grace
+
+insecure-fork-wanted
+ By default HAProxy tries hard to prevent any thread and process creation
+ after it starts. Doing so is particularly important when using Lua files of
+ uncertain origin, and when experimenting with development versions which may
+ still contain bugs whose exploitability is uncertain. And generally speaking
+ it's good hygiene to make sure that no unexpected background activity can be
+ triggered by traffic. But this prevents external checks from working, and may
+ break some very specific Lua scripts which actively rely on the ability to
+ fork. This option is there to disable this protection. Note that it is a bad
+ idea to disable it, as a vulnerability in a library or within HAProxy itself
+ will be easier to exploit once disabled. In addition, forking from Lua or
+ anywhere else is not reliable as the forked process may randomly embed a lock
+ set by another thread and never manage to finish an operation. As such it is
+ highly recommended that this option is never used and that any workload
+ requiring such a fork be reconsidered and moved to a safer solution (such as
+ agents instead of external checks). This option supports the "no" prefix to
+ disable it.
+
+insecure-setuid-wanted
+ HAProxy doesn't need to call executables at run time (except when using
+ external checks which are strongly recommended against), and is even expected
+ to isolate itself into an empty chroot. As such, there basically is no valid
+ reason to allow a setuid executable to be called without the user being fully
+ aware of the risks. In a situation where HAProxy would need to call external
+ checks and/or disable chroot, exploiting a vulnerability in a library or in
+ HAProxy itself could lead to the execution of an external program. On Linux
+ it is possible to lock the process so that any setuid bit present on such an
+ executable is ignored. This significantly reduces the risk of privilege
+ escalation in such a situation. This is what HAProxy does by default. In case
+ this causes a problem to an external check (for example one which would need
+ the "ping" command), then it is possible to disable this protection by
+ explicitly adding this directive in the global section. If enabled, it is
+ possible to turn it back off by prefixing it with the "no" keyword.
+
+issuers-chain-path <dir>
+ Assigns a directory to load certificate chain for issuer completion. All
+ files must be in PEM format. For certificates loaded with "crt" or "crt-list",
+ if certificate chain is not included in PEM (also commonly known as
+ intermediate certificate), HAProxy will complete chain if the issuer of the
+ certificate corresponds to the first certificate of the chain loaded with
+ "issuers-chain-path".
+ A "crt" file with PrivateKey+Certificate+IntermediateCA2+IntermediateCA1
+ could be replaced with PrivateKey+Certificate. HAProxy will complete the
+ chain if a file with IntermediateCA2+IntermediateCA1 is present in
+ "issuers-chain-path" directory. All other certificates with the same issuer
+ will share the chain in memory.
+
+limited-quic
+ This setting must be used to explicitly enable the QUIC listener bindings when
+ haproxy is compiled against a TLS/SSL stack without QUIC support, typically
+ OpenSSL. It has no effect when haproxy is compiled against a TLS/SSL stack
+ with QUIC support, quictls for instance. Note that QUIC 0-RTT is not supported
+ when this setting is set.
+
+localpeer <name>
+ Sets the local instance's peer name. It will be ignored if the "-L"
+ command line argument is specified or if used after "peers" section
+ definitions. In such cases, a warning message will be emitted during
+ the configuration parsing.
+
+ This option will also set the HAPROXY_LOCALPEER environment variable.
+ See also "-L" in the management guide and "peers" section below.
+
+log <target> [len <length>] [format <format>] [sample <ranges>:<sample_size>]
+ <facility> [max level [min level]]
+ Adds a global syslog server. Several global servers can be defined. They
+ will receive logs for starts and exits, as well as all logs from proxies
+ configured with "log global". See "log" option for proxies for more details.
+
+log-send-hostname [<string>]
+ Sets the hostname field in the syslog header. If optional "string" parameter
+ is set the header is set to the string contents, otherwise uses the hostname
+ of the system. Generally used if one is not relaying logs through an
+ intermediate syslog server or for simply customizing the hostname printed in
+ the logs.
+
+log-tag <string>
+ Sets the tag field in the syslog header to this string. It defaults to the
+ program name as launched from the command line, which usually is "haproxy".
+ Sometimes it can be useful to differentiate between multiple processes
+ running on the same host. See also the per-proxy "log-tag" directive.
+
+lua-load <file> [ <arg1> [ <arg2> [ ... ] ] ]
+ This global directive loads and executes a Lua file in the shared context
+ that is visible to all threads. Any variable set in such a context is visible
+ from any thread. This is the easiest and recommended way to load Lua programs
+ but it will not scale well if a lot of Lua calls are performed, as only one
+ thread may be running on the global state at a time. A program loaded this
+ way will always see 0 in the "core.thread" variable. This directive can be
+ used multiple times.
+
+ args are available in the lua file using the code below in the body of the
+ file. Do not forget that Lua arrays start at index 1. A "local" variable
+ declared in a file is available in the entire file and not available on
+ other files.
+
+ local args = table.pack(...)
+
+lua-load-per-thread <file> [ <arg1> [ <arg2> [ ... ] ] ]
+ This global directive loads and executes a Lua file into each started thread.
+ Any global variable has a thread-local visibility so that each thread could
+ see a different value. As such it is strongly recommended not to use global
+ variables in programs loaded this way. An independent copy is loaded and
+ initialized for each thread, everything is done sequentially and in the
+ thread's numeric order from 1 to nbthread. If some operations need to be
+ performed only once, the program should check the "core.thread" variable to
+ figure what thread is being initialized. Programs loaded this way will run
+ concurrently on all threads and will be highly scalable. This is the
+ recommended way to load simple functions that register sample-fetches,
+ converters, actions or services once it is certain the program doesn't depend
+ on global variables. For the sake of simplicity, the directive is available
+ even if only one thread is used and even if threads are disabled (in which
+ case it will be equivalent to lua-load). This directive can be used multiple
+ times.
+
+ See lua-load for usage of args.
+
+lua-prepend-path <string> [<type>]
+ Prepends the given string followed by a semicolon to Lua's package.<type>
+ variable.
+ <type> must either be "path" or "cpath". If <type> is not given it defaults
+ to "path".
+
+ Lua's paths are semicolon delimited lists of patterns that specify how the
+ `require` function attempts to find the source file of a library. Question
+ marks (?) within a pattern will be replaced by module name. The path is
+ evaluated left to right. This implies that paths that are prepended later
+ will be checked earlier.
+
+ As an example by specifying the following path:
+
+ lua-prepend-path /usr/share/haproxy-lua/?/init.lua
+ lua-prepend-path /usr/share/haproxy-lua/?.lua
+
+ When `require "example"` is being called Lua will first attempt to load the
+ /usr/share/haproxy-lua/example.lua script, if that does not exist the
+ /usr/share/haproxy-lua/example/init.lua will be attempted and the default
+ paths if that does not exist either.
+
+ See https://www.lua.org/pil/8.1.html for the details within the Lua
+ documentation.
+
+master-worker [no-exit-on-failure]
+ Master-worker mode. It is equivalent to the command line "-W" argument.
+ This mode will launch a "master" which will monitor the "workers". Using
+ this mode, you can reload HAProxy directly by sending a SIGUSR2 signal to
+ the master. The master-worker mode is compatible either with the foreground
+ or daemon mode.
+
+ By default, if a worker exits with a bad return code, in the case of a
+ segfault for example, all workers will be killed, and the master will leave.
+ It is convenient to combine this behavior with Restart=on-failure in a
+ systemd unit file in order to relaunch the whole process. If you don't want
+ this behavior, you must use the keyword "no-exit-on-failure".
+
+ See also "-W" in the management guide.
+
+mworker-max-reloads <number>
+ In master-worker mode, this option limits the number of time a worker can
+ survive to a reload. If the worker did not leave after a reload, once its
+ number of reloads is greater than this number, the worker will receive a
+ SIGTERM. This option helps to keep under control the number of workers.
+ See also "show proc" in the Management Guide.
+
+nbthread <number>
+ This setting is only available when support for threads was built in. It
+ makes HAProxy run on <number> threads. "nbthread" also works when HAProxy is
+ started in foreground. On some platforms supporting CPU affinity, the default
+ "nbthread" value is automatically set to the number of CPUs the process is
+ bound to upon startup. This means that the thread count can easily be
+ adjusted from the calling process using commands like "taskset" or "cpuset".
+ Otherwise, this value defaults to 1. The default value is reported in the
+ output of "haproxy -vv".
+
+no-quic
+ Disable QUIC transport protocol. All the QUIC listeners will still be created.
+ But they will not bind their addresses. Hence, no QUIC traffic will be
+ processed by haproxy. See also "quic_enabled" sample fetch.
+
+numa-cpu-mapping
+ If running on a NUMA-aware platform, HAProxy inspects on startup the CPU
+ topology of the machine. If a multi-socket machine is detected, the affinity
+ is automatically calculated to run on the CPUs of a single node. This is done
+ in order to not suffer from the performance penalties caused by the
+ inter-socket bus latency. However, if the applied binding is non optimal on a
+ particular architecture, it can be disabled with the statement 'no
+ numa-cpu-mapping'. This automatic binding is also not applied if a nbthread
+ statement is present in the configuration, or the affinity of the process is
+ already specified, for example via the 'cpu-map' directive or the taskset
+ utility.
+
+pidfile <pidfile>
+ Writes PIDs of all daemons into file <pidfile> when daemon mode or writes PID
+ of master process into file <pidfile> when master-worker mode. This option is
+ equivalent to the "-p" command line argument. The file must be accessible to
+ the user starting the process. See also "daemon" and "master-worker".
+
+pp2-never-send-local
+ A bug in the PROXY protocol v2 implementation was present in HAProxy up to
+ version 2.1, causing it to emit a PROXY command instead of a LOCAL command
+ for health checks. This is particularly minor but confuses some servers'
+ logs. Sadly, the bug was discovered very late and revealed that some servers
+ which possibly only tested their PROXY protocol implementation against
+ HAProxy fail to properly handle the LOCAL command, and permanently remain in
+ the "down" state when HAProxy checks them. When this happens, it is possible
+ to enable this global option to revert to the older (bogus) behavior for the
+ time it takes to contact the affected components' vendors and get them fixed.
+ This option is disabled by default and acts on all servers having the
+ "send-proxy-v2" statement.
+
+presetenv <name> <value>
+ Sets environment variable <name> to value <value>. If the variable exists, it
+ is NOT overwritten. The changes immediately take effect so that the next line
+ in the configuration file sees the new value. See also "setenv", "resetenv",
+ and "unsetenv".
+
+prealloc-fd
+ Performs a one-time open of the maximum file descriptor which results in a
+ pre-allocation of the kernel's data structures. This prevents short pauses
+ when nbthread>1 and HAProxy opens a file descriptor which requires the kernel
+ to expand its data structures.
+
+resetenv [<name> ...]
+ Removes all environment variables except the ones specified in argument. It
+ allows to use a clean controlled environment before setting new values with
+ setenv or unsetenv. Please note that some internal functions may make use of
+ some environment variables, such as time manipulation functions, but also
+ OpenSSL or even external checks. This must be used with extreme care and only
+ after complete validation. The changes immediately take effect so that the
+ next line in the configuration file sees the new environment. See also
+ "setenv", "presetenv", and "unsetenv".
+
+server-state-base <directory>
+ Specifies the directory prefix to be prepended in front of all servers state
+ file names which do not start with a '/'. See also "server-state-file",
+ "load-server-state-from-file" and "server-state-file-name".
+
+server-state-file <file>
+ Specifies the path to the file containing state of servers. If the path starts
+ with a slash ('/'), it is considered absolute, otherwise it is considered
+ relative to the directory specified using "server-state-base" (if set) or to
+ the current directory. Before reloading HAProxy, it is possible to save the
+ servers' current state using the stats command "show servers state". The
+ output of this command must be written in the file pointed by <file>. When
+ starting up, before handling traffic, HAProxy will read, load and apply state
+ for each server found in the file and available in its current running
+ configuration. See also "server-state-base" and "show servers state",
+ "load-server-state-from-file" and "server-state-file-name"
+
+set-dumpable
+ This option is better left disabled by default and enabled only upon a
+ developer's request. If it has been enabled, it may still be forcibly
+ disabled by prefixing it with the "no" keyword. It has no impact on
+ performance nor stability but will try hard to re-enable core dumps that were
+ possibly disabled by file size limitations (ulimit -f), core size limitations
+ (ulimit -c), or "dumpability" of a process after changing its UID/GID (such
+ as /proc/sys/fs/suid_dumpable on Linux). Core dumps might still be limited by
+ the current directory's permissions (check what directory the file is started
+ from), the chroot directory's permission (it may be needed to temporarily
+ disable the chroot directive or to move it to a dedicated writable location),
+ or any other system-specific constraint. For example, some Linux flavours are
+ notorious for replacing the default core file with a path to an executable
+ not even installed on the system (check /proc/sys/kernel/core_pattern). Often,
+ simply writing "core", "core.%p" or "/var/log/core/core.%p" addresses the
+ issue. When trying to enable this option waiting for a rare issue to
+ re-appear, it's often a good idea to first try to obtain such a dump by
+ issuing, for example, "kill -11" to the "haproxy" process and verify that it
+ leaves a core where expected when dying.
+
+set-var <var-name> <expr>
+ Sets the process-wide variable '<var-name>' to the result of the evaluation
+ of the sample expression <expr>. The variable '<var-name>' may only be a
+ process-wide variable (using the 'proc.' prefix). It works exactly like the
+ 'set-var' action in TCP or HTTP rules except that the expression is evaluated
+ at configuration parsing time and that the variable is instantly set. The
+ sample fetch functions and converters permitted in the expression are only
+ those using internal data, typically 'int(value)' or 'str(value)'. It is
+ possible to reference previously allocated variables as well. These variables
+ will then be readable (and modifiable) from the regular rule sets.
+
+ Example:
+ global
+ set-var proc.current_state str(primary)
+ set-var proc.prio int(100)
+ set-var proc.threshold int(200),sub(proc.prio)
+
+set-var-fmt <var-name> <fmt>
+ Sets the process-wide variable '<var-name>' to the string resulting from the
+ evaluation of the log-format <fmt>. The variable '<var-name>' may only be a
+ process-wide variable (using the 'proc.' prefix). It works exactly like the
+ 'set-var-fmt' action in TCP or HTTP rules except that the expression is
+ evaluated at configuration parsing time and that the variable is instantly
+ set. The sample fetch functions and converters permitted in the expression
+ are only those using internal data, typically 'int(value)' or 'str(value)'.
+ It is possible to reference previously allocated variables as well. These
+ variables will then be readable (and modifiable) from the regular rule sets.
+ Please see section 8.2.4 for details on the log-format syntax.
+
+ Example:
+ global
+ set-var-fmt proc.current_state "primary"
+ set-var-fmt proc.bootid "%pid|%t"
+
+setcap <name>[,<name>...]
+ Sets a list of capabilities that must be preserved when starting with uid 0
+ and switching to a non-zero uid. By default all permissions are lost by the
+ uid switch, but some are often needed when trying connecting to a server from
+ a foreign address during transparent proxying, or when binding to a port
+ below 1024, e.g. when using "tune.quic.socket-owner connection", resulting in
+ setups running entirely under uid 0. Setting capabilities generally is a
+ safer alternative, as only the required capabilities will be preserved. The
+ feature is OS-specific and only enabled on Linux when USE_LINUX_CAP=1 is set
+ at build time. The list of supported capabilities also depends on the OS and
+ is enumerated by the error message displayed when an invalid capability name
+ or an empty one is passed. Multiple capabilities may be passed, delimited by
+ commas. Among those commonly used, "cap_net_raw" allows to transparently bind
+ to a foreign address, and "cap_net_bind_service" allows to bind to a
+ privileged port and may be used by QUIC.
+
+setenv <name> <value>
+ Sets environment variable <name> to value <value>. If the variable exists, it
+ is overwritten. The changes immediately take effect so that the next line in
+ the configuration file sees the new value. See also "presetenv", "resetenv",
+ and "unsetenv".
+
+ssl-default-bind-ciphers <ciphers>
+ This setting is only available when support for OpenSSL was built in. It sets
+ the default string describing the list of cipher algorithms ("cipher suite")
+ that are negotiated during the SSL/TLS handshake up to TLSv1.2 for all
+ "bind" lines which do not explicitly define theirs. The format of the string
+ is defined in "man 1 ciphers" from OpenSSL man pages. For background
+ information and recommendations see e.g.
+ (https://wiki.mozilla.org/Security/Server_Side_TLS) and
+ (https://mozilla.github.io/server-side-tls/ssl-config-generator/). For TLSv1.3
+ cipher configuration, please check the "ssl-default-bind-ciphersuites" keyword.
+ Please check the "bind" keyword for more information.
+
+ssl-default-bind-ciphersuites <ciphersuites>
+ This setting is only available when support for OpenSSL was built in and
+ OpenSSL 1.1.1 or later was used to build HAProxy. It sets the default string
+ describing the list of cipher algorithms ("cipher suite") that are negotiated
+ during the TLSv1.3 handshake for all "bind" lines which do not explicitly define
+ theirs. The format of the string is defined in
+ "man 1 ciphers" from OpenSSL man pages under the section "ciphersuites". For
+ cipher configuration for TLSv1.2 and earlier, please check the
+ "ssl-default-bind-ciphers" keyword. Please check the "bind" keyword for more
+ information.
+
+ssl-default-bind-client-sigalgs <sigalgs>
+ This setting is only available when support for OpenSSL was built in. It sets
+ the default string describing the list of signature algorithms related to
+ client authentication for all "bind" lines which do not explicitly define
+ theirs. The format of the string is a colon-delimited list of signature
+ algorithms. Each signature algorithm can use one of two forms: TLS1.3 signature
+ scheme names ("rsa_pss_rsae_sha256") or the public key algorithm + digest form
+ ("ECDSA+SHA256"). A list can contain both forms. For more information on the
+ format, see SSL_CTX_set1_client_sigalgs(3). A list of signature algorithms is
+ also available in RFC8446 section 4.2.3 and in OpenSSL in the ssl/t1_lib.c
+ file. This setting is not applicable to TLSv1.1 and earlier versions of the
+ protocol as the signature algorithms aren't separately negotiated in these
+ versions. It is not recommended to change this setting unless compatibility
+ with a middlebox is required.
+
+ssl-default-bind-curves <curves>
+ This setting is only available when support for OpenSSL was built in. It sets
+ the default string describing the list of elliptic curves algorithms ("curve
+ suite") that are negotiated during the SSL/TLS handshake with ECDHE. The format
+ of the string is a colon-delimited list of curve name.
+ Please check the "bind" keyword for more information.
+
+ssl-default-bind-options [<option>]...
+ This setting is only available when support for OpenSSL was built in. It sets
+ default ssl-options to force on all "bind" lines. Please check the "bind"
+ keyword to see available options.
+
+ Example:
+ global
+ ssl-default-bind-options ssl-min-ver TLSv1.0 no-tls-tickets
+
+ssl-default-bind-sigalgs <sigalgs>
+ This setting is only available when support for OpenSSL was built in. It
+ sets the default string describing the list of signature algorithms that
+ are negotiated during the TLSv1.2 and TLSv1.3 handshake for all "bind" lines
+ which do not explicitly define theirs. The format of the string is a
+ colon-delimited list of signature algorithms. Each signature algorithm can
+ use one of two forms: TLS1.3 signature scheme names ("rsa_pss_rsae_sha256")
+ or the public key algorithm + digest form ("ECDSA+SHA256"). A list
+ can contain both forms. For more information on the format,
+ see SSL_CTX_set1_sigalgs(3). A list of signature algorithms is also
+ available in RFC8446 section 4.2.3 and in OpenSSL in the ssl/t1_lib.c file.
+ This setting is not applicable to TLSv1.1 and earlier versions of the
+ protocol as the signature algorithms aren't separately negotiated in these
+ versions. It is not recommended to change this setting unless compatibility
+ with a middlebox is required.
+
+ssl-default-server-ciphers <ciphers>
+ This setting is only available when support for OpenSSL was built in. It
+ sets the default string describing the list of cipher algorithms that are
+ negotiated during the SSL/TLS handshake up to TLSv1.2 with the server,
+ for all "server" lines which do not explicitly define theirs. The format of
+ the string is defined in "man 1 ciphers" from OpenSSL man pages. For background
+ information and recommendations see e.g.
+ (https://wiki.mozilla.org/Security/Server_Side_TLS) and
+ (https://mozilla.github.io/server-side-tls/ssl-config-generator/).
+ For TLSv1.3 cipher configuration, please check the
+ "ssl-default-server-ciphersuites" keyword. Please check the "server" keyword
+ for more information.
+
+ssl-default-server-ciphersuites <ciphersuites>
+ This setting is only available when support for OpenSSL was built in and
+ OpenSSL 1.1.1 or later was used to build HAProxy. It sets the default
+ string describing the list of cipher algorithms that are negotiated during
+ the TLSv1.3 handshake with the server, for all "server" lines which do not
+ explicitly define theirs. The format of the string is defined in
+ "man 1 ciphers" from OpenSSL man pages under the section "ciphersuites". For
+ cipher configuration for TLSv1.2 and earlier, please check the
+ "ssl-default-server-ciphers" keyword. Please check the "server" keyword for
+ more information.
+
+ssl-default-server-client-sigalgs <sigalgs>
+ This setting is only available when support for OpenSSL was built in. It sets
+ the default string describing the list of signature algorithms related to
+ client authentication for all "server" lines which do not explicitly define
+ theirs. The format of the string is a colon-delimited list of signature
+ algorithms. Each signature algorithm can use one of two forms: TLS1.3 signature
+ scheme names ("rsa_pss_rsae_sha256") or the public key algorithm + digest form
+ ("ECDSA+SHA256"). A list can contain both forms. For more information on the
+ format, see SSL_CTX_set1_client_sigalgs(3). A list of signature algorithms is
+ also available in RFC8446 section 4.2.3 and in OpenSSL in the ssl/t1_lib.c
+ file. This setting is not applicable to TLSv1.1 and earlier versions of the
+ protocol as the signature algorithms aren't separately negotiated in these
+ versions. It is not recommended to change this setting unless compatibility
+ with a middlebox is required.
+
+ssl-default-server-curves <curves>
+ This setting is only available when support for OpenSSL was built in. It sets
+ the default string describing the list of elliptic curves algorithms ("curve
+ suite") that are negotiated during the SSL/TLS handshake with ECDHE. The format
+ of the string is a colon-delimited list of curve name.
+ Please check the "server" keyword for more information.
+
+ssl-default-server-options [<option>]...
+ This setting is only available when support for OpenSSL was built in. It sets
+ default ssl-options to force on all "server" lines. Please check the "server"
+ keyword to see available options.
+
+ssl-default-server-sigalgs <sigalgs>
+ This setting is only available when support for OpenSSL was built in. It
+ sets the default string describing the list of signature algorithms that
+ are negotiated during the TLSv1.2 and TLSv1.3 handshake for all "server" lines
+ which do not explicitly define theirs. The format of the string is a
+ colon-delimited list of signature algorithms. Each signature algorithm can
+ use one of two forms: TLS1.3 signature scheme names ("rsa_pss_rsae_sha256")
+ or the public key algorithm + digest form ("ECDSA+SHA256"). A list
+ can contain both forms. For more information on the format,
+ see SSL_CTX_set1_sigalgs(3). A list of signature algorithms is also
+ available in RFC8446 section 4.2.3 and in OpenSSL in the ssl/t1_lib.c file.
+ This setting is not applicable to TLSv1.1 and earlier versions of the
+ protocol as the signature algorithms aren't separately negotiated in these
+ versions. It is not recommended to change this setting unless compatibility
+ with a middlebox is required.
+
+ssl-dh-param-file <file>
+ This setting is only available when support for OpenSSL was built in. It sets
+ the default DH parameters that are used during the SSL/TLS handshake when
+ ephemeral Diffie-Hellman (DHE) key exchange is used, for all "bind" lines
+ which do not explicitly define theirs. It will be overridden by custom DH
+ parameters found in a bind certificate file if any. If custom DH parameters
+ are not specified either by using ssl-dh-param-file or by setting them
+ directly in the certificate file, DHE ciphers will not be used, unless
+ tune.ssl.default-dh-param is set. In this latter case, pre-defined DH
+ parameters of the specified size will be used. Custom parameters are known to
+ be more secure and therefore their use is recommended.
+ Custom DH parameters may be generated by using the OpenSSL command
+ "openssl dhparam <size>", where size should be at least 2048, as 1024-bit DH
+ parameters should not be considered secure anymore.
+
+ssl-propquery <query>
+ This setting is only available when support for OpenSSL was built in and when
+ OpenSSL's version is at least 3.0. It allows to define a default property
+ string used when fetching algorithms in providers. It behave the same way as
+ the openssl propquery option and it follows the same syntax (described in
+ https://www.openssl.org/docs/man3.0/man7/property.html). For instance, if you
+ have two providers loaded, the foo one and the default one, the propquery
+ "?provider=foo" allows to pick the algorithm implementations provided by the
+ foo provider by default, and to fallback on the default provider's one if it
+ was not found.
+
+ssl-provider <name>
+ This setting is only available when support for OpenSSL was built in and when
+ OpenSSL's version is at least 3.0. It allows to load a provider during init.
+ If loading is successful, any capabilities provided by the loaded provider
+ might be used by HAProxy. Multiple 'ssl-provider' options can be specified in
+ a configuration file. The providers will be loaded in their order of
+ appearance.
+
+ Please note that loading a provider explicitly prevents OpenSSL from loading
+ the 'default' provider automatically. OpenSSL also allows to define the
+ providers that should be loaded directly in its configuration file
+ (openssl.cnf for instance) so it is not necessary to use this 'ssl-provider'
+ option to load providers. The "show ssl providers" CLI command can be used to
+ show all the providers that were successfully loaded.
+
+ The default search path of OpenSSL provider can be found in the output of the
+ "openssl version -a" command. If the provider is in another directory, you
+ can set the OPENSSL_MODULES environment variable, which takes the directory
+ where your provider can be found.
+
+ See also "ssl-propquery" and "ssl-provider-path".
+
+ssl-provider-path <path>
+ This setting is only available when support for OpenSSL was built in and when
+ OpenSSL's version is at least 3.0. It allows to specify the search path that
+ is to be used by OpenSSL for looking for providers. It behaves the same way
+ as the OPENSSL_MODULES environment variable. It will be used for any
+ following 'ssl-provider' option or until a new 'ssl-provider-path' is
+ defined.
+ See also "ssl-provider".
+
+ssl-load-extra-del-ext
+ This setting allows to configure the way HAProxy does the lookup for the
+ extra SSL files. By default HAProxy adds a new extension to the filename.
+ (ex: with "foobar.crt" load "foobar.crt.key"). With this option enabled,
+ HAProxy removes the extension before adding the new one (ex: with
+ "foobar.crt" load "foobar.key").
+
+ Your crt file must have a ".crt" extension for this option to work.
+
+ This option is not compatible with bundle extensions (.ecdsa, .rsa. .dsa)
+ and won't try to remove them.
+
+ This option is disabled by default. See also "ssl-load-extra-files".
+
+ssl-load-extra-files <none|all|bundle|sctl|ocsp|issuer|key>*
+ This setting alters the way HAProxy will look for unspecified files during
+ the loading of the SSL certificates. This option applies to certificates
+ associated to "bind" lines as well as "server" lines but some of the extra
+ files will not have any functional impact for "server" line certificates.
+
+ By default, HAProxy discovers automatically a lot of files not specified in
+ the configuration, and you may want to disable this behavior if you want to
+ optimize the startup time.
+
+ "none": Only load the files specified in the configuration. Don't try to load
+ a certificate bundle if the file does not exist. In the case of a directory,
+ it won't try to bundle the certificates if they have the same basename.
+
+ "all": This is the default behavior, it will try to load everything,
+ bundles, sctl, ocsp, issuer, key.
+
+ "bundle": When a file specified in the configuration does not exist, HAProxy
+ will try to load a "cert bundle". Certificate bundles are only managed on the
+ frontend side and will not work for backend certificates.
+
+ Starting from HAProxy 2.3, the bundles are not loaded in the same OpenSSL
+ certificate store, instead it will loads each certificate in a separate
+ store which is equivalent to declaring multiple "crt". OpenSSL 1.1.1 is
+ required to achieve this. Which means that bundles are now used only for
+ backward compatibility and are not mandatory anymore to do an hybrid RSA/ECC
+ bind configuration.
+
+ To associate these PEM files into a "cert bundle" that is recognized by
+ HAProxy, they must be named in the following way: All PEM files that are to
+ be bundled must have the same base name, with a suffix indicating the key
+ type. Currently, three suffixes are supported: rsa, dsa and ecdsa. For
+ example, if www.example.com has two PEM files, an RSA file and an ECDSA
+ file, they must be named: "example.pem.rsa" and "example.pem.ecdsa". The
+ first part of the filename is arbitrary; only the suffix matters. To load
+ this bundle into HAProxy, specify the base name only:
+
+ Example : bind :8443 ssl crt example.pem
+
+ Note that the suffix is not given to HAProxy; this tells HAProxy to look for
+ a cert bundle.
+
+ HAProxy will load all PEM files in the bundle as if they were configured
+ separately in several "crt".
+
+ The bundle loading does not have an impact anymore on the directory loading
+ since files are loading separately.
+
+ On the CLI, bundles are seen as separate files, and the bundle extension is
+ required to commit them.
+
+ OCSP files (.ocsp), issuer files (.issuer), Certificate Transparency (.sctl)
+ as well as private keys (.key) are supported with multi-cert bundling.
+
+ "sctl": Try to load "<basename>.sctl" for each crt keyword. If provided for
+ a backend certificate, it will be loaded but will not have any functional
+ impact.
+
+ "ocsp": Try to load "<basename>.ocsp" for each crt keyword. If provided for
+ a backend certificate, it will be loaded but will not have any functional
+ impact.
+
+ "issuer": Try to load "<basename>.issuer" if the issuer of the OCSP file is
+ not provided in the PEM file. If provided for a backend certificate, it will
+ be loaded but will not have any functional impact.
+
+ "key": If the private key was not provided by the PEM file, try to load a
+ file "<basename>.key" containing a private key.
+
+ The default behavior is "all".
+
+ Example:
+ ssl-load-extra-files bundle sctl
+ ssl-load-extra-files sctl ocsp issuer
+ ssl-load-extra-files none
+
+ See also: "crt", section 5.1 about bind options and section 5.2 about server
+ options.
+
+ssl-server-verify [none|required]
+ The default behavior for SSL verify on servers side. If specified to 'none',
+ servers certificates are not verified. The default is 'required' except if
+ forced using cmdline option '-dV'.
+
+ssl-skip-self-issued-ca
+ Self issued CA, aka x509 root CA, is the anchor for chain validation: as a
+ server is useless to send it, client must have it. Standard configuration
+ need to not include such CA in PEM file. This option allows you to keep such
+ CA in PEM file without sending it to the client. Use case is to provide
+ issuer for ocsp without the need for '.issuer' file and be able to share it
+ with 'issuers-chain-path'. This concerns all certificates without intermediate
+ certificates. It's useless for BoringSSL, .issuer is ignored because ocsp
+ bits does not need it. Requires at least OpenSSL 1.0.2.
+
+stats maxconn <connections>
+ By default, the stats socket is limited to 10 concurrent connections. It is
+ possible to change this value with "stats maxconn".
+
+stats socket [<address:port>|<path>] [param*]
+ Binds a UNIX socket to <path> or a TCPv4/v6 address to <address:port>.
+ Connections to this socket will return various statistics outputs and even
+ allow some commands to be issued to change some runtime settings. Please
+ consult section 9.3 "Unix Socket commands" of Management Guide for more
+ details.
+
+ All parameters supported by "bind" lines are supported, for instance to
+ restrict access to some users or their access rights. Please consult
+ section 5.1 for more information.
+
+stats timeout <timeout, in milliseconds>
+ The default timeout on the stats socket is set to 10 seconds. It is possible
+ to change this value with "stats timeout". The value must be passed in
+ milliseconds, or be suffixed by a time unit among { us, ms, s, m, h, d }.
+
+strict-limits
+ Makes process fail at startup when a setrlimit fails. HAProxy tries to set the
+ best setrlimit according to what has been calculated. If it fails, it will
+ emit a warning. This option is here to guarantee an explicit failure of
+ HAProxy when those limits fail. It is enabled by default. It may still be
+ forcibly disabled by prefixing it with the "no" keyword.
+
+thread-group <group> [<thread-range>...]
+ This setting is only available when support for threads was built in. It
+ enumerates the list of threads that will compose thread group <group>.
+ Thread numbers and group numbers start at 1. Thread ranges are defined either
+ using a single thread number at once, or by specifying the lower and upper
+ bounds delimited by a dash '-' (e.g. "1-16"). Unassigned threads will be
+ automatically assigned to unassigned thread groups, and thread groups
+ defined with this directive will never receive more threads than those
+ defined. Defining the same group multiple times overrides previous
+ definitions with the new one. See also "nbthread" and "thread-groups".
+
+thread-groups <number>
+ This setting is only available when support for threads was built in. It
+ makes HAProxy split its threads into <number> independent groups. At the
+ moment, the default value is 1. Thread groups make it possible to reduce
+ sharing between threads to limit contention, at the expense of some extra
+ configuration efforts. It is also the only way to use more than 64 threads
+ since up to 64 threads per group may be configured. The maximum number of
+ groups is configured at compile time and defaults to 16. See also "nbthread".
+
+trace <args...>
+ This command configures one "trace" subsystem statement. Each of them can be
+ found in the management manual, and follow the exact same syntax. Only one
+ statement per line is permitted (i.e. if some long trace configurations using
+ semi-colons are to be imported, they must be placed one per line). Any output
+ that the "trace" command would produce will be emitted during the parsing
+ step of the section. Most of the time these will be errors and warnings, but
+ certain incomplete commands might list permissible choices. This command is
+ not meant for regular use, it will generally only be suggested by developers
+ along complex debugging sessions. For this reason it is internally marked as
+ experimental, meaning that "expose-experimental-directives" must appear on a
+ line before any "trace" statement. Note that these directives are parsed on
+ the fly, so referencing a ring buffer that is only declared further will not
+ work. For such use cases it is suggested to place another "global" section
+ with only the "trace" statements after the declaration of that ring. It is
+ important to keep in mind that depending on the trace level and details,
+ enabling traces can severely degrade the global performance. Please refer to
+ the management manual for the statements syntax.
+
+uid <number>
+ Changes the process's user ID to <number>. It is recommended that the user ID
+ is dedicated to HAProxy or to a small set of similar daemons. HAProxy must
+ be started with superuser privileges in order to be able to switch to another
+ one. See also "gid" and "user".
+
+ulimit-n <number>
+ Sets the maximum number of per-process file-descriptors to <number>. By
+ default, it is automatically computed, so it is recommended not to use this
+ option. If the intent is only to limit the number of file descriptors, better
+ use "fd-hard-limit" instead.
+
+ Note that the dynamic servers are not taken into account in this automatic
+ resource calculation. If using a large number of them, it may be needed to
+ manually specify this value.
+
+ See also: fd-hard-limit, maxconn
+
+unix-bind [ prefix <prefix> ] [ mode <mode> ] [ user <user> ] [ uid <uid> ]
+ [ group <group> ] [ gid <gid> ]
+
+ Fixes common settings to UNIX listening sockets declared in "bind" statements.
+ This is mainly used to simplify declaration of those UNIX sockets and reduce
+ the risk of errors, since those settings are most commonly required but are
+ also process-specific. The <prefix> setting can be used to force all socket
+ path to be relative to that directory. This might be needed to access another
+ component's chroot. Note that those paths are resolved before HAProxy chroots
+ itself, so they are absolute. The <mode>, <user>, <uid>, <group> and <gid>
+ all have the same meaning as their homonyms used by the "bind" statement. If
+ both are specified, the "bind" statement has priority, meaning that the
+ "unix-bind" settings may be seen as process-wide default settings.
+
+unsetenv [<name> ...]
+ Removes environment variables specified in arguments. This can be useful to
+ hide some sensitive information that are occasionally inherited from the
+ user's environment during some operations. Variables which did not exist are
+ silently ignored so that after the operation, it is certain that none of
+ these variables remain. The changes immediately take effect so that the next
+ line in the configuration file will not see these variables. See also
+ "setenv", "presetenv", and "resetenv".
+
+user <user name>
+ Similar to "uid" but uses the UID of user name <user name> from /etc/passwd.
+ See also "uid" and "group".
+
+node <name>
+ Only letters, digits, hyphen and underscore are allowed, like in DNS names.
+
+ This statement is useful in HA configurations where two or more processes or
+ servers share the same IP address. By setting a different node-name on all
+ nodes, it becomes easy to immediately spot what server is handling the
+ traffic.
+
+wurfl-cache-size <size>
+ Sets the WURFL Useragent cache size. For faster lookups, already processed user
+ agents are kept in a LRU cache :
+ - "0" : no cache is used.
+ - <size> : size of lru cache in elements.
+
+ Please note that this option is only available when HAProxy has been compiled
+ with USE_WURFL=1.
+
+wurfl-data-file <file path>
+ The path of the WURFL data file to provide device detection services. The
+ file should be accessible by HAProxy with relevant permissions.
+
+ Please note that this option is only available when HAProxy has been compiled
+ with USE_WURFL=1.
+
+wurfl-information-list [<capability>]*
+ A space-delimited list of WURFL capabilities, virtual capabilities, property
+ names we plan to use in injected headers. A full list of capability and
+ virtual capability names is available on the Scientiamobile website :
+
+ https://www.scientiamobile.com/wurflCapability
+
+ Valid WURFL properties are:
+ - wurfl_id Contains the device ID of the matched device.
+
+ - wurfl_root_id Contains the device root ID of the matched
+ device.
+
+ - wurfl_isdevroot Tells if the matched device is a root device.
+ Possible values are "TRUE" or "FALSE".
+
+ - wurfl_useragent The original useragent coming with this
+ particular web request.
+
+ - wurfl_api_version Contains a string representing the currently
+ used Libwurfl API version.
+
+ - wurfl_info A string containing information on the parsed
+ wurfl.xml and its full path.
+
+ - wurfl_last_load_time Contains the UNIX timestamp of the last time
+ WURFL has been loaded successfully.
+
+ - wurfl_normalized_useragent The normalized useragent.
+
+ Please note that this option is only available when HAProxy has been compiled
+ with USE_WURFL=1.
+
+wurfl-information-list-separator <char>
+ A char that will be used to separate values in a response header containing
+ WURFL results. If not set that a comma (',') will be used by default.
+
+ Please note that this option is only available when HAProxy has been compiled
+ with USE_WURFL=1.
+
+wurfl-patch-file [<file path>]
+ A list of WURFL patch file paths. Note that patches are loaded during startup
+ thus before the chroot.
+
+ Please note that this option is only available when HAProxy has been compiled
+ with USE_WURFL=1.
+
+3.2. Performance tuning
+-----------------------
+
+busy-polling
+ In some situations, especially when dealing with low latency on processors
+ supporting a variable frequency or when running inside virtual machines, each
+ time the process waits for an I/O using the poller, the processor goes back
+ to sleep or is offered to another VM for a long time, and it causes
+ excessively high latencies. This option provides a solution preventing the
+ processor from sleeping by always using a null timeout on the pollers. This
+ results in a significant latency reduction (30 to 100 microseconds observed)
+ at the expense of a risk to overheat the processor. It may even be used with
+ threads, in which case improperly bound threads may heavily conflict,
+ resulting in a worse performance and high values for the CPU stolen fields
+ in "show info" output, indicating which threads are misconfigured. It is
+ important not to let the process run on the same processor as the network
+ interrupts when this option is used. It is also better to avoid using it on
+ multiple CPU threads sharing the same core. This option is disabled by
+ default. If it has been enabled, it may still be forcibly disabled by
+ prefixing it with the "no" keyword. It is ignored by the "select" and
+ "poll" pollers.
+
+ This option is automatically disabled on old processes in the context of
+ seamless reload; it avoids too much cpu conflicts when multiple processes
+ stay around for some time waiting for the end of their current connections.
+
+max-spread-checks <delay in milliseconds>
+ By default, HAProxy tries to spread the start of health checks across the
+ smallest health check interval of all the servers in a farm. The principle is
+ to avoid hammering services running on the same server. But when using large
+ check intervals (10 seconds or more), the last servers in the farm take some
+ time before starting to be tested, which can be a problem. This parameter is
+ used to enforce an upper bound on delay between the first and the last check,
+ even if the servers' check intervals are larger. When servers run with
+ shorter intervals, their intervals will be respected though.
+
+maxcompcpuusage <number>
+ Sets the maximum CPU usage HAProxy can reach before stopping the compression
+ for new requests or decreasing the compression level of current requests.
+ It works like 'maxcomprate' but measures CPU usage instead of incoming data
+ bandwidth. The value is expressed in percent of the CPU used by HAProxy. A
+ value of 100 disable the limit. The default value is 100. Setting a lower
+ value will prevent the compression work from slowing the whole process down
+ and from introducing high latencies.
+
+maxcomprate <number>
+ Sets the maximum per-process input compression rate to <number> kilobytes
+ per second. For each stream, if the maximum is reached, the compression
+ level will be decreased during the stream. If the maximum is reached at the
+ beginning of a stream, the stream will not compress at all. If the maximum
+ is not reached, the compression level will be increased up to
+ tune.comp.maxlevel. A value of zero means there is no limit, this is the
+ default value.
+
+maxconn <number>
+ Sets the maximum per-process number of concurrent connections to <number>. It
+ is equivalent to the command-line argument "-n". Proxies will stop accepting
+ connections when this limit is reached. The "ulimit-n" parameter is
+ automatically adjusted according to this value. See also "ulimit-n". Note:
+ the "select" poller cannot reliably use more than 1024 file descriptors on
+ some platforms. If your platform only supports select and reports "select
+ FAILED" on startup, you need to reduce maxconn until it works (slightly
+ below 500 in general). If this value is not set, it will automatically be
+ calculated based on the current file descriptors limit reported by the
+ "ulimit -n" command, possibly reduced to a lower value if a memory limit
+ is enforced, based on the buffer size, memory allocated to compression, SSL
+ cache size, and use or not of SSL and the associated maxsslconn (which can
+ also be automatic). In any case, the fd-hard-limit applies if set.
+
+ See also: fd-hard-limit, ulimit-n
+
+maxconnrate <number>
+ Sets the maximum per-process number of connections per second to <number>.
+ Proxies will stop accepting connections when this limit is reached. It can be
+ used to limit the global capacity regardless of each frontend capacity. It is
+ important to note that this can only be used as a service protection measure,
+ as there will not necessarily be a fair share between frontends when the
+ limit is reached, so it's a good idea to also limit each frontend to some
+ value close to its expected share. Also, lowering tune.maxaccept can improve
+ fairness.
+
+maxpipes <number>
+ Sets the maximum per-process number of pipes to <number>. Currently, pipes
+ are only used by kernel-based tcp splicing. Since a pipe contains two file
+ descriptors, the "ulimit-n" value will be increased accordingly. The default
+ value is maxconn/4, which seems to be more than enough for most heavy usages.
+ The splice code dynamically allocates and releases pipes, and can fall back
+ to standard copy, so setting this value too low may only impact performance.
+
+maxsessrate <number>
+ Sets the maximum per-process number of sessions per second to <number>.
+ Proxies will stop accepting connections when this limit is reached. It can be
+ used to limit the global capacity regardless of each frontend capacity. It is
+ important to note that this can only be used as a service protection measure,
+ as there will not necessarily be a fair share between frontends when the
+ limit is reached, so it's a good idea to also limit each frontend to some
+ value close to its expected share. Also, lowering tune.maxaccept can improve
+ fairness.
+
+maxsslconn <number>
+ Sets the maximum per-process number of concurrent SSL connections to
+ <number>. By default there is no SSL-specific limit, which means that the
+ global maxconn setting will apply to all connections. Setting this limit
+ avoids having openssl use too much memory and crash when malloc returns NULL
+ (since it unfortunately does not reliably check for such conditions). Note
+ that the limit applies both to incoming and outgoing connections, so one
+ connection which is deciphered then ciphered accounts for 2 SSL connections.
+ If this value is not set, but a memory limit is enforced, this value will be
+ automatically computed based on the memory limit, maxconn, the buffer size,
+ memory allocated to compression, SSL cache size, and use of SSL in either
+ frontends, backends or both. If neither maxconn nor maxsslconn are specified
+ when there is a memory limit, HAProxy will automatically adjust these values
+ so that 100% of the connections can be made over SSL with no risk, and will
+ consider the sides where it is enabled (frontend, backend, both).
+
+maxsslrate <number>
+ Sets the maximum per-process number of SSL sessions per second to <number>.
+ SSL listeners will stop accepting connections when this limit is reached. It
+ can be used to limit the global SSL CPU usage regardless of each frontend
+ capacity. It is important to note that this can only be used as a service
+ protection measure, as there will not necessarily be a fair share between
+ frontends when the limit is reached, so it's a good idea to also limit each
+ frontend to some value close to its expected share. It is also important to
+ note that the sessions are accounted before they enter the SSL stack and not
+ after, which also protects the stack against bad handshakes. Also, lowering
+ tune.maxaccept can improve fairness.
+
+maxzlibmem <number>
+ Sets the maximum amount of RAM in megabytes per process usable by the zlib.
+ When the maximum amount is reached, future streams will not compress as long
+ as RAM is unavailable. When sets to 0, there is no limit.
+ The default value is 0. The value is available in bytes on the UNIX socket
+ with "show info" on the line "MaxZlibMemUsage", the memory used by zlib is
+ "ZlibMemUsage" in bytes.
+
+no-memory-trimming
+ Disables memory trimming ("malloc_trim") at a few moments where attempts are
+ made to reclaim lots of memory (on memory shortage or on reload). Trimming
+ memory forces the system's allocator to scan all unused areas and to release
+ them. This is generally seen as nice action to leave more available memory to
+ a new process while the old one is unlikely to make significant use of it.
+ But some systems dealing with tens to hundreds of thousands of concurrent
+ connections may experience a lot of memory fragmentation, that may render
+ this release operation extremely long. During this time, no more traffic
+ passes through the process, new connections are not accepted anymore, some
+ health checks may even fail, and the watchdog may even trigger and kill the
+ unresponsive process, leaving a huge core dump. If this ever happens, then it
+ is suggested to use this option to disable trimming and stop trying to be
+ nice with the new process. Note that advanced memory allocators usually do
+ not suffer from such a problem.
+
+noepoll
+ Disables the use of the "epoll" event polling system on Linux. It is
+ equivalent to the command-line argument "-de". The next polling system
+ used will generally be "poll". See also "nopoll".
+
+noevports
+ Disables the use of the event ports event polling system on SunOS systems
+ derived from Solaris 10 and later. It is equivalent to the command-line
+ argument "-dv". The next polling system used will generally be "poll". See
+ also "nopoll".
+
+nogetaddrinfo
+ Disables the use of getaddrinfo(3) for name resolving. It is equivalent to
+ the command line argument "-dG". Deprecated gethostbyname(3) will be used.
+
+nokqueue
+ Disables the use of the "kqueue" event polling system on BSD. It is
+ equivalent to the command-line argument "-dk". The next polling system
+ used will generally be "poll". See also "nopoll".
+
+nopoll
+ Disables the use of the "poll" event polling system. It is equivalent to the
+ command-line argument "-dp". The next polling system used will be "select".
+ It should never be needed to disable "poll" since it's available on all
+ platforms supported by HAProxy. See also "nokqueue", "noepoll" and
+ "noevports".
+
+noreuseport
+ Disables the use of SO_REUSEPORT - see socket(7). It is equivalent to the
+ command line argument "-dR".
+
+nosplice
+ Disables the use of kernel tcp splicing between sockets on Linux. It is
+ equivalent to the command line argument "-dS". Data will then be copied
+ using conventional and more portable recv/send calls. Kernel tcp splicing is
+ limited to some very recent instances of kernel 2.6. Most versions between
+ 2.6.25 and 2.6.28 are buggy and will forward corrupted data, so they must not
+ be used. This option makes it easier to globally disable kernel splicing in
+ case of doubt. See also "option splice-auto", "option splice-request" and
+ "option splice-response".
+
+profiling.memory { on | off }
+ Enables ('on') or disables ('off') per-function memory profiling. This will
+ keep usage statistics of malloc/calloc/realloc/free calls anywhere in the
+ process (including libraries) which will be reported on the CLI using the
+ "show profiling" command. This is essentially meant to be used when an
+ abnormal memory usage is observed that cannot be explained by the pools and
+ other info are required. The performance hit will typically be around 1%,
+ maybe a bit more on highly threaded machines, so it is normally suitable for
+ use in production. The same may be achieved at run time on the CLI using the
+ "set profiling memory" command, please consult the management manual.
+
+profiling.tasks { auto | on | off }
+ Enables ('on') or disables ('off') per-task CPU profiling. When set to 'auto'
+ the profiling automatically turns on a thread when it starts to suffer from
+ an average latency of 1000 microseconds or higher as reported in the
+ "avg_loop_us" activity field, and automatically turns off when the latency
+ returns below 990 microseconds (this value is an average over the last 1024
+ loops so it does not vary quickly and tends to significantly smooth short
+ spikes). It may also spontaneously trigger from time to time on overloaded
+ systems, containers, or virtual machines, or when the system swaps (which
+ must absolutely never happen on a load balancer).
+
+ CPU profiling per task can be very convenient to report where the time is
+ spent and which requests have what effect on which other request. Enabling
+ it will typically affect the overall's performance by less than 1%, thus it
+ is recommended to leave it to the default 'auto' value so that it only
+ operates when a problem is identified. This feature requires a system
+ supporting the clock_gettime(2) syscall with clock identifiers
+ CLOCK_MONOTONIC and CLOCK_THREAD_CPUTIME_ID, otherwise the reported time will
+ be zero. This option may be changed at run time using "set profiling" on the
+ CLI.
+
+spread-checks <0..50, in percent>
+ Sometimes it is desirable to avoid sending agent and health checks to
+ servers at exact intervals, for instance when many logical servers are
+ located on the same physical server. With the help of this parameter, it
+ becomes possible to add some randomness in the check interval between 0
+ and +/- 50%. A value between 2 and 5 seems to show good results. The
+ default value remains at 0.
+
+ssl-engine <name> [algo <comma-separated list of algorithms>]
+ Sets the OpenSSL engine to <name>. List of valid values for <name> may be
+ obtained using the command "openssl engine". This statement may be used
+ multiple times, it will simply enable multiple crypto engines. Referencing an
+ unsupported engine will prevent HAProxy from starting. Note that many engines
+ will lead to lower HTTPS performance than pure software with recent
+ processors. The optional command "algo" sets the default algorithms an ENGINE
+ will supply using the OPENSSL function ENGINE_set_default_string(). A value
+ of "ALL" uses the engine for all cryptographic operations. If no list of
+ algo is specified then the value of "ALL" is used. A comma-separated list
+ of different algorithms may be specified, including: RSA, DSA, DH, EC, RAND,
+ CIPHERS, DIGESTS, PKEY, PKEY_CRYPTO, PKEY_ASN1. This is the same format that
+ openssl configuration file uses:
+ https://www.openssl.org/docs/man1.0.2/apps/config.html
+
+ HAProxy Version 2.6 disabled the support for engines in the default build.
+ This option is only available when HAProxy has been built with support for
+ it. In case the ssl-engine is required HAProxy can be rebuild with the
+ USE_ENGINE=1 flag.
+
+ssl-mode-async
+ Adds SSL_MODE_ASYNC mode to the SSL context. This enables asynchronous TLS
+ I/O operations if asynchronous capable SSL engines are used. The current
+ implementation supports a maximum of 32 engines. The Openssl ASYNC API
+ doesn't support moving read/write buffers and is not compliant with
+ HAProxy's buffer management. So the asynchronous mode is disabled on
+ read/write operations (it is only enabled during initial and renegotiation
+ handshakes).
+
+tune.buffers.limit <number>
+ Sets a hard limit on the number of buffers which may be allocated per process.
+ The default value is zero which means unlimited. The minimum non-zero value
+ will always be greater than "tune.buffers.reserve" and should ideally always
+ be about twice as large. Forcing this value can be particularly useful to
+ limit the amount of memory a process may take, while retaining a sane
+ behavior. When this limit is reached, streams which need a buffer wait for
+ another one to be released by another stream. Since buffers are dynamically
+ allocated and released, the waiting time is very short and not perceptible
+ provided that limits remain reasonable. In fact sometimes reducing the limit
+ may even increase performance by increasing the CPU cache's efficiency. Tests
+ have shown good results on average HTTP traffic with a limit to 1/10 of the
+ expected global maxconn setting, which also significantly reduces memory
+ usage. The memory savings come from the fact that a number of connections
+ will not allocate 2*tune.bufsize. It is best not to touch this value unless
+ advised to do so by an HAProxy core developer.
+
+tune.buffers.reserve <number>
+ Sets the number of buffers which are pre-allocated and reserved for use only
+ during memory shortage conditions resulting in failed memory allocations. The
+ minimum value is 2 and is also the default. There is no reason a user would
+ want to change this value, it's mostly aimed at HAProxy core developers.
+
+tune.bufsize <number>
+ Sets the buffer size to this size (in bytes). Lower values allow more
+ streams to coexist in the same amount of RAM, and higher values allow some
+ applications with very large cookies to work. The default value is 16384 and
+ can be changed at build time. It is strongly recommended not to change this
+ from the default value, as very low values will break some services such as
+ statistics, and values larger than default size will increase memory usage,
+ possibly causing the system to run out of memory. At least the global maxconn
+ parameter should be decreased by the same factor as this one is increased. In
+ addition, use of HTTP/2 mandates that this value must be 16384 or more. If an
+ HTTP request is larger than (tune.bufsize - tune.maxrewrite), HAProxy will
+ return HTTP 400 (Bad Request) error. Similarly if an HTTP response is larger
+ than this size, HAProxy will return HTTP 502 (Bad Gateway). Note that the
+ value set using this parameter will automatically be rounded up to the next
+ multiple of 8 on 32-bit machines and 16 on 64-bit machines.
+
+tune.comp.maxlevel <number>
+ Sets the maximum compression level. The compression level affects CPU
+ usage during compression. This value affects CPU usage during compression.
+ Each stream using compression initializes the compression algorithm with
+ this value. The default value is 1.
+
+tune.disable-fast-forward [ EXPERIMENTAL ]
+ Disables the data fast-forwarding. It is a mechanism to optimize the data
+ forwarding by passing data directly from a side to the other one without
+ waking the stream up. Thanks to this directive, it is possible to disable
+ this optimization. Note it also disable any kernel tcp splicing but also the
+ zero-copy forwarding. This command is not meant for regular use, it will
+ generally only be suggested by developers along complex debugging
+ sessions. For this reason it is internally marked as experimental, meaning
+ that "expose-experimental-directives" must appear on a line before this
+ directive.
+
+tune.disable-zero-copy-forwarding
+ Globally disables the zero-copy forwarding of data. It is a mechanism to
+ optimize the data fast-forwarding by avoiding to use the channel's buffer.
+ Thanks to this directive, it is possible to disable this optimization. Note
+ it also disable any kernel tcp splicing.
+
+ See also: tune.pt.zero-copy-forwarding,
+ tune.h1.zero-copy-fwd-recv, tune.h1.zero-copy-fwd-send,
+ tune.h2.zero-copy-fwd-send, tune.quic.zero-copy-fwd-send
+
+tune.events.max-events-at-once <number>
+ Sets the number of events that may be processed at once by an asynchronous
+ task handler (from event_hdl API). <number> should be included between 1
+ and 10000. Large number could cause thread contention as a result of the
+ task doing heavy work without interruption, and on the other hand, small
+ number could result in the task being constantly rescheduled because it
+ cannot consume enough events per run and is not able to catch up with the
+ event producer. The default value may be forced at build time, otherwise
+ defaults to 100.
+
+tune.fail-alloc
+ If compiled with DEBUG_FAIL_ALLOC or started with "-dMfail", gives the
+ percentage of chances an allocation attempt fails. Must be between 0 (no
+ failure) and 100 (no success). This is useful to debug and make sure memory
+ failures are handled gracefully. When not set, the ratio is 0. However the
+ command-line "-dMfail" option automatically sets it to 1% failure rate so that
+ it is not necessary to change the configuration for testing.
+
+tune.fd.edge-triggered { on | off } [ EXPERIMENTAL ]
+ Enables ('on') or disables ('off') the edge-triggered polling mode for FDs
+ that support it. This is currently only support with epoll. It may noticeably
+ reduce the number of epoll_ctl() calls and slightly improve performance in
+ certain scenarios. This is still experimental, it may result in frozen
+ connections if bugs are still present, and is disabled by default.
+
+tune.h1.zero-copy-fwd-recv { on | off }
+ Enables ('on') of disabled ('off') the zero-copy receives of data for the H1
+ multiplexer. It is enabled by default.
+
+ See also: tune.disable-zero-copy-forwarding, tune.h1.zero-copy-fwd-send
+
+tune.h1.zero-copy-fwd-send { on | off }
+ Enables ('on') of disabled ('off') the zero-copy sends of data for the H1
+ multiplexer. It is enabled by default.
+
+ See also: tune.disable-zero-copy-forwarding, tune.h1.zero-copy-fwd-recv
+
+tune.h2.be.initial-window-size <number>
+ Sets the HTTP/2 initial window size for outgoing connections, which is the
+ number of bytes the server can respond before waiting for an acknowledgment
+ from HAProxy. This setting only affects payload contents, not headers. When
+ not set, the common default value set by tune.h2.initial-window-size applies.
+ It can make sense to slightly increase this value to allow faster downloads
+ or to reduce CPU usage on the servers, at the expense of creating unfairness
+ between clients. It doesn't affect resource usage.
+
+ See also: tune.h2.initial-window-size.
+
+tune.h2.be.max-concurrent-streams <number>
+ Sets the HTTP/2 maximum number of concurrent streams per outgoing connection
+ (i.e. the number of outstanding requests on a single connection to a server).
+ When not set, the default set by tune.h2.max-concurrent-streams applies. A
+ smaller value than the default 100 may improve a site's responsiveness at the
+ expense of maintaining more established connections to the servers. When the
+ "http-reuse" setting is set to "always", it is recommended to reduce this
+ value so as not to mix too many different clients over the same connection,
+ because if a client is slower than others, a mechanism known as "head of
+ line blocking" tends to cause cascade effect on download speed for all
+ clients sharing a connection (keep tune.h2.be.initial-window-size low in this
+ case). It is highly recommended not to increase this value; some might find
+ it optimal to run at low values (1..5 typically).
+
+tune.h2.fe.initial-window-size <number>
+ Sets the HTTP/2 initial window size for incoming connections, which is the
+ number of bytes the client can upload before waiting for an acknowledgment
+ from HAProxy. This setting only affects payload contents (i.e. the body of
+ POST requests), not headers. When not set, the common default value set by
+ tune.h2.initial-window-size applies. It can make sense to increase this value
+ to allow faster uploads. The default value of 65536 allows up to 5 Mbps of
+ bandwidth per client over a 100 ms ping time, and 500 Mbps for 1 ms ping
+ time. It doesn't affect resource usage. Using too large values may cause
+ clients to experience a lack of responsiveness if pages are accessed in
+ parallel to large uploads.
+
+ See also: tune.h2.initial-window-size.
+
+tune.h2.fe.max-concurrent-streams <number>
+ Sets the HTTP/2 maximum number of concurrent streams per incoming connection
+ (i.e. the number of outstanding requests on a single connection from a
+ client). When not set, the default set by tune.h2.max-concurrent-streams
+ applies. A larger value than the default 100 may sometimes slightly improve
+ the page load time for complex sites with lots of small objects over high
+ latency networks but can also result in using more memory by allowing a
+ client to allocate more resources at once. The default value of 100 is
+ generally good and it is recommended not to change this value.
+
+tune.h2.fe.max-total-streams <number>
+ Sets the HTTP/2 maximum number of total streams processed per incoming
+ connection. Once this limit is reached, HAProxy will send a graceful GOAWAY
+ frame informing the client that it will close the connection after all
+ pending streams have been closed. In practice, clients tend to close as fast
+ as possible when receiving this, and to establish a new connection for next
+ requests. Doing this is sometimes useful and desired in situations where
+ clients stay connected for a very long time and cause some imbalance inside a
+ farm. For example, in some highly dynamic environments, it is possible that
+ new load balancers are instantiated on the fly to adapt to a load increase,
+ and that once the load goes down they should be stopped without breaking
+ established connections. By setting a limit here, the connections will have
+ a limited lifetime and will be frequently renewed, with some possibly being
+ established to other nodes, so that existing resources are quickly released.
+
+ It's important to understand that there is an implicit relation between this
+ limit and "tune.h2.fe.max-concurrent-streams" above. Indeed, HAProxy will
+ always accept to process any possibly pending streams that might be in flight
+ between the client and the frontend, so the advertised limit will always
+ automatically be raised by the value configured in max-concurrent-streams,
+ and this value will serve as a hard limit above which a violation by a non-
+ compliant client will result in the connection being closed. Thus when
+ counting the number of requests per connection from the logs, any number
+ between max-total-streams and (max-total-streams + max-concurrent-streams)
+ may be observed depending on how fast streams are created by the client.
+
+ The default value is zero, which enforces no limit beyond those implied by
+ the protocol (2^30 ~= 1.07 billion). Values around 1000 may already cause
+ frequent connection renewal without causing any perceptible latency to most
+ clients. Setting it too low may result in an increase of CPU usage due to
+ frequent TLS reconnections, in addition to increased page load time. Please
+ note that some load testing tools do not support reconnections and may report
+ errors with this setting; as such it may be needed to disable it when running
+ performance benchmarks. See also "tune.h2.fe.max-concurrent-streams".
+
+tune.h2.header-table-size <number>
+ Sets the HTTP/2 dynamic header table size. It defaults to 4096 bytes and
+ cannot be larger than 65536 bytes. A larger value may help certain clients
+ send more compact requests, depending on their capabilities. This amount of
+ memory is consumed for each HTTP/2 connection. It is recommended not to
+ change it.
+
+tune.h2.initial-window-size <number>
+ Sets the default value for the HTTP/2 initial window size, on both incoming
+ and outgoing connections. This value is used for incoming connections when
+ tune.h2.fe.initial-window-size is not set, and by outgoing connections when
+ tune.h2.be.initial-window-size is not set. The default value is 65536, which
+ for uploads roughly allows up to 5 Mbps of bandwidth per client over a
+ network showing a 100 ms ping time, or 500 Mbps over a 1-ms local network.
+ Given that changing the default value will both increase upload speeds and
+ cause more unfairness between clients on downloads, it is recommended to
+ instead use the side-specific settings tune.h2.fe.initial-window-size and
+ tune.h2.be.initial-window-size.
+
+tune.h2.max-concurrent-streams <number>
+ Sets the default HTTP/2 maximum number of concurrent streams per connection
+ (i.e. the number of outstanding requests on a single connection). This value
+ is used for incoming connections when tune.h2.fe.max-concurrent-streams is
+ not set, and for outgoing connections when tune.h2.be.max-concurrent-streams
+ is not set. The default value is 100. The impact varies depending on the side
+ so please see the two settings above for more details. It is recommended not
+ to use this setting and to switch to the per-side ones instead. A value of
+ zero disables the limit so a single client may create as many streams as
+ allocatable by HAProxy. It is highly recommended not to change this value.
+
+tune.h2.max-frame-size <number>
+ Sets the HTTP/2 maximum frame size that HAProxy announces it is willing to
+ receive to its peers. The default value is the largest between 16384 and the
+ buffer size (tune.bufsize). In any case, HAProxy will not announce support
+ for frame sizes larger than buffers. The main purpose of this setting is to
+ allow to limit the maximum frame size setting when using large buffers. Too
+ large frame sizes might have performance impact or cause some peers to
+ misbehave. It is highly recommended not to change this value.
+
+tune.h2.zero-copy-fwd-send { on | off }
+ Enables ('on') of disabled ('off') the zero-copy sends of data for the H2
+ multiplexer. It is enabled by default.
+
+ See also: tune.disable-zero-copy-forwarding
+
+tune.http.cookielen <number>
+ Sets the maximum length of captured cookies. This is the maximum value that
+ the "capture cookie xxx len yyy" will be allowed to take, and any upper value
+ will automatically be truncated to this one. It is important not to set too
+ high a value because all cookie captures still allocate this size whatever
+ their configured value (they share a same pool). This value is per request
+ per response, so the memory allocated is twice this value per connection.
+ When not specified, the limit is set to 63 characters. It is recommended not
+ to change this value.
+
+tune.http.logurilen <number>
+ Sets the maximum length of request URI in logs. This prevents truncating long
+ request URIs with valuable query strings in log lines. This is not related
+ to syslog limits. If you increase this limit, you may also increase the
+ 'log ... len yyy' parameter. Your syslog daemon may also need specific
+ configuration directives too.
+ The default value is 1024.
+
+tune.http.maxhdr <number>
+ Sets the maximum number of headers in a request. When a request comes with a
+ number of headers greater than this value (including the first line), it is
+ rejected with a "400 Bad Request" status code. Similarly, too large responses
+ are blocked with "502 Bad Gateway". The default value is 101, which is enough
+ for all usages, considering that the widely deployed Apache server uses the
+ same limit. It can be useful to push this limit further to temporarily allow
+ a buggy application to work by the time it gets fixed. The accepted range is
+ 1..32767. Keep in mind that each new header consumes 32bits of memory for
+ each stream, so don't push this limit too high.
+
+tune.idle-pool.shared { on | off }
+ Enables ('on') or disables ('off') sharing of idle connection pools between
+ threads for a same server. The default is to share them between threads in
+ order to minimize the number of persistent connections to a server, and to
+ optimize the connection reuse rate. But to help with debugging or when
+ suspecting a bug in HAProxy around connection reuse, it can be convenient to
+ forcefully disable this idle pool sharing between multiple threads, and force
+ this option to "off". The default is on. It is strongly recommended against
+ disabling this option without setting a conservative value on "pool-low-conn"
+ for all servers relying on connection reuse to achieve a high performance
+ level, otherwise connections might be closed very often as the thread count
+ increases.
+
+tune.idletimer <timeout>
+ Sets the duration after which HAProxy will consider that an empty buffer is
+ probably associated with an idle stream. This is used to optimally adjust
+ some packet sizes while forwarding large and small data alternatively. The
+ decision to use splice() or to send large buffers in SSL is modulated by this
+ parameter. The value is in milliseconds between 0 and 65535. A value of zero
+ means that HAProxy will not try to detect idle streams. The default is 1000,
+ which seems to correctly detect end user pauses (e.g. read a page before
+ clicking). There should be no reason for changing this value. Please check
+ tune.ssl.maxrecord below.
+
+tune.listener.default-shards { by-process | by-thread | by-group }
+ Normally, all "bind" lines will create a single shard, that is, a single
+ socket that all threads of the process will listen to. With many threads,
+ this is not very efficient, and may even induce some important overhead in
+ the kernel for updating the polling state or even distributing events to the
+ various threads. Modern operating systems support balancing of incoming
+ connections, a mechanism that will consist in permitting multiple sockets to
+ be bound to the same address and port, and to evenly distribute all incoming
+ connections to these sockets so that each thread only sees the connections
+ that are waiting in the socket it is bound to. This significantly reduces
+ kernel-side overhead and increases performance in the incoming connection
+ path. This is usually enabled in HAProxy using the "shards" setting on "bind"
+ lines, which defaults to 1, meaning that each listener will be unique in the
+ process. On systems with many processors, it may be more convenient to change
+ the default setting to "by-thread" in order to always create one listening
+ socket per thread, or "by-group" in order to always create one listening
+ socket per thread group. Be careful about the file descriptor usage with
+ "by-thread" as each listener will need as many sockets as there are threads.
+ Also some operating systems (e.g. FreeBSD) are limited to no more than 256
+ sockets on a same address. Note that "by-group" will remain equivalent to
+ "by-process" for default configurations involving a single thread group, and
+ will fall back to sharing the same socket on systems that do not support this
+ mechanism. The default is "by-group" with a fallback to "by-process" for
+ systems or socket families that do not support multiple bindings.
+
+tune.listener.multi-queue { on | fair | off }
+ Enables ('on' / 'fair') or disables ('off') the listener's multi-queue accept
+ which spreads the incoming traffic to all threads a "bind" line is allowed to
+ run on instead of taking them for itself. This provides a smoother traffic
+ distribution and scales much better, especially in environments where threads
+ may be unevenly loaded due to external activity (network interrupts colliding
+ with one thread for example). The default mode, "on", optimizes the choice of
+ a thread by picking in a sample the one with the less connections. It is
+ often the best choice when connections are long-lived as it manages to keep
+ all threads busy. A second mode, "fair", instead cycles through all threads
+ regardless of their instant load level. It can be better suited for short-
+ lived connections, or on machines with very large numbers of threads where
+ the probability to find the least loaded thread with the first mode is low.
+ Finally it is possible to forcefully disable the redistribution mechanism
+ using "off" for troubleshooting, or for situations where connections are
+ short-lived and it is estimated that the operating system already provides a
+ good enough distribution. The default is "on".
+
+tune.lua.forced-yield <number>
+ This directive forces the Lua engine to execute a yield each <number> of
+ instructions executed. This permits interrupting a long script and allows the
+ HAProxy scheduler to process other tasks like accepting connections or
+ forwarding traffic. The default value is 10000 instructions. If HAProxy often
+ executes some Lua code but more responsiveness is required, this value can be
+ lowered. If the Lua code is quite long and its result is absolutely required
+ to process the data, the <number> can be increased.
+
+tune.lua.maxmem <number>
+ Sets the maximum amount of RAM in megabytes per process usable by Lua. By
+ default it is zero which means unlimited. It is important to set a limit to
+ ensure that a bug in a script will not result in the system running out of
+ memory.
+
+tune.lua.session-timeout <timeout>
+ This is the execution timeout for the Lua sessions. This is useful for
+ preventing infinite loops or spending too much time in Lua. This timeout
+ counts only the pure Lua runtime. If the Lua does a sleep, the sleep is
+ not taken in account. The default timeout is 4s.
+
+tune.lua.burst-timeout <timeout>
+ The "burst" execution timeout applies to any Lua handler. If the handler
+ fails to finish or yield before timeout is reached, it will be aborted to
+ prevent thread contention, to prevent traffic from not being served for too
+ long, and ultimately to prevent the process from crashing because of the
+ watchdog kicking in. Unlike other lua timeouts which are yield-cumulative,
+ burst-timeout will ensure that the time spent in a single lua execution
+ window does not exceed the configured timeout.
+
+ Yielding here means that the lua execution is effectively interrupted
+ either through an explicit call to lua-yielding function such as
+ core.(m)sleep() or core.yield(), or following an automatic forced-yield
+ (see tune.lua.forced-yield) and that it will be resumed later when the
+ related task is set for rescheduling. Not all lua handlers may yield: we have
+ to make a distinction between yieldable handlers and unyieldable handlers.
+
+ For yieldable handlers (tasks, actions..), reaching the timeout means
+ "tune.lua.forced-yield" might be too high for the system, reducing it
+ could improve the situation, but it could also be a good idea to check if
+ adding manual yields at some key points within the lua function helps or not.
+ It may also indicate that the handler is spending too much time in a specific
+ lua library function that cannot be interrupted.
+
+ For unyieldable handlers (lua converters, sample fetches), it could simply
+ indicate that the handler is doing too much computation, which could result
+ from an improper design given that such handlers, which often block the
+ request execution flow, are expected to terminate quickly to allow the
+ request processing to go through. A common resolution approach here would be
+ to try to better optimize the lua function for speed since decreasing
+ "tune.lua.forced-yield" won't help.
+
+ This timeout only counts the pure Lua runtime. If the Lua does a core.sleep,
+ the sleeping time is not taken in account. The default timeout is 1000ms.
+
+ Note: if a lua GC cycle is initiated from the handler (either explicitly
+ requested or automatically triggered by lua after some time), the GC cycle
+ time will also be accounted for.
+
+ Indeed, there is no way to deduce the GC cycle time, so this could lead to
+ some false positives on saturated systems (where GC is having hard time to
+ catch up and consumes most of the available execution runtime). If it were
+ to be the case, here are some resolution leads:
+
+ - checking if the script could be optimized to reduce lua memory footprint
+ - fine-tuning lua GC parameters and / or requesting manual GC cycles
+ (see: https://www.lua.org/manual/5.4/manual.html#pdf-collectgarbage)
+ - increasing tune.lua.burst-timeout
+
+ Setting value to 0 completely disables this protection.
+
+tune.lua.service-timeout <timeout>
+ This is the execution timeout for the Lua services. This is useful for
+ preventing infinite loops or spending too much time in Lua. This timeout
+ counts only the pure Lua runtime. If the Lua does a sleep, the sleep is
+ not taken in account. The default timeout is 4s.
+
+tune.lua.task-timeout <timeout>
+ Purpose is the same as "tune.lua.session-timeout", but this timeout is
+ dedicated to the tasks. By default, this timeout isn't set because a task may
+ remain alive during of the lifetime of HAProxy. For example, a task used to
+ check servers.
+
+tune.lua.log.loggers { on | off }
+ Enables ('on') or disables ('off') logging the output of LUA scripts via the
+ loggers applicable to the current proxy, if any.
+
+ Defaults to 'on'.
+
+tune.lua.log.stderr { on | auto | off }
+ Enables ('on') or disables ('off') logging the output of LUA scripts via
+ stderr.
+ When set to 'auto', logging via stderr is conditionally 'on' if any of:
+
+ - tune.lua.log.loggers is set to 'off'
+ - the script is executed in a non-proxy context with no global logger
+ - the script is executed in a proxy context with no logger attached
+
+ Please note that, when enabled, this logging is in addition to the logging
+ configured via tune.lua.log.loggers.
+
+ Defaults to 'auto'.
+
+tune.max-checks-per-thread <number>
+ Sets the number of active checks per thread above which a thread will
+ actively try to search a less loaded thread to run the health check, or
+ queue it until the number of active checks running on it diminishes. The
+ default value is zero, meaning no such limit is set. It may be needed in
+ certain environments running an extremely large number of expensive checks
+ with many threads when the load appears unequal and may make health checks
+ to randomly time out on startup, typically when using OpenSSL 3.0 which is
+ about 20 times more CPU-intensive on health checks than older ones. This will
+ have for result to try to level the health check work across all threads. The
+ vast majority of configurations do not need to touch this parameter. Please
+ note that too low values may significantly slow down the health checking if
+ checks are slow to execute.
+
+tune.maxaccept <number>
+ Sets the maximum number of consecutive connections a process may accept in a
+ row before switching to other work. In single process mode, higher numbers
+ used to give better performance at high connection rates, though this is not
+ the case anymore with the multi-queue. This value applies individually to
+ each listener, so that the number of processes a listener is bound to is
+ taken into account. This value defaults to 4 which showed best results. If a
+ significantly higher value was inherited from an ancient config, it might be
+ worth removing it as it will both increase performance and lower response
+ time. In multi-process mode, it is divided by twice the number of processes
+ the listener is bound to. Setting this value to -1 completely disables the
+ limitation. It should normally not be needed to tweak this value.
+
+tune.maxpollevents <number>
+ Sets the maximum amount of events that can be processed at once in a call to
+ the polling system. The default value is adapted to the operating system. It
+ has been noticed that reducing it below 200 tends to slightly decrease
+ latency at the expense of network bandwidth, and increasing it above 200
+ tends to trade latency for slightly increased bandwidth.
+
+tune.maxrewrite <number>
+ Sets the reserved buffer space to this size in bytes. The reserved space is
+ used for header rewriting or appending. The first reads on sockets will never
+ fill more than bufsize-maxrewrite. Historically it has defaulted to half of
+ bufsize, though that does not make much sense since there are rarely large
+ numbers of headers to add. Setting it too high prevents processing of large
+ requests or responses. Setting it too low prevents addition of new headers
+ to already large requests or to POST requests. It is generally wise to set it
+ to about 1024. It is automatically readjusted to half of bufsize if it is
+ larger than that. This means you don't have to worry about it when changing
+ bufsize.
+
+tune.memory.hot-size <number>
+ Sets the per-thread amount of memory that will be kept hot in the local cache
+ and will never be recoverable by other threads. Access to this memory is very
+ fast (lockless), and having enough is critical to maintain a good performance
+ level under extreme thread contention. The value is expressed in bytes, and
+ the default value is configured at build time via CONFIG_HAP_POOL_CACHE_SIZE
+ which defaults to 524288 (512 kB). A larger value may increase performance in
+ some usage scenarios, especially when performance profiles show that memory
+ allocation is stressed a lot. Experience shows that a good value sits between
+ once to twice the per CPU core L2 cache size. Too large values will have a
+ negative impact on performance by making inefficient use of the L3 caches in
+ the CPUs, and will consume larger amounts of memory. It is recommended not to
+ change this value, or to proceed in small increments. In order to completely
+ disable the per-thread CPU caches, using a very small value could work, but
+ it is better to use "-dMno-cache" on the command-line.
+
+tune.pattern.cache-size <number>
+ Sets the size of the pattern lookup cache to <number> entries. This is an LRU
+ cache which reminds previous lookups and their results. It is used by ACLs
+ and maps on slow pattern lookups, namely the ones using the "sub", "reg",
+ "dir", "dom", "end", "bin" match methods as well as the case-insensitive
+ strings. It applies to pattern expressions which means that it will be able
+ to memorize the result of a lookup among all the patterns specified on a
+ configuration line (including all those loaded from files). It automatically
+ invalidates entries which are updated using HTTP actions or on the CLI. The
+ default cache size is set to 10000 entries, which limits its footprint to
+ about 5 MB per process/thread on 32-bit systems and 8 MB per process/thread
+ on 64-bit systems, as caches are thread/process local. There is a very low
+ risk of collision in this cache, which is in the order of the size of the
+ cache divided by 2^64. Typically, at 10000 requests per second with the
+ default cache size of 10000 entries, there's 1% chance that a brute force
+ attack could cause a single collision after 60 years, or 0.1% after 6 years.
+ This is considered much lower than the risk of a memory corruption caused by
+ aging components. If this is not acceptable, the cache can be disabled by
+ setting this parameter to 0.
+
+tune.peers.max-updates-at-once <number>
+ Sets the maximum number of stick-table updates that haproxy will try to
+ process at once when sending messages. Retrieving the data for these updates
+ requires some locking operations which can be CPU intensive on highly
+ threaded machines if unbound, and may also increase the traffic latency
+ during the initial batched transfer between an older and a newer process.
+ Conversely low values may also incur higher CPU overhead, and take longer
+ to complete. The default value is 200 and it is suggested not to change it.
+
+tune.pipesize <number>
+ Sets the kernel pipe buffer size to this size (in bytes). By default, pipes
+ are the default size for the system. But sometimes when using TCP splicing,
+ it can improve performance to increase pipe sizes, especially if it is
+ suspected that pipes are not filled and that many calls to splice() are
+ performed. This has an impact on the kernel's memory footprint, so this must
+ not be changed if impacts are not understood.
+
+tune.pool-high-fd-ratio <number>
+ This setting sets the max number of file descriptors (in percentage) used by
+ HAProxy globally against the maximum number of file descriptors HAProxy can
+ use before we start killing idle connections when we can't reuse a connection
+ and we have to create a new one. The default is 25 (one quarter of the file
+ descriptor will mean that roughly half of the maximum front connections can
+ keep an idle connection behind, anything beyond this probably doesn't make
+ much sense in the general case when targeting connection reuse).
+
+tune.pool-low-fd-ratio <number>
+ This setting sets the max number of file descriptors (in percentage) used by
+ HAProxy globally against the maximum number of file descriptors HAProxy can
+ use before we stop putting connection into the idle pool for reuse. The
+ default is 20.
+
+tune.pt.zero-copy-forwarding { on | off }
+ Enables ('on') of disabled ('off') the zero-copy forwarding of data for the
+ pass-through multiplexer. To be used, the kernel splicing must also be
+ configured. It is enabled by default.
+
+ See also: tune.disable-zero-copy-forwarding, option splice-auto,
+ option splice-request and option splice-response
+
+tune.quic.frontend.conn-tx-buffers.limit <number>
+ This settings defines the maximum number of buffers allocated for a QUIC
+ connection on data emission. By default, it is set to 30. QUIC buffers are
+ drained on ACK reception. This setting has a direct impact on the throughput
+ and memory consumption and can be adjusted according to an estimated round
+ time-trip. Each buffer is tune.bufsize.
+
+tune.quic.frontend.max-idle-timeout <timeout>
+ Sets the QUIC max_idle_timeout transport parameters in milliseconds for
+ frontends which determines the period of time after which a connection silently
+ closes if it has remained inactive during an effective period of time deduced
+ from the two max_idle_timeout values announced by the two endpoints:
+ - the minimum of the two values if both are not null,
+ - the maximum if only one of them is not null,
+ - if both values are null, this feature is disabled.
+
+ The default value is 30000.
+
+tune.quic.frontend.max-streams-bidi <number>
+ Sets the QUIC initial_max_streams_bidi transport parameter for frontends.
+ This is the initial maximum number of bidirectional streams the remote peer
+ will be authorized to open. This determines the number of concurrent client
+ requests.
+
+ The default value is 100.
+
+tune.quic.max-frame-loss <number>
+ Sets the limit for which a single QUIC frame can be marked as lost. If
+ exceeded, the connection is considered as failing and is closed immediately.
+
+ The default value is 10.
+
+tune.quic.reorder-ratio <0..100, in percent>
+ The ratio applied to the packet reordering threshold calculated. It may
+ trigger a high packet loss detection when too small.
+
+ The default value is 50.
+
+tune.quic.retry-threshold <number>
+ Dynamically enables the Retry feature for all the configured QUIC listeners
+ as soon as this number of half open connections is reached. A half open
+ connection is a connection whose handshake has not already successfully
+ completed or failed. To be functional this setting needs a cluster secret to
+ be set, if not it will be silently ignored (see "cluster-secret" setting).
+ This setting will be also silently ignored if the use of QUIC Retry was
+ forced (see "quic-force-retry").
+
+ The default value is 100.
+
+ See https://www.rfc-editor.org/rfc/rfc9000.html#section-8.1.2 for more
+ information about QUIC retry.
+
+tune.quic.socket-owner { connection | listener }
+ Specifies globally how QUIC connections will use socket for receive/send
+ operations. Connections can share listener socket or each connection can
+ allocate its own socket.
+
+ When default "connection" value is set, a dedicated socket will be allocated
+ by every QUIC connections. This option is the preferred one to achieve the
+ best performance with a large QUIC traffic. This is also the only way to
+ ensure soft-stop is conducted properly without data loss for QUIC connections
+ and cases of transient errors during sendto() operation are handled
+ efficiently. However, this relies on some advanced features from the UDP
+ network stack. If your platform is deemed not compatible, haproxy will
+ automatically switch to "listener" mode on startup. Please note that QUIC
+ listeners running on privileged ports may require to run as uid 0, or some
+ OS-specific tuning to permit the target uid to bind such ports, such as
+ system capabilities. See also the "setcap" global directive.
+
+ The "listener" value indicates that QUIC transfers will occur on the shared
+ listener socket. This option can be a good compromise for small traffic as it
+ allows to reduce FD consumption. However, performance won't be optimal due to
+ a higher CPU usage if listeners are shared across a lot of threads or a
+ large number of QUIC connections can be used simultaneously.
+
+ This setting is applied in conjunction with each "quic-socket" bind options.
+ If "connection" mode is used on global tuning, it will be activated for each
+ listener, unless its bind option is set to "listener". However, if "listener"
+ is used globally, it will be forced on every listener instance, regardless of
+ their individual configuration.
+
+tune.quic.zero-copy-fwd-send { on | off }
+ Enables ('on') of disabled ('off') the zero-copy sends of data for the QUIC
+ multiplexer. It is disabled by default.
+
+ See also: tune.disable-zero-copy-forwarding
+
+tune.rcvbuf.backend <number>
+tune.rcvbuf.frontend <number>
+ For the kernel socket receive buffer size on non-connected sockets to this
+ size. This can be used QUIC in listener mode and log-forward on the frontend.
+ The default system buffers might sometimes be too small for sockets receiving
+ lots of aggregated traffic, causing some losses and possibly retransmits (in
+ case of QUIC), possibly slowing down connection establishment under heavy
+ traffic. The value is expressed in bytes, applied to each socket. In listener
+ mode, sockets are shared between all connections, and the total number of
+ sockets depends on the "shards" value of the "bind" line. There's no good
+ value, a good one corresponds to an expected size per connection multiplied
+ by the expected number of connections. The kernel may trim large values. See
+ also "tune.rcvbuf.client" and "tune.rcvbuf.server" for their connected socket
+ counter parts, as well as "tune.sndbuf.backend" and "tune.sndbuf.frontend"
+ for the send setting.
+
+tune.rcvbuf.client <number>
+tune.rcvbuf.server <number>
+ Forces the kernel socket receive buffer size on the client or the server side
+ to the specified value in bytes. This value applies to all TCP/HTTP frontends
+ and backends. It should normally never be set, and the default size (0) lets
+ the kernel auto-tune this value depending on the amount of available memory.
+ However it can sometimes help to set it to very low values (e.g. 4096) in
+ order to save kernel memory by preventing it from buffering too large amounts
+ of received data. Lower values will significantly increase CPU usage though.
+
+tune.recv_enough <number>
+ HAProxy uses some hints to detect that a short read indicates the end of the
+ socket buffers. One of them is that a read returns more than <recv_enough>
+ bytes, which defaults to 10136 (7 segments of 1448 each). This default value
+ may be changed by this setting to better deal with workloads involving lots
+ of short messages such as telnet or SSH sessions.
+
+tune.runqueue-depth <number>
+ Sets the maximum amount of task that can be processed at once when running
+ tasks. The default value depends on the number of threads but sits between 35
+ and 280, which tend to show the highest request rates and lowest latencies.
+ Increasing it may incur latency when dealing with I/Os, making it too small
+ can incur extra overhead. Higher thread counts benefit from lower values.
+ When experimenting with much larger values, it may be useful to also enable
+ tune.sched.low-latency and possibly tune.fd.edge-triggered to limit the
+ maximum latency to the lowest possible.
+
+tune.sched.low-latency { on | off }
+ Enables ('on') or disables ('off') the low-latency task scheduler. By default
+ HAProxy processes tasks from several classes one class at a time as this is
+ the most efficient. But when running with large values of tune.runqueue-depth
+ this can have a measurable effect on request or connection latency. When this
+ low-latency setting is enabled, tasks of lower priority classes will always
+ be executed before other ones if they exist. This will permit to lower the
+ maximum latency experienced by new requests or connections in the middle of
+ massive traffic, at the expense of a higher impact on this large traffic.
+ For regular usage it is better to leave this off. The default value is off.
+
+tune.sndbuf.backend <number>
+tune.sndbuf.frontend <number>
+ For the kernel socket send buffer size on non-connected sockets to this size.
+ This can be used for UNIX socket and UDP logging on the backend side, and for
+ QUIC in listener mode on the frontend. The default system buffers might
+ sometimes be too small for sockets shared between many connections (or log
+ senders), causing some losses and possibly retransmits, slowing down new
+ connection establishment under high traffic. The value is expressed in bytes,
+ applied to each socket. In listener mode, sockets are shared between all
+ connections, and the total number of sockets depends on the "shards" value of
+ the "bind" line. There's no good value, a good one corresponds to an expected
+ size per connection multiplied by the expected number of connections. The
+ kernel may trim large values. See also "tune.sndbuf.client" and
+ "tune.sndbuf.server" for their connected socket counter parts, as well as
+ "tune.rcvbuf.backend" and "tune.rcvbuf.frontend" for the receive setting.
+
+tune.sndbuf.client <number>
+tune.sndbuf.server <number>
+ Forces the kernel socket send buffer size on the client or the server side to
+ the specified value in bytes. This value applies to all TCP/HTTP frontends
+ and backends. It should normally never be set, and the default size (0) lets
+ the kernel auto-tune this value depending on the amount of available memory.
+ However it can sometimes help to set it to very low values (e.g. 4096) in
+ order to save kernel memory by preventing it from buffering too large amounts
+ of received data. Lower values will significantly increase CPU usage though.
+ Another use case is to prevent write timeouts with extremely slow clients due
+ to the kernel waiting for a large part of the buffer to be read before
+ notifying HAProxy again.
+
+tune.ssl.cachesize <number>
+ Sets the size of the global SSL session cache, in a number of blocks. A block
+ is large enough to contain an encoded session without peer certificate. An
+ encoded session with peer certificate is stored in multiple blocks depending
+ on the size of the peer certificate. A block uses approximately 200 bytes of
+ memory (based on `sizeof(struct sh_ssl_sess_hdr) + SHSESS_BLOCK_MIN_SIZE`
+ calculation used for `shctx_init` function). The default value may be forced
+ at build time, otherwise defaults to 20000. When the cache is full, the most
+ idle entries are purged and reassigned. Higher values reduce the occurrence
+ of such a purge, hence the number of CPU-intensive SSL handshakes by ensuring
+ that all users keep their session as long as possible. All entries are
+ pre-allocated upon startup. Setting this value to 0 disables the SSL session
+ cache.
+
+tune.ssl.capture-buffer-size <number>
+tune.ssl.capture-cipherlist-size <number> (deprecated)
+ Sets the maximum size of the buffer used for capturing client hello cipher
+ list, extensions list, elliptic curves list and elliptic curve point
+ formats. If the value is 0 (default value) the capture is disabled,
+ otherwise a buffer is allocated for each SSL/TLS connection.
+
+tune.ssl.default-dh-param <number>
+ Sets the maximum size of the Diffie-Hellman parameters used for generating
+ the ephemeral/temporary Diffie-Hellman key in case of DHE key exchange. The
+ final size will try to match the size of the server's RSA (or DSA) key (e.g,
+ a 2048 bits temporary DH key for a 2048 bits RSA key), but will not exceed
+ this maximum value. Only 1024 or higher values are allowed. Higher values
+ will increase the CPU load, and values greater than 1024 bits are not
+ supported by Java 7 and earlier clients. This value is not used if static
+ Diffie-Hellman parameters are supplied either directly in the certificate
+ file or by using the ssl-dh-param-file parameter.
+ If there is neither a default-dh-param nor a ssl-dh-param-file defined, and
+ if the server's PEM file of a given frontend does not specify its own DH
+ parameters, then DHE ciphers will be unavailable for this frontend.
+
+tune.ssl.force-private-cache
+ This option disables SSL session cache sharing between all processes. It
+ should normally not be used since it will force many renegotiations due to
+ clients hitting a random process. But it may be required on some operating
+ systems where none of the SSL cache synchronization method may be used. In
+ this case, adding a first layer of hash-based load balancing before the SSL
+ layer might limit the impact of the lack of session sharing.
+
+tune.ssl.hard-maxrecord <number>
+ Sets the maximum amount of bytes passed to SSL_write() at any time. Default
+ value 0 means there is no limit. In contrast to tune.ssl.maxrecord this
+ settings will not be adjusted dynamically. Smaller records may decrease
+ throughput, but may be required when dealing with low-footprint clients.
+
+tune.ssl.keylog { on | off }
+ This option activates the logging of the TLS keys. It should be used with
+ care as it will consume more memory per SSL session and could decrease
+ performances. This is disabled by default.
+
+ These sample fetches should be used to generate the SSLKEYLOGFILE that is
+ required to decipher traffic with wireshark.
+
+ https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/Key_Log_Format
+
+ The SSLKEYLOG is a series of lines which are formatted this way:
+
+ <Label> <space> <ClientRandom> <space> <Secret>
+
+ The ClientRandom is provided by the %[ssl_fc_client_random,hex] sample
+ fetch, the secret and the Label could be find in the array below. You need
+ to generate a SSLKEYLOGFILE with all the labels in this array.
+
+ The following sample fetches are hexadecimal strings and does not need to be
+ converted.
+
+ SSLKEYLOGFILE Label | Sample fetches for the Secrets
+ --------------------------------|-----------------------------------------
+ CLIENT_EARLY_TRAFFIC_SECRET | %[ssl_fc_client_early_traffic_secret]
+ CLIENT_HANDSHAKE_TRAFFIC_SECRET | %[ssl_fc_client_handshake_traffic_secret]
+ SERVER_HANDSHAKE_TRAFFIC_SECRET | %[ssl_fc_server_handshake_traffic_secret]
+ CLIENT_TRAFFIC_SECRET_0 | %[ssl_fc_client_traffic_secret_0]
+ SERVER_TRAFFIC_SECRET_0 | %[ssl_fc_server_traffic_secret_0]
+ EXPORTER_SECRET | %[ssl_fc_exporter_secret]
+ EARLY_EXPORTER_SECRET | %[ssl_fc_early_exporter_secret]
+
+ This is only available with OpenSSL 1.1.1, and useful with TLS1.3 session.
+
+ If you want to generate the content of a SSLKEYLOGFILE with TLS < 1.3, you
+ only need this line:
+
+ "CLIENT_RANDOM %[ssl_fc_client_random,hex] %[ssl_fc_session_key,hex]"
+
+tune.ssl.lifetime <timeout>
+ Sets how long a cached SSL session may remain valid. This time is expressed
+ in seconds and defaults to 300 (5 min). It is important to understand that it
+ does not guarantee that sessions will last that long, because if the cache is
+ full, the longest idle sessions will be purged despite their configured
+ lifetime. The real usefulness of this setting is to prevent sessions from
+ being used for too long.
+
+tune.ssl.maxrecord <number>
+ Sets the maximum amount of bytes passed to SSL_write() at the beginning of
+ the data transfer. Default value 0 means there is no limit. Over SSL/TLS,
+ the client can decipher the data only once it has received a full record.
+ With large records, it means that clients might have to download up to 16kB
+ of data before starting to process them. Limiting the value can improve page
+ load times on browsers located over high latency or low bandwidth networks.
+ It is suggested to find optimal values which fit into 1 or 2 TCP segments
+ (generally 1448 bytes over Ethernet with TCP timestamps enabled, or 1460 when
+ timestamps are disabled), keeping in mind that SSL/TLS add some overhead.
+ Typical values of 1419 and 2859 gave good results during tests. Use
+ "strace -e trace=write" to find the best value. HAProxy will automatically
+ switch to this setting after an idle stream has been detected (see
+ tune.idletimer above). See also tune.ssl.hard-maxrecord.
+
+tune.ssl.ssl-ctx-cache-size <number>
+ Sets the size of the cache used to store generated certificates to <number>
+ entries. This is a LRU cache. Because generating a SSL certificate
+ dynamically is expensive, they are cached. The default cache size is set to
+ 1000 entries.
+
+tune.ssl.ocsp-update.maxdelay <number>
+ Sets the maximum interval between two automatic updates of the same OCSP
+ response. This time is expressed in seconds and defaults to 3600 (1 hour). It
+ must be set to a higher value than "tune.ssl.ocsp-update.mindelay". See
+ option "ocsp-update" for more information about the auto update mechanism.
+
+tune.ssl.ocsp-update.mindelay <number>
+ Sets the minimum interval between two automatic updates of the same OCSP
+ response. This time is expressed in seconds and defaults to 300 (5 minutes).
+ It is particularly useful for OCSP response that do not have explicit
+ expiration times. It must be set to a lower value than
+ "tune.ssl.ocsp-update.maxdelay". See option "ocsp-update" for more
+ information about the auto update mechanism.
+
+tune.stick-counters <number>
+ Sets the number of stick-counters that may be tracked at the same time by a
+ connection or a request via "track-sc*" actions in "tcp-request" or
+ "http-request" rules. The default value is set at build time by the macro
+ MAX_SESS_STK_CTR, and defaults to 3. With this setting it is possible to
+ change the value and ignore the one passed at build time. Increasing this
+ value may be needed when porting complex configurations to haproxy, but users
+ are warned against the costs: each entry takes 16 bytes per connection and
+ 16 bytes per request, all of which need to be allocated and zeroed for all
+ requests even when not used. As such a value of 10 will inflate the memory
+ consumption per request by 320 bytes and will cause this memory to be erased
+ for each request, which does have measurable CPU impacts. Conversely, when
+ no "track-sc" rules are used, the value may be lowered (0 being valid to
+ entirely disable stick-counters).
+
+tune.vars.global-max-size <size>
+tune.vars.proc-max-size <size>
+tune.vars.reqres-max-size <size>
+tune.vars.sess-max-size <size>
+tune.vars.txn-max-size <size>
+ These five tunes help to manage the maximum amount of memory used by the
+ variables system. "global" limits the overall amount of memory available for
+ all scopes. "proc" limits the memory for the process scope, "sess" limits the
+ memory for the session scope, "txn" for the transaction scope, and "reqres"
+ limits the memory for each request or response processing.
+ Memory accounting is hierarchical, meaning more coarse grained limits include
+ the finer grained ones: "proc" includes "sess", "sess" includes "txn", and
+ "txn" includes "reqres".
+
+ For example, when "tune.vars.sess-max-size" is limited to 100,
+ "tune.vars.txn-max-size" and "tune.vars.reqres-max-size" cannot exceed
+ 100 either. If we create a variable "txn.var" that contains 100 bytes,
+ all available space is consumed.
+ Notice that exceeding the limits at runtime will not result in an error
+ message, but values might be cut off or corrupted. So make sure to accurately
+ plan for the amount of space needed to store all your variables.
+
+tune.zlib.memlevel <number>
+ Sets the memLevel parameter in zlib initialization for each stream. It
+ defines how much memory should be allocated for the internal compression
+ state. A value of 1 uses minimum memory but is slow and reduces compression
+ ratio, a value of 9 uses maximum memory for optimal speed. Can be a value
+ between 1 and 9. The default value is 8.
+
+tune.zlib.windowsize <number>
+ Sets the window size (the size of the history buffer) as a parameter of the
+ zlib initialization for each stream. Larger values of this parameter result
+ in better compression at the expense of memory usage. Can be a value between
+ 8 and 15. The default value is 15.
+
+3.3. Debugging
+--------------
+
+anonkey <key>
+ This sets the global anonymizing key to <key>, which must be a 32-bit number
+ between 0 and 4294967295. This is the key that will be used by default by CLI
+ commands when anonymized mode is enabled. This key may also be set at runtime
+ from the CLI command "set anon global-key". See also command line argument
+ "-dC" in the management manual.
+
+quick-exit
+ This speeds up the old process exit upon reload by skipping the releasing of
+ memory objects and listeners, since all of these are reclaimed by the
+ operating system at the process' death. The gains are only marginal (in the
+ order of a few hundred milliseconds for huge configurations at most). The
+ main target usage in fact is when a bug is spotted in the deinit() code, as
+ this allows to bypass it. It is better not to use this unless instructed to
+ do so by developers.
+
+quiet
+ Do not display any message during startup. It is equivalent to the command-
+ line argument "-q".
+
+zero-warning
+ When this option is set, HAProxy will refuse to start if any warning was
+ emitted while processing the configuration. It is highly recommended to set
+ this option on configurations that are not changed often, as it helps detect
+ subtle mistakes and keep the configuration clean and forward-compatible. Note
+ that "haproxy -c" will also report errors in such a case. This option is
+ equivalent to command line argument "-dW".
+
+
+3.4. Userlists
+--------------
+It is possible to control access to frontend/backend/listen sections or to
+http stats by allowing only authenticated and authorized users. To do this,
+it is required to create at least one userlist and to define users.
+
+userlist <listname>
+ Creates new userlist with name <listname>. Many independent userlists can be
+ used to store authentication & authorization data for independent customers.
+
+group <groupname> [users <user>,<user>,(...)]
+ Adds group <groupname> to the current userlist. It is also possible to
+ attach users to this group by using a comma separated list of names
+ proceeded by "users" keyword.
+
+user <username> [password|insecure-password <password>]
+ [groups <group>,<group>,(...)]
+ Adds user <username> to the current userlist. Both secure (encrypted) and
+ insecure (unencrypted) passwords can be used. Encrypted passwords are
+ evaluated using the crypt(3) function, so depending on the system's
+ capabilities, different algorithms are supported. For example, modern Glibc
+ based Linux systems support MD5, SHA-256, SHA-512, and, of course, the
+ classic DES-based method of encrypting passwords.
+
+ Attention: Be aware that using encrypted passwords might cause significantly
+ increased CPU usage, depending on the number of requests, and the algorithm
+ used. For any of the hashed variants, the password for each request must
+ be processed through the chosen algorithm, before it can be compared to the
+ value specified in the config file. Most current algorithms are deliberately
+ designed to be expensive to compute to achieve resistance against brute
+ force attacks. They do not simply salt/hash the clear text password once,
+ but thousands of times. This can quickly become a major factor in HAProxy's
+ overall CPU consumption!
+
+ Example:
+ userlist L1
+ group G1 users tiger,scott
+ group G2 users xdb,scott
+
+ user tiger password $6$k6y3o.eP$JlKBx9za9667qe4(...)xHSwRv6J.C0/D7cV91
+ user scott insecure-password elgato
+ user xdb insecure-password hello
+
+ userlist L2
+ group G1
+ group G2
+
+ user tiger password $6$k6y3o.eP$JlKBx(...)xHSwRv6J.C0/D7cV91 groups G1
+ user scott insecure-password elgato groups G1,G2
+ user xdb insecure-password hello groups G2
+
+ Please note that both lists are functionally identical.
+
+
+3.5. Peers
+----------
+It is possible to propagate entries of any data-types in stick-tables between
+several HAProxy instances over TCP connections in a multi-master fashion. Each
+instance pushes its local updates and insertions to remote peers. The pushed
+values overwrite remote ones without aggregation. As an exception, the data
+type "conn_cur" is never learned from peers, as it is supposed to reflect local
+values. Earlier versions used to synchronize it and to cause negative values in
+active-active setups, and always-growing values upon reloads or active-passive
+switches because the local value would reflect more connections than locally
+present. This information, however, is pushed so that monitoring systems can
+watch it.
+
+Interrupted exchanges are automatically detected and recovered from the last
+known point. In addition, during a soft restart, the old process connects to
+the new one using such a TCP connection to push all its entries before the new
+process tries to connect to other peers. That ensures very fast replication
+during a reload, it typically takes a fraction of a second even for large
+tables.
+
+Note that Server IDs are used to identify servers remotely, so it is important
+that configurations look similar or at least that the same IDs are forced on
+each server on all participants.
+
+peers <peersect>
+ Creates a new peer list with name <peersect>. It is an independent section,
+ which is referenced by one or more stick-tables.
+
+bind [<address>]:port [param*]
+bind /<path> [param*]
+ Defines the binding parameters of the local peer of this "peers" section.
+ Such lines are not supported with "peer" line in the same "peers" section.
+
+disabled
+ Disables a peers section. It disables both listening and any synchronization
+ related to this section. This is provided to disable synchronization of stick
+ tables without having to comment out all "peers" references.
+
+default-bind [param*]
+ Defines the binding parameters for the local peer, excepted its address.
+
+default-server [param*]
+ Change default options for a server in a "peers" section.
+
+ Arguments:
+ <param*> is a list of parameters for this server. The "default-server"
+ keyword accepts an important number of options and has a complete
+ section dedicated to it. In a peers section, the transport
+ parameters of a "default-server" line are supported. Please refer
+ to section 5 for more details, and the "server" keyword below in
+ this section for some of the restrictions.
+
+ See also: "server" and section 5 about server options
+
+enabled
+ This re-enables a peers section which was previously disabled via the
+ "disabled" keyword.
+
+log <target> [len <length>] [format <format>] [sample <ranges>:<sample_size>]
+ <facility> [<level> [<minlevel>]]
+ "peers" sections support the same "log" keyword as for the proxies to
+ log information about the "peers" listener. See "log" option for proxies for
+ more details.
+
+peer <peername> [<address>]:port [param*]
+peer <peername> /<path> [param*]
+ Defines a peer inside a peers section.
+ If <peername> is set to the local peer name (by default hostname, or forced
+ using "-L" command line option or "localpeer" global configuration setting),
+ HAProxy will listen for incoming remote peer connection on the provided
+ address. Otherwise, the address defines where to connect to in order to join
+ the remote peer, and <peername> is used at the protocol level to identify and
+ validate the remote peer on the server side.
+
+ During a soft restart, local peer address is used by the old instance to
+ connect the new one and initiate a complete replication (teaching process).
+
+ It is strongly recommended to have the exact same peers declaration on all
+ peers and to only rely on the "-L" command line argument or the "localpeer"
+ global configuration setting to change the local peer name. This makes it
+ easier to maintain coherent configuration files across all peers.
+
+ You may want to reference some environment variables in the address
+ parameter, see section 2.3 about environment variables.
+
+ Note: "peer" keyword may transparently be replaced by "server" keyword (see
+ "server" keyword explanation below).
+
+server <peername> [<address>:<port>] [param*]
+server <peername> [/<path>] [param*]
+ As previously mentioned, "peer" keyword may be replaced by "server" keyword
+ with a support for all "server" parameters found in 5.2 paragraph that are
+ related to transport settings. If the underlying peer is local, the address
+ parameter must not be present; it must be provided on a "bind" line (see
+ "bind" keyword of this "peers" section).
+
+ A number of "server" parameters are irrelevant for "peers" sections. Peers by
+ nature do not support dynamic host name resolution nor health checks, hence
+ parameters like "init_addr", "resolvers", "check", "agent-check", or "track"
+ are not supported. Similarly, there is no load balancing nor stickiness, thus
+ parameters such as "weight" or "cookie" have no effect.
+
+ Example:
+ # The old way.
+ peers mypeers
+ peer haproxy1 192.168.0.1:1024
+ peer haproxy2 192.168.0.2:1024
+ peer haproxy3 10.2.0.1:1024
+
+ backend mybackend
+ mode tcp
+ balance roundrobin
+ stick-table type ip size 20k peers mypeers
+ stick on src
+
+ server srv1 192.168.0.30:80
+ server srv2 192.168.0.31:80
+
+ Example:
+ peers mypeers
+ bind 192.168.0.1:1024 ssl crt mycerts/pem
+ default-server ssl verify none
+ server haproxy1 #local peer
+ server haproxy2 192.168.0.2:1024
+ server haproxy3 10.2.0.1:1024
+
+shards <shards>
+
+ In some configurations, one would like to distribute the stick-table contents
+ to some peers in place of sending all the stick-table contents to each peer
+ declared in the "peers" section. In such cases, "shards" specifies the
+ number of peer involved in this stick-table contents distribution.
+ See also "shard" server parameter.
+
+table <tablename> type {ip | integer | string [len <length>] | binary [len <length>]}
+ size <size> [expire <expire>] [write-to <wtable>] [nopurge] [store <data_type>]*
+
+ Configure a stickiness table for the current section. This line is parsed
+ exactly the same way as the "stick-table" keyword in others section, except
+ for the "peers" argument which is not required here and with an additional
+ mandatory first parameter to designate the stick-table. Contrary to others
+ sections, there may be several "table" lines in "peers" sections (see also
+ "stick-table" keyword).
+
+ Also be aware of the fact that "peers" sections have their own stick-table
+ namespaces to avoid collisions between stick-table names identical in
+ different "peers" section. This is internally handled prepending the "peers"
+ sections names to the name of the stick-tables followed by a '/' character.
+ If somewhere else in the configuration file you have to refer to such
+ stick-tables declared in "peers" sections you must use the prefixed version
+ of the stick-table name as follows:
+
+ peers mypeers
+ peer A ...
+ peer B ...
+ table t1 ...
+
+ frontend fe1
+ tcp-request content track-sc0 src table mypeers/t1
+
+ This is also this prefixed version of the stick-table names which must be
+ used to refer to stick-tables through the CLI.
+
+ About "peers" protocol, as only "peers" belonging to the same section may
+ communicate with each others, there is no need to do such a distinction.
+ Several "peers" sections may declare stick-tables with the same name.
+ This is shorter version of the stick-table name which is sent over the network.
+ There is only a '/' character as prefix to avoid stick-table name collisions between
+ stick-tables declared as backends and stick-table declared in "peers" sections
+ as follows in this weird but supported configuration:
+
+ peers mypeers
+ peer A ...
+ peer B ...
+ table t1 type string size 10m store gpc0
+
+ backend t1
+ stick-table type string size 10m store gpc0 peers mypeers
+
+ Here "t1" table declared in "mypeers" section has "mypeers/t1" as global name.
+ "t1" table declared as a backend as "t1" as global name. But at peer protocol
+ level the former table is named "/t1", the latter is again named "t1".
+
+3.6. Mailers
+------------
+It is possible to send email alerts when the state of servers changes.
+If configured email alerts are sent to each mailer that is configured
+in a mailers section. Email is sent to mailers using SMTP.
+
+mailers <mailersect>
+ Creates a new mailer list with the name <mailersect>. It is an
+ independent section which is referenced by one or more proxies.
+
+mailer <mailername> <ip>:<port>
+ Defines a mailer inside a mailers section.
+
+ Example:
+ mailers mymailers
+ mailer smtp1 192.168.0.1:587
+ mailer smtp2 192.168.0.2:587
+
+ backend mybackend
+ mode tcp
+ balance roundrobin
+
+ email-alert mailers mymailers
+ email-alert from test1@horms.org
+ email-alert to test2@horms.org
+
+ server srv1 192.168.0.30:80
+ server srv2 192.168.0.31:80
+
+timeout mail <time>
+ Defines the time available for a mail/connection to be made and send to
+ the mail-server. If not defined the default value is 10 seconds. To allow
+ for at least two SYN-ACK packets to be send during initial TCP handshake it
+ is advised to keep this value above 4 seconds.
+
+ Example:
+ mailers mymailers
+ timeout mail 20s
+ mailer smtp1 192.168.0.1:587
+
+3.7. Programs
+-------------
+In master-worker mode, it is possible to launch external binaries with the
+master, these processes are called programs. These programs are launched and
+managed the same way as the workers.
+
+During a reload of HAProxy, those processes are dealing with the same
+sequence as a worker:
+
+ - the master is re-executed
+ - the master sends a SIGUSR1 signal to the program
+ - if "option start-on-reload" is not disabled, the master launches a new
+ instance of the program
+
+During a stop, or restart, a SIGTERM is sent to the programs.
+
+program <name>
+ This is a new program section, this section will create an instance <name>
+ which is visible in "show proc" on the master CLI. (See "9.4. Master CLI" in
+ the management guide).
+
+command <command> [arguments*]
+ Define the command to start with optional arguments. The command is looked
+ up in the current PATH if it does not include an absolute path. This is a
+ mandatory option of the program section. Arguments containing spaces must
+ be enclosed in quotes or double quotes or be prefixed by a backslash.
+
+user <user name>
+ Changes the executed command user ID to the <user name> from /etc/passwd.
+ See also "group".
+
+group <group name>
+ Changes the executed command group ID to the <group name> from /etc/group.
+ See also "user".
+
+option start-on-reload
+no option start-on-reload
+ Start (or not) a new instance of the program upon a reload of the master.
+ The default is to start a new instance. This option may only be used in a
+ program section.
+
+
+3.8. HTTP-errors
+----------------
+
+It is possible to globally declare several groups of HTTP errors, to be
+imported afterwards in any proxy section. Same group may be referenced at
+several places and can be fully or partially imported.
+
+http-errors <name>
+ Create a new http-errors group with the name <name>. It is an independent
+ section that may be referenced by one or more proxies using its name.
+
+errorfile <code> <file>
+ Associate a file contents to an HTTP error code
+
+ Arguments :
+ <code> is the HTTP status code. Currently, HAProxy is capable of
+ generating codes 200, 400, 401, 403, 404, 405, 407, 408, 410,
+ 425, 429, 500, 501, 502, 503, and 504.
+
+ <file> designates a file containing the full HTTP response. It is
+ recommended to follow the common practice of appending ".http" to
+ the filename so that people do not confuse the response with HTML
+ error pages, and to use absolute paths, since files are read
+ before any chroot is performed.
+
+ Please referrers to "errorfile" keyword in section 4 for details.
+
+ Example:
+ http-errors website-1
+ errorfile 400 /etc/haproxy/errorfiles/site1/400.http
+ errorfile 404 /etc/haproxy/errorfiles/site1/404.http
+ errorfile 408 /dev/null # work around Chrome pre-connect bug
+
+ http-errors website-2
+ errorfile 400 /etc/haproxy/errorfiles/site2/400.http
+ errorfile 404 /etc/haproxy/errorfiles/site2/404.http
+ errorfile 408 /dev/null # work around Chrome pre-connect bug
+
+3.9. Rings
+----------
+
+It is possible to globally declare ring-buffers, to be used as target for log
+servers or traces.
+
+ring <ringname>
+ Creates a new ring-buffer with name <ringname>.
+
+backing-file <path>
+ This replaces the regular memory allocation by a RAM-mapped file to store the
+ ring. This can be useful for collecting traces or logs for post-mortem
+ analysis, without having to attach a slow client to the CLI. Newer contents
+ will automatically replace older ones so that the latest contents are always
+ available. The contents written to the ring will be visible in that file once
+ the process stops (most often they will even be seen very soon after but
+ there is no such guarantee since writes are not synchronous).
+
+ When this option is used, the total storage area is reduced by the size of
+ the "struct ring" that starts at the beginning of the area, and that is
+ required to recover the area's contents. The file will be created with the
+ starting user's ownership, with mode 0600 and will be of the size configured
+ by the "size" directive. When the directive is parsed (thus even during
+ config checks), any existing non-empty file will first be renamed with the
+ extra suffix ".bak", and any previously existing file with suffix ".bak" will
+ be removed. This ensures that instant reload or restart of the process will
+ not wipe precious debugging information, and will leave time for an admin to
+ spot this new ".bak" file and to archive it if needed. As such, after a crash
+ the file designated by <path> will contain the freshest information, and if
+ the service is restarted, the "<path>.bak" file will have it instead. This
+ means that the total storage capacity required will be double of the ring
+ size. Failures to rotate the file are silently ignored, so placing the file
+ into a directory without write permissions will be sufficient to avoid the
+ backup file if not desired.
+
+ WARNING: there are stability and security implications in using this feature.
+ First, backing the ring to a slow device (e.g. physical hard drive) may cause
+ perceptible slowdowns during accesses, and possibly even panics if too many
+ threads compete for accesses. Second, an external process modifying the area
+ could cause the haproxy process to crash or to overwrite some of its own
+ memory with traces. Third, if the file system fills up before the ring,
+ writes to the ring may cause the process to crash.
+
+ The information present in this ring are structured and are NOT directly
+ readable using a text editor (even though most of it looks barely readable).
+ The output of this file is only intended for developers.
+
+description <text>
+ The description is an optional description string of the ring. It will
+ appear on CLI. By default, <name> is reused to fill this field.
+
+format <format>
+ Format used to store events into the ring buffer.
+
+ Arguments:
+ <format> is the log format used when generating syslog messages. It may be
+ one of the following :
+
+ iso A message containing only the ISO date, followed by the text.
+ The PID, process name and system name are omitted. This is
+ designed to be used with a local log server.
+
+ local Analog to rfc3164 syslog message format except that hostname
+ field is stripped. This is the default.
+ Note: option "log-send-hostname" switches the default to
+ rfc3164.
+
+ raw A message containing only the text. The level, PID, date, time,
+ process name and system name are omitted. This is designed to be
+ used in containers or during development, where the severity
+ only depends on the file descriptor used (stdout/stderr). This
+ is the default.
+
+ rfc3164 The RFC3164 syslog message format.
+ (https://tools.ietf.org/html/rfc3164)
+
+ rfc5424 The RFC5424 syslog message format.
+ (https://tools.ietf.org/html/rfc5424)
+
+ short A message containing only a level between angle brackets such as
+ '<3>', followed by the text. The PID, date, time, process name
+ and system name are omitted. This is designed to be used with a
+ local log server. This format is compatible with what the systemd
+ logger consumes.
+
+ priority A message containing only a level plus syslog facility between angle
+ brackets such as '<63>', followed by the text. The PID, date, time,
+ process name and system name are omitted. This is designed to be used
+ with a local log server.
+
+ timed A message containing only a level between angle brackets such as
+ '<3>', followed by ISO date and by the text. The PID, process
+ name and system name are omitted. This is designed to be
+ used with a local log server.
+
+maxlen <length>
+ The maximum length of an event message stored into the ring,
+ including formatted header. If an event message is longer than
+ <length>, it will be truncated to this length.
+
+server <name> <address> [param*]
+ Used to configure a syslog tcp server to forward messages from ring buffer.
+ This supports for all "server" parameters found in 5.2 paragraph. Some of
+ these parameters are irrelevant for "ring" sections. Important point: there
+ is little reason to add more than one server to a ring, because all servers
+ will receive the exact same copy of the ring contents, and as such the ring
+ will progress at the speed of the slowest server. If one server does not
+ respond, it will prevent old messages from being purged and may block new
+ messages from being inserted into the ring. The proper way to send messages
+ to multiple servers is to use one distinct ring per log server, not to
+ attach multiple servers to the same ring. Note that specific server directive
+ "log-proto" is used to set the protocol used to send messages.
+
+size <size>
+ This is the optional size in bytes for the ring-buffer. Default value is
+ set to BUFSIZE.
+
+timeout connect <timeout>
+ Set the maximum time to wait for a connection attempt to a server to succeed.
+
+ Arguments :
+ <timeout> is the timeout value specified in milliseconds by default, but
+ can be in any other unit if the number is suffixed by the unit,
+ as explained at the top of this document.
+
+timeout server <timeout>
+ Set the maximum time for pending data staying into output buffer.
+
+ Arguments :
+ <timeout> is the timeout value specified in milliseconds by default, but
+ can be in any other unit if the number is suffixed by the unit,
+ as explained at the top of this document.
+
+ Example:
+ global
+ log ring@myring local7
+
+ ring myring
+ description "My local buffer"
+ format rfc3164
+ maxlen 1200
+ size 32764
+ timeout connect 5s
+ timeout server 10s
+ server mysyslogsrv 127.0.0.1:6514 log-proto octet-count
+
+3.10. Log forwarding
+-------------------
+
+It is possible to declare one or multiple log forwarding section,
+HAProxy will forward all received log messages to a log servers list.
+
+log-forward <name>
+ Creates a new log forwarder proxy identified as <name>.
+
+backlog <conns>
+ Give hints to the system about the approximate listen backlog desired size
+ on connections accept.
+
+bind <addr> [param*]
+ Used to configure a stream log listener to receive messages to forward.
+ This supports the "bind" parameters found in 5.1 paragraph including
+ those about ssl but some statements such as "alpn" may be irrelevant for
+ syslog protocol over TCP.
+ Those listeners support both "Octet Counting" and "Non-Transparent-Framing"
+ modes as defined in rfc-6587.
+
+dgram-bind <addr> [param*]
+ Used to configure a datagram log listener to receive messages to forward.
+ Addresses must be in IPv4 or IPv6 form,followed by a port. This supports
+ for some of the "bind" parameters found in 5.1 paragraph among which
+ "interface", "namespace" or "transparent", the other ones being
+ silently ignored as irrelevant for UDP/syslog case.
+
+log global
+log <target> [len <length>] [format <format>] [sample <ranges>:<sample_size>]
+ <facility> [<level> [<minlevel>]]
+ Used to configure target log servers. See more details on proxies
+ documentation.
+ If no format specified, HAProxy tries to keep the incoming log format.
+ Configured facility is ignored, except if incoming message does not
+ present a facility but one is mandatory on the outgoing format.
+ If there is no timestamp available in the input format, but the field
+ exists in output format, HAProxy will use the local date.
+
+ Example:
+ global
+ log stderr format iso local7
+
+ ring myring
+ description "My local buffer"
+ format rfc5424
+ maxlen 1200
+ size 32764
+ timeout connect 5s
+ timeout server 10s
+ # syslog tcp server
+ server mysyslogsrv 127.0.0.1:514 log-proto octet-count
+
+ log-forward sylog-loadb
+ dgram-bind 127.0.0.1:1514
+ bind 127.0.0.1:1514
+ # all messages on stderr
+ log global
+ # all messages on local tcp syslog server
+ log ring@myring local0
+ # load balance messages on 4 udp syslog servers
+ log 127.0.0.1:10001 sample 1:4 local0
+ log 127.0.0.1:10002 sample 2:4 local0
+ log 127.0.0.1:10003 sample 3:4 local0
+ log 127.0.0.1:10004 sample 4:4 local0
+
+maxconn <conns>
+ Fix the maximum number of concurrent connections on a log forwarder.
+ 10 is the default.
+
+timeout client <timeout>
+ Set the maximum inactivity time on the client side.
+
+3.11. HTTPClient tuning
+-----------------------
+
+HTTPClient is an internal HTTP library, it can be used by various subsystems,
+for example in LUA scripts. HTTPClient is not used in the data path, in other
+words it has nothing with HTTP traffic passing through HAProxy.
+
+httpclient.resolvers.disabled <on|off>
+ Disable the DNS resolution of the httpclient. Prevent the creation of the
+ "default" resolvers section.
+
+ Default value is off.
+
+httpclient.resolvers.id <resolvers id>
+ This option defines the resolvers section with which the httpclient will try
+ to resolve.
+
+ Default option is the "default" resolvers ID. By default, if this option is
+ not used, it will simply disable the resolving if the section is not found.
+
+ However, when this option is explicitly enabled it will trigger a
+ configuration error if it fails to load.
+
+httpclient.resolvers.prefer <ipv4|ipv6>
+ This option allows to chose which family of IP you want when resolving,
+ which is convenient when IPv6 is not available on your network. Default
+ option is "ipv6".
+
+httpclient.retries <number>
+ This option allows to configure the number of retries attempt of the
+ httpclient when a request failed. This does the same as the "retries" keyword
+ in a backend.
+
+ Default value is 3.
+
+httpclient.ssl.ca-file <cafile>
+ This option defines the ca-file which should be used to verify the server
+ certificate. It takes the same parameters as the "ca-file" option on the
+ server line.
+
+ By default and when this option is not used, the value is
+ "@system-ca" which tries to load the CA of the system. If it fails the SSL
+ will be disabled for the httpclient.
+
+ However, when this option is explicitly enabled it will trigger a
+ configuration error if it fails.
+
+httpclient.ssl.verify [none|required]
+ Works the same way as the verify option on server lines. If specified to 'none',
+ servers certificates are not verified. Default option is "required".
+
+ By default and when this option is not used, the value is
+ "required". If it fails the SSL will be disabled for the httpclient.
+
+ However, when this option is explicitly enabled it will trigger a
+ configuration error if it fails.
+
+httpclient.timeout.connect <timeout>
+ Set the maximum time to wait for a connection attempt by default for the
+ httpclient.
+
+ Arguments :
+ <timeout> is the timeout value specified in milliseconds by default, but
+ can be in any other unit if the number is suffixed by the unit,
+ as explained at the top of this document.
+
+ The default value is 5000ms.
+
+4. Proxies
+----------
+
+Proxy configuration can be located in a set of sections :
+ - defaults [<name>] [ from <defaults_name> ]
+ - frontend <name> [ from <defaults_name> ]
+ - backend <name> [ from <defaults_name> ]
+ - listen <name> [ from <defaults_name> ]
+
+A "frontend" section describes a set of listening sockets accepting client
+connections.
+
+A "backend" section describes a set of servers to which the proxy will connect
+to forward incoming connections.
+
+A "listen" section defines a complete proxy with its frontend and backend
+parts combined in one section. It is generally useful for TCP-only traffic.
+
+A "defaults" section resets all settings to the documented ones and presets new
+ones for use by subsequent sections. All of "frontend", "backend" and "listen"
+sections always take their initial settings from a defaults section, by default
+the latest one that appears before the newly created section. It is possible to
+explicitly designate a specific "defaults" section to load the initial settings
+from by indicating its name on the section line after the optional keyword
+"from". While "defaults" section do not impose a name, this use is encouraged
+for better readability. It is also the only way to designate a specific section
+to use instead of the default previous one. Since "defaults" section names are
+optional, by default a very permissive check is applied on their name and these
+are even permitted to overlap. However if a "defaults" section is referenced by
+any other section, its name must comply with the syntax imposed on all proxy
+names, and this name must be unique among the defaults sections. Please note
+that regardless of what is currently permitted, it is recommended to avoid
+duplicate section names in general and to respect the same syntax as for proxy
+names. This rule might be enforced in a future version. In addition, a warning
+is emitted if a defaults section is explicitly used by a proxy while it is also
+implicitly used by another one because it is the last one defined. It is highly
+encouraged to not mix both usages by always using explicit references or by
+adding a last common defaults section reserved for all implicit uses.
+
+Note that it is even possible for a defaults section to take its initial
+settings from another one, and as such, inherit settings across multiple levels
+of defaults sections. This can be convenient to establish certain configuration
+profiles to carry groups of default settings (e.g. TCP vs HTTP or short vs long
+timeouts) but can quickly become confusing to follow.
+
+All proxy names must be formed from upper and lower case letters, digits,
+'-' (dash), '_' (underscore) , '.' (dot) and ':' (colon). ACL names are
+case-sensitive, which means that "www" and "WWW" are two different proxies.
+
+Historically, all proxy names could overlap, it just caused troubles in the
+logs. Since the introduction of content switching, it is mandatory that two
+proxies with overlapping capabilities (frontend/backend) have different names.
+However, it is still permitted that a frontend and a backend share the same
+name, as this configuration seems to be commonly encountered.
+
+Right now, two major proxy modes are supported : "tcp", also known as layer 4,
+and "http", also known as layer 7. In layer 4 mode, HAProxy simply forwards
+bidirectional traffic between two sides. In layer 7 mode, HAProxy analyzes the
+protocol, and can interact with it by allowing, blocking, switching, adding,
+modifying, or removing arbitrary contents in requests or responses, based on
+arbitrary criteria.
+
+In HTTP mode, the processing applied to requests and responses flowing over
+a connection depends in the combination of the frontend's HTTP options and
+the backend's. HAProxy supports 3 connection modes :
+
+ - KAL : keep alive ("option http-keep-alive") which is the default mode : all
+ requests and responses are processed, and connections remain open but idle
+ between responses and new requests.
+
+ - SCL: server close ("option http-server-close") : the server-facing
+ connection is closed after the end of the response is received, but the
+ client-facing connection remains open.
+
+ - CLO: close ("option httpclose"): the connection is closed after the end of
+ the response and "Connection: close" appended in both directions.
+
+The effective mode that will be applied to a connection passing through a
+frontend and a backend can be determined by both proxy modes according to the
+following matrix, but in short, the modes are symmetric, keep-alive is the
+weakest option and close is the strongest.
+
+ Backend mode
+
+ | KAL | SCL | CLO
+ ----+-----+-----+----
+ KAL | KAL | SCL | CLO
+ ----+-----+-----+----
+ mode SCL | SCL | SCL | CLO
+ ----+-----+-----+----
+ CLO | CLO | CLO | CLO
+
+It is possible to chain a TCP frontend to an HTTP backend. It is pointless if
+only HTTP traffic is handled. But it may be used to handle several protocols
+within the same frontend. In this case, the client's connection is first handled
+as a raw tcp connection before being upgraded to HTTP. Before the upgrade, the
+content processings are performend on raw data. Once upgraded, data is parsed
+and stored using an internal representation called HTX and it is no longer
+possible to rely on raw representation. There is no way to go back.
+
+There are two kind of upgrades, in-place upgrades and destructive upgrades. The
+first ones involves a TCP to HTTP/1 upgrade. In HTTP/1, the request
+processings are serialized, thus the applicative stream can be preserved. The
+second one involves a TCP to HTTP/2 upgrade. Because it is a multiplexed
+protocol, the applicative stream cannot be associated to any HTTP/2 stream and
+is destroyed. New applicative streams are then created when HAProxy receives
+new HTTP/2 streams at the lower level, in the H2 multiplexer. It is important
+to understand this difference because that drastically changes the way to
+process data. When an HTTP/1 upgrade is performed, the content processings
+already performed on raw data are neither lost nor reexecuted while for an
+HTTP/2 upgrade, applicative streams are distinct and all frontend rules are
+evaluated systematically on each one. And as said, the first stream, the TCP
+one, is destroyed, but only after the frontend rules were evaluated.
+
+There is another importnat point to understand when HTTP processings are
+performed from a TCP proxy. While HAProxy is able to parse HTTP/1 in-fly from
+tcp-request content rules, it is not possible for HTTP/2. Only the HTTP/2
+preface can be parsed. This is a huge limitation regarding the HTTP content
+analysis in TCP. Concretely it is only possible to know if received data are
+HTTP. For instance, it is not possible to choose a backend based on the Host
+header value while it is trivial in HTTP/1. Hopefully, there is a solution to
+mitigate this drawback.
+
+There are two ways to perform an HTTP upgrade. The first one, the historical
+method, is to select an HTTP backend. The upgrade happens when the backend is
+set. Thus, for in-place upgrades, only the backend configuration is considered
+in the HTTP data processing. For destructive upgrades, the applicative stream
+is destroyed, thus its processing is stopped. With this method, possibilities
+to choose a backend with an HTTP/2 connection are really limited, as mentioned
+above, and a bit useless because the stream is destroyed. The second method is
+to upgrade during the tcp-request content rules evaluation, thanks to the
+"switch-mode http" action. In this case, the upgrade is performed in the
+frontend context and it is possible to define HTTP directives in this
+frontend. For in-place upgrades, it offers all the power of the HTTP analysis
+as soon as possible. It is not that far from an HTTP frontend. For destructive
+upgrades, it does not change anything except it is useless to choose a backend
+on limited information. It is of course the recommended method. Thus, testing
+the request protocol from the tcp-request content rules to perform an HTTP
+upgrade is enough. All the remaining HTTP manipulation may be moved to the
+frontend http-request ruleset. But keep in mind that tcp-request content rules
+remains evaluated on each streams, that can't be changed.
+
+4.1. Proxy keywords matrix
+--------------------------
+
+The following list of keywords is supported. Most of them may only be used in a
+limited set of section types. Some of them are marked as "deprecated" because
+they are inherited from an old syntax which may be confusing or functionally
+limited, and there are new recommended keywords to replace them. Keywords
+marked with "(*)" can be optionally inverted using the "no" prefix, e.g. "no
+option contstats". This makes sense when the option has been enabled by default
+and must be disabled for a specific instance. Such options may also be prefixed
+with "default" in order to restore default settings regardless of what has been
+specified in a previous "defaults" section. Keywords supported in defaults
+sections marked with "(!)" are only supported in named defaults sections, not
+anonymous ones.
+
+
+ keyword defaults frontend listen backend
+------------------------------------+----------+----------+---------+---------
+acl X (!) X X X
+backlog X X X -
+balance X - X X
+bind - X X -
+capture cookie - X X -
+capture request header - X X -
+capture response header - X X -
+clitcpka-cnt X X X -
+clitcpka-idle X X X -
+clitcpka-intvl X X X -
+compression X X X X
+cookie X - X X
+declare capture - X X -
+default-server X - X X
+default_backend X X X -
+description - X X X
+disabled X X X X
+dispatch - - X X
+email-alert from X X X X
+email-alert level X X X X
+email-alert mailers X X X X
+email-alert myhostname X X X X
+email-alert to X X X X
+enabled X X X X
+errorfile X X X X
+errorfiles X X X X
+errorloc X X X X
+errorloc302 X X X X
+-- keyword -------------------------- defaults - frontend - listen -- backend -
+errorloc303 X X X X
+error-log-format X X X -
+force-persist - - X X
+filter - X X X
+fullconn X - X X
+hash-type X - X X
+http-after-response X (!) X X X
+http-check comment X - X X
+http-check connect X - X X
+http-check disable-on-404 X - X X
+http-check expect X - X X
+http-check send X - X X
+http-check send-state X - X X
+http-check set-var X - X X
+http-check unset-var X - X X
+http-error X X X X
+http-request X (!) X X X
+http-response X (!) X X X
+http-reuse X - X X
+http-send-name-header X - X X
+id - X X X
+ignore-persist - - X X
+load-server-state-from-file X - X X
+log (*) X X X X
+log-format X X X -
+log-format-sd X X X -
+log-tag X X X X
+max-keep-alive-queue X - X X
+max-session-srv-conns X X X -
+maxconn X X X -
+mode X X X X
+monitor fail - X X -
+monitor-uri X X X -
+option abortonclose (*) X - X X
+option accept-invalid-http-request (*) X X X -
+option accept-invalid-http-response (*) X - X X
+option allbackups (*) X - X X
+option checkcache (*) X - X X
+option clitcpka (*) X X X -
+option contstats (*) X X X -
+option disable-h2-upgrade (*) X X X -
+option dontlog-normal (*) X X X -
+option dontlognull (*) X X X -
+-- keyword -------------------------- defaults - frontend - listen -- backend -
+option forwardfor X X X X
+option forwarded (*) X - X X
+option h1-case-adjust-bogus-client (*) X X X -
+option h1-case-adjust-bogus-server (*) X - X X
+option http-buffer-request (*) X X X X
+option http-ignore-probes (*) X X X -
+option http-keep-alive (*) X X X X
+option http-no-delay (*) X X X X
+option http-pretend-keepalive (*) X - X X
+option http-restrict-req-hdr-names X X X X
+option http-server-close (*) X X X X
+option http-use-proxy-header (*) X X X -
+option httpchk X - X X
+option httpclose (*) X X X X
+option httplog X X X -
+option httpslog X X X -
+option independent-streams (*) X X X X
+option ldap-check X - X X
+option external-check X - X X
+option log-health-checks (*) X - X X
+option log-separate-errors (*) X X X -
+option logasap (*) X X X -
+option mysql-check X - X X
+option nolinger (*) X X X X
+option originalto X X X X
+option persist (*) X - X X
+option pgsql-check X - X X
+option prefer-last-server (*) X - X X
+option redispatch (*) X - X X
+option redis-check X - X X
+option smtpchk X - X X
+option socket-stats (*) X X X -
+option splice-auto (*) X X X X
+option splice-request (*) X X X X
+option splice-response (*) X X X X
+option spop-check X - X X
+option srvtcpka (*) X - X X
+option ssl-hello-chk X - X X
+-- keyword -------------------------- defaults - frontend - listen -- backend -
+option tcp-check X - X X
+option tcp-smart-accept (*) X X X -
+option tcp-smart-connect (*) X - X X
+option tcpka X X X X
+option tcplog X X X X
+option transparent (*) X - X X
+option idle-close-on-response (*) X X X -
+external-check command X - X X
+external-check path X - X X
+persist rdp-cookie X - X X
+rate-limit sessions X X X -
+redirect - X X X
+-- keyword -------------------------- defaults - frontend - listen -- backend -
+retries X - X X
+retry-on X - X X
+server - - X X
+server-state-file-name X - X X
+server-template - - X X
+source X - X X
+srvtcpka-cnt X - X X
+srvtcpka-idle X - X X
+srvtcpka-intvl X - X X
+stats admin - X X X
+stats auth X X X X
+stats enable X X X X
+stats hide-version X X X X
+stats http-request - X X X
+stats realm X X X X
+stats refresh X X X X
+stats scope X X X X
+stats show-desc X X X X
+stats show-legends X X X X
+stats show-node X X X X
+stats uri X X X X
+-- keyword -------------------------- defaults - frontend - listen -- backend -
+stick match - - X X
+stick on - - X X
+stick store-request - - X X
+stick store-response - - X X
+stick-table - X X X
+tcp-check comment X - X X
+tcp-check connect X - X X
+tcp-check expect X - X X
+tcp-check send X - X X
+tcp-check send-lf X - X X
+tcp-check send-binary X - X X
+tcp-check send-binary-lf X - X X
+tcp-check set-var X - X X
+tcp-check unset-var X - X X
+tcp-request connection X (!) X X -
+tcp-request content X (!) X X X
+tcp-request inspect-delay X (!) X X X
+tcp-request session X (!) X X -
+tcp-response content X (!) - X X
+tcp-response inspect-delay X (!) - X X
+timeout check X - X X
+timeout client X X X -
+timeout client-fin X X X -
+timeout client-hs X X X -
+timeout connect X - X X
+timeout http-keep-alive X X X X
+timeout http-request X X X X
+timeout queue X - X X
+timeout server X - X X
+timeout server-fin X - X X
+timeout tarpit X X X X
+timeout tunnel X - X X
+transparent (deprecated) X - X X
+unique-id-format X X X -
+unique-id-header X X X -
+use_backend - X X -
+use-fcgi-app - - X X
+use-server - - X X
+------------------------------------+----------+----------+---------+---------
+ keyword defaults frontend listen backend
+
+
+4.2. Alphabetically sorted keywords reference
+---------------------------------------------
+
+This section provides a description of each keyword and its usage.
+
+
+acl <aclname> <criterion> [flags] [operator] <value> ...
+ Declare or complete an access list.
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes(!) | yes | yes | yes
+
+ This directive is only available from named defaults sections, not anonymous
+ ones. ACLs defined in a defaults section are not visible from other sections
+ using it.
+
+ Example:
+ acl invalid_src src 0.0.0.0/7 224.0.0.0/3
+ acl invalid_src src_port 0:1023
+ acl local_dst hdr(host) -i localhost
+
+ See section 7 about ACL usage.
+
+
+backlog <conns>
+ Give hints to the system about the approximate listen backlog desired size
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments :
+ <conns> is the number of pending connections. Depending on the operating
+ system, it may represent the number of already acknowledged
+ connections, of non-acknowledged ones, or both.
+
+ This option is only meaningful for stream listeners, including QUIC ones. Its
+ behavior however is not identical with QUIC instances.
+
+ For all listeners but QUIC, in order to protect against SYN flood attacks,
+ one solution is to increase the system's SYN backlog size. Depending on the
+ system, sometimes it is just tunable via a system parameter, sometimes it is
+ not adjustable at all, and sometimes the system relies on hints given by the
+ application at the time of the listen() syscall. By default, HAProxy passes
+ the frontend's maxconn value to the listen() syscall. On systems which can
+ make use of this value, it can sometimes be useful to be able to specify a
+ different value, hence this backlog parameter.
+
+ On Linux 2.4, the parameter is ignored by the system. On Linux 2.6, it is
+ used as a hint and the system accepts up to the smallest greater power of
+ two, and never more than some limits (usually 32768).
+
+ For QUIC listeners, backlog sets a shared limits for both the maximum count
+ of active handshakes and connections waiting to be accepted. The handshake
+ phase relies primarily of the network latency with the remote peer, whereas
+ the second phase depends solely on haproxy load. When either one of this
+ limit is reached, haproxy starts to drop reception of INITIAL packets,
+ preventing any new connection allocation, until the connection excess starts
+ to decrease. This situation may cause browsers to silently downgrade the HTTP
+ versions and switching to TCP.
+
+ See also : "maxconn" and the target operating system's tuning guide.
+
+
+balance <algorithm> [ <arguments> ]
+balance url_param <param> [check_post]
+ Define the load balancing algorithm to be used in a backend.
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <algorithm> is the algorithm used to select a server when doing load
+ balancing. This only applies when no persistence information
+ is available, or when a connection is redispatched to another
+ server. <algorithm> may be one of the following :
+
+ roundrobin Each server is used in turns, according to their weights.
+ This is the smoothest and fairest algorithm when the server's
+ processing time remains equally distributed. This algorithm
+ is dynamic, which means that server weights may be adjusted
+ on the fly for slow starts for instance. It is limited by
+ design to 4095 active servers per backend. Note that in some
+ large farms, when a server becomes up after having been down
+ for a very short time, it may sometimes take a few hundreds
+ requests for it to be re-integrated into the farm and start
+ receiving traffic. This is normal, though very rare. It is
+ indicated here in case you would have the chance to observe
+ it, so that you don't worry. Note: weights are ignored for
+ backends in LOG mode.
+
+ static-rr Each server is used in turns, according to their weights.
+ This algorithm is as similar to roundrobin except that it is
+ static, which means that changing a server's weight on the
+ fly will have no effect. On the other hand, it has no design
+ limitation on the number of servers, and when a server goes
+ up, it is always immediately reintroduced into the farm, once
+ the full map is recomputed. It also uses slightly less CPU to
+ run (around -1%). This algorithm is not usable in LOG mode.
+
+ leastconn The server with the lowest number of connections receives the
+ connection. Round-robin is performed within groups of servers
+ of the same load to ensure that all servers will be used. Use
+ of this algorithm is recommended where very long sessions are
+ expected, such as LDAP, SQL, TSE, etc... but is not very well
+ suited for protocols using short sessions such as HTTP. This
+ algorithm is dynamic, which means that server weights may be
+ adjusted on the fly for slow starts for instance. It will
+ also consider the number of queued connections in addition to
+ the established ones in order to minimize queuing. This
+ algorithm is not usable in LOG mode.
+
+ first The first server with available connection slots receives the
+ connection. The servers are chosen from the lowest numeric
+ identifier to the highest (see server parameter "id"), which
+ defaults to the server's position in the farm. Once a server
+ reaches its maxconn value, the next server is used. It does
+ not make sense to use this algorithm without setting maxconn.
+ The purpose of this algorithm is to always use the smallest
+ number of servers so that extra servers can be powered off
+ during non-intensive hours. This algorithm ignores the server
+ weight, and brings more benefit to long session such as RDP
+ or IMAP than HTTP, though it can be useful there too. In
+ order to use this algorithm efficiently, it is recommended
+ that a cloud controller regularly checks server usage to turn
+ them off when unused, and regularly checks backend queue to
+ turn new servers on when the queue inflates. Alternatively,
+ using "http-check send-state" may inform servers on the load.
+
+ hash Takes a regular sample expression in argument. The expression
+ is evaluated for each request and hashed according to the
+ configured hash-type. The result of the hash is divided by
+ the total weight of the running servers to designate which
+ server will receive the request. This can be used in place of
+ "source", "uri", "hdr()", "url_param()", "rdp-cookie" to make
+ use of a converter, refine the evaluation, or be used to
+ extract data from local variables for example. When the data
+ is not available, round robin will apply. This algorithm is
+ static by default, which means that changing a server's
+ weight on the fly will have no effect, but this can be
+ changed using "hash-type". This algorithm is not usable for
+ backends in LOG mode, please use "log-hash" instead.
+
+ source The source IP address is hashed and divided by the total
+ weight of the running servers to designate which server will
+ receive the request. This ensures that the same client IP
+ address will always reach the same server as long as no
+ server goes down or up. If the hash result changes due to the
+ number of running servers changing, many clients will be
+ directed to a different server. This algorithm is generally
+ used in TCP mode where no cookie may be inserted. It may also
+ be used on the Internet to provide a best-effort stickiness
+ to clients which refuse session cookies. This algorithm is
+ static by default, which means that changing a server's
+ weight on the fly will have no effect, but this can be
+ changed using "hash-type". See also the "hash" option above.
+ This algorithm is not usable for backends in LOG mode.
+
+ uri This algorithm hashes either the left part of the URI (before
+ the question mark) or the whole URI (if the "whole" parameter
+ is present) and divides the hash value by the total weight of
+ the running servers. The result designates which server will
+ receive the request. This ensures that the same URI will
+ always be directed to the same server as long as no server
+ goes up or down. This is used with proxy caches and
+ anti-virus proxies in order to maximize the cache hit rate.
+ Note that this algorithm may only be used in an HTTP backend.
+ This algorithm is static by default, which means that
+ changing a server's weight on the fly will have no effect,
+ but this can be changed using "hash-type".
+
+ This algorithm supports two optional parameters "len" and
+ "depth", both followed by a positive integer number. These
+ options may be helpful when it is needed to balance servers
+ based on the beginning of the URI only. The "len" parameter
+ indicates that the algorithm should only consider that many
+ characters at the beginning of the URI to compute the hash.
+ Note that having "len" set to 1 rarely makes sense since most
+ URIs start with a leading "/".
+
+ The "depth" parameter indicates the maximum directory depth
+ to be used to compute the hash. One level is counted for each
+ slash in the request. If both parameters are specified, the
+ evaluation stops when either is reached.
+
+ A "path-only" parameter indicates that the hashing key starts
+ at the first '/' of the path. This can be used to ignore the
+ authority part of absolute URIs, and to make sure that HTTP/1
+ and HTTP/2 URIs will provide the same hash. See also the
+ "hash" option above.
+
+ url_param The URL parameter specified in argument will be looked up in
+ the query string of each HTTP GET request.
+
+ If the modifier "check_post" is used, then an HTTP POST
+ request entity will be searched for the parameter argument,
+ when it is not found in a query string after a question mark
+ ('?') in the URL. The message body will only start to be
+ analyzed once either the advertised amount of data has been
+ received or the request buffer is full. In the unlikely event
+ that chunked encoding is used, only the first chunk is
+ scanned. Parameter values separated by a chunk boundary, may
+ be randomly balanced if at all. This keyword used to support
+ an optional <max_wait> parameter which is now ignored.
+
+ If the parameter is found followed by an equal sign ('=') and
+ a value, then the value is hashed and divided by the total
+ weight of the running servers. The result designates which
+ server will receive the request.
+
+ This is used to track user identifiers in requests and ensure
+ that a same user ID will always be sent to the same server as
+ long as no server goes up or down. If no value is found or if
+ the parameter is not found, then a round robin algorithm is
+ applied. Note that this algorithm may only be used in an HTTP
+ backend. This algorithm is static by default, which means
+ that changing a server's weight on the fly will have no
+ effect, but this can be changed using "hash-type". See also
+ the "hash" option above.
+
+ hdr(<name>) The HTTP header <name> will be looked up in each HTTP
+ request. Just as with the equivalent ACL 'hdr()' function,
+ the header name in parenthesis is not case sensitive. If the
+ header is absent or if it does not contain any value, the
+ roundrobin algorithm is applied instead.
+
+ An optional 'use_domain_only' parameter is available, for
+ reducing the hash algorithm to the main domain part with some
+ specific headers such as 'Host'. For instance, in the Host
+ value "haproxy.1wt.eu", only "1wt" will be considered.
+
+ This algorithm is static by default, which means that
+ changing a server's weight on the fly will have no effect,
+ but this can be changed using "hash-type". See also the
+ "hash" option above.
+
+ random
+ random(<draws>)
+ A random number will be used as the key for the consistent
+ hashing function. This means that the servers' weights are
+ respected, dynamic weight changes immediately take effect, as
+ well as new server additions. Random load balancing can be
+ useful with large farms or when servers are frequently added
+ or removed as it may avoid the hammering effect that could
+ result from roundrobin or leastconn in this situation. The
+ hash-balance-factor directive can be used to further improve
+ fairness of the load balancing, especially in situations
+ where servers show highly variable response times. When an
+ argument <draws> is present, it must be an integer value one
+ or greater, indicating the number of draws before selecting
+ the least loaded of these servers. It was indeed demonstrated
+ that picking the least loaded of two servers is enough to
+ significantly improve the fairness of the algorithm, by
+ always avoiding to pick the most loaded server within a farm
+ and getting rid of any bias that could be induced by the
+ unfair distribution of the consistent list. Higher values N
+ will take away N-1 of the highest loaded servers at the
+ expense of performance. With very high values, the algorithm
+ will converge towards the leastconn's result but much slower.
+ The default value is 2, which generally shows very good
+ distribution and performance. This algorithm is also known as
+ the Power of Two Random Choices and is described here :
+ http://www.eecs.harvard.edu/~michaelm/postscripts/handbook2001.pdf
+
+ For backends in LOG mode, the number of draws is ignored and
+ a single random is picked since there is no notion of server
+ load. Random log balancing can be useful with large farms or
+ when servers are frequently added or removed from the pool of
+ available servers as it may avoid the hammering effect that
+ could result from roundrobin in this situation.
+
+ rdp-cookie
+ rdp-cookie(<name>)
+ The RDP cookie <name> (or "mstshash" if omitted) will be
+ looked up and hashed for each incoming TCP request. Just as
+ with the equivalent ACL 'req.rdp_cookie()' function, the name
+ is not case-sensitive. This mechanism is useful as a degraded
+ persistence mode, as it makes it possible to always send the
+ same user (or the same session ID) to the same server. If the
+ cookie is not found, the normal roundrobin algorithm is
+ used instead.
+
+ Note that for this to work, the frontend must ensure that an
+ RDP cookie is already present in the request buffer. For this
+ you must use 'tcp-request content accept' rule combined with
+ a 'req.rdp_cookie_cnt' ACL.
+
+ This algorithm is static by default, which means that
+ changing a server's weight on the fly will have no effect,
+ but this can be changed using "hash-type". See also the
+ "hash" option above.
+
+ log-hash Takes a comma-delimited list of converters in argument. These
+ converters are applied in sequence to the input log message,
+ and the result will be cast as a string then hashed according
+ to the configured hash-type. The resulting hash will be used
+ to select the destination server among the ones declared in
+ the log backend. The goal of this algorithm is to be able to
+ extract a key within the final log message using string
+ converters and then be able to stick to the same server thanks
+ to the hash. Only "map-based" hashes are supported for now.
+ This algorithm is only usable for backends in LOG mode, for
+ others, please use "hash" instead.
+
+ sticky Tries to stick to the same server as much as possible. The
+ first server in the list of available servers receives all
+ the log messages. When the server goes DOWN, the next server
+ in the list takes its place. When a previously DOWN server
+ goes back UP it is added at the end of the list so that the
+ sticky server doesn't change until it becomes DOWN. This
+ algorithm is only usable for backends in LOG mode.
+
+ <arguments> is an optional list of arguments which may be needed by some
+ algorithms. Right now, only "url_param", "uri" and "log-hash"
+ support an optional argument.
+
+ The load balancing algorithm of a backend is set to roundrobin when no other
+ algorithm, mode nor option have been set. The algorithm may only be set once
+ for each backend. In backends in LOG mode, server "weight" is always ignored.
+
+ With authentication schemes that require the same connection like NTLM, URI
+ based algorithms must not be used, as they would cause subsequent requests
+ to be routed to different backend servers, breaking the invalid assumptions
+ NTLM relies on.
+
+ TCP/HTTP Examples :
+ balance roundrobin
+ balance url_param userid
+ balance url_param session_id check_post 64
+ balance hdr(User-Agent)
+ balance hdr(host)
+ balance hdr(Host) use_domain_only
+ balance hash req.cookie(clientid)
+ balance hash var(req.client_id)
+ balance hash req.hdr_ip(x-forwarded-for,-1),ipmask(24)
+
+ LOG backend examples:
+ global
+ log backend@mylog-rrb local0 # send all logs to mylog-rrb backend
+ log backend@mylog-hash local0 # send all logs to mylog-hash backend
+
+ backend mylog-rrb
+ mode log
+ balance roundrobin
+
+ server s1 udp@127.0.0.1:514 # will receive 50% of log messages
+ server s2 udp@127.0.0.1:514
+
+ backend mylog-hash
+ mode log
+
+ # extract "METHOD URL PROTO" at the end of the log message,
+ # and let haproxy hash it so that log messages generated from
+ # similar requests get sent to the same syslog server:
+ balance log-hash 'field(-2,\")'
+
+ # server list here
+ server s1 127.0.0.1:514
+ #...
+
+ Note: the following caveats and limitations on using the "check_post"
+ extension with "url_param" must be considered :
+
+ - all POST requests are eligible for consideration, because there is no way
+ to determine if the parameters will be found in the body or entity which
+ may contain binary data. Therefore another method may be required to
+ restrict consideration of POST requests that have no URL parameters in
+ the body. (see acl http_end)
+
+ - using a <max_wait> value larger than the request buffer size does not
+ make sense and is useless. The buffer size is set at build time, and
+ defaults to 16 kB.
+
+ - Content-Encoding is not supported, the parameter search will probably
+ fail; and load balancing will fall back to Round Robin.
+
+ - Expect: 100-continue is not supported, load balancing will fall back to
+ Round Robin.
+
+ - Transfer-Encoding (RFC7230 3.3.1) is only supported in the first chunk.
+ If the entire parameter value is not present in the first chunk, the
+ selection of server is undefined (actually, defined by how little
+ actually appeared in the first chunk).
+
+ - This feature does not support generation of a 100, 411 or 501 response.
+
+ - In some cases, requesting "check_post" MAY attempt to scan the entire
+ contents of a message body. Scanning normally terminates when linear
+ white space or control characters are found, indicating the end of what
+ might be a URL parameter list. This is probably not a concern with SGML
+ type message bodies.
+
+ See also : "dispatch", "cookie", "transparent", "hash-type".
+
+
+bind [<address>]:<port_range> [, ...] [param*]
+bind /<path> [, ...] [param*]
+ Define one or several listening addresses and/or ports in a frontend.
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ no | yes | yes | no
+
+ Arguments :
+ <address> is optional and can be a host name, an IPv4 address, an IPv6
+ address, or '*'. It designates the address the frontend will
+ listen on. If unset, all IPv4 addresses of the system will be
+ listened on. The same will apply for '*' or the system's
+ special address "0.0.0.0". The IPv6 equivalent is '::'. Note
+ that if you bind a frontend to multiple UDP addresses you have
+ no guarantee about the address which will be used to respond.
+ This is why "0.0.0.0" addresses and lists of comma-separated
+ IP addresses have been forbidden to bind QUIC addresses.
+ Optionally, an address family prefix may be used before the
+ address to force the family regardless of the address format,
+ which can be useful to specify a path to a unix socket with
+ no slash ('/'). Currently supported prefixes are :
+ - 'ipv4@' -> address is always IPv4
+ - 'ipv6@' -> address is always IPv6
+ - 'udp@' -> address is resolved as IPv4 or IPv6 and
+ protocol UDP is used. Currently those listeners are
+ supported only in log-forward sections.
+ - 'udp4@' -> address is always IPv4 and protocol UDP
+ is used. Currently those listeners are supported
+ only in log-forward sections.
+ - 'udp6@' -> address is always IPv6 and protocol UDP
+ is used. Currently those listeners are supported
+ only in log-forward sections.
+ - 'unix@' -> address is a path to a local unix socket
+ - 'abns@' -> address is in abstract namespace (Linux only).
+ - 'fd@<n>' -> use file descriptor <n> inherited from the
+ parent. The fd must be bound and may or may not already
+ be listening.
+ - 'sockpair@<n>'-> like fd@ but you must use the fd of a
+ connected unix socket or of a socketpair. The bind waits
+ to receive a FD over the unix socket and uses it as if it
+ was the FD of an accept(). Should be used carefully.
+ - 'quic4@' -> address is resolved as IPv4 and protocol UDP
+ is used. Note that to achieve the best performance with a
+ large traffic you should keep "tune.quic.socket-owner" on
+ connection. Else QUIC connections will be multiplexed
+ over the listener socket. Another alternative would be to
+ duplicate QUIC listener instances over several threads,
+ for example using "shards" keyword to at least reduce
+ thread contention.
+ - 'quic6@' -> address is resolved as IPv6 and protocol UDP
+ is used. The performance note for QUIC over IPv4 applies
+ as well.
+ - 'rhttp@' [ EXPERIMENTAL ] -> used for reverse HTTP.
+ Address must be a server with the format
+ '<backend>/<server>'. The server will be used to
+ instantiate connections to a remote address. The listener
+ will try to maintain "nbconn" connections. This is an
+ experimental features which requires
+ "expose-experimental-directives" on a line before this
+ bind.
+
+ You may want to reference some environment variables in the
+ address parameter, see section 2.3 about environment
+ variables.
+
+ <port_range> is either a unique TCP port, or a port range for which the
+ proxy will accept connections for the IP address specified
+ above. The port is mandatory for TCP listeners. Note that in
+ the case of an IPv6 address, the port is always the number
+ after the last colon (':'). A range can either be :
+ - a numerical port (ex: '80')
+ - a dash-delimited ports range explicitly stating the lower
+ and upper bounds (ex: '2000-2100') which are included in
+ the range.
+
+ Particular care must be taken against port ranges, because
+ every <address:port> couple consumes one socket (= a file
+ descriptor), so it's easy to consume lots of descriptors
+ with a simple range, and to run out of sockets. Also, each
+ <address:port> couple must be used only once among all
+ instances running on a same system. Please note that binding
+ to ports lower than 1024 generally require particular
+ privileges to start the program, which are independent of
+ the 'uid' parameter.
+
+ <path> is a UNIX socket path beginning with a slash ('/'). This is
+ alternative to the TCP listening port. HAProxy will then
+ receive UNIX connections on the socket located at this place.
+ The path must begin with a slash and by default is absolute.
+ It can be relative to the prefix defined by "unix-bind" in
+ the global section. Note that the total length of the prefix
+ followed by the socket path cannot exceed some system limits
+ for UNIX sockets, which commonly are set to 107 characters.
+
+ <param*> is a list of parameters common to all sockets declared on the
+ same line. These numerous parameters depend on OS and build
+ options and have a complete section dedicated to them. Please
+ refer to section 5 to for more details.
+
+ It is possible to specify a list of address:port combinations delimited by
+ commas. The frontend will then listen on all of these addresses. There is no
+ fixed limit to the number of addresses and ports which can be listened on in
+ a frontend, as well as there is no limit to the number of "bind" statements
+ in a frontend.
+
+ Example :
+ listen http_proxy
+ bind :80,:443
+ bind 10.0.0.1:10080,10.0.0.1:10443
+ bind /var/run/ssl-frontend.sock user root mode 600 accept-proxy
+
+ listen http_https_proxy
+ bind :80
+ bind :443 ssl crt /etc/haproxy/site.pem
+
+ listen http_https_proxy_explicit
+ bind ipv6@:80
+ bind ipv4@public_ssl:443 ssl crt /etc/haproxy/site.pem
+ bind unix@ssl-frontend.sock user root mode 600 accept-proxy
+
+ listen external_bind_app1
+ bind "fd@${FD_APP1}"
+
+ listen h3_quic_proxy
+ bind quic4@10.0.0.1:8888 ssl crt /etc/mycrt
+
+ Note: regarding Linux's abstract namespace sockets, HAProxy uses the whole
+ sun_path length is used for the address length. Some other programs
+ such as socat use the string length only by default. Pass the option
+ ",unix-tightsocklen=0" to any abstract socket definition in socat to
+ make it compatible with HAProxy's.
+
+ See also : "source", "option forwardfor", "unix-bind" and the PROXY protocol
+ documentation, and section 5 about bind options.
+
+
+capture cookie <name> len <length>
+ Capture and log a cookie in the request and in the response.
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ no | yes | yes | no
+
+ Arguments :
+ <name> is the beginning of the name of the cookie to capture. In order
+ to match the exact name, simply suffix the name with an equal
+ sign ('='). The full name will appear in the logs, which is
+ useful with application servers which adjust both the cookie name
+ and value (e.g. ASPSESSIONXXX).
+
+ <length> is the maximum number of characters to report in the logs, which
+ include the cookie name, the equal sign and the value, all in the
+ standard "name=value" form. The string will be truncated on the
+ right if it exceeds <length>.
+
+ Only the first cookie is captured. Both the "cookie" request headers and the
+ "set-cookie" response headers are monitored. This is particularly useful to
+ check for application bugs causing session crossing or stealing between
+ users, because generally the user's cookies can only change on a login page.
+
+ When the cookie was not presented by the client, the associated log column
+ will report "-". When a request does not cause a cookie to be assigned by the
+ server, a "-" is reported in the response column.
+
+ The capture is performed in the frontend only because it is necessary that
+ the log format does not change for a given frontend depending on the
+ backends. This may change in the future. Note that there can be only one
+ "capture cookie" statement in a frontend. The maximum capture length is set
+ by the global "tune.http.cookielen" setting and defaults to 63 characters. It
+ is not possible to specify a capture in a "defaults" section.
+
+ Example:
+ capture cookie ASPSESSION len 32
+
+ See also : "capture request header", "capture response header" as well as
+ section 8 about logging.
+
+
+capture request header <name> len <length>
+ Capture and log the last occurrence of the specified request header.
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ no | yes | yes | no
+
+ Arguments :
+ <name> is the name of the header to capture. The header names are not
+ case-sensitive, but it is a common practice to write them as they
+ appear in the requests, with the first letter of each word in
+ upper case. The header name will not appear in the logs, only the
+ value is reported, but the position in the logs is respected.
+
+ <length> is the maximum number of characters to extract from the value and
+ report in the logs. The string will be truncated on the right if
+ it exceeds <length>.
+
+ The complete value of the last occurrence of the header is captured. The
+ value will be added to the logs between braces ('{}'). If multiple headers
+ are captured, they will be delimited by a vertical bar ('|') and will appear
+ in the same order they were declared in the configuration. Non-existent
+ headers will be logged just as an empty string. Common uses for request
+ header captures include the "Host" field in virtual hosting environments, the
+ "Content-length" when uploads are supported, "User-agent" to quickly
+ differentiate between real users and robots, and "X-Forwarded-For" in proxied
+ environments to find where the request came from.
+
+ Note that when capturing headers such as "User-agent", some spaces may be
+ logged, making the log analysis more difficult. Thus be careful about what
+ you log if you know your log parser is not smart enough to rely on the
+ braces.
+
+ There is no limit to the number of captured request headers nor to their
+ length, though it is wise to keep them low to limit memory usage per stream.
+ In order to keep log format consistent for a same frontend, header captures
+ can only be declared in a frontend. It is not possible to specify a capture
+ in a "defaults" section.
+
+ Example:
+ capture request header Host len 15
+ capture request header X-Forwarded-For len 15
+ capture request header Referer len 15
+
+ See also : "capture cookie", "capture response header" as well as section 8
+ about logging.
+
+
+capture response header <name> len <length>
+ Capture and log the last occurrence of the specified response header.
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ no | yes | yes | no
+
+ Arguments :
+ <name> is the name of the header to capture. The header names are not
+ case-sensitive, but it is a common practice to write them as they
+ appear in the response, with the first letter of each word in
+ upper case. The header name will not appear in the logs, only the
+ value is reported, but the position in the logs is respected.
+
+ <length> is the maximum number of characters to extract from the value and
+ report in the logs. The string will be truncated on the right if
+ it exceeds <length>.
+
+ The complete value of the last occurrence of the header is captured. The
+ result will be added to the logs between braces ('{}') after the captured
+ request headers. If multiple headers are captured, they will be delimited by
+ a vertical bar ('|') and will appear in the same order they were declared in
+ the configuration. Non-existent headers will be logged just as an empty
+ string. Common uses for response header captures include the "Content-length"
+ header which indicates how many bytes are expected to be returned, the
+ "Location" header to track redirections.
+
+ There is no limit to the number of captured response headers nor to their
+ length, though it is wise to keep them low to limit memory usage per stream.
+ In order to keep log format consistent for a same frontend, header captures
+ can only be declared in a frontend. It is not possible to specify a capture
+ in a "defaults" section.
+
+ Example:
+ capture response header Content-length len 9
+ capture response header Location len 15
+
+ See also : "capture cookie", "capture request header" as well as section 8
+ about logging.
+
+
+clitcpka-cnt <count>
+ Sets the maximum number of keepalive probes TCP should send before dropping
+ the connection on the client side.
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments :
+ <count> is the maximum number of keepalive probes.
+
+ This keyword corresponds to the socket option TCP_KEEPCNT. If this keyword
+ is not specified, system-wide TCP parameter (tcp_keepalive_probes) is used.
+ The availability of this setting depends on the operating system. It is
+ known to work on Linux.
+
+ See also : "option clitcpka", "clitcpka-idle", "clitcpka-intvl".
+
+
+clitcpka-idle <timeout>
+ Sets the time the connection needs to remain idle before TCP starts sending
+ keepalive probes, if enabled the sending of TCP keepalive packets on the
+ client side.
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments :
+ <timeout> is the time the connection needs to remain idle before TCP starts
+ sending keepalive probes. It is specified in seconds by default,
+ but can be in any other unit if the number is suffixed by the
+ unit, as explained at the top of this document.
+
+ This keyword corresponds to the socket option TCP_KEEPIDLE. If this keyword
+ is not specified, system-wide TCP parameter (tcp_keepalive_time) is used.
+ The availability of this setting depends on the operating system. It is
+ known to work on Linux.
+
+ See also : "option clitcpka", "clitcpka-cnt", "clitcpka-intvl".
+
+
+clitcpka-intvl <timeout>
+ Sets the time between individual keepalive probes on the client side.
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments :
+ <timeout> is the time between individual keepalive probes. It is specified
+ in seconds by default, but can be in any other unit if the number
+ is suffixed by the unit, as explained at the top of this
+ document.
+
+ This keyword corresponds to the socket option TCP_KEEPINTVL. If this keyword
+ is not specified, system-wide TCP parameter (tcp_keepalive_intvl) is used.
+ The availability of this setting depends on the operating system. It is
+ known to work on Linux.
+
+ See also : "option clitcpka", "clitcpka-cnt", "clitcpka-idle".
+
+
+compression algo <algorithm> ...
+compression algo-req <algorithm>
+compression algo-res <algorithm>
+compression type <mime type> ...
+ Enable HTTP compression.
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments :
+ algo is followed by the list of supported compression algorithms for
+ responses (legacy keyword)
+ algo-req is followed by compression algorithm for request (only one is
+ provided).
+ algo-res is followed by the list of supported compression algorithms for
+ responses.
+ type is followed by the list of MIME types that will be compressed for
+ responses (legacy keyword).
+ type-req is followed by the list of MIME types that will be compressed for
+ requests.
+ type-res is followed by the list of MIME types that will be compressed for
+ responses.
+
+ The currently supported algorithms are :
+ identity this is mostly for debugging, and it was useful for developing
+ the compression feature. Identity does not apply any change on
+ data.
+
+ gzip applies gzip compression. This setting is only available when
+ support for zlib or libslz was built in.
+
+ deflate same as "gzip", but with deflate algorithm and zlib format.
+ Note that this algorithm has ambiguous support on many
+ browsers and no support at all from recent ones. It is
+ strongly recommended not to use it for anything else than
+ experimentation. This setting is only available when support
+ for zlib or libslz was built in.
+
+ raw-deflate same as "deflate" without the zlib wrapper, and used as an
+ alternative when the browser wants "deflate". All major
+ browsers understand it and despite violating the standards,
+ it is known to work better than "deflate", at least on MSIE
+ and some versions of Safari. Do not use it in conjunction
+ with "deflate", use either one or the other since both react
+ to the same Accept-Encoding token. This setting is only
+ available when support for zlib or libslz was built in.
+
+ Compression will be activated depending on the Accept-Encoding request
+ header. With identity, it does not take care of that header.
+ If backend servers support HTTP compression, these directives
+ will be no-op: HAProxy will see the compressed response and will not
+ compress again. If backend servers do not support HTTP compression and
+ there is Accept-Encoding header in request, HAProxy will compress the
+ matching response.
+
+ Compression is disabled when:
+ * the request does not advertise a supported compression algorithm in the
+ "Accept-Encoding" header
+ * the response message is not HTTP/1.1 or above
+ * HTTP status code is not one of 200, 201, 202, or 203
+ * response contain neither a "Content-Length" header nor a
+ "Transfer-Encoding" whose last value is "chunked"
+ * response contains a "Content-Type" header whose first value starts with
+ "multipart"
+ * the response contains the "no-transform" value in the "Cache-control"
+ header
+ * User-Agent matches "Mozilla/4" unless it is MSIE 6 with XP SP2, or MSIE 7
+ and later
+ * The response contains a "Content-Encoding" header, indicating that the
+ response is already compressed (see compression offload)
+ * The response contains an invalid "ETag" header or multiple ETag headers
+
+ Note: The compression does not emit the Warning header.
+
+ Examples :
+ compression algo gzip
+ compression type text/html text/plain
+
+ See also : "compression offload", "compression direction"
+
+compression offload
+ Makes HAProxy work as a compression offloader only.
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ no | yes | yes | yes
+
+ The "offload" setting makes HAProxy remove the Accept-Encoding header to
+ prevent backend servers from compressing responses. It is strongly
+ recommended not to do this because this means that all the compression work
+ will be done on the single point where HAProxy is located. However in some
+ deployment scenarios, HAProxy may be installed in front of a buggy gateway
+ with broken HTTP compression implementation which can't be turned off.
+ In that case HAProxy can be used to prevent that gateway from emitting
+ invalid payloads. In this case, simply removing the header in the
+ configuration does not work because it applies before the header is parsed,
+ so that prevents HAProxy from compressing. The "offload" setting should
+ then be used for such scenarios.
+
+ If this setting is used in a defaults section, a warning is emitted and the
+ option is ignored.
+
+ See also : "compression type", "compression algo", "compression direction"
+
+compression direction <direction>
+ Makes haproxy able to compress both requests and responses.
+ Valid values are "request", to compress only requests, "response", to
+ compress only responses, or "both", when you want to compress both.
+ The default value is "response".
+
+ May be used in the following contexts: http
+
+ See also : "compression type", "compression algo", "compression offload"
+
+cookie <name> [ rewrite | insert | prefix ] [ indirect ] [ nocache ]
+ [ postonly ] [ preserve ] [ httponly ] [ secure ]
+ [ domain <domain> ]* [ maxidle <idle> ] [ maxlife <life> ]
+ [ dynamic ] [ attr <value> ]*
+ Enable cookie-based persistence in a backend.
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <name> is the name of the cookie which will be monitored, modified or
+ inserted in order to bring persistence. This cookie is sent to
+ the client via a "Set-Cookie" header in the response, and is
+ brought back by the client in a "Cookie" header in all requests.
+ Special care should be taken to choose a name which does not
+ conflict with any likely application cookie. Also, if the same
+ backends are subject to be used by the same clients (e.g.
+ HTTP/HTTPS), care should be taken to use different cookie names
+ between all backends if persistence between them is not desired.
+
+ rewrite This keyword indicates that the cookie will be provided by the
+ server and that HAProxy will have to modify its value to set the
+ server's identifier in it. This mode is handy when the management
+ of complex combinations of "Set-cookie" and "Cache-control"
+ headers is left to the application. The application can then
+ decide whether or not it is appropriate to emit a persistence
+ cookie. Since all responses should be monitored, this mode
+ doesn't work in HTTP tunnel mode. Unless the application
+ behavior is very complex and/or broken, it is advised not to
+ start with this mode for new deployments. This keyword is
+ incompatible with "insert" and "prefix".
+
+ insert This keyword indicates that the persistence cookie will have to
+ be inserted by HAProxy in server responses if the client did not
+
+ already have a cookie that would have permitted it to access this
+ server. When used without the "preserve" option, if the server
+ emits a cookie with the same name, it will be removed before
+ processing. For this reason, this mode can be used to upgrade
+ existing configurations running in the "rewrite" mode. The cookie
+ will only be a session cookie and will not be stored on the
+ client's disk. By default, unless the "indirect" option is added,
+ the server will see the cookies emitted by the client. Due to
+ caching effects, it is generally wise to add the "nocache" or
+ "postonly" keywords (see below). The "insert" keyword is not
+ compatible with "rewrite" and "prefix".
+
+ prefix This keyword indicates that instead of relying on a dedicated
+ cookie for the persistence, an existing one will be completed.
+ This may be needed in some specific environments where the client
+ does not support more than one single cookie and the application
+ already needs it. In this case, whenever the server sets a cookie
+ named <name>, it will be prefixed with the server's identifier
+ and a delimiter. The prefix will be removed from all client
+ requests so that the server still finds the cookie it emitted.
+ Since all requests and responses are subject to being modified,
+ this mode doesn't work with tunnel mode. The "prefix" keyword is
+ not compatible with "rewrite" and "insert". Note: it is highly
+ recommended not to use "indirect" with "prefix", otherwise server
+ cookie updates would not be sent to clients.
+
+ indirect When this option is specified, no cookie will be emitted to a
+ client which already has a valid one for the server which has
+ processed the request. If the server sets such a cookie itself,
+ it will be removed, unless the "preserve" option is also set. In
+ "insert" mode, this will additionally remove cookies from the
+ requests transmitted to the server, making the persistence
+ mechanism totally transparent from an application point of view.
+ Note: it is highly recommended not to use "indirect" with
+ "prefix", otherwise server cookie updates would not be sent to
+ clients.
+
+ nocache This option is recommended in conjunction with the insert mode
+ when there is a cache between the client and HAProxy, as it
+ ensures that a cacheable response will be tagged non-cacheable if
+ a cookie needs to be inserted. This is important because if all
+ persistence cookies are added on a cacheable home page for
+ instance, then all customers will then fetch the page from an
+ outer cache and will all share the same persistence cookie,
+ leading to one server receiving much more traffic than others.
+ See also the "insert" and "postonly" options.
+
+ postonly This option ensures that cookie insertion will only be performed
+ on responses to POST requests. It is an alternative to the
+ "nocache" option, because POST responses are not cacheable, so
+ this ensures that the persistence cookie will never get cached.
+ Since most sites do not need any sort of persistence before the
+ first POST which generally is a login request, this is a very
+ efficient method to optimize caching without risking to find a
+ persistence cookie in the cache.
+ See also the "insert" and "nocache" options.
+
+ preserve This option may only be used with "insert" and/or "indirect". It
+ allows the server to emit the persistence cookie itself. In this
+ case, if a cookie is found in the response, HAProxy will leave it
+ untouched. This is useful in order to end persistence after a
+ logout request for instance. For this, the server just has to
+ emit a cookie with an invalid value (e.g. empty) or with a date in
+ the past. By combining this mechanism with the "disable-on-404"
+ check option, it is possible to perform a completely graceful
+ shutdown because users will definitely leave the server after
+ they logout.
+
+ httponly This option tells HAProxy to add an "HttpOnly" cookie attribute
+ when a cookie is inserted. This attribute is used so that a
+ user agent doesn't share the cookie with non-HTTP components.
+ Please check RFC6265 for more information on this attribute.
+
+ secure This option tells HAProxy to add a "Secure" cookie attribute when
+ a cookie is inserted. This attribute is used so that a user agent
+ never emits this cookie over non-secure channels, which means
+ that a cookie learned with this flag will be presented only over
+ SSL/TLS connections. Please check RFC6265 for more information on
+ this attribute.
+
+ domain This option allows to specify the domain at which a cookie is
+ inserted. It requires exactly one parameter: a valid domain
+ name. If the domain begins with a dot, the browser is allowed to
+ use it for any host ending with that name. It is also possible to
+ specify several domain names by invoking this option multiple
+ times. Some browsers might have small limits on the number of
+ domains, so be careful when doing that. For the record, sending
+ 10 domains to MSIE 6 or Firefox 2 works as expected.
+
+ maxidle This option allows inserted cookies to be ignored after some idle
+ time. It only works with insert-mode cookies. When a cookie is
+ sent to the client, the date this cookie was emitted is sent too.
+ Upon further presentations of this cookie, if the date is older
+ than the delay indicated by the parameter (in seconds), it will
+ be ignored. Otherwise, it will be refreshed if needed when the
+ response is sent to the client. This is particularly useful to
+ prevent users who never close their browsers from remaining for
+ too long on the same server (e.g. after a farm size change). When
+ this option is set and a cookie has no date, it is always
+ accepted, but gets refreshed in the response. This maintains the
+ ability for admins to access their sites. Cookies that have a
+ date in the future further than 24 hours are ignored. Doing so
+ lets admins fix timezone issues without risking kicking users off
+ the site.
+
+ maxlife This option allows inserted cookies to be ignored after some life
+ time, whether they're in use or not. It only works with insert
+ mode cookies. When a cookie is first sent to the client, the date
+ this cookie was emitted is sent too. Upon further presentations
+ of this cookie, if the date is older than the delay indicated by
+ the parameter (in seconds), it will be ignored. If the cookie in
+ the request has no date, it is accepted and a date will be set.
+ Cookies that have a date in the future further than 24 hours are
+ ignored. Doing so lets admins fix timezone issues without risking
+ kicking users off the site. Contrary to maxidle, this value is
+ not refreshed, only the first visit date counts. Both maxidle and
+ maxlife may be used at the time. This is particularly useful to
+ prevent users who never close their browsers from remaining for
+ too long on the same server (e.g. after a farm size change). This
+ is stronger than the maxidle method in that it forces a
+ redispatch after some absolute delay.
+
+ dynamic Activate dynamic cookies. When used, a session cookie is
+ dynamically created for each server, based on the IP and port
+ of the server, and a secret key, specified in the
+ "dynamic-cookie-key" backend directive.
+ The cookie will be regenerated each time the IP address change,
+ and is only generated for IPv4/IPv6.
+
+ attr This option tells HAProxy to add an extra attribute when a
+ cookie is inserted. The attribute value can contain any
+ characters except control ones or ";". This option may be
+ repeated.
+
+ There can be only one persistence cookie per HTTP backend, and it can be
+ declared in a defaults section. The value of the cookie will be the value
+ indicated after the "cookie" keyword in a "server" statement. If no cookie
+ is declared for a given server, the cookie is not set.
+
+ Examples :
+ cookie JSESSIONID prefix
+ cookie SRV insert indirect nocache
+ cookie SRV insert postonly indirect
+ cookie SRV insert indirect nocache maxidle 30m maxlife 8h
+
+ See also : "balance source", "capture cookie", "server" and "ignore-persist".
+
+
+declare capture [ request | response ] len <length>
+ Declares a capture slot.
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ no | yes | yes | no
+
+ Arguments:
+ <length> is the length allowed for the capture.
+
+ This declaration is only available in the frontend or listen section, but the
+ reserved slot can be used in the backends. The "request" keyword allocates a
+ capture slot for use in the request, and "response" allocates a capture slot
+ for use in the response.
+
+ See also: "capture-req", "capture-res" (sample converters),
+ "capture.req.hdr", "capture.res.hdr" (sample fetches),
+ "http-request capture" and "http-response capture".
+
+
+default-server [param*]
+ Change default options for a server in a backend
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments:
+ <param*> is a list of parameters for this server. The "default-server"
+ keyword accepts an important number of options and has a complete
+ section dedicated to it. Please refer to section 5 for more
+ details.
+
+ Example :
+ default-server inter 1000 weight 13
+
+ See also: "server" and section 5 about server options
+
+
+default_backend <backend>
+ Specify the backend to use when no "use_backend" rule has been matched.
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments :
+ <backend> is the name of the backend to use.
+
+ When doing content-switching between frontend and backends using the
+ "use_backend" keyword, it is often useful to indicate which backend will be
+ used when no rule has matched. It generally is the dynamic backend which
+ will catch all undetermined requests.
+
+ Example :
+
+ use_backend dynamic if url_dyn
+ use_backend static if url_css url_img extension_img
+ default_backend dynamic
+
+ See also : "use_backend"
+
+
+description <string>
+ Describe a listen, frontend or backend.
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections : defaults | frontend | listen | backend
+ no | yes | yes | yes
+
+ Arguments : string
+
+ Allows to add a sentence to describe the related object in the HAProxy HTML
+ stats page. The description will be printed on the right of the object name
+ it describes.
+ No need to backslash spaces in the <string> arguments.
+
+
+disabled
+ Disable a proxy, frontend or backend.
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments : none
+
+ The "disabled" keyword is used to disable an instance, mainly in order to
+ liberate a listening port or to temporarily disable a service. The instance
+ will still be created and its configuration will be checked, but it will be
+ created in the "stopped" state and will appear as such in the statistics. It
+ will not receive any traffic nor will it send any health-checks or logs. It
+ is possible to disable many instances at once by adding the "disabled"
+ keyword in a "defaults" section.
+
+ See also : "enabled"
+
+
+dispatch <address>:<port>
+ Set a default server address
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ no | no | yes | yes
+
+ Arguments :
+
+ <address> is the IPv4 address of the default server. Alternatively, a
+ resolvable hostname is supported, but this name will be resolved
+ during start-up.
+
+ <ports> is a mandatory port specification. All connections will be sent
+ to this port, and it is not permitted to use port offsets as is
+ possible with normal servers.
+
+ The "dispatch" keyword designates a default server for use when no other
+ server can take the connection. In the past it was used to forward non
+ persistent connections to an auxiliary load balancer. Due to its simple
+ syntax, it has also been used for simple TCP relays. It is recommended not to
+ use it for more clarity, and to use the "server" directive instead.
+
+ See also : "server"
+
+
+dynamic-cookie-key <string>
+ Set the dynamic cookie secret key for a backend.
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments : The secret key to be used.
+
+ When dynamic cookies are enabled (see the "dynamic" directive for cookie),
+ a dynamic cookie is created for each server (unless one is explicitly
+ specified on the "server" line), using a hash of the IP address of the
+ server, the TCP port, and the secret key.
+ That way, we can ensure session persistence across multiple load-balancers,
+ even if servers are dynamically added or removed.
+
+enabled
+ Enable a proxy, frontend or backend.
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments : none
+
+ The "enabled" keyword is used to explicitly enable an instance, when the
+ defaults has been set to "disabled". This is very rarely used.
+
+ See also : "disabled"
+
+
+errorfile <code> <file>
+ Return a file contents instead of errors generated by HAProxy
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments :
+ <code> is the HTTP status code. Currently, HAProxy is capable of
+ generating codes 200, 400, 401, 403, 404, 405, 407, 408, 410,
+ 413, 425, 429, 500, 501, 502, 503, and 504.
+
+ <file> designates a file containing the full HTTP response. It is
+ recommended to follow the common practice of appending ".http" to
+ the filename so that people do not confuse the response with HTML
+ error pages, and to use absolute paths, since files are read
+ before any chroot is performed.
+
+ It is important to understand that this keyword is not meant to rewrite
+ errors returned by the server, but errors detected and returned by HAProxy.
+ This is why the list of supported errors is limited to a small set.
+
+ Code 200 is emitted in response to requests matching a "monitor-uri" rule.
+
+ The files are parsed when HAProxy starts and must be valid according to the
+ HTTP specification. They should not exceed the configured buffer size
+ (BUFSIZE), which generally is 16 kB, otherwise an internal error will be
+ returned. It is also wise not to put any reference to local contents
+ (e.g. images) in order to avoid loops between the client and HAProxy when all
+ servers are down, causing an error to be returned instead of an
+ image. Finally, The response cannot exceed (tune.bufsize - tune.maxrewrite)
+ so that "http-after-response" rules still have room to operate (see
+ "tune.maxrewrite").
+
+ The files are read at the same time as the configuration and kept in memory.
+ For this reason, the errors continue to be returned even when the process is
+ chrooted, and no file change is considered while the process is running. A
+ simple method for developing those files consists in associating them to the
+ 403 status code and interrogating a blocked URL.
+
+ See also : "http-error", "errorloc", "errorloc302", "errorloc303"
+
+ Example :
+ errorfile 400 /etc/haproxy/errorfiles/400badreq.http
+ errorfile 408 /dev/null # work around Chrome pre-connect bug
+ errorfile 403 /etc/haproxy/errorfiles/403forbid.http
+ errorfile 503 /etc/haproxy/errorfiles/503sorry.http
+
+
+errorfiles <name> [<code> ...]
+ Import, fully or partially, the error files defined in the <name> http-errors
+ section.
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments :
+ <name> is the name of an existing http-errors section.
+
+ <code> is a HTTP status code. Several status code may be listed.
+ Currently, HAProxy is capable of generating codes 200, 400, 401,
+ 403, 404, 405, 407, 408, 410, 413, 425, 429, 500, 501, 502, 503,
+ and 504.
+
+ Errors defined in the http-errors section with the name <name> are imported
+ in the current proxy. If no status code is specified, all error files of the
+ http-errors section are imported. Otherwise, only error files associated to
+ the listed status code are imported. Those error files override the already
+ defined custom errors for the proxy. And they may be overridden by following
+ ones. Functionally, it is exactly the same as declaring all error files by
+ hand using "errorfile" directives.
+
+ See also : "http-error", "errorfile", "errorloc", "errorloc302" ,
+ "errorloc303" and section 3.8 about http-errors.
+
+ Example :
+ errorfiles generic
+ errorfiles site-1 403 404
+
+
+errorloc <code> <url>
+errorloc302 <code> <url>
+ Return an HTTP redirection to a URL instead of errors generated by HAProxy
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments :
+ <code> is the HTTP status code. Currently, HAProxy is capable of
+ generating codes 200, 400, 401, 403, 404, 405, 407, 408, 410,
+ 413, 425, 429, 500, 501, 502, 503, and 504.
+
+ <url> it is the exact contents of the "Location" header. It may contain
+ either a relative URI to an error page hosted on the same site,
+ or an absolute URI designating an error page on another site.
+ Special care should be given to relative URIs to avoid redirect
+ loops if the URI itself may generate the same error (e.g. 500).
+
+ It is important to understand that this keyword is not meant to rewrite
+ errors returned by the server, but errors detected and returned by HAProxy.
+ This is why the list of supported errors is limited to a small set.
+
+ Code 200 is emitted in response to requests matching a "monitor-uri" rule.
+
+ Note that both keyword return the HTTP 302 status code, which tells the
+ client to fetch the designated URL using the same HTTP method. This can be
+ quite problematic in case of non-GET methods such as POST, because the URL
+ sent to the client might not be allowed for something other than GET. To
+ work around this problem, please use "errorloc303" which send the HTTP 303
+ status code, indicating to the client that the URL must be fetched with a GET
+ request.
+
+ See also : "http-error", "errorfile", "errorloc303"
+
+
+errorloc303 <code> <url>
+ Return an HTTP redirection to a URL instead of errors generated by HAProxy
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments :
+ <code> is the HTTP status code. Currently, HAProxy is capable of
+ generating codes 200, 400, 401, 403, 404, 405, 407, 408, 410,
+ 413, 425, 429, 500, 501, 502, 503, and 504.
+
+ <url> it is the exact contents of the "Location" header. It may contain
+ either a relative URI to an error page hosted on the same site,
+ or an absolute URI designating an error page on another site.
+ Special care should be given to relative URIs to avoid redirect
+ loops if the URI itself may generate the same error (e.g. 500).
+
+ It is important to understand that this keyword is not meant to rewrite
+ errors returned by the server, but errors detected and returned by HAProxy.
+ This is why the list of supported errors is limited to a small set.
+
+ Code 200 is emitted in response to requests matching a "monitor-uri" rule.
+
+ Note that both keyword return the HTTP 303 status code, which tells the
+ client to fetch the designated URL using the same HTTP GET method. This
+ solves the usual problems associated with "errorloc" and the 302 code. It is
+ possible that some very old browsers designed before HTTP/1.1 do not support
+ it, but no such problem has been reported till now.
+
+ See also : "http-error", "errorfile", "errorloc", "errorloc302"
+
+
+email-alert from <emailaddr>
+ Declare the from email address to be used in both the envelope and header
+ of email alerts. This is the address that email alerts are sent from.
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments :
+
+ <emailaddr> is the from email address to use when sending email alerts
+
+ Also requires "email-alert mailers" and "email-alert to" to be set
+ and if so sending email alerts is enabled for the proxy.
+
+ See also : "email-alert level", "email-alert mailers",
+ "email-alert myhostname", "email-alert to", section 3.6 about
+ mailers.
+
+
+email-alert level <level>
+ Declare the maximum log level of messages for which email alerts will be
+ sent. This acts as a filter on the sending of email alerts.
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments :
+
+ <level> One of the 8 syslog levels:
+ emerg alert crit err warning notice info debug
+ The above syslog levels are ordered from lowest to highest.
+
+ By default level is alert
+
+ Also requires "email-alert from", "email-alert mailers" and
+ "email-alert to" to be set and if so sending email alerts is enabled
+ for the proxy.
+
+ Alerts are sent when :
+
+ * An un-paused server is marked as down and <level> is alert or lower
+ * A paused server is marked as down and <level> is notice or lower
+ * A server is marked as up or enters the drain state and <level>
+ is notice or lower
+ * "option log-health-checks" is enabled, <level> is info or lower,
+ and a health check status update occurs
+
+ See also : "email-alert from", "email-alert mailers",
+ "email-alert myhostname", "email-alert to",
+ section 3.6 about mailers.
+
+
+email-alert mailers <mailersect>
+ Declare the mailers to be used when sending email alerts
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments :
+
+ <mailersect> is the name of the mailers section to send email alerts.
+
+ Also requires "email-alert from" and "email-alert to" to be set
+ and if so sending email alerts is enabled for the proxy.
+
+ See also : "email-alert from", "email-alert level", "email-alert myhostname",
+ "email-alert to", section 3.6 about mailers.
+
+
+email-alert myhostname <hostname>
+ Declare the to hostname address to be used when communicating with
+ mailers.
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments :
+
+ <hostname> is the hostname to use when communicating with mailers
+
+ By default the systems hostname is used.
+
+ Also requires "email-alert from", "email-alert mailers" and
+ "email-alert to" to be set and if so sending email alerts is enabled
+ for the proxy.
+
+ See also : "email-alert from", "email-alert level", "email-alert mailers",
+ "email-alert to", section 3.6 about mailers.
+
+
+email-alert to <emailaddr>
+ Declare both the recipient address in the envelope and to address in the
+ header of email alerts. This is the address that email alerts are sent to.
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments :
+
+ <emailaddr> is the to email address to use when sending email alerts
+
+ Also requires "email-alert mailers" and "email-alert to" to be set
+ and if so sending email alerts is enabled for the proxy.
+
+ See also : "email-alert from", "email-alert level", "email-alert mailers",
+ "email-alert myhostname", section 3.6 about mailers.
+
+
+error-log-format <string>
+ Specifies the log format string to use in case of connection error on the frontend side.
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ This directive specifies the log format string that will be used for logs
+ containing information related to errors, timeouts, retries redispatches or
+ HTTP status code 5xx. This format will in short be used for every log line
+ that would be concerned by the "log-separate-errors" option, including
+ connection errors described in section 8.2.5.
+
+ If the directive is used in a defaults section, all subsequent frontends will
+ use the same log format. Please see section 8.2.4 which covers the log format
+ string in depth.
+
+ "error-log-format" directive overrides previous "error-log-format"
+ directives.
+
+
+force-persist { if | unless } <condition>
+ Declare a condition to force persistence on down servers
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections: defaults | frontend | listen | backend
+ no | no | yes | yes
+
+ By default, requests are not dispatched to down servers. It is possible to
+ force this using "option persist", but it is unconditional and redispatches
+ to a valid server if "option redispatch" is set. That leaves with very little
+ possibilities to force some requests to reach a server which is artificially
+ marked down for maintenance operations.
+
+ The "force-persist" statement allows one to declare various ACL-based
+ conditions which, when met, will cause a request to ignore the down status of
+ a server and still try to connect to it. That makes it possible to start a
+ server, still replying an error to the health checks, and run a specially
+ configured browser to test the service. Among the handy methods, one could
+ use a specific source IP address, or a specific cookie. The cookie also has
+ the advantage that it can easily be added/removed on the browser from a test
+ page. Once the service is validated, it is then possible to open the service
+ to the world by returning a valid response to health checks.
+
+ The forced persistence is enabled when an "if" condition is met, or unless an
+ "unless" condition is met. The final redispatch is always disabled when this
+ is used.
+
+ See also : "option redispatch", "ignore-persist", "persist",
+ and section 7 about ACL usage.
+
+
+filter <name> [param*]
+ Add the filter <name> in the filter list attached to the proxy.
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ no | yes | yes | yes
+
+ Arguments :
+ <name> is the name of the filter. Officially supported filters are
+ referenced in section 9.
+
+ <param*> is a list of parameters accepted by the filter <name>. The
+ parsing of these parameters are the responsibility of the
+ filter. Please refer to the documentation of the corresponding
+ filter (section 9) for all details on the supported parameters.
+
+ Multiple occurrences of the filter line can be used for the same proxy. The
+ same filter can be referenced many times if needed.
+
+ Example:
+ listen
+ bind *:80
+
+ filter trace name BEFORE-HTTP-COMP
+ filter compression
+ filter trace name AFTER-HTTP-COMP
+
+ compression algo gzip
+ compression offload
+
+ server srv1 192.168.0.1:80
+
+ See also : section 9.
+
+
+fullconn <conns>
+ Specify at what backend load the servers will reach their maxconn
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <conns> is the number of connections on the backend which will make the
+ servers use the maximal number of connections.
+
+ When a server has a "maxconn" parameter specified, it means that its number
+ of concurrent connections will never go higher. Additionally, if it has a
+ "minconn" parameter, it indicates a dynamic limit following the backend's
+ load. The server will then always accept at least <minconn> connections,
+ never more than <maxconn>, and the limit will be on the ramp between both
+ values when the backend has less than <conns> concurrent connections. This
+ makes it possible to limit the load on the servers during normal loads, but
+ push it further for important loads without overloading the servers during
+ exceptional loads.
+
+ Since it's hard to get this value right, HAProxy automatically sets it to
+ 10% of the sum of the maxconns of all frontends that may branch to this
+ backend (based on "use_backend" and "default_backend" rules). That way it's
+ safe to leave it unset. However, "use_backend" involving dynamic names are
+ not counted since there is no way to know if they could match or not.
+
+ Example :
+ # The servers will accept between 100 and 1000 concurrent connections each
+ # and the maximum of 1000 will be reached when the backend reaches 10000
+ # connections.
+ backend dynamic
+ fullconn 10000
+ server srv1 dyn1:80 minconn 100 maxconn 1000
+ server srv2 dyn2:80 minconn 100 maxconn 1000
+
+ See also : "maxconn", "server"
+
+
+hash-balance-factor <factor>
+ Specify the balancing factor for bounded-load consistent hashing
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | no | yes
+
+ Arguments :
+ <factor> is the control for the maximum number of concurrent requests to
+ send to a server, expressed as a percentage of the average number
+ of concurrent requests across all of the active servers.
+
+ Specifying a "hash-balance-factor" for a server with "hash-type consistent"
+ enables an algorithm that prevents any one server from getting too many
+ requests at once, even if some hash buckets receive many more requests than
+ others. Setting <factor> to 0 (the default) disables the feature. Otherwise,
+ <factor> is a percentage greater than 100. For example, if <factor> is 150,
+ then no server will be allowed to have a load more than 1.5 times the average.
+ If server weights are used, they will be respected.
+
+ If the first-choice server is disqualified, the algorithm will choose another
+ server based on the request hash, until a server with additional capacity is
+ found. A higher <factor> allows more imbalance between the servers, while a
+ lower <factor> means that more servers will be checked on average, affecting
+ performance. Reasonable values are from 125 to 200.
+
+ This setting is also used by "balance random" which internally relies on the
+ consistent hashing mechanism.
+
+ See also : "balance" and "hash-type".
+
+
+hash-type <method> <function> <modifier>
+ Specify a method to use for mapping hashes to servers
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <method> is the method used to select a server from the hash computed by
+ the <function> :
+
+ map-based the hash table is a static array containing all alive servers.
+ The hashes will be very smooth, will consider weights, but
+ will be static in that weight changes while a server is up
+ will be ignored. This means that there will be no slow start.
+ Also, since a server is selected by its position in the array,
+ most mappings are changed when the server count changes. This
+ means that when a server goes up or down, or when a server is
+ added to a farm, most connections will be redistributed to
+ different servers. This can be inconvenient with caches for
+ instance.
+
+ consistent the hash table is a tree filled with many occurrences of each
+ server. The hash key is looked up in the tree and the closest
+ server is chosen. This hash is dynamic, it supports changing
+ weights while the servers are up, so it is compatible with the
+ slow start feature. It has the advantage that when a server
+ goes up or down, only its associations are moved. When a
+ server is added to the farm, only a few part of the mappings
+ are redistributed, making it an ideal method for caches.
+ However, due to its principle, the distribution will never be
+ very smooth and it may sometimes be necessary to adjust a
+ server's weight or its ID to get a more balanced distribution.
+ In order to get the same distribution on multiple load
+ balancers, it is important that all servers have the exact
+ same IDs. Note: consistent hash uses sdbm and avalanche if no
+ hash function is specified.
+
+ <function> is the hash function to be used :
+
+ sdbm this function was created initially for sdbm (a public-domain
+ reimplementation of ndbm) database library. It was found to do
+ well in scrambling bits, causing better distribution of the keys
+ and fewer splits. It also happens to be a good general hashing
+ function with good distribution, unless the total server weight
+ is a multiple of 64, in which case applying the avalanche
+ modifier may help.
+
+ djb2 this function was first proposed by Dan Bernstein many years ago
+ on comp.lang.c. Studies have shown that for certain workload this
+ function provides a better distribution than sdbm. It generally
+ works well with text-based inputs though it can perform extremely
+ poorly with numeric-only input or when the total server weight is
+ a multiple of 33, unless the avalanche modifier is also used.
+
+ wt6 this function was designed for HAProxy while testing other
+ functions in the past. It is not as smooth as the other ones, but
+ is much less sensible to the input data set or to the number of
+ servers. It can make sense as an alternative to sdbm+avalanche or
+ djb2+avalanche for consistent hashing or when hashing on numeric
+ data such as a source IP address or a visitor identifier in a URL
+ parameter.
+
+ crc32 this is the most common CRC32 implementation as used in Ethernet,
+ gzip, PNG, etc. It is slower than the other ones but may provide
+ a better distribution or less predictable results especially when
+ used on strings.
+
+ none don't hash the key, the key will be used as a hash, this can be
+ useful to manually hash the key using a converter for that purpose
+ and let haproxy use the result directly.
+
+ <modifier> indicates an optional method applied after hashing the key :
+
+ avalanche This directive indicates that the result from the hash
+ function above should not be used in its raw form but that
+ a 4-byte full avalanche hash must be applied first. The
+ purpose of this step is to mix the resulting bits from the
+ previous hash in order to avoid any undesired effect when
+ the input contains some limited values or when the number of
+ servers is a multiple of one of the hash's components (64
+ for SDBM, 33 for DJB2). Enabling avalanche tends to make the
+ result less predictable, but it's also not as smooth as when
+ using the original function. Some testing might be needed
+ with some workloads. This hash is one of the many proposed
+ by Bob Jenkins.
+
+ The default hash type is "map-based" and is recommended for most usages. The
+ default function is "sdbm", the selection of a function should be based on
+ the range of the values being hashed.
+
+ See also : "balance", "hash-balance-factor", "server"
+
+
+http-after-response <action> <options...> [ { if | unless } <condition> ]
+ Access control for all Layer 7 responses (server, applet/service and internal
+ ones).
+
+ May be used in the following contexts: http
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes(!) | yes | yes | yes
+
+ The http-after-response statement defines a set of rules which apply to layer
+ 7 processing. The rules are evaluated in their declaration order when they
+ are met in a frontend, listen or backend section. Since these rules apply on
+ responses, the backend rules are applied first, followed by the frontend's
+ rules. Any rule may optionally be followed by an ACL-based condition, in
+ which case it will only be evaluated if the condition evaluates true.
+
+ Unlike http-response rules, these ones are applied on all responses, the
+ server ones but also to all responses generated by HAProxy. These rules are
+ evaluated at the end of the responses analysis, before the data forwarding
+ phase.
+
+ The condition is evaluated just before the action is executed, and the action
+ is performed exactly once. As such, there is no problem if an action changes
+ an element which is checked as part of the condition. This also means that
+ multiple actions may rely on the same condition so that the first action that
+ changes the condition's evaluation is sufficient to implicitly disable the
+ remaining actions. This is used for example when trying to assign a value to
+ a variable from various sources when it's empty. There is no limit to the
+ number of "http-after-response" statements per instance.
+
+ The first keyword after "http-after-response" in the syntax is the rule's
+ action, optionally followed by a varying number of arguments for the action.
+ The supported actions and their respective syntaxes are enumerated in section
+ 4.3 "Actions" (look for actions which tick "HTTP Aft").
+
+ This directive is only available from named defaults sections, not anonymous
+ ones. Rules defined in the defaults section are evaluated before ones in the
+ associated proxy section. To avoid ambiguities, in this case the same
+ defaults section cannot be used by proxies with the frontend capability and
+ by proxies with the backend capability. It means a listen section cannot use
+ a defaults section defining such rules.
+
+ Note: Errors emitted in early stage of the request parsing are handled by the
+ multiplexer at a lower level, before any http analysis. Thus no
+ http-after-response ruleset is evaluated on these errors.
+
+ Example:
+ http-after-response set-header Strict-Transport-Security "max-age=31536000"
+ http-after-response set-header Cache-Control "no-store,no-cache,private"
+ http-after-response set-header Pragma "no-cache"
+
+
+http-check comment <string>
+ Defines a comment for the following the http-check rule, reported in logs if
+ it fails.
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <string> is the comment message to add in logs if the following http-check
+ rule fails.
+
+ It only works for connect, send and expect rules. It is useful to make
+ user-friendly error reporting.
+
+ See also : "option httpchk", "http-check connect", "http-check send" and
+ "http-check expect".
+
+
+http-check connect [default] [port <expr>] [addr <ip>] [send-proxy]
+ [via-socks4] [ssl] [sni <sni>] [alpn <alpn>] [linger]
+ [proto <name>] [comment <msg>]
+ Opens a new connection to perform an HTTP health check
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ comment <msg> defines a message to report if the rule evaluation fails.
+
+ default Use default options of the server line to do the health
+ checks. The server options are used only if not redefined.
+
+ port <expr> if not set, check port or server port is used.
+ It tells HAProxy where to open the connection to.
+ <port> must be a valid TCP port source integer, from 1 to
+ 65535 or an sample-fetch expression.
+
+ addr <ip> defines the IP address to do the health check.
+
+ send-proxy send a PROXY protocol string
+
+ via-socks4 enables outgoing health checks using upstream socks4 proxy.
+
+ ssl opens a ciphered connection
+
+ sni <sni> specifies the SNI to use to do health checks over SSL.
+
+ alpn <alpn> defines which protocols to advertise with ALPN. The protocol
+ list consists in a comma-delimited list of protocol names,
+ for instance: "h2,http/1.1". If it is not set, the server ALPN
+ is used.
+
+ proto <name> forces the multiplexer's protocol to use for this connection.
+ It must be an HTTP mux protocol and it must be usable on the
+ backend side. The list of available protocols is reported in
+ haproxy -vv.
+
+ linger cleanly close the connection instead of using a single RST.
+
+ Just like tcp-check health checks, it is possible to configure the connection
+ to use to perform HTTP health check. This directive should also be used to
+ describe a scenario involving several request/response exchanges, possibly on
+ different ports or with different servers.
+
+ When there are no TCP port configured on the server line neither server port
+ directive, then the first step of the http-check sequence must be to specify
+ the port with a "http-check connect".
+
+ In an http-check ruleset a 'connect' is required, it is also mandatory to start
+ the ruleset with a 'connect' rule. Purpose is to ensure admin know what they
+ do.
+
+ When a connect must start the ruleset, if may still be preceded by set-var,
+ unset-var or comment rules.
+
+ Examples :
+ # check HTTP and HTTPs services on a server.
+ # first open port 80 thanks to server line port directive, then
+ # tcp-check opens port 443, ciphered and run a request on it:
+ option httpchk
+
+ http-check connect
+ http-check send meth GET uri / ver HTTP/1.1 hdr host haproxy.1wt.eu
+ http-check expect status 200-399
+ http-check connect port 443 ssl sni haproxy.1wt.eu
+ http-check send meth GET uri / ver HTTP/1.1 hdr host haproxy.1wt.eu
+ http-check expect status 200-399
+
+ server www 10.0.0.1 check port 80
+
+ See also : "option httpchk", "http-check send", "http-check expect"
+
+
+http-check disable-on-404
+ Enable a maintenance mode upon HTTP/404 response to health-checks
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments : none
+
+ When this option is set, a server which returns an HTTP code 404 will be
+ excluded from further load-balancing, but will still receive persistent
+ connections. This provides a very convenient method for Web administrators
+ to perform a graceful shutdown of their servers. It is also important to note
+ that a server which is detected as failed while it was in this mode will not
+ generate an alert, just a notice. If the server responds 2xx or 3xx again, it
+ will immediately be reinserted into the farm. The status on the stats page
+ reports "NOLB" for a server in this mode. It is important to note that this
+ option only works in conjunction with the "httpchk" option. If this option
+ is used with "http-check expect", then it has precedence over it so that 404
+ responses will still be considered as soft-stop. Note also that a stopped
+ server will stay stopped even if it replies 404s. This option is only
+ evaluated for running servers.
+
+ See also : "option httpchk" and "http-check expect".
+
+
+http-check expect [min-recv <int>] [comment <msg>]
+ [ok-status <st>] [error-status <st>] [tout-status <st>]
+ [on-success <fmt>] [on-error <fmt>] [status-code <expr>]
+ [!] <match> <pattern>
+ Make HTTP health checks consider response contents or specific status codes
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ comment <msg> defines a message to report if the rule evaluation fails.
+
+ min-recv is optional and can define the minimum amount of data required to
+ evaluate the current expect rule. If the number of received bytes
+ is under this limit, the check will wait for more data. This
+ option can be used to resolve some ambiguous matching rules or to
+ avoid executing costly regex matches on content known to be still
+ incomplete. If an exact string is used, the minimum between the
+ string length and this parameter is used. This parameter is
+ ignored if it is set to -1. If the expect rule does not match,
+ the check will wait for more data. If set to 0, the evaluation
+ result is always conclusive.
+
+ ok-status <st> is optional and can be used to set the check status if
+ the expect rule is successfully evaluated and if it is
+ the last rule in the tcp-check ruleset. "L7OK", "L7OKC",
+ "L6OK" and "L4OK" are supported :
+ - L7OK : check passed on layer 7
+ - L7OKC : check conditionally passed on layer 7, set
+ server to NOLB state.
+ - L6OK : check passed on layer 6
+ - L4OK : check passed on layer 4
+ By default "L7OK" is used.
+
+ error-status <st> is optional and can be used to set the check status if
+ an error occurred during the expect rule evaluation.
+ "L7OKC", "L7RSP", "L7STS", "L6RSP" and "L4CON" are
+ supported :
+ - L7OKC : check conditionally passed on layer 7, set
+ server to NOLB state.
+ - L7RSP : layer 7 invalid response - protocol error
+ - L7STS : layer 7 response error, for example HTTP 5xx
+ - L6RSP : layer 6 invalid response - protocol error
+ - L4CON : layer 1-4 connection problem
+ By default "L7RSP" is used.
+
+ tout-status <st> is optional and can be used to set the check status if
+ a timeout occurred during the expect rule evaluation.
+ "L7TOUT", "L6TOUT", and "L4TOUT" are supported :
+ - L7TOUT : layer 7 (HTTP/SMTP) timeout
+ - L6TOUT : layer 6 (SSL) timeout
+ - L4TOUT : layer 1-4 timeout
+ By default "L7TOUT" is used.
+
+ on-success <fmt> is optional and can be used to customize the
+ informational message reported in logs if the expect
+ rule is successfully evaluated and if it is the last rule
+ in the tcp-check ruleset. <fmt> is a log-format string.
+
+ on-error <fmt> is optional and can be used to customize the
+ informational message reported in logs if an error
+ occurred during the expect rule evaluation. <fmt> is a
+ log-format string.
+
+ <match> is a keyword indicating how to look for a specific pattern in the
+ response. The keyword may be one of "status", "rstatus", "hdr",
+ "fhdr", "string", or "rstring". The keyword may be preceded by an
+ exclamation mark ("!") to negate the match. Spaces are allowed
+ between the exclamation mark and the keyword. See below for more
+ details on the supported keywords.
+
+ <pattern> is the pattern to look for. It may be a string, a regular
+ expression or a more complex pattern with several arguments. If
+ the string pattern contains spaces, they must be escaped with the
+ usual backslash ('\').
+
+ By default, "option httpchk" considers that response statuses 2xx and 3xx
+ are valid, and that others are invalid. When "http-check expect" is used,
+ it defines what is considered valid or invalid. Only one "http-check"
+ statement is supported in a backend. If a server fails to respond or times
+ out, the check obviously fails. The available matches are :
+
+ status <codes> : test the status codes found parsing <codes> string. it
+ must be a comma-separated list of status codes or range
+ codes. A health check response will be considered as
+ valid if the response's status code matches any status
+ code or is inside any range of the list. If the "status"
+ keyword is prefixed with "!", then the response will be
+ considered invalid if the status code matches.
+
+ rstatus <regex> : test a regular expression for the HTTP status code.
+ A health check response will be considered valid if the
+ response's status code matches the expression. If the
+ "rstatus" keyword is prefixed with "!", then the response
+ will be considered invalid if the status code matches.
+ This is mostly used to check for multiple codes.
+
+ hdr { name | name-lf } [ -m <meth> ] <name>
+ [ { value | value-lf } [ -m <meth> ] <value> :
+ test the specified header pattern on the HTTP response
+ headers. The name pattern is mandatory but the value
+ pattern is optional. If not specified, only the header
+ presence is verified. <meth> is the matching method,
+ applied on the header name or the header value. Supported
+ matching methods are "str" (exact match), "beg" (prefix
+ match), "end" (suffix match), "sub" (substring match) or
+ "reg" (regex match). If not specified, exact matching
+ method is used. If the "name-lf" parameter is used,
+ <name> is evaluated as a log-format string. If "value-lf"
+ parameter is used, <value> is evaluated as a log-format
+ string. These parameters cannot be used with the regex
+ matching method. Finally, the header value is considered
+ as comma-separated list. Note that matchings are case
+ insensitive on the header names.
+
+ fhdr { name | name-lf } [ -m <meth> ] <name>
+ [ { value | value-lf } [ -m <meth> ] <value> :
+ test the specified full header pattern on the HTTP
+ response headers. It does exactly the same than "hdr"
+ keyword, except the full header value is tested, commas
+ are not considered as delimiters.
+
+ string <string> : test the exact string match in the HTTP response body.
+ A health check response will be considered valid if the
+ response's body contains this exact string. If the
+ "string" keyword is prefixed with "!", then the response
+ will be considered invalid if the body contains this
+ string. This can be used to look for a mandatory word at
+ the end of a dynamic page, or to detect a failure when a
+ specific error appears on the check page (e.g. a stack
+ trace).
+
+ rstring <regex> : test a regular expression on the HTTP response body.
+ A health check response will be considered valid if the
+ response's body matches this expression. If the "rstring"
+ keyword is prefixed with "!", then the response will be
+ considered invalid if the body matches the expression.
+ This can be used to look for a mandatory word at the end
+ of a dynamic page, or to detect a failure when a specific
+ error appears on the check page (e.g. a stack trace).
+
+ string-lf <fmt> : test a log-format string match in the HTTP response body.
+ A health check response will be considered valid if the
+ response's body contains the string resulting of the
+ evaluation of <fmt>, which follows the log-format rules.
+ If prefixed with "!", then the response will be
+ considered invalid if the body contains the string.
+
+ It is important to note that the responses will be limited to a certain size
+ defined by the global "tune.bufsize" option, which defaults to 16384 bytes.
+ Thus, too large responses may not contain the mandatory pattern when using
+ "string" or "rstring". If a large response is absolutely required, it is
+ possible to change the default max size by setting the global variable.
+ However, it is worth keeping in mind that parsing very large responses can
+ waste some CPU cycles, especially when regular expressions are used, and that
+ it is always better to focus the checks on smaller resources.
+
+ In an http-check ruleset, the last expect rule may be implicit. If no expect
+ rule is specified after the last "http-check send", an implicit expect rule
+ is defined to match on 2xx or 3xx status codes. It means this rule is also
+ defined if there is no "http-check" rule at all, when only "option httpchk"
+ is set.
+
+ Last, if "http-check expect" is combined with "http-check disable-on-404",
+ then this last one has precedence when the server responds with 404.
+
+ Examples :
+ # only accept status 200 as valid
+ http-check expect status 200,201,300-310
+
+ # be sure a sessid coookie is set
+ http-check expect header name "set-cookie" value -m beg "sessid="
+
+ # consider SQL errors as errors
+ http-check expect ! string SQL\ Error
+
+ # consider status 5xx only as errors
+ http-check expect ! rstatus ^5
+
+ # check that we have a correct hexadecimal tag before /html
+ http-check expect rstring <!--tag:[0-9a-f]*--></html>
+
+ See also : "option httpchk", "http-check connect", "http-check disable-on-404"
+ and "http-check send".
+
+
+http-check send [meth <method>] [{ uri <uri> | uri-lf <fmt> }>] [ver <version>]
+ [hdr <name> <fmt>]* [{ body <string> | body-lf <fmt> }]
+ [comment <msg>]
+ Add a possible list of headers and/or a body to the request sent during HTTP
+ health checks.
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ comment <msg> defines a message to report if the rule evaluation fails.
+
+ meth <method> is the optional HTTP method used with the requests. When not
+ set, the "OPTIONS" method is used, as it generally requires
+ low server processing and is easy to filter out from the
+ logs. Any method may be used, though it is not recommended
+ to invent non-standard ones.
+
+ uri <uri> is optional and set the URI referenced in the HTTP requests
+ to the string <uri>. It defaults to "/" which is accessible
+ by default on almost any server, but may be changed to any
+ other URI. Query strings are permitted.
+
+ uri-lf <fmt> is optional and set the URI referenced in the HTTP requests
+ using the log-format string <fmt>. It defaults to "/" which
+ is accessible by default on almost any server, but may be
+ changed to any other URI. Query strings are permitted.
+
+ ver <version> is the optional HTTP version string. It defaults to
+ "HTTP/1.0" but some servers might behave incorrectly in HTTP
+ 1.0, so turning it to HTTP/1.1 may sometimes help. Note that
+ the Host field is mandatory in HTTP/1.1, use "hdr" argument
+ to add it.
+
+ hdr <name> <fmt> adds the HTTP header field whose name is specified in
+ <name> and whose value is defined by <fmt>, which follows
+ to the log-format rules.
+
+ body <string> add the body defined by <string> to the request sent during
+ HTTP health checks. If defined, the "Content-Length" header
+ is thus automatically added to the request.
+
+ body-lf <fmt> add the body defined by the log-format string <fmt> to the
+ request sent during HTTP health checks. If defined, the
+ "Content-Length" header is thus automatically added to the
+ request.
+
+ In addition to the request line defined by the "option httpchk" directive,
+ this one is the valid way to add some headers and optionally a body to the
+ request sent during HTTP health checks. If a body is defined, the associate
+ "Content-Length" header is automatically added. Thus, this header or
+ "Transfer-encoding" header should not be present in the request provided by
+ "http-check send". If so, it will be ignored. The old trick consisting to add
+ headers after the version string on the "option httpchk" line is now
+ deprecated.
+
+ Also "http-check send" doesn't support HTTP keep-alive. Keep in mind that it
+ will automatically append a "Connection: close" header, unless a Connection
+ header has already already been configured via a hdr entry.
+
+ Note that the Host header and the request authority, when both defined, are
+ automatically synchronized. It means when the HTTP request is sent, when a
+ Host is inserted in the request, the request authority is accordingly
+ updated. Thus, don't be surprised if the Host header value overwrites the
+ configured request authority.
+
+ Note also for now, no Host header is automatically added in HTTP/1.1 or above
+ requests. You should add it explicitly.
+
+ See also : "option httpchk", "http-check send-state" and "http-check expect".
+
+
+http-check send-state
+ Enable emission of a state header with HTTP health checks
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments : none
+
+ When this option is set, HAProxy will systematically send a special header
+ "X-Haproxy-Server-State" with a list of parameters indicating to each server
+ how they are seen by HAProxy. This can be used for instance when a server is
+ manipulated without access to HAProxy and the operator needs to know whether
+ HAProxy still sees it up or not, or if the server is the last one in a farm.
+
+ The header is composed of fields delimited by semi-colons, the first of which
+ is a word ("UP", "DOWN", "NOLB"), possibly followed by a number of valid
+ checks on the total number before transition, just as appears in the stats
+ interface. Next headers are in the form "<variable>=<value>", indicating in
+ no specific order some values available in the stats interface :
+ - a variable "address", containing the address of the backend server.
+ This corresponds to the <address> field in the server declaration. For
+ unix domain sockets, it will read "unix".
+
+ - a variable "port", containing the port of the backend server. This
+ corresponds to the <port> field in the server declaration. For unix
+ domain sockets, it will read "unix".
+
+ - a variable "name", containing the name of the backend followed by a slash
+ ("/") then the name of the server. This can be used when a server is
+ checked in multiple backends.
+
+ - a variable "node" containing the name of the HAProxy node, as set in the
+ global "node" variable, otherwise the system's hostname if unspecified.
+
+ - a variable "weight" indicating the weight of the server, a slash ("/")
+ and the total weight of the farm (just counting usable servers). This
+ helps to know if other servers are available to handle the load when this
+ one fails.
+
+ - a variable "scur" indicating the current number of concurrent connections
+ on the server, followed by a slash ("/") then the total number of
+ connections on all servers of the same backend.
+
+ - a variable "qcur" indicating the current number of requests in the
+ server's queue.
+
+ Example of a header received by the application server :
+ >>> X-Haproxy-Server-State: UP 2/3; name=bck/srv2; node=lb1; weight=1/2; \
+ scur=13/22; qcur=0
+
+ See also : "option httpchk", "http-check disable-on-404" and
+ "http-check send".
+
+
+http-check set-var(<var-name>[,<cond>...]) <expr>
+http-check set-var-fmt(<var-name>[,<cond>...]) <fmt>
+ This operation sets the content of a variable. The variable is declared inline.
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <var-name> The name of the variable starts with an indication about its
+ scope. The scopes allowed for http-check are:
+ "proc" : the variable is shared with the whole process.
+ "sess" : the variable is shared with the tcp-check session.
+ "check": the variable is declared for the lifetime of the tcp-check.
+ This prefix is followed by a name. The separator is a '.'.
+ The name may only contain characters 'a-z', 'A-Z', '0-9', '.',
+ and '-'.
+
+ <cond> A set of conditions that must all be true for the variable to
+ actually be set (such as "ifnotempty", "ifgt" ...). See the
+ set-var converter's description for a full list of possible
+ conditions.
+
+ <expr> Is a sample-fetch expression potentially followed by converters.
+
+ <fmt> This is the value expressed using log-format rules (see Custom
+ Log Format in section 8.2.4).
+
+ Examples :
+ http-check set-var(check.port) int(1234)
+ http-check set-var-fmt(check.port) "name=%H"
+
+
+http-check unset-var(<var-name>)
+ Free a reference to a variable within its scope.
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <var-name> The name of the variable starts with an indication about its
+ scope. The scopes allowed for http-check are:
+ "proc" : the variable is shared with the whole process.
+ "sess" : the variable is shared with the tcp-check session.
+ "check": the variable is declared for the lifetime of the tcp-check.
+ This prefix is followed by a name. The separator is a '.'.
+ The name may only contain characters 'a-z', 'A-Z', '0-9', '.',
+ and '-'.
+
+ Examples :
+ http-check unset-var(check.port)
+
+
+http-error status <code> [content-type <type>]
+ [ { default-errorfiles | errorfile <file> | errorfiles <name> |
+ file <file> | lf-file <file> | string <str> | lf-string <fmt> } ]
+ [ hdr <name> <fmt> ]*
+ Defines a custom error message to use instead of errors generated by HAProxy.
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments :
+ status <code> is the HTTP status code. It must be specified.
+ Currently, HAProxy is capable of generating codes
+ 200, 400, 401, 403, 404, 405, 407, 408, 410, 413, 425,
+ 429, 500, 501, 502, 503, and 504.
+
+ content-type <type> is the response content type, for instance
+ "text/plain". This parameter is ignored and should be
+ omitted when an errorfile is configured or when the
+ payload is empty. Otherwise, it must be defined.
+
+ default-errorfiles Reset the previously defined error message for current
+ proxy for the status <code>. If used on a backend, the
+ frontend error message is used, if defined. If used on
+ a frontend, the default error message is used.
+
+ errorfile <file> designates a file containing the full HTTP response.
+ It is recommended to follow the common practice of
+ appending ".http" to the filename so that people do
+ not confuse the response with HTML error pages, and to
+ use absolute paths, since files are read before any
+ chroot is performed.
+
+ errorfiles <name> designates the http-errors section to use to import
+ the error message with the status code <code>. If no
+ such message is found, the proxy's error messages are
+ considered.
+
+ file <file> specifies the file to use as response payload. If the
+ file is not empty, its content-type must be set as
+ argument to "content-type", otherwise, any
+ "content-type" argument is ignored. <file> is
+ considered as a raw string.
+
+ string <str> specifies the raw string to use as response payload.
+ The content-type must always be set as argument to
+ "content-type".
+
+ lf-file <file> specifies the file to use as response payload. If the
+ file is not empty, its content-type must be set as
+ argument to "content-type", otherwise, any
+ "content-type" argument is ignored. <file> is
+ evaluated as a log-format string.
+
+ lf-string <str> specifies the log-format string to use as response
+ payload. The content-type must always be set as
+ argument to "content-type".
+
+ hdr <name> <fmt> adds to the response the HTTP header field whose name
+ is specified in <name> and whose value is defined by
+ <fmt>, which follows to the log-format rules.
+ This parameter is ignored if an errorfile is used.
+
+ This directive may be used instead of "errorfile", to define a custom error
+ message. As "errorfile" directive, it is used for errors detected and
+ returned by HAProxy. If an errorfile is defined, it is parsed when HAProxy
+ starts and must be valid according to the HTTP standards. The generated
+ response must not exceed the configured buffer size (BUFFSIZE), otherwise an
+ internal error will be returned. Finally, if you consider to use some
+ http-after-response rules to rewrite these errors, the reserved buffer space
+ should be available (see "tune.maxrewrite").
+
+ The files are read at the same time as the configuration and kept in memory.
+ For this reason, the errors continue to be returned even when the process is
+ chrooted, and no file change is considered while the process is running.
+
+ Note: 400/408/500 errors emitted in early stage of the request parsing are
+ handled by the multiplexer at a lower level. No custom formatting is
+ supported at this level. Thus only static error messages, defined with
+ "errorfile" directive, are supported. However, this limitation only
+ exists during the request headers parsing or between two transactions.
+
+ See also : "errorfile", "errorfiles", "errorloc", "errorloc302",
+ "errorloc303" and section 3.8 about http-errors.
+
+
+http-request <action> [options...] [ { if | unless } <condition> ]
+ Access control for Layer 7 requests
+
+ May be used in the following contexts: http
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes(!) | yes | yes | yes
+
+ The http-request statement defines a set of rules which apply to layer 7
+ processing. The rules are evaluated in their declaration order when they are
+ met in a frontend, listen or backend section. Any rule may optionally be
+ followed by an ACL-based condition, in which case it will only be evaluated
+ if the condition evaluates to true.
+
+ The condition is evaluated just before the action is executed, and the action
+ is performed exactly once. As such, there is no problem if an action changes
+ an element which is checked as part of the condition. This also means that
+ multiple actions may rely on the same condition so that the first action that
+ changes the condition's evaluation is sufficient to implicitly disable the
+ remaining actions. This is used for example when trying to assign a value to
+ a variable from various sources when it's empty. There is no limit to the
+ number of "http-request" statements per instance.
+
+ The first keyword after "http-request" in the syntax is the rule's action,
+ optionally followed by a varying number of arguments for the action. The
+ supported actions and their respective syntaxes are enumerated in section 4.3
+ "Actions" (look for actions which tick "HTTP Req").
+
+ This directive is only available from named defaults sections, not anonymous
+ ones. Rules defined in the defaults section are evaluated before ones in the
+ associated proxy section. To avoid ambiguities, in this case the same
+ defaults section cannot be used by proxies with the frontend capability and
+ by proxies with the backend capability. It means a listen section cannot use
+ a defaults section defining such rules.
+
+ Example:
+ acl nagios src 192.168.129.3
+ acl local_net src 192.168.0.0/16
+ acl auth_ok http_auth(L1)
+
+ http-request allow if nagios
+ http-request allow if local_net auth_ok
+ http-request auth realm Gimme if local_net auth_ok
+ http-request deny
+
+ Example:
+ acl key req.hdr(X-Add-Acl-Key) -m found
+ acl add path /addacl
+ acl del path /delacl
+
+ acl myhost hdr(Host) -f myhost.lst
+
+ http-request add-acl(myhost.lst) %[req.hdr(X-Add-Acl-Key)] if key add
+ http-request del-acl(myhost.lst) %[req.hdr(X-Add-Acl-Key)] if key del
+
+ Example:
+ acl value req.hdr(X-Value) -m found
+ acl setmap path /setmap
+ acl delmap path /delmap
+
+ use_backend bk_appli if { hdr(Host),map_str(map.lst) -m found }
+
+ http-request set-map(map.lst) %[src] %[req.hdr(X-Value)] if setmap value
+ http-request del-map(map.lst) %[src] if delmap
+
+ See also : "stats http-request", section 3.4 about userlists and section 7
+ about ACL usage.
+
+http-response <action> <options...> [ { if | unless } <condition> ]
+ Access control for Layer 7 responses
+
+ May be used in the following contexts: http
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes(!) | yes | yes | yes
+
+ The http-response statement defines a set of rules which apply to layer 7
+ processing. The rules are evaluated in their declaration order when they are
+ met in a frontend, listen or backend section. Since these rules apply on
+ responses, the backend rules are applied first, followed by the frontend's
+ rules. Any rule may optionally be followed by an ACL-based condition, in
+ which case it will only be evaluated if the condition evaluates to true.
+
+ The condition is evaluated just before the action is executed, and the action
+ is performed exactly once. As such, there is no problem if an action changes
+ an element which is checked as part of the condition. This also means that
+ multiple actions may rely on the same condition so that the first action that
+ changes the condition's evaluation is sufficient to implicitly disable the
+ remaining actions. This is used for example when trying to assign a value to
+ a variable from various sources when it's empty. There is no limit to the
+ number of "http-response" statements per instance.
+
+ The first keyword after "http-response" in the syntax is the rule's action,
+ optionally followed by a varying number of arguments for the action. The
+ supported actions and their respective syntaxes are enumerated in section 4.3
+ "Actions" (look for actions which tick "HTTP Res").
+
+ This directive is only available from named defaults sections, not anonymous
+ ones. Rules defined in the defaults section are evaluated before ones in the
+ associated proxy section. To avoid ambiguities, in this case the same
+ defaults section cannot be used by proxies with the frontend capability and
+ by proxies with the backend capability. It means a listen section cannot use
+ a defaults section defining such rules.
+
+ Example:
+ acl key_acl res.hdr(X-Acl-Key) -m found
+
+ acl myhost hdr(Host) -f myhost.lst
+
+ http-response add-acl(myhost.lst) %[res.hdr(X-Acl-Key)] if key_acl
+ http-response del-acl(myhost.lst) %[res.hdr(X-Acl-Key)] if key_acl
+
+ Example:
+ acl value res.hdr(X-Value) -m found
+
+ use_backend bk_appli if { hdr(Host),map_str(map.lst) -m found }
+
+ http-response set-map(map.lst) %[src] %[res.hdr(X-Value)] if value
+ http-response del-map(map.lst) %[src] if ! value
+
+ See also : "http-request", section 3.4 about userlists and section 7 about
+ ACL usage.
+
+http-reuse { never | safe | aggressive | always }
+ Declare how idle HTTP connections may be shared between requests
+
+ May be used in the following contexts: http
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ By default, a connection established between HAProxy and the backend server
+ which is considered safe for reuse is moved back to the server's idle
+ connections pool so that any other request can make use of it. This is the
+ "safe" strategy below.
+
+ The argument indicates the desired connection reuse strategy :
+
+ - "never" : idle connections are never shared between sessions. This mode
+ may be enforced to cancel a different strategy inherited from
+ a defaults section or for troubleshooting. For example, if an
+ old bogus application considers that multiple requests over
+ the same connection come from the same client and it is not
+ possible to fix the application, it may be desirable to
+ disable connection sharing in a single backend. An example of
+ such an application could be an old HAProxy using cookie
+ insertion in tunnel mode and not checking any request past the
+ first one.
+
+ - "safe" : this is the default and the recommended strategy. The first
+ request of a session is always sent over its own connection,
+ and only subsequent requests may be dispatched over other
+ existing connections. This ensures that in case the server
+ closes the connection when the request is being sent, the
+ browser can decide to silently retry it. Since it is exactly
+ equivalent to regular keep-alive, there should be no side
+ effects. There is also a special handling for the connections
+ using protocols subject to Head-of-line blocking (backend with
+ h2 or fcgi). In this case, when at least one stream is
+ processed, the used connection is reserved to handle streams
+ of the same session. When no more streams are processed, the
+ connection is released and can be reused.
+
+ - "aggressive" : this mode may be useful in webservices environments where
+ all servers are not necessarily known and where it would be
+ appreciable to deliver most first requests over existing
+ connections. In this case, first requests are only delivered
+ over existing connections that have been reused at least once,
+ proving that the server correctly supports connection reuse.
+ It should only be used when it's sure that the client can
+ retry a failed request once in a while and where the benefit
+ of aggressive connection reuse significantly outweighs the
+ downsides of rare connection failures.
+
+ - "always" : this mode is only recommended when the path to the server is
+ known for never breaking existing connections quickly after
+ releasing them. It allows the first request of a session to be
+ sent to an existing connection. This can provide a significant
+ performance increase over the "safe" strategy when the backend
+ is a cache farm, since such components tend to show a
+ consistent behavior and will benefit from the connection
+ sharing. It is recommended that the "http-keep-alive" timeout
+ remains low in this mode so that no dead connections remain
+ usable. In most cases, this will lead to the same performance
+ gains as "aggressive" but with more risks. It should only be
+ used when it improves the situation over "aggressive".
+
+ When http connection sharing is enabled, a great care is taken to respect the
+ connection properties and compatibility. Indeed, some properties are specific
+ and it is not possibly to reuse it blindly. Those are the SSL SNI, source
+ and destination address and proxy protocol block. A connection is reused only
+ if it shares the same set of properties with the request.
+
+ Also note that connections with certain bogus authentication schemes (relying
+ on the connection) like NTLM are marked private and never shared.
+
+ A connection pool is involved and configurable with "pool-max-conn".
+
+ Note: connection reuse improves the accuracy of the "server maxconn" setting,
+ because almost no new connection will be established while idle connections
+ remain available. This is particularly true with the "always" strategy.
+
+ The rules to decide to keep an idle connection opened or to close it after
+ processing are also governed by the "tune.pool-low-fd-ratio" (default: 20%)
+ and "tune.pool-high-fd-ratio" (default: 25%). These correspond to the
+ percentage of total file descriptors spent in idle connections above which
+ haproxy will respectively refrain from keeping a connection opened after a
+ response, and actively kill idle connections. Some setups using a very high
+ ratio of idle connections, either because of too low a global "maxconn", or
+ due to a lot of HTTP/2 or HTTP/3 traffic on the frontend (few connections)
+ but HTTP/1 connections on the backend, may observe a lower reuse rate because
+ too few connections are kept open. It may be desirable in this case to adjust
+ such thresholds or simply to increase the global "maxconn" value.
+
+ Similarly, when thread groups are explicitly enabled, it is important to
+ understand that idle connections are only usable between threads from a same
+ group. As such it may happen that unfair load between groups leads to more
+ idle connections being needed, causing a lower reuse rate. The same solution
+ may then be applied (increase global "maxconn" or increase pool ratios).
+
+ See also : "option http-keep-alive", "server maxconn", "thread-groups",
+ "tune.pool-high-fd-ratio", "tune.pool-low-fd-ratio"
+
+
+http-send-name-header [<header>]
+ Add the server name to a request. Use the header string given by <header>
+
+ May be used in the following contexts: http
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <header> The header string to use to send the server name
+
+ The "http-send-name-header" statement causes the header field named <header>
+ to be set to the name of the target server at the moment the request is about
+ to be sent on the wire. Any existing occurrences of this header are removed.
+ Upon retries and redispatches, the header field is updated to always reflect
+ the server being attempted to connect to. Given that this header is modified
+ very late in the connection setup, it may have unexpected effects on already
+ modified headers. For example using it with transport-level header such as
+ connection, content-length, transfer-encoding and so on will likely result in
+ invalid requests being sent to the server. Additionally it has been reported
+ that this directive is currently being used as a way to overwrite the Host
+ header field in outgoing requests; while this trick has been known to work
+ as a side effect of the feature for some time, it is not officially supported
+ and might possibly not work anymore in a future version depending on the
+ technical difficulties this feature induces. A long-term solution instead
+ consists in fixing the application which required this trick so that it binds
+ to the correct host name.
+
+ See also : "server"
+
+id <value>
+ Set a persistent ID to a proxy.
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections : defaults | frontend | listen | backend
+ no | yes | yes | yes
+
+ Arguments : none
+
+ Set a persistent ID for the proxy. This ID must be unique and positive.
+ An unused ID will automatically be assigned if unset. The first assigned
+ value will be 1. This ID is currently only returned in statistics.
+
+
+ignore-persist { if | unless } <condition>
+ Declare a condition to ignore persistence
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections: defaults | frontend | listen | backend
+ no | no | yes | yes
+
+ By default, when cookie persistence is enabled, every requests containing
+ the cookie are unconditionally persistent (assuming the target server is up
+ and running).
+
+ The "ignore-persist" statement allows one to declare various ACL-based
+ conditions which, when met, will cause a request to ignore persistence.
+ This is sometimes useful to load balance requests for static files, which
+ often don't require persistence. This can also be used to fully disable
+ persistence for a specific User-Agent (for example, some web crawler bots).
+
+ The persistence is ignored when an "if" condition is met, or unless an
+ "unless" condition is met.
+
+ Example:
+ acl url_static path_beg /static /images /img /css
+ acl url_static path_end .gif .png .jpg .css .js
+ ignore-persist if url_static
+
+ See also : "force-persist", "cookie", and section 7 about ACL usage.
+
+load-server-state-from-file { global | local | none }
+ Allow seamless reload of HAProxy
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ This directive points HAProxy to a file where server state from previous
+ running process has been saved. That way, when starting up, before handling
+ traffic, the new process can apply old states to servers exactly has if no
+ reload occurred. The purpose of the "load-server-state-from-file" directive is
+ to tell HAProxy which file to use. For now, only 2 arguments to either prevent
+ loading state or load states from a file containing all backends and servers.
+ The state file can be generated by running the command "show servers state"
+ over the stats socket and redirect output.
+
+ The format of the file is versioned and is very specific. To understand it,
+ please read the documentation of the "show servers state" command (chapter
+ 9.3 of Management Guide).
+
+ Arguments:
+ global load the content of the file pointed by the global directive
+ named "server-state-file".
+
+ local load the content of the file pointed by the directive
+ "server-state-file-name" if set. If not set, then the backend
+ name is used as a file name.
+
+ none don't load any stat for this backend
+
+ Notes:
+ - server's IP address is preserved across reloads by default, but the
+ order can be changed thanks to the server's "init-addr" setting. This
+ means that an IP address change performed on the CLI at run time will
+ be preserved, and that any change to the local resolver (e.g. /etc/hosts)
+ will possibly not have any effect if the state file is in use.
+
+ - server's weight is applied from previous running process unless it has
+ has changed between previous and new configuration files.
+
+ Example: Minimal configuration
+
+ global
+ stats socket /tmp/socket
+ server-state-file /tmp/server_state
+
+ defaults
+ load-server-state-from-file global
+
+ backend bk
+ server s1 127.0.0.1:22 check weight 11
+ server s2 127.0.0.1:22 check weight 12
+
+
+ Then one can run :
+
+ socat /tmp/socket - <<< "show servers state" > /tmp/server_state
+
+ Content of the file /tmp/server_state would be like this:
+
+ 1
+ # <field names skipped for the doc example>
+ 1 bk 1 s1 127.0.0.1 2 0 11 11 4 6 3 4 6 0 0
+ 1 bk 2 s2 127.0.0.1 2 0 12 12 4 6 3 4 6 0 0
+
+ Example: Minimal configuration
+
+ global
+ stats socket /tmp/socket
+ server-state-base /etc/haproxy/states
+
+ defaults
+ load-server-state-from-file local
+
+ backend bk
+ server s1 127.0.0.1:22 check weight 11
+ server s2 127.0.0.1:22 check weight 12
+
+
+ Then one can run :
+
+ socat /tmp/socket - <<< "show servers state bk" > /etc/haproxy/states/bk
+
+ Content of the file /etc/haproxy/states/bk would be like this:
+
+ 1
+ # <field names skipped for the doc example>
+ 1 bk 1 s1 127.0.0.1 2 0 11 11 4 6 3 4 6 0 0
+ 1 bk 2 s2 127.0.0.1 2 0 12 12 4 6 3 4 6 0 0
+
+ See also: "server-state-file", "server-state-file-name", and
+ "show servers state"
+
+
+log global
+log <target> [len <length>] [format <format>] [sample <ranges>:<sample_size>]
+ <facility> [<level> [<minlevel>]]
+no log
+ Enable per-instance logging of events and traffic.
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Prefix :
+ no should be used when the logger list must be flushed. For example,
+ if you don't want to inherit from the default logger list. This
+ prefix does not allow arguments.
+
+ Arguments :
+ global should be used when the instance's logging parameters are the
+ same as the global ones. This is the most common usage. "global"
+ replaces <target>, <facility> and <level> with those of the log
+ entries found in the "global" section. Only one "log global"
+ statement may be used per instance, and this form takes no other
+ parameter.
+
+ <target> indicates where to send the logs. It takes the same format as
+ for the "global" section's logs, and can be one of :
+
+ - An IPv4 address optionally followed by a colon (':') and a UDP
+ port. If no port is specified, 514 is used by default (the
+ standard syslog port).
+
+ - An IPv6 address followed by a colon (':') and optionally a UDP
+ port. If no port is specified, 514 is used by default (the
+ standard syslog port).
+
+ - A filesystem path to a UNIX domain socket, keeping in mind
+ considerations for chroot (be sure the path is accessible
+ inside the chroot) and uid/gid (be sure the path is
+ appropriately writable).
+
+ - A file descriptor number in the form "fd@<number>", which may
+ point to a pipe, terminal, or socket. In this case unbuffered
+ logs are used and one writev() call per log is performed. This
+ is a bit expensive but acceptable for most workloads. Messages
+ sent this way will not be truncated but may be dropped, in
+ which case the DroppedLogs counter will be incremented. The
+ writev() call is atomic even on pipes for messages up to
+ PIPE_BUF size, which POSIX recommends to be at least 512 and
+ which is 4096 bytes on most modern operating systems. Any
+ larger message may be interleaved with messages from other
+ processes. Exceptionally for debugging purposes the file
+ descriptor may also be directed to a file, but doing so will
+ significantly slow HAProxy down as non-blocking calls will be
+ ignored. Also there will be no way to purge nor rotate this
+ file without restarting the process. Note that the configured
+ syslog format is preserved, so the output is suitable for use
+ with a TCP syslog server. See also the "short" and "raw"
+ formats below.
+
+ - "stdout" / "stderr", which are respectively aliases for "fd@1"
+ and "fd@2", see above.
+
+ - A ring buffer in the form "ring@<name>", which will correspond
+ to an in-memory ring buffer accessible over the CLI using the
+ "show events" command, which will also list existing rings and
+ their sizes. Such buffers are lost on reload or restart but
+ when used as a complement this can help troubleshooting by
+ having the logs instantly available.
+
+ - A log backend in the form "backend@<name>", which will send
+ log messages to the corresponding log backend responsible for
+ sending the message to the proper server according to the
+ backend's lb settings. A log backend is a backend section with
+ "mode log" set (see "mode" for more information).
+
+ - An explicit stream address prefix such as "tcp@","tcp6@",
+ "tcp4@" or "uxst@" will allocate an implicit ring buffer with
+ a stream forward server targeting the given address.
+
+ You may want to reference some environment variables in the
+ address parameter, see section 2.3 about environment variables.
+
+ <length> is an optional maximum line length. Log lines larger than this
+ value will be truncated before being sent. The reason is that
+ syslog servers act differently on log line length. All servers
+ support the default value of 1024, but some servers simply drop
+ larger lines while others do log them. If a server supports long
+ lines, it may make sense to set this value here in order to avoid
+ truncating long lines. Similarly, if a server drops long lines,
+ it is preferable to truncate them before sending them. Accepted
+ values are 80 to 65535 inclusive. The default value of 1024 is
+ generally fine for all standard usages. Some specific cases of
+ long captures or JSON-formatted logs may require larger values.
+ You may also need to increase "tune.http.logurilen" if your
+ request URIs are truncated.
+
+ <ranges> A list of comma-separated ranges to identify the logs to sample.
+ This is used to balance the load of the logs to send to the log
+ server. The limits of the ranges cannot be null. They are numbered
+ from 1. The size or period (in number of logs) of the sample must
+ be set with <sample_size> parameter.
+
+ <sample_size>
+ The size of the sample in number of logs to consider when balancing
+ their logging loads. It is used to balance the load of the logs to
+ send to the syslog server. This size must be greater or equal to the
+ maximum of the high limits of the ranges.
+ (see also <ranges> parameter).
+
+ <format> is the log format used when generating syslog messages. It may be
+ one of the following :
+
+ local Analog to rfc3164 syslog message format except that hostname
+ field is stripped. This is the default.
+ Note: option "log-send-hostname" switches the default to
+ rfc3164.
+
+ rfc3164 The RFC3164 syslog message format.
+ (https://tools.ietf.org/html/rfc3164)
+
+ rfc5424 The RFC5424 syslog message format.
+ (https://tools.ietf.org/html/rfc5424)
+
+ priority A message containing only a level plus syslog facility between
+ angle brackets such as '<63>', followed by the text. The PID,
+ date, time, process name and system name are omitted. This is
+ designed to be used with a local log server.
+
+ short A message containing only a level between angle brackets such as
+ '<3>', followed by the text. The PID, date, time, process name
+ and system name are omitted. This is designed to be used with a
+ local log server. This format is compatible with what the
+ systemd logger consumes.
+
+ timed A message containing only a level between angle brackets such as
+ '<3>', followed by ISO date and by the text. The PID, process
+ name and system name are omitted. This is designed to be
+ used with a local log server.
+
+ iso A message containing only the ISO date, followed by the text.
+ The PID, process name and system name are omitted. This is
+ designed to be used with a local log server.
+
+ raw A message containing only the text. The level, PID, date, time,
+ process name and system name are omitted. This is designed to
+ be used in containers or during development, where the severity
+ only depends on the file descriptor used (stdout/stderr).
+
+ <facility> must be one of the 24 standard syslog facilities :
+
+ kern user mail daemon auth syslog lpr news
+ uucp cron auth2 ftp ntp audit alert cron2
+ local0 local1 local2 local3 local4 local5 local6 local7
+
+ Note that the facility is ignored for the "short" and "raw"
+ formats, but still required as a positional field. It is
+ recommended to use "daemon" in this case to make it clear that
+ it's only supposed to be used locally.
+
+ <level> is optional and can be specified to filter outgoing messages. By
+ default, all messages are sent. If a level is specified, only
+ messages with a severity at least as important as this level
+ will be sent. An optional minimum level can be specified. If it
+ is set, logs emitted with a more severe level than this one will
+ be capped to this level. This is used to avoid sending "emerg"
+ messages on all terminals on some default syslog configurations.
+ Eight levels are known :
+
+ emerg alert crit err warning notice info debug
+
+ It is important to keep in mind that it is the frontend which decides what to
+ log from a connection, and that in case of content switching, the log entries
+ from the backend will be ignored. Connections are logged at level "info".
+
+ However, backend log declaration define how and where servers status changes
+ will be logged. Level "notice" will be used to indicate a server going up,
+ "warning" will be used for termination signals and definitive service
+ termination, and "alert" will be used for when a server goes down.
+
+ Note : According to RFC3164, messages are truncated to 1024 bytes before
+ being emitted.
+
+ Example :
+ log global
+ log stdout format short daemon # send log to systemd
+ log stdout format raw daemon # send everything to stdout
+ log stderr format raw daemon notice # send important events to stderr
+ log 127.0.0.1:514 local0 notice # only send important events
+ log tcp@127.0.0.1:514 local0 notice notice # same but limit output
+ # level and send in tcp
+ log "${LOCAL_SYSLOG}:514" local0 notice # send to local server
+
+log-format <string>
+ Specifies the log format string to use for traffic logs
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ This directive specifies the log format string that will be used for all logs
+ resulting from traffic passing through the frontend using this line. If the
+ directive is used in a defaults section, all subsequent frontends will use
+ the same log format. Please see section 8.2.4 which covers the log format
+ string in depth.
+ A specific log-format used only in case of connection error can also be
+ defined, see the "error-log-format" option.
+
+ "log-format" directive overrides previous "option tcplog", "log-format",
+ "option httplog" and "option httpslog" directives.
+
+log-format-sd <string>
+ Specifies the RFC5424 structured-data log format string
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ This directive specifies the RFC5424 structured-data log format string that
+ will be used for all logs resulting from traffic passing through the frontend
+ using this line. If the directive is used in a defaults section, all
+ subsequent frontends will use the same log format. Please see section 8.2.4
+ which covers the log format string in depth.
+
+ See https://tools.ietf.org/html/rfc5424#section-6.3 for more information
+ about the RFC5424 structured-data part.
+
+ Note : This log format string will be used only for loggers that have set
+ log format to "rfc5424".
+
+ Example :
+ log-format-sd [exampleSDID@1234\ bytes=\"%B\"\ status=\"%ST\"]
+
+
+log-tag <string>
+ Specifies the log tag to use for all outgoing logs
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Sets the tag field in the syslog header to this string. It defaults to the
+ log-tag set in the global section, otherwise the program name as launched
+ from the command line, which usually is "HAProxy". Sometimes it can be useful
+ to differentiate between multiple processes running on the same host, or to
+ differentiate customer instances running in the same process. In the backend,
+ logs about servers up/down will use this tag. As a hint, it can be convenient
+ to set a log-tag related to a hosted customer in a defaults section then put
+ all the frontends and backends for that customer, then start another customer
+ in a new defaults section. See also the global "log-tag" directive.
+
+max-keep-alive-queue <value>
+ Set the maximum server queue size for maintaining keep-alive connections
+
+ May be used in the following contexts: http
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ HTTP keep-alive tries to reuse the same server connection whenever possible,
+ but sometimes it can be counter-productive, for example if a server has a lot
+ of connections while other ones are idle. This is especially true for static
+ servers.
+
+ The purpose of this setting is to set a threshold on the number of queued
+ connections at which HAProxy stops trying to reuse the same server and prefers
+ to find another one. The default value, -1, means there is no limit. A value
+ of zero means that keep-alive requests will never be queued. For very close
+ servers which can be reached with a low latency and which are not sensible to
+ breaking keep-alive, a low value is recommended (e.g. local static server can
+ use a value of 10 or less). For remote servers suffering from a high latency,
+ higher values might be needed to cover for the latency and/or the cost of
+ picking a different server.
+
+ Note that this has no impact on responses which are maintained to the same
+ server consecutively to a 401 response. They will still go to the same server
+ even if they have to be queued.
+
+ See also : "option http-server-close", "option prefer-last-server", server
+ "maxconn" and cookie persistence.
+
+max-session-srv-conns <nb>
+ Set the maximum number of outgoing connections we can keep idling for a given
+ client session. The default is 5 (it precisely equals MAX_SRV_LIST which is
+ defined at build time).
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+maxconn <conns>
+ Fix the maximum number of concurrent connections on a frontend
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments :
+ <conns> is the maximum number of concurrent connections the frontend will
+ accept to serve. Excess connections will be queued by the system
+ in the socket's listen queue and will be served once a connection
+ closes.
+
+ If the system supports it, it can be useful on big sites to raise this limit
+ very high so that HAProxy manages connection queues, instead of leaving the
+ clients with unanswered connection attempts. This value should not exceed the
+ global maxconn. Also, keep in mind that a connection contains two buffers
+ of tune.bufsize (16kB by default) each, as well as some other data resulting
+ in about 33 kB of RAM being consumed per established connection. That means
+ that a medium system equipped with 1GB of RAM can withstand around
+ 20000-25000 concurrent connections if properly tuned.
+
+ Also, when <conns> is set to large values, it is possible that the servers
+ are not sized to accept such loads, and for this reason it is generally wise
+ to assign them some reasonable connection limits.
+
+ When this value is set to zero, which is the default, the global "maxconn"
+ value is used.
+
+ See also : "server", global section's "maxconn", "fullconn"
+
+
+mode { tcp|http|log }
+ Set the running mode or protocol of the instance
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+ Arguments :
+ tcp The instance will work in pure TCP mode. A full-duplex connection
+ will be established between clients and servers, and no layer 7
+ examination will be performed. This is the default mode. It
+ should be used for SSL, SSH, SMTP, ...
+
+ http The instance will work in HTTP mode. The client request will be
+ analyzed in depth before connecting to any server. Any request
+ which is not RFC-compliant will be rejected. Layer 7 filtering,
+ processing and switching will be possible. This is the mode which
+ brings HAProxy most of its value.
+
+ log When used in a backend section, it will turn the backend into a
+ log backend. Such backend can be used as a log destination for
+ any "log" directive by using the "backend@<name>" syntax. Log
+ messages will be distributed to the servers from the backend
+ according to the lb settings which can be configured using the
+ "balance" keyword. Log backends support UDP servers by prefixing
+ the server's address with the "udp@" prefix. Common backend and
+ server features are supported, but not TCP or HTTP specific ones.
+
+ When doing content switching, it is mandatory that the frontend and the
+ backend are in the same mode (generally HTTP), otherwise the configuration
+ will be refused.
+
+ Example :
+ defaults http_instances
+ mode http
+
+
+monitor fail { if | unless } <condition>
+ Add a condition to report a failure to a monitor HTTP request.
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ no | yes | yes | no
+
+ Arguments :
+ if <cond> the monitor request will fail if the condition is satisfied,
+ and will succeed otherwise. The condition should describe a
+ combined test which must induce a failure if all conditions
+ are met, for instance a low number of servers both in a
+ backend and its backup.
+
+ unless <cond> the monitor request will succeed only if the condition is
+ satisfied, and will fail otherwise. Such a condition may be
+ based on a test on the presence of a minimum number of active
+ servers in a list of backends.
+
+ This statement adds a condition which can force the response to a monitor
+ request to report a failure. By default, when an external component queries
+ the URI dedicated to monitoring, a 200 response is returned. When one of the
+ conditions above is met, HAProxy will return 503 instead of 200. This is
+ very useful to report a site failure to an external component which may base
+ routing advertisements between multiple sites on the availability reported by
+ HAProxy. In this case, one would rely on an ACL involving the "nbsrv"
+ criterion. Note that "monitor fail" only works in HTTP mode. Both status
+ messages may be tweaked using "errorfile" or "errorloc" if needed.
+
+ Example:
+ frontend www
+ mode http
+ acl site_dead nbsrv(dynamic) lt 2
+ acl site_dead nbsrv(static) lt 2
+ monitor-uri /site_alive
+ monitor fail if site_dead
+
+ See also : "monitor-uri", "errorfile", "errorloc"
+
+
+monitor-uri <uri>
+ Intercept a URI used by external components' monitor requests
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments :
+ <uri> is the exact URI which we want to intercept to return HAProxy's
+ health status instead of forwarding the request.
+
+ When an HTTP request referencing <uri> will be received on a frontend,
+ HAProxy will not forward it nor log it, but instead will return either
+ "HTTP/1.0 200 OK" or "HTTP/1.0 503 Service unavailable", depending on failure
+ conditions defined with "monitor fail". This is normally enough for any
+ front-end HTTP probe to detect that the service is UP and running without
+ forwarding the request to a backend server. Note that the HTTP method, the
+ version and all headers are ignored, but the request must at least be valid
+ at the HTTP level. This keyword may only be used with an HTTP-mode frontend.
+
+ Monitor requests are processed very early, just after the request is parsed
+ and even before any "http-request". The only rulesets applied before are the
+ tcp-request ones. They cannot be logged either, and it is the intended
+ purpose. Only one URI may be configured for monitoring; when multiple
+ "monitor-uri" statements are present, the last one will define the URI to
+ be used. They are only used to report HAProxy's health to an upper component,
+ nothing more. However, it is possible to add any number of conditions using
+ "monitor fail" and ACLs so that the result can be adjusted to whatever check
+ can be imagined (most often the number of available servers in a backend).
+
+ Note: if <uri> starts by a slash ('/'), the matching is performed against the
+ request's path instead of the request's uri. It is a workaround to let
+ the HTTP/2 requests match the monitor-uri. Indeed, in HTTP/2, clients
+ are encouraged to send absolute URIs only.
+
+ Example :
+ # Use /haproxy_test to report HAProxy's status
+ frontend www
+ mode http
+ monitor-uri /haproxy_test
+
+ See also : "monitor fail"
+
+
+option abortonclose
+no option abortonclose
+ Enable or disable early dropping of aborted requests pending in queues.
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments : none
+
+ In presence of very high loads, the servers will take some time to respond.
+ The per-instance connection queue will inflate, and the response time will
+ increase respective to the size of the queue times the average per-stream
+ response time. When clients will wait for more than a few seconds, they will
+ often hit the "STOP" button on their browser, leaving a useless request in
+ the queue, and slowing down other users, and the servers as well, because the
+ request will eventually be served, then aborted at the first error
+ encountered while delivering the response.
+
+ As there is no way to distinguish between a full STOP and a simple output
+ close on the client side, HTTP agents should be conservative and consider
+ that the client might only have closed its output channel while waiting for
+ the response. However, this introduces risks of congestion when lots of users
+ do the same, and is completely useless nowadays because probably no client at
+ all will close the stream while waiting for the response. Some HTTP agents
+ support this behavior (Squid, Apache, HAProxy), and others do not (TUX, most
+ hardware-based load balancers). So the probability for a closed input channel
+ to represent a user hitting the "STOP" button is close to 100%, and the risk
+ of being the single component to break rare but valid traffic is extremely
+ low, which adds to the temptation to be able to abort a stream early while
+ still not served and not pollute the servers.
+
+ In HAProxy, the user can choose the desired behavior using the option
+ "abortonclose". By default (without the option) the behavior is HTTP
+ compliant and aborted requests will be served. But when the option is
+ specified, a stream with an incoming channel closed will be aborted while
+ it is still possible, either pending in the queue for a connection slot, or
+ during the connection establishment if the server has not yet acknowledged
+ the connection request. This considerably reduces the queue size and the load
+ on saturated servers when users are tempted to click on STOP, which in turn
+ reduces the response time for other users.
+
+ If this option has been enabled in a "defaults" section, it can be disabled
+ in a specific instance by prepending the "no" keyword before it.
+
+ See also : "timeout queue" and server's "maxconn" and "maxqueue" parameters
+
+
+option accept-invalid-http-request
+no option accept-invalid-http-request
+ Enable or disable relaxing of HTTP request parsing
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments : none
+
+ By default, HAProxy complies with RFC7230 in terms of message parsing. This
+ means that invalid characters in header names are not permitted and cause an
+ error to be returned to the client. This is the desired behavior as such
+ forbidden characters are essentially used to build attacks exploiting server
+ weaknesses, and bypass security filtering. Sometimes, a buggy browser or
+ server will emit invalid header names for whatever reason (configuration,
+ implementation) and the issue will not be immediately fixed. In such a case,
+ it is possible to relax HAProxy's header name parser to accept any character
+ even if that does not make sense, by specifying this option. Similarly, the
+ list of characters allowed to appear in a URI is well defined by RFC3986, and
+ chars 0-31, 32 (space), 34 ('"'), 60 ('<'), 62 ('>'), 92 ('\'), 94 ('^'), 96
+ ('`'), 123 ('{'), 124 ('|'), 125 ('}'), 127 (delete) and anything above are
+ not allowed at all. HAProxy always blocks a number of them (0..32, 127). The
+ remaining ones are blocked by default unless this option is enabled. This
+ option also relaxes the test on the HTTP version, it allows HTTP/0.9 requests
+ to pass through (no version specified), as well as different protocol names
+ (e.g. RTSP), and multiple digits for both the major and the minor version.
+ Finally, this option also allows incoming URLs to contain fragment references
+ ('#' after the path).
+
+ This option should never be enabled by default as it hides application bugs
+ and open security breaches. It should only be deployed after a problem has
+ been confirmed.
+
+ When this option is enabled, erroneous header names will still be accepted in
+ requests, but the complete request will be captured in order to permit later
+ analysis using the "show errors" request on the UNIX stats socket. Similarly,
+ requests containing invalid chars in the URI part will be logged. Doing this
+ also helps confirming that the issue has been solved.
+
+ If this option has been enabled in a "defaults" section, it can be disabled
+ in a specific instance by prepending the "no" keyword before it.
+
+ See also : "option accept-invalid-http-response" and "show errors" on the
+ stats socket.
+
+
+option accept-invalid-http-response
+no option accept-invalid-http-response
+ Enable or disable relaxing of HTTP response parsing
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments : none
+
+ By default, HAProxy complies with RFC7230 in terms of message parsing. This
+ means that invalid characters in header names are not permitted and cause an
+ error to be returned to the client. This is the desired behavior as such
+ forbidden characters are essentially used to build attacks exploiting server
+ weaknesses, and bypass security filtering. Sometimes, a buggy browser or
+ server will emit invalid header names for whatever reason (configuration,
+ implementation) and the issue will not be immediately fixed. In such a case,
+ it is possible to relax HAProxy's header name parser to accept any character
+ even if that does not make sense, by specifying this option. This option also
+ relaxes the test on the HTTP version format, it allows multiple digits for
+ both the major and the minor version.
+
+ This option should never be enabled by default as it hides application bugs
+ and open security breaches. It should only be deployed after a problem has
+ been confirmed.
+
+ When this option is enabled, erroneous header names will still be accepted in
+ responses, but the complete response will be captured in order to permit
+ later analysis using the "show errors" request on the UNIX stats socket.
+ Doing this also helps confirming that the issue has been solved.
+
+ If this option has been enabled in a "defaults" section, it can be disabled
+ in a specific instance by prepending the "no" keyword before it.
+
+ See also : "option accept-invalid-http-request" and "show errors" on the
+ stats socket.
+
+
+option allbackups
+no option allbackups
+ Use either all backup servers at a time or only the first one
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments : none
+
+ By default, the first operational backup server gets all traffic when normal
+ servers are all down. Sometimes, it may be preferred to use multiple backups
+ at once, because one will not be enough. When "option allbackups" is enabled,
+ the load balancing will be performed among all backup servers when all normal
+ ones are unavailable. The same load balancing algorithm will be used and the
+ servers' weights will be respected. Thus, there will not be any priority
+ order between the backup servers anymore.
+
+ This option is mostly used with static server farms dedicated to return a
+ "sorry" page when an application is completely offline.
+
+ If this option has been enabled in a "defaults" section, it can be disabled
+ in a specific instance by prepending the "no" keyword before it.
+
+
+option checkcache
+no option checkcache
+ Analyze all server responses and block responses with cacheable cookies
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments : none
+
+ Some high-level frameworks set application cookies everywhere and do not
+ always let enough control to the developer to manage how the responses should
+ be cached. When a session cookie is returned on a cacheable object, there is a
+ high risk of session crossing or stealing between users traversing the same
+ caches. In some situations, it is better to block the response than to let
+ some sensitive session information go in the wild.
+
+ The option "checkcache" enables deep inspection of all server responses for
+ strict compliance with HTTP specification in terms of cacheability. It
+ carefully checks "Cache-control", "Pragma" and "Set-cookie" headers in server
+ response to check if there's a risk of caching a cookie on a client-side
+ proxy. When this option is enabled, the only responses which can be delivered
+ to the client are :
+ - all those without "Set-Cookie" header;
+ - all those with a return code other than 200, 203, 204, 206, 300, 301,
+ 404, 405, 410, 414, 501, provided that the server has not set a
+ "Cache-control: public" header field;
+ - all those that result from a request using a method other than GET, HEAD,
+ OPTIONS, TRACE, provided that the server has not set a 'Cache-Control:
+ public' header field;
+ - those with a 'Pragma: no-cache' header
+ - those with a 'Cache-control: private' header
+ - those with a 'Cache-control: no-store' header
+ - those with a 'Cache-control: max-age=0' header
+ - those with a 'Cache-control: s-maxage=0' header
+ - those with a 'Cache-control: no-cache' header
+ - those with a 'Cache-control: no-cache="set-cookie"' header
+ - those with a 'Cache-control: no-cache="set-cookie,' header
+ (allowing other fields after set-cookie)
+
+ If a response doesn't respect these requirements, then it will be blocked
+ just as if it was from an "http-response deny" rule, with an "HTTP 502 bad
+ gateway". The session state shows "PH--" meaning that the proxy blocked the
+ response during headers processing. Additionally, an alert will be sent in
+ the logs so that admins are informed that there's something to be fixed.
+
+ Due to the high impact on the application, the application should be tested
+ in depth with the option enabled before going to production. It is also a
+ good practice to always activate it during tests, even if it is not used in
+ production, as it will report potentially dangerous application behaviors.
+
+ If this option has been enabled in a "defaults" section, it can be disabled
+ in a specific instance by prepending the "no" keyword before it.
+
+
+option clitcpka
+no option clitcpka
+ Enable or disable the sending of TCP keepalive packets on the client side
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments : none
+
+ When there is a firewall or any session-aware component between a client and
+ a server, and when the protocol involves very long sessions with long idle
+ periods (e.g. remote desktops), there is a risk that one of the intermediate
+ components decides to expire a session which has remained idle for too long.
+
+ Enabling socket-level TCP keep-alives makes the system regularly send packets
+ to the other end of the connection, leaving it active. The delay between
+ keep-alive probes is controlled by the system only and depends both on the
+ operating system and its tuning parameters.
+
+ It is important to understand that keep-alive packets are neither emitted nor
+ received at the application level. It is only the network stacks which sees
+ them. For this reason, even if one side of the proxy already uses keep-alives
+ to maintain its connection alive, those keep-alive packets will not be
+ forwarded to the other side of the proxy.
+
+ Please note that this has nothing to do with HTTP keep-alive.
+
+ Using option "clitcpka" enables the emission of TCP keep-alive probes on the
+ client side of a connection, which should help when session expirations are
+ noticed between HAProxy and a client.
+
+ If this option has been enabled in a "defaults" section, it can be disabled
+ in a specific instance by prepending the "no" keyword before it.
+
+ See also : "option srvtcpka", "option tcpka"
+
+
+option contstats
+ Enable continuous traffic statistics updates
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments : none
+
+ By default, counters used for statistics calculation are incremented
+ only when a stream finishes. It works quite well when serving small
+ objects, but with big ones (for example large images or archives) or
+ with A/V streaming, a graph generated from HAProxy counters looks like
+ a hedgehog. With this option enabled counters get incremented frequently
+ along the stream, typically every 5 seconds, which is often enough to
+ produce clean graphs. Recounting touches a hotpath directly so it is not
+ not enabled by default, as it can cause a lot of wakeups for very large
+ session counts and cause a small performance drop.
+
+option disable-h2-upgrade
+no option disable-h2-upgrade
+ Enable or disable the implicit HTTP/2 upgrade from an HTTP/1.x client
+ connection.
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments : none
+
+ By default, HAProxy is able to implicitly upgrade an HTTP/1.x client
+ connection to an HTTP/2 connection if the first request it receives from a
+ given HTTP connection matches the HTTP/2 connection preface (i.e. the string
+ "PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n"). This way, it is possible to support
+ HTTP/1.x and HTTP/2 clients on a non-SSL connections. This option must be
+ used to disable the implicit upgrade. Note this implicit upgrade is only
+ supported for HTTP proxies, thus this option too. Note also it is possible to
+ force the HTTP/2 on clear connections by specifying "proto h2" on the bind
+ line. Finally, this option is applied on all bind lines. To disable implicit
+ HTTP/2 upgrades for a specific bind line, it is possible to use "proto h1".
+
+ If this option has been enabled in a "defaults" section, it can be disabled
+ in a specific instance by prepending the "no" keyword before it.
+
+option dontlog-normal
+no option dontlog-normal
+ Enable or disable logging of normal, successful connections
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments : none
+
+ There are large sites dealing with several thousand connections per second
+ and for which logging is a major pain. Some of them are even forced to turn
+ logs off and cannot debug production issues. Setting this option ensures that
+ normal connections, those which experience no error, no timeout, no retry nor
+ redispatch, will not be logged. This leaves disk space for anomalies. In HTTP
+ mode, the response status code is checked and return codes 5xx will still be
+ logged.
+
+ It is strongly discouraged to use this option as most of the time, the key to
+ complex issues is in the normal logs which will not be logged here. If you
+ need to separate logs, see the "log-separate-errors" option instead.
+
+ See also : "log", "dontlognull", "log-separate-errors" and section 8 about
+ logging.
+
+
+option dontlognull
+no option dontlognull
+ Enable or disable logging of null connections
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments : none
+
+ In certain environments, there are components which will regularly connect to
+ various systems to ensure that they are still alive. It can be the case from
+ another load balancer as well as from monitoring systems. By default, even a
+ simple port probe or scan will produce a log. If those connections pollute
+ the logs too much, it is possible to enable option "dontlognull" to indicate
+ that a connection on which no data has been transferred will not be logged,
+ which typically corresponds to those probes. Note that errors will still be
+ returned to the client and accounted for in the stats. If this is not what is
+ desired, option http-ignore-probes can be used instead.
+
+ It is generally recommended not to use this option in uncontrolled
+ environments (e.g. internet), otherwise scans and other malicious activities
+ would not be logged.
+
+ If this option has been enabled in a "defaults" section, it can be disabled
+ in a specific instance by prepending the "no" keyword before it.
+
+ See also : "log", "http-ignore-probes", "monitor-uri", and
+ section 8 about logging.
+
+option forwarded [ proto ]
+ [ host | host-expr <host_expr> ]
+ [ by | by-expr <by_expr> ] [ by_port | by_port-expr <by_port_expr>]
+ [ for | for-expr <for_expr> ] [ for_port | for_port-expr <for_port_expr>]
+no option forwarded
+ Enable insertion of the rfc 7239 forwarded header in requests sent to servers
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <host_expr> optional argument to specify a custom sample expression
+ those result will be used as 'host' parameter value
+
+ <by_expr> optional argument to specicy a custom sample expression
+ those result will be used as 'by' parameter nodename value
+
+ <for_expr> optional argument to specicy a custom sample expression
+ those result will be used as 'for' parameter nodename value
+
+ <by_port_expr> optional argument to specicy a custom sample expression
+ those result will be used as 'by' parameter nodeport value
+
+ <for_port_expr> optional argument to specicy a custom sample expression
+ those result will be used as 'for' parameter nodeport value
+
+
+ Since HAProxy works in reverse-proxy mode, servers are losing some request
+ context (request origin: client ip address, protocol used...)
+
+ A common way to address this limitation is to use the well known
+ x-forward-for and x-forward-* friends to expose some of this context to the
+ underlying servers/applications.
+ While this use to work and is widely deployed, it is not officially supported
+ by the IETF and can be the root of some interoperability as well as security
+ issues.
+
+ To solve this, a new HTTP extension has been described by the IETF:
+ forwarded header (RFC7239).
+ More information here: https://www.rfc-editor.org/rfc/rfc7239.html
+
+ The use of this single header allow to convey numerous details
+ within the same header, and most importantly, fixes the proxy chaining
+ issue. (the rfc allows for multiple chained proxies to append their own
+ values to an already existing header).
+
+ This option may be specified in defaults, listen or backend section, but it
+ will be ignored for frontend sections.
+
+ Setting option forwarded without arguments results in using default implicit
+ behavior.
+ Default behavior enables proto parameter and injects original client ip.
+
+ The equivalent explicit/manual configuration would be:
+ option forwarded proto for
+
+ The keyword 'by' is used to enable 'by' parameter ("nodename") in
+ forwarded header. It allows to embed request proxy information.
+ 'by' value will be set to proxy ip (destination address)
+ If not available (ie: UNIX listener), 'by' will be set to
+ "unknown".
+
+ The keyword 'by-expr' is used to enable 'by' parameter ("nodename") in
+ forwarded header. It allows to embed request proxy information.
+ 'by' value will be set to the result of the sample expression
+ <by_expr>, if valid, otherwise it will be set to "unknown".
+
+ The keyword 'for' is used to enable 'for' parameter ("nodename") in
+ forwarded header. It allows to embed request client information.
+ 'for' value will be set to client ip (source address)
+ If not available (ie: UNIX listener), 'for' will be set to
+ "unknown".
+
+ The keyword 'for-expr' is used to enable 'for' parameter ("nodename") in
+ forwarded header. It allows to embed request client information.
+ 'for' value will be set to the result of the sample expression
+ <for_expr>, if valid, otherwise it will be set to "unknown".
+
+ The keyword 'by_port' is used to provide "nodeport" info to
+ 'by' parameter. 'by_port' requires 'by' or 'by-expr' to be set or
+ it will be ignored.
+ "nodeport" will be set to proxy (destination) port if available,
+ otherwise it will be ignored.
+
+ The keyword 'by_port-expr' is used to provide "nodeport" info to
+ 'by' parameter. 'by_port-expr' requires 'by' or 'by-expr' to be set or
+ it will be ignored.
+ "nodeport" will be set to the result of the sample expression
+ <by_port_expr>, if valid, otherwise it will be ignored.
+
+ The keyword 'for_port' is used to provide "nodeport" info to
+ 'for' parameter. 'for_port' requires 'for' or 'for-expr' to be set or
+ it will be ignored.
+ "nodeport" will be set to client (source) port if available,
+ otherwise it will be ignored.
+
+ The keyword 'for_port-expr' is used to provide "nodeport" info to
+ 'for' parameter. 'for_port-expr' requires 'for' or 'for-expr' to be set or
+ it will be ignored.
+ "nodeport" will be set to the result of the sample expression
+ <for_port_expr>, if valid, otherwise it will be ignored.
+
+ Examples :
+ # Those servers want the ip address and protocol of the client request
+ # Resulting header would look like this:
+ # forwarded: proto=http;for=127.0.0.1
+ backend www_default
+ mode http
+ option forwarded
+ #equivalent to: option forwarded proto for
+
+ # Those servers want the requested host and hashed client ip address
+ # as well as client source port (you should use seed for xxh32 if ensuring
+ # ip privacy is a concern)
+ # Resulting header would look like this:
+ # forwarded: host="haproxy.org";for="_000000007F2F367E:60138"
+ backend www_host
+ mode http
+ option forwarded host for-expr src,xxh32,hex for_port
+
+ # Those servers want custom data in host, for and by parameters
+ # Resulting header would look like this:
+ # forwarded: host="host.com";by=_haproxy;for="[::1]:10"
+ backend www_custom
+ mode http
+ option forwarded host-expr str(host.com) by-expr str(_haproxy) for for_port-expr int(10)
+
+ # Those servers want random 'for' obfuscated identifiers for request
+ # tracing purposes while protecting sensitive IP information
+ # Resulting header would look like this:
+ # forwarded: for=_000000002B1F4D63
+ backend www_for_hide
+ mode http
+ option forwarded for-expr rand,hex
+
+ See also : "option forwardfor", "option originalto"
+
+option forwardfor [ except <network> ] [ header <name> ] [ if-none ]
+ Enable insertion of the X-Forwarded-For header to requests sent to servers
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments :
+ <network> is an optional argument used to disable this option for sources
+ matching <network>
+ <name> an optional argument to specify a different "X-Forwarded-For"
+ header name.
+
+ Since HAProxy works in reverse-proxy mode, the servers see its IP address as
+ their client address. This is sometimes annoying when the client's IP address
+ is expected in server logs. To solve this problem, the well-known HTTP header
+ "X-Forwarded-For" may be added by HAProxy to all requests sent to the server.
+ This header contains a value representing the client's IP address. Since this
+ header is always appended at the end of the existing header list, the server
+ must be configured to always use the last occurrence of this header only. See
+ the server's manual to find how to enable use of this standard header. Note
+ that only the last occurrence of the header must be used, since it is really
+ possible that the client has already brought one.
+
+ The keyword "header" may be used to supply a different header name to replace
+ the default "X-Forwarded-For". This can be useful where you might already
+ have a "X-Forwarded-For" header from a different application (e.g. stunnel),
+ and you need preserve it. Also if your backend server doesn't use the
+ "X-Forwarded-For" header and requires different one (e.g. Zeus Web Servers
+ require "X-Cluster-Client-IP").
+
+ Sometimes, a same HAProxy instance may be shared between a direct client
+ access and a reverse-proxy access (for instance when an SSL reverse-proxy is
+ used to decrypt HTTPS traffic). It is possible to disable the addition of the
+ header for a known source address or network by adding the "except" keyword
+ followed by the network address. In this case, any source IP matching the
+ network will not cause an addition of this header. Most common uses are with
+ private networks or 127.0.0.1. IPv4 and IPv6 are both supported.
+
+ Alternatively, the keyword "if-none" states that the header will only be
+ added if it is not present. This should only be used in perfectly trusted
+ environment, as this might cause a security issue if headers reaching HAProxy
+ are under the control of the end-user.
+
+ This option may be specified either in the frontend or in the backend. If at
+ least one of them uses it, the header will be added. Note that the backend's
+ setting of the header subargument takes precedence over the frontend's if
+ both are defined. In the case of the "if-none" argument, if at least one of
+ the frontend or the backend does not specify it, it wants the addition to be
+ mandatory, so it wins.
+
+ Example :
+ # Public HTTP address also used by stunnel on the same machine
+ frontend www
+ mode http
+ option forwardfor except 127.0.0.1 # stunnel already adds the header
+
+ # Those servers want the IP Address in X-Client
+ backend www
+ mode http
+ option forwardfor header X-Client
+
+ See also : "option httpclose", "option http-server-close",
+ "option http-keep-alive"
+
+
+option h1-case-adjust-bogus-client
+no option h1-case-adjust-bogus-client
+ Enable or disable the case adjustment of HTTP/1 headers sent to bogus clients
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments : none
+
+ There is no standard case for header names because, as stated in RFC7230,
+ they are case-insensitive. So applications must handle them in a case-
+ insensitive manner. But some bogus applications violate the standards and
+ erroneously rely on the cases most commonly used by browsers. This problem
+ becomes critical with HTTP/2 because all header names must be exchanged in
+ lower case, and HAProxy follows the same convention. All header names are
+ sent in lower case to clients and servers, regardless of the HTTP version.
+
+ When HAProxy receives an HTTP/1 response, its header names are converted to
+ lower case and manipulated and sent this way to the clients. If a client is
+ known to violate the HTTP standards and to fail to process a response coming
+ from HAProxy, it is possible to transform the lower case header names to a
+ different format when the response is formatted and sent to the client, by
+ enabling this option and specifying the list of headers to be reformatted
+ using the global directives "h1-case-adjust" or "h1-case-adjust-file". This
+ must only be a temporary workaround for the time it takes the client to be
+ fixed, because clients which require such workarounds might be vulnerable to
+ content smuggling attacks and must absolutely be fixed.
+
+ Please note that this option will not affect standards-compliant clients.
+
+ If this option has been enabled in a "defaults" section, it can be disabled
+ in a specific instance by prepending the "no" keyword before it.
+
+ See also: "option h1-case-adjust-bogus-server", "h1-case-adjust",
+ "h1-case-adjust-file".
+
+
+option h1-case-adjust-bogus-server
+no option h1-case-adjust-bogus-server
+ Enable or disable the case adjustment of HTTP/1 headers sent to bogus servers
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments : none
+
+ There is no standard case for header names because, as stated in RFC7230,
+ they are case-insensitive. So applications must handle them in a case-
+ insensitive manner. But some bogus applications violate the standards and
+ erroneously rely on the cases most commonly used by browsers. This problem
+ becomes critical with HTTP/2 because all header names must be exchanged in
+ lower case, and HAProxy follows the same convention. All header names are
+ sent in lower case to clients and servers, regardless of the HTTP version.
+
+ When HAProxy receives an HTTP/1 request, its header names are converted to
+ lower case and manipulated and sent this way to the servers. If a server is
+ known to violate the HTTP standards and to fail to process a request coming
+ from HAProxy, it is possible to transform the lower case header names to a
+ different format when the request is formatted and sent to the server, by
+ enabling this option and specifying the list of headers to be reformatted
+ using the global directives "h1-case-adjust" or "h1-case-adjust-file". This
+ must only be a temporary workaround for the time it takes the server to be
+ fixed, because servers which require such workarounds might be vulnerable to
+ content smuggling attacks and must absolutely be fixed.
+
+ Please note that this option will not affect standards-compliant servers.
+
+ If this option has been enabled in a "defaults" section, it can be disabled
+ in a specific instance by prepending the "no" keyword before it.
+
+ See also: "option h1-case-adjust-bogus-client", "h1-case-adjust",
+ "h1-case-adjust-file".
+
+
+option http-buffer-request
+no option http-buffer-request
+ Enable or disable waiting for whole HTTP request body before proceeding
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments : none
+
+ It is sometimes desirable to wait for the body of an HTTP request before
+ taking a decision. This is what is being done by "balance url_param" for
+ example. The first use case is to buffer requests from slow clients before
+ connecting to the server. Another use case consists in taking the routing
+ decision based on the request body's contents. This option placed in a
+ frontend or backend forces the HTTP processing to wait until either the whole
+ body is received or the request buffer is full. It can have undesired side
+ effects with some applications abusing HTTP by expecting unbuffered
+ transmissions between the frontend and the backend, so this should definitely
+ not be used by default.
+
+ See also : "option http-no-delay", "timeout http-request",
+ "http-request wait-for-body"
+
+
+option http-ignore-probes
+no option http-ignore-probes
+ Enable or disable logging of null connections and request timeouts
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments : none
+
+ Recently some browsers started to implement a "pre-connect" feature
+ consisting in speculatively connecting to some recently visited web sites
+ just in case the user would like to visit them. This results in many
+ connections being established to web sites, which end up in 408 Request
+ Timeout if the timeout strikes first, or 400 Bad Request when the browser
+ decides to close them first. These ones pollute the log and feed the error
+ counters. There was already "option dontlognull" but it's insufficient in
+ this case. Instead, this option does the following things :
+ - prevent any 400/408 message from being sent to the client if nothing
+ was received over a connection before it was closed;
+ - prevent any log from being emitted in this situation;
+ - prevent any error counter from being incremented
+
+ That way the empty connection is silently ignored. Note that it is better
+ not to use this unless it is clear that it is needed, because it will hide
+ real problems. The most common reason for not receiving a request and seeing
+ a 408 is due to an MTU inconsistency between the client and an intermediary
+ element such as a VPN, which blocks too large packets. These issues are
+ generally seen with POST requests as well as GET with large cookies. The logs
+ are often the only way to detect them.
+
+ If this option has been enabled in a "defaults" section, it can be disabled
+ in a specific instance by prepending the "no" keyword before it.
+
+ See also : "log", "dontlognull", "errorfile", and section 8 about logging.
+
+
+option http-keep-alive
+no option http-keep-alive
+ Enable or disable HTTP keep-alive from client to server for HTTP/1.x
+ connections
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments : none
+
+ By default HAProxy operates in keep-alive mode with regards to persistent
+ HTTP/1.x connections: for each connection it processes each request and
+ response, and leaves the connection idle on both sides. This mode may be
+ changed by several options such as "option http-server-close" or "option
+ httpclose". This option allows to set back the keep-alive mode, which can be
+ useful when another mode was used in a defaults section.
+
+ Setting "option http-keep-alive" enables HTTP keep-alive mode on the client-
+ and server- sides. This provides the lowest latency on the client side (slow
+ network) and the fastest session reuse on the server side at the expense
+ of maintaining idle connections to the servers. In general, it is possible
+ with this option to achieve approximately twice the request rate that the
+ "http-server-close" option achieves on small objects. There are mainly two
+ situations where this option may be useful :
+
+ - when the server is non-HTTP compliant and authenticates the connection
+ instead of requests (e.g. NTLM authentication)
+
+ - when the cost of establishing the connection to the server is significant
+ compared to the cost of retrieving the associated object from the server.
+
+ This last case can happen when the server is a fast static server of cache.
+
+ At the moment, logs will not indicate whether requests came from the same
+ session or not. The accept date reported in the logs corresponds to the end
+ of the previous request, and the request time corresponds to the time spent
+ waiting for a new request. The keep-alive request time is still bound to the
+ timeout defined by "timeout http-keep-alive" or "timeout http-request" if
+ not set.
+
+ This option disables and replaces any previous "option httpclose" or "option
+ http-server-close".
+
+ See also : "option httpclose",, "option http-server-close",
+ "option prefer-last-server" and "option http-pretend-keepalive".
+
+
+option http-no-delay
+no option http-no-delay
+ Instruct the system to favor low interactive delays over performance in HTTP
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments : none
+
+ In HTTP, each payload is unidirectional and has no notion of interactivity.
+ Any agent is expected to queue data somewhat for a reasonably low delay.
+ There are some very rare server-to-server applications that abuse the HTTP
+ protocol and expect the payload phase to be highly interactive, with many
+ interleaved data chunks in both directions within a single request. This is
+ absolutely not supported by the HTTP specification and will not work across
+ most proxies or servers. When such applications attempt to do this through
+ HAProxy, it works but they will experience high delays due to the network
+ optimizations which favor performance by instructing the system to wait for
+ enough data to be available in order to only send full packets. Typical
+ delays are around 200 ms per round trip. Note that this only happens with
+ abnormal uses. Normal uses such as CONNECT requests nor WebSockets are not
+ affected.
+
+ When "option http-no-delay" is present in either the frontend or the backend
+ used by a connection, all such optimizations will be disabled in order to
+ make the exchanges as fast as possible. Of course this offers no guarantee on
+ the functionality, as it may break at any other place. But if it works via
+ HAProxy, it will work as fast as possible. This option should never be used
+ by default, and should never be used at all unless such a buggy application
+ is discovered. The impact of using this option is an increase of bandwidth
+ usage and CPU usage, which may significantly lower performance in high
+ latency environments.
+
+ See also : "option http-buffer-request"
+
+
+option http-pretend-keepalive
+no option http-pretend-keepalive
+ Define whether HAProxy will announce keepalive for HTTP/1.x connection to the
+ server or not
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments : none
+
+ When running with "option http-server-close" or "option httpclose", HAProxy
+ adds a "Connection: close" header to the HTTP/1.x request forwarded to the
+ server. Unfortunately, when some servers see this header, they automatically
+ refrain from using the chunked encoding for responses of unknown length,
+ while this is totally unrelated. The effect is that a client or a cache could
+ receive an incomplete response without being aware of it, and consider the
+ response complete.
+
+ By setting "option http-pretend-keepalive", HAProxy will make the server
+ believe it will keep the connection alive. The server will then not fall back
+ to the abnormal undesired above. When HAProxy gets the whole response, it
+ will close the connection with the server just as it would do with the
+ "option httpclose". That way the client gets a normal response and the
+ connection is correctly closed on the server side.
+
+ It is recommended not to enable this option by default, because most servers
+ will more efficiently close the connection themselves after the last packet,
+ and release its buffers slightly earlier. Also, the added packet on the
+ network could slightly reduce the overall peak performance. However it is
+ worth noting that when this option is enabled, HAProxy will have slightly
+ less work to do. So if HAProxy is the bottleneck on the whole architecture,
+ enabling this option might save a few CPU cycles.
+
+ This option may be set in backend and listen sections. Using it in a frontend
+ section will be ignored and a warning will be reported during startup. It is
+ a backend related option, so there is no real reason to set it on a
+ frontend.
+
+ If this option has been enabled in a "defaults" section, it can be disabled
+ in a specific instance by prepending the "no" keyword before it.
+
+ See also : "option httpclose", "option http-server-close", and
+ "option http-keep-alive"
+
+option http-restrict-req-hdr-names { preserve | delete | reject }
+ Set HAProxy policy about HTTP request header names containing characters
+ outside the "[a-zA-Z0-9-]" charset
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments :
+ preserve disable the filtering. It is the default mode for HTTP proxies
+ with no FastCGI application configured.
+
+ delete remove request headers with a name containing a character
+ outside the "[a-zA-Z0-9-]" charset. It is the default mode for
+ HTTP backends with a configured FastCGI application.
+
+ reject reject the request with a 403-Forbidden response if it contains a
+ header name with a character outside the "[a-zA-Z0-9-]" charset.
+
+ This option may be used to restrict the request header names to alphanumeric
+ and hyphen characters ([A-Za-z0-9-]). This may be mandatory to interoperate
+ with non-HTTP compliant servers that fail to handle some characters in header
+ names. It may also be mandatory for FastCGI applications because all
+ non-alphanumeric characters in header names are replaced by an underscore
+ ('_'). Thus, it is easily possible to mix up header names and bypass some
+ rules. For instance, "X-Forwarded-For" and "X_Forwarded-For" headers are both
+ converted to "HTTP_X_FORWARDED_FOR" in FastCGI.
+
+ Note this option is evaluated per proxy and after the http-request rules
+ evaluation.
+
+option http-server-close
+no option http-server-close
+ Enable or disable HTTP/1.x connection closing on the server side
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments : none
+
+ By default HAProxy operates in keep-alive mode with regards to persistent
+ HTTP/1.x connections: for each connection it processes each request and
+ response, and leaves the connection idle on both sides. This mode may be
+ changed by several options such as "option http-server-close" or "option
+ httpclose". Setting "option http-server-close" enables HTTP connection-close
+ mode on the server side while keeping the ability to support HTTP keep-alive
+ and pipelining on the client side. This provides the lowest latency on the
+ client side (slow network) and the fastest session reuse on the server side
+ to save server resources, similarly to "option httpclose". It also permits
+ non-keepalive capable servers to be served in keep-alive mode to the clients
+ if they conform to the requirements of RFC7230. Please note that some servers
+ do not always conform to those requirements when they see "Connection: close"
+ in the request. The effect will be that keep-alive will never be used. A
+ workaround consists in enabling "option http-pretend-keepalive".
+
+ At the moment, logs will not indicate whether requests came from the same
+ session or not. The accept date reported in the logs corresponds to the end
+ of the previous request, and the request time corresponds to the time spent
+ waiting for a new request. The keep-alive request time is still bound to the
+ timeout defined by "timeout http-keep-alive" or "timeout http-request" if
+ not set.
+
+ This option may be set both in a frontend and in a backend. It is enabled if
+ at least one of the frontend or backend holding a connection has it enabled.
+ It disables and replaces any previous "option httpclose" or "option
+ http-keep-alive". Please check section 4 ("Proxies") to see how this option
+ combines with others when frontend and backend options differ.
+
+ If this option has been enabled in a "defaults" section, it can be disabled
+ in a specific instance by prepending the "no" keyword before it.
+
+ See also : "option httpclose", "option http-pretend-keepalive" and
+ "option http-keep-alive".
+
+option http-use-proxy-header
+no option http-use-proxy-header
+ Make use of non-standard Proxy-Connection header instead of Connection
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments : none
+
+ While RFC7230 explicitly states that HTTP/1.1 agents must use the
+ Connection header to indicate their wish of persistent or non-persistent
+ connections, both browsers and proxies ignore this header for proxied
+ connections and make use of the undocumented, non-standard Proxy-Connection
+ header instead. The issue begins when trying to put a load balancer between
+ browsers and such proxies, because there will be a difference between what
+ HAProxy understands and what the client and the proxy agree on.
+
+ By setting this option in a frontend, HAProxy can automatically switch to use
+ that non-standard header if it sees proxied requests. A proxied request is
+ defined here as one where the URI begins with neither a '/' nor a '*'. This
+ is incompatible with the HTTP tunnel mode. Note that this option can only be
+ specified in a frontend and will affect the request along its whole life.
+
+ Also, when this option is set, a request which requires authentication will
+ automatically switch to use proxy authentication headers if it is itself a
+ proxied request. That makes it possible to check or enforce authentication in
+ front of an existing proxy.
+
+ This option should normally never be used, except in front of a proxy.
+
+ See also : "option httpclose", and "option http-server-close".
+
+option httpchk
+option httpchk <uri>
+option httpchk <method> <uri>
+option httpchk <method> <uri> <version>
+ Enables HTTP protocol to check on the servers health
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <method> is the optional HTTP method used with the requests. When not set,
+ the "OPTIONS" method is used, as it generally requires low server
+ processing and is easy to filter out from the logs. Any method
+ may be used, though it is not recommended to invent non-standard
+ ones.
+
+ <uri> is the URI referenced in the HTTP requests. It defaults to " / "
+ which is accessible by default on almost any server, but may be
+ changed to any other URI. Query strings are permitted.
+
+ <version> is the optional HTTP version string. It defaults to "HTTP/1.0"
+ but some servers might behave incorrectly in HTTP 1.0, so turning
+ it to HTTP/1.1 may sometimes help. Note that the Host field is
+ mandatory in HTTP/1.1, use "http-check send" directive to add it.
+
+ By default, server health checks only consist in trying to establish a TCP
+ connection. When "option httpchk" is specified, a complete HTTP request is
+ sent once the TCP connection is established, and responses 2xx and 3xx are
+ considered valid, while all other ones indicate a server failure, including
+ the lack of any response.
+
+ Combined with "http-check" directives, it is possible to customize the
+ request sent during the HTTP health checks or the matching rules on the
+ response. It is also possible to configure a send/expect sequence, just like
+ with the directive "tcp-check" for TCP health checks.
+
+ The server configuration is used by default to open connections to perform
+ HTTP health checks. By it is also possible to overwrite server parameters
+ using "http-check connect" rules.
+
+ "httpchk" option does not necessarily require an HTTP backend, it also works
+ with plain TCP backends. This is particularly useful to check simple scripts
+ bound to some dedicated ports using the inetd daemon. However, it will always
+ internally relies on an HTX multiplexer. Thus, it means the request
+ formatting and the response parsing will be strict.
+
+ Examples :
+ # Relay HTTPS traffic to Apache instance and check service availability
+ # using HTTP request "OPTIONS * HTTP/1.1" on port 80.
+ backend https_relay
+ mode tcp
+ option httpchk OPTIONS * HTTP/1.1
+ http-check send hdr Host www
+ server apache1 192.168.1.1:443 check port 80
+
+ See also : "option ssl-hello-chk", "option smtpchk", "option mysql-check",
+ "option pgsql-check", "http-check" and the "check", "port" and
+ "inter" server options.
+
+
+option httpclose
+no option httpclose
+ Enable or disable HTTP/1.x connection closing
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments : none
+
+ By default HAProxy operates in keep-alive mode with regards to persistent
+ HTTP/1.x connections: for each connection it processes each request and
+ response, and leaves the connection idle on both sides. This mode may be
+ changed by several options such as "option http-server-close" or "option
+ httpclose".
+
+ If "option httpclose" is set, HAProxy will close the client or the server
+ connection, depending where the option is set. The frontend is considered for
+ client connections while the backend is considered for server ones. If the
+ option is set on a listener, it is applied both on client and server
+ connections. It will check if a "Connection: close" header is already set in
+ each direction, and will add one if missing.
+
+ This option may also be combined with "option http-pretend-keepalive", which
+ will disable sending of the "Connection: close" request header, but will
+ still cause the connection to be closed once the whole response is received.
+
+ It disables and replaces any previous "option http-server-close" or "option
+ http-keep-alive".
+
+ If this option has been enabled in a "defaults" section, it can be disabled
+ in a specific instance by prepending the "no" keyword before it.
+
+ See also : "option http-server-close".
+
+
+option httplog [ clf ]
+ Enable logging of HTTP request, stream state and timers
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments :
+ clf if the "clf" argument is added, then the output format will be
+ the CLF format instead of HAProxy's default HTTP format. You can
+ use this when you need to feed HAProxy's logs through a specific
+ log analyzer which only support the CLF format and which is not
+ extensible.
+
+ By default, the log output format is very poor, as it only contains the
+ source and destination addresses, and the instance name. By specifying
+ "option httplog", each log line turns into a much richer format including,
+ but not limited to, the HTTP request, the connection timers, the stream
+ status, the connections numbers, the captured headers and cookies, the
+ frontend, backend and server name, and of course the source address and
+ ports.
+
+ Specifying only "option httplog" will automatically clear the 'clf' mode
+ if it was set by default.
+
+ "option httplog" overrides any previous "log-format" directive.
+
+ See also : section 8 about logging.
+
+option httpslog
+ Enable logging of HTTPS request, stream state and timers
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ By default, the log output format is very poor, as it only contains the
+ source and destination addresses, and the instance name. By specifying
+ "option httpslog", each log line turns into a much richer format including,
+ but not limited to, the HTTP request, the connection timers, the stream
+ status, the connections numbers, the captured headers and cookies, the
+ frontend, backend and server name, the SSL certificate verification and SSL
+ handshake statuses, and of course the source address and ports.
+
+ "option httpslog" overrides any previous "log-format" directive.
+
+ See also : section 8 about logging.
+
+
+option independent-streams
+no option independent-streams
+ Enable or disable independent timeout processing for both directions
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments : none
+
+ By default, when data is sent over a socket, both the write timeout and the
+ read timeout for that socket are refreshed, because we consider that there is
+ activity on that socket, and we have no other means of guessing if we should
+ receive data or not.
+
+ While this default behavior is desirable for almost all applications, there
+ exists a situation where it is desirable to disable it, and only refresh the
+ read timeout if there are incoming data. This happens on streams with large
+ timeouts and low amounts of exchanged data such as telnet session. If the
+ server suddenly disappears, the output data accumulates in the system's
+ socket buffers, both timeouts are correctly refreshed, and there is no way
+ to know the server does not receive them, so we don't timeout. However, when
+ the underlying protocol always echoes sent data, it would be enough by itself
+ to detect the issue using the read timeout. Note that this problem does not
+ happen with more verbose protocols because data won't accumulate long in the
+ socket buffers.
+
+ When this option is set on the frontend, it will disable read timeout updates
+ on data sent to the client. There probably is little use of this case. When
+ the option is set on the backend, it will disable read timeout updates on
+ data sent to the server. Doing so will typically break large HTTP posts from
+ slow lines, so use it with caution.
+
+ See also : "timeout client", "timeout server" and "timeout tunnel"
+
+
+option ldap-check
+ Use LDAPv3 health checks for server testing
+
+ May be used in the following contexts: tcp
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments : none
+
+ It is possible to test that the server correctly talks LDAPv3 instead of just
+ testing that it accepts the TCP connection. When this option is set, an
+ LDAPv3 anonymous simple bind message is sent to the server, and the response
+ is analyzed to find an LDAPv3 bind response message.
+
+ The server is considered valid only when the LDAP response contains success
+ resultCode (http://tools.ietf.org/html/rfc4511#section-4.1.9).
+
+ Logging of bind requests is server dependent see your documentation how to
+ configure it.
+
+ Example :
+ option ldap-check
+
+ See also : "option httpchk"
+
+
+option external-check
+ Use external processes for server health checks
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ It is possible to test the health of a server using an external command.
+ This is achieved by running the executable set using "external-check
+ command".
+
+ Requires the "external-check" global to be set.
+
+ See also : "external-check", "external-check command", "external-check path"
+
+
+option idle-close-on-response
+no option idle-close-on-response
+ Avoid closing idle frontend connections if a soft stop is in progress
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments : none
+
+ By default, idle connections will be closed during a soft stop. In some
+ environments, a client talking to the proxy may have prepared some idle
+ connections in order to send requests later. If there is no proper retry on
+ write errors, this can result in errors while haproxy is reloading. Even
+ though a proper implementation should retry on connection/write errors, this
+ option was introduced to support backwards compatibility with haproxy prior
+ to version 2.4. Indeed before v2.4, haproxy used to wait for a last request
+ and response to add a "connection: close" header before closing, thus
+ notifying the client that the connection would not be reusable.
+
+ In a real life example, this behavior was seen in AWS using the ALB in front
+ of a haproxy. The end result was ALB sending 502 during haproxy reloads.
+
+ Users are warned that using this option may increase the number of old
+ processes if connections remain idle for too long. Adjusting the client
+ timeouts and/or the "hard-stop-after" parameter accordingly might be
+ needed in case of frequent reloads.
+
+ See also: "timeout client", "timeout client-fin", "timeout http-request",
+ "hard-stop-after"
+
+
+option log-health-checks
+no option log-health-checks
+ Enable or disable logging of health checks status updates
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments : none
+
+ By default, failed health check are logged if server is UP and successful
+ health checks are logged if server is DOWN, so the amount of additional
+ information is limited.
+
+ When this option is enabled, any change of the health check status or to
+ the server's health will be logged, so that it becomes possible to know
+ that a server was failing occasional checks before crashing, or exactly when
+ it failed to respond a valid HTTP status, then when the port started to
+ reject connections, then when the server stopped responding at all.
+
+ Note that status changes not caused by health checks (e.g. enable/disable on
+ the CLI) are intentionally not logged by this option.
+
+ See also: "option httpchk", "option ldap-check", "option mysql-check",
+ "option pgsql-check", "option redis-check", "option smtpchk",
+ "option tcp-check", "log" and section 8 about logging.
+
+
+option log-separate-errors
+no option log-separate-errors
+ Change log level for non-completely successful connections
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments : none
+
+ Sometimes looking for errors in logs is not easy. This option makes HAProxy
+ raise the level of logs containing potentially interesting information such
+ as errors, timeouts, retries, redispatches, or HTTP status codes 5xx. The
+ level changes from "info" to "err". This makes it possible to log them
+ separately to a different file with most syslog daemons. Be careful not to
+ remove them from the original file, otherwise you would lose ordering which
+ provides very important information.
+
+ Using this option, large sites dealing with several thousand connections per
+ second may log normal traffic to a rotating buffer and only archive smaller
+ error logs.
+
+ See also : "log", "dontlognull", "dontlog-normal" and section 8 about
+ logging.
+
+
+option logasap
+no option logasap
+ Enable or disable early logging.
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments : none
+
+ By default, logs are emitted when all the log format variables and sample
+ fetches used in the definition of the log-format string return a value, or
+ when the stream is terminated. This allows the built in log-format strings
+ to account for the transfer time, or the number of bytes in log messages.
+
+ When handling long lived connections such as large file transfers or RDP,
+ it may take a while for the request or connection to appear in the logs.
+ Using "option logasap", the log message is created as soon as the server
+ connection is established in mode tcp, or as soon as the server sends the
+ complete headers in mode http. Missing information in the logs will be the
+ total number of bytes which will only indicate the amount of data transferred
+ before the message was created and the total time which will not take the
+ remainder of the connection life or transfer time into account. For the case
+ of HTTP, it is good practice to capture the Content-Length response header
+ so that the logs at least indicate how many bytes are expected to be
+ transferred.
+
+ Examples :
+ listen http_proxy 0.0.0.0:80
+ mode http
+ option httplog
+ option logasap
+ log 192.168.2.200 local3
+
+ >>> Feb 6 12:14:14 localhost \
+ haproxy[14389]: 10.0.1.2:33317 [06/Feb/2009:12:14:14.655] http-in \
+ static/srv1 9/10/7/14/+30 200 +243 - - ---- 3/1/1/1/0 1/0 \
+ "GET /image.iso HTTP/1.0"
+
+ See also : "option httplog", "capture response header", and section 8 about
+ logging.
+
+
+option mysql-check [ user <username> [ { post-41 | pre-41 } ] ]
+ Use MySQL health checks for server testing
+
+ May be used in the following contexts: tcp
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <username> This is the username which will be used when connecting to MySQL
+ server.
+ post-41 Send post v4.1 client compatible checks (the default)
+ pre-41 Send pre v4.1 client compatible checks
+
+ If you specify a username, the check consists of sending two MySQL packet,
+ one Client Authentication packet, and one QUIT packet, to correctly close
+ MySQL session. We then parse the MySQL Handshake Initialization packet and/or
+ Error packet. It is a basic but useful test which does not produce error nor
+ aborted connect on the server. However, it requires an unlocked authorised
+ user without a password. To create a basic limited user in MySQL with
+ optional resource limits:
+
+ CREATE USER '<username>'@'<ip_of_haproxy|network_of_haproxy/netmask>'
+ /*!50701 WITH MAX_QUERIES_PER_HOUR 1 MAX_UPDATES_PER_HOUR 0 */
+ /*M!100201 MAX_STATEMENT_TIME 0.0001 */;
+
+ If you don't specify a username (it is deprecated and not recommended), the
+ check only consists in parsing the Mysql Handshake Initialization packet or
+ Error packet, we don't send anything in this mode. It was reported that it
+ can generate lockout if check is too frequent and/or if there is not enough
+ traffic. In fact, you need in this case to check MySQL "max_connect_errors"
+ value as if a connection is established successfully within fewer than MySQL
+ "max_connect_errors" attempts after a previous connection was interrupted,
+ the error count for the host is cleared to zero. If HAProxy's server get
+ blocked, the "FLUSH HOSTS" statement is the only way to unblock it.
+
+ Remember that this does not check database presence nor database consistency.
+ To do this, you can use an external check with xinetd for example.
+
+ The check requires MySQL >=3.22, for older version, please use TCP check.
+
+ Most often, an incoming MySQL server needs to see the client's IP address for
+ various purposes, including IP privilege matching and connection logging.
+ When possible, it is often wise to masquerade the client's IP address when
+ connecting to the server using the "usesrc" argument of the "source" keyword,
+ which requires the transparent proxy feature to be compiled in, and the MySQL
+ server to route the client via the machine hosting HAProxy.
+
+ See also: "option httpchk"
+
+
+option nolinger
+no option nolinger
+ Enable or disable immediate session resource cleaning after close
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments : none
+
+ When clients or servers abort connections in a dirty way (e.g. they are
+ physically disconnected), the session timeouts triggers and the session is
+ closed. But it will remain in FIN_WAIT1 state for some time in the system,
+ using some resources and possibly limiting the ability to establish newer
+ connections.
+
+ When this happens, it is possible to activate "option nolinger" which forces
+ the system to immediately remove any socket's pending data on close. Thus,
+ a TCP RST is emitted, any pending data are truncated, and the session is
+ instantly purged from the system's tables. The generally visible effect for
+ a client is that responses are truncated if the close happens with a last
+ block of data (e.g. on a redirect or error response). On the server side,
+ it may help release the source ports immediately when forwarding a client
+ aborts in tunnels. In both cases, TCP resets are emitted and given that
+ the session is instantly destroyed, there will be no retransmit. On a lossy
+ network this can increase problems, especially when there is a firewall on
+ the lossy side, because the firewall might see and process the reset (hence
+ purge its session) and block any further traffic for this session,, including
+ retransmits from the other side. So if the other side doesn't receive it,
+ it will never receive any RST again, and the firewall might log many blocked
+ packets.
+
+ For all these reasons, it is strongly recommended NOT to use this option,
+ unless absolutely needed as a last resort. In most situations, using the
+ "client-fin" or "server-fin" timeouts achieves similar results with a more
+ reliable behavior. On Linux it's also possible to use the "tcp-ut" bind or
+ server setting.
+
+ This option may be used both on frontends and backends, depending on the side
+ where it is required. Use it on the frontend for clients, and on the backend
+ for servers. While this option is technically supported in "defaults"
+ sections, it must really not be used there as it risks to accidentally
+ propagate to sections that must no use it and to cause problems there.
+
+ If this option has been enabled in a "defaults" section, it can be disabled
+ in a specific instance by prepending the "no" keyword before it.
+
+ See also: "timeout client-fin", "timeout server-fin", "tcp-ut" bind or server
+ keywords.
+
+option originalto [ except <network> ] [ header <name> ]
+ Enable insertion of the X-Original-To header to requests sent to servers
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments :
+ <network> is an optional argument used to disable this option for sources
+ matching <network>
+ <name> an optional argument to specify a different "X-Original-To"
+ header name.
+
+ Since HAProxy can work in transparent mode, every request from a client can
+ be redirected to the proxy and HAProxy itself can proxy every request to a
+ complex SQUID environment and the destination host from SO_ORIGINAL_DST will
+ be lost. This is annoying when you want access rules based on destination ip
+ addresses. To solve this problem, a new HTTP header "X-Original-To" may be
+ added by HAProxy to all requests sent to the server. This header contains a
+ value representing the original destination IP address. Since this must be
+ configured to always use the last occurrence of this header only. Note that
+ only the last occurrence of the header must be used, since it is really
+ possible that the client has already brought one.
+
+ The keyword "header" may be used to supply a different header name to replace
+ the default "X-Original-To". This can be useful where you might already
+ have a "X-Original-To" header from a different application, and you need
+ preserve it. Also if your backend server doesn't use the "X-Original-To"
+ header and requires different one.
+
+ Sometimes, a same HAProxy instance may be shared between a direct client
+ access and a reverse-proxy access (for instance when an SSL reverse-proxy is
+ used to decrypt HTTPS traffic). It is possible to disable the addition of the
+ header for a known destination address or network by adding the "except"
+ keyword followed by the network address. In this case, any destination IP
+ matching the network will not cause an addition of this header. Most common
+ uses are with private networks or 127.0.0.1. IPv4 and IPv6 are both
+ supported.
+
+ This option may be specified either in the frontend or in the backend. If at
+ least one of them uses it, the header will be added. Note that the backend's
+ setting of the header subargument takes precedence over the frontend's if
+ both are defined.
+
+ Examples :
+ # Original Destination address
+ frontend www
+ mode http
+ option originalto except 127.0.0.1
+
+ # Those servers want the IP Address in X-Client-Dst
+ backend www
+ mode http
+ option originalto header X-Client-Dst
+
+ See also : "option httpclose", "option http-server-close".
+
+
+option persist
+no option persist
+ Enable or disable forced persistence on down servers
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments : none
+
+ When an HTTP request reaches a backend with a cookie which references a dead
+ server, by default it is redispatched to another server. It is possible to
+ force the request to be sent to the dead server first using "option persist"
+ if absolutely needed. A common use case is when servers are under extreme
+ load and spend their time flapping. In this case, the users would still be
+ directed to the server they opened the session on, in the hope they would be
+ correctly served. It is recommended to use "option redispatch" in conjunction
+ with this option so that in the event it would not be possible to connect to
+ the server at all (server definitely dead), the client would finally be
+ redirected to another valid server.
+
+ If this option has been enabled in a "defaults" section, it can be disabled
+ in a specific instance by prepending the "no" keyword before it.
+
+ See also : "option redispatch", "retries", "force-persist"
+
+
+option pgsql-check user <username>
+ Use PostgreSQL health checks for server testing
+
+ May be used in the following contexts: tcp
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <username> This is the username which will be used when connecting to
+ PostgreSQL server.
+
+ The check sends a PostgreSQL StartupMessage and waits for either
+ Authentication request or ErrorResponse message. It is a basic but useful
+ test which does not produce error nor aborted connect on the server.
+ This check is identical with the "mysql-check".
+
+ See also: "option httpchk"
+
+
+option prefer-last-server
+no option prefer-last-server
+ Allow multiple load balanced requests to remain on the same server
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments : none
+
+ When the load balancing algorithm in use is not deterministic, and a previous
+ request was sent to a server to which HAProxy still holds a connection, it is
+ sometimes desirable that subsequent requests on a same session go to the same
+ server as much as possible. Note that this is different from persistence, as
+ we only indicate a preference which HAProxy tries to apply without any form
+ of warranty. The real use is for keep-alive connections sent to servers. When
+ this option is used, HAProxy will try to reuse the same connection that is
+ attached to the server instead of rebalancing to another server, causing a
+ close of the connection. This can make sense for static file servers. It does
+ not make much sense to use this in combination with hashing algorithms. Note,
+ HAProxy already automatically tries to stick to a server which sends a 401 or
+ to a proxy which sends a 407 (authentication required), when the load
+ balancing algorithm is not deterministic. This is mandatory for use with the
+ broken NTLM authentication challenge, and significantly helps in
+ troubleshooting some faulty applications. Option prefer-last-server might be
+ desirable in these environments as well, to avoid redistributing the traffic
+ after every other response.
+
+ If this option has been enabled in a "defaults" section, it can be disabled
+ in a specific instance by prepending the "no" keyword before it.
+
+ See also: "option http-keep-alive"
+
+
+option redispatch
+option redispatch <interval>
+no option redispatch
+ Enable or disable session redistribution in case of connection failure
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <interval> The optional integer value that controls how often redispatches
+ occur when retrying connections. Positive value P indicates a
+ redispatch is desired on every Pth retry, and negative value
+ N indicate a redispatch is desired on the Nth retry prior to the
+ last retry. For example, the default of -1 preserves the
+ historical behavior of redispatching on the last retry, a
+ positive value of 1 would indicate a redispatch on every retry,
+ and a positive value of 3 would indicate a redispatch on every
+ third retry. You can disable redispatches with a value of 0.
+
+
+ In HTTP mode, if a server designated by a cookie is down, clients may
+ definitely stick to it because they cannot flush the cookie, so they will not
+ be able to access the service anymore.
+
+ Specifying "option redispatch" will allow the proxy to break cookie or
+ consistent hash based persistence and redistribute them to a working server.
+
+ Active servers are selected from a subset of the list of available
+ servers. Active servers that are not down or in maintenance (i.e., whose
+ health is not checked or that have been checked as "up"), are selected in the
+ following order:
+
+ 1. Any active, non-backup server, if any, or,
+
+ 2. If the "allbackups" option is not set, the first backup server in the
+ list, or
+
+ 3. If the "allbackups" option is set, any backup server.
+
+ When a retry occurs, HAProxy tries to select another server than the last
+ one. The new server is selected from the current list of servers.
+
+ Sometimes, if the list is updated between retries (e.g., if numerous retries
+ occur and last longer than the time needed to check that a server is down,
+ remove it from the list and fall back on the list of backup servers),
+ connections may be redirected to a backup server, though.
+
+ It also allows to retry connections to another server in case of multiple
+ connection failures. Of course, it requires having "retries" set to a nonzero
+ value.
+
+ If this option has been enabled in a "defaults" section, it can be disabled
+ in a specific instance by prepending the "no" keyword before it.
+
+ See also : "retries", "force-persist"
+
+
+option redis-check
+ Use redis health checks for server testing
+
+ May be used in the following contexts: tcp
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments : none
+
+ It is possible to test that the server correctly talks REDIS protocol instead
+ of just testing that it accepts the TCP connection. When this option is set,
+ a PING redis command is sent to the server, and the response is analyzed to
+ find the "+PONG" response message.
+
+ Example :
+ option redis-check
+
+ See also : "option httpchk", "option tcp-check", "tcp-check expect"
+
+
+option smtpchk
+option smtpchk <hello> <domain>
+ Use SMTP health checks for server testing
+
+ May be used in the following contexts: tcp
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <hello> is an optional argument. It is the "hello" command to use. It can
+ be either "HELO" (for SMTP) or "EHLO" (for ESMTP). All other
+ values will be turned into the default command ("HELO").
+
+ <domain> is the domain name to present to the server. It may only be
+ specified (and is mandatory) if the hello command has been
+ specified. By default, "localhost" is used.
+
+ When "option smtpchk" is set, the health checks will consist in TCP
+ connections followed by an SMTP command. By default, this command is
+ "HELO localhost". The server's return code is analyzed and only return codes
+ starting with a "2" will be considered as valid. All other responses,
+ including a lack of response will constitute an error and will indicate a
+ dead server.
+
+ This test is meant to be used with SMTP servers or relays. Depending on the
+ request, it is possible that some servers do not log each connection attempt,
+ so you may want to experiment to improve the behavior. Using telnet on port
+ 25 is often easier than adjusting the configuration.
+
+ Most often, an incoming SMTP server needs to see the client's IP address for
+ various purposes, including spam filtering, anti-spoofing and logging. When
+ possible, it is often wise to masquerade the client's IP address when
+ connecting to the server using the "usesrc" argument of the "source" keyword,
+ which requires the transparent proxy feature to be compiled in.
+
+ Example :
+ option smtpchk HELO mydomain.org
+
+ See also : "option httpchk", "source"
+
+
+option socket-stats
+no option socket-stats
+
+ Enable or disable collecting & providing separate statistics for each socket.
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments : none
+
+
+option splice-auto
+no option splice-auto
+ Enable or disable automatic kernel acceleration on sockets in both directions
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments : none
+
+ When this option is enabled either on a frontend or on a backend, HAProxy
+ will automatically evaluate the opportunity to use kernel tcp splicing to
+ forward data between the client and the server, in either direction. HAProxy
+ uses heuristics to estimate if kernel splicing might improve performance or
+ not. Both directions are handled independently. Note that the heuristics used
+ are not much aggressive in order to limit excessive use of splicing. This
+ option requires splicing to be enabled at compile time, and may be globally
+ disabled with the global option "nosplice". Since splice uses pipes, using it
+ requires that there are enough spare pipes.
+
+ Important note: kernel-based TCP splicing is a Linux-specific feature which
+ first appeared in kernel 2.6.25. It offers kernel-based acceleration to
+ transfer data between sockets without copying these data to user-space, thus
+ providing noticeable performance gains and CPU cycles savings. Since many
+ early implementations are buggy, corrupt data and/or are inefficient, this
+ feature is not enabled by default, and it should be used with extreme care.
+ While it is not possible to detect the correctness of an implementation,
+ 2.6.29 is the first version offering a properly working implementation. In
+ case of doubt, splicing may be globally disabled using the global "nosplice"
+ keyword.
+
+ Example :
+ option splice-auto
+
+ If this option has been enabled in a "defaults" section, it can be disabled
+ in a specific instance by prepending the "no" keyword before it.
+
+ See also : "option splice-request", "option splice-response", and global
+ options "nosplice" and "maxpipes"
+
+
+option splice-request
+no option splice-request
+ Enable or disable automatic kernel acceleration on sockets for requests
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments : none
+
+ When this option is enabled either on a frontend or on a backend, HAProxy
+ will use kernel tcp splicing whenever possible to forward data going from
+ the client to the server. It might still use the recv/send scheme if there
+ are no spare pipes left. This option requires splicing to be enabled at
+ compile time, and may be globally disabled with the global option "nosplice".
+ Since splice uses pipes, using it requires that there are enough spare pipes.
+
+ Important note: see "option splice-auto" for usage limitations.
+
+ Example :
+ option splice-request
+
+ If this option has been enabled in a "defaults" section, it can be disabled
+ in a specific instance by prepending the "no" keyword before it.
+
+ See also : "option splice-auto", "option splice-response", and global options
+ "nosplice" and "maxpipes"
+
+
+option splice-response
+no option splice-response
+ Enable or disable automatic kernel acceleration on sockets for responses
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments : none
+
+ When this option is enabled either on a frontend or on a backend, HAProxy
+ will use kernel tcp splicing whenever possible to forward data going from
+ the server to the client. It might still use the recv/send scheme if there
+ are no spare pipes left. This option requires splicing to be enabled at
+ compile time, and may be globally disabled with the global option "nosplice".
+ Since splice uses pipes, using it requires that there are enough spare pipes.
+
+ Important note: see "option splice-auto" for usage limitations.
+
+ Example :
+ option splice-response
+
+ If this option has been enabled in a "defaults" section, it can be disabled
+ in a specific instance by prepending the "no" keyword before it.
+
+ See also : "option splice-auto", "option splice-request", and global options
+ "nosplice" and "maxpipes"
+
+
+option spop-check
+ Use SPOP health checks for server testing
+
+ May be used in the following contexts: tcp
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments : none
+
+ It is possible to test that the server correctly talks SPOP protocol instead
+ of just testing that it accepts the TCP connection. When this option is set,
+ a HELLO handshake is performed between HAProxy and the server, and the
+ response is analyzed to check no error is reported.
+
+ Example :
+ option spop-check
+
+ See also : "option httpchk"
+
+
+option srvtcpka
+no option srvtcpka
+ Enable or disable the sending of TCP keepalive packets on the server side
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments : none
+
+ When there is a firewall or any session-aware component between a client and
+ a server, and when the protocol involves very long sessions with long idle
+ periods (e.g. remote desktops), there is a risk that one of the intermediate
+ components decides to expire a session which has remained idle for too long.
+
+ Enabling socket-level TCP keep-alives makes the system regularly send packets
+ to the other end of the connection, leaving it active. The delay between
+ keep-alive probes is controlled by the system only and depends both on the
+ operating system and its tuning parameters.
+
+ It is important to understand that keep-alive packets are neither emitted nor
+ received at the application level. It is only the network stacks which sees
+ them. For this reason, even if one side of the proxy already uses keep-alives
+ to maintain its connection alive, those keep-alive packets will not be
+ forwarded to the other side of the proxy.
+
+ Please note that this has nothing to do with HTTP keep-alive.
+
+ Using option "srvtcpka" enables the emission of TCP keep-alive probes on the
+ server side of a connection, which should help when session expirations are
+ noticed between HAProxy and a server.
+
+ If this option has been enabled in a "defaults" section, it can be disabled
+ in a specific instance by prepending the "no" keyword before it.
+
+ See also : "option clitcpka", "option tcpka"
+
+
+option ssl-hello-chk
+ Use SSLv3 client hello health checks for server testing
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments : none
+
+ When some SSL-based protocols are relayed in TCP mode through HAProxy, it is
+ possible to test that the server correctly talks SSL instead of just testing
+ that it accepts the TCP connection. When "option ssl-hello-chk" is set, pure
+ SSLv3 client hello messages are sent once the connection is established to
+ the server, and the response is analyzed to find an SSL server hello message.
+ The server is considered valid only when the response contains this server
+ hello message.
+
+ All servers tested till there correctly reply to SSLv3 client hello messages,
+ and most servers tested do not even log the requests containing only hello
+ messages, which is appreciable.
+
+ Note that this check works even when SSL support was not built into HAProxy
+ because it forges the SSL message. When SSL support is available, it is best
+ to use native SSL health checks instead of this one.
+
+ See also: "option httpchk", "check-ssl"
+
+
+option tcp-check
+ Perform health checks using tcp-check send/expect sequences
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ This health check method is intended to be combined with "tcp-check" command
+ lists in order to support send/expect types of health check sequences.
+
+ TCP checks currently support 4 modes of operations :
+ - no "tcp-check" directive : the health check only consists in a connection
+ attempt, which remains the default mode.
+
+ - "tcp-check send" or "tcp-check send-binary" only is mentioned : this is
+ used to send a string along with a connection opening. With some
+ protocols, it helps sending a "QUIT" message for example that prevents
+ the server from logging a connection error for each health check. The
+ check result will still be based on the ability to open the connection
+ only.
+
+ - "tcp-check expect" only is mentioned : this is used to test a banner.
+ The connection is opened and HAProxy waits for the server to present some
+ contents which must validate some rules. The check result will be based
+ on the matching between the contents and the rules. This is suited for
+ POP, IMAP, SMTP, FTP, SSH, TELNET.
+
+ - both "tcp-check send" and "tcp-check expect" are mentioned : this is
+ used to test a hello-type protocol. HAProxy sends a message, the server
+ responds and its response is analyzed. the check result will be based on
+ the matching between the response contents and the rules. This is often
+ suited for protocols which require a binding or a request/response model.
+ LDAP, MySQL, Redis and SSL are example of such protocols, though they
+ already all have their dedicated checks with a deeper understanding of
+ the respective protocols.
+ In this mode, many questions may be sent and many answers may be
+ analyzed.
+
+ A fifth mode can be used to insert comments in different steps of the script.
+
+ For each tcp-check rule you create, you can add a "comment" directive,
+ followed by a string. This string will be reported in the log and stderr in
+ debug mode. It is useful to make user-friendly error reporting. The
+ "comment" is of course optional.
+
+ During the execution of a health check, a variable scope is made available to
+ store data samples, using the "tcp-check set-var" operation. Freeing those
+ variable is possible using "tcp-check unset-var".
+
+
+ Examples :
+ # perform a POP check (analyze only server's banner)
+ option tcp-check
+ tcp-check expect string +OK\ POP3\ ready comment POP\ protocol
+
+ # perform an IMAP check (analyze only server's banner)
+ option tcp-check
+ tcp-check expect string *\ OK\ IMAP4\ ready comment IMAP\ protocol
+
+ # look for the redis master server after ensuring it speaks well
+ # redis protocol, then it exits properly.
+ # (send a command then analyze the response 3 times)
+ option tcp-check
+ tcp-check comment PING\ phase
+ tcp-check send PING\r\n
+ tcp-check expect string +PONG
+ tcp-check comment role\ check
+ tcp-check send info\ replication\r\n
+ tcp-check expect string role:master
+ tcp-check comment QUIT\ phase
+ tcp-check send QUIT\r\n
+ tcp-check expect string +OK
+
+ forge a HTTP request, then analyze the response
+ (send many headers before analyzing)
+ option tcp-check
+ tcp-check comment forge\ and\ send\ HTTP\ request
+ tcp-check send HEAD\ /\ HTTP/1.1\r\n
+ tcp-check send Host:\ www.mydomain.com\r\n
+ tcp-check send User-Agent:\ HAProxy\ tcpcheck\r\n
+ tcp-check send \r\n
+ tcp-check expect rstring HTTP/1\..\ (2..|3..) comment check\ HTTP\ response
+
+
+ See also : "tcp-check connect", "tcp-check expect" and "tcp-check send".
+
+
+option tcp-smart-accept
+no option tcp-smart-accept
+ Enable or disable the saving of one ACK packet during the accept sequence
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments : none
+
+ When an HTTP connection request comes in, the system acknowledges it on
+ behalf of HAProxy, then the client immediately sends its request, and the
+ system acknowledges it too while it is notifying HAProxy about the new
+ connection. HAProxy then reads the request and responds. This means that we
+ have one TCP ACK sent by the system for nothing, because the request could
+ very well be acknowledged by HAProxy when it sends its response.
+
+ For this reason, in HTTP mode, HAProxy automatically asks the system to avoid
+ sending this useless ACK on platforms which support it (currently at least
+ Linux). It must not cause any problem, because the system will send it anyway
+ after 40 ms if the response takes more time than expected to come.
+
+ During complex network debugging sessions, it may be desirable to disable
+ this optimization because delayed ACKs can make troubleshooting more complex
+ when trying to identify where packets are delayed. It is then possible to
+ fall back to normal behavior by specifying "no option tcp-smart-accept".
+
+ It is also possible to force it for non-HTTP proxies by simply specifying
+ "option tcp-smart-accept". For instance, it can make sense with some services
+ such as SMTP where the server speaks first.
+
+ It is recommended to avoid forcing this option in a defaults section. In case
+ of doubt, consider setting it back to automatic values by prepending the
+ "default" keyword before it, or disabling it using the "no" keyword.
+
+ See also : "option tcp-smart-connect"
+
+
+option tcp-smart-connect
+no option tcp-smart-connect
+ Enable or disable the saving of one ACK packet during the connect sequence
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments : none
+
+ On certain systems (at least Linux), HAProxy can ask the kernel not to
+ immediately send an empty ACK upon a connection request, but to directly
+ send the buffer request instead. This saves one packet on the network and
+ thus boosts performance. It can also be useful for some servers, because they
+ immediately get the request along with the incoming connection.
+
+ This feature is enabled when "option tcp-smart-connect" is set in a backend.
+ It is not enabled by default because it makes network troubleshooting more
+ complex.
+
+ It only makes sense to enable it with protocols where the client speaks first
+ such as HTTP. In other situations, if there is no data to send in place of
+ the ACK, a normal ACK is sent.
+
+ If this option has been enabled in a "defaults" section, it can be disabled
+ in a specific instance by prepending the "no" keyword before it.
+
+ See also : "option tcp-smart-accept"
+
+
+option tcpka
+ Enable or disable the sending of TCP keepalive packets on both sides
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments : none
+
+ When there is a firewall or any session-aware component between a client and
+ a server, and when the protocol involves very long sessions with long idle
+ periods (e.g. remote desktops), there is a risk that one of the intermediate
+ components decides to expire a session which has remained idle for too long.
+
+ Enabling socket-level TCP keep-alives makes the system regularly send packets
+ to the other end of the connection, leaving it active. The delay between
+ keep-alive probes is controlled by the system only and depends both on the
+ operating system and its tuning parameters.
+
+ It is important to understand that keep-alive packets are neither emitted nor
+ received at the application level. It is only the network stacks which sees
+ them. For this reason, even if one side of the proxy already uses keep-alives
+ to maintain its connection alive, those keep-alive packets will not be
+ forwarded to the other side of the proxy.
+
+ Please note that this has nothing to do with HTTP keep-alive.
+
+ Using option "tcpka" enables the emission of TCP keep-alive probes on both
+ the client and server sides of a connection. Note that this is meaningful
+ only in "defaults" or "listen" sections. If this option is used in a
+ frontend, only the client side will get keep-alives, and if this option is
+ used in a backend, only the server side will get keep-alives. For this
+ reason, it is strongly recommended to explicitly use "option clitcpka" and
+ "option srvtcpka" when the configuration is split between frontends and
+ backends.
+
+ See also : "option clitcpka", "option srvtcpka"
+
+
+option tcplog
+ Enable advanced logging of TCP connections with stream state and timers
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments : none
+
+ By default, the log output format is very poor, as it only contains the
+ source and destination addresses, and the instance name. By specifying
+ "option tcplog", each log line turns into a much richer format including, but
+ not limited to, the connection timers, the stream status, the connections
+ numbers, the frontend, backend and server name, and of course the source
+ address and ports. This option is useful for pure TCP proxies in order to
+ find which of the client or server disconnects or times out. For normal HTTP
+ proxies, it's better to use "option httplog" which is even more complete.
+
+ "option tcplog" overrides any previous "log-format" directive.
+
+ See also : "option httplog", and section 8 about logging.
+
+
+option transparent
+no option transparent
+ Enable client-side transparent proxying
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments : none
+
+ This option was introduced in order to provide layer 7 persistence to layer 3
+ load balancers. The idea is to use the OS's ability to redirect an incoming
+ connection for a remote address to a local process (here HAProxy), and let
+ this process know what address was initially requested. When this option is
+ used, sessions without cookies will be forwarded to the original destination
+ IP address of the incoming request (which should match that of another
+ equipment), while requests with cookies will still be forwarded to the
+ appropriate server.
+
+ Note that contrary to a common belief, this option does NOT make HAProxy
+ present the client's IP to the server when establishing the connection.
+
+ See also: the "usesrc" argument of the "source" keyword, and the
+ "transparent" option of the "bind" keyword.
+
+
+external-check command <command>
+ Executable to run when performing an external-check
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <command> is the external command to run
+
+ The arguments passed to the to the command are:
+
+ <proxy_address> <proxy_port> <server_address> <server_port>
+
+ The <proxy_address> and <proxy_port> are derived from the first listener
+ that is either IPv4, IPv6 or a UNIX socket. In the case of a UNIX socket
+ listener the proxy_address will be the path of the socket and the
+ <proxy_port> will be the string "NOT_USED". In a backend section, it's not
+ possible to determine a listener, and both <proxy_address> and <proxy_port>
+ will have the string value "NOT_USED".
+
+ Some values are also provided through environment variables.
+
+ Environment variables :
+ HAPROXY_PROXY_ADDR The first bind address if available (or empty if not
+ applicable, for example in a "backend" section).
+
+ HAPROXY_PROXY_ID The backend id.
+
+ HAPROXY_PROXY_NAME The backend name.
+
+ HAPROXY_PROXY_PORT The first bind port if available (or empty if not
+ applicable, for example in a "backend" section or
+ for a UNIX socket).
+
+ HAPROXY_SERVER_ADDR The server address.
+
+ HAPROXY_SERVER_CURCONN The current number of connections on the server.
+
+ HAPROXY_SERVER_ID The server id.
+
+ HAPROXY_SERVER_MAXCONN The server max connections.
+
+ HAPROXY_SERVER_NAME The server name.
+
+ HAPROXY_SERVER_PORT The server port if available (or empty for a UNIX
+ socket).
+
+ HAPROXY_SERVER_SSL "0" when SSL is not used, "1" when it is used
+
+ HAPROXY_SERVER_PROTO The protocol used by this server, which can be one
+ of "cli" (the haproxy CLI), "syslog" (syslog TCP
+ server), "peers" (peers TCP server), "h1" (HTTP/1.x
+ server), "h2" (HTTP/2 server), or "tcp" (any other
+ TCP server).
+
+ PATH The PATH environment variable used when executing
+ the command may be set using "external-check path".
+
+ See also "2.3. Environment variables" for other variables.
+
+ If the command executed and exits with a zero status then the check is
+ considered to have passed, otherwise the check is considered to have
+ failed.
+
+ Example :
+ external-check command /bin/true
+
+ See also : "external-check", "option external-check", "external-check path"
+
+
+external-check path <path>
+ The value of the PATH environment variable used when running an external-check
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <path> is the path used when executing external command to run
+
+ The default path is "".
+
+ Example :
+ external-check path "/usr/bin:/bin"
+
+ See also : "external-check", "option external-check",
+ "external-check command"
+
+
+persist rdp-cookie
+persist rdp-cookie(<name>)
+ Enable RDP cookie-based persistence
+
+ May be used in the following contexts: tcp
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <name> is the optional name of the RDP cookie to check. If omitted, the
+ default cookie name "msts" will be used. There currently is no
+ valid reason to change this name.
+
+ This statement enables persistence based on an RDP cookie. The RDP cookie
+ contains all information required to find the server in the list of known
+ servers. So when this option is set in the backend, the request is analyzed
+ and if an RDP cookie is found, it is decoded. If it matches a known server
+ which is still UP (or if "option persist" is set), then the connection is
+ forwarded to this server.
+
+ Note that this only makes sense in a TCP backend, but for this to work, the
+ frontend must have waited long enough to ensure that an RDP cookie is present
+ in the request buffer. This is the same requirement as with the "rdp-cookie"
+ load-balancing method. Thus it is highly recommended to put all statements in
+ a single "listen" section.
+
+ Also, it is important to understand that the terminal server will emit this
+ RDP cookie only if it is configured for "token redirection mode", which means
+ that the "IP address redirection" option is disabled.
+
+ Example :
+ listen tse-farm
+ bind :3389
+ # wait up to 5s for an RDP cookie in the request
+ tcp-request inspect-delay 5s
+ tcp-request content accept if RDP_COOKIE
+ # apply RDP cookie persistence
+ persist rdp-cookie
+ # if server is unknown, let's balance on the same cookie.
+ # alternatively, "balance leastconn" may be useful too.
+ balance rdp-cookie
+ server srv1 1.1.1.1:3389
+ server srv2 1.1.1.2:3389
+
+ See also : "balance rdp-cookie", "tcp-request" and the "req.rdp_cookie" ACL.
+
+
+rate-limit sessions <rate>
+ Set a limit on the number of new sessions accepted per second on a frontend
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments :
+ <rate> The <rate> parameter is an integer designating the maximum number
+ of new sessions per second to accept on the frontend.
+
+ When the frontend reaches the specified number of new sessions per second, it
+ stops accepting new connections until the rate drops below the limit again.
+ During this time, the pending sessions will be kept in the socket's backlog
+ (in system buffers) and HAProxy will not even be aware that sessions are
+ pending. When applying very low limit on a highly loaded service, it may make
+ sense to increase the socket's backlog using the "backlog" keyword.
+
+ This feature is particularly efficient at blocking connection-based attacks
+ or service abuse on fragile servers. Since the session rate is measured every
+ millisecond, it is extremely accurate. Also, the limit applies immediately,
+ no delay is needed at all to detect the threshold.
+
+ Example : limit the connection rate on SMTP to 10 per second max
+ listen smtp
+ mode tcp
+ bind :25
+ rate-limit sessions 10
+ server smtp1 127.0.0.1:1025
+
+ Note : when the maximum rate is reached, the frontend's status is not changed
+ but its sockets appear as "WAITING" in the statistics if the
+ "socket-stats" option is enabled.
+
+ See also : the "backlog" keyword and the "fe_sess_rate" ACL criterion.
+
+
+redirect location <loc> [code <code>] <option> [{if | unless} <condition>]
+redirect prefix <pfx> [code <code>] <option> [{if | unless} <condition>]
+redirect scheme <sch> [code <code>] <option> [{if | unless} <condition>]
+ Return an HTTP redirection if/unless a condition is matched
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ no | yes | yes | yes
+
+ If/unless the condition is matched, the HTTP request will lead to a redirect
+ response. If no condition is specified, the redirect applies unconditionally.
+
+ Arguments :
+ <loc> With "redirect location", the exact value in <loc> is placed into
+ the HTTP "Location" header. When used in an "http-request" rule,
+ <loc> value follows the log-format rules and can include some
+ dynamic values (see Custom Log Format in section 8.2.4).
+
+ <pfx> With "redirect prefix", the "Location" header is built from the
+ concatenation of <pfx> and the complete URI path, including the
+ query string, unless the "drop-query" option is specified (see
+ below). As a special case, if <pfx> equals exactly "/", then
+ nothing is inserted before the original URI. It allows one to
+ redirect to the same URL (for instance, to insert a cookie). When
+ used in an "http-request" rule, <pfx> value follows the log-format
+ rules and can include some dynamic values (see Custom Log Format
+ in section 8.2.4).
+
+ <sch> With "redirect scheme", then the "Location" header is built by
+ concatenating <sch> with "://" then the first occurrence of the
+ "Host" header, and then the URI path, including the query string
+ unless the "drop-query" option is specified (see below). If no
+ path is found or if the path is "*", then "/" is used instead. If
+ no "Host" header is found, then an empty host component will be
+ returned, which most recent browsers interpret as redirecting to
+ the same host. This directive is mostly used to redirect HTTP to
+ HTTPS. When used in an "http-request" rule, <sch> value follows
+ the log-format rules and can include some dynamic values (see
+ Custom Log Format in section 8.2.4).
+
+ <code> The code is optional. It indicates which type of HTTP redirection
+ is desired. Only codes 301, 302, 303, 307 and 308 are supported,
+ with 302 used by default if no code is specified. 301 means
+ "Moved permanently", and a browser may cache the Location. 302
+ means "Moved temporarily" and means that the browser should not
+ cache the redirection. 303 is equivalent to 302 except that the
+ browser will fetch the location with a GET method. 307 is just
+ like 302 but makes it clear that the same method must be reused.
+ Likewise, 308 replaces 301 if the same method must be used.
+
+ <option> There are several options which can be specified to adjust the
+ expected behavior of a redirection :
+
+ - "drop-query"
+ When this keyword is used in a prefix-based redirection, then the
+ location will be set without any possible query-string, which is useful
+ for directing users to a non-secure page for instance. It has no effect
+ with a location-type redirect.
+
+ - "append-slash"
+ This keyword may be used in conjunction with "drop-query" to redirect
+ users who use a URL not ending with a '/' to the same one with the '/'.
+ It can be useful to ensure that search engines will only see one URL.
+ For this, a return code 301 is preferred.
+
+ - "ignore-empty"
+ This keyword only has effect when a location is produced using a log
+ format expression (i.e. when used in http-request or http-response).
+ It indicates that if the result of the expression is empty, the rule
+ should silently be skipped. The main use is to allow mass-redirects
+ of known paths using a simple map.
+
+ - "set-cookie NAME[=value]"
+ A "Set-Cookie" header will be added with NAME (and optionally "=value")
+ to the response. This is sometimes used to indicate that a user has
+ been seen, for instance to protect against some types of DoS. No other
+ cookie option is added, so the cookie will be a session cookie. Note
+ that for a browser, a sole cookie name without an equal sign is
+ different from a cookie with an equal sign.
+
+ - "clear-cookie NAME[=]"
+ A "Set-Cookie" header will be added with NAME (and optionally "="), but
+ with the "Max-Age" attribute set to zero. This will tell the browser to
+ delete this cookie. It is useful for instance on logout pages. It is
+ important to note that clearing the cookie "NAME" will not remove a
+ cookie set with "NAME=value". You have to clear the cookie "NAME=" for
+ that, because the browser makes the difference.
+
+ Example: move the login URL only to HTTPS.
+ acl clear dst_port 80
+ acl secure dst_port 8080
+ acl login_page url_beg /login
+ acl logout url_beg /logout
+ acl uid_given url_reg /login?userid=[^&]+
+ acl cookie_set hdr_sub(cookie) SEEN=1
+
+ redirect prefix https://mysite.com set-cookie SEEN=1 if !cookie_set
+ redirect prefix https://mysite.com if login_page !secure
+ redirect prefix http://mysite.com drop-query if login_page !uid_given
+ redirect location http://mysite.com/ if !login_page secure
+ redirect location / clear-cookie USERID= if logout
+
+ Example: send redirects for request for articles without a '/'.
+ acl missing_slash path_reg ^/article/[^/]*$
+ redirect code 301 prefix / drop-query append-slash if missing_slash
+
+ Example: redirect all HTTP traffic to HTTPS when SSL is handled by HAProxy.
+ redirect scheme https if !{ ssl_fc }
+
+ Example: append 'www.' prefix in front of all hosts not having it
+ http-request redirect code 301 location \
+ http://www.%[hdr(host)]%[capture.req.uri] \
+ unless { hdr_beg(host) -i www }
+
+ Example: permanently redirect only old URLs to new ones
+ http-request redirect code 301 location \
+ %[path,map_str(old-blog-articles.map)] ignore-empty
+
+ See section 7 about ACL usage.
+
+
+retries <value>
+ Set the number of retries to perform on a server after a failure
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <value> is the number of times a request or connection attempt should be
+ retried on a server after a failure.
+
+ By default, retries apply only to new connection attempts. However, when
+ the "retry-on" directive is used, other conditions might trigger a retry
+ (e.g. empty response, undesired status code), and each of them will count
+ one attempt, and when the total number attempts reaches the value here, an
+ error will be returned.
+
+ In order to avoid immediate reconnections to a server which is restarting,
+ a turn-around timer of min("timeout connect", one second) is applied before
+ a retry occurs on the same server.
+
+ When "option redispatch" is set, some retries may be performed on another
+ server even if a cookie references a different server. By default this will
+ only be the last retry unless an argument is passed to "option redispatch".
+
+ See also : "option redispatch"
+
+
+retry-on [space-delimited list of keywords]
+ Specify when to attempt to automatically retry a failed request.
+ This setting is only valid when "mode" is set to http and is silently ignored
+ otherwise.
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <keywords> is a space-delimited list of keywords or HTTP status codes, each
+ representing a type of failure event on which an attempt to
+ retry the request is desired. Please read the notes at the
+ bottom before changing this setting. The following keywords are
+ supported :
+
+ none never retry
+
+ conn-failure retry when the connection or the SSL handshake failed
+ and the request could not be sent. This is the default.
+
+ empty-response retry when the server connection was closed after part
+ of the request was sent, and nothing was received from
+ the server. This type of failure may be caused by the
+ request timeout on the server side, poor network
+ condition, or a server crash or restart while
+ processing the request.
+
+ junk-response retry when the server returned something not looking
+ like a complete HTTP response. This includes partial
+ responses headers as well as non-HTTP contents. It
+ usually is a bad idea to retry on such events, which
+ may be caused a configuration issue (wrong server port)
+ or by the request being harmful to the server (buffer
+ overflow attack for example).
+
+ response-timeout the server timeout stroke while waiting for the server
+ to respond to the request. This may be caused by poor
+ network condition, the reuse of an idle connection
+ which has expired on the path, or by the request being
+ extremely expensive to process. It generally is a bad
+ idea to retry on such events on servers dealing with
+ heavy database processing (full scans, etc) as it may
+ amplify denial of service attacks.
+
+ 0rtt-rejected retry requests which were sent over early data and were
+ rejected by the server. These requests are generally
+ considered to be safe to retry.
+
+ <status> any HTTP status code among "401" (Unauthorized), "403"
+ (Forbidden), "404" (Not Found), "408" (Request Timeout),
+ "425" (Too Early), "500" (Server Error), "501" (Not
+ Implemented), "502" (Bad Gateway), "503" (Service
+ Unavailable), "504" (Gateway Timeout).
+
+ all-retryable-errors
+ retry request for any error that are considered
+ retryable. This currently activates "conn-failure",
+ "empty-response", "junk-response", "response-timeout",
+ "0rtt-rejected", "500", "502", "503", and "504".
+
+ Using this directive replaces any previous settings with the new ones; it is
+ not cumulative.
+
+ Please note that using anything other than "none" and "conn-failure" requires
+ to allocate a buffer and copy the whole request into it, so it has memory and
+ performance impacts. Requests not fitting in a single buffer will never be
+ retried (see the global tune.bufsize setting).
+
+ You have to make sure the application has a replay protection mechanism built
+ in such as a unique transaction IDs passed in requests, or that replaying the
+ same request has no consequence, or it is very dangerous to use any retry-on
+ value beside "conn-failure" and "none". Static file servers and caches are
+ generally considered safe against any type of retry. Using a status code can
+ be useful to quickly leave a server showing an abnormal behavior (out of
+ memory, file system issues, etc), but in this case it may be a good idea to
+ immediately redispatch the connection to another server (please see "option
+ redispatch" for this). Last, it is important to understand that most causes
+ of failures are the requests themselves and that retrying a request causing a
+ server to misbehave will often make the situation even worse for this server,
+ or for the whole service in case of redispatch.
+
+ Unless you know exactly how the application deals with replayed requests, you
+ should not use this directive.
+
+ The default is "conn-failure".
+
+ Example:
+ retry-on 503 504
+
+ See also: "retries", "option redispatch", "tune.bufsize"
+
+server <name> <address>[:[port]] [param*]
+ Declare a server in a backend
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections : defaults | frontend | listen | backend
+ no | no | yes | yes
+
+ Arguments :
+ <name> is the internal name assigned to this server. This name will
+ appear in logs and alerts. If "http-send-name-header" is
+ set, it will be added to the request header sent to the server.
+
+ <address> is the IPv4 or IPv6 address of the server. Alternatively, a
+ resolvable hostname is supported, but this name will be resolved
+ during start-up. Address "0.0.0.0" or "*" has a special meaning.
+ It indicates that the connection will be forwarded to the same IP
+ address as the one from the client connection. This is useful in
+ transparent proxy architectures where the client's connection is
+ intercepted and HAProxy must forward to the original destination
+ address. This is more or less what the "transparent" keyword does
+ except that with a server it's possible to limit concurrency and
+ to report statistics. Optionally, an address family prefix may be
+ used before the address to force the family regardless of the
+ address format, which can be useful to specify a path to a unix
+ socket with no slash ('/'). Currently supported prefixes are :
+ - 'ipv4@' -> address is always IPv4
+ - 'ipv6@' -> address is always IPv6
+ - 'unix@' -> address is a path to a local unix socket
+ - 'abns@' -> address is in abstract namespace (Linux only)
+ - 'sockpair@' -> address is the FD of a connected unix
+ socket or of a socketpair. During a connection, the
+ backend creates a pair of connected sockets, and passes
+ one of them over the FD. The bind part will use the
+ received socket as the client FD. Should be used
+ carefully.
+ - 'rhttp@' [ EXPERIMENTAL ] -> custom address family for a
+ passive server in HTTP reverse context. This is an
+ experimental features which requires
+ "expose-experimental-directives" on a line before this
+ server.
+ You may want to reference some environment variables in the
+ address parameter, see section 2.3 about environment
+ variables. The "init-addr" setting can be used to modify the way
+ IP addresses should be resolved upon startup.
+
+ <port> is an optional port specification. If set, all connections will
+ be sent to this port. If unset, the same port the client
+ connected to will be used. The port may also be prefixed by a "+"
+ or a "-". In this case, the server's port will be determined by
+ adding this value to the client's port.
+
+ <param*> is a list of parameters for this server. The "server" keywords
+ accepts an important number of options and has a complete section
+ dedicated to it. Please refer to section 5 for more details.
+
+ Examples :
+ server first 10.1.1.1:1080 cookie first check inter 1000
+ server second 10.1.1.2:1080 cookie second check inter 1000
+ server transp ipv4@
+ server backup "${SRV_BACKUP}:1080" backup
+ server www1_dc1 "${LAN_DC1}.101:80"
+ server www1_dc2 "${LAN_DC2}.101:80"
+
+ Note: regarding Linux's abstract namespace sockets, HAProxy uses the whole
+ sun_path length is used for the address length. Some other programs
+ such as socat use the string length only by default. Pass the option
+ ",unix-tightsocklen=0" to any abstract socket definition in socat to
+ make it compatible with HAProxy's.
+
+ See also: "default-server", "http-send-name-header" and section 5 about
+ server options
+
+server-state-file-name [ { use-backend-name | <file> } ]
+ Set the server state file to read, load and apply to servers available in
+ this backend.
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections: defaults | frontend | listen | backend
+ no | no | yes | yes
+
+ It only applies when the directive "load-server-state-from-file" is set to
+ "local". When <file> is not provided, if "use-backend-name" is used or if
+ this directive is not set, then backend name is used. If <file> starts with a
+ slash '/', then it is considered as an absolute path. Otherwise, <file> is
+ concatenated to the global directive "server-state-base".
+
+ Example: the minimal configuration below would make HAProxy look for the
+ state server file '/etc/haproxy/states/bk':
+
+ global
+ server-state-file-base /etc/haproxy/states
+
+ backend bk
+ load-server-state-from-file
+
+ See also: "server-state-base", "load-server-state-from-file", and
+ "show servers state"
+
+server-template <prefix> <num | range> <fqdn>[:<port>] [params*]
+ Set a template to initialize servers with shared parameters.
+ The names of these servers are built from <prefix> and <num | range> parameters.
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections : defaults | frontend | listen | backend
+ no | no | yes | yes
+
+ Arguments:
+ <prefix> A prefix for the server names to be built.
+
+ <num | range>
+ If <num> is provided, this template initializes <num> servers
+ with 1 up to <num> as server name suffixes. A range of numbers
+ <num_low>-<num_high> may also be used to use <num_low> up to
+ <num_high> as server name suffixes.
+
+ <fqdn> A FQDN for all the servers this template initializes.
+
+ <port> Same meaning as "server" <port> argument (see "server" keyword).
+
+ <params*>
+ Remaining server parameters among all those supported by "server"
+ keyword.
+
+ Examples:
+ # Initializes 3 servers with srv1, srv2 and srv3 as names,
+ # google.com as FQDN, and health-check enabled.
+ server-template srv 1-3 google.com:80 check
+
+ # or
+ server-template srv 3 google.com:80 check
+
+ # would be equivalent to:
+ server srv1 google.com:80 check
+ server srv2 google.com:80 check
+ server srv3 google.com:80 check
+
+
+
+source <addr>[:<port>] [usesrc { <addr2>[:<port2>] | client | clientip } ]
+source <addr>[:<port>] [usesrc { <addr2>[:<port2>] | hdr_ip(<hdr>[,<occ>]) } ]
+source <addr>[:<port>] [interface <name>]
+ Set the source address for outgoing connections
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <addr> is the IPv4 address HAProxy will bind to before connecting to a
+ server. This address is also used as a source for health checks.
+
+ The default value of 0.0.0.0 means that the system will select
+ the most appropriate address to reach its destination. Optionally
+ an address family prefix may be used before the address to force
+ the family regardless of the address format, which can be useful
+ to specify a path to a unix socket with no slash ('/'). Currently
+ supported prefixes are :
+ - 'ipv4@' -> address is always IPv4
+ - 'ipv6@' -> address is always IPv6
+ - 'unix@' -> address is a path to a local unix socket
+ - 'abns@' -> address is in abstract namespace (Linux only)
+ You may want to reference some environment variables in the
+ address parameter, see section 2.3 about environment variables.
+
+ <port> is an optional port. It is normally not needed but may be useful
+ in some very specific contexts. The default value of zero means
+ the system will select a free port. Note that port ranges are not
+ supported in the backend. If you want to force port ranges, you
+ have to specify them on each "server" line.
+
+ <addr2> is the IP address to present to the server when connections are
+ forwarded in full transparent proxy mode. This is currently only
+ supported on some patched Linux kernels. When this address is
+ specified, clients connecting to the server will be presented
+ with this address, while health checks will still use the address
+ <addr>.
+
+ <port2> is the optional port to present to the server when connections
+ are forwarded in full transparent proxy mode (see <addr2> above).
+ The default value of zero means the system will select a free
+ port.
+
+ <hdr> is the name of a HTTP header in which to fetch the IP to bind to.
+ This is the name of a comma-separated header list which can
+ contain multiple IP addresses. By default, the last occurrence is
+ used. This is designed to work with the X-Forwarded-For header
+ and to automatically bind to the client's IP address as seen
+ by previous proxy, typically Stunnel. In order to use another
+ occurrence from the last one, please see the <occ> parameter
+ below. When the header (or occurrence) is not found, no binding
+ is performed so that the proxy's default IP address is used. Also
+ keep in mind that the header name is case insensitive, as for any
+ HTTP header.
+
+ <occ> is the occurrence number of a value to be used in a multi-value
+ header. This is to be used in conjunction with "hdr_ip(<hdr>)",
+ in order to specify which occurrence to use for the source IP
+ address. Positive values indicate a position from the first
+ occurrence, 1 being the first one. Negative values indicate
+ positions relative to the last one, -1 being the last one. This
+ is helpful for situations where an X-Forwarded-For header is set
+ at the entry point of an infrastructure and must be used several
+ proxy layers away. When this value is not specified, -1 is
+ assumed. Passing a zero here disables the feature.
+
+ <name> is an optional interface name to which to bind to for outgoing
+ traffic. On systems supporting this features (currently, only
+ Linux), this allows one to bind all traffic to the server to
+ this interface even if it is not the one the system would select
+ based on routing tables. This should be used with extreme care.
+ Note that using this option requires root privileges.
+
+ The "source" keyword is useful in complex environments where a specific
+ address only is allowed to connect to the servers. It may be needed when a
+ private address must be used through a public gateway for instance, and it is
+ known that the system cannot determine the adequate source address by itself.
+
+ An extension which is available on certain patched Linux kernels may be used
+ through the "usesrc" optional keyword. It makes it possible to connect to the
+ servers with an IP address which does not belong to the system itself. This
+ is called "full transparent proxy mode". For this to work, the destination
+ servers have to route their traffic back to this address through the machine
+ running HAProxy, and IP forwarding must generally be enabled on this machine.
+
+ In this "full transparent proxy" mode, it is possible to force a specific IP
+ address to be presented to the servers. This is not much used in fact. A more
+ common use is to tell HAProxy to present the client's IP address. For this,
+ there are two methods :
+
+ - present the client's IP and port addresses. This is the most transparent
+ mode, but it can cause problems when IP connection tracking is enabled on
+ the machine, because a same connection may be seen twice with different
+ states. However, this solution presents the huge advantage of not
+ limiting the system to the 64k outgoing address+port couples, because all
+ of the client ranges may be used.
+
+ - present only the client's IP address and select a spare port. This
+ solution is still quite elegant but slightly less transparent (downstream
+ firewalls logs will not match upstream's). It also presents the downside
+ of limiting the number of concurrent connections to the usual 64k ports.
+ However, since the upstream and downstream ports are different, local IP
+ connection tracking on the machine will not be upset by the reuse of the
+ same session.
+
+ This option sets the default source for all servers in the backend. It may
+ also be specified in a "defaults" section. Finer source address specification
+ is possible at the server level using the "source" server option. Refer to
+ section 5 for more information.
+
+ In order to work, "usesrc" requires root privileges, or on supported systems,
+ the "cap_net_raw" capability. See also the "setcap" global directive.
+
+ Examples :
+ backend private
+ # Connect to the servers using our 192.168.1.200 source address
+ source 192.168.1.200
+
+ backend transparent_ssl1
+ # Connect to the SSL farm from the client's source address
+ source 192.168.1.200 usesrc clientip
+
+ backend transparent_ssl2
+ # Connect to the SSL farm from the client's source address and port
+ # not recommended if IP conntrack is present on the local machine.
+ source 192.168.1.200 usesrc client
+
+ backend transparent_ssl3
+ # Connect to the SSL farm from the client's source address. It
+ # is more conntrack-friendly.
+ source 192.168.1.200 usesrc clientip
+
+ backend transparent_smtp
+ # Connect to the SMTP farm from the client's source address/port
+ # with Tproxy version 4.
+ source 0.0.0.0 usesrc clientip
+
+ backend transparent_http
+ # Connect to the servers using the client's IP as seen by previous
+ # proxy.
+ source 0.0.0.0 usesrc hdr_ip(x-forwarded-for,-1)
+
+ See also : the "source" server option in section 5, the Tproxy patches for
+ the Linux kernel on www.balabit.com, the "bind" keyword.
+
+
+srvtcpka-cnt <count>
+ Sets the maximum number of keepalive probes TCP should send before dropping
+ the connection on the server side.
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <count> is the maximum number of keepalive probes.
+
+ This keyword corresponds to the socket option TCP_KEEPCNT. If this keyword
+ is not specified, system-wide TCP parameter (tcp_keepalive_probes) is used.
+ The availability of this setting depends on the operating system. It is
+ known to work on Linux.
+
+ See also : "option srvtcpka", "srvtcpka-idle", "srvtcpka-intvl".
+
+
+srvtcpka-idle <timeout>
+ Sets the time the connection needs to remain idle before TCP starts sending
+ keepalive probes, if enabled the sending of TCP keepalive packets on the
+ server side.
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <timeout> is the time the connection needs to remain idle before TCP starts
+ sending keepalive probes. It is specified in seconds by default,
+ but can be in any other unit if the number is suffixed by the
+ unit, as explained at the top of this document.
+
+ This keyword corresponds to the socket option TCP_KEEPIDLE. If this keyword
+ is not specified, system-wide TCP parameter (tcp_keepalive_time) is used.
+ The availability of this setting depends on the operating system. It is
+ known to work on Linux.
+
+ See also : "option srvtcpka", "srvtcpka-cnt", "srvtcpka-intvl".
+
+
+srvtcpka-intvl <timeout>
+ Sets the time between individual keepalive probes on the server side.
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <timeout> is the time between individual keepalive probes. It is specified
+ in seconds by default, but can be in any other unit if the number
+ is suffixed by the unit, as explained at the top of this
+ document.
+
+ This keyword corresponds to the socket option TCP_KEEPINTVL. If this keyword
+ is not specified, system-wide TCP parameter (tcp_keepalive_intvl) is used.
+ The availability of this setting depends on the operating system. It is
+ known to work on Linux.
+
+ See also : "option srvtcpka", "srvtcpka-cnt", "srvtcpka-idle".
+
+
+stats admin { if | unless } <cond>
+ Enable statistics admin level if/unless a condition is matched
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ no | yes | yes | yes
+
+ This statement enables the statistics admin level if/unless a condition is
+ matched.
+
+ The admin level allows to enable/disable servers from the web interface. By
+ default, statistics page is read-only for security reasons.
+
+ Currently, the POST request is limited to the buffer size minus the reserved
+ buffer space, which means that if the list of servers is too long, the
+ request won't be processed. It is recommended to alter few servers at a
+ time.
+
+ Example :
+ # statistics admin level only for localhost
+ backend stats_localhost
+ stats enable
+ stats admin if LOCALHOST
+
+ Example :
+ # statistics admin level always enabled because of the authentication
+ backend stats_auth
+ stats enable
+ stats auth admin:AdMiN123
+ stats admin if TRUE
+
+ Example :
+ # statistics admin level depends on the authenticated user
+ userlist stats-auth
+ group admin users admin
+ user admin insecure-password AdMiN123
+ group readonly users haproxy
+ user haproxy insecure-password haproxy
+
+ backend stats_auth
+ stats enable
+ acl AUTH http_auth(stats-auth)
+ acl AUTH_ADMIN http_auth_group(stats-auth) admin
+ stats http-request auth unless AUTH
+ stats admin if AUTH_ADMIN
+
+ See also : "stats enable", "stats auth", "stats http-request", section 3.4
+ about userlists and section 7 about ACL usage.
+
+
+stats auth <user>:<passwd>
+ Enable statistics with authentication and grant access to an account
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments :
+ <user> is a user name to grant access to
+
+ <passwd> is the cleartext password associated to this user
+
+ This statement enables statistics with default settings, and restricts access
+ to declared users only. It may be repeated as many times as necessary to
+ allow as many users as desired. When a user tries to access the statistics
+ without a valid account, a "401 Forbidden" response will be returned so that
+ the browser asks the user to provide a valid user and password. The real
+ which will be returned to the browser is configurable using "stats realm".
+
+ Since the authentication method is HTTP Basic Authentication, the passwords
+ circulate in cleartext on the network. Thus, it was decided that the
+ configuration file would also use cleartext passwords to remind the users
+ that those ones should not be sensitive and not shared with any other account.
+
+ It is also possible to reduce the scope of the proxies which appear in the
+ report using "stats scope".
+
+ Though this statement alone is enough to enable statistics reporting, it is
+ recommended to set all other settings in order to avoid relying on default
+ unobvious parameters.
+
+ Example :
+ # public access (limited to this backend only)
+ backend public_www
+ server srv1 192.168.0.1:80
+ stats enable
+ stats hide-version
+ stats scope .
+ stats uri /admin?stats
+ stats realm HAProxy\ Statistics
+ stats auth admin1:AdMiN123
+ stats auth admin2:AdMiN321
+
+ # internal monitoring access (unlimited)
+ backend private_monitoring
+ stats enable
+ stats uri /admin?stats
+ stats refresh 5s
+
+ See also : "stats enable", "stats realm", "stats scope", "stats uri"
+
+
+stats enable
+ Enable statistics reporting with default settings
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments : none
+
+ This statement enables statistics reporting with default settings defined
+ at build time. Unless stated otherwise, these settings are used :
+ - stats uri : /haproxy?stats
+ - stats realm : "HAProxy Statistics"
+ - stats auth : no authentication
+ - stats scope : no restriction
+
+ Though this statement alone is enough to enable statistics reporting, it is
+ recommended to set all other settings in order to avoid relying on default
+ unobvious parameters.
+
+ Example :
+ # public access (limited to this backend only)
+ backend public_www
+ server srv1 192.168.0.1:80
+ stats enable
+ stats hide-version
+ stats scope .
+ stats uri /admin?stats
+ stats realm HAProxy\ Statistics
+ stats auth admin1:AdMiN123
+ stats auth admin2:AdMiN321
+
+ # internal monitoring access (unlimited)
+ backend private_monitoring
+ stats enable
+ stats uri /admin?stats
+ stats refresh 5s
+
+ See also : "stats auth", "stats realm", "stats uri"
+
+
+stats hide-version
+ Enable statistics and hide HAProxy version reporting
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments : none
+
+ By default, the stats page reports some useful status information along with
+ the statistics. Among them is HAProxy's version. However, it is generally
+ considered dangerous to report precise version to anyone, as it can help them
+ target known weaknesses with specific attacks. The "stats hide-version"
+ statement removes the version from the statistics report. This is recommended
+ for public sites or any site with a weak login/password.
+
+ Though this statement alone is enough to enable statistics reporting, it is
+ recommended to set all other settings in order to avoid relying on default
+ unobvious parameters.
+
+ Example :
+ # public access (limited to this backend only)
+ backend public_www
+ server srv1 192.168.0.1:80
+ stats enable
+ stats hide-version
+ stats scope .
+ stats uri /admin?stats
+ stats realm HAProxy\ Statistics
+ stats auth admin1:AdMiN123
+ stats auth admin2:AdMiN321
+
+ # internal monitoring access (unlimited)
+ backend private_monitoring
+ stats enable
+ stats uri /admin?stats
+ stats refresh 5s
+
+ See also : "stats auth", "stats enable", "stats realm", "stats uri"
+
+
+stats http-request { allow | deny | auth [realm <realm>] }
+ [ { if | unless } <condition> ]
+ Access control for statistics
+
+ May be used in the following contexts: http
+
+ May be used in sections: defaults | frontend | listen | backend
+ no | no | yes | yes
+
+ As "http-request", these set of options allow to fine control access to
+ statistics. Each option may be followed by if/unless and acl.
+ First option with matched condition (or option without condition) is final.
+ For "deny" a 403 error will be returned, for "allow" normal processing is
+ performed, for "auth" a 401/407 error code is returned so the client
+ should be asked to enter a username and password.
+
+ There is no fixed limit to the number of http-request statements per
+ instance.
+
+ See also : "http-request", section 3.4 about userlists and section 7
+ about ACL usage.
+
+
+stats realm <realm>
+ Enable statistics and set authentication realm
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments :
+ <realm> is the name of the HTTP Basic Authentication realm reported to
+ the browser. The browser uses it to display it in the pop-up
+ inviting the user to enter a valid username and password.
+
+ The realm is read as a single word, so any spaces in it should be escaped
+ using a backslash ('\').
+
+ This statement is useful only in conjunction with "stats auth" since it is
+ only related to authentication.
+
+ Though this statement alone is enough to enable statistics reporting, it is
+ recommended to set all other settings in order to avoid relying on default
+ unobvious parameters.
+
+ Example :
+ # public access (limited to this backend only)
+ backend public_www
+ server srv1 192.168.0.1:80
+ stats enable
+ stats hide-version
+ stats scope .
+ stats uri /admin?stats
+ stats realm HAProxy\ Statistics
+ stats auth admin1:AdMiN123
+ stats auth admin2:AdMiN321
+
+ # internal monitoring access (unlimited)
+ backend private_monitoring
+ stats enable
+ stats uri /admin?stats
+ stats refresh 5s
+
+ See also : "stats auth", "stats enable", "stats uri"
+
+
+stats refresh <delay>
+ Enable statistics with automatic refresh
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments :
+ <delay> is the suggested refresh delay, specified in seconds, which will
+ be returned to the browser consulting the report page. While the
+ browser is free to apply any delay, it will generally respect it
+ and refresh the page this every seconds. The refresh interval may
+ be specified in any other non-default time unit, by suffixing the
+ unit after the value, as explained at the top of this document.
+
+ This statement is useful on monitoring displays with a permanent page
+ reporting the load balancer's activity. When set, the HTML report page will
+ include a link "refresh"/"stop refresh" so that the user can select whether
+ they want automatic refresh of the page or not.
+
+ Though this statement alone is enough to enable statistics reporting, it is
+ recommended to set all other settings in order to avoid relying on default
+ unobvious parameters.
+
+ Example :
+ # public access (limited to this backend only)
+ backend public_www
+ server srv1 192.168.0.1:80
+ stats enable
+ stats hide-version
+ stats scope .
+ stats uri /admin?stats
+ stats realm HAProxy\ Statistics
+ stats auth admin1:AdMiN123
+ stats auth admin2:AdMiN321
+
+ # internal monitoring access (unlimited)
+ backend private_monitoring
+ stats enable
+ stats uri /admin?stats
+ stats refresh 5s
+
+ See also : "stats auth", "stats enable", "stats realm", "stats uri"
+
+
+stats scope { <name> | "." }
+ Enable statistics and limit access scope
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments :
+ <name> is the name of a listen, frontend or backend section to be
+ reported. The special name "." (a single dot) designates the
+ section in which the statement appears.
+
+ When this statement is specified, only the sections enumerated with this
+ statement will appear in the report. All other ones will be hidden. This
+ statement may appear as many times as needed if multiple sections need to be
+ reported. Please note that the name checking is performed as simple string
+ comparisons, and that it is never checked that a give section name really
+ exists.
+
+ Though this statement alone is enough to enable statistics reporting, it is
+ recommended to set all other settings in order to avoid relying on default
+ unobvious parameters.
+
+ Example :
+ # public access (limited to this backend only)
+ backend public_www
+ server srv1 192.168.0.1:80
+ stats enable
+ stats hide-version
+ stats scope .
+ stats uri /admin?stats
+ stats realm HAProxy\ Statistics
+ stats auth admin1:AdMiN123
+ stats auth admin2:AdMiN321
+
+ # internal monitoring access (unlimited)
+ backend private_monitoring
+ stats enable
+ stats uri /admin?stats
+ stats refresh 5s
+
+ See also : "stats auth", "stats enable", "stats realm", "stats uri"
+
+
+stats show-desc [ <desc> ]
+ Enable reporting of a description on the statistics page.
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ <desc> is an optional description to be reported. If unspecified, the
+ description from global section is automatically used instead.
+
+ This statement is useful for users that offer shared services to their
+ customers, where node or description should be different for each customer.
+
+ Though this statement alone is enough to enable statistics reporting, it is
+ recommended to set all other settings in order to avoid relying on default
+ unobvious parameters. By default description is not shown.
+
+ Example :
+ # internal monitoring access (unlimited)
+ backend private_monitoring
+ stats enable
+ stats show-desc Master node for Europe, Asia, Africa
+ stats uri /admin?stats
+ stats refresh 5s
+
+ See also: "show-node", "stats enable", "stats uri" and "description" in
+ global section.
+
+
+stats show-legends
+ Enable reporting additional information on the statistics page
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments : none
+
+ Enable reporting additional information on the statistics page :
+ - cap: capabilities (proxy)
+ - mode: one of tcp, http or health (proxy)
+ - id: SNMP ID (proxy, socket, server)
+ - IP (socket, server)
+ - cookie (backend, server)
+
+ Though this statement alone is enough to enable statistics reporting, it is
+ recommended to set all other settings in order to avoid relying on default
+ unobvious parameters. Default behavior is not to show this information.
+
+ See also: "stats enable", "stats uri".
+
+
+stats show-modules
+ Enable display of extra statistics module on the statistics page
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments : none
+
+ New columns are added at the end of the line containing the extra statistics
+ values as a tooltip.
+
+ Though this statement alone is enough to enable statistics reporting, it is
+ recommended to set all other settings in order to avoid relying on default
+ unobvious parameters. Default behavior is not to show this information.
+
+ See also: "stats enable", "stats uri".
+
+
+stats show-node [ <name> ]
+ Enable reporting of a host name on the statistics page.
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments:
+ <name> is an optional name to be reported. If unspecified, the
+ node name from global section is automatically used instead.
+
+ This statement is useful for users that offer shared services to their
+ customers, where node or description might be different on a stats page
+ provided for each customer. Default behavior is not to show host name.
+
+ Though this statement alone is enough to enable statistics reporting, it is
+ recommended to set all other settings in order to avoid relying on default
+ unobvious parameters.
+
+ Example:
+ # internal monitoring access (unlimited)
+ backend private_monitoring
+ stats enable
+ stats show-node Europe-1
+ stats uri /admin?stats
+ stats refresh 5s
+
+ See also: "show-desc", "stats enable", "stats uri", and "node" in global
+ section.
+
+
+stats uri <prefix>
+ Enable statistics and define the URI prefix to access them
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments :
+ <prefix> is the prefix of any URI which will be redirected to stats. This
+ prefix may contain a question mark ('?') to indicate part of a
+ query string.
+
+ The statistics URI is intercepted on the relayed traffic, so it appears as a
+ page within the normal application. It is strongly advised to ensure that the
+ selected URI will never appear in the application, otherwise it will never be
+ possible to reach it in the application.
+
+ The default URI compiled in HAProxy is "/haproxy?stats", but this may be
+ changed at build time, so it's better to always explicitly specify it here.
+ It is generally a good idea to include a question mark in the URI so that
+ intermediate proxies refrain from caching the results. Also, since any string
+ beginning with the prefix will be accepted as a stats request, the question
+ mark helps ensuring that no valid URI will begin with the same words.
+
+ It is sometimes very convenient to use "/" as the URI prefix, and put that
+ statement in a "listen" instance of its own. That makes it easy to dedicate
+ an address or a port to statistics only.
+
+ Though this statement alone is enough to enable statistics reporting, it is
+ recommended to set all other settings in order to avoid relying on default
+ unobvious parameters.
+
+ Example :
+ # public access (limited to this backend only)
+ backend public_www
+ server srv1 192.168.0.1:80
+ stats enable
+ stats hide-version
+ stats scope .
+ stats uri /admin?stats
+ stats realm HAProxy\ Statistics
+ stats auth admin1:AdMiN123
+ stats auth admin2:AdMiN321
+
+ # internal monitoring access (unlimited)
+ backend private_monitoring
+ stats enable
+ stats uri /admin?stats
+ stats refresh 5s
+
+ See also : "stats auth", "stats enable", "stats realm"
+
+
+stick match <pattern> [table <table>] [{if | unless} <cond>]
+ Define a request pattern matching condition to stick a user to a server
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ no | no | yes | yes
+
+ Arguments :
+ <pattern> is a sample expression rule as described in section 7.3. It
+ describes what elements of the incoming request or connection
+ will be analyzed in the hope to find a matching entry in a
+ stickiness table. This rule is mandatory.
+
+ <table> is an optional stickiness table name. If unspecified, the same
+ backend's table is used. A stickiness table is declared using
+ the "stick-table" statement.
+
+ <cond> is an optional matching condition. It makes it possible to match
+ on a certain criterion only when other conditions are met (or
+ not met). For instance, it could be used to match on a source IP
+ address except when a request passes through a known proxy, in
+ which case we'd match on a header containing that IP address.
+
+ Some protocols or applications require complex stickiness rules and cannot
+ always simply rely on cookies nor hashing. The "stick match" statement
+ describes a rule to extract the stickiness criterion from an incoming request
+ or connection. See section 7 for a complete list of possible patterns and
+ transformation rules.
+
+ The table has to be declared using the "stick-table" statement. It must be of
+ a type compatible with the pattern. By default it is the one which is present
+ in the same backend. It is possible to share a table with other backends by
+ referencing it using the "table" keyword. If another table is referenced,
+ the server's ID inside the backends are used. By default, all server IDs
+ start at 1 in each backend, so the server ordering is enough. But in case of
+ doubt, it is highly recommended to force server IDs using their "id" setting.
+
+ It is possible to restrict the conditions where a "stick match" statement
+ will apply, using "if" or "unless" followed by a condition. See section 7 for
+ ACL based conditions.
+
+ There is no limit on the number of "stick match" statements. The first that
+ applies and matches will cause the request to be directed to the same server
+ as was used for the request which created the entry. That way, multiple
+ matches can be used as fallbacks.
+
+ The stick rules are checked after the persistence cookies, so they will not
+ affect stickiness if a cookie has already been used to select a server. That
+ way, it becomes very easy to insert cookies and match on IP addresses in
+ order to maintain stickiness between HTTP and HTTPS.
+
+ Example :
+ # forward SMTP users to the same server they just used for POP in the
+ # last 30 minutes
+ backend pop
+ mode tcp
+ balance roundrobin
+ stick store-request src
+ stick-table type ip size 200k expire 30m
+ server s1 192.168.1.1:110
+ server s2 192.168.1.1:110
+
+ backend smtp
+ mode tcp
+ balance roundrobin
+ stick match src table pop
+ server s1 192.168.1.1:25
+ server s2 192.168.1.1:25
+
+ See also : "stick-table", "stick on", and section 7 about ACLs and samples
+ fetching.
+
+
+stick on <pattern> [table <table>] [{if | unless} <condition>]
+ Define a request pattern to associate a user to a server
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ no | no | yes | yes
+
+ Note : This form is exactly equivalent to "stick match" followed by
+ "stick store-request", all with the same arguments. Please refer
+ to both keywords for details. It is only provided as a convenience
+ for writing more maintainable configurations.
+
+ Examples :
+ # The following form ...
+ stick on src table pop if !localhost
+
+ # ...is strictly equivalent to this one :
+ stick match src table pop if !localhost
+ stick store-request src table pop if !localhost
+
+
+ # Use cookie persistence for HTTP, and stick on source address for HTTPS as
+ # well as HTTP without cookie. Share the same table between both accesses.
+ backend http
+ mode http
+ balance roundrobin
+ stick on src table https
+ cookie SRV insert indirect nocache
+ server s1 192.168.1.1:80 cookie s1
+ server s2 192.168.1.1:80 cookie s2
+
+ backend https
+ mode tcp
+ balance roundrobin
+ stick-table type ip size 200k expire 30m
+ stick on src
+ server s1 192.168.1.1:443
+ server s2 192.168.1.1:443
+
+ See also : "stick match", "stick store-request".
+
+
+stick store-request <pattern> [table <table>] [{if | unless} <condition>]
+ Define a request pattern used to create an entry in a stickiness table
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ no | no | yes | yes
+
+ Arguments :
+ <pattern> is a sample expression rule as described in section 7.3. It
+ describes what elements of the incoming request or connection
+ will be analyzed, extracted and stored in the table once a
+ server is selected.
+
+ <table> is an optional stickiness table name. If unspecified, the same
+ backend's table is used. A stickiness table is declared using
+ the "stick-table" statement.
+
+ <cond> is an optional storage condition. It makes it possible to store
+ certain criteria only when some conditions are met (or not met).
+ For instance, it could be used to store the source IP address
+ except when the request passes through a known proxy, in which
+ case we'd store a converted form of a header containing that IP
+ address.
+
+ Some protocols or applications require complex stickiness rules and cannot
+ always simply rely on cookies nor hashing. The "stick store-request" statement
+ describes a rule to decide what to extract from the request and when to do
+ it, in order to store it into a stickiness table for further requests to
+ match it using the "stick match" statement. Obviously the extracted part must
+ make sense and have a chance to be matched in a further request. Storing a
+ client's IP address for instance often makes sense. Storing an ID found in a
+ URL parameter also makes sense. Storing a source port will almost never make
+ any sense because it will be randomly matched. See section 7 for a complete
+ list of possible patterns and transformation rules.
+
+ The table has to be declared using the "stick-table" statement. It must be of
+ a type compatible with the pattern. By default it is the one which is present
+ in the same backend. It is possible to share a table with other backends by
+ referencing it using the "table" keyword. If another table is referenced,
+ the server's ID inside the backends are used. By default, all server IDs
+ start at 1 in each backend, so the server ordering is enough. But in case of
+ doubt, it is highly recommended to force server IDs using their "id" setting.
+
+ It is possible to restrict the conditions where a "stick store-request"
+ statement will apply, using "if" or "unless" followed by a condition. This
+ condition will be evaluated while parsing the request, so any criteria can be
+ used. See section 7 for ACL based conditions.
+
+ There is no limit on the number of "stick store-request" statements, but
+ there is a limit of 8 simultaneous stores per request or response. This
+ makes it possible to store up to 8 criteria, all extracted from either the
+ request or the response, regardless of the number of rules. Only the 8 first
+ ones which match will be kept. Using this, it is possible to feed multiple
+ tables at once in the hope to increase the chance to recognize a user on
+ another protocol or access method. Using multiple store-request rules with
+ the same table is possible and may be used to find the best criterion to rely
+ on, by arranging the rules by decreasing preference order. Only the first
+ extracted criterion for a given table will be stored. All subsequent store-
+ request rules referencing the same table will be skipped and their ACLs will
+ not be evaluated.
+
+ The "store-request" rules are evaluated once the server connection has been
+ established, so that the table will contain the real server that processed
+ the request.
+
+ Example :
+ # forward SMTP users to the same server they just used for POP in the
+ # last 30 minutes
+ backend pop
+ mode tcp
+ balance roundrobin
+ stick store-request src
+ stick-table type ip size 200k expire 30m
+ server s1 192.168.1.1:110
+ server s2 192.168.1.1:110
+
+ backend smtp
+ mode tcp
+ balance roundrobin
+ stick match src table pop
+ server s1 192.168.1.1:25
+ server s2 192.168.1.1:25
+
+ See also : "stick-table", "stick on", about ACLs and sample fetching.
+
+
+stick-table type {ip | integer | string [len <length>] | binary [len <length>]}
+ size <size> [expire <expire>] [nopurge] [peers <peersect>] [srvkey <srvkey>]
+ [write-to <wtable>] [store <data_type>]*
+ Configure the stickiness table for the current section
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ no | yes | yes | yes
+
+ Arguments :
+ ip a table declared with "type ip" will only store IPv4 addresses.
+ This form is very compact (about 50 bytes per entry) and allows
+ very fast entry lookup and stores with almost no overhead. This
+ is mainly used to store client source IP addresses.
+
+ ipv6 a table declared with "type ipv6" will only store IPv6 addresses.
+ This form is very compact (about 60 bytes per entry) and allows
+ very fast entry lookup and stores with almost no overhead. This
+ is mainly used to store client source IP addresses.
+
+ integer a table declared with "type integer" will store 32bit integers
+ which can represent a client identifier found in a request for
+ instance.
+
+ string a table declared with "type string" will store substrings of up
+ to <len> characters. If the string provided by the pattern
+ extractor is larger than <len>, it will be truncated before
+ being stored. During matching, at most <len> characters will be
+ compared between the string in the table and the extracted
+ pattern. When not specified, the string is automatically limited
+ to 32 characters.
+
+ binary a table declared with "type binary" will store binary blocks
+ of <len> bytes. If the block provided by the pattern
+ extractor is larger than <len>, it will be truncated before
+ being stored. If the block provided by the sample expression
+ is shorter than <len>, it will be padded by 0. When not
+ specified, the block is automatically limited to 32 bytes.
+
+ <length> is the maximum number of characters that will be stored in a
+ "string" type table (See type "string" above). Or the number
+ of bytes of the block in "binary" type table. Be careful when
+ changing this parameter as memory usage will proportionally
+ increase.
+
+ <size> is the maximum number of entries that can fit in the table. This
+ value directly impacts memory usage. Count approximately
+ 50 bytes per entry, plus the size of a string if any. The size
+ supports suffixes "k", "m", "g" for 2^10, 2^20 and 2^30 factors.
+
+ [nopurge] indicates that we refuse to purge older entries when the table
+ is full. When not specified and the table is full when HAProxy
+ wants to store an entry in it, it will flush a few of the oldest
+ entries in order to release some space for the new ones. This is
+ most often the desired behavior. In some specific cases, it
+ be desirable to refuse new entries instead of purging the older
+ ones. That may be the case when the amount of data to store is
+ far above the hardware limits and we prefer not to offer access
+ to new clients than to reject the ones already connected. When
+ using this parameter, be sure to properly set the "expire"
+ parameter (see below).
+
+ <peersect> is the name of the peers section to use for replication. Entries
+ which associate keys to server IDs are kept synchronized with
+ the remote peers declared in this section. All entries are also
+ automatically learned from the local peer (old process) during a
+ soft restart.
+
+ <wtable> is the name of the stick table where peers updates will be
+ written to in addition to the source table. <wtable> must be of
+ the same type as the table being defined and must have the same
+ key length, and source table cannot be used as a target table
+ itself. Every time an entry update will be received on the source
+ table through a peer, haproxy will try to refresh related
+ <wtable> entry. If the entry doesn't exist yet, it will be
+ created, else its values will be updated as well as its timer.
+ Note that only types that are not involved in arithmetic ops such
+ as server_id, server_key and gpt will be written to <wtable> to
+ prevent processed values from a remote table from interfering with
+ arithmetic operations performed on the local target table.
+ (ie: prevent shared cumulative counter from growing indefinitely)
+ One common use of this option is to be able to use sticking rules
+ (for server persistence) in a peers cluster setup, because
+ matching keys will be learned from remote tables.
+
+ <expire> defines the maximum duration of an entry in the table since it
+ was last created, refreshed using 'track-sc' or matched using
+ 'stick match' or 'stick on' rule. The expiration delay is
+ defined using the standard time format, similarly as the various
+ timeouts. The maximum duration is slightly above 24 days. See
+ section 2.5 for more information. If this delay is not specified,
+ the session won't automatically expire, but older entries will
+ be removed once full. Be sure not to use the "nopurge" parameter
+ if not expiration delay is specified.
+ Note: 'table_*' converters performs lookups but won't update touch
+ expire since they don't require 'track-sc'.
+
+ <srvkey> specifies how each server is identified for the purposes of the
+ stick table. The valid values are "name" and "addr". If "name" is
+ given, then <name> argument for the server (may be generated by
+ a template). If "addr" is given, then the server is identified
+ by its current network address, including the port. "addr" is
+ especially useful if you are using service discovery to generate
+ the addresses for servers with peered stick-tables and want
+ to consistently use the same host across peers for a stickiness
+ token.
+
+ <data_type> is used to store additional information in the stick-table. This
+ may be used by ACLs in order to control various criteria related
+ to the activity of the client matching the stick-table. For each
+ item specified here, the size of each entry will be inflated so
+ that the additional data can fit. Several data types may be
+ stored with an entry. Multiple data types may be specified after
+ the "store" keyword, as a comma-separated list. Alternatively,
+ it is possible to repeat the "store" keyword followed by one or
+ several data types. Except for the "server_id" type which is
+ automatically detected and enabled, all data types must be
+ explicitly declared to be stored. If an ACL references a data
+ type which is not stored, the ACL will simply not match. Some
+ data types require an argument which must be passed just after
+ the type between parenthesis. See below for the supported data
+ types and their arguments.
+
+ The data types that can be stored with an entry are the following :
+ - server_id : this is an integer which holds the numeric ID of the server a
+ request was assigned to. It is used by the "stick match", "stick store",
+ and "stick on" rules. It is automatically enabled when referenced.
+
+ - gpc(<nb>) : General Purpose Counters Array of <nb> elements. This is an
+ array of positive 32-bit integers which may be used to count anything.
+ Most of the time they will be used as a incremental counters on some
+ entries, for instance to note that a limit is reached and trigger some
+ actions. This array is limited to a maximum of 100 elements:
+ gpc0 to gpc99, to ensure that the build of a peer update
+ message can fit into the buffer. Users should take in consideration
+ that a large amount of counters will increase the data size and the
+ traffic load using peers protocol since all data/counters are pushed
+ each time any of them is updated.
+ This data_type will exclude the usage of the legacy data_types 'gpc0'
+ and 'gpc1' on the same table. Using the 'gpc' array data_type, all 'gpc0'
+ and 'gpc1' related fetches and actions will apply to the two first
+ elements of this array.
+
+ - gpc_rate(<nb>,<period>) : Array of increment rates of General Purpose
+ Counters over a period. Those elements are positive 32-bit integers which
+ may be used for anything. Just like <gpc>, the count events, but instead
+ of keeping a cumulative number, they maintain the rate at which the
+ counter is incremented. Most of the time it will be used to measure the
+ frequency of occurrence of certain events (e.g. requests to a specific
+ URL). This array is limited to a maximum of 100 elements: gpt(100)
+ allowing the storage of gpc0 to gpc99, to ensure that the build of a peer
+ update message can fit into the buffer.
+ The array cannot contain less than 1 element: use gpc(1) if you want to
+ store only the counter gpc0.
+ Users should take in consideration that a large amount of
+ counters will increase the data size and the traffic load using peers
+ protocol since all data/counters are pushed each time any of them is
+ updated.
+ This data_type will exclude the usage of the legacy data_types
+ 'gpc0_rate' and 'gpc1_rate' on the same table. Using the 'gpc_rate'
+ array data_type, all 'gpc0' and 'gpc1' related fetches and actions
+ will apply to the two first elements of this array.
+
+ - gpc0 : first General Purpose Counter. It is a positive 32-bit integer
+ integer which may be used for anything. Most of the time it will be used
+ to put a special tag on some entries, for instance to note that a
+ specific behavior was detected and must be known for future matches.
+
+ - gpc0_rate(<period>) : increment rate of the first General Purpose Counter
+ over a period. It is a positive 32-bit integer integer which may be used
+ for anything. Just like <gpc0>, it counts events, but instead of keeping
+ a cumulative number, it maintains the rate at which the counter is
+ incremented. Most of the time it will be used to measure the frequency of
+ occurrence of certain events (e.g. requests to a specific URL).
+
+ - gpc1 : second General Purpose Counter. It is a positive 32-bit integer
+ integer which may be used for anything. Most of the time it will be used
+ to put a special tag on some entries, for instance to note that a
+ specific behavior was detected and must be known for future matches.
+
+ - gpc1_rate(<period>) : increment rate of the second General Purpose Counter
+ over a period. It is a positive 32-bit integer integer which may be used
+ for anything. Just like <gpc1>, it counts events, but instead of keeping
+ a cumulative number, it maintains the rate at which the counter is
+ incremented. Most of the time it will be used to measure the frequency of
+ occurrence of certain events (e.g. requests to a specific URL).
+
+ - gpt(<nb>) : General Purpose Tags Array of <nb> elements. This is an array
+ of positive 32-bit integers which may be used for anything.
+ Most of the time they will be used to put a special tags on some entries,
+ for instance to note that a specific behavior was detected and must be
+ known for future matches. This array is limited to a maximum of 100
+ elements: gpt(100) allowing the storage of gpt0 to gpt99, to ensure that
+ the build of a peer update message can fit into the buffer.
+ The array cannot contain less than 1 element: use gpt(1) if you want to
+ to store only the tag gpt0.
+ Users should take in consideration that a large amount of counters will
+ increase the data size and the traffic load using peers protocol since
+ all data/counters are pushed each time any of them is updated.
+ This data_type will exclude the usage of the legacy data_type 'gpt0'
+ on the same table. Using the 'gpt' array data_type, all 'gpt0' related
+ fetches and actions will apply to the first element of this array.
+
+ - gpt0 : first General Purpose Tag. It is a positive 32-bit integer
+ integer which may be used for anything. Most of the time it will be used
+ to put a special tag on some entries, for instance to note that a
+ specific behavior was detected and must be known for future matches
+
+ - conn_cnt : Connection Count. It is a positive 32-bit integer which counts
+ the absolute number of connections received from clients which matched
+ this entry. It does not mean the connections were accepted, just that
+ they were received.
+
+ - conn_cur : Current Connections. It is a positive 32-bit integer which
+ stores the concurrent connection counts for the entry. It is incremented
+ once an incoming connection matches the entry, and decremented once the
+ connection leaves. That way it is possible to know at any time the exact
+ number of concurrent connections for an entry.
+
+ - conn_rate(<period>) : frequency counter (takes 12 bytes). It takes an
+ integer parameter <period> which indicates in milliseconds the length
+ of the period over which the average is measured. It reports the average
+ incoming connection rate over that period, in connections per period. The
+ result is an integer which can be matched using ACLs.
+
+ - sess_cnt : Session Count. It is a positive 32-bit integer which counts
+ the absolute number of sessions received from clients which matched this
+ entry. A session is a connection that was accepted by the layer 4 rules.
+
+ - sess_rate(<period>) : frequency counter (takes 12 bytes). It takes an
+ integer parameter <period> which indicates in milliseconds the length
+ of the period over which the average is measured. It reports the average
+ incoming session rate over that period, in sessions per period. The
+ result is an integer which can be matched using ACLs.
+
+ - http_req_cnt : HTTP request Count. It is a positive 32-bit integer which
+ counts the absolute number of HTTP requests received from clients which
+ matched this entry. It does not matter whether they are valid requests or
+ not. Note that this is different from sessions when keep-alive is used on
+ the client side.
+
+ - http_req_rate(<period>) : frequency counter (takes 12 bytes). It takes an
+ integer parameter <period> which indicates in milliseconds the length
+ of the period over which the average is measured. It reports the average
+ HTTP request rate over that period, in requests per period. The result is
+ an integer which can be matched using ACLs. It does not matter whether
+ they are valid requests or not. Note that this is different from sessions
+ when keep-alive is used on the client side.
+
+ - http_err_cnt : HTTP Error Count. It is a positive 32-bit integer which
+ counts the absolute number of HTTP requests errors induced by clients
+ which matched this entry. Errors are counted on invalid and truncated
+ requests, as well as on denied or tarpitted requests, and on failed
+ authentications. If the server responds with 4xx, then the request is
+ also counted as an error since it's an error triggered by the client
+ (e.g. vulnerability scan).
+
+ - http_err_rate(<period>) : frequency counter (takes 12 bytes). It takes an
+ integer parameter <period> which indicates in milliseconds the length
+ of the period over which the average is measured. It reports the average
+ HTTP request error rate over that period, in requests per period (see
+ http_err_cnt above for what is accounted as an error). The result is an
+ integer which can be matched using ACLs.
+
+ - http_fail_cnt : HTTP Failure Count. It is a positive 32-bit integer which
+ counts the absolute number of HTTP response failures induced by servers
+ which matched this entry. Errors are counted on invalid and truncated
+ responses, as well as any 5xx response other than 501 or 505. It aims at
+ being used combined with path or URI to detect service failures.
+
+ - http_fail_rate(<period>) : frequency counter (takes 12 bytes). It takes
+ an integer parameter <period> which indicates in milliseconds the length
+ of the period over which the average is measured. It reports the average
+ HTTP response failure rate over that period, in requests per period (see
+ http_fail_cnt above for what is accounted as a failure). The result is an
+ integer which can be matched using ACLs.
+
+ - bytes_in_cnt : client to server byte count. It is a positive 64-bit
+ integer which counts the cumulative number of bytes received from clients
+ which matched this entry. Headers are included in the count. This may be
+ used to limit abuse of upload features on photo or video servers.
+
+ - bytes_in_rate(<period>) : frequency counter (takes 12 bytes). It takes an
+ integer parameter <period> which indicates in milliseconds the length
+ of the period over which the average is measured. It reports the average
+ incoming bytes rate over that period, in bytes per period. It may be used
+ to detect users which upload too much and too fast. Warning: with large
+ uploads, it is possible that the amount of uploaded data will be counted
+ once upon termination, thus causing spikes in the average transfer speed
+ instead of having a smooth one. This may partially be smoothed with
+ "option contstats" though this is not perfect yet. Use of byte_in_cnt is
+ recommended for better fairness.
+
+ - bytes_out_cnt : server to client byte count. It is a positive 64-bit
+ integer which counts the cumulative number of bytes sent to clients which
+ matched this entry. Headers are included in the count. This may be used
+ to limit abuse of bots sucking the whole site.
+
+ - bytes_out_rate(<period>) : frequency counter (takes 12 bytes). It takes
+ an integer parameter <period> which indicates in milliseconds the length
+ of the period over which the average is measured. It reports the average
+ outgoing bytes rate over that period, in bytes per period. It may be used
+ to detect users which download too much and too fast. Warning: with large
+ transfers, it is possible that the amount of transferred data will be
+ counted once upon termination, thus causing spikes in the average
+ transfer speed instead of having a smooth one. This may partially be
+ smoothed with "option contstats" though this is not perfect yet. Use of
+ byte_out_cnt is recommended for better fairness.
+
+ There is only one stick-table per proxy. At the moment of writing this doc,
+ it does not seem useful to have multiple tables per proxy. If this happens
+ to be required, simply create a dummy backend with a stick-table in it and
+ reference it.
+
+ It is important to understand that stickiness based on learning information
+ has some limitations, including the fact that all learned associations are
+ lost upon restart unless peers are properly configured to transfer such
+ information upon restart (recommended). In general it can be good as a
+ complement but not always as an exclusive stickiness.
+
+ Last, memory requirements may be important when storing many data types.
+ Indeed, storing all indicators above at once in each entry requires 116 bytes
+ per entry, or 116 MB for a 1-million entries table. This is definitely not
+ something that can be ignored.
+
+ Example:
+ # Keep track of counters of up to 1 million IP addresses over 5 minutes
+ # and store a general purpose counter and the average connection rate
+ # computed over a sliding window of 30 seconds.
+ stick-table type ip size 1m expire 5m store gpc0,conn_rate(30s)
+
+ See also : "stick match", "stick on", "stick store-request", section 2.5
+ about time format and section 7 about ACLs.
+
+
+stick store-response <pattern> [table <table>] [{if | unless} <condition>]
+ Define a response pattern used to create an entry in a stickiness table
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ no | no | yes | yes
+
+ Arguments :
+ <pattern> is a sample expression rule as described in section 7.3. It
+ describes what elements of the response or connection will
+ be analyzed, extracted and stored in the table once a
+ server is selected.
+
+ <table> is an optional stickiness table name. If unspecified, the same
+ backend's table is used. A stickiness table is declared using
+ the "stick-table" statement.
+
+ <cond> is an optional storage condition. It makes it possible to store
+ certain criteria only when some conditions are met (or not met).
+ For instance, it could be used to store the SSL session ID only
+ when the response is a SSL server hello.
+
+ Some protocols or applications require complex stickiness rules and cannot
+ always simply rely on cookies nor hashing. The "stick store-response"
+ statement describes a rule to decide what to extract from the response and
+ when to do it, in order to store it into a stickiness table for further
+ requests to match it using the "stick match" statement. Obviously the
+ extracted part must make sense and have a chance to be matched in a further
+ request. Storing an ID found in a header of a response makes sense.
+ See section 7 for a complete list of possible patterns and transformation
+ rules.
+
+ The table has to be declared using the "stick-table" statement. It must be of
+ a type compatible with the pattern. By default it is the one which is present
+ in the same backend. It is possible to share a table with other backends by
+ referencing it using the "table" keyword. If another table is referenced,
+ the server's ID inside the backends are used. By default, all server IDs
+ start at 1 in each backend, so the server ordering is enough. But in case of
+ doubt, it is highly recommended to force server IDs using their "id" setting.
+
+ It is possible to restrict the conditions where a "stick store-response"
+ statement will apply, using "if" or "unless" followed by a condition. This
+ condition will be evaluated while parsing the response, so any criteria can
+ be used. See section 7 for ACL based conditions.
+
+ There is no limit on the number of "stick store-response" statements, but
+ there is a limit of 8 simultaneous stores per request or response. This
+ makes it possible to store up to 8 criteria, all extracted from either the
+ request or the response, regardless of the number of rules. Only the 8 first
+ ones which match will be kept. Using this, it is possible to feed multiple
+ tables at once in the hope to increase the chance to recognize a user on
+ another protocol or access method. Using multiple store-response rules with
+ the same table is possible and may be used to find the best criterion to rely
+ on, by arranging the rules by decreasing preference order. Only the first
+ extracted criterion for a given table will be stored. All subsequent store-
+ response rules referencing the same table will be skipped and their ACLs will
+ not be evaluated. However, even if a store-request rule references a table, a
+ store-response rule may also use the same table. This means that each table
+ may learn exactly one element from the request and one element from the
+ response at once.
+
+ The table will contain the real server that processed the request.
+
+ Example :
+ # Learn SSL session ID from both request and response and create affinity.
+ backend https
+ mode tcp
+ balance roundrobin
+ # maximum SSL session ID length is 32 bytes.
+ stick-table type binary len 32 size 30k expire 30m
+
+ acl clienthello req.ssl_hello_type 1
+ acl serverhello res.ssl_hello_type 2
+
+ # use tcp content accepts to detects ssl client and server hello.
+ tcp-request inspect-delay 5s
+ tcp-request content accept if clienthello
+
+ # no timeout on response inspect delay by default.
+ tcp-response content accept if serverhello
+
+ # SSL session ID (SSLID) may be present on a client or server hello.
+ # Its length is coded on 1 byte at offset 43 and its value starts
+ # at offset 44.
+
+ # Match and learn on request if client hello.
+ stick on req.payload_lv(43,1) if clienthello
+
+ # Learn on response if server hello.
+ stick store-response resp.payload_lv(43,1) if serverhello
+
+ server s1 192.168.1.1:443
+ server s2 192.168.1.1:443
+
+ See also : "stick-table", "stick on", and section 7 about ACLs and pattern
+ extraction.
+
+
+tcp-check comment <string>
+ Defines a comment for the following the tcp-check rule, reported in logs if
+ it fails.
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <string> is the comment message to add in logs if the following tcp-check
+ rule fails.
+
+ It only works for connect, send and expect rules. It is useful to make
+ user-friendly error reporting.
+
+ See also : "option tcp-check", "tcp-check connect", "tcp-check send" and
+ "tcp-check expect".
+
+
+tcp-check connect [default] [port <expr>] [addr <ip>] [send-proxy] [via-socks4]
+ [ssl] [sni <sni>] [alpn <alpn>] [linger]
+ [proto <name>] [comment <msg>]
+ Opens a new connection
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ comment <msg> defines a message to report if the rule evaluation fails.
+
+ default Use default options of the server line to do the health
+ checks. The server options are used only if not redefined.
+
+ port <expr> if not set, check port or server port is used.
+ It tells HAProxy where to open the connection to.
+ <port> must be a valid TCP port source integer, from 1 to
+ 65535 or an sample-fetch expression.
+
+ addr <ip> defines the IP address to do the health check.
+
+ send-proxy send a PROXY protocol string
+
+ via-socks4 enables outgoing health checks using upstream socks4 proxy.
+
+ ssl opens a ciphered connection
+
+ sni <sni> specifies the SNI to use to do health checks over SSL.
+
+ alpn <alpn> defines which protocols to advertise with ALPN. The protocol
+ list consists in a comma-delimited list of protocol names,
+ for instance: "http/1.1,http/1.0" (without quotes).
+ If it is not set, the server ALPN is used.
+
+ proto <name> forces the multiplexer's protocol to use for this connection.
+ It must be a TCP mux protocol and it must be usable on the
+ backend side. The list of available protocols is reported in
+ haproxy -vv.
+
+ linger cleanly close the connection instead of using a single RST.
+
+ When an application lies on more than a single TCP port or when HAProxy
+ load-balance many services in a single backend, it makes sense to probe all
+ the services individually before considering a server as operational.
+
+ When there are no TCP port configured on the server line neither server port
+ directive, then the 'tcp-check connect port <port>' must be the first step
+ of the sequence.
+
+ In a tcp-check ruleset a 'connect' is required, it is also mandatory to start
+ the ruleset with a 'connect' rule. Purpose is to ensure admin know what they
+ do.
+
+ When a connect must start the ruleset, if may still be preceded by set-var,
+ unset-var or comment rules.
+
+ Examples :
+ # check HTTP and HTTPs services on a server.
+ # first open port 80 thanks to server line port directive, then
+ # tcp-check opens port 443, ciphered and run a request on it:
+ option tcp-check
+ tcp-check connect
+ tcp-check send GET\ /\ HTTP/1.0\r\n
+ tcp-check send Host:\ haproxy.1wt.eu\r\n
+ tcp-check send \r\n
+ tcp-check expect rstring (2..|3..)
+ tcp-check connect port 443 ssl
+ tcp-check send GET\ /\ HTTP/1.0\r\n
+ tcp-check send Host:\ haproxy.1wt.eu\r\n
+ tcp-check send \r\n
+ tcp-check expect rstring (2..|3..)
+ server www 10.0.0.1 check port 80
+
+ # check both POP and IMAP from a single server:
+ option tcp-check
+ tcp-check connect port 110 linger
+ tcp-check expect string +OK\ POP3\ ready
+ tcp-check connect port 143
+ tcp-check expect string *\ OK\ IMAP4\ ready
+ server mail 10.0.0.1 check
+
+ See also : "option tcp-check", "tcp-check send", "tcp-check expect"
+
+
+tcp-check expect [min-recv <int>] [comment <msg>]
+ [ok-status <st>] [error-status <st>] [tout-status <st>]
+ [on-success <fmt>] [on-error <fmt>] [status-code <expr>]
+ [!] <match> <pattern>
+ Specify data to be collected and analyzed during a generic health check
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ comment <msg> defines a message to report if the rule evaluation fails.
+
+ min-recv is optional and can define the minimum amount of data required to
+ evaluate the current expect rule. If the number of received bytes
+ is under this limit, the check will wait for more data. This
+ option can be used to resolve some ambiguous matching rules or to
+ avoid executing costly regex matches on content known to be still
+ incomplete. If an exact string (string or binary) is used, the
+ minimum between the string length and this parameter is used.
+ This parameter is ignored if it is set to -1. If the expect rule
+ does not match, the check will wait for more data. If set to 0,
+ the evaluation result is always conclusive.
+
+ <match> is a keyword indicating how to look for a specific pattern in the
+ response. The keyword may be one of "string", "rstring", "binary" or
+ "rbinary".
+ The keyword may be preceded by an exclamation mark ("!") to negate
+ the match. Spaces are allowed between the exclamation mark and the
+ keyword. See below for more details on the supported keywords.
+
+ ok-status <st> is optional and can be used to set the check status if
+ the expect rule is successfully evaluated and if it is
+ the last rule in the tcp-check ruleset. "L7OK", "L7OKC",
+ "L6OK" and "L4OK" are supported :
+ - L7OK : check passed on layer 7
+ - L7OKC : check conditionally passed on layer 7, set
+ server to NOLB state.
+ - L6OK : check passed on layer 6
+ - L4OK : check passed on layer 4
+ By default "L7OK" is used.
+
+ error-status <st> is optional and can be used to set the check status if
+ an error occurred during the expect rule evaluation.
+ "L7OKC", "L7RSP", "L7STS", "L6RSP" and "L4CON" are
+ supported :
+ - L7OKC : check conditionally passed on layer 7, set
+ server to NOLB state.
+ - L7RSP : layer 7 invalid response - protocol error
+ - L7STS : layer 7 response error, for example HTTP 5xx
+ - L6RSP : layer 6 invalid response - protocol error
+ - L4CON : layer 1-4 connection problem
+ By default "L7RSP" is used.
+
+ tout-status <st> is optional and can be used to set the check status if
+ a timeout occurred during the expect rule evaluation.
+ "L7TOUT", "L6TOUT", and "L4TOUT" are supported :
+ - L7TOUT : layer 7 (HTTP/SMTP) timeout
+ - L6TOUT : layer 6 (SSL) timeout
+ - L4TOUT : layer 1-4 timeout
+ By default "L7TOUT" is used.
+
+ on-success <fmt> is optional and can be used to customize the
+ informational message reported in logs if the expect
+ rule is successfully evaluated and if it is the last rule
+ in the tcp-check ruleset. <fmt> is a log-format string.
+
+ on-error <fmt> is optional and can be used to customize the
+ informational message reported in logs if an error
+ occurred during the expect rule evaluation. <fmt> is a
+ log-format string.
+
+ status-code <expr> is optional and can be used to set the check status code
+ reported in logs, on success or on error. <expr> is a
+ standard HAProxy expression formed by a sample-fetch
+ followed by some converters.
+
+ <pattern> is the pattern to look for. It may be a string or a regular
+ expression. If the pattern contains spaces, they must be escaped
+ with the usual backslash ('\').
+ If the match is set to binary, then the pattern must be passed as
+ a series of hexadecimal digits in an even number. Each sequence of
+ two digits will represent a byte. The hexadecimal digits may be
+ used upper or lower case.
+
+ The available matches are intentionally similar to their http-check cousins :
+
+ string <string> : test the exact string matches in the response buffer.
+ A health check response will be considered valid if the
+ response's buffer contains this exact string. If the
+ "string" keyword is prefixed with "!", then the response
+ will be considered invalid if the body contains this
+ string. This can be used to look for a mandatory pattern
+ in a protocol response, or to detect a failure when a
+ specific error appears in a protocol banner.
+
+ rstring <regex> : test a regular expression on the response buffer.
+ A health check response will be considered valid if the
+ response's buffer matches this expression. If the
+ "rstring" keyword is prefixed with "!", then the response
+ will be considered invalid if the body matches the
+ expression.
+
+ string-lf <fmt> : test a log-format string match in the response's buffer.
+ A health check response will be considered valid if the
+ response's buffer contains the string resulting of the
+ evaluation of <fmt>, which follows the log-format rules.
+ If prefixed with "!", then the response will be
+ considered invalid if the buffer contains the string.
+
+ binary <hexstring> : test the exact string in its hexadecimal form matches
+ in the response buffer. A health check response will
+ be considered valid if the response's buffer contains
+ this exact hexadecimal string.
+ Purpose is to match data on binary protocols.
+
+ rbinary <regex> : test a regular expression on the response buffer, like
+ "rstring". However, the response buffer is transformed
+ into its hexadecimal form, including NUL-bytes. This
+ allows using all regex engines to match any binary
+ content. The hexadecimal transformation takes twice the
+ size of the original response. As such, the expected
+ pattern should work on at-most half the response buffer
+ size.
+
+ binary-lf <hexfmt> : test a log-format string in its hexadecimal form
+ match in the response's buffer. A health check response
+ will be considered valid if the response's buffer
+ contains the hexadecimal string resulting of the
+ evaluation of <fmt>, which follows the log-format
+ rules. If prefixed with "!", then the response will be
+ considered invalid if the buffer contains the
+ hexadecimal string. The hexadecimal string is converted
+ in a binary string before matching the response's
+ buffer.
+
+ It is important to note that the responses will be limited to a certain size
+ defined by the global "tune.bufsize" option, which defaults to 16384 bytes.
+ Thus, too large responses may not contain the mandatory pattern when using
+ "string", "rstring" or binary. If a large response is absolutely required, it
+ is possible to change the default max size by setting the global variable.
+ However, it is worth keeping in mind that parsing very large responses can
+ waste some CPU cycles, especially when regular expressions are used, and that
+ it is always better to focus the checks on smaller resources. Also, in its
+ current state, the check will not find any string nor regex past a null
+ character in the response. Similarly it is not possible to request matching
+ the null character.
+
+ Examples :
+ # perform a POP check
+ option tcp-check
+ tcp-check expect string +OK\ POP3\ ready
+
+ # perform an IMAP check
+ option tcp-check
+ tcp-check expect string *\ OK\ IMAP4\ ready
+
+ # look for the redis master server
+ option tcp-check
+ tcp-check send PING\r\n
+ tcp-check expect string +PONG
+ tcp-check send info\ replication\r\n
+ tcp-check expect string role:master
+ tcp-check send QUIT\r\n
+ tcp-check expect string +OK
+
+
+ See also : "option tcp-check", "tcp-check connect", "tcp-check send",
+ "tcp-check send-binary", "http-check expect", tune.bufsize
+
+
+tcp-check send <data> [comment <msg>]
+tcp-check send-lf <fmt> [comment <msg>]
+ Specify a string or a log-format string to be sent as a question during a
+ generic health check
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ comment <msg> defines a message to report if the rule evaluation fails.
+
+ <data> is the string that will be sent during a generic health
+ check session.
+
+ <fmt> is the log-format string that will be sent, once evaluated,
+ during a generic health check session.
+
+ Examples :
+ # look for the redis master server
+ option tcp-check
+ tcp-check send info\ replication\r\n
+ tcp-check expect string role:master
+
+ See also : "option tcp-check", "tcp-check connect", "tcp-check expect",
+ "tcp-check send-binary", tune.bufsize
+
+
+tcp-check send-binary <hexstring> [comment <msg>]
+tcp-check send-binary-lf <hexfmt> [comment <msg>]
+ Specify an hex digits string or an hex digits log-format string to be sent as
+ a binary question during a raw tcp health check
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ comment <msg> defines a message to report if the rule evaluation fails.
+
+ <hexstring> is the hexadecimal string that will be send, once converted
+ to binary, during a generic health check session.
+
+ <hexfmt> is the hexadecimal log-format string that will be send, once
+ evaluated and converted to binary, during a generic health
+ check session.
+
+ Examples :
+ # redis check in binary
+ option tcp-check
+ tcp-check send-binary 50494e470d0a # PING\r\n
+ tcp-check expect binary 2b504F4e47 # +PONG
+
+
+ See also : "option tcp-check", "tcp-check connect", "tcp-check expect",
+ "tcp-check send", tune.bufsize
+
+
+tcp-check set-var(<var-name>[,<cond>...]) <expr>
+tcp-check set-var-fmt(<var-name>[,<cond>...]) <fmt>
+ This operation sets the content of a variable. The variable is declared inline.
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <var-name> The name of the variable starts with an indication about its
+ scope. The scopes allowed for tcp-check are:
+ "proc" : the variable is shared with the whole process.
+ "sess" : the variable is shared with the tcp-check session.
+ "check": the variable is declared for the lifetime of the tcp-check.
+ This prefix is followed by a name. The separator is a '.'.
+ The name may only contain characters 'a-z', 'A-Z', '0-9', '.',
+ and '-'.
+
+ <cond> A set of conditions that must all be true for the variable to
+ actually be set (such as "ifnotempty", "ifgt" ...). See the
+ set-var converter's description for a full list of possible
+ conditions.
+
+ <expr> Is a sample-fetch expression potentially followed by converters.
+
+ <fmt> This is the value expressed using log-format rules (see Custom
+ Log Format in section 8.2.4).
+
+ Examples :
+ tcp-check set-var(check.port) int(1234)
+ tcp-check set-var-fmt(check.name) "%H"
+
+
+tcp-check unset-var(<var-name>)
+ Free a reference to a variable within its scope.
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <var-name> The name of the variable starts with an indication about its
+ scope. The scopes allowed for tcp-check are:
+ "proc" : the variable is shared with the whole process.
+ "sess" : the variable is shared with the tcp-check session.
+ "check": the variable is declared for the lifetime of the tcp-check.
+ This prefix is followed by a name. The separator is a '.'.
+ The name may only contain characters 'a-z', 'A-Z', '0-9', '.',
+ and '-'.
+
+ Examples :
+ tcp-check unset-var(check.port)
+
+
+tcp-request connection <action> <options...> [ { if | unless } <condition> ]
+ Perform an action on an incoming connection depending on a layer 4 condition
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes(!) | yes | yes | no
+
+ Arguments :
+ <action> defines the action to perform if the condition applies. See
+ below.
+
+ <condition> is a standard layer4-only ACL-based condition (see section 7).
+
+ Immediately after acceptance of a new incoming connection, it is possible to
+ evaluate some conditions to decide whether this connection must be accepted
+ or dropped or have its counters tracked. Those conditions cannot make use of
+ any data contents because the connection has not been read from yet, and the
+ buffers are not yet allocated. This is used to selectively and very quickly
+ accept or drop connections from various sources with a very low overhead. If
+ some contents need to be inspected in order to take the decision, the
+ "tcp-request content" statements must be used instead.
+
+ The "tcp-request connection" rules are evaluated in their exact declaration
+ order. If no rule matches or if there is no rule, the default action is to
+ accept the incoming connection. There is no specific limit to the number of
+ rules which may be inserted. Any rule may optionally be followed by an
+ ACL-based condition, in which case it will only be evaluated if the condition
+ evaluates to true.
+
+ The condition is evaluated just before the action is executed, and the action
+ is performed exactly once. As such, there is no problem if an action changes
+ an element which is checked as part of the condition. This also means that
+ multiple actions may rely on the same condition so that the first action that
+ changes the condition's evaluation is sufficient to implicitly disable the
+ remaining actions. This is used for example when trying to assign a value to
+ a variable from various sources when it's empty.
+
+ The first keyword after "tcp-request connection" in the syntax is the rule's
+ action, optionally followed by a varying number of arguments for the action.
+ The supported actions and their respective syntaxes are enumerated in
+ section 4.3 "Actions" (look for actions which tick "TCP RqCon").
+
+ This directive is only available from named defaults sections, not anonymous
+ ones. Rules defined in the defaults section are evaluated before ones in the
+ associated proxy section. To avoid ambiguities, in this case the same
+ defaults section cannot be used by proxies with the frontend capability and
+ by proxies with the backend capability. It means a listen section cannot use
+ a defaults section defining such rules.
+
+ Note that the "if/unless" condition is optional. If no condition is set on
+ the action, it is simply performed unconditionally. That can be useful for
+ "track-sc*" actions as well as for changing the default action to a reject.
+
+ Example: accept all connections from white-listed hosts, reject too fast
+ connection without counting them, and track accepted connections.
+ This results in connection rate being capped from abusive sources.
+
+ tcp-request connection accept if { src -f /etc/haproxy/whitelist.lst }
+ tcp-request connection reject if { src_conn_rate gt 10 }
+ tcp-request connection track-sc0 src
+
+ Example: accept all connections from white-listed hosts, count all other
+ connections and reject too fast ones. This results in abusive ones
+ being blocked as long as they don't slow down.
+
+ tcp-request connection accept if { src -f /etc/haproxy/whitelist.lst }
+ tcp-request connection track-sc0 src
+ tcp-request connection reject if { sc0_conn_rate gt 10 }
+
+ Example: enable the PROXY protocol for traffic coming from all known proxies.
+
+ tcp-request connection expect-proxy layer4 if { src -f proxies.lst }
+
+ See section 7 about ACL usage.
+
+ See also : "tcp-request session", "tcp-request content", "stick-table"
+
+tcp-request content <action> [{if | unless} <condition>]
+ Perform an action on a new session depending on a layer 4-7 condition
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes(!) | yes | yes | yes
+
+ Arguments :
+ <action> defines the action to perform if the condition applies. See
+ below.
+
+ <condition> is a standard layer 4-7 ACL-based condition (see section 7).
+
+ A request's contents can be analyzed at an early stage of request processing
+ called "TCP content inspection". During this stage, ACL-based rules are
+ evaluated every time the request contents are updated, until either an
+ "accept", a "reject" or a "switch-mode" rule matches, or the TCP request
+ inspection delay expires with no matching rule.
+
+ The first difference between these rules and "tcp-request connection" rules
+ is that "tcp-request content" rules can make use of contents to take a
+ decision. Most often, these decisions will consider a protocol recognition or
+ validity. The second difference is that content-based rules can be used in
+ both frontends and backends. In case of HTTP keep-alive with the client, all
+ tcp-request content rules are evaluated again, so HAProxy keeps a record of
+ what sticky counters were assigned by a "tcp-request connection" versus a
+ "tcp-request content" rule, and flushes all the content-related ones after
+ processing an HTTP request, so that they may be evaluated again by the rules
+ being evaluated again for the next request. This is of particular importance
+ when the rule tracks some L7 information or when it is conditioned by an
+ L7-based ACL, since tracking may change between requests.
+
+ Content-based rules are evaluated in their exact declaration order. If no
+ rule matches or if there is no rule, the default action is to accept the
+ contents. There is no specific limit to the number of rules which may be
+ inserted.
+
+ While there is nothing mandatory about it, it is recommended to use the
+ track-sc0 in "tcp-request connection" rules, track-sc1 for "tcp-request
+ content" rules in the frontend, and track-sc2 for "tcp-request content"
+ rules in the backend, because that makes the configuration more readable
+ and easier to troubleshoot, but this is just a guideline and all counters
+ may be used everywhere.
+
+ The first keyword after "tcp-request content" in the syntax is the rule's
+ action, optionally followed by a varying number of arguments for the action.
+ The supported actions and their respective syntaxes are enumerated in
+ section 4.3 "Actions" (look for actions which tick "TCP RqCnt").
+
+ This directive is only available from named defaults sections, not anonymous
+ ones. Rules defined in the defaults section are evaluated before ones in the
+ associated proxy section. To avoid ambiguities, in this case the same
+ defaults section cannot be used by proxies with the frontend capability and
+ by proxies with the backend capability. It means a listen section cannot use
+ a defaults section defining such rules.
+
+ Note that the "if/unless" condition is optional. If no condition is set on
+ the action, it is simply performed unconditionally. That can be useful for
+ "track-sc*" actions as well as for changing the default action to a reject.
+
+ Note also that it is recommended to use a "tcp-request session" rule to track
+ information that does *not* depend on Layer 7 contents, especially for HTTP
+ frontends. Some HTTP processing are performed at the session level and may
+ lead to an early rejection of the requests. Thus, the tracking at the content
+ level may be disturbed in such case. A warning is emitted during startup to
+ prevent, as far as possible, such unreliable usage.
+
+ It is perfectly possible to match layer 7 contents with "tcp-request content"
+ rules from a TCP proxy, since HTTP-specific ACL matches are able to
+ preliminarily parse the contents of a buffer before extracting the required
+ data. If the buffered contents do not parse as a valid HTTP message, then the
+ ACL does not match. The parser which is involved there is exactly the same
+ as for all other HTTP processing, so there is no risk of parsing something
+ differently. In an HTTP frontend or an HTTP backend, it is guaranteed that
+ HTTP contents will always be immediately present when the rule is evaluated
+ first because the HTTP parsing is performed in the early stages of the
+ connection processing, at the session level. But for such proxies, using
+ "http-request" rules is much more natural and recommended.
+
+ Tracking layer7 information is also possible provided that the information
+ are present when the rule is processed. The rule processing engine is able to
+ wait until the inspect delay expires when the data to be tracked is not yet
+ available.
+
+ Example:
+ tcp-request content use-service lua.deny if { src -f /etc/haproxy/blacklist.lst }
+
+ Example:
+ tcp-request content set-var(sess.my_var) src
+ tcp-request content set-var-fmt(sess.from) %[src]:%[src_port]
+ tcp-request content unset-var(sess.my_var2)
+
+ Example:
+ # Accept HTTP requests containing a Host header saying "example.com"
+ # and reject everything else. (Only works for HTTP/1 connections)
+ acl is_host_com hdr(Host) -i example.com
+ tcp-request inspect-delay 30s
+ tcp-request content accept if is_host_com
+ tcp-request content reject
+
+ # Accept HTTP requests containing a Host header saying "example.com"
+ # and reject everything else. (works for HTTP/1 and HTTP/2 connections)
+ acl is_host_com hdr(Host) -i example.com
+ tcp-request inspect-delay 5s
+ tcp-request switch-mode http if HTTP
+ tcp-request reject # non-HTTP traffic is implicit here
+ ...
+ http-request reject unless is_host_com
+
+ Example:
+ # reject SMTP connection if client speaks first
+ tcp-request inspect-delay 30s
+ acl content_present req.len gt 0
+ tcp-request content reject if content_present
+
+ # Forward HTTPS connection only if client speaks
+ tcp-request inspect-delay 30s
+ acl content_present req.len gt 0
+ tcp-request content accept if content_present
+ tcp-request content reject
+
+ Example:
+ # Track the last IP(stick-table type string) from X-Forwarded-For
+ tcp-request inspect-delay 10s
+ tcp-request content track-sc0 hdr(x-forwarded-for,-1)
+ # Or track the last IP(stick-table type ip|ipv6) from X-Forwarded-For
+ tcp-request content track-sc0 req.hdr_ip(x-forwarded-for,-1)
+
+ Example:
+ # track request counts per "base" (concatenation of Host+URL)
+ tcp-request inspect-delay 10s
+ tcp-request content track-sc0 base table req-rate
+
+ Example: track per-frontend and per-backend counters, block abusers at the
+ frontend when the backend detects abuse(and marks gpc0).
+
+ frontend http
+ # Use General Purpose Counter 0 in SC0 as a global abuse counter
+ # protecting all our sites
+ stick-table type ip size 1m expire 5m store gpc0
+ tcp-request connection track-sc0 src
+ tcp-request connection reject if { sc0_get_gpc0 gt 0 }
+ ...
+ use_backend http_dynamic if { path_end .php }
+
+ backend http_dynamic
+ # if a source makes too fast requests to this dynamic site (tracked
+ # by SC1), block it globally in the frontend.
+ stick-table type ip size 1m expire 5m store http_req_rate(10s)
+ acl click_too_fast sc1_http_req_rate gt 10
+ acl mark_as_abuser sc0_inc_gpc0(http) gt 0
+ tcp-request content track-sc1 src
+ tcp-request content reject if click_too_fast mark_as_abuser
+
+ See section 7 about ACL usage.
+
+ See also : "tcp-request connection", "tcp-request session",
+ "tcp-request inspect-delay", and "http-request".
+
+tcp-request inspect-delay <timeout>
+ Set the maximum allowed time to wait for data during content inspection
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes(!) | yes | yes | yes
+
+ Arguments :
+ <timeout> is the timeout value specified in milliseconds by default, but
+ can be in any other unit if the number is suffixed by the unit,
+ as explained at the top of this document.
+
+ People using HAProxy primarily as a TCP relay are often worried about the
+ risk of passing any type of protocol to a server without any analysis. In
+ order to be able to analyze the request contents, we must first withhold
+ the data then analyze them. This statement simply enables withholding of
+ data for at most the specified amount of time.
+
+ TCP content inspection applies very early when a connection reaches a
+ frontend, then very early when the connection is forwarded to a backend. This
+ means that a connection may experience a first delay in the frontend and a
+ second delay in the backend if both have tcp-request rules.
+
+ Note that when performing content inspection, HAProxy will evaluate the whole
+ rules for every new chunk which gets in, taking into account the fact that
+ those data are partial. If no rule matches before the aforementioned delay,
+ a last check is performed upon expiration, this time considering that the
+ contents are definitive. If no delay is set, HAProxy will not wait at all
+ and will immediately apply a verdict based on the available information.
+ Obviously this is unlikely to be very useful and might even be racy, so such
+ setups are not recommended.
+
+ Note the inspection delay is shortened if an connection error or shutdown is
+ experienced or if the request buffer appears as full.
+
+ As soon as a rule matches, the request is released and continues as usual. If
+ the timeout is reached and no rule matches, the default policy will be to let
+ it pass through unaffected.
+
+ For most protocols, it is enough to set it to a few seconds, as most clients
+ send the full request immediately upon connection. Add 3 or more seconds to
+ cover TCP retransmits but that's all. For some protocols, it may make sense
+ to use large values, for instance to ensure that the client never talks
+ before the server (e.g. SMTP), or to wait for a client to talk before passing
+ data to the server (e.g. SSL). Note that the client timeout must cover at
+ least the inspection delay, otherwise it will expire first. If the client
+ closes the connection or if the buffer is full, the delay immediately expires
+ since the contents will not be able to change anymore.
+
+ This directive is only available from named defaults sections, not anonymous
+ ones. Proxies inherit this value from their defaults section.
+
+ See also : "tcp-request content accept", "tcp-request content reject",
+ "timeout client".
+
+
+tcp-request session <action> [{if | unless} <condition>]
+ Perform an action on a validated session depending on a layer 5 condition
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes(!) | yes | yes | no
+
+ Arguments :
+ <action> defines the action to perform if the condition applies. See
+ below.
+
+ <condition> is a standard layer5-only ACL-based condition (see section 7).
+
+ Once a session is validated, (i.e. after all handshakes have been completed),
+ it is possible to evaluate some conditions to decide whether this session
+ must be accepted or dropped or have its counters tracked. Those conditions
+ cannot make use of any data contents because no buffers are allocated yet and
+ the processing cannot wait at this stage. The main use case is to copy some
+ early information into variables (since variables are accessible in the
+ session), or to keep track of some information collected after the handshake,
+ such as SSL-level elements (SNI, ciphers, client cert's CN) or information
+ from the PROXY protocol header (e.g. track a source forwarded this way). The
+ extracted information can thus be copied to a variable or tracked using
+ "track-sc" rules. Of course it is also possible to decide to accept/reject as
+ with other rulesets. Most operations performed here could also be performed
+ in "tcp-request content" rules, except that in HTTP these rules are evaluated
+ for each new request, and that might not always be acceptable. For example a
+ rule might increment a counter on each evaluation. It would also be possible
+ that a country is resolved by geolocation from the source IP address,
+ assigned to a session-wide variable, then the source address rewritten from
+ an HTTP header for all requests. If some contents need to be inspected in
+ order to take the decision, the "tcp-request content" statements must be used
+ instead.
+
+ The "tcp-request session" rules are evaluated in their exact declaration
+ order. If no rule matches or if there is no rule, the default action is to
+ accept the incoming session. There is no specific limit to the number of
+ rules which may be inserted.
+
+ The first keyword after "tcp-request session" in the syntax is the rule's
+ action, optionally followed by a varying number of arguments for the action.
+ The supported actions and their respective syntaxes are enumerated in
+ section 4.3 "Actions" (look for actions which tick "TCP RqSes").
+
+ This directive is only available from named defaults sections, not anonymous
+ ones. Rules defined in the defaults section are evaluated before ones in the
+ associated proxy section. To avoid ambiguities, in this case the same
+ defaults section cannot be used by proxies with the frontend capability and
+ by proxies with the backend capability. It means a listen section cannot use
+ a defaults section defining such rules.
+
+ Note that the "if/unless" condition is optional. If no condition is set on
+ the action, it is simply performed unconditionally. That can be useful for
+ "track-sc*" actions as well as for changing the default action to a reject.
+
+ Example: track the original source address by default, or the one advertised
+ in the PROXY protocol header for connection coming from the local
+ proxies. The first connection-level rule enables receipt of the
+ PROXY protocol for these ones, the second rule tracks whatever
+ address we decide to keep after optional decoding.
+
+ tcp-request connection expect-proxy layer4 if { src -f proxies.lst }
+ tcp-request session track-sc0 src
+
+ Example: accept all sessions from white-listed hosts, reject too fast
+ sessions without counting them, and track accepted sessions.
+ This results in session rate being capped from abusive sources.
+
+ tcp-request session accept if { src -f /etc/haproxy/whitelist.lst }
+ tcp-request session reject if { src_sess_rate gt 10 }
+ tcp-request session track-sc0 src
+
+ Example: accept all sessions from white-listed hosts, count all other
+ sessions and reject too fast ones. This results in abusive ones
+ being blocked as long as they don't slow down.
+
+ tcp-request session accept if { src -f /etc/haproxy/whitelist.lst }
+ tcp-request session track-sc0 src
+ tcp-request session reject if { sc0_sess_rate gt 10 }
+
+ See section 7 about ACL usage.
+
+ See also : "tcp-request connection", "tcp-request content", "stick-table"
+
+tcp-response content <action> [{if | unless} <condition>]
+ Perform an action on a session response depending on a layer 4-7 condition
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes(!) | no | yes | yes
+
+ Arguments :
+ <action> defines the action to perform if the condition applies. See
+ below.
+
+ <condition> is a standard layer 4-7 ACL-based condition (see section 7).
+
+ Response contents can be analyzed at an early stage of response processing
+ called "TCP content inspection". During this stage, ACL-based rules are
+ evaluated every time the response contents are updated, until either a final
+ rule matches, or a TCP response inspection delay is set and expires with no
+ matching rule.
+
+ Most often, these decisions will consider a protocol recognition or validity.
+
+ Content-based rules are evaluated in their exact declaration order. If no
+ rule matches or if there is no rule, the default action is to accept the
+ contents. There is no specific limit to the number of rules which may be
+ inserted.
+
+ The first keyword after "tcp-response content" in the syntax is the rule's
+ action, optionally followed by a varying number of arguments for the action.
+ The supported actions and their respective syntaxes are enumerated in
+ section 4.3 "Actions" (look for actions which tick "TCP RsCnt").
+
+ This directive is only available from named defaults sections, not anonymous
+ ones. Rules defined in the defaults section are evaluated before ones in the
+ associated proxy section. To avoid ambiguities, in this case the same
+ defaults section cannot be used by proxies with the frontend capability and
+ by proxies with the backend capability. It means a listen section cannot use
+ a defaults section defining such rules.
+
+ Note that the "if/unless" condition is optional. If no condition is set on
+ the action, it is simply performed unconditionally. That can be useful for
+ for changing the default action to a reject.
+
+ Several types of actions are supported :
+
+ It is perfectly possible to match layer 7 contents with "tcp-response
+ content" rules, but then it is important to ensure that a full response has
+ been buffered, otherwise no contents will match. In order to achieve this,
+ the best solution involves detecting the HTTP protocol during the inspection
+ period.
+
+ See section 7 about ACL usage.
+
+ See also : "tcp-request content", "tcp-response inspect-delay"
+
+tcp-response inspect-delay <timeout>
+ Set the maximum allowed time to wait for a response during content inspection
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes(!) | no | yes | yes
+
+ Arguments :
+ <timeout> is the timeout value specified in milliseconds by default, but
+ can be in any other unit if the number is suffixed by the unit,
+ as explained at the top of this document.
+
+ This directive is only available from named defaults sections, not anonymous
+ ones. Proxies inherit this value from their defaults section.
+
+ See also : "tcp-response content", "tcp-request inspect-delay".
+
+
+timeout check <timeout>
+ Set additional check timeout, but only after a connection has been already
+ established.
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections: defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments:
+ <timeout> is the timeout value specified in milliseconds by default, but
+ can be in any other unit if the number is suffixed by the unit,
+ as explained at the top of this document.
+
+ If set, HAProxy uses min("timeout connect", "inter") as a connect timeout
+ for check and "timeout check" as an additional read timeout. The "min" is
+ used so that people running with *very* long "timeout connect" (e.g. those
+ who needed this due to the queue or tarpit) do not slow down their checks.
+ (Please also note that there is no valid reason to have such long connect
+ timeouts, because "timeout queue" and "timeout tarpit" can always be used to
+ avoid that).
+
+ If "timeout check" is not set HAProxy uses "inter" for complete check
+ timeout (connect + read) exactly like all <1.3.15 version.
+
+ In most cases check request is much simpler and faster to handle than normal
+ requests and people may want to kick out laggy servers so this timeout should
+ be smaller than "timeout server".
+
+ This parameter is specific to backends, but can be specified once for all in
+ "defaults" sections. This is in fact one of the easiest solutions not to
+ forget about it.
+
+ See also: "timeout connect", "timeout queue", "timeout server",
+ "timeout tarpit".
+
+
+timeout client <timeout>
+ Set the maximum inactivity time on the client side.
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments :
+ <timeout> is the timeout value specified in milliseconds by default, but
+ can be in any other unit if the number is suffixed by the unit,
+ as explained at the top of this document.
+
+ The inactivity timeout applies when the client is expected to acknowledge or
+ send data. In HTTP mode, this timeout is particularly important to consider
+ during the first phase, when the client sends the request, and during the
+ response while it is reading data sent by the server. That said, for the
+ first phase, it is preferable to set the "timeout http-request" to better
+ protect HAProxy from Slowloris like attacks. The value is specified in
+ milliseconds by default, but can be in any other unit if the number is
+ suffixed by the unit, as specified at the top of this document. In TCP mode
+ (and to a lesser extent, in HTTP mode), it is highly recommended that the
+ client timeout remains equal to the server timeout in order to avoid complex
+ situations to debug. It is a good practice to cover one or several TCP packet
+ losses by specifying timeouts that are slightly above multiples of 3 seconds
+ (e.g. 4 or 5 seconds). If some long-lived streams are mixed with short-lived
+ streams (e.g. WebSocket and HTTP), it's worth considering "timeout tunnel",
+ which overrides "timeout client" and "timeout server" for tunnels, as well as
+ "timeout client-fin" for half-closed connections.
+
+ This parameter is specific to frontends, but can be specified once for all in
+ "defaults" sections. This is in fact one of the easiest solutions not to
+ forget about it. An unspecified timeout results in an infinite timeout, which
+ is not recommended. Such a usage is accepted and works but reports a warning
+ during startup because it may result in accumulation of expired sessions in
+ the system if the system's timeouts are not configured either.
+
+ See also : "timeout server", "timeout tunnel", "timeout http-request".
+
+
+timeout client-fin <timeout>
+ Set the inactivity timeout on the client side for half-closed connections.
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments :
+ <timeout> is the timeout value specified in milliseconds by default, but
+ can be in any other unit if the number is suffixed by the unit,
+ as explained at the top of this document.
+
+ The inactivity timeout applies when the client is expected to acknowledge or
+ send data while one direction is already shut down. This timeout is different
+ from "timeout client" in that it only applies to connections which are closed
+ in one direction. This is particularly useful to avoid keeping connections in
+ FIN_WAIT state for too long when clients do not disconnect cleanly. This
+ problem is particularly common long connections such as RDP or WebSocket.
+ Note that this timeout can override "timeout tunnel" when a connection shuts
+ down in one direction. It is applied to idle HTTP/2 connections once a GOAWAY
+ frame was sent, often indicating an expectation that the connection quickly
+ ends.
+
+ This parameter is specific to frontends, but can be specified once for all in
+ "defaults" sections. By default it is not set, so half-closed connections
+ will use the other timeouts (timeout.client or timeout.tunnel).
+
+ See also : "timeout client", "timeout server-fin", and "timeout tunnel".
+
+
+timeout client-hs <timeout>
+ Set the maximum time to wait for a client TLS handshake to complete. This is
+ usable both for TCP and QUIC connections.
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments :
+ <timeout> is the timeout value specified in milliseconds by default, but
+ can be in any other unit if the number is suffixed by the unit,
+ as explained at the top of this document.
+
+ If this handshake timeout is not set, this is the client timeout which is used
+ in place.
+
+
+timeout connect <timeout>
+ Set the maximum time to wait for a connection attempt to a server to succeed.
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <timeout> is the timeout value specified in milliseconds by default, but
+ can be in any other unit if the number is suffixed by the unit,
+ as explained at the top of this document.
+
+ If the server is located on the same LAN as HAProxy, the connection should be
+ immediate (less than a few milliseconds). Anyway, it is a good practice to
+ cover one or several TCP packet losses by specifying timeouts that are
+ slightly above multiples of 3 seconds (e.g. 4 or 5 seconds). By default, the
+ connect timeout also presets both queue and tarpit timeouts to the same value
+ if these have not been specified.
+
+ This parameter is specific to backends, but can be specified once for all in
+ "defaults" sections. This is in fact one of the easiest solutions not to
+ forget about it. An unspecified timeout results in an infinite timeout, which
+ is not recommended. Such a usage is accepted and works but reports a warning
+ during startup because it may result in accumulation of failed sessions in
+ the system if the system's timeouts are not configured either.
+
+ See also: "timeout check", "timeout queue", "timeout server", "timeout tarpit".
+
+
+timeout http-keep-alive <timeout>
+ Set the maximum allowed time to wait for a new HTTP request to appear
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments :
+ <timeout> is the timeout value specified in milliseconds by default, but
+ can be in any other unit if the number is suffixed by the unit,
+ as explained at the top of this document.
+
+ By default, the time to wait for a new request in case of keep-alive is set
+ by "timeout http-request". However this is not always convenient because some
+ people want very short keep-alive timeouts in order to release connections
+ faster, and others prefer to have larger ones but still have short timeouts
+ once the request has started to present itself.
+
+ The "http-keep-alive" timeout covers these needs. It will define how long to
+ wait for a new HTTP request to start coming after a response was sent. Once
+ the first byte of request has been seen, the "http-request" timeout is used
+ to wait for the complete request to come. Note that empty lines prior to a
+ new request do not refresh the timeout and are not counted as a new request.
+
+ There is also another difference between the two timeouts : when a connection
+ expires during timeout http-keep-alive, no error is returned, the connection
+ just closes. If the connection expires in "http-request" while waiting for a
+ connection to complete, a HTTP 408 error is returned.
+
+ In general it is optimal to set this value to a few tens to hundreds of
+ milliseconds, to allow users to fetch all objects of a page at once but
+ without waiting for further clicks. Also, if set to a very small value (e.g.
+ 1 millisecond) it will probably only accept pipelined requests but not the
+ non-pipelined ones. It may be a nice trade-off for very large sites running
+ with tens to hundreds of thousands of clients.
+
+ If this parameter is not set, the "http-request" timeout applies, and if both
+ are not set, "timeout client" still applies at the lower level. It should be
+ set in the frontend to take effect, unless the frontend is in TCP mode, in
+ which case the HTTP backend's timeout will be used.
+
+ See also : "timeout http-request", "timeout client".
+
+
+timeout http-request <timeout>
+ Set the maximum allowed time to wait for a complete HTTP request
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments :
+ <timeout> is the timeout value specified in milliseconds by default, but
+ can be in any other unit if the number is suffixed by the unit,
+ as explained at the top of this document.
+
+ In order to offer DoS protection, it may be required to lower the maximum
+ accepted time to receive a complete HTTP request without affecting the client
+ timeout. This helps protecting against established connections on which
+ nothing is sent. The client timeout cannot offer a good protection against
+ this abuse because it is an inactivity timeout, which means that if the
+ attacker sends one character every now and then, the timeout will not
+ trigger. With the HTTP request timeout, no matter what speed the client
+ types, the request will be aborted if it does not complete in time. When the
+ timeout expires, an HTTP 408 response is sent to the client to inform it
+ about the problem, and the connection is closed. The logs will report
+ termination codes "cR". Some recent browsers are having problems with this
+ standard, well-documented behavior, so it might be needed to hide the 408
+ code using "option http-ignore-probes" or "errorfile 408 /dev/null". See
+ more details in the explanations of the "cR" termination code in section 8.5.
+
+ By default, this timeout only applies to the header part of the request,
+ and not to any data. As soon as the empty line is received, this timeout is
+ not used anymore. When combined with "option http-buffer-request", this
+ timeout also applies to the body of the request..
+ It is used again on keep-alive connections to wait for a second
+ request if "timeout http-keep-alive" is not set.
+
+ Generally it is enough to set it to a few seconds, as most clients send the
+ full request immediately upon connection. Add 3 or more seconds to cover TCP
+ retransmits but that's all. Setting it to very low values (e.g. 50 ms) will
+ generally work on local networks as long as there are no packet losses. This
+ will prevent people from sending bare HTTP requests using telnet.
+
+ If this parameter is not set, the client timeout still applies between each
+ chunk of the incoming request. It should be set in the frontend to take
+ effect, unless the frontend is in TCP mode, in which case the HTTP backend's
+ timeout will be used.
+
+ See also : "errorfile", "http-ignore-probes", "timeout http-keep-alive", and
+ "timeout client", "option http-buffer-request".
+
+
+timeout queue <timeout>
+ Set the maximum time to wait in the queue for a connection slot to be free
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <timeout> is the timeout value specified in milliseconds by default, but
+ can be in any other unit if the number is suffixed by the unit,
+ as explained at the top of this document.
+
+ When a server's maxconn is reached, connections are left pending in a queue
+ which may be server-specific or global to the backend. In order not to wait
+ indefinitely, a timeout is applied to requests pending in the queue. If the
+ timeout is reached, it is considered that the request will almost never be
+ served, so it is dropped and a 503 error is returned to the client.
+
+ The "timeout queue" statement allows to fix the maximum time for a request to
+ be left pending in a queue. If unspecified, the same value as the backend's
+ connection timeout ("timeout connect") is used, for backwards compatibility
+ with older versions with no "timeout queue" parameter.
+
+ See also : "timeout connect".
+
+
+timeout server <timeout>
+ Set the maximum inactivity time on the server side.
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <timeout> is the timeout value specified in milliseconds by default, but
+ can be in any other unit if the number is suffixed by the unit,
+ as explained at the top of this document.
+
+ The inactivity timeout applies when the server is expected to acknowledge or
+ send data. In HTTP mode, this timeout is particularly important to consider
+ during the first phase of the server's response, when it has to send the
+ headers, as it directly represents the server's processing time for the
+ request. To find out what value to put there, it's often good to start with
+ what would be considered as unacceptable response times, then check the logs
+ to observe the response time distribution, and adjust the value accordingly.
+
+ The value is specified in milliseconds by default, but can be in any other
+ unit if the number is suffixed by the unit, as specified at the top of this
+ document. In TCP mode (and to a lesser extent, in HTTP mode), it is highly
+ recommended that the client timeout remains equal to the server timeout in
+ order to avoid complex situations to debug. Whatever the expected server
+ response times, it is a good practice to cover at least one or several TCP
+ packet losses by specifying timeouts that are slightly above multiples of 3
+ seconds (e.g. 4 or 5 seconds minimum). If some long-lived streams are mixed
+ with short-lived streams (e.g. WebSocket and HTTP), it's worth considering
+ "timeout tunnel", which overrides "timeout client" and "timeout server" for
+ tunnels.
+
+ This parameter is specific to backends, but can be specified once for all in
+ "defaults" sections. This is in fact one of the easiest solutions not to
+ forget about it. An unspecified timeout results in an infinite timeout, which
+ is not recommended. Such a usage is accepted and works but reports a warning
+ during startup because it may result in accumulation of expired sessions in
+ the system if the system's timeouts are not configured either.
+
+ See also : "timeout client" and "timeout tunnel".
+
+
+timeout server-fin <timeout>
+ Set the inactivity timeout on the server side for half-closed connections.
+
+ May be used in the following contexts: tcp, http, log
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <timeout> is the timeout value specified in milliseconds by default, but
+ can be in any other unit if the number is suffixed by the unit,
+ as explained at the top of this document.
+
+ The inactivity timeout applies when the server is expected to acknowledge or
+ send data while one direction is already shut down. This timeout is different
+ from "timeout server" in that it only applies to connections which are closed
+ in one direction. This is particularly useful to avoid keeping connections in
+ FIN_WAIT state for too long when a remote server does not disconnect cleanly.
+ This problem is particularly common long connections such as RDP or WebSocket.
+ Note that this timeout can override "timeout tunnel" when a connection shuts
+ down in one direction. This setting was provided for completeness, but in most
+ situations, it should not be needed.
+
+ This parameter is specific to backends, but can be specified once for all in
+ "defaults" sections. By default it is not set, so half-closed connections
+ will use the other timeouts (timeout.server or timeout.tunnel).
+
+ See also : "timeout client-fin", "timeout server", and "timeout tunnel".
+
+
+timeout tarpit <timeout>
+ Set the duration for which tarpitted connections will be maintained
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | yes
+
+ Arguments :
+ <timeout> is the tarpit duration specified in milliseconds by default, but
+ can be in any other unit if the number is suffixed by the unit,
+ as explained at the top of this document.
+
+ When a connection is tarpitted using "http-request tarpit", it is maintained
+ open with no activity for a certain amount of time, then closed. "timeout
+ tarpit" defines how long it will be maintained open.
+
+ The value is specified in milliseconds by default, but can be in any other
+ unit if the number is suffixed by the unit, as specified at the top of this
+ document. If unspecified, the same value as the backend's connection timeout
+ ("timeout connect") is used, for backwards compatibility with older versions
+ with no "timeout tarpit" parameter.
+
+ See also : "timeout connect".
+
+
+timeout tunnel <timeout>
+ Set the maximum inactivity time on the client and server side for tunnels.
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments :
+ <timeout> is the timeout value specified in milliseconds by default, but
+ can be in any other unit if the number is suffixed by the unit,
+ as explained at the top of this document.
+
+ The tunnel timeout applies when a bidirectional connection is established
+ between a client and a server, and the connection remains inactive in both
+ directions. This timeout supersedes both the client and server timeouts once
+ the connection becomes a tunnel. In TCP, this timeout is used as soon as no
+ analyzer remains attached to either connection (e.g. tcp content rules are
+ accepted). In HTTP, this timeout is used when a connection is upgraded (e.g.
+ when switching to the WebSocket protocol, or forwarding a CONNECT request
+ to a proxy), or after the first response when no keepalive/close option is
+ specified.
+
+ Since this timeout is usually used in conjunction with long-lived connections,
+ it usually is a good idea to also set "timeout client-fin" to handle the
+ situation where a client suddenly disappears from the net and does not
+ acknowledge a close, or sends a shutdown and does not acknowledge pending
+ data anymore. This can happen in lossy networks where firewalls are present,
+ and is detected by the presence of large amounts of sessions in a FIN_WAIT
+ state.
+
+ The value is specified in milliseconds by default, but can be in any other
+ unit if the number is suffixed by the unit, as specified at the top of this
+ document. Whatever the expected normal idle time, it is a good practice to
+ cover at least one or several TCP packet losses by specifying timeouts that
+ are slightly above multiples of 3 seconds (e.g. 4 or 5 seconds minimum).
+
+ This parameter is specific to backends, but can be specified once for all in
+ "defaults" sections. This is in fact one of the easiest solutions not to
+ forget about it.
+
+ Example :
+ defaults http
+ option http-server-close
+ timeout connect 5s
+ timeout client 30s
+ timeout client-fin 30s
+ timeout server 30s
+ timeout tunnel 1h # timeout to use with WebSocket and CONNECT
+
+ See also : "timeout client", "timeout client-fin", "timeout server".
+
+
+transparent (deprecated)
+ Enable client-side transparent proxying
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | no | yes | yes
+
+ Arguments : none
+
+ This keyword was introduced in order to provide layer 7 persistence to layer
+ 3 load balancers. The idea is to use the OS's ability to redirect an incoming
+ connection for a remote address to a local process (here HAProxy), and let
+ this process know what address was initially requested. When this option is
+ used, sessions without cookies will be forwarded to the original destination
+ IP address of the incoming request (which should match that of another
+ equipment), while requests with cookies will still be forwarded to the
+ appropriate server.
+
+ The "transparent" keyword is deprecated, use "option transparent" instead.
+
+ Note that contrary to a common belief, this option does NOT make HAProxy
+ present the client's IP to the server when establishing the connection.
+
+ See also: "option transparent"
+
+unique-id-format <string>
+ Generate a unique ID for each request.
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments :
+ <string> is a log-format string.
+
+ This keyword creates a ID for each request using the custom log format. A
+ unique ID is useful to trace a request passing through many components of
+ a complex infrastructure. The newly created ID may also be logged using the
+ %ID tag the log-format string.
+
+ The format should be composed from elements that are guaranteed to be
+ unique when combined together. For instance, if multiple HAProxy instances
+ are involved, it might be important to include the node name. It is often
+ needed to log the incoming connection's source and destination addresses
+ and ports. Note that since multiple requests may be performed over the same
+ connection, including a request counter may help differentiate them.
+ Similarly, a timestamp may protect against a rollover of the counter.
+ Logging the process ID will avoid collisions after a service restart.
+
+ It is recommended to use hexadecimal notation for many fields since it
+ makes them more compact and saves space in logs.
+
+ Example:
+
+ unique-id-format %{+X}o\ %ci:%cp_%fi:%fp_%Ts_%rt:%pid
+
+ will generate:
+
+ 7F000001:8296_7F00001E:1F90_4F7B0A69_0003:790A
+
+ See also: "unique-id-header"
+
+unique-id-header <name>
+ Add a unique ID header in the HTTP request.
+
+ May be used in the following contexts: http
+
+ May be used in sections : defaults | frontend | listen | backend
+ yes | yes | yes | no
+
+ Arguments :
+ <name> is the name of the header.
+
+ Add a unique-id header in the HTTP request sent to the server, using the
+ unique-id-format. It can't work if the unique-id-format doesn't exist.
+
+ Example:
+
+ unique-id-format %{+X}o\ %ci:%cp_%fi:%fp_%Ts_%rt:%pid
+ unique-id-header X-Unique-ID
+
+ will generate:
+
+ X-Unique-ID: 7F000001:8296_7F00001E:1F90_4F7B0A69_0003:790A
+
+ See also: "unique-id-format"
+
+use_backend <backend> [{if | unless} <condition>]
+ Switch to a specific backend if/unless an ACL-based condition is matched.
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ no | yes | yes | no
+
+ Arguments :
+ <backend> is the name of a valid backend or "listen" section, or a
+ "log-format" string resolving to a backend name.
+
+ <condition> is a condition composed of ACLs, as described in section 7. If
+ it is omitted, the rule is unconditionally applied.
+
+ When doing content-switching, connections arrive on a frontend and are then
+ dispatched to various backends depending on a number of conditions. The
+ relation between the conditions and the backends is described with the
+ "use_backend" keyword. While it is normally used with HTTP processing, it can
+ also be used in pure TCP, either without content using stateless ACLs (e.g.
+ source address validation) or combined with a "tcp-request" rule to wait for
+ some payload.
+
+ There may be as many "use_backend" rules as desired. All of these rules are
+ evaluated in their declaration order, and the first one which matches will
+ assign the backend.
+
+ In the first form, the backend will be used if the condition is met. In the
+ second form, the backend will be used if the condition is not met. If no
+ condition is valid, the backend defined with "default_backend" will be used.
+ If no default backend is defined, either the servers in the same section are
+ used (in case of a "listen" section) or, in case of a frontend, no server is
+ used and a 503 service unavailable response is returned.
+
+ Note that it is possible to switch from a TCP frontend to an HTTP backend. In
+ this case, either the frontend has already checked that the protocol is HTTP,
+ and backend processing will immediately follow, or the backend will wait for
+ a complete HTTP request to get in. This feature is useful when a frontend
+ must decode several protocols on a unique port, one of them being HTTP.
+
+ When <backend> is a simple name, it is resolved at configuration time, and an
+ error is reported if the specified backend does not exist. If <backend> is
+ a log-format string instead, no check may be done at configuration time, so
+ the backend name is resolved dynamically at run time. If the resulting
+ backend name does not correspond to any valid backend, no other rule is
+ evaluated, and the default_backend directive is applied instead. Note that
+ when using dynamic backend names, it is highly recommended to use a prefix
+ that no other backend uses in order to ensure that an unauthorized backend
+ cannot be forced from the request.
+
+ It is worth mentioning that "use_backend" rules with an explicit name are
+ used to detect the association between frontends and backends to compute the
+ backend's "fullconn" setting. This cannot be done for dynamic names.
+
+ See also: "default_backend", "tcp-request", "fullconn", "log-format", and
+ section 7 about ACLs.
+
+use-fcgi-app <name>
+ Defines the FastCGI application to use for the backend.
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ no | no | yes | yes
+
+ Arguments :
+ <name> is the name of the FastCGI application to use.
+
+ See section 10.1 about FastCGI application setup for details.
+
+use-server <server> if <condition>
+use-server <server> unless <condition>
+ Only use a specific server if/unless an ACL-based condition is matched.
+
+ May be used in the following contexts: tcp, http
+
+ May be used in sections : defaults | frontend | listen | backend
+ no | no | yes | yes
+
+ Arguments :
+ <server> is the name of a valid server in the same backend section
+ or a "log-format" string resolving to a server name.
+
+ <condition> is a condition composed of ACLs, as described in section 7.
+
+ By default, connections which arrive to a backend are load-balanced across
+ the available servers according to the configured algorithm, unless a
+ persistence mechanism such as a cookie is used and found in the request.
+
+ Sometimes it is desirable to forward a particular request to a specific
+ server without having to declare a dedicated backend for this server. This
+ can be achieved using the "use-server" rules. These rules are evaluated after
+ the "redirect" rules and before evaluating cookies, and they have precedence
+ on them. There may be as many "use-server" rules as desired. All of these
+ rules are evaluated in their declaration order, and the first one which
+ matches will assign the server.
+
+ If a rule designates a server which is down, and "option persist" is not used
+ and no force-persist rule was validated, it is ignored and evaluation goes on
+ with the next rules until one matches.
+
+ In the first form, the server will be used if the condition is met. In the
+ second form, the server will be used if the condition is not met. If no
+ condition is valid, the processing continues and the server will be assigned
+ according to other persistence mechanisms.
+
+ Note that even if a rule is matched, cookie processing is still performed but
+ does not assign the server. This allows prefixed cookies to have their prefix
+ stripped.
+
+ The "use-server" statement works both in HTTP and TCP mode. This makes it
+ suitable for use with content-based inspection. For instance, a server could
+ be selected in a farm according to the TLS SNI field when using protocols with
+ implicit TLS (also see "req.ssl_sni"). And if these servers have their weight
+ set to zero, they will not be used for other traffic.
+
+ Example :
+ # intercept incoming TLS requests based on the SNI field
+ use-server www if { req.ssl_sni -i www.example.com }
+ server www 192.168.0.1:443 weight 0
+ use-server mail if { req.ssl_sni -i mail.example.com }
+ server mail 192.168.0.1:465 weight 0
+ use-server imap if { req.ssl_sni -i imap.example.com }
+ server imap 192.168.0.1:993 weight 0
+ # all the rest is forwarded to this server
+ server default 192.168.0.2:443 check
+
+ When <server> is a simple name, it is checked against existing servers in the
+ configuration and an error is reported if the specified server does not exist.
+ If it is a log-format, no check is performed when parsing the configuration,
+ and if we can't resolve a valid server name at runtime but the use-server rule
+ was conditioned by an ACL returning true, no other use-server rule is applied
+ and we fall back to load balancing.
+
+ See also: "use_backend", section 5 about server and section 7 about ACLs.
+
+
+4.3. Actions keywords matrix
+----------------------------
+
+Several rule sets are evaluated at various stages of the request or response
+processing, and for each rule found in these rule sets, an action may be
+executed if the optional condition is met.
+
+A large number of actions are provided by default, they can modify contents,
+accept/block processing, change internal states etc. And it is possible to
+define new actions in Lua (in which case their names will always be prefixed
+with "lua.").
+
+While historically some actions did only exist in specific rule sets, nowadays
+many actions are usable with many rule sets. The listing in this section will
+indicate for which supported action where it may be used, by ticking the
+corresponding abbreviated entry names among the following rule sets:
+
+ - TCP RqCon: the action is valid for "tcp-request connection" rules
+ - TCP RqSes: the action is valid for "tcp-request session" rules
+ - TCP RqCnt: the action is valid for "tcp-request content" rules
+ - TCP RsCnt: the action is valid for "tcp-response content" rules
+ - HTTP Req: the action is valid for "http-request" rules
+ - HTTP Res: the action is valid for "http-response" rules
+ - HTTP Aft: the action is valid for "http-after-response" rules
+
+The same abbreviations are used in the reference section 4.4 below.
+
+
+ keyword TCP: RqCon RqSes RqCnt RsCnt HTTP: Req Res Aft
+----------------------+-----------+-----+-----+------+----------+---+----
+accept X X X X - - -
+add-acl - - - - X X -
+add-header - - - - X X X
+allow - - - - X X X
+attach-srv - X - - - - -
+auth - - - - X - -
+cache-store - - - - - X -
+cache-use - - - - X - -
+capture - - X - X X X
+close - - - X - - -
+del-acl - - - - X X -
+del-header - - - - X X X
+del-map - - - - X X X
+deny - - - - X X -
+disable-l7-retry - - - - X - -
+do-resolve - - X - X - -
+early-hint - - - - X - -
+expect-netscaler-cip X - - - - - -
+expect-proxy layer4 X - - - - - -
+normalize-uri - - - - X - -
+redirect - - - - X X -
+reject X X X X X - -
+replace-header - - - - X X X
+replace-path - - - - X - -
+replace-pathq - - - - X - -
+replace-uri - - - - X - -
+replace-value - - - - X X X
+return - - - - X X -
+sc-add-gpc X X X X X X X
+--keyword---------------TCP--RqCon-RqSes-RqCnt-RsCnt---HTTP--Req-Res-Aft-
+sc-inc-gpc X X X X X X X
+sc-inc-gpc0 X X X X X X X
+sc-inc-gpc1 X X X X X X X
+sc-set-gpt X X X X X X X
+sc-set-gpt0 X X X X X X X
+send-spoe-group - - X X X X -
+set-bandwidth-limit - - X X X X -
+set-dst X X X - X - -
+set-dst-port X X X - X - -
+set-header - - - - X X X
+set-log-level - - X X X X X
+set-map - - - - X X X
+set-mark X X X X X X -
+set-method - - - - X - -
+set-nice - - X X X X -
+set-path - - - - X - -
+set-pathq - - - - X - -
+set-priority-class - - X - X - -
+set-priority-offset - - X - X - -
+--keyword---------------TCP--RqCon-RqSes-RqCnt-RsCnt---HTTP--Req-Res-Aft-
+set-query - - - - X - -
+set-src X X X - X - -
+set-src-port X X X - X - -
+set-status - - - - - X X
+set-timeout - - - - X X -
+set-tos X X X X X X -
+set-uri - - - - X - -
+set-var X X X X X X X
+set-var-fmt X X X X X X X
+silent-drop X X X X X X -
+strict-mode - - - - X X X
+switch-mode - - X - - - -
+tarpit - - - - X - -
+track-sc1 X X X - X X -
+track-sc2 X X X - X X -
+unset-var X X X X X X X
+use-service - - X - X - -
+wait-for-body - - - - X X -
+wait-for-handshake - - - - X - -
+--keyword---------------TCP--RqCon-RqSes-RqCnt-RsCnt---HTTP--Req-Res-Aft-
+
+
+4.4. Alphabetically sorted actions reference
+--------------------------------------------
+
+This section provides a detailed description of each action and its usage,
+using the same ruleset terminology marking as described in section 4.3 above.
+
+
+accept
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt | HTTP Req| Res| Aft
+ X | X | X | X | - | - | -
+
+ This stops the evaluation of the rules and lets the request or response pass
+ the check. This action is final, i.e. no further rules from the same rule set
+ are evaluated for the current section. There is no difference between this
+ and the "allow" action except that for historical compatibility, "accept" is
+ used for TCP rules and "allow" for HTTP rules. See also the "allow" action
+ below.
+
+
+add-acl(<file-name>) <key fmt>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | X | -
+
+ This is used to add a new entry into an ACL. The ACL must be loaded from a
+ file (even a dummy empty file). The file name of the ACL to be updated is
+ passed between parentheses. It takes one argument: <key fmt>, which follows
+ log-format rules, to collect content of the new entry. It performs a lookup
+ in the ACL before insertion, to avoid duplicated (or more) values.
+ It is the equivalent of the "add acl" command from the stats socket, but can
+ be triggered by an HTTP request.
+
+
+add-header <name> <fmt>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | X | X
+
+ This appends an HTTP header field whose name is specified in <name> and
+ whose value is defined by <fmt> which follows the log-format rules (see
+ Custom Log Format in section 8.2.4). This is particularly useful to pass
+ connection-specific information to the server (e.g. the client's SSL
+ certificate), or to combine several headers into one. This rule is not
+ final, so it is possible to add other similar rules. Note that header
+ addition is performed immediately, so one rule might reuse the resulting
+ header from a previous rule.
+
+
+allow
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | X | X
+
+ This stops the evaluation of the rules and lets the request pass the check.
+ This action is final, i.e. no further rules from the same rule set are
+ evaluated for the current section. There is no difference between this and
+ the "accept" action except that for historical compatibility, "accept" is
+ used for TCP rules and "allow" for HTTP rules. See also the "accept" action
+ above.
+
+
+attach-srv <srv> [name <expr>] [ EXPERIMENTAL ]
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | X | - | - | - | - | -
+
+ This is used to intercept the connection after proper HTTP/2 establishment.
+ The connection is reversed to the backend side and inserted into the idle
+ pool of server <srv>. This may only be used with servers having an 'rhttp@'
+ address.
+
+ An extra parameter <expr> can be specified. Its value is interpreted as a
+ sample expression to name the connection inside the server idle pool. When
+ routing an outgoing request through this server, this name will be matched
+ against the 'sni' parameter of the server line. Otherwise, the connection
+ will have no name and will only match requests without SNI.
+
+ This rule is only valid for frontend in HTTP mode. Also all listeners must
+ not require a protocol different from HTTP/2.
+
+ Reverse HTTP is currently still in active development. Configuration
+ mechanism may change in the future. For this reason it is internally marked
+ as experimental, meaning that "expose-experimental-directives" must appear on
+ a line before this directive.
+
+auth [realm <realm>]
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | - | -
+
+ This stops the evaluation of the rules and immediately responds with an
+ HTTP 401 or 407 error code to invite the user to present a valid user name
+ and password. No further "http-request" rules are evaluated. An optional
+ "realm" parameter is supported, it sets the authentication realm that is
+ returned with the response (typically the application's name).
+
+ The corresponding proxy's error message is used. It may be customized using
+ an "errorfile" or an "http-error" directive. For 401 responses, all
+ occurrences of the WWW-Authenticate header are removed and replaced by a new
+ one with a basic authentication challenge for realm "<realm>". For 407
+ responses, the same is done on the Proxy-Authenticate header. If the error
+ message must not be altered, consider to use "http-request return" rule
+ instead.
+
+ Example:
+ acl auth_ok http_auth_group(L1) G1
+ http-request auth unless auth_ok
+
+
+cache-store <name>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | - | X | -
+
+ Store an http-response within the cache. The storage of the response headers
+ is done at this step, which means you can use others http-response actions
+ to modify headers before or after the storage of the response. This action
+ is responsible for the setup of the cache storage filter.
+
+ See section 6.2 about cache setup.
+
+
+cache-use <name>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | - | -
+
+ Try to deliver a cached object from the cache <name>. This directive is also
+ mandatory to store the cache as it calculates the cache hash. If you want to
+ use a condition for both storage and delivering that's a good idea to put it
+ after this one.
+
+ See section 6.2 about cache setup.
+
+
+capture <sample> [ len <length> | id <id> ]
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | X | - | X | X | X
+
+ This captures sample expression <sample> from the request or response buffer,
+ and converts it to a string of at most <len> characters. The resulting string
+ is stored into the next "capture" slot (either request or reponse), so it
+ will possibly appear next to some captured HTTP headers. It will then
+ automatically appear in the logs, and it will be possible to extract it using
+ sample fetch methods to feed it into headers or anything. The length should
+ be limited given that this size will be allocated for each capture during the
+ whole stream life. Note that the length is only usable with "http-request"
+ rules. Please check section 7.3 (Fetching samples), "capture request header"
+ and "capture response header" for more information.
+
+ If the keyword "id" is used instead of "len", the action tries to store the
+ captured string in a previously declared capture slot. This is useful to run
+ captures in backends. The slot id can be declared by a previous directive
+ "http-request capture" or with the "declare capture" keyword.
+
+ When using this action in a backend, please double check that the relevant
+ frontend(s) have the required capture slots otherwise, this rule will be
+ ignored at run time. This can't be detected at configuration parsing time due
+ to HAProxy's ability to dynamically resolve backend name at runtime.
+
+
+close
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | X | - | - | -
+
+ This is used to immediately close the connection with the server. No further
+ "tcp-response content" rules are evaluated. The main purpose of this action
+ is to force a connection to be finished between a client and a server after
+ an exchange when the application protocol expects some long time outs to
+ elapse first. The goal is to eliminate idle connections which take
+ significant resources on servers with certain protocols.
+
+
+del-acl(<file-name>) <key fmt>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | X | -
+
+ This is used to delete an entry from an ACL. The ACL must be loaded from a
+ file (even a dummy empty file). The file name of the ACL to be updated is
+ passed between parentheses. It takes one argument: <key fmt>, which follows
+ log-format rules, to collect content of the entry to delete.
+ It is the equivalent of the "del acl" command from the stats socket, but can
+ be triggered by an HTTP request or response.
+
+
+del-header <name> [ -m <meth> ]
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | X | X
+
+ This removes all HTTP header fields whose name is specified in <name>. <meth>
+ is the matching method, applied on the header name. Supported matching methods
+ are "str" (exact match), "beg" (prefix match), "end" (suffix match), "sub"
+ (substring match) and "reg" (regex match). If not specified, exact matching
+ method is used.
+
+
+del-map(<file-name>) <key fmt>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | X | X
+
+ This is used to delete an entry from a MAP. The MAP must be loaded from a
+ file (even a dummy empty file). The file name of the MAP to be updated is
+ passed between parentheses. It takes one argument: <key fmt>, which follows
+ log-format rules, to collect content of the entry to delete.
+ It takes one argument: "file name" It is the equivalent of the "del map"
+ command from the stats socket, but can be triggered by an HTTP request or
+ response.
+
+
+deny [ { status | deny_status } <code> ] [ content-type <type> ]
+ [ { default-errorfiles | errorfile <file> | errorfiles <name> |
+ file <file> | lf-file <file> | string <str> | lf-string <fmt> } ]
+ [ hdr <name> <fmt> ]*
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | X | -
+
+ This stops the evaluation of the rules and immediately rejects the request or
+ response. By default an HTTP 403 error is returned for requests, and 502 for
+ responses, but the returned response may be customized using same syntax as
+ for the "return" action. Thus, see "return" below for details. For
+ compatibility purposes, when no argument is defined, or only "deny_status",
+ the argument "default-errorfiles" is implied. It means "deny [deny_status
+ <status>]" is an alias of "deny [status <status>] default-errorfiles". This
+ action is final, i.e. no further rules from the same rule set are evaluated
+ for the current section. See also the "return" action for the advanced
+ syntax.
+
+
+disable-l7-retry
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | - | -
+
+ This disables any attempt to retry the request if it fails for any other
+ reason than a connection failure. This can be useful for example to make
+ sure POST requests aren't retried on failure.
+
+
+do-resolve(<var>,<resolvers>,[ipv4,ipv6]) <expr>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | X | - | X | - | -
+
+ This action performs a DNS resolution of the output of <expr> and stores
+ the result in the variable <var>. It uses the DNS resolvers section
+ pointed by <resolvers>.
+ It is possible to choose a resolution preference using the optional
+ arguments 'ipv4' or 'ipv6'.
+ When performing the DNS resolution, the client side connection is on
+ pause waiting till the end of the resolution.
+ If an IP address can be found, it is stored into <var>. If any kind of
+ error occurs, then <var> is not set.
+ One can use this action to discover a server IP address at run time and
+ based on information found in the request (IE a Host header).
+ If this action is used to find the server's IP address (using the
+ "set-dst" action), then the server IP address in the backend must be set
+ to 0.0.0.0. The do-resolve action takes an host-only parameter, any port must
+ be removed from the string.
+
+ Example:
+ resolvers mydns
+ nameserver local 127.0.0.53:53
+ nameserver google 8.8.8.8:53
+ timeout retry 1s
+ hold valid 10s
+ hold nx 3s
+ hold other 3s
+ hold obsolete 0s
+ accepted_payload_size 8192
+
+ frontend fe
+ bind 10.42.0.1:80
+ http-request do-resolve(txn.myip,mydns,ipv4) hdr(Host),host_only
+ http-request capture var(txn.myip) len 40
+
+ # return 503 when the variable is not set,
+ # which mean DNS resolution error
+ use_backend b_503 unless { var(txn.myip) -m found }
+
+ default_backend be
+
+ backend b_503
+ # dummy backend used to return 503.
+ # one can use the errorfile directive to send a nice
+ # 503 error page to end users
+
+ backend be
+ # rule to prevent HAProxy from reconnecting to services
+ # on the local network (forged DNS name used to scan the network)
+ http-request deny if { var(txn.myip) -m ip 127.0.0.0/8 10.0.0.0/8 }
+ http-request set-dst var(txn.myip)
+ server clear 0.0.0.0:0
+
+ NOTE: Don't forget to set the "protection" rules to ensure HAProxy won't
+ be used to scan the network or worst won't loop over itself...
+
+
+early-hint <name> <fmt>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | - | -
+
+ This is used to build an HTTP 103 Early Hints response prior to any other one.
+ This appends an HTTP header field to this response whose name is specified in
+ <name> and whose value is defined by <fmt> which follows the log-format rules
+ (see Custom Log Format in section 8.2.4). This is particularly useful to pass
+ to the client some Link headers to preload resources required to render the
+ HTML documents.
+
+ See RFC 8297 for more information.
+
+
+expect-netscaler-cip layer4
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ X | - | - | - | - | - | -
+
+ This configures the client-facing connection to receive a NetScaler Client IP
+ insertion protocol header before any byte is read from the socket. This is
+ equivalent to having the "accept-netscaler-cip" keyword on the "bind" line,
+ except that using the TCP rule allows the PROXY protocol to be accepted only
+ for certain IP address ranges using an ACL. This is convenient when multiple
+ layers of load balancers are passed through by traffic coming from public
+ hosts.
+
+
+expect-proxy layer4
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ X | - | - | - | - | - | -
+
+ This configures the client-facing connection to receive a PROXY protocol
+ header before any byte is read from the socket. This is equivalent to having
+ the "accept-proxy" keyword on the "bind" line, except that using the TCP rule
+ allows the PROXY protocol to be accepted only for certain IP address ranges
+ using an ACL. This is convenient when multiple layers of load balancers are
+ passed through by traffic coming from public hosts.
+
+
+normalize-uri <normalizer>
+normalize-uri fragment-encode
+normalize-uri fragment-strip
+normalize-uri path-merge-slashes
+normalize-uri path-strip-dot
+normalize-uri path-strip-dotdot [ full ]
+normalize-uri percent-decode-unreserved [ strict ]
+normalize-uri percent-to-uppercase [ strict ]
+normalize-uri query-sort-by-name
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | - | -
+
+ Performs normalization of the request's URI.
+
+ URI normalization in HAProxy 2.4 is currently available as an experimental
+ technical preview. As such, it requires the global directive
+ 'expose-experimental-directives' first to be able to invoke it. You should be
+ prepared that the behavior of normalizers might change to fix possible
+ issues, possibly breaking proper request processing in your infrastructure.
+
+ Each normalizer handles a single type of normalization to allow for a
+ fine-grained selection of the level of normalization that is appropriate for
+ the supported backend.
+
+ As an example the "path-strip-dotdot" normalizer might be useful for a static
+ fileserver that directly maps the requested URI to the path within the local
+ filesystem. However it might break routing of an API that expects a specific
+ number of segments in the path.
+
+ It is important to note that some normalizers might result in unsafe
+ transformations for broken URIs. It might also be possible that a combination
+ of normalizers that are safe by themselves results in unsafe transformations
+ when improperly combined.
+
+ As an example the "percent-decode-unreserved" normalizer might result in
+ unexpected results when a broken URI includes bare percent characters. One
+ such a broken URI is "/%%36%36" which would be decoded to "/%66" which in
+ turn is equivalent to "/f". By specifying the "strict" option requests to
+ such a broken URI would safely be rejected.
+
+ The following normalizers are available:
+
+ - fragment-encode: Encodes "#" as "%23".
+
+ The "fragment-strip" normalizer should be preferred, unless it is known
+ that broken clients do not correctly encode '#' within the path component.
+
+ Example:
+ - /#foo -> /%23foo
+
+ - fragment-strip: Removes the URI's "fragment" component.
+
+ According to RFC 3986#3.5 the "fragment" component of an URI should not
+ be sent, but handled by the User Agent after retrieving a resource.
+
+ This normalizer should be applied first to ensure that the fragment is
+ not interpreted as part of the request's path component.
+
+ Example:
+ - /#foo -> /
+
+ - path-strip-dot: Removes "/./" segments within the "path" component
+ (RFC 3986#6.2.2.3).
+
+ Segments including percent encoded dots ("%2E") will not be detected. Use
+ the "percent-decode-unreserved" normalizer first if this is undesired.
+
+ Example:
+ - /. -> /
+ - /./bar/ -> /bar/
+ - /a/./a -> /a/a
+ - /.well-known/ -> /.well-known/ (no change)
+
+ - path-strip-dotdot: Normalizes "/../" segments within the "path" component
+ (RFC 3986#6.2.2.3).
+
+ This merges segments that attempt to access the parent directory with
+ their preceding segment.
+
+ Empty segments do not receive special treatment. Use the "merge-slashes"
+ normalizer first if this is undesired.
+
+ Segments including percent encoded dots ("%2E") will not be detected. Use
+ the "percent-decode-unreserved" normalizer first if this is undesired.
+
+ Example:
+ - /foo/../ -> /
+ - /foo/../bar/ -> /bar/
+ - /foo/bar/../ -> /foo/
+ - /../bar/ -> /../bar/
+ - /bar/../../ -> /../
+ - /foo//../ -> /foo/
+ - /foo/%2E%2E/ -> /foo/%2E%2E/
+
+ If the "full" option is specified then "../" at the beginning will be
+ removed as well:
+
+ Example:
+ - /../bar/ -> /bar/
+ - /bar/../../ -> /
+
+ - path-merge-slashes: Merges adjacent slashes within the "path" component
+ into a single slash.
+
+ Example:
+ - // -> /
+ - /foo//bar -> /foo/bar
+
+ - percent-decode-unreserved: Decodes unreserved percent encoded characters to
+ their representation as a regular character (RFC 3986#6.2.2.2).
+
+ The set of unreserved characters includes all letters, all digits, "-",
+ ".", "_", and "~".
+
+ Example:
+ - /%61dmin -> /admin
+ - /foo%3Fbar=baz -> /foo%3Fbar=baz (no change)
+ - /%%36%36 -> /%66 (unsafe)
+ - /%ZZ -> /%ZZ
+
+ If the "strict" option is specified then invalid sequences will result
+ in a HTTP 400 Bad Request being returned.
+
+ Example:
+ - /%%36%36 -> HTTP 400
+ - /%ZZ -> HTTP 400
+
+ - percent-to-uppercase: Uppercases letters within percent-encoded sequences
+ (RFC 3986#6.2.2.1).
+
+ Example:
+ - /%6f -> /%6F
+ - /%zz -> /%zz
+
+ If the "strict" option is specified then invalid sequences will result
+ in a HTTP 400 Bad Request being returned.
+
+ Example:
+ - /%zz -> HTTP 400
+
+ - query-sort-by-name: Sorts the query string parameters by parameter name.
+ Parameters are assumed to be delimited by '&'. Shorter names sort before
+ longer names and identical parameter names maintain their relative order.
+
+ Example:
+ - /?c=3&a=1&b=2 -> /?a=1&b=2&c=3
+ - /?aaa=3&a=1&aa=2 -> /?a=1&aa=2&aaa=3
+ - /?a=3&b=4&a=1&b=5&a=2 -> /?a=3&a=1&a=2&b=4&b=5
+
+
+redirect <rule>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | X | -
+
+ This performs an HTTP redirection based on a redirect rule. This is exactly
+ the same as the "redirect" statement except that it inserts a redirect rule
+ which is processed in the middle of other "http-request" or "http-response"
+ rules and that these rules use the "log-format" strings. For responses, only
+ the "location" type of redirect is permitted. In addition, when a redirect is
+ performed during a response, the transfer from the server to HAProxy is
+ interrupted so that no payload can be forwarded to the client. This may cause
+ some connections to be closed on HTTP/1. This action is final, i.e. no
+ further rules from the same rule set are evaluated for the current section.
+ See the "redirect" keyword for the rule's syntax.
+
+
+reject
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ X | X | X | X | X | - | -
+
+ This stops the evaluation of the rules and immediately closes the connection
+ without sending any response. For HTTP rules, it acts similarly to the
+ "tcp-request content reject" rules. It can be useful to force an immediate
+ connection closure on HTTP/2 connections.
+
+ In "tcp-request connection" rules, rejected connections do not even become a
+ session, which is why they are accounted separately for in the stats, as
+ "denied connections". They are not considered for the session rate-limit and
+ are not logged either. The reason is that these rules should only be used to
+ filter extremely high connection rates such as the ones encountered during a
+ massive DDoS attack. Under these extreme conditions, the simple action of
+ logging each event would make the system collapse and would considerably
+ lower the filtering capacity. If logging is absolutely desired, then
+ "tcp-request content" rules should be used instead, as "tcp-request session"
+ rules will not log either.
+
+ When used in "tcp-response content" rules, the server connection will be
+ closed and the response aborted. This is generally used to prevent sensitive
+ information from leaking, typically after inspecting contents in conjunction
+ with the "wait-for-body" action.
+
+
+replace-header <name> <match-regex> <replace-fmt>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | X | X
+
+ This matches the value of all occurrences of header field <name> against
+ <match-regex>. Matching is performed case-sensitively. Matching values are
+ completely replaced by <replace-fmt>. Format characters are allowed in
+ <replace-fmt> and work like <fmt> arguments in "http-request add-header".
+ Standard back-references using the backslash ('\') followed by a number are
+ supported.
+
+ This action acts on whole header lines, regardless of the number of values
+ they may contain. Thus it is well-suited to process headers naturally
+ containing commas in their value, such as If-Modified-Since or Set-Cookie.
+ Headers that contain a comma-separated list of values, such as Accept, or
+ Cache-Control should be processed using the "replace-value" action instead.
+ See also the "replace-value" action.
+
+ Example:
+ http-request replace-header Cookie foo=([^;]*);(.*) foo=\1;ip=%bi;\2
+
+ # applied to:
+ Cookie: foo=foobar; expires=Tue, 14-Jun-2016 01:40:45 GMT;
+
+ # outputs:
+ Cookie: foo=foobar;ip=192.168.1.20; expires=Tue, 14-Jun-2016 01:40:45 GMT;
+
+ # assuming the backend IP is 192.168.1.20
+
+ http-request replace-header User-Agent curl foo
+
+ # applied to:
+ User-Agent: curl/7.47.0
+
+ # outputs:
+ User-Agent: foo
+
+ Example:
+ http-response replace-header Set-Cookie (C=[^;]*);(.*) \1;ip=%bi;\2
+
+ # applied to:
+ Set-Cookie: C=1; expires=Tue, 14-Jun-2016 01:40:45 GMT
+
+ # outputs:
+ Set-Cookie: C=1;ip=192.168.1.20; expires=Tue, 14-Jun-2016 01:40:45 GMT
+
+ # assuming the backend IP is 192.168.1.20.
+
+
+replace-path <match-regex> <replace-fmt>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | - | -
+
+ This works like "replace-header" except that it works on the request's path
+ component instead of a header. The path component starts at the first '/'
+ after an optional scheme+authority and ends before the question mark. Thus,
+ the replacement does not modify the scheme, the authority and the
+ query-string.
+
+ It is worth noting that regular expressions may be more expensive to evaluate
+ than certain ACLs, so rare replacements may benefit from a condition to avoid
+ performing the evaluation at all if it does not match.
+
+ Example:
+ # prefix /foo : turn /bar?q=1 into /foo/bar?q=1 :
+ http-request replace-path (.*) /foo\1
+
+ # strip /foo : turn /foo/bar?q=1 into /bar?q=1
+ http-request replace-path /foo/(.*) /\1
+ # or more efficient if only some requests match :
+ http-request replace-path /foo/(.*) /\1 if { url_beg /foo/ }
+
+
+replace-pathq <match-regex> <replace-fmt>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | - | -
+
+ This does the same as "http-request replace-path" except that the path
+ contains the query-string if any is present. Thus, the path and the
+ query-string are replaced.
+
+ Example:
+ # suffix /foo : turn /bar?q=1 into /bar/foo?q=1 :
+ http-request replace-pathq ([^?]*)(\?(.*))? \1/foo\2
+
+
+replace-uri <match-regex> <replace-fmt>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | - | -
+
+ This works like "replace-header" except that it works on the request's URI part
+ instead of a header. The URI part may contain an optional scheme, authority or
+ query string. These are considered to be part of the value that is matched
+ against.
+
+ It is worth noting that regular expressions may be more expensive to evaluate
+ than certain ACLs, so rare replacements may benefit from a condition to avoid
+ performing the evaluation at all if it does not match.
+
+ IMPORTANT NOTE: historically in HTTP/1.x, the vast majority of requests sent
+ by browsers use the "origin form", which differs from the "absolute form" in
+ that they do not contain a scheme nor authority in the URI portion. Mostly
+ only requests sent to proxies, those forged by hand and some emitted by
+ certain applications use the absolute form. As such, "replace-uri" usually
+ works fine most of the time in HTTP/1.x with rules starting with a "/". But
+ with HTTP/2, clients are encouraged to send absolute URIs only, which look
+ like the ones HTTP/1 clients use to talk to proxies. Such partial replace-uri
+ rules may then fail in HTTP/2 when they work in HTTP/1. Either the rules need
+ to be adapted to optionally match a scheme and authority, or replace-path
+ should be used.
+
+ Example:
+ # rewrite all "http" absolute requests to "https":
+ http-request replace-uri ^http://(.*) https://\1
+
+ # prefix /foo : turn /bar?q=1 into /foo/bar?q=1 :
+ http-request replace-uri ([^/:]*://[^/]*)?(.*) \1/foo\2
+
+
+replace-value <name> <match-regex> <replace-fmt>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | X | X
+
+ This works like "replace-header" except that it matches the regex against
+ every comma-delimited value of the header field <name> instead of the
+ entire header. This is suited for all headers which are allowed to carry
+ more than one value. An example could be the Accept request header, or
+ Cache-Control for requests or responses.
+
+ Example:
+ http-request replace-value X-Forwarded-For ^192\.168\.(.*)$ 172.16.\1
+
+ # applied to:
+ X-Forwarded-For: 192.168.10.1, 192.168.13.24, 10.0.0.37
+
+ # outputs:
+ X-Forwarded-For: 172.16.10.1, 172.16.13.24, 10.0.0.37
+
+ Example:
+ http-after-response replace-value Cache-control ^public$ private
+
+ # applied to:
+ Cache-Control: max-age=3600, public
+
+ # outputs:
+ Cache-Control: max-age=3600, private
+
+
+return [ status <code> ] [ content-type <type> ]
+ [ { default-errorfiles | errorfile <file> | errorfiles <name> |
+ file <file> | lf-file <file> | string <str> | lf-string <fmt> } ]
+ [ hdr <name> <fmt> ]*
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | X | -
+
+ This stops the evaluation of the rules and immediately returns a response. The
+ default status code used for the response is 200. It can be optionally
+ specified as an arguments to "status". The response content-type may also be
+ specified as an argument to "content-type". Finally the response itself may
+ be defined. It can be a full HTTP response specifying the errorfile to use,
+ or the response payload specifying the file or the string to use. These rules
+ are followed to create the response :
+
+ * If neither the errorfile nor the payload to use is defined, a dummy
+ response is returned. Only the "status" argument is considered. It can be
+ any code in the range [200, 599]. The "content-type" argument, if any, is
+ ignored.
+
+ * If "default-errorfiles" argument is set, the proxy's errorfiles are
+ considered. If the "status" argument is defined, it must be one of the
+ status code handled by HAProxy (200, 400, 403, 404, 405, 408, 410, 413,
+ 425, 429, 500, 501, 502, 503, and 504). The "content-type" argument, if
+ any, is ignored.
+
+ * If a specific errorfile is defined, with an "errorfile" argument, the
+ corresponding file, containing a full HTTP response, is returned. Only the
+ "status" argument is considered. It must be one of the status code handled
+ by HAProxy (200, 400, 403, 404, 405, 408, 410, 413, 425, 429, 500, 501,
+ 502, 503, and 504). The "content-type" argument, if any, is ignored.
+
+ * If an http-errors section is defined, with an "errorfiles" argument, the
+ corresponding file in the specified http-errors section, containing a full
+ HTTP response, is returned. Only the "status" argument is considered. It
+ must be one of the status code handled by HAProxy (200, 400, 403, 404, 405,
+ 408, 410, 413, 425, 429, 500, 501, 502, 503, and 504). The "content-type"
+ argument, if any, is ignored.
+
+ * If a "file" or a "lf-file" argument is specified, the file's content is
+ used as the response payload. If the file is not empty, its content-type
+ must be set as argument to "content-type". Otherwise, any "content-type"
+ argument is ignored. With a "lf-file" argument, the file's content is
+ evaluated as a log-format string. With a "file" argument, it is considered
+ as a raw content.
+
+ * If a "string" or "lf-string" argument is specified, the defined string is
+ used as the response payload. The content-type must always be set as
+ argument to "content-type". With a "lf-string" argument, the string is
+ evaluated as a log-format string. With a "string" argument, it is
+ considered as a raw string.
+
+ When the response is not based on an errorfile, it is possible to append HTTP
+ header fields to the response using "hdr" arguments. Otherwise, all "hdr"
+ arguments are ignored. For each one, the header name is specified in <name>
+ and its value is defined by <fmt> which follows the log-format rules.
+
+ Note that the generated response must be smaller than a buffer. And to avoid
+ any warning, when an errorfile or a raw file is loaded, the buffer space
+ reserved for the headers rewriting should also be free.
+
+ This action is final, i.e. no further rules from the same rule set are
+ evaluated for the current section.
+
+ Example:
+ http-request return errorfile /etc/haproxy/errorfiles/200.http \
+ if { path /ping }
+
+ http-request return content-type image/x-icon file /var/www/favicon.ico \
+ if { path /favicon.ico }
+
+ http-request return status 403 content-type text/plain \
+ lf-string "Access denied. IP %[src] is blacklisted." \
+ if { src -f /etc/haproxy/blacklist.lst }
+
+
+sc-add-gpc(<idx>,<sc-id>) { <int> | <expr> }
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ X | X | X | X | X | X | X
+
+ This action increments the General Purpose Counter at the index <idx> of the
+ array associated to the sticky counter designated by <sc-id> by the value of
+ either integer <int> or the integer evaluation of expression <expr>. Integers
+ and expressions are limited to unsigned 32-bit values. If an error occurs,
+ this action silently fails and the actions evaluation continues. <idx> is an
+ integer between 0 and 99 and <sc-id> is an integer between 0 and 2. It also
+ silently fails if the there is no GPC stored at this index. The entry in the
+ table is refreshed even if the value is zero. The 'gpc_rate' is automatically
+ adjusted to reflect the average growth rate of the gpc value.
+
+ This action applies only to the 'gpc' and 'gpc_rate' array data_types (and
+ not to the legacy 'gpc0', 'gpc1', 'gpc0_rate' nor 'gpc1_rate' data_types).
+ There is no equivalent function for legacy data types, but if the value is
+ always 1, please see 'sc-inc-gpc()', 'sc-inc-gpc0()' and 'sc-inc-gpc1()'.
+ There is no way to decrement the value either, but it is possible to store
+ exact values in a General Purpose Tag using 'sc-set-gpt()' instead.
+
+ The main use of this action is to count scores or total volumes (e.g.
+ estimated danger per source IP reported by the server or a WAF, total
+ uploaded bytes, etc).
+
+
+sc-inc-gpc(<idx>,<sc-id>)
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ X | X | X | X | X | X | X
+
+ This actions increments the General Purpose Counter at the index <idx> of the
+ array associated to the sticky counter designated by <sc-id>. If an error
+ occurs, this action silently fails and the actions evaluation continues.
+ <idx> is an integer between 0 and 99 and <sc-id> is an integer between 0 and
+ 2. It also silently fails if the there is no GPC stored at this index. This
+ action applies only to the 'gpc' and 'gpc_rate' array data_types (and not to
+ the legacy 'gpc0', 'gpc1', 'gpc0_rate' nor 'gpc1_rate' data_types).
+
+
+sc-inc-gpc0(<sc-id>)
+sc-inc-gpc1(<sc-id>)
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ X | X | X | X | X | X | X
+
+ This actions increments the GPC0 or GPC1 counter according with the sticky
+ counter designated by <sc-id>. If an error occurs, this action silently fails
+ and the actions evaluation continues.
+
+
+sc-set-gpt(<idx>,<sc-id>) { <int> | <expr> }
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ X | X | X | X | X | X | X
+
+ This action sets the 32-bit unsigned GPT at the index <idx> of the array
+ associated to the sticky counter designated by <sc-id> at the value of
+ <int>/<expr>. The expected result is a boolean.
+
+ If an error occurs, this action silently fails and the actions evaluation
+ continues. <idx> is an integer between 0 and 99 and <sc-id> is an integer
+ between 0 and 2. It also silently fails if the there is no GPT stored
+ at this index.
+
+ This action applies only to the 'gpt' array data_type (and not to the
+ legacy 'gpt0' data-type).
+
+
+sc-set-gpt0(<sc-id>) { <int> | <expr> }
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ X | X | X | X | X | X | X
+
+ This action sets the 32-bit unsigned GPT0 tag according to the sticky counter
+ designated by <sc-id> and the value of <int>/<expr>. The expected result is a
+ boolean. If an error occurs, this action silently fails and the actions
+ evaluation continues. This action is an alias for "sc-set-gpt(0,<sc-id>)".
+ See also the "sc-set-gpt" action.
+
+
+send-spoe-group <engine-name> <group-name>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | X | X | X | X | -
+
+ This action is used to trigger sending of a group of SPOE messages. To do so,
+ the SPOE engine used to send messages must be defined, as well as the SPOE
+ group to send. Of course, the SPOE engine must refer to an existing SPOE
+ filter. If not engine name is provided on the SPOE filter line, the SPOE
+ agent name must be used.
+
+ Arguments:
+ <engine-name> The SPOE engine name.
+
+ <group-name> The SPOE group name as specified in the engine
+ configuration.
+
+
+set-bandwidth-limit <name> [limit {<expr> | <size>}] [period {<expr> | <time>}]
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | X | X | X | X | -
+
+ This action is used to enable the bandwidth limitation filter <name>, either
+ on the upload or download direction depending on the filter type. Custom
+ limit and period may be defined, if and only if <name> references a
+ per-stream bandwidth limitation filter. When a set-bandwidth-limit rule is
+ executed, it first resets all settings of the filter to their defaults prior
+ to enabling it. As a consequence, if several "set-bandwidth-limit" actions
+ are executed for the same filter, only the last one is considered. Several
+ bandwidth limitation filters can be enabled on the same stream.
+
+ Note that this action cannot be used in a defaults section because bandwidth
+ limitation filters cannot be defined in defaults sections. In addition, only
+ the HTTP payload transfer is limited. The HTTP headers are not considered.
+
+ Arguments:
+ <expr> Is a standard HAProxy expression formed by a sample-fetch followed
+ by some converters. The result is converted to an integer. It is
+ interpreted as a size in bytes for the "limit" parameter and as a
+ duration in milliseconds for the "period" parameter.
+
+ <size> Is a number. It follows the HAProxy size format and is expressed in
+ bytes.
+
+ <time> Is a number. It follows the HAProxy time format and is expressed in
+ milliseconds.
+
+ Example:
+ http-request set-bandwidth-limit global-limit
+ http-request set-bandwidth-limit my-limit limit 1m period 10s
+
+ See section 9.7 about bandwidth limitation filter setup.
+
+
+set-dst <expr>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ X | X | X | - | X | - | -
+
+ This is used to set the destination IP address to the value of specified
+ expression. Useful when a proxy in front of HAProxy rewrites destination IP,
+ but provides the correct IP in a HTTP header; or you want to mask the IP for
+ privacy. If you want to connect to the new address/port, use '0.0.0.0:0' as a
+ server address in the backend.
+
+ Arguments:
+ <expr> Is a standard HAProxy expression formed by a sample-fetch followed
+ by some converters.
+
+ Example:
+ http-request set-dst hdr(x-dst)
+ http-request set-dst dst,ipmask(24)
+
+ When possible, set-dst preserves the original destination port as long as the
+ address family allows it, otherwise the destination port is set to 0.
+
+
+set-dst-port <expr>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ X | X | X | - | X | - | -
+
+ This is used to set the destination port address to the value of specified
+ expression. If you want to connect to the new address/port, use '0.0.0.0:0'
+ as a server address in the backend.
+
+ Arguments:
+ <expr> Is a standard HAProxy expression formed by a sample-fetch
+ followed by some converters.
+
+ Example:
+ http-request set-dst-port hdr(x-port)
+ http-request set-dst-port int(4000)
+
+ When possible, set-dst-port preserves the original destination address as
+ long as the address family supports a port, otherwise it forces the
+ destination address to IPv4 "0.0.0.0" before rewriting the port.
+
+
+set-header <name> <fmt>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | X | X
+
+ This does the same as the "add-header" action except that the header is first
+ removed if it existed. This is useful when passing security information to
+ the server, where the header must not be manipulated by external users, or to
+ force certain response headers such as "Server" to hide external information.
+ Note that the new value is computed before the removal so it is possible to
+ concatenate a value to an existing header.
+
+ Example:
+ http-request set-header X-Haproxy-Current-Date %T
+ http-request set-header X-SSL %[ssl_fc]
+ http-request set-header X-SSL-Session_ID %[ssl_fc_session_id,hex]
+ http-request set-header X-SSL-Client-Verify %[ssl_c_verify]
+ http-request set-header X-SSL-Client-DN %{+Q}[ssl_c_s_dn]
+ http-request set-header X-SSL-Client-CN %{+Q}[ssl_c_s_dn(cn)]
+ http-request set-header X-SSL-Issuer %{+Q}[ssl_c_i_dn]
+ http-request set-header X-SSL-Client-NotBefore %{+Q}[ssl_c_notbefore]
+ http-request set-header X-SSL-Client-NotAfter %{+Q}[ssl_c_notafter]
+
+
+set-log-level <level>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | X | X | X | X | X
+
+ This is used to change the log level of the current request when a certain
+ condition is met. Valid levels are the 8 syslog levels (see the "log"
+ keyword) plus the special level "silent" which disables logging for this
+ request. This rule is not final so the last matching rule wins. This rule
+ can be useful to disable health checks coming from another equipment.
+
+
+set-map(<file-name>) <key fmt> <value fmt>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | X | X
+
+ This is used to add a new entry into a map. The map must be loaded from a
+ file (even a dummy empty file). The file name of the map to be updated is
+ passed between parentheses. It takes 2 arguments: <key fmt>, which follows
+ log-format rules, used to collect map key, and <value fmt>, which follows
+ log-format rules, used to collect content for the new entry.
+ It performs a lookup in the map before insertion, to avoid duplicated (or
+ more) values. It is the equivalent of the "set map" command from the
+ stats socket, but can be triggered by an HTTP request.
+
+
+set-mark <mark>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ X | X | X | X | X | X | -
+
+ This is used to set the Netfilter/IPFW MARK on all packets sent to the client
+ to the value passed in <mark> on platforms which support it. This value is an
+ unsigned 32 bit value which can be matched by netfilter/ipfw and by the
+ routing table or monitoring the packets through DTrace. It can be expressed
+ both in decimal or hexadecimal format (prefixed by "0x").
+ This can be useful to force certain packets to take a different route (for
+ example a cheaper network path for bulk downloads). This works on Linux
+ kernels 2.6.32 and above and requires admin privileges, as well on FreeBSD
+ and OpenBSD.
+
+
+set-method <fmt>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | - | -
+
+ This rewrites the request method with the result of the evaluation of format
+ string <fmt>. There should be very few valid reasons for having to do so as
+ this is more likely to break something than to fix it.
+
+
+set-nice <nice>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | X | X | X | X | -
+
+ This sets the "nice" factor of the current request/response being processed.
+ It only has effect against the other requests being processed at the same
+ time. The default value is 0, unless altered by the "nice" setting on the
+ "bind" line. The accepted range is -1024..1024. The higher the value, the
+ nicest the request will be. Lower values will make the request more important
+ than other ones. This can be useful to improve the speed of some requests, or
+ lower the priority of non-important requests. Using this setting without
+ prior experimentation can cause some major slowdown.
+
+
+set-path <fmt>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | - | -
+
+ This rewrites the request path with the result of the evaluation of format
+ string <fmt>. The query string, if any, is left intact. If a scheme and
+ authority is found before the path, they are left intact as well. If the
+ request doesn't have a path ("*"), this one is replaced with the format.
+ This can be used to prepend a directory component in front of a path for
+ example. See also "http-request set-query" and "http-request set-uri".
+
+ Example :
+ # prepend the host name before the path
+ http-request set-path /%[hdr(host)]%[path]
+
+
+set-pathq <fmt>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | - | -
+
+ This does the same as "http-request set-path" except that the query-string is
+ also rewritten. It may be used to remove the query-string, including the
+ question mark (it is not possible using "http-request set-query").
+
+
+set-priority-class <expr>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | X | - | X | - | -
+
+ This is used to set the queue priority class of the current request.
+ The value must be a sample expression which converts to an integer in the
+ range -2047..2047. Results outside this range will be truncated.
+ The priority class determines the order in which queued requests are
+ processed. Lower values have higher priority.
+
+
+set-priority-offset <expr>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | X | - | X | - | -
+
+ This is used to set the queue priority timestamp offset of the current
+ request. The value must be a sample expression which converts to an integer
+ in the range -524287..524287. Results outside this range will be truncated.
+ When a request is queued, it is ordered first by the priority class, then by
+ the current timestamp adjusted by the given offset in milliseconds. Lower
+ values have higher priority.
+ Note that the resulting timestamp is is only tracked with enough precision
+ for 524,287ms (8m44s287ms). If the request is queued long enough to where the
+ adjusted timestamp exceeds this value, it will be misidentified as highest
+ priority. Thus it is important to set "timeout queue" to a value, where when
+ combined with the offset, does not exceed this limit.
+
+
+set-query <fmt>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | - | -
+
+ This rewrites the request's query string which appears after the first
+ question mark ("?") with the result of the evaluation of format string <fmt>.
+ The part prior to the question mark is left intact. If the request doesn't
+ contain a question mark and the new value is not empty, then one is added at
+ the end of the URI, followed by the new value. If a question mark was
+ present, it will never be removed even if the value is empty. This can be
+ used to add or remove parameters from the query string.
+
+ See also "http-request set-query" and "http-request set-uri".
+
+ Example:
+ # replace "%3D" with "=" in the query string
+ http-request set-query %[query,regsub(%3D,=,g)]
+
+
+set-src <expr>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ X | X | X | - | X | - | -
+
+ This is used to set the source IP address to the value of specified
+ expression. Useful when a proxy in front of HAProxy rewrites source IP, but
+ provides the correct IP in a HTTP header; or you want to mask source IP for
+ privacy. All subsequent calls to "src" fetch will return this value
+ (see example).
+
+ Arguments :
+ <expr> Is a standard HAProxy expression formed by a sample-fetch followed
+ by some converters.
+
+ See also "option forwardfor".
+
+ Example:
+ http-request set-src hdr(x-forwarded-for)
+ http-request set-src src,ipmask(24)
+
+ # After the masking this will track connections
+ # based on the IP address with the last byte zeroed out.
+ http-request track-sc0 src
+
+ When possible, set-src preserves the original source port as long as the
+ address family allows it, otherwise the source port is set to 0.
+
+
+set-src-port <expr>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ X | X | X | - | X | - | -
+
+ This is used to set the source port address to the value of specified
+ expression.
+
+ Arguments:
+ <expr> Is a standard HAProxy expression formed by a sample-fetch followed
+ by some converters.
+
+ Example:
+ http-request set-src-port hdr(x-port)
+ http-request set-src-port int(4000)
+
+ When possible, set-src-port preserves the original source address as long as
+ the address family supports a port, otherwise it forces the source address to
+ IPv4 "0.0.0.0" before rewriting the port.
+
+
+set-status <status> [reason <str>]
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | - | X | X
+
+ This replaces the response status code with <status> which must be an integer
+ between 100 and 999. Optionally, a custom reason text can be provided defined
+ by <str>, or the default reason for the specified code will be used as a
+ fallback. Note that the reason string only exists in HTTP/1.x and is ignored
+ by other versions of the protocol.
+
+ Example:
+ # return "431 Request Header Fields Too Large"
+ http-response set-status 431
+ # return "503 Slow Down", custom reason
+ http-response set-status 503 reason "Slow Down".
+
+
+set-timeout { client | server | tunnel } { <timeout> | <expr> }
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | X | -
+
+ This action overrides the specified "client", "server" or "tunnel" timeout
+ for the current stream only. The timeout can be specified in milliseconds or
+ with any other unit if the number is suffixed by the unit as explained at the
+ top of this document. It is also possible to write an expression which must
+ return a number interpreted as a timeout in milliseconds.
+
+ Note that the server/tunnel timeouts are only relevant on the backend side
+ and thus this rule is only available for the proxies with backend
+ capabilities. Likewise, client timeout is only relevant for frontend side.
+ Also the timeout value must be non-null to obtain the expected results.
+
+ Example:
+ http-request set-timeout tunnel 5s
+ http-request set-timeout server req.hdr(host),map_int(host.lst)
+
+ Example:
+ http-response set-timeout tunnel 5s
+ http-response set-timeout server res.hdr(X-Refresh-Seconds),mul(1000)
+
+
+set-tos <tos>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ X | X | X | X | X | X | -
+
+ This is used to set the TOS or DSCP field value of packets sent to the client
+ to the value passed in <tos> on platforms which support this. This value
+ represents the whole 8 bits of the IP TOS field, and can be expressed both in
+ decimal or hexadecimal format (prefixed by "0x"). Note that only the 6 higher
+ bits are used in DSCP or TOS, and the two lower bits are always 0. This can
+ be used to adjust some routing behavior on border routers based on some
+ information from the request.
+
+ See RFC 2474, 2597, 3260 and 4594 for more information.
+
+
+set-uri <fmt>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | - | -
+
+ This rewrites the request URI with the result of the evaluation of format
+ string <fmt>. The scheme, authority, path and query string are all replaced
+ at once. This can be used to rewrite hosts in front of proxies, or to perform
+ complex modifications to the URI such as moving parts between the path and
+ the query string. If an absolute URI is set, it will be sent as is to
+ HTTP/1.1 servers. If it is not the desired behavior, the host, the path
+ and/or the query string should be set separately.
+ See also "http-request set-path" and "http-request set-query".
+
+
+set-var(<var-name>[,<cond>...]) <expr>
+set-var-fmt(<var-name>[,<cond>...]) <fmt>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ X | X | X | X | X | X | X
+
+ This is used to set the contents of a variable. The variable is declared
+ inline.
+
+ Arguments:
+ <var-name> The name of the variable starts with an indication about its
+ scope. The scopes allowed are:
+ "proc" : the variable is shared with the whole process
+ "sess" : the variable is shared with the whole session
+ "txn" : the variable is shared with the transaction
+ (request and response)
+ "req" : the variable is shared only during request
+ processing
+ "res" : the variable is shared only during response
+ processing
+ This prefix is followed by a name. The separator is a '.'.
+ The name may only contain characters 'a-z', 'A-Z', '0-9'
+ and '_'.
+
+ <cond> A set of conditions that must all be true for the variable to
+ actually be set (such as "ifnotempty", "ifgt" ...). See the
+ set-var converter's description for a full list of possible
+ conditions.
+
+ <expr> Is a standard HAProxy expression formed by a sample-fetch
+ followed by some converters.
+
+ <fmt> This is the value expressed using log-format rules (see Custom
+ Log Format in section 8.2.4).
+
+ All scopes are usable for HTTP rules, but scopes "proc" and "sess" are the
+ only usable ones in rule sets which do not have access to contents such as
+ "tcp-request connection" and "tcp-request session".
+
+ Example:
+ http-request set-var(req.my_var) req.fhdr(user-agent),lower
+ http-request set-var-fmt(txn.from) %[src]:%[src_port]
+
+
+silent-drop [ rst-ttl <ttl> ]
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ X | X | X | X | X | X | -
+
+ This stops the evaluation of the rules and makes the client-facing connection
+ suddenly disappear using a system-dependent way that tries to prevent the
+ client from being notified. When called without the rst-ttl argument,
+ we try to prevent sending any FIN or RST packet back to the client by
+ using TCP_REPAIR. If this fails (mainly because of missing privileges),
+ we fall back to sending a RST packet with a TTL of 1.
+
+ The effect is that the client still sees an established connection while
+ there is none on HAProxy, saving resources. However, stateful equipment
+ placed between the HAProxy and the client (firewalls, proxies,
+ load balancers) will also keep the established connection in their
+ session tables.
+
+ The optional rst-ttl changes this behaviour: TCP_REPAIR is not used, and an
+ RST packet with a configurable TTL is sent. When set to a reasonable value,
+ the RST packet travels through the local infrastructure, deleting the
+ connection in firewalls and other systems, but disappears before reaching
+ the client. Future packets from the client will then be dropped already by
+ front equipments. These local RSTs protect local resources, but not the
+ client's. This must not be used unless the consequences of doing this are
+ fully understood.
+
+
+strict-mode { on | off }
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | X | X
+
+ This enables or disables the strict rewriting mode for following rules. It
+ does not affect rules declared before it and it is only applicable on rules
+ performing a rewrite on the requests. When the strict mode is enabled, any
+ rewrite failure triggers an internal error. Otherwise, such errors are
+ silently ignored. The purpose of the strict rewriting mode is to make some
+ rewrites optional while others must be performed to continue the request
+ processing.
+
+ By default, the strict rewriting mode is enabled. Its value is also reset
+ when a ruleset evaluation ends. So, for instance, if you change the mode on
+ the frontend, the default mode is restored when HAProxy starts the backend
+ rules evaluation.
+
+
+switch-mode http [ proto <name> ]
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | X | - | - | - | -
+
+ This action is used to perform a connection upgrade. Only HTTP upgrades are
+ supported for now. The protocol may optionally be specified. This action is
+ only available for a proxy with the frontend capability. The connection
+ upgrade is immediately performed, following "tcp-request content" rules are
+ not evaluated. This upgrade method should be preferred to the implicit one
+ consisting to rely on the backend mode. When used, it is possible to set HTTP
+ directives in a frontend without any warning. These directives will be
+ conditionally evaluated if the HTTP upgrade is performed. However, an HTTP
+ backend must still be selected. It remains unsupported to route an HTTP
+ connection (upgraded or not) to a TCP server.
+
+ See section 4 about Proxies for more details on HTTP upgrades.
+
+
+tarpit [ { status | deny_status } <code>] [content-type <type>]
+ [ { default-errorfiles | errorfile <file> | errorfiles <name> |
+ file <file> | lf-file <file> | string <str> | lf-string <fmt> } ]
+ [ hdr <name> <fmt> ]*
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | - | -
+
+ This stops the evaluation of the rules and immediately blocks the request
+ without responding for a delay specified by "timeout tarpit" or
+ "timeout connect" if the former is not set. After that delay, if the client
+ is still connected, a response is returned so that the client does not
+ suspect it has been tarpitted. Logs will report the flags "PT". The goal of
+ the tarpit rule is to slow down robots during an attack when they're limited
+ on the number of concurrent requests. It can be very efficient against very
+ dumb robots, and will significantly reduce the load on firewalls compared to
+ a "deny" rule. But when facing "correctly" developed robots, it can make
+ things worse by forcing HAProxy and the front firewall to support insane
+ number of concurrent connections. By default an HTTP error 500 is returned.
+ But the response may be customized using same syntax than
+ "http-request return" rules. Thus, see "http-request return" for details.
+
+ For compatibility purpose, when no argument is defined, or only "deny_status",
+ the argument "default-errorfiles" is implied. It means
+ "http-request tarpit [deny_status <status>]" is an alias of
+ "http-request tarpit [status <status>] default-errorfiles".
+ No further "http-request" rules are evaluated.
+ See also "http-request return" and "http-request silent-drop".
+
+
+track-sc0 <key> [table <table>]
+track-sc1 <key> [table <table>]
+track-sc2 <key> [table <table>]
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ X | X | X | - | X | X | -
+
+ This enables tracking of sticky counters from current request. These rules do
+ not stop evaluation and do not change default action. The number of counters
+ that may be simultaneously tracked by the same connection is set by the
+ global "tune.stick-counters" setting, which defaults to MAX_SESS_STKCTR if
+ set at build time (it is reported in haproxy -vv) and which defaults to 3,
+ so the track-sc number is between 0 and (tune.stick-counters-1). The first
+ "track-sc0" rule executed enables tracking of the counters of the specified
+ table as the first set. The first "track-sc1" rule executed enables tracking
+ of the counters of the specified table as the second set. The first
+ "track-sc2" rule executed enables tracking of the counters of the specified
+ table as the third set. It is a recommended practice to use the first set of
+ counters for the per-frontend counters and the second set for the per-backend
+ ones. But this is just a guideline, all may be used everywhere.
+
+ Arguments :
+
+ <key> is mandatory, and is a sample expression rule as described in
+ section 7.3. It describes what elements of the incoming connection,
+ request or reponse will be analyzed, extracted, combined, and used
+ to select which table entry to update the counters.
+
+ <table> is an optional table to be used instead of the default one, which
+ is the stick-table declared in the current proxy. All the counters
+ for the matches and updates for the key will then be performed in
+ that table until the session ends.
+
+ Once a "track-sc*" rule is executed, the key is looked up in the table and if
+ it is not found, an entry is allocated for it. Then a pointer to that entry
+ is kept during all the session's life, and this entry's counters are updated
+ as often as possible, every time the session's counters are updated, and also
+ systematically when the session ends. Counters are only updated for events
+ that happen after the tracking has been started. As an exception, connection
+ counters and request counters are systematically updated so that they reflect
+ useful information.
+
+ If the entry tracks concurrent connection counters, one connection is counted
+ for as long as the entry is tracked, and the entry will not expire during
+ that time. Tracking counters also provides a performance advantage over just
+ checking the keys, because only one table lookup is performed for all ACL
+ checks that make use of it.
+
+
+unset-var(<var-name>)
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ X | X | X | X | X | X | X
+
+ This is used to unset a variable. See the "set-var" action for details about
+ <var-name>.
+
+ Example:
+ http-request unset-var(req.my_var)
+
+
+use-service <service-name>
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | X | - | X | - | -
+
+ This action executes the configured TCP or HTTP service to reply to the
+ request, depending on the rule set it's used in. The rule is final, i.e.
+ no further rules are evaluated in the same rule set.
+
+ A service may choose to reply by sending any valid response or it may
+ immediately close the connection without sending any response. For HTTP
+ services, a valid response requires a valid HTTP response. Outside natives
+ services, for instance the Prometheus exporter for HTTP services, it is
+ possible to write custom TCP and HTTP services in Lua.
+
+ Arguments :
+ <service-name> is mandatory. It is the service to call
+
+ Example:
+ http-request use-service prometheus-exporter if { path /metrics }
+
+
+wait-for-body time <time> [ at-least <bytes> ]
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | X | -
+
+ This will delay the processing of the request or response until one of the
+ following conditions occurs:
+ - The full request body is received, in which case processing proceeds
+ normally.
+ - <bytes> bytes have been received, when the "at-least" argument is given and
+ <bytes> is non-zero, in which case processing proceeds normally.
+ - The request buffer is full, in which case processing proceeds normally. The
+ size of this buffer is determined by the "tune.bufsize" option.
+ - The request has been waiting for more than <time> milliseconds. In this
+ case HAProxy will respond with a 408 "Request Timeout" error to the client
+ and stop processing the request. Note that if any of the other conditions
+ happens first, this timeout will not occur even if the full body has
+ not yet been recieved.
+
+ This action may be used as a replacement for "option http-buffer-request".
+
+ Arguments :
+
+ <time> is mandatory. It is the maximum time to wait for the body. It
+ follows the HAProxy time format and is expressed in milliseconds.
+
+ <bytes> is optional. It is the minimum payload size to receive to stop to
+ wait. It follows the HAProxy size format and is expressed in
+ bytes. A value of 0 (the default) means no limit.
+
+ Example:
+ http-request wait-for-body time 1s at-least 1k if METH_POST
+
+ See also : "option http-buffer-request"
+
+
+wait-for-handshake
+ Usable in: TCP RqCon| RqSes| RqCnt| RsCnt| HTTP Req| Res| Aft
+ - | - | - | - | X | - | -
+
+ This will delay the processing of the request until the SSL handshake
+ happened. This is mostly useful to delay processing early data until we're
+ sure they are valid.
+
+
+5. Bind and server options
+--------------------------
+
+The "bind", "server" and "default-server" keywords support a number of settings
+depending on some build options and on the system HAProxy was built on. These
+settings generally each consist in one word sometimes followed by a value,
+written on the same line as the "bind" or "server" line. All these options are
+described in this section.
+
+
+5.1. Bind options
+-----------------
+
+The "bind" keyword supports a certain number of settings which are all passed
+as arguments on the same line. The order in which those arguments appear makes
+no importance, provided that they appear after the bind address. All of these
+parameters are optional. Some of them consist in a single words (booleans),
+while other ones expect a value after them. In this case, the value must be
+provided immediately after the setting name.
+
+The currently supported settings are the following ones.
+
+accept-netscaler-cip <magic number>
+ Enforces the use of the NetScaler Client IP insertion protocol over any
+ connection accepted by any of the TCP sockets declared on the same line. The
+ NetScaler Client IP insertion protocol dictates the layer 3/4 addresses of
+ the incoming connection to be used everywhere an address is used, with the
+ only exception of "tcp-request connection" rules which will only see the
+ real connection address. Logs will reflect the addresses indicated in the
+ protocol, unless it is violated, in which case the real address will still
+ be used. This keyword combined with support from external components can be
+ used as an efficient and reliable alternative to the X-Forwarded-For
+ mechanism which is not always reliable and not even always usable. See also
+ "tcp-request connection expect-netscaler-cip" for a finer-grained setting of
+ which client is allowed to use the protocol.
+
+accept-proxy
+ Enforces the use of the PROXY protocol over any connection accepted by any of
+ the sockets declared on the same line. Versions 1 and 2 of the PROXY protocol
+ are supported and correctly detected. The PROXY protocol dictates the layer
+ 3/4 addresses of the incoming connection to be used everywhere an address is
+ used, with the only exception of "tcp-request connection" rules which will
+ only see the real connection address. Logs will reflect the addresses
+ indicated in the protocol, unless it is violated, in which case the real
+ address will still be used. This keyword combined with support from external
+ components can be used as an efficient and reliable alternative to the
+ X-Forwarded-For mechanism which is not always reliable and not even always
+ usable. See also "tcp-request connection expect-proxy" for a finer-grained
+ setting of which client is allowed to use the protocol.
+
+allow-0rtt
+ Allow receiving early data when using TLSv1.3. This is disabled by default,
+ due to security considerations. Because it is vulnerable to replay attacks,
+ you should only allow if for requests that are safe to replay, i.e. requests
+ that are idempotent. You can use the "wait-for-handshake" action for any
+ request that wouldn't be safe with early data.
+
+alpn <protocols>
+ This enables the TLS ALPN extension and advertises the specified protocol
+ list as supported on top of ALPN. The protocol list consists in a comma-
+ delimited list of protocol names, for instance: "http/1.1,http/1.0" (without
+ quotes). This requires that the SSL library is built with support for TLS
+ extensions enabled (check with haproxy -vv). The ALPN extension replaces the
+ initial NPN extension. At the protocol layer, ALPN is required to enable
+ HTTP/2 on an HTTPS frontend and HTTP/3 on a QUIC frontend. However, when such
+ frontends have none of "npn", "alpn" and "no-alpn" set, a default value of
+ "h2,http/1.1" will be used for a regular HTTPS frontend, and "h3" for a QUIC
+ frontend. Versions of OpenSSL prior to 1.0.2 didn't support ALPN and only
+ supposed the now obsolete NPN extension. At the time of writing this, most
+ browsers still support both ALPN and NPN for HTTP/2 so a fallback to NPN may
+ still work for a while. But ALPN must be used whenever possible. Protocols
+ not advertised are not negotiated. For example it is possible to only accept
+ HTTP/2 connections with this:
+
+ bind :443 ssl crt pub.pem alpn h2 # explicitly disable HTTP/1.1
+
+ QUIC supports only h3 and hq-interop as ALPN. h3 is for HTTP/3 and hq-interop
+ is used for http/0.9 and QUIC interop runner (see https://interop.seemann.io).
+ Each "alpn" statement will replace a previous one. In order to remove them,
+ use "no-alpn".
+
+ Note that some old browsers such as Firefox 88 used to experience issues with
+ WebSocket over H2, and in case such a setup is encountered, it may be needed
+ to either explicitly disable HTTP/2 in the "alpn" string by forcing it to
+ "http/1.1" or "no-alpn", or to enable "h2-workaround-bogus-websocket-clients"
+ globally.
+
+backlog <backlog>
+ Sets the socket's backlog to this value. If unspecified or 0, the frontend's
+ backlog is used instead, which generally defaults to the maxconn value.
+
+curves <curves>
+ This setting is only available when support for OpenSSL was built in. It sets
+ the string describing the list of elliptic curves algorithms ("curve suite")
+ that are negotiated during the SSL/TLS handshake with ECDHE. The format of the
+ string is a colon-delimited list of curve name.
+ Example: "X25519:P-256" (without quote)
+ When "curves" is set, "ecdhe" parameter is ignored.
+
+ecdhe <named curve>
+ This setting is only available when support for OpenSSL was built in. It sets
+ the named curve (RFC 4492) used to generate ECDH ephemeral keys. By default,
+ used named curve is prime256v1.
+
+ca-file <cafile>
+ This setting is only available when support for OpenSSL was built in. It
+ designates a PEM file from which to load CA certificates used to verify
+ client's certificate. It is possible to load a directory containing multiple
+ CAs, in this case HAProxy will try to load every ".pem", ".crt", ".cer", and
+ .crl" available in the directory, files starting with a dot are ignored.
+
+ Warning: The "@system-ca" parameter could be used in place of the cafile
+ in order to use the trusted CAs of your system, like its done with the server
+ directive. But you mustn't use it unless you know what you are doing.
+ Configuring it this way basically mean that the bind will accept any client
+ certificate generated from one of the CA present on your system, which is
+ extremely insecure.
+
+ca-ignore-err [all|<errorID>,...]
+ This setting is only available when support for OpenSSL was built in.
+ Sets a comma separated list of errorIDs to ignore during verify at depth > 0.
+ It could be a numerical ID, or the constant name (X509_V_ERR) which is
+ available in the OpenSSL documentation:
+ https://www.openssl.org/docs/manmaster/man3/X509_STORE_CTX_get_error.html#ERROR-CODES
+ It is recommended to use the constant name as the numerical value can change
+ in new version of OpenSSL.
+ If set to 'all', all errors are ignored. SSL handshake is not aborted if an
+ error is ignored.
+
+ca-sign-file <cafile>
+ This setting is only available when support for OpenSSL was built in. It
+ designates a PEM file containing both the CA certificate and the CA private
+ key used to create and sign server's certificates. This is a mandatory
+ setting when the dynamic generation of certificates is enabled. See
+ 'generate-certificates' for details.
+
+ca-sign-pass <passphrase>
+ This setting is only available when support for OpenSSL was built in. It is
+ the CA private key passphrase. This setting is optional and used only when
+ the dynamic generation of certificates is enabled. See
+ 'generate-certificates' for details.
+
+ca-verify-file <cafile>
+ This setting designates a PEM file from which to load CA certificates used to
+ verify client's certificate. It designates CA certificates which must not be
+ included in CA names sent in server hello message. Typically, "ca-file" must
+ be defined with intermediate certificates, and "ca-verify-file" with
+ certificates to ending the chain, like root CA.
+
+ciphers <ciphers>
+ This setting is only available when support for OpenSSL was built in. It sets
+ the string describing the list of cipher algorithms ("cipher suite") that are
+ negotiated during the SSL/TLS handshake up to TLSv1.2. The format of the
+ string is defined in "man 1 ciphers" from OpenSSL man pages. For background
+ information and recommendations see e.g.
+ (https://wiki.mozilla.org/Security/Server_Side_TLS) and
+ (https://mozilla.github.io/server-side-tls/ssl-config-generator/). For TLSv1.3
+ cipher configuration, please check the "ciphersuites" keyword.
+
+ciphersuites <ciphersuites>
+ This setting is only available when support for OpenSSL was built in and
+ OpenSSL 1.1.1 or later was used to build HAProxy. It sets the string describing
+ the list of cipher algorithms ("cipher suite") that are negotiated during the
+ TLSv1.3 handshake. The format of the string is defined in "man 1 ciphers" from
+ OpenSSL man pages under the "ciphersuites" section. For cipher configuration
+ for TLSv1.2 and earlier, please check the "ciphers" keyword.
+
+client-sigalgs <sigalgs>
+ This setting is only available when support for OpenSSL was built in. It sets
+ the string describing the list of signature algorithms related to client
+ authentication that are negotiated . The format of the string is defined in
+ "man 3 SSL_CTX_set1_client_sigalgs" from the OpenSSL man pages. It is not
+ recommended to use this setting if no specific usecase was identified.
+
+crl-file <crlfile>
+ This setting is only available when support for OpenSSL was built in. It
+ designates a PEM file from which to load certificate revocation list used
+ to verify client's certificate. You need to provide a certificate revocation
+ list for every certificate of your certificate authority chain.
+
+crt <cert>
+ This setting is only available when support for OpenSSL was built in. It
+ designates a PEM file containing both the required certificates and any
+ associated private keys. This file can be built by concatenating multiple
+ PEM files into one (e.g. cat cert.pem key.pem > combined.pem). If your CA
+ requires an intermediate certificate, this can also be concatenated into this
+ file. Intermediate certificate can also be shared in a directory via
+ "issuers-chain-path" directive.
+
+ If the file does not contain a private key, HAProxy will try to load
+ the key at the same path suffixed by a ".key".
+
+ If the OpenSSL used supports Diffie-Hellman, parameters present in this file
+ are loaded.
+
+ If a directory name is used instead of a PEM file, then all files found in
+ that directory will be loaded in alphabetic order unless their name ends
+ with '.key', '.issuer', '.ocsp' or '.sctl' (reserved extensions). Files
+ starting with a dot are also ignored. This directive may be specified multiple
+ times in order to load certificates from multiple files or directories. The
+ certificates will be presented to clients who provide a valid TLS Server Name
+ Indication field matching one of their CN or alt subjects. Wildcards are
+ supported, where a wildcard character '*' is used instead of the first
+ hostname component (e.g. *.example.org matches www.example.org but not
+ www.sub.example.org). If an empty directory is used, HAProxy will not start
+ unless the "strict-sni" keyword is used.
+
+ If no SNI is provided by the client or if the SSL library does not support
+ TLS extensions, or if the client provides an SNI hostname which does not
+ match any certificate, then the first loaded certificate will be presented.
+ This means that when loading certificates from a directory, it is highly
+ recommended to load the default one first as a file or to ensure that it will
+ always be the first one in the directory.
+
+ Note that the same cert may be loaded multiple times without side effects.
+
+ Some CAs (such as GoDaddy) offer a drop down list of server types that do not
+ include HAProxy when obtaining a certificate. If this happens be sure to
+ choose a web server that the CA believes requires an intermediate CA (for
+ GoDaddy, selection Apache Tomcat will get the correct bundle, but many
+ others, e.g. nginx, result in a wrong bundle that will not work for some
+ clients).
+
+ For each PEM file, HAProxy checks for the presence of file at the same path
+ suffixed by ".ocsp". If such file is found, support for the TLS Certificate
+ Status Request extension (also known as "OCSP stapling") is automatically
+ enabled. The content of this file is optional. If not empty, it must contain
+ a valid OCSP Response in DER format. In order to be valid an OCSP Response
+ must comply with the following rules: it has to indicate a good status,
+ it has to be a single response for the certificate of the PEM file, and it
+ has to be valid at the moment of addition. If these rules are not respected
+ the OCSP Response is ignored and a warning is emitted. In order to identify
+ which certificate an OCSP Response applies to, the issuer's certificate is
+ necessary. If the issuer's certificate is not found in the PEM file, it will
+ be loaded from a file at the same path as the PEM file suffixed by ".issuer"
+ if it exists otherwise it will fail with an error.
+
+ For each PEM file, HAProxy also checks for the presence of file at the same
+ path suffixed by ".sctl". If such file is found, support for Certificate
+ Transparency (RFC6962) TLS extension is enabled. The file must contain a
+ valid Signed Certificate Timestamp List, as described in RFC. File is parsed
+ to check basic syntax, but no signatures are verified.
+
+ There are cases where it is desirable to support multiple key types, e.g. RSA
+ and ECDSA in the cipher suites offered to the clients. This allows clients
+ that support EC certificates to be able to use EC ciphers, while
+ simultaneously supporting older, RSA only clients.
+
+ To achieve this, OpenSSL 1.1.1 is required, you can configure this behavior
+ by providing one crt entry per certificate type, or by configuring a "cert
+ bundle" like it was required before HAProxy 1.8. See "ssl-load-extra-files".
+
+crt-ignore-err <errors>
+ This setting is only available when support for OpenSSL was built in. Sets a
+ comma separated list of errorIDs to ignore during verify at depth == 0.
+ It could be a numerical ID, or the constant name (X509_V_ERR) which is
+ available in the OpenSSL documentation:
+ https://www.openssl.org/docs/manmaster/man3/X509_STORE_CTX_get_error.html#ERROR-CODES
+ It is recommended to use the constant name as the numerical value can change
+ in new version of OpenSSL.
+ If set to 'all', all errors are ignored. SSL handshake is not aborted if an
+ error is ignored.
+
+crt-list <file>
+ This setting is only available when support for OpenSSL was built in. It
+ designates a list of PEM file with an optional ssl configuration and a SNI
+ filter per certificate, with the following format for each line :
+
+ <crtfile> [\[<sslbindconf> ...\]] [[!]<snifilter> ...]
+
+ sslbindconf supports "allow-0rtt", "alpn", "ca-file", "ca-verify-file",
+ "ciphers", "ciphersuites", "crl-file", "curves", "ecdhe", "no-ca-names",
+ "npn", "verify" configuration. With BoringSSL and Openssl >= 1.1.1
+ "ssl-min-ver" and "ssl-max-ver" are also supported. It overrides the
+ configuration set in bind line for the certificate.
+
+ Wildcards are supported in the SNI filter. Negative filter are also supported,
+ useful in combination with a wildcard filter to exclude a particular SNI, or
+ after the first certificate to exclude a pattern from its CN or Subject Alt
+ Name (SAN). The certificates will be presented to clients who provide a valid
+ TLS Server Name Indication field matching one of the SNI filters. If no SNI
+ filter is specified, the CN and SAN are used. This directive may be specified
+ multiple times. See the "crt" option for more information. The default
+ certificate is still needed to meet OpenSSL expectations. If it is not used,
+ the 'strict-sni' option may be used.
+
+ Multi-cert bundling (see "ssl-load-extra-files") is supported with crt-list,
+ as long as only the base name is given in the crt-list. SNI filter will do
+ the same work on all bundled certificates.
+
+ Empty lines as well as lines beginning with a hash ('#') will be ignored.
+
+ The first declared certificate of a bind line is used as the default
+ certificate, either from crt or crt-list option, which HAProxy should use in
+ the TLS handshake if no other certificate matches. This certificate will also
+ be used if the provided SNI matches its CN or SAN, even if a matching SNI
+ filter is found on any crt-list. The SNI filter !* can be used after the first
+ declared certificate to not include its CN and SAN in the SNI tree, so it will
+ never match except if no other certificate matches. This way the first
+ declared certificate act as a fallback.
+
+ When no ALPN is set, the "bind" line's default one is used. If a "bind" line
+ has no "no-alpn", "alpn" nor "npn" set, a default value will be used
+ depending on the protocol (see "alpn" above). However if the "bind" line has
+ a different default, or explicitly disables ALPN using "no-alpn", it is
+ possible to force a specific value for a certificate.
+
+ crt-list file example:
+ cert1.pem !*
+ # comment
+ cert2.pem [alpn h2,http/1.1]
+ certW.pem *.domain.tld !secure.domain.tld
+ certS.pem [curves X25519:P-256 ciphers ECDHE-ECDSA-AES256-GCM-SHA384] secure.domain.tld
+
+defer-accept
+ Is an optional keyword which is supported only on certain Linux kernels. It
+ states that a connection will only be accepted once some data arrive on it,
+ or at worst after the first retransmit. This should be used only on protocols
+ for which the client talks first (e.g. HTTP). It can slightly improve
+ performance by ensuring that most of the request is already available when
+ the connection is accepted. On the other hand, it will not be able to detect
+ connections which don't talk. It is important to note that this option is
+ broken in all kernels up to 2.6.31, as the connection is never accepted until
+ the client talks. This can cause issues with front firewalls which would see
+ an established connection while the proxy will only see it in SYN_RECV. This
+ option is only supported on TCPv4/TCPv6 sockets and ignored by other ones.
+
+expose-fd listeners
+ This option is only usable with the stats socket. It gives your stats socket
+ the capability to pass listeners FD to another HAProxy process.
+ In master-worker mode, this is not required anymore, the listeners will be
+ passed using the internal socketpairs between the master and the workers.
+ See also "-x" in the management guide.
+
+force-sslv3
+ This option enforces use of SSLv3 only on SSL connections instantiated from
+ this listener. SSLv3 is generally less expensive than the TLS counterparts
+ for high connection rates. This option is also available on global statement
+ "ssl-default-bind-options". See also "ssl-min-ver" and "ssl-max-ver".
+
+force-tlsv10
+ This option enforces use of TLSv1.0 only on SSL connections instantiated from
+ this listener. This option is also available on global statement
+ "ssl-default-bind-options". See also "ssl-min-ver" and "ssl-max-ver".
+
+force-tlsv11
+ This option enforces use of TLSv1.1 only on SSL connections instantiated from
+ this listener. This option is also available on global statement
+ "ssl-default-bind-options". See also "ssl-min-ver" and "ssl-max-ver".
+
+force-tlsv12
+ This option enforces use of TLSv1.2 only on SSL connections instantiated from
+ this listener. This option is also available on global statement
+ "ssl-default-bind-options". See also "ssl-min-ver" and "ssl-max-ver".
+
+force-tlsv13
+ This option enforces use of TLSv1.3 only on SSL connections instantiated from
+ this listener. This option is also available on global statement
+ "ssl-default-bind-options". See also "ssl-min-ver" and "ssl-max-ver".
+
+generate-certificates
+ This setting is only available when support for OpenSSL was built in. It
+ enables the dynamic SSL certificates generation. A CA certificate and its
+ private key are necessary (see 'ca-sign-file'). When HAProxy is configured as
+ a transparent forward proxy, SSL requests generate errors because of a common
+ name mismatch on the certificate presented to the client. With this option
+ enabled, HAProxy will try to forge a certificate using the SNI hostname
+ indicated by the client. This is done only if no certificate matches the SNI
+ hostname (see 'crt-list'). If an error occurs, the default certificate is
+ used, else the 'strict-sni' option is set.
+ It can also be used when HAProxy is configured as a reverse proxy to ease the
+ deployment of an architecture with many backends.
+
+ Creating a SSL certificate is an expensive operation, so a LRU cache is used
+ to store forged certificates (see 'tune.ssl.ssl-ctx-cache-size'). It
+ increases the HAProxy's memory footprint to reduce latency when the same
+ certificate is used many times.
+
+gid <gid>
+ Sets the group of the UNIX sockets to the designated system gid. It can also
+ be set by default in the global section's "unix-bind" statement. Note that
+ some platforms simply ignore this. This setting is equivalent to the "group"
+ setting except that the group ID is used instead of its name. This setting is
+ ignored by non UNIX sockets.
+
+group <group>
+ Sets the group of the UNIX sockets to the designated system group. It can
+ also be set by default in the global section's "unix-bind" statement. Note
+ that some platforms simply ignore this. This setting is equivalent to the
+ "gid" setting except that the group name is used instead of its gid. This
+ setting is ignored by non UNIX sockets.
+
+id <id>
+ Fixes the socket ID. By default, socket IDs are automatically assigned, but
+ sometimes it is more convenient to fix them to ease monitoring. This value
+ must be strictly positive and unique within the listener/frontend. This
+ option can only be used when defining only a single socket.
+
+interface <interface>
+ Restricts the socket to a specific interface. When specified, only packets
+ received from that particular interface are processed by the socket. This is
+ currently only supported on Linux. The interface must be a primary system
+ interface, not an aliased interface. It is also possible to bind multiple
+ frontends to the same address if they are bound to different interfaces. Note
+ that binding to a network interface requires root privileges. This parameter
+ is only compatible with TCPv4/TCPv6 sockets. When specified, return traffic
+ uses the same interface as inbound traffic, and its associated routing table,
+ even if there are explicit routes through different interfaces configured.
+ This can prove useful to address asymmetric routing issues when the same
+ client IP addresses need to be able to reach frontends hosted on different
+ interfaces.
+
+level <level>
+ This setting is used with the stats sockets only to restrict the nature of
+ the commands that can be issued on the socket. It is ignored by other
+ sockets. <level> can be one of :
+ - "user" is the least privileged level; only non-sensitive stats can be
+ read, and no change is allowed. It would make sense on systems where it
+ is not easy to restrict access to the socket.
+ - "operator" is the default level and fits most common uses. All data can
+ be read, and only non-sensitive changes are permitted (e.g. clear max
+ counters).
+ - "admin" should be used with care, as everything is permitted (e.g. clear
+ all counters).
+
+severity-output <format>
+ This setting is used with the stats sockets only to configure severity
+ level output prepended to informational feedback messages. Severity
+ level of messages can range between 0 and 7, conforming to syslog
+ rfc5424. Valid and successful socket commands requesting data
+ (i.e. "show map", "get acl foo" etc.) will never have a severity level
+ prepended. It is ignored by other sockets. <format> can be one of :
+ - "none" (default) no severity level is prepended to feedback messages.
+ - "number" severity level is prepended as a number.
+ - "string" severity level is prepended as a string following the
+ rfc5424 convention.
+
+maxconn <maxconn>
+ Limits the sockets to this number of concurrent connections. Extraneous
+ connections will remain in the system's backlog until a connection is
+ released. If unspecified, the limit will be the same as the frontend's
+ maxconn. Note that in case of port ranges or multiple addresses, the same
+ value will be applied to each socket. This setting enables different
+ limitations on expensive sockets, for instance SSL entries which may easily
+ eat all memory.
+
+mode <mode>
+ Sets the octal mode used to define access permissions on the UNIX socket. It
+ can also be set by default in the global section's "unix-bind" statement.
+ Note that some platforms simply ignore this. This setting is ignored by non
+ UNIX sockets.
+
+mss <maxseg>
+ Sets the TCP Maximum Segment Size (MSS) value to be advertised on incoming
+ connections. This can be used to force a lower MSS for certain specific
+ ports, for instance for connections passing through a VPN. Note that this
+ relies on a kernel feature which is theoretically supported under Linux but
+ was buggy in all versions prior to 2.6.28. It may or may not work on other
+ operating systems. It may also not change the advertised value but change the
+ effective size of outgoing segments. The commonly advertised value for TCPv4
+ over Ethernet networks is 1460 = 1500(MTU) - 40(IP+TCP). If this value is
+ positive, it will be used as the advertised MSS. If it is negative, it will
+ indicate by how much to reduce the incoming connection's advertised MSS for
+ outgoing segments. This parameter is only compatible with TCP v4/v6 sockets.
+
+name <name>
+ Sets an optional name for these sockets, which will be reported on the stats
+ page.
+
+namespace <name>
+ On Linux, it is possible to specify which network namespace a socket will
+ belong to. This directive makes it possible to explicitly bind a listener to
+ a namespace different from the default one. Please refer to your operating
+ system's documentation to find more details about network namespaces.
+
+nbconn <nbconn> [ EXPERIMENTAL ]
+ This setting is only valid for listener instances which uses reverse HTTP.
+ This will define the count of connections which will be mounted in parallel.
+ If not specified, a default value of 1 is used.
+
+ Reverse HTTP is currently still in active development. Configuration
+ mechanism may change in the future. For this reason it is internally marked
+ as expirmental, meaning that "expose-experimental-directives" must appear on
+ a line before this directive.
+
+nice <nice>
+ Sets the 'niceness' of connections initiated from the socket. Value must be
+ in the range -1024..1024 inclusive, and defaults to zero. Positive values
+ means that such connections are more friendly to others and easily offer
+ their place in the scheduler. On the opposite, negative values mean that
+ connections want to run with a higher priority than others. The difference
+ only happens under high loads when the system is close to saturation.
+ Negative values are appropriate for low-latency or administration services,
+ and high values are generally recommended for CPU intensive tasks such as SSL
+ processing or bulk transfers which are less sensible to latency. For example,
+ it may make sense to use a positive value for an SMTP socket and a negative
+ one for an RDP socket.
+
+no-alpn
+ Disables ALPN processing (technically speaking this sets the ALPN string to
+ an empty string that will not be advertised). It permits to cancel a previous
+ occurrence of an "alpn" setting and to disable application protocol
+ negotiation. It may also be used to prevent a listener from negotiating ALPN
+ with a client on an HTTPS or QUIC listener; by default, HTTPS listeners will
+ advertise "h2,http/1.1" and QUIC listeners will advertise "h3". See also
+ "alpn" bove. Note that when using "crt-list", a certificate may override the
+ "alpn" setting and re-enable its processing.
+
+no-ca-names
+ This setting is only available when support for OpenSSL was built in. It
+ prevents from send CA names in server hello message when ca-file is used.
+ Use "ca-verify-file" instead of "ca-file" with "no-ca-names".
+
+no-sslv3
+ This setting is only available when support for OpenSSL was built in. It
+ disables support for SSLv3 on any sockets instantiated from the listener when
+ SSL is supported. Note that SSLv2 is forced disabled in the code and cannot
+ be enabled using any configuration option. This option is also available on
+ global statement "ssl-default-bind-options". Use "ssl-min-ver" and
+ "ssl-max-ver" instead.
+
+no-tls-tickets
+ This setting is only available when support for OpenSSL was built in. It
+ disables the stateless session resumption (RFC 5077 TLS Ticket
+ extension) and force to use stateful session resumption. Stateless
+ session resumption is more expensive in CPU usage. This option is also
+ available on global statement "ssl-default-bind-options".
+ The TLS ticket mechanism is only used up to TLS 1.2.
+ Forward Secrecy is compromised with TLS tickets, unless ticket keys
+ are periodically rotated (via reload or by using "tls-ticket-keys").
+
+no-tlsv10
+ This setting is only available when support for OpenSSL was built in. It
+ disables support for TLSv1.0 on any sockets instantiated from the listener
+ when SSL is supported. Note that SSLv2 is forced disabled in the code and
+ cannot be enabled using any configuration option. This option is also
+ available on global statement "ssl-default-bind-options". Use "ssl-min-ver"
+ and "ssl-max-ver" instead.
+
+no-tlsv11
+ This setting is only available when support for OpenSSL was built in. It
+ disables support for TLSv1.1 on any sockets instantiated from the listener
+ when SSL is supported. Note that SSLv2 is forced disabled in the code and
+ cannot be enabled using any configuration option. This option is also
+ available on global statement "ssl-default-bind-options". Use "ssl-min-ver"
+ and "ssl-max-ver" instead.
+
+no-tlsv12
+ This setting is only available when support for OpenSSL was built in. It
+ disables support for TLSv1.2 on any sockets instantiated from the listener
+ when SSL is supported. Note that SSLv2 is forced disabled in the code and
+ cannot be enabled using any configuration option. This option is also
+ available on global statement "ssl-default-bind-options". Use "ssl-min-ver"
+ and "ssl-max-ver" instead.
+
+no-tlsv13
+ This setting is only available when support for OpenSSL was built in. It
+ disables support for TLSv1.3 on any sockets instantiated from the listener
+ when SSL is supported. Note that SSLv2 is forced disabled in the code and
+ cannot be enabled using any configuration option. This option is also
+ available on global statement "ssl-default-bind-options". Use "ssl-min-ver"
+ and "ssl-max-ver" instead.
+
+npn <protocols>
+ This enables the NPN TLS extension and advertises the specified protocol list
+ as supported on top of NPN. The protocol list consists in a comma-delimited
+ list of protocol names, for instance: "http/1.1,http/1.0" (without quotes).
+ This requires that the SSL library is built with support for TLS extensions
+ enabled (check with haproxy -vv). Note that the NPN extension has been
+ replaced with the ALPN extension (see the "alpn" keyword), though this one is
+ only available starting with OpenSSL 1.0.2. If HTTP/2 is desired on an older
+ version of OpenSSL, NPN might still be used as most clients still support it
+ at the time of writing this. It is possible to enable both NPN and ALPN
+ though it probably doesn't make any sense out of testing.
+
+ocsp-update [ off | on ] (crt-list only)
+ Enable automatic OCSP response update when set to 'on', disable it otherwise.
+ Its value defaults to 'off'.
+ Please note that for now, this option can only be used in a crt-list line, it
+ cannot be used directly on a bind line. It lies in this "Bind options"
+ section because it is still a frontend option. This limitation was set so
+ that the option applies to only one certificate at a time.
+ If a given certificate is used in multiple crt-lists with different values of
+ the 'ocsp-update' set, an error will be raised. Here is an example
+ configuration enabling it:
+
+ haproxy.cfg:
+ frontend fe
+ bind :443 ssl crt-list haproxy.list
+
+ haproxy.list:
+ server_cert.pem [ocsp-update on] foo.bar
+
+ When the option is set to 'on', we will try to get an ocsp response whenever
+ an ocsp uri is found in the frontend's certificate. The only limitation of
+ this mode is that the certificate's issuer will have to be known in order for
+ the OCSP certid to be built.
+ Each OCSP response will be updated at least once an hour, and even more
+ frequently if a given OCSP response has an expire date earlier than this one
+ hour limit. A minimum update interval of 5 minutes will still exist in order
+ to avoid updating too often responses that have a really short expire time or
+ even no 'Next Update' at all. Because of this hard limit, please note that
+ when auto update is set to 'on' or 'auto', any OCSP response loaded during
+ init will not be updated until at least 5 minutes, even if its expire time
+ ends before now+5m. This should not be too much of a hassle since an OCSP
+ response must be valid when it gets loaded during init (its expire time must
+ be in the future) so it is unlikely that this response expires in such a
+ short time after init.
+ On the other hand, if a certificate has an OCSP uri specified and no OCSP
+ response, setting this option to 'on' for the given certificate will ensure
+ that the OCSP response gets fetched automatically right after init.
+ The default minimum and maximum delays (5 minutes and 1 hour respectively)
+ can be configured by the "tune.ssl.ocsp-update.maxdelay" and
+ "tune.ssl.ocsp-update.mindelay" global options.
+
+ Whenever an OCSP response is updated by the auto update task or following a
+ call to the "update ssl ocsp-response" CLI command, a dedicated log line is
+ emitted. It follows a dedicated log-format that contains the following header
+ "%ci:%cp [%tr] %ft" and is followed by specific OCSP-related information:
+ - the path of the corresponding frontend certificate
+ - a numerical update status
+ - a textual update status
+ - the number of update failures for the given response
+ - the number of update successes for the givan response
+ See "show ssl ocsp-updates" CLI command for a full list of error codes and
+ error messages. This line is emitted regardless of the success or failure of
+ the concerned OCSP response update.
+ The OCSP request/response is sent and received through an http_client
+ instance that has the dontlog-normal option set and that uses the regular
+ HTTP log format in case of error (unreachable OCSP responder for instance).
+ If such an error occurs, another log line that contains HTTP-related
+ information will then be emitted alongside the "regular" OCSP one (which will
+ likely have "HTTP error" as text status). But if a purely HTTP error happens
+ (unreachable OCSP responder for instance), an extra log line that follows the
+ regular HTTP log-format will be emitted.
+ Here are two examples of such log lines, with a successful OCSP update log
+ line first and then an example of an HTTP error with the two different lines
+ (lines were spit and the URL was shortened for readability):
+ <134>Mar 6 11:16:53 haproxy[14872]: -:- [06/Mar/2023:11:16:52.808] \
+ <OCSP-UPDATE> /path_to_cert/foo.pem 1 "Update successful" 0 1
+
+ <134>Mar 6 11:18:55 haproxy[14872]: -:- [06/Mar/2023:11:18:54.207] \
+ <OCSP-UPDATE> /path_to_cert/bar.pem 2 "HTTP error" 1 0
+ <134>Mar 6 11:18:55 haproxy[14872]: -:- [06/Mar/2023:11:18:52.200] \
+ <OCSP-UPDATE> -/- 2/0/-1/-1/3009 503 217 - - SC-- 0/0/0/0/3 0/0 {} \
+ "GET http://127.0.0.1:12345/MEMwQT HTTP/1.1"
+
+ Troubleshooting:
+ A common error that can happen with let's encrypt certificates is if the DNS
+ resolution provides an IPv6 address and your system does not have a valid
+ outgoing IPv6 route. In such a case, you can either create the appropriate
+ route or set the "httpclient.resolvers.prefer ipv4" option in the global
+ section.
+ In case of "OCSP response check failure" error, you might want to check that
+ the issuer certificate that you provided is valid.
+
+prefer-client-ciphers
+ Use the client's preference when selecting the cipher suite, by default
+ the server's preference is enforced. This option is also available on
+ global statement "ssl-default-bind-options".
+ Note that with OpenSSL >= 1.1.1 ChaCha20-Poly1305 is reprioritized anyway
+ (without setting this option), if a ChaCha20-Poly1305 cipher is at the top of
+ the client cipher list.
+
+proto <name>
+ Forces the multiplexer's protocol to use for the incoming connections. It
+ must be compatible with the mode of the frontend (TCP or HTTP). It must also
+ be usable on the frontend side. The list of available protocols is reported
+ in haproxy -vv. The protocols properties are reported : the mode (TCP/HTTP),
+ the side (FE/BE), the mux name and its flags.
+
+ Some protocols are subject to the head-of-line blocking on server side
+ (flag=HOL_RISK). Finally some protocols don't support upgrades (flag=NO_UPG).
+ The HTX compatibility is also reported (flag=HTX).
+
+ Here are the protocols that may be used as argument to a "proto" directive on
+ a bind line :
+
+ h2 : mode=HTTP side=FE|BE mux=H2 flags=HTX|HOL_RISK|NO_UPG
+ h1 : mode=HTTP side=FE|BE mux=H1 flags=HTX|NO_UPG
+ none : mode=TCP side=FE|BE mux=PASS flags=NO_UPG
+
+ Idea behind this option is to bypass the selection of the best multiplexer's
+ protocol for all connections instantiated from this listening socket. For
+ instance, it is possible to force the http/2 on clear TCP by specifying "proto
+ h2" on the bind line.
+
+quic-cc-algo { cubic | newreno }
+quic-cc-algo { cubic | newreno }(max_window)
+ This is a QUIC specific setting to select the congestion control algorithm
+ for any connection attempts to the configured QUIC listeners. They are similar
+ to those used by TCP. An optional value in bytes may be used to specify the
+ maximum window size. It must be greater than 1k and smaller than 4g.
+
+ Default value: cubic
+ Default window value: tune.quic.frontend.conn-tx-buffers.limit * tune.bufsize
+
+ Example:
+ # newreno congestion control algorithm
+ quic-cc-algo newreno
+ # cubic congestion control algorithm with one megabytes as window
+ quic-cc-algo cubic(1m)
+
+quic-force-retry
+ This is a QUIC specific setting which forces the use of the QUIC Retry feature
+ for all the connection attempts to the configured QUIC listeners. It consists
+ in verifying the peers are able to receive packets at the transport address
+ they used to initiate a new connection, sending them a Retry packet which
+ contains a token. This token must be sent back to the Retry packet sender,
+ this latter being the only one to be able to validate the token. Note that QUIC
+ Retry will always be used even if a Retry threshold was set (see
+ "tune.quic.retry-threshold" setting).
+
+ This setting requires the cluster secret to be set or else an error will be
+ reported on startup (see "cluster-secret").
+
+ See https://www.rfc-editor.org/rfc/rfc9000.html#section-8.1.2 for more
+ information about QUIC retry.
+
+quic-socket [ connection | listener ]
+ This QUIC specific setting allows to define the socket allocation mode for
+ the specific listeners. See "tune.quic.socket-owner" for a full description
+ of its usage.
+
+shards <number> | by-thread | by-group
+ In multi-threaded mode, on operating systems supporting multiple listeners on
+ the same IP:port, this will automatically create this number of multiple
+ identical listeners for the same line, all bound to a fair share of the number
+ of the threads attached to this listener. This can sometimes be useful when
+ using very large thread counts where the in-kernel locking on a single socket
+ starts to cause a significant overhead. In this case the incoming traffic is
+ distributed over multiple sockets and the contention is reduced. Note that
+ doing this can easily increase the CPU usage by making more threads work a
+ little bit.
+
+ If the number of shards is higher than the number of available threads, it
+ will automatically be trimmed to the number of threads (i.e. one shard per
+ thread). The special "by-thread" value also creates as many shards as there
+ are threads on the "bind" line. Since the system will evenly distribute the
+ incoming traffic between all these shards, it is important that this number
+ is an integral divisor of the number of threads. Alternately, the other
+ special value "by-group" will create one shard per thread group. This can
+ be useful when dealing with many threads and not wanting to create too many
+ sockets. The load distribution will be a bit less optimal but the contention
+ (especially in the system) will still be lower than with a single socket.
+
+ On operating systems that do not support multiple sockets bound to the same
+ address, "by-thread" and "by-group" will automatically fall back to a single
+ shard. For "by-group" this is done without any warning since it doesn't
+ change anything for a single group, and will result in sockets being
+ duplicated for each group anyway. However, for "by-thread", a diagnostic
+ warning will be emitted if this happens since the resulting number of
+ listeners will not be the expected one.
+
+sigalgs <sigalgs>
+ This setting is only available when support for OpenSSL was built in. It sets
+ the string describing the list of signature algorithms that are negotiated
+ during the TLSv1.2 and TLSv1.3 handshake. The format of the string is defined
+ in "man 3 SSL_CTX_set1_sigalgs" from the OpenSSL man pages. It is not
+ recommended to use this setting unless compatibility with a middlebox is
+ required.
+
+ssl
+ This setting is only available when support for OpenSSL was built in. It
+ enables SSL deciphering on connections instantiated from this listener. A
+ certificate is necessary (see "crt" above). All contents in the buffers will
+ appear in clear text, so that ACLs and HTTP processing will only have access
+ to deciphered contents. SSLv3 is disabled per default, use "ssl-min-ver SSLv3"
+ to enable it.
+
+ssl-max-ver [ SSLv3 | TLSv1.0 | TLSv1.1 | TLSv1.2 | TLSv1.3 ]
+ This option enforces use of <version> or lower on SSL connections instantiated
+ from this listener. Using this setting without "ssl-min-ver" can be
+ ambiguous because the default ssl-min-ver value could change in future HAProxy
+ versions. This option is also available on global statement
+ "ssl-default-bind-options". See also "ssl-min-ver".
+
+ssl-min-ver [ SSLv3 | TLSv1.0 | TLSv1.1 | TLSv1.2 | TLSv1.3 ]
+ This option enforces use of <version> or upper on SSL connections
+ instantiated from this listener. The default value is "TLSv1.2". This option
+ is also available on global statement "ssl-default-bind-options".
+ See also "ssl-max-ver".
+
+strict-sni
+ This setting is only available when support for OpenSSL was built in. The
+ SSL/TLS negotiation is allow only if the client provided an SNI which match
+ a certificate. The default certificate is not used. This option also allows
+ to start without any certificate on a bind line, so an empty directory could
+ be used and filled later from the stats socket.
+ See the "crt" option for more information. See "add ssl crt-list" command in
+ the management guide.
+
+tcp-ut <delay>
+ Sets the TCP User Timeout for all incoming connections instantiated from this
+ listening socket. This option is available on Linux since version 2.6.37. It
+ allows HAProxy to configure a timeout for sockets which contain data not
+ receiving an acknowledgment for the configured delay. This is especially
+ useful on long-lived connections experiencing long idle periods such as
+ remote terminals or database connection pools, where the client and server
+ timeouts must remain high to allow a long period of idle, but where it is
+ important to detect that the client has disappeared in order to release all
+ resources associated with its connection (and the server's session). The
+ argument is a delay expressed in milliseconds by default. This only works
+ for regular TCP connections, and is ignored for other protocols.
+
+tfo
+ Is an optional keyword which is supported only on Linux kernels >= 3.7. It
+ enables TCP Fast Open on the listening socket, which means that clients which
+ support this feature will be able to send a request and receive a response
+ during the 3-way handshake starting from second connection, thus saving one
+ round-trip after the first connection. This only makes sense with protocols
+ that use high connection rates and where each round trip matters. This can
+ possibly cause issues with many firewalls which do not accept data on SYN
+ packets, so this option should only be enabled once well tested. This option
+ is only supported on TCPv4/TCPv6 sockets and ignored by other ones. You may
+ need to build HAProxy with USE_TFO=1 if your libc doesn't define
+ TCP_FASTOPEN.
+
+thread [<thread-group>/]<thread-set>[,...]
+ This restricts the list of threads on which this listener is allowed to run.
+ It does not enforce any of them but eliminates those which do not match. It
+ limits the threads allowed to process incoming connections for this listener.
+
+ There are two numbering schemes. By default, thread numbers are absolute in
+ the process, comprised between 1 and the value specified in global.nbthread.
+ It is also possible to designate a thread number using its relative number
+ inside its thread group, by specifying the thread group number first, then a
+ slash ('/') and the relative thread number(s). In this case thread numbers
+ also start at 1 and end at 32 or 64 depending on the platform. When absolute
+ thread numbers are specified, they will be automatically translated to
+ relative numbers once thread groups are known. Usually, absolute numbers are
+ preferred for simple configurations, and relative ones are preferred for
+ complex configurations where CPU arrangement matters for performance.
+
+ After the optional thread group number, the "thread-set" specification must
+ use the following format:
+
+ "all" | "odd" | "even" | [number][-[number]]
+
+ As their names imply, "all" validates all threads within the set (either all
+ of the group's when a group is specified, or all of the process' threads),
+ "odd" validates all odd-numberred threads (every other thread starting at 1)
+ either for the process or the group, and "even" validates all even-numberred
+ threads (every other thread starting at 2). If instead thread number ranges
+ are used, then all threads included in the range from the first to the last
+ thread number are validated. The numbers are either relative to the group
+ or absolute depending on the presence of a thread group number. If the first
+ thread number is omitted, "1" is used, representing either the first thread
+ of the group or the first thread of the process. If the last thread number is
+ omitted, either the last thread number of the group (32 or 64) is used, or
+ the last thread number of the process (global.nbthread).
+
+ These ranges may be repeated and delimited by a comma, so that non-contiguous
+ thread sets can be specified, and the group, if present, must be specified
+ again for each new range. Note that it is not permitted to mix group-relative
+ and absolute specifications because the whole "bind" line must use either
+ an absolute notation or a relative one, as those not set will be resolved at
+ the end of the parsing.
+
+ It is important to know that each listener described by a "bind" line creates
+ at least one socket represented by at least one file descriptor. Since file
+ descriptors cannot span multiple thread groups, if a "bind" line specifies a
+ thread range that covers more than one group, several file descriptors will
+ automatically be created so that there is at least one per group. Technically
+ speaking they all refer to the same socket in the kernel, but they will get a
+ distinct identifier in haproxy and will even have a dedicated stats entry if
+ "option socket-stats" is used.
+
+ The main purpose is to have multiple bind lines sharing the same IP:port but
+ not the same thread in a listener, so that the system can distribute the
+ incoming connections into multiple queues, bypassing haproxy's internal queue
+ load balancing. Currently Linux 3.9 and above is known for supporting this.
+ See also the "shards" keyword above that automates duplication of "bind"
+ lines and their assignment to multiple groups of threads.
+
+ This keyword is compatible with reverse HTTP binds. However, it is forbidden
+ to specify a thread set which spans accross several thread groups for such a
+ listener as this may caused "nbconn" to not work as intended.
+
+tls-ticket-keys <keyfile>
+ Sets the TLS ticket keys file to load the keys from. The keys need to be 48
+ or 80 bytes long, depending if aes128 or aes256 is used, encoded with base64
+ with one line per key (ex. openssl rand 80 | openssl base64 -A | xargs echo).
+ The first key determines the key length used for next keys: you can't mix
+ aes128 and aes256 keys. Number of keys is specified by the TLS_TICKETS_NO
+ build option (default 3) and at least as many keys need to be present in
+ the file. Last TLS_TICKETS_NO keys will be used for decryption and the
+ penultimate one for encryption. This enables easy key rotation by just
+ appending new key to the file and reloading the process. Keys must be
+ periodically rotated (ex. every 12h) or Perfect Forward Secrecy is
+ compromised. It is also a good idea to keep the keys off any permanent
+ storage such as hard drives (hint: use tmpfs and don't swap those files).
+ Lifetime hint can be changed using tune.ssl.timeout.
+
+transparent
+ Is an optional keyword which is supported only on certain Linux kernels. It
+ indicates that the addresses will be bound even if they do not belong to the
+ local machine, and that packets targeting any of these addresses will be
+ intercepted just as if the addresses were locally configured. This normally
+ requires that IP forwarding is enabled. Caution! do not use this with the
+ default address '*', as it would redirect any traffic for the specified port.
+ This keyword is available only when HAProxy is built with USE_LINUX_TPROXY=1.
+ This parameter is only compatible with TCPv4 and TCPv6 sockets, depending on
+ kernel version. Some distribution kernels include backports of the feature,
+ so check for support with your vendor.
+
+v4v6
+ Is an optional keyword which is supported only on most recent systems
+ including Linux kernels >= 2.4.21. It is used to bind a socket to both IPv4
+ and IPv6 when it uses the default address. Doing so is sometimes necessary
+ on systems which bind to IPv6 only by default. It has no effect on non-IPv6
+ sockets, and is overridden by the "v6only" option.
+
+v6only
+ Is an optional keyword which is supported only on most recent systems
+ including Linux kernels >= 2.4.21. It is used to bind a socket to IPv6 only
+ when it uses the default address. Doing so is sometimes preferred to doing it
+ system-wide as it is per-listener. It has no effect on non-IPv6 sockets and
+ has precedence over the "v4v6" option.
+
+uid <uid>
+ Sets the owner of the UNIX sockets to the designated system uid. It can also
+ be set by default in the global section's "unix-bind" statement. Note that
+ some platforms simply ignore this. This setting is equivalent to the "user"
+ setting except that the user numeric ID is used instead of its name. This
+ setting is ignored by non UNIX sockets.
+
+user <user>
+ Sets the owner of the UNIX sockets to the designated system user. It can also
+ be set by default in the global section's "unix-bind" statement. Note that
+ some platforms simply ignore this. This setting is equivalent to the "uid"
+ setting except that the user name is used instead of its uid. This setting is
+ ignored by non UNIX sockets.
+
+verify [none|optional|required]
+ This setting is only available when support for OpenSSL was built in. If set
+ to 'none', client certificate is not requested. This is the default. In other
+ cases, a client certificate is requested. If the client does not provide a
+ certificate after the request and if 'verify' is set to 'required', then the
+ handshake is aborted, while it would have succeeded if set to 'optional'. The
+ certificate provided by the client is always verified using CAs from
+ 'ca-file' and optional CRLs from 'crl-file'. On verify failure the handshake
+ is aborted, regardless of the 'verify' option, unless the error code exactly
+ matches one of those listed with 'ca-ignore-err' or 'crt-ignore-err'.
+
+5.2. Server and default-server options
+------------------------------------
+
+The "server" and "default-server" keywords support a certain number of settings
+which are all passed as arguments on the server line. The order in which those
+arguments appear does not count, and they are all optional. Some of those
+settings are single words (booleans) while others expect one or several values
+after them. In this case, the values must immediately follow the setting name.
+Except default-server, all those settings must be specified after the server's
+address if they are used:
+
+ server <name> <address>[:port] [settings ...]
+ default-server [settings ...]
+
+Note that all these settings are supported both by "server" and "default-server"
+keywords, except "id" which is only supported by "server".
+
+The currently supported settings are the following ones.
+
+addr <ipv4|ipv6>
+ Using the "addr" parameter, it becomes possible to use a different IP address
+ to send health-checks or to probe the agent-check. On some servers, it may be
+ desirable to dedicate an IP address to specific component able to perform
+ complex tests which are more suitable to health-checks than the application.
+ This parameter is ignored if the "check" parameter is not set. See also the
+ "port" parameter.
+
+agent-check
+ Enable an auxiliary agent check which is run independently of a regular
+ health check. An agent health check is performed by making a TCP connection
+ to the port set by the "agent-port" parameter and reading an ASCII string
+ terminated by the first '\r' or '\n' met. The string is made of a series of
+ words delimited by spaces, tabs or commas in any order, each consisting of :
+
+ - An ASCII representation of a positive integer percentage, e.g. "75%".
+ Values in this format will set the weight proportional to the initial
+ weight of a server as configured when HAProxy starts. Note that a zero
+ weight is reported on the stats page as "DRAIN" since it has the same
+ effect on the server (it's removed from the LB farm).
+
+ - The string "maxconn:" followed by an integer (no space between). Values
+ in this format will set the maxconn of a server. The maximum number of
+ connections advertised needs to be multiplied by the number of load
+ balancers and different backends that use this health check to get the
+ total number of connections the server might receive. Example: maxconn:30
+
+ - The word "ready". This will turn the server's administrative state to the
+ READY mode, thus canceling any DRAIN or MAINT state
+
+ - The word "drain". This will turn the server's administrative state to the
+ DRAIN mode, thus it will not accept any new connections other than those
+ that are accepted via persistence.
+
+ - The word "maint". This will turn the server's administrative state to the
+ MAINT mode, thus it will not accept any new connections at all, and health
+ checks will be stopped.
+
+ - The words "down", "fail", or "stopped", optionally followed by a
+ description string after a sharp ('#'). All of these mark the server's
+ operating state as DOWN, but since the word itself is reported on the stats
+ page, the difference allows an administrator to know if the situation was
+ expected or not : the service may intentionally be stopped, may appear up
+ but fail some validity tests, or may be seen as down (e.g. missing process,
+ or port not responding).
+
+ - The word "up" sets back the server's operating state as UP if health checks
+ also report that the service is accessible.
+
+ Parameters which are not advertised by the agent are not changed. For
+ example, an agent might be designed to monitor CPU usage and only report a
+ relative weight and never interact with the operating status. Similarly, an
+ agent could be designed as an end-user interface with 3 radio buttons
+ allowing an administrator to change only the administrative state. However,
+ it is important to consider that only the agent may revert its own actions,
+ so if a server is set to DRAIN mode or to DOWN state using the agent, the
+ agent must implement the other equivalent actions to bring the service into
+ operations again.
+
+ Failure to connect to the agent is not considered an error as connectivity
+ is tested by the regular health check which is enabled by the "check"
+ parameter. Warning though, it is not a good idea to stop an agent after it
+ reports "down", since only an agent reporting "up" will be able to turn the
+ server up again. Note that the CLI on the Unix stats socket is also able to
+ force an agent's result in order to work around a bogus agent if needed.
+
+ Requires the "agent-port" parameter to be set. See also the "agent-inter"
+ and "no-agent-check" parameters.
+
+agent-send <string>
+ If this option is specified, HAProxy will send the given string (verbatim)
+ to the agent server upon connection. You could, for example, encode
+ the backend name into this string, which would enable your agent to send
+ different responses based on the backend. Make sure to include a '\n' if
+ you want to terminate your request with a newline.
+
+agent-inter <delay>
+ The "agent-inter" parameter sets the interval between two agent checks
+ to <delay> milliseconds. If left unspecified, the delay defaults to 2000 ms.
+
+ Just as with every other time-based parameter, it may be entered in any
+ other explicit unit among { us, ms, s, m, h, d }. The "agent-inter"
+ parameter also serves as a timeout for agent checks "timeout check" is
+ not set. In order to reduce "resonance" effects when multiple servers are
+ hosted on the same hardware, the agent and health checks of all servers
+ are started with a small time offset between them. It is also possible to
+ add some random noise in the agent and health checks interval using the
+ global "spread-checks" keyword. This makes sense for instance when a lot
+ of backends use the same servers.
+
+ See also the "agent-check" and "agent-port" parameters.
+
+agent-addr <addr>
+ The "agent-addr" parameter sets address for agent check.
+
+ You can offload agent-check to another target, so you can make single place
+ managing status and weights of servers defined in HAProxy in case you can't
+ make self-aware and self-managing services. You can specify both IP or
+ hostname, it will be resolved.
+
+agent-port <port>
+ The "agent-port" parameter sets the TCP port used for agent checks.
+
+ See also the "agent-check" and "agent-inter" parameters.
+
+allow-0rtt
+ Allow sending early data to the server when using TLS 1.3.
+ Note that early data will be sent only if the client used early data, or
+ if the backend uses "retry-on" with the "0rtt-rejected" keyword.
+
+alpn <protocols>
+ This enables the TLS ALPN extension and advertises the specified protocol
+ list as supported on top of ALPN. The protocol list consists in a comma-
+ delimited list of protocol names, for instance: "http/1.1,http/1.0" (without
+ quotes). This requires that the SSL library is built with support for TLS
+ extensions enabled (check with haproxy -vv). The ALPN extension replaces the
+ initial NPN extension. ALPN is required to connect to HTTP/2 servers.
+ Versions of OpenSSL prior to 1.0.2 didn't support ALPN and only supposed the
+ now obsolete NPN extension.
+ If both HTTP/2 and HTTP/1.1 are expected to be supported, both versions can
+ be advertised, in order of preference, like below :
+
+ server 127.0.0.1:443 ssl crt pub.pem alpn h2,http/1.1
+
+ See also "ws" to use an alternative ALPN for websocket streams.
+
+backup
+ When "backup" is present on a server line, the server is only used in load
+ balancing when all other non-backup servers are unavailable. Requests coming
+ with a persistence cookie referencing the server will always be served
+ though. By default, only the first operational backup server is used, unless
+ the "allbackups" option is set in the backend. See also the "no-backup" and
+ "allbackups" options.
+
+ca-file <cafile>
+ This setting is only available when support for OpenSSL was built in. It
+ designates a PEM file from which to load CA certificates used to verify
+ server's certificate. It is possible to load a directory containing multiple
+ CAs, in this case HAProxy will try to load every ".pem", ".crt", ".cer", and
+ .crl" available in the directory, files starting with a dot are ignored.
+
+ In order to use the trusted CAs of your system, the "@system-ca" parameter
+ could be used in place of the cafile. The location of this directory could be
+ overwritten by setting the SSL_CERT_DIR environment variable.
+
+check
+ This option enables health checks on a server:
+ - when not set, no health checking is performed, and the server is always
+ considered available.
+ - when set and no other check method is configured, the server is considered
+ available when a connection can be established at the highest configured
+ transport layer. This means TCP by default, or SSL/TLS when "ssl" or
+ "check-ssl" are set, both possibly combined with connection prefixes such
+ as a PROXY protocol header when "send-proxy" or "check-send-proxy" are
+ set. This behavior is slightly different for dynamic servers, read the
+ following paragraphs for more details.
+ - when set and an application-level health check is defined, the
+ application-level exchanges are performed on top of the configured
+ transport layer and the server is considered available if all of the
+ exchanges succeed.
+
+ By default, health checks are performed on the same address and port as
+ configured on the server, using the same encapsulation parameters (SSL/TLS,
+ proxy-protocol header, etc... ). It is possible to change the destination
+ address using "addr" and the port using "port". When done, it is assumed the
+ server isn't checked on the service port, and configured encapsulation
+ parameters are not reused. One must explicitly set "check-send-proxy" to send
+ connection headers, "check-ssl" to use SSL/TLS.
+
+ Note that the implicit configuration of ssl and PROXY protocol is not
+ performed for dynamic servers. In this case, it is required to explicitly
+ use "check-ssl" and "check-send-proxy" when wanted, even if the check port is
+ not overridden.
+
+ When "sni" or "alpn" are set on the server line, their value is not used for
+ health checks and one must use "check-sni" or "check-alpn".
+
+ The default source address for health check traffic is the same as the one
+ defined in the backend. It can be changed with the "source" keyword.
+
+ The interval between checks can be set using the "inter" keyword, and the
+ "rise" and "fall" keywords can be used to define how many successful or
+ failed health checks are required to flag a server available or not
+ available.
+
+ Optional application-level health checks can be configured with "option
+ httpchk", "option mysql-check" "option smtpchk", "option pgsql-check",
+ "option ldap-check", or "option redis-check".
+
+ Example:
+ # simple tcp check
+ backend foo
+ server s1 192.168.0.1:80 check
+ # this does a tcp connect + tls handshake
+ backend foo
+ server s1 192.168.0.1:443 ssl check
+ # simple tcp check is enough for check success
+ backend foo
+ option tcp-check
+ tcp-check connect
+ server s1 192.168.0.1:443 ssl check
+
+check-send-proxy
+ This option forces emission of a PROXY protocol line with outgoing health
+ checks, regardless of whether the server uses send-proxy or not for the
+ normal traffic. By default, the PROXY protocol is enabled for health checks
+ if it is already enabled for normal traffic and if no "port" nor "addr"
+ directive is present. However, if such a directive is present, the
+ "check-send-proxy" option needs to be used to force the use of the
+ protocol. See also the "send-proxy" option for more information.
+
+check-alpn <protocols>
+ Defines which protocols to advertise with ALPN. The protocol list consists in
+ a comma-delimited list of protocol names, for instance: "http/1.1,http/1.0"
+ (without quotes). If it is not set, the server ALPN is used.
+
+check-proto <name>
+ Forces the multiplexer's protocol to use for the server's health-check
+ connections. It must be compatible with the health-check type (TCP or
+ HTTP). It must also be usable on the backend side. The list of available
+ protocols is reported in haproxy -vv. The protocols properties are
+ reported : the mode (TCP/HTTP), the side (FE/BE), the mux name and its flags.
+
+ Some protocols are subject to the head-of-line blocking on server side
+ (flag=HOL_RISK). Finally some protocols don't support upgrades (flag=NO_UPG).
+ The HTX compatibility is also reported (flag=HTX).
+
+ Here are the protocols that may be used as argument to a "check-proto"
+ directive on a server line:
+
+ h2 : mode=HTTP side=FE|BE mux=H2 flags=HTX|HOL_RISK|NO_UPG
+ fcgi : mode=HTTP side=BE mux=FCGI flags=HTX|HOL_RISK|NO_UPG
+ h1 : mode=HTTP side=FE|BE mux=H1 flags=HTX|NO_UPG
+ none : mode=TCP side=FE|BE mux=PASS flags=NO_UPG
+
+ Idea behind this option is to bypass the selection of the best multiplexer's
+ protocol for health-check connections established to this server.
+ If not defined, the server one will be used, if set.
+
+check-sni <sni>
+ This option allows you to specify the SNI to be used when doing health checks
+ over SSL. It is only possible to use a string to set <sni>. If you want to
+ set a SNI for proxied traffic, see "sni".
+
+check-ssl
+ This option forces encryption of all health checks over SSL, regardless of
+ whether the server uses SSL or not for the normal traffic. This is generally
+ used when an explicit "port" or "addr" directive is specified and SSL health
+ checks are not inherited. It is important to understand that this option
+ inserts an SSL transport layer below the checks, so that a simple TCP connect
+ check becomes an SSL connect, which replaces the old ssl-hello-chk. The most
+ common use is to send HTTPS checks by combining "httpchk" with SSL checks.
+ All SSL settings are common to health checks and traffic (e.g. ciphers).
+ See the "ssl" option for more information and "no-check-ssl" to disable
+ this option.
+
+check-via-socks4
+ This option enables outgoing health checks using upstream socks4 proxy. By
+ default, the health checks won't go through socks tunnel even it was enabled
+ for normal traffic.
+
+ciphers <ciphers>
+ This setting is only available when support for OpenSSL was built in. This
+ option sets the string describing the list of cipher algorithms that is
+ negotiated during the SSL/TLS handshake with the server. The format of the
+ string is defined in "man 1 ciphers" from OpenSSL man pages. For background
+ information and recommendations see e.g.
+ (https://wiki.mozilla.org/Security/Server_Side_TLS) and
+ (https://mozilla.github.io/server-side-tls/ssl-config-generator/). For TLSv1.3
+ cipher configuration, please check the "ciphersuites" keyword.
+
+ciphersuites <ciphersuites>
+ This setting is only available when support for OpenSSL was built in and
+ OpenSSL 1.1.1 or later was used to build HAProxy. This option sets the string
+ describing the list of cipher algorithms that is negotiated during the TLS
+ 1.3 handshake with the server. The format of the string is defined in
+ "man 1 ciphers" from OpenSSL man pages under the "ciphersuites" section.
+ For cipher configuration for TLSv1.2 and earlier, please check the "ciphers"
+ keyword.
+
+client-sigalgs <sigalgs>
+ This setting is only available when support for OpenSSL was built in. It sets
+ the string describing the list of signature algorithms related to client
+ authentication that are negotiated . The format of the string is defined in
+ "man 3 SSL_CTX_set1_client_sigalgs" from the OpenSSL man pages. It is not
+ recommended to use this setting if no specific usecase was identified.
+
+cookie <value>
+ The "cookie" parameter sets the cookie value assigned to the server to
+ <value>. This value will be checked in incoming requests, and the first
+ operational server possessing the same value will be selected. In return, in
+ cookie insertion or rewrite modes, this value will be assigned to the cookie
+ sent to the client. There is nothing wrong in having several servers sharing
+ the same cookie value, and it is in fact somewhat common between normal and
+ backup servers. See also the "cookie" keyword in backend section.
+
+crl-file <crlfile>
+ This setting is only available when support for OpenSSL was built in. It
+ designates a PEM file from which to load certificate revocation list used
+ to verify server's certificate.
+
+crt <cert>
+ This setting is only available when support for OpenSSL was built in.
+ It designates a PEM file from which to load both a certificate and the
+ associated private key. This file can be built by concatenating both PEM
+ files into one. This certificate will be sent if the server send a client
+ certificate request.
+
+ If the file does not contain a private key, HAProxy will try to load the key
+ at the same path suffixed by a ".key" (provided the "ssl-load-extra-files"
+ option is set accordingly).
+
+curves <curves>
+ This setting is only available when support for OpenSSL was built in. It sets
+ the string describing the list of elliptic curves algorithms ("curve suite")
+ that are negotiated during the SSL/TLS handshake with ECDHE. The format of the
+ string is a colon-delimited list of curve name.
+ Example: "X25519:P-256" (without quote)
+
+disabled
+ The "disabled" keyword starts the server in the "disabled" state. That means
+ that it is marked down in maintenance mode, and no connection other than the
+ ones allowed by persist mode will reach it. It is very well suited to setup
+ new servers, because normal traffic will never reach them, while it is still
+ possible to test the service by making use of the force-persist mechanism.
+ See also "enabled" setting.
+
+enabled
+ This option may be used as 'server' setting to reset any 'disabled'
+ setting which would have been inherited from 'default-server' directive as
+ default value.
+ It may also be used as 'default-server' setting to reset any previous
+ 'default-server' 'disabled' setting.
+
+error-limit <count>
+ If health observing is enabled, the "error-limit" parameter specifies the
+ number of consecutive errors that triggers event selected by the "on-error"
+ option. By default it is set to 10 consecutive errors.
+
+ See also the "check", "error-limit" and "on-error".
+
+fall <count>
+ The "fall" parameter states that a server will be considered as dead after
+ <count> consecutive unsuccessful health checks. This value defaults to 3 if
+ unspecified. See also the "check", "inter" and "rise" parameters.
+
+force-sslv3
+ This option enforces use of SSLv3 only when SSL is used to communicate with
+ the server. SSLv3 is generally less expensive than the TLS counterparts for
+ high connection rates. This option is also available on global statement
+ "ssl-default-server-options". See also "ssl-min-ver" and ssl-max-ver".
+
+force-tlsv10
+ This option enforces use of TLSv1.0 only when SSL is used to communicate with
+ the server. This option is also available on global statement
+ "ssl-default-server-options". See also "ssl-min-ver" and ssl-max-ver".
+
+force-tlsv11
+ This option enforces use of TLSv1.1 only when SSL is used to communicate with
+ the server. This option is also available on global statement
+ "ssl-default-server-options". See also "ssl-min-ver" and ssl-max-ver".
+
+force-tlsv12
+ This option enforces use of TLSv1.2 only when SSL is used to communicate with
+ the server. This option is also available on global statement
+ "ssl-default-server-options". See also "ssl-min-ver" and ssl-max-ver".
+
+force-tlsv13
+ This option enforces use of TLSv1.3 only when SSL is used to communicate with
+ the server. This option is also available on global statement
+ "ssl-default-server-options". See also "ssl-min-ver" and ssl-max-ver".
+
+id <value>
+ Set a persistent ID for the server. This ID must be positive and unique for
+ the proxy. An unused ID will automatically be assigned if unset. The first
+ assigned value will be 1. This ID is currently only returned in statistics.
+
+init-addr {last | libc | none | <ip>},[...]*
+ Indicate in what order the server's address should be resolved upon startup
+ if it uses an FQDN. Attempts are made to resolve the address by applying in
+ turn each of the methods mentioned in the comma-delimited list. The first
+ method which succeeds is used. If the end of the list is reached without
+ finding a working method, an error is thrown. Method "last" suggests to pick
+ the address which appears in the state file (see "server-state-file"). Method
+ "libc" uses the libc's internal resolver (gethostbyname() or getaddrinfo()
+ depending on the operating system and build options). Method "none"
+ specifically indicates that the server should start without any valid IP
+ address in a down state. It can be useful to ignore some DNS issues upon
+ startup, waiting for the situation to get fixed later. Finally, an IP address
+ (IPv4 or IPv6) may be provided. It can be the currently known address of the
+ server (e.g. filled by a configuration generator), or the address of a dummy
+ server used to catch old sessions and present them with a decent error
+ message for example. When the "first" load balancing algorithm is used, this
+ IP address could point to a fake server used to trigger the creation of new
+ instances on the fly. This option defaults to "last,libc" indicating that the
+ previous address found in the state file (if any) is used first, otherwise
+ the libc's resolver is used. This ensures continued compatibility with the
+ historic behavior.
+
+ Example:
+ defaults
+ # never fail on address resolution
+ default-server init-addr last,libc,none
+
+inter <delay>
+fastinter <delay>
+downinter <delay>
+ The "inter" parameter sets the interval between two consecutive health checks
+ to <delay> milliseconds. If left unspecified, the delay defaults to 2000 ms.
+ It is also possible to use "fastinter" and "downinter" to optimize delays
+ between checks depending on the server state :
+
+ Server state | Interval used
+ ----------------------------------------+----------------------------------
+ UP 100% (non-transitional) | "inter"
+ ----------------------------------------+----------------------------------
+ Transitionally UP (going down "fall"), | "fastinter" if set,
+ Transitionally DOWN (going up "rise"), | "inter" otherwise.
+ or yet unchecked. |
+ ----------------------------------------+----------------------------------
+ DOWN 100% (non-transitional) | "downinter" if set,
+ | "inter" otherwise.
+ ----------------------------------------+----------------------------------
+
+ Just as with every other time-based parameter, they can be entered in any
+ other explicit unit among { us, ms, s, m, h, d }. The "inter" parameter also
+ serves as a timeout for health checks sent to servers if "timeout check" is
+ not set. In order to reduce "resonance" effects when multiple servers are
+ hosted on the same hardware, the agent and health checks of all servers
+ are started with a small time offset between them. It is also possible to
+ add some random noise in the agent and health checks interval using the
+ global "spread-checks" keyword. This makes sense for instance when a lot
+ of backends use the same servers. The global "tune.max-checks-per-thread"
+ setting, if defined to a non-nul value, will limit the number of concurrent
+ checks being performed at once on any given thread. In order to achieve this,
+ haproxy will put in a queue the checks that were about to start on a thread
+ that has reached this limit, until another check finishes. This will have for
+ effect to extend the effective check interval. In such a case, reducing the
+ "inter" setting will have a very limited effect as it will not be able to
+ reduce the time spent in the queue.
+
+log-bufsize <bufsize>
+ The "log-bufsize" specifies the ring bufsize to use for the implicit ring
+ that will be associated to the log server in a log backend. When not
+ specified, this defaults to BUFSIZE. Use of a greater value will increase
+ memory usage but can help to prevent the loss of log messages with slow
+ servers since the buffer will be able to hold more pending messages.
+ This keyword may only be used in log backend sections (with "mode log")
+
+log-proto <logproto>
+ The "log-proto" specifies the protocol used to forward event messages to
+ a server configured in a log or ring section. Possible values are "legacy"
+ and "octet-count" corresponding respectively to "Non-transparent-framing"
+ and "Octet counting" in rfc6587. "legacy" is the default.
+
+maxconn <maxconn>
+ The "maxconn" parameter specifies the maximal number of concurrent
+ connections that will be sent to this server. If the number of incoming
+ concurrent connections goes higher than this value, they will be queued,
+ waiting for a slot to be released. This parameter is very important as it can
+ save fragile servers from going down under extreme loads. If a "minconn"
+ parameter is specified, the limit becomes dynamic. The default value is "0"
+ which means unlimited. See also the "minconn" and "maxqueue" parameters, and
+ the backend's "fullconn" keyword.
+
+ In HTTP mode this parameter limits the number of concurrent requests instead
+ of the number of connections. Multiple requests might be multiplexed over a
+ single TCP connection to the server. As an example if you specify a maxconn
+ of 50 you might see between 1 and 50 actual server connections, but no more
+ than 50 concurrent requests.
+
+maxqueue <maxqueue>
+ The "maxqueue" parameter specifies the maximal number of connections which
+ will wait in the queue for this server. If this limit is reached, next
+ requests will be redispatched to other servers instead of indefinitely
+ waiting to be served. This will break persistence but may allow people to
+ quickly re-log in when the server they try to connect to is dying. Some load
+ balancing algorithms such as leastconn take this into account and accept to
+ add requests into a server's queue up to this value if it is explicitly set
+ to a value greater than zero, which often allows to better smooth the load
+ when dealing with single-digit maxconn values. The default value is "0" which
+ means the queue is unlimited. See also the "maxconn" and "minconn" parameters
+ and "balance leastconn".
+
+max-reuse <count>
+ The "max-reuse" argument indicates the HTTP connection processors that they
+ should not reuse a server connection more than this number of times to send
+ new requests. Permitted values are -1 (the default), which disables this
+ limit, or any positive value. Value zero will effectively disable keep-alive.
+ This is only used to work around certain server bugs which cause them to leak
+ resources over time. The argument is not necessarily respected by the lower
+ layers as there might be technical limitations making it impossible to
+ enforce. At least HTTP/2 connections to servers will respect it.
+
+minconn <minconn>
+ When the "minconn" parameter is set, the maxconn limit becomes a dynamic
+ limit following the backend's load. The server will always accept at least
+ <minconn> connections, never more than <maxconn>, and the limit will be on
+ the ramp between both values when the backend has less than <fullconn>
+ concurrent connections. This makes it possible to limit the load on the
+ server during normal loads, but push it further for important loads without
+ overloading the server during exceptional loads. See also the "maxconn"
+ and "maxqueue" parameters, as well as the "fullconn" backend keyword.
+
+namespace <name>
+ On Linux, it is possible to specify which network namespace a socket will
+ belong to. This directive makes it possible to explicitly bind a server to
+ a namespace different from the default one. Please refer to your operating
+ system's documentation to find more details about network namespaces.
+
+no-agent-check
+ This option may be used as "server" setting to reset any "agent-check"
+ setting which would have been inherited from "default-server" directive as
+ default value.
+ It may also be used as "default-server" setting to reset any previous
+ "default-server" "agent-check" setting.
+
+no-backup
+ This option may be used as "server" setting to reset any "backup"
+ setting which would have been inherited from "default-server" directive as
+ default value.
+ It may also be used as "default-server" setting to reset any previous
+ "default-server" "backup" setting.
+
+no-check
+ This option may be used as "server" setting to reset any "check"
+ setting which would have been inherited from "default-server" directive as
+ default value.
+ It may also be used as "default-server" setting to reset any previous
+ "default-server" "check" setting.
+
+no-check-ssl
+ This option may be used as "server" setting to reset any "check-ssl"
+ setting which would have been inherited from "default-server" directive as
+ default value.
+ It may also be used as "default-server" setting to reset any previous
+ "default-server" "check-ssl" setting.
+
+no-send-proxy
+ This option may be used as "server" setting to reset any "send-proxy"
+ setting which would have been inherited from "default-server" directive as
+ default value.
+ It may also be used as "default-server" setting to reset any previous
+ "default-server" "send-proxy" setting.
+
+no-send-proxy-v2
+ This option may be used as "server" setting to reset any "send-proxy-v2"
+ setting which would have been inherited from "default-server" directive as
+ default value.
+ It may also be used as "default-server" setting to reset any previous
+ "default-server" "send-proxy-v2" setting.
+
+no-send-proxy-v2-ssl
+ This option may be used as "server" setting to reset any "send-proxy-v2-ssl"
+ setting which would have been inherited from "default-server" directive as
+ default value.
+ It may also be used as "default-server" setting to reset any previous
+ "default-server" "send-proxy-v2-ssl" setting.
+
+no-send-proxy-v2-ssl-cn
+ This option may be used as "server" setting to reset any "send-proxy-v2-ssl-cn"
+ setting which would have been inherited from "default-server" directive as
+ default value.
+ It may also be used as "default-server" setting to reset any previous
+ "default-server" "send-proxy-v2-ssl-cn" setting.
+
+no-ssl
+ This option may be used as "server" setting to reset any "ssl"
+ setting which would have been inherited from "default-server" directive as
+ default value.
+ It may also be used as "default-server" setting to reset any previous
+ "default-server" "ssl" setting.
+
+ Note that using `default-server ssl` setting and `no-ssl` on server will
+ however init SSL connection, so it can be later be enabled through the
+ runtime API: see `set server` commands in management doc.
+
+no-ssl-reuse
+ This option disables SSL session reuse when SSL is used to communicate with
+ the server. It will force the server to perform a full handshake for every
+ new connection. It's probably only useful for benchmarking, troubleshooting,
+ and for paranoid users.
+
+no-sslv3
+ This option disables support for SSLv3 when SSL is used to communicate with
+ the server. Note that SSLv2 is disabled in the code and cannot be enabled
+ using any configuration option. Use "ssl-min-ver" and "ssl-max-ver" instead.
+
+ Supported in default-server: No
+
+no-tls-tickets
+ This setting is only available when support for OpenSSL was built in. It
+ disables the stateless session resumption (RFC 5077 TLS Ticket
+ extension) and force to use stateful session resumption. Stateless
+ session resumption is more expensive in CPU usage for servers. This option
+ is also available on global statement "ssl-default-server-options".
+ The TLS ticket mechanism is only used up to TLS 1.2.
+ Forward Secrecy is compromised with TLS tickets, unless ticket keys
+ are periodically rotated (via reload or by using "tls-ticket-keys").
+ See also "tls-tickets".
+
+no-tlsv10
+ This option disables support for TLSv1.0 when SSL is used to communicate with
+ the server. Note that SSLv2 is disabled in the code and cannot be enabled
+ using any configuration option. TLSv1 is more expensive than SSLv3 so it
+ often makes sense to disable it when communicating with local servers. This
+ option is also available on global statement "ssl-default-server-options".
+ Use "ssl-min-ver" and "ssl-max-ver" instead.
+
+ Supported in default-server: No
+
+no-tlsv11
+ This option disables support for TLSv1.1 when SSL is used to communicate with
+ the server. Note that SSLv2 is disabled in the code and cannot be enabled
+ using any configuration option. TLSv1 is more expensive than SSLv3 so it
+ often makes sense to disable it when communicating with local servers. This
+ option is also available on global statement "ssl-default-server-options".
+ Use "ssl-min-ver" and "ssl-max-ver" instead.
+
+ Supported in default-server: No
+
+no-tlsv12
+ This option disables support for TLSv1.2 when SSL is used to communicate with
+ the server. Note that SSLv2 is disabled in the code and cannot be enabled
+ using any configuration option. TLSv1 is more expensive than SSLv3 so it
+ often makes sense to disable it when communicating with local servers. This
+ option is also available on global statement "ssl-default-server-options".
+ Use "ssl-min-ver" and "ssl-max-ver" instead.
+
+ Supported in default-server: No
+
+no-tlsv13
+ This option disables support for TLSv1.3 when SSL is used to communicate with
+ the server. Note that SSLv2 is disabled in the code and cannot be enabled
+ using any configuration option. TLSv1 is more expensive than SSLv3 so it
+ often makes sense to disable it when communicating with local servers. This
+ option is also available on global statement "ssl-default-server-options".
+ Use "ssl-min-ver" and "ssl-max-ver" instead.
+
+ Supported in default-server: No
+
+no-verifyhost
+ This option may be used as "server" setting to reset any "verifyhost"
+ setting which would have been inherited from "default-server" directive as
+ default value.
+ It may also be used as "default-server" setting to reset any previous
+ "default-server" "verifyhost" setting.
+
+no-tfo
+ This option may be used as "server" setting to reset any "tfo"
+ setting which would have been inherited from "default-server" directive as
+ default value.
+ It may also be used as "default-server" setting to reset any previous
+ "default-server" "tfo" setting.
+
+non-stick
+ Never add connections allocated to this sever to a stick-table.
+ This may be used in conjunction with backup to ensure that
+ stick-table persistence is disabled for backup servers.
+
+npn <protocols>
+ This enables the NPN TLS extension and advertises the specified protocol list
+ as supported on top of NPN. The protocol list consists in a comma-delimited
+ list of protocol names, for instance: "http/1.1,http/1.0" (without quotes).
+ This requires that the SSL library is built with support for TLS extensions
+ enabled (check with haproxy -vv). Note that the NPN extension has been
+ replaced with the ALPN extension (see the "alpn" keyword), though this one is
+ only available starting with OpenSSL 1.0.2.
+
+observe <mode>
+ This option enables health adjusting based on observing communication with
+ the server. By default this functionality is disabled and enabling it also
+ requires to enable health checks. There are two supported modes: "layer4" and
+ "layer7". In layer4 mode, only successful/unsuccessful tcp connections are
+ significant. In layer7, which is only allowed for http proxies, responses
+ received from server are verified, like valid/wrong http code, unparsable
+ headers, a timeout, etc. Valid status codes include 100 to 499, 501 and 505.
+
+ See also the "check", "on-error" and "error-limit".
+
+on-error <mode>
+ Select what should happen when enough consecutive errors are detected.
+ Currently, four modes are available:
+ - fastinter: force fastinter
+ - fail-check: simulate a failed check, also forces fastinter (default)
+ - sudden-death: simulate a pre-fatal failed health check, one more failed
+ check will mark a server down, forces fastinter
+ - mark-down: mark the server immediately down and force fastinter
+
+ See also the "check", "observe" and "error-limit".
+
+on-marked-down <action>
+ Modify what occurs when a server is marked down.
+ Currently one action is available:
+ - shutdown-sessions: Shutdown peer streams. When this setting is enabled,
+ all connections to the server are immediately terminated when the server
+ goes down. It might be used if the health check detects more complex cases
+ than a simple connection status, and long timeouts would cause the service
+ to remain unresponsive for too long a time. For instance, a health check
+ might detect that a database is stuck and that there's no chance to reuse
+ existing connections anymore. Connections killed this way are logged with
+ a 'D' termination code (for "Down").
+
+ Actions are disabled by default
+
+on-marked-up <action>
+ Modify what occurs when a server is marked up.
+ Currently one action is available:
+ - shutdown-backup-sessions: Shutdown streams on all backup servers. This is
+ done only if the server is not in backup state and if it is not disabled
+ (it must have an effective weight > 0). This can be used sometimes to force
+ an active server to take all the traffic back after recovery when dealing
+ with long sessions (e.g. LDAP, SQL, ...). Doing this can cause more trouble
+ than it tries to solve (e.g. incomplete transactions), so use this feature
+ with extreme care. Streams killed because a server comes up are logged
+ with an 'U' termination code (for "Up").
+
+ Actions are disabled by default
+
+pool-low-conn <max>
+ Set a low threshold on the number of idling connections for a server, below
+ which a thread will not try to steal a connection from another thread. This
+ can be useful to improve CPU usage patterns in scenarios involving many very
+ fast servers, in order to ensure all threads will keep a few idle connections
+ all the time instead of letting them accumulate over one thread and migrating
+ them from thread to thread. Typical values of twice the number of threads
+ seem to show very good performance already with sub-millisecond response
+ times. The default is zero, indicating that any idle connection can be used
+ at any time. It is the recommended setting for normal use. This only applies
+ to connections that can be shared according to the same principles as those
+ applying to "http-reuse". In case connection sharing between threads would
+ be disabled via "tune.idle-pool.shared", it can become very important to use
+ this setting to make sure each thread always has a few connections, or the
+ connection reuse rate will decrease as thread count increases.
+
+pool-max-conn <max>
+ Set the maximum number of idling connections for a server. -1 means unlimited
+ connections, 0 means no idle connections. The default is -1. When idle
+ connections are enabled, orphaned idle connections which do not belong to any
+ client session anymore are moved to a dedicated pool so that they remain
+ usable by future clients. This only applies to connections that can be shared
+ according to the same principles as those applying to "http-reuse".
+
+pool-purge-delay <delay>
+ Sets the delay to start purging idle connections. Each <delay> interval, half
+ of the idle connections are closed. 0 means we don't keep any idle connection.
+ The default is 5s.
+
+port <port>
+ Using the "port" parameter, it becomes possible to use a different port to
+ send health-checks or to probe the agent-check. On some servers, it may be
+ desirable to dedicate a port to a specific component able to perform complex
+ tests which are more suitable to health-checks than the application. It is
+ common to run a simple script in inetd for instance. This parameter is
+ ignored if the "check" parameter is not set. See also the "addr" parameter.
+
+proto <name>
+ Forces the multiplexer's protocol to use for the outgoing connections to this
+ server. It must be compatible with the mode of the backend (TCP or HTTP). It
+ must also be usable on the backend side. The list of available protocols is
+ reported in haproxy -vv.The protocols properties are reported : the mode
+ (TCP/HTTP), the side (FE/BE), the mux name and its flags.
+
+ Some protocols are subject to the head-of-line blocking on server side
+ (flag=HOL_RISK). Finally some protocols don't support upgrades (flag=NO_UPG).
+ The HTX compatibility is also reported (flag=HTX).
+
+ Here are the protocols that may be used as argument to a "proto" directive on
+ a server line :
+
+ h2 : mode=HTTP side=FE|BE mux=H2 flags=HTX|HOL_RISK|NO_UPG
+ fcgi : mode=HTTP side=BE mux=FCGI flags=HTX|HOL_RISK|NO_UPG
+ h1 : mode=HTTP side=FE|BE mux=H1 flags=HTX|NO_UPG
+ none : mode=TCP side=FE|BE mux=PASS flags=NO_UPG
+
+ Idea behind this option is to bypass the selection of the best multiplexer's
+ protocol for all connections established to this server.
+
+ See also "ws" to use an alternative protocol for websocket streams.
+
+redir <prefix>
+ The "redir" parameter enables the redirection mode for all GET and HEAD
+ requests addressing this server. This means that instead of having HAProxy
+ forward the request to the server, it will send an "HTTP 302" response with
+ the "Location" header composed of this prefix immediately followed by the
+ requested URI beginning at the leading '/' of the path component. That means
+ that no trailing slash should be used after <prefix>. All invalid requests
+ will be rejected, and all non-GET or HEAD requests will be normally served by
+ the server. Note that since the response is completely forged, no header
+ mangling nor cookie insertion is possible in the response. However, cookies in
+ requests are still analyzed, making this solution completely usable to direct
+ users to a remote location in case of local disaster. Main use consists in
+ increasing bandwidth for static servers by having the clients directly
+ connect to them. Note: never use a relative location here, it would cause a
+ loop between the client and HAProxy!
+
+ Example : server srv1 192.168.1.1:80 redir http://image1.mydomain.com check
+
+rise <count>
+ The "rise" parameter states that a server will be considered as operational
+ after <count> consecutive successful health checks. This value defaults to 2
+ if unspecified. See also the "check", "inter" and "fall" parameters.
+
+resolve-opts <option>,<option>,...
+ Comma separated list of options to apply to DNS resolution linked to this
+ server.
+
+ Available options:
+
+ * allow-dup-ip
+ By default, HAProxy prevents IP address duplication in a backend when DNS
+ resolution at runtime is in operation.
+ That said, for some cases, it makes sense that two servers (in the same
+ backend, being resolved by the same FQDN) have the same IP address.
+ For such case, simply enable this option.
+ This is the opposite of prevent-dup-ip.
+
+ * ignore-weight
+ Ignore any weight that is set within an SRV record. This is useful when
+ you would like to control the weights using an alternate method, such as
+ using an "agent-check" or through the runtime api.
+
+ * prevent-dup-ip
+ Ensure HAProxy's default behavior is enforced on a server: prevent re-using
+ an IP address already set to a server in the same backend and sharing the
+ same fqdn.
+ This is the opposite of allow-dup-ip.
+
+ Example:
+ backend b_myapp
+ default-server init-addr none resolvers dns
+ server s1 myapp.example.com:80 check resolve-opts allow-dup-ip
+ server s2 myapp.example.com:81 check resolve-opts allow-dup-ip
+
+ With the option allow-dup-ip set:
+ * if the nameserver returns a single IP address, then both servers will use
+ it
+ * If the nameserver returns 2 IP addresses, then each server will pick up a
+ different address
+
+ Default value: not set
+
+resolve-prefer <family>
+ When DNS resolution is enabled for a server and multiple IP addresses from
+ different families are returned, HAProxy will prefer using an IP address
+ from the family mentioned in the "resolve-prefer" parameter.
+ Available families: "ipv4" and "ipv6"
+
+ Default value: ipv6
+
+ Example:
+
+ server s1 app1.domain.com:80 resolvers mydns resolve-prefer ipv6
+
+resolve-net <network>[,<network[,...]]
+ This option prioritizes the choice of an ip address matching a network. This is
+ useful with clouds to prefer a local ip. In some cases, a cloud high
+ availability service can be announced with many ip addresses on many
+ different datacenters. The latency between datacenter is not negligible, so
+ this patch permits to prefer a local datacenter. If no address matches the
+ configured network, another address is selected.
+
+ Example:
+
+ server s1 app1.domain.com:80 resolvers mydns resolve-net 10.0.0.0/8
+
+resolvers <id>
+ Points to an existing "resolvers" section to resolve current server's
+ hostname.
+
+ Example:
+
+ server s1 app1.domain.com:80 check resolvers mydns
+
+ See also section 5.3
+
+send-proxy
+ The "send-proxy" parameter enforces use of the PROXY protocol over any
+ connection established to this server. The PROXY protocol informs the other
+ end about the layer 3/4 addresses of the incoming connection, so that it can
+ know the client's address or the public address it accessed to, whatever the
+ upper layer protocol. For connections accepted by an "accept-proxy" or
+ "accept-netscaler-cip" listener, the advertised address will be used. Only
+ TCPv4 and TCPv6 address families are supported. Other families such as
+ Unix sockets, will report an UNKNOWN family. Servers using this option can
+ fully be chained to another instance of HAProxy listening with an
+ "accept-proxy" setting. This setting must not be used if the server isn't
+ aware of the protocol. When health checks are sent to the server, the PROXY
+ protocol is automatically used when this option is set, unless there is an
+ explicit "port" or "addr" directive, in which case an explicit
+ "check-send-proxy" directive would also be needed to use the PROXY protocol.
+ See also the "no-send-proxy" option of this section and "accept-proxy" and
+ "accept-netscaler-cip" option of the "bind" keyword.
+
+send-proxy-v2
+ The "send-proxy-v2" parameter enforces use of the PROXY protocol version 2
+ over any connection established to this server. The PROXY protocol informs
+ the other end about the layer 3/4 addresses of the incoming connection, so
+ that it can know the client's address or the public address it accessed to,
+ whatever the upper layer protocol. It also send ALPN information if an alpn
+ have been negotiated. This setting must not be used if the server isn't aware
+ of this version of the protocol. See also the "no-send-proxy-v2" option of
+ this section and send-proxy" option of the "bind" keyword.
+
+set-proxy-v2-tlv-fmt(<id>) <fmt>
+ The "set-proxy-v2-tlv-fmt" parameter is used to send arbitrary PROXY protocol
+ version 2 TLVs. For the type (<id>) range of the defined TLV type please refer
+ to section 2.2.8. of the proxy protocol specification. However, the value can
+ be chosen freely as long as it does not exceed the maximum length of 65,535
+ bytes. It can also be used for forwarding TLVs by using the fetch "fc_pp_tlv"
+ to retrieve a received TLV from the frontend. It may be used as a server or
+ a default-server option. It must be used in combination with send-proxy-v2
+ such that PPv2 TLVs are actually sent out.
+
+ Example:
+ server srv1 192.168.1.1:80 send-proxy-v2 set-proxy-v2-tlv-fmt(0x20) %[fc_pp_tlv(0x20)]
+
+ In this case, we fetch the TLV with the type 0x20 as a string and set as the value
+ of a newly created TLV that also has the type 0x20.
+
+proxy-v2-options <option>[,<option>]*
+ The "proxy-v2-options" parameter add options to send in PROXY protocol
+ version 2 when "send-proxy-v2" is used. Options available are:
+
+ - ssl : See also "send-proxy-v2-ssl".
+ - cert-cn : See also "send-proxy-v2-ssl-cn".
+ - ssl-cipher: Name of the used cipher.
+ - cert-sig : Signature algorithm of the used certificate.
+ - cert-key : Key algorithm of the used certificate
+ - authority : Host name value passed by the client (only SNI from a TLS
+ connection is supported).
+ - crc32c : Checksum of the PROXYv2 header.
+ - unique-id : Send a unique ID generated using the frontend's
+ "unique-id-format" within the PROXYv2 header.
+ This unique-id is primarily meant for "mode tcp". It can
+ lead to unexpected results in "mode http", because the
+ generated unique ID is also used for the first HTTP request
+ within a Keep-Alive connection.
+
+send-proxy-v2-ssl
+ The "send-proxy-v2-ssl" parameter enforces use of the PROXY protocol version
+ 2 over any connection established to this server. The PROXY protocol informs
+ the other end about the layer 3/4 addresses of the incoming connection, so
+ that it can know the client's address or the public address it accessed to,
+ whatever the upper layer protocol. In addition, the SSL information extension
+ of the PROXY protocol is added to the PROXY protocol header. This setting
+ must not be used if the server isn't aware of this version of the protocol.
+ See also the "no-send-proxy-v2-ssl" option of this section and the
+ "send-proxy-v2" option of the "bind" keyword.
+
+send-proxy-v2-ssl-cn
+ The "send-proxy-v2-ssl" parameter enforces use of the PROXY protocol version
+ 2 over any connection established to this server. The PROXY protocol informs
+ the other end about the layer 3/4 addresses of the incoming connection, so
+ that it can know the client's address or the public address it accessed to,
+ whatever the upper layer protocol. In addition, the SSL information extension
+ of the PROXY protocol, along along with the Common Name from the subject of
+ the client certificate (if any), is added to the PROXY protocol header. This
+ setting must not be used if the server isn't aware of this version of the
+ protocol. See also the "no-send-proxy-v2-ssl-cn" option of this section and
+ the "send-proxy-v2" option of the "bind" keyword.
+
+shard <shard>
+ This parameter in used only in the context of stick-tables synchronisation
+ with peers protocol. The "shard" parameter identifies the peers which will
+ receive all the stick-table updates for keys with this shard as distribution
+ hash. The accepted values are 0 up to "shards" parameter value specified in
+ the "peers" section. 0 value is the default value meaning that the peer will
+ receive all the key updates. Greater values than "shards" will be ignored.
+ This is also the case for any value provided to the local peer.
+
+ Example :
+
+ peers mypeers
+ shards 3
+ peer A 127.0.0.1:40001 # local peer without shard value (0 internally)
+ peer B 127.0.0.1:40002 shard 1
+ peer C 127.0.0.1:40003 shard 2
+ peer D 127.0.0.1:40004 shard 3
+
+sigalgs <sigalgs>
+ This setting is only available when support for OpenSSL was built in. It sets
+ the string describing the list of signature algorithms that are negotiated
+ during the TLSv1.2 and TLSv1.3 handshake. The format of the string is defined
+ in "man 3 SSL_CTX_set1_sigalgs" from the OpenSSL man pages. It is not
+ recommended to use this setting unless compatibility with a middlebox is
+ required.
+
+slowstart <start_time_in_ms>
+ The "slowstart" parameter for a server accepts a value in milliseconds which
+ indicates after how long a server which has just come back up will run at
+ full speed. Just as with every other time-based parameter, it can be entered
+ in any other explicit unit among { us, ms, s, m, h, d }. The speed grows
+ linearly from 0 to 100% during this time. The limitation applies to two
+ parameters :
+
+ - maxconn: the number of connections accepted by the server will grow from 1
+ to 100% of the usual dynamic limit defined by (minconn,maxconn,fullconn).
+
+ - weight: when the backend uses a dynamic weighted algorithm, the weight
+ grows linearly from 1 to 100%. In this case, the weight is updated at every
+ health-check. For this reason, it is important that the "inter" parameter
+ is smaller than the "slowstart", in order to maximize the number of steps.
+
+ The slowstart never applies when HAProxy starts, otherwise it would cause
+ trouble to running servers. It only applies when a server has been previously
+ seen as failed.
+
+sni <expression>
+ The "sni" parameter evaluates the sample fetch expression, converts it to a
+ string and uses the result as the host name sent in the SNI TLS extension to
+ the server. A typical use case is to send the SNI received from the client in
+ a bridged TCP/SSL scenario, using the "ssl_fc_sni" sample fetch for the
+ expression. THIS MUST NOT BE USED FOR HTTPS, where req.hdr(host) should be
+ used instead, since SNI in HTTPS must always match the Host field and clients
+ are allowed to use different host names over the same connection). If
+ "verify required" is set (which is the recommended setting), the resulting
+ name will also be matched against the server certificate's names. See the
+ "verify" directive for more details. If you want to set a SNI for health
+ checks, see the "check-sni" directive for more details.
+
+source <addr>[:<pl>[-<ph>]] [usesrc { <addr2>[:<port2>] | client | clientip } ]
+source <addr>[:<port>] [usesrc { <addr2>[:<port2>] | hdr_ip(<hdr>[,<occ>]) } ]
+source <addr>[:<pl>[-<ph>]] [interface <name>] ...
+ The "source" parameter sets the source address which will be used when
+ connecting to the server. It follows the exact same parameters and principle
+ as the backend "source" keyword, except that it only applies to the server
+ referencing it. Please consult the "source" keyword for details.
+
+ Additionally, the "source" statement on a server line allows one to specify a
+ source port range by indicating the lower and higher bounds delimited by a
+ dash ('-'). Some operating systems might require a valid IP address when a
+ source port range is specified. It is permitted to have the same IP/range for
+ several servers. Doing so makes it possible to bypass the maximum of 64k
+ total concurrent connections. The limit will then reach 64k connections per
+ server.
+
+ Since Linux 4.2/libc 2.23 IP_BIND_ADDRESS_NO_PORT is set for connections
+ specifying the source address without port(s).
+
+ssl
+ This option enables SSL ciphering on outgoing connections to the server. It
+ is critical to verify server certificates using "verify" when using SSL to
+ connect to servers, otherwise the communication is prone to trivial man in
+ the-middle attacks rendering SSL useless. When this option is used, health
+ checks are automatically sent in SSL too unless there is a "port" or an
+ "addr" directive indicating the check should be sent to a different location.
+ See the "no-ssl" to disable "ssl" option and "check-ssl" option to force
+ SSL health checks.
+
+ssl-max-ver [ SSLv3 | TLSv1.0 | TLSv1.1 | TLSv1.2 | TLSv1.3 ]
+ This option enforces use of <version> or lower when SSL is used to communicate
+ with the server. This option is also available on global statement
+ "ssl-default-server-options". See also "ssl-min-ver".
+
+ssl-min-ver [ SSLv3 | TLSv1.0 | TLSv1.1 | TLSv1.2 | TLSv1.3 ]
+ This option enforces use of <version> or upper when SSL is used to communicate
+ with the server. This option is also available on global statement
+ "ssl-default-server-options". See also "ssl-max-ver".
+
+ssl-reuse
+ This option may be used as "server" setting to reset any "no-ssl-reuse"
+ setting which would have been inherited from "default-server" directive as
+ default value.
+ It may also be used as "default-server" setting to reset any previous
+ "default-server" "no-ssl-reuse" setting.
+
+stick
+ This option may be used as "server" setting to reset any "non-stick"
+ setting which would have been inherited from "default-server" directive as
+ default value.
+ It may also be used as "default-server" setting to reset any previous
+ "default-server" "non-stick" setting.
+
+socks4 <addr>:<port>
+ This option enables upstream socks4 tunnel for outgoing connections to the
+ server. Using this option won't force the health check to go via socks4 by
+ default. You will have to use the keyword "check-via-socks4" to enable it.
+
+tcp-ut <delay>
+ Sets the TCP User Timeout for all outgoing connections to this server. This
+ option is available on Linux since version 2.6.37. It allows HAProxy to
+ configure a timeout for sockets which contain data not receiving an
+ acknowledgment for the configured delay. This is especially useful on
+ long-lived connections experiencing long idle periods such as remote
+ terminals or database connection pools, where the client and server timeouts
+ must remain high to allow a long period of idle, but where it is important to
+ detect that the server has disappeared in order to release all resources
+ associated with its connection (and the client's session). One typical use
+ case is also to force dead server connections to die when health checks are
+ too slow or during a soft reload since health checks are then disabled. The
+ argument is a delay expressed in milliseconds by default. This only works for
+ regular TCP connections, and is ignored for other protocols.
+
+tfo
+ This option enables using TCP fast open when connecting to servers, on
+ systems that support it (currently only the Linux kernel >= 4.11).
+ See the "tfo" bind option for more information about TCP fast open.
+ Please note that when using tfo, you should also use the "conn-failure",
+ "empty-response" and "response-timeout" keywords for "retry-on", or HAProxy
+ won't be able to retry the connection on failure. See also "no-tfo".
+
+track [<backend>/]<server>
+ This option enables ability to set the current state of the server by tracking
+ another one. It is possible to track a server which itself tracks another
+ server, provided that at the end of the chain, a server has health checks
+ enabled. If <backend> is omitted the current one is used. If disable-on-404 is
+ used, it has to be enabled on both proxies.
+
+tls-tickets
+ This option may be used as "server" setting to reset any "no-tls-tickets"
+ setting which would have been inherited from "default-server" directive as
+ default value.
+ The TLS ticket mechanism is only used up to TLS 1.2.
+ Forward Secrecy is compromised with TLS tickets, unless ticket keys
+ are periodically rotated (via reload or by using "tls-ticket-keys").
+ It may also be used as "default-server" setting to reset any previous
+ "default-server" "no-tls-tickets" setting.
+
+verify [none|required]
+ This setting is only available when support for OpenSSL was built in. If set
+ to 'none', server certificate is not verified. In the other case, The
+ certificate provided by the server is verified using CAs from 'ca-file' and
+ optional CRLs from 'crl-file' after having checked that the names provided in
+ the certificate's subject and subjectAlternateNames attributes match either
+ the name passed using the "sni" directive, or if not provided, the static
+ host name passed using the "verifyhost" directive. When no name is found, the
+ certificate's names are ignored. For this reason, without SNI it's important
+ to use "verifyhost". On verification failure the handshake is aborted. It is
+ critically important to verify server certificates when using SSL to connect
+ to servers, otherwise the communication is prone to trivial man-in-the-middle
+ attacks rendering SSL totally useless. Unless "ssl_server_verify" appears in
+ the global section, "verify" is set to "required" by default.
+
+verifyhost <hostname>
+ This setting is only available when support for OpenSSL was built in, and
+ only takes effect if 'verify required' is also specified. This directive sets
+ a default static hostname to check the server's certificate against when no
+ SNI was used to connect to the server. If SNI is not used, this is the only
+ way to enable hostname verification. This static hostname, when set, will
+ also be used for health checks (which cannot provide an SNI value). If none
+ of the hostnames in the certificate match the specified hostname, the
+ handshake is aborted. The hostnames in the server-provided certificate may
+ include wildcards. See also "verify", "sni" and "no-verifyhost" options.
+
+weight <weight>
+ The "weight" parameter is used to adjust the server's weight relative to
+ other servers. All servers will receive a load proportional to their weight
+ relative to the sum of all weights, so the higher the weight, the higher the
+ load. The default weight is 1, and the maximal value is 256. A value of 0
+ means the server will not participate in load-balancing but will still accept
+ persistent connections. If this parameter is used to distribute the load
+ according to server's capacity, it is recommended to start with values which
+ can both grow and shrink, for instance between 10 and 100 to leave enough
+ room above and below for later adjustments.
+
+ws { auto | h1 | h2 }
+ This option allows to configure the protocol used when relaying websocket
+ streams. This is most notably useful when using an HTTP/2 backend without the
+ support for H2 websockets through the RFC8441.
+
+ The default mode is "auto". This will reuse the same protocol as the main
+ one. The only difference is when using ALPN. In this case, it can try to
+ downgrade the ALPN to "http/1.1" only for websocket streams if the configured
+ server ALPN contains it.
+
+ The value "h1" is used to force HTTP/1.1 for websockets streams, through ALPN
+ if SSL ALPN is activated for the server. Similarly, "h2" can be used to
+ force HTTP/2.0 websockets. Use this value with care : the server must support
+ RFC8441 or an error will be reported by haproxy when relaying websockets.
+
+ Note that NPN is not taken into account as its usage has been deprecated in
+ favor of the ALPN extension.
+
+ See also "alpn" and "proto".
+
+
+5.3. Server IP address resolution using DNS
+-------------------------------------------
+
+HAProxy allows using a host name on the server line to retrieve its IP address
+using name servers. By default, HAProxy resolves the name when parsing the
+configuration file, at startup and cache the result for the process's life.
+This is not sufficient in some cases, such as in Amazon where a server's IP
+can change after a reboot or an ELB Virtual IP can change based on current
+workload.
+This chapter describes how HAProxy can be configured to process server's name
+resolution at run time.
+Whether run time server name resolution has been enable or not, HAProxy will
+carry on doing the first resolution when parsing the configuration.
+
+
+5.3.1. Global overview
+----------------------
+
+As we've seen in introduction, name resolution in HAProxy occurs at two
+different steps of the process life:
+
+ 1. when starting up, HAProxy parses the server line definition and matches a
+ host name. It uses libc functions to get the host name resolved. This
+ resolution relies on /etc/resolv.conf file.
+
+ 2. at run time, HAProxy performs periodically name resolutions for servers
+ requiring DNS resolutions.
+
+A few other events can trigger a name resolution at run time:
+ - when a server's health check ends up in a connection timeout: this may be
+ because the server has a new IP address. So we need to trigger a name
+ resolution to know this new IP.
+
+When using resolvers, the server name can either be a hostname, or a SRV label.
+HAProxy considers anything that starts with an underscore as a SRV label. If a
+SRV label is specified, then the corresponding SRV records will be retrieved
+from the DNS server, and the provided hostnames will be used. The SRV label
+will be checked periodically, and if any server are added or removed, HAProxy
+will automatically do the same.
+
+A few things important to notice:
+ - all the name servers are queried in the meantime. HAProxy will process the
+ first valid response.
+
+ - a resolution is considered as invalid (NX, timeout, refused), when all the
+ servers return an error.
+
+
+5.3.2. The resolvers section
+----------------------------
+
+This section is dedicated to host information related to name resolution in
+HAProxy. There can be as many as resolvers section as needed. Each section can
+contain many name servers.
+
+At startup, HAProxy tries to generate a resolvers section named "default", if
+no section was named this way in the configuration. This section is used by
+default by the httpclient and uses the parse-resolv-conf keyword. If HAProxy
+failed to generate automatically this section, no error or warning are emitted.
+
+When multiple name servers are configured in a resolvers section, then HAProxy
+uses the first valid response. In case of invalid responses, only the last one
+is treated. Purpose is to give the chance to a slow server to deliver a valid
+answer after a fast faulty or outdated server.
+
+When each server returns a different error type, then only the last error is
+used by HAProxy. The following processing is applied on this error:
+
+ 1. HAProxy retries the same DNS query with a new query type. The A queries are
+ switch to AAAA or the opposite. SRV queries are not concerned here. Timeout
+ errors are also excluded.
+
+ 2. When the fallback on the query type was done (or not applicable), HAProxy
+ retries the original DNS query, with the preferred query type.
+
+ 3. HAProxy retries previous steps <resolve_retries> times. If no valid
+ response is received after that, it stops the DNS resolution and reports
+ the error.
+
+For example, with 2 name servers configured in a resolvers section, the
+following scenarios are possible:
+
+ - First response is valid and is applied directly, second response is
+ ignored
+
+ - First response is invalid and second one is valid, then second response is
+ applied
+
+ - First response is a NX domain and second one a truncated response, then
+ HAProxy retries the query with a new type
+
+ - First response is a NX domain and second one is a timeout, then HAProxy
+ retries the query with a new type
+
+ - Query timed out for both name servers, then HAProxy retries it with the
+ same query type
+
+As a DNS server may not answer all the IPs in one DNS request, HAProxy keeps
+a cache of previous answers, an answer will be considered obsolete after
+<hold obsolete> seconds without the IP returned.
+
+
+resolvers <resolvers id>
+ Creates a new name server list labeled <resolvers id>
+
+A resolvers section accept the following parameters:
+
+accepted_payload_size <nb>
+ Defines the maximum payload size accepted by HAProxy and announced to all the
+ name servers configured in this resolvers section.
+ <nb> is in bytes. If not set, HAProxy announces 512. (minimal value defined
+ by RFC 6891)
+
+ Note: the maximum allowed value is 65535. Recommended value for UDP is
+ 4096 and it is not recommended to exceed 8192 except if you are sure
+ that your system and network can handle this (over 65507 makes no sense
+ since is the maximum UDP payload size). If you are using only TCP
+ nameservers to handle huge DNS responses, you should put this value
+ to the max: 65535.
+
+nameserver <name> <address>[:port] [param*]
+ Used to configure a nameserver. <name> of the nameserver should ne unique.
+ By default the <address> is considered of type datagram. This means if an
+ IPv4 or IPv6 is configured without special address prefixes (paragraph 11.)
+ the UDP protocol will be used. If an stream protocol address prefix is used,
+ the nameserver will be considered as a stream server (TCP for instance) and
+ "server" parameters found in 5.2 paragraph which are relevant for DNS
+ resolving will be considered. Note: currently, in TCP mode, 4 queries are
+ pipelined on the same connections. A batch of idle connections are removed
+ every 5 seconds. "maxconn" can be configured to limit the amount of those
+ concurrent connections and TLS should also usable if the server supports.
+
+parse-resolv-conf
+ Adds all nameservers found in /etc/resolv.conf to this resolvers nameservers
+ list. Ordered as if each nameserver in /etc/resolv.conf was individually
+ placed in the resolvers section in place of this directive.
+
+hold <status> <period>
+ Upon receiving the DNS response <status>, determines whether a server's state
+ should change from UP to DOWN. To make that determination, it checks whether
+ any valid status has been received during the past <period> in order to
+ counteract the just received invalid status.
+
+ <status> : last name resolution status.
+ nx After receiving an NXDOMAIN status, check for any valid
+ status during the concluding period.
+
+ refused After receiving a REFUSED status, check for any valid
+ status during the concluding period.
+
+ timeout After the "timeout retry" has struck, check for any
+ valid status during the concluding period.
+
+ other After receiving any other invalid status, check for any
+ valid status during the concluding period.
+
+ valid Applies only to "http-request do-resolve" and
+ "tcp-request content do-resolve" actions. It defines the
+ period for which the server will maintain a valid response
+ before triggering another resolution. It does not affect
+ dynamic resolution of servers.
+
+ obsolete Defines how long to wait before removing obsolete DNS
+ records after an updated answer record is received. It
+ applies to SRV records.
+
+ <period> : Amount of time into the past during which a valid response must
+ have been received. It follows the HAProxy time format and is in
+ milliseconds by default.
+
+ For a server that relies on dynamic DNS resolution to determine its IP
+ address, receiving an invalid DNS response, such as NXDOMAIN, will lead to
+ changing the server's state from UP to DOWN. The hold directives define how
+ far into the past to look for a valid response. If a valid response has been
+ received within <period>, the just received invalid status will be ignored.
+
+ Unless a valid response has been receiving during the concluding period, the
+ server will be marked as DOWN. For example, if "hold nx 30s" is set and the
+ last received DNS response was NXDOMAIN, the server will be marked DOWN
+ unless a valid response has been received during the last 30 seconds.
+
+ A server in the DOWN state will be marked UP immediately upon receiving a
+ valid status from the DNS server.
+
+ A separate behavior exists for "hold valid" and "hold obsolete".
+
+resolve_retries <nb>
+ Defines the number <nb> of queries to send to resolve a server name before
+ giving up.
+ Default value: 3
+
+ A retry occurs on name server timeout or when the full sequence of DNS query
+ type failover is over and we need to start up from the default ANY query
+ type.
+
+timeout <event> <time>
+ Defines timeouts related to name resolution
+ <event> : the event on which the <time> timeout period applies to.
+ events available are:
+ - resolve : default time to trigger name resolutions when no
+ other time applied.
+ Default value: 1s
+ - retry : time between two DNS queries, when no valid response
+ have been received.
+ Default value: 1s
+ <time> : time related to the event. It follows the HAProxy time format.
+ <time> is expressed in milliseconds.
+
+ Example:
+
+ resolvers mydns
+ nameserver dns1 10.0.0.1:53
+ nameserver dns2 10.0.0.2:53
+ nameserver dns3 tcp@10.0.0.3:53
+ parse-resolv-conf
+ resolve_retries 3
+ timeout resolve 1s
+ timeout retry 1s
+ hold other 30s
+ hold refused 30s
+ hold nx 30s
+ hold timeout 30s
+ hold valid 10s
+ hold obsolete 30s
+
+
+6. Cache
+---------
+
+HAProxy provides a cache, which was designed to perform cache on small objects
+(favicon, css...). This is a minimalist low-maintenance cache which runs in
+RAM.
+
+The cache is based on a memory area shared between all threads, and split in 1kB
+blocks.
+
+If an object is not used anymore, it can be deleted to store a new object
+independently of its expiration date. The oldest objects are deleted first
+when we try to allocate a new one.
+
+The cache uses a hash of the host header and the URI as the key.
+
+It's possible to view the status of a cache using the Unix socket command
+"show cache" consult section 9.3 "Unix Socket commands" of Management Guide
+for more details.
+
+When an object is delivered from the cache, the server name in the log is
+replaced by "<CACHE>".
+
+
+6.1. Limitation
+----------------
+
+The cache won't store and won't deliver objects in these cases:
+
+- If the response is not a 200
+- If the response contains a Vary header and either the process-vary option is
+ disabled, or a currently unmanaged header is specified in the Vary value (only
+ accept-encoding, referer and origin are managed for now)
+- If the Content-Length + the headers size is greater than "max-object-size"
+- If the response is not cacheable
+- If the response does not have an explicit expiration time (s-maxage or max-age
+ Cache-Control directives or Expires header) or a validator (ETag or Last-Modified
+ headers)
+- If the process-vary option is enabled and there are already max-secondary-entries
+ entries with the same primary key as the current response
+- If the process-vary option is enabled and the response has an unknown encoding (not
+ mentioned in https://www.iana.org/assignments/http-parameters/http-parameters.xhtml)
+ while varying on the accept-encoding client header
+
+- If the request is not a GET
+- If the HTTP version of the request is smaller than 1.1
+- If the request contains an Authorization header
+
+
+6.2. Setup
+-----------
+
+To setup a cache, you must define a cache section and use it in a proxy with
+the corresponding http-request and http-response actions.
+
+
+6.2.1. Cache section
+---------------------
+
+cache <name>
+ Declare a cache section, allocate a shared cache memory named <name>, the
+ size of cache is mandatory.
+
+total-max-size <megabytes>
+ Define the size in RAM of the cache in megabytes. This size is split in
+ blocks of 1kB which are used by the cache entries. Its maximum value is 4095.
+
+max-object-size <bytes>
+ Define the maximum size of the objects to be cached. Must not be greater than
+ an half of "total-max-size". If not set, it equals to a 256th of the cache size.
+ All objects with sizes larger than "max-object-size" will not be cached.
+
+max-age <seconds>
+ Define the maximum expiration duration. The expiration is set as the lowest
+ value between the s-maxage or max-age (in this order) directive in the
+ Cache-Control response header and this value. The default value is 60
+ seconds, which means that you can't cache an object more than 60 seconds by
+ default.
+
+process-vary <on/off>
+ Enable or disable the processing of the Vary header. When disabled, a response
+ containing such a header will never be cached. When enabled, we need to calculate
+ a preliminary hash for a subset of request headers on all the incoming requests
+ (which might come with a cpu cost) which will be used to build a secondary
+ key for a given request (see RFC 7234#4.1). The secondary key is built out of
+ the contents of the 'accept-encoding', 'referer' and 'origin' headers for
+ now. The default value is off (disabled).
+
+max-secondary-entries <number>
+ Define the maximum number of simultaneous secondary entries with the same primary
+ key in the cache. This needs the vary support to be enabled. Its default value is 10
+ and should be passed a strictly positive integer.
+
+
+6.2.2. Proxy section
+---------------------
+
+The proxy section making use of the cache will need to involve the "cache-use"
+action in the "http-request" rule set in order to look up the requested object
+from the cache, and the "cache-store" action in the "http-response" rule set in
+order to store or update the retrieved object into the cache. Each of these
+actions may optionally involve conditions. For example, one could decide to
+skip the "cache-use" action for a certain sub-directory that is known not to
+be cacheable, or to skip the "cache-store" action for certain content-types
+that are known to be worthless. Please note that the cache indexing key is
+computed during the "cache-use" action, so if this action is skipped, no
+attempt to update the cache will be made on the response path anyway.
+
+Example:
+
+ backend bck1
+ mode http
+
+ http-request cache-use foobar
+ http-response cache-store foobar
+ server srv1 127.0.0.1:80
+
+ cache foobar
+ total-max-size 4
+ max-age 240
+
+
+7. Using ACLs and fetching samples
+----------------------------------
+
+HAProxy is capable of extracting data from request or response streams, from
+client or server information, from tables, environmental information etc...
+The action of extracting such data is called fetching a sample. Once retrieved,
+these samples may be used for various purposes such as a key to a stick-table,
+but most common usages consist in matching them against predefined constant
+data called patterns.
+
+
+7.1. ACL basics
+---------------
+
+The use of Access Control Lists (ACL) provides a flexible solution to perform
+content switching and generally to take decisions based on content extracted
+from the request, the response or any environmental status. The principle is
+simple :
+
+ - extract a data sample from a stream, table or the environment
+ - optionally apply some format conversion to the extracted sample
+ - apply one or multiple pattern matching methods on this sample
+ - perform actions only when a pattern matches the sample
+
+The actions generally consist in blocking a request, selecting a backend, or
+adding a header.
+
+In order to define a test, the "acl" keyword is used. The syntax is :
+
+ acl <aclname> <criterion> [flags] [operator] [<value>] ...
+
+This creates a new ACL <aclname> or completes an existing one with new tests.
+Those tests apply to the portion of request/response specified in <criterion>
+and may be adjusted with optional flags [flags]. Some criteria also support
+an operator which may be specified before the set of values. Optionally some
+conversion operators may be applied to the sample, and they will be specified
+as a comma-delimited list of keywords just after the first keyword. The values
+are of the type supported by the criterion, and are separated by spaces.
+
+ACL names must be formed from upper and lower case letters, digits, '-' (dash),
+'_' (underscore) , '.' (dot) and ':' (colon). ACL names are case-sensitive,
+which means that "my_acl" and "My_Acl" are two different ACLs.
+
+There is no enforced limit to the number of ACLs. The unused ones do not affect
+performance, they just consume a small amount of memory.
+
+The criterion generally is the name of a sample fetch method, or one of its ACL
+specific declinations. The default test method is implied by the output type of
+this sample fetch method. The ACL declinations can describe alternate matching
+methods of a same sample fetch method. The sample fetch methods are the only
+ones supporting a conversion.
+
+Sample fetch methods return data which can be of the following types :
+ - boolean
+ - integer (signed or unsigned)
+ - IPv4 or IPv6 address
+ - string
+ - data block
+
+Converters transform any of these data into any of these. For example, some
+converters might convert a string to a lower-case string while other ones
+would turn a string to an IPv4 address, or apply a netmask to an IP address.
+The resulting sample is of the type of the last converter applied to the list,
+which defaults to the type of the sample fetch method.
+
+Each sample or converter returns data of a specific type, specified with its
+keyword in this documentation. When an ACL is declared using a standard sample
+fetch method, certain types automatically involved a default matching method
+which are summarized in the table below :
+
+ +---------------------+-----------------+
+ | Sample or converter | Default |
+ | output type | matching method |
+ +---------------------+-----------------+
+ | boolean | bool |
+ +---------------------+-----------------+
+ | integer | int |
+ +---------------------+-----------------+
+ | ip | ip |
+ +---------------------+-----------------+
+ | string | str |
+ +---------------------+-----------------+
+ | binary | none, use "-m" |
+ +---------------------+-----------------+
+
+Note that in order to match a binary samples, it is mandatory to specify a
+matching method, see below.
+
+The ACL engine can match these types against patterns of the following types :
+ - boolean
+ - integer or integer range
+ - IP address / network
+ - string (exact, substring, suffix, prefix, subdir, domain)
+ - regular expression
+ - hex block
+
+The following ACL flags are currently supported :
+
+ -i : ignore case during matching of all subsequent patterns.
+ -f : load patterns from a file.
+ -m : use a specific pattern matching method
+ -n : forbid the DNS resolutions
+ -M : load the file pointed by -f like a map file.
+ -u : force the unique id of the ACL
+ -- : force end of flags. Useful when a string looks like one of the flags.
+
+The "-f" flag is followed by the name of a file from which all lines will be
+read as individual values. It is even possible to pass multiple "-f" arguments
+if the patterns are to be loaded from multiple files. Empty lines as well as
+lines beginning with a sharp ('#') will be ignored. All leading spaces and tabs
+will be stripped. If it is absolutely necessary to insert a valid pattern
+beginning with a sharp, just prefix it with a space so that it is not taken for
+a comment. Depending on the data type and match method, HAProxy may load the
+lines into a binary tree, allowing very fast lookups. This is true for IPv4 and
+exact string matching. In this case, duplicates will automatically be removed.
+
+The "-M" flag allows an ACL to use a map file. If this flag is set, the file is
+parsed as two column file. The first column contains the patterns used by the
+ACL, and the second column contain the samples. The sample can be used later by
+a map. This can be useful in some rare cases where an ACL would just be used to
+check for the existence of a pattern in a map before a mapping is applied.
+
+The "-u" flag forces the unique id of the ACL. This unique id is used with the
+socket interface to identify ACL and dynamically change its values. Note that a
+file is always identified by its name even if an id is set.
+
+Also, note that the "-i" flag applies to subsequent entries and not to entries
+loaded from files preceding it. For instance :
+
+ acl valid-ua hdr(user-agent) -f exact-ua.lst -i -f generic-ua.lst test
+
+In this example, each line of "exact-ua.lst" will be exactly matched against
+the "user-agent" header of the request. Then each line of "generic-ua" will be
+case-insensitively matched. Then the word "test" will be insensitively matched
+as well.
+
+The "-m" flag is used to select a specific pattern matching method on the input
+sample. All ACL-specific criteria imply a pattern matching method and generally
+do not need this flag. However, this flag is useful with generic sample fetch
+methods to describe how they're going to be matched against the patterns. This
+is required for sample fetches which return data type for which there is no
+obvious matching method (e.g. string or binary). When "-m" is specified and
+followed by a pattern matching method name, this method is used instead of the
+default one for the criterion. This makes it possible to match contents in ways
+that were not initially planned, or with sample fetch methods which return a
+string. The matching method also affects the way the patterns are parsed.
+
+The "-n" flag forbids the dns resolutions. It is used with the load of ip files.
+By default, if the parser cannot parse ip address it considers that the parsed
+string is maybe a domain name and try dns resolution. The flag "-n" disable this
+resolution. It is useful for detecting malformed ip lists. Note that if the DNS
+server is not reachable, the HAProxy configuration parsing may last many minutes
+waiting for the timeout. During this time no error messages are displayed. The
+flag "-n" disable this behavior. Note also that during the runtime, this
+function is disabled for the dynamic acl modifications.
+
+There are some restrictions however. Not all methods can be used with all
+sample fetch methods. Also, if "-m" is used in conjunction with "-f", it must
+be placed first. The pattern matching method must be one of the following :
+
+ - "found" : only check if the requested sample could be found in the stream,
+ but do not compare it against any pattern. It is recommended not
+ to pass any pattern to avoid confusion. This matching method is
+ particularly useful to detect presence of certain contents such
+ as headers, cookies, etc... even if they are empty and without
+ comparing them to anything nor counting them.
+
+ - "bool" : check the value as a boolean. It can only be applied to fetches
+ which return a boolean or integer value, and takes no pattern.
+ Value zero or false does not match, all other values do match.
+
+ - "int" : match the value as an integer. It can be used with integer and
+ boolean samples. Boolean false is integer 0, true is integer 1.
+
+ - "ip" : match the value as an IPv4 or IPv6 address. It is compatible
+ with IP address samples only, so it is implied and never needed.
+
+ - "bin" : match the contents against a hexadecimal string representing a
+ binary sequence. This may be used with binary or string samples.
+
+ - "len" : match the sample's length as an integer. This may be used with
+ binary or string samples.
+
+ - "str" : exact match : match the contents against a string. This may be
+ used with binary or string samples.
+
+ - "sub" : substring match : check that the contents contain at least one of
+ the provided string patterns. This may be used with binary or
+ string samples.
+
+ - "reg" : regex match : match the contents against a list of regular
+ expressions. This may be used with binary or string samples.
+
+ - "beg" : prefix match : check that the contents begin like the provided
+ string patterns. This may be used with binary or string samples.
+
+ - "end" : suffix match : check that the contents end like the provided
+ string patterns. This may be used with binary or string samples.
+
+ - "dir" : subdir match : check that a slash-delimited portion of the
+ contents exactly matches one of the provided string patterns.
+ This may be used with binary or string samples.
+
+ - "dom" : domain match : check that a dot-delimited portion of the contents
+ exactly match one of the provided string patterns. This may be
+ used with binary or string samples.
+
+For example, to quickly detect the presence of cookie "JSESSIONID" in an HTTP
+request, it is possible to do :
+
+ acl jsess_present req.cook(JSESSIONID) -m found
+
+In order to apply a regular expression on the 500 first bytes of data in the
+buffer, one would use the following acl :
+
+ acl script_tag req.payload(0,500) -m reg -i <script>
+
+On systems where the regex library is much slower when using "-i", it is
+possible to convert the sample to lowercase before matching, like this :
+
+ acl script_tag req.payload(0,500),lower -m reg <script>
+
+All ACL-specific criteria imply a default matching method. Most often, these
+criteria are composed by concatenating the name of the original sample fetch
+method and the matching method. For example, "hdr_beg" applies the "beg" match
+to samples retrieved using the "hdr" fetch method. This matching method is only
+usable when the keyword is used alone, without any converter. In case any such
+converter were to be applied after such an ACL keyword, the default matching
+method from the ACL keyword is simply ignored since what will matter for the
+matching is the output type of the last converter. Since all ACL-specific
+criteria rely on a sample fetch method, it is always possible instead to use
+the original sample fetch method and the explicit matching method using "-m".
+
+If an alternate match is specified using "-m" on an ACL-specific criterion,
+the matching method is simply applied to the underlying sample fetch method.
+For example, all ACLs below are exact equivalent :
+
+ acl short_form hdr_beg(host) www.
+ acl alternate1 hdr_beg(host) -m beg www.
+ acl alternate2 hdr_dom(host) -m beg www.
+ acl alternate3 hdr(host) -m beg www.
+
+
+The table below summarizes the compatibility matrix between sample or converter
+types and the pattern types to fetch against. It indicates for each compatible
+combination the name of the matching method to be used, surrounded with angle
+brackets ">" and "<" when the method is the default one and will work by
+default without "-m".
+
+ +-------------------------------------------------+
+ | Input sample type |
+ +----------------------+---------+---------+---------+---------+---------+
+ | pattern type | boolean | integer | ip | string | binary |
+ +----------------------+---------+---------+---------+---------+---------+
+ | none (presence only) | found | found | found | found | found |
+ +----------------------+---------+---------+---------+---------+---------+
+ | none (boolean value) |> bool <| bool | | bool | |
+ +----------------------+---------+---------+---------+---------+---------+
+ | integer (value) | int |> int <| int | int | |
+ +----------------------+---------+---------+---------+---------+---------+
+ | integer (length) | len | len | len | len | len |
+ +----------------------+---------+---------+---------+---------+---------+
+ | IP address | | |> ip <| ip | ip |
+ +----------------------+---------+---------+---------+---------+---------+
+ | exact string | str | str | str |> str <| str |
+ +----------------------+---------+---------+---------+---------+---------+
+ | prefix | beg | beg | beg | beg | beg |
+ +----------------------+---------+---------+---------+---------+---------+
+ | suffix | end | end | end | end | end |
+ +----------------------+---------+---------+---------+---------+---------+
+ | substring | sub | sub | sub | sub | sub |
+ +----------------------+---------+---------+---------+---------+---------+
+ | subdir | dir | dir | dir | dir | dir |
+ +----------------------+---------+---------+---------+---------+---------+
+ | domain | dom | dom | dom | dom | dom |
+ +----------------------+---------+---------+---------+---------+---------+
+ | regex | reg | reg | reg | reg | reg |
+ +----------------------+---------+---------+---------+---------+---------+
+ | hex block | | | | bin | bin |
+ +----------------------+---------+---------+---------+---------+---------+
+
+
+7.1.1. Matching booleans
+------------------------
+
+In order to match a boolean, no value is needed and all values are ignored.
+Boolean matching is used by default for all fetch methods of type "boolean".
+When boolean matching is used, the fetched value is returned as-is, which means
+that a boolean "true" will always match and a boolean "false" will never match.
+
+Boolean matching may also be enforced using "-m bool" on fetch methods which
+return an integer value. Then, integer value 0 is converted to the boolean
+"false" and all other values are converted to "true".
+
+
+7.1.2. Matching integers
+------------------------
+
+Integer matching applies by default to integer fetch methods. It can also be
+enforced on boolean fetches using "-m int". In this case, "false" is converted
+to the integer 0, and "true" is converted to the integer 1.
+
+Integer matching also supports integer ranges and operators. Note that integer
+matching only applies to positive values. A range is a value expressed with a
+lower and an upper bound separated with a colon, both of which may be omitted.
+
+For instance, "1024:65535" is a valid range to represent a range of
+unprivileged ports, and "1024:" would also work. "0:1023" is a valid
+representation of privileged ports, and ":1023" would also work.
+
+As a special case, some ACL functions support decimal numbers which are in fact
+two integers separated by a dot. This is used with some version checks for
+instance. All integer properties apply to those decimal numbers, including
+ranges and operators.
+
+For an easier usage, comparison operators are also supported. Note that using
+operators with ranges does not make much sense and is strongly discouraged.
+Similarly, it does not make much sense to perform order comparisons with a set
+of values.
+
+Available operators for integer matching are :
+
+ eq : true if the tested value equals at least one value
+ ge : true if the tested value is greater than or equal to at least one value
+ gt : true if the tested value is greater than at least one value
+ le : true if the tested value is less than or equal to at least one value
+ lt : true if the tested value is less than at least one value
+
+For instance, the following ACL matches any negative Content-Length header :
+
+ acl negative-length req.hdr_val(content-length) lt 0
+
+This one matches SSL versions between 3.0 and 3.1 (inclusive) :
+
+ acl sslv3 req.ssl_ver 3:3.1
+
+
+7.1.3. Matching strings
+-----------------------
+
+String matching applies to string or binary fetch methods, and exists in 6
+different forms :
+
+ - exact match (-m str) : the extracted string must exactly match the
+ patterns;
+
+ - substring match (-m sub) : the patterns are looked up inside the
+ extracted string, and the ACL matches if any of them is found inside;
+
+ - prefix match (-m beg) : the patterns are compared with the beginning of
+ the extracted string, and the ACL matches if any of them matches.
+
+ - suffix match (-m end) : the patterns are compared with the end of the
+ extracted string, and the ACL matches if any of them matches.
+
+ - subdir match (-m dir) : the patterns are looked up anywhere inside the
+ extracted string, delimited with slashes ("/"), the beginning or the end
+ of the string. The ACL matches if any of them matches. As such, the string
+ "/images/png/logo/32x32.png", would match "/images", "/images/png",
+ "images/png", "/png/logo", "logo/32x32.png" or "32x32.png" but not "png"
+ nor "32x32".
+
+ - domain match (-m dom) : the patterns are looked up anywhere inside the
+ extracted string, delimited with dots ("."), colons (":"), slashes ("/"),
+ question marks ("?"), the beginning or the end of the string. This is made
+ to be used with URLs. Leading and trailing delimiters in the pattern are
+ ignored. The ACL matches if any of them matches. As such, in the example
+ string "http://www1.dc-eu.example.com:80/blah", the patterns "http",
+ "www1", ".www1", "dc-eu", "example", "com", "80", "dc-eu.example",
+ "blah", ":www1:", "dc-eu.example:80" would match, but not "eu" nor "dc".
+ Using it to match domain suffixes for filtering or routing is generally
+ not a good idea, as the routing could easily be fooled by prepending the
+ matching prefix in front of another domain for example.
+
+String matching applies to verbatim strings as they are passed, with the
+exception of the backslash ("\") which makes it possible to escape some
+characters such as the space. If the "-i" flag is passed before the first
+string, then the matching will be performed ignoring the case. In order
+to match the string "-i", either set it second, or pass the "--" flag
+before the first string. Same applies of course to match the string "--".
+
+Do not use string matches for binary fetches which might contain null bytes
+(0x00), as the comparison stops at the occurrence of the first null byte.
+Instead, convert the binary fetch to a hex string with the hex converter first.
+
+Example:
+ # matches if the string <tag> is present in the binary sample
+ acl tag_found req.payload(0,0),hex -m sub 3C7461673E
+
+
+7.1.4. Matching regular expressions (regexes)
+---------------------------------------------
+
+Just like with string matching, regex matching applies to verbatim strings as
+they are passed, with the exception of the backslash ("\") which makes it
+possible to escape some characters such as the space. If the "-i" flag is
+passed before the first regex, then the matching will be performed ignoring
+the case. In order to match the string "-i", either set it second, or pass
+the "--" flag before the first string. Same principle applies of course to
+match the string "--".
+
+
+7.1.5. Matching arbitrary data blocks
+-------------------------------------
+
+It is possible to match some extracted samples against a binary block which may
+not safely be represented as a string. For this, the patterns must be passed as
+a series of hexadecimal digits in an even number, when the match method is set
+to binary. Each sequence of two digits will represent a byte. The hexadecimal
+digits may be used upper or lower case.
+
+Example :
+ # match "Hello\n" in the input stream (\x48 \x65 \x6c \x6c \x6f \x0a)
+ acl hello req.payload(0,6) -m bin 48656c6c6f0a
+
+
+7.1.6. Matching IPv4 and IPv6 addresses
+---------------------------------------
+
+IPv4 addresses values can be specified either as plain addresses or with a
+netmask appended, in which case the IPv4 address matches whenever it is
+within the network. Plain addresses may also be replaced with a resolvable
+host name, but this practice is generally discouraged as it makes it more
+difficult to read and debug configurations. If hostnames are used, you should
+at least ensure that they are present in /etc/hosts so that the configuration
+does not depend on any random DNS match at the moment the configuration is
+parsed.
+
+The dotted IPv4 address notation is supported in both regular as well as the
+abbreviated form with all-0-octets omitted:
+
+ +------------------+------------------+------------------+
+ | Example 1 | Example 2 | Example 3 |
+ +------------------+------------------+------------------+
+ | 192.168.0.1 | 10.0.0.12 | 127.0.0.1 |
+ | 192.168.1 | 10.12 | 127.1 |
+ | 192.168.0.1/22 | 10.0.0.12/8 | 127.0.0.1/8 |
+ | 192.168.1/22 | 10.12/8 | 127.1/8 |
+ +------------------+------------------+------------------+
+
+Notice that this is different from RFC 4632 CIDR address notation in which
+192.168.42/24 would be equivalent to 192.168.42.0/24.
+
+IPv6 may be entered in their usual form, with or without a netmask appended.
+Only bit counts are accepted for IPv6 netmasks. In order to avoid any risk of
+trouble with randomly resolved IP addresses, host names are never allowed in
+IPv6 patterns.
+
+HAProxy is also able to match IPv4 addresses with IPv6 addresses in the
+following situations :
+ - tested address is IPv4, pattern address is IPv4, the match applies
+ in IPv4 using the supplied mask if any.
+ - tested address is IPv6, pattern address is IPv6, the match applies
+ in IPv6 using the supplied mask if any.
+ - tested address is IPv6, pattern address is IPv4, the match applies in IPv4
+ using the pattern's mask if the IPv6 address matches with 2002:IPV4::,
+ ::IPV4 or ::ffff:IPV4, otherwise it fails.
+ - tested address is IPv4, pattern address is IPv6, the IPv4 address is first
+ converted to IPv6 by prefixing ::ffff: in front of it, then the match is
+ applied in IPv6 using the supplied IPv6 mask.
+
+
+7.2. Using ACLs to form conditions
+----------------------------------
+
+Some actions are only performed upon a valid condition. A condition is a
+combination of ACLs with operators. 3 operators are supported :
+
+ - AND (implicit)
+ - OR (explicit with the "or" keyword or the "||" operator)
+ - Negation with the exclamation mark ("!")
+
+A condition is formed as a disjunctive form:
+
+ [!]acl1 [!]acl2 ... [!]acln { or [!]acl1 [!]acl2 ... [!]acln } ...
+
+Such conditions are generally used after an "if" or "unless" statement,
+indicating when the condition will trigger the action.
+
+For instance, to block HTTP requests to the "*" URL with methods other than
+"OPTIONS", as well as POST requests without content-length, and GET or HEAD
+requests with a content-length greater than 0, and finally every request which
+is not either GET/HEAD/POST/OPTIONS !
+
+ acl missing_cl req.hdr_cnt(Content-length) eq 0
+ http-request deny if HTTP_URL_STAR !METH_OPTIONS || METH_POST missing_cl
+ http-request deny if METH_GET HTTP_CONTENT
+ http-request deny unless METH_GET or METH_POST or METH_OPTIONS
+
+To select a different backend for requests to static contents on the "www" site
+and to every request on the "img", "video", "download" and "ftp" hosts :
+
+ acl url_static path_beg /static /images /img /css
+ acl url_static path_end .gif .png .jpg .css .js
+ acl host_www hdr_beg(host) -i www
+ acl host_static hdr_beg(host) -i img. video. download. ftp.
+
+ # now use backend "static" for all static-only hosts, and for static URLs
+ # of host "www". Use backend "www" for the rest.
+ use_backend static if host_static or host_www url_static
+ use_backend www if host_www
+
+It is also possible to form rules using "anonymous ACLs". Those are unnamed ACL
+expressions that are built on the fly without needing to be declared. They must
+be enclosed between braces, with a space before and after each brace (because
+the braces must be seen as independent words). Example :
+
+ The following rule :
+
+ acl missing_cl req.hdr_cnt(Content-length) eq 0
+ http-request deny if METH_POST missing_cl
+
+ Can also be written that way :
+
+ http-request deny if METH_POST { req.hdr_cnt(Content-length) eq 0 }
+
+It is generally not recommended to use this construct because it's a lot easier
+to leave errors in the configuration when written that way. However, for very
+simple rules matching only one source IP address for instance, it can make more
+sense to use them than to declare ACLs with random names. Another example of
+good use is the following :
+
+ With named ACLs :
+
+ acl site_dead nbsrv(dynamic) lt 2
+ acl site_dead nbsrv(static) lt 2
+ monitor fail if site_dead
+
+ With anonymous ACLs :
+
+ monitor fail if { nbsrv(dynamic) lt 2 } || { nbsrv(static) lt 2 }
+
+See section 4.2 for detailed help on the "http-request deny" and "use_backend"
+keywords.
+
+
+7.3. Fetching samples
+---------------------
+
+Historically, sample fetch methods were only used to retrieve data to match
+against patterns using ACLs. With the arrival of stick-tables, a new class of
+sample fetch methods was created, most often sharing the same syntax as their
+ACL counterpart. These sample fetch methods are also known as "fetches". As
+of now, ACLs and fetches have converged. All ACL fetch methods have been made
+available as fetch methods, and ACLs may use any sample fetch method as well.
+
+This section details all available sample fetch methods and their output type.
+Some sample fetch methods have deprecated aliases that are used to maintain
+compatibility with existing configurations. They are then explicitly marked as
+deprecated and should not be used in new setups.
+
+The ACL derivatives are also indicated when available, with their respective
+matching methods. These ones all have a well defined default pattern matching
+method, so it is never necessary (though allowed) to pass the "-m" option to
+indicate how the sample will be matched using ACLs.
+
+As indicated in the sample type versus matching compatibility matrix above,
+when using a generic sample fetch method in an ACL, the "-m" option is
+mandatory unless the sample type is one of boolean, integer, IPv4 or IPv6. When
+the same keyword exists as an ACL keyword and as a standard fetch method, the
+ACL engine will automatically pick the ACL-only one by default.
+
+Some of these keywords support one or multiple mandatory arguments, and one or
+multiple optional arguments. These arguments are strongly typed and are checked
+when the configuration is parsed so that there is no risk of running with an
+incorrect argument (e.g. an unresolved backend name). Fetch function arguments
+are passed between parenthesis and are delimited by commas. When an argument
+is optional, it will be indicated below between square brackets ('[ ]'). When
+all arguments are optional, the parenthesis may be omitted.
+
+Thus, the syntax of a standard sample fetch method is one of the following :
+ - name
+ - name(arg1)
+ - name(arg1,arg2)
+
+
+7.3.1. Converters
+-----------------
+
+Sample fetch methods may be combined with transformations to be applied on top
+of the fetched sample (also called "converters"). These combinations form what
+is called "sample expressions" and the result is a "sample". Initially this
+was only supported by "stick on" and "stick store-request" directives but this
+has now be extended to all places where samples may be used (ACLs, log-format,
+unique-id-format, add-header, ...).
+
+These transformations are enumerated as a series of specific keywords after the
+sample fetch method. These keywords may equally be appended immediately after
+the fetch keyword's argument, delimited by a comma. These keywords can also
+support some arguments (e.g. a netmask) which must be passed in parenthesis.
+
+A certain category of converters are bitwise and arithmetic operators which
+support performing basic operations on integers. Some bitwise operations are
+supported (and, or, xor, cpl) and some arithmetic operations are supported
+(add, sub, mul, div, mod, neg). Some comparators are provided (odd, even, not,
+bool) which make it possible to report a match without having to write an ACL.
+
+The following keywords are supported:
+
+ keyword input type output type
+------------------------------------------------+-------------+----------------
+51d.single(prop[,prop*]) string string
+add(value) integer integer
+add_item(delim,[var][,suff]]) string string
+aes_gcm_dec(bits,nonce,key,aead_tag) binary binary
+and(value) integer integer
+b64dec string binary
+base64 binary string
+be2dec(separator,chunk_size,[truncate]) binary string
+be2hex([separator],[chunk_size],[truncate]) binary string
+bool integer boolean
+bytes(offset[,length]) binary binary
+capture-req(id) string string
+capture-res(id) string string
+concat([start],[var],[end]) string string
+cpl integer integer
+crc32([avalanche]) binary integer
+crc32c([avalanche]) binary integer
+cut_crlf string string
+da-csv-conv(prop[,prop*]) string string
+debug([prefix][,destination]) any same
+-- keyword -------------------------------------+- input type + output type -
+digest(algorithm) binary binary
+div(value) integer integer
+djb2([avalanche]) binary integer
+even integer boolean
+field(index,delimiters[,count]) string string
+fix_is_valid binary boolean
+fix_tag_value(tag) binary binary
+hex binary string
+hex2i binary integer
+hmac(algorithm,key) binary binary
+host_only string string
+htonl integer integer
+http_date([offset],[unit]) integer string
+iif(true,false) boolean string
+in_table(table) string boolean
+ipmask(mask4,[mask6]) address address
+json([input-code]) string string
+json_query(json_path,[output_type]) string _outtype_
+jwt_header_query([json_path],[output_type]) string string
+jwt_payload_query([json_path],[output_type]) string string
+-- keyword -------------------------------------+- input type + output type -
+jwt_verify(alg,key) string integer
+language(value[,default]) string string
+length string integer
+lower string string
+ltime(format[,offset]) integer string
+ltrim(chars) string string
+map(map_name[,default_value]) string string
+map_match(map_name[,default_value]) _match_ string
+map_match_output(map_name[,default_value]) _match_ _output_
+mod(value) integer integer
+mqtt_field_value(pkt_type,fieldname_or_prop_ID) binary binary
+mqtt_is_valid binary boolean
+ms_ltime(format[,offset]) integer string
+ms_utime(format[,offset]) integer string
+mul(value) integer integer
+nbsrv string integer
+neg integer integer
+not integer boolean
+odd integer boolean
+or(value) integer integer
+-- keyword -------------------------------------+- input type + output type -
+param(name,[delim]) string string
+port_only string integer
+protobuf(field_number,[field_type]) binary binary
+regsub(regex,subst[,flags]) string string
+rfc7239_field(field) string string
+rfc7239_is_valid string boolean
+rfc7239_n2nn string address / str
+rfc7239_n2np string integer / str
+rtrim(chars) string string
+sdbm([avalanche]) binary integer
+secure_memcmp(var) string boolean
+set-var(var[,cond...]) any same
+sha1 binary binary
+sha2([bits]) binary binary
+srv_queue string integer
+strcmp(var) string boolean
+sub(value) integer integer
+table_bytes_in_rate(table) string integer
+table_bytes_out_rate(table) string integer
+table_conn_cnt(table) string integer
+-- keyword -------------------------------------+- input type + output type -
+table_conn_cur(table) string integer
+table_conn_rate(table) string integer
+table_expire(table[,default_value]) string integer
+table_gpc(idx,table) string integer
+table_gpc0(table) string integer
+table_gpc0_rate(table) string integer
+table_gpc1(table) string integer
+table_gpc1_rate(table) string integer
+table_gpc_rate(idx,table) string integer
+table_gpt(idx,table) string integer
+table_gpt0(table) string integer
+table_http_err_cnt(table) string integer
+table_http_err_rate(table) string integer
+table_http_fail_cnt(table) string integer
+table_http_fail_rate(table) string integer
+table_http_req_cnt(table) string integer
+table_http_req_rate(table) string integer
+table_idle(table[,default_value]) string integer
+table_kbytes_in(table) string integer
+-- keyword -------------------------------------+- input type + output type -
+table_kbytes_out(table) string integer
+table_server_id(table) string integer
+table_sess_cnt(table) string integer
+table_sess_rate(table) string integer
+table_trackers(table) string integer
+ub64dec string string
+ub64enc string string
+ungrpc(field_number,[field_type]) binary binary / int
+unset-var(var) any same
+upper string string
+url_dec([in_form]) string string
+url_enc([enc_type]) string string
+us_ltime(format[,offset]) integer string
+us_utime(format[,offset]) integer string
+utime(format[,offset]) integer string
+word(index,delimiters[,count]) string string
+wt6([avalanche]) binary integer
+x509_v_err_str integer string
+xor(value) integer integer
+-- keyword -------------------------------------+- input type + output type -
+xxh3([seed]) binary integer
+xxh32([seed]) binary integer
+xxh64([seed]) binary integer
+
+The detailed list of converter keywords follows:
+
+51d.single(<prop>[,<prop>*])
+ Returns values for the properties requested as a string, where values are
+ separated by the delimiter specified with "51degrees-property-separator".
+ The device is identified using the User-Agent header passed to the
+ converter. The function can be passed up to five property names, and if a
+ property name can't be found, the value "NoData" is returned.
+
+ Example :
+ # Here the header "X-51D-DeviceTypeMobileTablet" is added to the request,
+ # containing values for the three properties requested by using the
+ # User-Agent passed to the converter.
+ frontend http-in
+ bind *:8081
+ default_backend servers
+ http-request set-header X-51D-DeviceTypeMobileTablet \
+ %[req.fhdr(User-Agent),51d.single(DeviceType,IsMobile,IsTablet)]
+
+add(<value>)
+ Adds <value> to the input value of type signed integer, and returns the
+ result as a signed integer. <value> can be a numeric value or a variable
+ name. The name of the variable starts with an indication about its scope. The
+ scopes allowed are:
+ "proc" : the variable is shared with the whole process
+ "sess" : the variable is shared with the whole session
+ "txn" : the variable is shared with the transaction (request and response)
+ "req" : the variable is shared only during request processing
+ "res" : the variable is shared only during response processing
+ This prefix is followed by a name. The separator is a '.'. The name may only
+ contain characters 'a-z', 'A-Z', '0-9', '.' and '_'.
+
+add_item(<delim>,[<var>][,<suff>]])
+ Concatenates a minimum of 2 and up to 3 fields after the current sample which
+ is then turned into a string. The first one, <delim>, is a constant string,
+ that will be appended immediately after the existing sample if an existing
+ sample is not empty and either the <var> or the <suff> is not empty. The
+ second one, <var>, is a variable name. The variable will be looked up, its
+ contents converted to a string, and it will be appended immediately after
+ the <delim> part. If the variable is not found, nothing is appended. It is
+ optional and may optionally be followed by a constant string <suff>, however
+ if <var> is omitted, then <suff> is mandatory. This converter is similar to
+ the concat converter and can be used to build new variables made of a
+ succession of other variables but the main difference is that it does the
+ checks if adding a delimiter makes sense as wouldn't be the case if e.g. the
+ current sample is empty. That situation would require 2 separate rules using
+ concat converter where the first rule would have to check if the current
+ sample string is empty before adding a delimiter. If commas or closing
+ parenthesis are needed as delimiters, they must be protected by quotes or
+ backslashes, themselves protected so that they are not stripped by the first
+ level parser (please see section 2.2 for quoting and escaping). See examples
+ below.
+
+ Example:
+ http-request set-var(req.tagged) 'var(req.tagged),add_item(",",req.score1,"(site1)") if src,in_table(site1)'
+ http-request set-var(req.tagged) 'var(req.tagged),add_item(",",req.score2,"(site2)") if src,in_table(site2)'
+ http-request set-var(req.tagged) 'var(req.tagged),add_item(",",req.score3,"(site3)") if src,in_table(site3)'
+ http-request set-header x-tagged %[var(req.tagged)]
+
+ http-request set-var(req.tagged) 'var(req.tagged),add_item(",",req.score1),add_item(",",req.score2)'
+ http-request set-var(req.tagged) 'var(req.tagged),add_item(",",,(site1))' if src,in_table(site1)
+
+aes_gcm_dec(<bits>,<nonce>,<key>,<aead_tag>)
+ Decrypts the raw byte input using the AES128-GCM, AES192-GCM or
+ AES256-GCM algorithm, depending on the <bits> parameter. All other parameters
+ need to be base64 encoded and the returned result is in raw byte format.
+ If the <aead_tag> validation fails, the converter doesn't return any data.
+ The <nonce>, <key> and <aead_tag> can either be strings or variables. This
+ converter requires at least OpenSSL 1.0.1.
+
+ Example:
+ http-response set-header X-Decrypted-Text %[var(txn.enc),\
+ aes_gcm_dec(128,txn.nonce,Zm9vb2Zvb29mb29wZm9vbw==,txn.aead_tag)]
+
+and(<value>)
+ Performs a bitwise "AND" between <value> and the input value of type signed
+ integer, and returns the result as an signed integer. <value> can be a
+ numeric value or a variable name. The name of the variable starts with an
+ indication about its scope. The scopes allowed are:
+ "proc" : the variable is shared with the whole process
+ "sess" : the variable is shared with the whole session
+ "txn" : the variable is shared with the transaction (request and response)
+ "req" : the variable is shared only during request processing
+ "res" : the variable is shared only during response processing
+ This prefix is followed by a name. The separator is a '.'. The name may only
+ contain characters 'a-z', 'A-Z', '0-9', '.' and '_'.
+
+b64dec
+ Converts (decodes) a base64 encoded input string to its binary
+ representation. It performs the inverse operation of base64().
+ For base64url("URL and Filename Safe Alphabet" (RFC 4648)) variant
+ see "ub64dec".
+
+base64
+ Converts a binary input sample to a base64 string. It is used to log or
+ transfer binary content in a way that can be reliably transferred (e.g.
+ an SSL ID can be copied in a header). For base64url("URL and Filename
+ Safe Alphabet" (RFC 4648)) variant see "ub64enc".
+
+be2dec(<separator>,<chunk_size>,[<truncate>])
+ Converts big-endian binary input sample to a string containing an unsigned
+ integer number per <chunk_size> input bytes. <separator> is put every
+ <chunk_size> binary input bytes if specified. <truncate> flag indicates
+ whatever binary input is truncated at <chunk_size> boundaries. <chunk_size>
+ maximum value is limited by the size of long long int (8 bytes).
+
+ Example:
+ bin(01020304050607),be2dec(:,2) # 258:772:1286:7
+ bin(01020304050607),be2dec(-,2,1) # 258-772-1286
+ bin(01020304050607),be2dec(,2,1) # 2587721286
+ bin(7f000001),be2dec(.,1) # 127.0.0.1
+
+be2hex([<separator>],[<chunk_size>],[<truncate>])
+ Converts big-endian binary input sample to a hex string containing two hex
+ digits per input byte. It is used to log or transfer hex dumps of some
+ binary input data in a way that can be reliably transferred (e.g. an SSL ID
+ can be copied in a header). <separator> is put every <chunk_size> binary
+ input bytes if specified. <truncate> flag indicates whatever binary input is
+ truncated at <chunk_size> boundaries.
+
+ Example:
+ bin(01020304050607),be2hex # 01020304050607
+ bin(01020304050607),be2hex(:,2) # 0102:0304:0506:07
+ bin(01020304050607),be2hex(--,2,1) # 0102--0304--0506
+ bin(0102030405060708),be2hex(,3,1) # 010203040506
+
+bool
+ Returns a boolean TRUE if the input value of type signed integer is
+ non-null, otherwise returns FALSE. Used in conjunction with and(), it can be
+ used to report true/false for bit testing on input values (e.g. verify the
+ presence of a flag).
+
+bytes(<offset>[,<length>])
+ Extracts some bytes from an input binary sample. The result is a binary
+ sample starting at an offset (in bytes) of the original sample and
+ optionally truncated at the given length. <offset> and <length> can be numeric
+ values or variable names. The converter returns an empty sample if either
+ <offset> or <length> is invalid. Invalid <offset> means a negative value or a
+ value >= length of the input sample. Invalid <length> means a negative value
+ or, in some cases, a value bigger than the length of the input sample.
+
+ Example:
+ http-request set-var(txn.input) req.hdr(input) # let's say input is "012345"
+
+ http-response set-header bytes_0 "%[var(txn.input),bytes(0)]" # outputs "012345"
+ http-response set-header bytes_1_3 "%[var(txn.input),bytes(1,3)]" # outputs "123"
+
+ http-response set-var(txn.var_start) int(1)
+ http-response set-var(txn.var_length) int(3)
+ http-response set-header bytes_var1_var3 "%[var(txn.input),bytes(txn.var_start,txn.var_length)]" # outputs "123"
+
+capture-req(<id>)
+ Capture the string entry in the request slot <id> and returns the entry as
+ is. If the slot doesn't exist, the capture fails silently.
+
+ See also: "declare capture", "http-request capture",
+ "http-response capture", "capture.req.hdr" and
+ "capture.res.hdr" (sample fetches).
+
+capture-res(<id>)
+ Capture the string entry in the response slot <id> and returns the entry as
+ is. If the slot doesn't exist, the capture fails silently.
+
+ See also: "declare capture", "http-request capture",
+ "http-response capture", "capture.req.hdr" and
+ "capture.res.hdr" (sample fetches).
+
+concat([<start>],[<var>],[<end>])
+ Concatenates up to 3 fields after the current sample which is then turned to
+ a string. The first one, <start>, is a constant string, that will be appended
+ immediately after the existing sample. It may be omitted if not used. The
+ second one, <var>, is a variable name. The variable will be looked up, its
+ contents converted to a string, and it will be appended immediately after the
+ <first> part. If the variable is not found, nothing is appended. It may be
+ omitted as well. The third field, <end> is a constant string that will be
+ appended after the variable. It may also be omitted. Together, these elements
+ allow to concatenate variables with delimiters to an existing set of
+ variables. This can be used to build new variables made of a succession of
+ other variables, such as colon-delimited values. If commas or closing
+ parenthesis are needed as delimiters, they must be protected by quotes or
+ backslashes, themselves protected so that they are not stripped by the first
+ level parser. This is often used to build composite variables from other
+ ones, but sometimes using a format string with multiple fields may be more
+ convenient. See examples below.
+
+ Example:
+ tcp-request session set-var(sess.src) src
+ tcp-request session set-var(sess.dn) ssl_c_s_dn
+ tcp-request session set-var(txn.sig) str(),concat(<ip=,sess.ip,>),concat(<dn=,sess.dn,>)
+ tcp-request session set-var(txn.ipport) "str(),concat('addr=(',sess.ip),concat(',',sess.port,')')"
+ tcp-request session set-var-fmt(txn.ipport) "addr=(%[sess.ip],%[sess.port])" ## does the same
+ http-request set-header x-hap-sig %[var(txn.sig)]
+
+cpl
+ Takes the input value of type signed integer, applies a ones-complement
+ (flips all bits) and returns the result as an signed integer.
+
+crc32([<avalanche>])
+ Hashes a binary input sample into an unsigned 32-bit quantity using the CRC32
+ hash function. Optionally, it is possible to apply a full avalanche hash
+ function to the output if the optional <avalanche> argument equals 1. This
+ converter uses the same functions as used by the various hash-based load
+ balancing algorithms, so it will provide exactly the same results. It is
+ provided for compatibility with other software which want a CRC32 to be
+ computed on some input keys, so it follows the most common implementation as
+ found in Ethernet, Gzip, PNG, etc... It is slower than the other algorithms
+ but may provide a better or at least less predictable distribution. It must
+ not be used for security purposes as a 32-bit hash is trivial to break. See
+ also "djb2", "sdbm", "wt6", "crc32c" and the "hash-type" directive.
+
+crc32c([<avalanche>])
+ Hashes a binary input sample into an unsigned 32-bit quantity using the CRC32C
+ hash function. Optionally, it is possible to apply a full avalanche hash
+ function to the output if the optional <avalanche> argument equals 1. This
+ converter uses the same functions as described in RFC4960, Appendix B [8].
+ It is provided for compatibility with other software which want a CRC32C to be
+ computed on some input keys. It is slower than the other algorithms and it must
+ not be used for security purposes as a 32-bit hash is trivial to break. See
+ also "djb2", "sdbm", "wt6", "crc32" and the "hash-type" directive.
+
+cut_crlf
+ Cuts the string representation of the input sample on the first carriage
+ return ('\r') or newline ('\n') character found. Only the string length is
+ updated.
+
+da-csv-conv(<prop>[,<prop>*])
+ Asks the DeviceAtlas converter to identify the User Agent string passed on
+ input, and to emit a string made of the concatenation of the properties
+ enumerated in argument, delimited by the separator defined by the global
+ keyword "deviceatlas-property-separator", or by default the pipe character
+ ('|'). There's a limit of 12 different properties imposed by the HAProxy
+ configuration language.
+
+ Example:
+ frontend www
+ bind *:8881
+ default_backend servers
+ http-request set-header X-DeviceAtlas-Data %[req.fhdr(User-Agent),da-csv(primaryHardwareType,osName,osVersion,browserName,browserVersion,browserRenderingEngine)]
+
+debug([<prefix][,<destination>])
+ This converter is used as debug tool. It takes a capture of the input sample
+ and sends it to event sink <destination>, which may designate a ring buffer
+ such as "buf0", as well as "stdout", or "stderr". Available sinks may be
+ checked at run time by issuing "show events" on the CLI. When not specified,
+ the output will be "buf0", which may be consulted via the CLI's "show events"
+ command. An optional prefix <prefix> may be passed to help distinguish
+ outputs from multiple expressions. It will then appear before the colon in
+ the output message. The input sample is passed as-is on the output, so that
+ it is safe to insert the debug converter anywhere in a chain, even with non-
+ printable sample types.
+
+ Example:
+ tcp-request connection track-sc0 src,debug(track-sc)
+
+digest(<algorithm>)
+ Converts a binary input sample to a message digest. The result is a binary
+ sample. The <algorithm> must be an OpenSSL message digest name (e.g. sha256).
+
+ Please note that this converter is only available when HAProxy has been
+ compiled with USE_OPENSSL.
+
+div(<value>)
+ Divides the input value of type signed integer by <value>, and returns the
+ result as an signed integer. If <value> is null, the largest unsigned
+ integer is returned (typically 2^63-1). <value> can be a numeric value or a
+ variable name. The name of the variable starts with an indication about its
+ scope. The scopes allowed are:
+ "proc" : the variable is shared with the whole process
+ "sess" : the variable is shared with the whole session
+ "txn" : the variable is shared with the transaction (request and response)
+ "req" : the variable is shared only during request processing
+ "res" : the variable is shared only during response processing
+ This prefix is followed by a name. The separator is a '.'. The name may only
+ contain characters 'a-z', 'A-Z', '0-9', '.' and '_'.
+
+djb2([<avalanche>])
+ Hashes a binary input sample into an unsigned 32-bit quantity using the DJB2
+ hash function. Optionally, it is possible to apply a full avalanche hash
+ function to the output if the optional <avalanche> argument equals 1. This
+ converter uses the same functions as used by the various hash-based load
+ balancing algorithms, so it will provide exactly the same results. It is
+ mostly intended for debugging, but can be used as a stick-table entry to
+ collect rough statistics. It must not be used for security purposes as a
+ 32-bit hash is trivial to break. See also "crc32", "sdbm", "wt6", "crc32c",
+ and the "hash-type" directive.
+
+even
+ Returns a boolean TRUE if the input value of type signed integer is even
+ otherwise returns FALSE. It is functionally equivalent to "not,and(1),bool".
+
+field(<index>,<delimiters>[,<count>])
+ Extracts the substring at the given index counting from the beginning
+ (positive index) or from the end (negative index) considering given delimiters
+ from an input string. Indexes start at 1 or -1 and delimiters are a string
+ formatted list of chars. Optionally you can specify <count> of fields to
+ extract (default: 1). Value of 0 indicates extraction of all remaining
+ fields.
+
+ Example :
+ str(f1_f2_f3__f5),field(4,_) # <empty>
+ str(f1_f2_f3__f5),field(5,_) # f5
+ str(f1_f2_f3__f5),field(2,_,0) # f2_f3__f5
+ str(f1_f2_f3__f5),field(2,_,2) # f2_f3
+ str(f1_f2_f3__f5),field(-2,_,3) # f2_f3_
+ str(f1_f2_f3__f5),field(-3,_,0) # f1_f2_f3
+
+fix_is_valid
+ Parses a binary payload and performs sanity checks regarding FIX (Financial
+ Information eXchange):
+
+ - checks that all tag IDs and values are not empty and the tags IDs are well
+ numeric
+ - checks the BeginString tag is the first tag with a valid FIX version
+ - checks the BodyLength tag is the second one with the right body length
+ - checks the MsgType tag is the third tag.
+ - checks that last tag in the message is the CheckSum tag with a valid
+ checksum
+
+ Due to current HAProxy design, only the first message sent by the client and
+ the server can be parsed.
+
+ This converter returns a boolean, true if the payload contains a valid FIX
+ message, false if not.
+
+ See also the fix_tag_value converter.
+
+ Example:
+ tcp-request inspect-delay 10s
+ tcp-request content reject unless { req.payload(0,0),fix_is_valid }
+
+fix_tag_value(<tag>)
+ Parses a FIX (Financial Information eXchange) message and extracts the value
+ from the tag <tag>. <tag> can be a string or an integer pointing to the
+ desired tag. Any integer value is accepted, but only the following strings
+ are translated into their integer equivalent: BeginString, BodyLength,
+ MsgType, SenderCompID, TargetCompID, CheckSum. More tag names can be easily
+ added.
+
+ Due to current HAProxy design, only the first message sent by the client and
+ the server can be parsed. No message validation is performed by this
+ converter. It is highly recommended to validate the message first using
+ fix_is_valid converter.
+
+ See also the fix_is_valid converter.
+
+ Example:
+ tcp-request inspect-delay 10s
+ tcp-request content reject unless { req.payload(0,0),fix_is_valid }
+ # MsgType tag ID is 35, so both lines below will return the same content
+ tcp-request content set-var(txn.foo) req.payload(0,0),fix_tag_value(35)
+ tcp-request content set-var(txn.bar) req.payload(0,0),fix_tag_value(MsgType)
+
+hex
+ Converts a binary input sample to a hex string containing two hex digits per
+ input byte. It is used to log or transfer hex dumps of some binary input data
+ in a way that can be reliably transferred (e.g. an SSL ID can be copied in a
+ header).
+
+hex2i
+ Converts a hex string containing two hex digits per input byte to an
+ integer. If the input value cannot be converted, then zero is returned.
+
+hmac(<algorithm>,<key>)
+ Converts a binary input sample to a message authentication code with the given
+ key. The result is a binary sample. The <algorithm> must be one of the
+ registered OpenSSL message digest names (e.g. sha256). The <key> parameter must
+ be base64 encoded and can either be a string or a variable.
+
+ Please note that this converter is only available when HAProxy has been
+ compiled with USE_OPENSSL.
+
+host_only
+ Converts a string which contains a Host header value and removes its port.
+ The input must respect the format of the host header value
+ (rfc9110#section-7.2). It will support that kind of input: hostname,
+ hostname:80, 127.0.0.1, 127.0.0.1:80, [::1], [::1]:80.
+
+ This converter also sets the string in lowercase.
+
+ See also: "port_only" converter which will return the port.
+
+htonl
+ Converts the input integer value to its 32-bit binary representation in the
+ network byte order. Because sample fetches own signed 64-bit integer, when
+ this converter is used, the input integer value is first casted to an
+ unsigned 32-bit integer.
+
+http_date([<offset],[<unit>])
+ Converts an integer supposed to contain a date since epoch to a string
+ representing this date in a format suitable for use in HTTP header fields. If
+ an offset value is specified, then it is added to the date before the
+ conversion is operated. This is particularly useful to emit Date header fields,
+ Expires values in responses when combined with a positive offset, or
+ Last-Modified values when the offset is negative.
+ If a unit value is specified, then consider the timestamp as either
+ "s" for seconds (default behavior), "ms" for milliseconds, or "us" for
+ microseconds since epoch. Offset is assumed to have the same unit as
+ input timestamp.
+
+iif(<true>,<false>)
+ Returns the <true> string if the input value is true. Returns the <false>
+ string otherwise.
+
+ Example:
+ http-request set-header x-forwarded-proto %[ssl_fc,iif(https,http)]
+
+in_table(<table>)
+ Uses the string representation of the input sample to perform a look up in
+ the specified table. If the key is not found in the table, a boolean false
+ is returned. Otherwise a boolean true is returned. This can be used to verify
+ the presence of a certain key in a table tracking some elements (e.g. whether
+ or not a source IP address or an Authorization header was already seen).
+
+ipmask(<mask4>,[<mask6>])
+ Apply a mask to an IP address, and use the result for lookups and storage.
+ This can be used to make all hosts within a certain mask to share the same
+ table entries and as such use the same server. The mask4 can be passed in
+ dotted form (e.g. 255.255.255.0) or in CIDR form (e.g. 24). The mask6 can
+ be passed in quadruplet form (e.g. ffff:ffff::) or in CIDR form (e.g. 64).
+ If no mask6 is given IPv6 addresses will fail to convert for backwards
+ compatibility reasons.
+
+json([<input-code>])
+ Escapes the input string and produces an ASCII output string ready to use as a
+ JSON string. The converter tries to decode the input string according to the
+ <input-code> parameter. It can be "ascii", "utf8", "utf8s", "utf8p" or
+ "utf8ps". The "ascii" decoder never fails. The "utf8" decoder detects 3 types
+ of errors:
+ - bad UTF-8 sequence (lone continuation byte, bad number of continuation
+ bytes, ...)
+ - invalid range (the decoded value is within a UTF-8 prohibited range),
+ - code overlong (the value is encoded with more bytes than necessary).
+
+ The UTF-8 JSON encoding can produce a "too long value" error when the UTF-8
+ character is greater than 0xffff because the JSON string escape specification
+ only authorizes 4 hex digits for the value encoding. The UTF-8 decoder exists
+ in 4 variants designated by a combination of two suffix letters : "p" for
+ "permissive" and "s" for "silently ignore". The behaviors of the decoders
+ are :
+ - "ascii" : never fails;
+ - "utf8" : fails on any detected errors;
+ - "utf8s" : never fails, but removes characters corresponding to errors;
+ - "utf8p" : accepts and fixes the overlong errors, but fails on any other
+ error;
+ - "utf8ps" : never fails, accepts and fixes the overlong errors, but removes
+ characters corresponding to the other errors.
+
+ This converter is particularly useful for building properly escaped JSON for
+ logging to servers which consume JSON-formatted traffic logs.
+
+ Example:
+ capture request header Host len 15
+ capture request header user-agent len 150
+ log-format '{"ip":"%[src]","user-agent":"%[capture.req.hdr(1),json(utf8s)]"}'
+
+ Input request from client 127.0.0.1:
+ GET / HTTP/1.0
+ User-Agent: Very "Ugly" UA 1/2
+
+ Output log:
+ {"ip":"127.0.0.1","user-agent":"Very \"Ugly\" UA 1\/2"}
+
+json_query(<json_path>,[<output_type>])
+ The json_query converter supports the JSON types string, boolean, number
+ and array. Floating point numbers will be returned as a string. By
+ specifying the output_type 'int' the value will be converted to an
+ Integer. Arrays will be returned as string, starting and ending with a
+ square brackets. The content is a CSV. Depending on the data type, the
+ array values might be quoted. If the array values are complex types,
+ the string contains the complete json representation of each value
+ separated by a comma. Example result for a roles query to a JWT:
+
+ ["manage-account","manage-account-links","view-profile"]
+
+ If conversion is not possible the json_query converter fails.
+
+ <json_path> must be a valid JSON Path string as defined in
+ https://datatracker.ietf.org/doc/draft-ietf-jsonpath-base/
+
+ Example:
+ # get a integer value from the request body
+ # "{"integer":4}" => 5
+ http-request set-var(txn.pay_int) req.body,json_query('$.integer','int'),add(1)
+
+ # get a key with '.' in the name
+ # {"my.key":"myvalue"} => myvalue
+ http-request set-var(txn.pay_mykey) req.body,json_query('$.my\\.key')
+
+ # {"boolean-false":false} => 0
+ http-request set-var(txn.pay_boolean_false) req.body,json_query('$.boolean-false')
+
+ # get the value of the key 'iss' from a JWT Bearer token
+ http-request set-var(txn.token_payload) req.hdr(Authorization),word(2,.),ub64dec,json_query('$.iss')
+
+jwt_header_query([<json_path>],[<output_type>])
+ When given a JSON Web Token (JWT) in input, either returns the decoded header
+ part of the token (the first base64-url encoded part of the JWT) if no
+ parameter is given, or performs a json_query on the decoded header part of
+ the token. See "json_query" converter for details about the accepted
+ json_path and output_type parameters.
+
+ Please note that this converter is only available when HAProxy has been
+ compiled with USE_OPENSSL.
+
+jwt_payload_query([<json_path>],[<output_type>])
+ When given a JSON Web Token (JWT) in input, either returns the decoded
+ payload part of the token (the second base64-url encoded part of the JWT) if
+ no parameter is given, or performs a json_query on the decoded payload part
+ of the token. See "json_query" converter for details about the accepted
+ json_path and output_type parameters.
+
+ Please note that this converter is only available when HAProxy has been
+ compiled with USE_OPENSSL.
+
+jwt_verify(<alg>,<key>)
+ Performs a signature verification for the JSON Web Token (JWT) given in input
+ by using the <alg> algorithm and the <key> parameter, which should either
+ hold a secret or a path to a public certificate. Returns 1 in case of
+ verification success, 0 in case of verification error and a strictly negative
+ value for any other error. Because of all those non-null error return values,
+ the result of this converter should never be converted to a boolean. See
+ below for a full list of the possible return values.
+
+ For now, only JWS tokens using the Compact Serialization format can be
+ processed (three dot-separated base64-url encoded strings). All the
+ algorithms mentioned in section 3.1 of RFC7518 are managed (HS, ES, RS and PS
+ with the 256, 384 or 512 key sizes, as well as the special "none" case).
+
+ If the used algorithm is of the HMAC family, <key> should be the secret used
+ in the HMAC signature calculation. Otherwise, <key> should be the path to the
+ public certificate that can be used to validate the token's signature. All
+ the certificates that might be used to verify JWTs must be known during init
+ in order to be added into a dedicated certificate cache so that no disk
+ access is required during runtime. For this reason, any used certificate must
+ be mentioned explicitly at least once in a jwt_verify call. Passing an
+ intermediate variable as second parameter is then not advised.
+
+ This converter only verifies the signature of the token and does not perform
+ a full JWT validation as specified in section 7.2 of RFC7519. We do not
+ ensure that the header and payload contents are fully valid JSON's once
+ decoded for instance, and no checks are performed regarding their respective
+ contents.
+
+ The possible return values are the following :
+
+ +----+----------------------------------------------------------------------+
+ | ID | message |
+ +----+----------------------------------------------------------------------+
+ | 0 | "Verification failure" |
+ | 1 | "Verification success" |
+ | -1 | "Unknown algorithm (not mentioned in RFC7518)" |
+ | -2 | "Unmanaged algorithm" |
+ | -3 | "Invalid token" |
+ | -4 | "Out of memory" |
+ | -5 | "Unknown certificate" |
+ +----+----------------------------------------------------------------------+
+
+ Please note that this converter is only available when HAProxy has been
+ compiled with USE_OPENSSL.
+
+ Example:
+ # Get a JWT from the authorization header, extract the "alg" field of its
+ # JOSE header and use a public certificate to verify a signature
+ http-request set-var(txn.bearer) http_auth_bearer
+ http-request set-var(txn.jwt_alg) var(txn.bearer),jwt_header_query('$.alg')
+ http-request deny unless { var(txn.jwt_alg) -m str "RS256" }
+ http-request deny unless { var(txn.bearer),jwt_verify(txn.jwt_alg,"/path/to/crt.pem") 1 }
+
+language(<value>[,<default>])
+ Returns the value with the highest q-factor from a list as extracted from the
+ "accept-language" header using "req.fhdr". Values with no q-factor have a
+ q-factor of 1. Values with a q-factor of 0 are dropped. Only values which
+ belong to the list of semi-colon delimited <values> will be considered. The
+ argument <value> syntax is "lang[;lang[;lang[;...]]]". If no value matches the
+ given list and a default value is provided, it is returned. Note that language
+ names may have a variant after a dash ('-'). If this variant is present in the
+ list, it will be matched, but if it is not, only the base language is checked.
+ The match is case-sensitive, and the output string is always one of those
+ provided in arguments. The ordering of arguments is meaningless, only the
+ ordering of the values in the request counts, as the first value among
+ multiple sharing the same q-factor is used.
+
+ Example :
+
+ # this configuration switches to the backend matching a
+ # given language based on the request :
+
+ acl es req.fhdr(accept-language),language(es;fr;en) -m str es
+ acl fr req.fhdr(accept-language),language(es;fr;en) -m str fr
+ acl en req.fhdr(accept-language),language(es;fr;en) -m str en
+ use_backend spanish if es
+ use_backend french if fr
+ use_backend english if en
+ default_backend choose_your_language
+
+length
+ Get the length of the string. This can only be placed after a string
+ sample fetch function or after a transformation keyword returning a string
+ type. The result is of type integer.
+
+lower
+ Convert a string sample to lower case. This can only be placed after a string
+ sample fetch function or after a transformation keyword returning a string
+ type. The result is of type string.
+
+ltime(<format>[,<offset>])
+ Converts an integer supposed to contain a date since epoch to a string
+ representing this date in local time using a format defined by the <format>
+ string using strftime(3). The purpose is to allow any date format to be used
+ in logs. An optional <offset> in seconds may be applied to the input date
+ (positive or negative). See the strftime() man page for the format supported
+ by your operating system. See also the utime converter.
+
+ Example :
+
+ # Emit two colons, one with the local time and another with ip:port
+ # e.g. 20140710162350 127.0.0.1:57325
+ log-format %[date,ltime(%Y%m%d%H%M%S)]\ %ci:%cp
+
+ltrim(<chars>)
+ Skips any characters from <chars> from the beginning of the string
+ representation of the input sample.
+
+map(<map_file>[,<default_value>])
+map_<match_type>(<map_file>[,<default_value>])
+map_<match_type>_<output_type>(<map_file>[,<default_value>])
+ Search the input value from <map_file> using the <match_type> matching method,
+ and return the associated value converted to the type <output_type>. If the
+ input value cannot be found in the <map_file>, the converter returns the
+ <default_value>. If the <default_value> is not set, the converter fails and
+ acts as if no input value could be fetched. If the <match_type> is not set, it
+ defaults to "str". Likewise, if the <output_type> is not set, it defaults to
+ "str". For convenience, the "map" keyword is an alias for "map_str" and maps a
+ string to another string.
+
+ It is important to avoid overlapping between the keys : IP addresses and
+ strings are stored in trees, so the first of the finest match will be used.
+ Other keys are stored in lists, so the first matching occurrence will be used.
+
+ The following array contains the list of all map functions available sorted by
+ input type, match type and output type.
+
+ input type | match method | output type str | output type int | output type ip
+ -----------+--------------+-----------------+-----------------+---------------
+ str | str | map_str | map_str_int | map_str_ip
+ -----------+--------------+-----------------+-----------------+---------------
+ str | beg | map_beg | map_beg_int | map_end_ip
+ -----------+--------------+-----------------+-----------------+---------------
+ str | sub | map_sub | map_sub_int | map_sub_ip
+ -----------+--------------+-----------------+-----------------+---------------
+ str | dir | map_dir | map_dir_int | map_dir_ip
+ -----------+--------------+-----------------+-----------------+---------------
+ str | dom | map_dom | map_dom_int | map_dom_ip
+ -----------+--------------+-----------------+-----------------+---------------
+ str | end | map_end | map_end_int | map_end_ip
+ -----------+--------------+-----------------+-----------------+---------------
+ str | reg | map_reg | map_reg_int | map_reg_ip
+ -----------+--------------+-----------------+-----------------+---------------
+ str | reg | map_regm | map_reg_int | map_reg_ip
+ -----------+--------------+-----------------+-----------------+---------------
+ int | int | map_int | map_int_int | map_int_ip
+ -----------+--------------+-----------------+-----------------+---------------
+ ip | ip | map_ip | map_ip_int | map_ip_ip
+ -----------+--------------+-----------------+-----------------+---------------
+
+ The special map called "map_regm" expect matching zone in the regular
+ expression and modify the output replacing back reference (like "\1") by
+ the corresponding match text.
+
+ The file contains one key + value per line. Lines which start with '#' are
+ ignored, just like empty lines. Leading tabs and spaces are stripped. The key
+ is then the first "word" (series of non-space/tabs characters), and the value
+ is what follows this series of space/tab till the end of the line excluding
+ trailing spaces/tabs.
+
+ Example :
+
+ # this is a comment and is ignored
+ 2.22.246.0/23 United Kingdom \n
+ <-><-----------><--><------------><---->
+ | | | | `- trailing spaces ignored
+ | | | `---------- value
+ | | `-------------------- middle spaces ignored
+ | `---------------------------- key
+ `------------------------------------ leading spaces ignored
+
+mod(<value>)
+ Divides the input value of type signed integer by <value>, and returns the
+ remainder as an signed integer. If <value> is null, then zero is returned.
+ <value> can be a numeric value or a variable name. The name of the variable
+ starts with an indication about its scope. The scopes allowed are:
+ "proc" : the variable is shared with the whole process
+ "sess" : the variable is shared with the whole session
+ "txn" : the variable is shared with the transaction (request and response)
+ "req" : the variable is shared only during request processing
+ "res" : the variable is shared only during response processing
+ This prefix is followed by a name. The separator is a '.'. The name may only
+ contain characters 'a-z', 'A-Z', '0-9', '.' and '_'.
+
+mqtt_field_value(<packettype>,<fieldname_or_property_ID>)
+ Returns value of <fieldname> found in input MQTT payload of type
+ <packettype>.
+ <packettype> can be either a string (case insensitive matching) or a numeric
+ value corresponding to the type of packet we're supposed to extract data
+ from.
+ Supported string and integers can be found here:
+ https://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html#_Toc398718021
+ https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901022
+
+ <fieldname> depends on <packettype> and can be any of the following below.
+ (note that <fieldname> matching is case insensitive).
+ <property id> can only be found in MQTT v5.0 streams. check this table:
+ https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901029
+
+ - CONNECT (or 1): flags, protocol_name, protocol_version, client_identifier,
+ will_topic, will_payload, username, password, keepalive
+ OR any property ID as a numeric value (for MQTT v5.0
+ packets only):
+ 17: Session Expiry Interval
+ 33: Receive Maximum
+ 39: Maximum Packet Size
+ 34: Topic Alias Maximum
+ 25: Request Response Information
+ 23: Request Problem Information
+ 21: Authentication Method
+ 22: Authentication Data
+ 18: Will Delay Interval
+ 1: Payload Format Indicator
+ 2: Message Expiry Interval
+ 3: Content Type
+ 8: Response Topic
+ 9: Correlation Data
+ Not supported yet:
+ 38: User Property
+
+ - CONNACK (or 2): flags, protocol_version, reason_code
+ OR any property ID as a numeric value (for MQTT v5.0
+ packets only):
+ 17: Session Expiry Interval
+ 33: Receive Maximum
+ 36: Maximum QoS
+ 37: Retain Available
+ 39: Maximum Packet Size
+ 18: Assigned Client Identifier
+ 34: Topic Alias Maximum
+ 31: Reason String
+ 40; Wildcard Subscription Available
+ 41: Subscription Identifiers Available
+ 42: Shared Subscription Available
+ 19: Server Keep Alive
+ 26: Response Information
+ 28: Server Reference
+ 21: Authentication Method
+ 22: Authentication Data
+ Not supported yet:
+ 38: User Property
+
+ Due to current HAProxy design, only the first message sent by the client and
+ the server can be parsed. Thus this converter can extract data only from
+ CONNECT and CONNACK packet types. CONNECT is the first message sent by the
+ client and CONNACK is the first response sent by the server.
+
+ Example:
+
+ acl data_in_buffer req.len ge 4
+ tcp-request content set-var(txn.username) \
+ req.payload(0,0),mqtt_field_value(connect,protocol_name) \
+ if data_in_buffer
+ # do the same as above
+ tcp-request content set-var(txn.username) \
+ req.payload(0,0),mqtt_field_value(1,protocol_name) \
+ if data_in_buffer
+
+mqtt_is_valid
+ Checks that the binary input is a valid MQTT packet. It returns a boolean.
+
+ Due to current HAProxy design, only the first message sent by the client and
+ the server can be parsed. Thus this converter can extract data only from
+ CONNECT and CONNACK packet types. CONNECT is the first message sent by the
+ client and CONNACK is the first response sent by the server.
+
+ Only MQTT 3.1, 3.1.1 and 5.0 are supported.
+
+ Example:
+
+ acl data_in_buffer req.len ge 4
+ tcp-request content reject unless { req.payload(0,0),mqtt_is_valid }
+
+ms_ltime(<format>[,<offset>])
+ This works like "ltime" but takes an input in milliseconds. It also supports
+ the %N conversion specifier inspired by date(1).
+ Converts an integer supposed to contain a date since epoch to a string
+ representing this date in local time using a format defined by the <format>
+ string using strftime(3). The purpose is to allow any date format to be used
+ in logs. An optional <offset> in milliseconds may be applied to the input date
+ (positive or negative). See the strftime() man page for the format supported
+ by your operating system.
+
+ The %N conversion specifier allows you to output the nanoseconds part of the
+ date, precision is limited since the input is milliseconds.
+ (000000000..999000000). %N can take a width argument between % and N. It is
+ useful to display milliseconds (%3N) or microseconds (%6N). The default and
+ maximum width is 9 (%N = %9N).
+
+ See also the utime converter for UTC as well as "ltime" and "us_ltime"
+ converters.
+
+ Example :
+
+ # Emit 3 colons, the local time, the timezone and another with ip:port
+ # e.g. 2023/07/24/11:53:02.196 +0200 127.0.0.1:41530
+ log-format %[accept_date(ms),ms_ltime("%Y/%m/%d/%H:%M:%S.%3N %z")]\ %ci:%cp
+
+ms_utime(<format>[,<offset>])
+ This works like "utime" but takes an input in milliseconds. It also supports
+ the %N conversion specifier inspired by date(1).
+ Converts an integer supposed to contain a date since epoch to a string
+ representing this date in UTC time using a format defined by the <format>
+ string using strftime(3). The purpose is to allow any date format to be used
+ in logs. An optional <offset> in milliseconds may be applied to the input date
+ (positive or negative). See the strftime() man page for the format supported
+ by your operating system.
+
+ The %N conversion specifier allows you to output the nanoseconds part of the
+ date, precision is limited since the input is milliseconds.
+ (000000000..999000000). %N can take a width argument between % and N. It is
+ useful to display milliseconds (%3N) or microseconds (%6N). The default and
+ maximum width is 9 (%N = %9N).
+
+ See also the ltime converter for local as well as "utime" and "us_utime"
+ converters.
+
+ Example :
+
+ # Emit 3 colons, the UTC time, the timezone and another with ip:port
+ # e.g. 2023/07/24/09:53:02.196 +0000 127.0.0.1:41530
+ log-format %[accept_date(ms),ms_utime("%Y/%m/%d/%H:%M:%S.%3N %z")]\ %ci:%cp
+
+mul(<value>)
+ Multiplies the input value of type signed integer by <value>, and returns
+ the product as an signed integer. In case of overflow, the largest possible
+ value for the sign is returned so that the operation doesn't wrap around.
+ <value> can be a numeric value or a variable name. The name of the variable
+ starts with an indication about its scope. The scopes allowed are:
+ "proc" : the variable is shared with the whole process
+ "sess" : the variable is shared with the whole session
+ "txn" : the variable is shared with the transaction (request and response)
+ "req" : the variable is shared only during request processing
+ "res" : the variable is shared only during response processing
+ This prefix is followed by a name. The separator is a '.'. The name may only
+ contain characters 'a-z', 'A-Z', '0-9', '.' and '_'.
+
+nbsrv
+ Takes an input value of type string, interprets it as a backend name and
+ returns the number of usable servers in that backend. Can be used in places
+ where we want to look up a backend from a dynamic name, like a result of a
+ map lookup.
+
+neg
+ Takes the input value of type signed integer, computes the opposite value,
+ and returns the remainder as an signed integer. 0 is identity. This operator
+ is provided for reversed subtracts : in order to subtract the input from a
+ constant, simply perform a "neg,add(value)".
+
+not
+ Returns a boolean FALSE if the input value of type signed integer is
+ non-null, otherwise returns TRUE. Used in conjunction with and(), it can be
+ used to report true/false for bit testing on input values (e.g. verify the
+ absence of a flag).
+
+odd
+ Returns a boolean TRUE if the input value of type signed integer is odd
+ otherwise returns FALSE. It is functionally equivalent to "and(1),bool".
+
+or(<value>)
+ Performs a bitwise "OR" between <value> and the input value of type signed
+ integer, and returns the result as an signed integer. <value> can be a
+ numeric value or a variable name. The name of the variable starts with an
+ indication about its scope. The scopes allowed are:
+ "proc" : the variable is shared with the whole process
+ "sess" : the variable is shared with the whole session
+ "txn" : the variable is shared with the transaction (request and response)
+ "req" : the variable is shared only during request processing
+ "res" : the variable is shared only during response processing
+ This prefix is followed by a name. The separator is a '.'. The name may only
+ contain characters 'a-z', 'A-Z', '0-9', '.' and '_'.
+
+param(<name>,[<delim>])
+ This extracts the first occurrence of the parameter <name> in the input string
+ where parameters are delimited by <delim>, which defaults to "&", and the name
+ and value of the parameter are separated by a "=". If there is no "=" and
+ value before the end of the parameter segment, it is treated as equivalent to
+ a value of an empty string.
+
+ This can be useful for extracting parameters from a query string, or possibly
+ a x-www-form-urlencoded body. In particular, `query,param(<name>)` can be used
+ as an alternative to `urlp(<name>)` which only uses "&" as a delimiter,
+ whereas "urlp" also uses "?" and ";".
+
+ Note that this converter doesn't do anything special with url encoded
+ characters. If you want to decode the value, you can use the url_dec converter
+ on the output. If the name of the parameter in the input might contain encoded
+ characters, you'll probably want do normalize the input before calling
+ "param". This can be done using "http-request normalize-uri", in particular
+ the percent-decode-unreserved and percent-to-uppercase options.
+
+ Example :
+ str(a=b&c=d&a=r),param(a) # b
+ str(a&b=c),param(a) # ""
+ str(a=&b&c=a),param(b) # ""
+ str(a=1;b=2;c=4),param(b,;) # 2
+ query,param(redirect_uri),urldec()
+
+port_only
+ Converts a string which contains a Host header value into an integer by
+ returning its port.
+ The input must respect the format of the host header value
+ (rfc9110#section-7.2). It will support that kind of input: hostname,
+ hostname:80, 127.0.0.1, 127.0.0.1:80, [::1], [::1]:80.
+
+ If no port were provided in the input, it will return 0.
+
+ See also: "host_only" converter which will return the host.
+
+protobuf(<field_number>,[<field_type>])
+ This extracts the protocol buffers message field in raw mode of an input binary
+ sample representation of a protocol buffer message with <field_number> as field
+ number (dotted notation) if <field_type> is not present, or as an integer sample
+ if this field is present (see also "ungrpc" below).
+ The list of the authorized types is the following one: "int32", "int64", "uint32",
+ "uint64", "sint32", "sint64", "bool", "enum" for the "varint" wire type 0
+ "fixed64", "sfixed64", "double" for the 64bit wire type 1, "fixed32", "sfixed32",
+ "float" for the wire type 5. Note that "string" is considered as a length-delimited
+ type, so it does not require any <field_type> argument to be extracted.
+ More information may be found here about the protocol buffers message field types:
+ https://developers.google.com/protocol-buffers/docs/encoding
+
+regsub(<regex>,<subst>[,<flags>])
+ Applies a regex-based substitution to the input string. It does the same
+ operation as the well-known "sed" utility with "s/<regex>/<subst>/". By
+ default it will replace in the input string the first occurrence of the
+ largest part matching the regular expression <regex> with the substitution
+ string <subst>. It is possible to replace all occurrences instead by adding
+ the flag "g" in the third argument <flags>. It is also possible to make the
+ regex case insensitive by adding the flag "i" in <flags>. Since <flags> is a
+ string, it is made up from the concatenation of all desired flags. Thus if
+ both "i" and "g" are desired, using "gi" or "ig" will have the same effect.
+ The first use of this converter is to replace certain characters or sequence
+ of characters with other ones.
+
+ It is highly recommended to enclose the regex part using protected quotes to
+ improve clarity and never have a closing parenthesis from the regex mixed up
+ with the parenthesis from the function. Just like in Bourne shell, the first
+ level of quotes is processed when delimiting word groups on the line, a
+ second level is usable for argument. It is recommended to use single quotes
+ outside since these ones do not try to resolve backslashes nor dollar signs.
+
+ Examples:
+
+ # de-duplicate "/" in header "x-path".
+ # input: x-path: /////a///b/c/xzxyz/
+ # output: x-path: /a/b/c/xzxyz/
+ http-request set-header x-path "%[hdr(x-path),regsub('/+','/','g')]"
+
+ # copy query string to x-query and drop all leading '?', ';' and '&'
+ http-request set-header x-query "%[query,regsub([?;&]*,'')]"
+
+ # capture groups and backreferences
+ # both lines do the same.
+ http-request redirect location %[url,'regsub("(foo|bar)([0-9]+)?","\2\1",i)']
+ http-request redirect location %[url,regsub(\"(foo|bar)([0-9]+)?\",\"\2\1\",i)]
+
+rfc7239_field(<field>)
+ Extracts a single field/parameter from RFC 7239 compliant header value input.
+
+ Supported fields are:
+ - proto: either 'http' or 'https'
+ - host: http compliant host
+ - for: RFC7239 node
+ - by: RFC7239 node
+
+ More info here:
+ https://www.rfc-editor.org/rfc/rfc7239.html#section-6
+
+ Example:
+ # extract host field from forwarded header and store it in req.fhost var
+ http-request set-var(req.fhost) req.hdr(forwarded),rfc7239_field(host)
+ #input: "proto=https;host=\"haproxy.org:80\""
+ # output: "haproxy.org:80"
+
+ # extract for field from forwarded header and store it in req.ffor var
+ http-request set-var(req.ffor) req.hdr(forwarded),rfc7239_field(for)
+ #input: "proto=https;host=\"haproxy.org:80\";for=\"127.0.0.1:9999\""
+ # output: "127.0.0.1:9999"
+
+rfc7239_is_valid
+ Returns true if input header is RFC 7239 compliant header value and false
+ otherwise.
+
+ Example:
+ acl valid req.hdr(forwarded),rfc7239_is_valid
+ #input: "for=127.0.0.1;proto=http"
+ # output: TRUE
+ #input: "proto=custom"
+ # output: FALSE
+
+rfc7239_n2nn
+ Converts RFC7239 node (provided by 'for' or 'by' 7239 header fields)
+ into its corresponding nodename final form:
+ - ipv4 address
+ - ipv6 address
+ - 'unknown'
+ - '_obfs' identifier
+
+ Example:
+ # extract 'for' field from forwarded header, extract nodename from
+ # resulting node identifier and store the result in req.fnn
+ http-request set-var(req.fnn) req.hdr(forwarded),rfc7239_field(for),rfc7239_n2nn
+ #input: "127.0.0.1:9999"
+ # output: 127.0.0.1 (ipv4)
+ #input: "[ab:cd:ff:ff:ff:ff:ff:ff]:9998"
+ # output: ab:cd:ff:ff:ff:ff:ff:ff (ipv6)
+ #input: "_name:_port"
+ # output: "_name" (string)
+
+rfc7239_n2np
+ Converts RFC7239 node (provided by 'for' or 'by' 7239 header fields)
+ into its corresponding nodeport final form:
+ - unsigned integer
+ - '_obfs' identifier
+
+ Example:
+ # extract 'by' field from forwarded header, extract node port from
+ # resulting node identifier and store the result in req.fnp
+ http-request set-var(req.fnp) req.hdr(forwarded),rfc7239_field(by),rfc7239_n2np
+ #input: "127.0.0.1:9999"
+ # output: 9999 (integer)
+ #input: "[ab:cd:ff:ff:ff:ff:ff:ff]:9998"
+ # output: 9998 (integer)
+ #input: "_name:_port"
+ # output: "_port" (string)
+
+rtrim(<chars>)
+ Skips any characters from <chars> from the end of the string representation
+ of the input sample.
+
+sdbm([<avalanche>])
+ Hashes a binary input sample into an unsigned 32-bit quantity using the SDBM
+ hash function. Optionally, it is possible to apply a full avalanche hash
+ function to the output if the optional <avalanche> argument equals 1. This
+ converter uses the same functions as used by the various hash-based load
+ balancing algorithms, so it will provide exactly the same results. It is
+ mostly intended for debugging, but can be used as a stick-table entry to
+ collect rough statistics. It must not be used for security purposes as a
+ 32-bit hash is trivial to break. See also "crc32", "djb2", "wt6", "crc32c",
+ and the "hash-type" directive.
+
+secure_memcmp(<var>)
+ Compares the contents of <var> with the input value. Both values are treated
+ as a binary string. Returns a boolean indicating whether both binary strings
+ match.
+
+ If both binary strings have the same length then the comparison will be
+ performed in constant time.
+
+ Please note that this converter is only available when HAProxy has been
+ compiled with USE_OPENSSL.
+
+ Example :
+
+ http-request set-var(txn.token) hdr(token)
+ # Check whether the token sent by the client matches the secret token
+ # value, without leaking the contents using a timing attack.
+ acl token_given str(my_secret_token),secure_memcmp(txn.token)
+
+set-var(<var>[,<cond>...])
+ Sets a variable with the input content and returns the content on the output
+ as-is if all of the specified conditions are true (see below for a list of
+ possible conditions). The variable keeps the value and the associated input
+ type. The name of the variable starts with an indication about its scope. The
+ scopes allowed are:
+ "proc" : the variable is shared with the whole process
+ "sess" : the variable is shared with the whole session
+ "txn" : the variable is shared with the transaction (request and
+ response),
+ "req" : the variable is shared only during request processing,
+ "res" : the variable is shared only during response processing.
+ This prefix is followed by a name. The separator is a '.'. The name may only
+ contain characters 'a-z', 'A-Z', '0-9', '.' and '_'.
+
+ You can pass at most four conditions to the converter among the following
+ possible conditions :
+ - "ifexists"/"ifnotexists":
+ Checks if the variable already existed before the current set-var call.
+ A variable is usually created through a successful set-var call.
+ Note that variables of scope "proc" are created during configuration
+ parsing so the "ifexists" condition will always be true for them.
+ - "ifempty"/"ifnotempty":
+ Checks if the input is empty or not.
+ Scalar types are never empty so the ifempty condition will be false for
+ them regardless of the input's contents (integers, booleans, IPs ...).
+ - "ifset"/"ifnotset":
+ Checks if the variable was previously set or not, or if unset-var was
+ called on the variable.
+ A variable that does not exist yet is considered as not set. A "proc"
+ variable can exist while not being set since they are created during
+ configuration parsing.
+ - "ifgt"/"iflt":
+ Checks if the content of the variable is "greater than" or "less than"
+ the input. This check can only be performed if both the input and
+ the variable are of type integer. Otherwise, the check is considered as
+ true by default.
+
+sha1
+ Converts a binary input sample to a SHA-1 digest. The result is a binary
+ sample with length of 20 bytes.
+
+sha2([<bits>])
+ Converts a binary input sample to a digest in the SHA-2 family. The result
+ is a binary sample with length of <bits>/8 bytes.
+
+ Valid values for <bits> are 224, 256, 384, 512, each corresponding to
+ SHA-<bits>. The default value is 256.
+
+ Please note that this converter is only available when HAProxy has been
+ compiled with USE_OPENSSL.
+
+srv_queue
+ Takes an input value of type string, either a server name or <backend>/<server>
+ format and returns the number of queued streams on that server. Can be used
+ in places where we want to look up queued streams from a dynamic name, like a
+ cookie value (e.g. req.cook(SRVID),srv_queue) and then make a decision to break
+ persistence or direct a request elsewhere.
+
+strcmp(<var>)
+ Compares the contents of <var> with the input value of type string. Returns
+ the result as a signed integer compatible with strcmp(3): 0 if both strings
+ are identical. A value less than 0 if the left string is lexicographically
+ smaller than the right string or if the left string is shorter. A value greater
+ than 0 otherwise (right string greater than left string or the right string is
+ shorter).
+
+ See also the secure_memcmp converter if you need to compare two binary
+ strings in constant time.
+
+ Example :
+
+ http-request set-var(txn.host) hdr(host)
+ # Check whether the client is attempting domain fronting.
+ acl ssl_sni_http_host_match ssl_fc_sni,strcmp(txn.host) eq 0
+
+
+sub(<value>)
+ Subtracts <value> from the input value of type signed integer, and returns
+ the result as an signed integer. Note: in order to subtract the input from
+ a constant, simply perform a "neg,add(value)". <value> can be a numeric value
+ or a variable name. The name of the variable starts with an indication about
+ its scope. The scopes allowed are:
+ "proc" : the variable is shared with the whole process
+ "sess" : the variable is shared with the whole session
+ "txn" : the variable is shared with the transaction (request and
+ response),
+ "req" : the variable is shared only during request processing,
+ "res" : the variable is shared only during response processing.
+ This prefix is followed by a name. The separator is a '.'. The name may only
+ contain characters 'a-z', 'A-Z', '0-9', '.' and '_'.
+
+table_bytes_in_rate(<table>)
+ Uses the string representation of the input sample to perform a look up in
+ the specified table. If the key is not found in the table, integer value zero
+ is returned. Otherwise the converter returns the average client-to-server
+ bytes rate associated with the input sample in the designated table, measured
+ in amount of bytes over the period configured in the table. See also the
+ sc_bytes_in_rate sample fetch keyword.
+
+
+table_bytes_out_rate(<table>)
+ Uses the string representation of the input sample to perform a look up in
+ the specified table. If the key is not found in the table, integer value zero
+ is returned. Otherwise the converter returns the average server-to-client
+ bytes rate associated with the input sample in the designated table, measured
+ in amount of bytes over the period configured in the table. See also the
+ sc_bytes_out_rate sample fetch keyword.
+
+table_conn_cnt(<table>)
+ Uses the string representation of the input sample to perform a look up in
+ the specified table. If the key is not found in the table, integer value zero
+ is returned. Otherwise the converter returns the cumulative number of incoming
+ connections associated with the input sample in the designated table. See
+ also the sc_conn_cnt sample fetch keyword.
+
+table_conn_cur(<table>)
+ Uses the string representation of the input sample to perform a look up in
+ the specified table. If the key is not found in the table, integer value zero
+ is returned. Otherwise the converter returns the current amount of concurrent
+ tracked connections associated with the input sample in the designated table.
+ See also the sc_conn_cur sample fetch keyword.
+
+table_conn_rate(<table>)
+ Uses the string representation of the input sample to perform a look up in
+ the specified table. If the key is not found in the table, integer value zero
+ is returned. Otherwise the converter returns the average incoming connection
+ rate associated with the input sample in the designated table. See also the
+ sc_conn_rate sample fetch keyword.
+
+table_expire(<table>[,<default_value>])
+ Uses the input sample to perform a look up in the specified table. If the key
+ is not found in the table, the converter fails except if <default_value> is
+ set: this makes the converter succeed and return <default_value>. If the key
+ is found the converter returns the key expiration delay associated with the
+ input sample in the designated table.
+ See also the table_idle sample fetch keyword.
+
+table_gpc(<idx>,<table>)
+ Uses the string representation of the input sample to perform a lookup in
+ the specified table. If the key is not found in the table, integer value zero
+ is returned. Otherwise the converter returns the current value of the
+ General Purpose Counter at the index <idx> of the array associated
+ to the input sample in the designated <table>. <idx> is an integer
+ between 0 and 99.
+ If there is no GPC stored at this index, it also returns the boolean value 0.
+ This applies only to the 'gpc' array data_type (and not to the legacy
+ 'gpc0' nor 'gpc1' data_types).
+ See also the sc_get_gpc sample fetch keyword.
+
+table_gpc0(<table>)
+ Uses the string representation of the input sample to perform a look up in
+ the specified table. If the key is not found in the table, integer value zero
+ is returned. Otherwise the converter returns the current value of the first
+ general purpose counter associated with the input sample in the designated
+ table. See also the sc_get_gpc0 sample fetch keyword.
+
+table_gpc0_rate(<table>)
+ Uses the string representation of the input sample to perform a look up in
+ the specified table. If the key is not found in the table, integer value zero
+ is returned. Otherwise the converter returns the frequency which the gpc0
+ counter was incremented over the configured period in the table, associated
+ with the input sample in the designated table. See also the sc_get_gpc0_rate
+ sample fetch keyword.
+
+table_gpc1(<table>)
+ Uses the string representation of the input sample to perform a look up in
+ the specified table. If the key is not found in the table, integer value zero
+ is returned. Otherwise the converter returns the current value of the second
+ general purpose counter associated with the input sample in the designated
+ table. See also the sc_get_gpc1 sample fetch keyword.
+
+table_gpc1_rate(<table>)
+ Uses the string representation of the input sample to perform a look up in
+ the specified table. If the key is not found in the table, integer value zero
+ is returned. Otherwise the converter returns the frequency which the gpc1
+ counter was incremented over the configured period in the table, associated
+ with the input sample in the designated table. See also the sc_get_gpc1_rate
+ sample fetch keyword.
+
+table_gpc_rate(<idx>,<table>)
+ Uses the string representation of the input sample to perform a lookup in
+ the specified table. If the key is not found in the table, integer value zero
+ is returned. Otherwise the converter returns the frequency which the Global
+ Purpose Counter at index <idx> of the array (associated to the input sample
+ in the designated stick-table <table>) was incremented over the
+ configured period. <idx> is an integer between 0 and 99.
+ If there is no gpc_rate stored at this index, it also returns the boolean
+ value 0.
+ This applies only to the 'gpc_rate' array data_type (and not to the
+ legacy 'gpc0_rate' nor 'gpc1_rate' data_types).
+ See also the sc_gpc_rate sample fetch keyword.
+
+table_gpt(<idx>,<table>)
+ Uses the string representation of the input sample to perform a lookup in
+ the specified table. If the key is not found in the table, boolean value zero
+ is returned. Otherwise the converter returns the current value of the general
+ purpose tag at the index <idx> of the array associated to the input sample
+ in the designated <table>. <idx> is an integer between 0 and 99.
+ If there is no GPT stored at this index, it also returns the boolean value 0.
+ This applies only to the 'gpt' array data_type (and not on the legacy 'gpt0'
+ data-type).
+ See also the sc_get_gpt sample fetch keyword.
+
+table_gpt0(<table>)
+ Uses the string representation of the input sample to perform a look up in
+ the specified table. If the key is not found in the table, boolean value zero
+ is returned. Otherwise the converter returns the current value of the first
+ general purpose tag associated with the input sample in the designated table.
+ See also the sc_get_gpt0 sample fetch keyword.
+
+table_http_err_cnt(<table>)
+ Uses the string representation of the input sample to perform a look up in
+ the specified table. If the key is not found in the table, integer value zero
+ is returned. Otherwise the converter returns the cumulative number of HTTP
+ errors associated with the input sample in the designated table. See also the
+ sc_http_err_cnt sample fetch keyword.
+
+table_http_err_rate(<table>)
+ Uses the string representation of the input sample to perform a look up in
+ the specified table. If the key is not found in the table, integer value zero
+ is returned. Otherwise the average rate of HTTP errors associated with the
+ input sample in the designated table, measured in amount of errors over the
+ period configured in the table. See also the sc_http_err_rate sample fetch
+ keyword.
+
+table_http_fail_cnt(<table>)
+ Uses the string representation of the input sample to perform a look up in
+ the specified table. If the key is not found in the table, integer value zero
+ is returned. Otherwise the converter returns the cumulative number of HTTP
+ failures associated with the input sample in the designated table. See also
+ the sc_http_fail_cnt sample fetch keyword.
+
+table_http_fail_rate(<table>)
+ Uses the string representation of the input sample to perform a look up in
+ the specified table. If the key is not found in the table, integer value zero
+ is returned. Otherwise the average rate of HTTP failures associated with the
+ input sample in the designated table, measured in amount of failures over the
+ period configured in the table. See also the sc_http_fail_rate sample fetch
+ keyword.
+
+table_http_req_cnt(<table>)
+ Uses the string representation of the input sample to perform a look up in
+ the specified table. If the key is not found in the table, integer value zero
+ is returned. Otherwise the converter returns the cumulative number of HTTP
+ requests associated with the input sample in the designated table. See also
+ the sc_http_req_cnt sample fetch keyword.
+
+table_http_req_rate(<table>)
+ Uses the string representation of the input sample to perform a look up in
+ the specified table. If the key is not found in the table, integer value zero
+ is returned. Otherwise the average rate of HTTP requests associated with the
+ input sample in the designated table, measured in amount of requests over the
+ period configured in the table. See also the sc_http_req_rate sample fetch
+ keyword.
+
+table_idle(<table>[,<default_value>])
+ Uses the input sample to perform a look up in the specified table. If the key
+ is not found in the table, the converter fails except if <default_value> is
+ set: this makes the converter succeed and return <default_value>. If the key
+ is found the converter returns the time the key entry associated with the
+ input sample in the designated table remained idle since the last time it was
+ updated.
+ See also the table_expire sample fetch keyword.
+
+table_kbytes_in(<table>)
+ Uses the string representation of the input sample to perform a look up in
+ the specified table. If the key is not found in the table, integer value zero
+ is returned. Otherwise the converter returns the cumulative number of client-
+ to-server data associated with the input sample in the designated table,
+ measured in kilobytes. The test is currently performed on 32-bit integers,
+ which limits values to 4 terabytes. See also the sc_kbytes_in sample fetch
+ keyword.
+
+table_kbytes_out(<table>)
+ Uses the string representation of the input sample to perform a look up in
+ the specified table. If the key is not found in the table, integer value zero
+ is returned. Otherwise the converter returns the cumulative number of server-
+ to-client data associated with the input sample in the designated table,
+ measured in kilobytes. The test is currently performed on 32-bit integers,
+ which limits values to 4 terabytes. See also the sc_kbytes_out sample fetch
+ keyword.
+
+table_server_id(<table>)
+ Uses the string representation of the input sample to perform a look up in
+ the specified table. If the key is not found in the table, integer value zero
+ is returned. Otherwise the converter returns the server ID associated with
+ the input sample in the designated table. A server ID is associated to a
+ sample by a "stick" rule when a connection to a server succeeds. A server ID
+ zero means that no server is associated with this key.
+
+table_sess_cnt(<table>)
+ Uses the string representation of the input sample to perform a look up in
+ the specified table. If the key is not found in the table, integer value zero
+ is returned. Otherwise the converter returns the cumulative number of incoming
+ sessions associated with the input sample in the designated table. Note that
+ a session here refers to an incoming connection being accepted by the
+ "tcp-request connection" rulesets. See also the sc_sess_cnt sample fetch
+ keyword.
+
+table_sess_rate(<table>)
+ Uses the string representation of the input sample to perform a look up in
+ the specified table. If the key is not found in the table, integer value zero
+ is returned. Otherwise the converter returns the average incoming session
+ rate associated with the input sample in the designated table. Note that a
+ session here refers to an incoming connection being accepted by the
+ "tcp-request connection" rulesets. See also the sc_sess_rate sample fetch
+ keyword.
+
+table_trackers(<table>)
+ Uses the string representation of the input sample to perform a look up in
+ the specified table. If the key is not found in the table, integer value zero
+ is returned. Otherwise the converter returns the current amount of concurrent
+ connections tracking the same key as the input sample in the designated
+ table. It differs from table_conn_cur in that it does not rely on any stored
+ information but on the table's reference count (the "use" value which is
+ returned by "show table" on the CLI). This may sometimes be more suited for
+ layer7 tracking. It can be used to tell a server how many concurrent
+ connections there are from a given address for example. See also the
+ sc_trackers sample fetch keyword.
+
+ub64dec
+ This converter is the base64url variant of b64dec converter. base64url
+ encoding is the "URL and Filename Safe Alphabet" variant of base64 encoding.
+ It is also the encoding used in JWT (JSON Web Token) standard.
+
+ Example:
+ # Decoding a JWT payload:
+ http-request set-var(txn.token_payload) req.hdr(Authorization),word(2,.),ub64dec
+
+ub64enc
+ This converter is the base64url variant of base64 converter.
+
+ungrpc(<field_number>,[<field_type>])
+ This extracts the protocol buffers message field in raw mode of an input binary
+ sample representation of a gRPC message with <field_number> as field number
+ (dotted notation) if <field_type> is not present, or as an integer sample if this
+ field is present.
+ The list of the authorized types is the following one: "int32", "int64", "uint32",
+ "uint64", "sint32", "sint64", "bool", "enum" for the "varint" wire type 0
+ "fixed64", "sfixed64", "double" for the 64bit wire type 1, "fixed32", "sfixed32",
+ "float" for the wire type 5. Note that "string" is considered as a length-delimited
+ type, so it does not require any <field_type> argument to be extracted.
+ More information may be found here about the protocol buffers message field types:
+ https://developers.google.com/protocol-buffers/docs/encoding
+
+ Example:
+ // with such a protocol buffer .proto file content adapted from
+ // https://github.com/grpc/grpc/blob/master/examples/protos/route_guide.proto
+
+ message Point {
+ int32 latitude = 1;
+ int32 longitude = 2;
+ }
+
+ message PPoint {
+ Point point = 59;
+ }
+
+ message Rectangle {
+ // One corner of the rectangle.
+ PPoint lo = 48;
+ // The other corner of the rectangle.
+ PPoint hi = 49;
+ }
+
+ let's say a body request is made of a "Rectangle" object value (two PPoint
+ protocol buffers messages), the four protocol buffers fields could be
+ extracted with these "ungrpc" directives:
+
+ req.body,ungrpc(48.59.1,int32) # "latitude" of "lo" first PPoint
+ req.body,ungrpc(48.59.2,int32) # "longitude" of "lo" first PPoint
+ req.body,ungrpc(49.59.1,int32) # "latitude" of "hi" second PPoint
+ req.body,ungrpc(49.59.2,int32) # "longitude" of "hi" second PPoint
+
+ We could also extract the intermediary 48.59 field as a binary sample as follows:
+
+ req.body,ungrpc(48.59)
+
+ As a gRPC message is always made of a gRPC header followed by protocol buffers
+ messages, in the previous example the "latitude" of "lo" first PPoint
+ could be extracted with these equivalent directives:
+
+ req.body,ungrpc(48.59),protobuf(1,int32)
+ req.body,ungrpc(48),protobuf(59.1,int32)
+ req.body,ungrpc(48),protobuf(59),protobuf(1,int32)
+
+ Note that the first convert must be "ungrpc", the remaining ones must be
+ "protobuf" and only the last one may have or not a second argument to
+ interpret the previous binary sample.
+
+
+unset-var(<var>)
+ Unsets a variable if the input content is defined. The name of the variable
+ starts with an indication about its scope. The scopes allowed are:
+ "proc" : the variable is shared with the whole process
+ "sess" : the variable is shared with the whole session
+ "txn" : the variable is shared with the transaction (request and
+ response),
+ "req" : the variable is shared only during request processing,
+ "res" : the variable is shared only during response processing.
+ This prefix is followed by a name. The separator is a '.'. The name may only
+ contain characters 'a-z', 'A-Z', '0-9', '.' and '_'.
+
+upper
+ Convert a string sample to upper case. This can only be placed after a string
+ sample fetch function or after a transformation keyword returning a string
+ type. The result is of type string.
+
+url_dec([<in_form>])
+ Takes an url-encoded string provided as input and returns the decoded version
+ as output. The input and the output are of type string. If the <in_form>
+ argument is set to a non-zero integer value, the input string is assumed to
+ be part of a form or query string and the '+' character will be turned into a
+ space (' '). Otherwise this will only happen after a question mark indicating
+ a query string ('?').
+
+url_enc([<enc_type>])
+ Takes a string provided as input and returns the encoded version as output.
+ The input and the output are of type string. By default the type of encoding
+ is meant for `query` type. There is no other type supported for now but the
+ optional argument is here for future changes.
+
+us_ltime(<format>[,<offset>])
+ This works like "ltime" but takes an input in microseconds. It also supports
+ the %N conversion specifier inspired by date(1).
+ Converts an integer supposed to contain a date since epoch to a string
+ representing this date in local time using a format defined by the <format>
+ string using strftime(3). The purpose is to allow any date format to be used
+ in logs. An optional <offset> in microseconds may be applied to the input
+ date (positive or negative). See the strftime() man page for the format
+ supported by your operating system.
+
+ The %N conversion specifier allows you to output the nanoseconds part of the
+ date, precision is limited since the input is microseconds.
+ (000000000..999999000). %N can take a width argument between % and N. It is
+ useful to display milliseconds (%3N) or microseconds (%6N). The default and
+ maximum width is 9 (%N = %9N).
+
+ See also the "utime" converter for UTC as well as "ltime" and "ms_ltime"
+ converters.
+
+ Example :
+
+ # Emit 3 colons, the local time, the timezone and another with ip:port
+ # e.g. 2023/07/24/09:53:02.196234 +0000 127.0.0.1:41530
+ log-format %[accept_date(us),us_ltime("%Y/%m/%d/%H:%M:%S.%6N %z")]\ %ci:%cp
+
+us_utime(<format>[,<offset>])
+ This works like "utime" but takes an input in microseconds. It also supports
+ the %N conversion specifier inspired by date(1).
+ Converts an integer supposed to contain a date since epoch to a string
+ representing this date in UTC time using a format defined by the <format>
+ string using strftime(3). The purpose is to allow any date format to be used
+ in logs. An optional <offset> in microseconds may be applied to the input
+ date (positive or negative). See the strftime() man page for the format
+ supported by your operating system.
+
+ The %N conversion specifier allows you to output the nanoseconds part of the
+ date, precision is limited since the input is microseconds.
+ (000000000..999999000). %N can take a width argument between % and N. It is
+ useful to display milliseconds (%3N) or microseconds (%6N). The default and
+ maximum width is 9 (%N = %9N).
+
+ See also the "ltime" converter for local as well as "utime" and "ms_utime"
+ converters.
+
+ Example :
+
+ # Emit 3 colons, the UTC time, the timezone and another with ip:port
+ # e.g. 2023/07/24/09:53:02.196234 +0000 127.0.0.1:41530
+ log-format %[accept_date(us),us_utime("%Y/%m/%d/%H:%M:%S.%6N %z")]\ %ci:%cp
+
+utime(<format>[,<offset>])
+ Converts an integer supposed to contain a date since epoch to a string
+ representing this date in UTC time using a format defined by the <format>
+ string using strftime(3). The purpose is to allow any date format to be used
+ in logs. An optional <offset> in seconds may be applied to the input date
+ (positive or negative). See the strftime() man page for the format supported
+ by your operating system. See also the "ltime" converter as well as "ms_utime"
+ and "us_utime".
+
+ Example :
+
+ # Emit two colons, one with the UTC time and another with ip:port
+ # e.g. 20140710162350 127.0.0.1:57325
+ log-format %[date,utime(%Y%m%d%H%M%S)]\ %ci:%cp
+
+word(<index>,<delimiters>[,<count>])
+ Extracts the nth word counting from the beginning (positive index) or from
+ the end (negative index) considering given delimiters from an input string.
+ Indexes start at 1 or -1 and delimiters are a string formatted list of chars.
+ Empty words are skipped. This means that delimiters at the start or end of
+ the input string are ignored and consecutive delimiters within the input
+ string are considered to be a single delimiter.
+ Optionally you can specify <count> of words to extract (default: 1).
+ Value of 0 indicates extraction of all remaining words.
+
+ Example :
+ str(f1_f2_f3__f5),word(4,_) # f5
+ str(f1_f2_f3__f5),word(5,_) # <not found>
+ str(f1_f2_f3__f5),word(2,_,0) # f2_f3__f5
+ str(f1_f2_f3__f5),word(3,_,2) # f3__f5
+ str(f1_f2_f3__f5),word(-2,_,3) # f1_f2_f3
+ str(f1_f2_f3__f5),word(-3,_,0) # f1_f2
+ str(/f1/f2/f3/f4),word(1,/) # f1
+ str(/f1////f2/f3/f4),word(1,/) # f2
+
+wt6([<avalanche>])
+ Hashes a binary input sample into an unsigned 32-bit quantity using the WT6
+ hash function. Optionally, it is possible to apply a full avalanche hash
+ function to the output if the optional <avalanche> argument equals 1. This
+ converter uses the same functions as used by the various hash-based load
+ balancing algorithms, so it will provide exactly the same results. It is
+ mostly intended for debugging, but can be used as a stick-table entry to
+ collect rough statistics. It must not be used for security purposes as a
+ 32-bit hash is trivial to break. See also "crc32", "djb2", "sdbm", "crc32c",
+ and the "hash-type" directive.
+
+
+x509_v_err_str
+ Convert a numerical value to its corresponding X509_V_ERR constant name. It
+ is useful in ACL in order to have a configuration which works with multiple
+ version of OpenSSL since some codes might change when changing version.
+
+ When the corresponding constant name was not found, outputs the numerical
+ value as a string.
+
+ The list of constant provided by OpenSSL can be found at
+ https://www.openssl.org/docs/manmaster/man3/X509_STORE_CTX_get_error.html#ERROR-CODES
+ Be careful to read the page for the right version of OpenSSL.
+
+ Example:
+
+ bind :443 ssl crt common.pem ca-file ca-auth.crt verify optional crt-ignore-err X509_V_ERR_CERT_REVOKED,X509_V_ERR_CERT_HAS_EXPIRED
+
+ acl cert_expired ssl_c_verify,x509_v_err_str -m str X509_V_ERR_CERT_HAS_EXPIRED
+ acl cert_revoked ssl_c_verify,x509_v_err_str -m str X509_V_ERR_CERT_REVOKED
+ acl cert_ok ssl_c_verify,x509_v_err_str -m str X509_V_OK
+
+ http-response add-header X-SSL Ok if cert_ok
+ http-response add-header X-SSL Expired if cert_expired
+ http-response add-header X-SSL Revoked if cert_revoked
+
+ http-response add-header X-SSL-verify %[ssl_c_verify,x509_v_err_str]
+
+xor(<value>)
+ Performs a bitwise "XOR" (exclusive OR) between <value> and the input value
+ of type signed integer, and returns the result as an signed integer.
+ <value> can be a numeric value or a variable name. The name of the variable
+ starts with an indication about its scope. The scopes allowed are:
+ "proc" : the variable is shared with the whole process
+ "sess" : the variable is shared with the whole session
+ "txn" : the variable is shared with the transaction (request and
+ response),
+ "req" : the variable is shared only during request processing,
+ "res" : the variable is shared only during response processing.
+ This prefix is followed by a name. The separator is a '.'. The name may only
+ contain characters 'a-z', 'A-Z', '0-9', '.' and '_'.
+
+xxh3([<seed>])
+ Hashes a binary input sample into a signed 64-bit quantity using the XXH3
+ 64-bit variant of the XXhash hash function. This hash supports a seed which
+ defaults to zero but a different value maybe passed as the <seed> argument.
+ This hash is known to be very good and very fast so it can be used to hash
+ URLs and/or URL parameters for use as stick-table keys to collect statistics
+ with a low collision rate, though care must be taken as the algorithm is not
+ considered as cryptographically secure.
+
+xxh32([<seed>])
+ Hashes a binary input sample into an unsigned 32-bit quantity using the 32-bit
+ variant of the XXHash hash function. This hash supports a seed which defaults
+ to zero but a different value maybe passed as the <seed> argument. This hash
+ is known to be very good and very fast so it can be used to hash URLs and/or
+ URL parameters for use as stick-table keys to collect statistics with a low
+ collision rate, though care must be taken as the algorithm is not considered
+ as cryptographically secure.
+
+xxh64([<seed>])
+ Hashes a binary input sample into a signed 64-bit quantity using the 64-bit
+ variant of the XXHash hash function. This hash supports a seed which defaults
+ to zero but a different value maybe passed as the <seed> argument. This hash
+ is known to be very good and very fast so it can be used to hash URLs and/or
+ URL parameters for use as stick-table keys to collect statistics with a low
+ collision rate, though care must be taken as the algorithm is not considered
+ as cryptographically secure.
+
+
+7.3.2. Fetching samples from internal states
+--------------------------------------------
+
+A first set of sample fetch methods applies to internal information which does
+not even relate to any client information. These ones are sometimes used with
+"monitor fail" directives to report an internal status to external watchers.
+The sample fetch methods described in this section are usable anywhere.
+
+Summary of sample fetch methods in this section and their respective types:
+
+ keyword output type
+-------------------------------------------------+-------------
+act_conn integer
+acl([!]<name>[,...]) boolean
+always_false boolean
+always_true boolean
+avg_queue([<backend>]) integer
+be_conn([<backend>]) integer
+be_conn_free([<backend>]) integer
+be_sess_rate([<backend>]) integer
+bin(<hex>) bin
+bool(<bool>) bool
+connslots([<backend>]) integer
+cpu_calls integer
+cpu_ns_avg integer
+cpu_ns_tot integer
+date([<offset>],[<unit>]) integer
+date_us integer
+env(<name>) string
+fe_conn([<frontend>]) integer
+fe_req_rate([<frontend>]) integer
+fe_sess_rate([<frontend>]) integer
+hostname string
+int(<integer>) signed
+ipv4(<ipv4>) ipv4
+ipv6(<ipv6>) ipv6
+last_rule_file string
+last_rule_line integer
+lat_ns_avg integer
+lat_ns_tot integer
+meth(<method>) method
+nbsrv([<backend>]) integer
+prio_class integer
+prio_offset integer
+pid integer
+proc integer
+queue([<backend>]) integer
+quic_enabled boolean
+rand([<range>]) integer
+srv_conn([<backend>/]<server>) integer
+srv_conn_free([<backend>/]<server>) integer
+srv_is_up([<backend>/]<server>) boolean
+srv_queue([<backend>/]<server>) integer
+srv_sess_rate([<backend>/]<server>) integer
+srv_iweight([<backend>/]<server>) integer
+srv_uweight([<backend>/]<server>) integer
+srv_weight([<backend>/]<server>) integer
+stopping boolean
+str(<string>) string
+table_avl([<table>]) integer
+table_cnt([<table>]) integer
+thread integer
+txn.id32 integer
+txn.conn_retries integer
+txn.sess_term_state string
+uuid([<version>]) string
+var(<var-name>[,<default>]) undefined
+-------------------------------------------------+-------------
+
+Detailed list:
+
+act_conn : integer
+ Returns the total number of active concurrent connections on the process.
+
+acl([!]<name>[,...]) : boolean
+ Returns true if the evaluation of all the named ACL(s) is true, otherwise
+ returns false. Up to 12 ACLs may be provided, each delimited by comma. Each
+ named ACL may be prefixed with a "!" to invert the result. If any evaluation
+ produces an error then the sample also returns an error.
+ Note that HAProxy does not perform any validation checks on the referenced
+ ACLs, such as whether an ACL which uses a http request sample is used in
+ response context. This behavior may be changed in the future.
+
+always_false : boolean
+ Always returns the boolean "false" value. It may be used with ACLs as a
+ temporary replacement for another one when adjusting configurations.
+
+always_true : boolean
+ Always returns the boolean "true" value. It may be used with ACLs as a
+ temporary replacement for another one when adjusting configurations.
+
+avg_queue([<backend>]) : integer
+ Returns the total number of queued connections of the designated backend
+ divided by the number of active servers. The current backend is used if no
+ backend is specified. This is very similar to "queue" except that the size of
+ the farm is considered, in order to give a more accurate measurement of the
+ time it may take for a new connection to be processed. The main usage is with
+ ACL to return a sorry page to new users when it becomes certain they will get
+ a degraded service, or to pass to the backend servers in a header so that
+ they decide to work in degraded mode or to disable some functions to speed up
+ the processing a bit. Note that in the event there would not be any active
+ server anymore, twice the number of queued connections would be considered as
+ the measured value. This is a fair estimate, as we expect one server to get
+ back soon anyway, but we still prefer to send new traffic to another backend
+ if in better shape. See also the "queue", "be_conn", and "be_sess_rate"
+ sample fetches.
+
+be_conn([<backend>]) : integer
+ Applies to the number of currently established connections on the backend,
+ possibly including the connection being evaluated. If no backend name is
+ specified, the current one is used. But it is also possible to check another
+ backend. It can be used to use a specific farm when the nominal one is full.
+ See also the "fe_conn", "queue", "be_conn_free", and "be_sess_rate" criteria.
+
+be_conn_free([<backend>]) : integer
+ Returns an integer value corresponding to the number of available connections
+ across available servers in the backend. Queue slots are not included. Backup
+ servers are also not included, unless all other servers are down. If no
+ backend name is specified, the current one is used. But it is also possible
+ to check another backend. It can be used to use a specific farm when the
+ nominal one is full. See also the "be_conn", "connslots", and "srv_conn_free"
+ criteria.
+
+ OTHER CAVEATS AND NOTES: if any of the server maxconn, or maxqueue is 0
+ (meaning unlimited), then this fetch clearly does not make sense, in which
+ case the value returned will be -1.
+
+be_sess_rate([<backend>]) : integer
+ Returns an integer value corresponding to the sessions creation rate on the
+ backend, in number of new sessions per second. This is used with ACLs to
+ switch to an alternate backend when an expensive or fragile one reaches too
+ high a session rate, or to limit abuse of service (e.g. prevent sucking of an
+ online dictionary). It can also be useful to add this element to logs using a
+ log-format directive.
+
+ Example :
+ # Redirect to an error page if the dictionary is requested too often
+ backend dynamic
+ mode http
+ acl being_scanned be_sess_rate gt 100
+ redirect location /denied.html if being_scanned
+
+bin(<hex>) : bin
+ Returns a binary chain. The input is the hexadecimal representation
+ of the string.
+
+bool(<bool>) : bool
+ Returns a boolean value. <bool> can be 'true', 'false', '1' or '0'.
+ 'false' and '0' are the same. 'true' and '1' are the same.
+
+connslots([<backend>]) : integer
+ Returns an integer value corresponding to the number of connection slots
+ still available in the backend, by totaling the maximum amount of
+ connections on all servers and the maximum queue size. This is probably only
+ used with ACLs.
+
+ The basic idea here is to be able to measure the number of connection "slots"
+ still available (connection + queue), so that anything beyond that (intended
+ usage; see "use_backend" keyword) can be redirected to a different backend.
+
+ 'connslots' = number of available server connection slots, + number of
+ available server queue slots.
+
+ Note that while "fe_conn" may be used, "connslots" comes in especially
+ useful when you have a case of traffic going to one single ip, splitting into
+ multiple backends (perhaps using ACLs to do name-based load balancing) and
+ you want to be able to differentiate between different backends, and their
+ available "connslots". Also, whereas "nbsrv" only measures servers that are
+ actually *down*, this fetch is more fine-grained and looks into the number of
+ available connection slots as well. See also "queue" and "avg_queue".
+
+ OTHER CAVEATS AND NOTES: at this point in time, the code does not take care
+ of dynamic connections. Also, if any of the server maxconn, or maxqueue is 0,
+ then this fetch clearly does not make sense, in which case the value returned
+ will be -1.
+
+cpu_calls : integer
+ Returns the number of calls to the task processing the stream or current
+ request since it was allocated. This number is reset for each new request on
+ the same connections in case of HTTP keep-alive. This value should usually be
+ low and stable (around 2 calls for a typically simple request) but may become
+ high if some processing (compression, caching or analysis) is performed. This
+ is purely for performance monitoring purposes.
+
+cpu_ns_avg : integer
+ Returns the average number of nanoseconds spent in each call to the task
+ processing the stream or current request. This number is reset for each new
+ request on the same connections in case of HTTP keep-alive. This value
+ indicates the overall cost of processing the request or the connection for
+ each call. There is no good nor bad value but the time spent in a call
+ automatically causes latency for other processing (see lat_ns_avg below),
+ and may affect other connection's apparent response time. Certain operations
+ like compression, complex regex matching or heavy Lua operations may directly
+ affect this value, and having it in the logs will make it easier to spot the
+ faulty processing that needs to be fixed to recover decent performance.
+ Note: this value is exactly cpu_ns_tot divided by cpu_calls.
+
+cpu_ns_tot : integer
+ Returns the total number of nanoseconds spent in each call to the task
+ processing the stream or current request. This number is reset for each new
+ request on the same connections in case of HTTP keep-alive. This value
+ indicates the overall cost of processing the request or the connection for
+ each call. There is no good nor bad value but the time spent in a call
+ automatically causes latency for other processing (see lat_ns_avg below),
+ induces CPU costs on the machine, and may affect other connection's apparent
+ response time. Certain operations like compression, complex regex matching or
+ heavy Lua operations may directly affect this value, and having it in the
+ logs will make it easier to spot the faulty processing that needs to be fixed
+ to recover decent performance. The value may be artificially high due to a
+ high cpu_calls count, for example when processing many HTTP chunks, and for
+ this reason it is often preferred to log cpu_ns_avg instead.
+
+date([<offset>],[<unit>]) : integer
+ Returns the current date as the epoch (number of seconds since 01/01/1970).
+
+ If an offset value is specified, then it is added to the current date before
+ returning the value. This is particularly useful to compute relative dates,
+ as both positive and negative offsets are allowed.
+ It is useful combined with the http_date converter.
+
+ <unit> is facultative, and can be set to "s" for seconds (default behavior),
+ "ms" for milliseconds or "us" for microseconds.
+ If unit is set, return value is an integer reflecting either seconds,
+ milliseconds or microseconds since epoch, plus offset.
+ It is useful when a time resolution of less than a second is needed.
+
+ Example :
+
+ # set an expires header to now+1 hour in every response
+ http-response set-header Expires %[date(3600),http_date]
+
+ # set an expires header to now+1 hour in every response, with
+ # millisecond granularity
+ http-response set-header Expires %[date(3600000,ms),http_date(0,ms)]
+
+date_us : integer
+ Return the microseconds part of the date (the "second" part is returned by
+ date sample). This sample is coherent with the date sample as it is comes
+ from the same timeval structure.
+
+env(<name>) : string
+ Returns a string containing the value of environment variable <name>. As a
+ reminder, environment variables are per-process and are sampled when the
+ process starts. This can be useful to pass some information to a next hop
+ server, or with ACLs to take specific action when the process is started a
+ certain way.
+
+ Examples :
+ # Pass the Via header to next hop with the local hostname in it
+ http-request add-header Via 1.1\ %[env(HOSTNAME)]
+
+ # reject cookie-less requests when the STOP environment variable is set
+ http-request deny if !{ req.cook(SESSIONID) -m found } { env(STOP) -m found }
+
+fe_conn([<frontend>]) : integer
+ Returns the number of currently established connections on the frontend,
+ possibly including the connection being evaluated. If no frontend name is
+ specified, the current one is used. But it is also possible to check another
+ frontend. It can be used to return a sorry page before hard-blocking, or to
+ use a specific backend to drain new requests when the farm is considered
+ full. This is mostly used with ACLs but can also be used to pass some
+ statistics to servers in HTTP headers. See also the "dst_conn", "be_conn",
+ "fe_sess_rate" fetches.
+
+fe_req_rate([<frontend>]) : integer
+ Returns an integer value corresponding to the number of HTTP requests per
+ second sent to a frontend. This number can differ from "fe_sess_rate" in
+ situations where client-side keep-alive is enabled.
+
+fe_sess_rate([<frontend>]) : integer
+ Returns an integer value corresponding to the sessions creation rate on the
+ frontend, in number of new sessions per second. This is used with ACLs to
+ limit the incoming session rate to an acceptable range in order to prevent
+ abuse of service at the earliest moment, for example when combined with other
+ layer 4 ACLs in order to force the clients to wait a bit for the rate to go
+ down below the limit. It can also be useful to add this element to logs using
+ a log-format directive. See also the "rate-limit sessions" directive for use
+ in frontends.
+
+ Example :
+ # This frontend limits incoming mails to 10/s with a max of 100
+ # concurrent connections. We accept any connection below 10/s, and
+ # force excess clients to wait for 100 ms. Since clients are limited to
+ # 100 max, there cannot be more than 10 incoming mails per second.
+ frontend mail
+ bind :25
+ mode tcp
+ maxconn 100
+ acl too_fast fe_sess_rate ge 10
+ tcp-request inspect-delay 100ms
+ tcp-request content accept if ! too_fast
+ tcp-request content accept if WAIT_END
+
+hostname : string
+ Returns the system hostname.
+
+int(<integer>) : signed integer
+ Returns a signed integer.
+
+ipv4(<ipv4>) : ipv4
+ Returns an ipv4.
+
+ipv6(<ipv6>) : ipv6
+ Returns an ipv6.
+
+last_rule_file : string
+ This returns the name of the configuration file containing the last final
+ rule that was matched during stream analysis. A final rule is one that
+ terminates the evaluation of the rule set (like an "accept", "deny" or
+ "redirect"). This works for TCP request and response rules acting on the
+ "content" rulesets, and on HTTP rules from "http-request", "http-response"
+ and "http-after-response" rule sets. The legacy "redirect" rulesets are not
+ supported (such information is not stored there), and neither "tcp-request
+ connection" nor "tcp-request session" rulesets are supported because the
+ information is stored at the stream level and streams do not exist during
+ these rules. The main purpose of this function is to be able to report in
+ logs where was the rule that gave the final verdict, in order to help
+ figure why a request was denied for example. See also "last_rule_line".
+
+last_rule_line : integer
+ This returns the line number in the configuration file where is located the
+ last final rule that was matched during stream analysis. A final rule is one
+ that terminates the evaluation of the rule set (like an "accept", "deny" or
+ "redirect"). This works for TCP request and response rules acting on the
+ "content" rulesets, and on HTTP rules from "http-request", "http-response"
+ and "http-after-response" rule sets. The legacy "redirect" rulesets are not
+ supported (such information is not stored there), and neither "tcp-request
+ connection" nor "tcp-request session" rulesets are supported because the
+ information is stored at the stream level and streams do not exist during
+ these rules. The main purpose of this function is to be able to report in
+ logs where was the rule that gave the final verdict, in order to help
+ figure why a request was denied for example. See also "last_rule_file".
+
+lat_ns_avg : integer
+ Returns the average number of nanoseconds spent between the moment the task
+ handling the stream is woken up and the moment it is effectively called. This
+ number is reset for each new request on the same connections in case of HTTP
+ keep-alive. This value indicates the overall latency inflicted to the current
+ request by all other requests being processed in parallel, and is a direct
+ indicator of perceived performance due to noisy neighbours. In order to keep
+ the value low, it is possible to reduce the scheduler's run queue depth using
+ "tune.runqueue-depth", to reduce the number of concurrent events processed at
+ once using "tune.maxpollevents", to decrease the stream's nice value using
+ the "nice" option on the "bind" lines or in the frontend, to enable low
+ latency scheduling using "tune.sched.low-latency", or to look for other heavy
+ requests in logs (those exhibiting large values of "cpu_ns_avg"), whose
+ processing needs to be adjusted or fixed. Compression of large buffers could
+ be a culprit, like heavy regex or long lists of regex. Note: this value is
+ exactly lat_ns_tot divided by cpu_calls.
+
+lat_ns_tot : integer
+ Returns the total number of nanoseconds spent between the moment the task
+ handling the stream is woken up and the moment it is effectively called. This
+ number is reset for each new request on the same connections in case of HTTP
+ keep-alive. This value indicates the overall latency inflicted to the current
+ request by all other requests being processed in parallel, and is a direct
+ indicator of perceived performance due to noisy neighbours. In order to keep
+ the value low, it is possible to reduce the scheduler's run queue depth using
+ "tune.runqueue-depth", to reduce the number of concurrent events processed at
+ once using "tune.maxpollevents", to decrease the stream's nice value using
+ the "nice" option on the "bind" lines or in the frontend, to enable low
+ latency scheduling using "tune.sched.low-latency", or to look for other heavy
+ requests in logs (those exhibiting large values of "cpu_ns_avg"), whose
+ processing needs to be adjusted or fixed. Compression of large buffers could
+ be a culprit, like heavy regex or long lists of regex. Note: while it
+ may intuitively seem that the total latency adds to a transfer time, it is
+ almost never true because while a task waits for the CPU, network buffers
+ continue to fill up and the next call will process more at once. The value
+ may be artificially high due to a high cpu_calls count, for example when
+ processing many HTTP chunks, and for this reason it is often preferred to log
+ lat_ns_avg instead, which is a more relevant performance indicator.
+
+meth(<method>) : method
+ Returns a method.
+
+nbsrv([<backend>]) : integer
+ Returns an integer value corresponding to the number of usable servers of
+ either the current backend or the named backend. This is mostly used with
+ ACLs but can also be useful when added to logs. This is normally used to
+ switch to an alternate backend when the number of servers is too low to
+ to handle some load. It is useful to report a failure when combined with
+ "monitor fail".
+
+prio_class : integer
+ Returns the priority class of the current stream for http mode or connection
+ for tcp mode. The value will be that set by the last call to "http-request
+ set-priority-class" or "tcp-request content set-priority-class".
+
+prio_offset : integer
+ Returns the priority offset of the current stream for http mode or
+ connection for tcp mode. The value will be that set by the last call to
+ "http-request set-priority-offset" or "tcp-request content
+ set-priority-offset".
+
+pid : integer
+ Return the PID of the current process. In most cases this is the PID of the
+ worker process.
+
+proc : integer
+ Always returns value 1 (historically it would return the calling process
+ number).
+
+queue([<backend>]) : integer
+ Returns the total number of queued connections of the designated backend,
+ including all the connections in server queues. If no backend name is
+ specified, the current one is used, but it is also possible to check another
+ one. This is useful with ACLs or to pass statistics to backend servers. This
+ can be used to take actions when queuing goes above a known level, generally
+ indicating a surge of traffic or a massive slowdown on the servers. One
+ possible action could be to reject new users but still accept old ones. See
+ also the "avg_queue", "be_conn", and "be_sess_rate" fetches.
+
+quic_enabled : boolean
+ Return true when the support for QUIC transport protocol was compiled and
+ if this procotol was not disabled by "no-quic" global option. See also "no-quic"
+ global option.
+
+rand([<range>]) : integer
+ Returns a random integer value within a range of <range> possible values,
+ starting at zero. If the range is not specified, it defaults to 2^32, which
+ gives numbers between 0 and 4294967295. It can be useful to pass some values
+ needed to take some routing decisions for example, or just for debugging
+ purposes. This random must not be used for security purposes.
+
+srv_conn([<backend>/]<server>) : integer
+ Returns an integer value corresponding to the number of currently established
+ connections on the designated server, possibly including the connection being
+ evaluated. If <backend> is omitted, then the server is looked up in the
+ current backend. It can be used to use a specific farm when one server is
+ full, or to inform the server about our view of the number of active
+ connections with it. See also the "fe_conn", "be_conn", "queue", and
+ "srv_conn_free" fetch methods.
+
+srv_conn_free([<backend>/]<server>) : integer
+ Returns an integer value corresponding to the number of available connections
+ on the designated server, possibly including the connection being evaluated.
+ The value does not include queue slots. If <backend> is omitted, then the
+ server is looked up in the current backend. It can be used to use a specific
+ farm when one server is full, or to inform the server about our view of the
+ number of active connections with it. See also the "be_conn_free" and
+ "srv_conn" fetch methods.
+
+ OTHER CAVEATS AND NOTES: If the server maxconn is 0, then this fetch clearly
+ does not make sense, in which case the value returned will be -1.
+
+srv_is_up([<backend>/]<server>) : boolean
+ Returns true when the designated server is UP, and false when it is either
+ DOWN or in maintenance mode. If <backend> is omitted, then the server is
+ looked up in the current backend. It is mainly used to take action based on
+ an external status reported via a health check (e.g. a geographical site's
+ availability). Another possible use which is more of a hack consists in
+ using dummy servers as boolean variables that can be enabled or disabled from
+ the CLI, so that rules depending on those ACLs can be tweaked in realtime.
+
+srv_queue([<backend>/]<server>) : integer
+ Returns an integer value corresponding to the number of connections currently
+ pending in the designated server's queue. If <backend> is omitted, then the
+ server is looked up in the current backend. It can sometimes be used together
+ with the "use-server" directive to force to use a known faster server when it
+ is not much loaded. See also the "srv_conn", "avg_queue" and "queue" sample
+ fetch methods.
+
+srv_sess_rate([<backend>/]<server>) : integer
+ Returns an integer corresponding to the sessions creation rate on the
+ designated server, in number of new sessions per second. If <backend> is
+ omitted, then the server is looked up in the current backend. This is mostly
+ used with ACLs but can make sense with logs too. This is used to switch to an
+ alternate backend when an expensive or fragile one reaches too high a session
+ rate, or to limit abuse of service (e.g. prevent latent requests from
+ overloading servers).
+
+ Example :
+ # Redirect to a separate back
+ acl srv1_full srv_sess_rate(be1/srv1) gt 50
+ acl srv2_full srv_sess_rate(be1/srv2) gt 50
+ use_backend be2 if srv1_full or srv2_full
+
+srv_iweight([<backend>/]<server>) : integer
+ Returns an integer corresponding to the server's initial weight. If <backend>
+ is omitted, then the server is looked up in the current backend. See also
+ "srv_weight" and "srv_uweight".
+
+srv_uweight([<backend>/]<server>) : integer
+ Returns an integer corresponding to the user visible server's weight. If
+ <backend> is omitted, then the server is looked up in the current
+ backend. See also "srv_weight" and "srv_iweight".
+
+srv_weight([<backend>/]<server>) : integer
+ Returns an integer corresponding to the current (or effective) server's
+ weight. If <backend> is omitted, then the server is looked up in the current
+ backend. See also "srv_iweight" and "srv_uweight".
+
+stopping : boolean
+ Returns TRUE if the process calling the function is currently stopping. This
+ can be useful for logging, or for relaxing certain checks or helping close
+ certain connections upon graceful shutdown.
+
+str(<string>) : string
+ Returns a string.
+
+table_avl([<table>]) : integer
+ Returns the total number of available entries in the current proxy's
+ stick-table or in the designated stick-table. See also table_cnt.
+
+table_cnt([<table>]) : integer
+ Returns the total number of entries currently in use in the current proxy's
+ stick-table or in the designated stick-table. See also src_conn_cnt and
+ table_avl for other entry counting methods.
+
+thread : integer
+ Returns an integer value corresponding to the position of the thread calling
+ the function, between 0 and (global.nbthread-1). This is useful for logging
+ and debugging purposes.
+
+txn.id32 : integer
+ Returns the internal transaction ID. It is a 32bits integer. So, in absolute,
+ its value is not unique, transaction IDs may wrap. The wrapping period
+ depends on the request rate. In practice, it should not be an issue. For a
+ true unique ID, see "unique-id-format" directive.
+
+txn.conn_retries : integer
+ Returns the the number of connection retries experienced by this stream when
+ trying to connect to the server. This value is subject to change while the
+ connection is not fully established. For HTTP connections, the value may be
+ affected by L7 retries.
+
+txn.sess_term_state : string
+ Retruns the TCP or HTTP stream termination state, as reported in the log. It
+ is a 2-characters string, The final stream state followed by the event which
+ caused its to terminate. See section 8.5 about stream state at disconnection
+ for the list of possible events. The current value at time the sample fetch
+ is evaluated is returned. It is subject to change. Except used with ACLs in
+ "http-after-response" rule sets or in log messages, it will always be "--".
+
+ Example:
+ # Return a 429-Too-Many-Requests if stream timed out in queue
+ http-after-response set-status 429 if { txn.sess_term_state "sQ" }
+
+uuid([<version>]) : string
+ Returns a UUID following the RFC4122 standard. If the version is not
+ specified, a UUID version 4 (fully random) is returned.
+ Currently, only version 4 is supported.
+
+var(<var-name>[,<default>]) : undefined
+ Returns a variable with the stored type. If the variable is not set, the
+ sample fetch fails, unless a default value is provided, in which case it will
+ return it as a string. Empty strings are permitted. The name of the variable
+ starts with an indication about its scope. The scopes allowed are:
+ "proc" : the variable is shared with the whole process
+ "sess" : the variable is shared with the whole session
+ "txn" : the variable is shared with the transaction (request and
+ response),
+ "req" : the variable is shared only during request processing,
+ "res" : the variable is shared only during response processing.
+ This prefix is followed by a name. The separator is a '.'. The name may only
+ contain characters 'a-z', 'A-Z', '0-9', '.' and '_'.
+
+7.3.3. Fetching samples at Layer 4
+----------------------------------
+
+The layer 4 usually describes just the transport layer which in HAProxy is
+closest to the connection, where no content is yet made available. The fetch
+methods described here are usable as low as the "tcp-request connection" rule
+sets unless they require some future information. Those generally include
+TCP/IP addresses and ports, as well as elements from stick-tables related to
+the incoming connection. For retrieving a value from a sticky counters, the
+counter number can be explicitly set as 0, 1, or 2 using the pre-defined
+"sc0_", "sc1_", or "sc2_" prefix. These three pre-defined prefixes can only be
+used if the global "tune.stick-counters" value does not exceed 3, otherwise the
+counter number can be specified as the first integer argument when using the
+"sc_" prefix starting from "sc_0" to "sc_N" where N is (tune.stick-counters-1).
+An optional table may be specified with the "sc*" form, in which case the
+currently tracked key will be looked up into this alternate table instead of
+the table currently being tracked.
+
+Summary of sample fetch methods in this section and their respective types:
+
+ keyword output type
+-------------------------------------------------+-------------
+accept_date([<unit>]) integer
+bc.timer.connect integer
+bc_dst ip
+bc_dst_port integer
+bc_err integer
+bc_err_str string
+bc_http_major integer
+bc_src ip
+bc_src_port integer
+be_id integer
+be_name string
+bc_rtt(<unit>) integer
+bc_rttvar(<unit>) integer
+be_server_timeout integer
+be_tunnel_timeout integer
+bytes_in integer
+bytes_out integer
+cur_server_timeout integer
+cur_tunnel_timeout integer
+cur_client_timeout integer
+dst ip
+dst_conn integer
+dst_is_local boolean
+dst_port integer
+fc.timer.handshake integer
+fc.timer.total integer
+fc_dst ip
+fc_dst_is_local boolean
+fc_dst_port integer
+fc_err integer
+fc_err_str string
+fc_fackets integer
+fc_http_major integer
+fc_lost integer
+fc_pp_authority string
+fc_pp_unique_id string
+fc_pp_tlv(<id>) string
+fc_rcvd_proxy boolean
+fc_reordering integer
+fc_retrans integer
+fc_rtt(<unit>) integer
+fc_rttvar(<unit>) integer
+fc_sacked integer
+fc_src ip
+fc_src_is_local boolean
+fc_src_port integer
+fc_unacked integer
+fe_defbe string
+fe_id integer
+fe_name string
+fe_client_timeout integer
+res.timer.data integer
+sc_bytes_in_rate(<ctr>[,<table>]) integer
+sc0_bytes_in_rate([<table>]) integer
+sc1_bytes_in_rate([<table>]) integer
+sc2_bytes_in_rate([<table>]) integer
+sc_bytes_out_rate(<ctr>[,<table>]) integer
+sc0_bytes_out_rate([<table>]) integer
+sc1_bytes_out_rate([<table>]) integer
+sc2_bytes_out_rate([<table>]) integer
+sc_clr_gpc(<idx>,<ctr>[,<table>]) integer
+sc_clr_gpc0(<ctr>[,<table>]) integer
+sc0_clr_gpc0([<table>]) integer
+sc1_clr_gpc0([<table>]) integer
+sc2_clr_gpc0([<table>]) integer
+sc_clr_gpc1(<ctr>[,<table>]) integer
+sc0_clr_gpc1([<table>]) integer
+sc1_clr_gpc1([<table>]) integer
+sc2_clr_gpc1([<table>]) integer
+sc_conn_cnt(<ctr>[,<table>]) integer
+sc0_conn_cnt([<table>]) integer
+sc1_conn_cnt([<table>]) integer
+sc2_conn_cnt([<table>]) integer
+sc_conn_cur(<ctr>[,<table>]) integer
+sc0_conn_cur([<table>]) integer
+sc1_conn_cur([<table>]) integer
+sc2_conn_cur([<table>]) integer
+sc_conn_rate(<ctr>[,<table>]) integer
+sc0_conn_rate([<table>]) integer
+sc1_conn_rate([<table>]) integer
+sc2_conn_rate([<table>]) integer
+sc_get_gpc(<idx>,<ctr>[,<table>]) integer
+sc_get_gpc0(<ctr>[,<table>]) integer
+sc0_get_gpc0([<table>]) integer
+sc1_get_gpc0([<table>]) integer
+sc2_get_gpc0([<table>]) integer
+sc_get_gpc1(<ctr>[,<table>]) integer
+sc0_get_gpc1([<table>]) integer
+sc1_get_gpc1([<table>]) integer
+sc2_get_gpc1([<table>]) integer
+sc_get_gpt(<idx>,<ctr>[,<table>]) integer
+sc_get_gpt0(<ctr>[,<table>]) integer
+sc0_get_gpt0([<table>]) integer
+sc1_get_gpt0([<table>]) integer
+sc2_get_gpt0([<table>]) integer
+sc_gpc_rate(<idx>,<ctr>[,<table>]) integer
+sc_gpc0_rate(<ctr>[,<table>]) integer
+sc0_gpc0_rate([<table>]) integer
+sc1_gpc0_rate([<table>]) integer
+sc2_gpc0_rate([<table>]) integer
+sc_gpc1_rate(<ctr>[,<table>]) integer
+sc0_gpc1_rate([<table>]) integer
+sc1_gpc1_rate([<table>]) integer
+sc2_gpc1_rate([<table>]) integer
+sc_http_err_cnt(<ctr>[,<table>]) integer
+sc0_http_err_cnt([<table>]) integer
+sc1_http_err_cnt([<table>]) integer
+sc2_http_err_cnt([<table>]) integer
+sc_http_err_rate(<ctr>[,<table>]) integer
+sc0_http_err_rate([<table>]) integer
+sc1_http_err_rate([<table>]) integer
+sc2_http_err_rate([<table>]) integer
+sc_http_fail_cnt(<ctr>[,<table>]) integer
+sc0_http_fail_cnt([<table>]) integer
+sc1_http_fail_cnt([<table>]) integer
+sc2_http_fail_cnt([<table>]) integer
+sc_http_fail_rate(<ctr>[,<table>]) integer
+sc0_http_fail_rate([<table>]) integer
+sc1_http_fail_rate([<table>]) integer
+sc2_http_fail_rate([<table>]) integer
+sc_http_req_cnt(<ctr>[,<table>]) integer
+sc0_http_req_cnt([<table>]) integer
+sc1_http_req_cnt([<table>]) integer
+sc2_http_req_cnt([<table>]) integer
+sc_http_req_rate(<ctr>[,<table>]) integer
+sc0_http_req_rate([<table>]) integer
+sc1_http_req_rate([<table>]) integer
+sc2_http_req_rate([<table>]) integer
+sc_inc_gpc(<idx>,<ctr>[,<table>]) integer
+sc_inc_gpc0(<ctr>[,<table>]) integer
+sc0_inc_gpc0([<table>]) integer
+sc1_inc_gpc0([<table>]) integer
+sc2_inc_gpc0([<table>]) integer
+sc_inc_gpc1(<ctr>[,<table>]) integer
+sc0_inc_gpc1([<table>]) integer
+sc1_inc_gpc1([<table>]) integer
+sc2_inc_gpc1([<table>]) integer
+sc_kbytes_in(<ctr>[,<table>]) integer
+sc0_kbytes_in([<table>]) integer
+sc1_kbytes_in([<table>]) integer
+sc2_kbytes_in([<table>]) integer
+sc_kbytes_out(<ctr>[,<table>]) integer
+sc0_kbytes_out([<table>]) integer
+sc1_kbytes_out([<table>]) integer
+sc2_kbytes_out([<table>]) integer
+sc_sess_cnt(<ctr>[,<table>]) integer
+sc0_sess_cnt([<table>]) integer
+sc1_sess_cnt([<table>]) integer
+sc2_sess_cnt([<table>]) integer
+sc_sess_rate(<ctr>[,<table>]) integer
+sc0_sess_rate([<table>]) integer
+sc1_sess_rate([<table>]) integer
+sc2_sess_rate([<table>]) integer
+sc_tracked(<ctr>[,<table>]) boolean
+sc0_tracked([<table>]) boolean
+sc1_tracked([<table>]) boolean
+sc2_tracked([<table>]) boolean
+sc_trackers(<ctr>[,<table>]) integer
+sc0_trackers([<table>]) integer
+sc1_trackers([<table>]) integer
+sc2_trackers([<table>]) integer
+so_id integer
+so_name string
+src ip
+src_bytes_in_rate([<table>]) integer
+src_bytes_out_rate([<table>]) integer
+src_clr_gpc(<idx>,[<table>]) integer
+src_clr_gpc0([<table>]) integer
+src_clr_gpc1([<table>]) integer
+src_conn_cnt([<table>]) integer
+src_conn_cur([<table>]) integer
+src_conn_rate([<table>]) integer
+src_get_gpc(<idx>,[<table>]) integer
+src_get_gpc0([<table>]) integer
+src_get_gpc1([<table>]) integer
+src_get_gpt(<idx>[,<table>]) integer
+src_get_gpt0([<table>]) integer
+src_gpc_rate(<idx>[,<table>]) integer
+src_gpc0_rate([<table>]) integer
+src_gpc1_rate([<table>]) integer
+src_http_err_cnt([<table>]) integer
+src_http_err_rate([<table>]) integer
+src_http_fail_cnt([<table>]) integer
+src_http_fail_rate([<table>]) integer
+src_http_req_cnt([<table>]) integer
+src_http_req_rate([<table>]) integer
+src_inc_gpc(<idx>,[<table>]) integer
+src_inc_gpc0([<table>]) integer
+src_inc_gpc1([<table>]) integer
+src_is_local boolean
+src_kbytes_in([<table>]) integer
+src_kbytes_out([<table>]) integer
+src_port integer
+src_sess_cnt([<table>]) integer
+src_sess_rate([<table>]) integer
+src_updt_conn_cnt([<table>]) integer
+srv_id integer
+srv_name string
+txn.conn_retries integer
+-------------------------------------------------+-------------
+
+Detailed list:
+
+accept_date([<unit>]) : integer
+ This is the exact date when the connection was received by HAProxy
+ (which might be very slightly different from the date observed on the
+ network if there was some queuing in the system's backlog). This is usually
+ the same date which may appear in any upstream firewall's log. When used in
+ HTTP mode, the accept_date field will be reset to the first moment the
+ connection is ready to receive a new request (end of previous response for
+ HTTP/1, immediately after previous request for HTTP/2).
+
+ Returns a value in number of seconds since epoch.
+
+ <unit> is facultative, and can be set to "s" for seconds (default behavior),
+ "ms" for milliseconds or "us" for microseconds.
+ If unit is set, return value is an integer reflecting either seconds,
+ milliseconds or microseconds since epoch.
+ It is useful when a time resolution of less than a second is needed.
+
+bc.timer.connect : integer
+ Total time to establish the TCP connection to the server. This is the
+ equivalent of %Tc in the log-format. This is reported in milliseconds (ms).
+ For more information see Section 8.4 "Timing events"
+
+bc_dst : ip
+ This is the destination ip address of the connection on the server side,
+ which is the server address HAProxy connected to. It is of type IP and works
+ on both IPv4 and IPv6 tables. On IPv6 tables, IPv4 address is mapped to its
+ IPv6 equivalent, according to RFC 4291.
+
+bc_dst_port : integer
+ Returns an integer value corresponding to the destination TCP port of the
+ connection on the server side, which is the port HAProxy connected to.
+
+bc_err : integer
+ Returns the ID of the error that might have occurred on the current backend
+ connection. See the "fc_err_str" fetch for a full list of error codes
+ and their corresponding error message.
+
+bc_err_str : string
+ Returns an error message describing what problem happened on the current
+ backend connection, resulting in a connection failure. See the
+ "fc_err_str" fetch for a full list of error codes and their
+ corresponding error message.
+
+bc_http_major : integer
+ Returns the backend connection's HTTP major version encoding, which may be 1
+ for HTTP/0.9 to HTTP/1.1 or 2 for HTTP/2. Note, this is based on the on-wire
+ encoding and not the version present in the request header.
+
+bc_src : ip
+ This is the source ip address of the connection on the server side, which is
+ the server address HAProxy connected from. It is of type IP and works on both
+ IPv4 and IPv6 tables. On IPv6 tables, IPv4 addresses are mapped to their IPv6
+ equivalent, according to RFC 4291.
+
+bc_src_port : integer
+ Returns an integer value corresponding to the TCP source port of the
+ connection on the server side, which is the port HAProxy connected from.
+
+be_id : integer
+ Returns an integer containing the current backend's id. It can be used in
+ frontends with responses to check which backend processed the request. If
+ used in a frontend and no backend was used, it returns the current
+ frontend's id. It can also be used in a tcp-check or an http-check ruleset.
+
+be_name : string
+ Returns a string containing the current backend's name. It can be used in
+ frontends with responses to check which backend processed the request. If
+ used in a frontend and no backend was used, it returns the current
+ frontend's name. It can also be used in a tcp-check or an http-check
+ ruleset.
+
+bc_rtt(<unit>) : integer
+ Returns the Round Trip Time (RTT) measured by the kernel for the backend
+ connection. <unit> is facultative, by default the unit is milliseconds. <unit>
+ can be set to "ms" for milliseconds or "us" for microseconds. If the server
+ connection is not established, if the connection is not TCP or if the
+ operating system does not support TCP_INFO, for example Linux kernels before
+ 2.4, the sample fetch fails.
+
+bc_rttvar(<unit>) : integer
+ Returns the Round Trip Time (RTT) variance measured by the kernel for the
+ backend connection. <unit> is facultative, by default the unit is milliseconds.
+ <unit> can be set to "ms" for milliseconds or "us" for microseconds. If the
+ server connection is not established, if the connection is not TCP or if the
+ operating system does not support TCP_INFO, for example Linux kernels before
+ 2.4, the sample fetch fails.
+
+be_server_timeout : integer
+ Returns the configuration value in millisecond for the server timeout of the
+ current backend. This timeout can be overwritten by a "set-timeout" rule. See
+ also the "cur_server_timeout".
+
+be_tunnel_timeout : integer
+ Returns the configuration value in millisecond for the tunnel timeout of the
+ current backend. This timeout can be overwritten by a "set-timeout" rule. See
+ also the "cur_tunnel_timeout".
+
+bytes_in : integer
+ This returns the number of bytes uploaded from the client to the server.
+
+bytes_out : integer
+ This is the number of bytes transmitted from the server to the client.
+
+cur_server_timeout : integer
+ Returns the currently applied server timeout in millisecond for the stream.
+ In the default case, this will be equal to be_server_timeout unless a
+ "set-timeout" rule has been applied. See also "be_server_timeout".
+
+cur_tunnel_timeout : integer
+ Returns the currently applied tunnel timeout in millisecond for the stream.
+ In the default case, this will be equal to be_tunnel_timeout unless a
+ "set-timeout" rule has been applied. See also "be_tunnel_timeout".
+
+cur_client_timeout : integer
+ Returns the currently applied client timeout in millisecond for the stream.
+ In the default case, this will be equal to fe_client_timeout unless a
+ "set-timeout" rule has been applied. See also "fe_client_timeout".
+
+dst : ip
+ This is the destination IP address of the connection on the client side,
+ which is the address the client connected to. Any tcp/http rules may alter
+ this address. It can be useful when running in transparent mode. It is of
+ type IP and works on both IPv4 and IPv6 tables. On IPv6 tables, IPv4 address
+ is mapped to its IPv6 equivalent, according to RFC 4291. When the incoming
+ connection passed through address translation or redirection involving
+ connection tracking, the original destination address before the redirection
+ will be reported. On Linux systems, the source and destination may seldom
+ appear reversed if the nf_conntrack_tcp_loose sysctl is set, because a late
+ response may reopen a timed out connection and switch what is believed to be
+ the source and the destination.
+
+dst_conn : integer
+ Returns an integer value corresponding to the number of currently established
+ connections on the same socket including the one being evaluated. It is
+ normally used with ACLs but can as well be used to pass the information to
+ servers in an HTTP header or in logs. It can be used to either return a sorry
+ page before hard-blocking, or to use a specific backend to drain new requests
+ when the socket is considered saturated. This offers the ability to assign
+ different limits to different listening ports or addresses. See also the
+ "fe_conn" and "be_conn" fetches.
+
+dst_is_local : boolean
+ Returns true if the destination address of the incoming connection is local
+ to the system, or false if the address doesn't exist on the system, meaning
+ that it was intercepted in transparent mode. It can be useful to apply
+ certain rules by default to forwarded traffic and other rules to the traffic
+ targeting the real address of the machine. For example the stats page could
+ be delivered only on this address, or SSH access could be locally redirected.
+ Please note that the check involves a few system calls, so it's better to do
+ it only once per connection.
+
+dst_port : integer
+ Returns an integer value corresponding to the destination TCP port of the
+ connection on the client side, which is the port the client connected to.
+ Any tcp/http rules may alter this address. This might be used when running in
+ transparent mode, when assigning dynamic ports to some clients for a whole
+ application session, to stick all users to a same server, or to pass the
+ destination port information to a server using an HTTP header.
+
+fc.timer.handshake : integer
+ Total time to accept tcp connection and execute handshakes for low level
+ protocols. Currently, these protocols are proxy-protocol and SSL. This is the
+ equivalent of %Th in the log-format. This is reported in milliseconds (ms).
+ For more information see Section 8.4 "Timing events"
+
+fc.timer.total : integer
+ Total stream duration time, between the moment the proxy accepted it and the
+ moment both ends were closed. This is the equivalent of %Tt in the log-format.
+ This is reported in milliseconds (ms). For more information see Section 8.4
+ "Timing events"
+
+fc_dst : ip
+ This is the original destination IP address of the connection on the client
+ side. Only "tcp-request connection" rules may alter this address. See "dst"
+ for details.
+
+fc_dst_is_local : boolean
+ Returns true if the original destination address of the incoming connection
+ is local to the system, or false if the address doesn't exist on the
+ system. See "dst_is_local" for details.
+
+fc_dst_port : integer
+ Returns an integer value corresponding to the original destination TCP port
+ of the connection on the client side. Only "tcp-request connection" rules may
+ alter this address. See "dst-port" for details.
+
+fc_err : integer
+ Returns the ID of the error that might have occurred on the current
+ connection. Any strictly positive value of this fetch indicates that the
+ connection did not succeed and would result in an error log being output (as
+ described in section 8.2.6). See the "fc_err_str" fetch for a full list of
+ error codes and their corresponding error message.
+
+fc_err_str : string
+ Returns an error message describing what problem happened on the current
+ connection, resulting in a connection failure. This string corresponds to the
+ "message" part of the error log format (see section 8.2.6). See below for a
+ full list of error codes and their corresponding error messages :
+
+ +----+---------------------------------------------------------------------------+
+ | ID | message |
+ +----+---------------------------------------------------------------------------+
+ | 0 | "Success" |
+ | 1 | "Reached configured maxconn value" |
+ | 2 | "Too many sockets on the process" |
+ | 3 | "Too many sockets on the system" |
+ | 4 | "Out of system buffers" |
+ | 5 | "Protocol or address family not supported" |
+ | 6 | "General socket error" |
+ | 7 | "Source port range exhausted" |
+ | 8 | "Can't bind to source address" |
+ | 9 | "Out of local source ports on the system" |
+ | 10 | "Local source address already in use" |
+ | 11 | "Connection closed while waiting for PROXY protocol header" |
+ | 12 | "Connection error while waiting for PROXY protocol header" |
+ | 13 | "Timeout while waiting for PROXY protocol header" |
+ | 14 | "Truncated PROXY protocol header received" |
+ | 15 | "Received something which does not look like a PROXY protocol header" |
+ | 16 | "Received an invalid PROXY protocol header" |
+ | 17 | "Received an unhandled protocol in the PROXY protocol header" |
+ | 18 | "Connection closed while waiting for NetScaler Client IP header" |
+ | 19 | "Connection error while waiting for NetScaler Client IP header" |
+ | 20 | "Timeout while waiting for a NetScaler Client IP header" |
+ | 21 | "Truncated NetScaler Client IP header received" |
+ | 22 | "Received an invalid NetScaler Client IP magic number" |
+ | 23 | "Received an unhandled protocol in the NetScaler Client IP header" |
+ | 24 | "Connection closed during SSL handshake" |
+ | 25 | "Connection error during SSL handshake" |
+ | 26 | "Timeout during SSL handshake" |
+ | 27 | "Too many SSL connections" |
+ | 28 | "Out of memory when initializing an SSL connection" |
+ | 29 | "Rejected a client-initiated SSL renegotiation attempt" |
+ | 30 | "SSL client CA chain cannot be verified" |
+ | 31 | "SSL client certificate not trusted" |
+ | 32 | "Server presented an SSL certificate different from the configured one" |
+ | 33 | "Server presented an SSL certificate different from the expected one" |
+ | 34 | "SSL handshake failure" |
+ | 35 | "SSL handshake failure after heartbeat" |
+ | 36 | "Stopped a TLSv1 heartbeat attack (CVE-2014-0160)" |
+ | 37 | "Attempt to use SSL on an unknown target (internal error)" |
+ | 38 | "Server refused early data" |
+ | 39 | "SOCKS4 Proxy write error during handshake" |
+ | 40 | "SOCKS4 Proxy read error during handshake" |
+ | 41 | "SOCKS4 Proxy deny the request" |
+ | 42 | "SOCKS4 Proxy handshake aborted by server" |
+ | 43 | "SSL fatal error" |
+ +----+---------------------------------------------------------------------------+
+
+fc_fackets : integer
+ Returns the fack counter measured by the kernel for the client
+ connection. If the server connection is not established, if the connection is
+ not TCP or if the operating system does not support TCP_INFO, for example
+ Linux kernels before 2.4, the sample fetch fails.
+
+fc_http_major : integer
+ Reports the front connection's HTTP major version encoding, which may be 1
+ for HTTP/0.9 to HTTP/1.1 or 2 for HTTP/2. Note, this is based on the on-wire
+ encoding and not on the version present in the request header.
+
+fc_lost : integer
+ Returns the lost counter measured by the kernel for the client
+ connection. If the server connection is not established, if the connection is
+ not TCP or if the operating system does not support TCP_INFO, for example
+ Linux kernels before 2.4, the sample fetch fails.
+
+fc_pp_authority : string
+ Returns the first authority TLV sent by the client in the PROXY protocol
+ header, if any.
+
+fc_pp_unique_id : string
+ Returns the first unique ID TLV sent by the client in the PROXY protocol
+ header, if any.
+
+fc_pp_tlv(<id>) : string
+ Returns the TLV value for the given TLV ID. The ID must either be a numeric
+ value between 0 and 255 or one of the following supported symbolic names
+ that correspond to the TLV constant suffixes in the PPv2 spec:
+ "ALPN": PP2_TYPE_ALPN, "AUTHORITY": PP2_TYPE_AUTHORITY,
+ "CRC32": PP2_TYPE_CRC32C, "NETNS": PP2_TYPE_NETNS, "NOOP: PP2_TYPE_NOOP",
+ "SSL": PP2_TYPE_SSL, "SSL_CIPHER": PP2_SUBTYPE_SSL_CIPHER,
+ "SSL_CN": PP2_SUBTYPE_SSL_CN, "SSL_KEY_ALG": PP2_SUBTYPE_SSL_KEY_ALG,
+ "SSL_SIG_ALG": PP2_SUBTYPE_SSL_SIG_ALG,
+ "SSL_VERSION": PP2_SUBTYPE_SSL_VERSION, "UNIQUE_ID": PP2_TYPE_UNIQUE_ID.
+
+ The received value must be smaller or equal to 1024 bytes. This is done to
+ prevent potential DoS attacks. Values smaller or equal to 256 bytes will be
+ able to be memory pooled. Therefore, try to restrict the length of sent
+ values to 256 bytes for optimal performance.
+
+ Note that unlike fc_pp_authority and fc_pp_unique_id, fc_pp_tlv is able to
+ iterate over all occurrences of a requested TLV in case there are duplicate
+ TLV IDs. The order of iteration matches the position in the PROXY protocol
+ header. However, relying on duplicates should mostly be avoided as TLVs are
+ typically assumed to be unique. Generally, finding duplicated TLV IDs
+ indicates an error on the sender side of the PROXY protocol header.
+
+fc_rcvd_proxy : boolean
+ Returns true if the client initiated the connection with a PROXY protocol
+ header.
+
+fc_reordering : integer
+ Returns the reordering counter measured by the kernel for the client
+ connection. If the server connection is not established, if the connection is
+ not TCP or if the operating system does not support TCP_INFO, for example
+ Linux kernels before 2.4, the sample fetch fails.
+
+fc_retrans : integer
+ Returns the retransmits counter measured by the kernel for the client
+ connection. If the server connection is not established, if the connection is
+ not TCP or if the operating system does not support TCP_INFO, for example
+ Linux kernels before 2.4, the sample fetch fails.
+
+fc_rtt(<unit>) : integer
+ Returns the Round Trip Time (RTT) measured by the kernel for the client
+ connection. <unit> is facultative, by default the unit is milliseconds. <unit>
+ can be set to "ms" for milliseconds or "us" for microseconds. If the server
+ connection is not established, if the connection is not TCP or if the
+ operating system does not support TCP_INFO, for example Linux kernels before
+ 2.4, the sample fetch fails.
+
+fc_rttvar(<unit>) : integer
+ Returns the Round Trip Time (RTT) variance measured by the kernel for the
+ client connection. <unit> is facultative, by default the unit is milliseconds.
+ <unit> can be set to "ms" for milliseconds or "us" for microseconds. If the
+ server connection is not established, if the connection is not TCP or if the
+ operating system does not support TCP_INFO, for example Linux kernels before
+ 2.4, the sample fetch fails.
+
+fc_sacked : integer
+ Returns the sacked counter measured by the kernel for the client connection.
+ If the server connection is not established, if the connection is not TCP or
+ if the operating system does not support TCP_INFO, for example Linux kernels
+ before 2.4, the sample fetch fails.
+
+fc_src : ip
+ This is the original source IP address of the connection on the client side
+ Only "tcp-request connection" rules may alter this address. See "src" for
+ details.
+
+fc_src_is_local : boolean
+ Returns true if the source address of incoming connection is local to the
+ system, or false if the address doesn't exist on the system. See
+ "src_is_local" for details.
+
+fc_src_port : integer
+
+ Returns an integer value corresponding to the TCP source port of the
+ connection on the client side. Only "tcp-request connection" rules may alter
+ this address. See "src-port" for details.
+
+
+fc_unacked : integer
+ Returns the unacked counter measured by the kernel for the client connection.
+ If the server connection is not established, if the connection is not TCP or
+ if the operating system does not support TCP_INFO, for example Linux kernels
+ before 2.4, the sample fetch fails.
+
+fe_defbe : string
+ Returns a string containing the frontend's default backend name. It can be
+ used in frontends to check which backend will handle requests by default.
+
+fe_id : integer
+ Returns an integer containing the current frontend's id. It can be used in
+ backends to check from which frontend it was called, or to stick all users
+ coming via a same frontend to the same server.
+
+fe_name : string
+ Returns a string containing the current frontend's name. It can be used in
+ backends to check from which frontend it was called, or to stick all users
+ coming via a same frontend to the same server.
+
+fe_client_timeout : integer
+ Returns the configuration value in millisecond for the client timeout of the
+ current frontend. This timeout can be overwritten by a "set-timeout" rule.
+
+res.timer.data : integer
+ this is the total transfer time of the response payload till the last byte
+ sent to the client. In HTTP it starts after the last response header (after
+ Tr). This is the equivalent of %Td in the log-format and is reported in
+ milliseconds (ms). For more information see Section 8.4 "Timing events"
+
+sc_bytes_in_rate(<ctr>[,<table>]) : integer
+sc0_bytes_in_rate([<table>]) : integer
+sc1_bytes_in_rate([<table>]) : integer
+sc2_bytes_in_rate([<table>]) : integer
+ Returns the average client-to-server bytes rate from the currently tracked
+ counters, measured in amount of bytes over the period configured in the
+ table. See also src_bytes_in_rate.
+
+sc_bytes_out_rate(<ctr>[,<table>]) : integer
+sc0_bytes_out_rate([<table>]) : integer
+sc1_bytes_out_rate([<table>]) : integer
+sc2_bytes_out_rate([<table>]) : integer
+ Returns the average server-to-client bytes rate from the currently tracked
+ counters, measured in amount of bytes over the period configured in the
+ table. See also src_bytes_out_rate.
+
+sc_clr_gpc(<idx>,<ctr>[,<table>]) : integer
+ Clears the General Purpose Counter at the index <idx> of the array
+ associated to the designated tracked counter of ID <ctr> from current
+ proxy's stick table or from the designated stick-table <table>, and
+ returns its previous value. <idx> is an integer between 0 and 99 and
+ <ctr> an integer between 0 and 2.
+ Before the first invocation, the stored value is zero, so first invocation
+ will always return zero.
+ This fetch applies only to the 'gpc' array data_type (and not to the legacy
+ 'gpc0' nor 'gpc1' data_types).
+
+sc_clr_gpc0(<ctr>[,<table>]) : integer
+sc0_clr_gpc0([<table>]) : integer
+sc1_clr_gpc0([<table>]) : integer
+sc2_clr_gpc0([<table>]) : integer
+ Clears the first General Purpose Counter associated to the currently tracked
+ counters, and returns its previous value. Before the first invocation, the
+ stored value is zero, so first invocation will always return zero. This is
+ typically used as a second ACL in an expression in order to mark a connection
+ when a first ACL was verified :
+
+ Example:
+ # block if 5 consecutive requests continue to come faster than 10 sess
+ # per second, and reset the counter as soon as the traffic slows down.
+ acl abuse sc0_http_req_rate gt 10
+ acl kill sc0_inc_gpc0 gt 5
+ acl save sc0_clr_gpc0 ge 0
+ tcp-request connection accept if !abuse save
+ tcp-request connection reject if abuse kill
+
+sc_clr_gpc1(<ctr>[,<table>]) : integer
+sc0_clr_gpc1([<table>]) : integer
+sc1_clr_gpc1([<table>]) : integer
+sc2_clr_gpc1([<table>]) : integer
+ Clears the second General Purpose Counter associated to the currently tracked
+ counters, and returns its previous value. Before the first invocation, the
+ stored value is zero, so first invocation will always return zero. This is
+ typically used as a second ACL in an expression in order to mark a connection
+ when a first ACL was verified.
+
+sc_conn_cnt(<ctr>[,<table>]) : integer
+sc0_conn_cnt([<table>]) : integer
+sc1_conn_cnt([<table>]) : integer
+sc2_conn_cnt([<table>]) : integer
+ Returns the cumulative number of incoming connections from currently tracked
+ counters. See also src_conn_cnt.
+
+sc_conn_cur(<ctr>[,<table>]) : integer
+sc0_conn_cur([<table>]) : integer
+sc1_conn_cur([<table>]) : integer
+sc2_conn_cur([<table>]) : integer
+ Returns the current amount of concurrent connections tracking the same
+ tracked counters. This number is automatically incremented when tracking
+ begins and decremented when tracking stops. See also src_conn_cur.
+
+sc_conn_rate(<ctr>[,<table>]) : integer
+sc0_conn_rate([<table>]) : integer
+sc1_conn_rate([<table>]) : integer
+sc2_conn_rate([<table>]) : integer
+ Returns the average connection rate from the currently tracked counters,
+ measured in amount of connections over the period configured in the table.
+ See also src_conn_rate.
+
+sc_get_gpc(<idx>,<ctr>[,<table>]) : integer
+ Returns the value of the General Purpose Counter at the index <idx>
+ in the GPC array and associated to the currently tracked counter of
+ ID <ctr> from the current proxy's stick-table or from the designated
+ stick-table <table>. <idx> is an integer between 0 and 99 and
+ <ctr> an integer between 0 and 2. If there is not gpc stored at this
+ index, zero is returned.
+ This fetch applies only to the 'gpc' array data_type (and not to the legacy
+ 'gpc0' nor 'gpc1' data_types). See also src_get_gpc and sc_inc_gpc.
+
+sc_get_gpc0(<ctr>[,<table>]) : integer
+sc0_get_gpc0([<table>]) : integer
+sc1_get_gpc0([<table>]) : integer
+sc2_get_gpc0([<table>]) : integer
+ Returns the value of the first General Purpose Counter associated to the
+ currently tracked counters. See also src_get_gpc0 and sc/sc0/sc1/sc2_inc_gpc0.
+
+sc_get_gpc1(<ctr>[,<table>]) : integer
+sc0_get_gpc1([<table>]) : integer
+sc1_get_gpc1([<table>]) : integer
+sc2_get_gpc1([<table>]) : integer
+ Returns the value of the second General Purpose Counter associated to the
+ currently tracked counters. See also src_get_gpc1 and sc/sc0/sc1/sc2_inc_gpc1.
+
+sc_get_gpt(<idx>,<ctr>[,<table>]) : integer
+ Returns the value of the first General Purpose Tag at the index <idx> of
+ the array associated to the tracked counter of ID <ctr> and from the
+ current proxy's sitck-table or the designated stick-table <table>. <idx>
+ is an integer between 0 and 99 and <ctr> an integer between 0 and 2.
+ If there is no GPT stored at this index, zero is returned.
+ This fetch applies only to the 'gpt' array data_type (and not on
+ the legacy 'gpt0' data-type). See also src_get_gpt.
+
+sc_get_gpt0(<ctr>[,<table>]) : integer
+sc0_get_gpt0([<table>]) : integer
+sc1_get_gpt0([<table>]) : integer
+sc2_get_gpt0([<table>]) : integer
+ Returns the value of the first General Purpose Tag associated to the
+ currently tracked counters. See also src_get_gpt0.
+
+sc_gpc_rate(<idx>,<ctr>[,<table>]) : integer
+ Returns the average increment rate of the General Purpose Counter at the
+ index <idx> of the array associated to the tracked counter of ID <ctr> from
+ the current proxy's table or from the designated stick-table <table>.
+ It reports the frequency which the gpc counter was incremented over the
+ configured period. <idx> is an integer between 0 and 99 and <ctr> an integer
+ between 0 and 2.
+ Note that the 'gpc_rate' counter array must be stored in the stick-table
+ for a value to be returned, as 'gpc' only holds the event count.
+ This fetch applies only to the 'gpc_rate' array data_type (and not to
+ the legacy 'gpc0_rate' nor 'gpc1_rate' data_types).
+ See also src_gpc_rate, sc_get_gpc, and sc_inc_gpc.
+
+sc_gpc0_rate(<ctr>[,<table>]) : integer
+sc0_gpc0_rate([<table>]) : integer
+sc1_gpc0_rate([<table>]) : integer
+sc2_gpc0_rate([<table>]) : integer
+ Returns the average increment rate of the first General Purpose Counter
+ associated to the currently tracked counters. It reports the frequency
+ which the gpc0 counter was incremented over the configured period. See also
+ src_gpc0_rate, sc/sc0/sc1/sc2_get_gpc0, and sc/sc0/sc1/sc2_inc_gpc0. Note
+ that the "gpc0_rate" counter must be stored in the stick-table for a value to
+ be returned, as "gpc0" only holds the event count.
+
+sc_gpc1_rate(<ctr>[,<table>]) : integer
+sc0_gpc1_rate([<table>]) : integer
+sc1_gpc1_rate([<table>]) : integer
+sc2_gpc1_rate([<table>]) : integer
+ Returns the average increment rate of the second General Purpose Counter
+ associated to the currently tracked counters. It reports the frequency
+ which the gpc1 counter was incremented over the configured period. See also
+ src_gpcA_rate, sc/sc0/sc1/sc2_get_gpc1, and sc/sc0/sc1/sc2_inc_gpc1. Note
+ that the "gpc1_rate" counter must be stored in the stick-table for a value to
+ be returned, as "gpc1" only holds the event count.
+
+sc_http_err_cnt(<ctr>[,<table>]) : integer
+sc0_http_err_cnt([<table>]) : integer
+sc1_http_err_cnt([<table>]) : integer
+sc2_http_err_cnt([<table>]) : integer
+ Returns the cumulative number of HTTP errors from the currently tracked
+ counters. This includes the both request errors and 4xx error responses.
+ See also src_http_err_cnt.
+
+sc_http_err_rate(<ctr>[,<table>]) : integer
+sc0_http_err_rate([<table>]) : integer
+sc1_http_err_rate([<table>]) : integer
+sc2_http_err_rate([<table>]) : integer
+ Returns the average rate of HTTP errors from the currently tracked counters,
+ measured in amount of errors over the period configured in the table. This
+ includes the both request errors and 4xx error responses. See also
+ src_http_err_rate.
+
+sc_http_fail_cnt(<ctr>[,<table>]) : integer
+sc0_http_fail_cnt([<table>]) : integer
+sc1_http_fail_cnt([<table>]) : integer
+sc2_http_fail_cnt([<table>]) : integer
+ Returns the cumulative number of HTTP response failures from the currently
+ tracked counters. This includes the both response errors and 5xx status codes
+ other than 501 and 505. See also src_http_fail_cnt.
+
+sc_http_fail_rate(<ctr>[,<table>]) : integer
+sc0_http_fail_rate([<table>]) : integer
+sc1_http_fail_rate([<table>]) : integer
+sc2_http_fail_rate([<table>]) : integer
+ Returns the average rate of HTTP response failures from the currently tracked
+ counters, measured in amount of failures over the period configured in the
+ table. This includes the both response errors and 5xx status codes other than
+ 501 and 505. See also src_http_fail_rate.
+
+sc_http_req_cnt(<ctr>[,<table>]) : integer
+sc0_http_req_cnt([<table>]) : integer
+sc1_http_req_cnt([<table>]) : integer
+sc2_http_req_cnt([<table>]) : integer
+ Returns the cumulative number of HTTP requests from the currently tracked
+ counters. This includes every started request, valid or not. See also
+ src_http_req_cnt.
+
+sc_http_req_rate(<ctr>[,<table>]) : integer
+sc0_http_req_rate([<table>]) : integer
+sc1_http_req_rate([<table>]) : integer
+sc2_http_req_rate([<table>]) : integer
+ Returns the average rate of HTTP requests from the currently tracked
+ counters, measured in amount of requests over the period configured in
+ the table. This includes every started request, valid or not. See also
+ src_http_req_rate.
+
+sc_inc_gpc(<idx>,<ctr>[,<table>]) : integer
+ Increments the General Purpose Counter at the index <idx> of the array
+ associated to the designated tracked counter of ID <ctr> from current
+ proxy's stick table or from the designated stick-table <table>, and
+ returns its new value. <idx> is an integer between 0 and 99 and
+ <ctr> an integer between 0 and 2.
+ Before the first invocation, the stored value is zero, so first invocation
+ will increase it to 1 and will return 1.
+ This fetch applies only to the 'gpc' array data_type (and not to the legacy
+ 'gpc0' nor 'gpc1' data_types).
+
+sc_inc_gpc0(<ctr>[,<table>]) : integer
+sc0_inc_gpc0([<table>]) : integer
+sc1_inc_gpc0([<table>]) : integer
+sc2_inc_gpc0([<table>]) : integer
+ Increments the first General Purpose Counter associated to the currently
+ tracked counters, and returns its new value. Before the first invocation,
+ the stored value is zero, so first invocation will increase it to 1 and will
+ return 1. This is typically used as a second ACL in an expression in order
+ to mark a connection when a first ACL was verified :
+
+ Example:
+ acl abuse sc0_http_req_rate gt 10
+ acl kill sc0_inc_gpc0 gt 0
+ tcp-request connection reject if abuse kill
+
+sc_inc_gpc1(<ctr>[,<table>]) : integer
+sc0_inc_gpc1([<table>]) : integer
+sc1_inc_gpc1([<table>]) : integer
+sc2_inc_gpc1([<table>]) : integer
+ Increments the second General Purpose Counter associated to the currently
+ tracked counters, and returns its new value. Before the first invocation,
+ the stored value is zero, so first invocation will increase it to 1 and will
+ return 1. This is typically used as a second ACL in an expression in order
+ to mark a connection when a first ACL was verified.
+
+sc_kbytes_in(<ctr>[,<table>]) : integer
+sc0_kbytes_in([<table>]) : integer
+sc1_kbytes_in([<table>]) : integer
+sc2_kbytes_in([<table>]) : integer
+ Returns the total amount of client-to-server data from the currently tracked
+ counters, measured in kilobytes. The test is currently performed on 32-bit
+ integers, which limits values to 4 terabytes. See also src_kbytes_in.
+
+sc_kbytes_out(<ctr>[,<table>]) : integer
+sc0_kbytes_out([<table>]) : integer
+sc1_kbytes_out([<table>]) : integer
+sc2_kbytes_out([<table>]) : integer
+ Returns the total amount of server-to-client data from the currently tracked
+ counters, measured in kilobytes. The test is currently performed on 32-bit
+ integers, which limits values to 4 terabytes. See also src_kbytes_out.
+
+sc_sess_cnt(<ctr>[,<table>]) : integer
+sc0_sess_cnt([<table>]) : integer
+sc1_sess_cnt([<table>]) : integer
+sc2_sess_cnt([<table>]) : integer
+ Returns the cumulative number of incoming connections that were transformed
+ into sessions, which means that they were accepted by a "tcp-request
+ connection" rule, from the currently tracked counters. A backend may count
+ more sessions than connections because each connection could result in many
+ backend sessions if some HTTP keep-alive is performed over the connection
+ with the client. See also src_sess_cnt.
+
+sc_sess_rate(<ctr>[,<table>]) : integer
+sc0_sess_rate([<table>]) : integer
+sc1_sess_rate([<table>]) : integer
+sc2_sess_rate([<table>]) : integer
+ Returns the average session rate from the currently tracked counters,
+ measured in amount of sessions over the period configured in the table. A
+ session is a connection that got past the early "tcp-request connection"
+ rules. A backend may count more sessions than connections because each
+ connection could result in many backend sessions if some HTTP keep-alive is
+ performed over the connection with the client. See also src_sess_rate.
+
+sc_tracked(<ctr>[,<table>]) : boolean
+sc0_tracked([<table>]) : boolean
+sc1_tracked([<table>]) : boolean
+sc2_tracked([<table>]) : boolean
+ Returns true if the designated session counter is currently being tracked by
+ the current session. This can be useful when deciding whether or not we want
+ to set some values in a header passed to the server.
+
+sc_trackers(<ctr>[,<table>]) : integer
+sc0_trackers([<table>]) : integer
+sc1_trackers([<table>]) : integer
+sc2_trackers([<table>]) : integer
+ Returns the current amount of concurrent connections tracking the same
+ tracked counters. This number is automatically incremented when tracking
+ begins and decremented when tracking stops. It differs from sc0_conn_cur in
+ that it does not rely on any stored information but on the table's reference
+ count (the "use" value which is returned by "show table" on the CLI). This
+ may sometimes be more suited for layer7 tracking. It can be used to tell a
+ server how many concurrent connections there are from a given address for
+ example.
+
+so_id : integer
+ Returns an integer containing the current listening socket's id. It is useful
+ in frontends involving many "bind" lines, or to stick all users coming via a
+ same socket to the same server.
+
+so_name : string
+ Returns a string containing the current listening socket's name, as defined
+ with name on a "bind" line. It can serve the same purposes as so_id but with
+ strings instead of integers.
+
+src : ip
+ This is the source IP address of the client of the session. Any tcp/http
+ rules may alter this address. It is of type IP and works on both IPv4 and
+ IPv6 tables. On IPv6 tables, IPv4 addresses are mapped to their IPv6
+ equivalent, according to RFC 4291. Note that it is the TCP-level source
+ address which is used, and not the address of a client behind a
+ proxy. However if the "accept-proxy" or "accept-netscaler-cip" bind directive
+ is used, it can be the address of a client behind another PROXY-protocol
+ compatible component for all rule sets except "tcp-request connection" which
+ sees the real address. When the incoming connection passed through address
+ translation or redirection involving connection tracking, the original
+ destination address before the redirection will be reported. On Linux
+ systems, the source and destination may seldom appear reversed if the
+ nf_conntrack_tcp_loose sysctl is set, because a late response may reopen a
+ timed out connection and switch what is believed to be the source and the
+ destination.
+
+ Example:
+ # add an HTTP header in requests with the originating address' country
+ http-request set-header X-Country %[src,map_ip(geoip.lst)]
+
+src_bytes_in_rate([<table>]) : integer
+ Returns the average bytes rate from the incoming connection's source address
+ in the current proxy's stick-table or in the designated stick-table, measured
+ in amount of bytes over the period configured in the table. If the address is
+ not found, zero is returned. See also sc/sc0/sc1/sc2_bytes_in_rate.
+
+src_bytes_out_rate([<table>]) : integer
+ Returns the average bytes rate to the incoming connection's source address in
+ the current proxy's stick-table or in the designated stick-table, measured in
+ amount of bytes over the period configured in the table. If the address is
+ not found, zero is returned. See also sc/sc0/sc1/sc2_bytes_out_rate.
+
+src_clr_gpc(<idx>,[<table>]) : integer
+ Clears the General Purpose Counter at the index <idx> of the array
+ associated to the incoming connection's source address in the current proxy's
+ stick-table or in the designated stick-table <table>, and returns its
+ previous value. <idx> is an integer between 0 and 99.
+ If the address is not found, an entry is created and 0 is returned.
+ This fetch applies only to the 'gpc' array data_type (and not to the legacy
+ 'gpc0' nor 'gpc1' data_types).
+ See also sc_clr_gpc.
+
+src_clr_gpc0([<table>]) : integer
+ Clears the first General Purpose Counter associated to the incoming
+ connection's source address in the current proxy's stick-table or in the
+ designated stick-table, and returns its previous value. If the address is not
+ found, an entry is created and 0 is returned. This is typically used as a
+ second ACL in an expression in order to mark a connection when a first ACL
+ was verified :
+
+ Example:
+ # block if 5 consecutive requests continue to come faster than 10 sess
+ # per second, and reset the counter as soon as the traffic slows down.
+ acl abuse src_http_req_rate gt 10
+ acl kill src_inc_gpc0 gt 5
+ acl save src_clr_gpc0 ge 0
+ tcp-request connection accept if !abuse save
+ tcp-request connection reject if abuse kill
+
+src_clr_gpc1([<table>]) : integer
+ Clears the second General Purpose Counter associated to the incoming
+ connection's source address in the current proxy's stick-table or in the
+ designated stick-table, and returns its previous value. If the address is not
+ found, an entry is created and 0 is returned. This is typically used as a
+ second ACL in an expression in order to mark a connection when a first ACL
+ was verified.
+
+src_conn_cnt([<table>]) : integer
+ Returns the cumulative number of connections initiated from the current
+ incoming connection's source address in the current proxy's stick-table or in
+ the designated stick-table. If the address is not found, zero is returned.
+ See also sc/sc0/sc1/sc2_conn_cnt.
+
+src_conn_cur([<table>]) : integer
+ Returns the current amount of concurrent connections initiated from the
+ current incoming connection's source address in the current proxy's
+ stick-table or in the designated stick-table. If the address is not found,
+ zero is returned. See also sc/sc0/sc1/sc2_conn_cur.
+
+src_conn_rate([<table>]) : integer
+ Returns the average connection rate from the incoming connection's source
+ address in the current proxy's stick-table or in the designated stick-table,
+ measured in amount of connections over the period configured in the table. If
+ the address is not found, zero is returned. See also sc/sc0/sc1/sc2_conn_rate.
+
+src_get_gpc(<idx>,[<table>]) : integer
+ Returns the value of the General Purpose Counter at the index <idx> of the
+ array associated to the incoming connection's source address in the
+ current proxy's stick-table or in the designated stick-table <table>. <idx>
+ is an integer between 0 and 99.
+ If the address is not found or there is no gpc stored at this index, zero
+ is returned.
+ This fetch applies only to the 'gpc' array data_type (and not on the legacy
+ 'gpc0' nor 'gpc1' data_types).
+ See also sc_get_gpc and src_inc_gpc.
+
+src_get_gpc0([<table>]) : integer
+ Returns the value of the first General Purpose Counter associated to the
+ incoming connection's source address in the current proxy's stick-table or in
+ the designated stick-table. If the address is not found, zero is returned.
+ See also sc/sc0/sc1/sc2_get_gpc0 and src_inc_gpc0.
+
+src_get_gpc1([<table>]) : integer
+ Returns the value of the second General Purpose Counter associated to the
+ incoming connection's source address in the current proxy's stick-table or in
+ the designated stick-table. If the address is not found, zero is returned.
+ See also sc/sc0/sc1/sc2_get_gpc1 and src_inc_gpc1.
+
+src_get_gpt(<idx>[,<table>]) : integer
+ Returns the value of the first General Purpose Tag at the index <idx> of
+ the array associated to the incoming connection's source address in the
+ current proxy's stick-table or in the designated stick-table <table>.
+ <idx> is an integer between 0 and 99.
+ If the address is not found or the GPT is not stored, zero is returned.
+ See also the sc_get_gpt sample fetch keyword.
+
+src_get_gpt0([<table>]) : integer
+ Returns the value of the first General Purpose Tag associated to the
+ incoming connection's source address in the current proxy's stick-table or in
+ the designated stick-table. If the address is not found, zero is returned.
+ See also sc/sc0/sc1/sc2_get_gpt0.
+
+src_gpc_rate(<idx>[,<table>]) : integer
+ Returns the average increment rate of the General Purpose Counter at the
+ index <idx> of the array associated to the incoming connection's
+ source address in the current proxy's stick-table or in the designated
+ stick-table <table>. It reports the frequency which the gpc counter was
+ incremented over the configured period. <idx> is an integer between 0 and 99.
+ Note that the 'gpc_rate' counter must be stored in the stick-table for a
+ value to be returned, as 'gpc' only holds the event count.
+ This fetch applies only to the 'gpc_rate' array data_type (and not to
+ the legacy 'gpc0_rate' nor 'gpc1_rate' data_types).
+ See also sc_gpc_rate, src_get_gpc, and sc_inc_gpc.
+
+src_gpc0_rate([<table>]) : integer
+ Returns the average increment rate of the first General Purpose Counter
+ associated to the incoming connection's source address in the current proxy's
+ stick-table or in the designated stick-table. It reports the frequency
+ which the gpc0 counter was incremented over the configured period. See also
+ sc/sc0/sc1/sc2_gpc0_rate, src_get_gpc0, and sc/sc0/sc1/sc2_inc_gpc0. Note
+ that the "gpc0_rate" counter must be stored in the stick-table for a value to
+ be returned, as "gpc0" only holds the event count.
+
+src_gpc1_rate([<table>]) : integer
+ Returns the average increment rate of the second General Purpose Counter
+ associated to the incoming connection's source address in the current proxy's
+ stick-table or in the designated stick-table. It reports the frequency
+ which the gpc1 counter was incremented over the configured period. See also
+ sc/sc0/sc1/sc2_gpc1_rate, src_get_gpc1, and sc/sc0/sc1/sc2_inc_gpc1. Note
+ that the "gpc1_rate" counter must be stored in the stick-table for a value to
+ be returned, as "gpc1" only holds the event count.
+
+src_http_err_cnt([<table>]) : integer
+ Returns the cumulative number of HTTP errors from the incoming connection's
+ source address in the current proxy's stick-table or in the designated
+ stick-table. This includes the both request errors and 4xx error responses.
+ See also sc/sc0/sc1/sc2_http_err_cnt. If the address is not found, zero is
+ returned.
+
+src_http_err_rate([<table>]) : integer
+ Returns the average rate of HTTP errors from the incoming connection's source
+ address in the current proxy's stick-table or in the designated stick-table,
+ measured in amount of errors over the period configured in the table. This
+ includes the both request errors and 4xx error responses. If the address is
+ not found, zero is returned. See also sc/sc0/sc1/sc2_http_err_rate.
+
+src_http_fail_cnt([<table>]) : integer
+ Returns the cumulative number of HTTP response failures triggered by the
+ incoming connection's source address in the current proxy's stick-table or in
+ the designated stick-table. This includes the both response errors and 5xx
+ status codes other than 501 and 505. See also sc/sc0/sc1/sc2_http_fail_cnt.
+ If the address is not found, zero is returned.
+
+src_http_fail_rate([<table>]) : integer
+ Returns the average rate of HTTP response failures triggered by the incoming
+ connection's source address in the current proxy's stick-table or in the
+ designated stick-table, measured in amount of failures over the period
+ configured in the table. This includes the both response errors and 5xx
+ status codes other than 501 and 505. If the address is not found, zero is
+ returned. See also sc/sc0/sc1/sc2_http_fail_rate.
+
+src_http_req_cnt([<table>]) : integer
+ Returns the cumulative number of HTTP requests from the incoming connection's
+ source address in the current proxy's stick-table or in the designated stick-
+ table. This includes every started request, valid or not. If the address is
+ not found, zero is returned. See also sc/sc0/sc1/sc2_http_req_cnt.
+
+src_http_req_rate([<table>]) : integer
+ Returns the average rate of HTTP requests from the incoming connection's
+ source address in the current proxy's stick-table or in the designated stick-
+ table, measured in amount of requests over the period configured in the
+ table. This includes every started request, valid or not. If the address is
+ not found, zero is returned. See also sc/sc0/sc1/sc2_http_req_rate.
+
+src_inc_gpc(<idx>,[<table>]) : integer
+ Increments the General Purpose Counter at index <idx> of the array
+ associated to the incoming connection's source address in the current proxy's
+ stick-table or in the designated stick-table <table>, and returns its new
+ value. <idx> is an integer between 0 and 99.
+ If the address is not found, an entry is created and 1 is returned.
+ This fetch applies only to the 'gpc' array data_type (and not to the legacy
+ 'gpc0' nor 'gpc1' data_types).
+ See also sc_inc_gpc.
+
+src_inc_gpc0([<table>]) : integer
+ Increments the first General Purpose Counter associated to the incoming
+ connection's source address in the current proxy's stick-table or in the
+ designated stick-table, and returns its new value. If the address is not
+ found, an entry is created and 1 is returned. See also sc0/sc2/sc2_inc_gpc0.
+ This is typically used as a second ACL in an expression in order to mark a
+ connection when a first ACL was verified :
+
+ Example:
+ acl abuse src_http_req_rate gt 10
+ acl kill src_inc_gpc0 gt 0
+ tcp-request connection reject if abuse kill
+
+src_inc_gpc1([<table>]) : integer
+ Increments the second General Purpose Counter associated to the incoming
+ connection's source address in the current proxy's stick-table or in the
+ designated stick-table, and returns its new value. If the address is not
+ found, an entry is created and 1 is returned. See also sc0/sc2/sc2_inc_gpc1.
+ This is typically used as a second ACL in an expression in order to mark a
+ connection when a first ACL was verified.
+
+src_is_local : boolean
+ Returns true if the source address of the incoming connection is local to the
+ system, or false if the address doesn't exist on the system, meaning that it
+ comes from a remote machine. Note that UNIX addresses are considered local.
+ It can be useful to apply certain access restrictions based on where the
+ client comes from (e.g. require auth or https for remote machines). Please
+ note that the check involves a few system calls, so it's better to do it only
+ once per connection.
+
+src_kbytes_in([<table>]) : integer
+ Returns the total amount of data received from the incoming connection's
+ source address in the current proxy's stick-table or in the designated
+ stick-table, measured in kilobytes. If the address is not found, zero is
+ returned. The test is currently performed on 32-bit integers, which limits
+ values to 4 terabytes. See also sc/sc0/sc1/sc2_kbytes_in.
+
+src_kbytes_out([<table>]) : integer
+ Returns the total amount of data sent to the incoming connection's source
+ address in the current proxy's stick-table or in the designated stick-table,
+ measured in kilobytes. If the address is not found, zero is returned. The
+ test is currently performed on 32-bit integers, which limits values to 4
+ terabytes. See also sc/sc0/sc1/sc2_kbytes_out.
+
+src_port : integer
+ Returns an integer value corresponding to the TCP source port of the
+ connection on the client side, which is the port the client connected
+ from. Any tcp/http rules may alter this address. Usage of this function is
+ very limited as modern protocols do not care much about source ports
+ nowadays.
+
+src_sess_cnt([<table>]) : integer
+ Returns the cumulative number of connections initiated from the incoming
+ connection's source IPv4 address in the current proxy's stick-table or in the
+ designated stick-table, that were transformed into sessions, which means that
+ they were accepted by "tcp-request" rules. If the address is not found, zero
+ is returned. See also sc/sc0/sc1/sc2_sess_cnt.
+
+src_sess_rate([<table>]) : integer
+ Returns the average session rate from the incoming connection's source
+ address in the current proxy's stick-table or in the designated stick-table,
+ measured in amount of sessions over the period configured in the table. A
+ session is a connection that went past the early "tcp-request" rules. If the
+ address is not found, zero is returned. See also sc/sc0/sc1/sc2_sess_rate.
+
+src_updt_conn_cnt([<table>]) : integer
+ Creates or updates the entry associated to the incoming connection's source
+ address in the current proxy's stick-table or in the designated stick-table.
+ This table must be configured to store the "conn_cnt" data type, otherwise
+ the match will be ignored. The current count is incremented by one, and the
+ expiration timer refreshed. The updated count is returned, so this match
+ can't return zero. This was used to reject service abusers based on their
+ source address. Note: it is recommended to use the more complete "track-sc*"
+ actions in "tcp-request" rules instead.
+
+ Example :
+ # This frontend limits incoming SSH connections to 3 per 10 second for
+ # each source address, and rejects excess connections until a 10 second
+ # silence is observed. At most 20 addresses are tracked.
+ listen ssh
+ bind :22
+ mode tcp
+ maxconn 100
+ stick-table type ip size 20 expire 10s store conn_cnt
+ tcp-request content reject if { src_updt_conn_cnt gt 3 }
+ server local 127.0.0.1:22
+
+srv_id : integer
+ Returns an integer containing the server's id when processing the response.
+ While it's almost only used with ACLs, it may be used for logging or
+ debugging. It can also be used in a tcp-check or an http-check ruleset.
+
+srv_name : string
+ Returns a string containing the server's name when processing the response.
+ While it's almost only used with ACLs, it may be used for logging or
+ debugging. It can also be used in a tcp-check or an http-check ruleset.
+
+txn.conn_retries : integer
+ Returns the the number of connection retries experienced by this stream when
+ trying to connect to the server. This value is subject to change while the
+ connection is not fully established. For HTTP connections, the value may be
+ affected by L7 retries.
+
+7.3.4. Fetching samples at Layer 5
+----------------------------------
+
+The layer 5 usually describes just the session layer which in HAProxy is
+closest to the session once all the connection handshakes are finished, but
+when no content is yet made available. The fetch methods described here are
+usable as low as the "tcp-request content" rule sets unless they require some
+future information. Those generally include the results of SSL negotiations.
+
+Summary of sample fetch methods in this section and their respective types:
+
+ keyword output type
+-------------------------------------------------+-------------
+51d.all(<prop>[,<prop>*]) string
+ssl_bc boolean
+ssl_bc_alg_keysize integer
+ssl_bc_alpn string
+ssl_bc_cipher string
+ssl_bc_client_random binary
+ssl_bc_curve string
+ssl_bc_err integer
+ssl_bc_err_str string
+ssl_bc_is_resumed boolean
+ssl_bc_npn string
+ssl_bc_protocol string
+ssl_bc_unique_id binary
+ssl_bc_server_random binary
+ssl_bc_session_id binary
+ssl_bc_session_key binary
+ssl_bc_use_keysize integer
+ssl_c_ca_err integer
+ssl_c_ca_err_depth integer
+ssl_c_chain_der binary
+ssl_c_der binary
+ssl_c_err integer
+ssl_c_i_dn([<entry>[,<occ>[,<format>]]]) string
+ssl_c_key_alg string
+ssl_c_notafter string
+ssl_c_notbefore string
+ssl_c_r_dn([<entry>[,<occ>[,<format>]]]) string
+ssl_c_s_dn([<entry>[,<occ>[,<format>]]]) string
+ssl_c_serial binary
+ssl_c_sha1 binary
+ssl_c_sig_alg string
+ssl_c_used boolean
+ssl_c_verify integer
+ssl_c_version integer
+ssl_f_der binary
+ssl_f_i_dn([<entry>[,<occ>[,<format>]]]) string
+ssl_f_key_alg string
+ssl_f_notafter string
+ssl_f_notbefore string
+ssl_f_s_dn([<entry>[,<occ>[,<format>]]]) string
+ssl_f_serial binary
+ssl_f_sha1 binary
+ssl_f_sig_alg string
+ssl_f_version integer
+ssl_fc boolean
+ssl_fc_alg_keysize integer
+ssl_fc_alpn string
+ssl_fc_cipher string
+ssl_fc_cipherlist_bin([<filter_option>]) binary
+ssl_fc_cipherlist_hex([<filter_option>]) string
+ssl_fc_cipherlist_str([<filter_option>]) string
+ssl_fc_cipherlist_xxh integer
+ssl_fc_curve string
+ssl_fc_ecformats_bin binary
+ssl_fc_eclist_bin([<filter_option>]) binary
+ssl_fc_extlist_bin([<filter_option>]) binary
+ssl_fc_client_random binary
+ssl_fc_client_early_traffic_secret string
+ssl_fc_client_handshake_traffic_secret string
+ssl_fc_client_traffic_secret_0 string
+ssl_fc_exporter_secret string
+ssl_fc_early_exporter_secret string
+ssl_fc_err integer
+ssl_fc_err_str string
+ssl_fc_has_crt boolean
+ssl_fc_has_early boolean
+ssl_fc_has_sni boolean
+ssl_fc_is_resumed boolean
+ssl_fc_npn string
+ssl_fc_protocol string
+ssl_fc_protocol_hello_id integer
+ssl_fc_unique_id binary
+ssl_fc_server_handshake_traffic_secret string
+ssl_fc_server_traffic_secret_0 string
+ssl_fc_server_random binary
+ssl_fc_session_id binary
+ssl_fc_session_key binary
+ssl_fc_sni string
+ssl_fc_use_keysize integer
+ssl_s_der binary
+ssl_s_chain_der binary
+ssl_s_key_alg string
+ssl_s_notafter string
+ssl_s_notbefore string
+ssl_s_i_dn([<entry>[,<occ>[,<format>]]]) string
+ssl_s_s_dn([<entry>[,<occ>[,<format>]]]) string
+ssl_s_serial binary
+ssl_s_sha1 binary
+ssl_s_sig_alg string
+ssl_s_version integer
+txn.timer.user integer
+-------------------------------------------------+-------------
+
+Detailed list:
+
+51d.all(<prop>[,<prop>*]) : string
+ Returns values for the properties requested as a string, where values are
+ separated by the delimiter specified with "51degrees-property-separator".
+ The device is identified using all the important HTTP headers from the
+ request. The function can be passed up to five property names, and if a
+ property name can't be found, the value "NoData" is returned.
+
+ Example :
+ # Here the header "X-51D-DeviceTypeMobileTablet" is added to the request
+ # containing the three properties requested using all relevant headers from
+ # the request.
+ frontend http-in
+ bind *:8081
+ default_backend servers
+ http-request set-header X-51D-DeviceTypeMobileTablet \
+ %[51d.all(DeviceType,IsMobile,IsTablet)]
+
+ssl_bc : boolean
+ Returns true when the back connection was made via an SSL/TLS transport
+ layer and is locally deciphered. This means the outgoing connection was made
+ to a server with the "ssl" option. It can be used in a tcp-check or an
+ http-check ruleset.
+
+ssl_bc_alg_keysize : integer
+ Returns the symmetric cipher key size supported in bits when the outgoing
+ connection was made over an SSL/TLS transport layer. It can be used in a
+ tcp-check or an http-check ruleset.
+
+ssl_bc_alpn : string
+ This extracts the Application Layer Protocol Negotiation field from an
+ outgoing connection made via a TLS transport layer.
+ The result is a string containing the protocol name negotiated with the
+ server. The SSL library must have been built with support for TLS
+ extensions enabled (check haproxy -vv). Note that the TLS ALPN extension is
+ not advertised unless the "alpn" keyword on the "server" line specifies a
+ protocol list. Also, nothing forces the server to pick a protocol from this
+ list, any other one may be requested. The TLS ALPN extension is meant to
+ replace the TLS NPN extension. See also "ssl_bc_npn". It can be used in a
+ tcp-check or an http-check ruleset.
+
+ssl_bc_cipher : string
+ Returns the name of the used cipher when the outgoing connection was made
+ over an SSL/TLS transport layer. It can be used in a tcp-check or an
+ http-check ruleset.
+
+ssl_bc_client_random : binary
+ Returns the client random of the back connection when the incoming connection
+ was made over an SSL/TLS transport layer. It is useful to to decrypt traffic
+ sent using ephemeral ciphers. This requires OpenSSL >= 1.1.0, or BoringSSL.
+ It can be used in a tcp-check or an http-check ruleset.
+
+ssl_bc_curve : string
+ Returns the name of the curve used in the key agreement when the outgoing
+ connection was made over an SSL/TLS transport layer. This requires
+ OpenSSL >= 3.0.0.
+
+ssl_bc_err : integer
+ When the outgoing connection was made over an SSL/TLS transport layer,
+ returns the ID of the last error of the first error stack raised on the
+ backend side. It can raise handshake errors as well as other read or write
+ errors occurring during the connection's lifetime. In order to get a text
+ description of this error code, you can either use the "ssl_bc_err_str"
+ sample fetch or use the "openssl errstr" command (which takes an error code
+ in hexadecimal representation as parameter). Please refer to your SSL
+ library's documentation to find the exhaustive list of error codes.
+
+ssl_bc_err_str : string
+ When the outgoing connection was made over an SSL/TLS transport layer,
+ returns a string representation of the last error of the first error stack
+ that was raised on the connection from the backend's perspective. See also
+ "ssl_fc_err".
+
+ssl_bc_is_resumed : boolean
+ Returns true when the back connection was made over an SSL/TLS transport
+ layer and the newly created SSL session was resumed using a cached
+ session or a TLS ticket. It can be used in a tcp-check or an http-check
+ ruleset.
+
+ssl_bc_npn : string
+ This extracts the Next Protocol Negotiation field from an outgoing connection
+ made via a TLS transport layer. The result is a string containing the
+ protocol name negotiated with the server . The SSL library must have been
+ built with support for TLS extensions enabled (check haproxy -vv). Note that
+ the TLS NPN extension is not advertised unless the "npn" keyword on the
+ "server" line specifies a protocol list. Also, nothing forces the server to
+ pick a protocol from this list, any other one may be used. Please note that
+ the TLS NPN extension was replaced with ALPN. It can be used in a tcp-check
+ or an http-check ruleset.
+
+ssl_bc_protocol : string
+ Returns the name of the used protocol when the outgoing connection was made
+ over an SSL/TLS transport layer. It can be used in a tcp-check or an
+ http-check ruleset.
+
+ssl_bc_unique_id : binary
+ When the outgoing connection was made over an SSL/TLS transport layer,
+ returns the TLS unique ID as defined in RFC5929 section 3. The unique id
+ can be encoded to base64 using the converter: "ssl_bc_unique_id,base64". It
+ can be used in a tcp-check or an http-check ruleset.
+
+ssl_bc_server_random : binary
+ Returns the server random of the back connection when the incoming connection
+ was made over an SSL/TLS transport layer. It is useful to to decrypt traffic
+ sent using ephemeral ciphers. This requires OpenSSL >= 1.1.0, or BoringSSL.
+ It can be used in a tcp-check or an http-check ruleset.
+
+ssl_bc_session_id : binary
+ Returns the SSL ID of the back connection when the outgoing connection was
+ made over an SSL/TLS transport layer. It is useful to log if we want to know
+ if session was reused or not. It can be used in a tcp-check or an http-check
+ ruleset.
+
+ssl_bc_session_key : binary
+ Returns the SSL session master key of the back connection when the outgoing
+ connection was made over an SSL/TLS transport layer. It is useful to decrypt
+ traffic sent using ephemeral ciphers. This requires OpenSSL >= 1.1.0, or
+ BoringSSL. It can be used in a tcp-check or an http-check ruleset.
+
+ssl_bc_use_keysize : integer
+ Returns the symmetric cipher key size used in bits when the outgoing
+ connection was made over an SSL/TLS transport layer. It can be used in a
+ tcp-check or an http-check ruleset.
+
+ssl_c_ca_err : integer
+ When the incoming connection was made over an SSL/TLS transport layer,
+ returns the ID of the first error detected during verification of the client
+ certificate at depth > 0, or 0 if no error was encountered during this
+ verification process. Please refer to your SSL library's documentation to
+ find the exhaustive list of error codes.
+
+ssl_c_ca_err_depth : integer
+ When the incoming connection was made over an SSL/TLS transport layer,
+ returns the depth in the CA chain of the first error detected during the
+ verification of the client certificate. If no error is encountered, 0 is
+ returned.
+
+ssl_c_chain_der : binary
+ Returns the DER formatted chain certificate presented by the client when the
+ incoming connection was made over an SSL/TLS transport layer. When used for
+ an ACL, the value(s) to match against can be passed in hexadecimal form. One
+ can parse the result with any lib accepting ASN.1 DER data. It currently
+ does not support resumed sessions.
+
+ssl_c_der : binary
+ Returns the DER formatted certificate presented by the client when the
+ incoming connection was made over an SSL/TLS transport layer. When used for
+ an ACL, the value(s) to match against can be passed in hexadecimal form.
+
+ssl_c_err : integer
+ When the incoming connection was made over an SSL/TLS transport layer,
+ returns the ID of the first error detected during verification at depth 0, or
+ 0 if no error was encountered during this verification process. Please refer
+ to your SSL library's documentation to find the exhaustive list of error
+ codes.
+
+ssl_c_i_dn([<entry>[,<occ>[,<format>]]]) : string
+ When the incoming connection was made over an SSL/TLS transport layer,
+ returns the full distinguished name of the issuer of the certificate
+ presented by the client when no <entry> is specified, or the value of the
+ first given entry found from the beginning of the DN. If a positive/negative
+ occurrence number is specified as the optional second argument, it returns
+ the value of the nth given entry value from the beginning/end of the DN.
+ For instance, "ssl_c_i_dn(OU,2)" the second organization unit, and
+ "ssl_c_i_dn(CN)" retrieves the common name.
+ The <format> parameter allows you to receive the DN suitable for
+ consumption by different protocols. Currently supported is rfc2253 for
+ LDAP v3.
+ If you'd like to modify the format only you can specify an empty string
+ and zero for the first two parameters. Example: ssl_c_i_dn(,0,rfc2253)
+
+ssl_c_key_alg : string
+ Returns the name of the algorithm used to generate the key of the certificate
+ presented by the client when the incoming connection was made over an SSL/TLS
+ transport layer.
+
+ssl_c_notafter : string
+ Returns the end date presented by the client as a formatted string
+ YYMMDDhhmmss[Z] when the incoming connection was made over an SSL/TLS
+ transport layer.
+
+ssl_c_notbefore : string
+ Returns the start date presented by the client as a formatted string
+ YYMMDDhhmmss[Z] when the incoming connection was made over an SSL/TLS
+ transport layer.
+
+ssl_c_r_dn([<entry>[,<occ>[,<format>]]]) : string
+ When the incoming connection was made over an SSL/TLS transport layer, and is
+ successfully validated with the configured ca-file, returns the full
+ distinguished name of the root CA of the certificate presented by the client
+ when no <entry> is specified, or the value of the first given entry found from
+ the beginning of the DN. If a positive/negative occurrence number is specified
+ as the optional second argument, it returns the value of the nth given entry
+ value from the beginning/end of the DN. For instance, "ssl_c_r_dn(OU,2)" the
+ second organization unit, and "ssl_c_r_dn(CN)" retrieves the common name. The
+ <format> parameter allows you to receive the DN suitable for consumption by
+ different protocols. Currently supported is rfc2253 for LDAP v3. If you'd like
+ to modify the format only you can specify an empty string and zero for the
+ first two parameters. Example: ssl_c_r_dn(,0,rfc2253)
+
+ssl_c_s_dn([<entry>[,<occ>[,<format>]]]) : string
+ When the incoming connection was made over an SSL/TLS transport layer,
+ returns the full distinguished name of the subject of the certificate
+ presented by the client when no <entry> is specified, or the value of the
+ first given entry found from the beginning of the DN. If a positive/negative
+ occurrence number is specified as the optional second argument, it returns
+ the value of the nth given entry value from the beginning/end of the DN.
+ For instance, "ssl_c_s_dn(OU,2)" the second organization unit, and
+ "ssl_c_s_dn(CN)" retrieves the common name.
+ The <format> parameter allows you to receive the DN suitable for
+ consumption by different protocols. Currently supported is rfc2253 for
+ LDAP v3.
+ If you'd like to modify the format only you can specify an empty string
+ and zero for the first two parameters. Example: ssl_c_s_dn(,0,rfc2253)
+
+ssl_c_serial : binary
+ Returns the serial of the certificate presented by the client when the
+ incoming connection was made over an SSL/TLS transport layer. When used for
+ an ACL, the value(s) to match against can be passed in hexadecimal form.
+
+ssl_c_sha1 : binary
+ Returns the SHA-1 fingerprint of the certificate presented by the client when
+ the incoming connection was made over an SSL/TLS transport layer. This can be
+ used to stick a client to a server, or to pass this information to a server.
+ Note that the output is binary, so if you want to pass that signature to the
+ server, you need to encode it in hex or base64, such as in the example below:
+
+ Example:
+ http-request set-header X-SSL-Client-SHA1 %[ssl_c_sha1,hex]
+
+ssl_c_sig_alg : string
+ Returns the name of the algorithm used to sign the certificate presented by
+ the client when the incoming connection was made over an SSL/TLS transport
+ layer.
+
+ssl_c_used : boolean
+ Returns true if current SSL session uses a client certificate even if current
+ connection uses SSL session resumption. See also "ssl_fc_has_crt".
+
+ssl_c_verify : integer
+ Returns the verify result error ID when the incoming connection was made over
+ an SSL/TLS transport layer, otherwise zero if no error is encountered. Please
+ refer to your SSL library's documentation for an exhaustive list of error
+ codes.
+
+ssl_c_version : integer
+ Returns the version of the certificate presented by the client when the
+ incoming connection was made over an SSL/TLS transport layer.
+
+ssl_f_der : binary
+ Returns the DER formatted certificate presented by the frontend when the
+ incoming connection was made over an SSL/TLS transport layer. When used for
+ an ACL, the value(s) to match against can be passed in hexadecimal form.
+
+ssl_f_i_dn([<entry>[,<occ>[,<format>]]]) : string
+ When the incoming connection was made over an SSL/TLS transport layer,
+ returns the full distinguished name of the issuer of the certificate
+ presented by the frontend when no <entry> is specified, or the value of the
+ first given entry found from the beginning of the DN. If a positive/negative
+ occurrence number is specified as the optional second argument, it returns
+ the value of the nth given entry value from the beginning/end of the DN.
+ For instance, "ssl_f_i_dn(OU,2)" the second organization unit, and
+ "ssl_f_i_dn(CN)" retrieves the common name.
+ The <format> parameter allows you to receive the DN suitable for
+ consumption by different protocols. Currently supported is rfc2253 for
+ LDAP v3.
+ If you'd like to modify the format only you can specify an empty string
+ and zero for the first two parameters. Example: ssl_f_i_dn(,0,rfc2253)
+
+ssl_f_key_alg : string
+ Returns the name of the algorithm used to generate the key of the certificate
+ presented by the frontend when the incoming connection was made over an
+ SSL/TLS transport layer.
+
+ssl_f_notafter : string
+ Returns the end date presented by the frontend as a formatted string
+ YYMMDDhhmmss[Z] when the incoming connection was made over an SSL/TLS
+ transport layer.
+
+ssl_f_notbefore : string
+ Returns the start date presented by the frontend as a formatted string
+ YYMMDDhhmmss[Z] when the incoming connection was made over an SSL/TLS
+ transport layer.
+
+ssl_f_s_dn([<entry>[,<occ>[,<format>]]]) : string
+ When the incoming connection was made over an SSL/TLS transport layer,
+ returns the full distinguished name of the subject of the certificate
+ presented by the frontend when no <entry> is specified, or the value of the
+ first given entry found from the beginning of the DN. If a positive/negative
+ occurrence number is specified as the optional second argument, it returns
+ the value of the nth given entry value from the beginning/end of the DN.
+ For instance, "ssl_f_s_dn(OU,2)" the second organization unit, and
+ "ssl_f_s_dn(CN)" retrieves the common name.
+ The <format> parameter allows you to receive the DN suitable for
+ consumption by different protocols. Currently supported is rfc2253 for
+ LDAP v3.
+ If you'd like to modify the format only you can specify an empty string
+ and zero for the first two parameters. Example: ssl_f_s_dn(,0,rfc2253)
+
+ssl_f_serial : binary
+ Returns the serial of the certificate presented by the frontend when the
+ incoming connection was made over an SSL/TLS transport layer. When used for
+ an ACL, the value(s) to match against can be passed in hexadecimal form.
+
+ssl_f_sha1 : binary
+ Returns the SHA-1 fingerprint of the certificate presented by the frontend
+ when the incoming connection was made over an SSL/TLS transport layer. This
+ can be used to know which certificate was chosen using SNI.
+
+ssl_f_sig_alg : string
+ Returns the name of the algorithm used to sign the certificate presented by
+ the frontend when the incoming connection was made over an SSL/TLS transport
+ layer.
+
+ssl_f_version : integer
+ Returns the version of the certificate presented by the frontend when the
+ incoming connection was made over an SSL/TLS transport layer.
+
+ssl_fc : boolean
+ Returns true when the front connection was made via an SSL/TLS transport
+ layer and is locally deciphered. This means it has matched a socket declared
+ with a "bind" line having the "ssl" option.
+
+ Example :
+ # This passes "X-Proto: https" to servers when client connects over SSL
+ listen http-https
+ bind :80
+ bind :443 ssl crt /etc/haproxy.pem
+ http-request add-header X-Proto https if { ssl_fc }
+
+ssl_fc_alg_keysize : integer
+ Returns the symmetric cipher key size supported in bits when the incoming
+ connection was made over an SSL/TLS transport layer.
+
+ssl_fc_alpn : string
+ This extracts the Application Layer Protocol Negotiation field from an
+ incoming connection made via a TLS transport layer and locally deciphered by
+ HAProxy. The result is a string containing the protocol name advertised by
+ the client. The SSL library must have been built with support for TLS
+ extensions enabled (check haproxy -vv). Note that the TLS ALPN extension is
+ not advertised unless the "alpn" keyword on the "bind" line specifies a
+ protocol list. Also, nothing forces the client to pick a protocol from this
+ list, any other one may be requested. The TLS ALPN extension is meant to
+ replace the TLS NPN extension. See also "ssl_fc_npn".
+
+ssl_fc_cipher : string
+ Returns the name of the used cipher when the incoming connection was made
+ over an SSL/TLS transport layer.
+
+ssl_fc_cipherlist_bin([<filter_option>]) : binary
+ Returns the binary form of the client hello cipher list. The maximum
+ returned value length is limited by the shared capture buffer size
+ controlled by "tune.ssl.capture-buffer-size" setting. Setting
+ <filter_option> allows to filter returned data. Accepted values:
+ 0 : return the full list of ciphers (default)
+ 1 : exclude GREASE (RFC8701) values from the output
+
+ Example:
+ http-request set-header X-SSL-JA3 %[ssl_fc_protocol_hello_id],\
+ %[ssl_fc_cipherlist_bin(1),be2dec(-,2)],\
+ %[ssl_fc_extlist_bin(1),be2dec(-,2)],\
+ %[ssl_fc_eclist_bin(1),be2dec(-,2)],\
+ %[ssl_fc_ecformats_bin,be2dec(-,1)]
+ acl is_malware req.fhdr(x-ssl-ja3),digest(md5),hex \
+ -f /path/to/file/with/malware-ja3.lst
+ http-request set-header X-Malware True if is_malware
+ http-request set-header X-Malware False if !is_malware
+
+ssl_fc_cipherlist_hex([<filter_option>]) : string
+ Returns the binary form of the client hello cipher list encoded as
+ hexadecimal. The maximum returned value length is limited by the shared
+ capture buffer size controlled by "tune.ssl.capture-buffer-size" setting.
+ Setting <filter_option> allows to filter returned data. Accepted values:
+ 0 : return the full list of ciphers (default)
+ 1 : exclude GREASE (RFC8701) values from the output
+
+ssl_fc_cipherlist_str([<filter_option>]) : string
+ Returns the decoded text form of the client hello cipher list. The maximum
+ returned value length is limited by the shared capture buffer size
+ controlled by "tune.ssl.capture-buffer-size" setting. Setting
+ <filter_option> allows to filter returned data. Accepted values:
+ 0 : return the full list of ciphers (default)
+ 1 : exclude GREASE (RFC8701) values from the output
+ Note that this sample-fetch is only available with OpenSSL >= 1.0.2. If the
+ function is not enabled, this sample-fetch returns the hash like
+ "ssl_fc_cipherlist_xxh".
+
+ssl_fc_cipherlist_xxh : integer
+ Returns a xxh64 of the cipher list. This hash can return only if the value
+ "tune.ssl.capture-buffer-size" is set greater than 0, however the hash take
+ into account all the data of the cipher list.
+
+ssl_fc_curve : string
+ Returns the name of the curve used in the key agreement when the incoming
+ connection was made over an SSL/TLS transport layer. This requires
+ OpenSSL >= 3.0.0.
+
+ssl_fc_ecformats_bin : binary
+ Return the binary form of the client hello supported elliptic curve point
+ formats. The maximum returned value length is limited by the shared capture
+ buffer size controlled by "tune.ssl.capture-buffer-size" setting.
+
+ Example:
+ http-request set-header X-SSL-JA3 %[ssl_fc_protocol_hello_id],\
+ %[ssl_fc_cipherlist_bin(1),be2dec(-,2)],\
+ %[ssl_fc_extlist_bin(1),be2dec(-,2)],\
+ %[ssl_fc_eclist_bin(1),be2dec(-,2)],\
+ %[ssl_fc_ecformats_bin,be2dec(-,1)]
+ acl is_malware req.fhdr(x-ssl-ja3),digest(md5),hex \
+ -f /path/to/file/with/malware-ja3.lst
+ http-request set-header X-Malware True if is_malware
+ http-request set-header X-Malware False if !is_malware
+
+ssl_fc_eclist_bin([<filter_option>]) : binary
+ Returns the binary form of the client hello supported elliptic curves. The
+ maximum returned value length is limited by the shared capture buffer size
+ controlled by "tune.ssl.capture-buffer-size" setting. Setting
+ <filter_option> allows to filter returned data. Accepted values:
+ 0 : return the full list of supported elliptic curves (default)
+ 1 : exclude GREASE (RFC8701) values from the output
+
+ Example:
+ http-request set-header X-SSL-JA3 %[ssl_fc_protocol_hello_id],\
+ %[ssl_fc_cipherlist_bin(1),be2dec(-,2)],\
+ %[ssl_fc_extlist_bin(1),be2dec(-,2)],\
+ %[ssl_fc_eclist_bin(1),be2dec(-,2)],\
+ %[ssl_fc_ecformats_bin,be2dec(-,1)]
+ acl is_malware req.fhdr(x-ssl-ja3),digest(md5),hex \
+ -f /path/to/file/with/malware-ja3.lst
+ http-request set-header X-Malware True if is_malware
+ http-request set-header X-Malware False if !is_malware
+
+ssl_fc_extlist_bin([<filter_option>]) : binary
+ Returns the binary form of the client hello extension list. The maximum
+ returned value length is limited by the shared capture buffer size
+ controlled by "tune.ssl.capture-buffer-size" setting. Setting
+ <filter_option> allows to filter returned data. Accepted values:
+ 0 : return the full list of extensions (default)
+ 1 : exclude GREASE (RFC8701) values from the output
+
+ Example:
+ http-request set-header X-SSL-JA3 %[ssl_fc_protocol_hello_id],\
+ %[ssl_fc_cipherlist_bin(1),be2dec(-,2)],\
+ %[ssl_fc_extlist_bin(1),be2dec(-,2)],\
+ %[ssl_fc_eclist_bin(1),be2dec(-,2)],\
+ %[ssl_fc_ecformats_bin,be2dec(-,1)]
+ acl is_malware req.fhdr(x-ssl-ja3),digest(md5),hex \
+ -f /path/to/file/with/malware-ja3.lst
+ http-request set-header X-Malware True if is_malware
+ http-request set-header X-Malware False if !is_malware
+
+ssl_fc_client_random : binary
+ Returns the client random of the front connection when the incoming connection
+ was made over an SSL/TLS transport layer. It is useful to to decrypt traffic
+ sent using ephemeral ciphers. This requires OpenSSL >= 1.1.0, or BoringSSL.
+
+ssl_fc_client_early_traffic_secret : string
+ Return the CLIENT_EARLY_TRAFFIC_SECRET as an hexadecimal string for the
+ front connection when the incoming connection was made over a TLS 1.3
+ transport layer.
+ Require OpenSSL >= 1.1.1. This is one of the keys dumped by the OpenSSL
+ keylog callback to generate the SSLKEYLOGFILE. The SSL Key logging must be
+ activated with "tune.ssl.keylog on" in the global section. See also
+ "tune.ssl.keylog"
+
+ssl_fc_client_handshake_traffic_secret : string
+ Return the CLIENT_HANDSHAKE_TRAFFIC_SECRET as an hexadecimal string for the
+ front connection when the incoming connection was made over a TLS 1.3
+ transport layer.
+ Require OpenSSL >= 1.1.1. This is one of the keys dumped by the OpenSSL
+ keylog callback to generate the SSLKEYLOGFILE. The SSL Key logging must be
+ activated with "tune.ssl.keylog on" in the global section. See also
+ "tune.ssl.keylog"
+
+ssl_fc_client_traffic_secret_0 : string
+ Return the CLIENT_TRAFFIC_SECRET_0 as an hexadecimal string for the
+ front connection when the incoming connection was made over a TLS 1.3
+ transport layer.
+ Require OpenSSL >= 1.1.1. This is one of the keys dumped by the OpenSSL
+ keylog callback to generate the SSLKEYLOGFILE. The SSL Key logging must be
+ activated with "tune.ssl.keylog on" in the global section. See also
+ "tune.ssl.keylog"
+
+ssl_fc_exporter_secret : string
+ Return the EXPORTER_SECRET as an hexadecimal string for the
+ front connection when the incoming connection was made over a TLS 1.3
+ transport layer.
+ Require OpenSSL >= 1.1.1. This is one of the keys dumped by the OpenSSL
+ keylog callback to generate the SSLKEYLOGFILE. The SSL Key logging must be
+ activated with "tune.ssl.keylog on" in the global section. See also
+ "tune.ssl.keylog"
+
+ssl_fc_early_exporter_secret : string
+ Return the EARLY_EXPORTER_SECRET as an hexadecimal string for the
+ front connection when the incoming connection was made over an TLS 1.3
+ transport layer.
+ Require OpenSSL >= 1.1.1. This is one of the keys dumped by the OpenSSL
+ keylog callback to generate the SSLKEYLOGFILE. The SSL Key logging must be
+ activated with "tune.ssl.keylog on" in the global section. See also
+ "tune.ssl.keylog"
+
+ssl_fc_err : integer
+ When the incoming connection was made over an SSL/TLS transport layer,
+ returns the ID of the last error of the first error stack raised on the
+ frontend side, or 0 if no error was encountered. It can be used to identify
+ handshake related errors other than verify ones (such as cipher mismatch), as
+ well as other read or write errors occurring during the connection's
+ lifetime. Any error happening during the client's certificate verification
+ process will not be raised through this fetch but via the existing
+ "ssl_c_err", "ssl_c_ca_err" and "ssl_c_ca_err_depth" fetches. In order to get
+ a text description of this error code, you can either use the
+ "ssl_fc_err_str" sample fetch or use the "openssl errstr" command (which
+ takes an error code in hexadecimal representation as parameter). Please refer
+ to your SSL library's documentation to find the exhaustive list of error
+ codes.
+
+ssl_fc_err_str : string
+ When the incoming connection was made over an SSL/TLS transport layer,
+ returns a string representation of the last error of the first error stack
+ that was raised on the frontend side. Any error happening during the client's
+ certificate verification process will not be raised through this fetch. See
+ also "ssl_fc_err".
+
+ssl_fc_has_crt : boolean
+ Returns true if a client certificate is present in an incoming connection over
+ SSL/TLS transport layer. Useful if 'verify' statement is set to 'optional'.
+ Note: on SSL session resumption with Session ID or TLS ticket, client
+ certificate is not present in the current connection but may be retrieved
+ from the cache or the ticket. So prefer "ssl_c_used" if you want to check if
+ current SSL session uses a client certificate.
+
+ssl_fc_has_early : boolean
+ Returns true if early data were sent, and the handshake didn't happen yet. As
+ it has security implications, it is useful to be able to refuse those, or
+ wait until the handshake happened.
+
+ssl_fc_has_sni : boolean
+ This checks for the presence of a Server Name Indication TLS extension (SNI)
+ in an incoming connection was made over an SSL/TLS transport layer. Returns
+ true when the incoming connection presents a TLS SNI field. This requires
+ that the SSL library is built with support for TLS extensions enabled (check
+ haproxy -vv).
+
+ssl_fc_is_resumed : boolean
+ Returns true if the SSL/TLS session has been resumed through the use of
+ SSL session cache or TLS tickets on an incoming connection over an SSL/TLS
+ transport layer.
+
+ssl_fc_npn : string
+ This extracts the Next Protocol Negotiation field from an incoming connection
+ made via a TLS transport layer and locally deciphered by HAProxy. The result
+ is a string containing the protocol name advertised by the client. The SSL
+ library must have been built with support for TLS extensions enabled (check
+ haproxy -vv). Note that the TLS NPN extension is not advertised unless the
+ "npn" keyword on the "bind" line specifies a protocol list. Also, nothing
+ forces the client to pick a protocol from this list, any other one may be
+ requested. Please note that the TLS NPN extension was replaced with ALPN.
+
+ssl_fc_protocol : string
+ Returns the name of the used protocol when the incoming connection was made
+ over an SSL/TLS transport layer.
+
+ssl_fc_protocol_hello_id : integer
+ The version of the TLS protocol by which the client wishes to communicate
+ during the session as indicated in client hello message. This value can
+ return only if the value "tune.ssl.capture-buffer-size" is set greater than
+ 0.
+
+ Example:
+ http-request set-header X-SSL-JA3 %[ssl_fc_protocol_hello_id],\
+ %[ssl_fc_cipherlist_bin(1),be2dec(-,2)],\
+ %[ssl_fc_extlist_bin(1),be2dec(-,2)],\
+ %[ssl_fc_eclist_bin(1),be2dec(-,2)],\
+ %[ssl_fc_ecformats_bin,be2dec(-,1)]
+ acl is_malware req.fhdr(x-ssl-ja3),digest(md5),hex \
+ -f /path/to/file/with/malware-ja3.lst
+ http-request set-header X-Malware True if is_malware
+ http-request set-header X-Malware False if !is_malware
+
+ssl_fc_unique_id : binary
+ When the incoming connection was made over an SSL/TLS transport layer,
+ returns the TLS unique ID as defined in RFC5929 section 3. The unique id
+ can be encoded to base64 using the converter: "ssl_fc_unique_id,base64".
+
+ssl_fc_server_handshake_traffic_secret : string
+ Return the SERVER_HANDSHAKE_TRAFFIC_SECRET as an hexadecimal string for the
+ front connection when the incoming connection was made over a TLS 1.3
+ transport layer.
+ Require OpenSSL >= 1.1.1. This is one of the keys dumped by the OpenSSL
+ keylog callback to generate the SSLKEYLOGFILE. The SSL Key logging must be
+ activated with "tune.ssl.keylog on" in the global section. See also
+ "tune.ssl.keylog"
+
+ssl_fc_server_traffic_secret_0 : string
+ Return the SERVER_TRAFFIC_SECRET_0 as an hexadecimal string for the
+ front connection when the incoming connection was made over an TLS 1.3
+ transport layer.
+ Require OpenSSL >= 1.1.1. This is one of the keys dumped by the OpenSSL
+ keylog callback to generate the SSLKEYLOGFILE. The SSL Key logging must be
+ activated with "tune.ssl.keylog on" in the global section. See also
+ "tune.ssl.keylog"
+
+ssl_fc_server_random : binary
+ Returns the server random of the front connection when the incoming connection
+ was made over an SSL/TLS transport layer. It is useful to to decrypt traffic
+ sent using ephemeral ciphers. This requires OpenSSL >= 1.1.0, or BoringSSL.
+
+ssl_fc_session_id : binary
+ Returns the SSL ID of the front connection when the incoming connection was
+ made over an SSL/TLS transport layer. It is useful to stick a given client to
+ a server. It is important to note that some browsers refresh their session ID
+ every few minutes.
+
+ssl_fc_session_key : binary
+ Returns the SSL session master key of the front connection when the incoming
+ connection was made over an SSL/TLS transport layer. It is useful to decrypt
+ traffic sent using ephemeral ciphers. This requires OpenSSL >= 1.1.0, or
+ BoringSSL.
+
+
+ssl_fc_sni : string
+ This extracts the Server Name Indication TLS extension (SNI) field from an
+ incoming connection made via an SSL/TLS transport layer and locally
+ deciphered by HAProxy. The result (when present) typically is a string
+ matching the HTTPS host name (253 chars or less). The SSL library must have
+ been built with support for TLS extensions enabled (check haproxy -vv).
+
+ This fetch is different from "req.ssl_sni" above in that it applies to the
+ connection being deciphered by HAProxy and not to SSL contents being blindly
+ forwarded. See also "ssl_fc_sni_end" and "ssl_fc_sni_reg" below. This
+ requires that the SSL library is built with support for TLS extensions
+ enabled (check haproxy -vv).
+
+ CAUTION! Except under very specific conditions, it is normally not correct to
+ use this field as a substitute for the HTTP "Host" header field. For example,
+ when forwarding an HTTPS connection to a server, the SNI field must be set
+ from the HTTP Host header field using "req.hdr(host)" and not from the front
+ SNI value. The reason is that SNI is solely used to select the certificate
+ the server side will present, and that clients are then allowed to send
+ requests with different Host values as long as they match the names in the
+ certificate. As such, "ssl_fc_sni" should normally not be used as an argument
+ to the "sni" server keyword, unless the backend works in TCP mode.
+
+ ACL derivatives :
+ ssl_fc_sni_end : suffix match
+ ssl_fc_sni_reg : regex match
+
+ssl_fc_use_keysize : integer
+ Returns the symmetric cipher key size used in bits when the incoming
+ connection was made over an SSL/TLS transport layer.
+
+ssl_s_der : binary
+ Returns the DER formatted certificate presented by the server when the
+ outgoing connection was made over an SSL/TLS transport layer. When used for
+ an ACL, the value(s) to match against can be passed in hexadecimal form.
+
+ssl_s_chain_der : binary
+ Returns the DER formatted chain certificate presented by the server when the
+ outgoing connection was made over an SSL/TLS transport layer. When used for
+ an ACL, the value(s) to match against can be passed in hexadecimal form. One
+ can parse the result with any lib accepting ASN.1 DER data. It currently
+ does not support resumed sessions.
+
+ssl_s_key_alg : string
+ Returns the name of the algorithm used to generate the key of the certificate
+ presented by the server when the outgoing connection was made over an
+ SSL/TLS transport layer.
+
+ssl_s_notafter : string
+ Returns the end date presented by the server as a formatted string
+ YYMMDDhhmmss[Z] when the outgoing connection was made over an SSL/TLS
+ transport layer.
+
+ssl_s_notbefore : string
+ Returns the start date presented by the server as a formatted string
+ YYMMDDhhmmss[Z] when the outgoing connection was made over an SSL/TLS
+ transport layer.
+
+ssl_s_i_dn([<entry>[,<occ>[,<format>]]]) : string
+ When the outgoing connection was made over an SSL/TLS transport layer,
+ returns the full distinguished name of the issuer of the certificate
+ presented by the server when no <entry> is specified, or the value of the
+ first given entry found from the beginning of the DN. If a positive/negative
+ occurrence number is specified as the optional second argument, it returns
+ the value of the nth given entry value from the beginning/end of the DN.
+ For instance, "ssl_s_i_dn(OU,2)" the second organization unit, and
+ "ssl_s_i_dn(CN)" retrieves the common name.
+ The <format> parameter allows you to receive the DN suitable for
+ consumption by different protocols. Currently supported is rfc2253 for
+ LDAP v3.
+ If you'd like to modify the format only you can specify an empty string
+ and zero for the first two parameters. Example: ssl_s_i_dn(,0,rfc2253)
+
+ssl_s_s_dn([<entry>[,<occ>[,<format>]]]) : string
+ When the outgoing connection was made over an SSL/TLS transport layer,
+ returns the full distinguished name of the subject of the certificate
+ presented by the server when no <entry> is specified, or the value of the
+ first given entry found from the beginning of the DN. If a positive/negative
+ occurrence number is specified as the optional second argument, it returns
+ the value of the nth given entry value from the beginning/end of the DN.
+ For instance, "ssl_s_s_dn(OU,2)" the second organization unit, and
+ "ssl_s_s_dn(CN)" retrieves the common name.
+ The <format> parameter allows you to receive the DN suitable for
+ consumption by different protocols. Currently supported is rfc2253 for
+ LDAP v3.
+ If you'd like to modify the format only you can specify an empty string
+ and zero for the first two parameters. Example: ssl_s_s_dn(,0,rfc2253)
+
+ssl_s_serial : binary
+ Returns the serial of the certificate presented by the server when the
+ outgoing connection was made over an SSL/TLS transport layer. When used for
+ an ACL, the value(s) to match against can be passed in hexadecimal form.
+
+ssl_s_sha1 : binary
+ Returns the SHA-1 fingerprint of the certificate presented by the server
+ when the outgoing connection was made over an SSL/TLS transport layer. This
+ can be used to know which certificate was chosen using SNI.
+
+ssl_s_sig_alg : string
+ Returns the name of the algorithm used to sign the certificate presented by
+ the server when the outgoing connection was made over an SSL/TLS transport
+ layer.
+
+ssl_s_version : integer
+ Returns the version of the certificate presented by the server when the
+ outgoing connection was made over an SSL/TLS transport layer.
+
+txn.timer.user : integer
+ Total estimated time as seen from client, between the moment the proxy
+ accepted it and the moment both ends were closed, without idle time.
+ This is the equivalent of %Tu in the log-format and is reported in
+ milliseconds (ms). For more details see Section 8.4 "Timing events"
+
+7.3.5. Fetching samples from buffer contents (Layer 6)
+------------------------------------------------------
+
+Fetching samples from buffer contents is a bit different from the previous
+sample fetches above because the sampled data are ephemeral. These data can
+only be used when they're available and will be lost when they're forwarded.
+For this reason, samples fetched from buffer contents during a request cannot
+be used in a response for example. Even while the data are being fetched, they
+can change. Sometimes it is necessary to set some delays or combine multiple
+sample fetch methods to ensure that the expected data are complete and usable,
+for example through TCP request content inspection. Please see the "tcp-request
+content" keyword for more detailed information on the subject.
+
+Warning : Following sample fetches are ignored if used from HTTP proxies. They
+ only deal with raw contents found in the buffers. On their side,
+ HTTP proxies use structured content. Thus raw representation of
+ these data are meaningless. A warning is emitted if an ACL relies on
+ one of the following sample fetches. But it is not possible to detect
+ all invalid usage (for instance inside a log-format string or a
+ sample expression). So be careful.
+
+Summary of sample fetch methods in this section and their respective types:
+
+ keyword output type
+----------------------------------------------------+-------------
+bs.id integer
+distcc_body(<token>[,<occ>]) binary
+distcc_param(<token>[,<occ>]) integer
+fs.id integer
+payload(<offset>,<length>) binary
+payload_lv(<offset1>,<length>[,<offset2>]) binary
+req.len integer
+req_len integer
+req.payload(<offset>,<length>) binary
+req.payload_lv(<offset1>,<length>[,<offset2>]) binary
+req.proto_http boolean
+req_proto_http boolean
+req.rdp_cookie([<name>]) string
+rdp_cookie([<name>]) string
+req.rdp_cookie_cnt([name]) integer
+rdp_cookie_cnt([name]) integer
+req.ssl_alpn string
+req.ssl_ec_ext boolean
+req.ssl_hello_type integer
+req_ssl_hello_type integer
+req.ssl_sni string
+req_ssl_sni string
+req.ssl_st_ext integer
+req.ssl_ver integer
+req_ssl_ver integer
+res.len integer
+res.payload(<offset>,<length>) binary
+res.payload_lv(<offset1>,<length>[,<offset2>]) binary
+res.ssl_hello_type integer
+rep_ssl_hello_type integer
+wait_end boolean
+----------------------------------------------------+-------------
+
+Detailed list:
+
+bs.id : integer
+ Returns the multiplexer's stream ID on the server side. It is the
+ multiplexer's responsibility to return the appropriate information.
+
+distcc_body(<token>[,<occ>]) : binary
+ Parses a distcc message and returns the body associated to occurrence #<occ>
+ of the token <token>. Occurrences start at 1, and when unspecified, any may
+ match though in practice only the first one is checked for now. This can be
+ used to extract file names or arguments in files built using distcc through
+ HAProxy. Please refer to distcc's protocol documentation for the complete
+ list of supported tokens.
+
+distcc_param(<token>[,<occ>]) : integer
+ Parses a distcc message and returns the parameter associated to occurrence
+ #<occ> of the token <token>. Occurrences start at 1, and when unspecified,
+ any may match though in practice only the first one is checked for now. This
+ can be used to extract certain information such as the protocol version, the
+ file size or the argument in files built using distcc through HAProxy.
+ Another use case consists in waiting for the start of the preprocessed file
+ contents before connecting to the server to avoid keeping idle connections.
+ Please refer to distcc's protocol documentation for the complete list of
+ supported tokens.
+
+ Example :
+ # wait up to 20s for the pre-processed file to be uploaded
+ tcp-request inspect-delay 20s
+ tcp-request content accept if { distcc_param(DOTI) -m found }
+ # send large files to the big farm
+ use_backend big_farm if { distcc_param(DOTI) gt 1000000 }
+
+fs.id : integer
+ Returns the multiplexer's stream ID on the client side. It is the
+ multiplexer's responsibility to return the appropriate information. For
+ instance, on a raw TCP, 0 is always returned because there is no stream.
+
+payload(<offset>,<length>) : binary (deprecated)
+ This is an alias for "req.payload" when used in the context of a request (e.g.
+ "stick on", "stick match"), and for "res.payload" when used in the context of
+ a response such as in "stick store response".
+
+payload_lv(<offset1>,<length>[,<offset2>]) : binary (deprecated)
+ This is an alias for "req.payload_lv" when used in the context of a request
+ (e.g. "stick on", "stick match"), and for "res.payload_lv" when used in the
+ context of a response such as in "stick store response".
+
+req.len : integer
+req_len : integer (deprecated)
+ Returns an integer value corresponding to the number of bytes present in the
+ request buffer. This is mostly used in ACL. It is important to understand
+ that this test does not return false as long as the buffer is changing. This
+ means that a check with equality to zero will almost always immediately match
+ at the beginning of the session, while a test for more data will wait for
+ that data to come in and return false only when HAProxy is certain that no
+ more data will come in. This test was designed to be used with TCP request
+ content inspection.
+
+req.payload(<offset>,<length>) : binary
+ This extracts a binary block of <length> bytes and starting at byte <offset>
+ in the request buffer. As a special case, if the <length> argument is zero,
+ the the whole buffer from <offset> to the end is extracted. This can be used
+ with ACLs in order to check for the presence of some content in a buffer at
+ any location.
+
+ ACL derivatives :
+ req.payload(<offset>,<length>) : hex binary match
+
+req.payload_lv(<offset1>,<length>[,<offset2>]) : binary
+ This extracts a binary block whose size is specified at <offset1> for <length>
+ bytes, and which starts at <offset2> if specified or just after the length in
+ the request buffer. The <offset2> parameter also supports relative offsets if
+ prepended with a '+' or '-' sign.
+
+ ACL derivatives :
+ req.payload_lv(<offset1>,<length>[,<offset2>]) : hex binary match
+
+ Example : please consult the example from the "stick store-response" keyword.
+
+req.proto_http : boolean
+req_proto_http : boolean (deprecated)
+ Returns true when data in the request buffer look like HTTP and correctly
+ parses as such. It is the same parser as the common HTTP request parser which
+ is used so there should be no surprises. The test does not match until the
+ request is complete, failed or timed out. This test may be used to report the
+ protocol in TCP logs, but the biggest use is to block TCP request analysis
+ until a complete HTTP request is present in the buffer, for example to track
+ a header.
+
+ Example:
+ # track request counts per "base" (concatenation of Host+URL)
+ tcp-request inspect-delay 10s
+ tcp-request content reject if !HTTP
+ tcp-request content track-sc0 base table req-rate
+
+req.rdp_cookie([<name>]) : string
+rdp_cookie([<name>]) : string (deprecated)
+ When the request buffer looks like the RDP protocol, extracts the RDP cookie
+ <name>, or any cookie if unspecified. The parser only checks for the first
+ cookie, as illustrated in the RDP protocol specification. The cookie name is
+ case insensitive. Generally the "MSTS" cookie name will be used, as it can
+ contain the user name of the client connecting to the server if properly
+ configured on the client. The "MSTSHASH" cookie is often used as well for
+ session stickiness to servers.
+
+ This differs from "balance rdp-cookie" in that any balancing algorithm may be
+ used and thus the distribution of clients to backend servers is not linked to
+ a hash of the RDP cookie. It is envisaged that using a balancing algorithm
+ such as "balance roundrobin" or "balance leastconn" will lead to a more even
+ distribution of clients to backend servers than the hash used by "balance
+ rdp-cookie".
+
+ ACL derivatives :
+ req.rdp_cookie([<name>]) : exact string match
+
+ Example :
+ listen tse-farm
+ bind 0.0.0.0:3389
+ # wait up to 5s for an RDP cookie in the request
+ tcp-request inspect-delay 5s
+ tcp-request content accept if RDP_COOKIE
+ # apply RDP cookie persistence
+ persist rdp-cookie
+ # Persist based on the mstshash cookie
+ # This is only useful makes sense if
+ # balance rdp-cookie is not used
+ stick-table type string size 204800
+ stick on req.rdp_cookie(mstshash)
+ server srv1 1.1.1.1:3389
+ server srv1 1.1.1.2:3389
+
+ See also : "balance rdp-cookie", "persist rdp-cookie", "tcp-request" and the
+ "req.rdp_cookie" ACL.
+
+req.rdp_cookie_cnt([name]) : integer
+rdp_cookie_cnt([name]) : integer (deprecated)
+ Tries to parse the request buffer as RDP protocol, then returns an integer
+ corresponding to the number of RDP cookies found. If an optional cookie name
+ is passed, only cookies matching this name are considered. This is mostly
+ used in ACL.
+
+ ACL derivatives :
+ req.rdp_cookie_cnt([<name>]) : integer match
+
+req.ssl_alpn : string
+ Returns a string containing the values of the Application-Layer Protocol
+ Negotiation (ALPN) TLS extension (RFC7301), sent by the client within the SSL
+ ClientHello message. Note that this only applies to raw contents found in the
+ request buffer and not to the contents deciphered via an SSL data layer, so
+ this will not work with "bind" lines having the "ssl" option. This is useful
+ in ACL to make a routing decision based upon the ALPN preferences of a TLS
+ client, like in the example below. See also "ssl_fc_alpn".
+
+ Examples :
+ # Wait for a client hello for at most 5 seconds
+ tcp-request inspect-delay 5s
+ tcp-request content accept if { req.ssl_hello_type 1 }
+ use_backend bk_acme if { req.ssl_alpn acme-tls/1 }
+ default_backend bk_default
+
+req.ssl_ec_ext : boolean
+ Returns a boolean identifying if client sent the Supported Elliptic Curves
+ Extension as defined in RFC4492, section 5.1. within the SSL ClientHello
+ message. This can be used to present ECC compatible clients with EC
+ certificate and to use RSA for all others, on the same IP address. Note that
+ this only applies to raw contents found in the request buffer and not to
+ contents deciphered via an SSL data layer, so this will not work with "bind"
+ lines having the "ssl" option.
+
+req.ssl_hello_type : integer
+req_ssl_hello_type : integer (deprecated)
+ Returns an integer value containing the type of the SSL hello message found
+ in the request buffer if the buffer contains data that parse as a complete
+ SSL (v3 or superior) client hello message. Note that this only applies to raw
+ contents found in the request buffer and not to contents deciphered via an
+ SSL data layer, so this will not work with "bind" lines having the "ssl"
+ option. This is mostly used in ACL to detect presence of an SSL hello message
+ that is supposed to contain an SSL session ID usable for stickiness.
+
+req.ssl_sni : string
+req_ssl_sni : string (deprecated)
+ Returns a string containing the value of the Server Name TLS extension sent
+ by a client in a TLS stream passing through the request buffer if the buffer
+ contains data that parse as a complete SSL (v3 or superior) client hello
+ message. Note that this only applies to raw contents found in the request
+ buffer and not to contents deciphered via an SSL data layer, so this will not
+ work with "bind" lines having the "ssl" option. This will only work for actual
+ implicit TLS based protocols like HTTPS (443), IMAPS (993), SMTPS (465),
+ however it will not work for explicit TLS based protocols, like SMTP (25/587)
+ or IMAP (143). SNI normally contains the name of the host the client tries to
+ connect to (for recent browsers). SNI is useful for allowing or denying access
+ to certain hosts when SSL/TLS is used by the client. This test was designed to
+ be used with TCP request content inspection. If content switching is needed,
+ it is recommended to first wait for a complete client hello (type 1), like in
+ the example below. See also "ssl_fc_sni".
+
+ ACL derivatives :
+ req.ssl_sni : exact string match
+
+ Examples :
+ # Wait for a client hello for at most 5 seconds
+ tcp-request inspect-delay 5s
+ tcp-request content accept if { req.ssl_hello_type 1 }
+ use_backend bk_allow if { req.ssl_sni -f allowed_sites }
+ default_backend bk_sorry_page
+
+req.ssl_st_ext : integer
+ Returns 0 if the client didn't send a SessionTicket TLS Extension (RFC5077)
+ Returns 1 if the client sent SessionTicket TLS Extension
+ Returns 2 if the client also sent non-zero length TLS SessionTicket
+ Note that this only applies to raw contents found in the request buffer and
+ not to contents deciphered via an SSL data layer, so this will not work with
+ "bind" lines having the "ssl" option. This can for example be used to detect
+ whether the client sent a SessionTicket or not and stick it accordingly, if
+ no SessionTicket then stick on SessionID or don't stick as there's no server
+ side state is there when SessionTickets are in use.
+
+req.ssl_ver : integer
+req_ssl_ver : integer (deprecated)
+ Returns an integer value containing the version of the SSL/TLS protocol of a
+ stream present in the request buffer. Both SSLv2 hello messages and SSLv3
+ messages are supported. TLSv1 is announced as SSL version 3.1. The value is
+ composed of the major version multiplied by 65536, added to the minor
+ version. Note that this only applies to raw contents found in the request
+ buffer and not to contents deciphered via an SSL data layer, so this will not
+ work with "bind" lines having the "ssl" option. The ACL version of the test
+ matches against a decimal notation in the form MAJOR.MINOR (e.g. 3.1). This
+ fetch is mostly used in ACL.
+
+ ACL derivatives :
+ req.ssl_ver : decimal match
+
+res.len : integer
+ Returns an integer value corresponding to the number of bytes present in the
+ response buffer. This is mostly used in ACL. It is important to understand
+ that this test does not return false as long as the buffer is changing. This
+ means that a check with equality to zero will almost always immediately match
+ at the beginning of the stream, while a test for more data will wait for
+ that data to come in and return false only when HAProxy is certain that no
+ more data will come in. This test was designed to be used with TCP response
+ content inspection. But it may also be used in tcp-check based expect rules.
+
+res.payload(<offset>,<length>) : binary
+ This extracts a binary block of <length> bytes and starting at byte <offset>
+ in the response buffer. As a special case, if the <length> argument is zero,
+ the whole buffer from <offset> to the end is extracted. This can be used
+ with ACLs in order to check for the presence of some content in a buffer at
+ any location. It may also be used in tcp-check based expect rules.
+
+res.payload_lv(<offset1>,<length>[,<offset2>]) : binary
+ This extracts a binary block whose size is specified at <offset1> for <length>
+ bytes, and which starts at <offset2> if specified or just after the length in
+ the response buffer. The <offset2> parameter also supports relative offsets
+ if prepended with a '+' or '-' sign. It may also be used in tcp-check based
+ expect rules.
+
+ Example : please consult the example from the "stick store-response" keyword.
+
+res.ssl_hello_type : integer
+rep_ssl_hello_type : integer (deprecated)
+ Returns an integer value containing the type of the SSL hello message found
+ in the response buffer if the buffer contains data that parses as a complete
+ SSL (v3 or superior) hello message. Note that this only applies to raw
+ contents found in the response buffer and not to contents deciphered via an
+ SSL data layer, so this will not work with "server" lines having the "ssl"
+ option. This is mostly used in ACL to detect presence of an SSL hello message
+ that is supposed to contain an SSL session ID usable for stickiness.
+
+wait_end : boolean
+ This fetch either returns true when the inspection period is over, or does
+ not fetch. It is only used in ACLs, in conjunction with content analysis to
+ avoid returning a wrong verdict early. It may also be used to delay some
+ actions, such as a delayed reject for some special addresses. Since it either
+ stops the rules evaluation or immediately returns true, it is recommended to
+ use this acl as the last one in a rule. Please note that the default ACL
+ "WAIT_END" is always usable without prior declaration. This test was designed
+ to be used with TCP request content inspection.
+
+ Examples :
+ # delay every incoming request by 2 seconds
+ tcp-request inspect-delay 2s
+ tcp-request content accept if WAIT_END
+
+ # don't immediately tell bad guys they are rejected
+ tcp-request inspect-delay 10s
+ acl goodguys src 10.0.0.0/24
+ acl badguys src 10.0.1.0/24
+ tcp-request content accept if goodguys
+ tcp-request content reject if badguys WAIT_END
+ tcp-request content reject
+
+
+7.3.6. Fetching HTTP samples (Layer 7)
+--------------------------------------
+
+It is possible to fetch samples from HTTP contents, requests and responses.
+This application layer is also called layer 7. It is only possible to fetch the
+data in this section when a full HTTP request or response has been parsed from
+its respective request or response buffer. This is always the case with all
+HTTP specific rules and for sections running with "mode http". When using TCP
+content inspection, it may be necessary to support an inspection delay in order
+to let the request or response come in first. These fetches may require a bit
+more CPU resources than the layer 4 ones, but not much since the request and
+response are indexed.
+
+Note : Regarding HTTP processing from the tcp-request content rules, everything
+ will work as expected from an HTTP proxy. However, from a TCP proxy,
+ without an HTTP upgrade, it will only work for HTTP/1 content. For
+ HTTP/2 content, only the preface is visible. Thus, it is only possible
+ to rely to "req.proto_http", "req.ver" and eventually "method" sample
+ fetches. All other L7 sample fetches will fail. After an HTTP upgrade,
+ they will work in the same manner than from an HTTP proxy.
+
+Summary of sample fetch methods in this section and their respective types:
+
+ keyword output type
+-------------------------------------------------+-------------
+base string
+base32 integer
+base32+src binary
+baseq string
+capture.req.hdr(<idx>) string
+capture.req.method string
+capture.req.uri string
+capture.req.ver string
+capture.res.hdr(<idx>) string
+capture.res.ver string
+req.body binary
+req.body_param([<name>[,i]]) string
+req.body_len integer
+req.body_size integer
+req.cook([<name>]) string
+cook([<name>]) string
+req.cook_cnt([<name>]) integer
+cook_cnt([<name>]) integer
+req.cook_val([<name>]) integer
+cook_val([<name>]) integer
+req.cook_names([<delim>]) string
+cookie([<name>]) string
+hdr([<name>[,<occ>]]) string
+request_date([<unit>]) integer
+req.fhdr(<name>[,<occ>]) string
+req.fhdr_cnt([<name>]) integer
+req.hdr([<name>[,<occ>]]) string
+req.hdr_cnt([<name>]) integer
+hdr_cnt([<header>]) integer
+req.hdr_ip([<name>[,<occ>]]) ip
+hdr_ip([<name>[,<occ>]]) ip
+req.hdr_val([<name>[,<occ>]]) integer
+hdr_val([<name>[,<occ>]]) integer
+req.hdrs string
+req.hdrs_bin binary
+req.timer.hdr integer
+req.timer.idle integer
+req.timer.queue integer
+req.timer.tq integer
+res.timer.hdr integer
+http_auth(<userlist>) boolean
+http_auth_bearer([<header>]) string
+http_auth_group(<userlist>) string
+http_auth_pass string
+http_auth_type string
+http_auth_user string
+http_first_req boolean
+method integer
+path string
+pathq string
+query string
+req.hdr_names([<delim>]) string
+req.ver string
+req_ver string
+res.body binary
+res.body_len integer
+res.body_size integer
+res.cache_hit boolean
+res.cache_name string
+res.comp boolean
+res.comp_algo string
+res.cook([<name>]) string
+scook([<name>]) string
+res.cook_cnt([<name>]) integer
+scook_cnt([<name>]) integer
+res.cook_val([<name>]) integer
+scook_val([<name>]) integer
+res.cook_names([<delim>]) string
+res.fhdr([<name>[,<occ>]]) string
+res.fhdr_cnt([<name>]) integer
+res.hdr([<name>[,<occ>]]) string
+shdr([<name>[,<occ>]]) string
+res.hdr_cnt([<name>]) integer
+shdr_cnt([<name>]) integer
+res.hdr_ip([<name>[,<occ>]]) ip
+shdr_ip([<name>[,<occ>]]) ip
+res.hdr_names([<delim>]) string
+res.hdr_val([<name>[,<occ>]]) integer
+shdr_val([<name>[,<occ>]]) integer
+res.hdrs string
+res.hdrs_bin binary
+res.ver string
+resp_ver string
+server_status integer
+set-cookie([<name>]) string
+status integer
+txn.status integer
+txn.timer.total integer
+unique-id string
+url string
+url_ip ip
+url_port integer
+urlp([<name>[,<delim>[,i]]]) string
+url_param([<name>[,<delim>[,i]]]) string
+urlp_val([<name>[,<delim>[,i]]]) integer
+url32 integer
+url32+src binary
+-------------------------------------------------+-------------
+
+Detailed list:
+
+base : string
+ This returns the concatenation of the first Host header and the path part of
+ the request, which starts at the first slash and ends before the question
+ mark. It can be useful in virtual hosted environments to detect URL abuses as
+ well as to improve shared caches efficiency. Using this with a limited size
+ stick table also allows one to collect statistics about most commonly
+ requested objects by host/path. With ACLs it can allow simple content
+ switching rules involving the host and the path at the same time, such as
+ "www.example.com/favicon.ico". See also "path" and "uri".
+
+ ACL derivatives :
+ base : exact string match
+ base_beg : prefix match
+ base_dir : subdir match
+ base_dom : domain match
+ base_end : suffix match
+ base_len : length match
+ base_reg : regex match
+ base_sub : substring match
+
+base32 : integer
+ This returns a 32-bit hash of the value returned by the "base" fetch method
+ above. This is useful to track per-URL activity on high traffic sites without
+ having to store all URLs. Instead a shorter hash is stored, saving a lot of
+ memory. The output type is an unsigned integer. The hash function used is
+ SDBM with full avalanche on the output. Technically, base32 is exactly equal
+ to "base,sdbm(1)".
+
+base32+src : binary
+ This returns the concatenation of the base32 fetch above and the src fetch
+ below. The resulting type is of type binary, with a size of 8 or 20 bytes
+ depending on the source address family. This can be used to track per-IP,
+ per-URL counters.
+
+baseq : string
+ This returns the concatenation of the first Host header and the path part of
+ the request with the query-string, which starts at the first slash. Using this
+ instead of "base" allows one to properly identify the target resource, for
+ statistics or caching use cases. See also "path", "pathq" and "base".
+
+capture.req.hdr(<idx>) : string
+ This extracts the content of the header captured by the "capture request
+ header", idx is the position of the capture keyword in the configuration.
+ The first entry is an index of 0. See also: "capture request header".
+
+capture.req.method : string
+ This extracts the METHOD of an HTTP request. It can be used in both request
+ and response. Unlike "method", it can be used in both request and response
+ because it's allocated.
+
+capture.req.uri : string
+ This extracts the request's URI, which starts at the first slash and ends
+ before the first space in the request (without the host part). Unlike "path"
+ and "url", it can be used in both request and response because it's
+ allocated.
+
+capture.req.ver : string
+ This extracts the request's HTTP version and returns either "HTTP/1.0" or
+ "HTTP/1.1". Unlike "req.ver", it can be used in both request, response, and
+ logs because it relies on a persistent flag.
+
+capture.res.hdr(<idx>) : string
+ This extracts the content of the header captured by the "capture response
+ header", idx is the position of the capture keyword in the configuration.
+ The first entry is an index of 0.
+ See also: "capture response header"
+
+capture.res.ver : string
+ This extracts the response's HTTP version and returns either "HTTP/1.0" or
+ "HTTP/1.1". Unlike "res.ver", it can be used in logs because it relies on a
+ persistent flag.
+
+req.body : binary
+ This returns the HTTP request's available body as a block of data. It is
+ recommended to use "option http-buffer-request" to be sure to wait, as much
+ as possible, for the request's body.
+
+req.body_param([<name>[,i]]) : string
+ This fetch assumes that the body of the POST request is url-encoded. The user
+ can check if the "content-type" contains the value
+ "application/x-www-form-urlencoded". This extracts the first occurrence of the
+ parameter <name> in the body, which ends before '&'. The parameter name is
+ case-sensitive, unless "i" is added as a second argument. If no name is
+ given, any parameter will match, and the first one will be returned. The
+ result is a string corresponding to the value of the parameter <name> as
+ presented in the request body (no URL decoding is performed). Note that the
+ ACL version of this fetch iterates over multiple parameters and will
+ iteratively report all parameters values if no name is given.
+
+req.body_len : integer
+ This returns the length of the HTTP request's available body in bytes. It may
+ be lower than the advertised length if the body is larger than the buffer. It
+ is recommended to use "option http-buffer-request" to be sure to wait, as
+ much as possible, for the request's body.
+
+req.body_size : integer
+ This returns the advertised length of the HTTP request's body in bytes. It
+ will represent the advertised Content-Length header, or the size of the
+ available data in case of chunked encoding.
+
+req.cook([<name>]) : string
+cook([<name>]) : string (deprecated)
+ This extracts the last occurrence of the cookie name <name> on a "Cookie"
+ header line from the request, and returns its value as string. If no name is
+ specified, the first cookie value is returned. When used with ACLs, all
+ matching cookies are evaluated. Spaces around the name and the value are
+ ignored as requested by the Cookie header specification (RFC6265). The cookie
+ name is case-sensitive. Empty cookies are valid, so an empty cookie may very
+ well return an empty value if it is present. Use the "found" match to detect
+ presence. Use the res.cook() variant for response cookies sent by the server.
+
+ ACL derivatives :
+ req.cook([<name>]) : exact string match
+ req.cook_beg([<name>]) : prefix match
+ req.cook_dir([<name>]) : subdir match
+ req.cook_dom([<name>]) : domain match
+ req.cook_end([<name>]) : suffix match
+ req.cook_len([<name>]) : length match
+ req.cook_reg([<name>]) : regex match
+ req.cook_sub([<name>]) : substring match
+
+req.cook_cnt([<name>]) : integer
+cook_cnt([<name>]) : integer (deprecated)
+ Returns an integer value representing the number of occurrences of the cookie
+ <name> in the request, or all cookies if <name> is not specified.
+
+req.cook_val([<name>]) : integer
+cook_val([<name>]) : integer (deprecated)
+ This extracts the last occurrence of the cookie name <name> on a "Cookie"
+ header line from the request, and converts its value to an integer which is
+ returned. If no name is specified, the first cookie value is returned. When
+ used in ACLs, all matching names are iterated over until a value matches.
+
+req.cook_names([<delim>]) : string
+ This builds a string made from the concatenation of all cookie names as they
+ appear in the request (Cookie header) when the rule is evaluated. The default
+ delimiter is the comma (',') but it may be overridden as an optional argument
+ <delim>. In this case, only the first character of <delim> is considered.
+
+cookie([<name>]) : string (deprecated)
+ This extracts the last occurrence of the cookie name <name> on a "Cookie"
+ header line from the request, or a "Set-Cookie" header from the response, and
+ returns its value as a string. A typical use is to get multiple clients
+ sharing a same profile use the same server. This can be similar to what
+ "appsession" did with the "request-learn" statement, but with support for
+ multi-peer synchronization and state keeping across restarts. If no name is
+ specified, the first cookie value is returned. This fetch should not be used
+ anymore and should be replaced by req.cook() or res.cook() instead as it
+ ambiguously uses the direction based on the context where it is used.
+
+hdr([<name>[,<occ>]]) : string
+ This is equivalent to req.hdr() when used on requests, and to res.hdr() when
+ used on responses. Please refer to these respective fetches for more details.
+ In case of doubt about the fetch direction, please use the explicit ones.
+ Note that contrary to the hdr() sample fetch method, the hdr_* ACL keywords
+ unambiguously apply to the request headers.
+
+request_date([<unit>]) : integer
+ This is the exact date when the first byte of the HTTP request was received
+ by HAProxy (log-format tag %tr). This is computed from accept_date +
+ handshake time (%Th) + idle time (%Ti).
+
+ Returns a value in number of seconds since epoch.
+
+ <unit> is facultative, and can be set to "s" for seconds (default behavior),
+ "ms" for milliseconds or "us" for microseconds.
+ If unit is set, return value is an integer reflecting either seconds,
+ milliseconds or microseconds since epoch.
+ It is useful when a time resolution of less than a second is needed.
+
+req.fhdr(<name>[,<occ>]) : string
+ This returns the full value of the last occurrence of header <name> in an
+ HTTP request. It differs from req.hdr() in that any commas present in the
+ value are returned and are not used as delimiters. This is sometimes useful
+ with headers such as User-Agent.
+
+ When used from an ACL, all occurrences are iterated over until a match is
+ found.
+
+ Optionally, a specific occurrence might be specified as a position number.
+ Positive values indicate a position from the first occurrence, with 1 being
+ the first one. Negative values indicate positions relative to the last one,
+ with -1 being the last one.
+
+req.fhdr_cnt([<name>]) : integer
+ Returns an integer value representing the number of occurrences of request
+ header field name <name>, or the total number of header fields if <name> is
+ not specified. Like req.fhdr() it differs from res.hdr_cnt() by not splitting
+ headers at commas.
+
+req.hdr([<name>[,<occ>]]) : string
+ This returns the last comma-separated value of the header <name> in an HTTP
+ request. The fetch considers any comma as a delimiter for distinct values.
+ This is useful if you need to process headers that are defined to be a list
+ of values, such as Accept, or X-Forwarded-For. If full-line headers are
+ desired instead, use req.fhdr(). Please carefully check RFC 7231 to know how
+ certain headers are supposed to be parsed. Also, some of them are case
+ insensitive (e.g. Connection).
+
+ When used from an ACL, all occurrences are iterated over until a match is
+ found.
+
+ Optionally, a specific occurrence might be specified as a position number.
+ Positive values indicate a position from the first occurrence, with 1 being
+ the first one. Negative values indicate positions relative to the last one,
+ with -1 being the last one.
+
+ A typical use is with the X-Forwarded-For header once converted to IP,
+ associated with an IP stick-table.
+
+ ACL derivatives :
+ hdr([<name>[,<occ>]]) : exact string match
+ hdr_beg([<name>[,<occ>]]) : prefix match
+ hdr_dir([<name>[,<occ>]]) : subdir match
+ hdr_dom([<name>[,<occ>]]) : domain match
+ hdr_end([<name>[,<occ>]]) : suffix match
+ hdr_len([<name>[,<occ>]]) : length match
+ hdr_reg([<name>[,<occ>]]) : regex match
+ hdr_sub([<name>[,<occ>]]) : substring match
+
+req.hdr_cnt([<name>]) : integer
+hdr_cnt([<header>]) : integer (deprecated)
+ Returns an integer value representing the number of occurrences of request
+ header field name <name>, or the total number of header field values if
+ <name> is not specified. Like req.hdr() it counts each comma separated
+ part of the header's value. If counting of full-line headers is desired,
+ then req.fhdr_cnt() should be used instead.
+
+ With ACLs, it can be used to detect presence, absence or abuse of a specific
+ header, as well as to block request smuggling attacks by rejecting requests
+ which contain more than one of certain headers.
+
+ Refer to req.hdr() for more information on header matching.
+
+req.hdr_ip([<name>[,<occ>]]) : ip
+hdr_ip([<name>[,<occ>]]) : ip (deprecated)
+ This extracts the last occurrence of header <name> in an HTTP request,
+ converts it to an IPv4 or IPv6 address and returns this address. When used
+ with ACLs, all occurrences are checked, and if <name> is omitted, every value
+ of every header is checked. The parser strictly adheres to the format
+ described in RFC7239, with the extension that IPv4 addresses may optionally
+ be followed by a colon (':') and a valid decimal port number (0 to 65535),
+ which will be silently dropped. All other forms will not match and will
+ cause the address to be ignored.
+
+ The <occ> parameter is processed as with req.hdr().
+
+ A typical use is with the X-Forwarded-For and X-Client-IP headers.
+
+req.hdr_val([<name>[,<occ>]]) : integer
+hdr_val([<name>[,<occ>]]) : integer (deprecated)
+ This extracts the last occurrence of header <name> in an HTTP request, and
+ converts it to an integer value. When used with ACLs, all occurrences are
+ checked, and if <name> is omitted, every value of every header is checked.
+
+ The <occ> parameter is processed as with req.hdr().
+
+ A typical use is with the X-Forwarded-For header.
+
+req.hdrs : string
+ Returns the current request headers as string including the last empty line
+ separating headers from the request body. The last empty line can be used to
+ detect a truncated header block. This sample fetch is useful for some SPOE
+ headers analyzers and for advanced logging.
+
+req.hdrs_bin : binary
+ Returns the current request headers contained in preparsed binary form. This
+ is useful for offloading some processing with SPOE. Each string is described
+ by a length followed by the number of bytes indicated in the length. The
+ length is represented using the variable integer encoding detailed in the
+ SPOE documentation. The end of the list is marked by a couple of empty header
+ names and values (length of 0 for both).
+
+ *(<str:header-name><str:header-value>)<empty string><empty string>
+
+ int: refer to the SPOE documentation for the encoding
+ str: <int:length><bytes>
+
+req.timer.hdr : integer
+ Total time to get the client request (HTTP mode only). It's the time elapsed
+ between the first bytes received and the moment the proxy received the empty
+ line marking the end of the HTTP headers. This is reported in milliseconds
+ (ms) and is equivalent to %TR in log-format. See section 8.4 "Timing events"
+ for more details.
+
+req.timer.idle : integer
+ This is the idle time before the HTTP request (HTTP mode only). This timer
+ counts between the end of the handshakes and the first byte of the HTTP
+ request. This is reported in milliseconds and is equivalent to %Ti in
+ log-format. See section 8.4 "Timing events" for more details.
+
+req.timer.queue : integer
+ Total time spent in the queues waiting for a connection slot.
+ This is reported in milliseconds and is equivalent to %Tw in
+ log-format. See section 8.4 "Timing events" for more details.
+
+req.timer.tq : integer
+ total time to get the client request from the accept date or since the
+ emission of the last byte of the previous response.
+ This is reported in milliseconds and is equivalent to %Tq in
+ log-format. See section 8.4 "Timing events" for more details.
+
+res.timer.hdr : integer
+ It's the time elapsed between the moment the TCP connection was established
+ to the server and the moment the server sent its complete response headers.
+ This is reported in milliseconds and is equivalent to %Tr in log-format. See
+ section 8.4 "Timing events" for more details.
+
+
+http_auth(<userlist>) : boolean
+ Returns a boolean indicating whether the authentication data received from
+ the client match a username & password stored in the specified userlist. This
+ fetch function is not really useful outside of ACLs. Currently only http
+ basic auth is supported.
+
+http_auth_bearer([<header>]) : string
+ Returns the client-provided token found in the authorization data when the
+ Bearer scheme is used (to send JSON Web Tokens for instance). No check is
+ performed on the data sent by the client.
+ If a specific <header> is supplied, it will parse this header instead of the
+ Authorization one.
+
+http_auth_group(<userlist>) : string
+ Returns a string corresponding to the user name found in the authentication
+ data received from the client if both the user name and password are valid
+ according to the specified userlist. The main purpose is to use it in ACLs
+ where it is then checked whether the user belongs to any group within a list.
+ This fetch function is not really useful outside of ACLs. Currently only http
+ basic auth is supported.
+
+ ACL derivatives :
+ http_auth_group(<userlist>) : group ...
+ Returns true when the user extracted from the request and whose password is
+ valid according to the specified userlist belongs to at least one of the
+ groups.
+
+http_auth_pass : string
+ Returns the user's password found in the authentication data received from
+ the client, as supplied in the Authorization header. Not checks are
+ performed by this sample fetch. Only Basic authentication is supported.
+
+http_auth_type : string
+ Returns the authentication method found in the authentication data received from
+ the client, as supplied in the Authorization header. Not checks are
+ performed by this sample fetch. Only Basic authentication is supported.
+
+http_auth_user : string
+ Returns the user name found in the authentication data received from the
+ client, as supplied in the Authorization header. Not checks are performed by
+ this sample fetch. Only Basic authentication is supported.
+
+http_first_req : boolean
+ Returns true when the request being processed is the first one of the
+ connection. This can be used to add or remove headers that may be missing
+ from some requests when a request is not the first one, or to help grouping
+ requests in the logs.
+
+method : integer + string
+ Returns an integer value corresponding to the method in the HTTP request. For
+ example, "GET" equals 1 (check sources to establish the matching). Value 9
+ means "other method" and may be converted to a string extracted from the
+ stream. This should not be used directly as a sample, this is only meant to
+ be used from ACLs, which transparently convert methods from patterns to these
+ integer + string values. Some predefined ACL already check for most common
+ methods.
+
+ ACL derivatives :
+ method : case insensitive method match
+
+ Example :
+ # only accept GET and HEAD requests
+ acl valid_method method GET HEAD
+ http-request deny if ! valid_method
+
+path : string
+ This extracts the request's URL path, which starts at the first slash and
+ ends before the question mark (without the host part). A typical use is with
+ prefetch-capable caches, and with portals which need to aggregate multiple
+ information from databases and keep them in caches. Note that with outgoing
+ caches, it would be wiser to use "url" instead. With ACLs, it's typically
+ used to match exact file names (e.g. "/login.php"), or directory parts using
+ the derivative forms. See also the "url" and "base" fetch methods. Please
+ note that any fragment reference in the URI ('#' after the path) is strictly
+ forbidden by the HTTP standard and will be rejected. However, if the frontend
+ receiving the request has "option accept-invalid-http-request", then this
+ fragment part will be accepted and will also appear in the path.
+
+ ACL derivatives :
+ path : exact string match
+ path_beg : prefix match
+ path_dir : subdir match
+ path_dom : domain match
+ path_end : suffix match
+ path_len : length match
+ path_reg : regex match
+ path_sub : substring match
+
+pathq : string
+ This extracts the request's URL path with the query-string, which starts at
+ the first slash. This sample fetch is pretty handy to always retrieve a
+ relative URI, excluding the scheme and the authority part, if any. Indeed,
+ while it is the common representation for an HTTP/1.1 request target, in
+ HTTP/2, an absolute URI is often used. This sample fetch will return the same
+ result in both cases. Please note that any fragment reference in the URI ('#'
+ after the path) is strictly forbidden by the HTTP standard and will be
+ rejected. However, if the frontend receiving the request has "option
+ accept-invalid-http-request", then this fragment part will be accepted and
+ will also appear in the path.
+
+query : string
+ This extracts the request's query string, which starts after the first
+ question mark. If no question mark is present, this fetch returns nothing. If
+ a question mark is present but nothing follows, it returns an empty string.
+ This means it's possible to easily know whether a query string is present
+ using the "found" matching method. This fetch is the complement of "path"
+ which stops before the question mark.
+
+req.hdr_names([<delim>]) : string
+ This builds a string made from the concatenation of all header names as they
+ appear in the request when the rule is evaluated. The default delimiter is
+ the comma (',') but it may be overridden as an optional argument <delim>. In
+ this case, only the first character of <delim> is considered.
+
+req.ver : string
+req_ver : string (deprecated)
+ Returns the version string from the HTTP request, for example "1.1". This can
+ be useful for ACL. For logs use the "%HV" log variable. Some predefined ACL
+ already check for versions 1.0 and 1.1.
+
+ Common values are "1.0", "1.1", "2.0" or "3.0".
+
+ In the case of http/2 and http/3, the value is not extracted from the HTTP
+ version in the request line but is determined by the negotiated protocol
+ version.
+
+ ACL derivatives :
+ req.ver : exact string match
+
+res.body : binary
+ This returns the HTTP response's available body as a block of data. Unlike
+ the request side, there is no directive to wait for the response's body. This
+ sample fetch is really useful (and usable) in the health-check context.
+
+ It may be used in tcp-check based expect rules.
+
+res.body_len : integer
+ This returns the length of the HTTP response available body in bytes. Unlike
+ the request side, there is no directive to wait for the response's body. This
+ sample fetch is really useful (and usable) in the health-check context.
+
+ It may be used in tcp-check based expect rules.
+
+res.body_size : integer
+ This returns the advertised length of the HTTP response body in bytes. It
+ will represent the advertised Content-Length header, or the size of the
+ available data in case of chunked encoding. Unlike the request side, there is
+ no directive to wait for the response body. This sample fetch is really
+ useful (and usable) in the health-check context.
+
+ It may be used in tcp-check based expect rules.
+
+res.cache_hit : boolean
+ Returns the boolean "true" value if the response has been built out of an
+ HTTP cache entry, otherwise returns boolean "false".
+
+res.cache_name : string
+ Returns a string containing the name of the HTTP cache that was used to
+ build the HTTP response if res.cache_hit is true, otherwise returns an
+ empty string.
+
+res.comp : boolean
+ Returns the boolean "true" value if the response has been compressed by
+ HAProxy, otherwise returns boolean "false". This may be used to add
+ information in the logs.
+
+res.comp_algo : string
+ Returns a string containing the name of the algorithm used if the response
+ was compressed by HAProxy, for example : "deflate". This may be used to add
+ some information in the logs.
+
+res.cook([<name>]) : string
+scook([<name>]) : string (deprecated)
+ This extracts the last occurrence of the cookie name <name> on a "Set-Cookie"
+ header line from the response, and returns its value as string. If no name is
+ specified, the first cookie value is returned.
+
+ It may be used in tcp-check based expect rules.
+
+ ACL derivatives :
+ res.scook([<name>] : exact string match
+
+res.cook_cnt([<name>]) : integer
+scook_cnt([<name>]) : integer (deprecated)
+ Returns an integer value representing the number of occurrences of the cookie
+ <name> in the response, or all cookies if <name> is not specified. This is
+ mostly useful when combined with ACLs to detect suspicious responses.
+
+ It may be used in tcp-check based expect rules.
+
+res.cook_val([<name>]) : integer
+scook_val([<name>]) : integer (deprecated)
+ This extracts the last occurrence of the cookie name <name> on a "Set-Cookie"
+ header line from the response, and converts its value to an integer which is
+ returned. If no name is specified, the first cookie value is returned.
+
+ It may be used in tcp-check based expect rules.
+
+res.cook_names([<delim>]) : string
+ This builds a string made from the concatenation of all cookie names as they
+ appear in the response (Set-Cookie headers) when the rule is evaluated. The
+ default delimiter is the comma (',') but it may be overridden as an optional
+ argument <delim>. In this case, only the first character of <delim> is
+ considered.
+
+ It may be used in tcp-check based expect rules.
+
+res.fhdr([<name>[,<occ>]]) : string
+ This fetch works like the req.fhdr() fetch with the difference that it acts
+ on the headers within an HTTP response.
+
+ Like req.fhdr() the res.fhdr() fetch returns full values. If the header is
+ defined to be a list you should use res.hdr().
+
+ This fetch is sometimes useful with headers such as Date or Expires.
+
+ It may be used in tcp-check based expect rules.
+
+res.fhdr_cnt([<name>]) : integer
+ This fetch works like the req.fhdr_cnt() fetch with the difference that it
+ acts on the headers within an HTTP response.
+
+ Like req.fhdr_cnt() the res.fhdr_cnt() fetch acts on full values. If the
+ header is defined to be a list you should use res.hdr_cnt().
+
+ It may be used in tcp-check based expect rules.
+
+res.hdr([<name>[,<occ>]]) : string
+shdr([<name>[,<occ>]]) : string (deprecated)
+ This fetch works like the req.hdr() fetch with the difference that it acts
+ on the headers within an HTTP response.
+
+ Like req.hdr() the res.hdr() fetch considers the comma to be a delimiter. If
+ this is not desired res.fhdr() should be used.
+
+ It may be used in tcp-check based expect rules.
+
+ ACL derivatives :
+ res.hdr([<name>[,<occ>]]) : exact string match
+ res.hdr_beg([<name>[,<occ>]]) : prefix match
+ res.hdr_dir([<name>[,<occ>]]) : subdir match
+ res.hdr_dom([<name>[,<occ>]]) : domain match
+ res.hdr_end([<name>[,<occ>]]) : suffix match
+ res.hdr_len([<name>[,<occ>]]) : length match
+ res.hdr_reg([<name>[,<occ>]]) : regex match
+ res.hdr_sub([<name>[,<occ>]]) : substring match
+
+res.hdr_cnt([<name>]) : integer
+shdr_cnt([<name>]) : integer (deprecated)
+ This fetch works like the req.hdr_cnt() fetch with the difference that it
+ acts on the headers within an HTTP response.
+
+ Like req.hdr_cnt() the res.hdr_cnt() fetch considers the comma to be a
+ delimiter. If this is not desired res.fhdr_cnt() should be used.
+
+ It may be used in tcp-check based expect rules.
+
+res.hdr_ip([<name>[,<occ>]]) : ip
+shdr_ip([<name>[,<occ>]]) : ip (deprecated)
+ This fetch works like the req.hdr_ip() fetch with the difference that it
+ acts on the headers within an HTTP response.
+
+ This can be useful to learn some data into a stick table.
+
+ It may be used in tcp-check based expect rules.
+
+res.hdr_names([<delim>]) : string
+ This builds a string made from the concatenation of all header names as they
+ appear in the response when the rule is evaluated. The default delimiter is
+ the comma (',') but it may be overridden as an optional argument <delim>. In
+ this case, only the first character of <delim> is considered.
+
+ It may be used in tcp-check based expect rules.
+
+res.hdr_val([<name>[,<occ>]]) : integer
+shdr_val([<name>[,<occ>]]) : integer (deprecated)
+ This fetch works like the req.hdr_val() fetch with the difference that it
+ acts on the headers within an HTTP response.
+
+ This can be useful to learn some data into a stick table.
+
+ It may be used in tcp-check based expect rules.
+
+res.hdrs : string
+ Returns the current response headers as string including the last empty line
+ separating headers from the request body. The last empty line can be used to
+ detect a truncated header block. This sample fetch is useful for some SPOE
+ headers analyzers and for advanced logging.
+
+ It may also be used in tcp-check based expect rules.
+
+res.hdrs_bin : binary
+ Returns the current response headers contained in preparsed binary form. This
+ is useful for offloading some processing with SPOE. It may be used in
+ tcp-check based expect rules. Each string is described by a length followed
+ by the number of bytes indicated in the length. The length is represented
+ using the variable integer encoding detailed in the SPOE documentation. The
+ end of the list is marked by a couple of empty header names and values
+ (length of 0 for both).
+
+ *(<str:header-name><str:header-value>)<empty string><empty string>
+
+ int: refer to the SPOE documentation for the encoding
+ str: <int:length><bytes>
+
+res.ver : string
+resp_ver : string (deprecated)
+ Returns the version string from the HTTP response, for example "1.1". This
+ can be useful for logs, but is mostly there for ACL.
+
+ It may be used in tcp-check based expect rules.
+
+ ACL derivatives :
+ resp.ver : exact string match
+
+server_status : integer
+ Return an integer containing the HTTP status code as received from the
+ server. If no response was received from the server, the sample fetch fails.
+
+set-cookie([<name>]) : string (deprecated)
+ This extracts the last occurrence of the cookie name <name> on a "Set-Cookie"
+ header line from the response and uses the corresponding value to match. This
+ can be comparable to what "appsession" did with default options, but with
+ support for multi-peer synchronization and state keeping across restarts.
+
+ This fetch function is deprecated and has been superseded by the "res.cook"
+ fetch. This keyword will disappear soon.
+
+status : integer
+ Returns an integer containing the HTTP status code in the HTTP response, for
+ example, 302. It is mostly used within ACLs and integer ranges, for example,
+ to remove any Location header if the response is not a 3xx. It will be the
+ status code received by the client if it is not changed, via a 'set-status'
+ action for instance.
+
+ It may be used in tcp-check based expect rules.
+
+txn.status : integer
+ Return an integer containing the HTTP status code of the transaction, as
+ reported in the log.
+
+txn.timer.total : integer
+ Total active time for the HTTP request, between the moment the proxy received
+ the first byte of the request header and the emission of the last byte of the
+ response body. This is the equivalent of %Ta in the log-format and is
+ reported in milliseconds (ms). For more information see Section 8.4 "Timing
+ events"
+
+unique-id : string
+ Returns the unique-id attached to the request. The directive
+ "unique-id-format" must be set. If it is not set, the unique-id sample fetch
+ fails. Note that the unique-id is usually used with HTTP requests, however this
+ sample fetch can be used with other protocols. Obviously, if it is used with
+ other protocols than HTTP, the unique-id-format directive must not contain
+ HTTP parts. See: unique-id-format and unique-id-header
+
+url : string
+ This extracts the request's URL as presented in the request. A typical use is
+ with prefetch-capable caches, and with portals which need to aggregate
+ multiple information from databases and keep them in caches. With ACLs, using
+ "path" is preferred over using "url", because clients may send a full URL as
+ is normally done with proxies. The only real use is to match "*" which does
+ not match in "path", and for which there is already a predefined ACL. See
+ also "path" and "base". Please note that any fragment reference in the URI
+ ('#' after the path) is strictly forbidden by the HTTP standard and will be
+ rejected. However, if the frontend receiving the request has "option
+ accept-invalid-http-request", then this fragment part will be accepted and
+ will also appear in the url.
+
+ ACL derivatives :
+ url : exact string match
+ url_beg : prefix match
+ url_dir : subdir match
+ url_dom : domain match
+ url_end : suffix match
+ url_len : length match
+ url_reg : regex match
+ url_sub : substring match
+
+url_ip : ip
+ This extracts the IP address from the request's URL when the host part is
+ presented as an IP address. Its use is very limited. For instance, a
+ monitoring system might use this field as an alternative for the source IP in
+ order to test what path a given source address would follow, or to force an
+ entry in a table for a given source address. It may be used in combination
+ with 'http-request set-dst' to emulate the older 'option http_proxy'.
+
+url_port : integer
+ This extracts the port part from the request's URL. Note that if the port is
+ not specified in the request, port 80 is assumed..
+
+urlp([<name>[,<delim>[,i]]]) : string
+url_param([<name>[,<delim>[,i]]]) : string
+ This extracts the first occurrence of the parameter <name> in the query
+ string, which begins after either '?' or <delim>, and which ends before '&',
+ ';' or <delim>. The parameter name is case-sensitive, unless"i" is added as a
+ third argument. If no name is given, any parameter will match, and the first
+ one will be returned. The result is a string corresponding to the value of the
+ parameter <name> as presented in the request (no URL decoding is performed).
+ This can be used for session stickiness based on a client ID, to extract an
+ application cookie passed as a URL parameter, or in ACLs to apply some checks.
+ Note that the ACL version of this fetch iterates over multiple parameters and
+ will iteratively report all parameters values if no name is given
+
+ ACL derivatives :
+ urlp(<name>[,<delim>]) : exact string match
+ urlp_beg(<name>[,<delim>]) : prefix match
+ urlp_dir(<name>[,<delim>]) : subdir match
+ urlp_dom(<name>[,<delim>]) : domain match
+ urlp_end(<name>[,<delim>]) : suffix match
+ urlp_len(<name>[,<delim>]) : length match
+ urlp_reg(<name>[,<delim>]) : regex match
+ urlp_sub(<name>[,<delim>]) : substring match
+
+
+ Example :
+ # match http://example.com/foo?PHPSESSIONID=some_id
+ stick on urlp(PHPSESSIONID)
+ # match http://example.com/foo;JSESSIONID=some_id
+ stick on urlp(JSESSIONID,;)
+
+urlp_val([<name>[,<delim>[,i]]]) : integer
+ See "urlp" above. This one extracts the URL parameter <name> in the request
+ and converts it to an integer value. This can be used for session stickiness
+ based on a user ID for example, or with ACLs to match a page number or price.
+
+url32 : integer
+ This returns a 32-bit hash of the value obtained by concatenating the first
+ Host header and the whole URL including parameters (not only the path part of
+ the request, as in the "base32" fetch above). This is useful to track per-URL
+ activity. A shorter hash is stored, saving a lot of memory. The output type
+ is an unsigned integer.
+
+url32+src : binary
+ This returns the concatenation of the "url32" fetch and the "src" fetch. The
+ resulting type is of type binary, with a size of 8 or 20 bytes depending on
+ the source address family. This can be used to track per-IP, per-URL counters.
+
+
+7.3.7. Fetching samples for developers
+---------------------------------------
+
+This set of sample fetch methods is reserved to developers and must never be
+used on a production environment, except on developer demand, for debugging
+purposes. Moreover, no special care will be taken on backwards compatibility.
+There is no warranty the following sample fetches will never change, be renamed
+or simply removed. So be really careful if you should use one of them. To avoid
+any ambiguity, these sample fetches are placed in the dedicated scope "internal",
+for instance "internal.strm.is_htx".
+
+Summary of sample fetch methods in this section and their respective types:
+
+ keyword output type
+-------------------------------------------------+-------------
+internal.htx.data integer
+internal.htx.free integer
+internal.htx.free_data integer
+internal.htx.has_eom boolean
+internal.htx.nbblks integer
+internal.htx.size integer
+internal.htx.used integer
+internal.htx_blk.size(<idx>) integer
+internal.htx_blk.type(<idx>) string
+internal.htx_blk.data(<idx>) binary
+internal.htx_blk.hdrname(<idx>) string
+internal.htx_blk.hdrval(<idx>) string
+internal.htx_blk.start_line(<idx>) string
+internal.strm.is_htx boolean
+-------------------------------------------------+-------------
+
+Detailed list:
+
+internal.htx.data : integer
+ Returns the size in bytes used by data in the HTX message associated to a
+ channel. The channel is chosen depending on the sample direction.
+
+internal.htx.free : integer
+ Returns the free space (size - used) in bytes in the HTX message associated
+ to a channel. The channel is chosen depending on the sample direction.
+
+internal.htx.free_data : integer
+ Returns the free space for the data in bytes in the HTX message associated to
+ a channel. The channel is chosen depending on the sample direction.
+
+internal.htx.has_eom : boolean
+ Returns true if the HTX message associated to a channel contains the
+ end-of-message flag (EOM). Otherwise, it returns false. The channel is chosen
+ depending on the sample direction.
+
+internal.htx.nbblks : integer
+ Returns the number of blocks present in the HTX message associated to a
+ channel. The channel is chosen depending on the sample direction.
+
+internal.htx.size : integer
+ Returns the total size in bytes of the HTX message associated to a
+ channel. The channel is chosen depending on the sample direction.
+
+internal.htx.used : integer
+ Returns the total size used in bytes (data + metadata) in the HTX message
+ associated to a channel. The channel is chosen depending on the sample
+ direction.
+
+internal.htx_blk.size(<idx>) : integer
+ Returns the size of the block at the position <idx> in the HTX message
+ associated to a channel or 0 if it does not exist. The channel is chosen
+ depending on the sample direction. <idx> may be any positive integer or one
+ of the special value :
+ * head : The oldest inserted block
+ * tail : The newest inserted block
+ * first : The first block where to (re)start the analysis
+
+internal.htx_blk.type(<idx>) : string
+ Returns the type of the block at the position <idx> in the HTX message
+ associated to a channel or "HTX_BLK_UNUSED" if it does not exist. The channel
+ is chosen depending on the sample direction. <idx> may be any positive
+ integer or one of the special value :
+ * head : The oldest inserted block
+ * tail : The newest inserted block
+ * first : The first block where to (re)start the analysis
+
+internal.htx_blk.data(<idx>) : binary
+ Returns the value of the DATA block at the position <idx> in the HTX message
+ associated to a channel or an empty string if it does not exist or if it is
+ not a DATA block. The channel is chosen depending on the sample direction.
+ <idx> may be any positive integer or one of the special value :
+
+ * head : The oldest inserted block
+ * tail : The newest inserted block
+ * first : The first block where to (re)start the analysis
+
+internal.htx_blk.hdrname(<idx>) : string
+ Returns the header name of the HEADER block at the position <idx> in the HTX
+ message associated to a channel or an empty string if it does not exist or if
+ it is not an HEADER block. The channel is chosen depending on the sample
+ direction. <idx> may be any positive integer or one of the special value :
+
+ * head : The oldest inserted block
+ * tail : The newest inserted block
+ * first : The first block where to (re)start the analysis
+
+internal.htx_blk.hdrval(<idx>) : string
+ Returns the header value of the HEADER block at the position <idx> in the HTX
+ message associated to a channel or an empty string if it does not exist or if
+ it is not an HEADER block. The channel is chosen depending on the sample
+ direction. <idx> may be any positive integer or one of the special value :
+
+ * head : The oldest inserted block
+ * tail : The newest inserted block
+ * first : The first block where to (re)start the analysis
+
+internal.htx_blk.start_line(<idx>) : string
+ Returns the value of the REQ_SL or RES_SL block at the position <idx> in the
+ HTX message associated to a channel or an empty string if it does not exist
+ or if it is not a SL block. The channel is chosen depending on the sample
+ direction. <idx> may be any positive integer or one of the special value :
+
+ * head : The oldest inserted block
+ * tail : The newest inserted block
+ * first : The first block where to (re)start the analysis
+
+internal.strm.is_htx : boolean
+ Returns true if the current stream is an HTX stream. It means the data in the
+ channels buffers are stored using the internal HTX representation. Otherwise,
+ it returns false.
+
+
+7.4. Pre-defined ACLs
+---------------------
+
+Some predefined ACLs are hard-coded so that they do not have to be declared in
+every frontend which needs them. They all have their names in upper case in
+order to avoid confusion. Their equivalence is provided below.
+
+ACL name Equivalent to Usage
+---------------+----------------------------------+------------------------------------------------------
+FALSE always_false never match
+HTTP req.proto_http match if request protocol is valid HTTP
+HTTP_1.0 req.ver 1.0 match if HTTP request version is 1.0
+HTTP_1.1 req.ver 1.1 match if HTTP request version is 1.1
+HTTP_2.0 req.ver 2.0 match if HTTP request version is 2.0
+HTTP_3.0 req.ver 3.0 match if HTTP request version is 3.0
+HTTP_CONTENT req.hdr_val(content-length) gt 0 match an existing content-length in the HTTP request
+HTTP_URL_ABS url_reg ^[^/:]*:// match absolute URL with scheme
+HTTP_URL_SLASH url_beg / match URL beginning with "/"
+HTTP_URL_STAR url * match URL equal to "*"
+LOCALHOST src 127.0.0.1/8 ::1 match connection from local host
+METH_CONNECT method CONNECT match HTTP CONNECT method
+METH_DELETE method DELETE match HTTP DELETE method
+METH_GET method GET HEAD match HTTP GET or HEAD method
+METH_HEAD method HEAD match HTTP HEAD method
+METH_OPTIONS method OPTIONS match HTTP OPTIONS method
+METH_POST method POST match HTTP POST method
+METH_PUT method PUT match HTTP PUT method
+METH_TRACE method TRACE match HTTP TRACE method
+RDP_COOKIE req.rdp_cookie_cnt gt 0 match presence of an RDP cookie in the request buffer
+REQ_CONTENT req.len gt 0 match data in the request buffer
+TRUE always_true always match
+WAIT_END wait_end wait for end of content analysis
+---------------+----------------------------------+------------------------------------------------------
+
+
+8. Logging
+----------
+
+One of HAProxy's strong points certainly lies is its precise logs. It probably
+provides the finest level of information available for such a product, which is
+very important for troubleshooting complex environments. Standard information
+provided in logs include client ports, TCP/HTTP state timers, precise stream
+state at termination and precise termination cause, information about decisions
+to direct traffic to a server, and of course the ability to capture arbitrary
+headers.
+
+In order to improve administrators reactivity, it offers a great transparency
+about encountered problems, both internal and external, and it is possible to
+send logs to different sources at the same time with different level filters :
+
+ - global process-level logs (system errors, start/stop, etc..)
+ - per-instance system and internal errors (lack of resource, bugs, ...)
+ - per-instance external troubles (servers up/down, max connections)
+ - per-instance activity (client connections), either at the establishment or
+ at the termination.
+ - per-request control of log-level, e.g.
+ http-request set-log-level silent if sensitive_request
+
+The ability to distribute different levels of logs to different log servers
+allow several production teams to interact and to fix their problems as soon
+as possible. For example, the system team might monitor system-wide errors,
+while the application team might be monitoring the up/down for their servers in
+real time, and the security team might analyze the activity logs with one hour
+delay.
+
+
+8.1. Log levels
+---------------
+
+TCP and HTTP connections can be logged with information such as the date, time,
+source IP address, destination address, connection duration, response times,
+HTTP request, HTTP return code, number of bytes transmitted, conditions
+in which the stream ended, and even exchanged cookies values. For example
+track a particular user's problems. All messages may be sent to up to two
+syslog servers. Check the "log" keyword in section 4.2 for more information
+about log facilities.
+
+
+8.2. Log formats
+----------------
+
+HAProxy supports 5 log formats. Several fields are common between these formats
+and will be detailed in the following sections. A few of them may vary
+slightly with the configuration, due to indicators specific to certain
+options. The supported formats are as follows :
+
+ - the default format, which is very basic and very rarely used. It only
+ provides very basic information about the incoming connection at the moment
+ it is accepted : source IP:port, destination IP:port, and frontend-name.
+ This mode will eventually disappear so it will not be described to great
+ extents.
+
+ - the TCP format, which is more advanced. This format is enabled when "option
+ tcplog" is set on the frontend. HAProxy will then usually wait for the
+ connection to terminate before logging. This format provides much richer
+ information, such as timers, connection counts, queue size, etc... This
+ format is recommended for pure TCP proxies.
+
+ - the HTTP format, which is the most advanced for HTTP proxying. This format
+ is enabled when "option httplog" is set on the frontend. It provides the
+ same information as the TCP format with some HTTP-specific fields such as
+ the request, the status code, and captures of headers and cookies. This
+ format is recommended for HTTP proxies.
+
+ - the CLF HTTP format, which is equivalent to the HTTP format, but with the
+ fields arranged in the same order as the CLF format. In this mode, all
+ timers, captures, flags, etc... appear one per field after the end of the
+ common fields, in the same order they appear in the standard HTTP format.
+
+ - the custom log format, allows you to make your own log line.
+
+Next sections will go deeper into details for each of these formats. Format
+specification will be performed on a "field" basis. Unless stated otherwise, a
+field is a portion of text delimited by any number of spaces. Since syslog
+servers are susceptible of inserting fields at the beginning of a line, it is
+always assumed that the first field is the one containing the process name and
+identifier.
+
+Note : Since log lines may be quite long, the log examples in sections below
+ might be broken into multiple lines. The example log lines will be
+ prefixed with 3 closing angle brackets ('>>>') and each time a log is
+ broken into multiple lines, each non-final line will end with a
+ backslash ('\') and the next line will start indented by two characters.
+
+
+8.2.1. Default log format
+-------------------------
+
+This format is used when no specific option is set. The log is emitted as soon
+as the connection is accepted. One should note that this currently is the only
+format which logs the request's destination IP and ports.
+
+ Example :
+ listen www
+ mode http
+ log global
+ server srv1 127.0.0.1:8000
+
+ >>> Feb 6 12:12:09 localhost \
+ haproxy[14385]: Connect from 10.0.1.2:33312 to 10.0.3.31:8012 \
+ (www/HTTP)
+
+ Field Format Extract from the example above
+ 1 process_name '[' pid ']:' haproxy[14385]:
+ 2 'Connect from' Connect from
+ 3 source_ip ':' source_port 10.0.1.2:33312
+ 4 'to' to
+ 5 destination_ip ':' destination_port 10.0.3.31:8012
+ 6 '(' frontend_name '/' mode ')' (www/HTTP)
+
+Detailed fields description :
+ - "source_ip" is the IP address of the client which initiated the connection.
+ - "source_port" is the TCP port of the client which initiated the connection.
+ - "destination_ip" is the IP address the client connected to.
+ - "destination_port" is the TCP port the client connected to.
+ - "frontend_name" is the name of the frontend (or listener) which received
+ and processed the connection.
+ - "mode is the mode the frontend is operating (TCP or HTTP).
+
+In case of a UNIX socket, the source and destination addresses are marked as
+"unix:" and the ports reflect the internal ID of the socket which accepted the
+connection (the same ID as reported in the stats).
+
+It is advised not to use this deprecated format for newer installations as it
+will eventually disappear.
+
+
+8.2.2. TCP log format
+---------------------
+
+The TCP format is used when "option tcplog" is specified in the frontend, and
+is the recommended format for pure TCP proxies. It provides a lot of precious
+information for troubleshooting. Since this format includes timers and byte
+counts, the log is normally emitted at the end of the session. It can be
+emitted earlier if "option logasap" is specified, which makes sense in most
+environments with long sessions such as remote terminals. Sessions which match
+the "monitor" rules are never logged. It is also possible not to emit logs for
+sessions for which no data were exchanged between the client and the server, by
+specifying "option dontlognull" in the frontend. Successful connections will
+not be logged if "option dontlog-normal" is specified in the frontend.
+
+The TCP log format is internally declared as a custom log format based on the
+exact following string, which may also be used as a basis to extend the format
+if required. Additionally the HAPROXY_TCP_LOG_FMT variable can be used instead.
+Refer to section 8.2.6 "Custom log format" to see how to use this:
+
+ # strict equivalent of "option tcplog"
+ log-format "%ci:%cp [%t] %ft %b/%s %Tw/%Tc/%Tt %B %ts \
+ %ac/%fc/%bc/%sc/%rc %sq/%bq"
+ # or using the HAPROXY_TCP_LOG_FMT variable
+ log-format "${HAPROXY_TCP_LOG_FMT}"
+
+A few fields may slightly vary depending on some configuration options, those
+are marked with a star ('*') after the field name below.
+
+ Example :
+ frontend fnt
+ mode tcp
+ option tcplog
+ log global
+ default_backend bck
+
+ backend bck
+ server srv1 127.0.0.1:8000
+
+ >>> Feb 6 12:12:56 localhost \
+ haproxy[14387]: 10.0.1.2:33313 [06/Feb/2009:12:12:51.443] fnt \
+ bck/srv1 0/0/5007 212 -- 0/0/0/0/3 0/0
+
+ Field Format Extract from the example above
+ 1 process_name '[' pid ']:' haproxy[14387]:
+ 2 client_ip ':' client_port 10.0.1.2:33313
+ 3 '[' accept_date ']' [06/Feb/2009:12:12:51.443]
+ 4 frontend_name fnt
+ 5 backend_name '/' server_name bck/srv1
+ 6 Tw '/' Tc '/' Tt* 0/0/5007
+ 7 bytes_read* 212
+ 8 termination_state --
+ 9 actconn '/' feconn '/' beconn '/' srv_conn '/' retries* 0/0/0/0/3
+ 10 srv_queue '/' backend_queue 0/0
+
+Detailed fields description :
+ - "client_ip" is the IP address of the client which initiated the TCP
+ connection to HAProxy. If the connection was accepted on a UNIX socket
+ instead, the IP address would be replaced with the word "unix". Note that
+ when the connection is accepted on a socket configured with "accept-proxy"
+ and the PROXY protocol is correctly used, or with a "accept-netscaler-cip"
+ and the NetScaler Client IP insertion protocol is correctly used, then the
+ logs will reflect the forwarded connection's information.
+
+ - "client_port" is the TCP port of the client which initiated the connection.
+ If the connection was accepted on a UNIX socket instead, the port would be
+ replaced with the ID of the accepting socket, which is also reported in the
+ stats interface.
+
+ - "accept_date" is the exact date when the connection was received by HAProxy
+ (which might be very slightly different from the date observed on the
+ network if there was some queuing in the system's backlog). This is usually
+ the same date which may appear in any upstream firewall's log. When used in
+ HTTP mode, the accept_date field will be reset to the first moment the
+ connection is ready to receive a new request (end of previous response for
+ HTTP/1, immediately after previous request for HTTP/2).
+
+ - "frontend_name" is the name of the frontend (or listener) which received
+ and processed the connection.
+
+ - "backend_name" is the name of the backend (or listener) which was selected
+ to manage the connection to the server. This will be the same as the
+ frontend if no switching rule has been applied, which is common for TCP
+ applications.
+
+ - "server_name" is the name of the last server to which the connection was
+ sent, which might differ from the first one if there were connection errors
+ and a redispatch occurred. Note that this server belongs to the backend
+ which processed the request. If the connection was aborted before reaching
+ a server, "<NOSRV>" is indicated instead of a server name.
+
+ - "Tw" is the total time in milliseconds spent waiting in the various queues.
+ It can be "-1" if the connection was aborted before reaching the queue.
+ See "Timers" below for more details.
+
+ - "Tc" is the total time in milliseconds spent waiting for the connection to
+ establish to the final server, including retries. It can be "-1" if the
+ connection was aborted before a connection could be established. See
+ "Timers" below for more details.
+
+ - "Tt" is the total time in milliseconds elapsed between the accept and the
+ last close. It covers all possible processing. There is one exception, if
+ "option logasap" was specified, then the time counting stops at the moment
+ the log is emitted. In this case, a '+' sign is prepended before the value,
+ indicating that the final one will be larger. See "Timers" below for more
+ details.
+
+ - "bytes_read" is the total number of bytes transmitted from the server to
+ the client when the log is emitted. If "option logasap" is specified, the
+ this value will be prefixed with a '+' sign indicating that the final one
+ may be larger. Please note that this value is a 64-bit counter, so log
+ analysis tools must be able to handle it without overflowing.
+
+ - "termination_state" is the condition the session was in when the session
+ ended. This indicates the session state, which side caused the end of
+ session to happen, and for what reason (timeout, error, ...). The normal
+ flags should be "--", indicating the session was closed by either end with
+ no data remaining in buffers. See below "Stream state at disconnection"
+ for more details.
+
+ - "actconn" is the total number of concurrent connections on the process when
+ the session was logged. It is useful to detect when some per-process system
+ limits have been reached. For instance, if actconn is close to 512 when
+ multiple connection errors occur, chances are high that the system limits
+ the process to use a maximum of 1024 file descriptors and that all of them
+ are used. See section 3 "Global parameters" to find how to tune the system.
+
+ - "feconn" is the total number of concurrent connections on the frontend when
+ the session was logged. It is useful to estimate the amount of resource
+ required to sustain high loads, and to detect when the frontend's "maxconn"
+ has been reached. Most often when this value increases by huge jumps, it is
+ because there is congestion on the backend servers, but sometimes it can be
+ caused by a denial of service attack.
+
+ - "beconn" is the total number of concurrent connections handled by the
+ backend when the session was logged. It includes the total number of
+ concurrent connections active on servers as well as the number of
+ connections pending in queues. It is useful to estimate the amount of
+ additional servers needed to support high loads for a given application.
+ Most often when this value increases by huge jumps, it is because there is
+ congestion on the backend servers, but sometimes it can be caused by a
+ denial of service attack.
+
+ - "srv_conn" is the total number of concurrent connections still active on
+ the server when the session was logged. It can never exceed the server's
+ configured "maxconn" parameter. If this value is very often close or equal
+ to the server's "maxconn", it means that traffic regulation is involved a
+ lot, meaning that either the server's maxconn value is too low, or that
+ there aren't enough servers to process the load with an optimal response
+ time. When only one of the server's "srv_conn" is high, it usually means
+ that this server has some trouble causing the connections to take longer to
+ be processed than on other servers.
+
+ - "retries" is the number of connection retries experienced by this session
+ when trying to connect to the server. It must normally be zero, unless a
+ server is being stopped at the same moment the connection was attempted.
+ Frequent retries generally indicate either a network problem between
+ HAProxy and the server, or a misconfigured system backlog on the server
+ preventing new connections from being queued. This field may optionally be
+ prefixed with a '+' sign, indicating that the session has experienced a
+ redispatch after the maximal retry count has been reached on the initial
+ server. In this case, the server name appearing in the log is the one the
+ connection was redispatched to, and not the first one, though both may
+ sometimes be the same in case of hashing for instance. So as a general rule
+ of thumb, when a '+' is present in front of the retry count, this count
+ should not be attributed to the logged server.
+
+ - "srv_queue" is the total number of requests which were processed before
+ this one in the server queue. It is zero when the request has not gone
+ through the server queue. It makes it possible to estimate the approximate
+ server's response time by dividing the time spent in queue by the number of
+ requests in the queue. It is worth noting that if a session experiences a
+ redispatch and passes through two server queues, their positions will be
+ cumulative. A request should not pass through both the server queue and the
+ backend queue unless a redispatch occurs.
+
+ - "backend_queue" is the total number of requests which were processed before
+ this one in the backend's global queue. It is zero when the request has not
+ gone through the global queue. It makes it possible to estimate the average
+ queue length, which easily translates into a number of missing servers when
+ divided by a server's "maxconn" parameter. It is worth noting that if a
+ session experiences a redispatch, it may pass twice in the backend's queue,
+ and then both positions will be cumulative. A request should not pass
+ through both the server queue and the backend queue unless a redispatch
+ occurs.
+
+
+8.2.3. HTTP log format
+----------------------
+
+The HTTP format is the most complete and the best suited for HTTP proxies. It
+is enabled by when "option httplog" is specified in the frontend. It provides
+the same level of information as the TCP format with additional features which
+are specific to the HTTP protocol. Just like the TCP format, the log is usually
+emitted at the end of the stream, unless "option logasap" is specified, which
+generally only makes sense for download sites. A stream which matches the
+"monitor" rules will never logged. It is also possible not to log streams for
+which no data were sent by the client by specifying "option dontlognull" in the
+frontend. Successful connections will not be logged if "option dontlog-normal"
+is specified in the frontend.
+
+The HTTP log format is internally declared as a custom log format based on the
+exact following string, which may also be used as a basis to extend the format
+if required. Additionally the HAPROXY_HTTP_LOG_FMT variable can be used
+instead. Refer to section 8.2.6 "Custom log format" to see how to use this:
+
+ # strict equivalent of "option httplog"
+ log-format "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC \
+ %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r"
+
+And the CLF log format is internally declared as a custom log format based on
+this exact string:
+
+ # strict equivalent of "option httplog clf"
+ log-format "%{+Q}o %{-Q}ci - - [%trg] %r %ST %B \"\" \"\" %cp \
+ %ms %ft %b %s %TR %Tw %Tc %Tr %Ta %tsc %ac %fc \
+ %bc %sc %rc %sq %bq %CC %CS %hrl %hsl"
+ # or using the HAPROXY_HTTP_LOG_FMT variable
+ log-format "${HAPROXY_HTTP_LOG_FMT}"
+
+Most fields are shared with the TCP log, some being different. A few fields may
+slightly vary depending on some configuration options. Those ones are marked
+with a star ('*') after the field name below.
+
+ Example :
+ frontend http-in
+ mode http
+ option httplog
+ log global
+ default_backend bck
+
+ backend static
+ server srv1 127.0.0.1:8000
+
+ >>> Feb 6 12:14:14 localhost \
+ haproxy[14389]: 10.0.1.2:33317 [06/Feb/2009:12:14:14.655] http-in \
+ static/srv1 10/0/30/69/109 200 2750 - - ---- 1/1/1/1/0 0/0 {1wt.eu} \
+ {} "GET /index.html HTTP/1.1"
+
+ Field Format Extract from the example above
+ 1 process_name '[' pid ']:' haproxy[14389]:
+ 2 client_ip ':' client_port 10.0.1.2:33317
+ 3 '[' request_date ']' [06/Feb/2009:12:14:14.655]
+ 4 frontend_name http-in
+ 5 backend_name '/' server_name static/srv1
+ 6 TR '/' Tw '/' Tc '/' Tr '/' Ta* 10/0/30/69/109
+ 7 status_code 200
+ 8 bytes_read* 2750
+ 9 captured_request_cookie -
+ 10 captured_response_cookie -
+ 11 termination_state ----
+ 12 actconn '/' feconn '/' beconn '/' srv_conn '/' retries* 1/1/1/1/0
+ 13 srv_queue '/' backend_queue 0/0
+ 14 '{' captured_request_headers* '}' {haproxy.1wt.eu}
+ 15 '{' captured_response_headers* '}' {}
+ 16 '"' http_request '"' "GET /index.html HTTP/1.1"
+
+Detailed fields description :
+ - "client_ip" is the IP address of the client which initiated the TCP
+ connection to HAProxy. If the connection was accepted on a UNIX socket
+ instead, the IP address would be replaced with the word "unix". Note that
+ when the connection is accepted on a socket configured with "accept-proxy"
+ and the PROXY protocol is correctly used, or with a "accept-netscaler-cip"
+ and the NetScaler Client IP insertion protocol is correctly used, then the
+ logs will reflect the forwarded connection's information.
+
+ - "client_port" is the TCP port of the client which initiated the connection.
+ If the connection was accepted on a UNIX socket instead, the port would be
+ replaced with the ID of the accepting socket, which is also reported in the
+ stats interface.
+
+ - "request_date" is the exact date when the first byte of the HTTP request
+ was received by HAProxy (log field %tr).
+
+ - "frontend_name" is the name of the frontend (or listener) which received
+ and processed the connection.
+
+ - "backend_name" is the name of the backend (or listener) which was selected
+ to manage the connection to the server. This will be the same as the
+ frontend if no switching rule has been applied.
+
+ - "server_name" is the name of the last server to which the connection was
+ sent, which might differ from the first one if there were connection errors
+ and a redispatch occurred. Note that this server belongs to the backend
+ which processed the request. If the request was aborted before reaching a
+ server, "<NOSRV>" is indicated instead of a server name. If the request was
+ intercepted by the stats subsystem, "<STATS>" is indicated instead.
+
+ - "TR" is the total time in milliseconds spent waiting for a full HTTP
+ request from the client (not counting body) after the first byte was
+ received. It can be "-1" if the connection was aborted before a complete
+ request could be received or a bad request was received. It should
+ always be very small because a request generally fits in one single packet.
+ Large times here generally indicate network issues between the client and
+ HAProxy or requests being typed by hand. See section 8.4 "Timing Events"
+ for more details.
+
+ - "Tw" is the total time in milliseconds spent waiting in the various queues.
+ It can be "-1" if the connection was aborted before reaching the queue.
+ See section 8.4 "Timing Events" for more details.
+
+ - "Tc" is the total time in milliseconds spent waiting for the connection to
+ establish to the final server, including retries. It can be "-1" if the
+ request was aborted before a connection could be established. See section
+ 8.4 "Timing Events" for more details.
+
+ - "Tr" is the total time in milliseconds spent waiting for the server to send
+ a full HTTP response, not counting data. It can be "-1" if the request was
+ aborted before a complete response could be received. It generally matches
+ the server's processing time for the request, though it may be altered by
+ the amount of data sent by the client to the server. Large times here on
+ "GET" requests generally indicate an overloaded server. See section 8.4
+ "Timing Events" for more details.
+
+ - "Ta" is the time the request remained active in HAProxy, which is the total
+ time in milliseconds elapsed between the first byte of the request was
+ received and the last byte of response was sent. It covers all possible
+ processing except the handshake (see Th) and idle time (see Ti). There is
+ one exception, if "option logasap" was specified, then the time counting
+ stops at the moment the log is emitted. In this case, a '+' sign is
+ prepended before the value, indicating that the final one will be larger.
+ See section 8.4 "Timing Events" for more details.
+
+ - "status_code" is the HTTP status code returned to the client. This status
+ is generally set by the server, but it might also be set by HAProxy when
+ the server cannot be reached or when its response is blocked by HAProxy.
+
+ - "bytes_read" is the total number of bytes transmitted to the client when
+ the log is emitted. This does include HTTP headers. If "option logasap" is
+ specified, this value will be prefixed with a '+' sign indicating that
+ the final one may be larger. Please note that this value is a 64-bit
+ counter, so log analysis tools must be able to handle it without
+ overflowing.
+
+ - "captured_request_cookie" is an optional "name=value" entry indicating that
+ the client had this cookie in the request. The cookie name and its maximum
+ length are defined by the "capture cookie" statement in the frontend
+ configuration. The field is a single dash ('-') when the option is not
+ set. Only one cookie may be captured, it is generally used to track session
+ ID exchanges between a client and a server to detect session crossing
+ between clients due to application bugs. For more details, please consult
+ the section "Capturing HTTP headers and cookies" below.
+
+ - "captured_response_cookie" is an optional "name=value" entry indicating
+ that the server has returned a cookie with its response. The cookie name
+ and its maximum length are defined by the "capture cookie" statement in the
+ frontend configuration. The field is a single dash ('-') when the option is
+ not set. Only one cookie may be captured, it is generally used to track
+ session ID exchanges between a client and a server to detect session
+ crossing between clients due to application bugs. For more details, please
+ consult the section "Capturing HTTP headers and cookies" below.
+
+ - "termination_state" is the condition the stream was in when the stream
+ ended. This indicates the stream state, which side caused the end of
+ stream to happen, for what reason (timeout, error, ...), just like in TCP
+ logs, and information about persistence operations on cookies in the last
+ two characters. The normal flags should begin with "--", indicating the
+ stream was closed by either end with no data remaining in buffers. See
+ below "Stream state at disconnection" for more details.
+
+ - "actconn" is the total number of concurrent connections on the process when
+ the stream was logged. It is useful to detect when some per-process system
+ limits have been reached. For instance, if actconn is close to 512 or 1024
+ when multiple connection errors occur, chances are high that the system
+ limits the process to use a maximum of 1024 file descriptors and that all
+ of them are used. See section 3 "Global parameters" to find how to tune the
+ system.
+
+ - "feconn" is the total number of concurrent connections on the frontend when
+ the stream was logged. It is useful to estimate the amount of resource
+ required to sustain high loads, and to detect when the frontend's "maxconn"
+ has been reached. Most often when this value increases by huge jumps, it is
+ because there is congestion on the backend servers, but sometimes it can be
+ caused by a denial of service attack.
+
+ - "beconn" is the total number of concurrent connections handled by the
+ backend when the stream was logged. It includes the total number of
+ concurrent connections active on servers as well as the number of
+ connections pending in queues. It is useful to estimate the amount of
+ additional servers needed to support high loads for a given application.
+ Most often when this value increases by huge jumps, it is because there is
+ congestion on the backend servers, but sometimes it can be caused by a
+ denial of service attack.
+
+ - "srv_conn" is the total number of concurrent connections still active on
+ the server when the stream was logged. It can never exceed the server's
+ configured "maxconn" parameter. If this value is very often close or equal
+ to the server's "maxconn", it means that traffic regulation is involved a
+ lot, meaning that either the server's maxconn value is too low, or that
+ there aren't enough servers to process the load with an optimal response
+ time. When only one of the server's "srv_conn" is high, it usually means
+ that this server has some trouble causing the requests to take longer to be
+ processed than on other servers.
+
+ - "retries" is the number of connection retries experienced by this stream
+ when trying to connect to the server. It must normally be zero, unless a
+ server is being stopped at the same moment the connection was attempted.
+ Frequent retries generally indicate either a network problem between
+ HAProxy and the server, or a misconfigured system backlog on the server
+ preventing new connections from being queued. This field may optionally be
+ prefixed with a '+' sign, indicating that the stream has experienced a
+ redispatch after the maximal retry count has been reached on the initial
+ server. In this case, the server name appearing in the log is the one the
+ connection was redispatched to, and not the first one, though both may
+ sometimes be the same in case of hashing for instance. So as a general rule
+ of thumb, when a '+' is present in front of the retry count, this count
+ should not be attributed to the logged server.
+
+ - "srv_queue" is the total number of requests which were processed before
+ this one in the server queue. It is zero when the request has not gone
+ through the server queue. It makes it possible to estimate the approximate
+ server's response time by dividing the time spent in queue by the number of
+ requests in the queue. It is worth noting that if a stream experiences a
+ redispatch and passes through two server queues, their positions will be
+ cumulative. A request should not pass through both the server queue and the
+ backend queue unless a redispatch occurs.
+
+ - "backend_queue" is the total number of requests which were processed before
+ this one in the backend's global queue. It is zero when the request has not
+ gone through the global queue. It makes it possible to estimate the average
+ queue length, which easily translates into a number of missing servers when
+ divided by a server's "maxconn" parameter. It is worth noting that if a
+ stream experiences a redispatch, it may pass twice in the backend's queue,
+ and then both positions will be cumulative. A request should not pass
+ through both the server queue and the backend queue unless a redispatch
+ occurs.
+
+ - "captured_request_headers" is a list of headers captured in the request due
+ to the presence of the "capture request header" statement in the frontend.
+ Multiple headers can be captured, they will be delimited by a vertical bar
+ ('|'). When no capture is enabled, the braces do not appear, causing a
+ shift of remaining fields. It is important to note that this field may
+ contain spaces, and that using it requires a smarter log parser than when
+ it's not used. Please consult the section "Capturing HTTP headers and
+ cookies" below for more details.
+
+ - "captured_response_headers" is a list of headers captured in the response
+ due to the presence of the "capture response header" statement in the
+ frontend. Multiple headers can be captured, they will be delimited by a
+ vertical bar ('|'). When no capture is enabled, the braces do not appear,
+ causing a shift of remaining fields. It is important to note that this
+ field may contain spaces, and that using it requires a smarter log parser
+ than when it's not used. Please consult the section "Capturing HTTP headers
+ and cookies" below for more details.
+
+ - "http_request" is the complete HTTP request line, including the method,
+ request and HTTP version string. Non-printable characters are encoded (see
+ below the section "Non-printable characters"). This is always the last
+ field, and it is always delimited by quotes and is the only one which can
+ contain quotes. If new fields are added to the log format, they will be
+ added before this field. This field might be truncated if the request is
+ huge and does not fit in the standard syslog buffer (1024 characters). This
+ is the reason why this field must always remain the last one.
+
+
+8.2.4. HTTPS log format
+----------------------
+
+The HTTPS format is the best suited for HTTP over SSL connections. It is an
+extension of the HTTP format (see section 8.2.3) to which SSL related
+information are added. It is enabled when "option httpslog" is specified in the
+frontend. Just like the TCP and HTTP formats, the log is usually emitted at the
+end of the stream, unless "option logasap" is specified. A stream which
+matches the "monitor" rules will never logged. It is also possible not to log
+streams for which no data were sent by the client by specifying "option
+dontlognull" in the frontend. Successful connections will not be logged if
+"option dontlog-normal" is specified in the frontend.
+
+The HTTPS log format is internally declared as a custom log format based on the
+exact following string, which may also be used as a basis to extend the format
+if required. Additionally the HAPROXY_HTTPS_LOG_FMT variable can be used
+instead. Refer to section 8.2.6 "Custom log format" to see how to use this:
+
+ # strict equivalent of "option httpslog"
+ log-format "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC \
+ %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r \
+ %[fc_err]/%[ssl_fc_err,hex]/%[ssl_c_err]/\
+ %[ssl_c_ca_err]/%[ssl_fc_is_resumed] %[ssl_fc_sni]/%sslv/%sslc"
+ # or using the HAPROXY_HTTPS_LOG_FMT variable
+ log-format "${HAPROXY_HTTPS_LOG_FMT}"
+
+This format is basically the HTTP one (see section 8.2.3) with new fields
+appended to it. The new fields (lines 17 and 18) will be detailed here. For the
+HTTP ones, refer to the HTTP section.
+
+ Example :
+ frontend https-in
+ mode http
+ option httpslog
+ log global
+ bind *:443 ssl crt mycerts/srv.pem ...
+ default_backend bck
+
+ backend static
+ server srv1 127.0.0.1:8000 ssl crt mycerts/clt.pem ...
+
+ >>> Feb 6 12:14:14 localhost \
+ haproxy[14389]: 10.0.1.2:33317 [06/Feb/2009:12:14:14.655] https-in \
+ static/srv1 10/0/30/69/109 200 2750 - - ---- 1/1/1/1/0 0/0 {1wt.eu} \
+ {} "GET /index.html HTTP/1.1" 0/0/0/0/0 \
+ 1wt.eu/TLSv1.3/TLS_AES_256_GCM_SHA384
+
+ Field Format Extract from the example above
+ 1 process_name '[' pid ']:' haproxy[14389]:
+ 2 client_ip ':' client_port 10.0.1.2:33317
+ 3 '[' request_date ']' [06/Feb/2009:12:14:14.655]
+ 4 frontend_name https-in
+ 5 backend_name '/' server_name static/srv1
+ 6 TR '/' Tw '/' Tc '/' Tr '/' Ta* 10/0/30/69/109
+ 7 status_code 200
+ 8 bytes_read* 2750
+ 9 captured_request_cookie -
+ 10 captured_response_cookie -
+ 11 termination_state ----
+ 12 actconn '/' feconn '/' beconn '/' srv_conn '/' retries* 1/1/1/1/0
+ 13 srv_queue '/' backend_queue 0/0
+ 14 '{' captured_request_headers* '}' {haproxy.1wt.eu}
+ 15 '{' captured_response_headers* '}' {}
+ 16 '"' http_request '"' "GET /index.html HTTP/1.1"
+ 17 fc_err '/' ssl_fc_err '/' ssl_c_err
+ '/' ssl_c_ca_err '/' ssl_fc_is_resumed 0/0/0/0/0
+ 18 ssl_fc_sni '/' ssl_version
+ '/' ssl_ciphers 1wt.eu/TLSv1.3/TLS_AES_256_GCM_SHA384
+
+Detailed fields description :
+ - "fc_err" is the status of the connection on the frontend's side. It
+ corresponds to the "fc_err" sample fetch. See the "fc_err" and "fc_err_str"
+ sample fetch functions for more information.
+
+ - "ssl_fc_err" is the last error of the first SSL error stack that was
+ raised on the connection from the frontend's perspective. It might be used
+ to detect SSL handshake errors for instance. It will be 0 if everything
+ went well. See the "ssl_fc_err" sample fetch's description for more
+ information.
+
+ - "ssl_c_err" is the status of the client's certificate verification process.
+ The handshake might be successful while having a non-null verification
+ error code if it is an ignored one. See the "ssl_c_err" sample fetch and
+ the "crt-ignore-err" option.
+
+ - "ssl_c_ca_err" is the status of the client's certificate chain verification
+ process. The handshake might be successful while having a non-null
+ verification error code if it is an ignored one. See the "ssl_c_ca_err"
+ sample fetch and the "ca-ignore-err" option.
+
+ - "ssl_fc_is_resumed" is true if the incoming TLS session was resumed with
+ the stateful cache or a stateless ticket. Don't forgot that a TLS session
+ can be shared by multiple requests.
+
+ - "ssl_fc_sni" is the SNI (Server Name Indication) presented by the client
+ to select the certificate to be used. It usually matches the host name for
+ the first request of a connection. An absence of this field may indicate
+ that the SNI was not sent by the client, and will lead haproxy to use the
+ default certificate, or to reject the connection in case of strict-sni.
+
+ - "ssl_version" is the SSL version of the frontend.
+
+ - "ssl_ciphers" is the SSL cipher used for the connection.
+
+
+8.2.5. Error log format
+-----------------------
+
+When an incoming connection fails due to an SSL handshake or an invalid PROXY
+protocol header, HAProxy will log the event using a shorter, fixed line format,
+unless a dedicated error log format is defined through an "error-log-format"
+line. By default, logs are emitted at the LOG_INFO level, unless the option
+"log-separate-errors" is set in the backend, in which case the LOG_ERR level
+will be used. Connections on which no data are exchanged (e.g. probes) are not
+logged if the "dontlognull" option is set.
+
+The default format looks like this :
+
+ >>> Dec 3 18:27:14 localhost \
+ haproxy[6103]: 127.0.0.1:56059 [03/Dec/2012:17:35:10.380] frt/f1: \
+ Connection error during SSL handshake
+
+ Field Format Extract from the example above
+ 1 process_name '[' pid ']:' haproxy[6103]:
+ 2 client_ip ':' client_port 127.0.0.1:56059
+ 3 '[' accept_date ']' [03/Dec/2012:17:35:10.380]
+ 4 frontend_name "/" bind_name ":" frt/f1:
+ 5 message Connection error during SSL handshake
+
+These fields just provide minimal information to help debugging connection
+failures.
+
+By using the "error-log-format" directive, the legacy log format described
+above will not be used anymore, and all error log lines will follow the
+defined format.
+
+An example of reasonably complete error-log-format follows, it will report the
+source address and port, the connection accept() date, the frontend name, the
+number of active connections on the process and on thit frontend, haproxy's
+internal error identifier on the front connection, the hexadecimal OpenSSL
+error number (that can be copy-pasted to "openssl errstr" for full decoding),
+the client certificate extraction status (0 indicates no error), the client
+certificate validation status using the CA (0 indicates no error), a boolean
+indicating if the connection is new or was resumed, the optional server name
+indication (SNI) provided by the client, the SSL version name and the SSL
+ciphers used on the connection, if any. Note that backend connection errors
+are never reported here since in order for a backend connection to fail, it
+would have passed through a successful stream, hence will be available as
+regular traffic log (see option httplog or option httpslog).
+
+ # detailed frontend connection error log
+ error-log-format "%ci:%cp [%tr] %ft %ac/%fc %[fc_err]/\
+ %[ssl_fc_err,hex]/%[ssl_c_err]/%[ssl_c_ca_err]/%[ssl_fc_is_resumed] \
+ %[ssl_fc_sni]/%sslv/%sslc"
+
+
+8.2.6. Custom log format
+------------------------
+
+When the default log formats are not sufficient, it is possible to define new
+ones in very fine details. As creating a log-format from scratch is not always
+a trivial task, it is strongly recommended to first have a look at the existing
+formats ("option tcplog", "option httplog", "option httpslog"), pick the one
+looking the closest to the expectation, copy its "log-format" equivalent string
+and adjust it.
+
+HAProxy understands some log format variables. % precedes log format variables.
+Variables can take arguments using braces ('{}'), and multiple arguments are
+separated by commas within the braces. Flags may be added or removed by
+prefixing them with a '+' or '-' sign.
+
+Special variable "%o" may be used to propagate its flags to all other
+variables on the same format string. This is particularly handy with quoted
+("Q") and escaped ("E") string formats.
+
+If a variable is named between square brackets ('[' .. ']') then it is used
+as a sample expression rule (see section 7.3). This it useful to add some
+less common information such as the client's SSL certificate's DN, or to log
+the key that would be used to store an entry into a stick table.
+
+Note: spaces must be escaped. In configuration directives "log-format",
+"log-format-sd" and "unique-id-format", spaces are considered as
+delimiters and are merged. In order to emit a verbatim '%', it must be
+preceded by another '%' resulting in '%%'.
+
+Note: when using the RFC5424 syslog message format, the characters '"',
+'\' and ']' inside PARAM-VALUE should be escaped with '\' as prefix (see
+https://tools.ietf.org/html/rfc5424#section-6.3.3 for more details). In
+such cases, the use of the flag "E" should be considered.
+
+Flags are :
+ * Q: quote a string
+ * X: hexadecimal representation (IPs, Ports, %Ts, %rt, %pid)
+ * E: escape characters '"', '\' and ']' in a string with '\' as prefix
+ (intended purpose is for the RFC5424 structured-data log formats)
+
+ Example:
+
+ log-format %T\ %t\ Some\ Text
+ log-format %{+Q}o\ %t\ %s\ %{-Q}r
+
+ log-format-sd %{+Q,+E}o\ [exampleSDID@1234\ header=%[capture.req.hdr(0)]]
+
+Please refer to the table below for currently defined variables :
+
+ +---+------+------------------------------------------------------+---------+
+ | R | var | field name (8.2.2 and 8.2.3 for description) | type |
+ | | | sample fetch alternative | |
+ +===+======+======================================================+=========+
+ | | %o | special variable, apply flags on all next var | |
+ +---+------+------------------------------------------------------+---------+
+ | date formats |
+ +---+------+------------------------------------------------------+---------+
+ | | %T | Accept date UTC + timezone | |
+ | | | %[accept_date,utime("%d/%b/%Y:%H:%M:%S %z")] | date |
+ +---+------+------------------------------------------------------+---------+
+ | | %Tl | Accept date local + timezone | |
+ | | | %[accept_date,ltime("%d/%b/%Y:%H:%M:%S %z")] | date |
+ +---+------+------------------------------------------------------+---------+
+ | | %Ts | Accept date as a UNIX timestamp | numeric |
+ +---+------+------------------------------------------------------+---------+
+ | | %t | Accept date local (with millisecond resolution) | |
+ | | | %[accept_date(ms),ms_ltime("%d/%b/%Y:%H:%M:%S.%3N")] | date |
+ +---+------+------------------------------------------------------+---------+
+ | | %ms | Accept date milliseconds | |
+ | | | %[accept_date(ms),ms_utime("%3N") | numeric |
+ +---+------+------------------------------------------------------+---------+
+ | H | %tr | Request date local (with millisecond resolution) | |
+ | | | %[request_date(ms),ms_ltime("%d/%b/%Y:%H:%M:%S.%3N")]| date |
+ +---+------+------------------------------------------------------+---------+
+ | H | %trg | Request date UTC + timezone | |
+ | | | %[request_date,utime("%d/%b/%Y:%H:%M:%S %z")] | date |
+ +---+------+------------------------------------------------------+---------+
+ | H | %trl | Request date local + timezone | |
+ | | | %[request_date,ltime("%d/%b/%Y:%H:%M:%S %z")] | date |
+ +---+------+------------------------------------------------------+---------+
+ | Timing events |
+ +---+------+------------------------------------------------------+---------+
+ | H | %Ta | Active time of the request (from TR to end) | |
+ | | | %[txn.timer.total] | numeric |
+ +---+------+------------------------------------------------------+---------+
+ | | %Tc | Tc | |
+ | | | %[bc.timer.connect] | numeric |
+ +---+------+------------------------------------------------------+---------+
+ | | %Td | Td = Tt - (Tq + Tw + Tc + Tr) | |
+ | | | %[res.timer.data] | numeric |
+ +---+------+------------------------------------------------------+---------+
+ | | %Th | connection handshake time (SSL, PROXY proto) | |
+ | | | %[fc.timer.handshake] | numeric |
+ +---+------+------------------------------------------------------+---------+
+ | H | %Ti | idle time before the HTTP request | |
+ | | | %[req.timer.idle] | numeric |
+ +---+------+------------------------------------------------------+---------+
+ | H | %Tq | Th + Ti + TR | |
+ | | | %[req.timer.tq] | numeric |
+ +---+------+------------------------------------------------------+---------+
+ | H | %TR | time to receive the full request from 1st byte | |
+ | | | %[req.timer.hdr] | numeric |
+ +---+------+------------------------------------------------------+---------+
+ | H | %Tr | Tr (response time) | |
+ | | | %[res.timer.hdr] | numeric |
+ +---+------+------------------------------------------------------+---------+
+ | | %Tt | Tt | |
+ | | | %[fc.timer.total] | numeric |
+ +---+------+------------------------------------------------------+---------+
+ | | %Tu | Tu | |
+ | | | %[txn.timer.user] | numeric |
+ +---+------+------------------------------------------------------+---------+
+ | | %Tw | Tw | |
+ | | | %[req.timer.queue] | numeric |
+ +---+------+------------------------------------------------------+---------+
+ | Others |
+ +---+------+------------------------------------------------------+---------+
+ | | %B | bytes_read (from server to client) | numeric |
+ | | | %[bytes_out] | |
+ +---+------+------------------------------------------------------+---------+
+ | H | %CC | captured_request_cookie | string |
+ +---+------+------------------------------------------------------+---------+
+ | H | %CS | captured_response_cookie | string |
+ +---+------+------------------------------------------------------+---------+
+ | | %H | hostname | string |
+ +---+------+------------------------------------------------------+---------+
+ | H | %HM | HTTP method (ex: POST) | string |
+ +---+------+------------------------------------------------------+---------+
+ | H | %HP | HTTP request URI without query string | string |
+ +---+------+------------------------------------------------------+---------+
+ | H | %HPO | HTTP path only (without host nor query string) | string |
+ +---+------+------------------------------------------------------+---------+
+ | H | %HQ | HTTP request URI query string (ex: ?bar=baz) | string |
+ | | | ?%[query] | |
+ +---+------+------------------------------------------------------+---------+
+ | H | %HU | HTTP request URI (ex: /foo?bar=baz) | string |
+ +---+------+------------------------------------------------------+---------+
+ | H | %HV | HTTP version (ex: HTTP/1.0) | string |
+ | | | HTTP/%[req.ver] | |
+ +---+------+------------------------------------------------------+---------+
+ | | %ID | unique-id | string |
+ +---+------+------------------------------------------------------+---------+
+ | | %ST | status_code | numeric |
+ | | | %[txn.status] | |
+ +---+------+------------------------------------------------------+---------+
+ | | %U | bytes_uploaded (from client to server) | numeric |
+ | | | %[bytes_in] | |
+ +---+------+------------------------------------------------------+---------+
+ | | %ac | actconn | |
+ | | | %[act_conn] | numeric |
+ +---+------+------------------------------------------------------+---------+
+ | | %b | backend_name | |
+ | | | %[be_name] | string |
+ +---+------+------------------------------------------------------+---------+
+ | | %bc | beconn (backend concurrent connections) | numeric |
+ +---+------+------------------------------------------------------+---------+
+ | | %bi | backend_source_ip (connecting address) | |
+ | | | %[bc_src] | IP |
+ +---+------+------------------------------------------------------+---------+
+ | | %bp | backend_source_port (connecting address) | |
+ | | | %[bc_src_port] | numeric |
+ +---+------+------------------------------------------------------+---------+
+ | | %bq | backend_queue | numeric |
+ +---+------+------------------------------------------------------+---------+
+ | | %ci | client_ip (accepted address) | |
+ | | | %[src] | IP |
+ +---+------+------------------------------------------------------+---------+
+ | | %cp | client_port (accepted address) | |
+ | | | %[src_port] | numeric |
+ +---+------+------------------------------------------------------+---------+
+ | | %f | frontend_name | string |
+ +---+------+------------------------------------------------------+---------+
+ | | %fc | feconn (frontend concurrent connections) | numeric |
+ +---+------+------------------------------------------------------+---------+
+ | | %fi | frontend_ip (accepting address) | |
+ | | | %[dst] | IP |
+ +---+------+------------------------------------------------------+---------+
+ | | %fp | frontend_port (accepting address) | |
+ | | | %[dst_port] | numeric |
+ +---+------+------------------------------------------------------+---------+
+ | | %ft | frontend_name_transport ('~' suffix for SSL) | string |
+ +---+------+------------------------------------------------------+---------+
+ | | %lc | frontend_log_counter | numeric |
+ +---+------+------------------------------------------------------+---------+
+ | | %hr | captured_request_headers default style | string |
+ +---+------+------------------------------------------------------+---------+
+ | | %hrl | captured_request_headers CLF style | string |
+ | | | | list |
+ +---+------+------------------------------------------------------+---------+
+ | | %hs | captured_response_headers default style | string |
+ +---+------+------------------------------------------------------+---------+
+ | | %hsl | captured_response_headers CLF style | string |
+ | | | | list |
+ +---+------+------------------------------------------------------+---------+
+ | | %pid | PID | |
+ | | | %[pid] | numeric |
+ +---+------+------------------------------------------------------+---------+
+ | H | %r | http_request | string |
+ +---+------+------------------------------------------------------+---------+
+ | | %rc | retries | numeric |
+ | | | %[txn.conn_retries] | |
+ +---+------+------------------------------------------------------+---------+
+ | | %rt | request_counter (HTTP req or TCP session) | numeric |
+ | | | %[txn.id32] | |
+ +---+------+------------------------------------------------------+---------+
+ | | %s | server_name | string |
+ +---+------+------------------------------------------------------+---------+
+ | | %sc | srv_conn (server concurrent connections) | numeric |
+ +---+------+------------------------------------------------------+---------+
+ | | %si | server_IP (target address) | |
+ | | | %[bc_dst] | IP |
+ +---+------+------------------------------------------------------+---------+
+ | | %sp | server_port (target address) | |
+ | | | %[bc_dst_port] | numeric |
+ +---+------+------------------------------------------------------+---------+
+ | | %sq | srv_queue | numeric |
+ +---+------+------------------------------------------------------+---------+
+ | S | %sslc| ssl_ciphers (ex: AES-SHA) | |
+ | | | %[ssl_fc_cipher] | string |
+ +---+------+------------------------------------------------------+---------+
+ | S | %sslv| ssl_version (ex: TLSv1) | |
+ | | | %[ssl_fc_protocol] | string |
+ +---+------+------------------------------------------------------+---------+
+ | | %ts | termination_state | string |
+ | | | %[txn.sess_term_state] | |
+ +---+------+------------------------------------------------------+---------+
+ | H | %tsc | termination_state with cookie status | string |
+ +---+------+------------------------------------------------------+---------+
+
+ R = Restrictions : H = mode http only ; S = SSL only
+
+
+8.3. Advanced logging options
+-----------------------------
+
+Some advanced logging options are often looked for but are not easy to find out
+just by looking at the various options. Here is an entry point for the few
+options which can enable better logging. Please refer to the keywords reference
+for more information about their usage.
+
+
+8.3.1. Disabling logging of external tests
+------------------------------------------
+
+It is quite common to have some monitoring tools perform health checks on
+HAProxy. Sometimes it will be a layer 3 load-balancer such as LVS or any
+commercial load-balancer, and sometimes it will simply be a more complete
+monitoring system such as Nagios. When the tests are very frequent, users often
+ask how to disable logging for those checks. There are three possibilities :
+
+ - if connections come from everywhere and are just TCP probes, it is often
+ desired to simply disable logging of connections without data exchange, by
+ setting "option dontlognull" in the frontend. It also disables logging of
+ port scans, which may or may not be desired.
+
+ - it is possible to use the "http-request set-log-level silent" action using
+ a variety of conditions (source networks, paths, user-agents, etc).
+
+ - if the tests are performed on a known URI, use "monitor-uri" to declare
+ this URI as dedicated to monitoring. Any host sending this request will
+ only get the result of a health-check, and the request will not be logged.
+
+
+8.3.2. Logging before waiting for the stream to terminate
+----------------------------------------------------------
+
+The problem with logging at end of connection is that you have no clue about
+what is happening during very long streams, such as remote terminal sessions
+or large file downloads. This problem can be worked around by specifying
+"option logasap" in the frontend. HAProxy will then log as soon as possible,
+just before data transfer begins. This means that in case of TCP, it will still
+log the connection status to the server, and in case of HTTP, it will log just
+after processing the server headers. In this case, the number of bytes reported
+is the number of header bytes sent to the client. In order to avoid confusion
+with normal logs, the total time field and the number of bytes are prefixed
+with a '+' sign which means that real numbers are certainly larger.
+
+
+8.3.3. Raising log level upon errors
+------------------------------------
+
+Sometimes it is more convenient to separate normal traffic from errors logs,
+for instance in order to ease error monitoring from log files. When the option
+"log-separate-errors" is used, connections which experience errors, timeouts,
+retries, redispatches or HTTP status codes 5xx will see their syslog level
+raised from "info" to "err". This will help a syslog daemon store the log in
+a separate file. It is very important to keep the errors in the normal traffic
+file too, so that log ordering is not altered. You should also be careful if
+you already have configured your syslog daemon to store all logs higher than
+"notice" in an "admin" file, because the "err" level is higher than "notice".
+
+
+8.3.4. Disabling logging of successful connections
+--------------------------------------------------
+
+Although this may sound strange at first, some large sites have to deal with
+multiple thousands of logs per second and are experiencing difficulties keeping
+them intact for a long time or detecting errors within them. If the option
+"dontlog-normal" is set on the frontend, all normal connections will not be
+logged. In this regard, a normal connection is defined as one without any
+error, timeout, retry nor redispatch. In HTTP, the status code is checked too,
+and a response with a status 5xx is not considered normal and will be logged
+too. Of course, doing is is really discouraged as it will remove most of the
+useful information from the logs. Do this only if you have no other
+alternative.
+
+
+8.4. Timing events
+------------------
+
+Timers provide a great help in troubleshooting network problems. All values are
+reported in milliseconds (ms). These timers should be used in conjunction with
+the stream termination flags. In TCP mode with "option tcplog" set on the
+frontend, 3 control points are reported under the form "Tw/Tc/Tt", and in HTTP
+mode, 5 control points are reported under the form "TR/Tw/Tc/Tr/Ta". In
+addition, three other measures are provided, "Th", "Ti", and "Tq".
+
+Timings events in HTTP mode:
+
+ first request 2nd request
+ |<-------------------------------->|<-------------- ...
+ t tr t tr ...
+ ---|----|----|----|----|----|----|----|----|--
+ : Th Ti TR Tw Tc Tr Td : Ti ...
+ :<---- Tq ---->: :
+ :<-------------- Tt -------------->:
+ :<-- -----Tu--------------->:
+ :<--------- Ta --------->:
+
+Timings events in TCP mode:
+
+ TCP session
+ |<----------------->|
+ t t
+ ---|----|----|----|----|---
+ | Th Tw Tc Td |
+ |<------ Tt ------->|
+
+ - Th: total time to accept tcp connection and execute handshakes for low level
+ protocols. Currently, these protocols are proxy-protocol and SSL. This may
+ only happen once during the whole connection's lifetime. A large time here
+ may indicate that the client only pre-established the connection without
+ speaking, that it is experiencing network issues preventing it from
+ completing a handshake in a reasonable time (e.g. MTU issues), or that an
+ SSL handshake was very expensive to compute. Please note that this time is
+ reported only before the first request, so it is safe to average it over
+ all request to calculate the amortized value. The second and subsequent
+ request will always report zero here.
+
+ This timer is named %Th as a log-format tag, and fc.timer.handshake as a
+ sample fetch.
+
+ - Ti: is the idle time before the HTTP request (HTTP mode only). This timer
+ counts between the end of the handshakes and the first byte of the HTTP
+ request. When dealing with a second request in keep-alive mode, it starts
+ to count after the end of the transmission the previous response. When a
+ multiplexed protocol such as HTTP/2 is used, it starts to count immediately
+ after the previous request. Some browsers pre-establish connections to a
+ server in order to reduce the latency of a future request, and keep them
+ pending until they need it. This delay will be reported as the idle time. A
+ value of -1 indicates that nothing was received on the connection.
+
+ This timer is named %Ti as a log-format tag, and req.timer.idle as a
+ sample fetch.
+
+ - TR: total time to get the client request (HTTP mode only). It's the time
+ elapsed between the first bytes received and the moment the proxy received
+ the empty line marking the end of the HTTP headers. The value "-1"
+ indicates that the end of headers has never been seen. This happens when
+ the client closes prematurely or times out. This time is usually very short
+ since most requests fit in a single packet. A large time may indicate a
+ request typed by hand during a test.
+
+ This timer is named %TR as a log-format tag, and req.timer.hdr as a
+ sample fetch.
+
+ - Tq: total time to get the client request from the accept date or since the
+ emission of the last byte of the previous response (HTTP mode only). It's
+ exactly equal to Th + Ti + TR unless any of them is -1, in which case it
+ returns -1 as well. This timer used to be very useful before the arrival of
+ HTTP keep-alive and browsers' pre-connect feature. It's recommended to drop
+ it in favor of TR nowadays, as the idle time adds a lot of noise to the
+ reports.
+
+ This timer is named %Tq as a log-format tag, and req.timer.tq as a
+ sample fetch.
+
+ - Tw: total time spent in the queues waiting for a connection slot. It
+ accounts for backend queue as well as the server queues, and depends on the
+ queue size, and the time needed for the server to complete previous
+ requests. The value "-1" means that the request was killed before reaching
+ the queue, which is generally what happens with invalid or denied requests.
+
+ This timer is named %Tw as a log-format tag, and req.timer.queue as a
+ sample fetch.
+
+ - Tc: total time to establish the TCP connection to the server. It's the time
+ elapsed between the moment the proxy sent the connection request, and the
+ moment it was acknowledged by the server, or between the TCP SYN packet and
+ the matching SYN/ACK packet in return. The value "-1" means that the
+ connection never established.
+
+ This timer is named %Tc as a log-format tag, and bc.timer.connect as a
+ sample fetch.
+
+ - Tr: server response time (HTTP mode only). It's the time elapsed between
+ the moment the TCP connection was established to the server and the moment
+ the server sent its complete response headers. It purely shows its request
+ processing time, without the network overhead due to the data transmission.
+ It is worth noting that when the client has data to send to the server, for
+ instance during a POST request, the time already runs, and this can distort
+ apparent response time. For this reason, it's generally wise not to trust
+ too much this field for POST requests initiated from clients behind an
+ untrusted network. A value of "-1" here means that the last the response
+ header (empty line) was never seen, most likely because the server timeout
+ stroke before the server managed to process the request.
+
+ This timer is named %Tr as a log-format tag, and res.timer.hdr as a
+ sample fetch.
+
+ - Td: this is the total transfer time of the response payload till the last
+ byte sent to the client. In HTTP it starts after the last response header
+ (after Tr).
+
+ The data sent are not guaranteed to be received by the client, they can be
+ stuck in either the kernel or the network.
+
+ This timer is named %Td as a log-format tag, and res.timer.data as a
+ sample fetch.
+
+ - Ta: total active time for the HTTP request, between the moment the proxy
+ received the first byte of the request header and the emission of the last
+ byte of the response body. The exception is when the "logasap" option is
+ specified. In this case, it only equals (TR+Tw+Tc+Tr), and is prefixed with
+ a '+' sign. From this field, we can deduce "Td", the data transmission time,
+ by subtracting other timers when valid :
+
+ Td = Ta - (TR + Tw + Tc + Tr)
+
+ Timers with "-1" values have to be excluded from this equation. Note that
+ "Ta" can never be negative.
+
+ This timer is named %Ta as a log-format tag, and txn.timer.total as a
+ sample fetch.
+
+ - Tt: total stream duration time, between the moment the proxy accepted it
+ and the moment both ends were closed. The exception is when the "logasap"
+ option is specified. In this case, it only equals (Th+Ti+TR+Tw+Tc+Tr), and
+ is prefixed with a '+' sign. From this field, we can deduce "Td", the data
+ transmission time, by subtracting other timers when valid :
+
+ Td = Tt - (Th + Ti + TR + Tw + Tc + Tr)
+
+ Timers with "-1" values have to be excluded from this equation. In TCP
+ mode, "Ti", "Tq" and "Tr" have to be excluded too. Note that "Tt" can never
+ be negative and that for HTTP, Tt is simply equal to (Th+Ti+Ta).
+
+ This timer is named %Tt as a log-format tag, and fc.timer.total as a
+ sample fetch.
+
+ - Tu: total estimated time as seen from client, between the moment the proxy
+ accepted it and the moment both ends were closed, without idle time.
+ This is useful to roughly measure end-to-end time as a user would see it,
+ without idle time pollution from keep-alive time between requests. This
+ timer in only an estimation of time seen by user as it assumes network
+ latency is the same in both directions. The exception is when the "logasap"
+ option is specified. In this case, it only equals (Th+TR+Tw+Tc+Tr), and is
+ prefixed with a '+' sign.
+
+ This timer is named %Tu as a log-format tag, and txn.timer.user as a
+ sample fetch.
+
+These timers provide precious indications on trouble causes. Since the TCP
+protocol defines retransmit delays of 3, 6, 12... seconds, we know for sure
+that timers close to multiples of 3s are nearly always related to lost packets
+due to network problems (wires, negotiation, congestion). Moreover, if "Ta" or
+"Tt" is close to a timeout value specified in the configuration, it often means
+that a stream has been aborted on timeout.
+
+Most common cases :
+
+ - If "Th" or "Ti" are close to 3000, a packet has probably been lost between
+ the client and the proxy. This is very rare on local networks but might
+ happen when clients are on far remote networks and send large requests. It
+ may happen that values larger than usual appear here without any network
+ cause. Sometimes, during an attack or just after a resource starvation has
+ ended, HAProxy may accept thousands of connections in a few milliseconds.
+ The time spent accepting these connections will inevitably slightly delay
+ processing of other connections, and it can happen that request times in the
+ order of a few tens of milliseconds are measured after a few thousands of
+ new connections have been accepted at once. Using one of the keep-alive
+ modes may display larger idle times since "Ti" measures the time spent
+ waiting for additional requests.
+
+ - If "Tc" is close to 3000, a packet has probably been lost between the
+ server and the proxy during the server connection phase. This value should
+ always be very low, such as 1 ms on local networks and less than a few tens
+ of ms on remote networks.
+
+ - If "Tr" is nearly always lower than 3000 except some rare values which seem
+ to be the average majored by 3000, there are probably some packets lost
+ between the proxy and the server.
+
+ - If "Ta" is large even for small byte counts, it generally is because
+ neither the client nor the server decides to close the connection while
+ HAProxy is running in tunnel mode and both have agreed on a keep-alive
+ connection mode. In order to solve this issue, it will be needed to specify
+ one of the HTTP options to manipulate keep-alive or close options on either
+ the frontend or the backend. Having the smallest possible 'Ta' or 'Tt' is
+ important when connection regulation is used with the "maxconn" option on
+ the servers, since no new connection will be sent to the server until
+ another one is released.
+
+Other noticeable HTTP log cases ('xx' means any value to be ignored) :
+
+ TR/Tw/Tc/Tr/+Ta The "option logasap" is present on the frontend and the log
+ was emitted before the data phase. All the timers are valid
+ except "Ta" which is shorter than reality.
+
+ -1/xx/xx/xx/Ta The client was not able to send a complete request in time
+ or it aborted too early. Check the stream termination flags
+ then "timeout http-request" and "timeout client" settings.
+
+ TR/-1/xx/xx/Ta It was not possible to process the request, maybe because
+ servers were out of order, because the request was invalid
+ or forbidden by ACL rules. Check the stream termination
+ flags.
+
+ TR/Tw/-1/xx/Ta The connection could not establish on the server. Either it
+ actively refused it or it timed out after Ta-(TR+Tw) ms.
+ Check the stream termination flags, then check the
+ "timeout connect" setting. Note that the tarpit action might
+ return similar-looking patterns, with "Tw" equal to the time
+ the client connection was maintained open.
+
+ TR/Tw/Tc/-1/Ta The server has accepted the connection but did not return
+ a complete response in time, or it closed its connection
+ unexpectedly after Ta-(TR+Tw+Tc) ms. Check the stream
+ termination flags, then check the "timeout server" setting.
+
+
+8.5. Stream state at disconnection
+-----------------------------------
+
+TCP and HTTP logs provide a stream termination indicator in the
+"termination_state" field, just before the number of active connections. It is
+2-characters long in TCP mode, and is extended to 4 characters in HTTP mode,
+each of which has a special meaning :
+
+ - On the first character, a code reporting the first event which caused the
+ stream to terminate :
+
+ C : the TCP session was unexpectedly aborted by the client.
+
+ S : the TCP session was unexpectedly aborted by the server, or the
+ server explicitly refused it.
+
+ P : the stream or session was prematurely aborted by the proxy, because
+ of a connection limit enforcement, because a DENY filter was
+ matched, because of a security check which detected and blocked a
+ dangerous error in server response which might have caused
+ information leak (e.g. cacheable cookie).
+
+ L : the stream was locally processed by HAProxy.
+
+ R : a resource on the proxy has been exhausted (memory, sockets, source
+ ports, ...). Usually, this appears during the connection phase, and
+ system logs should contain a copy of the precise error. If this
+ happens, it must be considered as a very serious anomaly which
+ should be fixed as soon as possible by any means.
+
+ I : an internal error was identified by the proxy during a self-check.
+ This should NEVER happen, and you are encouraged to report any log
+ containing this, because this would almost certainly be a bug. It
+ would be wise to preventively restart the process after such an
+ event too, in case it would be caused by memory corruption.
+
+ D : the stream was killed by HAProxy because the server was detected
+ as down and was configured to kill all connections when going down.
+
+ U : the stream was killed by HAProxy on this backup server because an
+ active server was detected as up and was configured to kill all
+ backup connections when going up.
+
+ K : the stream was actively killed by an admin operating on HAProxy.
+
+ c : the client-side timeout expired while waiting for the client to
+ send or receive data.
+
+ s : the server-side timeout expired while waiting for the server to
+ send or receive data.
+
+ - : normal stream completion, both the client and the server closed
+ with nothing left in the buffers.
+
+ - on the second character, the TCP or HTTP stream state when it was closed :
+
+ R : the proxy was waiting for a complete, valid REQUEST from the client
+ (HTTP mode only). Nothing was sent to any server.
+
+ Q : the proxy was waiting in the QUEUE for a connection slot. This can
+ only happen when servers have a 'maxconn' parameter set. It can
+ also happen in the global queue after a redispatch consecutive to
+ a failed attempt to connect to a dying server. If no redispatch is
+ reported, then no connection attempt was made to any server.
+
+ C : the proxy was waiting for the CONNECTION to establish on the
+ server. The server might at most have noticed a connection attempt.
+
+ H : the proxy was waiting for complete, valid response HEADERS from the
+ server (HTTP only).
+
+ D : the stream was in the DATA phase.
+
+ L : the proxy was still transmitting LAST data to the client while the
+ server had already finished. This one is very rare as it can only
+ happen when the client dies while receiving the last packets.
+
+ T : the request was tarpitted. It has been held open with the client
+ during the whole "timeout tarpit" duration or until the client
+ closed, both of which will be reported in the "Tw" timer.
+
+ - : normal stream completion after end of data transfer.
+
+ - the third character tells whether the persistence cookie was provided by
+ the client (only in HTTP mode) :
+
+ N : the client provided NO cookie. This is usually the case for new
+ visitors, so counting the number of occurrences of this flag in the
+ logs generally indicate a valid trend for the site frequentation.
+
+ I : the client provided an INVALID cookie matching no known server.
+ This might be caused by a recent configuration change, mixed
+ cookies between HTTP/HTTPS sites, persistence conditionally
+ ignored, or an attack.
+
+ D : the client provided a cookie designating a server which was DOWN,
+ so either "option persist" was used and the client was sent to
+ this server, or it was not set and the client was redispatched to
+ another server.
+
+ V : the client provided a VALID cookie, and was sent to the associated
+ server.
+
+ E : the client provided a valid cookie, but with a last date which was
+ older than what is allowed by the "maxidle" cookie parameter, so
+ the cookie is consider EXPIRED and is ignored. The request will be
+ redispatched just as if there was no cookie.
+
+ O : the client provided a valid cookie, but with a first date which was
+ older than what is allowed by the "maxlife" cookie parameter, so
+ the cookie is consider too OLD and is ignored. The request will be
+ redispatched just as if there was no cookie.
+
+ U : a cookie was present but was not used to select the server because
+ some other server selection mechanism was used instead (typically a
+ "use-server" rule).
+
+ - : does not apply (no cookie set in configuration).
+
+ - the last character reports what operations were performed on the persistence
+ cookie returned by the server (only in HTTP mode) :
+
+ N : NO cookie was provided by the server, and none was inserted either.
+
+ I : no cookie was provided by the server, and the proxy INSERTED one.
+ Note that in "cookie insert" mode, if the server provides a cookie,
+ it will still be overwritten and reported as "I" here.
+
+ U : the proxy UPDATED the last date in the cookie that was presented by
+ the client. This can only happen in insert mode with "maxidle". It
+ happens every time there is activity at a different date than the
+ date indicated in the cookie. If any other change happens, such as
+ a redispatch, then the cookie will be marked as inserted instead.
+
+ P : a cookie was PROVIDED by the server and transmitted as-is.
+
+ R : the cookie provided by the server was REWRITTEN by the proxy, which
+ happens in "cookie rewrite" or "cookie prefix" modes.
+
+ D : the cookie provided by the server was DELETED by the proxy.
+
+ - : does not apply (no cookie set in configuration).
+
+The combination of the two first flags gives a lot of information about what
+was happening when the stream or session terminated, and why it did terminate.
+It can be helpful to detect server saturation, network troubles, local system
+resource starvation, attacks, etc...
+
+The most common termination flags combinations are indicated below. They are
+alphabetically sorted, with the lowercase set just after the upper case for
+easier finding and understanding.
+
+ Flags Reason
+
+ -- Normal termination.
+
+ CC The client aborted before the connection could be established to the
+ server. This can happen when HAProxy tries to connect to a recently
+ dead (or unchecked) server, and the client aborts while HAProxy is
+ waiting for the server to respond or for "timeout connect" to expire.
+
+ CD The client unexpectedly aborted during data transfer. This can be
+ caused by a browser crash, by an intermediate equipment between the
+ client and HAProxy which decided to actively break the connection,
+ by network routing issues between the client and HAProxy, or by a
+ keep-alive stream between the server and the client terminated first
+ by the client.
+
+ cD The client did not send nor acknowledge any data for as long as the
+ "timeout client" delay. This is often caused by network failures on
+ the client side, or the client simply leaving the net uncleanly.
+
+ CH The client aborted while waiting for the server to start responding.
+ It might be the server taking too long to respond or the client
+ clicking the 'Stop' button too fast.
+
+ cH The "timeout client" stroke while waiting for client data during a
+ POST request. This is sometimes caused by too large TCP MSS values
+ for PPPoE networks which cannot transport full-sized packets. It can
+ also happen when client timeout is smaller than server timeout and
+ the server takes too long to respond.
+
+ CQ The client aborted while its stream was queued, waiting for a server
+ with enough empty slots to accept it. It might be that either all the
+ servers were saturated or that the assigned server was taking too
+ long a time to respond.
+
+ CR The client aborted before sending a full HTTP request. Most likely
+ the request was typed by hand using a telnet client, and aborted
+ too early. The HTTP status code is likely a 400 here. Sometimes this
+ might also be caused by an IDS killing the connection between HAProxy
+ and the client. "option http-ignore-probes" can be used to ignore
+ connections without any data transfer.
+
+ cR The "timeout http-request" stroke before the client sent a full HTTP
+ request. This is sometimes caused by too large TCP MSS values on the
+ client side for PPPoE networks which cannot transport full-sized
+ packets, or by clients sending requests by hand and not typing fast
+ enough, or forgetting to enter the empty line at the end of the
+ request. The HTTP status code is likely a 408 here. Note: recently,
+ some browsers started to implement a "pre-connect" feature consisting
+ in speculatively connecting to some recently visited web sites just
+ in case the user would like to visit them. This results in many
+ connections being established to web sites, which end up in 408
+ Request Timeout if the timeout strikes first, or 400 Bad Request when
+ the browser decides to close them first. These ones pollute the log
+ and feed the error counters. Some versions of some browsers have even
+ been reported to display the error code. It is possible to work
+ around the undesirable effects of this behavior by adding "option
+ http-ignore-probes" in the frontend, resulting in connections with
+ zero data transfer to be totally ignored. This will definitely hide
+ the errors of people experiencing connectivity issues though.
+
+ CT The client aborted while its stream was tarpitted. It is important to
+ check if this happens on valid requests, in order to be sure that no
+ wrong tarpit rules have been written. If a lot of them happen, it
+ might make sense to lower the "timeout tarpit" value to something
+ closer to the average reported "Tw" timer, in order not to consume
+ resources for just a few attackers.
+
+ LC The request was intercepted and locally handled by HAProxy. The
+ request was not sent to the server. It only happens with a redirect
+ because of a "redir" parameter on the server line.
+
+ LR The request was intercepted and locally handled by HAProxy. The
+ request was not sent to the server. Generally it means a redirect was
+ returned, an HTTP return statement was processed or the request was
+ handled by an applet (stats, cache, Prometheus exported, lua applet...).
+
+ LH The response was intercepted and locally handled by HAProxy. Generally
+ it means a redirect was returned or an HTTP return statement was
+ processed.
+
+ SC The server or an equipment between it and HAProxy explicitly refused
+ the TCP connection (the proxy received a TCP RST or an ICMP message
+ in return). Under some circumstances, it can also be the network
+ stack telling the proxy that the server is unreachable (e.g. no route,
+ or no ARP response on local network). When this happens in HTTP mode,
+ the status code is likely a 502 or 503 here.
+
+ sC The "timeout connect" stroke before a connection to the server could
+ complete. When this happens in HTTP mode, the status code is likely a
+ 503 or 504 here.
+
+ SD The connection to the server died with an error during the data
+ transfer. This usually means that HAProxy has received an RST from
+ the server or an ICMP message from an intermediate equipment while
+ exchanging data with the server. This can be caused by a server crash
+ or by a network issue on an intermediate equipment.
+
+ sD The server did not send nor acknowledge any data for as long as the
+ "timeout server" setting during the data phase. This is often caused
+ by too short timeouts on L4 equipment before the server (firewalls,
+ load-balancers, ...), as well as keep-alive sessions maintained
+ between the client and the server expiring first on HAProxy.
+
+ SH The server aborted before sending its full HTTP response headers, or
+ it crashed while processing the request. Since a server aborting at
+ this moment is very rare, it would be wise to inspect its logs to
+ control whether it crashed and why. The logged request may indicate a
+ small set of faulty requests, demonstrating bugs in the application.
+ Sometimes this might also be caused by an IDS killing the connection
+ between HAProxy and the server.
+
+ sH The "timeout server" stroke before the server could return its
+ response headers. This is the most common anomaly, indicating too
+ long transactions, probably caused by server or database saturation.
+ The immediate workaround consists in increasing the "timeout server"
+ setting, but it is important to keep in mind that the user experience
+ will suffer from these long response times. The only long term
+ solution is to fix the application.
+
+ sQ The stream spent too much time in queue and has been expired. See
+ the "timeout queue" and "timeout connect" settings to find out how to
+ fix this if it happens too often. If it often happens massively in
+ short periods, it may indicate general problems on the affected
+ servers due to I/O or database congestion, or saturation caused by
+ external attacks.
+
+ PC The proxy refused to establish a connection to the server because the
+ process's socket limit has been reached while attempting to connect.
+ The global "maxconn" parameter may be increased in the configuration
+ so that it does not happen anymore. This status is very rare and
+ might happen when the global "ulimit-n" parameter is forced by hand.
+
+ PD The proxy blocked an incorrectly formatted chunked encoded message in
+ a request or a response, after the server has emitted its headers. In
+ most cases, this will indicate an invalid message from the server to
+ the client. HAProxy supports chunk sizes of up to 2GB - 1 (2147483647
+ bytes). Any larger size will be considered as an error.
+
+ PH The proxy blocked the server's response, because it was invalid,
+ incomplete, dangerous (cache control), or matched a security filter.
+ In any case, an HTTP 502 error is sent to the client. One possible
+ cause for this error is an invalid syntax in an HTTP header name
+ containing unauthorized characters. It is also possible but quite
+ rare, that the proxy blocked a chunked-encoding request from the
+ client due to an invalid syntax, before the server responded. In this
+ case, an HTTP 400 error is sent to the client and reported in the
+ logs. Finally, it may be due to an HTTP header rewrite failure on the
+ response. In this case, an HTTP 500 error is sent (see
+ "tune.maxrewrite" and "http-response strict-mode" for more
+ inforomation).
+
+ PR The proxy blocked the client's HTTP request, either because of an
+ invalid HTTP syntax, in which case it returned an HTTP 400 error to
+ the client, or because a deny filter matched, in which case it
+ returned an HTTP 403 error. It may also be due to an HTTP header
+ rewrite failure on the request. In this case, an HTTP 500 error is
+ sent (see "tune.maxrewrite" and "http-request strict-mode" for more
+ inforomation).
+
+ PT The proxy blocked the client's request and has tarpitted its
+ connection before returning it a 500 server error. Nothing was sent
+ to the server. The connection was maintained open for as long as
+ reported by the "Tw" timer field.
+
+ RC A local resource has been exhausted (memory, sockets, source ports)
+ preventing the connection to the server from establishing. The error
+ logs will tell precisely what was missing. This is very rare and can
+ only be solved by proper system tuning.
+
+The combination of the two last flags gives a lot of information about how
+persistence was handled by the client, the server and by HAProxy. This is very
+important to troubleshoot disconnections, when users complain they have to
+re-authenticate. The commonly encountered flags are :
+
+ -- Persistence cookie is not enabled.
+
+ NN No cookie was provided by the client, none was inserted in the
+ response. For instance, this can be in insert mode with "postonly"
+ set on a GET request.
+
+ II A cookie designating an invalid server was provided by the client,
+ a valid one was inserted in the response. This typically happens when
+ a "server" entry is removed from the configuration, since its cookie
+ value can be presented by a client when no other server knows it.
+
+ NI No cookie was provided by the client, one was inserted in the
+ response. This typically happens for first requests from every user
+ in "insert" mode, which makes it an easy way to count real users.
+
+ VN A cookie was provided by the client, none was inserted in the
+ response. This happens for most responses for which the client has
+ already got a cookie.
+
+ VU A cookie was provided by the client, with a last visit date which is
+ not completely up-to-date, so an updated cookie was provided in
+ response. This can also happen if there was no date at all, or if
+ there was a date but the "maxidle" parameter was not set, so that the
+ cookie can be switched to unlimited time.
+
+ EI A cookie was provided by the client, with a last visit date which is
+ too old for the "maxidle" parameter, so the cookie was ignored and a
+ new cookie was inserted in the response.
+
+ OI A cookie was provided by the client, with a first visit date which is
+ too old for the "maxlife" parameter, so the cookie was ignored and a
+ new cookie was inserted in the response.
+
+ DI The server designated by the cookie was down, a new server was
+ selected and a new cookie was emitted in the response.
+
+ VI The server designated by the cookie was not marked dead but could not
+ be reached. A redispatch happened and selected another one, which was
+ then advertised in the response.
+
+
+8.6. Non-printable characters
+-----------------------------
+
+In order not to cause trouble to log analysis tools or terminals during log
+consulting, non-printable characters are not sent as-is into log files, but are
+converted to the two-digits hexadecimal representation of their ASCII code,
+prefixed by the character '#'. The only characters that can be logged without
+being escaped are comprised between 32 and 126 (inclusive). Obviously, the
+escape character '#' itself is also encoded to avoid any ambiguity ("#23"). It
+is the same for the character '"' which becomes "#22", as well as '{', '|' and
+'}' when logging headers.
+
+Note that the space character (' ') is not encoded in headers, which can cause
+issues for tools relying on space count to locate fields. A typical header
+containing spaces is "User-Agent".
+
+Last, it has been observed that some syslog daemons such as syslog-ng escape
+the quote ('"') with a backslash ('\'). The reverse operation can safely be
+performed since no quote may appear anywhere else in the logs.
+
+
+8.7. Capturing HTTP cookies
+---------------------------
+
+Cookie capture simplifies the tracking a complete user session. This can be
+achieved using the "capture cookie" statement in the frontend. Please refer to
+section 4.2 for more details. Only one cookie can be captured, and the same
+cookie will simultaneously be checked in the request ("Cookie:" header) and in
+the response ("Set-Cookie:" header). The respective values will be reported in
+the HTTP logs at the "captured_request_cookie" and "captured_response_cookie"
+locations (see section 8.2.3 about HTTP log format). When either cookie is
+not seen, a dash ('-') replaces the value. This way, it's easy to detect when a
+user switches to a new session for example, because the server will reassign it
+a new cookie. It is also possible to detect if a server unexpectedly sets a
+wrong cookie to a client, leading to session crossing.
+
+ Examples :
+ # capture the first cookie whose name starts with "ASPSESSION"
+ capture cookie ASPSESSION len 32
+
+ # capture the first cookie whose name is exactly "vgnvisitor"
+ capture cookie vgnvisitor= len 32
+
+
+8.8. Capturing HTTP headers
+---------------------------
+
+Header captures are useful to track unique request identifiers set by an upper
+proxy, virtual host names, user-agents, POST content-length, referrers, etc. In
+the response, one can search for information about the response length, how the
+server asked the cache to behave, or an object location during a redirection.
+
+Header captures are performed using the "capture request header" and "capture
+response header" statements in the frontend. Please consult their definition in
+section 4.2 for more details.
+
+It is possible to include both request headers and response headers at the same
+time. Non-existent headers are logged as empty strings, and if one header
+appears more than once, only its last occurrence will be logged. Request headers
+are grouped within braces '{' and '}' in the same order as they were declared,
+and delimited with a vertical bar '|' without any space. Response headers
+follow the same representation, but are displayed after a space following the
+request headers block. These blocks are displayed just before the HTTP request
+in the logs.
+
+As a special case, it is possible to specify an HTTP header capture in a TCP
+frontend. The purpose is to enable logging of headers which will be parsed in
+an HTTP backend if the request is then switched to this HTTP backend.
+
+ Example :
+ # This instance chains to the outgoing proxy
+ listen proxy-out
+ mode http
+ option httplog
+ option logasap
+ log global
+ server cache1 192.168.1.1:3128
+
+ # log the name of the virtual server
+ capture request header Host len 20
+
+ # log the amount of data uploaded during a POST
+ capture request header Content-Length len 10
+
+ # log the beginning of the referrer
+ capture request header Referer len 20
+
+ # server name (useful for outgoing proxies only)
+ capture response header Server len 20
+
+ # logging the content-length is useful with "option logasap"
+ capture response header Content-Length len 10
+
+ # log the expected cache behavior on the response
+ capture response header Cache-Control len 8
+
+ # the Via header will report the next proxy's name
+ capture response header Via len 20
+
+ # log the URL location during a redirection
+ capture response header Location len 20
+
+ >>> Aug 9 20:26:09 localhost \
+ haproxy[2022]: 127.0.0.1:34014 [09/Aug/2004:20:26:09] proxy-out \
+ proxy-out/cache1 0/0/0/162/+162 200 +350 - - ---- 0/0/0/0/0 0/0 \
+ {fr.adserver.yahoo.co||http://fr.f416.mail.} {|864|private||} \
+ "GET http://fr.adserver.yahoo.com/"
+
+ >>> Aug 9 20:30:46 localhost \
+ haproxy[2022]: 127.0.0.1:34020 [09/Aug/2004:20:30:46] proxy-out \
+ proxy-out/cache1 0/0/0/182/+182 200 +279 - - ---- 0/0/0/0/0 0/0 \
+ {w.ods.org||} {Formilux/0.1.8|3495|||} \
+ "GET http://trafic.1wt.eu/ HTTP/1.1"
+
+ >>> Aug 9 20:30:46 localhost \
+ haproxy[2022]: 127.0.0.1:34028 [09/Aug/2004:20:30:46] proxy-out \
+ proxy-out/cache1 0/0/2/126/+128 301 +223 - - ---- 0/0/0/0/0 0/0 \
+ {www.sytadin.equipement.gouv.fr||http://trafic.1wt.eu/} \
+ {Apache|230|||http://www.sytadin.} \
+ "GET http://www.sytadin.equipement.gouv.fr/ HTTP/1.1"
+
+
+8.9. Examples of logs
+---------------------
+
+These are real-world examples of logs accompanied with an explanation. Some of
+them have been made up by hand. The syslog part has been removed for better
+reading. Their sole purpose is to explain how to decipher them.
+
+ >>> haproxy[674]: 127.0.0.1:33318 [15/Oct/2003:08:31:57.130] px-http \
+ px-http/srv1 6559/0/7/147/6723 200 243 - - ---- 5/3/3/1/0 0/0 \
+ "HEAD / HTTP/1.0"
+
+ => long request (6.5s) entered by hand through 'telnet'. The server replied
+ in 147 ms, and the session ended normally ('----')
+
+ >>> haproxy[674]: 127.0.0.1:33319 [15/Oct/2003:08:31:57.149] px-http \
+ px-http/srv1 6559/1230/7/147/6870 200 243 - - ---- 324/239/239/99/0 \
+ 0/9 "HEAD / HTTP/1.0"
+
+ => Idem, but the request was queued in the global queue behind 9 other
+ requests, and waited there for 1230 ms.
+
+ >>> haproxy[674]: 127.0.0.1:33320 [15/Oct/2003:08:32:17.654] px-http \
+ px-http/srv1 9/0/7/14/+30 200 +243 - - ---- 3/3/3/1/0 0/0 \
+ "GET /image.iso HTTP/1.0"
+
+ => request for a long data transfer. The "logasap" option was specified, so
+ the log was produced just before transferring data. The server replied in
+ 14 ms, 243 bytes of headers were sent to the client, and total time from
+ accept to first data byte is 30 ms.
+
+ >>> haproxy[674]: 127.0.0.1:33320 [15/Oct/2003:08:32:17.925] px-http \
+ px-http/srv1 9/0/7/14/30 502 243 - - PH-- 3/2/2/0/0 0/0 \
+ "GET /cgi-bin/bug.cgi? HTTP/1.0"
+
+ => the proxy blocked a server response either because of an "http-response
+ deny" rule, or because the response was improperly formatted and not
+ HTTP-compliant, or because it blocked sensitive information which risked
+ being cached. In this case, the response is replaced with a "502 bad
+ gateway". The flags ("PH--") tell us that it was HAProxy who decided to
+ return the 502 and not the server.
+
+ >>> haproxy[18113]: 127.0.0.1:34548 [15/Oct/2003:15:18:55.798] px-http \
+ px-http/<NOSRV> -1/-1/-1/-1/8490 -1 0 - - CR-- 2/2/2/0/0 0/0 ""
+
+ => the client never completed its request and aborted itself ("C---") after
+ 8.5s, while the proxy was waiting for the request headers ("-R--").
+ Nothing was sent to any server.
+
+ >>> haproxy[18113]: 127.0.0.1:34549 [15/Oct/2003:15:19:06.103] px-http \
+ px-http/<NOSRV> -1/-1/-1/-1/50001 408 0 - - cR-- 2/2/2/0/0 0/0 ""
+
+ => The client never completed its request, which was aborted by the
+ time-out ("c---") after 50s, while the proxy was waiting for the request
+ headers ("-R--"). Nothing was sent to any server, but the proxy could
+ send a 408 return code to the client.
+
+ >>> haproxy[18989]: 127.0.0.1:34550 [15/Oct/2003:15:24:28.312] px-tcp \
+ px-tcp/srv1 0/0/5007 0 cD 0/0/0/0/0 0/0
+
+ => This log was produced with "option tcplog". The client timed out after
+ 5 seconds ("c----").
+
+ >>> haproxy[18989]: 10.0.0.1:34552 [15/Oct/2003:15:26:31.462] px-http \
+ px-http/srv1 3183/-1/-1/-1/11215 503 0 - - SC-- 205/202/202/115/3 \
+ 0/0 "HEAD / HTTP/1.0"
+
+ => The request took 3s to complete (probably a network problem), and the
+ connection to the server failed ('SC--') after 4 attempts of 2 seconds
+ (config says 'retries 3'), and no redispatch (otherwise we would have
+ seen "/+3"). Status code 503 was returned to the client. There were 115
+ connections on this server, 202 connections on this proxy, and 205 on
+ the global process. It is possible that the server refused the
+ connection because of too many already established.
+
+
+9. Supported filters
+--------------------
+
+Here are listed officially supported filters with the list of parameters they
+accept. Depending on compile options, some of these filters might be
+unavailable. The list of available filters is reported in haproxy -vv.
+
+See also : "filter"
+
+9.1. Trace
+----------
+
+filter trace [name <name>] [random-forwarding] [hexdump]
+
+ Arguments:
+ <name> is an arbitrary name that will be reported in
+ messages. If no name is provided, "TRACE" is used.
+
+ <quiet> inhibits trace messages.
+
+ <random-forwarding> enables the random forwarding of parsed data. By
+ default, this filter forwards all previously parsed
+ data. With this parameter, it only forwards a random
+ amount of the parsed data.
+
+ <hexdump> dumps all forwarded data to the server and the client.
+
+This filter can be used as a base to develop new filters. It defines all
+callbacks and print a message on the standard error stream (stderr) with useful
+information for all of them. It may be useful to debug the activity of other
+filters or, quite simply, HAProxy's activity.
+
+Using <random-parsing> and/or <random-forwarding> parameters is a good way to
+tests the behavior of a filter that parses data exchanged between a client and
+a server by adding some latencies in the processing.
+
+
+9.2. HTTP compression
+---------------------
+
+filter compression
+
+The HTTP compression has been moved in a filter in HAProxy 1.7. "compression"
+keyword must still be used to enable and configure the HTTP compression. And
+when no other filter is used, it is enough. When used with the cache or the
+fcgi-app enabled, it is also enough. In this case, the compression is always
+done after the response is stored in the cache. But it is mandatory to
+explicitly use a filter line to enable the HTTP compression when at least one
+filter other than the cache or the fcgi-app is used for the same
+listener/frontend/backend. This is important to know the filters evaluation
+order.
+
+See also : "compression", section 9.4 about the cache filter and section 9.5
+ about the fcgi-app filter.
+
+
+9.3. Stream Processing Offload Engine (SPOE)
+--------------------------------------------
+
+filter spoe [engine <name>] config <file>
+
+ Arguments :
+
+ <name> is the engine name that will be used to find the right scope in
+ the configuration file. If not provided, all the file will be
+ parsed.
+
+ <file> is the path of the engine configuration file. This file can
+ contain configuration of several engines. In this case, each
+ part must be placed in its own scope.
+
+The Stream Processing Offload Engine (SPOE) is a filter communicating with
+external components. It allows the offload of some specifics processing on the
+streams in tiered applications. These external components and information
+exchanged with them are configured in dedicated files, for the main part. It
+also requires dedicated backends, defined in HAProxy configuration.
+
+SPOE communicates with external components using an in-house binary protocol,
+the Stream Processing Offload Protocol (SPOP).
+
+For all information about the SPOE configuration and the SPOP specification, see
+"doc/SPOE.txt".
+
+9.4. Cache
+----------
+
+filter cache <name>
+
+ Arguments :
+
+ <name> is name of the cache section this filter will use.
+
+The cache uses a filter to store cacheable responses. The HTTP rules
+"cache-store" and "cache-use" must be used to define how and when to use a
+cache. By default the corresponding filter is implicitly defined. And when no
+other filters than fcgi-app or compression are used, it is enough. In such
+case, the compression filter is always evaluated after the cache filter. But it
+is mandatory to explicitly use a filter line to use a cache when at least one
+filter other than the compression or the fcgi-app is used for the same
+listener/frontend/backend. This is important to know the filters evaluation
+order.
+
+See also : section 9.2 about the compression filter, section 9.5 about the
+ fcgi-app filter and section 6 about cache.
+
+
+9.5. Fcgi-app
+-------------
+
+filter fcgi-app <name>
+
+ Arguments :
+
+ <name> is name of the fcgi-app section this filter will use.
+
+The FastCGI application uses a filter to evaluate all custom parameters on the
+request path, and to process the headers on the response path. the <name> must
+reference an existing fcgi-app section. The directive "use-fcgi-app" should be
+used to define the application to use. By default the corresponding filter is
+implicitly defined. And when no other filters than cache or compression are
+used, it is enough. But it is mandatory to explicitly use a filter line to a
+fcgi-app when at least one filter other than the compression or the cache is
+used for the same backend. This is important to know the filters evaluation
+order.
+
+See also: "use-fcgi-app", section 9.2 about the compression filter, section 9.4
+ about the cache filter and section 10 about FastCGI application.
+
+
+9.6. OpenTracing
+----------------
+
+The OpenTracing filter adds native support for using distributed tracing in
+HAProxy. This is enabled by sending an OpenTracing compliant request to one
+of the supported tracers such as Datadog, Jaeger, Lightstep and Zipkin tracers.
+Please note: tracers are not listed by any preference, but alphabetically.
+
+This feature is only enabled when HAProxy was built with USE_OT=1.
+
+The OpenTracing filter activation is done explicitly by specifying it in the
+HAProxy configuration. If this is not done, the OpenTracing filter in no way
+participates in the work of HAProxy.
+
+filter opentracing [id <id>] config <file>
+
+ Arguments :
+
+ <id> is the OpenTracing filter id that will be used to find the
+ right scope in the configuration file. If no filter id is
+ specified, 'ot-filter' is used as default. If scope is not
+ specified in the configuration file, it applies to all defined
+ OpenTracing filters.
+
+ <file> is the path of the OpenTracing configuration file. The same
+ file can contain configurations for multiple OpenTracing
+ filters simultaneously. In that case we do not need to define
+ scope so the same configuration applies to all filters or each
+ filter must have its own scope defined.
+
+More detailed documentation related to the operation, configuration and use
+of the filter can be found in the addons/ot directory.
+
+9.7. Bandwidth limitation
+--------------------------
+
+filter bwlim-in <name> default-limit <size> default-period <time> [min-size <sz>]
+filter bwlim-out <name> default-limit <size> default-period <time> [min-size <sz>]
+filter bwlim-in <name> limit <size> key <pattern> [table <table>] [min-size <sz>]
+filter bwlim-out <name> limit <size> key <pattern> [table <table>] [min-size <sz>]
+
+ Arguments :
+
+ <name> is the filter name that will be used by 'set-bandwidth-limit'
+ actions to reference a specific bandwidth limitation filter.
+
+ <size> is max number of bytes that can be forwarded over the period.
+ The value must be specified for per-stream and shared bandwidth
+ limitation filters. It follows the HAProxy size format and is
+ expressed in bytes.
+
+ <pattern> is a sample expression rule as described in section 7.3. It
+ describes what elements will be analyzed, extracted, combined,
+ and used to select which table entry to update the counters. It
+ must be specified for shared bandwidth limitation filters only.
+
+ <table> is an optional table to be used instead of the default one,
+ which is the stick-table declared in the current proxy. It can
+ be specified for shared bandwidth limitation filters only.
+
+ <time> is the default time period used to evaluate the bandwidth
+ limitation rate. It can be specified for per-stream bandwidth
+ limitation filters only. It follows the HAProxy time format and
+ is expressed in milliseconds.
+
+ <min-size> is the optional minimum number of bytes forwarded at a time by
+ a stream excluding the last packet that may be smaller. This
+ value can be specified for per-stream and shared bandwidth
+ limitation filters. It follows the HAProxy size format and is
+ expressed in bytes.
+
+Bandwidth limitation filters should be used to restrict the data forwarding
+speed at the stream level. By extension, such filters limit the network
+bandwidth consumed by a resource. Several bandwidth limitation filters can be
+used. For instance, it is possible to define a limit per source address to be
+sure a client will never consume all the network bandwidth, thereby penalizing
+other clients, and another one per stream to be able to fairly handle several
+connections for a given client.
+
+The definition order of these filters is important. If several bandwidth
+filters are enabled on a stream, the filtering will be applied in their
+definition order. It is also important to understand the definition order of
+the other filters have an influence. For instance, depending on the HTTP
+compression filter is defined before or after a bandwidth limitation filter,
+the limit will be applied on the compressed payload or not. The same is true
+for the cache filter.
+
+There are two kinds of bandwidth limitation filters. The first one enforces a
+default limit and is applied per stream. The second one uses a stickiness table
+to enforce a limit equally divided between all streams sharing the same entry in
+the table.
+
+In addition, for a given filter, depending on the filter keyword used, the
+limitation can be applied on incoming data, received from the client and
+forwarded to a server, or on outgoing data, received from a server and sent to
+the client. To apply a limit on incoming data, "bwlim-in" keyword must be
+used. To apply it on outgoing data, "bwlim-out" keyword must be used. In both
+cases, the bandwidth limitation is applied on forwarded data, at the stream
+level.
+
+The bandwidth limitation is applied at the stream level and not at the
+connection level. For multiplexed protocols (H2, H3 and FastCGI), the streams
+of the same connection may have different limits.
+
+For a per-stream bandwidth limitation filter, default period and limit must be
+defined. As their names suggest, they are the default values used to setup the
+bandwidth limitation rate for a stream. However, for this kind of filter and
+only this one, it is possible to redefine these values using sample expressions
+when the filter is enabled with a TCP/HTTP "set-bandwidth-limit" action.
+
+For a shared bandwidth limitation filter, depending on whether it is applied on
+incoming or outgoing data, the stickiness table used must store the
+corresponding bytes rate information. "bytes_in_rate(<period>)" counter must be
+stored to limit incoming data and "bytes_out_rate(<period>)" counter must be
+used to limit outgoing data.
+
+Finally, it is possible to set the minimum number of bytes that a bandwidth
+limitation filter can forward at a time for a given stream. It should be used
+to not forward too small amount of data, to reduce the CPU usage. It must
+carefully be defined. Too small, a value can increase the CPU usage. Too high,
+it can increase the latency. It is also highly linked to the defined bandwidth
+limit. If it is too close to the bandwidth limit, some pauses may be
+experienced to not exceed the limit because too many bytes will be consumed at
+a time. It is highly dependent on the filter configuration. A good idea is to
+start with something around 2 TCP MSS, typically 2896 bytes, and tune it after
+some experimentations.
+
+ Example:
+ frontend http
+ bind *:80
+ mode http
+
+ # If this filter is enabled, the stream will share the download limit
+ # of 10m/s with all other streams with the same source address.
+ filter bwlim-out limit-by-src key src table limit-by-src limit 10m
+
+ # If this filter is enabled, the stream will be limited to download at 1m/s,
+ # independently of all other streams.
+ filter bwlim-out limit-by-strm default-limit 1m default-period 1s
+
+ # Limit all streams to 1m/s (the default limit) and those accessing the
+ # internal API to 100k/s. Limit each source address to 10m/s. The shared
+ # limit is applied first. Both are limiting the download rate.
+ http-request set-bandwidth-limit limit-by-strm
+ http-request set-bandwidth-limit limit-by-strm limit 100k if { path_beg /internal }
+ http-request set-bandwidth-limit limit-by-src
+ ...
+
+ backend limit-by-src
+ # The stickiness table used by <limit-by-src> filter
+ stick-table type ip size 1m expire 3600s store bytes_out_rate(1s)
+
+See also : "tcp-request content set-bandwidth-limit",
+ "tcp-response content set-bandwidth-limit",
+ "http-request set-bandwidth-limit" and
+ "http-response set-bandwidth-limit".
+
+10. FastCGI applications
+-------------------------
+
+HAProxy is able to send HTTP requests to Responder FastCGI applications. This
+feature was added in HAProxy 2.1. To do so, servers must be configured to use
+the FastCGI protocol (using the keyword "proto fcgi" on the server line) and a
+FastCGI application must be configured and used by the backend managing these
+servers (using the keyword "use-fcgi-app" into the proxy section). Several
+FastCGI applications may be defined, but only one can be used at a time by a
+backend.
+
+HAProxy implements all features of the FastCGI specification for Responder
+application. Especially it is able to multiplex several requests on a simple
+connection.
+
+10.1. Setup
+-----------
+
+10.1.1. Fcgi-app section
+--------------------------
+
+fcgi-app <name>
+ Declare a FastCGI application named <name>. To be valid, at least the
+ document root must be defined.
+
+acl <aclname> <criterion> [flags] [operator] <value> ...
+ Declare or complete an access list.
+
+ See "acl" keyword in section 4.2 and section 7 about ACL usage for
+ details. ACLs defined for a FastCGI application are private. They cannot be
+ used by any other application or by any proxy. In the same way, ACLs defined
+ in any other section are not usable by a FastCGI application. However,
+ Pre-defined ACLs are available.
+
+docroot <path>
+ Define the document root on the remote host. <path> will be used to build
+ the default value of FastCGI parameters SCRIPT_FILENAME and
+ PATH_TRANSLATED. It is a mandatory setting.
+
+index <script-name>
+ Define the script name that will be appended after an URI that ends with a
+ slash ("/") to set the default value of the FastCGI parameter SCRIPT_NAME. It
+ is an optional setting.
+
+ Example :
+ index index.php
+
+log-stderr global
+log-stderr <target> [len <length>] [format <format>]
+ [sample <ranges>:<sample_size>] <facility> [<level> [<minlevel>]]
+ Enable logging of STDERR messages reported by the FastCGI application.
+
+ See "log" keyword in section 4.2 for details. It is an optional setting. By
+ default STDERR messages are ignored.
+
+pass-header <name> [ { if | unless } <condition> ]
+ Specify the name of a request header which will be passed to the FastCGI
+ application. It may optionally be followed by an ACL-based condition, in
+ which case it will only be evaluated if the condition is true.
+
+ Most request headers are already available to the FastCGI application,
+ prefixed with "HTTP_". Thus, this directive is only required to pass headers
+ that are purposefully omitted. Currently, the headers "Authorization",
+ "Proxy-Authorization" and hop-by-hop headers are omitted.
+
+ Note that the headers "Content-type" and "Content-length" are never passed to
+ the FastCGI application because they are already converted into parameters.
+
+path-info <regex>
+ Define a regular expression to extract the script-name and the path-info from
+ the URL-decoded path. Thus, <regex> may have two captures: the first one to
+ capture the script name and the second one to capture the path-info. The
+ first one is mandatory, the second one is optional. This way, it is possible
+ to extract the script-name from the path ignoring the path-info. It is an
+ optional setting. If it is not defined, no matching is performed on the
+ path. and the FastCGI parameters PATH_INFO and PATH_TRANSLATED are not
+ filled.
+
+ For security reason, when this regular expression is defined, the newline and
+ the null characters are forbidden from the path, once URL-decoded. The reason
+ to such limitation is because otherwise the matching always fails (due to a
+ limitation one the way regular expression are executed in HAProxy). So if one
+ of these two characters is found in the URL-decoded path, an error is
+ returned to the client. The principle of least astonishment is applied here.
+
+ Example :
+ path-info ^(/.+\.php)(/.*)?$ # both script-name and path-info may be set
+ path-info ^(/.+\.php) # the path-info is ignored
+
+option get-values
+no option get-values
+ Enable or disable the retrieve of variables about connection management.
+
+ HAProxy is able to send the record FCGI_GET_VALUES on connection
+ establishment to retrieve the value for following variables:
+
+ * FCGI_MAX_REQS The maximum number of concurrent requests this
+ application will accept.
+
+ * FCGI_MPXS_CONNS "0" if this application does not multiplex connections,
+ "1" otherwise.
+
+ Some FastCGI applications does not support this feature. Some others close
+ the connection immediately after sending their response. So, by default, this
+ option is disabled.
+
+ Note that the maximum number of concurrent requests accepted by a FastCGI
+ application is a connection variable. It only limits the number of streams
+ per connection. If the global load must be limited on the application, the
+ server parameters "maxconn" and "pool-max-conn" must be set. In addition, if
+ an application does not support connection multiplexing, the maximum number
+ of concurrent requests is automatically set to 1.
+
+option keep-conn
+no option keep-conn
+ Instruct the FastCGI application to keep the connection open or not after
+ sending a response.
+
+ If disabled, the FastCGI application closes the connection after responding
+ to this request. By default, this option is enabled.
+
+option max-reqs <reqs>
+ Define the maximum number of concurrent requests this application will
+ accept.
+
+ This option may be overwritten if the variable FCGI_MAX_REQS is retrieved
+ during connection establishment. Furthermore, if the application does not
+ support connection multiplexing, this option will be ignored. By default set
+ to 1.
+
+option mpxs-conns
+no option mpxs-conns
+ Enable or disable the support of connection multiplexing.
+
+ This option may be overwritten if the variable FCGI_MPXS_CONNS is retrieved
+ during connection establishment. It is disabled by default.
+
+set-param <name> <fmt> [ { if | unless } <condition> ]
+ Set a FastCGI parameter that should be passed to this application. Its
+ value, defined by <fmt> must follows the log-format rules (see section 8.2.4
+ "Custom Log format"). It may optionally be followed by an ACL-based
+ condition, in which case it will only be evaluated if the condition is true.
+
+ With this directive, it is possible to overwrite the value of default FastCGI
+ parameters. If the value is evaluated to an empty string, the rule is
+ ignored. These directives are evaluated in their declaration order.
+
+ Example :
+ # PHP only, required if PHP was built with --enable-force-cgi-redirect
+ set-param REDIRECT_STATUS 200
+
+ set-param PHP_AUTH_DIGEST %[req.hdr(Authorization)]
+
+
+10.1.2. Proxy section
+---------------------
+
+use-fcgi-app <name>
+ Define the FastCGI application to use for the backend.
+
+ Arguments :
+ <name> is the name of the FastCGI application to use.
+
+ This keyword is only available for HTTP proxies with the backend capability
+ and with at least one FastCGI server. However, FastCGI servers can be mixed
+ with HTTP servers. But except there is a good reason to do so, it is not
+ recommended (see section 10.3 about the limitations for details). Only one
+ application may be defined at a time per backend.
+
+ Note that, once a FastCGI application is referenced for a backend, depending
+ on the configuration some processing may be done even if the request is not
+ sent to a FastCGI server. Rules to set parameters or pass headers to an
+ application are evaluated.
+
+
+10.1.3. Example
+---------------
+
+ frontend front-http
+ mode http
+ bind *:80
+ bind *:
+
+ use_backend back-dynamic if { path_reg ^/.+\.php(/.*)?$ }
+ default_backend back-static
+
+ backend back-static
+ mode http
+ server www A.B.C.D:80
+
+ backend back-dynamic
+ mode http
+ use-fcgi-app php-fpm
+ server php-fpm A.B.C.D:9000 proto fcgi
+
+ fcgi-app php-fpm
+ log-stderr global
+ option keep-conn
+
+ docroot /var/www/my-app
+ index index.php
+ path-info ^(/.+\.php)(/.*)?$
+
+
+10.2. Default parameters
+------------------------
+
+A Responder FastCGI application has the same purpose as a CGI/1.1 program. In
+the CGI/1.1 specification (RFC3875), several variables must be passed to the
+script. So HAProxy set them and some others commonly used by FastCGI
+applications. All these variables may be overwritten, with caution though.
+
+ +-------------------+-----------------------------------------------------+
+ | AUTH_TYPE | Identifies the mechanism, if any, used by HAProxy |
+ | | to authenticate the user. Concretely, only the |
+ | | BASIC authentication mechanism is supported. |
+ | | |
+ +-------------------+-----------------------------------------------------+
+ | CONTENT_LENGTH | Contains the size of the message-body attached to |
+ | | the request. It means only requests with a known |
+ | | size are considered as valid and sent to the |
+ | | application. |
+ | | |
+ +-------------------+-----------------------------------------------------+
+ | CONTENT_TYPE | Contains the type of the message-body attached to |
+ | | the request. It may not be set. |
+ | | |
+ +-------------------+-----------------------------------------------------+
+ | DOCUMENT_ROOT | Contains the document root on the remote host under |
+ | | which the script should be executed, as defined in |
+ | | the application's configuration. |
+ | | |
+ +-------------------+-----------------------------------------------------+
+ | GATEWAY_INTERFACE | Contains the dialect of CGI being used by HAProxy |
+ | | to communicate with the FastCGI application. |
+ | | Concretely, it is set to "CGI/1.1". |
+ | | |
+ +-------------------+-----------------------------------------------------+
+ | PATH_INFO | Contains the portion of the URI path hierarchy |
+ | | following the part that identifies the script |
+ | | itself. To be set, the directive "path-info" must |
+ | | be defined. |
+ | | |
+ +-------------------+-----------------------------------------------------+
+ | PATH_TRANSLATED | If PATH_INFO is set, it is its translated version. |
+ | | It is the concatenation of DOCUMENT_ROOT and |
+ | | PATH_INFO. If PATH_INFO is not set, this parameters |
+ | | is not set too. |
+ | | |
+ +-------------------+-----------------------------------------------------+
+ | QUERY_STRING | Contains the request's query string. It may not be |
+ | | set. |
+ | | |
+ +-------------------+-----------------------------------------------------+
+ | REMOTE_ADDR | Contains the network address of the client sending |
+ | | the request. |
+ | | |
+ +-------------------+-----------------------------------------------------+
+ | REMOTE_USER | Contains the user identification string supplied by |
+ | | client as part of user authentication. |
+ | | |
+ +-------------------+-----------------------------------------------------+
+ | REQUEST_METHOD | Contains the method which should be used by the |
+ | | script to process the request. |
+ | | |
+ +-------------------+-----------------------------------------------------+
+ | REQUEST_URI | Contains the request's URI. |
+ | | |
+ +-------------------+-----------------------------------------------------+
+ | SCRIPT_FILENAME | Contains the absolute pathname of the script. it is |
+ | | the concatenation of DOCUMENT_ROOT and SCRIPT_NAME. |
+ | | |
+ +-------------------+-----------------------------------------------------+
+ | SCRIPT_NAME | Contains the name of the script. If the directive |
+ | | "path-info" is defined, it is the first part of the |
+ | | URI path hierarchy, ending with the script name. |
+ | | Otherwise, it is the entire URI path. |
+ | | |
+ +-------------------+-----------------------------------------------------+
+ | SERVER_NAME | Contains the name of the server host to which the |
+ | | client request is directed. It is the value of the |
+ | | header "Host", if defined. Otherwise, the |
+ | | destination address of the connection on the client |
+ | | side. |
+ | | |
+ +-------------------+-----------------------------------------------------+
+ | SERVER_PORT | Contains the destination TCP port of the connection |
+ | | on the client side, which is the port the client |
+ | | connected to. |
+ | | |
+ +-------------------+-----------------------------------------------------+
+ | SERVER_PROTOCOL | Contains the request's protocol. |
+ | | |
+ +-------------------+-----------------------------------------------------+
+ | SERVER_SOFTWARE | Contains the string "HAProxy" followed by the |
+ | | current HAProxy version. |
+ | | |
+ +-------------------+-----------------------------------------------------+
+ | HTTPS | Set to a non-empty value ("on") if the script was |
+ | | queried through the HTTPS protocol. |
+ | | |
+ +-------------------+-----------------------------------------------------+
+
+
+10.3. Limitations
+------------------
+
+The current implementation have some limitations. The first one is about the
+way some request headers are hidden to the FastCGI applications. This happens
+during the headers analysis, on the backend side, before the connection
+establishment. At this stage, HAProxy know the backend is using a FastCGI
+application but it don't know if the request will be routed to a FastCGI server
+or not. But to hide request headers, it simply removes them from the HTX
+message. So, if the request is finally routed to an HTTP server, it never see
+these headers. For this reason, it is not recommended to mix FastCGI servers
+and HTTP servers under the same backend.
+
+Similarly, the rules "set-param" and "pass-header" are evaluated during the
+request headers analysis. So the evaluation is always performed, even if the
+requests is finally forwarded to an HTTP server.
+
+About the rules "set-param", when a rule is applied, a pseudo header is added
+into the HTX message. So, the same way than for HTTP header rewrites, it may
+fail if the buffer is full. The rules "set-param" will compete with
+"http-request" ones.
+
+Finally, all FastCGI params and HTTP headers are sent into a unique record
+FCGI_PARAM. Encoding of this record must be done in one pass, otherwise a
+processing error is returned. It means the record FCGI_PARAM, once encoded,
+must not exceeds the size of a buffer. However, there is no reserve to respect
+here.
+
+
+11. Address formats
+-------------------
+
+Several statements as "bind, "server", "nameserver" and "log" requires an
+address.
+
+This address can be a host name, an IPv4 address, an IPv6 address, or '*'.
+The '*' is equal to the special address "0.0.0.0" and can be used, in the case
+of "bind" or "dgram-bind" to listen on all IPv4 of the system.The IPv6
+equivalent is '::'.
+
+Depending of the statement, a port or port range follows the IP address. This
+is mandatory on 'bind' statement, optional on 'server'.
+
+This address can also begin with a slash '/'. It is considered as the "unix"
+family, and '/' and following characters must be present the path.
+
+Default socket type or transport method "datagram" or "stream" depends on the
+configuration statement showing the address. Indeed, 'bind' and 'server' will
+use a "stream" socket type by default whereas 'log', 'nameserver' or
+'dgram-bind' will use a "datagram".
+
+Optionally, a prefix could be used to force the address family and/or the
+socket type and the transport method.
+
+
+11.1. Address family prefixes
+-----------------------------
+
+'abns@<name>' following <name> is an abstract namespace (Linux only).
+
+'fd@<n>' following address is a file descriptor <n> inherited from the
+ parent. The fd must be bound and may or may not already be
+ listening.
+
+'ip@<address>[:port1[-port2]]' following <address> is considered as an IPv4 or
+ IPv6 address depending on the syntax. Depending
+ on the statement using this address, a port or
+ a port range may or must be specified.
+
+'ipv4@<address>[:port1[-port2]]' following <address> is always considered as
+ an IPv4 address. Depending on the statement
+ using this address, a port or a port range
+ may or must be specified.
+
+'ipv6@<address>[:port1[-port2]]' following <address> is always considered as
+ an IPv6 address. Depending on the statement
+ using this address, a port or a port range
+ may or must be specified.
+
+'sockpair@<n>' following address is the file descriptor of a connected unix
+ socket or of a socketpair. During a connection, the initiator
+ creates a pair of connected sockets, and passes one of them
+ over the FD to the other end. The listener waits to receive
+ the FD from the unix socket and uses it as if it were the FD
+ of an accept(). Should be used carefully.
+
+'unix@<path>' following string is considered as a UNIX socket <path>. this
+ prefix is useful to declare an UNIX socket path which don't
+ start by slash '/'.
+
+
+11.2. Socket type prefixes
+--------------------------
+
+Previous "Address family prefixes" can also be prefixed to force the socket
+type and the transport method. The default depends of the statement using
+this address but in some cases the user may force it to a different one.
+This is the case for "log" statement where the default is syslog over UDP
+but we could force to use syslog over TCP.
+
+Those prefixes were designed for internal purpose and users should instead use
+use aliases of the next section "11.3 Protocol prefixes". However these can
+sometimes be convenient, for example in combination with inherited sockets
+known by their file descriptor number, in which case the address family is "fd"
+and the socket type must be declared.
+
+If users need one those prefixes to perform what they expect because
+they can not configure the same using the protocol prefixes, they should
+report this to the maintainers.
+
+'stream+<family>@<address>' forces socket type and transport method
+ to "stream"
+
+'dgram+<family>@<address>' forces socket type and transport method
+ to "datagram".
+
+'quic+<family>@<address>' forces socket type to "datagram" and transport
+ method to "stream".
+
+
+
+11.3. Protocol prefixes
+-----------------------
+
+'quic4@<address>[:port1[-port2]]' following <address> is always considered as
+ an IPv4 address but socket type is forced to
+ "datagram" and the transport method is forced
+ to "stream". Depending on the statement using
+ this address, a UDP port or port range can or
+ must be specified. It is equivalent to
+ "quic+ipv4@".
+
+'quic6@<address>[:port1[-port2]]' following <address> is always considered as
+ an IPv6 address but socket type is forced to
+ "datagram" and the transport method is forced
+ to "stream". Depending on the statement using
+ this address, a UDP port or port range can or
+ must be specified. It is equivalent to
+ "quic+ipv6@".
+
+'tcp@<address>[:port1[-port2]]' following <address> is considered as an IPv4
+ or IPv6 address depending of the syntax but
+ socket type and transport method is forced to
+ "stream". Depending on the statement using
+ this address, a port or a port range can or
+ must be specified. It is considered as an alias
+ of 'stream+ip@'.
+
+'tcp4@<address>[:port1[-port2]]' following <address> is always considered as
+ an IPv4 address but socket type and transport
+ method is forced to "stream". Depending on the
+ statement using this address, a port or port
+ range can or must be specified.
+ It is considered as an alias of 'stream+ipv4@'.
+
+'tcp6@<address>[:port1[-port2]]' following <address> is always considered as
+ an IPv6 address but socket type and transport
+ method is forced to "stream". Depending on the
+ statement using this address, a port or port
+ range can or must be specified.
+ It is considered as an alias of 'stream+ipv4@'.
+
+'udp@<address>[:port1[-port2]]' following <address> is considered as an IPv4
+ or IPv6 address depending of the syntax but
+ socket type and transport method is forced to
+ "datagram". Depending on the statement using
+ this address, a port or a port range can or
+ must be specified. It is considered as an alias
+ of 'dgram+ip@'.
+
+'udp4@<address>[:port1[-port2]]' following <address> is always considered as
+ an IPv4 address but socket type and transport
+ method is forced to "datagram". Depending on
+ the statement using this address, a port or
+ port range can or must be specified.
+ It is considered as an alias of 'dgram+ipv4@'.
+
+'udp6@<address>[:port1[-port2]]' following <address> is always considered as
+ an IPv6 address but socket type and transport
+ method is forced to "datagram". Depending on
+ the statement using this address, a port or
+ port range can or must be specified.
+ It is considered as an alias of 'dgram+ipv4@'.
+
+'uxdg@<path>' following string is considered as a unix socket <path> but
+ transport method is forced to "datagram". It is considered as
+ an alias of 'dgram+unix@'.
+
+'uxst@<path>' following string is considered as a unix socket <path> but
+ transport method is forced to "stream". It is considered as
+ an alias of 'stream+unix@'.
+
+In future versions, other prefixes could be used to specify protocols like
+QUIC which proposes stream transport based on socket of type "datagram".
+
+/*
+ * Local variables:
+ * fill-column: 79
+ * End:
+ */
diff --git a/doc/cookie-options.txt b/doc/cookie-options.txt
new file mode 100644
index 0000000..b3badf3
--- /dev/null
+++ b/doc/cookie-options.txt
@@ -0,0 +1,25 @@
+2011/04/13 : List of possible cookie settings with associated behaviours.
+
+PSV="preserve", PFX="prefix", INS="insert", REW="rewrite", IND="indirect"
+0 = option not set
+1 = option is set
+* = option doesn't matter
+
+PSV PFX INS REW IND Behaviour
+ 0 0 0 0 0 passive mode
+ 0 0 0 0 1 passive + indirect : remove response if not needed
+ 0 0 0 1 0 always rewrite response
+ 0 0 1 0 0 always insert or replace response
+ 0 0 1 0 1 insert + indirect : remove req and also resp if not needed
+ * * 1 1 * [ forbidden ]
+ 0 1 0 0 0 prefix
+ 0 1 0 0 1 !! prefix on request, remove response cookie if not needed
+ * 1 * 1 * [ forbidden ]
+ * 1 1 * * [ forbidden ]
+ * * * 1 1 [ forbidden ]
+ 1 * 0 * 0 [ forbidden ]
+ 1 0 0 0 1 passive mode (alternate form)
+ 1 0 1 0 0 insert only, and preserve server response cookie if any
+ 1 0 1 0 1 conditional insert only for new requests
+ 1 1 0 0 1 prefix on requests only (passive prefix)
+
diff --git a/doc/design-thoughts/binding-possibilities.txt b/doc/design-thoughts/binding-possibilities.txt
new file mode 100644
index 0000000..3f5e432
--- /dev/null
+++ b/doc/design-thoughts/binding-possibilities.txt
@@ -0,0 +1,167 @@
+2013/10/10 - possibilities for setting source and destination addresses
+
+
+When establishing a connection to a remote device, this device is designated
+as a target, which designates an entity defined in the configuration. A same
+target appears only once in a configuration, and multiple targets may share
+the same settings if needed.
+
+The following types of targets are currently supported :
+
+ - listener : all connections with this type of target come from clients ;
+ - server : connections to such targets are for "server" lines ;
+ - peer : connections to such target address "peer" lines in "peers"
+ sections ;
+ - proxy : these targets are used by "dispatch", "option transparent"
+ or "option http_proxy" statements.
+
+A connection might not be reused between two different targets, even if all
+parameters seem similar. One of the reason is that some parameters are specific
+to the target and are not easy or not cheap to compare (eg: bind to interface,
+mss, ...).
+
+A number of source and destination addresses may be set for a given target.
+
+ - listener :
+ - the "from" address:port is set by accept()
+
+ - the "to" address:port is set if conn_get_to_addr() is called
+
+ - peer :
+ - the "from" address:port is not set
+
+ - the "to" address:port is static and dependent only on the peer
+
+ - server :
+ - the "from" address may be set alone when "source" is used with
+ a forced IP address, or when "usesrc clientip" is used.
+
+ - the "from" port may be set only combined with the address when
+ "source" is used with IP:port, IP:port-range or "usesrc client" is
+ used. Note that in this case, both the address and the port may be
+ 0, meaning that the kernel will pick the address or port and that
+ the final value might not match the one explicitly set (eg:
+ important for logging).
+
+ - the "from" address may be forced from a header which implies it
+ may change between two consecutive requests on the same connection.
+
+ - the "to" address and port are set together when connecting to a
+ regular server, or by copying the client's IP address when
+ "server 0.0.0.0" is used. Note that the destination port may be
+ an offset applied to the original destination port.
+
+ - proxy :
+ - the "from" address may be set alone when "source" is used with a
+ forced IP address or when "usesrc clientip" is used.
+
+ - the "from" port may be set only combined with the address when
+ "source" is used with IP:port or with "usesrc client". There is
+ no ip:port range for a proxy as of now. Same comment applies as
+ above when port and/or address are 0.
+
+ - the "from" address may be forced from a header which implies it
+ may change between two consecutive requests on the same connection.
+
+ - the "to" address and port are set together, either by configuration
+ when "dispatch" is used, or dynamically when "transparent" is used
+ (1:1 with client connection) or "option http_proxy" is used, where
+ each client request may lead to a different destination address.
+
+
+At the moment, there are some limits in what might happen between multiple
+concurrent requests to a same target.
+
+ - peers parameter do not change, so no problem.
+
+ - server parameters may change in this way :
+ - a connection may require a source bound to an IP address found in a
+ header, which will fall back to the "source" settings if the address
+ is not found in this header. This means that the source address may
+ switch between a dynamically forced IP address and another forced
+ IP and/or port range.
+
+ - if the element is not found (eg: header), the remaining "forced"
+ source address might very well be empty (unset), so the connection
+ reuse is acceptable when switching in that direction.
+
+ - it is not possible to switch between client and clientip or any of
+ these and hdr_ip() because they're exclusive.
+
+ - using a source address/port belonging to a port range is compatible
+ with connection reuse because there is a single range per target, so
+ switching from a range to another range means we remain in the same
+ range.
+
+ - destination address may currently not change since the only possible
+ case for dynamic destination address setting is the transparent mode,
+ reproducing the client's destination address.
+
+ - proxy parameters may change in this way :
+ - a connection may require a source bound to an IP address found in a
+ header, which will fall back to the "source" settings if the address
+ is not found in this header. This means that the source address may
+ switch between a dynamically forced IP address and another forced
+ IP and/or port range.
+
+ - if the element is not found (eg: header), the remaining "forced"
+ source address might very well be empty (unset), so the connection
+ reuse is acceptable when switching in that direction.
+
+ - it is not possible to switch between client and clientip or any of
+ these and hdr_ip() because they're exclusive.
+
+ - proxies do not support port ranges at the moment.
+
+ - destination address might change in the case where "option http_proxy"
+ is used.
+
+So, for each source element (IP, port), we want to know :
+ - if the element was assigned by static configuration (eg: ":80")
+ - if the element was assigned from a connection-specific value (eg: usesrc clientip)
+ - if the element was assigned from a configuration-specific range (eg: 1024-65535)
+ - if the element was assigned from a request-specific value (eg: hdr_ip(xff))
+ - if the element was not assigned at all
+
+For the destination, we want to know :
+ - if the element was assigned by static configuration (eg: ":80")
+ - if the element was assigned from a connection-specific value (eg: transparent)
+ - if the element was assigned from a request-specific value (eg: http_proxy)
+
+We don't need to store the information about the origin of the dynamic value
+since we have the value itself. So in practice we have :
+ - default value, unknown (not yet checked with getsockname/getpeername)
+ - default value, known (check done)
+ - forced value (known)
+ - forced range (known)
+
+We can't do that on an ip:port basis because the port may be fixed regardless
+of the address and conversely.
+
+So that means :
+
+ enum {
+ CO_ADDR_NONE = 0, /* not set, unknown value */
+ CO_ADDR_KNOWN = 1, /* not set, known value */
+ CO_ADDR_FIXED = 2, /* fixed value, known */
+ CO_ADDR_RANGE = 3, /* from assigned range, known */
+ } conn_addr_values;
+
+ unsigned int new_l3_src_status:2;
+ unsigned int new_l4_src_status:2;
+ unsigned int new_l3_dst_status:2;
+ unsigned int new_l4_dst_status:2;
+
+ unsigned int cur_l3_src_status:2;
+ unsigned int cur_l4_src_status:2;
+ unsigned int cur_l3_dsp_status:2;
+ unsigned int cur_l4_dst_status:2;
+
+ unsigned int new_family:2;
+ unsigned int cur_family:2;
+
+Note: this obsoletes CO_FL_ADDR_FROM_SET and CO_FL_ADDR_TO_SET. These flags
+must be changed to individual l3+l4 checks ORed between old and new values,
+or better, set to cur only which will inherit new.
+
+In the connection, these values may be merged in the same word as err_code.
diff --git a/doc/design-thoughts/connection-reuse.txt b/doc/design-thoughts/connection-reuse.txt
new file mode 100644
index 0000000..4eb22f6
--- /dev/null
+++ b/doc/design-thoughts/connection-reuse.txt
@@ -0,0 +1,224 @@
+2015/08/06 - server connection sharing
+
+Improvements on the connection sharing strategies
+-------------------------------------------------
+
+4 strategies are currently supported :
+ - never
+ - safe
+ - aggressive
+ - always
+
+The "aggressive" and "always" strategies take into account the fact that the
+connection has already been reused at least once or not. The principle is that
+second requests can be used to safely "validate" connection reuse on newly
+added connections, and that such validated connections may be used even by
+first requests from other sessions. A validated connection is a connection
+which has already been reused, hence proving that it definitely supports
+multiple requests. Such connections are easy to verify : after processing the
+response, if the txn already had the TX_NOT_FIRST flag, then it was not the
+first request over that connection, and it is validated as safe for reuse.
+Validated connections are put into a distinct list : server->safe_conns.
+
+Incoming requests with TX_NOT_FIRST first pick from the regular idle_conns
+list so that any new idle connection is validated as soon as possible.
+
+Incoming requests without TX_NOT_FIRST only pick from the safe_conns list for
+strategy "aggressive", guaranteeing that the server properly supports connection
+reuse, or first from the safe_conns list, then from the idle_conns list for
+strategy "always".
+
+Connections are always stacked into the list (LIFO) so that there are higher
+changes to convert recent connections and to use them. This will first optimize
+the likeliness that the connection works, and will avoid TCP metrics from being
+lost due to an idle state, and/or the congestion window to drop and the
+connection going to slow start mode.
+
+
+Handling connections in pools
+-----------------------------
+
+A per-server "pool-max" setting should be added to permit disposing unused idle
+connections not attached anymore to a session for use by future requests. The
+principle will be that attached connections are queued from the front of the
+list while the detached connections will be queued from the tail of the list.
+
+This way, most reused connections will be fairly recent and detached connections
+will most often be ignored. The number of detached idle connections in the lists
+should be accounted for (pool_used) and limited (pool_max).
+
+After some time, a part of these detached idle connections should be killed.
+For this, the list is walked from tail to head and connections without an owner
+may be evicted. It may be useful to have a per-server pool_min setting
+indicating how many idle connections should remain in the pool, ready for use
+by new requests. Conversely, a pool_low metric should be kept between eviction
+runs, to indicate the lowest amount of detached connections that were found in
+the pool.
+
+For eviction, the principle of a half-life is appealing. The principle is
+simple : over a period of time, half of the connections between pool_min and
+pool_low should be gone. Since pool_low indicates how many connections were
+remaining unused over a period, it makes sense to kill some of them.
+
+In order to avoid killing thousands of connections in one run, the purge
+interval should be split into smaller batches. Let's call N the ratio of the
+half-life interval and the effective interval.
+
+The algorithm consists in walking over them from the end every interval and
+killing ((pool_low - pool_min) + 2 * N - 1) / (2 * N). It ensures that half
+of the unused connections are killed over the half-life period, in N batches
+of population/2N entries at most.
+
+Unsafe connections should be evicted first. There should be quite few of them
+since most of them are probed and become safe. Since detached connections are
+quickly recycled and attached to a new session, there should not be too many
+detached connections in the pool, and those present there may be killed really
+quickly.
+
+Another interesting point of pools is that when a pool-max is not null, then it
+makes sense to automatically enable pretend-keep-alive on non-private connections
+going to the server in order to be able to feed them back into the pool. With
+the "aggressive" or "always" strategies, it can allow clients making a single
+request over their connection to share persistent connections to the servers.
+
+
+
+2013/10/17 - server connection management and reuse
+
+Current state
+-------------
+
+At the moment, a connection entity is needed to carry any address
+information. This means in the following situations, we need a server
+connection :
+
+- server is elected and the server's destination address is set
+
+- transparent mode is elected and the destination address is set from
+ the incoming connection
+
+- proxy mode is enabled, and the destination's address is set during
+ the parsing of the HTTP request
+
+- connection to the server fails and must be retried on the same
+ server using the same parameters, especially the destination
+ address (SN_ADDR_SET not removed)
+
+
+On the accepting side, we have further requirements :
+
+- allocate a clean connection without a stream interface
+
+- incrementally set the accepted connection's parameters without
+ clearing it, and keep track of what is set (eg: getsockname).
+
+- initialize a stream interface in established mode
+
+- attach the accepted connection to a stream interface
+
+
+This means several things :
+
+- the connection has to be allocated on the fly the first time it is
+ needed to store the source or destination address ;
+
+- the connection has to be attached to the stream interface at this
+ moment ;
+
+- it must be possible to incrementally set some settings on the
+ connection's addresses regardless of the connection's current state
+
+- the connection must not be released across connection retries ;
+
+- it must be possible to clear a connection's parameters for a
+ redispatch without having to detach/attach the connection ;
+
+- we need to allocate a connection without an existing stream interface
+
+So on the accept() side, it looks like this :
+
+ fd = accept();
+ conn = new_conn();
+ get_some_addr_info(&conn->addr);
+ ...
+ si = new_si();
+ si_attach_conn(si, conn);
+ si_set_state(si, SI_ST_EST);
+ ...
+ get_more_addr_info(&conn->addr);
+
+On the connect() side, it looks like this :
+
+ si = new_si();
+ while (!properly_connected) {
+ if (!(conn = si->end)) {
+ conn = new_conn();
+ conn_clear(conn);
+ si_attach_conn(si, conn);
+ }
+ else {
+ if (connected) {
+ f = conn->flags & CO_FL_XPRT_TRACKED;
+ conn->flags &= ~CO_FL_XPRT_TRACKED;
+ conn_close(conn);
+ conn->flags |= f;
+ }
+ if (!correct_dest)
+ conn_clear(conn);
+ }
+ set_some_addr_info(&conn->addr);
+ si_set_state(si, SI_ST_CON);
+ ...
+ set_more_addr_info(&conn->addr);
+ conn->connect();
+ if (must_retry) {
+ close_conn(conn);
+ }
+ }
+
+Note: we need to be able to set the control and transport protocols.
+On outgoing connections, this is set once we know the destination address.
+On incoming connections, this is set the earliest possible (once we know
+the source address).
+
+The problem analysed below was solved on 2013/10/22
+
+| ==> the real requirement is to know whether a connection is still valid or not
+| before deciding to close it. CO_FL_CONNECTED could be enough, though it
+| will not indicate connections that are still waiting for a connect to occur.
+| This combined with CO_FL_WAIT_L4_CONN and CO_FL_WAIT_L6_CONN should be OK.
+|
+| Alternatively, conn->xprt could be used for this, but needs some careful checks
+| (it's used by conn_full_close at least).
+|
+| Right now, conn_xprt_close() checks conn->xprt and sets it to NULL.
+| conn_full_close() also checks conn->xprt and sets it to NULL, except
+| that the check on ctrl is performed within xprt. So conn_xprt_close()
+| followed by conn_full_close() will not close the file descriptor.
+| Note that conn_xprt_close() is never called, maybe we should kill it ?
+|
+| Note: at the moment, it's problematic to leave conn->xprt to NULL before doing
+| xprt_init() because we might end up with a pending file descriptor. Or at
+| least with some transport not de-initialized. We might thus need
+| conn_xprt_close() when conn_xprt_init() fails.
+|
+| The fd should be conditioned by ->ctrl only, and the transport layer by ->xprt.
+|
+| - conn_prepare_ctrl(conn, ctrl)
+| - conn_prepare_xprt(conn, xprt)
+| - conn_prepare_data(conn, data)
+|
+| Note: conn_xprt_init() needs conn->xprt so it's not a problem to set it early.
+|
+| One problem might be with conn_xprt_close() not being able to know if xprt_init()
+| was called or not. That's where it might make sense to only set ->xprt during init.
+| Except that it does not fly with outgoing connections (xprt_init is called after
+| connect()).
+|
+| => currently conn_xprt_close() is only used by ssl_sock.c and decides whether
+| to do something based on ->xprt_ctx which is set by ->init() from xprt_init().
+| So there is nothing to worry about. We just need to restore conn_xprt_close()
+| and rely on ->ctrl to close the fd instead of ->xprt.
+|
+| => we have the same issue with conn_ctrl_close() : when is the fd supposed to be
+| valid ? On outgoing connections, the control is set much before the fd...
diff --git a/doc/design-thoughts/http_load_time.url b/doc/design-thoughts/http_load_time.url
new file mode 100644
index 0000000..f178e46
--- /dev/null
+++ b/doc/design-thoughts/http_load_time.url
@@ -0,0 +1,5 @@
+Excellent paper about page load time for keepalive on/off, pipelining,
+multiple host names, etc...
+
+http://www.die.net/musings/page_load_time/
+
diff --git a/doc/design-thoughts/pool-debugging.txt b/doc/design-thoughts/pool-debugging.txt
new file mode 100644
index 0000000..106e41c
--- /dev/null
+++ b/doc/design-thoughts/pool-debugging.txt
@@ -0,0 +1,243 @@
+2022-02-22 - debugging options with pools
+
+Two goals:
+ - help developers spot bugs as early as possible
+
+ - make the process more reliable in field, by killing sick ones as soon as
+ possible instead of letting them corrupt data, cause trouble, or even be
+ exploited.
+
+An allocated object may exist in 5 forms:
+ - in use: currently referenced and used by haproxy, 100% of its size are
+ dedicated to the application which can do absolutely anything with it,
+ but it may never touch anything before nor after that area.
+
+ - in cache: the object is neither referenced nor used anymore, but it sits
+ in a thread's cache. The application may not touch it at all anymore, and
+ some parts of it could even be unmapped. Only the current thread may safely
+ reach it, though others might find/release it when under thread isolation.
+ The thread cache needs some LRU linking that may be stored anywhere, either
+ inside the area, or outside. The parts surrounding the <size> parts remain
+ invisible to the application layer, and can serve as a protection.
+
+ - in shared cache: the object is neither referenced nor used anymore, but it
+ may be reached by any thread. Some parts of it could be unmapped. Any
+ thread may pick it but only one may find it, hence once grabbed, it is
+ guaranteed no other one will find it. The shared cache needs to set up a
+ linked list and a single pointer needs to be stored anywhere, either inside
+ or outside the area. The parts surrounding the <size> parts remain
+ invisible to the application layer, and can serve as a protection.
+
+ - in the system's memory allocator: the object is not known anymore from
+ haproxy. It may be reassigned in parts or totally to other pools or other
+ subsystems (e.g. crypto library). Some or all of it may be unmapped. The
+ areas surrounding the <size> parts are also part of the object from the
+ library's point of view and may be delivered to other areas. Tampering
+ with these may cause any other part to malfunction in dirty ways.
+
+ - in the OS only: the memory allocator gave it back to the OS.
+
+The following options need to be configurable:
+ - detect improper initialization: this is done by poisonning objects before
+ delivering them to the application.
+
+ - help figure where an object was allocated when in use: a pointer to the
+ call place will help. Pointing to the last pool_free() as well for the
+ same reasons when dealing with a UAF.
+
+ - detection of wrong pointer/pool when in use: a pointer to the pool before
+ or after the area will definitely help.
+
+ - detection of overflows when in use: a canary at the end of the area
+ (closest possible to <size>) will definitely help. The pool above can do
+ that job. Ideally, we should fill some data at the end so that even
+ unaligned sizes can be checked (e.g. a buffer that gets a zero appended).
+ If we just align on 2 pointers, writing the same pointer twice at the end
+ may do the job, but we won't necessarily have our bytes. Thus a particular
+ end-of-string pattern would be useful (e.g. ff55aa01) to fill it.
+
+ - detection of double free when in cache: similar to detection of wrong
+ pointer/pool when in use: the pointer at the end may simply be changed so
+ that it cannot match the pool anymore. By using a pointer to the caller of
+ the previous free() operation, we have the guarantee to see different
+ pointers, and this pointer can be inspected to figure where the object was
+ previously freed. An extra check may even distinguish a perfect double-free
+ (same caller) from just a wrong free (pointer differs from pool).
+
+ - detection of late corruption when in cache: keeping a copy of the
+ checksum of the whole area upon free() will do the job, but requires one
+ extra storage area for the checksum. Filling the area with a pattern also
+ does the job and doesn't require extra storage, but it loses the contents
+ and can be a bit slower. Sometimes losing the contents can be a feature,
+ especially when trying to detect late reads. Probably that both need to
+ be implemented. Note that if contents are not strictly needed, storing a
+ checksum inside the area does the job.
+
+ - preserve total contents in cache for debugging: losing some precious
+ information can be a problem.
+
+ - pattern filling of the area helps detect use-after-free in read-only mode.
+
+ - allocate cold first helps with both cases above.
+
+Uncovered:
+ - overflow/underflow when in cache/shared/libc: it belongs to use-after-free
+ pattern and such an error during regular use ought to be caught while the
+ object was still in use.
+
+ - integrity when in libc: not under our control anymore, this is a libc
+ problem.
+
+Arbitrable:
+ - integrity when in shared cache: unlikely to happen only then if it could
+ have happened in the local cache. Shared cache not often used anymore, thus
+ probably not worth the effort
+
+ - protection against double-free when in shared cache/libc: might be done for
+ a cheap price, probably worth being able to quickly tell that such an
+ object left the local cache (e.g. the mark points to the caller, but could
+ possibly just be incremented, hence still point to the same code location+1
+ byte when released. Calls are 4 bytes min on RISC, 5 on x86 so we do have
+ some margin by having a caller's location be +0,+1,+2 or +3.
+
+ - underflow when in use: hasn't been really needed over time but may change.
+
+ - detection of late corruption when in shared cache: checksum or area filling
+ are possible, but is this as relevant as it used to considering the less
+ common use of the shared cache ?
+
+Design considerations:
+ - object allocation when in use must remain minimal
+
+ - when in cache, there are 2 lists which the compiler expect to be at least
+ aligned each (e.g. if/when we start to use DWCAS).
+
+ - the original "pool debugging" feature covers both pool tracking, double-
+ free detection, overflow detection and caller info at the cost of a single
+ pointer placed immediately after the area.
+
+ - preserving the contents might be done by placing the cache links and the
+ shared cache's list outside of the area (either before or after). Placing
+ it before has the merit that the allocated object preserves the 4-ptr
+ alignment. But when a larger alignment is desired this often does not work
+ anymore. Placing it after requires some dynamic adjustment depending on the
+ object's size. If any protection is installed, this protection must be
+ placed before the links so that the list doesn't get randomly corrupted and
+ corrupts adjacent elements. Note that if protection is desired, the extra
+ waste is probably less critical.
+
+ - a link to the last caller might have to be stored somewhere. Without
+ preservation the free() caller may be placed anywhere while the alloc()
+ caller may only be placed outside. With preservation, again the free()
+ caller may be placed either before the object or after the mark at the end.
+ There is no particular need that both share the same location though it may
+ help. Note that when debugging is enabled, the free() caller doesn't need
+ to be duplicated and can continue to serve as the double-free detection.
+ Thus maybe in the end we only need to store the caller to the last alloc()
+ but not the free() since if we want it it's available via the pool debug.
+
+ - use-after-free detection: contents may be erased on free() and checked on
+ alloc(), but they can also be checksummed on free() and rechecked on
+ alloc(). In the latter case we need to store a checksum somewhere. Note
+ that with pure checksum we don't know what part was modified, but seeing
+ previous contents can be useful.
+
+Possibilities:
+
+1) Linked lists inside the area:
+
+ V size alloc
+ ---+------------------------------+-----------------+--
+ in use |##############################| (Pool) (Tracer) |
+ ---+------------------------------+-----------------+--
+
+ ---+--+--+------------------------+-----------------+--
+ in cache |L1|L2|########################| (Caller) (Sum) |
+ ---+--+--+------------------------+-----------------+--
+or:
+ ---+--+--+------------------------+-----------------+--
+ in cache |L1|L2|###################(sum)| (Caller) |
+ ---+--+--+------------------------+-----------------+--
+
+ ---+-+----------------------------+-----------------+--
+ in global |N|XXXX########################| (Caller) |
+ ---+-+----------------------------+-----------------+--
+
+
+2) Linked lists before the the area leave room for tracer and pool before
+ the area, but the canary must remain at the end, however the area will
+ be more difficult to keep aligned:
+
+ V head size alloc
+ ----+-+-+------------------------------+-----------------+--
+ in use |T|P|##############################| (canary) |
+ ----+-+-+------------------------------+-----------------+--
+
+ --+-----+------------------------------+-----------------+--
+ in cache |L1|L2|##############################| (Caller) (Sum) |
+ --+-----+------------------------------+-----------------+--
+
+ ------+-+------------------------------+-----------------+--
+ in global |N|##############################| (Caller) |
+ ------+-+------------------------------+-----------------+--
+
+
+3) Linked lists at the end of the area, might be shared with extra data
+ depending on the state:
+
+ V size alloc
+ ---+------------------------------+-----------------+--
+ in use |##############################| (Pool) (Tracer) |
+ ---+------------------------------+-----------------+--
+
+ ---+------------------------------+--+--+-----------+--
+ in cache |##############################|L1|L2| (Caller) (Sum)
+ ---+------------------------------+--+--+-----------+--
+
+ ---+------------------------------+-+---------------+--
+ in global |##############################|N| (Caller) |
+ ---+------------------------------+-+---------------+--
+
+This model requires a little bit of alignment at the end of the area, which is
+not incompatible with pattern filling and/or checksumming:
+ - preserving the area for post-mortem analysis means nothing may be placed
+ inside. In this case it could make sense to always store the last releaser.
+ - detecting late corruption may be done either with filling or checksumming,
+ but the simple fact of assuming a risk of corruption that needs to be
+ chased means we must not store the lists nor caller inside the area.
+
+Some models imply dedicating some place when in cache:
+ - preserving contents forces the lists to be prefixed or appended, which
+ leaves unused places when in use. Thus we could systematically place the
+ pool pointer and the caller in this case.
+
+ - if preserving contents is not desired, almost everything can be stored
+ inside when not in use. Then each situation's size should be calculated
+ so that the allocated size is known, and entries are filled from the
+ beginning while not in use, or after the size when in use.
+
+ - if poisonning is requested, late corruption might be detected but then we
+ don't want the list to be stored inside at the risk of being corrupted.
+
+Maybe just implement a few models:
+ - compact/optimal: put l1/l2 inside
+ - detect late corruption: fill/sum, put l1/l2 out
+ - preserve contents: put l1/l2 out
+ - corruption+preserve: do not fill, sum out
+ - poisonning: not needed on free if pattern filling is done.
+
+try2:
+ - poison on alloc to detect missing initialization: yes/no
+ (note: nothing to do if filling done)
+ - poison on free to detect use-after-free: yes/no
+ (note: nothing to do if filling done)
+ - check on alloc for corruption-after-free: yes/no
+ If content-preserving => sum, otherwise pattern filling; in
+ any case, move L1/L2 out.
+ - check for overflows: yes/no: use a canary after the area. The
+ canary can be the pointer to the pool.
+ - check for alloc caller: yes/no => always after the area
+ - content preservation: yes/no
+ (disables filling, moves lists out)
+ - improved caller tracking: used to detect double-free, may benefit
+ from content-preserving but not only.
diff --git a/doc/design-thoughts/thread-group.txt b/doc/design-thoughts/thread-group.txt
new file mode 100644
index 0000000..e845230
--- /dev/null
+++ b/doc/design-thoughts/thread-group.txt
@@ -0,0 +1,655 @@
+Thread groups
+#############
+
+2021-07-13 - first draft
+==========
+
+Objective
+---------
+- support multi-socket systems with limited cache-line bouncing between
+ physical CPUs and/or L3 caches
+
+- overcome the 64-thread limitation
+
+- Support a reasonable number of groups. I.e. if modern CPUs arrive with
+ core complexes made of 8 cores, with 8 CC per chip and 2 chips in a
+ system, it makes sense to support 16 groups.
+
+
+Non-objective
+-------------
+- no need to optimize to the last possible cycle. I.e. some algos like
+ leastconn will remain shared across all threads, servers will keep a
+ single queue, etc. Global information remains global.
+
+- no stubborn enforcement of FD sharing. Per-server idle connection lists
+ can become per-group; listeners can (and should probably) be per-group.
+ Other mechanisms (like SO_REUSEADDR) can already overcome this.
+
+- no need to go beyond 64 threads per group.
+
+
+Identified tasks
+================
+
+General
+-------
+Everywhere tid_bit is used we absolutely need to find a complement using
+either the current group or a specific one. Thread debugging will need to
+be extended as masks are extensively used.
+
+
+Scheduler
+---------
+The global run queue and global wait queue must become per-group. This
+means that a task may only be queued into one of them at a time. It
+sounds like tasks may only belong to a given group, but doing so would
+bring back the original issue that it's impossible to perform remote wake
+ups.
+
+We could probably ignore the group if we don't need to set the thread mask
+in the task. the task's thread_mask is never manipulated using atomics so
+it's safe to complement it with a group.
+
+The sleeping_thread_mask should become per-group. Thus possibly that a
+wakeup may only be performed on the assigned group, meaning that either
+a task is not assigned, in which case it be self-assigned (like today),
+otherwise the tg to be woken up will be retrieved from the task itself.
+
+Task creation currently takes a thread mask of either tid_bit, a specific
+mask, or MAX_THREADS_MASK. How to create a task able to run anywhere
+(checks, Lua, ...) ?
+
+Profiling -> completed
+---------
+There should be one task_profiling_mask per thread group. Enabling or
+disabling profiling should be made per group (possibly by iterating).
+-> not needed anymore, one flag per thread in each thread's context.
+
+Thread isolation
+----------------
+Thread isolation is difficult as we solely rely on atomic ops to figure
+who can complete. Such operation is rare, maybe we could have a global
+read_mostly flag containing a mask of the groups that require isolation.
+Then the threads_want_rdv_mask etc can become per-group. However setting
+and clearing the bits will become problematic as this will happen in two
+steps hence will require careful ordering.
+
+FD
+--
+Tidbit is used in a number of atomic ops on the running_mask. If we have
+one fdtab[] per group, the mask implies that it's within the group.
+Theoretically we should never face a situation where an FD is reported nor
+manipulated for a remote group.
+
+There will still be one poller per thread, except that this time all
+operations will be related to the current thread_group. No fd may appear
+in two thread_groups at once, but we can probably not prevent that (e.g.
+delayed close and reopen). Should we instead have a single shared fdtab[]
+(less memory usage also) ? Maybe adding the group in the fdtab entry would
+work, but when does a thread know it can leave it ? Currently this is
+solved by running_mask and by update_mask. Having two tables could help
+with this (each table sees the FD in a different group with a different
+mask) but this looks overkill.
+
+There's polled_mask[] which needs to be decided upon. Probably that it
+should be doubled as well. Note, polled_mask left fdtab[] for cacheline
+alignment reasons in commit cb92f5cae4.
+
+If we have one fdtab[] per group, what *really* prevents from using the
+same FD in multiple groups ? _fd_delete_orphan() and fd_update_events()
+need to check for no-thread usage before closing the FD. This could be
+a limiting factor. Enabling could require to wake every poller.
+
+Shouldn't we remerge fdinfo[] with fdtab[] (one pointer + one int/short,
+used only during creation and close) ?
+
+Other problem, if we have one fdtab[] per TG, disabling/enabling an FD
+(e.g. pause/resume on listener) can become a problem if it's not necessarily
+on the current TG. We'll then need a way to figure that one. It sounds like
+FDs from listeners and receivers are very specific and suffer from problems
+all other ones under high load do not suffer from. Maybe something specific
+ought to be done for them, if we can guarantee there is no risk of accidental
+reuse (e.g. locate the TG info in the receiver and have a "MT" bit in the
+FD's flags). The risk is always that a close() can result in instant pop-up
+of the same FD on any other thread of the same process.
+
+Observations: right now fdtab[].thread_mask more or less corresponds to a
+declaration of interest, it's very close to meaning "active per thread". It is
+in fact located in the FD while it ought to do nothing there, as it should be
+where the FD is used as it rules accesses to a shared resource that is not
+the FD but what uses it. Indeed, if neither polled_mask nor running_mask have
+a thread's bit, the FD is unknown to that thread and the element using it may
+only be reached from above and not from the FD. As such we ought to have a
+thread_mask on a listener and another one on connections. These ones will
+indicate who uses them. A takeover could then be simplified (atomically set
+exclusivity on the FD's running_mask, upon success, takeover the connection,
+clear the running mask). Probably that the change ought to be performed on
+the connection level first, not the FD level by the way. But running and
+polled are the two relevant elements, one indicates userland knowledge,
+the other one kernel knowledge. For listeners there's no exclusivity so it's
+a bit different but the rule remains the same that we don't have to know
+what threads are *interested* in the FD, only its holder.
+
+Not exact in fact, see FD notes below.
+
+activity
+--------
+There should be one activity array per thread group. The dump should
+simply scan them all since the cumuled values are not very important
+anyway.
+
+applets
+-------
+They use tid_bit only for the task. It looks like the appctx's thread_mask
+is never used (now removed). Furthermore, it looks like the argument is
+*always* tid_bit.
+
+CPU binding
+-----------
+This is going to be tough. It will be needed to detect that threads overlap
+and are not bound (i.e. all threads on same mask). In this case, if the number
+of threads is higher than the number of threads per physical socket, one must
+try hard to evenly spread them among physical sockets (e.g. one thread group
+per physical socket) and start as many threads as needed on each, bound to
+all threads/cores of each socket. If there is a single socket, the same job
+may be done based on L3 caches. Maybe it could always be done based on L3
+caches. The difficulty behind this is the number of sockets to be bound: it
+is not possible to bind several FDs per listener. Maybe with a new bind
+keyword we can imagine to automatically duplicate listeners ? In any case,
+the initially bound cpumap (via taskset) must always be respected, and
+everything should probably start from there.
+
+Frontend binding
+----------------
+We'll have to define a list of threads and thread-groups per frontend.
+Probably that having a group mask and a same thread-mask for each group
+would suffice.
+
+Threads should have two numbers:
+ - the per-process number (e.g. 1..256)
+ - the per-group number (1..64)
+
+The "bind-thread" lines ought to use the following syntax:
+ - bind 45 ## bind to process' thread 45
+ - bind 1/45 ## bind to group 1's thread 45
+ - bind all/45 ## bind to thread 45 in each group
+ - bind 1/all ## bind to all threads in group 1
+ - bind all ## bind to all threads
+ - bind all/all ## bind to all threads in all groups (=all)
+ - bind 1/65 ## rejected
+ - bind 65 ## OK if there are enough
+ - bind 35-45 ## depends. Rejected if it crosses a group boundary.
+
+The global directive "nbthread 28" means 28 total threads for the process. The
+number of groups will sub-divide this. E.g. 4 groups will very likely imply 7
+threads per group. At the beginning, the nbgroup should be manual since it
+implies config adjustments to bind lines.
+
+There should be a trivial way to map a global thread to a group and local ID
+and to do the opposite.
+
+
+Panic handler + watchdog
+------------------------
+Will probably depend on what's done for thread_isolate
+
+Per-thread arrays inside structures
+-----------------------------------
+- listeners have a thr_conn[] array, currently limited to MAX_THREADS. Should
+ we simply bump the limit ?
+- same for servers with idle connections.
+=> doesn't seem very practical.
+- another solution might be to point to dynamically allocated arrays of
+ arrays (e.g. nbthread * nbgroup) or a first level per group and a second
+ per thread.
+=> dynamic allocation based on the global number
+
+Other
+-----
+- what about dynamic thread start/stop (e.g. for containers/VMs) ?
+ E.g. if we decide to start $MANY threads in 4 groups, and only use
+ one, in the end it will not be possible to use less than one thread
+ per group, and at most 64 will be present in each group.
+
+
+FD Notes
+--------
+ - updt_fd_polling() uses thread_mask to figure where to send the update,
+ the local list or a shared list, and which bits to set in update_mask.
+ This could be changed so that it takes the update mask in argument. The
+ call from the poller's fork would just have to broadcast everywhere.
+
+ - pollers use it to figure whether they're concerned or not by the activity
+ update. This looks important as otherwise we could re-enable polling on
+ an FD that changed to another thread.
+
+ - thread_mask being a per-thread active mask looks more exact and is
+ precisely used this way by _update_fd(). In this case using it instead
+ of running_mask to gauge a change or temporarily lock it during a
+ removal could make sense.
+
+ - running should be conditioned by thread. Polled not (since deferred
+ or migrated). In this case testing thread_mask can be enough most of
+ the time, but this requires synchronization that will have to be
+ extended to tgid.. But migration seems a different beast that we shouldn't
+ care about here: if first performed at the higher level it ought to
+ be safe.
+
+In practice the update_mask can be dropped to zero by the first fd_delete()
+as the only authority allowed to fd_delete() is *the* owner, and as soon as
+all running_mask are gone, the FD will be closed, hence removed from all
+pollers. This will be the only way to make sure that update_mask always
+refers to the current tgid.
+
+However, it may happen that a takeover within the same group causes a thread
+to read the update_mask late, while the FD is being wiped by another thread.
+That other thread may close it, causing another thread in another group to
+catch it, and change the tgid and start to update the update_mask. This means
+that it would be possible for a thread entering do_poll() to see the correct
+tgid, then the fd would be closed, reopened and reassigned to another tgid,
+and the thread would see its bit in the update_mask, being confused. Right
+now this should already happen when the update_mask is not cleared, except
+that upon wakeup a migration would be detected and that would be all.
+
+Thus we might need to set the running bit to prevent the FD from migrating
+before reading update_mask, which also implies closing on fd_clr_running() == 0 :-(
+
+Also even fd_update_events() leaves a risk of updating update_mask after
+clearing running, thus affecting the wrong one. Probably that update_mask
+should be updated before clearing running_mask there. Also, how about not
+creating an update on a close ? Not trivial if done before running, unless
+thread_mask==0.
+
+Note that one situation that is currently visible is that a thread closes a
+file descriptor that it's the last one to own and to have an update for. In
+fd_delete_orphan() it does call poller.clo() but this one is not sufficient
+as it doesn't drop the update_mask nor does it clear the polled_mask. The
+typical problem that arises is that the close() happens before processing
+the last update (e.g. a close() just after a partial read), thus it still
+has *at least* one bit set for the current thread in both update_mask and
+polled_mask, and it is present in the update_list. Not handling it would
+mean that the event is lost on update() from the concerned threads and
+that some resource might leak. Handling it means zeroing the update_mask
+and polled_mask, and deleting the update entry from the update_list, thus
+losing the update event. And as indicated above, if the FD switches twice
+between 2 groups, the finally called thread does not necessarily know that
+the FD isn't the same anymore, thus it's difficult to decide whether to
+delete it or not, because deleting the event might in fact mean deleting
+something that was just re-added for the same thread with the same FD but
+a different usage.
+
+Also it really seems unrealistic to scan a single shared update_list like
+this using write operations. There should likely be one per thread-group.
+But in this case there is no more choice than deleting the update event
+upon fd_delete_orphan(). This also means that poller->clo() must do the
+job for all of the group's threads at once. This would mean a synchronous
+removal before the close(), which doesn't seem ridiculously expensive. It
+just requires that any thread of a group may manipulate any other thread's
+status for an FD and a poller.
+
+Note about our currently supported pollers:
+
+ - epoll: our current code base relies on the modern version which
+ automatically removes closed FDs, so we don't have anything to do
+ when closing and we don't need the update.
+
+ - kqueue: according to https://www.freebsd.org/cgi/man.cgi?query=kqueue, just
+ like epoll, a close() implies a removal. Our poller doesn't perform
+ any bookkeeping either so it's OK to directly close.
+
+ - evports: https://docs.oracle.com/cd/E86824_01/html/E54766/port-dissociate-3c.html
+ says the same, i.e. close() implies a removal of all events. No local
+ processing nor bookkeeping either, we can close.
+
+ - poll: the fd_evts[] array is global, thus shared by all threads. As such,
+ a single removal is needed to flush it for all threads at once. The
+ operation is already performed like this.
+
+ - select: works exactly like poll() above, hence already handled.
+
+As a preliminary conclusion, it's safe to delete the event and reset
+update_mask just after calling poller->clo(). If extremely unlucky (changing
+thread mask due to takeover ?), the same FD may appear at the same time:
+ - in one or several thread-local fd_updt[] arrays. These ones are just work
+ queues, there's nothing to do to ignore them, just leave the holes with an
+ outdated FD which will be ignored once met. As a bonus, poller->clo() could
+ check if the last fd_updt[] points to this specific FD and decide to kill
+ it.
+
+ - in the global update_list. In this case, fd_rm_from_fd_list() already
+ performs an attachment check, so it's safe to always call it before closing
+ (since no one else may be in the process of changing anything).
+
+
+###########################################################
+
+Current state:
+
+
+Mux / takeover / fd_delete() code ||| poller code
+-------------------------------------------------|||---------------------------------------------------
+ \|/
+mux_takeover(): | fd_set_running():
+ if (fd_takeover()<0) | old = {running, thread};
+ return fail; | new = {tid_bit, tid_bit};
+ ... |
+fd_takeover(): | do {
+ atomic_or(running, tid_bit); | if (!(old.thread & tid_bit))
+ old = {running, thread}; | return -1;
+ new = {tid_bit, tid_bit}; | new = { running | tid_bit, old.thread }
+ if (owner != expected) { | } while (!dwcas({running, thread}, &old, &new));
+ atomic_and(running, ~tid_bit); |
+ return -1; // fail | fd_clr_running():
+ } | return atomic_and_fetch(running, ~tid_bit);
+ |
+ while (old == {tid_bit, !=0 }) | poll():
+ if (dwcas({running, thread}, &old, &new)) { | if (!owner)
+ atomic_and(running, ~tid_bit); | continue;
+ return 0; // success |
+ } | if (!(thread_mask & tid_bit)) {
+ } | epoll_ctl_del();
+ | continue;
+ atomic_and(running, ~tid_bit); | }
+ return -1; // fail |
+ | // via fd_update_events()
+fd_delete(): | if (fd_set_running() != -1) {
+ atomic_or(running, tid_bit); | iocb();
+ atomic_store(thread, 0); | if (fd_clr_running() == 0 && !thread_mask)
+ if (fd_clr_running(fd) = 0) | fd_delete_orphan();
+ fd_delete_orphan(); | }
+
+
+The idle_conns_lock prevents the connection from being *picked* and released
+while someone else is reading it. What it does is guarantee that on idle
+connections, the caller of the IOCB will not dereference the task's context
+while the connection is still in the idle list, since it might be picked then
+freed at the same instant by another thread. As soon as the IOCB manages to
+get that lock, it removes the connection from the list so that it cannot be
+taken over anymore. Conversely, the mux's takeover() code runs under that
+lock so that if it frees the connection and task, this will appear atomic
+to the IOCB. The timeout task (which is another entry point for connection
+deletion) does the same. Thus, when coming from the low-level (I/O or timeout):
+ - task always exists, but ctx checked under lock validates; conn removal
+ from list prevents takeover().
+ - t->context is stable, except during changes under takeover lock. So
+ h2_timeout_task may well run on a different thread than h2_io_cb().
+
+Coming from the top:
+ - takeover() done under lock() clears task's ctx and possibly closes the FD
+ (unless some running remains present).
+
+Unlikely but currently possible situations:
+ - multiple pollers (up to N) may have an idle connection's FD being
+ polled, if the connection was passed from thread to thread. The first
+ event on the connection would wake all of them. Most of them would
+ see fdtab[].owner set (the late ones might miss it). All but one would
+ see that their bit is missing from fdtab[].thread_mask and give up.
+ However, just after this test, others might take over the connection,
+ so in practice if terribly unlucky, all but 1 could see their bit in
+ thread_mask just before it gets removed, all of them set their bit
+ in running_mask, and all of them call iocb() (sock_conn_iocb()).
+ Thus all of them dereference the connection and touch the subscriber
+ with no protection, then end up in conn_notify_mux() that will call
+ the mux's wake().
+
+ - multiple pollers (up to N-1) might still be in fd_update_events()
+ manipulating fdtab[].state. The cause is that the "locked" variable
+ is determined by atleast2(thread_mask) but that thread_mask is read
+ at a random instant (i.e. it may be stolen by another one during a
+ takeover) since we don't yet hold running to prevent this from being
+ done. Thus we can arrive here with thread_mask==something_else (1bit),
+ locked==0 and fdtab[].state assigned non-atomically.
+
+ - it looks like nothing prevents h2_release() from being called on a
+ thread (e.g. from the top or task timeout) while sock_conn_iocb()
+ dereferences the connection on another thread. Those killing the
+ connection don't yet consider the fact that it's an FD that others
+ might currently be waking up on.
+
+###################
+
+pb with counter:
+
+users count doesn't say who's using the FD and two users can do the same
+close in turn. The thread_mask should define who's responsible for closing
+the FD, and all those with a bit in it ought to do it.
+
+
+2021-08-25 - update with minimal locking on tgid value
+==========
+
+ - tgid + refcount at once using CAS
+ - idle_conns lock during updates
+ - update:
+ if tgid differs => close happened, thus drop update
+ otherwise normal stuff. Lock tgid until running if needed.
+ - poll report:
+ if tgid differs => closed
+ if thread differs => stop polling (migrated)
+ keep tgid lock until running
+ - test on thread_id:
+ if (xadd(&tgid,65536) != my_tgid) {
+ // was closed
+ sub(&tgid, 65536)
+ return -1
+ }
+ if !(thread_id & tidbit) => migrated/closed
+ set_running()
+ sub(tgid,65536)
+ - note: either fd_insert() or the final close() ought to set
+ polled and update to 0.
+
+2021-09-13 - tid / tgroups etc.
+==========
+
+ * tid currently is the thread's global ID. It's essentially used as an index
+ for arrays. It must be clearly stated that it works this way.
+
+ * tasklets use the global thread id, and __tasklet_wakeup_on() must use a
+ global ID as well. It's capital that tinfo[] provides instant access to
+ local/global bits/indexes/arrays
+
+ - tid_bit makes no sense process-wide, so it must be redefined to represent
+ the thread's tid within its group. The name is not much welcome though, but
+ there are 286 of it that are not going to be changed that fast.
+ => now we have ltid and ltid_bit in thread_info. thread-local tid_bit still
+ not changed though. If renamed we must make sure the older one vanishes.
+ Why not rename "ptid, ptid_bit" for the process-wide tid and "gtid,
+ gtid_bit" for the group-wide ones ? This removes the ambiguity on "tid"
+ which is half the time not the one we expect.
+
+ * just like "ti" is the thread_info, we need to have "tg" pointing to the
+ thread_group.
+
+ - other less commonly used elements should be retrieved from ti->xxx. E.g.
+ the thread's local ID.
+
+ - lock debugging must reproduce tgid
+
+ * task profiling must be made per-group (annoying), unless we want to add a
+ per-thread TH_FL_* flag and have the rare places where the bit is changed
+ iterate over all threads if needed. Sounds preferable overall.
+
+ * an offset might be placed in the tgroup so that even with 64 threads max
+ we could have completely separate tid_bits over several groups.
+ => base and count now
+
+2021-09-15 - bind + listen() + rx
+==========
+
+ - thread_mask (in bind_conf->rx_settings) should become an array of
+ MAX_TGROUP longs.
+ - when parsing "thread 123" or "thread 2/37", the proper bit is set,
+ assuming the array is either a contiguous bitfield or a tgroup array.
+ An option RX_O_THR_PER_GRP or RX_O_THR_PER_PROC is set depending on
+ how the thread num was parsed, so that we reject mixes.
+ - end of parsing: entries translated to the cleanest form (to be determined)
+ - binding: for each socket()/bind()/listen()... just perform one extra dup()
+ for each tgroup and store the multiple FDs into an FD array indexed on
+ MAX_TGROUP. => allows to use one FD per tgroup for the same socket, hence
+ to have multiple entries in all tgroup pollers without requiring the user
+ to duplicate the bind line.
+
+2021-09-15 - global thread masks
+==========
+
+Some global variables currently expect to know about thread IDs and it's
+uncertain what must be done with them:
+ - global_tasks_mask /* Mask of threads with tasks in the global runqueue */
+ => touched under the rq lock. Change it per-group ? What exact use is made ?
+
+ - sleeping_thread_mask /* Threads that are about to sleep in poll() */
+ => seems that it can be made per group
+
+ - all_threads_mask: a bit complicated, derived from nbthread and used with
+ masks and with my_ffsl() to wake threads up. Should probably be per-group
+ but we might miss something for global.
+
+ - stopping_thread_mask: used in combination with all_threads_mask, should
+ move per-group.
+
+ - threads_harmless_mask: indicates all threads that are currently harmless in
+ that they promise not to access a shared resource. Must be made per-group
+ but then we'll likely need a second stage to have the harmless groups mask.
+ threads_idle_mask, threads_sync_mask, threads_want_rdv_mask go with the one
+ above. Maybe the right approach will be to request harmless on a group mask
+ so that we can detect collisions and arbiter them like today, but on top of
+ this it becomes possible to request harmless only on the local group if
+ desired. The subtlety is that requesting harmless at the group level does
+ not mean it's achieved since the requester cannot vouch for the other ones
+ in the same group.
+
+In addition, some variables are related to the global runqueue:
+ __decl_aligned_spinlock(rq_lock); /* spin lock related to run queue */
+ struct eb_root rqueue; /* tree constituting the global run queue, accessed under rq_lock */
+ unsigned int grq_total; /* total number of entries in the global run queue, atomic */
+ static unsigned int global_rqueue_ticks; /* insertion count in the grq, use rq_lock */
+
+And others to the global wait queue:
+ struct eb_root timers; /* sorted timers tree, global, accessed under wq_lock */
+ __decl_aligned_rwlock(wq_lock); /* RW lock related to the wait queue */
+ struct eb_root timers; /* sorted timers tree, global, accessed under wq_lock */
+
+
+2022-06-14 - progress on task affinity
+==========
+
+The particularity of the current global run queue is to be usable for remote
+wakeups because it's protected by a lock. There is no need for a global run
+queue beyond this, and there could already be a locked queue per thread for
+remote wakeups, with a random selection at wakeup time. It's just that picking
+a pending task in a run queue among a number is convenient (though it
+introduces some excessive locking). A task will either be tied to a single
+group or will be allowed to run on any group. As such it's pretty clear that we
+don't need a global run queue. When a run-anywhere task expires, either it runs
+on the current group's runqueue with any thread, or a target thread is selected
+during the wakeup and it's directly assigned.
+
+A global wait queue seems important for scheduled repetitive tasks however. But
+maybe it's more a task for a cron-like job and there's no need for the task
+itself to wake up anywhere, because once the task wakes up, it must be tied to
+one (or a set of) thread(s). One difficulty if the task is temporarily assigned
+a thread group is that it's impossible to know where it's running when trying
+to perform a second wakeup or when trying to kill it. Maybe we'll need to have
+two tgid for a task (desired, effective). Or maybe we can restrict the ability
+of such a task to stay in wait queue in case of wakeup, though that sounds
+difficult. Other approaches would be to set the GID to the current one when
+waking up the task, and to have a flag (or sign on the GID) indicating that the
+task is still queued in the global timers queue. We already have TASK_SHARED_WQ
+so it seems that antoher similar flag such as TASK_WAKE_ANYWHERE could make
+sense. But when is TASK_SHARED_WQ really used, except for the "anywhere" case ?
+All calls to task_new() use either 1<<thr, tid_bit, all_threads_mask, or come
+from appctx_new which does exactly the same. The only real user of non-global,
+non-unique task_new() call is debug_parse_cli_sched() which purposely allows to
+use an arbitrary mask.
+
+ +----------------------------------------------------------------------------+
+ | => we don't need one WQ per group, only a global and N local ones, hence |
+ | the TASK_SHARED_WQ flag can continue to be used for this purpose. |
+ +----------------------------------------------------------------------------+
+
+Having TASK_SHARED_WQ should indicate that a task will always be queued to the
+shared queue and will always have a temporary gid and thread mask in the run
+queue.
+
+Going further, as we don't have any single case of a task bound to a small set
+of threads, we could decide to wake up only expired tasks for ourselves by
+looking them up using eb32sc and adopting them. Thus, there's no more need for
+a shared runqueue nor a global_runqueue_ticks counter, and we can simply have
+the ability to wake up a remote task. The task's thread_mask will then change
+so that it's only a thread ID, except when the task has TASK_SHARED_WQ, in
+which case it corresponds to the running thread. That's very close to what is
+already done with tasklets in fact.
+
+
+2021-09-29 - group designation and masks
+==========
+
+Neither FDs nor tasks will belong to incomplete subsets of threads spanning
+over multiple thread groups. In addition there may be a difference between
+configuration and operation (for FDs). This allows to fix the following rules:
+
+ group mask description
+ 0 0 bind_conf: groups & thread not set. bind to any/all
+ task: it would be nice to mean "run on the same as the caller".
+
+ 0 xxx bind_conf: thread set but not group: thread IDs are global
+ FD/task: group 0, mask xxx
+
+ G>0 0 bind_conf: only group is set: bind to all threads of group G
+ FD/task: mask 0 not permitted (= not owned). May be used to
+ mention "any thread of this group", though already covered by
+ G/xxx like today.
+
+ G>0 xxx bind_conf: Bind to these threads of this group
+ FD/task: group G, mask xxx
+
+It looks like keeping groups starting at zero internally complicates everything
+though. But forcing it to start at 1 might also require that we rescan all tasks
+to replace 0 with 1 upon startup. This would also allow group 0 to be special and
+be used as the default group for any new thread creation, so that group0.count
+would keep the number of unassigned threads. Let's try:
+
+ group mask description
+ 0 0 bind_conf: groups & thread not set. bind to any/all
+ task: "run on the same group & thread as the caller".
+
+ 0 xxx bind_conf: thread set but not group: thread IDs are global
+ FD/task: invalid. Or maybe for a task we could use this to
+ mean "run on current group, thread XXX", which would cover
+ the need for health checks (g/t 0/0 while sleeping, 0/xxx
+ while running) and have wake_expired_tasks() detect 0/0 and
+ wake them up to a random group.
+
+ G>0 0 bind_conf: only group is set: bind to all threads of group G
+ FD/task: mask 0 not permitted (= not owned). May be used to
+ mention "any thread of this group", though already covered by
+ G/xxx like today.
+
+ G>0 xxx bind_conf: Bind to these threads of this group
+ FD/task: group G, mask xxx
+
+With a single group declared in the config, group 0 would implicitly find the
+first one.
+
+
+The problem with the approach above is that a task queued in one group+thread's
+wait queue could very well receive a signal from another thread and/or group,
+and that there is no indication about where the task is queued, nor how to
+dequeue it. Thus it seems that it's up to the application itself to unbind/
+rebind a task. This contradicts the principle of leaving a task waiting in a
+wait queue and waking it anywhere.
+
+Another possibility might be to decide that a task having a defined group but
+a mask of zero is shared and will always be queued into its group's wait queue.
+However, upon expiry, the scheduler would notice the thread-mask 0 and would
+broadcast it to any group.
+
+Right now in the code we have:
+ - 18 calls of task_new(tid_bit)
+ - 17 calls of task_new_anywhere()
+ - 2 calls with a single bit
+
+Thus it looks like "task_new_anywhere()", "task_new_on()" and
+"task_new_here()" would be sufficient.
diff --git a/doc/gpl.txt b/doc/gpl.txt
new file mode 100644
index 0000000..f90922e
--- /dev/null
+++ b/doc/gpl.txt
@@ -0,0 +1,340 @@
+ GNU GENERAL PUBLIC LICENSE
+ Version 2, June 1991
+
+ Copyright (C) 1989, 1991 Free Software Foundation, Inc.
+ 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+ Preamble
+
+ The licenses for most software are designed to take away your
+freedom to share and change it. By contrast, the GNU General Public
+License is intended to guarantee your freedom to share and change free
+software--to make sure the software is free for all its users. This
+General Public License applies to most of the Free Software
+Foundation's software and to any other program whose authors commit to
+using it. (Some other Free Software Foundation software is covered by
+the GNU Lesser General Public License instead.) You can apply it to
+your programs, too.
+
+ When we speak of free software, we are referring to freedom, not
+price. Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+this service if you wish), that you receive source code or can get it
+if you want it, that you can change the software or use pieces of it
+in new free programs; and that you know you can do these things.
+
+ To protect your rights, we need to make restrictions that forbid
+anyone to deny you these rights or to ask you to surrender the rights.
+These restrictions translate to certain responsibilities for you if you
+distribute copies of the software, or if you modify it.
+
+ For example, if you distribute copies of such a program, whether
+gratis or for a fee, you must give the recipients all the rights that
+you have. You must make sure that they, too, receive or can get the
+source code. And you must show them these terms so they know their
+rights.
+
+ We protect your rights with two steps: (1) copyright the software, and
+(2) offer you this license which gives you legal permission to copy,
+distribute and/or modify the software.
+
+ Also, for each author's protection and ours, we want to make certain
+that everyone understands that there is no warranty for this free
+software. If the software is modified by someone else and passed on, we
+want its recipients to know that what they have is not the original, so
+that any problems introduced by others will not reflect on the original
+authors' reputations.
+
+ Finally, any free program is threatened constantly by software
+patents. We wish to avoid the danger that redistributors of a free
+program will individually obtain patent licenses, in effect making the
+program proprietary. To prevent this, we have made it clear that any
+patent must be licensed for everyone's free use or not licensed at all.
+
+ The precise terms and conditions for copying, distribution and
+modification follow.
+
+ GNU GENERAL PUBLIC LICENSE
+ TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
+
+ 0. This License applies to any program or other work which contains
+a notice placed by the copyright holder saying it may be distributed
+under the terms of this General Public License. The "Program", below,
+refers to any such program or work, and a "work based on the Program"
+means either the Program or any derivative work under copyright law:
+that is to say, a work containing the Program or a portion of it,
+either verbatim or with modifications and/or translated into another
+language. (Hereinafter, translation is included without limitation in
+the term "modification".) Each licensee is addressed as "you".
+
+Activities other than copying, distribution and modification are not
+covered by this License; they are outside its scope. The act of
+running the Program is not restricted, and the output from the Program
+is covered only if its contents constitute a work based on the
+Program (independent of having been made by running the Program).
+Whether that is true depends on what the Program does.
+
+ 1. You may copy and distribute verbatim copies of the Program's
+source code as you receive it, in any medium, provided that you
+conspicuously and appropriately publish on each copy an appropriate
+copyright notice and disclaimer of warranty; keep intact all the
+notices that refer to this License and to the absence of any warranty;
+and give any other recipients of the Program a copy of this License
+along with the Program.
+
+You may charge a fee for the physical act of transferring a copy, and
+you may at your option offer warranty protection in exchange for a fee.
+
+ 2. You may modify your copy or copies of the Program or any portion
+of it, thus forming a work based on the Program, and copy and
+distribute such modifications or work under the terms of Section 1
+above, provided that you also meet all of these conditions:
+
+ a) You must cause the modified files to carry prominent notices
+ stating that you changed the files and the date of any change.
+
+ b) You must cause any work that you distribute or publish, that in
+ whole or in part contains or is derived from the Program or any
+ part thereof, to be licensed as a whole at no charge to all third
+ parties under the terms of this License.
+
+ c) If the modified program normally reads commands interactively
+ when run, you must cause it, when started running for such
+ interactive use in the most ordinary way, to print or display an
+ announcement including an appropriate copyright notice and a
+ notice that there is no warranty (or else, saying that you provide
+ a warranty) and that users may redistribute the program under
+ these conditions, and telling the user how to view a copy of this
+ License. (Exception: if the Program itself is interactive but
+ does not normally print such an announcement, your work based on
+ the Program is not required to print an announcement.)
+
+These requirements apply to the modified work as a whole. If
+identifiable sections of that work are not derived from the Program,
+and can be reasonably considered independent and separate works in
+themselves, then this License, and its terms, do not apply to those
+sections when you distribute them as separate works. But when you
+distribute the same sections as part of a whole which is a work based
+on the Program, the distribution of the whole must be on the terms of
+this License, whose permissions for other licensees extend to the
+entire whole, and thus to each and every part regardless of who wrote it.
+
+Thus, it is not the intent of this section to claim rights or contest
+your rights to work written entirely by you; rather, the intent is to
+exercise the right to control the distribution of derivative or
+collective works based on the Program.
+
+In addition, mere aggregation of another work not based on the Program
+with the Program (or with a work based on the Program) on a volume of
+a storage or distribution medium does not bring the other work under
+the scope of this License.
+
+ 3. You may copy and distribute the Program (or a work based on it,
+under Section 2) in object code or executable form under the terms of
+Sections 1 and 2 above provided that you also do one of the following:
+
+ a) Accompany it with the complete corresponding machine-readable
+ source code, which must be distributed under the terms of Sections
+ 1 and 2 above on a medium customarily used for software interchange; or,
+
+ b) Accompany it with a written offer, valid for at least three
+ years, to give any third party, for a charge no more than your
+ cost of physically performing source distribution, a complete
+ machine-readable copy of the corresponding source code, to be
+ distributed under the terms of Sections 1 and 2 above on a medium
+ customarily used for software interchange; or,
+
+ c) Accompany it with the information you received as to the offer
+ to distribute corresponding source code. (This alternative is
+ allowed only for noncommercial distribution and only if you
+ received the program in object code or executable form with such
+ an offer, in accord with Subsection b above.)
+
+The source code for a work means the preferred form of the work for
+making modifications to it. For an executable work, complete source
+code means all the source code for all modules it contains, plus any
+associated interface definition files, plus the scripts used to
+control compilation and installation of the executable. However, as a
+special exception, the source code distributed need not include
+anything that is normally distributed (in either source or binary
+form) with the major components (compiler, kernel, and so on) of the
+operating system on which the executable runs, unless that component
+itself accompanies the executable.
+
+If distribution of executable or object code is made by offering
+access to copy from a designated place, then offering equivalent
+access to copy the source code from the same place counts as
+distribution of the source code, even though third parties are not
+compelled to copy the source along with the object code.
+
+ 4. You may not copy, modify, sublicense, or distribute the Program
+except as expressly provided under this License. Any attempt
+otherwise to copy, modify, sublicense or distribute the Program is
+void, and will automatically terminate your rights under this License.
+However, parties who have received copies, or rights, from you under
+this License will not have their licenses terminated so long as such
+parties remain in full compliance.
+
+ 5. You are not required to accept this License, since you have not
+signed it. However, nothing else grants you permission to modify or
+distribute the Program or its derivative works. These actions are
+prohibited by law if you do not accept this License. Therefore, by
+modifying or distributing the Program (or any work based on the
+Program), you indicate your acceptance of this License to do so, and
+all its terms and conditions for copying, distributing or modifying
+the Program or works based on it.
+
+ 6. Each time you redistribute the Program (or any work based on the
+Program), the recipient automatically receives a license from the
+original licensor to copy, distribute or modify the Program subject to
+these terms and conditions. You may not impose any further
+restrictions on the recipients' exercise of the rights granted herein.
+You are not responsible for enforcing compliance by third parties to
+this License.
+
+ 7. If, as a consequence of a court judgment or allegation of patent
+infringement or for any other reason (not limited to patent issues),
+conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License. If you cannot
+distribute so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you
+may not distribute the Program at all. For example, if a patent
+license would not permit royalty-free redistribution of the Program by
+all those who receive copies directly or indirectly through you, then
+the only way you could satisfy both it and this License would be to
+refrain entirely from distribution of the Program.
+
+If any portion of this section is held invalid or unenforceable under
+any particular circumstance, the balance of the section is intended to
+apply and the section as a whole is intended to apply in other
+circumstances.
+
+It is not the purpose of this section to induce you to infringe any
+patents or other property right claims or to contest validity of any
+such claims; this section has the sole purpose of protecting the
+integrity of the free software distribution system, which is
+implemented by public license practices. Many people have made
+generous contributions to the wide range of software distributed
+through that system in reliance on consistent application of that
+system; it is up to the author/donor to decide if he or she is willing
+to distribute software through any other system and a licensee cannot
+impose that choice.
+
+This section is intended to make thoroughly clear what is believed to
+be a consequence of the rest of this License.
+
+ 8. If the distribution and/or use of the Program is restricted in
+certain countries either by patents or by copyrighted interfaces, the
+original copyright holder who places the Program under this License
+may add an explicit geographical distribution limitation excluding
+those countries, so that distribution is permitted only in or among
+countries not thus excluded. In such case, this License incorporates
+the limitation as if written in the body of this License.
+
+ 9. The Free Software Foundation may publish revised and/or new versions
+of the General Public License from time to time. Such new versions will
+be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+Each version is given a distinguishing version number. If the Program
+specifies a version number of this License which applies to it and "any
+later version", you have the option of following the terms and conditions
+either of that version or of any later version published by the Free
+Software Foundation. If the Program does not specify a version number of
+this License, you may choose any version ever published by the Free Software
+Foundation.
+
+ 10. If you wish to incorporate parts of the Program into other free
+programs whose distribution conditions are different, write to the author
+to ask for permission. For software which is copyrighted by the Free
+Software Foundation, write to the Free Software Foundation; we sometimes
+make exceptions for this. Our decision will be guided by the two goals
+of preserving the free status of all derivatives of our free software and
+of promoting the sharing and reuse of software generally.
+
+ NO WARRANTY
+
+ 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
+FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
+OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
+PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
+OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
+TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
+PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
+REPAIR OR CORRECTION.
+
+ 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
+REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
+INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
+OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
+TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
+YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
+PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGES.
+
+ END OF TERMS AND CONDITIONS
+
+ How to Apply These Terms to Your New Programs
+
+ If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+ To do so, attach the following notices to the program. It is safest
+to attach them to the start of each source file to most effectively
+convey the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+ <one line to give the program's name and a brief idea of what it does.>
+ Copyright (C) <year> <name of author>
+
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 2 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; if not, write to the Free Software
+ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+
+
+Also add information on how to contact you by electronic and paper mail.
+
+If the program is interactive, make it output a short notice like this
+when it starts in an interactive mode:
+
+ Gnomovision version 69, Copyright (C) year name of author
+ Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
+ This is free software, and you are welcome to redistribute it
+ under certain conditions; type `show c' for details.
+
+The hypothetical commands `show w' and `show c' should show the appropriate
+parts of the General Public License. Of course, the commands you use may
+be called something other than `show w' and `show c'; they could even be
+mouse-clicks or menu items--whatever suits your program.
+
+You should also get your employer (if you work as a programmer) or your
+school, if any, to sign a "copyright disclaimer" for the program, if
+necessary. Here is a sample; alter the names:
+
+ Yoyodyne, Inc., hereby disclaims all copyright interest in the program
+ `Gnomovision' (which makes passes at compilers) written by James Hacker.
+
+ <signature of Ty Coon>, 1 April 1989
+ Ty Coon, President of Vice
+
+This General Public License does not permit incorporating your program into
+proprietary programs. If your program is a subroutine library, you may
+consider it more useful to permit linking proprietary applications with the
+library. If this is what you want to do, use the GNU Lesser General
+Public License instead of this License.
diff --git a/doc/haproxy.1 b/doc/haproxy.1
new file mode 100644
index 0000000..4c2d786
--- /dev/null
+++ b/doc/haproxy.1
@@ -0,0 +1,227 @@
+.TH HAPROXY 1 "17 August 2007"
+
+.SH NAME
+
+HAProxy \- fast and reliable http reverse proxy and load balancer
+
+.SH SYNOPSIS
+
+haproxy \-f <configuration\ file|dir> [\-L\ <name>] [\-n\ maxconn] [\-N\ maxconn] [\-C\ <dir>] [\-v|\-vv] [\-d] [\-D] [\-W] [\-Ws] [\-q] [\-V] [\-c] [\-p\ <pidfile>] [\-dk] [\-ds] [\-de] [\-dp] [\-db] [\-dM[<byte>]] [\-m\ <megs>] [\-x <unix_socket>] [{\-sf|\-st}\ pidlist...]
+
+.SH DESCRIPTION
+
+HAProxy is a TCP/HTTP reverse proxy which is particularly suited for
+high availability environments. Indeed, it can:
+ \- route HTTP requests depending on statically assigned cookies ;
+ \- spread the load among several servers while assuring server
+ persistence through the use of HTTP cookies ;
+ \- switch to backup servers in the event a main one fails ;
+ \- accept connections to special ports dedicated to service
+ monitoring ;
+ \- stop accepting connections without breaking existing ones ;
+ \- add/modify/delete HTTP headers both ways ;
+ \- block requests matching a particular pattern ;
+ \- hold clients to the right application server depending on
+ application cookies
+ \- report detailed status as HTML pages to authenticated users from an
+ URI intercepted from the application.
+
+It needs very little resource. Its event-driven architecture allows it
+to easily handle thousands of simultaneous connections on hundreds of
+instances without risking the system's stability.
+
+.SH OPTIONS
+
+.TP
+\fB\-f <configuration file|dir>\fP
+Specify configuration file or directory path. If the argument is a directory
+the files (and only files) it contains are added in lexical order (using
+LC_COLLATE=C) ; only non hidden files with ".cfg" extension are added.
+
+.TP
+\fB\-L <name>\fP
+Set the local instance's peer name. Peers are defined in the \fBpeers\fP
+configuration section and used for syncing stick tables between different
+instances. If this option is not specified, the local hostname is used as peer
+name. This name is exported in the $HAPROXY_LOCALPEER environment variable and
+can be used in the configuration file.
+
+.TP
+\fB\-n <maxconn>\fP
+Set the high limit for the total number of simultaneous connections.
+
+.TP
+\fB\-N <maxconn>\fP
+Set the high limit for the per-listener number of simultaneous connections.
+
+.TP
+\fB\-C <dir>\fP
+Change directory to <\fIdir\fP> before loading any files.
+
+.TP
+\fB\-v\fP
+Display HAProxy's version.
+
+.TP
+\fB\-vv\fP
+Display HAProxy's version and all build options.
+
+.TP
+\fB\-d\fP
+Start in foreground with debugging mode enabled.
+When the proxy runs in this mode, it dumps every connections,
+disconnections, timestamps, and HTTP headers to stdout. This should
+NEVER be used in an init script since it will prevent the system from
+starting up.
+
+.TP
+\fB\-D\fP
+Start in daemon mode.
+
+.TP
+\fB\-W\fP
+Start in master-worker mode. Could be used either with foreground or daemon
+mode.
+
+.TP
+\fB\-Ws\fP
+Start in master-worker mode with systemd notify support. It tells systemd when
+the process is ready. This mode forces foreground.
+
+.TP
+\fB\-q\fP
+Disable messages on output.
+
+.TP
+\fB\-V\fP
+Displays messages on output even when \-q or 'quiet' are specified. Some
+information about pollers and config file are displayed during startup.
+
+.TP
+\fB\-c\fP
+Only checks config file and exits with code 0 if no error was found, or
+exits with code 1 if a syntax error was found.
+
+.TP
+\fB\-p <pidfile>\fP
+Ask the process to write down each of its children's pids to this file
+in daemon mode or ask the process to write down its master's pid to
+this file in master-worker mode.
+
+.TP
+\fB\-dk\fP
+Disable use of \fBkqueue\fP(2). \fBkqueue\fP(2) is available only on BSD systems.
+
+.TP
+\fB\-dv\fP
+Disable use of event ports. Event ports are available only on SunOS systems
+derived from Solaris 10 and later (including illumos systems).
+
+.TP
+\fB\-ds\fP
+Disable use of speculative \fBepoll\fP(7). \fBepoll\fP(7) is available only on
+Linux 2.6 and some custom Linux 2.4 systems.
+
+.TP
+\fB\-de\fP
+Disable use of \fBepoll\fP(7). \fBepoll\fP(7) is available only on Linux 2.6
+and some custom Linux 2.4 systems.
+
+.TP
+\fB\-dp\fP
+Disables use of \fBpoll\fP(2). \fBselect\fP(2) might be used instead.
+
+.TP
+\fB\-dS\fP
+Disables use of \fBsplice\fP(2), which is broken on older kernels.
+
+.TP
+\fB\-db\fP
+Disables background mode (stays in foreground, useful for debugging).
+For debugging, the '\-db' option is very useful as it temporarily
+disables daemon mode and multi-process mode. The service can then be
+stopped by simply pressing Ctrl-C, without having to edit the config nor
+run full debug.
+
+.TP
+\fB\-dM[<byte>]\fP
+Initializes all allocated memory areas with the given <\fIbyte\fP>. This makes
+it easier to detect bugs resulting from uninitialized memory accesses, at the
+expense of touching all allocated memory once. If <\fIbyte\fP> is not
+specified, it defaults to 0x50 (ASCII 'P').
+
+.TP
+\fB\-m <megs>\fP
+Enforce a memory usage limit to a maximum of <megs> megabytes.
+
+.TP
+\fB\-sf <pidlist>\fP
+Send FINISH signal to the pids in pidlist after startup. The processes
+which receive this signal will wait for all sessions to finish before
+exiting. This option must be specified last, followed by any number of
+PIDs. Technically speaking, \fBSIGTTOU\fP and \fBSIGUSR1\fP are sent.
+
+.TP
+\fB\-st <pidlist>\fP
+Send TERMINATE signal to the pids in pidlist after startup. The processes
+which receive this signal will terminate immediately, closing all active
+sessions. This option must be specified last, followed by any number of
+PIDs. Technically speaking, \fBSIGTTOU\fP and \fBSIGTERM\fP are sent.
+
+.TP
+\f8\-x <unix_socket>\fP
+Attempt to connect to the unix socket, and retrieve all the listening sockets
+from the old process. Those sockets will then be used if possible instead of
+binding new ones.
+
+.TP
+\fB\-S <bind>[,<bind options>...]\fP
+In master-worker mode, create a master CLI. This CLI will enable access to the
+CLI of every worker. Useful for debugging, it's a convenient way of accessing a
+leaving process.
+
+.SH LOGGING
+Since HAProxy can run inside a chroot, it cannot reliably access /dev/log.
+For this reason, it uses the UDP protocol to send its logs to the server,
+even if it is the local server. People who experience trouble receiving
+logs should ensure that their syslog daemon listens to the UDP socket.
+Several Linux distributions which ship with syslogd from the sysklogd
+package have UDP disabled by default. The \fB\-r\fP option must be passed
+to the daemon in order to enable UDP.
+
+.SH SIGNALS
+Some signals have a special meaning for the haproxy daemon. Generally, they are used between daemons and need not be used by the administrator.
+.TP
+\- \fBSIGUSR1\fP
+Tells the daemon to stop all proxies and exit once all sessions are closed. It is often referred to as the "soft-stop" signal.
+.TP
+\- \fBSIGUSR2\fP
+In master-worker mode, reloads the configuration and sends a soft-stop signal to old processes.
+.TP
+\- \fBSIGTTOU\fP
+Tells the daemon to stop listening to all sockets. Used internally by \fB\-sf\fP and \fB\-st\fP.
+.TP
+\- \fBSIGTTIN\fP
+Tells the daemon to restart listening to all sockets after a \fBSIGTTOU\fP. Used internally when there was a problem during hot reconfiguration.
+.TP
+\- \fBSIGINT\fP and \fBSIGTERM\fP
+Both signals can be used to quickly stop the daemon.
+.TP
+\- \fBSIGHUP\fP
+Dumps the status of all proxies and servers into the logs. Mostly used for trouble-shooting purposes.
+.TP
+\- \fBSIGQUIT\fP
+Dumps information about memory pools on stderr. Mostly used for debugging purposes.
+.TP
+\- \fBSIGPIPE\fP
+This signal is intercepted and ignored on systems without \fBMSG_NOSIGNAL\fP.
+
+.SH SEE ALSO
+
+A much better documentation can be found in configuration.txt. On Debian
+systems, you can find this file in /usr/share/doc/haproxy/configuration.txt.gz.
+
+.SH AUTHOR
+
+HAProxy was written by Willy Tarreau. This man page was written by Arnaud Cornet and Willy Tarreau.
+
diff --git a/doc/internals/acl.txt b/doc/internals/acl.txt
new file mode 100644
index 0000000..0379331
--- /dev/null
+++ b/doc/internals/acl.txt
@@ -0,0 +1,82 @@
+2011/12/16 - How ACLs work internally in haproxy - w@1wt.eu
+
+An ACL is declared by the keyword "acl" followed by a name, followed by a
+matching method, followed by one or multiple pattern values :
+
+ acl internal src 127.0.0.0/8 10.0.0.0/8 192.168.0.0/16
+
+In the statement above, "internal" is the ACL's name (acl->name), "src" is the
+ACL keyword defining the matching method (acl_expr->kw) and the IP addresses
+are patterns of type acl_pattern to match against the source address.
+
+The acl_pattern struct may define one single pattern, a range of values or a
+tree of values to match against. The type of the patterns is implied by the
+ACL keyword. For instance, the "src" keyword implies IPv4 patterns.
+
+The line above constitutes an ACL expression (acl_expr). ACL expressions are
+formed of a keyword, an optional argument for the keyword, and a list of
+patterns (in fact, both a list and a root tree).
+
+Dynamic values are extracted according to a fetch function defined by the ACL
+keyword. This fetch function fills or updates a struct acl_test with all the
+extracted information so that a match function can compare it against all the
+patterns. The fetch function is called iteratively by the ACL engine until it
+reports no more value. This makes sense for instance when checking IP addresses
+found in HTTP headers, which can appear multiple times. The acl_test is kept
+intact between calls and even holds a context so that the fetch function knows
+where to start from for subsequent calls. The match function may also use the
+context even though it was not designed for that purpose.
+
+An ACL is defined only by its name and can be a series of ACL expressions. The
+ACL is deemed true when any of its expressions is true. They are evaluated in
+the declared order and can involve multiple matching methods.
+
+So in summary :
+
+ - an ACL is a series of tests to perform on a stream, any of which is enough
+ to validate the result.
+
+ - each test is defined by an expression associating a keyword and a series of
+ patterns.
+
+ - a keyword implies several things at once :
+ - the type of the patterns and how to parse them
+ - the method to fetch the required information from the stream
+ - the method to match the fetched information against the patterns
+
+ - a fetch function fills an acl_test struct which is passed to the match
+ function defined by the keyword
+
+ - the match function tries to match the value in the acl_test against the
+ pattern list declared in the expression which involved its acl_keyword.
+
+
+ACLs are used by conditional processing rules. A rule generally uses an "if" or
+"unless" keyword followed by an ACL condition (acl_cond). This condition is a
+series of term suites which are ORed together. Each term suite is a series of
+terms which are ANDed together. Terms may be negated before being evaluated in
+a suite. A term simply is a pointer to an ACL.
+
+We could then represent a rule by the following BNF :
+
+ rule = if-cond
+ | unless-cond
+
+ if-cond (struct acl_cond with ->pol = ACL_COND_IF)
+ = "if" condition
+
+ unless-cond (struct acl_cond with ->pol = ACL_COND_UNLESS)
+ = "unless" condition
+
+ condition
+ = term-suite
+ | term-suite "||" term-suite
+ | term-suite "or" term-suite
+
+ term-suite (struct acl_term_suite)
+ = term
+ | term term
+
+ term = acl
+ | "!" acl
+
diff --git a/doc/internals/api/appctx.txt b/doc/internals/api/appctx.txt
new file mode 100644
index 0000000..137ec7b
--- /dev/null
+++ b/doc/internals/api/appctx.txt
@@ -0,0 +1,142 @@
+Instantiation of applet contexts (appctx) in 2.6.
+
+
+1. Background
+
+Most applets are in fact simplified services that are called by the CLI when a
+registered keyword is matched. Some of them only have a ->parse() function
+which immediately returns with a final result, while others will return zero
+asking for the->io_handler() one to be called till the end. For these ones, a
+context is generally needed between calls to know where to restart from.
+
+Other applets are completely autonomous applets with their init function and
+an I/O handler, and these ones also need a persistent context between calls to
+the I/O handler. These ones are typically instantiated by "use-service" or by
+other means.
+
+Originally a few integers were provided to keep a trivial state (st0, st1, st2)
+and these ones progressively proved insufficient, leading to a "ctx.cli" sub-
+context that was allowed to use extra fields of various types. Other applets
+preferred to use their own context definition.
+
+All this resulted in the appctx->ctx to contain a myriad of definitions of
+various service contexts, and in some services abusing other services'
+definitions by laziness, and others being extended to use their own definition
+after having run for a long time on the generic types, some of which were not
+noticed and mistakenly used the same storage locations by accident. A massive
+cleanup was needed.
+
+
+2. New approach in 2.6
+
+In 2.6, there's an "svcctx" pointer that's initialized to NULL before any
+instantiation of an applet or of a CLI keyword's function. Applets and keyword
+handlers are free to make it point wherever they want, and to find it unaltered
+between subsequent calls, including up to the ->release() call. The "st2" state
+that was totally abused with random enums is not used anymore and was marked as
+deprecated. It's still initialized to zero before the first call though.
+
+One special area, "svc.storage[]", is large enough to contain any of the
+contexts that used to be present under "appctx->ctx". The "svcctx" may be set
+to point to this area so that a small structure can be allocated for free and
+without requiring error checking. In order to make this easier, a specially
+purposed function is provided: "applet_reserve_svcctx()". This function will
+require the caller to indicate how large an area it needs, and will return a
+pointer to this area after checking that it fits. If it does not, haproxy will
+crash. This is purposely done so that it's known during development that if a
+small structure doesn't fit, a different approach is required.
+
+As such, for the vast majority of commands, the process is the following one:
+
+ struct foo_ctx {
+ int myfield1;
+ int myfield2;
+ char *myfield3;
+ };
+
+ int io_handler(struct appctx *appctx)
+ {
+ struct foo_ctx *ctx = applet_reserve_svcctx(appctx, sizeof(*ctx));
+
+ if (!ctx->myfield1) {
+ /* first call */
+ ctx->myfield1++;
+ }
+ ...
+ }
+
+The pointer may be directly accessed from the I/O handler if it's known that it
+was already reserved by the init handler or parsing function. Otherwise it's
+guaranteed to be NULL so that can also serve as a test for a first call:
+
+ int parse_handler(struct appctx *appctx)
+ {
+ struct foo_ctx *ctx = applet_reserve_svcctx(appctx, sizeof(*ctx));
+ ctx->myfield1 = 12;
+ return 0;
+ }
+
+ int io_handler(struct appctx *appctx)
+ {
+ struct foo_ctx *ctx = appctx->svcctx;
+
+ for (; !ctx->myfield1; ctx->myfield1--) {
+ do_something();
+ }
+ ...
+ }
+
+There is no need to free anything because that space is not allocated but just
+points to a reserved area.
+
+If it is too small (its size is APPLET_MAX_SVCCTX bytes), it is preferable to
+use it with dynamically allocated structures (pools, malloc, etc). For example:
+
+ int io_handler(struct appctx *appctx)
+ {
+ struct foo_ctx *ctx = appctx->svcctx;
+
+ if (!ctx) {
+ /* first call */
+ ctx = pool_alloc(pool_foo_ctx);
+ if (!ctx)
+ return 1;
+ }
+ ...
+ }
+
+ void io_release(struct appctx *appctx)
+ {
+ pool_free(pool_foo_ctx, appctx->svcctx);
+ }
+
+The CLI code itself uses this mechanism for the cli_print_*() functions. Since
+these functions are terminal (i.e. not meant to be used in the middle of an I/O
+handler as they share the same contextual space), they always reset the svcctx
+pointer to place it to the "cli_print_ctx" mapped in ->svc.storage.
+
+
+3. Transition for old code
+
+A lot of care was taken to make the transition as smooth as possible for
+out-of-tree code since that's an API change. A dummy "ctx.cli" struct still
+exists in the appctx struct, and it happens to map perfectly to the one set by
+cli_print_*, so that if some code uses a mix of both, it will still work.
+However, it will build with "deprecated" warnings allowing to spot the
+remaining places. It's a good exercise to rename "ctx.cli" in "appctx" and see
+if the code still compiles.
+
+Regarding the "st2" sub-state, it will disappear as well after 2.6, but is
+still provided and initialized so that code relying on it will still work even
+if it builds with deprecation warnings. The correct approach is to move this
+state into the newly defined applet's context, and to stop using the stats
+enums STAT_ST_* that often barely match the needs and result in code that is
+more complicated than desired (the STAT_ST_* enum values have also been marked
+as deprecated).
+
+The code dealing with "show fd", "show sess" and the peers applet show good
+examples of how to convert a registered keyword or an applet.
+
+All this transition code requires complex layouts that will be removed during
+2.7-dev so there is no other long-term option but to update the code (or better
+get it merged if it can be useful to other users).
diff --git a/doc/internals/api/buffer-api.txt b/doc/internals/api/buffer-api.txt
new file mode 100644
index 0000000..ac35300
--- /dev/null
+++ b/doc/internals/api/buffer-api.txt
@@ -0,0 +1,653 @@
+2018-07-13 - HAProxy Internal Buffer API
+
+
+1. Background
+
+HAProxy uses a "struct buffer" internally to store data received from external
+agents, as well as data to be sent to external agents. These buffers are also
+used during data transformation such as compression, header insertion or
+defragmentation, and are used to carry intermediary representations between the
+various internal layers. They support wrapping at the end, and they carry their
+own size information so that in theory it would be possible to use different
+buffer sizes in parallel even though this is not currently implemented.
+
+The format of this structure has evolved over time, to reach a point where it
+is convenient and versatile enough to have permitted to make several internal
+types converge into a single one (specifically the struct chunk disappeared).
+
+
+2. Representation as of 1.9-dev1
+
+The current buffer representation consists in a linear storage area of known
+size, with a head position indicating the oldest data, and a total data count
+expressed in bytes. The head position, data count and size are expressed as
+integers and are positive or null. By convention, the head position is strictly
+smaller than the buffer size and the data count is smaller than or equal to the
+size, so that wrapping can be resolved with a single subtract. A buffer not
+respecting these rules is said to be degenerate. Unless specified otherwise,
+the various API functions will adopt an undefined behaviour when passed such a
+degenerate buffer.
+
+ Buffer declaration :
+
+ struct buffer {
+ size_t size; // size of the storage area (wrapping point)
+ char *area; // start of the storage area
+ size_t data; // contents length after head
+ size_t head; // start offset of remaining data relative to area
+ };
+
+
+ Linear buffer representation :
+
+ area
+ |
+ V<--------------------------------------------------------->| size
+ +-----------+---------------------------------+-------------+
+ | |/////////////////////////////////| |
+ +-----------+---------------------------------+-------------+
+ |<--------->|<------------------------------->|
+ head data ^
+ |
+ tail
+
+
+ Wrapping buffer representation :
+
+ area
+ |
+ V<--------------------------------------------------------->| size
+ +---------------+------------------------+------------------+
+ |///////////////| |//////////////////|
+ +---------------+------------------------+------------------+
+ |<-------------------------------------->| head
+ |-------------->| ...data data...|<-----------------|
+ ^
+ |
+ tail
+
+
+3. Terminology
+
+Manipulating a buffer just based on a head and a wrapping data count is not
+very convenient, so we define a certain number of terms for important elements
+characterizing a buffer :
+
+ - origin : pointer to relative position 0 in the storage area. Undefined
+ when the buffer is not allocated.
+
+ - size : the allocated size of the storage area starting at the origin,
+ expressed in bytes. A buffer whose size is zero is said not to
+ be allocated, and its origin in this case is undefined.
+
+ - data : the amount of data the buffer contains, in bytes. It is always
+ lower than or equal to the buffer's size, hence it is always 0
+ for an unallocated buffer.
+
+ - emptiness : a buffer is said to be empty when it contains no data, hence
+ data == 0. It is possible for such buffers not to be allocated
+ and to have size == 0 as well.
+
+ - room : the available space in the buffer. This is its size minus data.
+
+ - head : position relative to origin where the oldest data byte is found
+ (it typically is what send() uses to pick outgoing data). The
+ head is strictly smaller than the size.
+
+ - tail : position relative to origin where the first spare byte is found
+ (it typically is what recv() uses to store incoming data). It
+ is always equal to the buffer's data added to its head modulo
+ the buffer's size.
+
+ - wrapping : the byte following the last one of the storage area loops back
+ to position 0. This is called wrapping. The wrapping point is
+ the first position relative to origin which doesn't belong to
+ the storage area. There is no wrapping when a buffer is not
+ allocated. Wrapping requires special care and means that the
+ regular string manipulation functions are not usable on most
+ buffers, unless it is known that no wrapping happens. Free
+ space may wrap as well if the buffer only contains data in the
+ middle.
+
+ - alignment : a buffer is said to be aligned if its data do not wrap. That
+ is, its head is strictly before the tail, or the buffer is
+ empty and the head is null. Aligning a buffer may be required
+ to use regular string manipulation functions which have no
+ support for wrapping.
+
+
+A buffer may be in three different states :
+ - unallocated : size == 0, area == 0 (b_is_null() is true)
+ - waiting : size == 0, area != 0
+ - allocated : size > 0, area > 0
+
+It is not permitted to have area == 0 with a non-null size. In addition, the
+waiting state may also be used to indicate a read-only buffer which does not
+wrap and which must not be freed (e.g. for use with error messages).
+
+The basic API only covers allocated buffers. Switching to/from the other states
+is covered by the management API since it requires specific allocation and free
+calls.
+
+
+4. Using buffers
+
+Buffers are defined in a few files :
+ - include/common/buf.h : structure definition, and manipulation functions
+ - include/common/buffer.h : resource management (alloc/free/wait lists)
+ - include/common/istbuf.h : advanced string manipulation
+
+
+4.1. Basic API
+
+The basic API is made of the functions which abstract accesses to the buffers
+and which help calculating their state, free space or used space.
+
+====================+==================+=======================================
+Function | Arguments/Return | Description
+--------------------+------------------+---------------------------------------
+b_is_null() | const buffer *buf| returns true if (and only if) the
+ | ret: int | buffer is not yet allocated and thus
+ | | points to a NULL area
+--------------------+------------------+---------------------------------------
+b_orig() | const buffer *buf| returns the pointer to the origin of
+ | ret: char * | the storage, which is the location of
+ | | byte at offset zero. This is mostly
+ | | used by functions which handle the
+ | | wrapping by themselves
+--------------------+------------------+---------------------------------------
+b_size() | const buffer *buf| returns the size of the buffer
+ | ret: size_t |
+--------------------+------------------+---------------------------------------
+b_wrap() | const buffer *buf| returns the pointer to the wrapping
+ | ret: char * | position of the buffer area, which is
+ | | by definition the first byte not part
+ | | of the buffer
+--------------------+------------------+---------------------------------------
+b_data() | const buffer *buf| returns the number of bytes present in
+ | ret: size_t | the buffer
+--------------------+------------------+---------------------------------------
+b_room() | const buffer *buf| returns the amount of room left in the
+ | ret: size_t | buffer
+--------------------+------------------+---------------------------------------
+b_full() | const buffer *buf| returns true if the buffer is full
+ | ret: int |
+--------------------+------------------+---------------------------------------
+__b_stop() | const buffer *buf| returns a pointer to the byte
+ | ret: char * | following the end of the buffer, which
+ | | may be out of the buffer if the buffer
+ | | ends on the last byte of the area. It
+ | | is the caller's responsibility to
+ | | either know that the buffer does not
+ | | wrap or to check that the result does
+ | | not wrap
+--------------------+------------------+---------------------------------------
+__b_stop_ofs() | const buffer *buf| returns an origin-relative offset
+ | ret: size_t | pointing to the byte following the end
+ | | of the buffer, which may be out of the
+ | | buffer if the buffer ends on the last
+ | | byte of the area. It's the caller's
+ | | responsibility to either know that the
+ | | buffer does not wrap or to check that
+ | | the result does not wrap
+--------------------+------------------+---------------------------------------
+b_stop() | const buffer *buf| returns the pointer to the byte
+ | ret: char * | following the end of the buffer, which
+ | | may be out of the buffer if the buffer
+ | | ends on the last byte of the area
+--------------------+------------------+---------------------------------------
+b_stop_ofs() | const buffer *buf| returns an origin-relative offset
+ | ret: size_t | pointing to the byte following the end
+ | | of the buffer, which may be out of the
+ | | buffer if the buffer ends on the last
+ | | byte of the area
+--------------------+------------------+---------------------------------------
+__b_peek() | const buffer *buf| returns a pointer to the data at
+ | size_t ofs | position <ofs> relative to the head of
+ | ret: char * | the buffer. Will typically point to
+ | | input data if called with the amount
+ | | of output data. It's the caller's
+ | | responsibility to either know that the
+ | | buffer does not wrap or to check that
+ | | the result does not wrap
+--------------------+------------------+---------------------------------------
+__b_peek_ofs() | const buffer *buf| returns an origin-relative offset
+ | size_t ofs | pointing to the data at position <ofs>
+ | ret: size_t | relative to the head of the
+ | | buffer. Will typically point to input
+ | | data if called with the amount of
+ | | output data. It's the caller's
+ | | responsibility to either know that the
+ | | buffer does not wrap or to check that
+ | | the result does not wrap
+--------------------+------------------+---------------------------------------
+b_peek() | const buffer *buf| returns a pointer to the data at
+ | size_t ofs | position <ofs> relative to the head of
+ | ret: char * | the buffer. Will typically point to
+ | | input data if called with the amount
+ | | of output data. If applying <ofs> to
+ | | the buffers' head results in a
+ | | position between <size> and 2*>size>-1
+ | | included, a wrapping compensation is
+ | | applied to the result
+--------------------+------------------+---------------------------------------
+b_peek_ofs() | const buffer *buf| returns an origin-relative offset
+ | size_t ofs | pointing to the data at position <ofs>
+ | ret: size_t | relative to the head of the
+ | | buffer. Will typically point to input
+ | | data if called with the amount of
+ | | output data. If applying <ofs> to the
+ | | buffers' head results in a position
+ | | between <size> and 2*>size>-1
+ | | included, a wrapping compensation is
+ | | applied to the result
+--------------------+------------------+---------------------------------------
+__b_head() | const buffer *buf| returns the pointer to the buffer's
+ | ret: char * | head, which is the location of the
+ | | next byte to be dequeued. The result
+ | | is undefined for unallocated buffers
+--------------------+------------------+---------------------------------------
+__b_head_ofs() | const buffer *buf| returns an origin-relative offset
+ | ret: size_t | pointing to the buffer's head, which
+ | | is the location of the next byte to be
+ | | dequeued. The result is undefined for
+ | | unallocated buffers
+--------------------+------------------+---------------------------------------
+b_head() | const buffer *buf| returns the pointer to the buffer's
+ | ret: char * | head, which is the location of the
+ | | next byte to be dequeued. The result
+ | | is undefined for unallocated
+ | | buffers. If applying <ofs> to the
+ | | buffers' head results in a position
+ | | between <size> and 2*>size>-1
+ | | included, a wrapping compensation is
+ | | applied to the result
+--------------------+------------------+---------------------------------------
+b_head_ofs() | const buffer *buf| returns an origin-relative offset
+ | ret: size_t | pointing to the buffer's head, which
+ | | is the location of the next byte to be
+ | | dequeued. The result is undefined for
+ | | unallocated buffers. If applying
+ | | <ofs> to the buffers' head results in
+ | | a position between <size> and
+ | | 2*>size>-1 included, a wrapping
+ | | compensation is applied to the result
+--------------------+------------------+---------------------------------------
+__b_tail() | const buffer *buf| returns the pointer to the tail of the
+ | ret: char * | buffer, which is the location of the
+ | | first byte where it is possible to
+ | | enqueue new data. The result is
+ | | undefined for unallocated buffers
+--------------------+------------------+---------------------------------------
+__b_tail_ofs() | const buffer *buf| returns an origin-relative offset
+ | ret: size_t | pointing to the tail of the buffer,
+ | | which is the location of the first
+ | | byte where it is possible to enqueue
+ | | new data. The result is undefined for
+ | | unallocated buffers
+--------------------+------------------+---------------------------------------
+b_tail() | const buffer *buf| returns the pointer to the tail of the
+ | ret: char * | buffer, which is the location of the
+ | | first byte where it is possible to
+ | | enqueue new data. The result is
+ | | undefined for unallocated buffers
+--------------------+------------------+---------------------------------------
+b_tail_ofs() | const buffer *buf| returns an origin-relative offset
+ | ret: size_t | pointing to the tail of the buffer,
+ | | which is the location of the first
+ | | byte where it is possible to enqueue
+ | | new data. The result is undefined for
+ | | unallocated buffers
+--------------------+------------------+---------------------------------------
+b_next() | const buffer *buf| for an absolute pointer <p> pointing
+ | const char *p | to a valid location within buffer <b>,
+ | ret: char * | returns the absolute pointer to the
+ | | next byte, which usually is at (p + 1)
+ | | unless p reaches the wrapping point
+ | | and wrapping is needed
+--------------------+------------------+---------------------------------------
+b_next_ofs() | const buffer *buf| for an origin-relative offset <o>
+ | size_t o | pointing to a valid location within
+ | ret: size_t | buffer <b>, returns either the
+ | | relative offset pointing to the next
+ | | byte, which usually is at (o + 1)
+ | | unless o reaches the wrapping point
+ | | and wrapping is needed
+--------------------+------------------+---------------------------------------
+b_dist() | const buffer *buf| returns the distance between two
+ | const char *from | pointers, taking into account the
+ | const char *to | ability to wrap around the buffer's
+ | ret: size_t | end. The operation is not defined if
+ | | either of the pointers does not belong
+ | | to the buffer or if their distance is
+ | | greater than the buffer's size
+--------------------+------------------+---------------------------------------
+b_almost_full() | const buffer *buf| returns 1 if the buffer uses at least
+ | ret: int | 3/4 of its capacity, otherwise
+ | | zero. Buffers of size zero are
+ | | considered full
+--------------------+------------------+---------------------------------------
+b_space_wraps() | const buffer *buf| returns non-zero only if the buffer's
+ | ret: int | free space wraps, which means that the
+ | | buffer contains data that are not
+ | | touching at least one edge
+--------------------+------------------+---------------------------------------
+b_contig_data() | const buffer *buf| returns the amount of data that can
+ | size_t start | contiguously be read at once starting
+ | ret: size_t | from a relative offset <start> (which
+ | | allows to easily pre-compute blocks
+ | | for memcpy). The start point will
+ | | typically contain the amount of past
+ | | data already returned by a previous
+ | | call to this function
+--------------------+------------------+---------------------------------------
+b_contig_space() | const buffer *buf| returns the amount of bytes that can
+ | ret: size_t | be appended to the buffer at once
+--------------------+------------------+---------------------------------------
+b_getblk() | const buffer *buf| gets one full block of data at once
+ | char *blk | from a buffer, starting from offset
+ | size_t len | <offset> after the buffer's head, and
+ | size_t offset | limited to no more than <len> bytes.
+ | ret: size_t | The caller is responsible for ensuring
+ | | that neither <offset> nor <offset> +
+ | | <len> exceed the total number of bytes
+ | | available in the buffer. Return zero
+ | | if not enough data was available, in
+ | | which case blk is left undefined, or
+ | | the number of bytes read which is
+ | | equal to the requested size
+--------------------+------------------+---------------------------------------
+b_getblk_nc() | const buffer *buf| gets one or two blocks of data at once
+ | const char **blk1| from a buffer, starting from offset
+ | size_t *len1 | <ofs> after the beginning of its
+ | const char **blk2| output, and limited to no more than
+ | size_t *len2 | <max> bytes. The caller is responsible
+ | size_t ofs | for ensuring that neither <ofs> nor
+ | size_t max | <ofs>+<max> exceed the total number of
+ | ret: int | bytes available in the buffer. Returns
+ | | 0 if not enough data were available,
+ | | or the number of blocks filled (1 or
+ | | 2). <blk1> is always filled before
+ | | <blk2>. The unused blocks are left
+ | | undefined, and the buffer is left
+ | | unaffected. Unused buffers are left in
+ | | an undefined state
+--------------------+------------------+---------------------------------------
+b_reset() | buffer *buf | resets a buffer. The size is not
+ | ret: void | touched. In practice it resets the
+ | | head and the data length
+--------------------+------------------+---------------------------------------
+b_sub() | buffer *buf | decreases the buffer length by <count>
+ | size_t count | without touching the head position
+ | ret: void | (only the tail moves). this may mostly
+ | | be used to trim pending data before
+ | | reusing a buffer. The caller is
+ | | responsible for not removing more than
+ | | the available data
+--------------------+------------------+---------------------------------------
+b_add() | buffer *buf | increase the buffer length by <count>
+ | size_t count | without touching the head position
+ | ret: void | (only the tail moves). This is used
+ | | when adding data at the tail of a
+ | | buffer. The caller is responsible for
+ | | not adding more than the available
+ | | room
+--------------------+------------------+---------------------------------------
+b_set_data() | buffer *buf | sets the buffer's length, by adjusting
+ | size_t len | the buffer's tail only. The caller is
+ | ret: void | responsible for passing a valid length
+--------------------+------------------+---------------------------------------
+b_del() | buffer *buf | deletes <del> bytes at the head of
+ | size_t del | buffer <b> and updates the head. The
+ | ret: void | caller is responsible for not removing
+ | | more than the available data. This is
+ | | used after sending data from the
+ | | buffer
+--------------------+------------------+---------------------------------------
+b_realign_if_empty()| buffer *buf | realigns a buffer if it's empty, does
+ | ret: void | nothing otherwise. This is mostly used
+ | | after b_del() to make an empty
+ | | buffer's free space contiguous
+--------------------+------------------+---------------------------------------
+b_slow_realign() | buffer *buf | realigns a possibly wrapping buffer so
+ | size_t output | that the part remaining to be parsed
+ | ret: void | is contiguous and starts at the
+ | | beginning of the buffer and the
+ | | already parsed output part ends at the
+ | | end of the buffer. This provides the
+ | | best conditions since it allows the
+ | | largest inputs to be processed at once
+ | | and ensures that once the output data
+ | | leaves, the whole buffer is available
+ | | at once. The number of output bytes
+ | | supposedly present at the beginning of
+ | | the buffer and which need to be moved
+ | | to the end must be passed in <output>.
+ | | It will effectively make this offset
+ | | the new wrapping point. A temporary
+ | | swap area at least as large as b->size
+ | | must be provided in <swap>. It's up
+ | | to the caller to ensure <output> is no
+ | | larger than the difference between the
+ | | whole buffer's length and its input
+--------------------+------------------+---------------------------------------
+b_putchar() | buffer *buf | tries to append char <c> at the end of
+ | char c | buffer <b>. Supports wrapping. New
+ | ret: void | data are silently discarded if the
+ | | buffer is already full
+--------------------+------------------+---------------------------------------
+b_putblk() | buffer *buf | tries to append block <blk> at the end
+ | const char *blk | of buffer <b>. Supports wrapping. Data
+ | size_t len | are truncated if the buffer is too
+ | ret: size_t | short or if not enough space is
+ | | available. It returns the number of
+ | | bytes really copied
+--------------------+------------------+---------------------------------------
+b_move() | buffer *buf | moves block (src,len) left or right
+ | size_t src | by <shift> bytes, supporting wrapping
+ | size_t len | and overlapping.
+ | size_t shift |
+--------------------+------------------+---------------------------------------
+b_rep_blk() | buffer *buf | writes the block <blk> at position
+ | char *pos | <pos> which must be in buffer <b>, and
+ | char *end | moves the part between <end> and the
+ | const char *blk | buffer's tail just after the end of
+ | size_t len | the copy of <blk>. This effectively
+ | ret: int | replaces the part located between
+ | | <pos> and <end> with a copy of <blk>
+ | | of length <len>. The buffer's length
+ | | is automatically updated. This is used
+ | | to replace a block with another one
+ | | inside a buffer. The shift value
+ | | (positive or negative) is returned. If
+ | | there's no space left, the move is not
+ | | done. If <len> is null, the <blk>
+ | | pointer is allowed to be null, in
+ | | order to erase a block
+--------------------+------------------+---------------------------------------
+b_xfer() | buffer *src | transfers at most <count> bytes from
+ | buffer *dst | buffer <src> to buffer <dst> and
+ | size_t cout | returns the number of bytes copied.
+ | ret: size_t | The bytes are removed from <src> and
+ | | added to <dst>. The caller guarantees
+ | | that <count> is <= b_room(dst)
+====================+==================+=======================================
+
+
+4.2. String API
+
+The string API aims at providing both convenient and efficient ways to read and
+write to/from buffers using indirect strings (ist). These strings and some
+associated functions are defined in ist.h.
+
+====================+==================+=======================================
+Function | Arguments/Return | Description
+--------------------+------------------+---------------------------------------
+b_isteq() | const buffer *b | b_isteq() : returns > 0 if the first
+ | size_t o | <n> characters of buffer <b> starting
+ | size_t n | at offset <o> relative to the buffer's
+ | const ist ist | head match <ist>. (empty strings do
+ | ret: int | match). It is designed to be used with
+ | | reasonably small strings (it matches a
+ | | single byte per loop iteration). It is
+ | | expected to be used with an offset to
+ | | skip old data. Return value number of
+ | | matching bytes if >0, not enough bytes
+ | | or empty string if 0, or non-matching
+ | | byte found if <0.
+--------------------+------------------+---------------------------------------
+b_isteat | struct buffer *b | b_isteat() : "eats" string <ist> from
+ | const ist ist | the head of buffer <b>. Wrapping data
+ | ret: ssize_t | is explicitly supported. It matches a
+ | | single byte per iteration so strings
+ | | should remain reasonably small.
+ | | Returns the number of bytes matched
+ | | and eaten if >0, not enough bytes or
+ | | matched empty string if 0, or non
+ | | matching byte found if <0.
+--------------------+------------------+---------------------------------------
+b_istput | struct buffer *b | b_istput() : injects string <ist> at
+ | const ist ist | the tail of output buffer <b> provided
+ | ret: ssize_t | that it fits. Wrapping is supported.
+ | | It's designed for small strings as it
+ | | only writes a single byte per
+ | | iteration. Returns the number of
+ | | characters copied (ist.len), 0 if it
+ | | temporarily does not fit, or -1 if it
+ | | will never fit. It will only modify
+ | | the buffer upon success. In all cases,
+ | | the contents are copied prior to
+ | | reporting an error, so that the
+ | | destination at least contains a valid
+ | | but truncated string.
+--------------------+------------------+---------------------------------------
+b_putist | struct buffer *b | b_putist() : tries to copy as much as
+ | const ist ist | possible of string <ist> into buffer
+ | ret: size_t | <b> and returns the number of bytes
+ | | copied (truncation is possible). It
+ | | uses b_putblk() and is suitable for
+ | | large blocks.
+====================+==================+=======================================
+
+
+4.3. Management API
+
+The management API makes a distinction between an empty buffer, which by
+definition is not allocated but is ready to be allocated at any time, and a
+buffer which failed an allocation and is waiting for an available area to be
+offered. The functions allow to register on a list to be notified about buffer
+availability, to notify others of a number of buffers just released, and to be
+and to be notified of buffer availability. All allocations are made through the
+standard buffer pools.
+
+====================+==================+=======================================
+Function | Arguments/Return | Description
+--------------------+------------------+---------------------------------------
+buffer_almost_full | const buffer *buf| returns true if the buffer is not null
+ | ret: int | and at least 3/4 of the buffer's space
+ | | are used. A waiting buffer will match.
+--------------------+------------------+---------------------------------------
+b_alloc | buffer *buf | ensures that <buf> is allocated or
+ | ret: buffer * | allocates a buffer and assigns it to
+ | | *buf. If no memory is available, (1)
+ | | is assigned instead with a zero size.
+ | | The allocated buffer is returned, or
+ | | NULL in case no memory is available
+--------------------+------------------+---------------------------------------
+__b_free | buffer *buf | releases <buf> which must be allocated
+ | ret: void | and marks it empty
+--------------------+------------------+---------------------------------------
+b_free | buffer *buf | releases <buf> only if it is allocated
+ | ret: void | and marks it empty
+--------------------+------------------+---------------------------------------
+offer_buffers() | void *from | offer a buffer currently belonging to
+ | uint threshold | target <from> to whoever needs
+ | ret: void | one. Any pointer is valid for <from>,
+ | | including NULL. Its purpose is to
+ | | avoid passing a buffer to oneself in
+ | | case of failed allocations (e.g. need
+ | | two buffers, get one, fail, release it
+ | | and wake up self again). In case of
+ | | normal buffer release where it is
+ | | expected that the caller is not
+ | | waiting for a buffer, NULL is fine
+====================+==================+=======================================
+
+
+5. Porting code from older versions
+
+The previous buffer API introduced in 1.5-dev9 (May 2012) used to look like the
+following (with the struct renamed to old_buffer here to avoid confusion during
+quick lookups at the doc). It's worth noting that the "data" field used to be
+part of the struct but with a different type and meaning. It's important to be
+careful about potential code making use of &b->data as it will silently compile
+but fail.
+
+ Previous buffer declaration :
+
+ struct old_buffer {
+ char *p; /* buffer's start pointer, separates in and out data */
+ unsigned int size; /* buffer size in bytes */
+ unsigned int i; /* number of input bytes pending for analysis in the buffer */
+ unsigned int o; /* number of out bytes the sender can consume from this buffer */
+ char data[0]; /* <size> bytes */
+ };
+
+ Previous linear buffer representation :
+
+ data p
+ | |
+ V V
+ +-----------+--------------------+------------+-------------+
+ | |////////////////////|////////////| |
+ +-----------+--------------------+------------+-------------+
+ <---------------------------------------------------------> size
+ <------------------> <---------->
+ o i
+
+There is this correspondence between old and new fields (some will involve a
+knowledge of a channel when the output byte count is required) :
+
+ Old | New
+ --------+----------------------------------------------------
+ p | data + head + co_data(channel) // ci_head(channel)
+ size | size
+ i | data - co_data(channel) // ci_data(channel)
+ o | co_data(channel) // channel->output
+ data | area
+ --------+-----------------------------------------------------
+
+Then some common expressions can be mapped like this :
+
+ Old | New
+ -----------------------+---------------------------------------
+ b->data | b_orig(b)
+ &b->data | b_orig(b)
+ bi_ptr(b) | ci_head(channel)
+ bi_end(b) | b_tail(b)
+ bo_ptr(b) | b_head(b)
+ bo_end(b) | co_tail(channel)
+ bi_putblk(b,s,l) | b_putblk(b,s,l)
+ bo_getblk(b,s,l,o) | b_getblk(b,s,l,o)
+ bo_getblk_nc(b,s,l,o) | b_getblk_nc(b,s,l,o,0,co_data(channel))
+ b->i + b->o | b_data(b)
+ b->data + b->size | b_wrap(b)
+ b->i += len | b_add(b, len)
+ b->i -= len | b_sub(b, len)
+ b->i = len | b_set_data(b, co_data(channel) + len)
+ b->o += len | b_add(b, len); channel->output += len
+ b->o -= len | b_del(b, len); channel->output -= len
+ -----------------------+---------------------------------------
+
+The buffer modification functions are less straightforward and depend a lot on
+the context where they are used. It is strongly advised to figure in the list
+of functions above what is available based on what is attempted to be done in
+the existing code.
+
+Note that it is very likely that any out-of-tree code relying on buffers will
+not use both ->i and ->o but instead will use exclusively ->i on the side
+producing data and use exclusively ->o on the side consuming data (such as in a
+mux or in an applet). In both cases, it should be assumed that the other side
+is always zero and that either ->i or ->o is replaced with ->data, making the
+remaining code much simpler (no more code duplication based on the data
+direction).
diff --git a/doc/internals/api/event_hdl.txt b/doc/internals/api/event_hdl.txt
new file mode 100644
index 0000000..72eeff8
--- /dev/null
+++ b/doc/internals/api/event_hdl.txt
@@ -0,0 +1,1015 @@
+ -----------------------------------------
+ event_hdl Guide - version 2.8
+ ( Last update: 2022-11-14 )
+ ------------------------------------------
+
+ABSTRACT
+--------
+
+The event_hdl support is a new feature of HAProxy 2.7. It is a way to easily
+handle general events in a simple to maintain fashion, while keeping core code
+impact to the bare minimum.
+
+This document first describes how to use already supported events,
+then how to add support for your very own events.
+
+This feature is quite new for now. The API is not frozen and will be
+updated/modified/improved/extended as needed.
+
+SUMMARY
+-------
+
+ 1. event_hdl introduction
+ 2. How to handle existing events
+ 2.1 SYNC mode
+ 2.2 ASYNC mode
+ 2.2.1 normal version
+ 2.2.2 task version
+ 2.3 Advanced features
+ 2.3.1 sub_mgmt
+ 2.3.2 subscription external lookups
+ 2.3.3 subscription ptr
+ 2.3.4 private_free
+ 3. How to add support for new events
+ 3.1 Declaring a new event data structure
+ 3.2 Publishing an event
+ 4. Subscription lists
+ 5. misc/helper functions
+
+
+1. EVENT_HDL INTRODUCTION
+-----------------------
+
+EVENT_HDL provides two complementary APIs, both are implemented
+in src/event_hdl.c and include/haproxy/event_hdl(-t).h:
+
+One API targeting developers that want to register event
+handlers that will be notified when specific events occur in the process.
+(See section 2.)
+
+One API targeting developers that want to notify registered handlers about
+an event that is happening in the process.
+(See section 3.)
+
+2. HOW TO HANDLE EXISTING EVENTS
+---------------------
+
+To handle existing events, you must first decide which events you're
+interested in.
+
+event types are defined as follow:
+
+```
+ /* type for storing event subscription type */
+ typedef struct event_hdl_sub_type
+ {
+ /* up to 256 families, non cumulative, adjust if needed */
+ uint8_t family;
+ /* up to 16 sub types using bitmasks, adjust if needed */
+ uint16_t subtype;
+ } event_hdl_sub_type;
+```
+
+For an up to date list of already supported events,
+please refer to include/haproxy/event_hdl-t.h
+At the end of the file you will find existing event types.
+
+Each event family provides an unique data structure that will
+be provided to the event handler (registered to one or more
+event subtypes) when such events occur.
+
+An event handler can subscribe to a single event family type at a time, but
+within the family type it can subscribe to multiple event subtypes.
+
+ For example, let's consider the SERVER family type.
+
+ Let's assume it provides the event_hdl_cb_data_server data structure.
+
+ We can register a handler that will be notified for
+ every SERVER event types using:
+ EVENT_HDL_SUB_SERVER
+
+ This will include EVENT_HDL_SUB_SERVER_ADD,
+ EVENT_HDL_SUB_SERVER_DEL [...]
+
+ But we can also subscribe to a specific subtype only,
+ for example server deletion:
+ EVENT_HDL_SUB_SERVER_DEL
+
+ You can even combine multiple SERVER subtypes using
+ event_hdl_sub_type_add function helper:
+ event_hdl_sub_type_add(EVENT_HDL_SUB_SERVER_DEL,
+ EVENT_HDL_SUB_SERVER_ADD)
+
+ (will refer to server deletion as well as server addition)
+
+Registering a handler comes into multiple flavors:
+
+ SYNC mode:
+ handler is called in a blocking manner directly from the
+ thread that publishes the event.
+ This mode should be used with precaution because it could
+ slow the caller or cause deadlocks if used improperly.
+
+ Sync mode is useful when you directly depend on data or
+ state consistency from the caller.
+
+ Sync mode gives you access to unsafe elements in the data structure
+ provided by the caller (again, see event_hdl-t.h for more details).
+ The data structure may provide lock hints in the unsafe section
+ so that you know which locks are already held within the
+ calling context, hopefully preventing you from relocking
+ an already locked element and preventing deadlocks.
+
+ ASYNC mode:
+ handler is called in a non-blocking manner
+ (in a dedicated tasklet),
+ thus, the caller (that published the event) is not affected
+ by the handler. (time wise and data wise)
+
+ This is the safest way to handle events,
+ but it also comes with a limitation:
+
+ unsafe elements in the data structure provided by
+ the caller SHOULD be used under NO circumstances.
+ Indeed, only safe elements are meant to be used
+ when handling the event in async mode.
+
+ ASYNC mode is declined in 2 different versions:
+ normal:
+ handler is simply a function pointer
+ (same prototype as sync mode),
+ that is called asynchronously with relevant data
+ when the event is published. Only difference with
+ sync mode here is that 'unsafe' data provided
+ by the data structure may not be used.
+ task:
+ handler is a user defined task(let) that uses an event
+ queue to consume pending events.
+ This mode is interesting when you need to perform
+ advanced operations or you need to handle the event
+ in an already existing task context.
+ It is a bit more complicated to setup, but really
+ nothing to worry about, some examples will be
+ provided later in this document.
+
+event subscription is performed using the function:
+
+ event_hdl_subscribe(list, event, hdl);
+
+ The function returns 1 in case of success,
+ and 0 in case of failure (bad arguments, or memory error)
+
+ The function may BUG_ON if used improperly (invalid arguments)
+
+ <list> is either user specified list used to store the
+ new subscription, or NULL if you want to store the subscription
+ in the process global list.
+
+ <list> is also asked when publishing an event,
+ so specifying list could be useful, if, for example,
+ you only want to subscribe to a specific subscription list
+ (see this as a scope for example, NULL being full scope,
+ and specific list being limited scope)
+
+ We will use server events as an example:
+
+ You could register to events for ALL servers by using the
+ global list (NULL), or only to a specific server events
+ by using the subscription list dedicated to a single server.
+
+ <event> are the events (family.subtypes) you're subscribing to
+
+ <hdl> contains required handler options, it must be provided using
+ EVENT_HDL_(TASK_)(A)SYNC() and EVENT_HDL_ID_(TASK_)(A)SYNC()
+ helper macros.
+
+ See include/haproxy/event_hdl.h or below to know which macro
+ best suits your needs.
+
+ When registering a handler, you have the ability to provide an
+ unique ID (using EVENT_HDL_ID_ macro family) that could be used
+ later to perform lookups on the subscription.
+ ID is stored as an uint64_t hash that is expected to be computed using
+ general purpose event_hdl_id inline function provided by event_hdl.h.
+ Not providing an ID (using EVENT_HDL_ macro family)
+ results in the subscription being considered as anonymous.
+ As the name implies, anonymous subscriptions don't support lookups.
+
+2.1 SYNC MODE
+---------------------
+
+Example, you want to register a sync handler that will be called when
+a new server is added.
+
+Here is what the handler function will look like:
+```
+void my_sync_handler(const struct event_hdl_cb *cb, void *private)
+{
+ const struct event_hdl_cb_data_server *server = cb->e_data;
+
+ /* using EVENT_HDL_ASSERT_SYNC is a good practice to ensure
+ * that the function breaks if used in async mode
+ * (because we will access unsafe data in this function that
+ * is sync mode only)
+ */
+ EVENT_HDL_ASSERT_SYNC(cb);
+ printf("I've been called for '%s', private = %p\n",
+ event_hdl_sub_type_to_string(cb->e_type), private);
+ printf("server name is '%s'\n", server->safe.name);
+
+ /* here it is safe to use unsafe data */
+ printf("server ptr is '%p'\n", server->unsafe.ptr);
+
+ /* from here you have the possibility to manage the subscription
+ * cb->sub_mgmt->unsub(cb->sub_mgmt);
+ * // hdl will be removed from the subscription list
+ */
+}
+```
+
+Here is how you perform the subscription:
+
+anonymous subscription:
+```
+ int private = 10;
+
+ event_hdl_subscribe(NULL, EVENT_HDL_SUB_SERVER_ADD,
+ EVENT_HDL_SYNC(my_sync_handler, &private, NULL));
+```
+
+identified subscription:
+```
+ int private = 10;
+ uint64_t id = event_hdl_id("test", "sync");
+
+ event_hdl_subscribe(NULL, EVENT_HDL_SUB_SERVER_ADD,
+ EVENT_HDL_ID_SYNC(id,
+ my_sync_handler,
+ &private,
+ NULL));
+
+```
+
+identified subscription where freeing private is required when subscription ends:
+(also works for anonymous)
+(more on this feature in 2.3.4)
+```
+ int *private = malloc(sizeof(*private));
+ uint64_t id = event_hdl_id("test", "sync_free");
+
+ BUG_ON(!private);
+ *private = 10;
+
+ /* passing free as 'private_free' function so that
+ * private can be freed when unregistering is performed
+ */
+ event_hdl_subscribe(NULL, EVENT_HDL_SUB_SERVER_ADD,
+ EVENT_HDL_ID_SYNC(id,
+ my_sync_handler,
+ private,
+ free));
+
+
+ /* ... */
+
+ // unregistering the identified hdl
+ if (event_hdl_lookup_unsubscribe(NULL, id)) {
+ printf("private will automatically be freed!\n");
+ }
+```
+
+2.2 ASYNC MODE
+---------------------
+
+As mentioned before, async mode comes in 2 flavors, normal and task.
+
+2.2.1 NORMAL VERSION
+---------------------
+
+Normal is meant to be really easy to use, and highly compatible with sync mode.
+
+(Handler can easily be converted or copy pasted from async to sync mode
+and vice versa)
+
+Quick warning about sync to async handler conversion:
+
+please always use EVENT_HDL_ASSERT_SYNC whenever you develop a
+sync handler that performs unsafe data access.
+
+This way, if the handler were to be converted or copy pasted as is to
+async mode without removing unsafe data accesses,
+the handler will forcefully fail to indicate an error so that you
+know something has to be fixed in your handler code.
+
+Back to our async handler, let's say you want to declare an
+async handler that will be called when a new server is added.
+
+Here is what the handler function will look like:
+```
+void my_async_handler(const struct event_hdl_cb *cb, void *private)
+{
+ const struct event_hdl_cb_data_server *server = cb->e_data;
+
+ printf("I've been called for '%s', private = %p\n",
+ event_hdl_sub_type_to_string(cb->e_type), private);
+ printf("server name is '%s'\n", server->safe.name);
+
+ /* here it is not safe to use unsafe data */
+
+ /* from here you have the possibility to manage the subscription
+ * cb->sub_mgmt->unsub(cb->sub_mgmt);
+ * // hdl will be removed from the subscription list
+ */
+}
+```
+
+Note that it is pretty similar to sync handler, except
+for unsafe data access.
+
+Here is how you declare the subscription:
+
+anonymous subscription:
+```
+ int private = 10;
+
+ event_hdl_subscribe(NULL, EVENT_HDL_SUB_SERVER_ADD,
+ EVENT_HDL_ASYNC(my_async_handler, &private, NULL));
+```
+
+identified subscription:
+```
+ int private = 10;
+ uint64_t id = event_hdl_id("test", "async");
+
+ event_hdl_subscribe(NULL, EVENT_HDL_SUB_SERVER_ADD,
+ EVENT_HDL_ID_ASYNC(id,
+ my_async_handler,
+ &private,
+ NULL));
+
+```
+
+identified subscription where freeing private is required when subscription ends:
+(also works for anonymous)
+```
+ int *private = malloc(sizeof(*private));
+ uint64_t id = event_hdl_id("test", "async_free");
+
+ BUG_ON(!private);
+ *private = 10;
+
+ /* passing free as 'private_free' function so that
+ * private can be freed when unregistering is performed
+ */
+ event_hdl_subscribe(NULL, EVENT_HDL_SUB_SERVER_ADD,
+ EVENT_HDL_ID_ASYNC(id,
+ my_async_handler,
+ private,
+ free));
+
+ /* ... */
+
+ // unregistering the identified hdl
+ if (event_hdl_lookup_unsubscribe(NULL, id)) {
+ printf("private will automatically be freed when "
+ "all pending events referencing private "
+ "are consumed!\n");
+ }
+```
+
+2.2.2 TASK VERSION
+---------------------
+
+task version requires a bit more setup, but it's pretty
+straightforward actually.
+
+
+First, you need to initialize an event queue that will be used
+by event_hdl facility to push you events according to your subscription:
+
+```
+ event_hdl_async_equeue my_q;
+
+ event_hdl_async_equeue_init(&my_q);
+```
+
+
+Then, you need to declare a task(let) (or reuse existing task(let))
+
+It is your responsibility to make sure that the task(let) still exists
+(is not freed) when calling the subscribe function
+(and that the task remains valid as long as the subscription is).
+
+When a subscription referencing your task is over
+(either ended because of list purge, external code or from the handler itself),
+you will receive the EVENT_HDL_SUB_END event.
+When you receive this event, you must free it as usual and you can safely
+assume that the related subscription won't be sending you any more events.
+
+Here is what your task will look like (involving a single event queue):
+
+```
+struct task *event_hdl_async_task_my(struct task *task,
+ void *ctx, unsigned int state)
+{
+ struct tasklet *tl = (struct tasklet *)task;
+ event_hdl_async_equeue *queue = ctx;
+ struct event_hdl_async_event *event;
+ struct event_hdl_cb_data_server *srv;
+ uint8_t done = 0;
+
+ while ((event = event_hdl_async_equeue_pop(queue)))
+ {
+ if (event_hdl_sub_type_equal(event->type, EVENT_HDL_SUB_END)) {
+ done = 1;
+ event_hdl_async_free_event(event);
+ printf("no more events to come, "
+ "subscription is over\n");
+ break;
+ }
+
+ srv = event->data;
+
+ printf("task event %s, %d (name = %s)\n",
+ event_hdl_sub_type_to_string(event->type),
+ *((int *)event->private), srv->safe.name);
+ event_hdl_async_free_event(event);
+ }
+
+ if (done) {
+ /* our job is done, subscription is over:
+ * no more events to come
+ */
+ tasklet_free(tl);
+ return NULL;
+ }
+ return task;
+}
+
+```
+
+Here is how we would initialize the task event_hdl_async_task_my:
+```
+ struct tasklet *my_task;
+
+ my_task = tasklet_new();
+ BUG_ON(!my_task);
+ my_task->context = &my_q; // we declared my_q previously in this example
+ /* we declared event_hdl_async_task_my previously
+ * in this example
+ */
+ my_task->process = event_hdl_async_task_my;
+
+```
+
+Given our task and our previously initialized event queue, here is how
+to perform the subscription:
+```
+ int test_val = 11;
+ uint64_t id = event_hdl_id("test", "my_task");
+
+ /* anonymous variant */
+ event_hdl_subscribe(NULL, EVENT_HDL_SUB_SERVER_ADD,
+ EVENT_HDL_ASYNC_TASK(&my_q,
+ my_task,
+ &test_val,
+ NULL));
+ /* identified variant */
+ event_hdl_subscribe(NULL, EVENT_HDL_SUB_SERVER_ADD,
+ EVENT_HDL_ID_ASYNC_TASK(id,
+ &my_q,
+ my_task,
+ &test_val,
+ NULL));
+```
+
+Note: it is not recommended to perform multiple subscriptions
+ that share the same event queue or same task(let) (or both)
+
+ That is, having more than one subscription waking a task(let)
+ and/or feeding the same event queue.
+
+ No check is performed on this when registering, so the API
+ won't prevent you from doing it.
+
+ If you are going to do this anyway despite this warning:
+
+ In the case you need to stop the task prematurely
+ (if this is not going to happen please skip this paragraph):
+ You are responsible for acknowledging the end of every
+ active subscriptions that refer to your task or
+ your event queue(s).
+ And you really don't want a subscription associated with
+ your task or event queue to keep going when the task
+ is not active anymore because:
+ 1: there will be memory leak
+ (event queue might continue to receive new events)
+ 2: there is a 100% chance of process crash in case of event
+ because we will try to wake a task (your task)
+ that might already be freed. Thus UAF will occur.
+
+2.3 ADVANCED FEATURES
+-----------------------
+
+We've already covered some of these features in the previous examples.
+Here is a documented recap.
+
+
+2.3.1 SUB MGMT
+-----------------------
+
+From an event handler context, either sync or async mode:
+ You have the ability to directly manage the subscription
+ that provided the event.
+
+As of today, these actions are supported:
+ - Consulting the subscription.
+ - Modifying the subscription (resubscribing within same family)
+ - Unregistering the subscription (unsubscribing).
+
+To do this, consider the following structure:
+```
+ struct event_hdl_sub_mgmt
+ {
+ /* manage subscriptions from event
+ * this must not be used directly because
+ * locking might be required
+ */
+ struct event_hdl_sub *this;
+ /* safe functions than can be used from
+ * event context (sync and async mode)
+ */
+ struct event_hdl_sub_type (*getsub)(const struct event_hdl_sub_mgmt *);
+ int (*resub)(const struct event_hdl_sub_mgmt *, struct event_hdl_sub_type);
+ void (*unsub)(const struct event_hdl_sub_mgmt *);
+ };
+
+```
+A reference to this structure is provided in every handler mode.
+
+Sync mode and normal async mode (directly from the callback data pointer):
+```
+ const struct event_hdl_cb *cb;
+ // cb->sub_mgmt
+ // cb->sub_mgmt->getsub(cb->sub_mgmt);
+ // cb->sub_mgmt->unsub(cb->sub_mgmt);
+```
+
+task and notify async modes (from the event):
+```
+ struct event_hdl_async_event *event;
+ // event->sub_mgmt
+ // event->sub_mgmt.getsub(&event->sub_mgmt);
+ // event->sub_mgmt.unsub(&event->sub_mgmt);
+```
+
+2.3.2 SUBSCRIPTION EXTERNAL LOOKUPS
+-----------------------
+
+As you've seen in 2.3.1, managing the subscription directly
+from the handler is a possibility.
+
+But for identified subscriptions, you also have the ability to
+perform lookups and management operations on specific subscriptions
+within a list based on their ID, anywhere in the code.
+
+/!\ This feature is not available for anonymous subscriptions /!\
+
+Here are the actions already supported:
+
+ - unregistering a subscription (unsubscribing)
+ - updating a subscription (resubscribing within same family)
+ - getting a ptr/reference to the subscription
+
+Those functions are documented in event_hdl.h
+(search for EVENT_HDL_LOOKUP section).
+
+To select a specific subscription, you must provide
+the unique identifier (uint64_t hash) that was provided when subscribing.
+(using event_hdl_id(scope, name) function)
+
+Notes:
+ "id" is only unique within a given subscription list.
+
+ When using event_hdl_id to provide the id:
+ It is your responsibility to make sure that you "own"
+ the scope if you rely on name to be "free".
+
+ As ID computation is backed by xxhash hash API,
+ you should be aware that hash collisions could occur,
+ but are extremely rare and are thus considered safe
+ enough for this usage.
+ (see event_hdl.h for implementation details)
+
+ Please consider ptr based subscription management if
+ these limitations don't fit your requirements.
+
+Here are some examples:
+
+unsubscribing:
+```
+ /* registering "scope":"name" subscription */
+ event_hdl_subscribe(NULL, EVENT_HDL_SUB_SERVER_ADD,
+ EVENT_HDL_ID_SYNC(event_hdl_id("scope", "name"),
+ my_sync_handler,
+ NULL,
+ NULL));
+ /* unregistering "scope":"name" subscription */
+ event_hdl_lookup_unsubscribe(NULL, event_hdl_id("scope", "name"));
+```
+
+2.3.3 SUBSCRIPTION PTR
+-----------------------
+
+To manage existing subscriptions from external code,
+we already talked about identified subscriptions that
+allow lookups within list.
+
+But there is another way to accomplish this.
+
+When subscribing, you can use the event_hdl_subscribe_ptr() function
+variant (same arguments as event_hdl_subscribe()).
+
+What this function does, is instead of returning 1 in case of
+success and 0 in case of failure: it returns a valid subscription ptr
+for success and NULL for failure.
+
+Returned ptr is guaranteed to remain valid even if subscription
+is ended meanwhile because the ptr is internally guarded with a refcount.
+
+Thus, as long as you don't explicitly unregister the subscription with
+event_hdl_unsubscribe() or drop the reference using event_hdl_drop(),
+subscription ptr won't be freed.
+
+This ptr will allow you to use the following subscription
+management functions from external code:
+
+ - event_hdl_take() to increment subscription ptr refcount
+ (automatically incremented when using event_hdl_subscribe_ptr)
+ - event_hdl_drop() to decrement subscription ptr refcount
+ - event_hdl_resubscribe() to modify subscription subtype
+ - event_hdl_unsubscribe() to end the subscription
+ (refcount will be automatically decremented)
+
+Here is an example:
+```
+ struct event_hdl_sub *sub_ptr;
+
+ /* registering a subscription with subscribe_ptr */
+ sub_ptr = event_hdl_subscribe_ptr(NULL, EVENT_HDL_SUB_SERVER_ADD,
+ EVENT_HDL_SYNC(my_sync_handler,
+ NULL,
+ NULL));
+
+ /* ... */
+
+ /* unregistering the subscription */
+ event_hdl_unsubscribe(sub_ptr);
+```
+
+Regarding identified subscriptions that were registered using the non ptr
+subscribe function:
+
+You still have the ability to get a reference to the related subscription
+(if it still exists), by using event_hdl_lookup_take(list, id) function.
+event_hdl_lookup_take will return a subscription ptr in case of success
+and NULL in case of failure.
+Returned ptr reference is automatically incremented, so it is safe to use.
+
+Please don't forget to drop the reference
+when holding the ptr is no longer needed.
+
+Example:
+```
+ struct event_hdl_sub *sub_ptr = NULL;
+
+ /* registering subscription id "test":"ptr" with normal subscribe */
+ if (event_hdl_subscribe(NULL, EVENT_HDL_SUB_SERVER_ADD,
+ EVENT_HDL_ID_SYNC(event_hdl_id("test", "ptr"),
+ my_sync_handler,
+ NULL,
+ NULL))) {
+ /* fetch ref to subscription "test":"ptr" */
+ sub_ptr = event_hdl_lookup_take(NULL,
+ event_hdl_id("test", "ptr"));
+
+ /* unregister the subscription using lookup */
+ event_hdl_lookup_unsubscribe(NULL,
+ event_hdl_id("test", "ptr"));
+ }
+
+ /* ... */
+
+ /* unregistering the subscription with ptr
+ * will do nothing because subscription was
+ * already ended by lookup_unsubscribe, but
+ * here the catch is that sub_ptr is still
+ * valid so this won't crash the program
+ */
+ if (sub_ptr) {
+ event_hdl_unsubscribe(sub_ptr);
+ /* unsubscribe will also result in subscription
+ * reference drop, thus subscription will be freed here
+ * because sub_ptr was the last active reference.
+ * You must not use sub_ptr anymore past this point
+ * or UAF could occur
+ */
+ }
+
+```
+
+2.3.4 PRIVATE FREE
+-----------------------
+
+Upon handler subscription, you have the ability to provide
+a private data pointer that will be passed to the handler
+when subscribed events occur.
+
+Sometimes this private data pointer will rely on dynamically allocated memory.
+And in such cases, you have no way of knowing when
+freeing this pointer can be done safely.
+
+You could be tempted to think that freeing right after performing
+the unsubscription could be safe.
+But this is not the case, remember we could be dealing with async handlers
+that might still consume pending events even though unsubscription
+has been performed from external code.
+
+To deal with this, you may want to provide the private_free
+function pointer upon subscription.
+This way, private_free function will automatically be called
+(with private as argument) when private is no longer be used.
+
+Example:
+First we declare our private free function:
+```
+void my_private_free(void *my_private_data) {
+ /* here we only call free,
+ * but you could do more sophisticated stuff
+ */
+ free(my_private_data);
+}
+```
+Then:
+```
+ char *my_private_data = strdup("this string needs to be freed");
+
+ BUG_ON(!my_private_data);
+
+ event_hdl_subscribe(NULL, EVENT_HDL_SUB_SERVER_DEL,
+ EVENT_HDL_ID_ASYNC(event_hdl_id("test", "private"),
+ my_async_handler,
+ my_private_data,
+ my_private_free));
+
+ /* freeing my_private_data is not required anymore,
+ * it will be automatically freed by our private free
+ * function when subscription ends
+ */
+
+ /* unregistering "test":"private" subscription */
+ event_hdl_lookup_unsubscribe(NULL, event_hdl_id("test", "private"));
+
+ /* my_private_free will be automatically summoned when my_private_data
+ * is not referenced anymore
+ */
+```
+
+3 HOW TO ADD SUPPORT FOR NEW EVENTS
+-----------------------
+
+Adding support for a new event is pretty straightforward.
+
+First, you need to declare a new event subtype in event_hdl-t.h file
+(bottom of the file).
+
+You might want to declare a whole new event family, in which case
+you declare both the new family and the associated subtypes (if any).
+
+```
+ #define EVENT_HDL_SUB_NEW_FAMILY EVENT_HDL_SUB_FAMILY(4)
+ #define EVENT_HDL_SUB_NEW_FAMILY_SUBTYPE_1 EVENT_HDL_SUB_TYPE(4,0)
+```
+
+Then, you need to update the event_hdl_sub_type_map map,
+defined in src/event_hdl.c file (top of the file)
+to add string to event type and event type to string conversion support.
+You just need to add the missing entries corresponding to
+the event family / subtypes you've defined.
+
+Please follow this procedure:
+ You only added a new subtype to existing family: go to section 3.2
+ You added a new family: go to section 3.1
+
+3.1 DECLARING A NEW EVENT DATA STRUCTURE
+-----------------------
+
+You have the ability to provide additional data for a given
+event family when such events occur.
+
+Note that it is not mandatory: you could simply declare a new event family
+that does not provide any data.
+If this is your case, you can skip this section and go to 3.2 section.
+
+Now, take a look at this event data structure template
+(also defined at the top of event_hdl-t.h file):
+```
+ /* event data struct are defined as followed */
+ struct event_hdl_cb_data_template {
+ struct {
+ /* safe data can be safely used from both
+ * sync and async functions
+ * data consistency is guaranteed
+ */
+ } safe;
+ struct {
+ /* unsafe data may only be used from sync functions:
+ * in async mode, data consistency cannot be guaranteed
+ * and unsafe data may already be stale, thus using
+ * it is highly discouraged because it
+ * could lead to undefined behavior
+ * (UAF, null dereference...)
+ */
+ } unsafe;
+ };
+```
+
+This structure template allows you to easily create a new event
+data structure that can be provided with your new event family.
+
+You should name it after 'struct event_hdl_cb_data_new_family' so that it is
+easy to guess the event family it relates to.
+
+Indeed, each event data structure is to be associated with an
+unique event family type.
+For each subtypes within a family type, the associated data structure
+should be provided when publishing the event.
+
+The event data struct declaration should not be performed
+directly under event_hdl-t.h file:
+
+ It should be done in the header files of the corresponding
+ facility that will publish/provide this event.
+
+ Example: struct event_hdl_cb_data_server, provided for the
+ EVENT_HDL_SUB_SERVER event family, is going to be declared in
+ include/haproxy/server-t.h file.
+
+ However, in event_hdl-t.h, where you declare event family/subtypes,
+ you should add comments or links to the file containing the relevant
+ data struct declaration. This way we make sure all events related
+ information is centralized in event_hdl-t.h while keeping it clean
+ and not depending on any additional includes (you are free to
+ depend on specific data types within your custom event data structure).
+
+Please make sure that EVENT_HDL_ASYNC_EVENT_DATA (defined in event_hdl-t.h)
+is greater than sizeof(event_hdl_cb_data_new_family).
+
+It is required for async handlers to properly consume event data.
+
+You are free to adjust EVENT_HDL_ASYNC_EVENT_DATA size if needed.
+
+If EVENT_HDL_ASYNC_EVENT_DATA is not big enough to store your new
+event family struct, a compilation assert triggered by EVENT_HDL_CB_DATA
+will occur. In addition to this, an extra runtime BUG_ON will make
+sure the condition is met when publishing the event.
+The goal here is to force haproxy to fail explicitly so you know that
+something must be done on your side.
+
+3.1 PUBLISHING AN EVENT
+-----------------------
+
+Publishing an event is really simple.
+It relies on the event_hdl_publish function.
+
+The function is defined as follow:
+```
+ int event_hdl_publish(event_hdl_sub_list *sub_list,
+ event_hdl_sub_type e_type,
+ const struct event_hdl_cb_data *data);
+```
+
+We will ignore sub_list argument for now.
+In the examples below, we will use sub_list = NULL.
+Go to section 4 for a full picture about this feature.
+
+<e_type>: the event type that should be published.
+ All subscriptions referring to this event within
+ a subscription list context will be notified about the event.
+<data>: data provided for the event family of <e_type>
+ If <e_type>.family does not provide additional data,
+ data should be set to NULL.
+ If <e_type>.family does provide additional data, data should be set
+ using EVENT_HDL_CB_DATA macro.
+ (see the example below)
+
+The function returns 1 in case of SUCCESS (handlers successfully notified)
+and 0 in case of FAILURE (no handlers notified, because of memory error).
+
+Event publishing can be performed from anywhere in the code.
+(this example does not compile)
+```
+ struct event_hdl_cb_data_new_family event_data;
+
+ /* first we need to prepare event data
+ * that will be provided to event handlers
+ */
+
+ /* safe data, available from both sync and async contexts */
+ event_data.safe.my_custom_data = x;
+
+ /* unsafe data, only available from sync contexts */
+ event_data.unsafe.my_unsafe_data = y;
+
+ /* once data is prepared, we can publish the event */
+ event_hdl_publish(NULL,
+ EVENT_HDL_SUB_NEW_FAMILY_SUBTYPE_1,
+ EVENT_HDL_CB_DATA(&event_data));
+
+ /* EVENT_HDL_SUB_NEW_FAMILY_SUBTYPE_1 event was
+ * successfully published in global subscription list
+ */
+```
+
+--------------------------------------------------------------------------------
+|You should know that there is currently a limitation about publish function: |
+|The function should not be used from critical places |
+|(where the calling frequency is high |
+|or where timing sensitivity is high). |
+| |
+|Because in current implementation, subscription list lookups are not |
+|optimized for such uses cases. |
+--------------------------------------------------------------------------------
+
+4 SUBSCRIPTION LISTS
+-----------------------
+
+As you may already know, EVENT_HDL API main functions rely on
+subscription lists.
+Providing NULL where subscription list argument is required
+allows to use the implicit global subscription list.
+
+But you can also provide a specific subscription list, example:
+ subscription list associated with a single entity so that you only
+ subscribe to events of this single entity
+
+A subscription list is of type event_hdl_sub_list.
+It is defined in event_hdl-t.h
+
+To make use of this feature, you should know about these 2 functions:
+
+event_hdl_sub_list_init(list): use this fcn to initialize
+ a new subscription list.
+
+Example:
+```
+ event_hdl_sub_list my_custom_list;
+
+ event_hdl_sub_list_init(&my_custom_list);
+```
+
+event_hdl_sub_list_destroy(list): use this fcn to destroy
+ an existing subscription list.
+
+Example:
+```
+ event_hdl_sub_list_init(&my_custom_list);
+```
+
+ Using this function will cause all the existing subscriptions
+ within the provided sub_list to be properly unregistered
+ and deleted according to their types.
+
+Now we'll take another quick look at event_hdl_publish() function:
+
+Remember that the function is defined as follow:
+```
+ int event_hdl_publish(event_hdl_sub_list *sub_list,
+ event_hdl_sub_type e_type,
+ const struct event_hdl_cb_data *data);
+```
+
+In the previous examples, we used sub_list = NULL.
+
+if sub_list is NULL:
+ event will be published in in global list
+else
+ event will be published in user specified sub_list
+
+5 MISC/HELPER FUNCTIONS
+-----------------------
+
+Don't forget to take a look at MISC/HELPER FUNCTIONS in
+include/haproxy/event_hdl.h (end of the file) for a
+complete list of helper functions / macros.
+
+We've already used some, if not the vast majority
+in the examples shown in this document.
+
+This includes, to name a few:
+ - event types manipulation
+ - event types comparison
+ - lookup id computing
+ - subscriber list management (covered in section 4)
+ - sync/async handler helpers
diff --git a/doc/internals/api/filters.txt b/doc/internals/api/filters.txt
new file mode 100644
index 0000000..f1d2f34
--- /dev/null
+++ b/doc/internals/api/filters.txt
@@ -0,0 +1,1188 @@
+ -----------------------------------------
+ Filters Guide - version 2.9
+ ( Last update: 2021-02-24 )
+ ------------------------------------------
+ Author : Christopher Faulet
+ Contact : christopher dot faulet at capflam dot org
+
+
+ABSTRACT
+--------
+
+The filters support is a new feature of HAProxy 1.7. It is a way to extend
+HAProxy without touching its core code and, in certain extent, without knowing
+its internals. This feature will ease contributions, reducing impact of
+changes. Another advantage will be to simplify HAProxy by replacing some parts
+by filters. As we will see, and as an example, the HTTP compression is the first
+feature moved in a filter.
+
+This document describes how to write a filter and what to keep in mind to do
+so. It also talks about the known limits and the pitfalls to avoid.
+
+As said, filters are quite new for now. The API is not freezed and will be
+updated/modified/improved/extended as needed.
+
+
+
+SUMMARY
+-------
+
+ 1. Filters introduction
+ 2. How to use filters
+ 3. How to write a new filter
+ 3.1. API Overview
+ 3.2. Defining the filter name and its configuration
+ 3.3. Managing the filter lifecycle
+ 3.3.1. Dealing with threads
+ 3.4. Handling the streams activity
+ 3.5. Analyzing the channels activity
+ 3.6. Filtering the data exchanged
+ 4. FAQ
+
+
+
+1. FILTERS INTRODUCTION
+-----------------------
+
+First of all, to fully understand how filters work and how to create one, it is
+best to know, at least from a distance, what is a proxy (frontend/backend), a
+stream and a channel in HAProxy and how these entities are linked to each other.
+In doc/internals/api/layers.txt is a good overview of the different layers in
+HAProxy and in doc/internals/muxes.pdf is described the flow between the
+different muxes.
+
+Then, to support filters, many callbacks has been added to HAProxy at different
+places, mainly around channel analyzers. Their purpose is to allow filters to
+be involved in the data processing, from the stream creation/destruction to
+the data forwarding. Depending of what it should do, a filter can implement all
+or part of these callbacks. For now, existing callbacks are focused on
+streams. But future improvements could enlarge filters scope. For instance, it
+could be useful to handle events at the connection level.
+
+In HAProxy configuration file, a filter is declared in a proxy section, except
+default. So the configuration corresponding to a filter declaration is attached
+to a specific proxy, and will be shared by all its instances. it is opaque from
+the HAProxy point of view, this is the filter responsibility to manage it. For
+each filter declaration matches a uniq configuration. Several declarations of
+the same filter in the same proxy will be handle as different filters by
+HAProxy.
+
+A filter instance is represented by a partially opaque context (or a state)
+attached to a stream and passed as arguments to callbacks. Through this context,
+filter instances are stateful. Depending the filter is declared in a frontend or
+a backend section, its instances will be created, respectively, when a stream is
+created or when a backend is selected. Their behaviors will also be
+different. Only instances of filters declared in a frontend section will be
+aware of the creation and the destruction of the stream, and will take part in
+the channels analyzing before the backend is defined.
+
+It is important to remember the configuration of a filter is shared by all its
+instances, while the context of an instance is owned by a uniq stream.
+
+Filters are designed to be chained. It is possible to declare several filters in
+the same proxy section. The declaration order is important because filters will
+be called one after the other respecting this order. Frontend and backend
+filters are also chained, frontend ones called first. Even if the filters
+processing is serialized, each filter will bahave as it was alone (unless it was
+developed to be aware of other filters). For all that, some constraints are
+imposed to filters, especially when data exchanged between the client and the
+server are processed. We will discuss again these constraints when we will tackle
+the subject of writing a filter.
+
+
+
+2. HOW TO USE FILTERS
+---------------------
+
+To use a filter, the parameter 'filter' should be used, followed by the filter
+name and, optionally, its configuration in the desired listen, frontend or
+backend section. For instance :
+
+ listen test
+ ...
+ filter trace name TST
+ ...
+
+
+See doc/configuration.txt for a formal definition of the parameter 'filter'.
+Note that additional parameters on the filter line must be parsed by the filter
+itself.
+
+The list of available filters is reported by 'haproxy -vv' :
+
+ $> haproxy -vv
+ HAProxy version 1.7-dev2-3a1d4a-33 2016/03/21
+ Copyright 2000-2016 Willy Tarreau <willy@haproxy.org>
+
+ [...]
+
+ Available filters :
+ [COMP] compression
+ [TRACE] trace
+
+
+Multiple filter lines can be used in a proxy section to chain filters. Filters
+will be called in the declaration order.
+
+Some filters can support implicit declarations in certain circumstances
+(without the filter line). This is not recommended for new features but are
+useful for existing ones moved in a filter, for backward compatibility
+reasons. Implicit declarations are supported when there is only one filter used
+on a proxy. When several filters are used, explicit declarations are mandatory.
+The HTTP compression filter is one of these filters. Alone, using 'compression'
+keywords is enough to use it. But when at least a second filter is used, a
+filter line must be added.
+
+ # filter line is optional
+ listen t1
+ bind *:80
+ compression algo gzip
+ compression offload
+ server srv x.x.x.x:80
+
+ # filter line is mandatory for the compression filter
+ listen t2
+ bind *:81
+ filter trace name T2
+ filter compression
+ compression algo gzip
+ compression offload
+ server srv x.x.x.x:80
+
+
+
+
+3. HOW TO WRITE A NEW FILTER
+----------------------------
+
+To write a filter, there are 2 header files to explore :
+
+ * include/haproxy/filters-t.h : This is the main header file, containing all
+ important structures to use. It represents the
+ filter API.
+
+ * include/haproxy/filters.h : This header file contains helper functions that
+ may be used. It also contains the internal API
+ used by HAProxy to handle filters.
+
+To ease the filters integration, it is better to follow some conventions :
+
+ * Use 'flt_' prefix to name the filter (e.g flt_http_comp or flt_trace).
+
+ * Keep everything related to the filter in a same file.
+
+The filter 'trace' can be used as a template to write new filter. It is a good
+start to see how filters really work.
+
+3.1 API OVERVIEW
+----------------
+
+Writing a filter can be summarized to write functions and attach them to the
+existing callbacks. Available callbacks are listed in the following structure :
+
+ struct flt_ops {
+ /*
+ * Callbacks to manage the filter lifecycle
+ */
+ int (*init) (struct proxy *p, struct flt_conf *fconf);
+ void (*deinit) (struct proxy *p, struct flt_conf *fconf);
+ int (*check) (struct proxy *p, struct flt_conf *fconf);
+ int (*init_per_thread) (struct proxy *p, struct flt_conf *fconf);
+ void (*deinit_per_thread)(struct proxy *p, struct flt_conf *fconf);
+
+ /*
+ * Stream callbacks
+ */
+ int (*attach) (struct stream *s, struct filter *f);
+ int (*stream_start) (struct stream *s, struct filter *f);
+ int (*stream_set_backend)(struct stream *s, struct filter *f, struct proxy *be);
+ void (*stream_stop) (struct stream *s, struct filter *f);
+ void (*detach) (struct stream *s, struct filter *f);
+ void (*check_timeouts) (struct stream *s, struct filter *f);
+
+ /*
+ * Channel callbacks
+ */
+ int (*channel_start_analyze)(struct stream *s, struct filter *f,
+ struct channel *chn);
+ int (*channel_pre_analyze) (struct stream *s, struct filter *f,
+ struct channel *chn,
+ unsigned int an_bit);
+ int (*channel_post_analyze) (struct stream *s, struct filter *f,
+ struct channel *chn,
+ unsigned int an_bit);
+ int (*channel_end_analyze) (struct stream *s, struct filter *f,
+ struct channel *chn);
+
+ /*
+ * HTTP callbacks
+ */
+ int (*http_headers) (struct stream *s, struct filter *f,
+ struct http_msg *msg);
+ int (*http_payload) (struct stream *s, struct filter *f,
+ struct http_msg *msg, unsigned int offset,
+ unsigned int len);
+ int (*http_end) (struct stream *s, struct filter *f,
+ struct http_msg *msg);
+
+ void (*http_reset) (struct stream *s, struct filter *f,
+ struct http_msg *msg);
+ void (*http_reply) (struct stream *s, struct filter *f,
+ short status,
+ const struct buffer *msg);
+
+ /*
+ * TCP callbacks
+ */
+ int (*tcp_payload) (struct stream *s, struct filter *f,
+ struct channel *chn, unsigned int offset,
+ unsigned int len);
+ };
+
+
+We will explain in following parts when these callbacks are called and what they
+should do.
+
+Filters are declared in proxy sections. So each proxy have an ordered list of
+filters, possibly empty if no filter is used. When the configuration of a proxy
+is parsed, each filter line represents an entry in this list. In the structure
+'proxy', the filters configurations are stored in the field 'filter_configs',
+each one of type 'struct flt_conf *' :
+
+ /*
+ * Structure representing the filter configuration, attached to a proxy and
+ * accessible from a filter when instantiated in a stream
+ */
+ struct flt_conf {
+ const char *id; /* The filter id */
+ struct flt_ops *ops; /* The filter callbacks */
+ void *conf; /* The filter configuration */
+ struct list list; /* Next filter for the same proxy */
+ unsigned int flags; /* FLT_CFG_FL_* */
+ };
+
+ * 'flt_conf.id' is an identifier, defined by the filter. It can be
+ NULL. HAProxy does not use this field. Filters can use it in log messages or
+ as a uniq identifier to check multiple declarations. It is the filter
+ responsibility to free it, if necessary.
+
+ * 'flt_conf.conf' is opaque. It is the internal configuration of a filter,
+ generally allocated and filled by its parsing function (See § 3.2). It is
+ the filter responsibility to free it.
+
+ * 'flt_conf.ops' references the callbacks implemented by the filter. This
+ field must be set during the parsing phase (See § 3.2) and can be refine
+ during the initialization phase (See § 3.3). If it is dynamically allocated,
+ it is the filter responsibility to free it.
+
+ * 'flt_conf.flags' is a bitfield to specify the filter capabilities. For now,
+ only FLT_CFG_FL_HTX may be set when a filter is able to process HTX
+ streams. If not set, the filter is excluded from the HTTP filtering.
+
+
+The filter configuration is global and shared by all its instances. A filter
+instance is created in the context of a stream and attached to this stream. in
+the structure 'stream', the field 'strm_flt' is the state of all filter
+instances attached to a stream :
+
+ /*
+ * Structure representing the "global" state of filters attached to a
+ * stream.
+ */
+ struct strm_flt {
+ struct list filters; /* List of filters attached to a stream */
+ struct filter *current[2]; /* From which filter resume processing, for a specific channel.
+ * This is used for resumable callbacks only,
+ * If NULL, we start from the first filter.
+ * 0: request channel, 1: response channel */
+ unsigned short flags; /* STRM_FL_* */
+ unsigned char nb_req_data_filters; /* Number of data filters registered on the request channel */
+ unsigned char nb_rsp_data_filters; /* Number of data filters registered on the response channel */
+ unsigned long long offset[2]; /* gloal offset of input data already filtered for a specific channel
+ * 0: request channel, 1: response channel */
+ };
+
+
+Filter instances attached to a stream are stored in the field
+'strm_flt.filters', each instance is of type 'struct filter *' :
+
+ /*
+ * Structure representing a filter instance attached to a stream
+ *
+ * 2D-Array fields are used to store info per channel. The first index
+ * stands for the request channel, and the second one for the response
+ * channel. Especially, <next> and <fwd> are offsets representing amount of
+ * data that the filter are, respectively, parsed and forwarded on a
+ * channel. Filters can access these values using FLT_NXT and FLT_FWD
+ * macros.
+ */
+ struct filter {
+ struct flt_conf *config; /* the filter's configuration */
+ void *ctx; /* The filter context (opaque) */
+ unsigned short flags; /* FLT_FL_* */
+ unsigned long long offset[2]; /* Offset of input data already filtered for a specific channel
+ * 0: request channel, 1: response channel */
+ unsigned int pre_analyzers; /* bit field indicating analyzers to
+ * pre-process */
+ unsigned int post_analyzers; /* bit field indicating analyzers to
+ * post-process */
+ struct list list; /* Next filter for the same proxy/stream */
+ };
+
+ * 'filter.config' is the filter configuration previously described. All
+ instances of a filter share it.
+
+ * 'filter.ctx' is an opaque context. It is managed by the filter, so it is its
+ responsibility to free it.
+
+ * 'filter.pre_analyzers and 'filter.post_analyzers will be described later
+ (See § 3.5).
+
+ * 'filter.offset' will be described later (See § 3.6).
+
+
+3.2. DEFINING THE FILTER NAME AND ITS CONFIGURATION
+---------------------------------------------------
+
+During the filter development, the first thing to do is to add it in the
+supported filters. To do so, its name must be registered as a valid keyword on
+the filter line :
+
+ /* Declare the filter parser for "my_filter" keyword */
+ static struct flt_kw_list flt_kws = { "MY_FILTER_SCOPE", { }, {
+ { "my_filter", parse_my_filter_cfg, NULL /* private data */ },
+ { NULL, NULL, NULL },
+ }
+ };
+ INITCALL1(STG_REGISTER, flt_register_keywords, &flt_kws);
+
+
+Then the filter internal configuration must be defined. For instance :
+
+ struct my_filter_config {
+ struct proxy *proxy;
+ char *name;
+ /* ... */
+ };
+
+
+All callbacks implemented by the filter must then be declared. Here, a global
+variable is used :
+
+ struct flt_ops my_filter_ops {
+ .init = my_filter_init,
+ .deinit = my_filter_deinit,
+ .check = my_filter_config_check,
+
+ /* ... */
+ };
+
+
+Finally, the function to parse the filter configuration must be written, here
+'parse_my_filter_cfg'. This function must parse all remaining keywords on the
+filter line :
+
+ /* Return -1 on error, else 0 */
+ static int
+ parse_my_filter_cfg(char **args, int *cur_arg, struct proxy *px,
+ struct flt_conf *flt_conf, char **err, void *private)
+ {
+ struct my_filter_config *my_conf;
+ int pos = *cur_arg;
+
+ /* Allocate the internal configuration used by the filter */
+ my_conf = calloc(1, sizeof(*my_conf));
+ if (!my_conf) {
+ memprintf(err, "%s : out of memory", args[*cur_arg]);
+ return -1;
+ }
+ my_conf->proxy = px;
+
+ /* ... */
+
+ /* Parse all keywords supported by the filter and fill the internal
+ * configuration */
+ pos++; /* Skip the filter name */
+ while (*args[pos]) {
+ if (!strcmp(args[pos], "name")) {
+ if (!*args[pos + 1]) {
+ memprintf(err, "'%s' : '%s' option without value",
+ args[*cur_arg], args[pos]);
+ goto error;
+ }
+ my_conf->name = strdup(args[pos + 1]);
+ if (!my_conf->name) {
+ memprintf(err, "%s : out of memory", args[*cur_arg]);
+ goto error;
+ }
+ pos += 2;
+ }
+
+ /* ... parse other keywords ... */
+ }
+ *cur_arg = pos;
+
+ /* Set callbacks supported by the filter */
+ flt_conf->ops = &my_filter_ops;
+
+ /* Last, save the internal configuration */
+ flt_conf->conf = my_conf;
+ return 0;
+
+ error:
+ if (my_conf->name)
+ free(my_conf->name);
+ free(my_conf);
+ return -1;
+ }
+
+
+WARNING : In this parsing function, 'flt_conf->ops' must be initialized. All
+ arguments of the filter line must also be parsed. This is mandatory.
+
+In the previous example, the filter lne should be read as follows :
+
+ filter my_filter name MY_NAME ...
+
+
+Optionally, by implementing the 'flt_ops.check' callback, an extra set is added
+to check the internal configuration of the filter after the parsing phase, when
+the HAProxy configuration is fully defined. For instance :
+
+ /* Check configuration of a trace filter for a specified proxy.
+ * Return 1 on error, else 0. */
+ static int
+ my_filter_config_check(struct proxy *px, struct flt_conf *my_conf)
+ {
+ if (px->mode != PR_MODE_HTTP) {
+ Alert("The filter 'my_filter' cannot be used in non-HTTP mode.\n");
+ return 1;
+ }
+
+ /* ... */
+
+ return 0;
+ }
+
+
+
+3.3. MANAGING THE FILTER LIFECYCLE
+----------------------------------
+
+Once the configuration parsed and checked, filters are ready to by used. There
+are two main callbacks to manage the filter lifecycle :
+
+ * 'flt_ops.init' : It initializes the filter for a proxy. This callback may be
+ defined to finish the filter configuration.
+
+ * 'flt_ops.deinit' : It cleans up what the parsing function and the init
+ callback have done. This callback is useful to release
+ memory allocated for the filter configuration.
+
+Here is an example :
+
+ /* Initialize the filter. Returns -1 on error, else 0. */
+ static int
+ my_filter_init(struct proxy *px, struct flt_conf *fconf)
+ {
+ struct my_filter_config *my_conf = fconf->conf;
+
+ /* ... */
+
+ return 0;
+ }
+
+ /* Free resources allocated by the trace filter. */
+ static void
+ my_filter_deinit(struct proxy *px, struct flt_conf *fconf)
+ {
+ struct my_filter_config *my_conf = fconf->conf;
+
+ if (my_conf) {
+ free(my_conf->name);
+ /* ... */
+ free(my_conf);
+ }
+ fconf->conf = NULL;
+ }
+
+
+3.3.1 DEALING WITH THREADS
+--------------------------
+
+When HAProxy is compiled with the threads support and started with more that one
+thread (global.nbthread > 1), then it is possible to manage the filter per
+thread with following callbacks :
+
+ * 'flt_ops.init_per_thread': It initializes the filter for each thread. It
+ works the same way than 'flt_ops.init' but in the
+ context of a thread. This callback is called
+ after the thread creation.
+
+ * 'flt_ops.deinit_per_thread': It cleans up what the init_per_thread callback
+ have done. It is called in the context of a
+ thread, before exiting it.
+
+It is the filter responsibility to deal with concurrency. check, init and deinit
+callbacks are called on the main thread. All others are called on a "worker"
+thread (not always the same). It is also the filter responsibility to know if
+HAProxy is started with more than one thread. If it is started with one thread
+(or compiled without the threads support), these callbacks will be silently
+ignored (in this case, global.nbthread will be always equal to one).
+
+
+3.4. HANDLING THE STREAMS ACTIVITY
+-----------------------------------
+
+It may be interesting to handle streams activity. For now, there is three
+callbacks that should define to do so :
+
+ * 'flt_ops.stream_start' : It is called when a stream is started. This
+ callback can fail by returning a negative value. It
+ will be considered as a critical error by HAProxy
+ which disabled the listener for a short time.
+
+ * 'flt_ops.stream_set_backend' : It is called when a backend is set for a
+ stream. This callbacks will be called for all
+ filters attached to a stream (frontend and
+ backend). Note this callback is not called if
+ the frontend and the backend are the same.
+
+ * 'flt_ops.stream_stop' : It is called when a stream is stopped. This callback
+ always succeed. Anyway, it is too late to return an
+ error.
+
+For instance :
+
+ /* Called when a stream is created. Returns -1 on error, else 0. */
+ static int
+ my_filter_stream_start(struct stream *s, struct filter *filter)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+
+ /* ... */
+
+ return 0;
+ }
+
+ /* Called when a backend is set for a stream */
+ static int
+ my_filter_stream_set_backend(struct stream *s, struct filter *filter,
+ struct proxy *be)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+
+ /* ... */
+
+ return 0;
+ }
+
+ /* Called when a stream is destroyed */
+ static void
+ my_filter_stream_stop(struct stream *s, struct filter *filter)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+
+ /* ... */
+ }
+
+
+WARNING : Handling the streams creation and destruction is only possible for
+ filters defined on proxies with the frontend capability.
+
+In addition, it is possible to handle creation and destruction of filter
+instances using following callbacks:
+
+ * 'flt_ops.attach' : It is called after a filter instance creation, when it is
+ attached to a stream. This happens when the stream is
+ started for filters defined on the stream's frontend and
+ when the backend is set for filters declared on the
+ stream's backend. It is possible to ignore the filter, if
+ needed, by returning 0. This could be useful to have
+ conditional filtering.
+
+ * 'flt_ops.detach' : It is called when a filter instance is detached from a
+ stream, before its destruction. This happens when the
+ stream is stopped for filters defined on the stream's
+ frontend and when the analyze ends for filters defined on
+ the stream's backend.
+
+For instance :
+
+ /* Called when a filter instance is created and attach to a stream */
+ static int
+ my_filter_attach(struct stream *s, struct filter *filter)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+
+ if (/* ... */)
+ return 0; /* Ignore the filter here */
+ return 1;
+ }
+
+ /* Called when a filter instance is detach from a stream, just before its
+ * destruction */
+ static void
+ my_filter_detach(struct stream *s, struct filter *filter)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+
+ /* ... */
+ }
+
+Finally, it may be interesting to notify the filter when the stream is woken up
+because of an expired timer. This could let a chance to check some internal
+timeouts, if any. To do so the following callback must be used :
+
+ * 'flt_opt.check_timeouts' : It is called when a stream is woken up because of
+ an expired timer.
+
+For instance :
+
+ /* Called when a stream is woken up because of an expired timer */
+ static void
+ my_filter_check_timeouts(struct stream *s, struct filter *filter)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+
+ /* ... */
+ }
+
+
+3.5. ANALYZING THE CHANNELS ACTIVITY
+------------------------------------
+
+The main purpose of filters is to take part in the channels analyzing. To do so,
+there is 2 callbacks, 'flt_ops.channel_pre_analyze' and
+'flt_ops.channel_post_analyze', called respectively before and after each
+analyzer attached to a channel, except analyzers responsible for the data
+forwarding (TCP or HTTP). Concretely, on the request channel, these callbacks
+could be called before following analyzers :
+
+ * tcp_inspect_request (AN_REQ_INSPECT_FE and AN_REQ_INSPECT_BE)
+ * http_wait_for_request (AN_REQ_WAIT_HTTP)
+ * http_wait_for_request_body (AN_REQ_HTTP_BODY)
+ * http_process_req_common (AN_REQ_HTTP_PROCESS_FE)
+ * process_switching_rules (AN_REQ_SWITCHING_RULES)
+ * http_process_req_ common (AN_REQ_HTTP_PROCESS_BE)
+ * http_process_tarpit (AN_REQ_HTTP_TARPIT)
+ * process_server_rules (AN_REQ_SRV_RULES)
+ * http_process_request (AN_REQ_HTTP_INNER)
+ * tcp_persist_rdp_cookie (AN_REQ_PRST_RDP_COOKIE)
+ * process_sticking_rules (AN_REQ_STICKING_RULES)
+
+And on the response channel :
+
+ * tcp_inspect_response (AN_RES_INSPECT)
+ * http_wait_for_response (AN_RES_WAIT_HTTP)
+ * process_store_rules (AN_RES_STORE_RULES)
+ * http_process_res_common (AN_RES_HTTP_PROCESS_BE)
+
+Unlike the other callbacks previously seen before, 'flt_ops.channel_pre_analyze'
+can interrupt the stream processing. So a filter can decide to not execute the
+analyzer that follows and wait the next iteration. If there are more than one
+filter, following ones are skipped. On the next iteration, the filtering resumes
+where it was stopped, i.e. on the filter that has previously stopped the
+processing. So it is possible for a filter to stop the stream processing on a
+specific analyzer for a while before continuing. Moreover, this callback can be
+called many times for the same analyzer, until it finishes its processing. For
+instance :
+
+ /* Called before a processing happens on a given channel.
+ * Returns a negative value if an error occurs, 0 if it needs to wait,
+ * any other value otherwise. */
+ static int
+ my_filter_chn_pre_analyze(struct stream *s, struct filter *filter,
+ struct channel *chn, unsigned an_bit)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+
+ switch (an_bit) {
+ case AN_REQ_WAIT_HTTP:
+ if (/* wait that a condition is verified before continuing */)
+ return 0;
+ break;
+ /* ... * /
+ }
+ return 1;
+ }
+
+ * 'an_bit' is the analyzer id. All analyzers are listed in
+ 'include/haproxy/channels-t.h'.
+
+ * 'chn' is the channel on which the analyzing is done. It is possible to
+ determine if it is the request or the response channel by testing if
+ CF_ISRESP flag is set :
+
+ │ ((chn->flags & CF_ISRESP) == CF_ISRESP)
+
+
+In previous example, the stream processing is blocked before receipt of the HTTP
+request until a condition is verified.
+
+'flt_ops.channel_post_analyze', for its part, is not resumable. It returns a
+negative value if an error occurs, any other value otherwise. It is called when
+a filterable analyzer finishes its processing, so once for the same analyzer.
+For instance :
+
+ /* Called after a processing happens on a given channel.
+ * Returns a negative value if an error occurs, any other
+ * value otherwise. */
+ static int
+ my_filter_chn_post_analyze(struct stream *s, struct filter *filter,
+ struct channel *chn, unsigned an_bit)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+ struct http_msg *msg;
+
+ switch (an_bit) {
+ case AN_REQ_WAIT_HTTP:
+ if (/* A test on received headers before any other treatment */) {
+ msg = ((chn->flags & CF_ISRESP) ? &s->txn->rsp : &s->txn->req);
+ txn->status = 400;
+ msg->msg_state = HTTP_MSG_ERROR;
+ http_reply_and_close(s, s->txn->status, http_error_message(s));
+ return -1; /* This is an error ! */
+ }
+ break;
+ /* ... * /
+ }
+ return 1;
+ }
+
+
+Pre and post analyzer callbacks of a filter are not automatically called. They
+must be regiesterd explicitly on analyzers, updating the value of
+'filter.pre_analyzers' and 'filter.post_analyzers' bit fields. All analyzer bits
+are listed in 'include/types/channels.h'. Here is an example :
+
+ static int
+ my_filter_stream_start(struct stream *s, struct filter *filter)
+ {
+ /* ... * /
+
+ /* Register the pre analyzer callback on all request and response
+ * analyzers */
+ filter->pre_analyzers |= (AN_REQ_ALL | AN_RES_ALL)
+
+ /* Register the post analyzer callback of only on AN_REQ_WAIT_HTTP and
+ * AN_RES_WAIT_HTTP analyzers */
+ filter->post_analyzers |= (AN_REQ_WAIT_HTTP | AN_RES_WAIT_HTTP)
+
+ /* ... * /
+ return 0;
+ }
+
+
+To surround activity of a filter during the channel analyzing, two new analyzers
+has been added :
+
+ * 'flt_start_analyze' (AN_REQ/RES_FLT_START_FE/AN_REQ_RES_FLT_START_BE) : For
+ a specific filter, this analyzer is called before any call to the
+ 'channel_analyze' callback. From the filter point of view, it calls the
+ 'flt_ops.channel_start_analyze' callback.
+
+ * 'flt_end_analyze' (AN_REQ/RES_FLT_END) : For a specific filter, this
+ analyzer is called when all other analyzers have finished their
+ processing. From the filter point of view, it calls the
+ 'flt_ops.channel_end_analyze' callback.
+
+These analyzers are called only once per streams.
+
+'flt_ops.channel_start_analyze' and 'flt_ops.channel_end_analyze' callbacks can
+interrupt the stream processing, as 'flt_ops.channel_analyze'. Here is an
+example :
+
+ /* Called when analyze starts for a given channel
+ * Returns a negative value if an error occurs, 0 if it needs to wait,
+ * any other value otherwise. */
+ static int
+ my_filter_chn_start_analyze(struct stream *s, struct filter *filter,
+ struct channel *chn)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+
+ /* ... TODO ... */
+
+ return 1;
+ }
+
+ /* Called when analyze ends for a given channel
+ * Returns a negative value if an error occurs, 0 if it needs to wait,
+ * any other value otherwise. */
+ static int
+ my_filter_chn_end_analyze(struct stream *s, struct filter *filter,
+ struct channel *chn)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+
+ /* ... TODO ... */
+
+ return 1;
+ }
+
+
+Workflow on channels can be summarized as following :
+
+ FE: Called for filters defined on the stream's frontend
+ BE: Called for filters defined on the stream's backend
+
+ +------->---------+
+ | | |
+ +----------------------+ | +----------------------+
+ | flt_ops.attach (FE) | | | flt_ops.attach (BE) |
+ +----------------------+ | +----------------------+
+ | | |
+ V | V
+ +--------------------------+ | +------------------------------------+
+ | flt_ops.stream_start (FE)| | | flt_ops.stream_set_backend (FE+BE) |
+ +--------------------------+ | +------------------------------------+
+ | | |
+ ... | ...
+ | | |
+ | ^ |
+ | --+ | | --+
+ +------<----------+ | | +--------<--------+ |
+ | | | | | | |
+ V | | | V | |
++-------------------------------+ | | | +-------------------------------+ | |
+| flt_start_analyze (FE) +-+ | | | flt_start_analyze (BE) +-+ |
+|(flt_ops.channel_start_analyze)| | F | |(flt_ops.channel_start_analyze)| |
++---------------+---------------+ | R | +-------------------------------+ |
+ | | O | | |
+ +------<---------+ | N ^ +--------<-------+ | B
+ | | | T | | | | A
++---------------|------------+ | | E | +---------------|------------+ | | C
+|+--------------V-------------+ | | N | |+--------------V-------------+ | | K
+||+----------------------------+ | | D | ||+----------------------------+ | | E
+|||flt_ops.channel_pre_analyze | | | | |||flt_ops.channel_pre_analyze | | | N
+||| V | | | | ||| V | | | D
+||| analyzer (FE) +-+ | | ||| analyzer (FE+BE) +-+ |
++|| V | | | +|| V | |
+ +|flt_ops.channel_post_analyze| | | +|flt_ops.channel_post_analyze| |
+ +----------------------------+ | | +----------------------------+ |
+ | --+ | | |
+ +------------>------------+ ... |
+ | |
+ [ data filtering (see below) ] |
+ | |
+ ... |
+ | |
+ +--------<--------+ |
+ | | |
+ V | |
+ +-------------------------------+ | |
+ | flt_end_analyze (FE+BE) +-+ |
+ | (flt_ops.channel_end_analyze) | |
+ +---------------+---------------+ |
+ | --+
+ V
+ +----------------------+
+ | flt_ops.detach (BE) |
+ +----------------------+
+ |
+ V
+ +--------------------------+
+ | flt_ops.stream_stop (FE) |
+ +--------------------------+
+ |
+ V
+ +----------------------+
+ | flt_ops.detach (FE) |
+ +----------------------+
+ |
+ V
+
+By zooming on an analyzer box we have:
+
+ ...
+ |
+ V
+ |
+ +-----------<-----------+
+ | |
+ +-----------------+--------------------+ |
+ | | | |
+ | +--------<---------+ | |
+ | | | | |
+ | V | | |
+ | flt_ops.channel_pre_analyze ->-+ | ^
+ | | | |
+ | | | |
+ | V | |
+ | analyzer --------->-----+--+
+ | | |
+ | | |
+ | V |
+ | flt_ops.channel_post_analyze |
+ | | |
+ | | |
+ +-----------------+--------------------+
+ |
+ V
+ ...
+
+
+ 3.6. FILTERING THE DATA EXCHANGED
+-----------------------------------
+
+WARNING : To fully understand this part, it is important to be aware on how the
+ buffers work in HAProxy. For the HTTP part, it is also important to
+ understand how data are parsed and structured, and how the internal
+ representation, called HTX, works. See doc/internals/buffer-api.txt
+ and doc/internals/htx-api.txt for details.
+
+An extended feature of the filters is the data filtering. By default a filter
+does not look into data exchanged between the client and the server because it
+is expensive. Indeed, instead of forwarding data without any processing, each
+byte need to be buffered.
+
+So, to enable the data filtering on a channel, at any time, in one of previous
+callbacks, 'register_data_filter' function must be called. And conversely, to
+disable it, 'unregister_data_filter' function must be called. For instance :
+
+ my_filter_http_headers(struct stream *s, struct filter *filter,
+ struct http_msg *msg)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+
+ /* 'chn' must be the request channel */
+ if (!(msg->chn->flags & CF_ISRESP)) {
+ struct htx *htx;
+ struct ist hdr;
+ struct http_hdr_ctx ctx;
+
+ htx = htxbuf(msg->chn->buf);
+
+ /* Enable the data filtering for the request if 'X-Filter' header
+ * is set to 'true'. */
+ hdr = ist("X-Filter);
+ ctx.blk = NULL;
+ if (http_find_header(htx, hdr, &ctx, 0) &&
+ ctx.value.len >= 4 && memcmp(ctx.value.ptr, "true", 4) == 0)
+ register_data_filter(s, chn, filter);
+ }
+
+ return 1;
+ }
+
+Here, the data filtering is enabled if the HTTP header 'X-Filter' is found and
+set to 'true'.
+
+If several filters are declared, the evaluation order remains the same,
+regardless the order of the registrations to the data filtering. Data
+registrations must be performed before the data forwarding step. However, a
+filter may be unregistered from the data filtering at any time.
+
+Depending on the stream type, TCP or HTTP, the way to handle data filtering is
+different. HTTP data are structured while TCP data are raw. And there are more
+callbacks for HTTP streams to fully handle all steps of an HTTP transaction. But
+the main part is the same. The data filtering is performed in one callback,
+called in loop on input data starting at a specific offset for a given
+length. Data analyzed by a filter are considered as forwarded from its point of
+view. Because filters are chained, a filter never analyzes more data than its
+predecessors. Thus only data analyzed by the last filter are effectively
+forwarded. This means, at any time, any filter may choose to not analyze all
+available data (available from its point of view), blocking the data forwarding.
+
+Internally, filters own 2 offsets representing the number of bytes already
+analyzed in the available input data, one per channel. There is also an offset
+couple at the stream level, in the strm_flt object, representing the total
+number of bytes already forwarded. These offsets may be retrieved and updated
+using following macros :
+
+ * FLT_OFF(flt, chn)
+
+ * FLT_STRM_OFF(s, chn)
+
+where 'flt' is the 'struct filter' passed as argument in all callbacks, 's' the
+filtered stream and 'chn' is the considered channel. However, there is no reason
+for a filter to use these macros or take care of these offsets.
+
+
+3.6.1 FILTERING DATA ON TCP STREAMS
+-----------------------------------
+
+The TCP data filtering for TCP streams is the easy case, because HAProxy do not
+parse these data. Data are stored in raw in the buffer. So there is only one
+callback to consider:
+
+ * 'flt_ops.tcp_payload : This callback is called when input data are
+ available. If not defined, all available data will be considered as analyzed
+ and forwarded from the filter point of view.
+
+This callback is called only if the filter is registered to analyze TCP
+data. Here is an example :
+
+ /* Returns a negative value if an error occurs, else the number of
+ * consumed bytes. */
+ static int
+ my_filter_tcp_payload(struct stream *s, struct filter *filter,
+ struct channel *chn, unsigned int offset,
+ unsigned int len)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+ int ret = len;
+
+ /* Do not parse more than 'my_conf->max_parse' bytes at a time */
+ if (my_conf->max_parse != 0 && ret > my_conf->max_parse)
+ ret = my_conf->max_parse;
+
+ /* if available data are not completely parsed, wake up the stream to
+ * be sure to not freeze it. The best is probably to set a
+ * chn->analyse_exp timer */
+ if (ret != len)
+ task_wakeup(s->task, TASK_WOKEN_MSG);
+ return ret;
+ }
+
+But it is important to note that tunnelled data of an HTTP stream may also be
+filtered via this callback. Tunnelled data are data exchange after an HTTP tunnel
+is established between the client and the server, via an HTTP CONNECT or via a
+protocol upgrade. In this case, the data are structured. Of course, to do so,
+the filter must be able to parse HTX data and must have the FLT_CFG_FL_HTX flag
+set. At any time, the IS_HTX_STRM() macros may be used on the stream to know if
+it is an HTX stream or a TCP stream.
+
+
+3.6.2 FILTERING DATA ON HTTP STREAMS
+------------------------------------
+
+The HTTP data filtering is a bit more complex because HAProxy data are
+structutred and represented to an internal format, called HTX. So basically
+there is the HTTP counterpart to the previous callback :
+
+ * 'flt_ops.http_payload' : This callback is called when input data are
+ available. If not defined, all available data will be considered as analyzed
+ and forwarded for the filter.
+
+But the prototype for this callbacks is slightly different. Instead of having
+the channel as parameter, we have the HTTP message (struct http_msg). This
+callback is called only if the filter is registered to analyze TCP data. Here is
+an example :
+
+ /* Returns a negative value if an error occurs, else the number of
+ * consumed bytes. */
+ static int
+ my_filter_http_payload(struct stream *s, struct filter *filter,
+ struct http_msg *msg, unsigned int offset,
+ unsigned int len)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+ struct htx *htx = htxbuf(&msg->chn->buf);
+ struct htx_ret htxret = htx_find_offset(htx, offset);
+ struct htx_blk *blk;
+
+ blk = htxret.blk;
+ offset = htxret.ret;
+ for (; blk; blk = htx_get_next_blk(blk, htx)) {
+ enum htx_blk_type type = htx_get_blk_type(blk);
+
+ if (type == HTX_BLK_UNUSED)
+ continue;
+ else if (type == HTX_BLK_DATA) {
+ /* filter data */
+ }
+ else
+ break;
+ }
+
+ return len;
+ }
+
+In addition, there are two others callbacks :
+
+ * 'flt_ops.http_headers' : This callback is called just before the HTTP body
+ forwarding and after any processing on the request/response HTTP
+ headers. When defined, this callback is always called for HTTP streams
+ (i.e. without needs of a registration on data filtering).
+ Here is an example :
+
+
+ /* Returns a negative value if an error occurs, 0 if it needs to wait,
+ * any other value otherwise. */
+ static int
+ my_filter_http_headers(struct stream *s, struct filter *filter,
+ struct http_msg *msg)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+ struct htx *htx = htxbuf(&msg->chn->buf);
+ struct htx_sl *sl = http_get_stline(htx);
+ int32_t pos;
+
+ for (pos = htx_get_first(htx); pos != -1; pos = htx_get_next(htx, pos)) {
+ struct htx_blk *blk = htx_get_blk(htx, pos);
+ enum htx_blk_type type = htx_get_blk_type(blk);
+ struct ist n, v;
+
+ if (type == HTX_BLK_EOH)
+ break;
+ if (type != HTX_BLK_HDR)
+ continue;
+
+ n = htx_get_blk_name(htx, blk);
+ v = htx_get_blk_value(htx, blk);
+ /* Do something on the header name/value */
+ }
+
+ return 1;
+ }
+
+ * 'flt_ops.http_end' : This callback is called when the whole HTTP message was
+ processed. It may interrupt the stream processing. So, it could be used to
+ synchronize the HTTP request with the HTTP response, for instance :
+
+ /* Returns a negative value if an error occurs, 0 if it needs to wait,
+ * any other value otherwise. */
+ static int
+ my_filter_http_end(struct stream *s, struct filter *filter,
+ struct http_msg *msg)
+ {
+ struct my_filter_ctx *my_ctx = filter->ctx;
+
+
+ if (!(msg->chn->flags & CF_ISRESP)) /* The request */
+ my_ctx->end_of_req = 1;
+ else /* The response */
+ my_ctx->end_of_rsp = 1;
+
+ /* Both the request and the response are finished */
+ if (my_ctx->end_of_req == 1 && my_ctx->end_of_rsp == 1)
+ return 1;
+
+ /* Wait */
+ return 0;
+ }
+
+Then, to finish, there are 2 informational callbacks :
+
+ * 'flt_ops.http_reset' : This callback is called when an HTTP message is
+ reset. This happens either when a 1xx informational response is received, or
+ if we're retrying to send the request to the server after it failed. It
+ could be useful to reset the filter context before receiving the true
+ response.
+ By checking s->txn->status, it is possible to know why this callback is
+ called. If it's a 1xx, we're called because of an informational
+ message. Otherwise, it is a L7 retry.
+
+ * 'flt_ops.http_reply' : This callback is called when, at any time, HAProxy
+ decides to stop the processing on a HTTP message and to send an internal
+ response to the client. This mainly happens when an error or a redirect
+ occurs.
+
+
+3.6.3 REWRITING DATA
+--------------------
+
+The last part, and the trickiest one about the data filtering, is about the data
+rewriting. For now, the filter API does not offer a lot of functions to handle
+it. There are only functions to notify HAProxy that the data size has changed to
+let it update internal state of filters. This is the developer responsibility to
+update data itself, i.e. the buffer offsets, using following function :
+
+ * 'flt_update_offsets()' : This function must be called when a filter alter
+ incoming data. It updates offsets of the stream and of all filters
+ preceding the calling one. Do not call this function when a filter change
+ the size of incoming data leads to an undefined behavior.
+
+A good example of filter changing the data size is the HTTP compression filter.
diff --git a/doc/internals/api/htx-api.txt b/doc/internals/api/htx-api.txt
new file mode 100644
index 0000000..62b3093
--- /dev/null
+++ b/doc/internals/api/htx-api.txt
@@ -0,0 +1,570 @@
+ -----------------------------------------------
+ HTX API
+ Version 1.1
+ ( Last update: 2021-02-24 )
+ -----------------------------------------------
+ Author : Christopher Faulet
+ Contact : cfaulet at haproxy dot com
+
+1. Background
+
+Historically, HAProxy stored HTTP messages in a raw fashion in buffers, keeping
+parsing information separately in a "struct http_msg" owned by the stream. It was
+optimized to the data transfer, but not so much for rewrites. It was also HTTP/1
+centered. While it was the only HTTP version supported, it was not a
+problem. But with the rise of HTTP/2, it starts to be hard to still use this
+representation.
+
+At the first age of the HTTP/2 in HAProxy, H2 messages were converted into
+H1. This was terribly unefficient because it required two parsing passes, a
+first one in H2 and a second one in H1, with a conversion in the middle. And of
+course, the same was also true in the opposite direction. outgoing H1 messages
+had to be converted back in H2 to be sent. Even worse, because the H2->H1
+conversion, only client H2 connections were supported.
+
+So, to address all these problems, we decided to replace the old raw
+representation by a version-agnostic and self-structured internal HTTP
+representation, the HTX. As an additional benefit, with this new representation,
+the message parsing and its processing are now separated, making all the HTTP
+analysis simpler and cleaner. The parsing of HTTP messages is now handled by
+the multiplexers (h1 or h2).
+
+
+2. The HTX message
+
+The HTX is a structure containing useful information about an HTTP message
+followed by a contiguous array with some parts of the message. These parts are
+called blocks. A block is composed of metadata (htx_blk) and an associated
+payload. Blocks' metadata are stored starting from the end of the array while
+their payload are stored at the beginning. Blocks' metadata are often simply
+called blocks. it is a misuse of language that's simplify explanations.
+
+Internally, this structure is "hidden" in a buffer. This way, there are few
+changes into intermediate layers (stream-interface and channels). They still
+manipulate buffers. Only the multiplexer and the stream have to know how data
+are really stored. From the HTX perspective, a buffer is just a memory
+area. When an HTX message is stored in a buffer, this one appears as full.
+
+ * General view of an HTX message :
+
+
+ buffer->area
+ |
+ |<------------ buffer->size == buffer->data ----------------------|
+ | |
+ | |<------------- Blocks array (htx->size) ------------------>|
+ V | |
+ +-----+-----------------+-------------------------+---------------+
+ | HTX | PAYLOADS ==> | | <== HTX_BLKs |
+ +-----+-----------------+-------------------------+---------------+
+ | | | |
+ |<-payloads part->|<----- free space ------>|<-blocks part->|
+ (htx->data)
+
+
+The blocks part remains linear and sorted. It may be see as an array with
+negative indexes. But, instead of using negative indexes, we use positive
+positions to identify a block. This position is then converted to an address
+relatively to the beginning of the blocks array.
+
+ tail head
+ | |
+ V V
+ .....--+----+-----------------------+------+------+
+ | Bn | ... | B1 | B0 |
+ .....--+----+-----------------------+------+------+
+ ^ ^ ^
+ Addr of the block Addr of the block Addr of the block
+ at the position N at the position 1 at the position 0
+
+
+In the HTX structure, 3 "special" positions are stored :
+
+ - tail : Position of the newest inserted block
+ - head : Position of the oldest inserted block
+ - first : Position of the first block to (re)start the analyse
+
+The blocks part never wrap. If we have no space to allocate a new block and if
+there is a hole at the beginning of the blocks part (so at the end of the blocks
+array), we move back all blocks.
+
+
+ tail head tail head
+ | | | |
+ V V V V
+ ...+--------------+---------+ blocks ...----------+--------------+
+ | X== HTX_BLKS | | defrag | <== HTX_BLKS |
+ ...+--------------+---------+ =====> ...----------+--------------+
+
+
+The payloads part is a raw space that may wrap. A block's payload must never be
+accessed directly. Instead a block must be selected to retrieve the address of
+its payload.
+
+
+ +------------------------( B0.addr )--------------------------+
+ | +-------------------( B1.addr )----------------------+ |
+ | | +-----------( B2.addr )----------------+ | |
+ V V V | | |
+ +-----+----+-------+----+--------+-------------+-------+----+----+----+
+ | HTX | P0 | P1 | P2 | ...==> | | <=... | B2 | B1 | B0 |
+ +-----+----+-------+----+--------+-------------+-------+----+----+----+
+
+
+Because the payloads part may wrap, there are 2 usable free spaces :
+
+ - The free space in front of the blocks part. This one is used if and only if
+ the other one was not used yet.
+
+ - The free space at the beginning of the message. Once this one is used, the
+ other one is never used again, until a message defragmentation.
+
+
+ * Linear payloads part :
+
+
+ head_addr end_addr tail_addr
+ | | |
+ V V V
+ +-----+--------------------+-------------+--------------------+-------...
+ | HTX | | PAYLOADS | | HTX_BLKs
+ +-----+--------------------+-------------+--------------------+-------...
+ |<-- free space 2 -->| |<-- free space 1 -->|
+ (used if the other is too small) (used in priority)
+
+
+ * Wrapping payloads part :
+
+
+ head_addr end_addr tail_addr
+ | | |
+ V V V
+ +-----+----+----------------+--------+----------------+-------+-------...
+ | HTX | | PAYLOADS part2 | | PAYLOADS part1 | | HTX_BLKs
+ +-----+----+----------------+--------+----------------+-------+-------...
+ |<-->| |<------>| |<----->|
+ unusable free space unusable
+ free space free space
+
+
+Finally, when the usable free space is not enough to store a new block, unusable
+parts may be get back with a full defragmentation. The payloads part is then
+realigned at the beginning of the blocks array and the free space becomes
+continuous again.
+
+
+3. The HTX blocks
+
+An HTX block can be as well a start-line as a header, a body part or a
+trailer. For all these types of block, a payload is attached to the block. It
+can also be a marker, the end-of-headers or end-of-trailers. For these blocks,
+there is no payload but it counts for a byte. It is important to not skip it
+when data are forwarded.
+
+As already said, a block is composed of metadata and a payload. Metadata are
+stored in the blocks part and are composed of 2 fields :
+
+ - info : It a 32 bits field containing the block's type on 4 bits followed
+ by the payload length. See below for details.
+
+ - addr : The payload's address, if any, relatively to the beginning the
+ array used to store part of the HTTP message itself.
+
+
+ * Block's info representation :
+
+ 0b 0000 0000 0000 0000 0000 0000 0000 0000
+ ---- ------------------------ ---------
+ type value (1 MB max) name length (header/trailer - 256B max)
+ ----------------------------------
+ data length (256 MB max)
+ (body, method, path, version, status, reason)
+
+
+Supported types are :
+
+ - 0000 (0) : The request start-line
+ - 0001 (1) : The response start-line
+ - 0010 (2) : A header block
+ - 0011 (3) : The end-of-headers marker
+ - 0100 (4) : A data block
+ - 0101 (5) : A trailer block
+ - 0110 (6) : The end-of-trailers marker
+ - 1111 (15) : An unused block
+
+Other types are unused for now and reserved for futur extensions.
+
+An HTX message is typically composed of following blocks, in this order :
+
+ - a start-line
+ - zero or more header blocks
+ - an end-of-headers marker
+ - zero or more data blocks
+ - zero or more trailer blocks (optional)
+ - an end-of-trailers marker (optional but always set if there is at least
+ one trailer block)
+
+Only one HTTP request at a time can be stored in an HTX message. For HTTP
+response, it is more complicated. Only one "final" response can be stored in an
+HTX message. It is a response with status-code 101 or greater or equal to
+200. But it may be preceded by several 1xx informational responses. Such
+responses are part of the same HTX message.
+
+When the end of the message is reached a special flag is set on the message
+(HTX_FL_EOM). It means no more data are expected for this message, except
+tunneled data. But tunneled data will never be mixed with message data to avoid
+ambiguities. Thus once the flag marking the end of the message is set, it is
+easy to know the message ends. The end is reached if the HTX message is empty or
+on the tail HTX block in the HTX message. Once all blocks of the HTX message are
+consumed, tunneled data, if any, may be transferred.
+
+
+3.1. The start-line
+
+Every HTX message starts with a start-line. Its payload is a "struct htx_sl". In
+addition to the parts of the HTTP start-line, this structure contains some
+information about the represented HTTP message, mainly in the form of flags
+(HTX_SL_F_*). For instance, if an HTTP message contains the header
+"conten-length", then the flag HTX_SL_F_CLEN is set.
+
+Each HTTP message has its own start-line. So an HTX request has one and only one
+start-line because it must contain only one HTTP request at a time. But an HTX
+response may have more than one start-line if the final HTTP response is
+precedeed by some 1xx informational responses.
+
+In HTTP/2, there is no start-line. So the H2 multiplexer must create one when it
+converts an H2 message to HTX :
+
+ - For the request, it uses the pseudo headers ":method", ":path" or
+ ":authority" depending on the method and the hardcoded version "HTTP/2.0".
+
+ - For the response, it used the hardcoded version "HTTP/2.0", the
+ pseudo-header ":status" and an empty reason.
+
+
+3.2. The headers and trailers
+
+HTX Headers and trailers are quite similar. Different types are used to simplify
+headers processing. But from the HTX point of view, there is no real difference,
+except their position in the HTX message. The header blocks always follow an HTX
+start-line while trailer blocks come after the data. If there is no data, they
+follow the end-of-headers marker.
+
+Headers and trailers are the only blocks containing a Key/Value payload. The
+corresponding end-of marker must always be placed after each group to mark, as
+it name suggests, the end.
+
+In HTTP/1, trailers are only present on chunked messages. But chunked messages
+do not always have trailers. In this case, the end-of-trailers block may or may
+not be present. Multiplexers must be able to handle both situations. In HTTP/2,
+trailers are only present if a HEADERS frame is sent after DATA frames.
+
+
+3.3. The data
+
+The payload body of an HTTP message is stored as DATA blocks in the HTX
+message. For HTTP/1 messages, it is the message body without the chunks
+formatting, if any. For HTTP/2, it is the payload of DATA frames.
+
+The DATA blocks are the only HTX blocks that may be partially processed (copied
+or removed). All other types of block must be entirely processed. This means
+DATA blocks can be resized.
+
+
+3.4. The end-of markers
+
+These blocks are used to delimit parts of an HTX message. It exists two
+markers :
+
+ - end-of-headers (EOH)
+ - end-of-trailers (EOT)
+
+EOH is always present in an HTX message. EOT is optional.
+
+
+4. The HTX API
+
+
+4.1. Get/set HTX message from/to the underlying buffer
+
+The first thing to do to process an HTX message is to get it from the underlying
+buffer. There are 2 functions to do so, the second one relying on the first :
+
+ - htxbuf() returns an HTX message from a buffer. It does not modify the
+ buffer. It only initialize the HTX message if the buffer is empty.
+
+ - htx_from_buf() uses htxbuf(). But it also updates the underlying buffer so
+ that it appears as full.
+
+Both functions return a "zero-sized" HTX message if the buffer is null. This
+way, the HTX message is always valid. The first function is the default function
+to use. The second one is only useful when some content will be added. For
+instance, it used by the HTX analyzers when HAProxy generates a response. Thus,
+the buffer is in a right state.
+
+Once the processing done, if the HTX message has been modified, the underlying
+buffer must be also updated, except htx_from_buf() was used _AND_ data was only
+added. For all other cases, the function htx_to_buf() must be called.
+
+Finally, the function htx_reset() may be called at any time to reset an HTX
+message. And the function buf_room_for_htx_data() may be called to know if a raw
+buffer is full from the HTX perspective. It is used during conversion from/to
+the HTX.
+
+
+4.2. Helpers to deal with free space in an HTX message
+
+Once with an HTX message, following functions may help to process it :
+
+ - htx_used_space() and htx_meta_space() return, respectively, the total
+ space used in an HTX message and the space used by block's metadata only.
+
+ - htx_free_space() and htx_free_data_space() return, respectively, the total
+ free space in an HTX message and the free space available for the payload
+ if a new HTX block is stored (so it is the total free space minus the size
+ of an HTX block).
+
+ - htx_is_empty() and htx_is_not_empty() are boolean functions to know if an
+ HTX message is empty or not.
+
+ - htx_get_max_blksz() returns the maximum size available for the payload,
+ not exceeding a maximum, metadata included.
+
+ - htx_almost_full() should be used to know if an HTX message uses at least
+ 3/4 of its capacity.
+
+
+4.3. HTX Blocks manipulations
+
+Once the available sapce in an HTX message is known, the next step is to add HTX
+blocks. First of all the function htx_nbblks() returns the number of blocks
+allocated in an HTX message. Then, there is an add function per block's type :
+
+ - htx_add_stline() adds a start-line. The type (request or response) and the
+ flags of the start-line must be provided, as well as its three parts
+ (method,uri,version or version,status-code,reason).
+
+ - htx_add_header() and htx_add_trailers() are similar. The name and the
+ value must be provided. The inserted HTX block is returned on success or
+ NULL if an error occurred.
+
+ - htx_add_endof() must be used to add any end-of marker. The block's type
+ (EOH or EOT) must be specified. The inserted HTX block is returned on
+ success or NULL if an error occurred.
+
+ - htx_add_all_headers() and htx_add_all_trailers() add, respectively, a list
+ of headers and a list of trailers, followed by the appropriate end-of
+ marker. On success, this marker is returned. Otherwise, NULL is
+ returned. Note there is no rollback on the HTX message when an error
+ occurred. Some headers or trailers may have been added. So it is the
+ caller responsibility to take care of that.
+
+ - htx_add_data() must be used to add a DATA block. Unlike previous
+ functions, this one returns the number of bytes copied or 0 if nothing was
+ copied. If possible, the data are appended to the tail block if it is a
+ DATA block. Only a part of the payload may be copied because this function
+ will try to limit the message defragmentation and the wrapping of blocks
+ as far as possible.
+
+ - htx_add_data_atonce() must be used if all data must be added or nothing.
+ It tries to insert all the payload, this function returns the inserted
+ block on success. Otherwise it returns NULL.
+
+When an HTX block is added, it is always the last one (the tail). But, if a
+block must be added at a specific place, it is not really handy. 2 functions may
+help (others could be added) :
+
+ - htx_add_last_data() adds a DATA block just after all other DATA blocks and
+ before any trailers and EOT marker. It relies on htx_add_data_atonce(), so
+ a defragmentation may be performed.
+
+ - htx_move_blk_before() moves a specific block just after another one. Both
+ blocks must already be in the HTX message and the block to move must
+ always be placed after the "pivot".
+
+Once added, there are three functions to update the block's payload :
+
+ - htx_replace_stline() updates a start-line. The HTX block must be passed as
+ argument. Only string parts of the start-line are updated by this
+ function. On success, it returns the new start-line. So it is pretty easy
+ to update its flags. NULL is returned if an error occurred.
+
+ - htx_replace_header() fully replaces a header (its name and its value) by a
+ new one. The HTX block must be passed a argument, as well as its new name
+ and its new value. The new header can be smaller or larger than the old
+ one. This function returns the new HTX block on success, or NULL is an
+ error occurred.
+
+ - htx_replace_blk_value() replaces a part of a block's payload or its
+ totality. It works for HEADERS, TRAILERS or DATA blocks. The HTX block
+ must be provided with the part to remove and the new one. The new part can
+ be smaller or larger than the old one. This function returns the new HTX
+ block on success, or NULL is an error occurred.
+
+ - htx_change_blk_value_len() changes the size of the value. It is the caller
+ responsibility to change the value itself, make sure there is enough space
+ and update allocated value. This function updates the HTX message
+ accordingly.
+
+ - htx_set_blk_value_len() changes the size of the value. It is the caller
+ responsibility to change the value itself, make sure there is enough space
+ and update allocated value. Unlike the function
+ htx_change_blk_value_len(), this one does not update the HTX message. So
+ it should be used with caution.
+
+ - htx_cut_data_blk() removes <n> bytes from the beginning of a DATA
+ block. The block's start address and its length are adjusted, and the
+ htx's total data count is updated. This is used to mark that part of some
+ data were transferred from a DATA block without removing this DATA
+ block. No sanity check is performed, the caller is responsible for doing
+ this exclusively on DATA blocks, and never removing more than the block's
+ size.
+
+ - htx_remove_blk() removes a block from an HTX message. It returns the
+ following block or NULL if it is the tail block.
+
+Finally, a block may be removed using the function htx_remove_blk(). This
+function returns the block following the one removed or NULL if it is the tail
+block.
+
+
+4.4. The HTX start-line
+
+Unlike other HTX blocks, the start-line is a bit special because its payload is
+a structure followed by its three parts :
+
+ +--------+-------+-------+-------+
+ | HTX_SL | PART1 | PART2 | PART3 |
+ +--------+-------+-------+-------+
+
+Some macros and functions may help to manipulate these parts :
+
+ - HTX_SL_P{N}_LEN() and HTX_SL_P{N}_PTR() are macros to get the length of a
+ part and a pointer on it. {N} should be 1, 2 or 3.
+
+ - HTX_SL_REQ_MLEN(), HTX_SL_REQ_ULEN(), HTX_SL_REQ_VLEN(),
+ HTX_SL_REQ_MPTR(), HTX_SL_REQ_UPTR() and HTX_SL_REQ_VPTR() are macros to
+ get info about a request start-line. These macros only wrap HTX_SL_P*
+ ones.
+
+ - HTX_SL_RES_VLEN(), HTX_SL_RES_CLEN(), HTX_SL_RES_RLEN(),
+ HTX_SL_RES_VPTR(), HTX_SL_RES_CPTR() and HTX_SL_RES_RPTR() are macros to
+ get info about a response start-line. These macros only wrap HTX_SL_P*
+ ones.
+
+ - htx_sl_p1(), htx_sl_p2() and htx_sl_p2() are functions to get the ist
+ corresponding to the right part of a start-line.
+
+ - htx_sl_req_meth(), htx_sl_req_uri() and htx_sl_req_vsn() get the ist
+ corresponding to the right part of a request start-line.
+
+ - htx_sl_res_vsn(), htx_sl_res_code() and htx_sl_res_reason() get the ist
+ corresponding to the right part of a response start-line.
+
+
+4.5. Iterate on the HTX message
+
+To iterate on an HTX message, the first thing to do is to get the HTX block to
+start the loop. There are three special blocks in an HTX message that may be
+good candidates to start a loop :
+
+ - the head block. It is the oldest inserted block. Multiplexers always start
+ to consume an HTX message from this block. The function htx_get_head()
+ returns its position and htx_get_head_blk() returns the blocks itself. In
+ addition, the function htx_get_head_type() returns its block's type.
+
+ - the tail block. It is the newest inserted block. The function
+ htx_get_tail() returns its position and htx_get_tail_blk() returns the
+ blocks itself. In addition, the function htx_get_tail_type() returns its
+ block's type.
+
+ - the first block. It is the block where to (re)start the analyse. It is
+ used as start point by HTX analyzers. The function htx_get_first() returns
+ its position and htx_get_first_blk() returns the blocks itself. In
+ addition, the function htx_get_first_type() returns its block's type.
+
+For all these functions, if the HTX message is empty, -1 is returned for the
+block's position, NULL instead of a block and HTX_BLK_UNUSED for its type.
+
+Then to iterate on blocks, foreword or backward :
+
+ - htx_get_prev() and htx_get_next() return, respectively, the position of
+ the previous block or the next block, given a specific position. Or -1 if
+ an edge is reached.
+
+ - htx_get_prev_blk() and htx_get_next_blk() return, respectively, the
+ previous block or the next one, given a specific block. Or NULL if an edge
+ is reached.
+
+4.6. Access block content and info
+
+Following functions may be used to retrieve information about a specific HTX
+block :
+
+ - htx_get_blk_pos() returns the position of a block. It must be in the HTX
+ message.
+
+ - htx_get_blk_ptr() returns a pointer on the payload of a block.
+
+ - htx_get_blk_type() returns the type of a block.
+
+ - htx_get_blksz() returns the payload size of a block
+
+ - htx_get_blk_name() returns the name of a block, only if it is a header or
+ a trailer. Otherwise, it returns an empty string.
+
+ - htx_get_blk_value() returns the value of a block, depending on its
+ type. For header and trailer blocks, it is the value field. For markers
+ (EOH or EOT), an empty string is returned. For other blocks an ist
+ pointing on the block payload is returned.
+
+ - htx_is_unique_blk() may be used to know if a block is the only one
+ remaining inside an HTX message, excluding unused blocks. This function is
+ pretty useful to determine the end of a HTX message, in conjunction with
+ HTX_FL_EOM flag.
+
+4.7. Advanced functions
+
+Some more advanced functions may be used to do complex processing on the HTX
+message. These functions are used by HTX analyzers or by multiplexers.
+
+ - htx_truncate() removes all blocks after the one containing a specific
+ offset relatively to the head block of the HTX message. If the offset is
+ inside a DATA block, it is truncated. For all other blocks, the removal
+ starts to the next block.
+
+ - htx_drain() tries to remove a specific amount of bytes of payload. If the
+ tail block is a DATA block, it may be truncated if necessary. All other
+ block are removed at once or kept. This function returns a mixed value,
+ with the first block not removed, or NULL if everything was removed, and
+ the amount of data drained.
+
+ - htx_xfer_blks() transfers HTX blocks from an HTX message to another,
+ stopping on the first block of a specified type or when a specific amount
+ of bytes, including meta-data, was moved. If the tail block is a DATA
+ block, it may be partially moved. All other block are transferred at once
+ or kept. This function returns a mixed value, with the last block moved,
+ or NULL if nothing was moved, and the amount of data transferred. When
+ HEADERS or TRAILERS blocks must be transferred, this function transfers
+ all of them. Otherwise, if it is not possible, it triggers an error. It is
+ the caller responsibility to transfer all headers or trailers at once.
+
+ - htx_append_msg() append an HTX message to another one. All the message is
+ copied or nothing. So, if an error occurred, a rollback is performed. This
+ function returns 1 on success and 0 on error.
+
+ - htx_reserve_max_data() Reserves the maximum possible size for an HTX data
+ block, by extending an existing one or by creating a new one. It returns a
+ compound result with the HTX block and the position where new data must be
+ inserted (0 for a new block). If an error occurs or if there is no space
+ left, NULL is returned instead of a pointer on an HTX block.
+
+ - htx_find_offset() looks for the HTX block containing a specific offset,
+ starting at the HTX message's head. The function returns the found HTX
+ block and the position inside this block where the offset is. If the
+ offset is outside of the HTX message, NULL is returned.
+
+ - htx_defrag() defragments an HTX message. It removes unused blocks and
+ unwraps the payloads part. A temporary buffer is used to do so. This
+ function never fails. A referenced block may be provided. If so, the
+ corresponding new block is returned. Otherwise, NULL is returned.
diff --git a/doc/internals/api/initcalls.txt b/doc/internals/api/initcalls.txt
new file mode 100644
index 0000000..a341edc
--- /dev/null
+++ b/doc/internals/api/initcalls.txt
@@ -0,0 +1,366 @@
+Initialization stages aka how to get your code initialized at the right moment
+
+
+1. Background
+
+Originally all subsystems were initialized via a dedicated function call
+from the huge main() function. Then some code started to become conditional
+or a bit more modular and the #ifdef placed there became a mess, resulting
+in init code being moved to function constructors in each subsystem's own
+file. Then pools of various things were introduced, starting to make the
+whole init sequence more complicated due to some forms of internal
+dependencies. Later epoll was introduced, requiring a post-fork callback,
+and finally threads arrived also requiring some post-thread init/deinit
+and allocation, marking the old architecture's last breath. Finally the
+whole thing resulted in lots of init code duplication and was simplified
+in 1.9 with the introduction of initcalls and initialization stages.
+
+
+2. New architecture
+
+The new architecture relies on two layers :
+ - the registration functions
+ - the INITCALL macros and initialization stages
+
+The first ones are mostly used to add a callback to a list. The second ones
+are used to specify when to call a function. Both are totally independent,
+however they are generally combined via another set consisting in the REGISTER
+macros which make some registration functions be called at some specific points
+during the init sequence.
+
+
+3. Registration functions
+
+Registration functions never fail. Or more precisely, if they fail it will only
+be on out-of-memory condition, and they will cause the process to immediately
+exit. As such they do not return any status and the caller doesn't have to care
+about their success.
+
+All available functions are described below in alphanumeric ordering. Please
+make sure to respect this ordering when adding new ones.
+
+- void hap_register_build_opts(const char *str, int must_free)
+
+ This appends the zero-terminated constant string <str> to the list of known
+ build options that will be reported on the output of "haproxy -vv". A line
+ feed character ('\n') will automatically be appended after the string when it
+ is displayed. The <must_free> argument must be zero, unless the string was
+ allocated by any malloc-compatible function such as malloc()/calloc()/
+ realloc()/strdup() or memprintf(), in which case it's better to pass a
+ non-null value so that the string is freed upon exit. Note that despite the
+ function's prototype taking a "const char *", the pointer will actually be
+ cast and freed. The const char* is here to leave more freedom to use consts
+ when making such options lists.
+
+- void hap_register_per_thread_alloc(int (*fct)())
+
+ This adds a call to function <fct> to the list of functions to be called when
+ threads are started, at the beginning of the polling loop. This is also valid
+ for the main thread and will be called even if threads are disabled, so that
+ it is guaranteed that this function will be called in any circumstance. Each
+ thread will first call all these functions exactly once when it starts. Calls
+ are serialized by the init_mutex, so that locking is not necessary in these
+ functions. There is no relation between the thread numbers and the callback
+ ordering. The function is expected to return non-zero on success, or zero on
+ failure. A failure will make the process emit a succinct error message and
+ immediately exit. See also hap_register_per_thread_free() for functions
+ called after these ones.
+
+- void hap_register_per_thread_deinit(void (*fct)());
+
+ This adds a call to function <fct> to the list of functions to be called when
+ threads are gracefully stopped, at the end of the polling loop. This is also
+ valid for the main thread and will be called even if threads are disabled, so
+ that it is guaranteed that this function will be called in any circumstance
+ if the process experiences a soft stop. Each thread will call this function
+ exactly once when it stops. However contrary to _alloc() and _init(), the
+ calls are made without any protection, thus if any shared resource if touched
+ by the function, the function is responsible for protecting it. The reason
+ behind this is that such resources are very likely to be still in use in one
+ other thread and that most of the time the functions will in fact only touch
+ a refcount or deinitialize their private resources. See also
+ hap_register_per_thread_free() for functions called after these ones.
+
+- void hap_register_per_thread_free(void (*fct)());
+
+ This adds a call to function <fct> to the list of functions to be called when
+ threads are gracefully stopped, at the end of the polling loop, after all calls
+ to _deinit() callbacks are done for this thread. This is also valid for the
+ main thread and will be called even if threads are disabled, so that it is
+ guaranteed that this function will be called in any circumstance if the
+ process experiences a soft stop. Each thread will call this function exactly
+ once when it stops. However contrary to _alloc() and _init(), the calls are
+ made without any protection, thus if any shared resource if touched by the
+ function, the function is responsible for protecting it. The reason behind
+ this is that such resources are very likely to be still in use in one other
+ thread and that most of the time the functions will in fact only touch a
+ refcount or deinitialize their private resources. See also
+ hap_register_per_thread_deinit() for functions called before these ones.
+
+- void hap_register_per_thread_init(int (*fct)())
+
+ This adds a call to function <fct> to the list of functions to be called when
+ threads are started, at the beginning of the polling loop, right after the
+ list of _alloc() functions. This is also valid for the main thread and will
+ be called even if threads are disabled, so that it is guaranteed that this
+ function will be called in any circumstance. Each thread will call this
+ function exactly once when it starts, and calls are serialized by the
+ init_mutex which is held over all _alloc() and _init() calls, so that locking
+ is not necessary in these functions. In other words for all threads but the
+ current one, the sequence of _alloc() and _init() calls will be atomic. There
+ is no relation between the thread numbers and the callback ordering. The
+ function is expected to return non-zero on success, or zero on failure. A
+ failure will make the process emit a succinct error message and immediately
+ exit. See also hap_register_per_thread_alloc() for functions called before
+ these ones.
+
+- void hap_register_pre_check(int (*fct)())
+
+ This adds a call to function <fct> to the list of functions to be called at
+ the step just before the configuration validity checks. This is useful when you
+ need to create things like it would have been done during the configuration
+ parsing and where the initialization should continue in the configuration
+ check.
+ It could be used for example to generate a proxy with multiple servers using
+ the configuration parser itself. At this step the final trash buffers are
+ allocated. Threads are not yet started so no protection is required. The
+ function is expected to return non-zero on success, or zero on failure. A
+ failure will make the process emit a succinct error message and immediately
+ exit.
+
+- void hap_register_post_check(int (*fct)())
+
+ This adds a call to function <fct> to the list of functions to be called at
+ the end of the configuration validity checks, just at the point where the
+ program either forks or exits depending whether it's called with "-c" or not.
+ Such calls are suited for memory allocation or internal table pre-computation
+ that would preferably not be done on the fly to avoid inducing extra time to
+ a pure configuration check. Threads are not yet started so no protection is
+ required. The function is expected to return non-zero on success, or zero on
+ failure. A failure will make the process emit a succinct error message and
+ immediately exit.
+
+- void hap_register_post_deinit(void (*fct)())
+
+ This adds a call to function <fct> to the list of functions to be called when
+ freeing the global sections at the end of deinit(), after everything is
+ stopped. The process is single-threaded at this point, thus these functions
+ are suitable for releasing configuration elements provided that no other
+ _deinit() function uses them, i.e. only close/release what is strictly
+ private to the subsystem. Since such functions are mostly only called during
+ soft stops (reloads) or failed startups, they tend to experience much less
+ test coverage than others despite being more exposed, and as such a lot of
+ care must be taken to test them especially when facing partial subsystem
+ initializations followed by errors.
+
+- void hap_register_post_proxy_check(int (*fct)(struct proxy *))
+
+ This adds a call to function <fct> to the list of functions to be called for
+ each proxy, after the calls to _post_server_check(). This can allow, for
+ example, to pre-configure default values for an option in a frontend based on
+ the "bind" lines or something in a backend based on the "server" lines. It's
+ worth being aware that such a function must be careful not to waste too much
+ time in order not to significantly slow down configurations with tens of
+ thousands of backends. The function is expected to return non-zero on
+ success, or zero on failure. A failure will make the process emit a succinct
+ error message and immediately exit.
+
+- void hap_register_post_server_check(int (*fct)(struct server *))
+
+ This adds a call to function <fct> to the list of functions to be called for
+ each server, after the call to check_config_validity(). This can allow, for
+ example, to preset a health state on a server or to allocate a protocol-
+ specific memory area. It's worth being aware that such a function must be
+ careful not to waste too much time in order not to significantly slow down
+ configurations with tens of thousands of servers. The function is expected
+ to return non-zero on success, or zero on failure. A failure will make the
+ process emit a succinct error message and immediately exit.
+
+- void hap_register_proxy_deinit(void (*fct)(struct proxy *))
+
+ This adds a call to function <fct> to the list of functions to be called when
+ freeing the resources during deinit(). These functions will be called as part
+ of the proxy's resource cleanup. Note that some of the proxy's fields will
+ already have been freed and others not, so such a function must not use any
+ information from the proxy that is subject to being released. In particular,
+ all servers have already been deleted. Since such functions are mostly only
+ called during soft stops (reloads) or failed startups, they tend to
+ experience much less test coverage than others despite being more exposed,
+ and as such a lot of care must be taken to test them especially when facing
+ partial subsystem initializations followed by errors. It's worth mentioning
+ that too slow functions could have a significant impact on the configuration
+ check or exit time especially on large configurations.
+
+- void hap_register_server_deinit(void (*fct)(struct server *))
+
+ This adds a call to function <fct> to the list of functions to be called when
+ freeing the resources during deinit(). These functions will be called as part
+ of the server's resource cleanup. Note that some of the server's fields will
+ already have been freed and others not, so such a function must not use any
+ information from the server that is subject to being released. Since such
+ functions are mostly only called during soft stops (reloads) or failed
+ startups, they tend to experience much less test coverage than others despite
+ being more exposed, and as such a lot of care must be taken to test them
+ especially when facing partial subsystem initializations followed by errors.
+ It's worth mentioning that too slow functions could have a significant impact
+ on the configuration check or exit time especially on large configurations.
+
+
+4. Initialization stages
+
+In order to offer some guarantees, the startup of the program is split into
+several stages. Some callbacks can be placed into each of these stages using
+an INITCALL macro, with 0 to 3 arguments, respectively called INITCALL0 to
+INITCALL3. These macros must be placed anywhere at the top level of a C file,
+preferably at the end so that the referenced symbols have already been met,
+but it may also be fine to place them right after the callbacks themselves.
+
+Such callbacks are referenced into small structures containing a pointer to the
+function and 3 arguments. NULL replaces unused arguments. The callbacks are
+cast to (void (*)(void *, void *, void *)) and the arguments to (void *).
+
+The first argument to the INITCALL macro is the initialization stage. The
+second one is the callback function, and others if any are the arguments.
+The init stage must be among the values of the "init_stage" enum, currently,
+and in this execution order:
+
+ - STG_PREPARE : used to preset variables, pre-initialize lookup tables and
+ pre-initialize list heads
+ - STG_LOCK : used to pre-initialize locks
+ - STG_REGISTER : used to register static lists such as keywords
+ - STG_ALLOC : used to allocate the required structures
+ - STG_POOL : used to create pools
+ - STG_INIT : used to initialize subsystems
+
+Each stage is guaranteed that previous stages have successfully completed. This
+means that an INITCALL placed at stage STG_INIT is guaranteed that all pools
+were already created and will be usable. Conversely, an INITCALL placed at
+stage STG_REGISTER must not rely on any field that requires preliminary
+allocation nor initialization. A callback cannot rely on other callbacks of the
+same stage, as the execution order within a stage is undefined and essentially
+depends on the linking order.
+
+The STG_REGISTER level is made for run-time linking of the various modules that
+compose the executable. Keywords, protocols and various other elements that are
+local known to each compilation unit can will be appended into common lists at
+boot time. This is why this call is placed just before STG_ALLOC.
+
+Note that trash is needed in various functions. Trash is a pool and is
+allocated during STG_POOL, so it's not permitted to use it before STG_INIT,
+where it will only use the default size, and may be reallocated later with a
+different size.
+
+Example: register a very early call to init_log() with no argument, and another
+ call to cli_register_kw(&cli_kws) much later:
+
+ INITCALL0(STG_PREPARE, init_log);
+ INITCALL1(STG_REGISTER, cli_register_kw, &cli_kws);
+
+Technically speaking, each call to such a macro adds a distinct local symbol
+whose dynamic name involves the line number. These symbols are placed into a
+separate section and the beginning and end section pointers are provided by the
+linker. When too old a linker is used, a fallback is applied consisting in
+placing them into a linked list which is built by a constructor function for
+each initcall (this takes more room).
+
+Due to the symbols internally using the line number, it is very important not
+to place more than one INITCALL per line in the source file.
+
+It is also strongly recommended that functions and referenced arguments are
+static symbols local to the source file, unless they are global registration
+functions like in the example above with cli_register_kw(), where only the
+argument is a local keywords table.
+
+INITCALLs do not expect the callback function to return anything and as such
+do not perform any error check. As such, they are very similar to constructors
+offered by the compiler except that they are segmented in stages. It is thus
+the responsibility of the called functions to perform their own error checking
+and to exit in case of error. This may change in the future.
+
+
+5. REGISTER family of macros
+
+The association of INITCALLs and registration functions allows to perform some
+early dynamic registration of functions to be used anywhere, as well as values
+to be added to existing lists without having to manipulate list elements. For
+the sake of simplification, these combinations are available as a set of
+REGISTER macros which register calls to certain functions at the appropriate
+init stage. Such macros must be used at the top level in a file, just like
+INITCALL macros. The following macros are currently supported. Please keep them
+alphanumerically ordered:
+
+- REGISTER_BUILD_OPTS(str)
+
+ Adds the constant string <str> to the list of build options. This is done by
+ registering a call to hap_register_build_opts(str, 0) at stage STG_REGISTER.
+ The string will not be freed.
+
+- REGISTER_CONFIG_POSTPARSER(name, parser)
+
+ Adds a call to function <parser> at the end of the config parsing. The
+ function is called at the very end of check_config_validity() and may be used
+ to initialize a subsystem based on global settings for example. This is done
+ by registering a call to cfg_register_postparser(name, parser) at stage
+ STG_REGISTER.
+
+- REGISTER_CONFIG_SECTION(name, parse, post)
+
+ Registers a new config section name <name> which will be parsed by function
+ <parse> (if not null), and with an optional call to function <post> at the
+ end of the section. Function <parse> must be of type (int (*parse)(const char
+ *file, int linenum, char **args, int inv)), and returns 0 on success or an
+ error code among the ERR_* set on failure. The <post> callback takes no
+ argument and returns a similar error code. This is achieved by registering a
+ call to cfg_register_section() with the three arguments at stage
+ STG_REGISTER.
+
+- REGISTER_PER_THREAD_ALLOC(fct)
+
+ Registers a call to register_per_thread_alloc(fct) at stage STG_REGISTER.
+
+- REGISTER_PER_THREAD_DEINIT(fct)
+
+ Registers a call to register_per_thread_deinit(fct) at stage STG_REGISTER.
+
+- REGISTER_PER_THREAD_FREE(fct)
+
+ Registers a call to register_per_thread_free(fct) at stage STG_REGISTER.
+
+- REGISTER_PER_THREAD_INIT(fct)
+
+ Registers a call to register_per_thread_init(fct) at stage STG_REGISTER.
+
+- REGISTER_POOL(ptr, name, size)
+
+ Used internally to declare a new pool. This is made by calling function
+ create_pool_callback() with these arguments at stage STG_POOL. Do not use it
+ directly, use either DECLARE_POOL() or DECLARE_STATIC_POOL() instead.
+
+- REGISTER_PRE_CHECK(fct)
+
+ Registers a call to register_pre_check(fct) at stage STG_REGISTER.
+
+- REGISTER_POST_CHECK(fct)
+
+ Registers a call to register_post_check(fct) at stage STG_REGISTER.
+
+- REGISTER_POST_DEINIT(fct)
+
+ Registers a call to register_post_deinit(fct) at stage STG_REGISTER.
+
+- REGISTER_POST_PROXY_CHECK(fct)
+
+ Registers a call to register_post_proxy_check(fct) at stage STG_REGISTER.
+
+- REGISTER_POST_SERVER_CHECK(fct)
+
+ Registers a call to register_post_server_check(fct) at stage STG_REGISTER.
+
+- REGISTER_PROXY_DEINIT(fct)
+
+ Registers a call to register_proxy_deinit(fct) at stage STG_REGISTER.
+
+- REGISTER_SERVER_DEINIT(fct)
+
+ Registers a call to register_server_deinit(fct) at stage STG_REGISTER.
+
diff --git a/doc/internals/api/ist.txt b/doc/internals/api/ist.txt
new file mode 100644
index 0000000..0f118d6
--- /dev/null
+++ b/doc/internals/api/ist.txt
@@ -0,0 +1,167 @@
+2021-11-08 - Indirect Strings (IST) API
+
+
+1. Background
+-------------
+
+When parsing traffic, most of the standard C string functions are unusable
+since they rely on a trailing zero. In addition, for the rare ones that support
+a length, we have to constantly maintain both the pointer and the length. But
+then, it's easy to come up with complex lengths and offsets calculations all
+over the place, rendering the code hard to read and bugs hard to avoid or spot.
+
+IST provides a solution to this by defining a structure made of exactly two
+word size elements, that most C ABIs know how to handle as a register when
+used as a function argument or a function's return value. The functions are
+inlined to leave a maximum set of opportunities to the compiler or optimization
+and expression reduction, and as a result they are often inexpensive to use. It
+is important however to keep in mind that all of these are designed for minimal
+code size when dealing with short strings (i.e. parsing tokens in protocols),
+and they are not optimal for processing large blocks.
+
+
+2. API description
+------------------
+
+IST are defined like this:
+
+ struct ist {
+ char *ptr; // pointer to the string's first byte
+ size_t len; // number of valid bytes starting from ptr
+ };
+
+A string is not set if its ->ptr member is NULL. In this case .len is undefined
+and is recommended to be zero.
+
+Declaring a function returning an IST:
+
+ struct ist produce_ist(int ok)
+ {
+ return ok ? IST("OK") : IST("KO");
+ }
+
+Declaring a function consuming an IST:
+
+ void say_ist(struct ist i)
+ {
+ write(1, istptr(i), istlen(i));
+ }
+
+Chaining the two:
+
+ void say_ok(int ok)
+ {
+ say_ist(produce_ist(ok));
+ }
+
+Notes:
+ - the arguments are passed as value, not reference, so there's no need for
+ any "const" in their declaration (except to catch coding mistakes).
+ Pointers to ist may benefit from being marked "const" however.
+
+ - similarly for the return value, there's no point is marking it "const" as
+ this would protect the pointer and length, not the data.
+
+ - use ist0() to append a trailing zero to a variable string for use with
+ printf()'s "%s" format, or for use with functions that work on NUL-
+ terminated strings, but beware of not doing this with constants.
+
+ - the API provides a starting pointer and current length, but does not
+ provide an allocated size. It remains up to the caller to know how large
+ the allocated area is when adding data, though most functions make this
+ easy.
+
+The following macros and functions are defined. Those whose name starts with
+underscores require special care and must not be used without being certain
+they are properly used (typically subject to buffer overflows if misused). Note
+that most functions were added over time depending on instant needs, and some
+are very close to each other. Many useful functions are still missing and would
+deserve being added.
+
+Below, arguments "i1","i2" are all of type "ist". Arguments "s" are
+NUL-terminated strings of type "char*", and "cs" are of type "const char *".
+Arguments "c" are of type "char", and "n" are of type size_t.
+
+ IST(cs):ist make constant IST from a NUL-terminated const string
+ IST_NULL:ist return an unset IST = ist2(NULL,0)
+ __istappend(i1,c):ist append character <c> at the end of ist <i1>
+ ist(s):ist return an IST from a nul-terminated string
+ ist0(i1):char* write a \0 at the end of an IST, return the string
+ ist2(cs,l):ist return a variable IST from a const string and length
+ ist2bin(s,i1):ist copy IST into a buffer, return the result
+ ist2bin_lc(s,i1):ist like ist2bin() but turning turning to lower case
+ ist2bin_uc(s,i1):ist like ist2bin() but turning turning to upper case
+ ist2str(s,i1):ist copy IST into a buffer, add NUL and return the result
+ ist2str_lc(s,i1):ist like ist2str() but turning turning to lower case
+ ist2str_uc(s,i1):ist like ist2str() but turning turning to upper case
+ ist_find(i1,c):ist return first occurrence of char <c> in <i1>
+ ist_find_ctl(i1):char* return pointer to first CTL char in <i1> or NULL
+ ist_skip(i1,c):ist return first occurrence of char not <c> in <i1>
+ istadv(i1,n):ist advance the string by <n> characters
+ istalloc(n):ist return allocated string of zero initial length
+ istcat(d,s,n):ssize_t copy <s> after <d> for <n> chars max, return len or -1
+ istchr(i1,c):char* return pointer to first occurrence of <c> in <i1>
+ istclear(i1*):size_t return previous size and set size to zero
+ istcpy(d,s,n):ssize_t copy <s> over <d> for <n> chars max, return len or -1
+ istdiff(i1,i2):int return the ordinal difference, like strcmp()
+ istdup(i1):ist allocate new ist and copy original one into it
+ istend(i1):char* return pointer to first character after the IST
+ isteq(i1,i2):int return non-zero if strings are equal
+ isteqi(i1,i2):int like isteq() but case-insensitive
+ istfree(i1*) free of allocated <i1>/IST_NULL and set it to IST_NULL
+ istissame(i1,i2):int return true if pointers and lengths are equal
+ istist(i1,i2):ist return first occurrence of <i2> in <i1>
+ istlen(i1):size_t return the length of the IST (number of characters)
+ istmatch(i1,i2):int return non-zero if i1 starts like i2 (empty OK)
+ istmatchi(i1,i2):int like istmatch() but case insensitive
+ istneq(i1,i2,n):int like isteq() but limited to the first <n> chars
+ istnext(i1):ist return the IST advanced by one character
+ istnmatch(i1,i2,n):int like istmatch() but limited to the first <n> chars
+ istpad(s,i1):ist copy IST into a buffer, add a NUL, return the result
+ istptr(i1):char* return the starting pointer of the IST
+ istscat(d,s,n):ssize_t same as istcat() but always place a NUL at the end
+ istscpy(d,s,n):ssize_t same as istcpy() but always place a NUL at the end
+ istshift(i1*):char return the first character and advance the IST by one
+ istsplit(i1*,c):ist return part before <c>, make ist start from <c>
+ iststop(i1,c):ist truncate ist before first occurrence of <c>
+ isttest(i1):int return true if ist is not NULL, false otherwise
+ isttrim(i1,n):ist return ist trimmed to no more than <n> characters
+ istzero(i1,n):ist trim to <n> chars, trailing zero included.
+
+
+3. Quick index by typical C construct or function
+-------------------------------------------------
+
+Some common C constructs may be adjusted to use ist instead. The mapping is not
+always one-to-one, but usually the computations on the length part tends to
+disappear in the refactoring, allowing to directly chain function calls. The
+entries below are hints to figure what function to look for in order to rewrite
+some common use cases.
+
+ char* IST equivalent
+
+ strchr() istchr(), ist_find(), iststop()
+ strstr() istist()
+ strcpy() istcpy()
+ strscpy() istscpy()
+ strlcpy() istscpy()
+ strcat() istcat()
+ strscat() istscat()
+ strlcat() istscat()
+ strcmp() istdiff()
+ strdup() istdup()
+ !strcmp() isteq()
+ !strncmp() istneq(), istmatch(), istnmatch()
+ !strcasecmp() isteqi()
+ !strncasecmp() istneqi(), istmatchi()
+ strtok() istsplit()
+ return NULL return IST_NULL
+ s = malloc() s = istalloc()
+ free(s); s = NULL istfree(&s)
+ p != NULL isttest(p)
+ c = *(p++) c = istshift(p)
+ *(p++) = c __istappend(p, c)
+ p += n istadv(p, n)
+ p + strlen(p) istend(p)
+ p[max] = 0 isttrim(p, max)
+ p[max+1] = 0 istzero(p, max)
diff --git a/doc/internals/api/layers.txt b/doc/internals/api/layers.txt
new file mode 100644
index 0000000..b5c35f4
--- /dev/null
+++ b/doc/internals/api/layers.txt
@@ -0,0 +1,190 @@
+2022-05-27 - Stream layers in HAProxy 2.6
+
+
+1. Background
+
+There are streams at plenty of levels in haproxy, essentially due to the
+introduction of multiplexed protocols which provide high-level streams on top
+of low-level streams, themselves either based on stream-oriented protocols or
+datagram-oriented protocols.
+
+The refactoring of the appctx and muxes that allowed to drop a lot of duplicate
+code between 2.5 and 2.6-dev6 raised another concern with some entities like
+"conn_stream" that were not specific to connections anymore, "endpoints" that
+became entities on their own, and "targets" whose life had been extended to
+last all along a connection.
+
+It was time to rename all such legacy entities introduced in 1.8 and which had
+turned particularly confusing over time as their roles evolved.
+
+
+2. Naming principles
+
+The global renaming of some entities between streams and connections was
+articulated around several principles:
+
+ - avoid the confusing use of "context" in shared places. For example, the
+ endpoint's connection is in "ctx" and nothing makes it obvious that the
+ endpoint's context is a connection, especially when an applet is there.
+
+ - reserve relative nouns for pointers and not for types. "endpoint", just
+ like "owner" or "peer" is relative, but when accessed from a different
+ layer it starts to make no sense at all, or to make one believe it's
+ something else, particularly with void*.
+
+ - avoid too generic terms that have multiple meanings, or words that are
+ synonyms in a same place (e.g. "peer" and "remote", or "endpoint" and
+ "target"). If two synonyms are needed to designate two distinct entities,
+ there's probably a problem elsewhere, or the problem is poorly defined.
+
+ - make it clearer that all that is manipulated is related to streams. This
+ particularly important in sample fetch functions for example, which tend
+ to require low-level access and could be mislead in trying to follow the
+ wrong chain when trying to get information about a connection.
+
+ - use easily spellable short names that abbreviate unambiguously when used
+ together in adjacent contexts
+
+
+3. Current state as of 2.6
+
+- when a name is required to designate the lower block that starts at the mux
+ stream or the appctx, it is spoken of as a "stream endpoint", and abbreviated
+ "se". It's okay because while "endpoint" itself is relative, "stream
+ endpoint" unequivocally designates one extremity of a stream. If a type is
+ needed for this in the future (e.g. via obj_type), then the type "stendp"
+ may be used. Before 2.6-dev6 there was no name for this, it was known as
+ conn_stream->ctx.
+
+- the 2.6-dev6 cs_endpoint which preserves the state of a mux stream or an
+ appctx and abstracts them in front of a conn_stream becomes a "stream
+ endpoint descriptor", of type "sedesc" and often abbreviated "sd", "sed"
+ or "ed". Its "target" pointer became "se" as per the rule above. Before
+ 2.6-dev6, these elements were mixed with others inside conn_stream. From
+ the appctx it's called "sedesc" (few occurrences hence long name OK).
+
+- the conn_stream which is always attached to either a stream or a health check
+ and that is used to reach a mux or an applet becomes a "stream connector" of
+ type "stconn", generally abbreviated "sc". Its "endp" pointer becomes
+ "sedesc" as per the rule above, and that one has a back pointer "sc". The
+ stream uses "scf" and "scb" as the respective front and back pointers to the
+ stconns. Prior to 2.6-dev6, these parts were split between conn_stream and
+ stream_interface.
+
+- the sedesc's "ctx" which is solely used to store the connection as of now, is
+ renamed "conn" to void any doubt in the context of applets or even muxes. In
+ the future the connection should be attached to the "se" instead and this
+ pointer should disappear (or be recycled for anything else).
+
+The new 2.6 model looks like this:
+
+ +------------------------+
+ | stream or health check |
+ +------------------------+
+ ^ \ scf, scb
+ / \
+ | |
+ \ /
+ app \ v
+ +----------+
+ | stconn |
+ +----------+
+ ^ \ sedesc
+ / \
+ . . . . | . . . | . . . . . split point (retries etc)
+ \ /
+ sc \ v
+ +----------+
+ flags <--| sedesc | : sedesc :
+ +----------+ ... +----------+
+ conn / ^ \ se ^ \
+ +------------+ / / \ | \
+ | connection |<--' | | ... OR ... | |
+ +------------+ \ / \ |
+ mux| ^ |ctx sd \ v : sedesc \ v
+ | | | +----------------------+ \ # +----------+ svcctx
+ | | | | mux stream or appctx | | # | appctx |--.
+ | | | +----------------------+ | # +----------+ |
+ | | | ^ | / private # : : |
+ v | | | v > to the # +----------+ |
+ mux_ops | | +----------------+ \ mux # | svcctx |<-'
+ | +---->| mux connection | ) # +----------+
+ +------ +----------------+ / #
+
+Stream descriptors may exist in the following modes:
+ - .conn = NULL, .se = NULL : backend, not connection attempt yet
+ - .conn = NULL, .se = <appctx> : frontend or backend, applet
+ - .conn = <conn>, .se = NULL : backend, connection in progress
+ - .conn = <conn>, .se = <muxs> : frontend or backend, connected
+
+Notes:
+ - for historical reasons (connect, forced protocol upgrades, etc), during a
+ connection setup or a rule-based protocol upgrade, the connection's "ctx"
+ may temporarily point to the stconn
+
+
+4. Invariants and cardinalities
+
+Usually a stream is created from an existing stconn from a mux or some applets,
+but may also be allocated first by other applets schedulers. After stream_new()
+a stream always has exactly one stconn per side (scf, scb), each of which has
+one ->sedesc. Each side is initialized with either one or no stream endpoint
+attached to the descriptor.
+
+Both applets and a mux stream always have a stream endpoint descriptor. AS SUCH
+IT IS NEVER NECESSARY TO TEST FOR THE EXISTENCE OF THE SEDESC FROM ANY SIDE, IT
+ALWAYS EXISTS. This explains why as much as possible it's preferable to use the
+sedesc to access flags and statuses from any side, rather than bouncing via the
+stconn.
+
+An applet's app layer is always a stream, which means that there are always
+channels accessible above, and there is always an opposite stream connector and
+a stream endpoint descriptor. As such, it's always safe for an applet to access
+the other side using sc_opposite().
+
+When an outgoing connection is in the process of being established, the backend
+side sedesc has its ->conn pointer pointing to the pending connection, and no
+->se. Once the connection is established and a mux is chosen, it's attached to
+the ->se. If an applet is used instead of a mux, the appctx is attached to the
+sedesc's ->se and ->conn remains NULL.
+
+If either side wants to detach from the other, it must allocate a new virgin
+sedesc to replace the existing one, and leave the existing one to the endpoint,
+since it continues to describe the stream endpoint. The stconn keeps its state
+(modulo the updates related to the disconnection). The previous sedesc points
+to a NULL stconn. For example, disconnecting from a backend mux will leave the
+entities like this:
+
+ +------------------------+
+ | stream or health check |
+ +------------------------+
+ ^ \ scf, scb
+ / \
+ | |
+ \ /
+ app \ v
+ +----------+
+ | stconn |
+ +----------+
+ ^ \ sedesc
+ / \
+ NULL | |
+ ^ \ /
+ sc | / sc \ v
+ +----------+ / +----------+
+ flags <--| sedesc1 | . . . . . | sedesc2 |--> flags
+ +----------+ / +----------+
+ conn / ^ \ se / conn / \ se
+ +------------+ / / \ | |
+ | connection |<--' | | v v
+ +------------+ \ / NULL NULL
+ mux| ^ |ctx sd \ v
+ | | | +----------------------+
+ | | | | mux stream or appctx |
+ | | | +----------------------+
+ | | | ^ |
+ v | | | v
+ mux_ops | | +----------------+
+ | +---->| mux connection |
+ +------ +----------------+
+
diff --git a/doc/internals/api/list.txt b/doc/internals/api/list.txt
new file mode 100644
index 0000000..d03cf03
--- /dev/null
+++ b/doc/internals/api/list.txt
@@ -0,0 +1,195 @@
+2021-11-09 - List API
+
+
+1. Background
+-------------
+
+HAProxy's lists are almost all doubly-linked and circular so that it is always
+possible to insert at the beginning, append at the end, scan them in any order
+and delete any element without having to scan to search the predecessor nor the
+successor.
+
+A list's head is just a regular list element, and an element always points to
+another list element. Such elements only have two pointers, the next and the
+previous elements. The object being pointed to is retrieved by subtracting the
+list element's offset in its structure from the list element's pointer. This
+way there is no need for any separate allocation for the list element, for a
+pointer to the object in the list, nor for a pointer to the list element from
+the object, as the list is embedded into the object.
+
+All basic operations are provided, as well as some iterators. Some iterators
+are safe for removal of the current element within the loop, others not. In any
+case a list cannot be freely modified while iterating over it (e.g. the current
+element's successor cannot not be freed if it's saved as the restart point).
+
+Extreme care is taken nowadays in HAProxy to make sure that no dangling
+pointers are left in elements, so it is important to always initialize list
+heads and list elements, as well as elements that are removed from a list if
+they are not immediately freed, so that their deletion is idempotent. A rule of
+thumb is that a list pointer's validity never has to be checked, it is always
+valid to dereference it. A lot of complex bugs have been caused in the past by
+incorrect list manipulation, such as an element being deleted twice, resulting
+in damaging previously adjacent elements' neighbours. This usually has serious
+consequences at locations that are totally different from the one of the bug,
+and that are only detected much later, so it is required to be particularly
+strict on using lists safely.
+
+The lists are not thread-safe, but mt_lists may be used instead.
+
+
+2. API description
+------------------
+
+A list is defined like this, both for the list's head, and for any other
+element:
+
+ struct list {
+ struct list *n; /* next */
+ struct list *p; /* prev */
+ };
+
+An empty list points to itself for both pointers. I.e. a list's head is both
+its own successor and its own predecessor. This guarantees that insertions
+and deletions can be done without any check and that deletion is idempotent.
+For this reason and by convention, a detached element ought to be represented
+like an empty head.
+
+Lists are manipulated using a set of macros which are used to initialize, add,
+remove, or iterate over elements. Most of these macros are extremely simple and
+are not even protected against multiple evaluation, so it is fundamentally
+important that the expressions used in the arguments are idempotent and that
+the result does not depend on the evaluation order of the arguments.
+
+Macro Description
+
+ILH
+ Initialized List Head : this is a non-NULL, non-empty list element used
+ to prevent the compiler from moving an empty list head declaration to
+ BSS, typically when it appears in an array of keywords Without this,
+ some older versions of gcc tend to trim all the array and cause
+ corruption.
+
+LIST_INIT(l)
+ Initialize the list as an empty list head
+
+LIST_HEAD_INIT(l)
+ Return a valid initialized empty list head pointing to this
+ element. Essentially used with assignments in declarations.
+
+LIST_INSERT(l, e)
+ Add an element at the beginning of a list and return it
+
+LIST_APPEND(l, e)
+ Add an element at the end of a list and return it
+
+LIST_SPLICE(n, o)
+ Add the contents of a list <o> at the beginning of another list <n>.
+ The old list head remains untouched.
+
+LIST_SPLICE_END_DETACHED(n, o)
+ Add the contents of a list whose first element is is <o> and last one
+ is <o->p> at the end of another list <n>. The old list DOES NOT have
+ any head here.
+
+LIST_DELETE(e)
+ Remove an element from a list and return it. Safe to call on
+ initialized elements, but will not change the element itself so it is
+ not idempotent. Consider using LIST_DEL_INIT() instead unless called
+ immediately after a free().
+
+LIST_DEL_INIT(e)
+ Remove an element from a list, initialize it and return it so that a
+ subsequent LIST_DELETE() is safe. This is faster than performing a
+ LIST_DELETE() followed by a LIST_INIT() as pointers are not reloaded.
+
+LIST_ELEM(l, t, m)
+ Return a pointer of type <t> to a structure containing a list head
+ member called <m> at address <l>. Note that <l> can be the result of a
+ function or macro since it's used only once.
+
+LIST_ISEMPTY(l)
+ Check if the list head <l> is empty (=initialized) or not, and return
+ non-zero only if so.
+
+LIST_INLIST(e)
+ Check if the list element <e> was added to a list or not, thus return
+ true unless the element was initialized.
+
+LIST_INLIST_ATOMIC(e)
+ Atomically check if the list element's next pointer points to anything
+ different from itself, implying the element should be part of a
+ list. This usually is similar to LIST_INLIST() except that while that
+ one might be instrumented using debugging code to perform further
+ consistency checks, the macro below guarantees to always perform a
+ single atomic test and is safe to use with barriers.
+
+LIST_NEXT(l, t, m)
+ Return a pointer of type <t> to a structure following the element which
+ contains list head <l>, which is known as member <m> in struct <t>.
+
+LIST_PREV(l, t, m)
+ Return a pointer of type <t> to a structure preceding the element which
+ contains list head <l>, which is known as member <m> in struct <t>.
+ Note that this macro is first undefined as it happened to already exist
+ on some old OSes.
+
+list_for_each_entry(i, l, m)
+ Iterate local variable <i> through a list of items of type "typeof(*i)"
+ which are linked via a "struct list" member named <m>. A pointer to the
+ head of the list is passed in <l>. No temporary variable is needed.
+ Note that <i> must not be modified during the loop.
+
+list_for_each_entry_from(i, l, m)
+ Same as list_for_each_entry() but starting from current value of <i>
+ instead of the list's head.
+
+list_for_each_entry_from_rev(i, l, m)
+ Same as list_for_each_entry_rev() but starting from current value of <i>
+ instead of the list's head.
+
+list_for_each_entry_rev(i, l, m)
+ Iterate backwards local variable <i> through a list of items of type
+ "typeof(*i)" which are linked via a "struct list" member named <m>. A
+ pointer to the head of the list is passed in <l>. No temporary variable
+ is needed. Note that <i> must not be modified during the loop.
+
+list_for_each_entry_safe(i, b, l, m)
+ Iterate variable <i> through a list of items of type "typeof(*i)" which
+ are linked via a "struct list" member named <m>. A pointer to the head
+ of the list is passed in <l>. A temporary backup variable <b> of same
+ type as <i> is needed so that <i> may safely be deleted if needed. Note
+ that it is only permitted to delete <i> and no other element during
+ this operation!
+
+list_for_each_entry_safe_from(i, b, l, m)
+ Same as list_for_each_entry_safe() but starting from current value of
+ <i> instead of the list's head.
+
+list_for_each_entry_safe_from_rev(i, b, l, m)
+ Same as list_for_each_entry_safe_rev() but starting from current value
+ of <i> instead of the list's head.
+
+list_for_each_entry_safe_rev(i, b, l, m)
+ Iterate backwards local variable <i> through a list of items of type
+ "typeof(*i)" which are linked via a "struct list" member named <m>. A
+ pointer to the head of the list is passed in <l>. A temporary variable
+ <b> of same type as <i> is needed so that <i> may safely be deleted if
+ needed. Note that it is only permitted to delete <i> and no other
+ element during this operation!
+
+3. Notes
+--------
+
+- This API is quite old and some macros are missing. For example there's still
+ no list_first() so it's common to use LIST_ELEM(head->n, ...) instead. Some
+ older parts of the code also used to rely on list_for_each() followed by a
+ break to stop on the first element.
+
+- Some parts were recently renamed because LIST_ADD() used to do what
+ LIST_INSERT() currently does and was often mistaken with LIST_ADDQ() which is
+ what LIST_APPEND() now is. As such it is not totally impossible that some
+ places use a LIST_INSERT() where a LIST_APPEND() would be desired.
+
+- The structure must not be modified at all (even to add debug info). Some
+ parts of the code assume that its layout is exactly this one, particularly
+ the parts ensuring the casting between MT lists and lists.
diff --git a/doc/internals/api/pools.txt b/doc/internals/api/pools.txt
new file mode 100644
index 0000000..d84fb9d
--- /dev/null
+++ b/doc/internals/api/pools.txt
@@ -0,0 +1,585 @@
+2022-02-24 - Pools structure and API
+
+1. Background
+-------------
+
+Memory allocation is a complex problem covered by a massive amount of
+literature. Memory allocators found in field cover a broad spectrum of
+capabilities, performance, fragmentation, efficiency etc.
+
+The main difficulty of memory allocation comes from finding the optimal chunks
+for arbitrary sized requests, that will still preserve a low fragmentation
+level. Doing this well is often expensive in CPU usage and/or memory usage.
+
+In programs like HAProxy that deal with a large number of fixed size objects,
+there is no point having to endure all this risk of fragmentation, and the
+associated costs (sometimes up to several milliseconds with certain minimalist
+allocators) are simply not acceptable. A better approach consists in grouping
+frequently used objects by size, knowing that due to the high repetitiveness of
+operations, a freed object will immediately be needed for another operation.
+
+This grouping of objects by size is what is called a pool. Pools are created
+for certain frequently allocated objects, are usually merged together when they
+are of the same size (or almost the same size), and significantly reduce the
+number of calls to the memory allocator.
+
+With the arrival of threads, pools started to become a bottleneck so they now
+implement an optional thread-local lockless cache. Finally with the arrival of
+really efficient memory allocator in modern operating systems, the shared part
+has also become optional so that it doesn't consume memory if it does not bring
+any value.
+
+In 2.6-dev2, a number of debugging options that used to be configured at build
+time only changed to boot-time and can be modified using keywords passed after
+"-dM" on the command line, which sets or clears bits in the pool_debugging
+variable. The build-time options still affect the default settings however.
+Default values may be consulted using "haproxy -dMhelp".
+
+
+2. Principles
+-------------
+
+The pools architecture is selected at build time. The main options are:
+
+ - thread-local caches and process-wide shared pool enabled (1)
+
+ This is the default situation on most operating systems. Each thread has
+ its own local cache, and when depleted it refills from the process-wide
+ pool that avoids calling the standard allocator too often. It is possible
+ to force this mode at build time by setting CONFIG_HAP_GLOBAL_POOLS or at
+ boot time with "-dMglobal".
+
+ - thread-local caches only are enabled (2)
+
+ This is the situation on operating systems where a fast and modern memory
+ allocator is detected and when it is estimated that the process-wide shared
+ pool will not bring any benefit. This detection is automatic at build time,
+ but may also be forced at build tmie by setting CONFIG_HAP_NO_GLOBAL_POOLS
+ or at boot time with "-dMno-global".
+
+ - pass-through to the standard allocator (3)
+
+ This is used when one absolutely wants to disable pools and rely on regular
+ malloc() and free() calls, essentially in order to trace memory allocations
+ by call points, either internally via DEBUG_MEM_STATS, or externally via
+ tools such as Valgrind. This mode of operation may be forced at build time
+ by setting DEBUG_NO_POOLS or at boot time with "-dMno-cache".
+
+ - pass-through to an mmap-based allocator for debugging (4)
+
+ This is used only during deep debugging when trying to detect various
+ conditions such as use-after-free. In this case each allocated object's
+ size is rounded up to a multiple of a page size (4096 bytes) and an
+ integral number of pages is allocated for each object using mmap(),
+ surrounded by two unaccessible holes that aim to detect some out-of-bounds
+ accesses. Released objects are instantly freed using munmap() so that any
+ immediate subsequent access to the memory area crashes the process if the
+ area had not been reallocated yet. This mode can be enabled at build time
+ by setting DEBUG_UAF, or at run time by disabling pools and enabling UAF
+ with "-dMuaf". It tends to consume a lot of memory and not to scale at all
+ with concurrent calls, that tends to make the system stall. The watchdog
+ may even trigger on some slow allocations.
+
+There are no more provisions for running with a shared pool but no thread-local
+cache: the shared pool's main goal is to compensate for the expensive calls to
+the memory allocator. This gain may be huge on tiny systems using basic
+allocators, but the thread-local cache will already achieve this. And on larger
+threaded systems, the shared pool's benefit is visible when the underlying
+allocator scales poorly, but in this case the shared pool would suffer from
+the same limitations without its thread-local cache and wouldn't provide any
+benefit.
+
+Summary of the various operation modes:
+
+ (1) (2) (3) (4)
+
+ User User User User
+ | | | |
+ pool_alloc() V V | |
+ +---------+ +---------+ | |
+ | Thread | | Thread | | |
+ | Local | | Local | | |
+ | Cache | | Cache | | |
+ +---------+ +---------+ | |
+ | | | |
+ pool_refill*() V | | |
+ +---------+ | | |
+ | Shared | | | |
+ | Pool | | | |
+ +---------+ | | |
+ | | | |
+ malloc() V V V |
+ +---------+ +---------+ +---------+ |
+ | Library | | Library | | Library | |
+ +---------+ +---------+ +---------+ |
+ | | | |
+ mmap() V V V V
+ +---------+ +---------+ +---------+ +---------+
+ | OS | | OS | | OS | | OS |
+ +---------+ +---------+ +---------+ +---------+
+
+One extra build define, DEBUG_FAIL_ALLOC, is used to enforce random allocation
+failure in pool_alloc() by randomly returning NULL, to test that callers
+properly handle allocation failures. It may also be enabled at boot time using
+"-dMfail". In this case the desired average rate of allocation failures can be
+fixed by global setting "tune.fail-alloc" expressed in percent.
+
+The thread-local caches contain the freshest objects. Its total size amounts to
+the number of bytes set in global.tune.pool_cache_size and that may be adjusted
+by the "tune.memory.hot-size" global option, which itself defaults to build
+time setting CONFIG_HAP_POOL_CACHE_SIZE, which was 1MB before 2.6 and 512kB
+after. The aim is to keep hot objects that still fit in the CPU core's private
+L2 cache. Once these objects do not fit into the cache anymore, there's no
+benefit keeping them local to the thread, so they'd rather be returned to the
+shared pool or the main allocator so that any other thread may make use of
+them. Under extreme thread contention the cost of accessing shared structures
+in the global cache or in malloc() may still be important and it may prove
+useful to increase the thread-local cache size.
+
+
+3. Storage in thread-local caches
+---------------------------------
+
+This section describes how objects are linked in thread local caches. This is
+not meant to be a concern for users of the pools API but it can be useful when
+inspecting post-mortem dumps or when trying to figure certain size constraints.
+
+Objects are stored in the local cache using a doubly-linked list. This ensures
+that they can be visited by freshness order like a stack, while at the same
+time being able to access them from oldest to newest when it is needed to
+evict coldest ones first:
+
+ - releasing an object to the cache always puts it on the top.
+
+ - allocating an object from the cache always takes the topmost one, hence the
+ freshest one.
+
+ - scanning for older objects to evict starts from the bottom, where the
+ oldest ones are located
+
+To that end, each thread-local cache keeps a list head in the "list" member of
+its "pool_cache_head" descriptor, that links all objects cast to type
+"pool_cache_item" via their "by_pool" member.
+
+Note that the mechanism described above only works for a single pool. When
+trying to limit the total cache size to a certain value, all pools included,
+there is also a need to arrange all objects from all pools together in the
+local caches. For this, each thread_ctx maintains a list head of recently
+released objects, all pools included, in its member "pool_lru_head". All items
+in a thread-local cache are linked there via their "by_lru" member.
+
+This means that releasing an object using pool_free() consists in inserting
+it at the beginning of two lists:
+ - the local pool_cache_head's "list" list head
+ - the thread context's "pool_lru_head" list head
+
+Allocating an object consists in picking the first entry from the pool's "list"
+and deleting its "by_pool" and "by_lru" links.
+
+Evicting an object consists in scanning the thread context's "pool_lru_head"
+backwards and deleting the object's "by_pool" and "by_lru" links.
+
+Given that entries are both inserted and removed synchronously, we have the
+guarantee that the oldest object in the thread's LRU list is always the oldest
+object in its pool, and that the next element is the cache's list head. This is
+what allows the LRU eviction mechanism to figure what pool an object belongs to
+when releasing it.
+
+Note:
+ | Since a pool_cache_item has two list entries, on 64-bit systems it will be
+ | 32-bytes long. This is the smallest size that a pool may be, and any smaller
+ | size will automatically be rounded up to this size.
+
+When build option DEBUG_POOL_INTEGRITY is set, or the boot-time option
+"-dMintegrity" is passed on the command line, the area of the object between
+the two list elements and the end according to pool->size will be filled with
+pseudo-random words during pool_put_to_cache(), and these words will be
+compared between each other during pool_get_from_cache(), and the process will
+crash in case any bit differs, as this would indicate that the memory area was
+modified after the free. The pseudo-random pattern is in fact incremented by
+(~0)/3 upon each free so that roughly half of the bits change each time and we
+maximize the likelihood of detecting a single bit flip in either direction. In
+order to avoid an immediate reuse and maximize the time the object spends in
+the cache, when this option is set, objects are picked from the cache from the
+oldest one instead of the freshest one. This way even late memory corruptions
+have a chance to be detected.
+
+When build option DEBUG_MEMORY_POOLS is set, or the boot-time option "-dMtag"
+is passed on the executable's command line, pool objects are allocated with
+one extra pointer compared to the requested size, so that the bytes that follow
+the memory area point to the pool descriptor itself as long as the object is
+allocated via pool_alloc(). Upon releasing via pool_free(), the pointer is
+compared and the code will crash in if it differs. This allows to detect both
+memory overflows and object released to the wrong pool (code bug resulting from
+a copy-paste error typically).
+
+Thus an object will look like this depending whether it's in the cache or is
+currently in use:
+
+ in cache in use
+ +------------+ +------------+
+ <--+ by_pool.p | | N bytes |
+ | by_pool.n +--> | |
+ +------------+ |N=16 min on |
+ <--+ by_lru.p | | 32-bit, |
+ | by_lru.n +--> | 32 min on |
+ +------------+ | 64-bit |
+ : : : :
+ | N bytes | | |
+ +------------+ +------------+ \ optional, only if
+ : (unused) : : pool ptr : > DEBUG_MEMORY_POOLS
+ +------------+ +------------+ / is set at build time
+ or -dMtag at boot time
+
+Right now no provisions are made to return objects aligned on larger boundaries
+than those currently covered by malloc() (i.e. two pointers). This need appears
+from time to time and the layout above might evolve a little bit if needed.
+
+
+4. Storage in the process-wide shared pool
+------------------------------------------
+
+In order for the shared pool not to be a contention point in a multi-threaded
+environment, objects are allocated from or released to shared pools by clusters
+of a few objects at once. The maximum number of objects that may be moved to or
+from a shared pool at once is defined by CONFIG_HAP_POOL_CLUSTER_SIZE at build
+time, and currently defaults to 8.
+
+In order to remain scalable, the shared pool has to make some tradeoffs to
+limit the number of atomic operations and the duration of any locked operation.
+As such, it's composed of a single-linked list of clusters, themselves made of
+a single-linked list of objects.
+
+Clusters and objects are of the same type "pool_item" and are accessed from the
+pool's "free_list" member. This member points to the latest pool_item inserted
+into the pool by a release operation. And the pool_item's "next" member points
+to the next pool_item, which was the one present in the pool's free_list just
+before the pool_item was inserted, and the last pool_item in the list simply
+has a NULL "next" field.
+
+The pool_item's "down" pointer points down to the next objects part of the same
+cluster, that will be released or allocated at the same time as the first one.
+Each of these items also has a NULL "next" field, and are chained by their
+respective "down" pointers until the last one is detected by a NULL value.
+
+This results in the following layout:
+
+ pool pool_item pool_item pool_item
+ +-----------+ +------+ +------+ +------+
+ | free_list +--> | next +--> | next +--> | NULL |
+ +-----------+ +------+ +------+ +------+
+ | down | | NULL | | down |
+ +--+---+ +------+ +--+---+
+ | |
+ V V
+ +------+ +------+
+ | NULL | | NULL |
+ +------+ +------+
+ | down | | NULL |
+ +--+---+ +------+
+ |
+ V
+ +------+
+ | NULL |
+ +------+
+ | NULL |
+ +------+
+
+Allocating an entry is only a matter of performing two atomic allocations on
+the free_list and reading the pool's "next" value:
+
+ - atomically mark the free_list as being updated by writing a "magic" pointer
+ - read the first pool_item's "next" field
+ - atomically replace the free_list with this value
+
+This results in a fast operation that instantly retrieves a cluster at once.
+Then outside of the critical section entries are walked over and inserted into
+the local cache one at a time. In order to keep the code simple and efficient,
+objects allocated from the shared pool are all placed into the local cache, and
+only then the first one is allocated from the cache. This operation is
+performed by the dedicated function pool_refill_local_from_shared() which is
+called from pool_get_from_cache() when the cache is empty. It means there is an
+overhead of two list insert/delete operations for the first object and that
+could be avoided at the expense of more complex code in the fast path, but this
+is negligible since it only concerns objects that need to be visited anyway.
+
+Freeing a group of objects consists in performing the operation the other way
+around:
+
+ - atomically mark the free_list as being updated by writing a "magic" pointer
+ - write the free_list value to the to-be-released item's "next" entry
+ - atomically replace the free_list with the pool_item's pointer
+
+The cluster will simply have to be prepared before being sent to the shared
+pool. The operation of releasing a cluster at once is performed by function
+pool_put_to_shared_cache() which is called from pool_evict_last_items() which
+itself is responsible for building the clusters.
+
+Due to the way objects are stored, it is important to try to group objects as
+much as possible when releasing them because this is what will condition their
+retrieval as groups as well. This is the reason why pool_evict_last_items()
+uses the LRU to find a first entry but tries to pick several items at once from
+a single cache. Tests have shown that CONFIG_HAP_POOL_CLUSTER_SIZE set to 8
+achieves up to 6-6.5 objects on average per operation, which effectively
+divides by as much the average time spent per object by each thread and pushes
+the contention point further.
+
+Also, grouping items in clusters is a property of the process-wide shared pool
+and not of the thread-local caches. This means that there is no grouped
+operation when not using the shared pool (mode "2" in the diagram above).
+
+
+5. API
+------
+
+The following functions are public and available for user code:
+
+struct pool_head *create_pool(char *name, uint size, uint flags)
+ Create a new pool named <name> for objects of size <size> bytes. Pool
+ names are truncated to their first 11 characters. Pools of very similar
+ size will usually be merged if both have set the flag MEM_F_SHARED in
+ <flags>. When DEBUG_DONT_SHARE_POOLS was set at build time, or
+ "-dMno-merge" is passed on the executable's command line, the pools
+ also need to have the exact same name to be merged. In addition, unless
+ MEM_F_EXACT is set in <flags>, the object size will usually be rounded
+ up to the size of pointers (16 or 32 bytes). The name that will appear
+ in the pool upon merging is the name of the first created pool. The
+ returned pointer is the new (or reused) pool head, or NULL upon error.
+ Pools created this way must be destroyed using pool_destroy().
+
+void *pool_destroy(struct pool_head *pool)
+ Destroy pool <pool>, that is, all of its unused objects are freed and
+ the structure is freed as well if the pool didn't have any used objects
+ anymore. In this case NULL is returned. If some objects remain in use,
+ the pool is preserved and its pointer is returned. This ought to be
+ used essentially on exit or in rare situations where some internal
+ entities that hold pools have to be destroyed.
+
+void pool_destroy_all(void)
+ Destroy all pools, without checking which ones still have used entries.
+ This is only meant for use on exit.
+
+void *__pool_alloc(struct pool_head *pool, uint flags)
+ Allocate an entry from the pool <pool>. The allocator will first look
+ for an object in the thread-local cache if enabled, then in the shared
+ pool if enabled, then will fall back to the operating system's default
+ allocator. NULL is returned if the object couldn't be allocated (due to
+ configured limits or lack of memory). Object allocated this way have to
+ be released using pool_free(). Like with malloc(), by default the
+ contents of the returned object are undefined. If memory poisonning is
+ enabled, the object will be filled with the poisonning byte. If the
+ global "pool.fail-alloc" setting is non-zero and DEBUG_FAIL_ALLOC is
+ enabled, a random number generator will be called to randomly return a
+ NULL. The allocator's behavior may be adjusted using a few flags passed
+ in <flags>:
+ - POOL_F_NO_POISON : when set, disables memory poisonning (e.g. when
+ pointless and expensive, like for buffers)
+ - POOL_F_MUST_ZERO : when set, the memory area will be zeroed before
+ being returned, similar to what calloc() does
+ - POOL_F_NO_FAIL : when set, disables the random allocation failure,
+ e.g. for use during early init code or critical sections.
+
+void *pool_alloc(struct pool_head *pool)
+ This is an exact equivalent of __pool_alloc(pool, 0). It is the regular
+ way to allocate entries from a pool.
+
+void *pool_alloc_nocache(struct pool_head *pool)
+ Allocate an entry from the pool <pool>, bypassing the cache. If shared
+ pools are enabled, they will be consulted first. Otherwise the object
+ is allocated using the operating system's default allocator. This is
+ essentially used during early boot to pre-allocate a number of objects
+ for pools which require a minimum number of entries to exist.
+
+void *pool_zalloc(struct pool_head *pool)
+ This is an exact equivalent of __pool_alloc(pool, POOL_F_MUST_ZERO).
+
+void pool_free(struct pool_head *pool, void *ptr)
+ Free an entry allocate from one of the pool_alloc() functions above
+ from pool <pool>. The object will be placed into the thread-local cache
+ if enabled, or in the shared pool if enabled, or will be released using
+ the operating system's default allocator. When a local cache is
+ enabled, if the local cache size becomes larger than 75% of the maximum
+ size configured at build time, some objects will be evicted to the
+ shared pool. Such objects are taken first from the same pool, but if
+ the total size is really huge, other pools might be checked as well.
+ Some extra checks enabled at build time may enforce extra checks so
+ that the process will immediately crash if the object was not allocated
+ from this pool or experienced an overflow or some memory corruption.
+
+void pool_flush(struct pool_head *pool)
+ Free all unused objects from shared pool <pool>. Thread-local caches
+ are not affected. This is essentially used when running low on memory
+ or when stopping, in order to release a maximum amount of memory for
+ the new process.
+
+void pool_gc(struct pool_head *pool)
+ Free all unused objects from all pools, but respecting the minimum
+ number of spare objects required for each of them. Then, for operating
+ systems which support it, indicate the system that all unused memory
+ can be released. Thread-local caches are not affected. This operation
+ differs from pool_flush() in that it is run locklessly, under thread
+ isolation, and on all pools in a row. It is called by the SIGQUIT
+ signal handler and upon exit. Note that the obsolete argument <pool> is
+ not used and the convention is to pass NULL there.
+
+void dump_pools_to_trash(void)
+ Dump the current status of all pools into the trash buffer. This is
+ essentially used by the "show pools" CLI command or the SIGQUIT signal
+ handler to dump them on stderr. The total report size may not exceed
+ the size of the trash buffer. If it does, some entries will be missing.
+
+void dump_pools(void)
+ Dump the current status of all pools to stderr. This just calls
+ dump_pools_to_trash() and writes the trash to stderr.
+
+int pool_total_failures(void)
+ Report the total number of failed allocations. This is solely used to
+ report the "PoolFailed" metrics of the "show info" output. The total
+ is calculated on the fly by summing the number of failures in all pools
+ and is only meant to be used as an indicator rather than a precise
+ measure.
+
+ullong pool_total_allocated(void)
+ Report the total number of bytes allocated in all pools, for reporting
+ in the "PoolAlloc_MB" field of the "show info" output. The total is
+ calculated on the fly by summing the number of allocated bytes in all
+ pools and is only meant to be used as an indicator rather than a
+ precise measure.
+
+ullong pool_total_used(void)
+ Report the total number of bytes used in all pools, for reporting in
+ the "PoolUsed_MB" field of the "show info" output. The total is
+ calculated on the fly by summing the number of used bytes in all pools
+ and is only meant to be used as an indicator rather than a precise
+ measure. Note that objects present in caches are accounted as used.
+
+Some other functions exist and are only used by the pools code itself. While
+not strictly forbidden to use outside of this code, it is generally recommended
+to avoid touching them in order not to create undesired dependencies that will
+complicate maintenance.
+
+A few macros exist to ease the declaration of pools:
+
+DECLARE_POOL(ptr, name, size)
+ Placed at the top level of a file, this declares a global memory pool
+ as variable <ptr>, name <name> and size <size> bytes per element. This
+ is made via a call to REGISTER_POOL() and by assigning the resulting
+ pointer to variable <ptr>. <ptr> will be created of type "struct
+ pool_head *". If the pool needs to be visible outside of the function
+ (which is likely), it will also need to be declared somewhere as
+ "extern struct pool_head *<ptr>;". It is recommended to place such
+ declarations very early in the source file so that the variable is
+ already known to all subsequent functions which may use it.
+
+DECLARE_STATIC_POOL(ptr, name, size)
+ Placed at the top level of a file, this declares a static memory pool
+ as variable <ptr>, name <name> and size <size> bytes per element. This
+ is made via a call to REGISTER_POOL() and by assigning the resulting
+ pointer to local variable <ptr>. <ptr> will be created of type "static
+ struct pool_head *". It is recommended to place such declarations very
+ early in the source file so that the variable is already known to all
+ subsequent functions which may use it.
+
+
+6. Build options
+----------------
+
+A number of build-time defines allow to tune the pools behavior. All of them
+have to be enabled using "-Dxxx" or "-Dxxx=yyy" in the makefile's DEBUG
+variable.
+
+DEBUG_NO_POOLS
+ When this is set, pools are entirely disabled, and allocations are made
+ using malloc() instead. This is not recommended for production but may
+ be useful for tracing allocations. It corresponds to "-dMno-cache" at
+ boot time.
+
+DEBUG_MEMORY_POOLS
+ When this is set, an extra pointer is allocated at the end of each
+ object to reference the pool the object was allocated from and detect
+ buffer overflows. Then, pool_free() will provoke a crash in case it
+ detects an anomaly (pointer at the end not matching the pool). It
+ corresponds to "-dMtag" at boot time.
+
+DEBUG_FAIL_ALLOC
+ When enabled, a global setting "tune.fail-alloc" may be set to a non-
+ zero value representing a percentage of memory allocations that will be
+ made to fail in order to stress the calling code. It corresponds to
+ "-dMfail" at boot time.
+
+DEBUG_DONT_SHARE_POOLS
+ When enabled, pools of similar sizes are not merged unless the have the
+ exact same name. It corresponds to "-dMno-merge" at boot time.
+
+DEBUG_UAF
+ When enabled, pools are disabled and all allocations and releases pass
+ through mmap() and munmap(). The memory usage significantly inflates
+ and the performance degrades, but this allows to detect a lot of
+ use-after-free conditions by crashing the program at the first abnormal
+ access. This should not be used in production. It corresponds to
+ boot-time options "-dMuaf". Caching is disabled but may be re-enabled
+ using "-dMcache".
+
+DEBUG_POOL_INTEGRITY
+ When enabled, objects picked from the cache are checked for corruption
+ by comparing their contents against a pattern that was placed when they
+ were inserted into the cache. Objects are also allocated in the reverse
+ order, from the oldest one to the most recent, so as to maximize the
+ ability to detect such a corruption. The goal is to detect writes after
+ free (or possibly hardware memory corruptions). Contrary to DEBUG_UAF
+ this cannot detect reads after free, but may possibly detect later
+ corruptions and will not consume extra memory. The CPU usage will
+ increase a bit due to the cost of filling/checking the area and for the
+ preference for cold cache instead of hot cache, though not as much as
+ with DEBUG_UAF. This option is meant to be usable in production. It
+ corresponds to boot-time options "-dMcold-first,integrity".
+
+DEBUG_POOL_TRACING
+ When enabled, the callers of pool_alloc() and pool_free() will be
+ recorded into an extra memory area placed after the end of the object.
+ This may only be required by developers who want to get a few more
+ hints about code paths involved in some crashes, but will serve no
+ purpose outside of this. It remains compatible (and completes well)
+ DEBUG_POOL_INTEGRITY above. Such information become meaningless once
+ the objects leave the thread-local cache. It corresponds to boot-time
+ option "-dMcaller".
+
+DEBUG_MEM_STATS
+ When enabled, all malloc/calloc/realloc/strdup/free calls are accounted
+ for per call place (file+line number), and may be displayed or reset on
+ the CLI using "debug dev memstats". This is essentially used to detect
+ potential leaks or abnormal usages. When pools are enabled (default),
+ such calls are rare and the output will mostly contain calls induced by
+ libraries. When pools are disabled, about all calls to pool_alloc() and
+ pool_free() will also appear since they will be remapped to standard
+ functions.
+
+CONFIG_HAP_GLOBAL_POOLS
+ When enabled, process-wide shared pools will be forcefully enabled even
+ if not considered useful on the platform. The default is to let haproxy
+ decide based on the OS and C library. It corresponds to boot-time
+ option "-dMglobal".
+
+CONFIG_HAP_NO_GLOBAL_POOLS
+ When enabled, process-wide shared pools will be forcefully disabled
+ even if considered useful on the platform. The default is to let
+ haproxy decide based on the OS and C library. It corresponds to
+ boot-time option "-dMno-global".
+
+CONFIG_HAP_POOL_CACHE_SIZE
+ This allows one to define the default size of the per-thread cache, in
+ bytes. The default value is 512 kB (524288). Smaller values will use
+ less memory at the expense of a possibly higher CPU usage when using
+ many threads. Higher values will give diminishing returns on
+ performance while using much more memory. Usually there is no benefit
+ in using more than a per-core L2 cache size. It would be better not to
+ set this value lower than a few times the size of a buffer (bufsize,
+ defaults to 16 kB). In addition, keep in mind that this option may be
+ changed at runtime using "tune.memory.hot-size".
+
+CONFIG_HAP_POOL_CLUSTER_SIZE
+ This allows one to define the maximum number of objects that will be
+ groupped together in an allocation from the shared pool. Values 4 to 8
+ have experimentally shown good results with 16 threads. On systems with
+ more cores or loosely coupled caches exhibiting slow atomic operations,
+ it could possibly make sense to slightly increase this value.
diff --git a/doc/internals/api/scheduler.txt b/doc/internals/api/scheduler.txt
new file mode 100644
index 0000000..dd1ad5f
--- /dev/null
+++ b/doc/internals/api/scheduler.txt
@@ -0,0 +1,228 @@
+2021-11-17 - Scheduler API
+
+
+1. Background
+-------------
+
+The scheduler relies on two major parts:
+ - the wait queue or timers queue, which contains an ordered tree of the next
+ timers to expire
+
+ - the run queue, which contains tasks that were already woken up and are
+ waiting for a CPU slot to execute.
+
+There are two types of schedulable objects in HAProxy:
+ - tasks: they contain one timer and can be in the run queue without leaving
+ their place in the timers queue.
+
+ - tasklets: they do not have the timers part and are either sleeping or
+ running.
+
+Both the timers queue and run queue in fact exist both shared between all
+threads and per-thread. A task or tasklet may only be queued in a single of
+each at a time. The thread-local queues are not thread-safe while the shared
+ones are. This means that it is only permitted to manipulate an object which
+is in the local queue or in a shared queue, but then after locking it. As such
+tasks and tasklets are usually pinned to threads and do not move, or only in
+very specific ways not detailed here.
+
+In case of doubt, keep in mind that it's not permitted to manipulate another
+thread's private task or tasklet, and that any task held by another thread
+might vanish while it's being looked at.
+
+Internally a large part of the task and tasklet struct is shared between
+the two types, which reduces code duplication and eases the preservation
+of fairness in the run queue by interleaving all of them. As such, some
+fields or flags may not always be relevant to tasklets and may be ignored.
+
+
+Tasklets do not use a thread mask but use a thread ID instead, to which they
+are bound. If the thread ID is negative, the tasklet is not bound but may only
+be run on the calling thread.
+
+
+2. API
+------
+
+There are few functions exposed by the scheduler. A few more ones are in fact
+accessible but if not documented there they'd rather be avoided or used only
+when absolutely certain they're suitable, as some have delicate corner cases.
+In doubt, checking the sched.pdf diagram may help.
+
+int total_run_queues()
+ Return the approximate number of tasks in run queues. This is racy
+ and a bit inaccurate as it iterates over all queues, but it is
+ sufficient for stats reporting.
+
+int task_in_rq(t)
+ Return non-zero if the designated task is in the run queue (i.e. it was
+ already woken up).
+
+int task_in_wq(t)
+ Return non-zero if the designated task is in the timers queue (i.e. it
+ has a valid timeout and will eventually expire).
+
+int thread_has_tasks()
+ Return non-zero if the current thread has some work to be done in the
+ run queue. This is used to decide whether or not to sleep in poll().
+
+void task_wakeup(t, f)
+ Will make sure task <t> will wake up, that is, will execute at least
+ once after the start of the function is called. The task flags <f> will
+ be ORed on the task's state, among TASK_WOKEN_* flags exclusively. In
+ multi-threaded environments it is safe to wake up another thread's task
+ and even if the thread is sleeping it will be woken up. Users have to
+ keep in mind that a task running on another thread might very well
+ finish and go back to sleep before the function returns. It is
+ permitted to wake the current task up, in which case it will be
+ scheduled to run another time after it returns to the scheduler.
+
+struct task *task_unlink_wq(t)
+ Remove the task from the timers queue if it was in it, and return it.
+ It may only be done for the local thread, or for a shared thread that
+ might be in the shared queue. It must not be done for another thread's
+ task.
+
+void task_queue(t)
+ Place or update task <t> into the timers queue, where it may already
+ be, scheduling it for an expiration at date t->expire. If t->expire is
+ infinite, nothing is done, so it's safe to call this function without
+ prior checking the expiration date. It is only valid to call this
+ function for local tasks or for shared tasks who have the calling
+ thread in their thread mask.
+
+void task_set_thread(t, id)
+ Change task <t>'s thread ID to new value <id>. This may only be
+ performed by the task itself while running. This is only used to let a
+ task voluntarily migrate to another thread. Thread id -1 is used to
+ indicate "any thread". It's ignored and replaced by zero when threads
+ are disabled.
+
+void tasklet_wakeup(tl)
+ Make sure that tasklet <tl> will wake up, that is, will execute at
+ least once. The tasklet will run on its assigned thread, or on any
+ thread if its TID is negative.
+
+void tasklet_wakeup_on(tl, thr)
+ Make sure that tasklet <tl> will wake up on thread <thr>, that is, will
+ execute at least once. The designated thread may only differ from the
+ calling one if the tasklet is already configured to run on another
+ thread, and it is not permitted to self-assign a tasklet if its tid is
+ negative, as it may already be scheduled to run somewhere else. Just in
+ case, only use tasklet_wakeup() which will pick the tasklet's assigned
+ thread ID.
+
+struct tasklet *tasklet_new()
+ Allocate a new tasklet and set it to run by default on the calling
+ thread. The caller may change its tid to another one before using it.
+ The new tasklet is returned.
+
+struct task *task_new_anywhere()
+ Allocate a new task to run on any thread, and return the task, or NULL
+ in case of allocation issue. Note that such tasks will be marked as
+ shared and will go through the locked queues, thus their activity will
+ be heavier than for other ones. See also task_new_here().
+
+struct task *task_new_here()
+ Allocate a new task to run on the calling thread, and return the task,
+ or NULL in case of allocation issue.
+
+struct task *task_new_on(t)
+ Allocate a new task to run on thread <t>, and return the task, or NULL
+ in case of allocation issue.
+
+void task_destroy(t)
+ Destroy this task. The task will be unlinked from any timers queue,
+ and either immediately freed, or asynchronously killed if currently
+ running. This may only be done by one of the threads this task is
+ allowed to run on. Developers must not forget that the task's memory
+ area is not always immediately freed, and that certain misuses could
+ only have effect later down the chain (e.g. use-after-free).
+
+void tasklet_free()
+ Free this tasklet, which must not be running, so that may only be
+ called by the thread responsible for the tasklet, typically the
+ tasklet's process() function itself.
+
+void task_schedule(t, d)
+ Schedule task <t> to run no later than date <d>. If the task is already
+ running, or scheduled for an earlier instant, nothing is done. If the
+ task was not in queued or was scheduled to run later, its timer entry
+ will be updated. This function assumes that it will never be called
+ with a timer in the past nor with TICK_ETERNITY. Only one of the
+ threads assigned to the task may call this function.
+
+The task's ->process() function receives the following arguments:
+
+ - struct task *t: a pointer to the task itself. It is always valid.
+
+ - void *ctx : a copy of the task's ->context pointer at the moment
+ the ->process() function was called by the scheduler. A
+ function must use this and not task->context, because
+ task->context might possibly be changed by another thread.
+ For instance, the muxes' takeover() function do this.
+
+ - uint state : a copy of the task's ->state field at the moment the
+ ->process() function was executed. A function must use
+ this and not task->state as the latter misses the wakeup
+ reasons and may constantly change during execution along
+ concurrent wakeups (threads or signals).
+
+The possible state flags to use during a call to task_wakeup() or seen by the
+task being called are the following; they're automatically cleaned from the
+state field before the call to ->process()
+
+ - TASK_WOKEN_INIT each creation of a task causes a first wakeup with this
+ flag set. Applications should not set it themselves.
+
+ - TASK_WOKEN_TIMER this indicates the task's expire date was reached in the
+ timers queue. Applications should not set it themselves.
+
+ - TASK_WOKEN_IO indicates the wake-up happened due to I/O activity. Now
+ that all low-level I/O processing happens on tasklets,
+ this notion of I/O is now application-defined (for
+ example stream-interfaces use it to notify the stream).
+
+ - TASK_WOKEN_SIGNAL indicates that a signal the task was subscribed to was
+ received. Applications should not set it themselves.
+
+ - TASK_WOKEN_MSG any application-defined wake-up reason, usually for
+ inter-task communication (e.g filters vs streams).
+
+ - TASK_WOKEN_RES a resource the task was waiting for was finally made
+ available, allowing the task to continue its work. This
+ is essentially used by buffers and queues. Applications
+ may carefully use it for their own purpose if they're
+ certain not to rely on existing ones.
+
+ - TASK_WOKEN_OTHER any other application-defined wake-up reason.
+
+
+In addition, a few persistent flags may be observed or manipulated by the
+application, both for tasks and tasklets:
+
+ - TASK_SELF_WAKING when set, indicates that this task was found waking
+ itself up, and its class will change to bulk processing.
+ If this behavior is under control temporarily expected,
+ and it is not expected to happen again, it may make
+ sense to reset this flag from the ->process() function
+ itself.
+
+ - TASK_HEAVY when set, indicates that this task does so heavy
+ processing that it will become mandatory to give back
+ control to I/Os otherwise big latencies might occur. It
+ may be set by an application that expects something
+ heavy to happen (tens to hundreds of microseconds), and
+ reset once finished. An example of user is the TLS stack
+ which sets it when an imminent crypto operation is
+ expected.
+
+ - TASK_F_USR1 This is the first application-defined persistent flag.
+ It is always zero unless the application changes it. An
+ example of use cases is the I/O handler for backend
+ connections, to mention whether the connection is safe
+ to use or might have recently been migrated.
+
+Finally, when built with -DDEBUG_TASK, an extra sub-structure "debug" is added
+to both tasks and tasklets to note the code locations of the last two calls to
+task_wakeup() and tasklet_wakeup().
diff --git a/doc/internals/body-parsing.txt b/doc/internals/body-parsing.txt
new file mode 100644
index 0000000..be209af
--- /dev/null
+++ b/doc/internals/body-parsing.txt
@@ -0,0 +1,165 @@
+2014/04/16 - Pointer assignments during processing of the HTTP body
+
+In HAProxy, a struct http_msg is a descriptor for an HTTP message, which stores
+the state of an HTTP parser at any given instant, relative to a buffer which
+contains part of the message being inspected.
+
+Currently, an http_msg holds a few pointers and offsets to some important
+locations in a message depending on the state the parser is in. Some of these
+pointers and offsets may move when data are inserted into or removed from the
+buffer, others won't move.
+
+An important point is that the state of the parser only translates what the
+parser is reading, and not at all what is being done on the message (eg:
+forwarding).
+
+For an HTTP message <msg> and a buffer <buf>, we have the following elements
+to work with :
+
+
+Buffer :
+--------
+
+buf.size : the allocated size of the buffer. A message cannot be larger than
+ this size. In general, a message will even be smaller because the
+ size is almost always reduced by global.maxrewrite bytes.
+
+buf.data : memory area containing the part of the message being worked on. This
+ area is exactly <buf.size> bytes long. It should be seen as a sliding
+ window over the message, but in terms of implementation, it's closer
+ to a wrapping window. For ease of processing, new messages (requests
+ or responses) are aligned to the beginning of the buffer so that they
+ never wrap and common string processing functions can be used.
+
+buf.p : memory pointer (char *) to the beginning of the buffer as the parser
+ understands it. It commonly refers to the first character of an HTTP
+ request or response, but during forwarding, it can point to other
+ locations. This pointer always points to a location in <buf.data>.
+
+buf.i : number of bytes after <buf.p> that are available in the buffer. If
+ <buf.p + buf.i> exceeds <buf.data + buf.size>, then the pending data
+ wrap at the end of the buffer and continue at <buf.data>.
+
+buf.o : number of bytes already processed before <buf.p> that are pending
+ for departure. These bytes may leave at any instant once a connection
+ is established. These ones may wrap before <buf.data> to start before
+ <buf.data + buf.size>.
+
+It's common to call the part between buf.p and buf.p+buf.i the input buffer, and
+the part between buf.p-buf.o and buf.p the output buffer. This design permits
+efficient forwarding without copies. As a result, forwarding one byte from the
+input buffer to the output buffer only consists in :
+ - incrementing buf.p
+ - incrementing buf.o
+ - decrementing buf.i
+
+
+Message :
+---------
+Unless stated otherwise, all values are relative to <buf.p>, and are always
+comprised between 0 and <buf.i>. These values are relative offsets and they do
+not need to take wrapping into account, they are used as if the buffer was an
+infinite length sliding window. The buffer management functions handle the
+wrapping automatically.
+
+msg.next : points to the next byte to inspect. This offset is automatically
+ adjusted when inserting/removing some headers. In data states, it is
+ automatically adjusted to the number of bytes already inspected.
+
+msg.sov : start of value. First character of the header's value in the header
+ states, start of the body in the data states. Strictly positive
+ values indicate that headers were not forwarded yet (<buf.p> is
+ before the start of the body), and null or negative values are seen
+ after headers are forwarded (<buf.p> is at or past the start of the
+ body). The value stops changing when data start to leave the buffer
+ (in order to avoid integer overflows). So the maximum possible range
+ is -<buf.size> to +<buf.size>. This offset is automatically adjusted
+ when inserting or removing some headers. It is useful to rewind the
+ request buffer to the beginning of the body at any phase. The
+ response buffer does not really use it since it is immediately
+ forwarded to the client.
+
+msg.sol : start of line. Points to the beginning of the current header line
+ while parsing headers. It is cleared to zero in the BODY state,
+ and contains exactly the number of bytes comprising the preceding
+ chunk size in the DATA state (which can be zero), so that the sum of
+ msg.sov + msg.sol always points to the beginning of data for all
+ states starting with DATA. For chunked encoded messages, this sum
+ always corresponds to the beginning of the current chunk of data as
+ it appears in the buffer, or to be more precise, it corresponds to
+ the first of the remaining bytes of chunked data to be inspected. In
+ TRAILERS state, it contains the length of the last parsed part of
+ the trailer headers.
+
+msg.eoh : end of headers. Points to the CRLF (or LF) preceding the body and
+ marking the end of headers. It is where new headers are appended.
+ This offset is automatically adjusted when inserting/removing some
+ headers. It always contains the size of the headers excluding the
+ trailing CRLF even after headers have been forwarded.
+
+msg.eol : end of line. Points to the CRLF or LF of the current header line
+ being inspected during the various header states. In data states, it
+ holds the trailing CRLF length (1 or 2) so that msg.eoh + msg.eol
+ always equals the exact header length. It is not affected during data
+ states nor by forwarding.
+
+The beginning of the message headers can always be found this way even after
+headers or data have been forwarded, provided that everything is still present
+in the buffer :
+
+ headers = buf.p + msg->sov - msg->eoh - msg->eol
+
+
+Message length :
+----------------
+msg.chunk_len : amount of bytes of the current chunk or total message body
+ remaining to be inspected after msg.next. It is automatically
+ incremented when parsing a chunk size, and decremented as data
+ are forwarded.
+
+msg.body_len : total message body length, for logging. Equals Content-Length
+ when used, otherwise is the sum of all correctly parsed chunks.
+
+
+Message state :
+---------------
+msg.msg_state contains the current parser state, one of HTTP_MSG_*. The state
+indicates what byte is expected at msg->next.
+
+HTTP_MSG_BODY : all headers have been parsed, parsing of body has not
+ started yet.
+
+HTTP_MSG_100_SENT : parsing of body has started. If a 100-Continue was needed
+ it has already been sent.
+
+HTTP_MSG_DATA : some bytes are remaining for either the whole body when
+ the message size is determined by Content-Length, or for
+ the current chunk in chunked-encoded mode.
+
+HTTP_MSG_CHUNK_CRLF : msg->next points to the CRLF after the current data chunk.
+
+HTTP_MSG_TRAILERS : msg->next points to the beginning of a possibly empty
+ trailer line after the final empty chunk.
+
+HTTP_MSG_DONE : all the Content-Length data has been inspected, or the
+ final CRLF after trailers has been met.
+
+
+Message forwarding :
+--------------------
+Forwarding part of a message consists in advancing buf.p up to the point where
+it points to the byte following the last one to be forwarded. This can be done
+inline if enough bytes are present in the buffer, or in multiple steps if more
+buffers need to be forwarded (possibly including splicing). Thus by definition,
+after a block has been scheduled for being forwarded, msg->next and msg->sov
+must be reset.
+
+The communication channel between the producer and the consumer holds a counter
+of extra bytes remaining to be forwarded directly without consulting analysers,
+after buf.p. This counter is called to_forward. It commonly holds the advertised
+chunk length or content-length that does not fit in the buffer. For example, if
+2000 bytes are to be forwarded, and 10 bytes are present after buf.p as reported
+by buf.i, then both buf.o and buf.p will advance by 10, buf.i will be reset, and
+to_forward will be set to 1990 so that in total, 2000 bytes will be forwarded.
+At the end of the forwarding, buf.p will point to the first byte to be inspected
+after the 2000 forwarded bytes.
diff --git a/doc/internals/connect-status.txt b/doc/internals/connect-status.txt
new file mode 100644
index 0000000..70bbcc5
--- /dev/null
+++ b/doc/internals/connect-status.txt
@@ -0,0 +1,28 @@
+Normally, we should use getsockopt(fd, SOL_SOCKET, SO_ERROR) on a pending
+connect() to detect whether the connection correctly established or not.
+
+Unfortunately, getsockopt() does not report the status of a pending connection,
+which means that it returns 0 if the connection is still pending. This has to
+be expected, because as the name implies it, it only returns errors.
+
+With the speculative I/O, a new problem was introduced : if we pretend the
+socket was indicated as ready and we go to the socket's write() function,
+a pending connection will then inevitably be identified as established.
+
+In fact, there are solutions to this issue :
+
+ - send() returns -EAGAIN if it cannot write, so that as long as there are
+ pending data in the buffer, we'll be informed about the status of the
+ connection
+
+ - connect() on an already pending connection will return -1 with errno set to
+ one of the following values :
+ - EALREADY : connection already in progress
+ - EISCONN : connection already established
+ - anything else will indicate an error.
+
+=> So instead of using getsockopt() on a pending connection with no data, we
+ will switch to connect(). This implies that the connection address must be
+ known within the socket's write() function.
+
+
diff --git a/doc/internals/connection-header.txt b/doc/internals/connection-header.txt
new file mode 100644
index 0000000..b74cea0
--- /dev/null
+++ b/doc/internals/connection-header.txt
@@ -0,0 +1,196 @@
+2010/01/16 - Connection header adjustments depending on the transaction mode.
+
+
+HTTP transactions supports 5 possible modes :
+
+ WANT_TUN : default, nothing changed
+ WANT_TUN + httpclose : headers set for close in both dirs
+ WANT_KAL : keep-alive desired in both dirs
+ WANT_SCL : want close with the server and KA with the client
+ WANT_CLO : want close on both sides.
+
+When only WANT_TUN is set, nothing is changed nor analysed, so for commodity
+below, we'll refer to WANT_TUN+httpclose as WANT_TUN.
+
+The mode is adjusted in 3 steps :
+ - configuration sets initial mode
+ - request headers set required request mode
+ - response headers set the final mode
+
+
+1) Adjusting the initial mode via the configuration
+
+ option httpclose => TUN
+ option http-keep-alive => KAL
+ option http-server-close => SCL
+ option forceclose => CLO
+
+Note that option httpclose combined with any other option is equivalent to
+forceclose.
+
+
+2) Adjusting the request mode once the request is parsed
+
+If we cannot determine the body length from the headers, we set the mode to CLO
+but later we'll switch to tunnel mode once forwarding the body. That way, all
+parties are informed of the correct mode.
+
+Depending on the request version and request Connection header, we may have to
+adjust the current transaction mode and to update the connection header.
+
+mode req_ver req_hdr new_mode hdr_change
+TUN 1.0 - TUN -
+TUN 1.0 ka TUN del_ka
+TUN 1.0 close TUN del_close
+TUN 1.0 both TUN del_ka, del_close
+
+TUN 1.1 - TUN add_close
+TUN 1.1 ka TUN del_ka, add_close
+TUN 1.1 close TUN -
+TUN 1.1 both TUN del_ka
+
+KAL 1.0 - CLO -
+KAL 1.0 ka KAL -
+KAL 1.0 close CLO del_close
+KAL 1.0 both CLO del_ka, del_close
+
+KAL 1.1 - KAL -
+KAL 1.1 ka KAL del_ka
+KAL 1.1 close CLO -
+KAL 1.1 both CLO del_ka
+
+SCL 1.0 - CLO -
+SCL 1.0 ka SCL del_ka
+SCL 1.0 close CLO del_close
+SCL 1.0 both CLO del_ka, del_close
+
+SCL 1.1 - SCL add_close
+SCL 1.1 ka SCL del_ka, add_close
+SCL 1.1 close CLO -
+SCL 1.1 both CLO del_ka
+
+CLO 1.0 - CLO -
+CLO 1.0 ka CLO del_ka
+CLO 1.0 close CLO del_close
+CLO 1.0 both CLO del_ka, del_close
+
+CLO 1.1 - CLO add_close
+CLO 1.1 ka CLO del_ka, add_close
+CLO 1.1 close CLO -
+CLO 1.1 both CLO del_ka
+
+=> Summary:
+ - KAL and SCL are only possible with the same requests :
+ - 1.0 + ka
+ - 1.1 + ka or nothing
+
+ - CLO is assumed for any non-TUN request which contains at least a close
+ header, as well as for any 1.0 request without a keep-alive header.
+
+ - del_ka is set whenever we want a CLO or SCL or TUN and req contains a KA,
+ or when the req is 1.1 and contains a KA.
+
+ - del_close is set whenever a 1.0 request contains a close.
+
+ - add_close is set whenever a 1.1 request must be switched to TUN, SCL, CLO
+ and did not have a close hdr.
+
+Note that the request processing is performed in two passes, one with the
+frontend's config and a second one with the backend's config. It is only
+possible to "raise" the mode between them, so during the second pass, we have
+no reason to re-add a header that we previously removed. As an exception, the
+TUN mode is converted to CLO once combined because in fact it's an httpclose
+option set on a TUN mode connection :
+
+ BE (2)
+ | TUN KAL SCL CLO
+ ----+----+----+----+----
+ TUN | TUN CLO CLO CLO
+ +
+ KAL | CLO KAL SCL CLO
+ FE +
+ (1) SCL | CLO SCL SCL CLO
+ +
+ CLO | CLO CLO CLO CLO
+
+
+3) Adjusting the final mode once the response is parsed
+
+This part becomes trickier. It is possible that the server responds with a
+version that the client does not necessarily understand. Obviously, 1.1 clients
+are asusmed to understand 1.0 responses. The problematic case is a 1.0 client
+receiving a 1.1 response without any Connection header. Some 1.0 clients might
+know that in 1.1 this means "keep-alive" while others might ignore the version
+and assume a "close". Since we know the version on both sides, we may have to
+adjust some responses to remove any ambiguous case. That's the reason why the
+following table considers both the request and the response version. If the
+response length cannot be determined, we switch to CLO mode.
+
+mode res_ver res_hdr req_ver new_mode hdr_change
+TUN 1.0 - any TUN -
+TUN 1.0 ka any TUN del_ka
+TUN 1.0 close any TUN del_close
+TUN 1.0 both any TUN del_ka, del_close
+
+TUN 1.1 - any TUN add_close
+TUN 1.1 ka any TUN del_ka, add_close
+TUN 1.1 close any TUN -
+TUN 1.1 both any TUN del_ka
+
+KAL 1.0 - any SCL add_ka
+KAL 1.0 ka any KAL -
+KAL 1.0 close any SCL del_close, add_ka
+KAL 1.0 both any SCL del_close
+
+KAL 1.1 - 1.0 KAL add_ka
+KAL 1.1 - 1.1 KAL -
+KAL 1.1 ka 1.0 KAL -
+KAL 1.1 ka 1.1 KAL del_ka
+KAL 1.1 close 1.0 SCL del_close, add_ka
+KAL 1.1 close 1.1 SCL del_close
+KAL 1.1 both 1.0 SCL del_close
+KAL 1.1 both 1.1 SCL del_ka, del_close
+
+SCL 1.0 - any SCL add_ka
+SCL 1.0 ka any SCL -
+SCL 1.0 close any SCL del_close, add_ka
+SCL 1.0 both any SCL del_close
+
+SCL 1.1 - 1.0 SCL add_ka
+SCL 1.1 - 1.1 SCL -
+SCL 1.1 ka 1.0 SCL -
+SCL 1.1 ka 1.1 SCL del_ka
+SCL 1.1 close 1.0 SCL del_close, add_ka
+SCL 1.1 close 1.1 SCL del_close
+SCL 1.1 both 1.0 SCL del_close
+SCL 1.1 both 1.1 SCL del_ka, del_close
+
+CLO 1.0 - any CLO -
+CLO 1.0 ka any CLO del_ka
+CLO 1.0 close any CLO del_close
+CLO 1.0 both any CLO del_ka, del_close
+
+CLO 1.1 - any CLO add_close
+CLO 1.1 ka any CLO del_ka, add_close
+CLO 1.1 close any CLO -
+CLO 1.1 both any CLO del_ka
+
+=> in summary :
+ - the header operations do not depend on the initial mode, they only depend
+ on versions and current connection header(s).
+
+ - both CLO and TUN modes work similarly, they need to set a close mode on the
+ response. A 1.1 response will exclusively need the close header, while a 1.0
+ response will have it removed. Any keep-alive header is always removed when
+ found.
+
+ - a KAL request where the server wants to close turns into an SCL response so
+ that we release the server but still maintain the connection to the client.
+
+ - the KAL and SCL modes work the same way as we need to set keep-alive on the
+ response. So a 1.0 response will only have the keep-alive header with any
+ close header removed. A 1.1 response will have the keep-alive header added
+ for 1.0 requests and the close header removed for all requests.
+
+Note that the SCL and CLO modes will automatically cause the server connection
+to be closed at the end of the data transfer.
diff --git a/doc/internals/connection-scale.txt b/doc/internals/connection-scale.txt
new file mode 100644
index 0000000..7c3d902
--- /dev/null
+++ b/doc/internals/connection-scale.txt
@@ -0,0 +1,44 @@
+Problčme des connexions simultanées avec un backend
+
+Pour chaque serveur, 3 cas possibles :
+
+ - pas de limite (par défaut)
+ - limite statique (maxconn)
+ - limite dynamique (maxconn/(ratio de px->conn), avec minconn)
+
+On a donc besoin d'une limite sur le proxy dans le cas de la limite
+dynamique, afin de fixer un seuil et un ratio. Ce qui compte, c'est
+le point aprčs lequel on passe d'un régime linéaire ŕ un régime
+saturé.
+
+On a donc 3 phases :
+
+ - régime minimal (0..srv->minconn)
+ - régime linéaire (srv->minconn..srv->maxconn)
+ - régime saturé (srv->maxconn..)
+
+Le minconn pourrait aussi ressortir du serveur ?
+En pratique, on veut :
+ - un max par serveur
+ - un seuil global auquel les serveurs appliquent le max
+ - un seuil minimal en-dessous duquel le nb de conn est
+ maintenu. Cette limite a un sens par serveur (jamais moins de X conns)
+ mais aussi en global (pas la peine de faire du dynamique en dessous de
+ X conns ŕ répartir). La difficulté en global, c'est de savoir comment
+ on calcule le nombre min associé ŕ chaque serveur, vu que c'est un ratio
+ défini ŕ partir du max.
+
+Ca revient ŕ peu prčs ŕ la męme chose que de faire 2 états :
+
+ - régime linéaire avec un offset (srv->minconn..srv->maxconn)
+ - régime saturé (srv->maxconn..)
+
+Sauf que dans ce cas, le min et le max sont bien par serveur, et le seuil est
+global et correspond ŕ la limite de connexions au-delŕ de laquel on veut
+tourner ŕ plein régime sur l'ensemble des serveurs. On peut donc parler de
+passage en mode "full", "saturated", "optimal". On peut également parler de
+la fin de la partie "scalable", "dynamique".
+
+=> fullconn 1000 par exemple ?
+
+
diff --git a/doc/internals/fd-migration.txt b/doc/internals/fd-migration.txt
new file mode 100644
index 0000000..aaddad3
--- /dev/null
+++ b/doc/internals/fd-migration.txt
@@ -0,0 +1,138 @@
+2021-07-30 - File descriptor migration between threads
+
+An FD migration may happen on any idle connection that experiences a takeover()
+operation by another thread. In this case the acting thread becomes the owner
+of the connection (and FD) while previous one(s) need to forget about it.
+
+File descriptor migration between threads is a fairly complex operation because
+it is required to maintain a durable consistency between the pollers states and
+the haproxy's desired state. Indeed, very often the FD is registered within one
+thread's poller and that thread might be waiting in the system, so there is no
+way to synchronously update it. This is where thread_mask, polled_mask and per
+thread updates are used:
+
+ - a thread knows if it's allowed to manipulate an FD by looking at its bit in
+ the FD's thread_mask ;
+
+ - each thread knows if it was polling an FD by looking at its bit in the
+ polled_mask field ; a recent migration is usually indicated by a bit being
+ present in polled_mask and absent from thread_mask.
+
+ - other threads know whether it's safe to take over an FD by looking at the
+ running mask: if it contains any other thread's bit, then other threads are
+ using it and it's not safe to take it over.
+
+ - sleeping threads are notified about the need to update their polling via
+ local or global updates to the FD. Each thread has its own local update
+ list and its own bit in the update_mask to know whether there are pending
+ updates for it. This allows to reconverge polling with the desired state
+ at the last instant before polling.
+
+While the description above could be seen as "progressive" (it technically is)
+in that there is always a transition and convergence period in a migrated FD's
+life, functionally speaking it's perfectly atomic thanks to the running bit and
+to the per-thread idle connections lock: no takeover is permitted without
+holding the idle_conns lock, and takeover may only happen by atomically picking
+a connection from the list that is also protected by this lock. In practice, an
+FD is never taken over by itself, but always in the context of a connection,
+and by atomically removing a connection from an idle list, it is possible to
+guarantee that a connection will not be picked, hence that its FD will not be
+taken over.
+
+same thread as list!
+
+The possible entry points to a race to use a file descriptor are the following
+ones, with their respective sequences:
+
+ 1) takeover: requested by conn_backend_get() on behalf of connect_server()
+ - take the idle_conns_lock, protecting against a parallel access from the
+ I/O tasklet or timeout task
+ - pick the first connection from the list
+ - attempt an fd_takeover() on this connection's fd. Usually it works,
+ unless a late wakeup of the owning thread shows up in the FD's running
+ mask. The operation is performed in fd_takeover() using a DWCAS which
+ tries to switch both running and thread_mask to the caller's tid_bit. A
+ concurrent bit in running is enough to make it fail. This guarantees
+ another thread does not wakeup from I/O in the middle of the takeover.
+ In case of conflict, this FD is skipped and the attempt is tried again
+ with the next connection.
+ - resets the task/tasklet contexts to NULL, as a signal that they are not
+ allowed to run anymore. The tasks retrieve their execution context from
+ the scheduler in the arguments, but will check the tasks' context from
+ the structure under the lock to detect this possible change, and abort.
+ - at this point the takeover succeeded, the idle_conns_lock is released and
+ the connection and its FD are now owned by the caller
+
+ 2) poll report: happens on late rx, shutdown or error on idle conns
+ - fd_set_running() is called to atomically set the running_mask and check
+ that the caller's tid_bit is still present in the thread_mask. Upon
+ failure the caller arranges itself to stop reporting that FD (e.g. by
+ immediate removal or by an asynchronous update). Upon success, it's
+ guaranteed that any concurrent fd_takeover() will fail the DWCAS and that
+ another connection will need to be picked instead.
+ - FD's state is possibly updated
+ - the iocb is called if needed (almost always)
+ - if the iocb didn't kill the connection, release the bit from running_mask
+ making the connection possibly available to a subsequent fd_takeover().
+
+ 3) I/O tasklet, timeout task: timeout or subscribed wakeup
+ - start by taking the idle_conns_lock, ensuring no takeover() will pick the
+ same connection from this point.
+ - check the task/tasklet's context to verify that no recently completed
+ takeover() stole the connection. If it's NULL, the connection was lost,
+ the lock is released and the task/tasklet killed. Otherwise it is
+ guaranteed that no other thread may use that connection (current takeover
+ candidates are waiting on the lock, previous owners waking from poll()
+ lost their bit in the thread_mask and will not touch the FD).
+ - the connection is removed from the idle conns list. From this point on,
+ no other thread will even find it there nor even try fd_takeover() on it.
+ - the idle_conns_lock is now released, the connection is protected and its
+ FD is not reachable by other threads anymore.
+ - the task does what it has to do
+ - if the connection is still usable (i.e. not upon timeout), it's inserted
+ again into the idle conns list, meaning it may instantly be taken over
+ by a competing thread.
+
+ 4) wake() callback: happens on last user after xfers (may free() the conn)
+ - the connection is still owned by the caller, it's still subscribed to
+ polling but the connection is idle thus inactive. Errors or shutdowns
+ may be reported late, via sock_conn_iocb() and conn_notify_mux(), thus
+ the running bit is set (i.e. a concurrent fd_takeover() will fail).
+ - if the connection is in the list, the idle_conns_lock is grabbed, the
+ connection is removed from the list, and the lock is released.
+ - mux->wake() is called
+ - if the connection previously was in the list, it's reinserted under the
+ idle_conns_lock.
+
+
+With the DWCAS removal between running_mask & thread_mask:
+
+fd_takeover:
+ 1 if (!CAS(&running_mask, 0, tid_bit))
+ 2 return fail;
+ 3 atomic_store(&thread_mask, tid_bit);
+ 4 atomic_and(&running_mask, ~tid_bit);
+
+poller:
+ 1 do {
+ 2 /* read consistent running_mask & thread_mask */
+ 3 do {
+ 4 run = atomic_load(&running_mask);
+ 5 thr = atomic_load(&thread_mask);
+ 6 } while (run & ~thr);
+ 7
+ 8 if (!(thr & tid_bit)) {
+ 9 /* takeover has started */
+ 10 goto disable_fd;
+ 11 }
+ 12 } while (!CAS(&running_mask, run, run | tid_bit));
+
+fd_delete:
+ 1 atomic_or(&running_mask, tid_bit);
+ 2 atomic_store(&thread_mask, 0);
+ 3 atomic_and(&running_mask, ~tid_bit);
+
+The loop in poller:3-6 is used to make sure the thread_mask we read matches
+the last updated running_mask. If nobody can give up on fd_takeover(), it
+might even be possible to spin on thread_mask only. Late pollers will not
+set running anymore with this.
diff --git a/doc/internals/hashing.txt b/doc/internals/hashing.txt
new file mode 100644
index 0000000..260b6af
--- /dev/null
+++ b/doc/internals/hashing.txt
@@ -0,0 +1,83 @@
+2013/11/20 - How hashing works internally in haproxy - maddalab@gmail.com
+
+This document describes how HAProxy implements hashing both map-based and
+consistent hashing, both prior to versions 1.5 and the motivation and tests
+that were done when providing additional options starting in version 2.4
+
+A note on hashing in general, hash functions strive to have little
+correlation between input and output. The heart of a hash function is its
+mixing step. The behavior of the mixing step largely determines whether the
+hash function is collision-resistant. Hash functions that are collision
+resistant are more likely to have an even distribution of load.
+
+The purpose of the mixing function is to spread the effect of each message
+bit throughout all the bits of the internal state. Ideally every bit in the
+hash state is affected by every bit in the message. And we want to do that
+as quickly as possible simply for the sake of program performance. A
+function is said to satisfy the strict avalanche criterion if, whenever a
+single input bit is complemented (toggled between 0 and 1), each of the
+output bits should change with a probability of one half for an arbitrary
+selection of the remaining input bits.
+
+To guard against a combination of hash function and input that results in
+high rate of collisions, haproxy implements an avalanche algorithm on the
+result of the hashing function. In all versions 1.4 and prior avalanche is
+always applied when using the consistent hashing directive. It is intended
+to provide quite a good distribution for little input variations. The result
+is quite suited to fit over a 32-bit space with enough variations so that
+a randomly picked number falls equally before any server position, which is
+ideal for consistently hashed backends, a common use case for caches.
+
+In all versions 1.4 and prior HAProxy implements the SDBM hashing function.
+However tests show that alternatives to SDBM have a better cache
+distribution on different hashing criteria. Additional tests involving
+alternatives for hash input and an option to trigger avalanche, we found
+different algorithms perform better on different criteria. DJB2 performs
+well when hashing ascii text and is a good choice when hashing on host
+header. Other alternatives perform better on numbers and are a good choice
+when using source ip. The results also vary by use of the avalanche flag.
+
+The results of the testing can be found under the tests folder. Here is
+a summary of the discussion on the results on 1 input criteria and the
+methodology used to generate the results.
+
+A note of the setup when validating the results independently, one
+would want to avoid backend server counts that may skew the results. As
+an example with DJB2 avoid 33 servers. Please see the implementations of
+the hashing function, which can be found in the links under references.
+
+The following was the set up used
+
+(a) hash-type consistent/map-based
+(b) avalanche on/off
+(c) balanche host(hdr)
+(d) 3 criteria for inputs
+ - ~ 10K requests, including duplicates
+ - ~ 46K requests, unique requests from 1 MM requests were obtained
+ - ~ 250K requests, including duplicates
+(e) 17 servers in backend, all servers were assigned the same weight
+
+Result of the hashing were obtained across the server via monitoring log
+files for haproxy. Population Standard deviation was used to evaluate the
+efficacy of the hashing algorithm. Lower standard deviation, indicates
+a better distribution of load across the backends.
+
+On 10K requests, when using consistent hashing with avalanche on host
+headers, DJB2 significantly out performs SDBM. Std dev on SDBM was 48.95
+and DJB2 was 26.29. This relationship is inverted with avalanche disabled,
+however DJB2 with avalanche enabled out performs SDBM with avalanche
+disabled.
+
+On map-based hashing SDBM out performs DJB2 irrespective of the avalanche
+option. SDBM without avalanche is marginally better than with avalanche.
+DJB2 performs significantly worse with avalanche enabled.
+
+Summary: The results of the testing indicate that there isn't a hashing
+algorithm that can be applied across all input criteria. It is necessary
+to support alternatives to SDBM, which is generally the best option, with
+algorithms that are better for different inputs. Avalanche is not always
+applicable and may result in less smooth distribution.
+
+References:
+Mixing Functions/Avalanche: https://papa.bretmulvey.com/post/124027987928/hash-functions
+Hash Functions: http://www.cse.yorku.ca/~oz/hash.html
diff --git a/doc/internals/list.fig b/doc/internals/list.fig
new file mode 100644
index 0000000..aeb1f1d
--- /dev/null
+++ b/doc/internals/list.fig
@@ -0,0 +1,599 @@
+#FIG 3.2 Produced by xfig version 2.4
+Landscape
+Center
+Metric
+A4
+119.50
+Single
+-2
+1200 2
+6 3960 3420 4320 4230
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 4230 3860 4005 3860
+2 2 0 2 0 2 53 0 20 0.000 0 0 -1 0 0 5
+ 4005 3510 4230 3510 4230 4185 4005 4185 4005 3510
+4 1 0 50 0 14 10 0.0000 4 105 105 4120 4062 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 4118 3735 N\001
+-6
+6 4185 5580 4545 6390
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 4455 6020 4230 6020
+2 2 0 2 0 4 53 0 20 0.000 0 0 -1 0 0 5
+ 4230 5670 4455 5670 4455 6345 4230 6345 4230 5670
+4 1 0 50 0 14 10 0.0000 4 105 105 4345 6222 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 4343 5895 N\001
+-6
+6 4905 5445 5445 6525
+6 4905 5445 5445 6525
+6 5085 5580 5445 6390
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 5355 6020 5130 6020
+2 2 0 2 0 6 53 0 20 0.000 0 0 -1 0 0 5
+ 5130 5670 5355 5670 5355 6345 5130 6345 5130 5670
+4 1 0 50 0 14 10 0.0000 4 105 105 5245 6222 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 5243 5895 N\001
+-6
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 5355 5670 4905 5670
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 4905 6345 5355 6345
+2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 4905 5445 5355 5445 5355 6525 4905 6525 4905 5445
+4 1 0 50 0 14 12 0.0000 4 120 120 5040 6075 L\001
+-6
+-6
+6 5805 5445 6345 6525
+6 5805 5445 6345 6525
+6 5985 5580 6345 6390
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 6255 6020 6030 6020
+2 2 0 2 0 6 53 0 20 0.000 0 0 -1 0 0 5
+ 6030 5670 6255 5670 6255 6345 6030 6345 6030 5670
+4 1 0 50 0 14 10 0.0000 4 105 105 6145 6222 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 6143 5895 N\001
+-6
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 6255 5670 5805 5670
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 5805 6345 6255 6345
+2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 5805 5445 6255 5445 6255 6525 5805 6525 5805 5445
+4 1 0 50 0 14 12 0.0000 4 120 120 5940 6075 L\001
+-6
+-6
+6 6705 5445 7245 6525
+6 6705 5445 7245 6525
+6 6885 5580 7245 6390
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 7155 6020 6930 6020
+2 2 0 2 0 6 53 0 20 0.000 0 0 -1 0 0 5
+ 6930 5670 7155 5670 7155 6345 6930 6345 6930 5670
+4 1 0 50 0 14 10 0.0000 4 105 105 7045 6222 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 7043 5895 N\001
+-6
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 7155 5670 6705 5670
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 6705 6345 7155 6345
+2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 6705 5445 7155 5445 7155 6525 6705 6525 6705 5445
+4 1 0 50 0 14 12 0.0000 4 120 120 6840 6075 L\001
+-6
+-6
+6 450 5580 810 6390
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 720 6020 495 6020
+2 2 0 2 0 4 53 0 20 0.000 0 0 -1 0 0 5
+ 495 5670 720 5670 720 6345 495 6345 495 5670
+4 1 0 50 0 14 10 0.0000 4 105 105 610 6222 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 608 5895 N\001
+-6
+6 1170 5445 1710 6525
+6 1170 5445 1710 6525
+6 1350 5580 1710 6390
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 1620 6020 1395 6020
+2 2 0 2 0 6 53 0 20 0.000 0 0 -1 0 0 5
+ 1395 5670 1620 5670 1620 6345 1395 6345 1395 5670
+4 1 0 50 0 14 10 0.0000 4 105 105 1510 6222 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 1508 5895 N\001
+-6
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 1620 5670 1170 5670
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 1170 6345 1620 6345
+2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 1170 5445 1620 5445 1620 6525 1170 6525 1170 5445
+4 1 0 50 0 14 12 0.0000 4 120 120 1305 6075 L\001
+-6
+-6
+6 2070 5445 2610 6525
+6 2070 5445 2610 6525
+6 2250 5580 2610 6390
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 2520 6020 2295 6020
+2 2 0 2 0 6 53 0 20 0.000 0 0 -1 0 0 5
+ 2295 5670 2520 5670 2520 6345 2295 6345 2295 5670
+4 1 0 50 0 14 10 0.0000 4 105 105 2410 6222 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 2408 5895 N\001
+-6
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 2520 5670 2070 5670
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 2070 6345 2520 6345
+2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 2070 5445 2520 5445 2520 6525 2070 6525 2070 5445
+4 1 0 50 0 14 12 0.0000 4 120 120 2205 6075 L\001
+-6
+-6
+6 2970 5445 3510 6525
+6 2970 5445 3510 6525
+6 3150 5580 3510 6390
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 3420 6020 3195 6020
+2 2 0 2 0 6 53 0 20 0.000 0 0 -1 0 0 5
+ 3195 5670 3420 5670 3420 6345 3195 6345 3195 5670
+4 1 0 50 0 14 10 0.0000 4 105 105 3310 6222 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 3308 5895 N\001
+-6
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 3420 5670 2970 5670
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 2970 6345 3420 6345
+2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 2970 5445 3420 5445 3420 6525 2970 6525 2970 5445
+4 1 0 50 0 14 12 0.0000 4 120 120 3105 6075 L\001
+-6
+-6
+6 720 3420 1080 4230
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 990 3860 765 3860
+2 2 0 2 0 2 53 0 20 0.000 0 0 -1 0 0 5
+ 765 3510 990 3510 990 4185 765 4185 765 3510
+4 1 0 50 0 14 10 0.0000 4 105 105 880 4062 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 878 3735 N\001
+-6
+6 2700 3420 3060 4230
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 2970 3860 2745 3860
+2 2 0 2 0 6 53 0 20 0.000 0 0 -1 0 0 5
+ 2745 3510 2970 3510 2970 4185 2745 4185 2745 3510
+4 1 0 50 0 14 10 0.0000 4 105 105 2860 4062 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 2858 3735 N\001
+-6
+6 1620 3465 1935 4230
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 1890 3860 1665 3860
+2 2 0 2 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 1665 3510 1890 3510 1890 4185 1665 4185 1665 3510
+4 1 0 50 0 14 10 0.0000 4 105 105 1780 4062 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 1778 3735 N\001
+-6
+6 10485 3330 11025 4410
+6 10665 3465 11025 4275
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 10935 3905 10710 3905
+2 2 0 2 0 6 53 0 20 0.000 0 0 -1 0 0 5
+ 10710 3555 10935 3555 10935 4230 10710 4230 10710 3555
+4 1 0 50 0 14 10 0.0000 4 105 105 10825 4107 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 10823 3780 N\001
+-6
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 10935 3555 10485 3555
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 10485 4230 10935 4230
+2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 10485 3330 10935 3330 10935 4410 10485 4410 10485 3330
+4 1 0 50 0 14 12 0.0000 4 120 120 10620 3960 L\001
+-6
+6 7110 3105 7650 4185
+6 7110 3105 7650 4185
+6 7290 3240 7650 4050
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 7560 3680 7335 3680
+2 2 0 2 0 6 53 0 20 0.000 0 0 -1 0 0 5
+ 7335 3330 7560 3330 7560 4005 7335 4005 7335 3330
+4 1 0 50 0 14 10 0.0000 4 105 105 7450 3882 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 7448 3555 N\001
+-6
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 7560 3330 7110 3330
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 7110 4005 7560 4005
+2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 7110 3105 7560 3105 7560 4185 7110 4185 7110 3105
+4 1 0 50 0 14 12 0.0000 4 120 120 7245 3735 L\001
+-6
+-6
+6 8010 3105 8550 4185
+6 8010 3105 8550 4185
+6 8190 3240 8550 4050
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 8460 3680 8235 3680
+2 2 0 2 0 6 53 0 20 0.000 0 0 -1 0 0 5
+ 8235 3330 8460 3330 8460 4005 8235 4005 8235 3330
+4 1 0 50 0 14 10 0.0000 4 105 105 8350 3882 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 8348 3555 N\001
+-6
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 8460 3330 8010 3330
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 8010 4005 8460 4005
+2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 8010 3105 8460 3105 8460 4185 8010 4185 8010 3105
+4 1 0 50 0 14 12 0.0000 4 120 120 8145 3735 L\001
+-6
+-6
+6 9315 990 12195 2160
+6 9675 1080 10035 1890
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 9945 1520 9720 1520
+2 2 0 2 0 2 53 0 20 0.000 0 0 -1 0 0 5
+ 9720 1170 9945 1170 9945 1845 9720 1845 9720 1170
+4 1 0 50 0 14 10 0.0000 4 105 105 9835 1722 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 9833 1395 N\001
+-6
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 10935 1520 10710 1520
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 11925 1520 11700 1520
+2 2 0 2 0 7 52 0 20 0.000 0 0 -1 0 0 5
+ 10710 1170 10935 1170 10935 1845 10710 1845 10710 1170
+2 2 0 2 0 6 52 0 20 0.000 0 0 -1 0 0 5
+ 11700 1170 11925 1170 11925 1845 11700 1845 11700 1170
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 9945 1350 10665 1350
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 10935 1350 11655 1350
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 8
+ 1 1 1.00 60.00 120.00
+ 11925 1350 12105 1350 12195 1350 12195 990 9315 990 9315 1350
+ 9495 1350 9675 1350
+ 0.000 1.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 8
+ 1 1 1.00 60.00 120.00
+ 9675 1710 9495 1710 9315 1710 9405 2160 12195 2160 12195 1710
+ 12105 1710 11925 1710
+ 0.000 1.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 11655 1710 10935 1710
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 10665 1710 9945 1710
+ 0.000 0.000
+4 1 0 50 0 14 10 0.0000 4 105 105 10825 1722 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 10823 1395 N\001
+4 1 0 50 0 14 10 0.0000 4 105 105 11815 1722 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 11813 1395 N\001
+-6
+6 6345 1080 6705 1890
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 6615 1520 6390 1520
+2 2 0 2 0 2 53 0 20 0.000 0 0 -1 0 0 5
+ 6390 1170 6615 1170 6615 1845 6390 1845 6390 1170
+4 1 0 50 0 14 10 0.0000 4 105 105 6505 1722 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 6503 1395 N\001
+-6
+6 7335 1080 7695 1890
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 7605 1520 7380 1520
+2 2 0 2 0 6 52 0 20 0.000 0 0 -1 0 0 5
+ 7380 1170 7605 1170 7605 1845 7380 1845 7380 1170
+4 1 0 50 0 14 10 0.0000 4 105 105 7495 1722 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 7493 1395 N\001
+-6
+6 8325 1080 8685 1890
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 8595 1520 8370 1520
+2 2 0 2 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 8370 1170 8595 1170 8595 1845 8370 1845 8370 1170
+4 1 0 50 0 14 10 0.0000 4 105 105 8485 1722 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 8483 1395 N\001
+-6
+6 3870 1215 4185 1980
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 4140 1610 3915 1610
+2 2 0 2 0 2 53 0 20 0.000 0 0 -1 0 0 5
+ 3915 1260 4140 1260 4140 1935 3915 1935 3915 1260
+4 1 0 50 0 14 10 0.0000 4 105 105 4030 1812 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 4028 1485 N\001
+-6
+6 4770 1215 5085 1980
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 5040 1610 4815 1610
+2 2 0 2 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 4815 1260 5040 1260 5040 1935 4815 1935 4815 1260
+4 1 0 50 0 14 10 0.0000 4 105 105 4930 1812 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 4928 1485 N\001
+-6
+6 2205 990 2925 2160
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 2655 1610 2430 1610
+2 2 0 2 0 2 53 0 20 0.000 0 0 -1 0 0 5
+ 2430 1260 2655 1260 2655 1935 2430 1935 2430 1260
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 6
+ 1 1 1.00 60.00 120.00
+ 2655 1440 2880 1440 2880 1035 2205 1035 2205 1440 2430 1440
+ 0.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 6
+ 1 1 1.00 60.00 120.00
+ 2655 1755 2880 1755 2880 2160 2205 2160 2205 1755 2430 1755
+ 0.000 1.000 1.000 1.000 1.000 0.000
+4 1 0 50 0 14 10 0.0000 4 105 105 2545 1812 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 2543 1485 N\001
+-6
+6 525 1350 1455 1830
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 540 1590 1440 1590
+2 2 0 2 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 540 1365 1440 1365 1440 1815 540 1815 540 1365
+4 1 0 50 0 14 10 0.0000 4 105 735 990 1545 list *N\001
+4 1 0 50 0 14 10 0.0000 4 105 735 990 1770 list *P\001
+-6
+6 4815 3420 5175 4230
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 5085 3860 4860 3860
+2 2 0 2 0 7 53 0 20 0.000 0 0 -1 0 0 5
+ 4860 3510 5085 3510 5085 4185 4860 4185 4860 3510
+4 1 0 50 0 14 10 0.0000 4 105 105 4975 4062 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 4973 3735 N\001
+-6
+6 5715 3285 6390 4410
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 6165 3860 5940 3860
+2 2 0 2 0 6 53 0 20 0.000 0 0 -1 0 0 5
+ 5940 3510 6165 3510 6165 4185 5940 4185 5940 3510
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 6
+ 1 1 1.00 60.00 120.00
+ 6165 3690 6390 3690 6390 3285 5715 3285 5715 3690 5940 3690
+ 0.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 6
+ 1 1 1.00 60.00 120.00
+ 6165 4005 6390 4005 6390 4410 5715 4410 5715 4005 5940 4005
+ 0.000 1.000 1.000 1.000 1.000 0.000
+4 1 0 50 0 14 10 0.0000 4 105 105 6055 4062 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 6053 3735 N\001
+-6
+2 4 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 5
+ 4050 4725 7605 4725 7605 6840 4050 6840 4050 4725
+2 4 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 5
+ 315 4725 3870 4725 3870 6840 315 6840 315 4725
+2 4 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 5
+ 3150 4500 315 4500 315 2475 3150 2475 3150 4500
+2 4 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 5
+ 6660 2475 8910 2475 8910 4500 6660 4500 6660 2475
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 10035 3375 10485 3330
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 10080 3735 10485 3555
+2 4 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 5
+ 9135 2475 12285 2475 12285 4500 9135 4500 9135 2475
+2 4 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 5
+ 9270 270 12285 270 12285 2250 9270 2250 9270 270
+2 4 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 5
+ 5760 270 9045 270 9045 2250 5760 2250 5760 270
+2 4 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 5
+ 3465 270 5535 270 5535 2250 3465 2250 3465 270
+2 4 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 5
+ 1845 270 3240 270 3240 2250 1845 2250 1845 270
+2 4 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 5
+ 315 270 1620 270 1620 2250 315 2250 315 270
+2 4 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 5
+ 3330 2475 6435 2475 6435 4500 3330 4500 3330 2475
+2 4 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 5
+ 12285 6840 12285 4725 7785 4725 7785 6840 12285 6840
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 4230 3690 4860 3690
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 4860 4050 4230 4050
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 7
+ 1 1 1.00 60.00 120.00
+ 3960 4050 3780 4050 3600 4050 3600 4410 5580 4410 5580 4050
+ 5130 4050
+ 0.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 6261 5805 6711 5670
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 4461 5805 4911 5670
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 5358 5805 5808 5670
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 6705 6210 6255 6210
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 5805 6210 5355 6210
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 4905 6210 4455 6210
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 7
+ 1 1 1.00 60.00 120.00
+ 4320 6345 4320 6525 4320 6750 7470 6750 7470 6480 7470 6210
+ 7155 6210
+ 0.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 8
+ 1 1 1.00 60.00 120.00
+ 7155 5850 7335 5850 7470 5850 7470 5355 7470 5085 4590 5085
+ 4590 5355 4860 5625
+ 0.000 1.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2526 5805 2976 5670
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 726 5805 1176 5670
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 1623 5805 2073 5670
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2970 6210 2520 6210
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2070 6210 1620 6210
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 1170 6210 720 6210
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 7
+ 1 1 1.00 60.00 120.00
+ 585 6345 585 6525 585 6750 3735 6750 3735 6480 3735 6210
+ 3420 6210
+ 0.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 8
+ 1 1 1.00 60.00 120.00
+ 3420 5850 3600 5850 3735 5850 3735 5355 3735 5085 585 5085
+ 585 5265 585 5670
+ 0.000 1.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 990 3690 1620 3690
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 1620 4050 990 4050
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 7
+ 1 1 1.00 60.00 120.00
+ 1890 3690 2340 3690 2340 3240 360 3240 360 3690 540 3690
+ 720 3690
+ 0.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 7
+ 1 1 1.00 60.00 120.00
+ 720 4050 540 4050 360 4050 360 4410 2340 4410 2340 4050
+ 1890 4050
+ 0.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 7560 3465 8010 3330
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 7560 3915 8010 3375
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 7
+ 1 1 1.00 60.00 120.00
+ 8460 3465 8775 3465 8820 3060 8730 2745 6750 2745 6705 3330
+ 7110 3330
+ 0.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 8
+ 1 1 1.00 60.00 120.00
+ 8460 3870 8820 3870 8820 4230 8640 4365 6930 4365 6750 4230
+ 6705 3510 7065 3375
+ 0.000 1.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 6615 1350 7335 1350
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 7605 1350 8325 1350
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 8
+ 1 1 1.00 60.00 120.00
+ 8595 1350 8775 1350 8865 1350 8865 990 5985 990 5985 1350
+ 6165 1350 6345 1350
+ 0.000 1.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 8
+ 1 1 1.00 60.00 120.00
+ 6345 1710 6165 1710 5985 1710 6075 2160 8865 2160 8865 1710
+ 8775 1710 8595 1710
+ 0.000 1.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 8325 1710 7605 1710
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 7335 1710 6615 1710
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 4140 1440 4770 1440
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 4770 1800 4140 1800
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 7
+ 1 1 1.00 60.00 120.00
+ 5040 1440 5490 1440 5490 990 3510 990 3510 1440 3690 1440
+ 3870 1440
+ 0.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 7
+ 1 1 1.00 60.00 120.00
+ 3870 1800 3690 1800 3510 1800 3510 2160 5490 2160 5490 1800
+ 5040 1800
+ 0.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 7
+ 1 1 1.00 60.00 120.00
+ 5130 3690 5580 3690 5580 3240 3600 3240 3600 3690 3780 3690
+ 3960 3690
+ 0.000 1.000 1.000 1.000 1.000 1.000 0.000
+4 1 0 50 0 14 10 0.0000 4 135 3780 5805 4950 Asymmetrical list starting at R(red)\001
+4 1 0 50 0 14 10 0.0000 4 135 4095 10215 4950 Symmetrical lists vs Asymmetrical lists\001
+4 1 0 50 0 12 10 0.0000 4 135 525 5130 5355 foo_0\001
+4 1 0 50 0 12 10 0.0000 4 135 525 6030 5355 foo_1\001
+4 1 0 50 0 12 10 0.0000 4 135 525 6930 5355 foo_2\001
+4 1 0 50 0 14 10 0.0000 4 135 3675 2070 4950 Symmetrical list starting at R(red)\001
+4 1 0 50 0 12 10 0.0000 4 135 525 3195 5355 foo_2\001
+4 1 0 50 0 12 10 0.0000 4 135 525 2295 5355 foo_1\001
+4 1 0 50 0 12 10 0.0000 4 135 525 1395 5355 foo_0\001
+4 1 0 50 0 12 10 0.0000 4 105 315 9855 3420 foo\001
+4 1 0 50 0 12 10 0.0000 4 105 105 9990 3825 E\001
+4 1 0 50 0 14 10 0.0000 4 135 1680 7785 2655 Linking elements\001
+4 1 0 50 0 12 10 0.0000 4 135 525 8235 3015 foo_1\001
+4 1 0 50 0 12 10 0.0000 4 135 525 7335 3015 foo_0\001
+4 1 0 50 0 14 10 0.0000 4 105 1470 2565 675 struct list *G\001
+4 1 0 50 0 14 10 0.0000 4 135 1470 2565 495 LIST_INIT(G):G\001
+4 1 0 50 0 14 10 0.0000 4 135 1890 4500 495 LIST_INSERT(G,W):W\001
+4 1 0 50 0 14 10 0.0000 4 135 3360 10665 2700 foo=LIST_ELEM(E, struct foo*, L)\001
+4 1 0 50 0 14 10 0.0000 4 135 1890 4500 675 LIST_APPEND(G,W):W\001
+4 1 0 50 0 14 10 0.0000 4 135 1890 7425 540 LIST_INSERT(G,Y):Y\001
+4 1 0 50 0 14 10 0.0000 4 135 1890 10755 540 LIST_APPEND(G,Y):Y\001
+4 1 0 50 0 14 10 0.0000 4 135 1680 1755 2790 LIST_DELETE(Y):Y\001
+4 1 0 50 0 14 10 0.0000 4 135 1890 4905 2745 LIST_DEL_INIT(Y):Y\001
+4 1 0 50 0 12 9 0.0000 4 120 2880 10665 2925 Returns a pointer to struct foo*\001
+4 1 0 50 0 12 9 0.0000 4 120 2790 10665 3105 containing header E as member L\001
+4 1 0 50 0 12 9 0.0000 4 120 2700 10755 810 adds Y at the queue (before G)\001
+4 1 0 50 0 12 9 0.0000 4 120 3060 7425 810 adds Y(yellow) just after G(green)\001
+4 1 0 50 0 12 9 0.0000 4 120 1170 4500 855 adds W(white)\001
+4 1 0 50 0 12 9 0.0000 4 105 1080 2565 900 Terminates G\001
+4 1 0 50 0 12 9 0.0000 4 90 540 990 855 N=next\001
+4 1 0 50 0 12 9 0.0000 4 105 540 990 1080 P=prev\001
+4 1 0 50 0 12 9 0.0000 4 120 2610 1755 3060 unlinks and returns Y(yellow)\001
+4 1 0 50 0 12 9 0.0000 4 120 2610 4905 3060 unlinks, inits, and returns Y\001
+4 0 0 50 0 12 8 0.0000 4 105 2175 7875 5265 - both are empty if R->P == R\001
+4 0 0 50 0 12 8 0.0000 4 90 2175 7875 5490 - last element has R->P == &L\001
+4 0 0 50 0 12 8 0.0000 4 105 3150 7875 5715 - FOREACH_ITEM(it, R, end, struct foo*, L)\001
+4 0 0 50 0 12 8 0.0000 4 105 3300 7875 5940 iterates <it> through foo{0,1,2} and stops\001
+4 0 0 50 0 12 8 0.0000 4 105 3900 7875 6165 - FOREACH_ITEM_SAFE(it, bck, R, end, struct foo*, L)\001
+4 0 0 50 0 12 8 0.0000 4 105 3750 7875 6390 does the same except that <bck> allows to delete\001
+4 0 0 50 0 12 8 0.0000 4 105 1950 7875 6570 any node, including <it>\001
+4 1 0 50 0 14 11 0.0000 4 135 1155 945 585 struct list\001
diff --git a/doc/internals/list.png b/doc/internals/list.png
new file mode 100644
index 0000000..ec41a6b
--- /dev/null
+++ b/doc/internals/list.png
Binary files differ
diff --git a/doc/internals/listener-states.fig b/doc/internals/listener-states.fig
new file mode 100644
index 0000000..863e7f5
--- /dev/null
+++ b/doc/internals/listener-states.fig
@@ -0,0 +1,150 @@
+#FIG 3.2 Produced by xfig version 2.3
+Portrait
+Center
+Metric
+A4
+300.00
+Single
+-2
+1200 2
+0 32 #ff60e0
+0 33 #ff8020
+0 34 #56c5ff
+0 35 #55d941
+0 36 #f8e010
+1 1 0 3 0 7 51 -1 20 0.000 1 0.0000 900 450 495 225 900 450 1395 450
+1 1 0 3 0 7 51 -1 20 0.000 1 0.0000 2700 450 495 225 2700 450 3195 450
+1 1 0 3 0 7 51 -1 20 0.000 1 0.0000 4500 450 495 225 4500 450 4995 450
+1 1 0 3 0 7 51 -1 20 0.000 1 0.0000 900 3465 495 225 900 3465 1395 3465
+1 1 0 3 0 7 51 -1 20 0.000 1 0.0000 2700 2475 495 225 2700 2475 3195 2475
+1 1 0 3 0 7 51 -1 20 0.000 1 0.0000 3645 1575 495 225 3645 1575 4140 1575
+1 1 0 3 0 7 51 -1 20 0.000 1 0.0000 4500 2475 495 225 4500 2475 4995 2475
+1 1 0 3 0 7 51 -1 20 0.000 1 0.0000 2700 3471 495 225 2700 3471 3195 3471
+2 1 1 3 1 7 52 -1 -1 8.000 1 0 -1 0 0 2
+ 270 1980 5355 1350
+2 2 0 2 32 32 52 -1 20 0.000 1 0 -1 0 0 5
+ 2070 3060 3330 3060 3330 3870 2070 3870 2070 3060
+2 3 0 1 33 33 53 -1 20 0.000 1 0 -1 0 0 5
+ 2070 990 5130 990 5130 2880 2070 2880 2070 990
+2 2 0 2 35 35 52 -1 20 0.000 1 0 -1 0 0 5
+ 270 90 5130 90 5130 855 270 855 270 90
+2 2 0 2 36 36 52 -1 20 0.000 1 0 -1 0 0 5
+ 270 3060 1530 3060 1530 3870 270 3870 270 3060
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 1395 450 2250 450
+ 0.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 3195 450 4050 450
+ 0.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 4095 1665 4455 2025 4500 2250
+ 0.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 3195 3510 3600 3465 4140 2655
+ 0.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 4410 2250 4365 2070 4050 1710
+ 0.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 945 3240 936 2142 2961 1917 3240 1710
+ 0.000 1.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 3195 1665 2835 1845 855 2115 855 3240
+ 0.000 1.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 5
+ 1 1 1.00 60.00 120.00
+ 990 3690 1035 3960 2880 4050 4365 3915 4410 2700
+ 0.000 1.000 1.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2700 2700 2700 3240
+ 0.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 4095 2610 3600 3375 3150 3420
+ 0.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 5
+ 1 1 1.00 60.00 120.00
+ 4500 2700 4455 4005 2655 4140 945 4005 900 3690
+ 0.000 1.000 1.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 2205 2520 1395 2745 1260 2970 1125 3240
+ 0.000 1.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 3510 1800 3330 2025 3330 2835 2970 3285
+ 0.000 1.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 1170 3285 1305 3015 1485 2790 2250 2610
+ 0.000 1.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 2205 3420 1710 3420 1395 3465
+ 0.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 1395 3510 1800 3510 2205 3465
+ 0.000 1.000 0.000
+3 0 0 3 0 7 51 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 2925 2295 3060 1980 3330 1755
+ 0.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 4500 675 4455 990 3960 1395
+ 0.000 1.000 0.000
+4 1 0 50 -1 18 10 0.0000 4 120 375 900 450 NEW\001
+4 1 0 50 -1 18 10 0.0000 4 120 315 2700 450 INIT\001
+4 1 0 50 -1 18 10 0.0000 4 120 810 4500 450 ASSIGNED\001
+4 1 1 50 -1 16 10 0.0000 4 120 90 900 630 0\001
+4 1 1 50 -1 16 10 0.0000 4 120 90 2700 630 1\001
+4 1 1 50 -1 16 10 0.0000 4 120 90 4500 630 2\001
+4 1 0 50 -1 16 7 0.0000 4 120 420 1755 405 create()\001
+4 2 0 50 -1 16 7 0.0000 4 120 660 1215 2160 enable() &&\001
+4 2 0 50 -1 16 7 0.0000 4 90 540 1080 2295 !maxconn\001
+4 2 1 51 -1 16 7 1.5708 4 105 600 5355 1485 transitions\001
+4 0 1 51 -1 16 7 1.5708 4 105 600 5355 1260 transitions\001
+4 2 1 51 -1 16 7 1.5708 4 105 795 5265 1485 multi-threaded\001
+4 0 1 51 -1 16 7 1.5708 4 120 870 5265 1260 single-threaded\001
+4 0 0 52 -1 17 7 0.0000 4 90 345 315 765 no FD\001
+4 0 0 52 -1 17 7 0.0000 4 135 315 315 3825 polled\001
+4 1 0 50 -1 18 10 0.0000 4 120 555 900 3465 READY\001
+4 0 0 50 -1 16 7 0.0000 4 120 255 1170 3825 full()\001
+4 2 0 50 -1 16 7 0.0000 4 90 540 2205 3375 !maxconn\001
+4 2 0 50 -1 16 7 0.0000 4 105 675 2295 3240 resume() &&\001
+4 0 0 50 -1 16 7 0.0000 4 105 405 1395 3645 pause()\001
+4 0 0 52 -1 17 7 0.0000 4 135 585 2115 3825 shut(sock)\001
+4 2 0 50 -1 16 7 0.0000 4 120 480 4320 2205 disable()\001
+4 2 0 50 -1 16 7 0.0000 4 105 405 4005 2655 pause()\001
+4 0 0 50 -1 16 7 0.0000 4 105 465 4545 2835 resume()\001
+4 2 0 50 -1 16 7 0.0000 4 120 480 2925 2160 disable()\001
+4 0 0 50 -1 16 7 0.0000 4 105 405 3465 1980 pause()\001
+4 0 0 50 -1 16 7 0.0000 4 120 660 4230 1710 enable() &&\001
+4 0 0 50 -1 16 7 0.0000 4 75 510 4320 1845 maxconn\001
+4 2 0 50 -1 16 7 0.0000 4 105 405 2655 2835 pause()\001
+4 0 0 50 -1 16 7 0.0000 4 105 675 3375 3555 resume() &&\001
+4 0 0 50 -1 16 7 0.0000 4 75 510 3375 3645 maxconn\001
+4 0 0 50 -1 16 7 0.0000 4 120 480 1080 2655 disable()\001
+4 2 0 50 -1 16 7 0.0000 4 105 465 2160 2475 resume()\001
+4 1 0 50 -1 16 7 0.0000 4 120 330 3555 405 .add()\001
+4 0 0 50 -1 16 7 0.0000 4 120 375 4545 810 .bind()\001
+4 0 0 52 -1 17 7 0.0000 4 135 1080 2115 1125 FD ready, not polled\001
+4 0 0 50 -1 16 7 0.0000 4 120 315 1305 3240 limit()\001
+4 1 0 50 -1 18 10 0.0000 4 120 630 2700 2475 LIMITED\001
+4 1 0 50 -1 18 10 0.0000 4 120 555 3645 1575 LISTEN\001
+4 1 0 50 -1 18 10 0.0000 4 120 375 4500 2475 FULL\001
+4 1 0 50 -1 18 10 0.0000 4 120 630 2700 3465 PAUSED\001
+4 1 1 50 -1 16 10 0.0000 4 120 90 2700 3645 3\001
+4 1 1 50 -1 16 10 0.0000 4 120 90 2700 2655 7\001
+4 1 1 50 -1 16 10 0.0000 4 120 90 4500 2655 6\001
+4 1 1 50 -1 16 10 0.0000 4 120 90 900 3645 5\001
+4 1 1 50 -1 16 10 0.0000 4 120 90 3645 1755 4\001
diff --git a/doc/internals/listener-states.png b/doc/internals/listener-states.png
new file mode 100644
index 0000000..8757a12
--- /dev/null
+++ b/doc/internals/listener-states.png
Binary files differ
diff --git a/doc/internals/lua_socket.fig b/doc/internals/lua_socket.fig
new file mode 100644
index 0000000..7da3294
--- /dev/null
+++ b/doc/internals/lua_socket.fig
@@ -0,0 +1,113 @@
+#FIG 3.2 Produced by xfig version 1.8
+Landscape
+Center
+Metric
+A4
+100.00
+Single
+-2
+1200 2
+6 1125 2745 2565 3555
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 1125 2745 2565 2745 2565 3555 1125 3555 1125 2745
+4 0 0 50 -1 16 12 0.0000 4 180 1080 1215 3195 lua_State *T\001
+4 0 0 50 -1 18 12 0.0000 4 150 990 1215 2925 struct hlua\001
+4 0 0 50 -1 16 12 0.0000 4 195 1245 1215 3465 stop_list *stop\001
+-6
+6 7560 4365 10620 5265
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 7650 4635 10530 4635 10530 5175 7650 5175 7650 4635
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 7560 4365 10620 4365 10620 5265 7560 5265 7560 4365
+4 0 0 50 -1 18 12 0.0000 4 195 2565 7740 4815 struct stream_interface si[0]\001
+4 0 0 50 -1 16 12 0.0000 4 195 1725 7740 5085 enum obj_type *end\001
+4 0 0 50 -1 18 12 0.0000 4 150 1215 7650 4545 struct stream\001
+-6
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 225 4500 2745 4500
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 225 5040 2745 5040
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 225 4770 2745 4770
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 1935 5715 7740 6705
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2520 3420 3600 4095
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 225 4230 2745 4230 2745 7020 225 7020 225 4230
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 225 6300 2745 6300
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 225 6660 2745 6660
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 1035 2205 2655 2205 2655 3645 1035 3645 1035 2205
+2 1 1 4 4 7 500 -1 -1 4.000 0 0 -1 0 0 2
+ 4860 1935 4860 9225
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 7695 6435 5760 4410
+2 2 0 1 0 7 50 -1 20 0.000 0 0 -1 0 0 5
+ 3600 3915 6075 3915 6075 4410 3600 4410 3600 3915
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 9450 5040 9225 5670
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 4
+ 7740 6300 7695 6345 7695 6525 7740 6570
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 7560 5670 9765 5670 9765 7200 7560 7200 7560 5670
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 7650 5940 9675 5940 9675 7110 7650 7110 7650 5940
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 315 5310 2655 5310 2655 6165 315 6165 315 5310
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 7830 6840 2565 5580
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 7740 6705 9540 6705 9540 6930 7740 6930 7740 6705
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 405 5580 2565 5580 2565 5805 405 5805 405 5580
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 5
+ 1 1 1.00 60.00 120.00
+ 1215 3105 765 3330 720 3555 765 3915 810 4230
+ 0.000 1.000 1.000 1.000 0.000
+3 0 1 1 13 7 50 -1 -1 1.000 0 1 0 3
+ 5 1 1.00 60.00 120.00
+ 675 7020 675 7785 900 8104
+ 0.000 1.000 0.000
+3 0 1 1 13 7 50 -1 -1 1.000 0 1 0 2
+ 5 1 1.00 60.00 120.00
+ 7740 7200 7740 8100
+ 0.000 0.000
+3 0 1 1 13 7 50 -1 -1 1.000 0 1 0 3
+ 5 1 1.00 60.00 120.00
+ 7605 7200 7605 8865 7740 9000
+ 0.000 1.000 0.000
+4 0 0 50 -1 18 12 0.0000 4 150 885 315 4410 stack Lua\001
+4 0 0 50 -1 16 12 0.0000 4 195 1140 315 4680 stack entry 0\001
+4 0 0 50 -1 16 12 0.0000 4 195 1140 315 4950 stack entry 1\001
+4 0 0 50 -1 16 12 0.0000 4 195 1140 315 5220 stack entry 2\001
+4 0 0 50 -1 18 12 0.0000 4 195 1695 405 5490 struct hlua_socket\001
+4 0 0 50 -1 16 12 0.0000 4 195 1140 315 6570 stack entry 3\001
+4 0 0 50 -1 16 12 0.0000 4 195 1140 315 6930 stack entry 4\001
+4 1 12 50 -1 12 9 5.6723 4 135 540 3150 3735 (list)\001
+4 0 0 50 -1 18 12 0.0000 4 150 1305 1125 2430 struct session\001
+4 0 0 50 -1 16 12 0.0000 4 150 1440 1125 2655 struct task *task\001
+4 0 0 50 -1 12 12 0.0000 4 165 1560 990 8100 hlua_tcp_gc()\001
+4 0 0 50 -1 16 12 0.0000 4 195 2430 990 8295 Called just before the object\001
+4 0 0 50 -1 16 12 0.0000 4 195 840 990 8535 garbaging\001
+4 1 12 50 -1 12 9 5.5327 4 135 540 6390 4905 (list)\001
+4 0 0 50 -1 18 12 0.0000 4 195 2205 3690 4095 struct hlua_socket_com\001
+4 0 0 50 -1 16 12 0.0000 4 150 1440 3690 4320 struct task *task\001
+4 0 0 50 -1 18 12 0.0000 4 195 1200 7650 5850 struct appctx\001
+4 0 0 50 -1 18 12 0.0000 4 150 1110 7740 6120 struct <lua>\001
+4 0 0 50 -1 16 12 0.0000 4 195 1620 7740 6615 struct hlua_tcp *wr\001
+4 0 0 50 -1 16 12 0.0000 4 195 1590 7740 6390 struct hlua_tcp *rd\001
+4 0 0 50 -1 12 12 0.0000 4 165 2160 7875 9000 hlua_tcp_release()\001
+4 0 0 50 -1 16 12 0.0000 4 195 3150 7875 9195 Called when the applet is destroyed.\001
+4 0 0 50 -1 12 12 0.0000 4 165 2400 7875 8100 update_tcp_handler()\001
+4 0 0 50 -1 16 12 0.0000 4 195 2640 7875 8295 Called on each change on the \001
+4 0 0 50 -1 16 12 0.0000 4 195 1830 7875 8535 tcp connection state.\001
+4 0 0 50 -1 16 12 0.0000 4 150 1350 495 5760 struct xref *xref\001
+4 0 0 50 -1 16 12 0.0000 4 150 1350 7830 6885 struct xref *xref\001
diff --git a/doc/internals/lua_socket.pdf b/doc/internals/lua_socket.pdf
new file mode 100644
index 0000000..e3b80ee
--- /dev/null
+++ b/doc/internals/lua_socket.pdf
Binary files differ
diff --git a/doc/internals/muxes.fig b/doc/internals/muxes.fig
new file mode 100644
index 0000000..babdd55
--- /dev/null
+++ b/doc/internals/muxes.fig
@@ -0,0 +1,401 @@
+#FIG 3.2 Produced by xfig version 3.2.8b
+Landscape
+Center
+Inches
+Letter
+100.00
+Single
+-1
+1200 2
+0 32 #bbf2e2
+0 33 #a7ceb3
+0 34 #dae8fc
+0 35 #458dba
+0 36 #ffe6cc
+0 37 #e9b000
+0 38 #1a1a1a
+0 39 #8e8e8e
+0 40 #ffc1e7
+6 4200 8700 4800 9825
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 4261 9751 4261 8751 4761 8751 4761 9751
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 4761 9751 4761 8751 4261 8751 4261 9751 4761 9751
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 4261 8850 4761 8850
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 4261 8925 4761 8925
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 4261 9000 4761 9000
+-6
+6 1425 3525 2025 4650
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 1486 4576 1486 3576 1986 3576 1986 4576
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 1986 4576 1986 3576 1486 3576 1486 4576 1986 4576
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 1486 3675 1986 3675
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 1486 3750 1986 3750
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 1486 3825 1986 3825
+-6
+6 3225 3525 3825 4650
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 3286 4576 3286 3576 3786 3576 3786 4576
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 3786 4576 3786 3576 3286 3576 3286 4576 3786 4576
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 3286 3675 3786 3675
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 3286 3750 3786 3750
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 3286 3825 3786 3825
+-6
+6 5025 3525 5625 4650
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 5086 4576 5086 3576 5586 3576 5586 4576
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 5586 4576 5586 3576 5086 3576 5086 4576 5586 4576
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 5086 3675 5586 3675
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 5086 3750 5586 3750
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 5086 3825 5586 3825
+-6
+6 6900 3525 7500 4650
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 6961 4576 6961 3576 7461 3576 7461 4576
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 7461 4576 7461 3576 6961 3576 6961 4576 7461 4576
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 6961 3675 7461 3675
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 6961 3750 7461 3750
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 6961 3825 7461 3825
+-6
+6 11925 10725 13875 11475
+2 4 0 3 0 35 50 -1 20 0.000 1 0 7 0 0 5
+ 13800 11400 12000 11400 12000 10800 13800 10800 13800 11400
+4 1 0 49 -1 4 18 0.0000 4 285 1335 12900 11175 Transport\001
+-6
+6 6600 1200 10050 1800
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 6692 1261 9959 1261 9959 1761 6692 1761
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 6692 1761 9959 1761 9959 1261 6692 1261 6692 1761
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9750 1261 9750 1761
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9525 1261 9525 1761
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9300 1261 9300 1761
+4 1 0 46 -1 4 16 0.0000 4 210 1605 8025 1575 channel buf\001
+-6
+6 12375 8100 12900 8700
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 12600 8161 12600 8661
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 12425 8161 12825 8161 12825 8661 12425 8661
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 12425 8661 12825 8661 12825 8161 12425 8161 12425 8661
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 12675 8161 12675 8661
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 12750 8161 12750 8661
+-6
+6 11700 8100 12225 8700
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 11925 8161 11925 8661
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 11750 8161 12150 8161 12150 8661 11750 8661
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 11750 8661 12150 8661 12150 8161 11750 8161 11750 8661
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 12000 8161 12000 8661
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 12075 8161 12075 8661
+-6
+6 11025 8100 11550 8700
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 11250 8161 11250 8661
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 11075 8161 11475 8161 11475 8661 11075 8661
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 11075 8661 11475 8661 11475 8161 11075 8161 11075 8661
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 11325 8161 11325 8661
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 11400 8161 11400 8661
+-6
+6 10350 8100 10875 8700
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 10575 8161 10575 8661
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 10400 8161 10800 8161 10800 8661 10400 8661
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 10400 8661 10800 8661 10800 8161 10400 8161 10400 8661
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 10650 8161 10650 8661
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 10725 8161 10725 8661
+-6
+6 13050 8100 13575 8700
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 13275 8161 13275 8661
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 13100 8161 13500 8161 13500 8661 13100 8661
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 13100 8661 13500 8661 13500 8161 13100 8161 13100 8661
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 13350 8161 13350 8661
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 13425 8161 13425 8661
+-6
+6 13725 8100 14250 8700
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 13950 8161 13950 8661
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 13775 8161 14175 8161 14175 8661 13775 8661
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 13775 8661 14175 8661 14175 8161 13775 8161 13775 8661
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 14025 8161 14025 8661
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 14100 8161 14100 8661
+-6
+6 11100 11700 13050 12150
+1 1 0 4 20 40 49 -1 20 0.000 1 0.0000 11400 11925 225 150 11400 11925 11625 12075
+4 0 0 49 -1 4 12 0.0000 4 165 960 11850 12000 I/O tasklet\001
+-6
+6 11100 12300 11700 12600
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 11357 12331 11357 12581
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 11157 12331 11614 12331 11614 12581 11157 12581
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 11157 12581 11614 12581 11614 12331 11157 12331 11157 12581
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 11443 12331 11443 12581
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 11529 12331 11529 12581
+-6
+1 3 0 3 0 0 49 -1 20 0.000 1 0.0000 10725 5700 75 75 10725 5700 10800 5700
+1 3 0 3 0 0 49 -1 20 0.000 1 0.0000 12750 5700 75 75 12750 5700 12825 5700
+1 3 0 3 0 0 49 -1 20 0.000 1 0.0000 13875 5700 75 75 13875 5700 13950 5700
+1 3 0 3 0 0 49 -1 20 0.000 1 0.0000 11700 5700 75 75 11700 5700 11775 5700
+1 3 0 3 0 0 49 -1 20 0.000 1 0.0000 2925 6750 75 75 2925 6750 3000 6750
+1 3 0 3 0 0 49 -1 20 0.000 1 0.0000 4950 6750 75 75 4950 6750 5025 6750
+1 3 0 3 0 0 49 -1 20 0.000 1 0.0000 6075 6750 75 75 6075 6750 6150 6750
+1 3 0 3 0 0 49 -1 20 0.000 1 0.0000 3900 6750 75 75 3900 6750 3975 6750
+1 1 0 4 37 36 49 -1 20 0.000 1 0.0000 9525 4140 583 250 9525 4140 10108 3890
+1 1 0 4 37 36 49 -1 20 0.000 1 0.0000 11341 4140 583 250 11341 4140 11924 3890
+1 1 0 4 37 36 49 -1 20 0.000 1 0.0000 13154 4140 583 250 13154 4140 13737 3890
+1 1 0 4 37 36 49 -1 20 0.000 1 0.0000 15033 4140 583 250 15033 4140 15616 3890
+1 1 0 4 37 36 49 -1 20 0.000 1 0.0000 7182 5173 583 250 7182 5173 7765 4923
+1 1 0 4 37 36 49 -1 20 0.000 1 0.0000 3507 5173 583 250 3507 5173 4090 4923
+1 1 0 4 37 36 49 -1 20 0.000 1 0.0000 1719 5173 583 250 1719 5173 2302 4923
+1 1 0 4 37 36 49 -1 20 0.000 1 0.0000 5325 5175 583 250 5325 5175 5908 4925
+1 1 0 4 10 11 45 -1 20 0.000 1 0.0000 4488 8082 612 250 4488 8082 5100 8082
+1 1 0 4 10 11 49 -1 20 0.000 1 0.0000 12333 7025 417 250 12333 7025 12750 7025
+1 1 0 4 20 40 49 -1 20 0.000 1 0.0000 12392 9240 808 210 12392 9240 13200 9240
+1 1 0 4 20 40 49 -1 20 0.000 1 0.0000 3167 9240 808 210 3167 9240 3975 9240
+1 1 0 4 37 36 49 -1 20 0.000 1 0.0000 1800 11925 225 150 1800 11925 2025 12075
+1 1 0 4 10 11 45 -1 20 0.000 1 0.0000 6600 11925 225 150 6600 11925 6825 12075
+1 1 0 4 20 40 49 -1 20 0.000 1 0.0000 8400 600 900 210 8400 600 9300 600
+2 1 1 1 0 7 49 -1 -1 4.000 1 0 -1 0 0 2
+ 2550 3300 2550 6150
+2 1 1 1 0 7 49 -1 -1 4.000 1 0 -1 0 0 2
+ 4500 3300 4500 6150
+2 1 1 1 0 7 49 -1 -1 4.000 1 0 -1 0 0 2
+ 6300 3300 6300 6150
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 90.00 180.00
+ 600 8025 600 12225
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 90.00 180.00
+ 600 3150 600 1800
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 90.00 180.00
+ 600 1500 600 150
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 1 1 4
+ 1 1 1.00 90.00 180.00
+ 1 1 1.00 90.00 180.00
+ 3000 3300 3000 1425 3675 600 7500 600
+2 3 0 4 33 32 50 -1 20 0.000 0 0 -1 0 0 5
+ 900 3300 900 9900 8100 9900 8100 3300 900 3300
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 1 0 4
+ 1 1 1.00 90.00 180.00
+ 3525 3525 3525 2625 4500 1500 6750 1500
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 1 0 3
+ 1 1 1.00 90.00 180.00
+ 11295 4425 11295 4725 11700 5625
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 1 0 3
+ 1 1 1.00 90.00 180.00
+ 9495 4425 9495 4725 10695 5700
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 1 0 3
+ 1 1 1.00 90.00 180.00
+ 13163 4425 13163 4725 12788 5625
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 1 0 3
+ 1 1 1.00 90.00 180.00
+ 15013 4427 15013 4725 13888 5702
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 9525 3525 9525 3825
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 13125 3525 13125 3825
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 15000 3525 15000 3825
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 5
+ 1 1 1.00 90.00 180.00
+ 12300 7275 12300 7725 9975 7725 9975 8400 10425 8400
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 1 0 3
+ 1 1 1.00 90.00 180.00
+ 11775 5850 12300 6450 12300 6825
+2 1 1 3 0 7 49 -1 -1 8.000 1 0 -1 0 0 3
+ 11475 6150 13200 6150 13200 6825
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 0 1 3
+ 1 1 1.00 90.00 180.00
+ 3975 6900 4500 7650 4500 7875
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 0 1 3
+ 1 1 1.00 90.00 180.00
+ 3495 5475 3495 5775 3900 6675
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 0 1 3
+ 1 1 1.00 90.00 180.00
+ 1695 5475 1695 5775 2895 6750
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 0 1 3
+ 1 1 1.00 90.00 180.00
+ 7213 5477 7213 5775 6088 6752
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 1725 4875 1725 4575
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 3525 4875 3525 4575
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 5325 4875 5325 4575
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 7200 4875 7200 4575
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 0 1 2
+ 1 1 1.00 90.00 180.00
+ 4500 8325 4500 8721
+2 1 1 3 0 7 49 -1 -1 8.000 1 0 -1 0 0 3
+ 3225 7875 3225 7350 4725 7350
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 1 1 2
+ 1 1 1.00 90.00 180.00
+ 1 1 1.00 90.00 180.00
+ 3900 10800 3225 9450
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 90.00 180.00
+ 4500 10800 4500 9750
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 1 1 3
+ 1 1 1.00 90.00 180.00
+ 1 1 1.00 90.00 180.00
+ 12375 10800 12375 9750 12375 9450
+2 1 1 1 0 7 49 -1 -1 4.000 1 0 -1 0 0 2
+ 12225 3300 12225 5025
+2 1 1 1 0 7 49 -1 -1 4.000 1 0 -1 0 0 2
+ 10425 3300 10425 5025
+2 1 1 1 0 7 49 -1 -1 4.000 1 0 -1 0 0 2
+ 14025 3300 14025 5025
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 4
+ 1 1 1.00 90.00 180.00
+ 9975 1500 10800 1500 11325 2100 11325 3825
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 1 4
+ 1 1 1.00 90.00 180.00
+ 1 1 1.00 90.00 180.00
+ 9300 600 11175 600 11775 1275 11775 3300
+2 3 0 4 33 32 50 -1 20 0.000 0 0 -1 0 0 5
+ 8700 3300 8700 9900 15900 9900 15900 3300 8700 3300
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 0 1 5
+ 1 1 1.00 90.00 180.00
+ 13200 10800 13200 10200 14625 9750 14625 8400 14175 8400
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 0 1 3
+ 1 1 1.00 90.00 180.00
+ 5325 5475 5325 5775 4950 6675
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 90.00 180.00
+ 600 5400 600 3300
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 0 0 2
+ 600 7800 600 5700
+2 4 0 3 0 35 50 -1 20 0.000 1 0 7 0 0 5
+ 5400 11400 3600 11400 3600 10800 5400 10800 5400 11400
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 12150 8400 12450 8400
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 11475 8400 11775 8400
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 10800 8400 11100 8400
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 12825 8400 13125 8400
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 13500 8400 13800 8400
+2 4 0 3 0 35 50 -1 20 0.000 1 0 7 0 0 5
+ 2100 12600 1575 12600 1575 12300 2100 12300 2100 12600
+2 4 0 3 33 32 50 -1 20 0.000 1 0 7 0 0 5
+ 6900 12600 6375 12600 6375 12300 6900 12300 6900 12600
+4 1 0 49 -1 4 14 1.5708 4 225 1335 450 825 application\001
+4 0 0 49 -1 4 12 1.5708 4 180 2595 2850 3225 mux->subscribe(SUB_RECV)\001
+4 1 0 46 -1 4 16 1.5708 4 210 645 3600 4200 rxbuf\001
+4 1 0 46 -1 4 16 1.5708 4 210 615 4575 9375 dbuf\001
+4 1 0 49 -1 4 16 0.0000 4 210 600 12300 7125 MUX\001
+4 1 0 44 -1 4 16 0.0000 4 210 945 4500 8175 DEMUX\001
+4 2 0 49 -1 4 12 0.0000 4 150 915 3600 8100 Stream ID\001
+4 0 0 49 -1 4 12 0.0000 4 150 915 12825 7125 Stream ID\001
+4 2 0 49 -1 4 12 0.0000 4 180 1635 3300 10125 tasklet_wakeup()\001
+4 2 0 49 -1 4 12 0.0000 4 180 1635 12150 10125 tasklet_wakeup()\001
+4 2 0 49 -1 4 12 0.0000 4 180 1470 11175 3150 mux->snd_buf()\001
+4 0 0 49 -1 4 12 0.0000 4 180 1425 3675 3225 mux->rcv_buf()\001
+4 0 0 49 -1 4 12 0.0000 4 180 1920 13425 10575 xprt->snd_buf(mbuf)\001
+4 0 0 49 -1 4 12 0.0000 4 180 1830 4725 10500 xprt->rcv_buf(dbuf)\001
+4 1 0 49 -1 4 12 0.0000 4 150 3105 8400 2100 HTX contents when mode==HTTP\001
+4 2 0 49 -1 4 12 0.0000 4 180 1635 7500 450 tasklet_wakeup()\001
+4 0 0 49 -1 4 12 0.0000 4 180 1635 9300 450 tasklet_wakeup()\001
+4 1 38 48 -1 4 12 0.0000 4 150 750 9534 4200 encode\001
+4 1 38 48 -1 4 12 0.0000 4 150 750 11325 4200 encode\001
+4 1 38 48 -1 4 12 0.0000 4 150 750 13134 4200 encode\001
+4 1 38 48 -1 4 12 0.0000 4 150 750 15009 4200 encode\001
+4 1 38 48 -1 4 12 0.0000 4 150 765 1725 5250 decode\001
+4 1 38 48 -1 4 12 0.0000 4 150 765 3525 5250 decode\001
+4 1 38 48 -1 4 12 0.0000 4 150 765 5325 5250 decode\001
+4 1 38 48 -1 4 12 0.0000 4 150 765 7200 5250 decode\001
+4 1 38 48 -1 4 12 0.0000 4 180 1035 12375 9300 mux_io_cb\001
+4 0 0 49 -1 4 12 1.5708 4 180 2580 12075 3225 mux->subscribe(SUB_SEND)\001
+4 1 0 49 -1 4 14 1.5708 4 180 1425 450 4500 mux streams\001
+4 1 0 49 -1 4 14 1.5708 4 135 1980 450 6750 mux=conn->mux\001
+4 1 0 49 -1 4 18 0.0000 4 285 1335 4500 11175 Transport\001
+4 1 0 46 -1 4 16 0.0000 4 210 690 14625 8175 mbuf\001
+4 1 38 48 -1 4 12 0.0000 4 180 1035 3159 9300 mux_io_cb\001
+4 0 0 49 -1 4 12 0.0000 4 195 2805 2250 12000 encoding/decoding function\001
+4 0 0 49 -1 4 12 0.0000 4 180 1365 2250 12525 transport layer\001
+4 0 0 49 -1 4 12 0.0000 4 180 2445 7050 12525 multiplexer (MUX/DEMUX)\001
+4 0 0 49 -1 4 12 0.0000 4 195 2655 7050 12000 general processing function\001
+4 0 0 49 -1 4 12 0.0000 4 180 2820 11775 12525 stream buffer (byte-level FIFO)\001
+4 2 0 49 -1 4 12 0.0000 4 180 2550 3675 10725 xprt->subscribe(SUB_RECV)\001
+4 2 0 49 -1 4 12 0.0000 4 180 2535 12225 10725 xprt->subscribe(SUB_SEND)\001
+4 1 0 49 -1 4 14 1.5708 4 180 780 450 2550 stconn\001
+4 1 0 49 -1 4 12 1.5708 4 195 2010 900 1125 (eg: checks, streams)\001
+4 1 0 49 -1 4 14 1.5708 4 180 3720 450 10125 connection = sc->sedesc->conn\001
+4 0 0 49 -1 4 12 0.0000 4 150 600 12225 225 Notes:\001
+4 0 0 49 -1 4 12 0.0000 4 180 2220 12975 675 snd_buf() will move the\001
+4 0 0 49 -1 4 12 0.0000 4 180 2310 12975 975 buffer (zero-copy) when\001
+4 0 0 49 -1 4 12 0.0000 4 180 2310 12975 1275 the destination is empty.\001
+4 0 0 49 -1 4 12 0.0000 4 180 2220 12825 1650 - the application is also\001
+4 0 0 49 -1 4 12 0.0000 4 180 2700 12975 2250 is sc->app with sc->app_ops\001
+4 0 0 49 -1 4 12 0.0000 4 180 2490 12825 2550 - transport layers (xprt) are\001
+4 0 0 49 -1 4 12 0.0000 4 180 2250 12975 2775 stackable. conn->xprt is\001
+4 0 0 49 -1 4 12 0.0000 4 180 1635 12975 3000 the topmost one.\001
+4 0 0 49 -1 4 12 0.0000 4 180 2400 12975 1950 called the app layer and\001
+4 0 0 49 -1 4 12 0.0000 4 180 1995 12825 375 - mux->rcv_buf() and\001
+4 1 38 48 -1 4 12 0.0000 4 180 1440 8409 657 sc_conn_io_cb\001
diff --git a/doc/internals/muxes.pdf b/doc/internals/muxes.pdf
new file mode 100644
index 0000000..54f8cc7
--- /dev/null
+++ b/doc/internals/muxes.pdf
Binary files differ
diff --git a/doc/internals/muxes.png b/doc/internals/muxes.png
new file mode 100644
index 0000000..a58f42f
--- /dev/null
+++ b/doc/internals/muxes.png
Binary files differ
diff --git a/doc/internals/muxes.svg b/doc/internals/muxes.svg
new file mode 100644
index 0000000..3feaa4d
--- /dev/null
+++ b/doc/internals/muxes.svg
@@ -0,0 +1,911 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Creator: fig2dev Version 3.2.8b -->
+<!-- CreationDate: 2022-05-27 11:37:43 -->
+<!-- Magnification: 1 -->
+<svg xmlns="http://www.w3.org/2000/svg"
+ xmlns:xlink="http://www.w3.org/1999/xlink"
+ width="942pt" height="755pt"
+ viewBox="254 60 15690 12573">
+<g fill="none">
+<!-- Line -->
+<rect x="12000" y="10800" width="1800" height="600" rx="105" fill="#458dba"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Line -->
+<polygon points=" 900,3300 900,9900 8100,9900 8100,3300" fill="#bbf2e2"
+ stroke="#a7ceb3" stroke-width="45px"/>
+<!-- Line -->
+<polygon points=" 8700,3300 8700,9900 15900,9900 15900,3300" fill="#bbf2e2"
+ stroke="#a7ceb3" stroke-width="45px"/>
+<!-- Line -->
+<rect x="3600" y="10800" width="1800" height="600" rx="105" fill="#458dba"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Line -->
+<rect x="1575" y="12300" width="525" height="300" rx="105" fill="#458dba"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Line -->
+<rect x="6375" y="12300" width="525" height="300" rx="105" fill="#bbf2e2"
+ stroke="#a7ceb3" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Line -->
+<polygon points=" 4761,9751 4761,8751 4261,8751 4261,9751" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 1986,4576 1986,3576 1486,3576 1486,4576" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 3786,4576 3786,3576 3286,3576 3286,4576" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 5586,4576 5586,3576 5086,3576 5086,4576" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 7461,4576 7461,3576 6961,3576 6961,4576" fill="#dae8fc"/>
+<!-- Text -->
+<text xml:space="preserve" x="12900" y="11175" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="216" text-anchor="middle">Transport</text>
+<!-- Line -->
+<polygon points=" 6692,1761 9959,1761 9959,1261 6692,1261" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 12425,8661 12825,8661 12825,8161 12425,8161" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 11750,8661 12150,8661 12150,8161 11750,8161" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 11075,8661 11475,8661 11475,8161 11075,8161" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 10400,8661 10800,8661 10800,8161 10400,8161" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 13100,8661 13500,8661 13500,8161 13100,8161" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 13775,8661 14175,8661 14175,8161 13775,8161" fill="#dae8fc"/>
+<!-- Ellipse -->
+<ellipse cx="11400" cy="11925" rx="225" ry="150" fill="#ffc1e7"
+ stroke="#d10000" stroke-width="45px"/>
+<!-- Text -->
+<text xml:space="preserve" x="11850" y="12000" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">I/O tasklet</text>
+<!-- Line -->
+<polygon points=" 11157,12581 11614,12581 11614,12331 11157,12331" fill="#dae8fc"/>
+<!-- Circle -->
+<circle cx="10725" cy="5700" r="75" fill="#000000"
+ stroke="#000000" stroke-width="30px"/>
+<!-- Circle -->
+<circle cx="12750" cy="5700" r="75" fill="#000000"
+ stroke="#000000" stroke-width="30px"/>
+<!-- Circle -->
+<circle cx="13875" cy="5700" r="75" fill="#000000"
+ stroke="#000000" stroke-width="30px"/>
+<!-- Circle -->
+<circle cx="11700" cy="5700" r="75" fill="#000000"
+ stroke="#000000" stroke-width="30px"/>
+<!-- Circle -->
+<circle cx="2925" cy="6750" r="75" fill="#000000"
+ stroke="#000000" stroke-width="30px"/>
+<!-- Circle -->
+<circle cx="4950" cy="6750" r="75" fill="#000000"
+ stroke="#000000" stroke-width="30px"/>
+<!-- Circle -->
+<circle cx="6075" cy="6750" r="75" fill="#000000"
+ stroke="#000000" stroke-width="30px"/>
+<!-- Circle -->
+<circle cx="3900" cy="6750" r="75" fill="#000000"
+ stroke="#000000" stroke-width="30px"/>
+<!-- Ellipse -->
+<ellipse cx="9525" cy="4140" rx="583" ry="250" fill="#ffe6cc"
+ stroke="#e9b000" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="11341" cy="4140" rx="583" ry="250" fill="#ffe6cc"
+ stroke="#e9b000" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="13154" cy="4140" rx="583" ry="250" fill="#ffe6cc"
+ stroke="#e9b000" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="15033" cy="4140" rx="583" ry="250" fill="#ffe6cc"
+ stroke="#e9b000" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="7182" cy="5173" rx="583" ry="250" fill="#ffe6cc"
+ stroke="#e9b000" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="3507" cy="5173" rx="583" ry="250" fill="#ffe6cc"
+ stroke="#e9b000" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="1719" cy="5173" rx="583" ry="250" fill="#ffe6cc"
+ stroke="#e9b000" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="5325" cy="5175" rx="583" ry="250" fill="#ffe6cc"
+ stroke="#e9b000" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="12333" cy="7025" rx="417" ry="250" fill="#87cfff"
+ stroke="#0000d1" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="12392" cy="9240" rx="808" ry="210" fill="#ffc1e7"
+ stroke="#d10000" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="3167" cy="9240" rx="808" ry="210" fill="#ffc1e7"
+ stroke="#d10000" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="1800" cy="11925" rx="225" ry="150" fill="#ffe6cc"
+ stroke="#e9b000" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="8400" cy="600" rx="900" ry="210" fill="#ffc1e7"
+ stroke="#d10000" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 2550,3300 2550,6150"
+ stroke="#000000" stroke-width="8px" stroke-linejoin="round" stroke-dasharray="40 40"/>
+<!-- Line -->
+<polyline points=" 4500,3300 4500,6150"
+ stroke="#000000" stroke-width="8px" stroke-linejoin="round" stroke-dasharray="40 40"/>
+<!-- Line -->
+<polyline points=" 6300,3300 6300,6150"
+ stroke="#000000" stroke-width="8px" stroke-linejoin="round" stroke-dasharray="40 40"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp0">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 645,12029 555,12029 582,12243 618,12243z"/>
+</clipPath>
+</defs>
+<polyline points=" 600,8025 600,12225" clip-path="url(#cp0)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 600,12225 -->
+<polygon points=" 555,12029 600,12209 645,12029 555,12029"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp1">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 555,1996 645,1996 618,1782 582,1782z"/>
+</clipPath>
+</defs>
+<polyline points=" 600,3150 600,1800" clip-path="url(#cp1)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 600,1800 -->
+<polygon points=" 645,1996 600,1816 555,1996 645,1996"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp2">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 555,346 645,346 618,132 582,132z"/>
+</clipPath>
+</defs>
+<polyline points=" 600,1500 600,150" clip-path="url(#cp2)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 600,150 -->
+<polygon points=" 645,346 600,166 555,346 645,346"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp3">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 7304,555 7304,645 7518,618 7518,582z
+ M 3045,3104 2955,3104 2982,3318 3018,3318z"/>
+</clipPath>
+</defs>
+<polyline points=" 3000,3300 3000,1425 3675,600 7500,600" clip-path="url(#cp3)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 7500,600 -->
+<polygon points=" 7304,645 7484,600 7304,555 7304,645"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Backward arrow to point 3000,3300 -->
+<polygon points=" 2955,3104 3000,3284 3045,3104 2955,3104"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp4">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 6554,1455 6554,1545 6768,1518 6768,1482z
+ M 3045,3104 2955,3104 2982,3318 3018,3318z"/>
+</clipPath>
+</defs>
+<polyline points=" 3525,3525 3525,2625 4500,1500 6750,1500" clip-path="url(#cp4)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 6750,1500 -->
+<polygon points=" 6554,1545 6734,1500 6554,1455 6554,1545"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp5">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 11661,5428 11578,5465 11691,5649 11724,5634z
+ M 3045,3104 2955,3104 2982,3318 3018,3318z"/>
+</clipPath>
+</defs>
+<polyline points=" 11295,4425 11295,4725 11700,5625" clip-path="url(#cp5)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 11700,5625 -->
+<polygon points=" 11578,5465 11693,5610 11661,5428 11578,5465"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp6">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 10571,5541 10514,5611 10698,5725 10720,5697z
+ M 3045,3104 2955,3104 2982,3318 3018,3318z"/>
+</clipPath>
+</defs>
+<polyline points=" 9495,4425 9495,4725 10695,5700" clip-path="url(#cp6)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 10695,5700 -->
+<polygon points=" 10514,5611 10682,5690 10571,5541 10514,5611"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp7">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 12905,5461 12822,5427 12764,5635 12798,5649z
+ M 3045,3104 2955,3104 2982,3318 3018,3318z"/>
+</clipPath>
+</defs>
+<polyline points=" 13163,4425 13163,4725 12788,5625" clip-path="url(#cp7)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 12788,5625 -->
+<polygon points=" 12822,5427 12794,5610 12905,5461 12822,5427"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp8">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 14066,5607 14007,5539 13863,5700 13886,5727z
+ M 3045,3104 2955,3104 2982,3318 3018,3318z"/>
+</clipPath>
+</defs>
+<polyline points=" 15013,4427 15013,4725 13888,5702" clip-path="url(#cp8)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 13888,5702 -->
+<polygon points=" 14007,5539 13900,5691 14066,5607 14007,5539"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp9">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 9555,3689 9495,3689 9507,3843 9543,3843z
+ M 3045,3104 2955,3104 2982,3318 3018,3318z"/>
+</clipPath>
+</defs>
+<polyline points=" 9525,3525 9525,3825" clip-path="url(#cp9)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 9525,3825 -->
+<polygon points=" 9495,3689 9525,3809 9555,3689 9495,3689"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp10">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 13155,3689 13095,3689 13107,3843 13143,3843z
+ M 3045,3104 2955,3104 2982,3318 3018,3318z"/>
+</clipPath>
+</defs>
+<polyline points=" 13125,3525 13125,3825" clip-path="url(#cp10)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 13125,3825 -->
+<polygon points=" 13095,3689 13125,3809 13155,3689 13095,3689"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp11">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 15030,3689 14970,3689 14982,3843 15018,3843z
+ M 3045,3104 2955,3104 2982,3318 3018,3318z"/>
+</clipPath>
+</defs>
+<polyline points=" 15000,3525 15000,3825" clip-path="url(#cp11)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 15000,3825 -->
+<polygon points=" 14970,3689 15000,3809 15030,3689 14970,3689"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp12">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 10229,8355 10229,8445 10443,8418 10443,8382z
+ M 3045,3104 2955,3104 2982,3318 3018,3318z"/>
+</clipPath>
+</defs>
+<polyline points=" 12300,7275 12300,7725 9975,7725 9975,8400 10425,8400" clip-path="url(#cp12)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 10425,8400 -->
+<polygon points=" 10229,8445 10409,8400 10229,8355 10229,8445"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp13">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 12345,6629 12255,6629 12282,6843 12318,6843z
+ M 3045,3104 2955,3104 2982,3318 3018,3318z"/>
+</clipPath>
+</defs>
+<polyline points=" 11775,5850 12300,6450 12300,6825" clip-path="url(#cp13)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 12300,6825 -->
+<polygon points=" 12255,6629 12300,6809 12345,6629 12255,6629"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<polyline points=" 11475,6150 13200,6150 13200,6825"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round" stroke-dasharray="80 80"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp14">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 12345,6629 12255,6629 12282,6843 12318,6843z
+ M 4051,7087 4124,7035 3979,6875 3950,6896z"/>
+</clipPath>
+</defs>
+<polyline points=" 3975,6900 4500,7650 4500,7875" clip-path="url(#cp14)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Backward arrow to point 3975,6900 -->
+<polygon points=" 4124,7035 3984,6913 4051,7087 4124,7035"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp15">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 12345,6629 12255,6629 12282,6843 12318,6843z
+ M 3450,5671 3540,5671 3513,5457 3477,5457z"/>
+</clipPath>
+</defs>
+<polyline points=" 3495,5475 3495,5775 3900,6675" clip-path="url(#cp15)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Backward arrow to point 3495,5475 -->
+<polygon points=" 3540,5671 3495,5491 3450,5671 3540,5671"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp16">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 12345,6629 12255,6629 12282,6843 12318,6843z
+ M 1650,5671 1740,5671 1713,5457 1677,5457z"/>
+</clipPath>
+</defs>
+<polyline points=" 1695,5475 1695,5775 2895,6750" clip-path="url(#cp16)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Backward arrow to point 1695,5475 -->
+<polygon points=" 1740,5671 1695,5491 1650,5671 1740,5671"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp17">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 12345,6629 12255,6629 12282,6843 12318,6843z
+ M 7168,5673 7258,5673 7231,5459 7195,5459z"/>
+</clipPath>
+</defs>
+<polyline points=" 7213,5477 7213,5775 6088,6752" clip-path="url(#cp17)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Backward arrow to point 7213,5477 -->
+<polygon points=" 7258,5673 7213,5493 7168,5673 7258,5673"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp18">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 1695,4711 1755,4711 1743,4557 1707,4557z
+ M 7168,5673 7258,5673 7231,5459 7195,5459z"/>
+</clipPath>
+</defs>
+<polyline points=" 1725,4875 1725,4575" clip-path="url(#cp18)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 1725,4575 -->
+<polygon points=" 1755,4711 1725,4591 1695,4711 1755,4711"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp19">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 3495,4711 3555,4711 3543,4557 3507,4557z
+ M 7168,5673 7258,5673 7231,5459 7195,5459z"/>
+</clipPath>
+</defs>
+<polyline points=" 3525,4875 3525,4575" clip-path="url(#cp19)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 3525,4575 -->
+<polygon points=" 3555,4711 3525,4591 3495,4711 3555,4711"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp20">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 5295,4711 5355,4711 5343,4557 5307,4557z
+ M 7168,5673 7258,5673 7231,5459 7195,5459z"/>
+</clipPath>
+</defs>
+<polyline points=" 5325,4875 5325,4575" clip-path="url(#cp20)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 5325,4575 -->
+<polygon points=" 5355,4711 5325,4591 5295,4711 5355,4711"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp21">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 7170,4711 7230,4711 7218,4557 7182,4557z
+ M 7168,5673 7258,5673 7231,5459 7195,5459z"/>
+</clipPath>
+</defs>
+<polyline points=" 7200,4875 7200,4575" clip-path="url(#cp21)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 7200,4575 -->
+<polygon points=" 7230,4711 7200,4591 7170,4711 7230,4711"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp22">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 7170,4711 7230,4711 7218,4557 7182,4557z
+ M 4455,8521 4545,8521 4518,8307 4482,8307z"/>
+</clipPath>
+</defs>
+<polyline points=" 4500,8325 4500,8721" clip-path="url(#cp22)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Backward arrow to point 4500,8325 -->
+<polygon points=" 4545,8521 4500,8341 4455,8521 4545,8521"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<polyline points=" 3225,7875 3225,7350 4725,7350"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round" stroke-dasharray="80 80"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp23">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 3272,9646 3353,9605 3233,9426 3201,9442z
+ M 3853,10604 3772,10645 3892,10824 3924,10808z"/>
+</clipPath>
+</defs>
+<polyline points=" 3900,10800 3225,9450" clip-path="url(#cp23)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 3225,9450 -->
+<polygon points=" 3353,9605 3232,9464 3272,9646 3353,9605"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Backward arrow to point 3900,10800 -->
+<polygon points=" 3772,10645 3893,10786 3853,10604 3772,10645"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp24">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 4455,9946 4545,9946 4518,9732 4482,9732z
+ M 3853,10604 3772,10645 3892,10824 3924,10808z"/>
+</clipPath>
+</defs>
+<polyline points=" 4500,10800 4500,9750" clip-path="url(#cp24)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 4500,9750 -->
+<polygon points=" 4545,9946 4500,9766 4455,9946 4545,9946"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp25">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 12330,9646 12420,9646 12393,9432 12357,9432z
+ M 12420,10604 12330,10604 12357,10818 12393,10818z"/>
+</clipPath>
+</defs>
+<polyline points=" 12375,10800 12375,9750 12375,9450" clip-path="url(#cp25)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 12375,9450 -->
+<polygon points=" 12420,9646 12375,9466 12330,9646 12420,9646"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Backward arrow to point 12375,10800 -->
+<polygon points=" 12330,10604 12375,10784 12420,10604 12330,10604"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<polyline points=" 12225,3300 12225,5025"
+ stroke="#000000" stroke-width="8px" stroke-linejoin="round" stroke-dasharray="40 40"/>
+<!-- Line -->
+<polyline points=" 10425,3300 10425,5025"
+ stroke="#000000" stroke-width="8px" stroke-linejoin="round" stroke-dasharray="40 40"/>
+<!-- Line -->
+<polyline points=" 14025,3300 14025,5025"
+ stroke="#000000" stroke-width="8px" stroke-linejoin="round" stroke-dasharray="40 40"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp26">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 11370,3629 11280,3629 11307,3843 11343,3843z
+ M 12420,10604 12330,10604 12357,10818 12393,10818z"/>
+</clipPath>
+</defs>
+<polyline points=" 9975,1500 10800,1500 11325,2100 11325,3825" clip-path="url(#cp26)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 11325,3825 -->
+<polygon points=" 11280,3629 11325,3809 11370,3629 11280,3629"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp27">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 11820,3104 11730,3104 11757,3318 11793,3318z
+ M 9496,645 9496,555 9282,582 9282,618z"/>
+</clipPath>
+</defs>
+<polyline points=" 9300,600 11175,600 11775,1275 11775,3300" clip-path="url(#cp27)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 11775,3300 -->
+<polygon points=" 11730,3104 11775,3284 11820,3104 11730,3104"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Backward arrow to point 9300,600 -->
+<polygon points=" 9496,555 9316,600 9496,645 9496,555"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp28">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 11820,3104 11730,3104 11757,3318 11793,3318z
+ M 13245,10604 13155,10604 13182,10818 13218,10818z"/>
+</clipPath>
+</defs>
+<polyline points=" 13200,10800 13200,10200 14625,9750 14625,8400 14175,8400" clip-path="url(#cp28)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Backward arrow to point 13200,10800 -->
+<polygon points=" 13155,10604 13200,10784 13245,10604 13155,10604"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp29">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 11820,3104 11730,3104 11757,3318 11793,3318z
+ M 5280,5671 5370,5671 5343,5457 5307,5457z"/>
+</clipPath>
+</defs>
+<polyline points=" 5325,5475 5325,5775 4950,6675" clip-path="url(#cp29)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Backward arrow to point 5325,5475 -->
+<polygon points=" 5370,5671 5325,5491 5280,5671 5370,5671"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp30">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 555,3496 645,3496 618,3282 582,3282z
+ M 5280,5671 5370,5671 5343,5457 5307,5457z"/>
+</clipPath>
+</defs>
+<polyline points=" 600,5400 600,3300" clip-path="url(#cp30)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 600,3300 -->
+<polygon points=" 645,3496 600,3316 555,3496 645,3496"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<polyline points=" 600,7800 600,5700"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp31">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 12314,8370 12314,8430 12468,8418 12468,8382z
+ M 5280,5671 5370,5671 5343,5457 5307,5457z"/>
+</clipPath>
+</defs>
+<polyline points=" 12150,8400 12450,8400" clip-path="url(#cp31)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 12450,8400 -->
+<polygon points=" 12314,8430 12434,8400 12314,8370 12314,8430"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp32">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 11639,8370 11639,8430 11793,8418 11793,8382z
+ M 5280,5671 5370,5671 5343,5457 5307,5457z"/>
+</clipPath>
+</defs>
+<polyline points=" 11475,8400 11775,8400" clip-path="url(#cp32)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 11775,8400 -->
+<polygon points=" 11639,8430 11759,8400 11639,8370 11639,8430"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp33">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 10964,8370 10964,8430 11118,8418 11118,8382z
+ M 5280,5671 5370,5671 5343,5457 5307,5457z"/>
+</clipPath>
+</defs>
+<polyline points=" 10800,8400 11100,8400" clip-path="url(#cp33)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 11100,8400 -->
+<polygon points=" 10964,8430 11084,8400 10964,8370 10964,8430"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp34">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 12989,8370 12989,8430 13143,8418 13143,8382z
+ M 5280,5671 5370,5671 5343,5457 5307,5457z"/>
+</clipPath>
+</defs>
+<polyline points=" 12825,8400 13125,8400" clip-path="url(#cp34)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 13125,8400 -->
+<polygon points=" 12989,8430 13109,8400 12989,8370 12989,8430"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp35">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 13664,8370 13664,8430 13818,8418 13818,8382z
+ M 5280,5671 5370,5671 5343,5457 5307,5457z"/>
+</clipPath>
+</defs>
+<polyline points=" 13500,8400 13800,8400" clip-path="url(#cp35)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 13800,8400 -->
+<polygon points=" 13664,8430 13784,8400 13664,8370 13664,8430"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Text -->
+<g transform="translate(450,825) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="168" text-anchor="middle">application</text>
+</g><!-- Text -->
+<g transform="translate(2850,3225) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">mux-&gt;subscribe(SUB_RECV)</text>
+</g><!-- Text -->
+<text xml:space="preserve" x="12300" y="7125" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="192" text-anchor="middle">MUX</text>
+<!-- Text -->
+<text xml:space="preserve" x="3600" y="8100" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="end">Stream ID</text>
+<!-- Text -->
+<text xml:space="preserve" x="12825" y="7125" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">Stream ID</text>
+<!-- Text -->
+<text xml:space="preserve" x="3300" y="10125" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="end">tasklet_wakeup()</text>
+<!-- Text -->
+<text xml:space="preserve" x="12150" y="10125" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="end">tasklet_wakeup()</text>
+<!-- Text -->
+<text xml:space="preserve" x="11175" y="3150" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="end">mux-&gt;snd_buf()</text>
+<!-- Text -->
+<text xml:space="preserve" x="3675" y="3225" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">mux-&gt;rcv_buf()</text>
+<!-- Text -->
+<text xml:space="preserve" x="13425" y="10575" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">xprt-&gt;snd_buf(mbuf)</text>
+<!-- Text -->
+<text xml:space="preserve" x="4725" y="10500" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">xprt-&gt;rcv_buf(dbuf)</text>
+<!-- Text -->
+<text xml:space="preserve" x="8400" y="2100" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">HTX contents when mode==HTTP</text>
+<!-- Text -->
+<text xml:space="preserve" x="7500" y="450" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="end">tasklet_wakeup()</text>
+<!-- Text -->
+<text xml:space="preserve" x="9300" y="450" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">tasklet_wakeup()</text>
+<!-- Text -->
+<g transform="translate(12075,3225) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">mux-&gt;subscribe(SUB_SEND)</text>
+</g><!-- Text -->
+<g transform="translate(450,4500) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="168" text-anchor="middle">mux streams</text>
+</g><!-- Text -->
+<g transform="translate(450,6750) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="168" text-anchor="middle">mux=conn-&gt;mux</text>
+</g><!-- Text -->
+<text xml:space="preserve" x="4500" y="11175" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="216" text-anchor="middle">Transport</text>
+<!-- Text -->
+<text xml:space="preserve" x="2250" y="12000" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">encoding/decoding function</text>
+<!-- Text -->
+<text xml:space="preserve" x="2250" y="12525" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">transport layer</text>
+<!-- Text -->
+<text xml:space="preserve" x="7050" y="12525" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">multiplexer (MUX/DEMUX)</text>
+<!-- Text -->
+<text xml:space="preserve" x="7050" y="12000" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">general processing function</text>
+<!-- Text -->
+<text xml:space="preserve" x="11775" y="12525" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">stream buffer (byte-level FIFO)</text>
+<!-- Text -->
+<text xml:space="preserve" x="3675" y="10725" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="end">xprt-&gt;subscribe(SUB_RECV)</text>
+<!-- Text -->
+<text xml:space="preserve" x="12225" y="10725" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="end">xprt-&gt;subscribe(SUB_SEND)</text>
+<!-- Text -->
+<g transform="translate(450,2550) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="168" text-anchor="middle">stconn</text>
+</g><!-- Text -->
+<g transform="translate(900,1125) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">(eg: checks, streams)</text>
+</g><!-- Text -->
+<g transform="translate(450,10125) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="168" text-anchor="middle">connection = sc-&gt;sedesc-&gt;conn</text>
+</g><!-- Text -->
+<text xml:space="preserve" x="12225" y="225" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">Notes:</text>
+<!-- Text -->
+<text xml:space="preserve" x="12975" y="675" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">snd_buf() will move the</text>
+<!-- Text -->
+<text xml:space="preserve" x="12975" y="975" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">buffer (zero-copy) when</text>
+<!-- Text -->
+<text xml:space="preserve" x="12975" y="1275" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">the destination is empty.</text>
+<!-- Text -->
+<text xml:space="preserve" x="12825" y="1650" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">- the application is also</text>
+<!-- Text -->
+<text xml:space="preserve" x="12975" y="2250" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">is sc-&gt;app with sc-&gt;app_ops</text>
+<!-- Text -->
+<text xml:space="preserve" x="12825" y="2550" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">- transport layers (xprt) are</text>
+<!-- Text -->
+<text xml:space="preserve" x="12975" y="2775" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">stackable. conn-&gt;xprt is</text>
+<!-- Text -->
+<text xml:space="preserve" x="12975" y="3000" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">the topmost one.</text>
+<!-- Text -->
+<text xml:space="preserve" x="12975" y="1950" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">called the app layer and</text>
+<!-- Text -->
+<text xml:space="preserve" x="12825" y="375" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">- mux-&gt;rcv_buf() and</text>
+<!-- Line -->
+<polyline points=" 4261,9751 4261,8751 4761,8751 4761,9751"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 1486,4576 1486,3576 1986,3576 1986,4576"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 3286,4576 3286,3576 3786,3576 3786,4576"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 5086,4576 5086,3576 5586,3576 5586,4576"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 6961,4576 6961,3576 7461,3576 7461,4576"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 6692,1261 9959,1261 9959,1761 6692,1761"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 12425,8161 12825,8161 12825,8661 12425,8661"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 11750,8161 12150,8161 12150,8661 11750,8661"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 11075,8161 11475,8161 11475,8661 11075,8661"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 10400,8161 10800,8161 10800,8661 10400,8661"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 13100,8161 13500,8161 13500,8661 13100,8661"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 13775,8161 14175,8161 14175,8661 13775,8661"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 11157,12331 11614,12331 11614,12581 11157,12581"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Text -->
+<text xml:space="preserve" x="9534" y="4200" fill="#1a1a1a" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">encode</text>
+<!-- Text -->
+<text xml:space="preserve" x="11325" y="4200" fill="#1a1a1a" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">encode</text>
+<!-- Text -->
+<text xml:space="preserve" x="13134" y="4200" fill="#1a1a1a" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">encode</text>
+<!-- Text -->
+<text xml:space="preserve" x="15009" y="4200" fill="#1a1a1a" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">encode</text>
+<!-- Text -->
+<text xml:space="preserve" x="1725" y="5250" fill="#1a1a1a" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">decode</text>
+<!-- Text -->
+<text xml:space="preserve" x="3525" y="5250" fill="#1a1a1a" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">decode</text>
+<!-- Text -->
+<text xml:space="preserve" x="5325" y="5250" fill="#1a1a1a" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">decode</text>
+<!-- Text -->
+<text xml:space="preserve" x="7200" y="5250" fill="#1a1a1a" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">decode</text>
+<!-- Text -->
+<text xml:space="preserve" x="12375" y="9300" fill="#1a1a1a" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">mux_io_cb</text>
+<!-- Text -->
+<text xml:space="preserve" x="3159" y="9300" fill="#1a1a1a" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">mux_io_cb</text>
+<!-- Text -->
+<text xml:space="preserve" x="8409" y="657" fill="#1a1a1a" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">sc_conn_io_cb</text>
+<!-- Line -->
+<polyline points=" 4261,8850 4761,8850"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 4261,8925 4761,8925"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 4261,9000 4761,9000"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 1486,3675 1986,3675"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 1486,3750 1986,3750"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 1486,3825 1986,3825"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 3286,3675 3786,3675"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 3286,3750 3786,3750"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 3286,3825 3786,3825"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 5086,3675 5586,3675"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 5086,3750 5586,3750"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 5086,3825 5586,3825"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 6961,3675 7461,3675"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 6961,3750 7461,3750"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 6961,3825 7461,3825"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9750,1261 9750,1761"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9525,1261 9525,1761"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9300,1261 9300,1761"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 12600,8161 12600,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 12675,8161 12675,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 12750,8161 12750,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 11925,8161 11925,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 12000,8161 12000,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 12075,8161 12075,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 11250,8161 11250,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 11325,8161 11325,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 11400,8161 11400,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 10575,8161 10575,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 10650,8161 10650,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 10725,8161 10725,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 13275,8161 13275,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 13350,8161 13350,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 13425,8161 13425,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 13950,8161 13950,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 14025,8161 14025,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 14100,8161 14100,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 11357,12331 11357,12581"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 11443,12331 11443,12581"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 11529,12331 11529,12581"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Text -->
+<text xml:space="preserve" x="8025" y="1575" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="192" text-anchor="middle">channel buf</text>
+<!-- Text -->
+<g transform="translate(3600,4200) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="192" text-anchor="middle">rxbuf</text>
+</g><!-- Text -->
+<g transform="translate(4575,9375) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="192" text-anchor="middle">dbuf</text>
+</g><!-- Text -->
+<text xml:space="preserve" x="14625" y="8175" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="192" text-anchor="middle">mbuf</text>
+<!-- Ellipse -->
+<ellipse cx="4488" cy="8082" rx="612" ry="250" fill="#87cfff"
+ stroke="#0000d1" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="6600" cy="11925" rx="225" ry="150" fill="#87cfff"
+ stroke="#0000d1" stroke-width="45px"/>
+<!-- Text -->
+<text xml:space="preserve" x="4500" y="8175" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="192" text-anchor="middle">DEMUX</text>
+</g>
+</svg>
diff --git a/doc/internals/notes-layers.txt b/doc/internals/notes-layers.txt
new file mode 100644
index 0000000..541c125
--- /dev/null
+++ b/doc/internals/notes-layers.txt
@@ -0,0 +1,330 @@
+2018-02-21 - Layering in haproxy 1.9
+------------------------------------
+
+2 main zones :
+ - application : reads from conn_streams, writes to conn_streams, often uses
+ streams
+
+ - connection : receives data from the network, presented into buffers
+ available via conn_streams, sends data to the network
+
+
+The connection zone contains multiple layers which behave independently in each
+direction. The Rx direction is activated upon callbacks from the lower layers.
+The Tx direction is activated recursively from the upper layers. Between every
+two layers there may be a buffer, in each direction. When a buffer is full
+either in Tx or Rx direction, this direction is paused from the network layer
+and the location where the congestion is encountered. Upon end of congestion
+(cs_recv() from the upper layer, of sendto() at the lower layers), a
+tasklet_wakeup() is performed on the blocked layer so that suspended operations
+can be resumed. In this case, the Rx side restarts propagating data upwards
+from the lowest blocked level, while the Tx side restarts propagating data
+downwards from the highest blocked level. Proceeding like this ensures that
+information known to the producer may always be used to tailor the buffer sizes
+or decide of a strategy to best aggregate data. Additionally, each time a layer
+is crossed without transformation, it becomes possible to send without copying.
+
+The Rx side notifies the application of data readiness using a wakeup or a
+callback. The Tx side notifies the application of room availability once data
+have been moved resulting in the uppermost buffer having some free space.
+
+When crossing a mux downwards, it is possible that the sender is not allowed to
+access the buffer because it is not yet its turn. It is not a problem, the data
+remains in the conn_stream's buffer (or the stream one) and will be restarted
+once the mux is ready to consume these data.
+
+
+ cs_recv() -------. cs_send()
+ ^ +--------> |||||| -------------+ ^
+ | | -------' | | stream
+ --|----------|-------------------------------|-------|-------------------
+ | | V | connection
+ data .---. | | room
+ ready! |---| |---| available!
+ |---| |---|
+ |---| |---|
+ | | '---'
+ ^ +------------+-------+ |
+ | | ^ | /
+ / V | V /
+ / recvfrom() | sendto() |
+ -------------|----------------|--------------|---------------------------
+ | | poll! V kernel
+
+
+The cs_recv() function should act on pointers to buffer pointers, so that the
+callee may decide to pass its own buffer directly by simply swapping pointers.
+Similarly for cs_send() it is desirable to let the callee steal the buffer by
+swapping the pointers. This way it remains possible to implement zero-copy
+forwarding.
+
+Some operation flags will be needed on cs_recv() :
+ - RECV_ZERO_COPY : refuse to merge new data into the current buffer if it
+ will result in a data copy (ie the buffer is not empty), unless no more
+ than XXX bytes have to be copied (eg: copying 2 cache lines may be cheaper
+ than waiting and playing with pointers)
+
+ - RECV_AT_ONCE : only perform the operation if it will result in the source
+ buffer to become empty at the end of the operation so that no two buffers
+ remain allocated at the end. It will most of the time result in either a
+ small read or a zero-copy operation.
+
+ - RECV_PEEK : retrieve a copy of pending data without removing these data
+ from the source buffer. Maybe an alternate solution could consist in
+ finding the pointer to the source buffer and accessing these data directly,
+ except that it might be less interesting for the long term, thread-wise.
+
+ - RECV_MIN : receive minimum X bytes (or less with a shutdown), or fail.
+ This should help various protocol parsers which need to receive a complete
+ frame before proceeding.
+
+ - RECV_ENOUGH : no more data expected after this read if it's of the
+ requested size, thus no need to re-enable receiving on the lower layers.
+
+ - RECV_ONE_SHOT : perform a single read without re-enabling reading on the
+ lower layers, like we currently do when receiving an HTTP/1 request. Like
+ RECV_ENOUGH where any size is enough. Probably that the two could be merged
+ (eg: by having a MIN argument like RECV_MIN).
+
+
+Some operation flags will be needed on cs_send() :
+ - SEND_ZERO_COPY : refuse to merge the presented data with existing data and
+ prefer to wait for current data to leave and try again, unless the consumer
+ considers the amount of data acceptable for a copy.
+
+ - SEND_AT_ONCE : only perform the operation if it will result in the source
+ buffer to become empty at the end of the operation so that no two buffers
+ remain allocated at the end. It will most of the time result in either a
+ small write or a zero-copy operation.
+
+
+Both operations should return a composite status :
+ - number of bytes transferred
+ - status flags (shutr, shutw, reset, empty, full, ...)
+
+
+2018-07-23 - Update after merging rxbuf
+---------------------------------------
+
+It becomes visible that the mux will not always be welcome to decode incoming
+data because it will sometimes imply extra memory copies and/or usage for no
+benefit.
+
+Ideally, when when a stream is instantiated based on incoming data, these
+incoming data should be passed and the upper layers called, but it should then
+be up these upper layers to peek more data in certain circumstances. Typically
+if the pending connection data are larger than what is expected to be passed
+above, it means some data may cause head-of-line blocking (HOL) to other
+streams, and needs to be pushed up through the layers to let other streams
+continue to work. Similarly very large H2 data frames after header frames
+should probably not be passed as they may require copies that could be avoided
+if passed later. However if the decoded frame fits into the conn_stream's
+buffer, there is an opportunity to use a single buffer for the conn_stream
+and the channel. The H2 demux could set a blocking flag indicating it's waiting
+for the upper stream to take over demuxing. This flag would be purged once the
+upper stream would start reading, or when extra data come and change the
+conditions.
+
+Forcing structured headers and raw data to coexist within a single buffer is
+quite challenging for many code parts. For example it's perfectly possible to
+see a fragmented buffer containing series of headers, then a small data chunk
+that was received at the same time, then a few other headers added by request
+processing, then another data block received afterwards, then possibly yet
+another header added by option http-send-name-header, and yet another data
+block. This causes some pain for compression which still needs to know where
+compressed and uncompressed data start/stop. It also makes it very difficult
+to account the exact bytes to pass through the various layers.
+
+One solution consists in thinking about buffers using 3 representations :
+
+ - a structured message, which is used for the internal HTTP representation.
+ This message may only be atomically processed. It has no clear byte count,
+ it's a message.
+
+ - a raw stream, consisting in sequences of bytes. That's typically what
+ happens in data sequences or in tunnel.
+
+ - a pipe, which contains data to be forwarded, and that haproxy cannot have
+ access to.
+
+The processing efficiency decreases with the higher complexity above, but the
+capabilities increase. The structured message can contain anything including
+serialized data blocks to be processed or forwarded. The raw stream contains
+data blocks to be processed or forwarded. The pipe only contains data blocks
+to be forwarded. The the latter ones are only an optimization of the former
+ones.
+
+Thus ideally a channel should have access to all such 3 storage areas at once,
+depending on the use case :
+ (1) a structured message,
+ (2) a raw stream,
+ (3) a pipe
+
+Right now a channel only has (2) and (3) but after the native HTTP rework, it
+will only have (1) and (3). Placing a raw stream exclusively in (1) comes with
+some performance drawbacks which are not easily recovered, and with some quite
+difficult management still involving the reserve to ensure that a data block
+doesn't prevent headers from being appended. But during header processing, the
+payload may be necessary so we cannot decide to drop this option.
+
+A long-term approach would consist in ensuring that a single channel may have
+access to all 3 representations at once, and to enumerate priority rules to
+define how they interact together. That's exactly what is currently being done
+with the pipe and the raw buffer right now. Doing so would also save the need
+for storing payload in the structured message and void the requirement for the
+reserve. But it would cost more memory to process POST data and server
+responses. Thus an intermediary step consists in keeping this model in mind but
+not implementing everything yet.
+
+Short term proposal : a channel has access to a buffer and a pipe. A non-empty
+buffer is either in structured message format OR raw stream format. Only the
+channel knows. However a structured buffer MAY contain raw data in a properly
+formatted way (using the envelope defined by the structured message format).
+
+By default, when a demux writes to a CS rxbuf, it will try to use the lowest
+possible level for what is being done (i.e. splice if possible, otherwise raw
+stream, otherwise structured message). If the buffer already contains a
+structured message, then this format is exclusive. From this point the MUX has
+two options : either encode the incoming data to match the structured message
+format, or refrain from receiving into the CS's rxbuf and wait until the upper
+layer request those data.
+
+This opens a simplified option which could be suited even for the long term :
+ - cs_recv() will take one or two flags to indicate if a buffer already
+ contains a structured message or not ; the upper layer knows it.
+
+ - cs_recv() will take two flags to indicate what the upper layer is willing
+ to take :
+ - structured message only
+ - raw stream only
+ - any of them
+
+ From this point the mux can decide to either pass anything or refrain from
+ doing so.
+
+ - the demux stores the knowledge it has from the contents into some CS flags
+ to indicate whether or not some structured message are still available, and
+ whether or not some raw data are still available. Thus the caller knows
+ whether or not extra data are available.
+
+ - when the demux works on its own, it refrains from passing structured data
+ to a non-empty buffer, unless these data are causing trouble to other
+ streams (HOL).
+
+ - when a demux has to encapsulate raw data into a structured message, it will
+ always have to respect a configured reserve so that extra header processing
+ can be done on the structured message inside the buffer, regardless of the
+ supposed available room. In addition, the upper layer may indicate using an
+ extra recv() flag whether it wants the demux to defragment serialized data
+ (for example by moving trailing headers apart) or if it's not necessary.
+ This flag will be set by the stream interface if compression is required or
+ if the http-buffer-request option is set for example. Probably that using
+ to_forward==0 is a stronger indication that the reserve must be respected.
+
+ - cs_recv() and cs_send() when fed with a message, should not return byte
+ counts but message counts (i.e. 0 or 1). This implies that a single call to
+ either of these functions cannot mix raw data and structured messages at
+ the same time.
+
+At this point it looks like the conn_stream will have some encapsulation work
+to do for the payload if it needs to be encapsulated into a message. This
+further magnifies the importance of *not* decoding DATA frames into the CS's
+rxbuf until really needed.
+
+The CS will probably need to hold indication of what is available at the mux
+level, not only in the CS. Eg: we know that payload is still available.
+
+Using these elements, it should be possible to ensure that full header frames
+may be received without enforcing any reserve, that too large frames that do
+not fit will be detected because they return 0 message and indicate that such
+a message is still pending, and that data availability is correctly detected
+(later we may expect that the stream-interface allocates a larger or second
+buffer to place the payload).
+
+Regarding the ability for the channel to forward data, it looks like having a
+new function "cs_xfer(src_cs, dst_cs, count)" could be very productive in
+optimizing the forwarding to make use of splicing when available. It is not yet
+totally clear whether it will split into "cs_xfer_in(src_cs, pipe, count)"
+followed by "cs_xfer_out(dst_cs, pipe, count)" or anything different, and it
+still needs to be studied. The general idea seems to be that the receiver might
+have to call the sender directly once they agree on how to transfer data (pipe
+or buffer). If the transfer is incomplete, the cs_xfer() return value and/or
+flags will indicate the current situation (src empty, dst full, etc) so that
+the caller may register for notifications on the appropriate event and wait to
+be called again to continue.
+
+Short term implementation :
+ 1) add new CS flags to qualify what the buffer contains and what we expect
+ to read into it;
+
+ 2) set these flags to pretend we have a structured message when receiving
+ headers (after all, H1 is an atomic header as well) and see what it
+ implies for the code; for H1 it's unclear whether it makes sense to try
+ to set it without the H1 mux.
+
+ 3) use these flags to refrain from sending DATA frames after HEADERS frames
+ in H2.
+
+ 4) flush the flags at the stream interface layer when performing a cs_send().
+
+ 5) use the flags to enforce receipt of data only when necessary
+
+We should be able to end up with sequential receipt in H2 modelling what is
+needed for other protocols without interfering with the native H1 devs.
+
+
+2018-08-17 - Considerations after killing cs_recv()
+---------------------------------------------------
+
+With the ongoing reorganisation of the I/O layers, it's visible that cs_recv()
+will have to transfer data between the cs' rxbuf and the channel's buffer while
+not being aware of the data format. Moreover, in case there's no data there, it
+needs to recursively call the mux's rcv_buf() to trigger a decoding, while this
+function is sometimes replaced with cs_recv(). All this shows that cs_recv() is
+in fact needed while data are pushed upstream from the lower layers, and is not
+suitable for the "pull" mode. Thus it was decided to remove this function and
+put its code back into h2_rcv_buf(). The H1 mux's rcv_buf() already couldn't be
+replaced with cs_recv() since it is the only one knowing about the buffer's
+format.
+
+This opportunity simplified something : if the cs's rxbuf is only read by the
+mux's rcv_buf() method, then it doesn't need to be located into the CS and is
+well placed into the mux's representation of the stream. This has an important
+impact for H2 as it offers more freedom to the mux to allocate/free/reallocate
+this buffer, and it ensures the mux always has access to it.
+
+Furthermore, the conn_stream's txbuf experienced the same fate. Indeed, the H1
+mux has already uncovered the difficulty related to the channel shutting down
+on output, with data stuck into the CS's txbuf. Since the CS is tightly coupled
+to the stream and the stream can close immediately once its buffers are empty,
+it required a way to support orphaned CS with pending data in their txbuf. This
+is something that the H2 mux already has to deal with, by carefully leaving the
+data in the channel's buffer. But due to the snd_buf() call being top-down, it
+is always possible to push the stream's data via the mux's snd_buf() call
+without requiring a CS txbuf anymore. Thus the txbuf (when needed) is only
+implemented in the mux and attached to the mux's representation of the stream,
+and doing so allows to immediately release the channel once the data are safe
+in the mux's buffer.
+
+This is an important change which clarifies the roles and responsibilities of
+each layer in the chain : when receiving data from a mux, it's the mux's
+responsibility to make sure it can correctly decode the incoming data and to
+buffer the possible excess of data it cannot pass to the requester. This means
+that decoding an H2 frame, which is not retryable since it has an impact on the
+HPACK decompression context, and which cannot be reordered for the same reason,
+simply needs to be performed to the H2 stream's rxbuf which will then be passed
+to the stream when this one calls h2_rcv_buf(), even if it reads one byte at a
+time. Similarly when calling h2_snd_buf(), it's the mux's responsibility to
+read as much as it needs to be able to restart later, possibly by buffering
+some data into a local buffer. And it's only once all the output data has been
+consumed by snd_buf() that the stream is free to disappear.
+
+This model presents the nice benefit of being infinitely stackable and solving
+the last identified showstoppers to move towards a structured message internal
+representation, as it will give full power to the rcv_buf() and snd_buf() to
+process what they need.
+
+For now the conn_stream's flags indicating whether a shutdown has been seen in
+any direction or if an end of stream was seen will remain in the conn_stream,
+though it's likely that some of them will move to the mux's representation of
+the stream after structured messages are implemented.
diff --git a/doc/internals/notes-poll-connect.txt b/doc/internals/notes-poll-connect.txt
new file mode 100644
index 0000000..5cb0885
--- /dev/null
+++ b/doc/internals/notes-poll-connect.txt
@@ -0,0 +1,93 @@
+2022-11-17 - Tests involving poll() return states upon a pending connect().
+
+- connect() to a closed port returns OUT and HUP:
+
+ $ dev/poll/poll -v -l clo -c pol
+ #### BEGIN ####
+ cmd #1 stp #1: clo(l=3): ret=0
+ cmd #2 stp #0: con(c=4): ret=-1 (Connection refused)
+ cmd #2 stp #1: pol(c=4): ret=1 ev=0x14 (OUT HUP)
+ #### END ####
+
+=> with HUP we *know* the connection failed, since we never asked for a
+ SHUTW before connecting. It is indeed an error as can be seen with
+ connect() returning -1 ECONNREFUSED.
+
+- connect() to a port that does close(accept()) does return IN and RDHUP:
+
+ $ dev/poll/poll -v -s clo -c pol
+ #### BEGIN ####
+ cmd #1 stp #0: con(c=4): ret=0
+ cmd #1 stp #0: acc(l=3): ret=5
+ cmd #1 stp #1: clo(s=5): ret=0
+ cmd #2 stp #1: pol(c=4): ret=1 ev=0x2005 (IN OUT RDHUP)
+ #### END ####
+
+=> here there's no HUP, only RDHUP because the FIN is pending in the
+ socket buffers, waiting to be read.
+
+- for a HUP to happen after a connect() to a valid port, one would have to
+ perform a shutw() on the client, which is normally not the case, indicating
+ that HUP is reliable here:
+
+ $ dev/poll/poll -v -s clo -c shw,pol
+ #### BEGIN ####
+ cmd #1 stp #0: con(c=4): ret=0
+ cmd #1 stp #0: acc(l=3): ret=5
+ cmd #1 stp #1: clo(s=5): ret=0
+ cmd #2 stp #1: shw(c=4): ret=0
+ cmd #2 stp #2: pol(c=4): ret=1 ev=0x2015 (IN OUT HUP RDHUP)
+ #### END ####
+
+- one case that may happen is when sending a request and immediately shutw()
+ (which leaves a TIME_WAIT so not recommended):
+
+ $ dev/poll/poll -v -c snd,shw -s clo -c pol,rcv,pol
+ #### BEGIN ####
+ cmd #1 stp #0: con(c=4): ret=0
+ cmd #1 stp #1: snd(c=4): ret=3
+ cmd #1 stp #2: shw(c=4): ret=0
+ cmd #2 stp #0: acc(l=3): ret=5
+ cmd #2 stp #1: clo(s=5): ret=0
+ cmd #3 stp #1: pol(c=4): ret=1 ev=0x201d (IN OUT ERR HUP RDHUP)
+ cmd #3 stp #2: rcv(c=4): ret=-1 (Connection reset by peer)
+ cmd #3 stp #3: pol(c=4): ret=1 ev=0x2015 (IN OUT HUP RDHUP)
+ #### END ####
+
+=> here it's impossible to know from the client whether the server consumed the
+ data or not, which is normal since a close on the server causes an RST to be
+ emitted for the data in flight, hence the ERR here. It's also worth noting
+ that once POLL_ERR is consumed by recv() it disappears.
+
+- for the server, sending a shutw() before closing here delivers an ACK in time
+ that prevents the RST from being sent, thus connect() is not notified (but if
+ the server has too much to send, it will truncate and emit an RST):
+
+ $ dev/poll/poll -v -c snd,shw -s shw,clo -c pol,rcv,pol
+ #### BEGIN ####
+ cmd #1 stp #0: con(c=4): ret=0
+ cmd #1 stp #1: snd(c=4): ret=3
+ cmd #1 stp #2: shw(c=4): ret=0
+ cmd #2 stp #0: acc(l=3): ret=5
+ cmd #2 stp #1: shw(s=5): ret=0
+ cmd #2 stp #2: clo(s=5): ret=0
+ cmd #3 stp #1: pol(c=4): ret=1 ev=0x2015 (IN OUT HUP RDHUP)
+ cmd #3 stp #2: rcv(c=4): ret=0
+ cmd #3 stp #3: pol(c=4): ret=1 ev=0x2015 (IN OUT HUP RDHUP)
+ #### END ####
+
+- if the server sends a response, disables lingering and closes with RST, it is
+ possible to get HUP and ERR at the same time during the connect() phase, and
+ recv() can still receive the pending response:
+
+ $ dev/poll/poll -v -s snd,lin,clo -c pol,rcv,pol
+ #### BEGIN ####
+ cmd #1 stp #0: con(c=4): ret=0
+ cmd #1 stp #0: acc(l=3): ret=5
+ cmd #1 stp #1: snd(s=5): ret=3
+ cmd #1 stp #2: lin(s=5): ret=0
+ cmd #1 stp #3: clo(s=5): ret=0
+ cmd #2 stp #1: pol(c=4): ret=1 ev=0x201d (IN OUT ERR HUP RDHUP)
+ cmd #2 stp #2: rcv(c=4): ret=3
+ cmd #2 stp #3: pol(c=4): ret=1 ev=0x201d (IN OUT ERR HUP RDHUP)
+ #### END ####
diff --git a/doc/internals/notes-pollhup.txt b/doc/internals/notes-pollhup.txt
new file mode 100644
index 0000000..ced332b
--- /dev/null
+++ b/doc/internals/notes-pollhup.txt
@@ -0,0 +1,281 @@
+tcp mode 8001->8008
+
+
+Remote test:
+============
+
+willy@up1:~$ echo bar | ncat -lp8008
+willy@wtap:haproxy$ echo foo | ncat 127.1 8001
+
+17:09:53.663154 epoll_wait(3, [{EPOLLIN, {u32=5, u64=5}}], 200, 1000) = 1
+17:09:54.582146 accept4(5, {sa_family=AF_INET, sin_port=htons(33378), sin_addr=inet_addr("127.0.0.1")}, [128->16], SOCK_NONBLOCK) = 8
+17:09:54.582299 setsockopt(8, SOL_TCP, TCP_NODELAY, [1], 4) = 0
+17:09:54.582527 accept4(5, 0x7ffc4a8bf330, [128], SOCK_NONBLOCK) = -1 EAGAIN (Resource temporarily unavailable)
+17:09:54.582655 recvfrom(8, "foo\n", 15360, 0, NULL, NULL) = 4
+17:09:54.582727 recvfrom(8, "", 15356, 0, NULL, NULL) = 0
+17:09:54.582827 socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 9
+17:09:54.582878 setsockopt(9, SOL_TCP, TCP_NODELAY, [1], 4) = 0
+17:09:54.582897 connect(9, {sa_family=AF_INET, sin_port=htons(8008), sin_addr=inet_addr("10.0.3.82")}, 16) = -1 EINPROGRESS (Operation now in progress)
+17:09:54.582941 sendto(9, "foo\n", 4, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = -1 EAGAIN (Resource temporarily unavailable)
+17:09:54.582968 epoll_ctl(3, EPOLL_CTL_ADD, 9, {EPOLLOUT, {u32=9, u64=9}}) = 0
+17:09:54.582997 epoll_wait(3, [{EPOLLOUT, {u32=9, u64=9}}], 200, 1000) = 1
+17:09:54.583686 connect(9, {sa_family=AF_INET, sin_port=htons(8008), sin_addr=inet_addr("10.0.3.82")}, 16) = 0
+17:09:54.583706 sendto(9, "foo\n", 4, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 4
+17:09:54.583733 recvfrom(9, 0x19c2300, 15360, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
+17:09:54.583755 shutdown(9, SHUT_WR) = 0
+17:09:54.583775 epoll_ctl(3, EPOLL_CTL_MOD, 9, {EPOLLIN|EPOLLRDHUP, {u32=9, u64=9}}) = 0
+17:09:54.583802 epoll_wait(3, [{EPOLLIN, {u32=9, u64=9}}], 200, 1000) = 1
+17:09:54.584672 recvfrom(9, "bar\n", 16384, 0, NULL, NULL) = 4
+17:09:54.584713 recvfrom(9, "", 16380, 0, NULL, NULL) = 0
+17:09:54.584743 sendto(8, "bar\n", 4, MSG_DONTWAIT|MSG_NOSIGNAL|MSG_MORE, NULL, 0) = 4
+17:09:54.584819 epoll_wait(3, [], 200, 0) = 0
+17:09:54.584901 epoll_wait(3, [], 200, 1000) = 0
+
+
+Notes:
+ - we had data available to try the connect() (see first attempt), despite
+ this during the retry we sent the connect again!
+
+ - why do we wait before sending the shutw to the server if we already know
+ it's needed ? missing CF_SHUTW_NOW ? Missing request forwarding ? Missing
+ auto-close ?
+
+ - response didn't feature HUP nor RDHUP
+
+
+Local:
+
+17:15:43.010786 accept4(5, {sa_family=AF_INET, sin_port=htons(33506), sin_addr=inet_addr("127.0.0.1")}, [128->16], SOCK_NONBLOCK) = 8
+17:15:43.011013 setsockopt(8, SOL_TCP, TCP_NODELAY, [1], 4) = 0
+17:15:43.011181 accept4(5, 0x7ffcd9092cd0, [128], SOCK_NONBLOCK) = -1 EAGAIN (Resource temporarily unavailable)
+17:15:43.011231 recvfrom(8, "foo\n", 15360, 0, NULL, NULL) = 4
+17:15:43.011296 recvfrom(8, "", 15356, 0, NULL, NULL) = 0
+17:15:43.011318 socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 9
+17:15:43.011340 setsockopt(9, SOL_TCP, TCP_NODELAY, [1], 4) = 0
+17:15:43.011353 connect(9, {sa_family=AF_INET, sin_port=htons(8008), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EINPROGRESS (Operation now in progress)
+17:15:43.011395 sendto(9, "foo\n", 4, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 4
+17:15:43.011425 shutdown(9, SHUT_WR) = 0
+17:15:43.011459 recvfrom(9, "bar\n", 16384, 0, NULL, NULL) = 4
+17:15:43.011491 recvfrom(9, "", 16380, 0, NULL, NULL) = 0
+17:15:43.011525 sendto(8, "bar\n", 4, MSG_DONTWAIT|MSG_NOSIGNAL|MSG_MORE, NULL, 0) = 4
+17:15:43.011584 epoll_wait(3, [], 200, 0) = 0
+
+Notes:
+ - the shutdown() was properly done right after the sendto(), proving that
+ CF_SHUTW_NOW and auto-close were present. Maybe difference is sync vs async
+ send.
+
+
+Local with delay before closing client:
+
+17:18:17.155349 epoll_wait(3, [{EPOLLIN, {u32=5, u64=5}}], 200, 1000) = 1
+17:18:17.727327 accept4(5, {sa_family=AF_INET, sin_port=htons(33568), sin_addr=inet_addr("127.0.0.1")}, [128->16], SOCK_NONBLOCK) = 8
+17:18:17.727553 setsockopt(8, SOL_TCP, TCP_NODELAY, [1], 4) = 0
+17:18:17.727661 accept4(5, 0x7fff4eb9a0b0, [128], SOCK_NONBLOCK) = -1 EAGAIN (Resource temporarily unavailable)
+17:18:17.727798 recvfrom(8, 0xbda300, 15360, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
+17:18:17.727830 socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 9
+17:18:17.727858 setsockopt(9, SOL_TCP, TCP_NODELAY, [1], 4) = 0
+17:18:17.727877 connect(9, {sa_family=AF_INET, sin_port=htons(8008), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EINPROGRESS (Operation now in progress)
+17:18:17.727923 epoll_ctl(3, EPOLL_CTL_ADD, 8, {EPOLLIN|EPOLLRDHUP, {u32=8, u64=8}}) = 0
+17:18:17.727945 epoll_ctl(3, EPOLL_CTL_ADD, 9, {EPOLLOUT, {u32=9, u64=9}}) = 0
+17:18:17.727989 epoll_wait(3, [{EPOLLOUT, {u32=9, u64=9}}], 200, 1000) = 1
+17:18:17.728010 connect(9, {sa_family=AF_INET, sin_port=htons(8008), sin_addr=inet_addr("127.0.0.1")}, 16) = 0
+17:18:17.728027 recvfrom(9, "bar\n", 15360, 0, NULL, NULL) = 4
+17:18:17.728055 recvfrom(9, 0xbd62f4, 15356, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
+17:18:17.728073 sendto(8, "bar\n", 4, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 4
+17:18:17.728104 epoll_ctl(3, EPOLL_CTL_MOD, 9, {EPOLLIN|EPOLLRDHUP, {u32=9, u64=9}}) = 0
+17:18:17.728127 epoll_wait(3, [], 200, 1000) = 0
+17:18:18.729411 epoll_wait(3, [], 200, 1000) = 0
+17:18:19.730654 epoll_wait(3, [{EPOLLIN|EPOLLRDHUP, {u32=8, u64=8}}], 200, 1000) = 1
+17:18:20.299268 recvfrom(8, "", 16384, 0, NULL, NULL) = 0
+17:18:20.299336 epoll_ctl(3, EPOLL_CTL_DEL, 8, 0x7ff3a969f7d0) = 0
+17:18:20.299379 epoll_wait(3, [], 200, 0) = 0
+17:18:20.299401 shutdown(9, SHUT_WR) = 0
+17:18:20.299523 epoll_wait(3, [{EPOLLIN|EPOLLHUP|EPOLLRDHUP, {u32=9, u64=9}}], 200, 1000) = 1
+17:18:20.299678 recvfrom(9, "", 16384, 0, NULL, NULL) = 0
+17:18:20.299761 epoll_wait(3, [], 200, 0) = 0
+
+Notes: server sent the response in two parts ("bar" then EOF) just due to
+netcat's implementation. The second epoll_wait() caught it.
+
+Here we clearly see that :
+ - read0 alone returns EPOLLIN|EPOLLRDHUP
+ - read0 after shutw returns EPOLLIN|EPOLLRDHUP|EPOLLHUP
+ => difference indeed is "cannot write"
+
+
+Local with a delay before closing the server:
+
+17:30:32.527157 epoll_wait(3, [{EPOLLIN, {u32=5, u64=5}}], 200, 1000) = 1
+17:30:33.216827 accept4(5, {sa_family=AF_INET, sin_port=htons(33908), sin_addr=inet_addr("127.0.0.1")}, [128->16], SOCK_NONBLOCK) = 8
+17:30:33.216957 setsockopt(8, SOL_TCP, TCP_NODELAY, [1], 4) = 0
+17:30:33.216984 accept4(5, 0x7ffc1a1fb0c0, [128], SOCK_NONBLOCK) = -1 EAGAIN (Resource temporarily unavailable)
+17:30:33.217071 recvfrom(8, "GET / HTTP/1.0\r\n\r\n\n", 15360, 0, NULL, NULL) = 19
+17:30:33.217115 recvfrom(8, "", 15341, 0, NULL, NULL) = 0
+17:30:33.217135 socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 9
+17:30:33.217176 setsockopt(9, SOL_TCP, TCP_NODELAY, [1], 4) = 0
+17:30:33.217190 connect(9, {sa_family=AF_INET, sin_port=htons(8000), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EINPROGRESS (Operation now in progress)
+17:30:33.217233 sendto(9, "GET / HTTP/1.0\r\n\r\n\n", 19, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 19
+17:30:33.217272 shutdown(9, SHUT_WR) = 0
+17:30:33.217318 recvfrom(9, 0x109b2f0, 16384, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
+17:30:33.217332 epoll_ctl(3, EPOLL_CTL_ADD, 9, {EPOLLIN|EPOLLRDHUP, {u32=9, u64=9}}) = 0
+17:30:33.217355 epoll_wait(3, [{EPOLLIN|EPOLLHUP|EPOLLRDHUP, {u32=9, u64=9}}], 200, 1000) = 1
+17:30:33.217377 recvfrom(9, "HTTP/1.0 200\r\nContent-length: 0\r\nX-req: size=19, time=0 ms\r\nX-rsp: id=dummy, code=200, cache=1, size=0, time=0 ms (0 real)\r\n\r\n", 16384, 0, NULL, NULL) = 126
+17:30:33.217395 close(9) = 0
+17:30:33.217411 sendto(8, "HTTP/1.0 200\r\nContent-length: 0\r\nX-req: size=19, time=0 ms\r\nX-rsp: id=dummy, code=200, cache=1, size=0, time=0 ms (0 real)\r\n\r\n", 126, MSG_DONTWAIT|MSG_NOSIGNAL|MSG_MORE, NULL, 0) = 126
+17:30:33.217464 close(8) = 0
+17:30:33.217496 epoll_wait(3, [], 200, 0) = 0
+
+
+Notes:
+ - RDHUP is properly present while some data remain pending.
+ - HUP is present since RDHUP + shutw
+
+It could be concluded that HUP indicates RDHUP+shutw and in no way indicates
+the ability to avoid reading.
+
+Below HUP|ERR|OUT are reported on connection failures, thus WITHOUT read:
+
+accept4(5, {sa_family=AF_INET, sin_port=htons(39080), sin_addr=inet_addr("127.0.0.1")}, [128->16], SOCK_NONBLOCK) = 8
+setsockopt(8, SOL_TCP, TCP_NODELAY, [1], 4) = 0
+accept4(5, 0x7ffffba55730, [128], SOCK_NONBLOCK) = -1 EAGAIN (Resource temporarily unavailable)
+recvfrom(8, "foo\n", 15360, 0, NULL, NULL) = 4
+recvfrom(8, 0x7f634dcfeff4, 15356, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
+socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 9
+fcntl(9, F_SETFL, O_RDONLY|O_NONBLOCK) = 0
+setsockopt(9, SOL_TCP, TCP_NODELAY, [1], 4) = 0
+connect(9, {sa_family=AF_INET, sin_port=htons(8008), sin_addr=inet_addr("10.0.3.82")}, 16) = -1 EINPROGRESS (Operation now in progress)
+sendto(9, "foo\n", 4, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = -1 EAGAIN (Resource temporarily unavailable)
+epoll_ctl(3, EPOLL_CTL_ADD, 8, {EPOLLIN|EPOLLRDHUP, {u32=8, u64=8}}) = 0
+epoll_ctl(3, EPOLL_CTL_ADD, 9, {EPOLLOUT, {u32=9, u64=9}}) = 0
+epoll_wait(3, [{EPOLLOUT|EPOLLERR|EPOLLHUP, {u32=9, u64=9}}], 200, 1000) = 1
+getsockopt(9, SOL_SOCKET, SO_ERROR, [111], [4]) = 0
+recvfrom(9, "", 15360, 0, NULL, NULL) = 0
+close(9) = 0
+
+
+On a failed connect attempt immediately followed by a failed recv (all flags
+set), we can see this:
+
+socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 8
+fcntl(8, F_SETFL, O_RDONLY|O_NONBLOCK) = 0
+setsockopt(8, SOL_TCP, TCP_NODELAY, [1], 4) = 0
+connect(8, {sa_family=AF_INET, sin_port=htons(8008), sin_addr=inet_addr("10.0.3.82")}, 16) = -1 EINPROGRESS (Operation now in progress)
+recvfrom(8, 0x1084a20, 16384, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
+epoll_ctl(3, EPOLL_CTL_ADD, 8, {EPOLLIN|EPOLLOUT|EPOLLRDHUP, {u32=8, u64=8}}) = 0
+epoll_wait(3, [{EPOLLIN|EPOLLOUT|EPOLLERR|EPOLLHUP|EPOLLRDHUP, {u32=8, u64=8}}], 200, 1000) = 1
+connect(8, {sa_family=AF_INET, sin_port=htons(8008), sin_addr=inet_addr("10.0.3.82")}, 16) = -1 ECONNREFUSED (Connection refused)
+close(8) = 0
+
+=> all flags are reported in case of error.
+
+It's also interesting to note that POLLOUT is still reported after a shutw,
+and no send error is ever reported after shutw:
+
+ shutdown(4, SHUT_WR) = 0
+ poll([{fd=4, events=POLLIN|POLLOUT|POLLRDHUP}], 1, 0) = 1 ([{fd=4, revents=POLLOUT}])
+
+and:
+ shutdown(4, SHUT_WR) = 0
+ sendto(5, "foo", 3, MSG_NOSIGNAL, NULL, 0) = 3
+ poll([{fd=4, events=POLLIN|POLLOUT|POLLRDHUP}], 1, 0) = 1 ([{fd=4, revents=POLLIN|POLLOUT}])
+
+and:
+ shutdown(4, SHUT_WR) = 0
+ sendto(4, "bar", 3, MSG_NOSIGNAL, NULL, 0) = -1 EPIPE (Broken pipe)
+ poll([{fd=4, events=POLLIN|POLLOUT|POLLRDHUP}], 1, 0) = 1 ([{fd=4, revents=POLLOUT}])
+
+
+POLLOUT is still reported after a SHUTWR:
+
+socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
+setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
+bind(3, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("0.0.0.0")}, 16) = 0
+listen(3, 1000) = 0
+getsockname(3, {sa_family=AF_INET, sin_port=htons(34729), sin_addr=inet_addr("0.0.0.0")}, [16]) = 0
+socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 4
+connect(4, {sa_family=AF_INET, sin_port=htons(34729), sin_addr=inet_addr("0.0.0.0")}, 16) = 0
+accept(3, 0x7ffcd6a68300, [0->16]) = 5
+fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 3), ...}) = 0
+brk(NULL) = 0xc4e000
+brk(0xc6f000) = 0xc6f000
+write(1, "\n", 1
+) = 1
+shutdown(4, SHUT_WR) = 0
+poll([{fd=4, events=POLLIN|POLLOUT|POLLRDHUP}], 1, 0) = 1 ([{fd=4, revents=POLLOUT}])
+write(1, "ret=1 ev={fd:4 ev:4}\n", 21ret=1 ev={fd:4 ev:4}
+) = 21
+close(5) = 0
+close(4) = 0
+close(3) = 0
+
+Performing a write() on it reports a SIGPIPE:
+
+shutdown(4, SHUT_WR) = 0
+sendto(4, "bar", 3, MSG_NOSIGNAL, NULL, 0) = -1 EPIPE (Broken pipe)
+poll([{fd=4, events=POLLIN|POLLOUT|POLLRDHUP}], 1, 0) = 1 ([{fd=4, revents=POLLOUT}])
+
+
+On SHUT_RD we see POLLIN|POLLOUT|POLLRDHUP (there's no data pending here) :
+shutdown(4, SHUT_RD) = 0
+poll([{fd=4, events=POLLIN|POLLOUT|POLLRDHUP}], 1, 0) = 1 ([{fd=4, revents=POLLIN|POLLOUT|POLLRDHUP}])
+
+
+What is observed in the end :
+ - POLLOUT is always reported for anything SHUT_WR even if it would cause a broken pipe, including listeners if they're also SHUT_RD
+ - POLLHUP is always reported for anything SHUT_WR + having a SHUT_RD pending with or without anything to read, including listeners
+ - POLLIN is always reported for anything to read or a pending zero
+ - POLLIN is NOT reported for SHUT_RD listeners, even with pending connections, only OUT+HUP are reported
+ - POLLIN and POLLRDHUP are always reported after a SHUTR
+ - POLLERR also enables IN,OUT,HUP,RHUP
+
+
+
+
+
+
+
+Currently there's a bit of an issue with connect() being too impatient to read:
+
+16:26:06.818521 connect(9, {sa_family=AF_INET, sin_port=htons(8000), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EINPROGRESS (Operation now in progress)
+16:26:06.818558 recvfrom(9, 0x1db9400, 16320, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
+16:26:06.818571 epoll_ctl(3, EPOLL_CTL_ADD, 9, {EPOLLIN|EPOLLOUT|EPOLLRDHUP, {u32=9, u64=9}}) = 0
+16:26:06.818588 epoll_wait(3, [{EPOLLOUT, {u32=9, u64=9}}], 200, 1000) = 1
+16:26:06.818603 connect(9, {sa_family=AF_INET, sin_port=htons(8000), sin_addr=inet_addr("127.0.0.1")}, 16) = 0
+16:26:06.818617 sendto(9, "GET /?s=10k HTTP/1.1\r\nhost: 127.0.0.1:4445\r\nuser-agent: curl/7.54.1\r\naccept: */*\r\n\r\n", 84, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 84
+16:26:06.818660 epoll_ctl(3, EPOLL_CTL_MOD, 9, {EPOLLIN|EPOLLRDHUP, {u32=9, u64=9}}) = 0
+16:26:06.818696 epoll_wait(3, [{EPOLLIN, {u32=9, u64=9}}], 200, 1000) = 1
+16:26:06.818711 recvfrom(9, "HTTP/1.1 200\r\nContent-length: 10240\r\nX-req: size=84, time=0 ms\r\nX-rsp: id=dummy, code=200, cache=1, size=10240, time=0 ms (0 real)\r\n\r\n89.123456789.12345678\n.123456789.123456789.123456789.123456789.123"..., 16320, 0, NULL, NULL) = 10374
+16:26:06.818735 recvfrom(9, 0x1dd75f6, 5946, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
+16:26:06.818790 epoll_ctl(3, EPOLL_CTL_DEL, 9, 0x7ffa818fd7d0) = 0
+16:26:06.818804 epoll_wait(3, [], 200, 0) = 0
+
+
+
+
+This one shows that the error is not definitive, it disappears once it's
+been signaled, then only shut remains! Also it's a proof that an error
+may well be reported after a shutw, so the r/w error may not be merged
+with a shutw since it may appear after an deliberate shutw.
+
+$ ./contrib/debug/poll -v -c snd,shw -s pol,rcv,pol,rcv,pol,snd,lin,clo -c pol,rcv,pol,rcv,pol,rcv,pol
+#### BEGIN ####
+cmd #1 stp #1: do_snd(4): ret=3
+cmd #1 stp #2: do_shw(4): ret=0
+cmd #2 stp #0: do_acc(3): ret=5
+cmd #2 stp #1: do_pol(5): ret=1 ev=0x2005 (IN OUT RDHUP)
+cmd #2 stp #2: do_rcv(5): ret=3
+cmd #2 stp #3: do_pol(5): ret=1 ev=0x2005 (IN OUT RDHUP)
+cmd #2 stp #4: do_rcv(5): ret=0
+cmd #2 stp #5: do_pol(5): ret=1 ev=0x2005 (IN OUT RDHUP)
+cmd #2 stp #6: do_snd(5): ret=3
+cmd #2 stp #7: do_lin(5): ret=0
+cmd #2 stp #8: do_clo(5): ret=0
+cmd #3 stp #1: do_pol(4): ret=1 ev=0x201d (IN OUT ERR HUP RDHUP)
+cmd #3 stp #2: do_rcv(4): ret=3
+cmd #3 stp #3: do_pol(4): ret=1 ev=0x201d (IN OUT ERR HUP RDHUP)
+cmd #3 stp #4: do_rcv(4): ret=-1 (Connection reset by peer)
+cmd #3 stp #5: do_pol(4): ret=1 ev=0x2015 (IN OUT HUP RDHUP)
+cmd #3 stp #6: do_rcv(4): ret=0
+cmd #3 stp #7: do_pol(4): ret=1 ev=0x2015 (IN OUT HUP RDHUP)
+#### END ####
diff --git a/doc/internals/notes-polling.txt b/doc/internals/notes-polling.txt
new file mode 100644
index 0000000..e7741a6
--- /dev/null
+++ b/doc/internals/notes-polling.txt
@@ -0,0 +1,192 @@
+2019-09-03
+
+u8 fd.state;
+u8 fd.ev;
+
+
+ev = one of :
+ #define FD_POLL_IN 0x01
+ #define FD_POLL_PRI 0x02
+ #define FD_POLL_OUT 0x04
+ #define FD_POLL_ERR 0x08
+ #define FD_POLL_HUP 0x10
+
+Could we instead have :
+
+ FD_WAIT_IN 0x01
+ FD_WAIT_OUT 0x02
+ FD_WAIT_PRI 0x04
+ FD_SEEN_HUP 0x08
+ FD_SEEN_HUP 0x10
+ FD_WAIT_CON 0x20 <<= shouldn't this be in the connection itself in fact ?
+
+=> not needed, covered by the state instead.
+
+What is missing though is :
+ - FD_DATA_PENDING -- overlaps with READY_R, OK if passed by pollers only
+ - FD_EOI_PENDING
+ - FD_ERR_PENDING
+ - FD_EOI
+ - FD_SHW
+ - FD_ERR
+
+fd_update_events() could do that :
+
+ if ((fd_data_pending|fd_eoi_pending|fd_err_pending) && !(fd_err|fd_eoi))
+ may_recv()
+
+ if (fd_send_ok && !(fd_err|fd_shw))
+ may_send()
+
+ if (fd_err)
+ wake()
+
+the poller could do that :
+ HUP+OUT => always indicates a failed connect(), it should not lack ERR. Is this err_pending ?
+
+ ERR HUP OUT IN
+ 0 0 0 0 => nothing
+ 0 0 0 1 => FD_DATA_PENDING
+ 0 0 1 0 => FD_SEND_OK
+ 0 0 1 1 => FD_DATA_PENDING|FD_SEND_OK
+ 0 1 0 0 => FD_EOI (|FD_SHW)
+ 0 1 0 1 => FD_DATA_PENDING|FD_EOI_PENDING (|FD_SHW)
+ 0 1 1 0 => FD_EOI |FD_ERR (|FD_SHW)
+ 0 1 1 1 => FD_EOI_PENDING (|FD_ERR_PENDING) |FD_DATA_PENDING (|FD_SHW)
+ 1 X 0 0 => FD_ERR | FD_EOI (|FD_SHW)
+ 1 X X 1 => FD_ERR_PENDING | FD_EOI_PENDING | FD_DATA_PENDING (|FD_SHW)
+ 1 X 1 0 => FD_ERR | FD_EOI (|FD_SHW)
+
+ OUT+HUP,OUT+HUP+ERR => FD_ERR
+
+This reorders to:
+
+ IN ERR HUP OUT
+ 0 0 0 0 => nothing
+ 0 0 0 1 => FD_SEND_OK
+ 0 0 1 0 => FD_EOI (|FD_SHW)
+
+ 0 X 1 1 => FD_ERR | FD_EOI (|FD_SHW)
+ 0 1 X 0 => FD_ERR | FD_EOI (|FD_SHW)
+ 0 1 X 1 => FD_ERR | FD_EOI (|FD_SHW)
+
+ 1 0 0 0 => FD_DATA_PENDING
+ 1 0 0 1 => FD_DATA_PENDING|FD_SEND_OK
+ 1 0 1 0 => FD_DATA_PENDING|FD_EOI_PENDING (|FD_SHW)
+ 1 0 1 1 => FD_EOI_PENDING (|FD_ERR_PENDING) |FD_DATA_PENDING (|FD_SHW)
+ 1 1 X X => FD_ERR_PENDING | FD_EOI_PENDING | FD_DATA_PENDING (|FD_SHW)
+
+Regarding "|SHW", it's normally useless since it will already have been done,
+except on connect() error where this indicates there's no need for SHW.
+
+FD_EOI and FD_SHW could be part of the state (FD_EV_SHUT_R, FD_EV_SHUT_W).
+Then all states having these bit and another one would be transient and need
+to resync. We could then have "fd_shut_recv" and "fd_shut_send" to turn these
+states.
+
+The FD's ev then only needs to update EOI_PENDING, ERR_PENDING, ERR, DATA_PENDING.
+With this said, these are not exactly polling states either, as err/eoi/shw are
+orthogonal to the other states and are required to update them so that the polling
+state really is DISABLED in the end. So we need more of an operational status for
+the FD containing EOI_PENDING, EOI, ERR_PENDING, ERR, SHW, CLO?. These could be
+classified in 3 categories: read:(OPEN, EOI_PENDING, EOI); write:(OPEN,SHW),
+ctrl:(OPEN,ERR_PENDING,ERR,CLO). That would be 2 bits for R, 1 for W, 2 for ctrl
+or total 5 vs 6 for individual ones, but would be harder to manipulate.
+
+Proposal:
+ - rename fdtab[].state to "polling_state"
+ - rename fdtab[].ev to "status"
+
+Note: POLLHUP is also reported is a listen() socket has gone in shutdown()
+TEMPORARILY! Thus we may not always consider this as a final error.
+
+
+Work hypothesis:
+
+SHUT RDY ACT
+ 0 0 0 => disabled
+ 0 0 1 => active
+ 0 1 0 => stopped
+ 0 1 1 => ready
+ 1 0 0 => final shut
+ 1 0 1 => shut pending without data
+ 1 1 0 => shut pending, stopped
+ 1 1 1 => shut pending
+
+PB: we can land into final shut if one thread disables the FD while another
+ one that was waiting on it reports it as shut. Theorically it should be
+ implicitly ready though, since reported. But if no data is reported, it
+ will be reportedly shut only. And no event will be reported then. This
+ might still make sense since it's not active, thus we don't want events.
+ But it will not be enabled later either in this case so the shut really
+ risks not to be properly reported. The issue is that there's no difference
+ between a shut coming from the bottom and a shut coming from the top, and
+ we need an event to report activity here. Or we may consider that a poller
+ never leaves a final shut by itself (100) and always reports it as
+ shut+stop (thus ready) if it was not active. Alternately, if active is
+ disabled, shut should possibly be ignored, then a poller cannot report
+ shut. But shut+stopped seems the most suitable as it corresponds to
+ disabled->stopped transition.
+
+Now let's add ERR. ERR necessarily implies SHUT as there doesn't seem to be a
+valid case of ERR pending without shut pending.
+
+ERR SHUT RDY ACT
+ 0 0 0 0 => disabled
+ 0 0 0 1 => active
+ 0 0 1 0 => stopped
+ 0 0 1 1 => ready
+
+ 0 1 0 0 => final shut, no error
+ 0 1 0 1 => shut pending without data
+ 0 1 1 0 => shut pending, stopped
+ 0 1 1 1 => shut pending
+
+ 1 0 X X => invalid
+
+ 1 1 0 0 => final shut, error encountered
+ 1 1 0 1 => error pending without data
+ 1 1 1 0 => error pending after data, stopped
+ 1 1 1 1 => error pending
+
+So the algorithm for the poller is:
+ - if (shutdown_pending or error) reported and ACT==0,
+ report SHUT|RDY or SHUT|ERR|RDY
+
+For read handlers :
+ - if (!(flags & (RDY|ACT)))
+ return
+ - if (ready)
+ try_to_read
+ - if (err)
+ report error
+ - if (shut)
+ read0
+
+For write handlers:
+ - if (!(flags & (RDY|ACT)))
+ return
+ - if (err||shut)
+ report error
+ - if (ready)
+ try_to_write
+
+For listeners:
+ - if (!(flags & (RDY|ACT)))
+ return
+ - if (err||shut)
+ pause
+ - if (ready)
+ try_to_accept
+
+Kqueue reports events differently, it says EV_EOF() on READ or WRITE, that
+we currently map to FD_POLL_HUP and FD_POLL_ERR. Thus kqueue reports only
+POLLRDHUP and not POLLHUP, so for now a direct mapping of POLLHUP to
+FD_POLL_HUP does NOT imply write closed with kqueue while it does for others.
+
+Other approach, use the {RD,WR}_{ERR,SHUT,RDY} flags to build a composite
+status in each poller and pass this to fd_update_events(). We normally
+have enough to be precise, and this latter will rework the events.
+
+FIXME: Normally on KQUEUE we're supposed to look at kev[].fflags to get the error
+on EV_EOF() on read or write.
diff --git a/doc/internals/pattern.dia b/doc/internals/pattern.dia
new file mode 100644
index 0000000..3d13215
--- /dev/null
+++ b/doc/internals/pattern.dia
Binary files differ
diff --git a/doc/internals/pattern.pdf b/doc/internals/pattern.pdf
new file mode 100644
index 0000000..a8d8bc9
--- /dev/null
+++ b/doc/internals/pattern.pdf
Binary files differ
diff --git a/doc/internals/polling-states.fig b/doc/internals/polling-states.fig
new file mode 100644
index 0000000..3b2c782
--- /dev/null
+++ b/doc/internals/polling-states.fig
@@ -0,0 +1,59 @@
+#FIG 3.2 Produced by xfig version 2.3
+Portrait
+Center
+Metric
+A4
+100.00
+Single
+-2
+1200 2
+2 1 0 1 0 7 50 -1 -1 0.000 1 0 -1 1 0 2
+ 1 1 1.00 90.00 180.00
+ 1125 1350 1125 1800
+2 1 0 1 0 7 50 -1 -1 0.000 1 0 -1 1 0 2
+ 1 1 1.00 90.00 180.00
+ 1125 2250 1125 2700
+2 1 0 1 0 7 50 -1 -1 0.000 1 0 -1 1 0 2
+ 1 1 1.00 90.00 180.00
+ 1125 3150 1125 3600
+2 1 0 1 0 7 50 -1 -1 0.000 1 0 -1 1 0 2
+ 1 1 1.00 90.00 180.00
+ 1575 1800 1575 1350
+2 1 0 1 0 7 50 -1 -1 0.000 1 0 -1 1 0 2
+ 1 1 1.00 90.00 180.00
+ 1575 3600 1575 3150
+2 4 0 1 0 7 51 -1 20 0.000 0 0 7 0 0 5
+ 1800 1350 900 1350 900 900 1800 900 1800 1350
+2 4 0 1 0 7 51 -1 20 0.000 0 0 7 0 0 5
+ 1800 2250 900 2250 900 1800 1800 1800 1800 2250
+2 4 0 1 0 7 51 -1 20 0.000 0 0 7 0 0 5
+ 1800 4050 900 4050 900 3600 1800 3600 1800 4050
+2 4 0 1 0 7 51 -1 20 0.000 0 0 7 0 0 5
+ 1800 3150 900 3150 900 2700 1800 2700 1800 3150
+2 1 0 1 0 7 50 -1 -1 0.000 1 0 -1 1 0 2
+ 1 1 1.00 90.00 180.00
+ 1350 450 1350 900
+2 1 0 1 0 7 50 -1 -1 0.000 1 0 -1 1 0 2
+ 1 1 1.00 90.00 180.00
+ 1575 2700 1575 2250
+4 2 0 50 -1 16 8 0.0000 4 105 270 1080 1485 want\001
+4 2 0 50 -1 16 8 0.0000 4 120 255 1035 3285 stop\001
+4 0 0 50 -1 16 8 0.0000 4 105 270 1665 3510 want\001
+4 1 0 50 -1 16 10 0.0000 4 120 735 1350 1080 STOPPED\001
+4 1 0 50 -1 16 10 0.0000 4 120 795 1350 3780 DISABLED\001
+4 1 0 50 -1 16 10 0.0000 4 120 555 1350 2880 ACTIVE\001
+4 1 0 50 -1 16 10 0.0000 4 120 540 1350 1980 READY\001
+4 0 0 50 -1 16 8 0.0000 4 90 210 1665 2565 may\001
+4 2 0 50 -1 16 8 0.0000 4 105 240 1035 2430 cant\001
+4 1 0 50 -1 16 8 0.0000 4 120 240 1350 1260 R,!A\001
+4 1 0 50 -1 16 8 0.0000 4 120 210 1350 2160 R,A\001
+4 1 0 50 -1 16 8 0.0000 4 120 240 1350 3060 !R,A\001
+4 1 0 50 -1 16 8 0.0000 4 120 270 1350 3960 !R,!A\001
+4 0 0 50 -1 16 8 0.0000 4 120 255 1665 1710 stop\001
+4 0 0 50 -1 16 10 0.0000 4 150 855 2520 1125 R=ready flag\001
+4 0 0 50 -1 16 10 0.0000 4 150 885 2520 1290 A=active flag\001
+4 0 0 50 -1 16 10 0.0000 4 150 1365 2520 2475 fd_want sets A flag\001
+4 0 0 50 -1 16 10 0.0000 4 150 1440 2520 2640 fd_stop clears A flag\001
+4 0 0 50 -1 16 10 0.0000 4 150 1905 2520 3300 update() updates the poller.\001
+4 0 0 50 -1 16 10 0.0000 4 150 2190 2520 2970 fd_cant clears R flag (EAGAIN)\001
+4 0 0 50 -1 16 10 0.0000 4 150 2115 2520 3135 fd_rdy sets R flag (poll return)\001
diff --git a/doc/internals/sched.fig b/doc/internals/sched.fig
new file mode 100644
index 0000000..4134420
--- /dev/null
+++ b/doc/internals/sched.fig
@@ -0,0 +1,748 @@
+#FIG 3.2 Produced by xfig version 2.4
+Landscape
+Center
+Metric
+A4
+150.00
+Single
+-2
+1200 2
+0 32 #c5ebe1
+0 33 #86c8a2
+0 34 #ffebac
+0 35 #cbb366
+0 36 #c7b696
+0 37 #effbff
+0 38 #dfcba6
+0 39 #414141
+0 40 #aeaaae
+0 41 #595559
+0 42 #414141
+0 43 #868286
+0 44 #bec3be
+0 45 #868286
+0 46 #bec3be
+0 47 #dfe3df
+0 48 #8e8e8e
+0 49 #8e8e8e
+0 50 #414141
+0 51 #868286
+0 52 #bec3be
+0 53 #dfe3df
+0 54 #414141
+0 55 #868286
+0 56 #bec3be
+0 57 #dfe3df
+0 58 #868286
+0 59 #bec3be
+0 60 #dfe3df
+0 61 #c7b696
+0 62 #effbff
+0 63 #dfcba6
+0 64 #c7b696
+0 65 #effbff
+0 66 #dfcba6
+0 67 #aeaaae
+0 68 #595559
+0 69 #8e8e8e
+0 70 #414141
+0 71 #868286
+0 72 #bec3be
+0 73 #dfe3df
+0 74 #414141
+0 75 #868286
+0 76 #bec3be
+0 77 #dfe3df
+0 78 #868286
+0 79 #bec3be
+0 80 #dfe3df
+0 81 #414141
+0 82 #868286
+0 83 #bec3be
+0 84 #414141
+0 85 #bec3be
+0 86 #dfe3df
+0 87 #414141
+0 88 #868286
+0 89 #bec3be
+0 90 #8e8e8e
+0 91 #414141
+0 92 #868286
+0 93 #bec3be
+0 94 #dfe3df
+0 95 #414141
+0 96 #868286
+0 97 #bec3be
+0 98 #dfe3df
+0 99 #bebebe
+0 100 #515151
+0 101 #e7e3e7
+0 102 #000049
+0 103 #797979
+0 104 #303430
+0 105 #414541
+0 106 #414141
+0 107 #868286
+0 108 #bec3be
+0 109 #dfe3df
+0 110 #cfcfcf
+0 111 #cfcfcf
+0 112 #cfcfcf
+0 113 #cfcfcf
+0 114 #cfcfcf
+0 115 #cfcfcf
+0 116 #cfcfcf
+0 117 #cfcfcf
+0 118 #cfcfcf
+0 119 #cfcfcf
+0 120 #cfcfcf
+0 121 #cfcfcf
+0 122 #cfcfcf
+0 123 #cfcfcf
+0 124 #cfcfcf
+0 125 #cfcfcf
+0 126 #cfcfcf
+0 127 #cfcfcf
+0 128 #cfcfcf
+0 129 #cfcfcf
+0 130 #cfcfcf
+0 131 #cfcfcf
+0 132 #cfcfcf
+0 133 #cfcfcf
+0 134 #cfcfcf
+0 135 #cfcfcf
+0 136 #cfcfcf
+0 137 #cfcfcf
+0 138 #cfcfcf
+0 139 #cfcfcf
+0 140 #cfcfcf
+0 141 #cfcfcf
+0 142 #cfcfcf
+0 143 #cfcfcf
+0 144 #cfcfcf
+0 145 #cfcfcf
+0 146 #cfcfcf
+0 147 #cfcfcf
+0 148 #cfcfcf
+0 149 #cfcfcf
+0 150 #c7c3c7
+0 151 #868286
+0 152 #bec3be
+0 153 #dfe3df
+0 154 #8e8e8e
+0 155 #8e8e8e
+0 156 #494549
+0 157 #868686
+0 158 #c7c7c7
+0 159 #e7e7e7
+0 160 #f7f7f7
+0 161 #9e9e9e
+0 162 #717571
+0 163 #aeaaae
+0 164 #494549
+0 165 #aeaaae
+0 166 #595559
+0 167 #bec3be
+0 168 #dfe3df
+0 169 #494549
+0 170 #616561
+0 171 #494549
+0 172 #868286
+0 173 #bec3be
+0 174 #dfe3df
+0 175 #bec3be
+0 176 #dfe3df
+0 177 #c7b696
+0 178 #effbff
+0 179 #dfcba6
+0 180 #414141
+0 181 #868286
+0 182 #bec3be
+0 183 #dfe3df
+0 184 #8e8e8e
+0 185 #aeaaae
+0 186 #595559
+0 187 #414141
+0 188 #868286
+0 189 #bec3be
+0 190 #868286
+0 191 #bec3be
+0 192 #dfe3df
+0 193 #8e8e8e
+0 194 #8e8e8e
+0 195 #414141
+0 196 #868286
+0 197 #bec3be
+0 198 #dfe3df
+0 199 #414141
+0 200 #868286
+0 201 #bec3be
+0 202 #dfe3df
+0 203 #868286
+0 204 #bec3be
+0 205 #dfe3df
+0 206 #c7b696
+0 207 #effbff
+0 208 #dfcba6
+0 209 #c7b696
+0 210 #effbff
+0 211 #dfcba6
+0 212 #aeaaae
+0 213 #595559
+0 214 #8e8e8e
+0 215 #414141
+0 216 #868286
+0 217 #bec3be
+0 218 #dfe3df
+0 219 #414141
+0 220 #868286
+0 221 #bec3be
+0 222 #dfe3df
+0 223 #868286
+0 224 #bec3be
+0 225 #dfe3df
+0 226 #414141
+0 227 #868286
+0 228 #bec3be
+0 229 #414141
+0 230 #bec3be
+0 231 #dfe3df
+0 232 #414141
+0 233 #868286
+0 234 #bec3be
+0 235 #8e8e8e
+0 236 #414141
+0 237 #868286
+0 238 #bec3be
+0 239 #dfe3df
+0 240 #414141
+0 241 #868286
+0 242 #bec3be
+0 243 #dfe3df
+0 244 #414141
+0 245 #868286
+0 246 #bec3be
+0 247 #dfe3df
+0 248 #868286
+0 249 #bec3be
+0 250 #dfe3df
+0 251 #8e8e8e
+0 252 #8e8e8e
+0 253 #494549
+0 254 #aeaaae
+0 255 #494549
+0 256 #aeaaae
+0 257 #595559
+0 258 #bec3be
+0 259 #dfe3df
+0 260 #494549
+0 261 #616561
+0 262 #494549
+0 263 #868286
+0 264 #bec3be
+0 265 #dfe3df
+0 266 #bec3be
+0 267 #dfe3df
+0 268 #dfe3ef
+0 269 #96969e
+0 270 #d7dbd7
+0 271 #9ea2b6
+0 272 #9e0000
+0 273 #efefef
+0 274 #86aeff
+0 275 #7171ff
+0 276 #bbf2e2
+0 277 #a7ceb3
+0 278 #dae8fc
+0 279 #458dba
+0 280 #ffe6cc
+0 281 #e9b000
+0 282 #1a1a1a
+0 283 #ffc1e7
+0 284 #009ed7
+0 285 #006d9e
+0 286 #00719e
+0 287 #9e9a9e
+0 288 #000000
+0 289 #595959
+0 290 #006596
+0 291 #00a6d7
+0 292 #b6b6b6
+0 293 #8edbef
+0 294 #00699e
+0 295 #595d59
+0 296 #69d3e7
+0 297 #a6e3ef
+0 298 #9ec7d7
+0 299 #aeb2ae
+0 300 #00b6df
+0 301 #00aed7
+0 302 #797d79
+0 303 #00a2d7
+0 304 #303030
+0 305 #006996
+0 306 #086d9e
+0 307 #86b6cf
+0 308 #f7fbf7
+0 309 #9ec3d7
+0 310 #ffff96
+0 311 #ff600a
+5 1 0 2 0 7 50 -1 -1 0.000 0 0 1 0 11301.000 3060.000 11205 3825 10530 3060 11205 2295
+ 0 0 1.00 60.00 120.00
+5 1 0 2 0 7 50 -1 -1 0.000 0 1 0 1 11289.000 3060.000 11385 3825 12060 3060 11385 2295
+ 0 0 1.00 60.00 120.00
+5 1 0 2 0 7 50 -1 -1 0.000 0 1 0 1 11293.750 3060.000 10890 3105 11700 3060 10890 3015
+ 2 1 1.00 60.00 120.00
+5 1 0 2 0 7 50 -1 -1 0.000 0 0 1 0 7611.000 3060.000 7515 3825 6840 3060 7515 2295
+ 0 0 1.00 60.00 120.00
+5 1 0 2 0 7 50 -1 -1 0.000 0 1 0 1 7599.000 3060.000 7695 3825 8370 3060 7695 2295
+ 0 0 1.00 60.00 120.00
+5 1 0 2 0 7 50 -1 -1 0.000 0 1 0 1 7603.750 3060.000 7200 3105 8010 3060 7200 3015
+ 2 1 1.00 60.00 120.00
+5 1 0 2 0 7 50 -1 -1 0.000 0 0 1 0 4956.000 3060.000 4860 3825 4185 3060 4860 2295
+ 0 0 1.00 60.00 120.00
+5 1 0 2 0 7 50 -1 -1 0.000 0 1 0 1 4944.000 3060.000 5040 3825 5715 3060 5040 2295
+ 0 0 1.00 60.00 120.00
+5 1 0 2 0 7 50 -1 -1 0.000 0 1 0 1 4948.750 3060.000 4545 3105 5355 3060 4545 3015
+ 2 1 1.00 60.00 120.00
+5 1 0 2 0 7 50 -1 -1 0.000 0 0 1 0 1266.000 3060.000 1170 3825 495 3060 1170 2295
+ 0 0 1.00 60.00 120.00
+5 1 0 2 0 7 50 -1 -1 0.000 0 1 0 1 1254.000 3060.000 1350 3825 2025 3060 1350 2295
+ 0 0 1.00 60.00 120.00
+5 1 0 2 0 7 50 -1 -1 0.000 0 1 0 1 1258.750 3060.000 855 3105 1665 3060 855 3015
+ 2 1 1.00 60.00 120.00
+6 10606 2371 11985 3749
+2 3 0 2 0 31 50 -1 20 0.000 0 0 -1 0 0 15
+ 11768 3060 11970 3060 11967 3119 11959 3177 11946 3234 11929 3291
+ 11907 3345 11879 3397 11704 3296 11723 3259 11738 3222 11751 3182
+ 11760 3142 11765 3101 11768 3060
+2 3 0 2 0 31 50 -1 20 0.000 0 0 -1 0 0 15
+ 11704 3296 11879 3397 11848 3447 11812 3494 11772 3537 11729 3577
+ 11682 3613 11633 3644 11531 3469 11566 3447 11599 3422 11628 3393
+ 11657 3364 11682 3331 11704 3296
+2 3 0 2 0 31 50 -1 20 0.000 0 0 -1 0 0 15
+ 11531 3469 11633 3644 11580 3672 11526 3694 11469 3711 11412 3724
+ 11354 3732 11295 3734 11295 3532 11336 3530 11377 3525 11417 3516
+ 11457 3503 11494 3488 11531 3469
+2 3 0 2 0 31 50 -1 20 0.000 0 0 -1 0 0 15
+ 11295 3532 11295 3734 11236 3732 11178 3724 11121 3711 11064 3694
+ 11010 3672 10958 3644 11059 3469 11096 3488 11133 3503 11173 3516
+ 11213 3525 11254 3530 11295 3532
+2 3 0 2 0 31 50 -1 20 0.000 0 0 -1 0 0 15
+ 11059 3469 10958 3644 10908 3613 10861 3577 10818 3537 10778 3494
+ 10742 3447 10711 3398 10886 3296 10908 3331 10933 3364 10962 3393
+ 10991 3422 11024 3447 11059 3469
+2 3 0 2 0 31 50 -1 20 0.000 0 0 -1 0 0 15
+ 10886 3296 10711 3398 10683 3345 10661 3291 10644 3234 10631 3177
+ 10623 3119 10621 3060 10823 3060 10825 3101 10830 3142 10839 3182
+ 10852 3222 10867 3259 10886 3296
+2 3 0 2 0 31 50 -1 20 0.000 0 0 -1 0 0 15
+ 10823 3060 10621 3060 10623 3001 10631 2943 10644 2886 10661 2829
+ 10683 2775 10711 2723 10886 2824 10867 2861 10852 2898 10839 2938
+ 10830 2978 10825 3019 10823 3060
+2 3 0 2 0 31 50 -1 20 0.000 0 0 -1 0 0 15
+ 10886 2824 10711 2723 10742 2673 10778 2626 10818 2583 10861 2543
+ 10908 2507 10958 2476 11059 2651 11024 2673 10991 2698 10962 2727
+ 10933 2756 10908 2789 10886 2824
+2 3 0 2 0 31 50 -1 20 0.000 0 0 -1 0 0 15
+ 11059 2651 10958 2476 11010 2448 11064 2426 11121 2409 11178 2396
+ 11236 2388 11295 2386 11295 2588 11254 2590 11213 2595 11173 2604
+ 11133 2617 11096 2632 11059 2651
+2 3 0 2 0 31 50 -1 20 0.000 0 0 -1 0 0 15
+ 11295 2588 11295 2386 11354 2388 11412 2396 11469 2409 11526 2426
+ 11580 2448 11632 2476 11531 2651 11494 2632 11457 2617 11417 2604
+ 11377 2595 11336 2590 11295 2588
+2 3 0 2 0 31 50 -1 20 0.000 0 0 -1 0 0 15
+ 11531 2651 11632 2476 11682 2507 11729 2543 11772 2583 11812 2626
+ 11848 2673 11879 2723 11704 2824 11682 2789 11657 2756 11628 2727
+ 11599 2698 11566 2673 11531 2651
+2 3 0 2 0 31 50 -1 20 0.000 0 0 -1 0 0 15
+ 11704 2824 11879 2723 11907 2775 11929 2829 11946 2886 11959 2943
+ 11967 3001 11969 3060 11767 3060 11765 3019 11760 2978 11751 2938
+ 11738 2898 11723 2861 11704 2824
+-6
+6 4261 2371 5640 3749
+2 3 0 2 0 13 50 -1 20 0.000 0 0 -1 0 0 15
+ 5423 3060 5625 3060 5622 3119 5614 3177 5601 3234 5584 3291
+ 5562 3345 5534 3397 5359 3296 5378 3259 5393 3222 5406 3182
+ 5415 3142 5420 3101 5423 3060
+2 3 0 2 0 13 50 -1 20 0.000 0 0 -1 0 0 15
+ 5359 3296 5534 3397 5503 3447 5467 3494 5427 3537 5384 3577
+ 5337 3613 5288 3644 5186 3469 5221 3447 5254 3422 5283 3393
+ 5312 3364 5337 3331 5359 3296
+2 3 0 2 0 13 50 -1 20 0.000 0 0 -1 0 0 15
+ 5186 3469 5288 3644 5235 3672 5181 3694 5124 3711 5067 3724
+ 5009 3732 4950 3734 4950 3532 4991 3530 5032 3525 5072 3516
+ 5112 3503 5149 3488 5186 3469
+2 3 0 2 0 13 50 -1 20 0.000 0 0 -1 0 0 15
+ 4950 3532 4950 3734 4891 3732 4833 3724 4776 3711 4719 3694
+ 4665 3672 4613 3644 4714 3469 4751 3488 4788 3503 4828 3516
+ 4868 3525 4909 3530 4950 3532
+2 3 0 2 0 13 50 -1 20 0.000 0 0 -1 0 0 15
+ 4714 3469 4613 3644 4563 3613 4516 3577 4473 3537 4433 3494
+ 4397 3447 4366 3398 4541 3296 4563 3331 4588 3364 4617 3393
+ 4646 3422 4679 3447 4714 3469
+2 3 0 2 0 13 50 -1 20 0.000 0 0 -1 0 0 15
+ 4541 3296 4366 3398 4338 3345 4316 3291 4299 3234 4286 3177
+ 4278 3119 4276 3060 4478 3060 4480 3101 4485 3142 4494 3182
+ 4507 3222 4522 3259 4541 3296
+2 3 0 2 0 13 50 -1 20 0.000 0 0 -1 0 0 15
+ 4478 3060 4276 3060 4278 3001 4286 2943 4299 2886 4316 2829
+ 4338 2775 4366 2723 4541 2824 4522 2861 4507 2898 4494 2938
+ 4485 2978 4480 3019 4478 3060
+2 3 0 2 0 13 50 -1 20 0.000 0 0 -1 0 0 15
+ 4541 2824 4366 2723 4397 2673 4433 2626 4473 2583 4516 2543
+ 4563 2507 4613 2476 4714 2651 4679 2673 4646 2698 4617 2727
+ 4588 2756 4563 2789 4541 2824
+2 3 0 2 0 13 50 -1 20 0.000 0 0 -1 0 0 15
+ 4714 2651 4613 2476 4665 2448 4719 2426 4776 2409 4833 2396
+ 4891 2388 4950 2386 4950 2588 4909 2590 4868 2595 4828 2604
+ 4788 2617 4751 2632 4714 2651
+2 3 0 2 0 13 50 -1 20 0.000 0 0 -1 0 0 15
+ 4950 2588 4950 2386 5009 2388 5067 2396 5124 2409 5181 2426
+ 5235 2448 5287 2476 5186 2651 5149 2632 5112 2617 5072 2604
+ 5032 2595 4991 2590 4950 2588
+2 3 0 2 0 13 50 -1 20 0.000 0 0 -1 0 0 15
+ 5186 2651 5287 2476 5337 2507 5384 2543 5427 2583 5467 2626
+ 5503 2673 5534 2723 5359 2824 5337 2789 5312 2756 5283 2727
+ 5254 2698 5221 2673 5186 2651
+2 3 0 2 0 13 50 -1 20 0.000 0 0 -1 0 0 15
+ 5359 2824 5534 2723 5562 2775 5584 2829 5601 2886 5614 2943
+ 5622 3001 5624 3060 5422 3060 5420 3019 5415 2978 5406 2938
+ 5393 2898 5378 2861 5359 2824
+-6
+6 2250 4815 3960 5265
+1 1 0 3 8 11 52 -1 20 0.000 1 0.0000 3105 5049 810 171 3105 5049 3915 5049
+4 1 0 50 -1 6 10 0.0000 4 150 1125 3105 5130 Most Urgent\001
+-6
+6 6916 2371 8295 3749
+2 3 0 2 0 31 50 -1 43 0.000 0 0 -1 0 0 15
+ 8078 3060 8280 3060 8277 3119 8269 3177 8256 3234 8239 3291
+ 8217 3345 8189 3397 8014 3296 8033 3259 8048 3222 8061 3182
+ 8070 3142 8075 3101 8078 3060
+2 3 0 2 0 31 50 -1 43 0.000 0 0 -1 0 0 15
+ 8014 3296 8189 3397 8158 3447 8122 3494 8082 3537 8039 3577
+ 7992 3613 7943 3644 7841 3469 7876 3447 7909 3422 7938 3393
+ 7967 3364 7992 3331 8014 3296
+2 3 0 2 0 31 50 -1 43 0.000 0 0 -1 0 0 15
+ 7841 3469 7943 3644 7890 3672 7836 3694 7779 3711 7722 3724
+ 7664 3732 7605 3734 7605 3532 7646 3530 7687 3525 7727 3516
+ 7767 3503 7804 3488 7841 3469
+2 3 0 2 0 31 50 -1 43 0.000 0 0 -1 0 0 15
+ 7605 3532 7605 3734 7546 3732 7488 3724 7431 3711 7374 3694
+ 7320 3672 7268 3644 7369 3469 7406 3488 7443 3503 7483 3516
+ 7523 3525 7564 3530 7605 3532
+2 3 0 2 0 31 50 -1 43 0.000 0 0 -1 0 0 15
+ 7369 3469 7268 3644 7218 3613 7171 3577 7128 3537 7088 3494
+ 7052 3447 7021 3398 7196 3296 7218 3331 7243 3364 7272 3393
+ 7301 3422 7334 3447 7369 3469
+2 3 0 2 0 31 50 -1 43 0.000 0 0 -1 0 0 15
+ 7196 3296 7021 3398 6993 3345 6971 3291 6954 3234 6941 3177
+ 6933 3119 6931 3060 7133 3060 7135 3101 7140 3142 7149 3182
+ 7162 3222 7177 3259 7196 3296
+2 3 0 2 0 31 50 -1 43 0.000 0 0 -1 0 0 15
+ 7133 3060 6931 3060 6933 3001 6941 2943 6954 2886 6971 2829
+ 6993 2775 7021 2723 7196 2824 7177 2861 7162 2898 7149 2938
+ 7140 2978 7135 3019 7133 3060
+2 3 0 2 0 31 50 -1 43 0.000 0 0 -1 0 0 15
+ 7196 2824 7021 2723 7052 2673 7088 2626 7128 2583 7171 2543
+ 7218 2507 7268 2476 7369 2651 7334 2673 7301 2698 7272 2727
+ 7243 2756 7218 2789 7196 2824
+2 3 0 2 0 31 50 -1 43 0.000 0 0 -1 0 0 15
+ 7369 2651 7268 2476 7320 2448 7374 2426 7431 2409 7488 2396
+ 7546 2388 7605 2386 7605 2588 7564 2590 7523 2595 7483 2604
+ 7443 2617 7406 2632 7369 2651
+2 3 0 2 0 31 50 -1 43 0.000 0 0 -1 0 0 15
+ 7605 2588 7605 2386 7664 2388 7722 2396 7779 2409 7836 2426
+ 7890 2448 7942 2476 7841 2651 7804 2632 7767 2617 7727 2604
+ 7687 2595 7646 2590 7605 2588
+2 3 0 2 0 31 50 -1 43 0.000 0 0 -1 0 0 15
+ 7841 2651 7942 2476 7992 2507 8039 2543 8082 2583 8122 2626
+ 8158 2673 8189 2723 8014 2824 7992 2789 7967 2756 7938 2727
+ 7909 2698 7876 2673 7841 2651
+2 3 0 2 0 31 50 -1 43 0.000 0 0 -1 0 0 15
+ 8014 2824 8189 2723 8217 2775 8239 2829 8256 2886 8269 2943
+ 8277 3001 8279 3060 8077 3060 8075 3019 8070 2978 8061 2938
+ 8048 2898 8033 2861 8014 2824
+-6
+6 571 2371 1950 3749
+2 3 0 2 0 13 50 -1 43 0.000 0 0 -1 0 0 15
+ 1733 3060 1935 3060 1932 3119 1924 3177 1911 3234 1894 3291
+ 1872 3345 1844 3397 1669 3296 1688 3259 1703 3222 1716 3182
+ 1725 3142 1730 3101 1733 3060
+2 3 0 2 0 13 50 -1 43 0.000 0 0 -1 0 0 15
+ 1669 3296 1844 3397 1813 3447 1777 3494 1737 3537 1694 3577
+ 1647 3613 1598 3644 1496 3469 1531 3447 1564 3422 1593 3393
+ 1622 3364 1647 3331 1669 3296
+2 3 0 2 0 13 50 -1 43 0.000 0 0 -1 0 0 15
+ 1496 3469 1598 3644 1545 3672 1491 3694 1434 3711 1377 3724
+ 1319 3732 1260 3734 1260 3532 1301 3530 1342 3525 1382 3516
+ 1422 3503 1459 3488 1496 3469
+2 3 0 2 0 13 50 -1 43 0.000 0 0 -1 0 0 15
+ 1260 3532 1260 3734 1201 3732 1143 3724 1086 3711 1029 3694
+ 975 3672 923 3644 1024 3469 1061 3488 1098 3503 1138 3516
+ 1178 3525 1219 3530 1260 3532
+2 3 0 2 0 13 50 -1 43 0.000 0 0 -1 0 0 15
+ 1024 3469 923 3644 873 3613 826 3577 783 3537 743 3494
+ 707 3447 676 3398 851 3296 873 3331 898 3364 927 3393
+ 956 3422 989 3447 1024 3469
+2 3 0 2 0 13 50 -1 43 0.000 0 0 -1 0 0 15
+ 851 3296 676 3398 648 3345 626 3291 609 3234 596 3177
+ 588 3119 586 3060 788 3060 790 3101 795 3142 804 3182
+ 817 3222 832 3259 851 3296
+2 3 0 2 0 13 50 -1 43 0.000 0 0 -1 0 0 15
+ 788 3060 586 3060 588 3001 596 2943 609 2886 626 2829
+ 648 2775 676 2723 851 2824 832 2861 817 2898 804 2938
+ 795 2978 790 3019 788 3060
+2 3 0 2 0 13 50 -1 43 0.000 0 0 -1 0 0 15
+ 851 2824 676 2723 707 2673 743 2626 783 2583 826 2543
+ 873 2507 923 2476 1024 2651 989 2673 956 2698 927 2727
+ 898 2756 873 2789 851 2824
+2 3 0 2 0 13 50 -1 43 0.000 0 0 -1 0 0 15
+ 1024 2651 923 2476 975 2448 1029 2426 1086 2409 1143 2396
+ 1201 2388 1260 2386 1260 2588 1219 2590 1178 2595 1138 2604
+ 1098 2617 1061 2632 1024 2651
+2 3 0 2 0 13 50 -1 43 0.000 0 0 -1 0 0 15
+ 1260 2588 1260 2386 1319 2388 1377 2396 1434 2409 1491 2426
+ 1545 2448 1597 2476 1496 2651 1459 2632 1422 2617 1382 2604
+ 1342 2595 1301 2590 1260 2588
+2 3 0 2 0 13 50 -1 43 0.000 0 0 -1 0 0 15
+ 1496 2651 1597 2476 1647 2507 1694 2543 1737 2583 1777 2626
+ 1813 2673 1844 2723 1669 2824 1647 2789 1622 2756 1593 2727
+ 1564 2698 1531 2673 1496 2651
+2 3 0 2 0 13 50 -1 43 0.000 0 0 -1 0 0 15
+ 1669 2824 1844 2723 1872 2775 1894 2829 1911 2886 1924 2943
+ 1932 3001 1934 3060 1732 3060 1730 3019 1725 2978 1716 2938
+ 1703 2898 1688 2861 1669 2824
+-6
+6 1800 1845 2520 2385
+4 1 0 50 -1 6 10 0.0000 4 120 570 2160 1980 Global\001
+4 1 0 50 -1 6 10 0.0000 4 120 495 2160 2160 tasks\001
+4 1 0 50 -1 6 10 0.0000 4 150 705 2160 2340 (locked)\001
+-6
+6 3960 1935 4500 2250
+4 1 0 50 -1 6 10 0.0000 4 120 495 4230 2250 tasks\001
+4 1 0 50 -1 6 10 0.0000 4 120 465 4230 2070 Local\001
+-6
+6 8190 1845 8910 2385
+4 1 0 50 -1 6 10 0.0000 4 150 705 8550 2340 (locked)\001
+4 1 0 50 -1 6 10 0.0000 4 120 585 8550 2160 timers\001
+4 1 0 50 -1 6 10 0.0000 4 120 570 8550 1980 Global\001
+-6
+6 10215 1935 10845 2250
+4 1 0 50 -1 6 10 0.0000 4 120 585 10530 2250 timers\001
+4 1 0 50 -1 6 10 0.0000 4 120 465 10530 2070 Local\001
+-6
+6 2430 945 3735 1530
+1 1 0 3 20 29 52 -1 20 0.000 1 0.0000 3083 1180 607 170 3083 1180 3690 1350
+4 1 0 50 -1 6 10 0.0000 4 120 615 3105 1260 Local ?\001
+4 0 0 50 -1 6 9 0.0000 4 105 315 3375 1530 Yes\001
+4 2 0 50 -1 6 9 0.0000 4 105 225 2790 1530 No\001
+-6
+6 8775 945 10080 1530
+1 1 0 3 20 29 52 -1 20 0.000 1 0.0000 9428 1180 607 170 9428 1180 10035 1350
+4 1 0 50 -1 6 10 0.0000 4 120 615 9450 1260 Local ?\001
+4 0 0 50 -1 6 9 0.0000 4 105 315 9720 1530 Yes\001
+4 2 0 50 -1 6 9 0.0000 4 105 225 9135 1530 No\001
+-6
+6 7200 6345 9810 6885
+2 1 0 4 279 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 7234 6398 9776 6398 9776 6838 7234 6838
+2 3 0 0 -1 278 49 -1 20 0.000 0 0 -1 0 0 5
+ 7234 6838 9776 6838 9776 6398 7234 6398 7234 6838
+2 1 0 2 279 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9613 6398 9613 6838
+2 1 0 2 279 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9438 6398 9438 6838
+2 1 0 2 279 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9264 6398 9264 6838
+4 1 0 46 -1 4 16 0.0000 4 210 1620 8460 6705 TL_URGENT\001
+-6
+6 4140 7830 4545 9045
+1 1 0 3 20 29 52 -1 20 0.000 1 1.5708 4330 8437 607 170 4330 8437 4500 7830
+4 1 0 50 -1 6 10 1.5708 4 120 585 4410 8415 Class?\001
+-6
+1 1 0 3 8 11 52 -1 20 0.000 1 0.0000 9450 5049 540 171 9450 5049 9990 5049
+1 1 0 3 20 29 52 -1 20 0.000 1 1.5708 2440 7672 607 170 2440 7672 2610 7065
+1 1 0 3 8 11 52 -1 20 0.000 1 1.5708 10755 7695 810 171 10755 7695 10755 6885
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 4
+ 2 1 1.00 90.00 180.00
+ 7605 3870 7605 4185 9270 4545 9270 4905
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 4
+ 2 1 1.00 90.00 180.00
+ 11301 3870 11301 4185 9636 4545 9636 4905
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 4
+ 2 1 1.00 90.00 180.00
+ 9630 1395 9626 1591 11291 1800 11295 2295
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 4
+ 2 1 1.00 90.00 180.00
+ 9270 1395 9270 1575 7605 1800 7605 2295
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 2
+ 2 1 1.00 90.00 180.00
+ 9450 360 9450 1035
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 4
+ 2 1 1.00 90.00 180.00
+ 1260 3870 1260 4185 2925 4545 2925 4905
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 4
+ 2 1 1.00 90.00 180.00
+ 4956 3870 4956 4185 3291 4545 3291 4905
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 2
+ 2 1 1.00 90.00 180.00
+ 3105 360 3105 1035
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 4
+ 2 1 1.00 90.00 180.00
+ 3285 1395 3285 1575 4950 1845 4950 2385
+2 1 0 3 22 7 54 -1 -1 0.000 1 0 -1 0 0 2
+ 9180 5535 9000 5805
+2 1 0 5 13 7 54 -1 -1 0.000 1 0 -1 1 0 4
+ 2 1 1.00 120.00 240.00
+ 3105 5220 3105 5850 3105 7200 7200 7200
+2 1 0 5 22 7 54 -1 -1 0.000 1 0 -1 1 0 5
+ 2 1 1.00 120.00 240.00
+ 9450 5220 9450 5670 6300 5670 6300 1215 3690 1170
+2 1 0 3 13 7 54 -1 -1 0.000 1 0 -1 0 0 2
+ 3195 5535 3015 5805
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 4
+ 2 1 1.00 90.00 180.00
+ 2925 1395 2925 1575 1260 1845 1260 2385
+2 2 0 3 35 34 100 -1 20 0.000 1 0 -1 0 0 5
+ 6570 720 12330 720 12330 5400 6570 5400 6570 720
+2 2 0 3 33 32 100 -1 20 0.000 1 0 -1 0 0 5
+ 270 720 6030 720 6030 5400 270 5400 270 720
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 2
+ 2 1 1.00 90.00 180.00
+ 315 7650 2250 7650
+2 1 0 5 4 7 54 -1 -1 0.000 1 0 -1 1 0 2
+ 2 1 1.00 120.00 240.00
+ 10890 7695 12285 7695
+2 1 0 3 0 7 54 -1 -1 0.000 1 0 -1 1 0 3
+ 2 1 1.00 90.00 180.00
+ 4455 8775 4725 8910 7200 8910
+2 1 0 4 279 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 7234 7118 9776 7118 9776 7558 7234 7558
+2 3 0 0 -1 278 49 -1 20 0.000 0 0 -1 0 0 5
+ 7234 7558 9776 7558 9776 7118 7234 7118 7234 7558
+2 1 0 2 279 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9613 7118 9613 7558
+2 1 0 2 279 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9438 7118 9438 7558
+2 1 0 2 279 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9264 7118 9264 7558
+2 3 0 0 -1 278 49 -1 20 0.000 0 0 -1 0 0 5
+ 7234 8278 9776 8278 9776 7838 7234 7838 7234 8278
+2 1 0 2 279 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9613 7838 9613 8278
+2 1 0 2 279 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9438 7838 9438 8278
+2 1 0 2 279 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9264 7838 9264 8278
+2 1 0 4 279 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 7234 8558 9776 8558 9776 8998 7234 8998
+2 3 0 0 -1 278 49 -1 20 0.000 0 0 -1 0 0 5
+ 7234 8998 9776 8998 9776 8558 7234 8558 7234 8998
+2 1 0 2 279 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9613 8558 9613 8998
+2 1 0 2 279 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9438 8558 9438 8998
+2 1 0 2 279 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9264 8558 9264 8998
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 2
+ 2 1 1.00 90.00 180.00
+ 6075 6480 7200 6480
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 3
+ 2 1 1.00 90.00 180.00
+ 2610 7830 3195 8415 4140 8415
+2 1 0 4 45 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 4166 6398 6094 6398 6094 6838 4166 6838
+2 1 0 2 45 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 5923 6398 5923 6838
+2 1 0 2 45 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 5748 6398 5748 6838
+2 1 0 2 45 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 5574 6398 5574 6838
+2 1 0 3 0 7 54 -1 -1 0.000 1 0 -1 1 0 4
+ 2 1 1.00 90.00 180.00
+ 2610 7515 2925 6660 3645 6660 4140 6660
+2 3 0 0 277 276 49 -1 43 0.000 0 0 -1 0 0 5
+ 4166 6838 6094 6838 6094 6398 4166 6398 4166 6838
+2 1 0 3 0 7 54 -1 -1 0.000 1 0 -1 1 0 3
+ 2 1 1.00 90.00 180.00
+ 9765 8775 10350 8775 10665 8280
+2 1 0 3 0 7 54 -1 -1 0.000 1 0 -1 1 0 3
+ 2 1 1.00 90.00 180.00
+ 9765 8055 10305 8055 10620 7875
+2 1 0 3 0 7 54 -1 -1 0.000 1 0 -1 1 0 3
+ 2 1 1.00 90.00 180.00
+ 9806 6605 10350 6615 10665 7155
+2 1 0 3 0 7 54 -1 -1 0.000 1 0 -1 1 0 3
+ 2 1 1.00 90.00 180.00
+ 9720 7335 10350 7335 10620 7560
+2 1 1 5 4 7 57 -1 -1 12.000 1 0 -1 1 0 2
+ 2 1 1.00 120.00 240.00
+ 9900 6165 9900 9450
+2 1 0 2 0 7 54 -1 -1 0.000 1 0 -1 0 0 2
+ 10080 7245 9990 7425
+2 1 0 2 0 7 54 -1 -1 0.000 1 0 -1 0 0 2
+ 10080 7965 9990 8145
+2 1 0 2 0 7 54 -1 -1 0.000 1 0 -1 0 0 2
+ 10080 8685 9990 8865
+2 1 0 3 0 7 54 -1 -1 0.000 1 0 -1 1 0 4
+ 2 1 1.00 90.00 180.00
+ 4500 8550 6255 8550 6705 8190 7200 8190
+2 1 0 3 0 7 54 -1 -1 0.000 1 0 -1 1 0 5
+ 2 1 1.00 90.00 180.00
+ 4500 8280 4725 8100 6435 8100 6750 7470 7200 7470
+2 1 0 3 0 7 54 -1 -1 0.000 1 0 -1 1 0 5
+ 2 1 1.00 90.00 180.00
+ 4455 8055 4635 7740 6390 7740 6750 6750 7200 6750
+2 1 0 4 279 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 7234 7838 9776 7838 9776 8278 7234 8278
+2 1 0 2 0 7 54 -1 -1 0.000 1 0 -1 0 0 2
+ 10080 6525 9990 6705
+2 2 0 3 43 47 100 -1 20 0.000 1 0 -1 0 0 5
+ 1935 5985 11070 5985 11070 9585 1935 9585 1935 5985
+4 1 0 50 -1 4 9 1.5708 4 135 315 12240 3060 past\001
+4 1 0 50 -1 4 9 1.5708 4 120 465 10440 3060 future\001
+4 1 0 50 -1 4 9 1.5708 4 135 315 8550 3060 past\001
+4 1 0 50 -1 4 9 1.5708 4 120 465 6750 3060 future\001
+4 1 0 50 -1 6 10 0.0000 4 120 600 9450 5130 Oldest\001
+4 1 0 50 -1 4 9 1.5708 4 105 540 405 3060 newest\001
+4 1 0 50 -1 4 9 1.5708 4 120 450 2205 3060 oldest\001
+4 1 0 50 -1 4 9 1.5708 4 105 540 4095 3060 newest\001
+4 1 0 50 -1 4 9 1.5708 4 120 450 5895 3060 oldest\001
+4 0 0 50 -1 14 10 0.0000 4 135 1470 9135 5850 runqueue-depth\001
+4 0 0 50 -1 14 10 0.0000 4 135 1470 3195 5715 runqueue-depth\001
+4 1 0 50 -1 6 12 0.0000 4 165 1320 9450 3600 Time-based\001
+4 1 0 50 -1 6 12 0.0000 4 195 1395 9450 3780 Wait queues\001
+4 0 0 50 -1 6 12 0.0000 4 195 1050 9000 4005 - 1 global\001
+4 0 0 50 -1 6 12 0.0000 4 195 1605 9000 4185 - 1 per thread\001
+4 1 0 50 -1 6 12 0.0000 4 195 1650 3105 3600 Priority-based\001
+4 1 0 50 -1 6 12 0.0000 4 180 1365 3105 3780 Run queues\001
+4 0 0 50 -1 6 12 0.0000 4 195 1050 2655 4005 - 1 global\001
+4 0 0 50 -1 6 12 0.0000 4 195 1605 2655 4185 - 1 per thread\001
+4 0 0 50 -1 14 10 0.0000 4 135 1365 3240 585 task_wakeup()\001
+4 0 0 50 -1 14 10 0.0000 4 135 1575 9585 630 task_schedule()\001
+4 0 0 50 -1 14 10 0.0000 4 135 1260 9585 450 task_queue()\001
+4 0 0 50 -1 14 10 0.0000 4 135 1680 315 7560 tasklet_wakeup()\001
+4 2 0 50 -1 14 10 0.0000 4 135 1260 12285 7515 t->process()\001
+4 2 4 50 -1 6 12 0.0000 4 150 525 12285 7335 Run!\001
+4 1 0 46 -1 4 16 0.0000 4 210 1695 8460 7425 TL_NORMAL\001
+4 1 0 46 -1 4 16 0.0000 4 210 1200 8460 8145 TL_BULK\001
+4 1 0 46 -1 4 16 0.0000 4 210 1425 8460 8865 TL_HEAVY\001
+4 1 0 46 -1 4 16 0.0000 4 195 1095 4950 6705 SHARED\001
+4 0 0 50 -1 6 9 0.0000 4 105 345 10035 7515 37%\001
+4 0 0 50 -1 6 9 0.0000 4 105 210 10080 8955 =1\001
+4 1 0 50 -1 4 10 0.0000 4 150 2280 5085 6255 (accessed using atomic ops)\001
+4 0 0 50 -1 6 9 0.0000 4 105 345 10035 6795 50%\001
+4 0 0 50 -1 6 9 0.0000 4 105 345 10035 8235 13%\001
+4 2 0 50 -1 6 9 1.5708 4 105 315 2745 8100 Yes\001
+4 1 0 50 -1 6 10 1.5708 4 120 615 2520 7650 Local ?\001
+4 0 0 50 -1 6 9 1.5708 4 105 225 2700 7110 No\001
+4 0 0 50 -1 14 10 0.0000 4 135 1680 4725 8460 TASK_SELF_WAKING\001
+4 0 0 50 -1 14 10 0.0000 4 135 1050 4725 8820 TASK_HEAVY\001
+4 0 0 50 -1 4 10 0.0000 4 165 675 4725 8010 (default)\001
+4 0 0 50 -1 4 10 0.0000 4 150 1290 4725 7650 In I/O or signals\001
+4 1 0 50 -1 6 10 1.5708 4 150 1125 10815 7695 Most Urgent\001
+4 0 4 50 -1 6 10 0.0000 4 120 480 9990 6480 order\001
+4 0 4 50 -1 6 10 0.0000 4 120 420 9990 6300 Scan\001
+4 1 0 50 -1 6 12 0.0000 4 195 9075 6030 9450 5 class-based tasklet queues per thread (one accessible from remote threads)\001
diff --git a/doc/internals/sched.pdf b/doc/internals/sched.pdf
new file mode 100644
index 0000000..d1ce3de
--- /dev/null
+++ b/doc/internals/sched.pdf
Binary files differ
diff --git a/doc/internals/sched.png b/doc/internals/sched.png
new file mode 100644
index 0000000..65c97a1
--- /dev/null
+++ b/doc/internals/sched.png
Binary files differ
diff --git a/doc/internals/sched.svg b/doc/internals/sched.svg
new file mode 100644
index 0000000..0fa329a
--- /dev/null
+++ b/doc/internals/sched.svg
@@ -0,0 +1,1204 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Creator: fig2dev Version 3.2.7b -->
+<!-- CreationDate: 2021-02-26 17:49:00 -->
+<!-- Magnification: 1.57 -->
+<svg xmlns="http://www.w3.org/2000/svg"
+ xmlns:xlink="http://www.w3.org/1999/xlink"
+ width="1146pt" height="878pt"
+ viewBox="237 327 12126 9291">
+<g fill="none">
+<!-- Line -->
+<rect x="6570" y="720" width="5760" height="4680" fill="#ffebac"
+ stroke="#cbb366" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Line -->
+<rect x="270" y="720" width="5760" height="4680" fill="#c5ebe1"
+ stroke="#86c8a2" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Line -->
+<rect x="1935" y="5985" width="9135" height="3600" fill="#dfe3df"
+ stroke="#868286" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp0">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 9960,9130 9900,9190 9840,9130 9867,9483 9933,9483z"/>
+</clipPath>
+</defs>
+<polyline points=" 9900,6165 9900,9450" clip-path="url(#cp0)"
+ stroke="#ff0000" stroke-width="60px" stroke-linejoin="round" stroke-dasharray="120 120"/>
+<!-- Forward arrow to point 9900,9450 -->
+<polygon points=" 9840,9130 9900,9430 9960,9130 9900,9190 9840,9130"
+ stroke="#ff0000" stroke-width="8px" stroke-miterlimit="8" fill="#ff0000"/>
+<!-- Line -->
+<polyline points=" 9180,5535 9000,5805"
+ stroke="#b000b0" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp1">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 6880,7140 6940,7200 6880,7260 7233,7233 7233,7167z"/>
+</clipPath>
+</defs>
+<polyline points=" 3105,5220 3105,5850 3105,7200 7200,7200" clip-path="url(#cp1)"
+ stroke="#00b000" stroke-width="60px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 7200,7200 -->
+<polygon points=" 6880,7260 7180,7200 6880,7140 6940,7200 6880,7260"
+ stroke="#00b000" stroke-width="8px" stroke-miterlimit="8" fill="#00b000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp2">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 4009,1236 3950,1174 4011,1116 3658,1136 3656,1202z"/>
+</clipPath>
+</defs>
+<polyline points=" 9450,5220 9450,5670 6300,5670 6300,1215 3690,1170" clip-path="url(#cp2)"
+ stroke="#b000b0" stroke-width="60px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 3690,1170 -->
+<polygon points=" 4011,1116 3710,1170 4009,1236 3950,1174 4011,1116"
+ stroke="#b000b0" stroke-width="8px" stroke-miterlimit="8" fill="#b000b0"/>
+<!-- Line -->
+<polyline points=" 3195,5535 3015,5805"
+ stroke="#00b000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp3">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 11965,7635 12025,7695 11965,7755 12318,7728 12318,7662z"/>
+</clipPath>
+</defs>
+<polyline points=" 10890,7695 12285,7695" clip-path="url(#cp3)"
+ stroke="#ff0000" stroke-width="60px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 12285,7695 -->
+<polygon points=" 11965,7755 12265,7695 11965,7635 12025,7695 11965,7755"
+ stroke="#ff0000" stroke-width="8px" stroke-miterlimit="8" fill="#ff0000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp4">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 6955,8865 7000,8910 6955,8955 7218,8928 7218,8892z"/>
+</clipPath>
+</defs>
+<polyline points=" 4455,8775 4725,8910 7200,8910" clip-path="url(#cp4)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 7200,8910 -->
+<polygon points=" 6955,8955 7180,8910 6955,8865 7000,8910 6955,8955"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp5">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 3895,6615 3940,6660 3895,6705 4158,6678 4158,6642z"/>
+</clipPath>
+</defs>
+<polyline points=" 2610,7515 2925,6660 3645,6660 4140,6660" clip-path="url(#cp5)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 4140,6660 -->
+<polygon points=" 3895,6705 4120,6660 3895,6615 3940,6660 3895,6705"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp6">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 10496,8462 10558,8449 10572,8511 10690,8274 10659,8255z"/>
+</clipPath>
+</defs>
+<polyline points=" 9765,8775 10350,8775 10665,8280" clip-path="url(#cp6)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 10665,8280 -->
+<polygon points=" 10572,8511 10654,8297 10496,8462 10558,8449 10572,8511"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp7">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 10385,7957 10446,7974 10430,8036 10645,7882 10627,7850z"/>
+</clipPath>
+</defs>
+<polyline points=" 9765,8055 10305,8055 10620,7875" clip-path="url(#cp7)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 10620,7875 -->
+<polygon points=" 10430,8036 10603,7885 10385,7957 10446,7974 10430,8036"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp8">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 10580,6921 10564,6982 10503,6966 10659,7180 10690,7161z"/>
+</clipPath>
+</defs>
+<polyline points=" 9806,6605 10350,6615 10665,7155" clip-path="url(#cp8)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 10665,7155 -->
+<polygon points=" 10503,6966 10655,7138 10580,6921 10564,6982 10503,6966"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp9">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 10461,7369 10466,7432 10403,7438 10622,7585 10645,7558z"/>
+</clipPath>
+</defs>
+<polyline points=" 9720,7335 10350,7335 10620,7560" clip-path="url(#cp9)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 10620,7560 -->
+<polygon points=" 10403,7438 10605,7547 10461,7369 10466,7432 10403,7438"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<polyline points=" 10080,7245 9990,7425"
+ stroke="#000000" stroke-width="15px" stroke-linejoin="round"/>
+<!-- Line -->
+<polyline points=" 10080,7965 9990,8145"
+ stroke="#000000" stroke-width="15px" stroke-linejoin="round"/>
+<!-- Line -->
+<polyline points=" 10080,8685 9990,8865"
+ stroke="#000000" stroke-width="15px" stroke-linejoin="round"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp10">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 6955,8145 7000,8190 6955,8235 7218,8208 7218,8172z"/>
+</clipPath>
+</defs>
+<polyline points=" 4500,8550 6255,8550 6705,8190 7200,8190" clip-path="url(#cp10)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 7200,8190 -->
+<polygon points=" 6955,8235 7180,8190 6955,8145 7000,8190 6955,8235"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp11">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 6955,7425 7000,7470 6955,7515 7218,7488 7218,7452z"/>
+</clipPath>
+</defs>
+<polyline points=" 4500,8280 4725,8100 6435,8100 6750,7470 7200,7470" clip-path="url(#cp11)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 7200,7470 -->
+<polygon points=" 6955,7515 7180,7470 6955,7425 7000,7470 6955,7515"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp12">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 6955,6705 7000,6750 6955,6795 7218,6768 7218,6732z"/>
+</clipPath>
+</defs>
+<polyline points=" 4455,8055 4635,7740 6390,7740 6750,6750 7200,6750" clip-path="url(#cp12)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 7200,6750 -->
+<polygon points=" 6955,6795 7180,6750 6955,6705 7000,6750 6955,6795"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<polyline points=" 10080,6525 9990,6705"
+ stroke="#000000" stroke-width="15px" stroke-linejoin="round"/>
+<!-- Ellipse -->
+<ellipse cx="3105" cy="5049" rx="810" ry="171" fill="#87cfff"
+ stroke="#00008f" stroke-width="30px"/>
+<!-- Ellipse -->
+<ellipse cx="3083" cy="1180" rx="607" ry="170" fill="#ffbfbf"
+ stroke="#d10000" stroke-width="30px"/>
+<!-- Ellipse -->
+<ellipse cx="9428" cy="1180" rx="607" ry="170" fill="#ffbfbf"
+ stroke="#d10000" stroke-width="30px"/>
+<!-- Ellipse -->
+<ellipse transform="translate(4330,8437) rotate(-90)" rx="607" ry="170" fill="#ffbfbf"
+ stroke="#d10000" stroke-width="30px"/>
+<!-- Ellipse -->
+<ellipse cx="9450" cy="5049" rx="540" ry="171" fill="#87cfff"
+ stroke="#00008f" stroke-width="30px"/>
+<!-- Ellipse -->
+<ellipse transform="translate(2440,7672) rotate(-90)" rx="607" ry="170" fill="#ffbfbf"
+ stroke="#d10000" stroke-width="30px"/>
+<!-- Ellipse -->
+<ellipse transform="translate(10755,7695) rotate(-90)" rx="810" ry="171" fill="#87cfff"
+ stroke="#00008f" stroke-width="30px"/>
+<!-- Line -->
+<polygon points=" 11768,3060 11970,3060 11967,3119 11959,3177 11946,3234 11929,3291 11907,3345
+ 11879,3397 11704,3296 11723,3259 11738,3222 11751,3182 11760,3142 11765,3101
+" fill="#ffd600"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 11704,3296 11879,3397 11848,3447 11812,3494 11772,3537 11729,3577 11682,3613
+ 11633,3644 11531,3469 11566,3447 11599,3422 11628,3393 11657,3364 11682,3331
+" fill="#ffd600"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 11531,3469 11633,3644 11580,3672 11526,3694 11469,3711 11412,3724 11354,3732
+ 11295,3734 11295,3532 11336,3530 11377,3525 11417,3516 11457,3503 11494,3488
+" fill="#ffd600"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 11295,3532 11295,3734 11236,3732 11178,3724 11121,3711 11064,3694 11010,3672
+ 10958,3644 11059,3469 11096,3488 11133,3503 11173,3516 11213,3525 11254,3530
+" fill="#ffd600"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 11059,3469 10958,3644 10908,3613 10861,3577 10818,3537 10778,3494 10742,3447
+ 10711,3398 10886,3296 10908,3331 10933,3364 10962,3393 10991,3422 11024,3447
+" fill="#ffd600"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 10886,3296 10711,3398 10683,3345 10661,3291 10644,3234 10631,3177 10623,3119
+ 10621,3060 10823,3060 10825,3101 10830,3142 10839,3182 10852,3222 10867,3259
+" fill="#ffd600"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 10823,3060 10621,3060 10623,3001 10631,2943 10644,2886 10661,2829 10683,2775
+ 10711,2723 10886,2824 10867,2861 10852,2898 10839,2938 10830,2978 10825,3019
+" fill="#ffd600"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 10886,2824 10711,2723 10742,2673 10778,2626 10818,2583 10861,2543 10908,2507
+ 10958,2476 11059,2651 11024,2673 10991,2698 10962,2727 10933,2756 10908,2789
+" fill="#ffd600"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 11059,2651 10958,2476 11010,2448 11064,2426 11121,2409 11178,2396 11236,2388
+ 11295,2386 11295,2588 11254,2590 11213,2595 11173,2604 11133,2617 11096,2632
+" fill="#ffd600"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 11295,2588 11295,2386 11354,2388 11412,2396 11469,2409 11526,2426 11580,2448
+ 11632,2476 11531,2651 11494,2632 11457,2617 11417,2604 11377,2595 11336,2590
+" fill="#ffd600"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 11531,2651 11632,2476 11682,2507 11729,2543 11772,2583 11812,2626 11848,2673
+ 11879,2723 11704,2824 11682,2789 11657,2756 11628,2727 11599,2698 11566,2673
+" fill="#ffd600"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 11704,2824 11879,2723 11907,2775 11929,2829 11946,2886 11959,2943 11967,3001
+ 11969,3060 11767,3060 11765,3019 11760,2978 11751,2938 11738,2898 11723,2861
+" fill="#ffd600"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 5423,3060 5625,3060 5622,3119 5614,3177 5601,3234 5584,3291 5562,3345 5534,3397
+ 5359,3296 5378,3259 5393,3222 5406,3182 5415,3142 5420,3101" fill="#00b000"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 5359,3296 5534,3397 5503,3447 5467,3494 5427,3537 5384,3577 5337,3613 5288,3644
+ 5186,3469 5221,3447 5254,3422 5283,3393 5312,3364 5337,3331" fill="#00b000"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 5186,3469 5288,3644 5235,3672 5181,3694 5124,3711 5067,3724 5009,3732 4950,3734
+ 4950,3532 4991,3530 5032,3525 5072,3516 5112,3503 5149,3488" fill="#00b000"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 4950,3532 4950,3734 4891,3732 4833,3724 4776,3711 4719,3694 4665,3672 4613,3644
+ 4714,3469 4751,3488 4788,3503 4828,3516 4868,3525 4909,3530" fill="#00b000"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 4714,3469 4613,3644 4563,3613 4516,3577 4473,3537 4433,3494 4397,3447 4366,3398
+ 4541,3296 4563,3331 4588,3364 4617,3393 4646,3422 4679,3447" fill="#00b000"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 4541,3296 4366,3398 4338,3345 4316,3291 4299,3234 4286,3177 4278,3119 4276,3060
+ 4478,3060 4480,3101 4485,3142 4494,3182 4507,3222 4522,3259" fill="#00b000"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 4478,3060 4276,3060 4278,3001 4286,2943 4299,2886 4316,2829 4338,2775 4366,2723
+ 4541,2824 4522,2861 4507,2898 4494,2938 4485,2978 4480,3019" fill="#00b000"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 4541,2824 4366,2723 4397,2673 4433,2626 4473,2583 4516,2543 4563,2507 4613,2476
+ 4714,2651 4679,2673 4646,2698 4617,2727 4588,2756 4563,2789" fill="#00b000"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 4714,2651 4613,2476 4665,2448 4719,2426 4776,2409 4833,2396 4891,2388 4950,2386
+ 4950,2588 4909,2590 4868,2595 4828,2604 4788,2617 4751,2632" fill="#00b000"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 4950,2588 4950,2386 5009,2388 5067,2396 5124,2409 5181,2426 5235,2448 5287,2476
+ 5186,2651 5149,2632 5112,2617 5072,2604 5032,2595 4991,2590" fill="#00b000"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 5186,2651 5287,2476 5337,2507 5384,2543 5427,2583 5467,2626 5503,2673 5534,2723
+ 5359,2824 5337,2789 5312,2756 5283,2727 5254,2698 5221,2673" fill="#00b000"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 5359,2824 5534,2723 5562,2775 5584,2829 5601,2886 5614,2943 5622,3001 5624,3060
+ 5422,3060 5420,3019 5415,2978 5406,2938 5393,2898 5378,2861" fill="#00b000"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Text -->
+<text xml:space="preserve" x="3105" y="5130" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">Most Urgent</text>
+<!-- Line -->
+<defs>
+<polygon points=" 8078,3060 8280,3060 8277,3119 8269,3177 8256,3234 8239,3291 8217,3345 8189,3397
+ 8014,3296 8033,3259 8048,3222 8061,3182 8070,3142 8075,3101" id="p0"/>
+<pattern id="tile0" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p0" fill="#ffd600"/>
+<use xlink:href="#p0" fill="url(#tile0)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 8014,3296 8189,3397 8158,3447 8122,3494 8082,3537 8039,3577 7992,3613 7943,3644
+ 7841,3469 7876,3447 7909,3422 7938,3393 7967,3364 7992,3331" id="p1"/>
+<pattern id="tile1" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p1" fill="#ffd600"/>
+<use xlink:href="#p1" fill="url(#tile1)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 7841,3469 7943,3644 7890,3672 7836,3694 7779,3711 7722,3724 7664,3732 7605,3734
+ 7605,3532 7646,3530 7687,3525 7727,3516 7767,3503 7804,3488" id="p2"/>
+<pattern id="tile2" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p2" fill="#ffd600"/>
+<use xlink:href="#p2" fill="url(#tile2)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 7605,3532 7605,3734 7546,3732 7488,3724 7431,3711 7374,3694 7320,3672 7268,3644
+ 7369,3469 7406,3488 7443,3503 7483,3516 7523,3525 7564,3530" id="p3"/>
+<pattern id="tile3" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p3" fill="#ffd600"/>
+<use xlink:href="#p3" fill="url(#tile3)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 7369,3469 7268,3644 7218,3613 7171,3577 7128,3537 7088,3494 7052,3447 7021,3398
+ 7196,3296 7218,3331 7243,3364 7272,3393 7301,3422 7334,3447" id="p4"/>
+<pattern id="tile4" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p4" fill="#ffd600"/>
+<use xlink:href="#p4" fill="url(#tile4)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 7196,3296 7021,3398 6993,3345 6971,3291 6954,3234 6941,3177 6933,3119 6931,3060
+ 7133,3060 7135,3101 7140,3142 7149,3182 7162,3222 7177,3259" id="p5"/>
+<pattern id="tile5" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p5" fill="#ffd600"/>
+<use xlink:href="#p5" fill="url(#tile5)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 7133,3060 6931,3060 6933,3001 6941,2943 6954,2886 6971,2829 6993,2775 7021,2723
+ 7196,2824 7177,2861 7162,2898 7149,2938 7140,2978 7135,3019" id="p6"/>
+<pattern id="tile6" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p6" fill="#ffd600"/>
+<use xlink:href="#p6" fill="url(#tile6)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 7196,2824 7021,2723 7052,2673 7088,2626 7128,2583 7171,2543 7218,2507 7268,2476
+ 7369,2651 7334,2673 7301,2698 7272,2727 7243,2756 7218,2789" id="p7"/>
+<pattern id="tile7" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p7" fill="#ffd600"/>
+<use xlink:href="#p7" fill="url(#tile7)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 7369,2651 7268,2476 7320,2448 7374,2426 7431,2409 7488,2396 7546,2388 7605,2386
+ 7605,2588 7564,2590 7523,2595 7483,2604 7443,2617 7406,2632" id="p8"/>
+<pattern id="tile8" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p8" fill="#ffd600"/>
+<use xlink:href="#p8" fill="url(#tile8)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 7605,2588 7605,2386 7664,2388 7722,2396 7779,2409 7836,2426 7890,2448 7942,2476
+ 7841,2651 7804,2632 7767,2617 7727,2604 7687,2595 7646,2590" id="p9"/>
+<pattern id="tile9" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p9" fill="#ffd600"/>
+<use xlink:href="#p9" fill="url(#tile9)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 7841,2651 7942,2476 7992,2507 8039,2543 8082,2583 8122,2626 8158,2673 8189,2723
+ 8014,2824 7992,2789 7967,2756 7938,2727 7909,2698 7876,2673" id="p10"/>
+<pattern id="tile10" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p10" fill="#ffd600"/>
+<use xlink:href="#p10" fill="url(#tile10)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 8014,2824 8189,2723 8217,2775 8239,2829 8256,2886 8269,2943 8277,3001 8279,3060
+ 8077,3060 8075,3019 8070,2978 8061,2938 8048,2898 8033,2861" id="p11"/>
+<pattern id="tile11" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p11" fill="#ffd600"/>
+<use xlink:href="#p11" fill="url(#tile11)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 1733,3060 1935,3060 1932,3119 1924,3177 1911,3234 1894,3291 1872,3345 1844,3397
+ 1669,3296 1688,3259 1703,3222 1716,3182 1725,3142 1730,3101" id="p12"/>
+<pattern id="tile12" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p12" fill="#00b000"/>
+<use xlink:href="#p12" fill="url(#tile12)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 1669,3296 1844,3397 1813,3447 1777,3494 1737,3537 1694,3577 1647,3613 1598,3644
+ 1496,3469 1531,3447 1564,3422 1593,3393 1622,3364 1647,3331" id="p13"/>
+<pattern id="tile13" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p13" fill="#00b000"/>
+<use xlink:href="#p13" fill="url(#tile13)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 1496,3469 1598,3644 1545,3672 1491,3694 1434,3711 1377,3724 1319,3732 1260,3734
+ 1260,3532 1301,3530 1342,3525 1382,3516 1422,3503 1459,3488" id="p14"/>
+<pattern id="tile14" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p14" fill="#00b000"/>
+<use xlink:href="#p14" fill="url(#tile14)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 1260,3532 1260,3734 1201,3732 1143,3724 1086,3711 1029,3694 975,3672 923,3644
+ 1024,3469 1061,3488 1098,3503 1138,3516 1178,3525 1219,3530" id="p15"/>
+<pattern id="tile15" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p15" fill="#00b000"/>
+<use xlink:href="#p15" fill="url(#tile15)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 1024,3469 923,3644 873,3613 826,3577 783,3537 743,3494 707,3447 676,3398 851,3296
+ 873,3331 898,3364 927,3393 956,3422 989,3447" id="p16"/>
+<pattern id="tile16" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p16" fill="#00b000"/>
+<use xlink:href="#p16" fill="url(#tile16)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 851,3296 676,3398 648,3345 626,3291 609,3234 596,3177 588,3119 586,3060 788,3060
+ 790,3101 795,3142 804,3182 817,3222 832,3259" id="p17"/>
+<pattern id="tile17" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p17" fill="#00b000"/>
+<use xlink:href="#p17" fill="url(#tile17)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 788,3060 586,3060 588,3001 596,2943 609,2886 626,2829 648,2775 676,2723 851,2824
+ 832,2861 817,2898 804,2938 795,2978 790,3019" id="p18"/>
+<pattern id="tile18" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p18" fill="#00b000"/>
+<use xlink:href="#p18" fill="url(#tile18)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 851,2824 676,2723 707,2673 743,2626 783,2583 826,2543 873,2507 923,2476 1024,2651
+ 989,2673 956,2698 927,2727 898,2756 873,2789" id="p19"/>
+<pattern id="tile19" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p19" fill="#00b000"/>
+<use xlink:href="#p19" fill="url(#tile19)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 1024,2651 923,2476 975,2448 1029,2426 1086,2409 1143,2396 1201,2388 1260,2386
+ 1260,2588 1219,2590 1178,2595 1138,2604 1098,2617 1061,2632" id="p20"/>
+<pattern id="tile20" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p20" fill="#00b000"/>
+<use xlink:href="#p20" fill="url(#tile20)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 1260,2588 1260,2386 1319,2388 1377,2396 1434,2409 1491,2426 1545,2448 1597,2476
+ 1496,2651 1459,2632 1422,2617 1382,2604 1342,2595 1301,2590" id="p21"/>
+<pattern id="tile21" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p21" fill="#00b000"/>
+<use xlink:href="#p21" fill="url(#tile21)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 1496,2651 1597,2476 1647,2507 1694,2543 1737,2583 1777,2626 1813,2673 1844,2723
+ 1669,2824 1647,2789 1622,2756 1593,2727 1564,2698 1531,2673" id="p22"/>
+<pattern id="tile22" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p22" fill="#00b000"/>
+<use xlink:href="#p22" fill="url(#tile22)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 1669,2824 1844,2723 1872,2775 1894,2829 1911,2886 1924,2943 1932,3001 1934,3060
+ 1732,3060 1730,3019 1725,2978 1716,2938 1703,2898 1688,2861" id="p23"/>
+<pattern id="tile23" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p23" fill="#00b000"/>
+<use xlink:href="#p23" fill="url(#tile23)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Text -->
+<text xml:space="preserve" x="2160" y="1980" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">Global</text>
+<!-- Text -->
+<text xml:space="preserve" x="2160" y="2160" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">tasks</text>
+<!-- Text -->
+<text xml:space="preserve" x="2160" y="2340" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">(locked)</text>
+<!-- Text -->
+<text xml:space="preserve" x="4230" y="2250" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">tasks</text>
+<!-- Text -->
+<text xml:space="preserve" x="4230" y="2070" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">Local</text>
+<!-- Text -->
+<text xml:space="preserve" x="8550" y="2340" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">(locked)</text>
+<!-- Text -->
+<text xml:space="preserve" x="8550" y="2160" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">timers</text>
+<!-- Text -->
+<text xml:space="preserve" x="8550" y="1980" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">Global</text>
+<!-- Text -->
+<text xml:space="preserve" x="10530" y="2250" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">timers</text>
+<!-- Text -->
+<text xml:space="preserve" x="10530" y="2070" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">Local</text>
+<!-- Text -->
+<text xml:space="preserve" x="3105" y="1260" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">Local ?</text>
+<!-- Text -->
+<text xml:space="preserve" x="3375" y="1530" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="108" text-anchor="start">Yes</text>
+<!-- Text -->
+<text xml:space="preserve" x="2790" y="1530" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="108" text-anchor="end">No</text>
+<!-- Text -->
+<text xml:space="preserve" x="9450" y="1260" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">Local ?</text>
+<!-- Text -->
+<text xml:space="preserve" x="9720" y="1530" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="108" text-anchor="start">Yes</text>
+<!-- Text -->
+<text xml:space="preserve" x="9135" y="1530" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="108" text-anchor="end">No</text>
+<!-- Text -->
+<g transform="translate(4410,8415) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">Class?</text>
+</g><!-- Arc -->
+<defs>
+<clipPath id="cp13">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 11065,2294 11189,2298 11078,2353 11217,2303 11213,2283z"/>
+</clipPath>
+</defs>
+<path d="M 11205,3825 A 771 771 0 0 1 11205 2295" clip-path="url(#cp13)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Forward arrow to point 11205,2295 -->
+<polyline points=" 11065,2294 11189,2298 11078,2353"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8"/>
+<!-- Arc -->
+<defs>
+<clipPath id="cp14">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 11065,2294 11189,2298 11078,2353 11217,2303 11213,2283z
+ M 11525,3826 11401,3822 11512,3767 11373,3817 11377,3837z"/>
+</clipPath>
+</defs>
+<path d="M 11385,3825 A 771 771 0 0 0 11385 2295" clip-path="url(#cp14)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Backward arrow to point 11385,3825 -->
+<polyline points=" 11525,3826 11401,3822 11512,3767"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8"/>
+<!-- Arc -->
+<defs>
+<clipPath id="cp15">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 11065,2294 11189,2298 11078,2353 11217,2303 11213,2283z
+ M 10908,3277 10928,3240 10965,3260 10897,3092 10877,3098z"/>
+</clipPath>
+</defs>
+<path d="M 10890,3105 A 406 406 0 1 0 10890 3015" clip-path="url(#cp15)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Backward arrow to point 10890,3105 -->
+<polygon points=" 10965,3260 10895,3124 10908,3277 10928,3240 10965,3260"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Arc -->
+<defs>
+<clipPath id="cp16">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 7375,2294 7499,2298 7388,2353 7527,2303 7523,2283z
+ M 10908,3277 10928,3240 10965,3260 10897,3092 10877,3098z"/>
+</clipPath>
+</defs>
+<path d="M 7515,3825 A 771 771 0 0 1 7515 2295" clip-path="url(#cp16)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Forward arrow to point 7515,2295 -->
+<polyline points=" 7375,2294 7499,2298 7388,2353"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8"/>
+<!-- Arc -->
+<defs>
+<clipPath id="cp17">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 7375,2294 7499,2298 7388,2353 7527,2303 7523,2283z
+ M 7835,3826 7711,3822 7822,3767 7683,3817 7687,3837z"/>
+</clipPath>
+</defs>
+<path d="M 7695,3825 A 771 771 0 0 0 7695 2295" clip-path="url(#cp17)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Backward arrow to point 7695,3825 -->
+<polyline points=" 7835,3826 7711,3822 7822,3767"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8"/>
+<!-- Arc -->
+<defs>
+<clipPath id="cp18">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 7375,2294 7499,2298 7388,2353 7527,2303 7523,2283z
+ M 7218,3277 7238,3240 7275,3260 7207,3092 7187,3098z"/>
+</clipPath>
+</defs>
+<path d="M 7200,3105 A 406 406 0 1 0 7200 3015" clip-path="url(#cp18)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Backward arrow to point 7200,3105 -->
+<polygon points=" 7275,3260 7205,3124 7218,3277 7238,3240 7275,3260"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Arc -->
+<defs>
+<clipPath id="cp19">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 4720,2294 4844,2298 4733,2353 4872,2303 4868,2283z
+ M 7218,3277 7238,3240 7275,3260 7207,3092 7187,3098z"/>
+</clipPath>
+</defs>
+<path d="M 4860,3825 A 771 771 0 0 1 4860 2295" clip-path="url(#cp19)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Forward arrow to point 4860,2295 -->
+<polyline points=" 4720,2294 4844,2298 4733,2353"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8"/>
+<!-- Arc -->
+<defs>
+<clipPath id="cp20">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 4720,2294 4844,2298 4733,2353 4872,2303 4868,2283z
+ M 5180,3826 5056,3822 5167,3767 5028,3817 5032,3837z"/>
+</clipPath>
+</defs>
+<path d="M 5040,3825 A 771 771 0 0 0 5040 2295" clip-path="url(#cp20)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Backward arrow to point 5040,3825 -->
+<polyline points=" 5180,3826 5056,3822 5167,3767"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8"/>
+<!-- Arc -->
+<defs>
+<clipPath id="cp21">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 4720,2294 4844,2298 4733,2353 4872,2303 4868,2283z
+ M 4563,3277 4583,3240 4620,3260 4552,3092 4532,3098z"/>
+</clipPath>
+</defs>
+<path d="M 4545,3105 A 406 406 0 1 0 4545 3015" clip-path="url(#cp21)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Backward arrow to point 4545,3105 -->
+<polygon points=" 4620,3260 4550,3124 4563,3277 4583,3240 4620,3260"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Arc -->
+<defs>
+<clipPath id="cp22">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 1030,2294 1154,2298 1043,2353 1182,2303 1178,2283z
+ M 4563,3277 4583,3240 4620,3260 4552,3092 4532,3098z"/>
+</clipPath>
+</defs>
+<path d="M 1170,3825 A 771 771 0 0 1 1170 2295" clip-path="url(#cp22)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Forward arrow to point 1170,2295 -->
+<polyline points=" 1030,2294 1154,2298 1043,2353"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8"/>
+<!-- Arc -->
+<defs>
+<clipPath id="cp23">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 1030,2294 1154,2298 1043,2353 1182,2303 1178,2283z
+ M 1490,3826 1366,3822 1477,3767 1338,3817 1342,3837z"/>
+</clipPath>
+</defs>
+<path d="M 1350,3825 A 771 771 0 0 0 1350 2295" clip-path="url(#cp23)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Backward arrow to point 1350,3825 -->
+<polyline points=" 1490,3826 1366,3822 1477,3767"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8"/>
+<!-- Arc -->
+<defs>
+<clipPath id="cp24">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 1030,2294 1154,2298 1043,2353 1182,2303 1178,2283z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<path d="M 855,3105 A 406 406 0 1 0 855 3015" clip-path="url(#cp24)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Backward arrow to point 855,3105 -->
+<polygon points=" 930,3260 860,3124 873,3277 893,3240 930,3260"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp25">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 9315,4660 9270,4705 9225,4660 9252,4923 9288,4923z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 7605,3870 7605,4185 9270,4545 9270,4905" clip-path="url(#cp25)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 9270,4905 -->
+<polygon points=" 9225,4660 9270,4885 9315,4660 9270,4705 9225,4660"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp26">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 9681,4660 9636,4705 9591,4660 9618,4923 9654,4923z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 11301,3870 11301,4185 9636,4545 9636,4905" clip-path="url(#cp26)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 9636,4905 -->
+<polygon points=" 9591,4660 9636,4885 9681,4660 9636,4705 9591,4660"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp27">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 11338,2050 11293,2095 11248,2051 11277,2313 11313,2313z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 9630,1395 9626,1591 11291,1800 11295,2295" clip-path="url(#cp27)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 11295,2295 -->
+<polygon points=" 11248,2051 11295,2275 11338,2050 11293,2095 11248,2051"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp28">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 7650,2050 7605,2095 7560,2050 7587,2313 7623,2313z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 9270,1395 9270,1575 7605,1800 7605,2295" clip-path="url(#cp28)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 7605,2295 -->
+<polygon points=" 7560,2050 7605,2275 7650,2050 7605,2095 7560,2050"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp29">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 9495,790 9450,835 9405,790 9432,1053 9468,1053z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 9450,360 9450,1035" clip-path="url(#cp29)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 9450,1035 -->
+<polygon points=" 9405,790 9450,1015 9495,790 9450,835 9405,790"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp30">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 2970,4660 2925,4705 2880,4660 2907,4923 2943,4923z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 1260,3870 1260,4185 2925,4545 2925,4905" clip-path="url(#cp30)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 2925,4905 -->
+<polygon points=" 2880,4660 2925,4885 2970,4660 2925,4705 2880,4660"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp31">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 3336,4660 3291,4705 3246,4660 3273,4923 3309,4923z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 4956,3870 4956,4185 3291,4545 3291,4905" clip-path="url(#cp31)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 3291,4905 -->
+<polygon points=" 3246,4660 3291,4885 3336,4660 3291,4705 3246,4660"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp32">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 3150,790 3105,835 3060,790 3087,1053 3123,1053z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 3105,360 3105,1035" clip-path="url(#cp32)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 3105,1035 -->
+<polygon points=" 3060,790 3105,1015 3150,790 3105,835 3060,790"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp33">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 4995,2140 4950,2185 4905,2140 4932,2403 4968,2403z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 3285,1395 3285,1575 4950,1845 4950,2385" clip-path="url(#cp33)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 4950,2385 -->
+<polygon points=" 4905,2140 4950,2365 4995,2140 4950,2185 4905,2140"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp34">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 1305,2140 1260,2185 1215,2140 1242,2403 1278,2403z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 2925,1395 2925,1575 1260,1845 1260,2385" clip-path="url(#cp34)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 1260,2385 -->
+<polygon points=" 1215,2140 1260,2365 1305,2140 1260,2185 1215,2140"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp35">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 2005,7605 2050,7650 2005,7695 2268,7668 2268,7632z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 315,7650 2250,7650" clip-path="url(#cp35)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 2250,7650 -->
+<polygon points=" 2005,7695 2230,7650 2005,7605 2050,7650 2005,7695"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp36">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 6955,6435 7000,6480 6955,6525 7218,6498 7218,6462z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 6075,6480 7200,6480" clip-path="url(#cp36)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 7200,6480 -->
+<polygon points=" 6955,6525 7180,6480 6955,6435 7000,6480 6955,6525"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp37">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 3895,8370 3940,8415 3895,8460 4158,8433 4158,8397z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 2610,7830 3195,8415 4140,8415" clip-path="url(#cp37)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 4140,8415 -->
+<polygon points=" 3895,8460 4120,8415 3895,8370 3940,8415 3895,8460"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Text -->
+<g transform="translate(12240,3060) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="108" text-anchor="middle">past</text>
+</g><!-- Text -->
+<g transform="translate(10440,3060) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="108" text-anchor="middle">future</text>
+</g><!-- Text -->
+<g transform="translate(8550,3060) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="108" text-anchor="middle">past</text>
+</g><!-- Text -->
+<g transform="translate(6750,3060) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="108" text-anchor="middle">future</text>
+</g><!-- Text -->
+<text xml:space="preserve" x="9450" y="5130" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">Oldest</text>
+<!-- Text -->
+<g transform="translate(405,3060) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="108" text-anchor="middle">newest</text>
+</g><!-- Text -->
+<g transform="translate(2205,3060) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="108" text-anchor="middle">oldest</text>
+</g><!-- Text -->
+<g transform="translate(4095,3060) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="108" text-anchor="middle">newest</text>
+</g><!-- Text -->
+<g transform="translate(5895,3060) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="108" text-anchor="middle">oldest</text>
+</g><!-- Text -->
+<text xml:space="preserve" x="9135" y="5850" fill="#000000" font-family="Courier" font-style="normal" font-weight="bold" font-size="120" text-anchor="start">runqueue-depth</text>
+<!-- Text -->
+<text xml:space="preserve" x="3195" y="5715" fill="#000000" font-family="Courier" font-style="normal" font-weight="bold" font-size="120" text-anchor="start">runqueue-depth</text>
+<!-- Text -->
+<text xml:space="preserve" x="9450" y="3600" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="144" text-anchor="middle">Time-based</text>
+<!-- Text -->
+<text xml:space="preserve" x="9450" y="3780" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="144" text-anchor="middle">Wait queues</text>
+<!-- Text -->
+<text xml:space="preserve" x="9000" y="4005" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="144" text-anchor="start">- 1 global</text>
+<!-- Text -->
+<text xml:space="preserve" x="9000" y="4185" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="144" text-anchor="start">- 1 per thread</text>
+<!-- Text -->
+<text xml:space="preserve" x="3105" y="3600" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="144" text-anchor="middle">Priority-based</text>
+<!-- Text -->
+<text xml:space="preserve" x="3105" y="3780" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="144" text-anchor="middle">Run queues</text>
+<!-- Text -->
+<text xml:space="preserve" x="2655" y="4005" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="144" text-anchor="start">- 1 global</text>
+<!-- Text -->
+<text xml:space="preserve" x="2655" y="4185" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="144" text-anchor="start">- 1 per thread</text>
+<!-- Text -->
+<text xml:space="preserve" x="3240" y="585" fill="#000000" font-family="Courier" font-style="normal" font-weight="bold" font-size="120" text-anchor="start">task_wakeup()</text>
+<!-- Text -->
+<text xml:space="preserve" x="9585" y="630" fill="#000000" font-family="Courier" font-style="normal" font-weight="bold" font-size="120" text-anchor="start">task_schedule()</text>
+<!-- Text -->
+<text xml:space="preserve" x="9585" y="450" fill="#000000" font-family="Courier" font-style="normal" font-weight="bold" font-size="120" text-anchor="start">task_queue()</text>
+<!-- Text -->
+<text xml:space="preserve" x="315" y="7560" fill="#000000" font-family="Courier" font-style="normal" font-weight="bold" font-size="120" text-anchor="start">tasklet_wakeup()</text>
+<!-- Text -->
+<text xml:space="preserve" x="12285" y="7515" fill="#000000" font-family="Courier" font-style="normal" font-weight="bold" font-size="120" text-anchor="end">t-&gt;process()</text>
+<!-- Text -->
+<text xml:space="preserve" x="12285" y="7335" fill="#ff0000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="144" text-anchor="end">Run!</text>
+<!-- Text -->
+<text xml:space="preserve" x="10035" y="7515" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="108" text-anchor="start">37%</text>
+<!-- Text -->
+<text xml:space="preserve" x="10080" y="8955" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="108" text-anchor="start">=1</text>
+<!-- Text -->
+<text xml:space="preserve" x="5085" y="6255" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="120" text-anchor="middle">(accessed using atomic ops)</text>
+<!-- Text -->
+<text xml:space="preserve" x="10035" y="6795" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="108" text-anchor="start">50%</text>
+<!-- Text -->
+<text xml:space="preserve" x="10035" y="8235" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="108" text-anchor="start">13%</text>
+<!-- Text -->
+<g transform="translate(2745,8100) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="108" text-anchor="end">Yes</text>
+</g><!-- Text -->
+<g transform="translate(2520,7650) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">Local ?</text>
+</g><!-- Text -->
+<g transform="translate(2700,7110) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="108" text-anchor="start">No</text>
+</g><!-- Text -->
+<text xml:space="preserve" x="4725" y="8460" fill="#000000" font-family="Courier" font-style="normal" font-weight="bold" font-size="120" text-anchor="start">TASK_SELF_WAKING</text>
+<!-- Text -->
+<text xml:space="preserve" x="4725" y="8820" fill="#000000" font-family="Courier" font-style="normal" font-weight="bold" font-size="120" text-anchor="start">TASK_HEAVY</text>
+<!-- Text -->
+<text xml:space="preserve" x="4725" y="8010" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="120" text-anchor="start">(default)</text>
+<!-- Text -->
+<text xml:space="preserve" x="4725" y="7650" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="120" text-anchor="start">In I/O or signals</text>
+<!-- Text -->
+<g transform="translate(10815,7695) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">Most Urgent</text>
+</g><!-- Text -->
+<text xml:space="preserve" x="9990" y="6480" fill="#ff0000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="start">order</text>
+<!-- Text -->
+<text xml:space="preserve" x="9990" y="6300" fill="#ff0000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="start">Scan</text>
+<!-- Text -->
+<text xml:space="preserve" x="6030" y="9450" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="144" text-anchor="middle">5 class-based tasklet queues per thread (one accessible from remote threads)</text>
+<!-- Line -->
+<polygon points=" 7234,6838 9776,6838 9776,6398 7234,6398" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 7234,7558 9776,7558 9776,7118 7234,7118" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 7234,8278 9776,8278 9776,7838 7234,7838" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 7234,8998 9776,8998 9776,8558 7234,8558" fill="#dae8fc"/>
+<!-- Line -->
+<defs>
+<polygon points=" 4166,6838 6094,6838 6094,6398 4166,6398" id="p24"/>
+<pattern id="tile24" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#a7ceb3" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p24" fill="#bbf2e2"/>
+<use xlink:href="#p24" fill="url(#tile24)"/>
+<!-- Line -->
+<polyline points=" 7234,6398 9776,6398 9776,6838 7234,6838"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 7234,7118 9776,7118 9776,7558 7234,7558"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 7234,8558 9776,8558 9776,8998 7234,8998"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 4166,6398 6094,6398 6094,6838 4166,6838"
+ stroke="#868286" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 7234,7838 9776,7838 9776,8278 7234,8278"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 9613,6398 9613,6838"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9438,6398 9438,6838"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9264,6398 9264,6838"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9613,7118 9613,7558"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9438,7118 9438,7558"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9264,7118 9264,7558"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9613,7838 9613,8278"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9438,7838 9438,8278"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9264,7838 9264,8278"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9613,8558 9613,8998"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9438,8558 9438,8998"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9264,8558 9264,8998"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 5923,6398 5923,6838"
+ stroke="#868286" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 5748,6398 5748,6838"
+ stroke="#868286" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 5574,6398 5574,6838"
+ stroke="#868286" stroke-width="15px"/>
+<!-- Text -->
+<text xml:space="preserve" x="8460" y="6705" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="192" text-anchor="middle">TL_URGENT</text>
+<!-- Text -->
+<text xml:space="preserve" x="8460" y="7425" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="192" text-anchor="middle">TL_NORMAL</text>
+<!-- Text -->
+<text xml:space="preserve" x="8460" y="8145" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="192" text-anchor="middle">TL_BULK</text>
+<!-- Text -->
+<text xml:space="preserve" x="8460" y="8865" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="192" text-anchor="middle">TL_HEAVY</text>
+<!-- Text -->
+<text xml:space="preserve" x="4950" y="6705" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="192" text-anchor="middle">SHARED</text>
+</g>
+</svg>
diff --git a/doc/internals/ssl_cert.dia b/doc/internals/ssl_cert.dia
new file mode 100644
index 0000000..52496a1
--- /dev/null
+++ b/doc/internals/ssl_cert.dia
Binary files differ
diff --git a/doc/internals/stats-v2.txt b/doc/internals/stats-v2.txt
new file mode 100644
index 0000000..7d2ae76
--- /dev/null
+++ b/doc/internals/stats-v2.txt
@@ -0,0 +1,8 @@
+
+ Qcur Qmax Scur Smax Slim Scum Fin Fout Bin Bout Ereq Econ Ersp Sts Wght Act Bck EChk Down
+Frontend - - X maxX Y totX I O I O Q - - - - - - - -
+Server X maxX X maxX Y totX I O I O - C R S W A B E D
+Server X maxX X maxX Y totX I O I O - C R S W A B E D
+Server X maxX X maxX Y totX I O I O - C R S W A B E D
+Backend X maxX X maxX Y totX I O I O - C R S totW totA totB totE totD
+
diff --git a/doc/internals/stconn-close.txt b/doc/internals/stconn-close.txt
new file mode 100644
index 0000000..fe1ddca
--- /dev/null
+++ b/doc/internals/stconn-close.txt
@@ -0,0 +1,74 @@
+2023-05-23 - closing states on the stream endpoint descriptor
+
+This document deals with the current flags on the SE desc:
+
+ - SE_FL_ERR_PENDING: an error was met while sending, but some incoming data
+ might still be pending. This flag will be promoted to SE_FL_ERROR when the
+ SE_FL_EOI or SE_FL_EOS flags are set via the standard API (se_fl_set()).
+
+ - SE_FL_ERROR ("ERR"): an error was met, last data were received if any, and no
+ more progress will happen.
+
+ - SE_FL_EOI ("EOI"): the end of the input message was seen, without implying
+ an end of the connection nor the end of event reporting for this stream. For
+ example and end of HTTP request or response will set EOI, after which it's
+ still possible (in case of a request) to bring an abort or error. Said
+ differently, the expected end of the message was seen.
+
+ - SE_FL_EOS ("EOS"): the definitive end of the input data was detected. It may
+ result from an error, an abort, a connection shutdown, and no more receive
+ events will be reported.
+
+The different muxes (H1,H2,H3) can face slightly different situations due to
+the nature, properties, and limitations of their underlying protocols, and will
+set these 3 flags to best translate the lower layer's situation and report it
+to the upper layer:
+
+ +-----------+-----------------------------------------------------------------
+ |ERR EOS EOI| Description per mux
+ +-----------+-----------------------------------------------------------------
+ | 0 0 0 | all: transfer still in progress
+ +-----------+-----------------------------------------------------------------
+ | 0 0 1 | H1: end of message reached.
+ | | H2: "ES" flag seen on a frame.
+ | | H3: not set
+ +-----------+-----------------------------------------------------------------
+ | 0 1 0 | H1: not set (*1)
+ | | H2: not set (*2)
+ | | H3: RST received before FIN (client stops uploading)
+ +-----------+-----------------------------------------------------------------
+ | 0 1 1 | H1: end of message + read0, such as close response or aborted
+ | | request
+ | | H2: not set (*2)
+ | | H3: end of message reached (any subsequent RSTs are ignored)
+ +-----------+-----------------------------------------------------------------
+ | 1 0 0 | all: could be used to report a protocol error (ex: invalid chunk
+ | | encoding, forbidden response header seen from a server).
+ +-----------+-----------------------------------------------------------------
+ | 1 0 1 | all: could be used to report an internal error or a downstream
+ | | protocol error, such as a forbidden header in an HTX block
+ | | coming from the stream layer or the impossibility to encode
+ | | a message. Seems unused right now.
+ +-----------+-----------------------------------------------------------------
+ | 1 1 0 | H1: truncated client input data before response, or truncated
+ | | response from the server
+ | | H2: RST or read0 received before end of input message
+ | | H3: RST + STOP_SENDING before FIN
+ +-----------+-----------------------------------------------------------------
+ | 1 1 1 | H1: error face while sending after end of input message
+ | | H2: RST or read0 received after end of input message
+ | | H3: STOP_SENDING received after a frame with FIN
+ +-----------+-----------------------------------------------------------------
+
+*1: EOS alone is currently not set by H1, however this situation could best
+ describe an H1 upload that was interrupted by the client while receiving
+ an early response, a reused persistent server connection that delivered a
+ read0 immediately after the request was sent, or a truncated server
+ response (or possibly one in close mode when no C-L was advertised). Right
+ now these situations are always accompanied with an ERR flag in addition to
+ the EOS one.
+
+*2: H2 doesn't set EOS without ERR because currently the only ways to close a
+ stream in H2 are by resetting the stream (which conveys an error) or
+ closing the connection (which renders it unusable in both directions and
+ prevents from sending as well).
diff --git a/doc/internals/stream-sock-states.fig b/doc/internals/stream-sock-states.fig
new file mode 100644
index 0000000..79131e5
--- /dev/null
+++ b/doc/internals/stream-sock-states.fig
@@ -0,0 +1,535 @@
+#FIG 3.2 Produced by xfig version 2.0
+Portrait
+Center
+Metric
+A4
+100.00
+Single
+-2
+1200 2
+0 32 #8e8e8e
+6 2295 1260 2430 1395
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 2363 1328 68 68 2430 1328 2295 1328
+4 1 0 50 -1 18 5 0.0000 4 60 60 2363 1361 1\001
+-6
+6 1845 2295 1980 2430
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 1913 2363 68 68 1980 2363 1845 2363
+4 1 0 50 -1 18 5 0.0000 4 60 60 1913 2396 2\001
+-6
+6 2475 2340 2610 2475
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 2543 2408 68 68 2610 2408 2475 2408
+4 1 0 50 -1 18 5 0.0000 4 60 60 2543 2441 9\001
+-6
+6 2835 2610 2970 2745
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 2903 2678 68 68 2970 2678 2835 2678
+4 1 0 50 -1 18 5 0.0000 4 60 60 2903 2711 7\001
+-6
+6 3195 2025 3330 2160
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 3263 2093 68 68 3330 2093 3195 2093
+4 1 0 50 -1 18 5 0.0000 4 60 60 3263 2126 8\001
+-6
+6 2745 2160 2880 2295
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 2813 2228 68 68 2880 2228 2745 2228
+4 1 0 50 -1 18 5 0.0000 4 60 60 2813 2261 6\001
+-6
+6 990 2700 1125 2835
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 1058 2768 68 68 1125 2768 990 2768
+4 1 0 50 -1 18 5 0.0000 4 60 120 1058 2801 13\001
+-6
+6 1305 2970 1440 3105
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 1373 3038 68 68 1440 3038 1305 3038
+4 1 0 50 -1 18 5 0.0000 4 60 120 1373 3071 12\001
+-6
+6 3105 1710 3240 1845
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 3173 1778 68 68 3240 1778 3105 1778
+4 1 0 50 -1 18 5 0.0000 4 60 120 3173 1811 15\001
+-6
+6 4275 1260 4410 1395
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 1328 68 68 4410 1328 4275 1328
+4 1 0 50 -1 18 5 0.0000 4 60 60 4343 1361 1\001
+-6
+6 4275 1440 4410 1575
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 1508 68 68 4410 1508 4275 1508
+4 1 0 50 -1 18 5 0.0000 4 60 60 4343 1541 2\001
+-6
+6 4275 1620 4410 1755
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 1688 68 68 4410 1688 4275 1688
+4 1 0 50 -1 18 5 0.0000 4 60 60 4343 1721 3\001
+-6
+6 4275 1800 4410 1935
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 1868 68 68 4410 1868 4275 1868
+4 1 0 50 -1 18 5 0.0000 4 60 60 4343 1901 4\001
+-6
+6 3240 2835 3375 2970
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 3308 2903 68 68 3375 2903 3240 2903
+4 1 0 50 -1 18 5 0.0000 4 60 120 3308 2936 16\001
+-6
+6 2835 3015 2970 3150
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 2903 3083 68 68 2970 3083 2835 3083
+4 1 0 50 -1 18 5 0.0000 4 60 120 2903 3116 17\001
+-6
+6 2295 3195 2430 3330
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 2363 3263 68 68 2430 3263 2295 3263
+4 1 0 50 -1 18 5 0.0000 4 60 60 2363 3296 3\001
+-6
+6 1440 4815 1620 4995
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 1508 4883 68 68 1575 4883 1440 4883
+4 1 0 50 -1 18 5 0.0000 4 60 120 1508 4916 19\001
+-6
+6 1800 3960 1980 4140
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 1868 4028 68 68 1935 4028 1800 4028
+4 1 0 50 -1 18 5 0.0000 4 60 120 1868 4061 18\001
+-6
+6 4275 1980 4410 2115
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 2048 68 68 4410 2048 4275 2048
+4 1 0 50 -1 18 5 0.0000 4 60 60 4343 2081 5\001
+-6
+6 4275 2340 4410 2475
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 2408 68 68 4410 2408 4275 2408
+4 1 0 50 -1 18 5 0.0000 4 60 60 4343 2441 6\001
+-6
+6 4275 2520 4410 2655
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 2588 68 68 4410 2588 4275 2588
+4 1 0 50 -1 18 5 0.0000 4 60 60 4343 2621 7\001
+-6
+6 4275 2700 4410 2835
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 2768 68 68 4410 2768 4275 2768
+4 1 0 50 -1 18 5 0.0000 4 60 60 4343 2801 8\001
+-6
+6 4275 2880 4410 3015
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 2948 68 68 4410 2948 4275 2948
+4 1 0 50 -1 18 5 0.0000 4 60 60 4343 2981 9\001
+-6
+6 4275 3060 4410 3195
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 3128 68 68 4410 3128 4275 3128
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 3161 10\001
+-6
+6 4275 3240 4410 3375
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 3308 68 68 4410 3308 4275 3308
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 3341 11\001
+-6
+6 4275 3420 4410 3555
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 3488 68 68 4410 3488 4275 3488
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 3521 12\001
+-6
+6 4275 3600 4410 3735
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 3668 68 68 4410 3668 4275 3668
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 3701 13\001
+-6
+6 4275 3960 4410 4095
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 4028 68 68 4410 4028 4275 4028
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 4061 15\001
+-6
+6 4275 4140 4410 4275
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 4208 68 68 4410 4208 4275 4208
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 4241 16\001
+-6
+6 4275 4320 4410 4455
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 4388 68 68 4410 4388 4275 4388
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 4421 17\001
+-6
+6 4275 3780 4455 3960
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 3848 68 68 4410 3848 4275 3848
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 3881 14\001
+-6
+6 4275 4590 4455 4770
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 4658 68 68 4410 4658 4275 4658
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 4691 18\001
+-6
+6 4275 4770 4455 4950
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 4838 68 68 4410 4838 4275 4838
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 4871 19\001
+-6
+6 4275 4950 4455 5130
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 5018 68 68 4410 5018 4275 5018
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 5051 20\001
+-6
+6 1170 3690 1350 3870
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 1238 3758 68 68 1305 3758 1170 3758
+4 1 0 50 -1 18 5 0.0000 4 60 120 1238 3791 11\001
+-6
+6 1530 3555 1710 3735
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 1598 3623 68 68 1665 3623 1530 3623
+4 1 0 50 -1 18 5 0.0000 4 60 120 1598 3656 10\001
+-6
+6 720 4095 900 4275
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 788 4163 68 68 855 4163 720 4163
+4 1 0 50 -1 18 5 0.0000 4 60 120 788 4196 14\001
+-6
+6 855 3645 1035 3825
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 923 3713 68 68 990 3713 855 3713
+4 1 0 50 -1 18 5 0.0000 4 60 120 923 3746 21\001
+-6
+6 4275 5130 4455 5310
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 5198 68 68 4410 5198 4275 5198
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 5231 21\001
+-6
+6 2295 4140 2430 4275
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 2363 4208 68 68 2430 4208 2295 4208
+4 1 0 50 -1 18 5 0.0000 4 60 60 2363 4241 4\001
+-6
+6 2475 3870 2655 4050
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 2543 3938 68 68 2610 3938 2475 3938
+4 1 0 50 -1 18 5 0.0000 4 60 120 2543 3971 22\001
+-6
+6 4275 5310 4455 5490
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 5378 68 68 4410 5378 4275 5378
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 5411 22\001
+-6
+6 2295 5625 2430 5760
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 2363 5693 68 68 2430 5693 2295 5693
+4 1 0 50 -1 18 5 0.0000 4 60 60 2363 5726 5\001
+-6
+6 2295 6480 2475 6660
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 2363 6548 68 68 2430 6548 2295 6548
+4 1 0 50 -1 18 5 0.0000 4 60 120 2363 6581 20\001
+-6
+1 2 0 1 0 6 50 -1 20 0.000 1 0.0000 1350 4612 225 112 1125 4612 1575 4612
+1 2 0 1 0 6 50 -1 20 0.000 1 0.0000 2250 1912 225 112 2025 1912 2475 1912
+1 2 0 1 0 7 50 -1 20 0.000 1 0.0000 1125 3487 225 112 900 3487 1350 3487
+1 2 0 1 0 7 50 -1 20 0.000 1 0.0000 2250 3712 225 112 2025 3712 2475 3712
+1 2 0 1 0 6 50 -1 20 0.000 1 0.0000 2250 2812 225 112 2025 2812 2475 2812
+1 2 0 1 0 7 50 -1 20 0.000 1 0.0000 3375 2362 225 112 3150 2362 3600 2362
+1 2 0 1 0 7 50 -1 20 0.000 1 0.0000 2250 1012 225 112 2025 1012 2475 1012
+1 2 0 1 0 6 50 -1 20 0.000 1 0.0000 2250 6232 225 112 2025 6232 2475 6232
+1 2 0 1 0 7 50 -1 20 0.000 1 0.0000 2250 5422 225 112 2025 5422 2475 5422
+1 2 0 1 0 7 50 -1 20 0.000 1 0.0000 2250 6997 225 112 2025 6997 2475 6997
+1 2 0 1 0 6 50 -1 20 0.000 1 0.0000 2250 4587 225 112 2025 4587 2475 4587
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2250 1125 2250 1800
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8910 5805 4500 5805
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 3
+ 6885 5900 6930 5990 6975 5810
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 3
+ 6885 6570 6930 6660 6975 6480
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 5310 5589 5310 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 5670 5589 5670 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 6030 5589 6030 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 6390 5589 6390 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 6750 5589 6750 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 7110 5589 7110 6921
+2 1 0 2 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 4950 5589 4950 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8910 6705 4500 6705
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2250 5535 2250 6120
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2250 6345 2250 6885
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 4500 5580 8910 5580 8910 6930 4500 6930 4500 5580
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 4500 5580 8910 5580 8910 6930 4500 6930 4500 5580
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8910 6030 4500 6030
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8910 6255 4500 6255
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8910 6480 4500 6480
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 5310 5589 5310 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 5670 5589 5670 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 6030 5589 6030 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 6390 5589 6390 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 6750 5589 6750 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 7110 5589 7110 6921
+2 1 0 2 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 4950 5589 4950 6921
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 4500 5580 8910 5580 8910 6930 4500 6930 4500 5580
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8865 5805 4500 5805
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8865 6030 4500 6030
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8865 6255 4500 6255
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8865 6480 4500 6480
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 5310 5589 5310 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 5670 5589 5670 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 6030 5589 6030 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 6390 5589 6390 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 6750 5589 6750 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 7110 5589 7110 6921
+2 1 0 2 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 4950 5589 4950 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8865 6705 4500 6705
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8865 5805 4500 5805
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8865 6030 4500 6030
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8865 6255 4500 6255
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8865 6480 4500 6480
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 5310 5589 5310 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 5670 5589 5670 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 6030 5589 6030 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 6390 5589 6390 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 6750 5589 6750 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 7110 5589 7110 6921
+2 1 0 2 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 4950 5589 4950 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8865 6705 4500 6705
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 3
+ 7605 5890 7650 5980 7695 5800
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 3
+ 7605 6570 7650 6660 7695 6480
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 7470 5589 7470 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 3
+ 7965 5890 8010 5980 8055 5800
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 3
+ 7965 6570 8010 6660 8055 6480
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 7830 5589 7830 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 3
+ 8325 6570 8370 6660 8415 6480
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8190 5589 8190 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8550 5589 8550 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8190 5589 8190 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8550 5589 8550 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8190 5589 8190 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8550 5589 8550 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8190 5589 8190 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8550 5589 8550 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 1
+ 4500 5805
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 1
+ 4500 6030
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 1
+ 4500 6255
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 1
+ 4500 6480
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 1
+ 4500 6705
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 2250 2700 2475 2475 2475 2250 2250 2025
+ 0.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 3375 2250 2925 2025 2475 1935
+ 0.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 3375 2475 3375 2700 2475 2835
+ 0.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 3420 2475 3420 4320 3150 5850 2475 6165
+ 0.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 1125 3375 1125 2925 2025 2790
+ 0.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 1125 3375 1125 2250 2025 1935
+ 0.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 6
+ 1 1 1.00 60.00 120.00
+ 2475 1890 3825 1800 3825 2520 3825 4500 3150 6075 2475 6210
+ 0.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 2250 2025 2025 2250 2025 2475 2250 2700
+ 0.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2250 3825 2250 4500
+ 0.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 2475 1980 2880 2115 3150 2340
+ 0.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2250 2925 2250 3600
+ 0.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 2205 3825 2070 4140 1622 4221 1440 4500
+ 0.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 1350 4725 1350 4950 1485 5760 2025 6165
+ 0.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 7
+ 1 1 1.00 60.00 120.00
+ 1125 4590 720 4455 675 4050 675 3600 675 2250 1350 1800
+ 2025 1935
+ 0.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 1260 4500 1125 4320 1125 3600
+ 0.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 1350 4500 1440 3645 1575 3330 2070 2880
+ 0.000 1.000 1.000 0.000
+3 0 0 1 32 7 51 -1 -1 0.000 0 1 0 5
+ 1 1 1.00 60.00 120.00
+ 1035 3600 990 4365 990 5040 1395 5895 2025 6210
+ 0.000 1.000 1.000 1.000 0.000
+3 0 0 1 32 7 51 -1 -1 0.000 0 1 0 5
+ 1 1 1.00 60.00 120.00
+ 2340 3825 2385 4005 2925 4275 2655 4815 2295 5310
+ 0.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 2475 2835 3150 3375 3150 5625 2475 6120
+ 0.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2250 4725 2250 5310
+ 0.000 0.000
+4 0 0 50 -1 14 6 0.0000 4 105 2880 4500 1710 ASS-CON: ssui(): connect_server() == SN_ERR_NONE\001
+4 0 0 50 -1 14 6 0.0000 4 90 540 4500 1350 INI-REQ: \001
+4 0 0 50 -1 14 6 0.0000 4 120 3720 4500 1530 REQ-ASS: prepare_conn_request(): srv_redispatch_connect() == 0\001
+4 0 4 50 -1 14 10 0.0000 4 105 105 2475 2700 4\001
+4 0 4 50 -1 14 10 0.0000 4 105 105 1620 4500 6\001
+4 0 0 50 -1 14 6 0.0000 4 120 3360 4500 1890 CON-EST: sess_update_st_con_tcp(): !timeout && !conn_err\001
+4 0 0 50 -1 14 6 0.0000 4 105 2460 4500 3510 TAR-ASS: ssui(): SI_FL_EXP && SN_ASSIGNED\001
+4 0 0 50 -1 14 6 0.0000 4 105 3420 4500 2970 ASS-REQ: connect_server: conn_retries == 0 && PR_O_REDISP\001
+4 0 0 50 -1 14 6 0.0000 4 120 2460 4500 2610 QUE-REQ: ssui(): !pend_pos && SN_ASSIGNED\001
+4 0 0 50 -1 14 6 0.0000 4 120 2520 4500 2790 QUE-REQ: ssui(): !pend_pos && !SN_ASSIGNED\001
+4 0 0 50 -1 14 6 0.0000 4 120 3300 4500 4230 QUE-CLO: ssui(): pend_pos && (SI_FL_EXP || req_aborted)\001
+4 0 0 50 -1 14 6 0.0000 4 105 2520 4500 3690 TAR-REQ: ssui(): SI_FL_EXP && !SN_ASSIGNED\001
+4 0 0 50 -1 14 6 0.0000 4 120 3960 4500 4545 ASS-CLO: PR_O_REDISP && SN_REDIRECTABLE && perform_http_redirect()\001
+4 0 0 50 -1 14 6 0.0000 4 120 4440 4500 2430 REQ-QUE: prepare_conn_request(): srv_redispatch_connect() != 0 (SI_ST_QUE)\001
+4 0 0 50 -1 14 6 0.0000 4 120 4200 4500 4050 REQ-CLO: prepare_conn_request(): srv_redispatch_connect() != 0 (error)\001
+4 0 0 50 -1 14 6 0.0000 4 105 4320 4500 4410 ASS-CLO: ssui(): connect_server() == SN_ERR_INTERNAL || conn_retries < 0\001
+4 0 0 50 -1 14 6 0.0000 4 120 3120 4500 4680 CON-CER: sess_update_st_con_tcp(): timeout/SI_FL_ERR\001
+4 0 0 50 -1 14 6 0.0000 4 120 3600 4500 4860 CER-CLO: sess_update_st_cer(): (ERR/EXP) && conn_retries < 0\001
+4 0 0 50 -1 14 6 0.0000 4 120 4200 4500 3870 CER-REQ: sess_update_st_cer(): timeout && !conn_retries && PR_O_REDISP\001
+4 0 0 50 -1 14 6 0.0000 4 120 3600 4500 3330 CER-TAR: sess_update_st_cer(): conn_err && conn_retries >= 0\001
+4 0 0 50 -1 14 6 0.0000 4 120 4620 4500 3150 CER-ASS: sess_update_st_cer(): timeout && (conn_retries >= 0 || !PR_O_REDISP)\001
+4 0 4 50 -1 14 10 0.0000 4 105 105 1305 3375 3\001
+4 0 4 50 -1 14 10 0.0000 4 105 105 2430 3600 5\001
+4 0 4 50 -1 14 10 0.0000 4 105 105 3555 2250 2\001
+4 0 4 50 -1 14 10 0.0000 4 105 105 2430 1800 1\001
+4 0 4 50 -1 14 10 0.0000 4 105 105 2430 900 0\001
+4 0 0 50 -1 14 6 0.0000 4 105 3000 4500 2070 EST-DIS: stream_sock_read/write/shutr/shutw: close\001
+4 0 0 50 -1 14 6 0.0000 4 120 1980 4500 2250 EST-DIS: process_session(): error\001
+4 0 0 50 -1 14 6 0.0000 4 120 2100 4500 5040 DIS-CLO: process_session(): cleanup\001
+4 1 0 50 -1 14 10 0.0000 4 105 315 1350 4680 CER\001
+4 1 0 50 -1 14 10 0.0000 4 135 315 2250 1980 REQ\001
+4 1 0 50 -1 14 10 0.0000 4 105 315 1125 3555 TAR\001
+4 1 0 50 -1 14 10 0.0000 4 105 315 2250 2880 ASS\001
+4 1 0 50 -1 14 10 0.0000 4 135 315 3375 2430 QUE\001
+4 1 0 50 -1 14 10 0.0000 4 105 315 2250 3780 CON\001
+4 1 0 50 -1 14 10 0.0000 4 105 315 2250 1080 INI\001
+4 0 0 50 -1 14 6 0.0000 4 120 2820 4500 5220 TAR-CLO: sess_update_stream_int(): client abort\001
+4 0 0 50 -1 14 6 0.0000 4 120 2820 4500 5400 CON-DIS: sess_update_st_con_tcp(): client abort\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 5130 5985 -\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 5490 5985 -\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 5850 5985 -\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 6210 5985 -\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 6570 5985 -\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 7290 5985 -\001
+4 1 0 50 -1 16 7 0.0000 4 105 120 4725 5985 fd\001
+4 1 0 50 -1 14 8 0.0000 4 90 225 5130 5760 INI\001
+4 1 0 50 -1 16 7 0.0000 4 105 270 4725 5760 state\001
+4 1 0 50 -1 14 8 0.0000 4 120 225 5490 5760 REQ\001
+4 1 0 50 -1 14 8 0.0000 4 120 225 5850 5760 QUE\001
+4 1 0 50 -1 14 8 0.0000 4 90 225 6210 5760 TAR\001
+4 1 0 50 -1 14 8 0.0000 4 90 225 6570 5760 ASS\001
+4 1 0 50 -1 14 8 0.0000 4 90 225 6930 5760 CON\001
+4 1 0 50 -1 14 8 0.0000 4 90 225 7290 5760 CER\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 5850 6210 0\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 5130 6210 0\001
+4 1 0 50 -1 16 7 0.0000 4 90 270 4725 6210 ERR\001
+4 1 0 50 -1 16 7 0.0000 4 90 270 4725 6435 EXP\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 5490 6210 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 6210 6210 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 6570 6210 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 6570 6435 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 5490 6435 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 5130 6435 0\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 5850 6435 0\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 6210 6435 0\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 7290 6435 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 6930 6435 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 7290 6210 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 6930 6210 X\001
+4 1 0 50 -1 16 7 0.0000 4 75 240 4725 6660 sess\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 5130 6660 -\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 5490 6660 -\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 5850 6660 -\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 6210 6660 -\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 6570 6660 -\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 7290 6660 -\001
+4 0 0 50 -1 16 6 0.0000 4 120 5970 675 7335 Note: states painted yellow above are transient ; process_session() will never leave a stream interface in any of those upon return.\001
+4 1 0 50 -1 16 7 0.0000 4 90 330 4725 6840 SHUT\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 7290 6840 -\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 6930 6840 0\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 6570 6840 0\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 6210 6840 0\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 5850 6840 0\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 5490 6840 0\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 5130 6840 0\001
+4 1 0 50 -1 14 10 0.0000 4 105 315 2250 6300 DIS\001
+4 1 0 50 -1 14 10 0.0000 4 105 315 2250 5490 EST\001
+4 1 0 50 -1 14 10 0.0000 4 105 315 2250 7065 CLO\001
+4 1 0 50 -1 14 10 0.0000 4 105 315 2250 4635 RDY\001
+4 0 4 50 -1 14 10 0.0000 4 105 105 2430 4455 7\001
+4 0 4 50 -1 14 10 0.0000 4 105 105 2430 5310 8\001
+4 0 4 50 -1 14 10 0.0000 4 105 105 2385 6120 9\001
+4 0 4 50 -1 14 10 0.0000 4 105 210 2385 6840 10\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 7650 6210 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 7650 6435 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 7650 6840 0\001
+4 1 0 50 -1 14 8 0.0000 4 90 225 8010 5760 EST\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 8010 6210 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 8010 6435 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 8010 6840 0\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 8370 5985 -\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 8370 6210 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 8370 6435 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 225 8730 5760 CLO\001
+4 1 0 50 -1 14 8 0.0000 4 90 225 8370 5760 DIS\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 8730 5985 -\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 8730 6210 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 8730 6435 X\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 8730 6660 -\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 8370 6840 1\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 8730 6840 1\001
+4 1 0 50 -1 14 8 0.0000 4 90 225 7650 5760 RDY\001
diff --git a/doc/intro.txt b/doc/intro.txt
new file mode 100644
index 0000000..f4133a1
--- /dev/null
+++ b/doc/intro.txt
@@ -0,0 +1,1700 @@
+ -----------------------
+ HAProxy Starter Guide
+ -----------------------
+ version 2.9
+
+
+This document is an introduction to HAProxy for all those who don't know it, as
+well as for those who want to re-discover it when they know older versions. Its
+primary focus is to provide users with all the elements to decide if HAProxy is
+the product they're looking for or not. Advanced users may find here some parts
+of solutions to some ideas they had just because they were not aware of a given
+new feature. Some sizing information is also provided, the product's lifecycle
+is explained, and comparisons with partially overlapping products are provided.
+
+This document doesn't provide any configuration help or hints, but it explains
+where to find the relevant documents. The summary below is meant to help you
+search sections by name and navigate through the document.
+
+Note to documentation contributors :
+ This document is formatted with 80 columns per line, with even number of
+ spaces for indentation and without tabs. Please follow these rules strictly
+ so that it remains easily printable everywhere. If you add sections, please
+ update the summary below for easier searching.
+
+
+Summary
+-------
+
+1. Available documentation
+
+2. Quick introduction to load balancing and load balancers
+
+3. Introduction to HAProxy
+3.1. What HAProxy is and is not
+3.2. How HAProxy works
+3.3. Basic features
+3.3.1. Proxying
+3.3.2. SSL
+3.3.3. Monitoring
+3.3.4. High availability
+3.3.5. Load balancing
+3.3.6. Stickiness
+3.3.7. Logging
+3.3.8. Statistics
+3.4. Standard features
+3.4.1. Sampling and converting information
+3.4.2. Maps
+3.4.3. ACLs and conditions
+3.4.4. Content switching
+3.4.5. Stick-tables
+3.4.6. Formatted strings
+3.4.7. HTTP rewriting and redirection
+3.4.8. Server protection
+3.5. Advanced features
+3.5.1. Management
+3.5.2. System-specific capabilities
+3.5.3. Scripting
+3.6. Sizing
+3.7. How to get HAProxy
+
+4. Companion products and alternatives
+4.1. Apache HTTP server
+4.2. NGINX
+4.3. Varnish
+4.4. Alternatives
+
+5. Contacts
+
+
+1. Available documentation
+--------------------------
+
+The complete HAProxy documentation is contained in the following documents.
+Please ensure to consult the relevant documentation to save time and to get the
+most accurate response to your needs. Also please refrain from sending questions
+to the mailing list whose responses are present in these documents.
+
+ - intro.txt (this document) : it presents the basics of load balancing,
+ HAProxy as a product, what it does, what it doesn't do, some known traps to
+ avoid, some OS-specific limitations, how to get it, how it evolves, how to
+ ensure you're running with all known fixes, how to update it, complements
+ and alternatives.
+
+ - management.txt : it explains how to start haproxy, how to manage it at
+ runtime, how to manage it on multiple nodes, and how to proceed with
+ seamless upgrades.
+
+ - configuration.txt : the reference manual details all configuration keywords
+ and their options. It is used when a configuration change is needed.
+
+ - coding-style.txt : this is for developers who want to propose some code to
+ the project. It explains the style to adopt for the code. It is not very
+ strict and not all the code base completely respects it, but contributions
+ which diverge too much from it will be rejected.
+
+ - proxy-protocol.txt : this is the de-facto specification of the PROXY
+ protocol which is implemented by HAProxy and a number of third party
+ products.
+
+ - README : how to build HAProxy from sources
+
+
+2. Quick introduction to load balancing and load balancers
+----------------------------------------------------------
+
+Load balancing consists in aggregating multiple components in order to achieve
+a total processing capacity above each component's individual capacity, without
+any intervention from the end user and in a scalable way. This results in more
+operations being performed simultaneously by the time it takes a component to
+perform only one. A single operation however will still be performed on a single
+component at a time and will not get faster than without load balancing. It
+always requires at least as many operations as available components and an
+efficient load balancing mechanism to make use of all components and to fully
+benefit from the load balancing. A good example of this is the number of lanes
+on a highway which allows as many cars to pass during the same time frame
+without increasing their individual speed.
+
+Examples of load balancing :
+
+ - Process scheduling in multi-processor systems
+ - Link load balancing (e.g. EtherChannel, Bonding)
+ - IP address load balancing (e.g. ECMP, DNS round-robin)
+ - Server load balancing (via load balancers)
+
+The mechanism or component which performs the load balancing operation is
+called a load balancer. In web environments these components are called a
+"network load balancer", and more commonly a "load balancer" given that this
+activity is by far the best known case of load balancing.
+
+A load balancer may act :
+
+ - at the link level : this is called link load balancing, and it consists in
+ choosing what network link to send a packet to;
+
+ - at the network level : this is called network load balancing, and it
+ consists in choosing what route a series of packets will follow;
+
+ - at the server level : this is called server load balancing and it consists
+ in deciding what server will process a connection or request.
+
+Two distinct technologies exist and address different needs, though with some
+overlapping. In each case it is important to keep in mind that load balancing
+consists in diverting the traffic from its natural flow and that doing so always
+requires a minimum of care to maintain the required level of consistency between
+all routing decisions.
+
+The first one acts at the packet level and processes packets more or less
+individually. There is a 1-to-1 relation between input and output packets, so
+it is possible to follow the traffic on both sides of the load balancer using a
+regular network sniffer. This technology can be very cheap and extremely fast.
+It is usually implemented in hardware (ASICs) allowing to reach line rate, such
+as switches doing ECMP. Usually stateless, it can also be stateful (consider
+the session a packet belongs to and called layer4-LB or L4), may support DSR
+(direct server return, without passing through the LB again) if the packets
+were not modified, but provides almost no content awareness. This technology is
+very well suited to network-level load balancing, though it is sometimes used
+for very basic server load balancing at high speed.
+
+The second one acts on session contents. It requires that the input streams is
+reassembled and processed as a whole. The contents may be modified, and the
+output stream is segmented into new packets. For this reason it is generally
+performed by proxies and they're often called layer 7 load balancers or L7.
+This implies that there are two distinct connections on each side, and that
+there is no relation between input and output packets sizes nor counts. Clients
+and servers are not required to use the same protocol (for example IPv4 vs
+IPv6, clear vs SSL). The operations are always stateful, and the return traffic
+must pass through the load balancer. The extra processing comes with a cost so
+it's not always possible to achieve line rate, especially with small packets.
+On the other hand, it offers wide possibilities and is generally achieved by
+pure software, even if embedded into hardware appliances. This technology is
+very well suited for server load balancing.
+
+Packet-based load balancers are generally deployed in cut-through mode, so they
+are installed on the normal path of the traffic and divert it according to the
+configuration. The return traffic doesn't necessarily pass through the load
+balancer. Some modifications may be applied to the network destination address
+in order to direct the traffic to the proper destination. In this case, it is
+mandatory that the return traffic passes through the load balancer. If the
+routes doesn't make this possible, the load balancer may also replace the
+packets' source address with its own in order to force the return traffic to
+pass through it.
+
+Proxy-based load balancers are deployed as a server with their own IP addresses
+and ports, without architecture changes. Sometimes this requires to perform some
+adaptations to the applications so that clients are properly directed to the
+load balancer's IP address and not directly to the server's. Some load balancers
+may have to adjust some servers' responses to make this possible (e.g. the HTTP
+Location header field used in HTTP redirects). Some proxy-based load balancers
+may intercept traffic for an address they don't own, and spoof the client's
+address when connecting to the server. This allows them to be deployed as if
+they were a regular router or firewall, in a cut-through mode very similar to
+the packet based load balancers. This is particularly appreciated for products
+which combine both packet mode and proxy mode. In this case DSR is obviously
+still not possible and the return traffic still has to be routed back to the
+load balancer.
+
+A very scalable layered approach would consist in having a front router which
+receives traffic from multiple load balanced links, and uses ECMP to distribute
+this traffic to a first layer of multiple stateful packet-based load balancers
+(L4). These L4 load balancers in turn pass the traffic to an even larger number
+of proxy-based load balancers (L7), which have to parse the contents to decide
+what server will ultimately receive the traffic.
+
+The number of components and possible paths for the traffic increases the risk
+of failure; in very large environments, it is even normal to permanently have
+a few faulty components being fixed or replaced. Load balancing done without
+awareness of the whole stack's health significantly degrades availability. For
+this reason, any sane load balancer will verify that the components it intends
+to deliver the traffic to are still alive and reachable, and it will stop
+delivering traffic to faulty ones. This can be achieved using various methods.
+
+The most common one consists in periodically sending probes to ensure the
+component is still operational. These probes are called "health checks". They
+must be representative of the type of failure to address. For example a ping-
+based check will not detect that a web server has crashed and doesn't listen to
+a port anymore, while a connection to the port will verify this, and a more
+advanced request may even validate that the server still works and that the
+database it relies on is still accessible. Health checks often involve a few
+retries to cover for occasional measuring errors. The period between checks
+must be small enough to ensure the faulty component is not used for too long
+after an error occurs.
+
+Other methods consist in sampling the production traffic sent to a destination
+to observe if it is processed correctly or not, and to evict the components
+which return inappropriate responses. However this requires to sacrifice a part
+of the production traffic and this is not always acceptable. A combination of
+these two mechanisms provides the best of both worlds, with both of them being
+used to detect a fault, and only health checks to detect the end of the fault.
+A last method involves centralized reporting : a central monitoring agent
+periodically updates all load balancers about all components' state. This gives
+a global view of the infrastructure to all components, though sometimes with
+less accuracy or responsiveness. It's best suited for environments with many
+load balancers and many servers.
+
+Layer 7 load balancers also face another challenge known as stickiness or
+persistence. The principle is that they generally have to direct multiple
+subsequent requests or connections from a same origin (such as an end user) to
+the same target. The best known example is the shopping cart on an online
+store. If each click leads to a new connection, the user must always be sent
+to the server which holds his shopping cart. Content-awareness makes it easier
+to spot some elements in the request to identify the server to deliver it to,
+but that's not always enough. For example if the source address is used as a
+key to pick a server, it can be decided that a hash-based algorithm will be
+used and that a given IP address will always be sent to the same server based
+on a divide of the address by the number of available servers. But if one
+server fails, the result changes and all users are suddenly sent to a different
+server and lose their shopping cart. The solution against this issue consists
+in memorizing the chosen target so that each time the same visitor is seen,
+he's directed to the same server regardless of the number of available servers.
+The information may be stored in the load balancer's memory, in which case it
+may have to be replicated to other load balancers if it's not alone, or it may
+be stored in the client's memory using various methods provided that the client
+is able to present this information back with every request (cookie insertion,
+redirection to a sub-domain, etc). This mechanism provides the extra benefit of
+not having to rely on unstable or unevenly distributed information (such as the
+source IP address). This is in fact the strongest reason to adopt a layer 7
+load balancer instead of a layer 4 one.
+
+In order to extract information such as a cookie, a host header field, a URL
+or whatever, a load balancer may need to decrypt SSL/TLS traffic and even
+possibly to re-encrypt it when passing it to the server. This expensive task
+explains why in some high-traffic infrastructures, sometimes there may be a
+lot of load balancers.
+
+Since a layer 7 load balancer may perform a number of complex operations on the
+traffic (decrypt, parse, modify, match cookies, decide what server to send to,
+etc), it can definitely cause some trouble and will very commonly be accused of
+being responsible for a lot of trouble that it only revealed. Often it will be
+discovered that servers are unstable and periodically go up and down, or for
+web servers, that they deliver pages with some hard-coded links forcing the
+clients to connect directly to one specific server without passing via the load
+balancer, or that they take ages to respond under high load causing timeouts.
+That's why logging is an extremely important aspect of layer 7 load balancing.
+Once a trouble is reported, it is important to figure if the load balancer took
+a wrong decision and if so why so that it doesn't happen anymore.
+
+
+3. Introduction to HAProxy
+--------------------------
+
+HAProxy is written as "HAProxy" to designate the product, and as "haproxy" to
+designate the executable program, software package or a process. However, both
+are commonly used for both purposes, and are pronounced H-A-Proxy. Very early,
+"haproxy" used to stand for "high availability proxy" and the name was written
+in two separate words, though by now it means nothing else than "HAProxy".
+
+
+3.1. What HAProxy is and isn't
+------------------------------
+
+HAProxy is :
+
+ - a TCP proxy : it can accept a TCP connection from a listening socket,
+ connect to a server and attach these sockets together allowing traffic to
+ flow in both directions; IPv4, IPv6 and even UNIX sockets are supported on
+ either side, so this can provide an easy way to translate addresses between
+ different families.
+
+ - an HTTP reverse-proxy (called a "gateway" in HTTP terminology) : it presents
+ itself as a server, receives HTTP requests over connections accepted on a
+ listening TCP socket, and passes the requests from these connections to
+ servers using different connections. It may use any combination of HTTP/1.x
+ or HTTP/2 on any side and will even automatically detect the protocol
+ spoken on each side when ALPN is used over TLS.
+
+ - an SSL terminator / initiator / offloader : SSL/TLS may be used on the
+ connection coming from the client, on the connection going to the server,
+ or even on both connections. A lot of settings can be applied per name
+ (SNI), and may be updated at runtime without restarting. Such setups are
+ extremely scalable and deployments involving tens to hundreds of thousands
+ of certificates were reported.
+
+ - a TCP normalizer : since connections are locally terminated by the operating
+ system, there is no relation between both sides, so abnormal traffic such as
+ invalid packets, flag combinations, window advertisements, sequence numbers,
+ incomplete connections (SYN floods), or so will not be passed to the other
+ side. This protects fragile TCP stacks from protocol attacks, and also
+ allows to optimize the connection parameters with the client without having
+ to modify the servers' TCP stack settings.
+
+ - an HTTP normalizer : when configured to process HTTP traffic, only valid
+ complete requests are passed. This protects against a lot of protocol-based
+ attacks. Additionally, protocol deviations for which there is a tolerance
+ in the specification are fixed so that they don't cause problem on the
+ servers (e.g. multiple-line headers).
+
+ - an HTTP fixing tool : it can modify / fix / add / remove / rewrite the URL
+ or any request or response header. This helps fixing interoperability issues
+ in complex environments.
+
+ - a content-based switch : it can consider any element from the request to
+ decide what server to pass the request or connection to. Thus it is possible
+ to handle multiple protocols over a same port (e.g. HTTP, HTTPS, SSH).
+
+ - a server load balancer : it can load balance TCP connections and HTTP
+ requests. In TCP mode, load balancing decisions are taken for the whole
+ connection. In HTTP mode, decisions are taken per request.
+
+ - a traffic regulator : it can apply some rate limiting at various points,
+ protect the servers against overloading, adjust traffic priorities based on
+ the contents, and even pass such information to lower layers and outer
+ network components by marking packets.
+
+ - a protection against DDoS and service abuse : it can maintain a wide number
+ of statistics per IP address, URL, cookie, etc and detect when an abuse is
+ happening, then take action (slow down the offenders, block them, send them
+ to outdated contents, etc).
+
+ - an observation point for network troubleshooting : due to the precision of
+ the information reported in logs, it is often used to narrow down some
+ network-related issues.
+
+ - an HTTP compression offloader : it can compress responses which were not
+ compressed by the server, thus reducing the page load time for clients with
+ poor connectivity or using high-latency, mobile networks.
+
+ - a caching proxy : it may cache responses in RAM so that subsequent requests
+ for the same object avoid the cost of another network transfer from the
+ server as long as the object remains present and valid. It will however not
+ store objects to any persistent storage. Please note that this caching
+ feature is designed to be maintenance free and focuses solely on saving
+ haproxy's precious resources and not on save the server's resources. Caches
+ designed to optimize servers require much more tuning and flexibility. If
+ you instead need such an advanced cache, please use Varnish Cache, which
+ integrates perfectly with haproxy, especially when SSL/TLS is needed on any
+ side.
+
+ - a FastCGI gateway : FastCGI can be seen as a different representation of
+ HTTP, and as such, HAProxy can directly load-balance a farm comprising any
+ combination of FastCGI application servers without requiring to insert
+ another level of gateway between them. This results in resource savings and
+ a reduction of maintenance costs.
+
+HAProxy is not :
+
+ - an explicit HTTP proxy, i.e. the proxy that browsers use to reach the
+ internet. There are excellent open-source software dedicated for this task,
+ such as Squid. However HAProxy can be installed in front of such a proxy to
+ provide load balancing and high availability.
+
+ - a data scrubber : it will not modify the body of requests nor responses.
+
+ - a static web server : during startup, it isolates itself inside a chroot
+ jail and drops its privileges, so that it will not perform any single file-
+ system access once started. As such it cannot be turned into a static web
+ server (dynamic servers are supported through FastCGI however). There are
+ excellent open-source software for this such as Apache or Nginx, and
+ HAProxy can be easily installed in front of them to provide load balancing,
+ high availability and acceleration.
+
+ - a packet-based load balancer : it will not see IP packets nor UDP datagrams,
+ will not perform NAT or even less DSR. These are tasks for lower layers.
+ Some kernel-based components such as IPVS (Linux Virtual Server) already do
+ this pretty well and complement perfectly with HAProxy.
+
+
+3.2. How HAProxy works
+----------------------
+
+HAProxy is an event-driven, non-blocking engine combining a very fast I/O layer
+with a priority-based, multi-threaded scheduler. As it is designed with a data
+forwarding goal in mind, its architecture is optimized to move data as fast as
+possible with the least possible operations. It focuses on optimizing the CPU
+cache's efficiency by sticking connections to the same CPU as long as possible.
+As such it implements a layered model offering bypass mechanisms at each level
+ensuring data doesn't reach higher levels unless needed. Most of the processing
+is performed in the kernel, and HAProxy does its best to help the kernel do the
+work as fast as possible by giving some hints or by avoiding certain operation
+when it guesses they could be grouped later. As a result, typical figures show
+15% of the processing time spent in HAProxy versus 85% in the kernel in TCP or
+HTTP close mode, and about 30% for HAProxy versus 70% for the kernel in HTTP
+keep-alive mode.
+
+A single process can run many proxy instances; configurations as large as
+300000 distinct proxies in a single process were reported to run fine. A single
+core, single CPU setup is far more than enough for more than 99% users, and as
+such, users of containers and virtual machines are encouraged to use the
+absolute smallest images they can get to save on operational costs and simplify
+troubleshooting. However the machine HAProxy runs on must never ever swap, and
+its CPU must not be artificially throttled (sub-CPU allocation in hypervisors)
+nor be shared with compute-intensive processes which would induce a very high
+context-switch latency.
+
+Threading allows to exploit all available processing capacity by using one
+thread per CPU core. This is mostly useful for SSL or when data forwarding
+rates above 40 Gbps are needed. In such cases it is critically important to
+avoid communications between multiple physical CPUs, which can cause strong
+bottlenecks in the network stack and in HAProxy itself. While counter-intuitive
+to some, the first thing to do when facing some performance issues is often to
+reduce the number of CPUs HAProxy runs on.
+
+HAProxy only requires the haproxy executable and a configuration file to run.
+For logging it is highly recommended to have a properly configured syslog daemon
+and log rotations in place. Logs may also be sent to stdout/stderr, which can be
+useful inside containers. The configuration files are parsed before starting,
+then HAProxy tries to bind all listening sockets, and refuses to start if
+anything fails. Past this point it cannot fail anymore. This means that there
+are no runtime failures and that if it accepts to start, it will work until it
+is stopped.
+
+Once HAProxy is started, it does exactly 3 things :
+
+ - process incoming connections;
+
+ - periodically check the servers' status (known as health checks);
+
+ - exchange information with other haproxy nodes.
+
+Processing incoming connections is by far the most complex task as it depends
+on a lot of configuration possibilities, but it can be summarized as the 9 steps
+below :
+
+ - accept incoming connections from listening sockets that belong to a
+ configuration entity known as a "frontend", which references one or multiple
+ listening addresses;
+
+ - apply the frontend-specific processing rules to these connections that may
+ result in blocking them, modifying some headers, or intercepting them to
+ execute some internal applets such as the statistics page or the CLI;
+
+ - pass these incoming connections to another configuration entity representing
+ a server farm known as a "backend", which contains the list of servers and
+ the load balancing strategy for this server farm;
+
+ - apply the backend-specific processing rules to these connections;
+
+ - decide which server to forward the connection to according to the load
+ balancing strategy;
+
+ - apply the backend-specific processing rules to the response data;
+
+ - apply the frontend-specific processing rules to the response data;
+
+ - emit a log to report what happened in fine details;
+
+ - in HTTP, loop back to the second step to wait for a new request, otherwise
+ close the connection.
+
+Frontends and backends are sometimes considered as half-proxies, since they only
+look at one side of an end-to-end connection; the frontend only cares about the
+clients while the backend only cares about the servers. HAProxy also supports
+full proxies which are exactly the union of a frontend and a backend. When HTTP
+processing is desired, the configuration will generally be split into frontends
+and backends as they open a lot of possibilities since any frontend may pass a
+connection to any backend. With TCP-only proxies, using frontends and backends
+rarely provides a benefit and the configuration can be more readable with full
+proxies.
+
+
+3.3. Basic features
+-------------------
+
+This section will enumerate a number of features that HAProxy implements, some
+of which are generally expected from any modern load balancer, and some of
+which are a direct benefit of HAProxy's architecture. More advanced features
+will be detailed in the next section.
+
+
+3.3.1. Basic features : Proxying
+--------------------------------
+
+Proxying is the action of transferring data between a client and a server over
+two independent connections. The following basic features are supported by
+HAProxy regarding proxying and connection management :
+
+ - Provide the server with a clean connection to protect them against any
+ client-side defect or attack;
+
+ - Listen to multiple IP addresses and/or ports, even port ranges;
+
+ - Transparent accept : intercept traffic targeting any arbitrary IP address
+ that doesn't even belong to the local system;
+
+ - Server port doesn't need to be related to listening port, and may even be
+ translated by a fixed offset (useful with ranges);
+
+ - Transparent connect : spoof the client's (or any) IP address if needed
+ when connecting to the server;
+
+ - Provide a reliable return IP address to the servers in multi-site LBs;
+
+ - Offload the server thanks to buffers and possibly short-lived connections
+ to reduce their concurrent connection count and their memory footprint;
+
+ - Optimize TCP stacks (e.g. SACK), congestion control, and reduce RTT impacts;
+
+ - Support different protocol families on both sides (e.g. IPv4/IPv6/Unix);
+
+ - Timeout enforcement : HAProxy supports multiple levels of timeouts depending
+ on the stage the connection is, so that a dead client or server, or an
+ attacker cannot be granted resources for too long;
+
+ - Protocol validation: HTTP, SSL, or payload are inspected and invalid
+ protocol elements are rejected, unless instructed to accept them anyway;
+
+ - Policy enforcement : ensure that only what is allowed may be forwarded;
+
+ - Both incoming and outgoing connections may be limited to certain network
+ namespaces (Linux only), making it easy to build a cross-container,
+ multi-tenant load balancer;
+
+ - PROXY protocol presents the client's IP address to the server even for
+ non-HTTP traffic. This is an HAProxy extension that was adopted by a number
+ of third-party products by now, at least these ones at the time of writing :
+ - client : haproxy, stud, stunnel, exaproxy, ELB, squid
+ - server : haproxy, stud, postfix, exim, nginx, squid, node.js, varnish
+
+
+3.3.2. Basic features : SSL
+---------------------------
+
+HAProxy's SSL stack is recognized as one of the most featureful according to
+Google's engineers (http://istlsfastyet.com/). The most commonly used features
+making it quite complete are :
+
+ - SNI-based multi-hosting with no limit on sites count and focus on
+ performance. At least one deployment is known for running 50000 domains
+ with their respective certificates;
+
+ - support for wildcard certificates reduces the need for many certificates ;
+
+ - certificate-based client authentication with configurable policies on
+ failure to present a valid certificate. This allows to present a different
+ server farm to regenerate the client certificate for example;
+
+ - authentication of the backend server ensures the backend server is the real
+ one and not a man in the middle;
+
+ - authentication with the backend server lets the backend server know it's
+ really the expected haproxy node that is connecting to it;
+
+ - TLS NPN and ALPN extensions make it possible to reliably offload SPDY/HTTP2
+ connections and pass them in clear text to backend servers;
+
+ - OCSP stapling further reduces first page load time by delivering inline an
+ OCSP response when the client requests a Certificate Status Request;
+
+ - Dynamic record sizing provides both high performance and low latency, and
+ significantly reduces page load time by letting the browser start to fetch
+ new objects while packets are still in flight;
+
+ - permanent access to all relevant SSL/TLS layer information for logging,
+ access control, reporting etc. These elements can be embedded into HTTP
+ header or even as a PROXY protocol extension so that the offloaded server
+ gets all the information it would have had if it performed the SSL
+ termination itself.
+
+ - Detect, log and block certain known attacks even on vulnerable SSL libs,
+ such as the Heartbleed attack affecting certain versions of OpenSSL.
+
+ - support for stateless session resumption (RFC 5077 TLS Ticket extension).
+ TLS tickets can be updated from CLI which provides them means to implement
+ Perfect Forward Secrecy by frequently rotating the tickets.
+
+
+3.3.3. Basic features : Monitoring
+----------------------------------
+
+HAProxy focuses a lot on availability. As such it cares about servers state,
+and about reporting its own state to other network components :
+
+ - Servers' state is continuously monitored using per-server parameters. This
+ ensures the path to the server is operational for regular traffic;
+
+ - Health checks support two hysteresis for up and down transitions in order
+ to protect against state flapping;
+
+ - Checks can be sent to a different address/port/protocol : this makes it
+ easy to check a single service that is considered representative of multiple
+ ones, for example the HTTPS port for an HTTP+HTTPS server.
+
+ - Servers can track other servers and go down simultaneously : this ensures
+ that servers hosting multiple services can fail atomically and that no one
+ will be sent to a partially failed server;
+
+ - Agents may be deployed on the server to monitor load and health : a server
+ may be interested in reporting its load, operational status, administrative
+ status independently from what health checks can see. By running a simple
+ agent on the server, it's possible to consider the server's view of its own
+ health in addition to the health checks validating the whole path;
+
+ - Various check methods are available : TCP connect, HTTP request, SMTP hello,
+ SSL hello, LDAP, SQL, Redis, send/expect scripts, all with/without SSL;
+
+ - State change is notified in the logs and stats page with the failure reason
+ (e.g. the HTTP response received at the moment the failure was detected). An
+ e-mail can also be sent to a configurable address upon such a change ;
+
+ - Server state is also reported on the stats interface and can be used to take
+ routing decisions so that traffic may be sent to different farms depending
+ on their sizes and/or health (e.g. loss of an inter-DC link);
+
+ - HAProxy can use health check requests to pass information to the servers,
+ such as their names, weight, the number of other servers in the farm etc.
+ so that servers can adjust their response and decisions based on this
+ knowledge (e.g. postpone backups to keep more CPU available);
+
+ - Servers can use health checks to report more detailed state than just on/off
+ (e.g. I would like to stop, please stop sending new visitors);
+
+ - HAProxy itself can report its state to external components such as routers
+ or other load balancers, allowing to build very complete multi-path and
+ multi-layer infrastructures.
+
+
+3.3.4. Basic features : High availability
+-----------------------------------------
+
+Just like any serious load balancer, HAProxy cares a lot about availability to
+ensure the best global service continuity :
+
+ - Only valid servers are used ; the other ones are automatically evicted from
+ load balancing farms ; under certain conditions it is still possible to
+ force to use them though;
+
+ - Support for a graceful shutdown so that it is possible to take servers out
+ of a farm without affecting any connection;
+
+ - Backup servers are automatically used when active servers are down and
+ replace them so that sessions are not lost when possible. This also allows
+ to build multiple paths to reach the same server (e.g. multiple interfaces);
+
+ - Ability to return a global failed status for a farm when too many servers
+ are down. This, combined with the monitoring capabilities makes it possible
+ for an upstream component to choose a different LB node for a given service;
+
+ - Stateless design makes it easy to build clusters : by design, HAProxy does
+ its best to ensure the highest service continuity without having to store
+ information that could be lost in the event of a failure. This ensures that
+ a takeover is the most seamless possible;
+
+ - Integrates well with standard VRRP daemon keepalived : HAProxy easily tells
+ keepalived about its state and copes very well with floating virtual IP
+ addresses. Note: only use IP redundancy protocols (VRRP/CARP) over cluster-
+ based solutions (Heartbeat, ...) as they're the ones offering the fastest,
+ most seamless, and most reliable switchover.
+
+
+3.3.5. Basic features : Load balancing
+--------------------------------------
+
+HAProxy offers a fairly complete set of load balancing features, most of which
+are unfortunately not available in a number of other load balancing products :
+
+ - no less than 10 load balancing algorithms are supported, some of which apply
+ to input data to offer an infinite list of possibilities. The most common
+ ones are round-robin (for short connections, pick each server in turn),
+ leastconn (for long connections, pick the least recently used of the servers
+ with the lowest connection count), source (for SSL farms or terminal server
+ farms, the server directly depends on the client's source address), URI (for
+ HTTP caches, the server directly depends on the HTTP URI), hdr (the server
+ directly depends on the contents of a specific HTTP header field), first
+ (for short-lived virtual machines, all connections are packed on the
+ smallest possible subset of servers so that unused ones can be powered
+ down);
+
+ - all algorithms above support per-server weights so that it is possible to
+ accommodate from different server generations in a farm, or direct a small
+ fraction of the traffic to specific servers (debug mode, running the next
+ version of the software, etc);
+
+ - dynamic weights are supported for round-robin, leastconn and consistent
+ hashing ; this allows server weights to be modified on the fly from the CLI
+ or even by an agent running on the server;
+
+ - slow-start is supported whenever a dynamic weight is supported; this allows
+ a server to progressively take the traffic. This is an important feature
+ for fragile application servers which require to compile classes at runtime
+ as well as cold caches which need to fill up before being run at full
+ throttle;
+
+ - hashing can apply to various elements such as client's source address, URL
+ components, query string element, header field values, POST parameter, RDP
+ cookie;
+
+ - consistent hashing protects server farms against massive redistribution when
+ adding or removing servers in a farm. That's very important in large cache
+ farms and it allows slow-start to be used to refill cold caches;
+
+ - a number of internal metrics such as the number of connections per server,
+ per backend, the amount of available connection slots in a backend etc makes
+ it possible to build very advanced load balancing strategies.
+
+
+3.3.6. Basic features : Stickiness
+----------------------------------
+
+Application load balancing would be useless without stickiness. HAProxy provides
+a fairly comprehensive set of possibilities to maintain a visitor on the same
+server even across various events such as server addition/removal, down/up
+cycles, and some methods are designed to be resistant to the distance between
+multiple load balancing nodes in that they don't require any replication :
+
+ - stickiness information can be individually matched and learned from
+ different places if desired. For example a JSESSIONID cookie may be matched
+ both in a cookie and in the URL. Up to 8 parallel sources can be learned at
+ the same time and each of them may point to a different stick-table;
+
+ - stickiness information can come from anything that can be seen within a
+ request or response, including source address, TCP payload offset and
+ length, HTTP query string elements, header field values, cookies, and so
+ on.
+
+ - stick-tables are replicated between all nodes in a multi-master fashion;
+
+ - commonly used elements such as SSL-ID or RDP cookies (for TSE farms) are
+ directly accessible to ease manipulation;
+
+ - all sticking rules may be dynamically conditioned by ACLs;
+
+ - it is possible to decide not to stick to certain servers, such as backup
+ servers, so that when the nominal server comes back, it automatically takes
+ the load back. This is often used in multi-path environments;
+
+ - in HTTP it is often preferred not to learn anything and instead manipulate
+ a cookie dedicated to stickiness. For this, it's possible to detect,
+ rewrite, insert or prefix such a cookie to let the client remember what
+ server was assigned;
+
+ - the server may decide to change or clean the stickiness cookie on logout,
+ so that leaving visitors are automatically unbound from the server;
+
+ - using ACL-based rules it is also possible to selectively ignore or enforce
+ stickiness regardless of the server's state; combined with advanced health
+ checks, that helps admins verify that the server they're installing is up
+ and running before presenting it to the whole world;
+
+ - an innovative mechanism to set a maximum idle time and duration on cookies
+ ensures that stickiness can be smoothly stopped on devices which are never
+ closed (smartphones, TVs, home appliances) without having to store them on
+ persistent storage;
+
+ - multiple server entries may share the same stickiness keys so that
+ stickiness is not lost in multi-path environments when one path goes down;
+
+ - soft-stop ensures that only users with stickiness information will continue
+ to reach the server they've been assigned to but no new users will go there.
+
+
+3.3.7. Basic features : Logging
+-------------------------------
+
+Logging is an extremely important feature for a load balancer, first because a
+load balancer is often wrongly accused of causing the problems it reveals, and
+second because it is placed at a critical point in an infrastructure where all
+normal and abnormal activity needs to be analyzed and correlated with other
+components.
+
+HAProxy provides very detailed logs, with millisecond accuracy and the exact
+connection accept time that can be searched in firewalls logs (e.g. for NAT
+correlation). By default, TCP and HTTP logs are quite detailed and contain
+everything needed for troubleshooting, such as source IP address and port,
+frontend, backend, server, timers (request receipt duration, queue duration,
+connection setup time, response headers time, data transfer time), global
+process state, connection counts, queue status, retries count, detailed
+stickiness actions and disconnect reasons, header captures with a safe output
+encoding. It is then possible to extend or replace this format to include any
+sampled data, variables, captures, resulting in very detailed information. For
+example it is possible to log the number of cumulative requests or number of
+different URLs visited by a client.
+
+The log level may be adjusted per request using standard ACLs, so it is possible
+to automatically silent some logs considered as pollution and instead raise
+warnings when some abnormal behavior happen for a small part of the traffic
+(e.g. too many URLs or HTTP errors for a source address). Administrative logs
+are also emitted with their own levels to inform about the loss or recovery of a
+server for example.
+
+Each frontend and backend may use multiple independent log outputs, which eases
+multi-tenancy. Logs are preferably sent over UDP, maybe JSON-encoded, and are
+truncated after a configurable line length in order to guarantee delivery. But
+it is also possible to send them to stdout/stderr or any file descriptor, as
+well as to a ring buffer that a client can subscribe to in order to retrieve
+them.
+
+
+3.3.8. Basic features : Statistics
+----------------------------------
+
+HAProxy provides a web-based statistics reporting interface with authentication,
+security levels and scopes. It is thus possible to provide each hosted customer
+with his own page showing only his own instances. This page can be located in a
+hidden URL part of the regular web site so that no new port needs to be opened.
+This page may also report the availability of other HAProxy nodes so that it is
+easy to spot if everything works as expected at a glance. The view is synthetic
+with a lot of details accessible (such as error causes, last access and last
+change duration, etc), which are also accessible as a CSV table that other tools
+may import to draw graphs. The page may self-refresh to be used as a monitoring
+page on a large display. In administration mode, the page also allows to change
+server state to ease maintenance operations.
+
+A Prometheus exporter is also provided so that the statistics can be consumed
+in a different format depending on the deployment.
+
+
+3.4. Standard features
+----------------------
+
+In this section, some features that are very commonly used in HAProxy but are
+not necessarily present on other load balancers are enumerated.
+
+
+3.4.1. Standard features : Sampling and converting information
+--------------------------------------------------------------
+
+HAProxy supports information sampling using a wide set of "sample fetch
+functions". The principle is to extract pieces of information known as samples,
+for immediate use. This is used for stickiness, to build conditions, to produce
+information in logs or to enrich HTTP headers.
+
+Samples can be fetched from various sources :
+
+ - constants : integers, strings, IP addresses, binary blocks;
+
+ - the process : date, environment variables, server/frontend/backend/process
+ state, byte/connection counts/rates, queue length, random generator, ...
+
+ - variables : per-session, per-request, per-response variables;
+
+ - the client connection : source and destination addresses and ports, and all
+ related statistics counters;
+
+ - the SSL client session : protocol, version, algorithm, cipher, key size,
+ session ID, all client and server certificate fields, certificate serial,
+ SNI, ALPN, NPN, client support for certain extensions;
+
+ - request and response buffers contents : arbitrary payload at offset/length,
+ data length, RDP cookie, decoding of SSL hello type, decoding of TLS SNI;
+
+ - HTTP (request and response) : method, URI, path, query string arguments,
+ status code, headers values, positional header value, cookies, captures,
+ authentication, body elements;
+
+A sample may then pass through a number of operators known as "converters" to
+experience some transformation. A converter consumes a sample and produces a
+new one, possibly of a completely different type. For example, a converter may
+be used to return only the integer length of the input string, or could turn a
+string to upper case. Any arbitrary number of converters may be applied in
+series to a sample before final use. Among all available sample converters, the
+following ones are the most commonly used :
+
+ - arithmetic and logic operators : they make it possible to perform advanced
+ computation on input data, such as computing ratios, percentages or simply
+ converting from one unit to another one;
+
+ - IP address masks are useful when some addresses need to be grouped by larger
+ networks;
+
+ - data representation : URL-decode, base64, hex, JSON strings, hashing;
+
+ - string conversion : extract substrings at fixed positions, fixed length,
+ extract specific fields around certain delimiters, extract certain words,
+ change case, apply regex-based substitution;
+
+ - date conversion : convert to HTTP date format, convert local to UTC and
+ conversely, add or remove offset;
+
+ - lookup an entry in a stick table to find statistics or assigned server;
+
+ - map-based key-to-value conversion from a file (mostly used for geolocation).
+
+
+3.4.2. Standard features : Maps
+-------------------------------
+
+Maps are a powerful type of converter consisting in loading a two-columns file
+into memory at boot time, then looking up each input sample from the first
+column and either returning the corresponding pattern on the second column if
+the entry was found, or returning a default value. The output information also
+being a sample, it can in turn experience other transformations including other
+map lookups. Maps are most commonly used to translate the client's IP address
+to an AS number or country code since they support a longest match for network
+addresses but they can be used for various other purposes.
+
+Part of their strength comes from being updatable on the fly either from the CLI
+or from certain actions using other samples, making them capable of storing and
+retrieving information between subsequent accesses. Another strength comes from
+the binary tree based indexation which makes them extremely fast even when they
+contain hundreds of thousands of entries, making geolocation very cheap and easy
+to set up.
+
+
+3.4.3. Standard features : ACLs and conditions
+----------------------------------------------
+
+Most operations in HAProxy can be made conditional. Conditions are built by
+combining multiple ACLs using logic operators (AND, OR, NOT). Each ACL is a
+series of tests based on the following elements :
+
+ - a sample fetch method to retrieve the element to test ;
+
+ - an optional series of converters to transform the element ;
+
+ - a list of patterns to match against ;
+
+ - a matching method to indicate how to compare the patterns with the sample
+
+For example, the sample may be taken from the HTTP "Host" header, it could then
+be converted to lower case, then matched against a number of regex patterns
+using the regex matching method.
+
+Technically, ACLs are built on the same core as the maps, they share the exact
+same internal structure, pattern matching methods and performance. The only real
+difference is that instead of returning a sample, they only return "found" or
+or "not found". In terms of usage, ACL patterns may be declared inline in the
+configuration file and do not require their own file. ACLs may be named for ease
+of use or to make configurations understandable. A named ACL may be declared
+multiple times and it will evaluate all definitions in turn until one matches.
+
+About 13 different pattern matching methods are provided, among which IP address
+mask, integer ranges, substrings, regex. They work like functions, and just like
+with any programming language, only what is needed is evaluated, so when a
+condition involving an OR is already true, next ones are not evaluated, and
+similarly when a condition involving an AND is already false, the rest of the
+condition is not evaluated.
+
+There is no practical limit to the number of declared ACLs, and a handful of
+commonly used ones are provided. However experience has shown that setups using
+a lot of named ACLs are quite hard to troubleshoot and that sometimes using
+anonymous ACLs inline is easier as it requires less references out of the scope
+being analyzed.
+
+
+3.4.4. Standard features : Content switching
+--------------------------------------------
+
+HAProxy implements a mechanism known as content-based switching. The principle
+is that a connection or request arrives on a frontend, then the information
+carried with this request or connection are processed, and at this point it is
+possible to write ACLs-based conditions making use of these information to
+decide what backend will process the request. Thus the traffic is directed to
+one backend or another based on the request's contents. The most common example
+consists in using the Host header and/or elements from the path (sub-directories
+or file-name extensions) to decide whether an HTTP request targets a static
+object or the application, and to route static objects traffic to a backend made
+of fast and light servers, and all the remaining traffic to a more complex
+application server, thus constituting a fine-grained virtual hosting solution.
+This is quite convenient to make multiple technologies coexist as a more global
+solution.
+
+Another use case of content-switching consists in using different load balancing
+algorithms depending on various criteria. A cache may use a URI hash while an
+application would use round-robin.
+
+Last but not least, it allows multiple customers to use a small share of a
+common resource by enforcing per-backend (thus per-customer connection limits).
+
+Content switching rules scale very well, though their performance may depend on
+the number and complexity of the ACLs in use. But it is also possible to write
+dynamic content switching rules where a sample value directly turns into a
+backend name and without making use of ACLs at all. Such configurations have
+been reported to work fine at least with 300000 backends in production.
+
+
+3.4.5. Standard features : Stick-tables
+---------------------------------------
+
+Stick-tables are commonly used to store stickiness information, that is, to keep
+a reference to the server a certain visitor was directed to. The key is then the
+identifier associated with the visitor (its source address, the SSL ID of the
+connection, an HTTP or RDP cookie, the customer number extracted from the URL or
+from the payload, ...) and the stored value is then the server's identifier.
+
+Stick tables may use 3 different types of samples for their keys : integers,
+strings and addresses. Only one stick-table may be referenced in a proxy, and it
+is designated everywhere with the proxy name. Up to 8 keys may be tracked in
+parallel. The server identifier is committed during request or response
+processing once both the key and the server are known.
+
+Stick-table contents may be replicated in active-active mode with other HAProxy
+nodes known as "peers" as well as with the new process during a reload operation
+so that all load balancing nodes share the same information and take the same
+routing decision if client's requests are spread over multiple nodes.
+
+Since stick-tables are indexed on what allows to recognize a client, they are
+often also used to store extra information such as per-client statistics. The
+extra statistics take some extra space and need to be explicitly declared. The
+type of statistics that may be stored includes the input and output bandwidth,
+the number of concurrent connections, the connection rate and count over a
+period, the amount and frequency of errors, some specific tags and counters,
+etc. In order to support keeping such information without being forced to
+stick to a given server, a special "tracking" feature is implemented and allows
+to track up to 3 simultaneous keys from different tables at the same time
+regardless of stickiness rules. Each stored statistics may be searched, dumped
+and cleared from the CLI and adds to the live troubleshooting capabilities.
+
+While this mechanism can be used to surclass a returning visitor or to adjust
+the delivered quality of service depending on good or bad behavior, it is
+mostly used to fight against service abuse and more generally DDoS as it allows
+to build complex models to detect certain bad behaviors at a high processing
+speed.
+
+
+3.4.6. Standard features : Formatted strings
+--------------------------------------------
+
+There are many places where HAProxy needs to manipulate character strings, such
+as logs, redirects, header additions, and so on. In order to provide the
+greatest flexibility, the notion of Formatted strings was introduced, initially
+for logging purposes, which explains why it's still called "log-format". These
+strings contain escape characters allowing to introduce various dynamic data
+including variables and sample fetch expressions into strings, and even to
+adjust the encoding while the result is being turned into a string (for example,
+adding quotes). This provides a powerful way to build header contents, to build
+response data or even response templates, or to customize log lines.
+Additionally, in order to remain simple to build most common strings, about 50
+special tags are provided as shortcuts for information commonly used in logs.
+
+
+3.4.7. Standard features : HTTP rewriting and redirection
+---------------------------------------------------------
+
+Installing a load balancer in front of an application that was never designed
+for this can be a challenging task without the proper tools. One of the most
+commonly requested operation in this case is to adjust requests and response
+headers to make the load balancer appear as the origin server and to fix hard
+coded information. This comes with changing the path in requests (which is
+strongly advised against), modifying Host header field, modifying the Location
+response header field for redirects, modifying the path and domain attribute
+for cookies, and so on. It also happens that a number of servers are somewhat
+verbose and tend to leak too much information in the response, making them more
+vulnerable to targeted attacks. While it's theoretically not the role of a load
+balancer to clean this up, in practice it's located at the best place in the
+infrastructure to guarantee that everything is cleaned up.
+
+Similarly, sometimes the load balancer will have to intercept some requests and
+respond with a redirect to a new target URL. While some people tend to confuse
+redirects and rewriting, these are two completely different concepts, since the
+rewriting makes the client and the server see different things (and disagree on
+the location of the page being visited) while redirects ask the client to visit
+the new URL so that it sees the same location as the server.
+
+In order to do this, HAProxy supports various possibilities for rewriting and
+redirects, among which :
+
+ - regex-based URL and header rewriting in requests and responses. Regex are
+ the most commonly used tool to modify header values since they're easy to
+ manipulate and well understood;
+
+ - headers may also be appended, deleted or replaced based on formatted strings
+ so that it is possible to pass information there (e.g. client side TLS
+ algorithm and cipher);
+
+ - HTTP redirects can use any 3xx code to a relative, absolute, or completely
+ dynamic (formatted string) URI;
+
+ - HTTP redirects also support some extra options such as setting or clearing
+ a specific cookie, dropping the query string, appending a slash if missing,
+ and so on;
+
+ - a powerful "return" directive allows to customize every part of a response
+ like status, headers, body using dynamic contents or even template files.
+
+ - all operations support ACL-based conditions;
+
+
+3.4.8. Standard features : Server protection
+--------------------------------------------
+
+HAProxy does a lot to maximize service availability, and for this it takes
+large efforts to protect servers against overloading and attacks. The first
+and most important point is that only complete and valid requests are forwarded
+to the servers. The initial reason is that HAProxy needs to find the protocol
+elements it needs to stay synchronized with the byte stream, and the second
+reason is that until the request is complete, there is no way to know if some
+elements will change its semantics. The direct benefit from this is that servers
+are not exposed to invalid or incomplete requests. This is a very effective
+protection against slowloris attacks, which have almost no impact on HAProxy.
+
+Another important point is that HAProxy contains buffers to store requests and
+responses, and that by only sending a request to a server when it's complete and
+by reading the whole response very quickly from the local network, the server
+side connection is used for a very short time and this preserves server
+resources as much as possible.
+
+A direct extension to this is that HAProxy can artificially limit the number of
+concurrent connections or outstanding requests to a server, which guarantees
+that the server will never be overloaded even if it continuously runs at 100% of
+its capacity during traffic spikes. All excess requests will simply be queued to
+be processed when one slot is released. In the end, this huge resource savings
+most often ensures so much better server response times that it ends up actually
+being faster than by overloading the server. Queued requests may be redispatched
+to other servers, or even aborted in queue when the client aborts, which also
+protects the servers against the "reload effect", where each click on "reload"
+by a visitor on a slow-loading page usually induces a new request and maintains
+the server in an overloaded state.
+
+The slow-start mechanism also protects restarting servers against high traffic
+levels while they're still finalizing their startup or compiling some classes.
+
+Regarding the protocol-level protection, it is possible to relax the HTTP parser
+to accept non standard-compliant but harmless requests or responses and even to
+fix them. This allows bogus applications to be accessible while a fix is being
+developed. In parallel, offending messages are completely captured with a
+detailed report that help developers spot the issue in the application. The most
+dangerous protocol violations are properly detected and dealt with and fixed.
+For example malformed requests or responses with two Content-length headers are
+either fixed if the values are exactly the same, or rejected if they differ,
+since it becomes a security problem. Protocol inspection is not limited to HTTP,
+it is also available for other protocols like TLS or RDP.
+
+When a protocol violation or attack is detected, there are various options to
+respond to the user, such as returning the common "HTTP 400 bad request",
+closing the connection with a TCP reset, or faking an error after a long delay
+("tarpit") to confuse the attacker. All of these contribute to protecting the
+servers by discouraging the offending client from pursuing an attack that
+becomes very expensive to maintain.
+
+HAProxy also proposes some more advanced options to protect against accidental
+data leaks and session crossing. Not only it can log suspicious server responses
+but it will also log and optionally block a response which might affect a given
+visitors' confidentiality. One such example is a cacheable cookie appearing in a
+cacheable response and which may result in an intermediary cache to deliver it
+to another visitor, causing an accidental session sharing.
+
+
+3.5. Advanced features
+----------------------
+
+3.5.1. Advanced features : Management
+-------------------------------------
+
+HAProxy is designed to remain extremely stable and safe to manage in a regular
+production environment. It is provided as a single executable file which doesn't
+require any installation process. Multiple versions can easily coexist, meaning
+that it's possible (and recommended) to upgrade instances progressively by
+order of importance instead of migrating all of them at once. Configuration
+files are easily versioned. Configuration checking is done off-line so it
+doesn't require to restart a service that will possibly fail. During
+configuration checks, a number of advanced mistakes may be detected (e.g. a rule
+hiding another one, or stickiness that will not work) and detailed warnings and
+configuration hints are proposed to fix them. Backwards configuration file
+compatibility goes very far away in time, with version 1.5 still fully
+supporting configurations for versions 1.1 written 13 years before, and 1.6
+only dropping support for almost unused, obsolete keywords that can be done
+differently. The configuration and software upgrade mechanism is smooth and non
+disruptive in that it allows old and new processes to coexist on the system,
+each handling its own connections. System status, build options, and library
+compatibility are reported on startup.
+
+Some advanced features allow an application administrator to smoothly stop a
+server, detect when there's no activity on it anymore, then take it off-line,
+stop it, upgrade it and ensure it doesn't take any traffic while being upgraded,
+then test it again through the normal path without opening it to the public, and
+all of this without touching HAProxy at all. This ensures that even complicated
+production operations may be done during opening hours with all technical
+resources available.
+
+The process tries to save resources as much as possible, uses memory pools to
+save on allocation time and limit memory fragmentation, releases payload buffers
+as soon as their contents are sent, and supports enforcing strong memory limits
+above which connections have to wait for a buffer to become available instead of
+allocating more memory. This system helps guarantee memory usage in certain
+strict environments.
+
+A command line interface (CLI) is available as a UNIX or TCP socket, to perform
+a number of operations and to retrieve troubleshooting information. Everything
+done on this socket doesn't require a configuration change, so it is mostly used
+for temporary changes. Using this interface it is possible to change a server's
+address, weight and status, to consult statistics and clear counters, dump and
+clear stickiness tables, possibly selectively by key criteria, dump and kill
+client-side and server-side connections, dump captured errors with a detailed
+analysis of the exact cause and location of the error, dump, add and remove
+entries from ACLs and maps, update TLS shared secrets, apply connection limits
+and rate limits on the fly to arbitrary frontends (useful in shared hosting
+environments), and disable a specific frontend to release a listening port
+(useful when daytime operations are forbidden and a fix is needed nonetheless).
+Updating certificates and their configuration on the fly is permitted, as well
+as enabling and consulting traces of every processing step of the traffic.
+
+For environments where SNMP is mandatory, at least two agents exist, one is
+provided with the HAProxy sources and relies on the Net-SNMP Perl module.
+Another one is provided with the commercial packages and doesn't require Perl.
+Both are roughly equivalent in terms of coverage.
+
+It is often recommended to install 4 utilities on the machine where HAProxy is
+deployed :
+
+ - socat (in order to connect to the CLI, though certain forks of netcat can
+ also do it to some extents);
+
+ - halog from the latest HAProxy version : this is the log analysis tool, it
+ parses native TCP and HTTP logs extremely fast (1 to 2 GB per second) and
+ extracts useful information and statistics such as requests per URL, per
+ source address, URLs sorted by response time or error rate, termination
+ codes etc. It was designed to be deployed on the production servers to
+ help troubleshoot live issues so it has to be there ready to be used;
+
+ - tcpdump : this is highly recommended to take the network traces needed to
+ troubleshoot an issue that was made visible in the logs. There is a moment
+ where application and haproxy's analysis will diverge and the network traces
+ are the only way to say who's right and who's wrong. It's also fairly common
+ to detect bugs in network stacks and hypervisors thanks to tcpdump;
+
+ - strace : it is tcpdump's companion. It will report what HAProxy really sees
+ and will help sort out the issues the operating system is responsible for
+ from the ones HAProxy is responsible for. Strace is often requested when a
+ bug in HAProxy is suspected;
+
+
+3.5.2. Advanced features : System-specific capabilities
+-------------------------------------------------------
+
+Depending on the operating system HAProxy is deployed on, certain extra features
+may be available or needed. While it is supported on a number of platforms,
+HAProxy is primarily developed on Linux, which explains why some features are
+only available on this platform.
+
+The transparent bind and connect features, the support for binding connections
+to a specific network interface, as well as the ability to bind multiple
+processes to the same IP address and ports are only available on Linux and BSD
+systems, though only Linux performs a kernel-side load balancing of the incoming
+requests between the available processes.
+
+On Linux, there are also a number of extra features and optimizations including
+support for network namespaces (also known as "containers") allowing HAProxy to
+be a gateway between all containers, the ability to set the MSS, Netfilter marks
+and IP TOS field on the client side connection, support for TCP FastOpen on the
+listening side, TCP user timeouts to let the kernel quickly kill connections
+when it detects the client has disappeared before the configured timeouts, TCP
+splicing to let the kernel forward data between the two sides of a connections
+thus avoiding multiple memory copies, the ability to enable the "defer-accept"
+bind option to only get notified of an incoming connection once data become
+available in the kernel buffers, and the ability to send the request with the
+ACK confirming a connect (sometimes called "piggy-back") which is enabled with
+the "tcp-smart-connect" option. On Linux, HAProxy also takes great care of
+manipulating the TCP delayed ACKs to save as many packets as possible on the
+network.
+
+Some systems have an unreliable clock which jumps back and forth in the past
+and in the future. This used to happen with some NUMA systems where multiple
+processors didn't see the exact same time of day, and recently it became more
+common in virtualized environments where the virtual clock has no relation with
+the real clock, resulting in huge time jumps (sometimes up to 30 seconds have
+been observed). This causes a lot of trouble with respect to timeout enforcement
+in general. Due to this flaw of these systems, HAProxy maintains its own
+monotonic clock which is based on the system's clock but where drift is measured
+and compensated for. This ensures that even with a very bad system clock, timers
+remain reasonably accurate and timeouts continue to work. Note that this problem
+affects all the software running on such systems and is not specific to HAProxy.
+The common effects are spurious timeouts or application freezes. Thus if this
+behavior is detected on a system, it must be fixed, regardless of the fact that
+HAProxy protects itself against it.
+
+On Linux, a new starting process may communicate with the previous one to reuse
+its listening file descriptors so that the listening sockets are never
+interrupted during the process's replacement.
+
+
+3.5.3. Advanced features : Scripting
+------------------------------------
+
+HAProxy can be built with support for the Lua embedded language, which opens a
+wide area of new possibilities related to complex manipulation of requests or
+responses, routing decisions, statistics processing and so on. Using Lua it is
+even possible to establish parallel connections to other servers to exchange
+information. This way it becomes possible (though complex) to develop an
+authentication system for example. Please refer to the documentation in the file
+"doc/lua-api/index.rst" for more information on how to use Lua.
+
+
+3.5.4. Advanced features: Tracing
+---------------------------------
+
+At any moment an administrator may connect over the CLI and enable tracing in
+various internal subsystems. Various levels of details are provided by default
+so that in practice anything between one line per request to 500 lines per
+request can be retrieved. Filters as well as an automatic capture on/off/pause
+mechanism are available so that it really is possible to wait for a certain
+event and watch it in detail. This is extremely convenient to diagnose protocol
+violations from faulty servers and clients, or denial of service attacks.
+
+
+3.6. Sizing
+-----------
+
+Typical CPU usage figures show 15% of the processing time spent in HAProxy
+versus 85% in the kernel in TCP or HTTP close mode, and about 30% for HAProxy
+versus 70% for the kernel in HTTP keep-alive mode. This means that the operating
+system and its tuning have a strong impact on the global performance.
+
+Usages vary a lot between users, some focus on bandwidth, other ones on request
+rate, others on connection concurrency, others on SSL performance. This section
+aims at providing a few elements to help with this task.
+
+It is important to keep in mind that every operation comes with a cost, so each
+individual operation adds its overhead on top of the other ones, which may be
+negligible in certain circumstances, and which may dominate in other cases.
+
+When processing the requests from a connection, we can say that :
+
+ - forwarding data costs less than parsing request or response headers;
+
+ - parsing request or response headers cost less than establishing then closing
+ a connection to a server;
+
+ - establishing an closing a connection costs less than a TLS resume operation;
+
+ - a TLS resume operation costs less than a full TLS handshake with a key
+ computation;
+
+ - an idle connection costs less CPU than a connection whose buffers hold data;
+
+ - a TLS context costs even more memory than a connection with data;
+
+So in practice, it is cheaper to process payload bytes than header bytes, thus
+it is easier to achieve high network bandwidth with large objects (few requests
+per volume unit) than with small objects (many requests per volume unit). This
+explains why maximum bandwidth is always measured with large objects, while
+request rate or connection rates are measured with small objects.
+
+Some operations scale well on multiple processes spread over multiple CPUs,
+and others don't scale as well. Network bandwidth doesn't scale very far because
+the CPU is rarely the bottleneck for large objects, it's mostly the network
+bandwidth and data buses to reach the network interfaces. The connection rate
+doesn't scale well over multiple processors due to a few locks in the system
+when dealing with the local ports table. The request rate over persistent
+connections scales very well as it doesn't involve much memory nor network
+bandwidth and doesn't require to access locked structures. TLS key computation
+scales very well as it's totally CPU-bound. TLS resume scales moderately well,
+but reaches its limits around 4 processes where the overhead of accessing the
+shared table offsets the small gains expected from more power.
+
+The performance numbers one can expect from a very well tuned system are in the
+following range. It is important to take them as orders of magnitude and to
+expect significant variations in any direction based on the processor, IRQ
+setting, memory type, network interface type, operating system tuning and so on.
+
+The following numbers were found on a Core i7 running at 3.7 GHz equipped with
+a dual-port 10 Gbps NICs running Linux kernel 3.10, HAProxy 1.6 and OpenSSL
+1.0.2. HAProxy was running as a single process on a single dedicated CPU core,
+and two extra cores were dedicated to network interrupts :
+
+ - 20 Gbps of maximum network bandwidth in clear text for objects 256 kB or
+ higher, 10 Gbps for 41kB or higher;
+
+ - 4.6 Gbps of TLS traffic using AES256-GCM cipher with large objects;
+
+ - 83000 TCP connections per second from client to server;
+
+ - 82000 HTTP connections per second from client to server;
+
+ - 97000 HTTP requests per second in server-close mode (keep-alive with the
+ client, close with the server);
+
+ - 243000 HTTP requests per second in end-to-end keep-alive mode;
+
+ - 300000 filtered TCP connections per second (anti-DDoS)
+
+ - 160000 HTTPS requests per second in keep-alive mode over persistent TLS
+ connections;
+
+ - 13100 HTTPS requests per second using TLS resumed connections;
+
+ - 1300 HTTPS connections per second using TLS connections renegotiated with
+ RSA2048;
+
+ - 20000 concurrent saturated connections per GB of RAM, including the memory
+ required for system buffers; it is possible to do better with careful tuning
+ but this result it easy to achieve.
+
+ - about 8000 concurrent TLS connections (client-side only) per GB of RAM,
+ including the memory required for system buffers;
+
+ - about 5000 concurrent end-to-end TLS connections (both sides) per GB of
+ RAM including the memory required for system buffers;
+
+A more recent benchmark featuring the multi-thread enabled HAProxy 2.4 on a
+64-core ARM Graviton2 processor in AWS reached 2 million HTTPS requests per
+second at sub-millisecond response time, and 100 Gbps of traffic:
+
+ https://www.haproxy.com/blog/haproxy-forwards-over-2-million-http-requests-per-second-on-a-single-aws-arm-instance/
+
+Thus a good rule of thumb to keep in mind is that the request rate is divided
+by 10 between TLS keep-alive and TLS resume, and between TLS resume and TLS
+renegotiation, while it's only divided by 3 between HTTP keep-alive and HTTP
+close. Another good rule of thumb is to remember that a high frequency core
+with AES instructions can do around 20 Gbps of AES-GCM per core.
+
+Another good rule of thumb is to consider that on the same server, HAProxy will
+be able to saturate :
+
+ - about 5-10 static file servers or caching proxies;
+
+ - about 100 anti-virus proxies;
+
+ - and about 100-1000 application servers depending on the technology in use.
+
+
+3.7. How to get HAProxy
+-----------------------
+
+HAProxy is an open source project covered by the GPLv2 license, meaning that
+everyone is allowed to redistribute it provided that access to the sources is
+also provided upon request, especially if any modifications were made.
+
+HAProxy evolves as a main development branch called "master" or "mainline", from
+which new branches are derived once the code is considered stable. A lot of web
+sites run some development branches in production on a voluntarily basis, either
+to participate to the project or because they need a bleeding edge feature, and
+their feedback is highly valuable to fix bugs and judge the overall quality and
+stability of the version being developed.
+
+The new branches that are created when the code is stable enough constitute a
+stable version and are generally maintained for several years, so that there is
+no emergency to migrate to a newer branch even when you're not on the latest.
+Once a stable branch is issued, it may only receive bug fixes, and very rarely
+minor feature updates when that makes users' life easier. All fixes that go into
+a stable branch necessarily come from the master branch. This guarantees that no
+fix will be lost after an upgrade. For this reason, if you fix a bug, please
+make the patch against the master branch, not the stable branch. You may even
+discover it was already fixed. This process also ensures that regressions in a
+stable branch are extremely rare, so there is never any excuse for not upgrading
+to the latest version in your current branch.
+
+Branches are numbered with two digits delimited with a dot, such as "1.6".
+Since 1.9, branches with an odd second digit are mostly focused on sensitive
+technical updates and more aimed at advanced users because they are likely to
+trigger more bugs than the other ones. They are maintained for about a year
+only and must not be deployed where they cannot be rolled back in emergency. A
+complete version includes one or two sub-version numbers indicating the level of
+fix. For example, version 1.5.14 is the 14th fix release in branch 1.5 after
+version 1.5.0 was issued. It contains 126 fixes for individual bugs, 24 updates
+on the documentation, and 75 other backported patches, most of which were needed
+to fix the aforementioned 126 bugs. An existing feature may never be modified
+nor removed in a stable branch, in order to guarantee that upgrades within the
+same branch will always be harmless.
+
+HAProxy is available from multiple sources, at different release rhythms :
+
+ - The official community web site : http://www.haproxy.org/ : this site
+ provides the sources of the latest development release, all stable releases,
+ as well as nightly snapshots for each branch. The release cycle is not fast,
+ several months between stable releases, or between development snapshots.
+ Very old versions are still supported there. Everything is provided as
+ sources only, so whatever comes from there needs to be rebuilt and/or
+ repackaged;
+
+ - GitHub : https://github.com/haproxy/haproxy/ : this is the mirror for the
+ development branch only, which provides integration with the issue tracker,
+ continuous integration and code coverage tools. This is exclusively for
+ contributors;
+
+ - A number of operating systems such as Linux distributions and BSD ports.
+ These systems generally provide long-term maintained versions which do not
+ always contain all the fixes from the official ones, but which at least
+ contain the critical fixes. It often is a good option for most users who do
+ not seek advanced configurations and just want to keep updates easy;
+
+ - Commercial versions from http://www.haproxy.com/ : these are supported
+ professional packages built for various operating systems or provided as
+ appliances, based on the latest stable versions and including a number of
+ features backported from the next release for which there is a strong
+ demand. It is the best option for users seeking the latest features with
+ the reliability of a stable branch, the fastest response time to fix bugs,
+ or simply support contracts on top of an open source product;
+
+
+In order to ensure that the version you're using is the latest one in your
+branch, you need to proceed this way :
+
+ - verify which HAProxy executable you're running : some systems ship it by
+ default and administrators install their versions somewhere else on the
+ system, so it is important to verify in the startup scripts which one is
+ used;
+
+ - determine which source your HAProxy version comes from. For this, it's
+ generally sufficient to type "haproxy -v". A development version will
+ appear like this, with the "dev" word after the branch number :
+
+ HAProxy version 2.4-dev18-a5357c-137 2021/05/09 - https://haproxy.org/
+
+ A stable version will appear like this, as well as unmodified stable
+ versions provided by operating system vendors :
+
+ HAProxy version 1.5.14 2015/07/02
+
+ And a nightly snapshot of a stable version will appear like this with an
+ hexadecimal sequence after the version, and with the date of the snapshot
+ instead of the date of the release :
+
+ HAProxy version 1.5.14-e4766ba 2015/07/29
+
+ Any other format may indicate a system-specific package with its own
+ patch set. For example HAProxy Enterprise versions will appear with the
+ following format (<branch>-<latest commit>-<revision>) :
+
+ HAProxy version 1.5.0-994126-357 2015/07/02
+
+ Please note that historically versions prior to 2.4 used to report the
+ process name with a hyphen between "HA" and "Proxy", including those above
+ which were adjusted to show the correct format only, so better ignore this
+ word or use a relaxed match in scripts. Additionally, modern versions add
+ a URL linking to the project's home.
+
+ Finally, versions 2.1 and above will include a "Status" line indicating
+ whether the version is safe for production or not, and if so, till when, as
+ well as a link to the list of known bugs affecting this version.
+
+ - for system-specific packages, you have to check with your vendor's package
+ repository or update system to ensure that your system is still supported,
+ and that fixes are still provided for your branch. For community versions
+ coming from haproxy.org, just visit the site, verify the status of your
+ branch and compare the latest version with yours to see if you're on the
+ latest one. If not you can upgrade. If your branch is not maintained
+ anymore, you're definitely very late and will have to consider an upgrade
+ to a more recent branch (carefully read the README when doing so).
+
+HAProxy will have to be updated according to the source it came from. Usually it
+follows the system vendor's way of upgrading a package. If it was taken from
+sources, please read the README file in the sources directory after extracting
+the sources and follow the instructions for your operating system.
+
+
+4. Companion products and alternatives
+--------------------------------------
+
+HAProxy integrates fairly well with certain products listed below, which is why
+they are mentioned here even if not directly related to HAProxy.
+
+
+4.1. Apache HTTP server
+-----------------------
+
+Apache is the de-facto standard HTTP server. It's a very complete and modular
+project supporting both file serving and dynamic contents. It can serve as a
+frontend for some application servers. It can even proxy requests and cache
+responses. In all of these use cases, a front load balancer is commonly needed.
+Apache can work in various modes, some being heavier than others. Certain
+modules still require the heavier pre-forked model and will prevent Apache from
+scaling well with a high number of connections. In this case HAProxy can provide
+a tremendous help by enforcing the per-server connection limits to a safe value
+and will significantly speed up the server and preserve its resources that will
+be better used by the application.
+
+Apache can extract the client's address from the X-Forwarded-For header by using
+the "mod_rpaf" extension. HAProxy will automatically feed this header when
+"option forwardfor" is specified in its configuration. HAProxy may also offer a
+nice protection to Apache when exposed to the internet, where it will better
+resist a wide number of types of DoS attacks.
+
+
+4.2. NGINX
+----------
+
+NGINX is the second de-facto standard HTTP server. Just like Apache, it covers a
+wide range of features. NGINX is built on a similar model as HAProxy so it has
+no problem dealing with tens of thousands of concurrent connections. When used
+as a gateway to some applications (e.g. using the included PHP FPM) it can often
+be beneficial to set up some frontend connection limiting to reduce the load
+on the PHP application. HAProxy will clearly be useful there both as a regular
+load balancer and as the traffic regulator to speed up PHP by decongesting
+it. Also since both products use very little CPU thanks to their event-driven
+architecture, it's often easy to install both of them on the same system. NGINX
+implements HAProxy's PROXY protocol, thus it is easy for HAProxy to pass the
+client's connection information to NGINX so that the application gets all the
+relevant information. Some benchmarks have also shown that for large static
+file serving, implementing consistent hash on HAProxy in front of NGINX can be
+beneficial by optimizing the OS' cache hit ratio, which is basically multiplied
+by the number of server nodes.
+
+
+4.3. Varnish
+------------
+
+Varnish is a smart caching reverse-proxy, probably best described as a web
+application accelerator. Varnish doesn't implement SSL/TLS and wants to dedicate
+all of its CPU cycles to what it does best. Varnish also implements HAProxy's
+PROXY protocol so that HAProxy can very easily be deployed in front of Varnish
+as an SSL offloader as well as a load balancer and pass it all relevant client
+information. Also, Varnish naturally supports decompression from the cache when
+a server has provided a compressed object, but doesn't compress however. HAProxy
+can then be used to compress outgoing data when backend servers do not implement
+compression, though it's rarely a good idea to compress on the load balancer
+unless the traffic is low.
+
+When building large caching farms across multiple nodes, HAProxy can make use of
+consistent URL hashing to intelligently distribute the load to the caching nodes
+and avoid cache duplication, resulting in a total cache size which is the sum of
+all caching nodes. In addition, caching of very small dumb objects for a short
+duration on HAProxy can sometimes save network round trips and reduce the CPU
+load on both the HAProxy and the Varnish nodes. This is only possible is no
+processing is done on these objects on Varnish (this is often referred to as
+the notion of "favicon cache", by which a sizeable percentage of useless
+downstream requests can sometimes be avoided). However do not enable HAProxy
+caching for a long time (more than a few seconds) in front of any other cache,
+that would significantly complicate troubleshooting without providing really
+significant savings.
+
+
+4.4. Alternatives
+-----------------
+
+Linux Virtual Server (LVS or IPVS) is the layer 4 load balancer included within
+the Linux kernel. It works at the packet level and handles TCP and UDP. In most
+cases it's more a complement than an alternative since it doesn't have layer 7
+knowledge at all.
+
+Pound is another well-known load balancer. It's much simpler and has much less
+features than HAProxy but for many very basic setups both can be used. Its
+author has always focused on code auditability first and wants to maintain the
+set of features low. Its thread-based architecture scales less well with high
+connection counts, but it's a good product.
+
+Pen is a quite light load balancer. It supports SSL, maintains persistence using
+a fixed-size table of its clients' IP addresses. It supports a packet-oriented
+mode allowing it to support direct server return and UDP to some extents. It is
+meant for small loads (the persistence table only has 2048 entries).
+
+NGINX can do some load balancing to some extents, though it's clearly not its
+primary function. Production traffic is used to detect server failures, the
+load balancing algorithms are more limited, and the stickiness is very limited.
+But it can make sense in some simple deployment scenarios where it is already
+present. The good thing is that since it integrates very well with HAProxy,
+there's nothing wrong with adding HAProxy later when its limits have been
+reached.
+
+Varnish also does some load balancing of its backend servers and does support
+real health checks. It doesn't implement stickiness however, so just like with
+NGINX, as long as stickiness is not needed that can be enough to start with.
+And similarly, since HAProxy and Varnish integrate so well together, it's easy
+to add it later into the mix to complement the feature set.
+
+
+5. Contacts
+-----------
+
+If you want to contact the developers or any community member about anything,
+the best way to do it usually is via the mailing list by sending your message
+to haproxy@formilux.org. Please note that this list is public and its archives
+are public as well so you should avoid disclosing sensitive information. A
+thousand of users of various experience levels are present there and even the
+most complex questions usually find an optimal response relatively quickly.
+Suggestions are welcome too. For users having difficulties with e-mail, a
+Discourse platform is available at http://discourse.haproxy.org/ . However
+please keep in mind that there are less people reading questions there and that
+most are handled by a really tiny team. In any case, please be patient and
+respectful with those who devote their spare time helping others.
+
+I you believe you've found a bug but are not sure, it's best reported on the
+mailing list. If you're quite convinced you've found a bug, that your version
+is up-to-date in its branch, and you already have a GitHub account, feel free
+to go directly to https://github.com/haproxy/haproxy/ and file an issue with
+all possibly available details. Again, this is public so be careful not to post
+information you might later regret. Since the issue tracker presents itself as
+a very long thread, please avoid pasting very long dumps (a few hundreds lines
+or more) and attach them instead.
+
+If you've found what you're absolutely certain can be considered a critical
+security issue that would put many users in serious trouble if discussed in a
+public place, then you can send it with the reproducer to security@haproxy.org.
+A small team of trusted developers will receive it and will be able to propose
+a fix. We usually don't use embargoes and once a fix is available it gets
+merged. In some rare circumstances it can happen that a release is coordinated
+with software vendors. Please note that this process usually messes up with
+eveyone's work, and that rushed up releases can sometimes introduce new bugs,
+so it's best avoided unless strictly necessary; as such, there is often little
+consideration for reports that needlessly cause such extra burden, and the best
+way to see your work credited usually is to provide a working fix, which will
+appear in changelogs.
diff --git a/doc/lgpl.txt b/doc/lgpl.txt
new file mode 100644
index 0000000..5ab7695
--- /dev/null
+++ b/doc/lgpl.txt
@@ -0,0 +1,504 @@
+ GNU LESSER GENERAL PUBLIC LICENSE
+ Version 2.1, February 1999
+
+ Copyright (C) 1991, 1999 Free Software Foundation, Inc.
+ 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+[This is the first released version of the Lesser GPL. It also counts
+ as the successor of the GNU Library Public License, version 2, hence
+ the version number 2.1.]
+
+ Preamble
+
+ The licenses for most software are designed to take away your
+freedom to share and change it. By contrast, the GNU General Public
+Licenses are intended to guarantee your freedom to share and change
+free software--to make sure the software is free for all its users.
+
+ This license, the Lesser General Public License, applies to some
+specially designated software packages--typically libraries--of the
+Free Software Foundation and other authors who decide to use it. You
+can use it too, but we suggest you first think carefully about whether
+this license or the ordinary General Public License is the better
+strategy to use in any particular case, based on the explanations below.
+
+ When we speak of free software, we are referring to freedom of use,
+not price. Our General Public Licenses are designed to make sure that
+you have the freedom to distribute copies of free software (and charge
+for this service if you wish); that you receive source code or can get
+it if you want it; that you can change the software and use pieces of
+it in new free programs; and that you are informed that you can do
+these things.
+
+ To protect your rights, we need to make restrictions that forbid
+distributors to deny you these rights or to ask you to surrender these
+rights. These restrictions translate to certain responsibilities for
+you if you distribute copies of the library or if you modify it.
+
+ For example, if you distribute copies of the library, whether gratis
+or for a fee, you must give the recipients all the rights that we gave
+you. You must make sure that they, too, receive or can get the source
+code. If you link other code with the library, you must provide
+complete object files to the recipients, so that they can relink them
+with the library after making changes to the library and recompiling
+it. And you must show them these terms so they know their rights.
+
+ We protect your rights with a two-step method: (1) we copyright the
+library, and (2) we offer you this license, which gives you legal
+permission to copy, distribute and/or modify the library.
+
+ To protect each distributor, we want to make it very clear that
+there is no warranty for the free library. Also, if the library is
+modified by someone else and passed on, the recipients should know
+that what they have is not the original version, so that the original
+author's reputation will not be affected by problems that might be
+introduced by others.
+
+ Finally, software patents pose a constant threat to the existence of
+any free program. We wish to make sure that a company cannot
+effectively restrict the users of a free program by obtaining a
+restrictive license from a patent holder. Therefore, we insist that
+any patent license obtained for a version of the library must be
+consistent with the full freedom of use specified in this license.
+
+ Most GNU software, including some libraries, is covered by the
+ordinary GNU General Public License. This license, the GNU Lesser
+General Public License, applies to certain designated libraries, and
+is quite different from the ordinary General Public License. We use
+this license for certain libraries in order to permit linking those
+libraries into non-free programs.
+
+ When a program is linked with a library, whether statically or using
+a shared library, the combination of the two is legally speaking a
+combined work, a derivative of the original library. The ordinary
+General Public License therefore permits such linking only if the
+entire combination fits its criteria of freedom. The Lesser General
+Public License permits more lax criteria for linking other code with
+the library.
+
+ We call this license the "Lesser" General Public License because it
+does Less to protect the user's freedom than the ordinary General
+Public License. It also provides other free software developers Less
+of an advantage over competing non-free programs. These disadvantages
+are the reason we use the ordinary General Public License for many
+libraries. However, the Lesser license provides advantages in certain
+special circumstances.
+
+ For example, on rare occasions, there may be a special need to
+encourage the widest possible use of a certain library, so that it becomes
+a de-facto standard. To achieve this, non-free programs must be
+allowed to use the library. A more frequent case is that a free
+library does the same job as widely used non-free libraries. In this
+case, there is little to gain by limiting the free library to free
+software only, so we use the Lesser General Public License.
+
+ In other cases, permission to use a particular library in non-free
+programs enables a greater number of people to use a large body of
+free software. For example, permission to use the GNU C Library in
+non-free programs enables many more people to use the whole GNU
+operating system, as well as its variant, the GNU/Linux operating
+system.
+
+ Although the Lesser General Public License is Less protective of the
+users' freedom, it does ensure that the user of a program that is
+linked with the Library has the freedom and the wherewithal to run
+that program using a modified version of the Library.
+
+ The precise terms and conditions for copying, distribution and
+modification follow. Pay close attention to the difference between a
+"work based on the library" and a "work that uses the library". The
+former contains code derived from the library, whereas the latter must
+be combined with the library in order to run.
+
+ GNU LESSER GENERAL PUBLIC LICENSE
+ TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
+
+ 0. This License Agreement applies to any software library or other
+program which contains a notice placed by the copyright holder or
+other authorized party saying it may be distributed under the terms of
+this Lesser General Public License (also called "this License").
+Each licensee is addressed as "you".
+
+ A "library" means a collection of software functions and/or data
+prepared so as to be conveniently linked with application programs
+(which use some of those functions and data) to form executables.
+
+ The "Library", below, refers to any such software library or work
+which has been distributed under these terms. A "work based on the
+Library" means either the Library or any derivative work under
+copyright law: that is to say, a work containing the Library or a
+portion of it, either verbatim or with modifications and/or translated
+straightforwardly into another language. (Hereinafter, translation is
+included without limitation in the term "modification".)
+
+ "Source code" for a work means the preferred form of the work for
+making modifications to it. For a library, complete source code means
+all the source code for all modules it contains, plus any associated
+interface definition files, plus the scripts used to control compilation
+and installation of the library.
+
+ Activities other than copying, distribution and modification are not
+covered by this License; they are outside its scope. The act of
+running a program using the Library is not restricted, and output from
+such a program is covered only if its contents constitute a work based
+on the Library (independent of the use of the Library in a tool for
+writing it). Whether that is true depends on what the Library does
+and what the program that uses the Library does.
+
+ 1. You may copy and distribute verbatim copies of the Library's
+complete source code as you receive it, in any medium, provided that
+you conspicuously and appropriately publish on each copy an
+appropriate copyright notice and disclaimer of warranty; keep intact
+all the notices that refer to this License and to the absence of any
+warranty; and distribute a copy of this License along with the
+Library.
+
+ You may charge a fee for the physical act of transferring a copy,
+and you may at your option offer warranty protection in exchange for a
+fee.
+
+ 2. You may modify your copy or copies of the Library or any portion
+of it, thus forming a work based on the Library, and copy and
+distribute such modifications or work under the terms of Section 1
+above, provided that you also meet all of these conditions:
+
+ a) The modified work must itself be a software library.
+
+ b) You must cause the files modified to carry prominent notices
+ stating that you changed the files and the date of any change.
+
+ c) You must cause the whole of the work to be licensed at no
+ charge to all third parties under the terms of this License.
+
+ d) If a facility in the modified Library refers to a function or a
+ table of data to be supplied by an application program that uses
+ the facility, other than as an argument passed when the facility
+ is invoked, then you must make a good faith effort to ensure that,
+ in the event an application does not supply such function or
+ table, the facility still operates, and performs whatever part of
+ its purpose remains meaningful.
+
+ (For example, a function in a library to compute square roots has
+ a purpose that is entirely well-defined independent of the
+ application. Therefore, Subsection 2d requires that any
+ application-supplied function or table used by this function must
+ be optional: if the application does not supply it, the square
+ root function must still compute square roots.)
+
+These requirements apply to the modified work as a whole. If
+identifiable sections of that work are not derived from the Library,
+and can be reasonably considered independent and separate works in
+themselves, then this License, and its terms, do not apply to those
+sections when you distribute them as separate works. But when you
+distribute the same sections as part of a whole which is a work based
+on the Library, the distribution of the whole must be on the terms of
+this License, whose permissions for other licensees extend to the
+entire whole, and thus to each and every part regardless of who wrote
+it.
+
+Thus, it is not the intent of this section to claim rights or contest
+your rights to work written entirely by you; rather, the intent is to
+exercise the right to control the distribution of derivative or
+collective works based on the Library.
+
+In addition, mere aggregation of another work not based on the Library
+with the Library (or with a work based on the Library) on a volume of
+a storage or distribution medium does not bring the other work under
+the scope of this License.
+
+ 3. You may opt to apply the terms of the ordinary GNU General Public
+License instead of this License to a given copy of the Library. To do
+this, you must alter all the notices that refer to this License, so
+that they refer to the ordinary GNU General Public License, version 2,
+instead of to this License. (If a newer version than version 2 of the
+ordinary GNU General Public License has appeared, then you can specify
+that version instead if you wish.) Do not make any other change in
+these notices.
+
+ Once this change is made in a given copy, it is irreversible for
+that copy, so the ordinary GNU General Public License applies to all
+subsequent copies and derivative works made from that copy.
+
+ This option is useful when you wish to copy part of the code of
+the Library into a program that is not a library.
+
+ 4. You may copy and distribute the Library (or a portion or
+derivative of it, under Section 2) in object code or executable form
+under the terms of Sections 1 and 2 above provided that you accompany
+it with the complete corresponding machine-readable source code, which
+must be distributed under the terms of Sections 1 and 2 above on a
+medium customarily used for software interchange.
+
+ If distribution of object code is made by offering access to copy
+from a designated place, then offering equivalent access to copy the
+source code from the same place satisfies the requirement to
+distribute the source code, even though third parties are not
+compelled to copy the source along with the object code.
+
+ 5. A program that contains no derivative of any portion of the
+Library, but is designed to work with the Library by being compiled or
+linked with it, is called a "work that uses the Library". Such a
+work, in isolation, is not a derivative work of the Library, and
+therefore falls outside the scope of this License.
+
+ However, linking a "work that uses the Library" with the Library
+creates an executable that is a derivative of the Library (because it
+contains portions of the Library), rather than a "work that uses the
+library". The executable is therefore covered by this License.
+Section 6 states terms for distribution of such executables.
+
+ When a "work that uses the Library" uses material from a header file
+that is part of the Library, the object code for the work may be a
+derivative work of the Library even though the source code is not.
+Whether this is true is especially significant if the work can be
+linked without the Library, or if the work is itself a library. The
+threshold for this to be true is not precisely defined by law.
+
+ If such an object file uses only numerical parameters, data
+structure layouts and accessors, and small macros and small inline
+functions (ten lines or less in length), then the use of the object
+file is unrestricted, regardless of whether it is legally a derivative
+work. (Executables containing this object code plus portions of the
+Library will still fall under Section 6.)
+
+ Otherwise, if the work is a derivative of the Library, you may
+distribute the object code for the work under the terms of Section 6.
+Any executables containing that work also fall under Section 6,
+whether or not they are linked directly with the Library itself.
+
+ 6. As an exception to the Sections above, you may also combine or
+link a "work that uses the Library" with the Library to produce a
+work containing portions of the Library, and distribute that work
+under terms of your choice, provided that the terms permit
+modification of the work for the customer's own use and reverse
+engineering for debugging such modifications.
+
+ You must give prominent notice with each copy of the work that the
+Library is used in it and that the Library and its use are covered by
+this License. You must supply a copy of this License. If the work
+during execution displays copyright notices, you must include the
+copyright notice for the Library among them, as well as a reference
+directing the user to the copy of this License. Also, you must do one
+of these things:
+
+ a) Accompany the work with the complete corresponding
+ machine-readable source code for the Library including whatever
+ changes were used in the work (which must be distributed under
+ Sections 1 and 2 above); and, if the work is an executable linked
+ with the Library, with the complete machine-readable "work that
+ uses the Library", as object code and/or source code, so that the
+ user can modify the Library and then relink to produce a modified
+ executable containing the modified Library. (It is understood
+ that the user who changes the contents of definitions files in the
+ Library will not necessarily be able to recompile the application
+ to use the modified definitions.)
+
+ b) Use a suitable shared library mechanism for linking with the
+ Library. A suitable mechanism is one that (1) uses at run time a
+ copy of the library already present on the user's computer system,
+ rather than copying library functions into the executable, and (2)
+ will operate properly with a modified version of the library, if
+ the user installs one, as long as the modified version is
+ interface-compatible with the version that the work was made with.
+
+ c) Accompany the work with a written offer, valid for at
+ least three years, to give the same user the materials
+ specified in Subsection 6a, above, for a charge no more
+ than the cost of performing this distribution.
+
+ d) If distribution of the work is made by offering access to copy
+ from a designated place, offer equivalent access to copy the above
+ specified materials from the same place.
+
+ e) Verify that the user has already received a copy of these
+ materials or that you have already sent this user a copy.
+
+ For an executable, the required form of the "work that uses the
+Library" must include any data and utility programs needed for
+reproducing the executable from it. However, as a special exception,
+the materials to be distributed need not include anything that is
+normally distributed (in either source or binary form) with the major
+components (compiler, kernel, and so on) of the operating system on
+which the executable runs, unless that component itself accompanies
+the executable.
+
+ It may happen that this requirement contradicts the license
+restrictions of other proprietary libraries that do not normally
+accompany the operating system. Such a contradiction means you cannot
+use both them and the Library together in an executable that you
+distribute.
+
+ 7. You may place library facilities that are a work based on the
+Library side-by-side in a single library together with other library
+facilities not covered by this License, and distribute such a combined
+library, provided that the separate distribution of the work based on
+the Library and of the other library facilities is otherwise
+permitted, and provided that you do these two things:
+
+ a) Accompany the combined library with a copy of the same work
+ based on the Library, uncombined with any other library
+ facilities. This must be distributed under the terms of the
+ Sections above.
+
+ b) Give prominent notice with the combined library of the fact
+ that part of it is a work based on the Library, and explaining
+ where to find the accompanying uncombined form of the same work.
+
+ 8. You may not copy, modify, sublicense, link with, or distribute
+the Library except as expressly provided under this License. Any
+attempt otherwise to copy, modify, sublicense, link with, or
+distribute the Library is void, and will automatically terminate your
+rights under this License. However, parties who have received copies,
+or rights, from you under this License will not have their licenses
+terminated so long as such parties remain in full compliance.
+
+ 9. You are not required to accept this License, since you have not
+signed it. However, nothing else grants you permission to modify or
+distribute the Library or its derivative works. These actions are
+prohibited by law if you do not accept this License. Therefore, by
+modifying or distributing the Library (or any work based on the
+Library), you indicate your acceptance of this License to do so, and
+all its terms and conditions for copying, distributing or modifying
+the Library or works based on it.
+
+ 10. Each time you redistribute the Library (or any work based on the
+Library), the recipient automatically receives a license from the
+original licensor to copy, distribute, link with or modify the Library
+subject to these terms and conditions. You may not impose any further
+restrictions on the recipients' exercise of the rights granted herein.
+You are not responsible for enforcing compliance by third parties with
+this License.
+
+ 11. If, as a consequence of a court judgment or allegation of patent
+infringement or for any other reason (not limited to patent issues),
+conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License. If you cannot
+distribute so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you
+may not distribute the Library at all. For example, if a patent
+license would not permit royalty-free redistribution of the Library by
+all those who receive copies directly or indirectly through you, then
+the only way you could satisfy both it and this License would be to
+refrain entirely from distribution of the Library.
+
+If any portion of this section is held invalid or unenforceable under any
+particular circumstance, the balance of the section is intended to apply,
+and the section as a whole is intended to apply in other circumstances.
+
+It is not the purpose of this section to induce you to infringe any
+patents or other property right claims or to contest validity of any
+such claims; this section has the sole purpose of protecting the
+integrity of the free software distribution system which is
+implemented by public license practices. Many people have made
+generous contributions to the wide range of software distributed
+through that system in reliance on consistent application of that
+system; it is up to the author/donor to decide if he or she is willing
+to distribute software through any other system and a licensee cannot
+impose that choice.
+
+This section is intended to make thoroughly clear what is believed to
+be a consequence of the rest of this License.
+
+ 12. If the distribution and/or use of the Library is restricted in
+certain countries either by patents or by copyrighted interfaces, the
+original copyright holder who places the Library under this License may add
+an explicit geographical distribution limitation excluding those countries,
+so that distribution is permitted only in or among countries not thus
+excluded. In such case, this License incorporates the limitation as if
+written in the body of this License.
+
+ 13. The Free Software Foundation may publish revised and/or new
+versions of the Lesser General Public License from time to time.
+Such new versions will be similar in spirit to the present version,
+but may differ in detail to address new problems or concerns.
+
+Each version is given a distinguishing version number. If the Library
+specifies a version number of this License which applies to it and
+"any later version", you have the option of following the terms and
+conditions either of that version or of any later version published by
+the Free Software Foundation. If the Library does not specify a
+license version number, you may choose any version ever published by
+the Free Software Foundation.
+
+ 14. If you wish to incorporate parts of the Library into other free
+programs whose distribution conditions are incompatible with these,
+write to the author to ask for permission. For software which is
+copyrighted by the Free Software Foundation, write to the Free
+Software Foundation; we sometimes make exceptions for this. Our
+decision will be guided by the two goals of preserving the free status
+of all derivatives of our free software and of promoting the sharing
+and reuse of software generally.
+
+ NO WARRANTY
+
+ 15. BECAUSE THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO
+WARRANTY FOR THE LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW.
+EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR
+OTHER PARTIES PROVIDE THE LIBRARY "AS IS" WITHOUT WARRANTY OF ANY
+KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE
+LIBRARY IS WITH YOU. SHOULD THE LIBRARY PROVE DEFECTIVE, YOU ASSUME
+THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
+
+ 16. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN
+WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY
+AND/OR REDISTRIBUTE THE LIBRARY AS PERMITTED ABOVE, BE LIABLE TO YOU
+FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR
+CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE
+LIBRARY (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING
+RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A
+FAILURE OF THE LIBRARY TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF
+SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH
+DAMAGES.
+
+ END OF TERMS AND CONDITIONS
+
+ How to Apply These Terms to Your New Libraries
+
+ If you develop a new library, and you want it to be of the greatest
+possible use to the public, we recommend making it free software that
+everyone can redistribute and change. You can do so by permitting
+redistribution under these terms (or, alternatively, under the terms of the
+ordinary General Public License).
+
+ To apply these terms, attach the following notices to the library. It is
+safest to attach them to the start of each source file to most effectively
+convey the exclusion of warranty; and each file should have at least the
+"copyright" line and a pointer to where the full notice is found.
+
+ <one line to give the library's name and a brief idea of what it does.>
+ Copyright (C) <year> <name of author>
+
+ This library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ This library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with this library; if not, write to the Free Software
+ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+
+Also add information on how to contact you by electronic and paper mail.
+
+You should also get your employer (if you work as a programmer) or your
+school, if any, to sign a "copyright disclaimer" for the library, if
+necessary. Here is a sample; alter the names:
+
+ Yoyodyne, Inc., hereby disclaims all copyright interest in the
+ library `Frob' (a library for tweaking knobs) written by James Random Hacker.
+
+ <signature of Ty Coon>, 1 April 1990
+ Ty Coon, President of Vice
+
+That's all there is to it!
+
+
diff --git a/doc/linux-syn-cookies.txt b/doc/linux-syn-cookies.txt
new file mode 100644
index 0000000..ca13066
--- /dev/null
+++ b/doc/linux-syn-cookies.txt
@@ -0,0 +1,106 @@
+SYN cookie analysis on 3.10
+
+include/net/request_sock.h:
+
+static inline int reqsk_queue_is_full(const struct request_sock_queue *queue)
+{
+ return queue->listen_opt->qlen >> queue->listen_opt->max_qlen_log;
+}
+
+include/net/inet_connection_sock.h:
+
+static inline int inet_csk_reqsk_queue_is_full(const struct sock *sk)
+{
+ return reqsk_queue_is_full(&inet_csk(sk)->icsk_accept_queue);
+}
+
+max_qlen_log is computed to equal log2(min(min(listen_backlog,somaxconn), sysctl_max_syn_backlog),
+and this is done this way following this path :
+
+ socket.c:listen(fd, backlog) :
+
+ backlog = min(backlog, somaxconn)
+ => af_inet.c:inet_listen(sock, backlog)
+
+ => inet_connection_sock.c:inet_csk_listen_start(sk, backlog)
+
+ sk_max_ack_backlog = backlog
+ => request_sock.c:reqsk_queue_alloc(sk, backlog (=nr_table_entries))
+
+ nr_table_entries = min_t(u32, nr_table_entries, sysctl_max_syn_backlog);
+ nr_table_entries = max_t(u32, nr_table_entries, 8);
+ nr_table_entries = roundup_pow_of_two(nr_table_entries + 1);
+ for (lopt->max_qlen_log = 3;
+ (1 << lopt->max_qlen_log) < nr_table_entries;
+ lopt->max_qlen_log++);
+
+
+tcp_ipv4.c:tcp_v4_conn_request()
+ - inet_csk_reqsk_queue_is_full() returns true when the listening socket's
+ qlen is larger than 1 << max_qlen_log, so basically qlen >= min(backlog,max_backlog)
+
+ - tcp_syn_flood_action() returns true when sysctl_tcp_syncookies is set. It
+ also emits a warning once per listening socket when activating the feature.
+
+ if (inet_csk_reqsk_queue_is_full(sk) && !isn) {
+ want_cookie = tcp_syn_flood_action(sk, skb, "TCP");
+ if (!want_cookie)
+ goto drop;
+ }
+
+ => when the socket's current backlog is >= min(backlog,max_backlog),
+ either tcp_syn_cookies is set so we set want_cookie to 1, or we drop.
+
+
+ /* Accept backlog is full. If we have already queued enough
+ * of warm entries in syn queue, drop request. It is better than
+ * clogging syn queue with openreqs with exponentially increasing
+ * timeout.
+ */
+
+sock.h:sk_acceptq_is_full() = sk_ack_backlog > sk_max_ack_backlog
+ = sk_ack_backlog > min(somaxconn, listen_backlog)
+
+ if (sk_acceptq_is_full(sk) && inet_csk_reqsk_queue_young(sk) > 1) {
+ NET_INC_STATS_BH(sock_net(sk), LINUX_MIB_LISTENOVERFLOWS);
+ goto drop;
+ }
+
+====> the following algorithm is applied in the reverse order but with these
+ priorities :
+
+ 1) IF socket's accept queue >= min(somaxconn, listen_backlog) THEN drop
+
+ 2) IF socket's SYN backlog < min(somaxconn, listen_backlog, tcp_max_syn_backlog) THEN accept
+
+ 3) IF tcp_syn_cookies THEN send_syn_cookie
+
+ 4) otherwise drop
+
+====> the problem is the accept queue being filled, but it's supposed to be
+ filled only with validated client requests (step 1).
+
+
+
+ req = inet_reqsk_alloc(&tcp_request_sock_ops);
+ if (!req)
+ goto drop;
+
+ ...
+ if (!sysctl_tcp_syncookies &&
+ (sysctl_max_syn_backlog - inet_csk_reqsk_queue_len(sk) <
+ (sysctl_max_syn_backlog >> 2)) &&
+ !tcp_peer_is_proven(req, dst, false)) {
+ /* Without syncookies last quarter of
+ * backlog is filled with destinations,
+ * proven to be alive.
+ * It means that we continue to communicate
+ * to destinations, already remembered
+ * to the moment of synflood.
+ */
+ LIMIT_NETDEBUG(KERN_DEBUG pr_fmt("drop open request from %pI4/%u\n"),
+ &saddr, ntohs(tcp_hdr(skb)->source));
+ goto drop_and_release;
+ }
+
+
diff --git a/doc/lua-api/Makefile b/doc/lua-api/Makefile
new file mode 100644
index 0000000..b21857d
--- /dev/null
+++ b/doc/lua-api/Makefile
@@ -0,0 +1,153 @@
+# Makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line.
+SPHINXOPTS =
+SPHINXBUILD = sphinx-build
+PAPER =
+BUILDDIR = _build
+
+# Internal variables.
+PAPEROPT_a4 = -D latex_paper_size=a4
+PAPEROPT_letter = -D latex_paper_size=letter
+ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
+# the i18n builder cannot share the environment and doctrees with the others
+I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
+
+.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext
+
+help:
+ @echo "Please use \`make <target>' where <target> is one of"
+ @echo " html to make standalone HTML files"
+ @echo " dirhtml to make HTML files named index.html in directories"
+ @echo " singlehtml to make a single large HTML file"
+ @echo " pickle to make pickle files"
+ @echo " json to make JSON files"
+ @echo " htmlhelp to make HTML files and a HTML help project"
+ @echo " qthelp to make HTML files and a qthelp project"
+ @echo " devhelp to make HTML files and a Devhelp project"
+ @echo " epub to make an epub"
+ @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
+ @echo " latexpdf to make LaTeX files and run them through pdflatex"
+ @echo " text to make text files"
+ @echo " man to make manual pages"
+ @echo " texinfo to make Texinfo files"
+ @echo " info to make Texinfo files and run them through makeinfo"
+ @echo " gettext to make PO message catalogs"
+ @echo " changes to make an overview of all changed/added/deprecated items"
+ @echo " linkcheck to check all external links for integrity"
+ @echo " doctest to run all doctests embedded in the documentation (if enabled)"
+
+clean:
+ -rm -rf $(BUILDDIR)/*
+
+html:
+ $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
+ @echo
+ @echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
+
+dirhtml:
+ $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
+ @echo
+ @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
+
+singlehtml:
+ $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
+ @echo
+ @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."
+
+pickle:
+ $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
+ @echo
+ @echo "Build finished; now you can process the pickle files."
+
+json:
+ $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
+ @echo
+ @echo "Build finished; now you can process the JSON files."
+
+htmlhelp:
+ $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
+ @echo
+ @echo "Build finished; now you can run HTML Help Workshop with the" \
+ ".hhp project file in $(BUILDDIR)/htmlhelp."
+
+qthelp:
+ $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
+ @echo
+ @echo "Build finished; now you can run "qcollectiongenerator" with the" \
+ ".qhcp project file in $(BUILDDIR)/qthelp, like this:"
+ @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/haproxy-lua.qhcp"
+ @echo "To view the help file:"
+ @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/haproxy-lua.qhc"
+
+devhelp:
+ $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp
+ @echo
+ @echo "Build finished."
+ @echo "To view the help file:"
+ @echo "# mkdir -p $$HOME/.local/share/devhelp/haproxy-lua"
+ @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/haproxy-lua"
+ @echo "# devhelp"
+
+epub:
+ $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
+ @echo
+ @echo "Build finished. The epub file is in $(BUILDDIR)/epub."
+
+latex:
+ $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
+ @echo
+ @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
+ @echo "Run \`make' in that directory to run these through (pdf)latex" \
+ "(use \`make latexpdf' here to do that automatically)."
+
+latexpdf:
+ $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
+ @echo "Running LaTeX files through pdflatex..."
+ $(MAKE) -C $(BUILDDIR)/latex all-pdf
+ @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
+
+text:
+ $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
+ @echo
+ @echo "Build finished. The text files are in $(BUILDDIR)/text."
+
+man:
+ $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
+ @echo
+ @echo "Build finished. The manual pages are in $(BUILDDIR)/man."
+
+texinfo:
+ $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
+ @echo
+ @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo."
+ @echo "Run \`make' in that directory to run these through makeinfo" \
+ "(use \`make info' here to do that automatically)."
+
+info:
+ $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
+ @echo "Running Texinfo files through makeinfo..."
+ make -C $(BUILDDIR)/texinfo info
+ @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo."
+
+gettext:
+ $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale
+ @echo
+ @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale."
+
+changes:
+ $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
+ @echo
+ @echo "The overview file is in $(BUILDDIR)/changes."
+
+linkcheck:
+ $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
+ @echo
+ @echo "Link check complete; look for any errors in the above output " \
+ "or in $(BUILDDIR)/linkcheck/output.txt."
+
+doctest:
+ $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
+ @echo "Testing of doctests in the sources finished, look at the " \
+ "results in $(BUILDDIR)/doctest/output.txt."
diff --git a/doc/lua-api/_static/channel.fig b/doc/lua-api/_static/channel.fig
new file mode 100644
index 0000000..8a6c0a1
--- /dev/null
+++ b/doc/lua-api/_static/channel.fig
@@ -0,0 +1,55 @@
+#FIG 3.2 Produced by xfig version 3.2.5b
+Landscape
+Center
+Metric
+A4
+100.00
+Single
+-2
+1200 2
+1 1 0 1 0 7 50 -1 -1 0.000 1 0.0000 4500 1620 1260 585 4500 1620 5760 2205
+2 3 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 8
+ 1170 1350 1170 1890 2790 1890 2790 2070 3240 1620 2790 1170
+ 2790 1350 1170 1350
+2 3 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 8
+ 5760 1350 5760 1890 7380 1890 7380 2070 7830 1620 7380 1170
+ 7380 1350 5760 1350
+2 1 1 1 0 7 50 -1 -1 1.000 0 0 -1 1 0 2
+ 5 1 1.00 60.00 120.00
+ 6210 540 6210 1440
+2 1 1 1 0 7 50 -1 -1 1.000 0 0 -1 1 0 2
+ 5 1 1.00 60.00 120.00
+ 6210 2340 6210 1800
+2 1 1 1 0 7 50 -1 -1 1.000 0 0 -1 1 0 2
+ 5 1 1.00 60.00 120.00
+ 1350 2520 1350 1800
+2 1 1 1 0 7 50 -1 -1 1.000 0 0 -1 1 0 2
+ 5 1 1.00 60.00 120.00
+ 1350 360 1350 1440
+3 0 1 1 0 7 50 -1 -1 1.000 0 0 1 5
+ 5 1 1.00 60.00 120.00
+ 2970 1665 3105 1125 3330 900 3600 765 3915 720
+ 0.000 1.000 1.000 1.000 0.000
+3 0 1 1 0 7 50 -1 -1 1.000 0 0 1 5
+ 5 1 1.00 60.00 120.00
+ 6030 1665 5895 1125 5670 900 5400 765 5040 720
+ 0.000 1.000 1.000 1.000 0.000
+4 2 0 50 -1 16 12 0.0000 4 195 750 1080 1665 producer\001
+4 1 0 50 -1 16 12 0.0000 4 195 1785 4500 1575 HAProxy processing\001
+4 1 0 50 -1 16 12 0.0000 4 195 1260 4500 1815 (including Lua)\001
+4 0 0 50 -1 16 12 0.0000 4 105 855 7920 1665 consumer\001
+4 0 0 50 -1 12 12 0.0000 4 150 600 1440 2205 set()\001
+4 0 0 50 -1 12 12 0.0000 4 165 960 1440 2400 append()\001
+4 0 0 50 -1 16 12 0.0000 4 150 1260 1260 2700 write functions\001
+4 0 0 50 -1 16 12 0.0000 4 150 1230 1260 315 read functions\001
+4 0 0 50 -1 12 12 0.0000 4 165 600 1440 540 dup()\001
+4 0 0 50 -1 12 12 0.0000 4 165 600 1440 735 get()\001
+4 0 0 50 -1 12 12 0.0000 4 165 1200 1440 930 get_line()\001
+4 0 0 50 -1 12 12 0.0000 4 165 1440 1440 1125 get_in_len()\001
+4 1 0 50 -1 12 12 0.0000 4 150 1080 4500 765 forward()\001
+4 0 0 50 -1 16 12 0.0000 4 150 1260 6120 495 write functions\001
+4 0 0 50 -1 12 12 0.0000 4 150 720 6300 1110 send()\001
+4 0 0 50 -1 12 12 0.0000 4 165 1560 6255 2205 get_out_len()\001
+4 0 0 50 -1 16 12 0.0000 4 150 1230 6120 2520 read functions\001
+4 1 0 50 -1 16 12 0.0000 4 150 1650 4500 315 both side functions\001
+4 1 0 50 -1 12 12 0.0000 4 150 1080 4500 540 is_full()\001
diff --git a/doc/lua-api/_static/channel.png b/doc/lua-api/_static/channel.png
new file mode 100644
index 0000000..e12a26e
--- /dev/null
+++ b/doc/lua-api/_static/channel.png
Binary files differ
diff --git a/doc/lua-api/conf.py b/doc/lua-api/conf.py
new file mode 100644
index 0000000..fd7e0ee
--- /dev/null
+++ b/doc/lua-api/conf.py
@@ -0,0 +1,242 @@
+# -*- coding: utf-8 -*-
+#
+# haproxy-lua documentation build configuration file, created by
+# sphinx-quickstart on Tue Mar 10 11:15:09 2015.
+#
+# This file is execfile()d with the current directory set to its containing dir.
+#
+# Note that not all possible configuration values are present in this
+# autogenerated file.
+#
+# All configuration values have a default; values that are commented out
+# serve to show the default.
+
+import sys, os
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#sys.path.insert(0, os.path.abspath('.'))
+
+# -- General configuration -----------------------------------------------------
+
+# If your documentation needs a minimal Sphinx version, state it here.
+#needs_sphinx = '1.0'
+
+# Add any Sphinx extension module names here, as strings. They can be extensions
+# coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
+extensions = []
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# The suffix of source filenames.
+source_suffix = '.rst'
+
+# The encoding of source files.
+#source_encoding = 'utf-8-sig'
+
+# The master toctree document.
+master_doc = 'index'
+
+# General information about the project.
+project = u'haproxy-lua'
+copyright = u'2015, Thierry FOURNIER'
+
+# The version info for the project you're documenting, acts as replacement for
+# |version| and |release|, also used in various other places throughout the
+# built documents.
+#
+# The short X.Y version.
+version = '1.0'
+# The full version, including alpha/beta/rc tags.
+release = '1.0'
+
+# The language for content autogenerated by Sphinx. Refer to documentation
+# for a list of supported languages.
+#language = None
+
+# There are two options for replacing |today|: either, you set today to some
+# non-false value, then it is used:
+#today = ''
+# Else, today_fmt is used as the format for a strftime call.
+#today_fmt = '%B %d, %Y'
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+exclude_patterns = ['_build']
+
+# The reST default role (used for this markup: `text`) to use for all documents.
+#default_role = None
+
+# If true, '()' will be appended to :func: etc. cross-reference text.
+#add_function_parentheses = True
+
+# If true, the current module name will be prepended to all description
+# unit titles (such as .. function::).
+#add_module_names = True
+
+# If true, sectionauthor and moduleauthor directives will be shown in the
+# output. They are ignored by default.
+#show_authors = False
+
+# The name of the Pygments (syntax highlighting) style to use.
+pygments_style = 'sphinx'
+
+# A list of ignored prefixes for module index sorting.
+#modindex_common_prefix = []
+
+
+# -- Options for HTML output ---------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages. See the documentation for
+# a list of builtin themes.
+html_theme = 'default'
+
+# Theme options are theme-specific and customize the look and feel of a theme
+# further. For a list of options available for each theme, see the
+# documentation.
+#html_theme_options = {}
+
+# Add any paths that contain custom themes here, relative to this directory.
+#html_theme_path = []
+
+# The name for this set of Sphinx documents. If None, it defaults to
+# "<project> v<release> documentation".
+#html_title = None
+
+# A shorter title for the navigation bar. Default is the same as html_title.
+#html_short_title = None
+
+# The name of an image file (relative to this directory) to place at the top
+# of the sidebar.
+#html_logo = None
+
+# The name of an image file (within the static path) to use as favicon of the
+# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
+# pixels large.
+#html_favicon = None
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+
+# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
+# using the given strftime format.
+#html_last_updated_fmt = '%b %d, %Y'
+
+# If true, SmartyPants will be used to convert quotes and dashes to
+# typographically correct entities.
+#html_use_smartypants = True
+
+# Custom sidebar templates, maps document names to template names.
+#html_sidebars = {}
+
+# Additional templates that should be rendered to pages, maps page names to
+# template names.
+#html_additional_pages = {}
+
+# If false, no module index is generated.
+#html_domain_indices = True
+
+# If false, no index is generated.
+#html_use_index = True
+
+# If true, the index is split into individual pages for each letter.
+#html_split_index = False
+
+# If true, links to the reST sources are added to the pages.
+#html_show_sourcelink = True
+
+# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
+#html_show_sphinx = True
+
+# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
+#html_show_copyright = True
+
+# If true, an OpenSearch description file will be output, and all pages will
+# contain a <link> tag referring to it. The value of this option must be the
+# base URL from which the finished HTML is served.
+#html_use_opensearch = ''
+
+# This is the file name suffix for HTML files (e.g. ".xhtml").
+#html_file_suffix = None
+
+# Output file base name for HTML help builder.
+htmlhelp_basename = 'haproxy-luadoc'
+
+
+# -- Options for LaTeX output --------------------------------------------------
+
+latex_elements = {
+# The paper size ('letterpaper' or 'a4paper').
+#'papersize': 'letterpaper',
+
+# The font size ('10pt', '11pt' or '12pt').
+#'pointsize': '10pt',
+
+# Additional stuff for the LaTeX preamble.
+#'preamble': '',
+}
+
+# Grouping the document tree into LaTeX files. List of tuples
+# (source start file, target name, title, author, documentclass [howto/manual]).
+latex_documents = [
+ ('index', 'haproxy-lua.tex', u'haproxy-lua Documentation',
+ u'Thierry FOURNIER', 'manual'),
+]
+
+# The name of an image file (relative to this directory) to place at the top of
+# the title page.
+#latex_logo = None
+
+# For "manual" documents, if this is true, then toplevel headings are parts,
+# not chapters.
+#latex_use_parts = False
+
+# If true, show page references after internal links.
+#latex_show_pagerefs = False
+
+# If true, show URL addresses after external links.
+#latex_show_urls = False
+
+# Documents to append as an appendix to all manuals.
+#latex_appendices = []
+
+# If false, no module index is generated.
+#latex_domain_indices = True
+
+
+# -- Options for manual page output --------------------------------------------
+
+# One entry per manual page. List of tuples
+# (source start file, name, description, authors, manual section).
+man_pages = [
+ ('index', 'haproxy-lua', u'haproxy-lua Documentation',
+ [u'Thierry FOURNIER'], 1)
+]
+
+# If true, show URL addresses after external links.
+#man_show_urls = False
+
+
+# -- Options for Texinfo output ------------------------------------------------
+
+# Grouping the document tree into Texinfo files. List of tuples
+# (source start file, target name, title, author,
+# dir menu entry, description, category)
+texinfo_documents = [
+ ('index', 'haproxy-lua', u'haproxy-lua Documentation',
+ u'Thierry FOURNIER', 'haproxy-lua', 'One line description of project.',
+ 'Miscellaneous'),
+]
+
+# Documents to append as an appendix to all manuals.
+#texinfo_appendices = []
+
+# If false, no module index is generated.
+#texinfo_domain_indices = True
+
+# How to display URL addresses: 'footnote', 'no', or 'inline'.
+#texinfo_show_urls = 'footnote'
diff --git a/doc/lua-api/index.rst b/doc/lua-api/index.rst
new file mode 100644
index 0000000..e8df63e
--- /dev/null
+++ b/doc/lua-api/index.rst
@@ -0,0 +1,4491 @@
+.. toctree::
+ :maxdepth: 2
+
+
+How Lua runs in HAProxy
+=======================
+
+HAProxy Lua running contexts
+----------------------------
+
+The Lua code executed in HAProxy can be processed in 2 main modes. The first one
+is the **initialisation mode**, and the second is the **runtime mode**.
+
+* In the **initialisation mode**, we can perform DNS solves, but we cannot
+ perform socket I/O. In this initialisation mode, HAProxy still blocked during
+ the execution of the Lua program.
+
+* In the **runtime mode**, we cannot perform DNS solves, but we can use sockets.
+ The execution of the Lua code is multiplexed with the requests processing, so
+ the Lua code seems to be run in blocking, but it is not the case.
+
+The Lua code is loaded in one or more files. These files contains main code and
+functions. Lua has 8 execution contexts.
+
+1. The Lua file **body context**. It is executed during the load of the Lua file
+ in the HAProxy `[global]` section with the directive `lua-load`. It is
+ executed in initialisation mode. This section is use for configuring Lua
+ bindings in HAProxy.
+
+2. The Lua **init context**. It is a Lua function executed just after the
+ HAProxy configuration parsing. The execution is in initialisation mode. In
+ this context the HAProxy environment are already initialized. It is useful to
+ check configuration, or initializing socket connections or tasks. These
+ functions are declared in the body context with the Lua function
+ `core.register_init()`. The prototype of the function is a simple function
+ without return value and without parameters, like this: `function fcn()`.
+
+3. The Lua **task context**. It is a Lua function executed after the start
+ of the HAProxy scheduler, and just after the declaration of the task with the
+ Lua function `core.register_task()`. This context can be concurrent with the
+ traffic processing. It is executed in runtime mode. The prototype of the
+ function is a simple function without return value and without parameters,
+ like this: `function fcn()`.
+
+4. The **action context**. It is a Lua function conditionally executed. These
+ actions are registered by the Lua directives "`core.register_action()`". The
+ prototype of the Lua called function is a function with doesn't returns
+ anything and that take an object of class TXN as entry. `function fcn(txn)`.
+
+5. The **sample-fetch context**. This function takes a TXN object as entry
+ argument and returns a string. These types of function cannot execute any
+ blocking function. They are useful to aggregate some of original HAProxy
+ sample-fetches and return the result. The prototype of the function is
+ `function string fcn(txn)`. These functions can be registered with the Lua
+ function `core.register_fetches()`. Each declared sample-fetch is prefixed by
+ the string "lua.".
+
+ .. note::
+ It is possible that this function cannot found the required data in the
+ original HAProxy sample-fetches, in this case, it cannot return the
+ result. This case is not yet supported
+
+6. The **converter context**. It is a Lua function that takes a string as input
+ and returns another string as output. These types of function are stateless,
+ it cannot access to any context. They don't execute any blocking function.
+ The call prototype is `function string fcn(string)`. This function can be
+ registered with the Lua function `core.register_converters()`. Each declared
+ converter is prefixed by the string "lua.".
+
+7. The **filter context**: It is a Lua object based on a class defining filter
+ callback functions. Lua filters are registered using
+ `core.register_filter()`. Each declared filter is prefixed by the string
+ "lua.".
+
+8. The **event context**: Inside a function that handles events subscribed
+ through `core.event_sub()` or `Server.event_sub()`.
+
+
+HAProxy Lua Hello world
+-----------------------
+
+HAProxy configuration file (`hello_world.conf`):
+
+::
+
+ global
+ lua-load hello_world.lua
+
+ listen proxy
+ bind 127.0.0.1:10001
+ tcp-request inspect-delay 1s
+ tcp-request content use-service lua.hello_world
+
+HAProxy Lua file (`hello_world.lua`):
+
+.. code-block:: lua
+
+ core.register_service("hello_world", "tcp", function(applet)
+ applet:send("hello world\n")
+ end)
+
+How to start HAProxy for testing this configuration:
+
+::
+
+ ./haproxy -f hello_world.conf
+
+On other terminal, you can test with telnet:
+
+::
+
+ #:~ telnet 127.0.0.1 10001
+ hello world
+
+Usage of load parameters
+------------------------
+
+HAProxy lua-load(-per-thread) directives allow a list of parameters after
+the lua file name. These parameters are accessible through an array of args
+using this code `local args = table.pack(...)` in the body of loaded file.
+
+Below, a new version of the hello world using load parameters
+
+HAProxy configuration file (`hello_world.conf`):
+
+::
+
+ global
+ lua-load hello_world.lua "this is not an hello world"
+
+ listen proxy
+ bind 127.0.0.1:10001
+ tcp-request inspect-delay 1s
+ tcp-request content use-service lua.hello_world
+
+HAProxy Lua file (`hello_world.lua`):
+
+.. code-block:: lua
+
+ local args = table.pack(...)
+
+ core.register_service("hello_world", "tcp", function(applet)
+ applet:send(args[1] .. "\n")
+ end)
+
+
+Core class
+==========
+
+.. js:class:: core
+
+ The "core" class contains all the HAProxy core functions. These function are
+ useful for the controlling of the execution flow, registering hooks,
+ manipulating global maps or ACL, ...
+
+ "core" class is basically provided with HAProxy. No `require` line is
+ required to uses these function.
+
+ The "core" class is static, it is not possible to create a new object of this
+ type.
+
+.. js:attribute:: core.emerg
+
+ :returns: integer
+
+ This attribute is an integer, it contains the value of the loglevel
+ "emergency" (0).
+
+.. js:attribute:: core.alert
+
+ :returns: integer
+
+ This attribute is an integer, it contains the value of the loglevel
+ "alert" (1).
+
+.. js:attribute:: core.crit
+
+ :returns: integer
+
+ This attribute is an integer, it contains the value of the loglevel
+ "critical" (2).
+
+.. js:attribute:: core.err
+
+ :returns: integer
+
+ This attribute is an integer, it contains the value of the loglevel
+ "error" (3).
+
+.. js:attribute:: core.warning
+
+ :returns: integer
+
+ This attribute is an integer, it contains the value of the loglevel
+ "warning" (4).
+
+.. js:attribute:: core.notice
+
+ :returns: integer
+
+ This attribute is an integer, it contains the value of the loglevel
+ "notice" (5).
+
+.. js:attribute:: core.info
+
+ :returns: integer
+
+ This attribute is an integer, it contains the value of the loglevel
+ "info" (6).
+
+.. js:attribute:: core.debug
+
+ :returns: integer
+
+ This attribute is an integer, it contains the value of the loglevel
+ "debug" (7).
+
+.. js:attribute:: core.proxies
+
+ **context**: init, task, action, sample-fetch, converter
+
+ This attribute is a table of declared proxies (frontend and backends). Each
+ proxy give an access to his list of listeners and servers. The table is
+ indexed by proxy name, and each entry is of type :ref:`proxy_class`.
+
+ .. Warning::
+ if you declared a frontend and backend with the same name, only one of
+ them will be listed.
+
+ :see: :js:attr:`core.backends`
+ :see: :js:attr:`core.frontends`
+
+.. js:attribute:: core.backends
+
+ **context**: init, task, action, sample-fetch, converter
+
+ This attribute is a table of declared proxies with backend capability. Each
+ proxy give an access to his list of listeners and servers. The table is
+ indexed by the backend name, and each entry is of type :ref:`proxy_class`.
+
+ :see: :js:attr:`core.proxies`
+ :see: :js:attr:`core.frontends`
+
+.. js:attribute:: core.frontends
+
+ **context**: init, task, action, sample-fetch, converter
+
+ This attribute is a table of declared proxies with frontend capability. Each
+ proxy give an access to his list of listeners and servers. The table is
+ indexed by the frontend name, and each entry is of type :ref:`proxy_class`.
+
+ :see: :js:attr:`core.proxies`
+ :see: :js:attr:`core.backends`
+
+.. js:attribute:: core.thread
+
+ **context**: task, action, sample-fetch, converter, applet
+
+ This variable contains the executing thread number starting at 1. 0 is a
+ special case for the common lua context. So, if thread is 0, Lua scope is
+ shared by all threads, otherwise the scope is dedicated to a single thread.
+ A program which needs to execute some parts exactly once regardless of the
+ number of threads can check that core.thread is 0 or 1.
+
+.. js:function:: core.log(loglevel, msg)
+
+ **context**: body, init, task, action, sample-fetch, converter
+
+ This function sends a log. The log is sent, according with the HAProxy
+ configuration file, to the loggers relevant to the current context and/or
+ to stderr if it is allowed.
+
+ The exact behaviour depends on tune.lua.log.loggers and tune.lua.log.stderr.
+
+ :param integer loglevel: Is the log level associated with the message. It is a
+ number between 0 and 7.
+ :param string msg: The log content.
+ :see: :js:attr:`core.emerg`, :js:attr:`core.alert`, :js:attr:`core.crit`,
+ :js:attr:`core.err`, :js:attr:`core.warning`, :js:attr:`core.notice`,
+ :js:attr:`core.info`, :js:attr:`core.debug` (log level definitions)
+ :see: :js:func:`core.Debug`
+ :see: :js:func:`core.Info`
+ :see: :js:func:`core.Warning`
+ :see: :js:func:`core.Alert`
+
+.. js:function:: core.Debug(msg)
+
+ **context**: body, init, task, action, sample-fetch, converter
+
+ :param string msg: The log content.
+ :see: :js:func:`core.log`
+
+ Does the same job than:
+
+.. code-block:: lua
+
+ function Debug(msg)
+ core.log(core.debug, msg)
+ end
+..
+
+.. js:function:: core.Info(msg)
+
+ **context**: body, init, task, action, sample-fetch, converter
+
+ :param string msg: The log content.
+ :see: :js:func:`core.log`
+
+.. code-block:: lua
+
+ function Info(msg)
+ core.log(core.info, msg)
+ end
+..
+
+.. js:function:: core.Warning(msg)
+
+ **context**: body, init, task, action, sample-fetch, converter
+
+ :param string msg: The log content.
+ :see: :js:func:`core.log`
+
+.. code-block:: lua
+
+ function Warning(msg)
+ core.log(core.warning, msg)
+ end
+..
+
+.. js:function:: core.Alert(msg)
+
+ **context**: body, init, task, action, sample-fetch, converter
+
+ :param string msg: The log content.
+ :see: :js:func:`core.log`
+
+.. code-block:: lua
+
+ function Alert(msg)
+ core.log(core.alert, msg)
+ end
+..
+
+.. js:function:: core.add_acl(filename, key)
+
+ **context**: init, task, action, sample-fetch, converter
+
+ Add the ACL *key* in the ACLs list referenced by the file *filename*.
+
+ :param string filename: the filename that reference the ACL entries.
+ :param string key: the key which will be added.
+
+.. js:function:: core.del_acl(filename, key)
+
+ **context**: init, task, action, sample-fetch, converter
+
+ Delete the ACL entry referenced by the key *key* in the list of ACLs
+ referenced by *filename*.
+
+ :param string filename: the filename that reference the ACL entries.
+ :param string key: the key which will be deleted.
+
+.. js:function:: core.del_map(filename, key)
+
+ **context**: init, task, action, sample-fetch, converter
+
+ Delete the map entry indexed with the specified key in the list of maps
+ referenced by his filename.
+
+ :param string filename: the filename that reference the map entries.
+ :param string key: the key which will be deleted.
+
+.. js:function:: core.get_info()
+
+ **context**: body, init, task, action, sample-fetch, converter
+
+ Returns HAProxy core information. We can find information like the uptime,
+ the pid, memory pool usage, tasks number, ...
+
+ This information is also returned by the management socket via the command
+ "show info". See the management socket documentation for more information
+ about the content of these variables.
+
+ :returns: an array of values.
+
+.. js:function:: core.get_var()
+
+ **context**: body, init, task, action, sample-fetch, converter
+
+ Returns data stored in the variable <var> converter in Lua type.
+ This is limited to "proc." scoped variables.
+
+ :param string var: The variable name in "proc." scope according with the
+ HAProxy variable syntax.
+
+.. js:function:: core.now()
+
+ **context**: body, init, task, action
+
+ This function returns the current time. The time returned is fixed by the
+ HAProxy core and assures than the hour will be monotonic and that the system
+ call 'gettimeofday' will not be called too. The time is refreshed between each
+ Lua execution or resume, so two consecutive call to the function "now" will
+ probably returns the same result.
+
+ :returns: a table which contains two entries "sec" and "usec". "sec"
+ contains the current at the epoch format, and "usec" contains the
+ current microseconds.
+
+.. js:function:: core.http_date(date)
+
+ **context**: body, init, task, action
+
+ This function take a string representing http date, and returns an integer
+ containing the corresponding date with a epoch format. A valid http date
+ me respect the format IMF, RFC850 or ASCTIME.
+
+ :param string date: a date http-date formatted
+ :returns: integer containing epoch date
+ :see: :js:func:`core.imf_date`.
+ :see: :js:func:`core.rfc850_date`.
+ :see: :js:func:`core.asctime_date`.
+ :see: https://tools.ietf.org/html/rfc7231#section-7.1.1.1
+
+.. js:function:: core.imf_date(date)
+
+ **context**: body, init, task, action
+
+ This function take a string representing IMF date, and returns an integer
+ containing the corresponding date with a epoch format.
+
+ :param string date: a date IMF formatted
+ :returns: integer containing epoch date
+ :see: https://tools.ietf.org/html/rfc7231#section-7.1.1.1
+
+ The IMF format is like this:
+
+.. code-block:: text
+
+ Sun, 06 Nov 1994 08:49:37 GMT
+..
+
+.. js:function:: core.rfc850_date(date)
+
+ **context**: body, init, task, action
+
+ This function take a string representing RFC850 date, and returns an integer
+ containing the corresponding date with a epoch format.
+
+ :param string date: a date RFC859 formatted
+ :returns: integer containing epoch date
+ :see: https://tools.ietf.org/html/rfc7231#section-7.1.1.1
+
+ The RFC850 format is like this:
+
+.. code-block:: text
+
+ Sunday, 06-Nov-94 08:49:37 GMT
+..
+
+.. js:function:: core.asctime_date(date)
+
+ **context**: body, init, task, action
+
+ This function take a string representing ASCTIME date, and returns an integer
+ containing the corresponding date with a epoch format.
+
+ :param string date: a date ASCTIME formatted
+ :returns: integer containing epoch date
+ :see: https://tools.ietf.org/html/rfc7231#section-7.1.1.1
+
+ The ASCTIME format is like this:
+
+.. code-block:: text
+
+ Sun Nov 6 08:49:37 1994
+..
+
+.. js:function:: core.msleep(milliseconds)
+
+ **context**: body, init, task, action
+
+ The `core.msleep()` stops the Lua execution between specified milliseconds.
+
+ :param integer milliseconds: the required milliseconds.
+
+.. js:function:: core.register_action(name, actions, func [, nb_args])
+
+ **context**: body
+
+ Register a Lua function executed as action. All the registered action can be
+ used in HAProxy with the prefix "lua.". An action gets a TXN object class as
+ input.
+
+ :param string name: is the name of the action.
+ :param table actions: is a table of string describing the HAProxy actions
+ facilities where to expose the new action. Expected facilities are:
+ 'tcp-req', 'tcp-res', 'http-req', 'http-res', 'http-after-res'.
+ :param function func: is the Lua function called to work as an action.
+ :param integer nb_args: is the expected number of argument for the action.
+ By default the value is 0.
+
+ The prototype of the Lua function used as argument is:
+
+.. code-block:: lua
+
+ function(txn [, arg1 [, arg2]])
+..
+
+ * **txn** (:ref:`txn_class`): this is a TXN object used for manipulating the
+ current request or TCP stream.
+
+ * **argX**: this is argument provided through the HAProxy configuration file.
+
+ Here, an example of action registration. The action just send an 'Hello world'
+ in the logs.
+
+.. code-block:: lua
+
+ core.register_action("hello-world", { "tcp-req", "http-req" }, function(txn)
+ txn:Info("Hello world")
+ end)
+..
+
+ This example code is used in HAProxy configuration like this:
+
+::
+
+ frontend tcp_frt
+ mode tcp
+ tcp-request content lua.hello-world
+
+ frontend http_frt
+ mode http
+ http-request lua.hello-world
+
+..
+
+ A second example using arguments
+
+.. code-block:: lua
+
+ function hello_world(txn, arg)
+ txn:Info("Hello world for " .. arg)
+ end
+ core.register_action("hello-world", { "tcp-req", "http-req" }, hello_world, 2)
+
+..
+
+ This example code is used in HAProxy configuration like this:
+
+::
+
+ frontend tcp_frt
+ mode tcp
+ tcp-request content lua.hello-world everybody
+
+..
+
+.. js:function:: core.register_converters(name, func)
+
+ **context**: body
+
+ Register a Lua function executed as converter. All the registered converters
+ can be used in HAProxy with the prefix "lua.". A converter gets a string as
+ input and returns a string as output. The registered function can take up to 9
+ values as parameter. All the values are strings.
+
+ :param string name: is the name of the converter.
+ :param function func: is the Lua function called to work as converter.
+
+ The prototype of the Lua function used as argument is:
+
+.. code-block:: lua
+
+ function(str, [p1 [, p2 [, ... [, p5]]]])
+..
+
+ * **str** (*string*): this is the input value automatically converted in
+ string.
+ * **p1** .. **p5** (*string*): this is a list of string arguments declared in
+ the HAProxy configuration file. The number of arguments doesn't exceed 5.
+ The order and the nature of these is conventionally chosen by the
+ developer.
+
+.. js:function:: core.register_fetches(name, func)
+
+ **context**: body
+
+ Register a Lua function executed as sample fetch. All the registered sample
+ fetch can be used in HAProxy with the prefix "lua.". A Lua sample fetch
+ returns a string as output. The registered function can take up to 9 values as
+ parameter. All the values are strings.
+
+ :param string name: is the name of the sample fetch.
+ :param function func: is the Lua function called to work as sample fetch.
+
+ The prototype of the Lua function used as argument is:
+
+.. code-block:: lua
+
+ string function(txn, [p1 [, p2 [, ... [, p5]]]])
+..
+
+ * **txn** (:ref:`txn_class`): this is the txn object associated with the
+ current request.
+ * **p1** .. **p5** (*string*): this is a list of string arguments declared in
+ the HAProxy configuration file. The number of arguments doesn't exceed 5.
+ The order and the nature of these is conventionally chosen by the
+ developer.
+ * **Returns**: A string containing some data, or nil if the value cannot be
+ returned now.
+
+ lua example code:
+
+.. code-block:: lua
+
+ core.register_fetches("hello", function(txn)
+ return "hello"
+ end)
+..
+
+ HAProxy example configuration:
+
+::
+
+ frontend example
+ http-request redirect location /%[lua.hello]
+
+.. js:function:: core.register_filter(name, Flt, func)
+
+ **context**: body
+
+ Register a Lua function used to declare a filter. All the registered filters
+ can by used in HAProxy with the prefix "lua.".
+
+ :param string name: is the name of the filter.
+ :param table Flt: is a Lua class containing the filter definition (id, flags,
+ callbacks).
+ :param function func: is the Lua function called to create the Lua filter.
+
+ The prototype of the Lua function used as argument is:
+
+.. code-block:: lua
+
+ function(flt, args)
+..
+
+ * **flt** : Is a filter object based on the class provided in
+ :js:func:`core.register_filter()` function.
+
+ * **args**: Is a table of strings containing all arguments provided through
+ the HAProxy configuration file, on the filter line.
+
+ It must return the filter to use or nil to ignore it. Here, an example of
+ filter registration.
+
+.. code-block:: lua
+
+ core.register_filter("my-filter", MyFilter, function(flt, args)
+ flt.args = args -- Save arguments
+ return flt
+ end)
+..
+
+ This example code is used in HAProxy configuration like this:
+
+::
+
+ frontend http
+ mode http
+ filter lua.my-filter arg1 arg2 arg3
+
+..
+
+ :see: :js:class:`Filter`
+
+.. js:function:: core.register_service(name, mode, func)
+
+ **context**: body
+
+ Register a Lua function executed as a service. All the registered services
+ can be used in HAProxy with the prefix "lua.". A service gets an object class
+ as input according with the required mode.
+
+ :param string name: is the name of the service.
+ :param string mode: is string describing the required mode. Only 'tcp' or
+ 'http' are allowed.
+ :param function func: is the Lua function called to work as service.
+
+ The prototype of the Lua function used as argument is:
+
+.. code-block:: lua
+
+ function(applet)
+..
+
+ * **applet** *applet* will be a :ref:`applettcp_class` or a
+ :ref:`applethttp_class`. It depends the type of registered applet. An applet
+ registered with the 'http' value for the *mode* parameter will gets a
+ :ref:`applethttp_class`. If the *mode* value is 'tcp', the applet will gets
+ a :ref:`applettcp_class`.
+
+ .. warning::
+ Applets of type 'http' cannot be called from 'tcp-*' rulesets. Only the
+ 'http-*' rulesets are authorized, this means that is not possible to call
+ a HTTP applet from a proxy in tcp mode. Applets of type 'tcp' can be
+ called from anywhere.
+
+ Here, an example of service registration. The service just send an
+ 'Hello world' as an http response.
+
+.. code-block:: lua
+
+ core.register_service("hello-world", "http", function(applet)
+ local response = "Hello World !"
+ applet:set_status(200)
+ applet:add_header("content-length", string.len(response))
+ applet:add_header("content-type", "text/plain")
+ applet:start_response()
+ applet:send(response)
+ end)
+..
+
+ This example code is used in HAProxy configuration like this:
+
+::
+
+ frontend example
+ http-request use-service lua.hello-world
+
+.. js:function:: core.register_init(func)
+
+ **context**: body
+
+ Register a function executed after the configuration parsing. This is useful
+ to check any parameters.
+
+ :param function func: is the Lua function called to work as initializer.
+
+ The prototype of the Lua function used as argument is:
+
+.. code-block:: lua
+
+ function()
+..
+
+ It takes no input, and no output is expected.
+
+.. js:function:: core.register_task(func[, arg1[, arg2[, ...[, arg4]]]])
+
+ **context**: body, init, task, action, sample-fetch, converter, event
+
+ Register and start independent task. The task is started when the HAProxy
+ main scheduler starts. For example this type of tasks can be executed to
+ perform complex health checks.
+
+ :param function func: is the Lua function called to work as an async task.
+
+ Up to 4 optional arguments (all types supported) may be passed to the
+ function. (They will be passed as-is to the task function)
+
+ The prototype of the Lua function used as argument is:
+
+.. code-block:: lua
+
+ function([arg1[, arg2[, ...[, arg4]]]])
+..
+
+ It takes up to 4 optional arguments (provided when registering), and no
+ output is expected.
+
+ See also :js:func:`core.queue` to dynamically pass data between main context
+ and tasks or even between tasks.
+
+.. js:function:: core.register_cli([path], usage, func)
+
+ **context**: body
+
+ Register a custom cli that will be available from haproxy stats socket.
+
+ :param array path: is the sequence of word for which the cli execute the Lua
+ binding.
+ :param string usage: is the usage message displayed in the help.
+ :param function func: is the Lua function called to handle the CLI commands.
+
+ The prototype of the Lua function used as argument is:
+
+.. code-block:: lua
+
+ function(AppletTCP, [arg1, [arg2, [...]]])
+..
+
+ I/O are managed with the :ref:`applettcp_class` object. Args are given as
+ parameter. The args embed the registered path. If the path is declared like
+ this:
+
+.. code-block:: lua
+
+ core.register_cli({"show", "ssl", "stats"}, "Display SSL stats..", function(applet, arg1, arg2, arg3, arg4, arg5)
+ end)
+..
+
+ And we execute this in the prompt:
+
+.. code-block:: text
+
+ > prompt
+ > show ssl stats all
+..
+
+ Then, arg1, arg2 and arg3 will contains respectively "show", "ssl" and
+ "stats".
+ arg4 will contain "all". arg5 contains nil.
+
+.. js:function:: core.set_nice(nice)
+
+ **context**: task, action, sample-fetch, converter
+
+ Change the nice of the current task or current session.
+
+ :param integer nice: the nice value, it must be between -1024 and 1024.
+
+.. js:function:: core.set_map(filename, key, value)
+
+ **context**: init, task, action, sample-fetch, converter
+
+ Set the value *value* associated to the key *key* in the map referenced by
+ *filename*.
+
+ :param string filename: the Map reference
+ :param string key: the key to set or replace
+ :param string value: the associated value
+
+.. js:function:: core.sleep(int seconds)
+
+ **context**: body, init, task, action
+
+ The `core.sleep()` functions stop the Lua execution between specified seconds.
+
+ :param integer seconds: the required seconds.
+
+.. js:function:: core.tcp()
+
+ **context**: init, task, action
+
+ This function returns a new object of a *socket* class.
+
+ :returns: A :ref:`socket_class` object.
+
+.. js:function:: core.httpclient()
+
+ **context**: init, task, action
+
+ This function returns a new object of a *httpclient* class.
+
+ :returns: A :ref:`httpclient_class` object.
+
+.. js:function:: core.concat()
+
+ **context**: body, init, task, action, sample-fetch, converter
+
+ This function returns a new concat object.
+
+ :returns: A :ref:`concat_class` object.
+
+.. js:function:: core.queue()
+
+ **context**: body, init, task, event, action, sample-fetch, converter
+
+ This function returns a new queue object.
+
+ :returns: A :ref:`queue_class` object.
+
+.. js:function:: core.done(data)
+
+ **context**: body, init, task, action, sample-fetch, converter
+
+ :param any data: Return some data for the caller. It is useful with
+ sample-fetches and sample-converters.
+
+ Immediately stops the current Lua execution and returns to the caller which
+ may be a sample fetch, a converter or an action and returns the specified
+ value (ignored for actions and init). It is used when the LUA process finishes
+ its work and wants to give back the control to HAProxy without executing the
+ remaining code. It can be seen as a multi-level "return".
+
+.. js:function:: core.yield()
+
+ **context**: task, action, sample-fetch, converter
+
+ Give back the hand at the HAProxy scheduler. It is used when the LUA
+ processing consumes a lot of processing time.
+
+.. js:function:: core.parse_addr(address)
+
+ **context**: body, init, task, action, sample-fetch, converter
+
+ :param network: is a string describing an ipv4 or ipv6 address and optionally
+ its network length, like this: "127.0.0.1/8" or "aaaa::1234/32".
+ :returns: a userdata containing network or nil if an error occurs.
+
+ Parse ipv4 or ipv6 addresses and its facultative associated network.
+
+.. js:function:: core.match_addr(addr1, addr2)
+
+ **context**: body, init, task, action, sample-fetch, converter
+
+ :param addr1: is an address created with "core.parse_addr".
+ :param addr2: is an address created with "core.parse_addr".
+ :returns: boolean, true if the network of the addresses match, else returns
+ false.
+
+ Match two networks. For example "127.0.0.1/32" matches "127.0.0.0/8". The
+ order of network is not important.
+
+.. js:function:: core.tokenize(str, separators [, noblank])
+
+ **context**: body, init, task, action, sample-fetch, converter
+
+ This function is useful for tokenizing an entry, or splitting some messages.
+ :param string str: The string which will be split.
+ :param string separators: A string containing a list of separators.
+ :param boolean noblank: Ignore empty entries.
+ :returns: an array of string.
+
+ For example:
+
+.. code-block:: lua
+
+ local array = core.tokenize("This function is useful, for tokenizing an entry.", "., ", true)
+ print_r(array)
+..
+
+ Returns this array:
+
+.. code-block:: text
+
+ (table) table: 0x21c01e0 [
+ 1: (string) "This"
+ 2: (string) "function"
+ 3: (string) "is"
+ 4: (string) "useful"
+ 5: (string) "for"
+ 6: (string) "tokenizing"
+ 7: (string) "an"
+ 8: (string) "entry"
+ ]
+..
+
+.. js:function:: core.event_sub(event_types, func)
+
+ **context**: body, init, task, action, sample-fetch, converter
+
+ Register a function that will be called on specific system events.
+
+ :param array event_types: array of string containing the event types you want
+ to subscribe to
+ :param function func: is the Lua function called when one of the subscribed
+ events occur.
+ :returns: A :ref:`event_sub_class` object.
+ :see: :js:func:`Server.event_sub()`.
+
+ List of available event types :
+
+ **SERVER** Family:
+
+ * **SERVER_ADD**: when a server is added
+ * **SERVER_DEL**: when a server is removed
+ * **SERVER_DOWN**: when a server state goes from UP to DOWN
+ * **SERVER_UP**: when a server state goes from DOWN to UP
+ * **SERVER_STATE**: when a server state changes
+ * **SERVER_ADMIN**: when a server administrative state changes
+ * **SERVER_CHECK**: when a server's check status change is reported.
+ Be careful when subscribing to this type since many events might be
+ generated.
+
+ .. Note::
+ Use **SERVER** in **event_types** to subscribe to all server events types
+ at once. Note that this should only be used for testing purposes since a
+ single event source could result in multiple events types being generated.
+ (e.g.: SERVER_STATE will always be generated for each SERVER_DOWN or
+ SERVER_UP)
+
+ The prototype of the Lua function used as argument is:
+
+.. code-block:: lua
+
+ function(event, event_data, sub, when)
+..
+
+ * **event** (*string*): the event type (one of the **event_types** specified
+ when subscribing)
+ * **event_data**: specific to each event family (For **SERVER** family,
+ a :ref:`server_event_class` object)
+ * **sub**: class to manage the subscription from within the event
+ (a :ref:`event_sub_class` object)
+ * **when**: timestamp corresponding to the date when the event was generated.
+ It is an integer representing the number of seconds elapsed since Epoch.
+ It may be provided as optional argument to `os.date()` lua function to
+ convert it to a string according to a given format string.
+
+ .. Warning::
+ The callback function will only be scheduled on the very same thread that
+ performed the subscription.
+
+ Moreover, each thread treats events sequentially. It means that if you
+ have, let's say SERVER_UP followed by a SERVER_DOWN in a short timelapse,
+ then the cb function will first be called with SERVER_UP, and once it's
+ done handling the event, the cb function will be called again with
+ SERVER_DOWN.
+
+ This is to ensure event consistency when it comes to logging / triggering
+ logic from lua.
+
+ Your lua cb function may yield if needed, but you're pleased to process the
+ event as fast as possible to prevent the event queue from growing up,
+ depending on the event flow that is expected for the given subscription.
+
+ To prevent abuses, if the event queue for the current subscription goes
+ over a certain amount of unconsumed events, the subscription will pause
+ itself automatically for as long as it takes for your handler to catch up.
+ This would lead to events being missed, so an error will be reported in the
+ logs to warn you about that.
+ This is not something you want to let happen too often, it may indicate
+ that you subscribed to an event that is occurring too frequently or/and
+ that your callback function is too slow to keep up the pace and you should
+ review it.
+
+ If you want to do some parallel processing because your callback functions
+ are slow: you might want to create subtasks from lua using
+ :js:func:`core.register_task()` from within your callback function to
+ perform the heavy job in a dedicated task and allow remaining events to be
+ processed more quickly.
+
+.. js:function:: core.disable_legacy_mailers()
+
+ **LEGACY**
+
+ **context**: body, init
+
+ Disable the sending of email alerts through the legacy email sending
+ function when mailers are used in the configuration.
+
+ Use this when sending email alerts directly from lua.
+
+ :see: :js:func:`Proxy.get_mailers()`
+
+.. _proxy_class:
+
+Proxy class
+============
+
+.. js:class:: Proxy
+
+ This class provides a way for manipulating proxy and retrieving information
+ like statistics.
+
+.. js:attribute:: Proxy.name
+
+ Contain the name of the proxy.
+
+ .. warning::
+ This attribute is now deprecated and will eventually be removed.
+ Please use :js:func:`Proxy.get_name()` function instead.
+
+.. js:function:: Proxy.get_name()
+
+ Returns the name of the proxy.
+
+.. js:attribute:: Proxy.uuid
+
+ Contain the unique identifier of the proxy.
+
+ .. warning::
+ This attribute is now deprecated and will eventually be removed.
+ Please use :js:func:`Proxy.get_uuid()` function instead.
+
+.. js:function:: Proxy.get_uuid()
+
+ Returns the unique identifier of the proxy.
+
+.. js:attribute:: Proxy.servers
+
+ Contain a table with the attached servers. The table is indexed by server
+ name, and each server entry is an object of type :ref:`server_class`.
+
+.. js:attribute:: Proxy.stktable
+
+ Contains a stick table object of type :ref:`sticktable_class` attached to the
+ proxy.
+
+.. js:attribute:: Proxy.listeners
+
+ Contain a table with the attached listeners. The table is indexed by listener
+ name, and each each listeners entry is an object of type
+ :ref:`listener_class`.
+
+.. js:function:: Proxy.pause(px)
+
+ Pause the proxy. See the management socket documentation for more information.
+
+ :param class_proxy px: A :ref:`proxy_class` which indicates the manipulated
+ proxy.
+
+.. js:function:: Proxy.resume(px)
+
+ Resume the proxy. See the management socket documentation for more
+ information.
+
+ :param class_proxy px: A :ref:`proxy_class` which indicates the manipulated
+ proxy.
+
+.. js:function:: Proxy.stop(px)
+
+ Stop the proxy. See the management socket documentation for more information.
+
+ :param class_proxy px: A :ref:`proxy_class` which indicates the manipulated
+ proxy.
+
+.. js:function:: Proxy.shut_bcksess(px)
+
+ Kill the session attached to a backup server. See the management socket
+ documentation for more information.
+
+ :param class_proxy px: A :ref:`proxy_class` which indicates the manipulated
+ proxy.
+
+.. js:function:: Proxy.get_cap(px)
+
+ Returns a string describing the capabilities of the proxy.
+
+ :param class_proxy px: A :ref:`proxy_class` which indicates the manipulated
+ proxy.
+ :returns: a string "frontend", "backend", "proxy" or "ruleset".
+
+.. js:function:: Proxy.get_mode(px)
+
+ Returns a string describing the mode of the current proxy.
+
+ :param class_proxy px: A :ref:`proxy_class` which indicates the manipulated
+ proxy.
+ :returns: a string "tcp", "http", "syslog" or "unknown"
+
+.. js:function:: Proxy.get_srv_act(px)
+
+ Returns the number of current active servers for the current proxy that are
+ eligible for LB.
+
+ :param class_proxy px: A :ref:`proxy_class` which indicates the manipulated
+ proxy.
+ :returns: an integer
+
+.. js:function:: Proxy.get_srv_bck(px)
+
+ Returns the number backup servers for the current proxy that are eligible
+ for LB.
+
+ :param class_proxy px: A :ref:`proxy_class` which indicates the manipulated
+ proxy.
+ :returns: an integer
+
+.. js:function:: Proxy.get_stats(px)
+
+ Returns a table containing the proxy statistics. The statistics returned are
+ not the same if the proxy is frontend or a backend.
+
+ :param class_proxy px: A :ref:`proxy_class` which indicates the manipulated
+ proxy.
+ :returns: a key/value table containing stats
+
+.. js:function:: Proxy.get_mailers(px)
+
+ **LEGACY**
+
+ Returns a table containing mailers config for the current proxy or nil
+ if mailers are not available for the proxy.
+
+ :param class_proxy px: A :ref:`proxy_class` which indicates the manipulated
+ proxy.
+ :returns: a :ref:`proxy_mailers_class` containing proxy mailers config
+
+.. _proxy_mailers_class:
+
+ProxyMailers class
+==================
+
+**LEGACY**
+
+.. js:class:: ProxyMailers
+
+ This class provides mailers config for a given proxy.
+
+ If sending emails directly from lua, please consider
+ :js:func:`core.disable_legacy_mailers()` to disable the email sending from
+ haproxy. (Or email alerts will be sent twice...)
+
+.. js:attribute:: ProxyMailers.track_server_health
+
+ Boolean set to true if the option "log-health-checks" is configured on
+ the proxy, meaning that all server checks event should trigger email alerts.
+
+.. js:attribute:: ProxyMailers.log_level
+
+ An integer, the maximum log level that triggers email alerts. It is a number
+ between 0 and 7 as defined by option "email-alert level".
+
+.. js:attribute:: ProxyMailers.mailservers
+
+ An array containing the list of mail servers that should receive email alerts.
+ Each array entry is a name:desc pair where desc represents the full server
+ address (including port) as described in haproxy's configuration file.
+
+.. js:attribute:: ProxyMailers.mailservers_timeout
+
+ An integer representing the maximum time in milliseconds to wait for the
+ email to be sent. See "timeout mail" directive from "mailers" section in
+ haproxy configuration file.
+
+.. js:attribute:: ProxyMailers.smtp_hostname
+
+ A string containing the hostname to use for the SMTP transaction.
+ (option "email-alert myhostname")
+
+.. js:attribute:: ProxyMailers.smtp_from
+
+ A string containing the "MAIL FROM" address to use for the SMTP transaction.
+ (option "email-alert from")
+
+.. js:attribute:: ProxyMailers.smtp_to
+
+ A string containing the "RCPT TO" address to use for the SMTP transaction.
+ (option "email-alert to")
+
+.. _server_class:
+
+Server class
+============
+
+.. js:class:: Server
+
+ This class provides a way for manipulating servers and retrieving information.
+
+.. js:attribute:: Server.name
+
+ Contain the name of the server.
+
+ .. warning::
+ This attribute is now deprecated and will eventually be removed.
+ Please use :js:func:`Server.get_name()` function instead.
+
+.. js:function:: Server.get_name(sv)
+
+ Returns the name of the server.
+
+.. js:attribute:: Server.puid
+
+ Contain the proxy unique identifier of the server.
+
+ .. warning::
+ This attribute is now deprecated and will eventually be removed.
+ Please use :js:func:`Server.get_puid()` function instead.
+
+.. js:function:: Server.get_puid(sv)
+
+ Returns the proxy unique identifier of the server.
+
+.. js:function:: Server.get_rid(sv)
+
+ Returns the rid (revision ID) of the server.
+ It is an unsigned integer that is set upon server creation. Value is derived
+ from a global counter that starts at 0 and is incremented each time one or
+ multiple server deletions are followed by a server addition (meaning that
+ old name/id reuse could occur).
+
+ Combining server name/id with server rid yields a process-wide unique
+ identifier.
+
+.. js:function:: Server.is_draining(sv)
+
+ Return true if the server is currently draining sticky connections.
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+ :returns: a boolean
+
+.. js:function:: Server.is_backup(sv)
+
+ Return true if the server is a backup server
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+ :returns: a boolean
+
+.. js:function:: Server.is_dynamic(sv)
+
+ Return true if the server was instantiated at runtime (e.g.: from the cli)
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+ :returns: a boolean
+
+.. js:function:: Server.get_cur_sess(sv)
+
+ Return the number of currently active sessions on the server
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+ :returns: an integer
+
+.. js:function:: Server.get_pend_conn(sv)
+
+ Return the number of pending connections to the server
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+ :returns: an integer
+
+.. js:function:: Server.set_maxconn(sv, weight)
+
+ Dynamically change the maximum connections of the server. See the management
+ socket documentation for more information about the format of the string.
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+ :param string maxconn: A string describing the server maximum connections.
+
+.. js:function:: Server.get_maxconn(sv, weight)
+
+ This function returns an integer representing the server maximum connections.
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+ :returns: an integer.
+
+.. js:function:: Server.set_weight(sv, weight)
+
+ Dynamically change the weight of the server. See the management socket
+ documentation for more information about the format of the string.
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+ :param string weight: A string describing the server weight.
+
+.. js:function:: Server.get_weight(sv)
+
+ This function returns an integer representing the server weight.
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+ :returns: an integer.
+
+.. js:function:: Server.set_addr(sv, addr[, port])
+
+ Dynamically change the address of the server. See the management socket
+ documentation for more information about the format of the string.
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+ :param string addr: A string describing the server address.
+
+.. js:function:: Server.get_addr(sv)
+
+ Returns a string describing the address of the server.
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+ :returns: A string
+
+.. js:function:: Server.get_stats(sv)
+
+ Returns server statistics.
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+ :returns: a key/value table containing stats
+
+.. js:function:: Server.get_proxy(sv)
+
+ Returns the parent proxy to which the server belongs.
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+ :returns: a :ref:`proxy_class` or nil if not available
+
+.. js:function:: Server.shut_sess(sv)
+
+ Shutdown all the sessions attached to the server. See the management socket
+ documentation for more information about this function.
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+
+.. js:function:: Server.set_drain(sv)
+
+ Drain sticky sessions. See the management socket documentation for more
+ information about this function.
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+
+.. js:function:: Server.set_maint(sv)
+
+ Set maintenance mode. See the management socket documentation for more
+ information about this function.
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+
+.. js:function:: Server.set_ready(sv)
+
+ Set normal mode. See the management socket documentation for more information
+ about this function.
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+
+.. js:function:: Server.check_enable(sv)
+
+ Enable health checks. See the management socket documentation for more
+ information about this function.
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+
+.. js:function:: Server.check_disable(sv)
+
+ Disable health checks. See the management socket documentation for more
+ information about this function.
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+
+.. js:function:: Server.check_force_up(sv)
+
+ Force health-check up. See the management socket documentation for more
+ information about this function.
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+
+.. js:function:: Server.check_force_nolb(sv)
+
+ Force health-check nolb mode. See the management socket documentation for more
+ information about this function.
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+
+.. js:function:: Server.check_force_down(sv)
+
+ Force health-check down. See the management socket documentation for more
+ information about this function.
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+
+.. js:function:: Server.agent_enable(sv)
+
+ Enable agent check. See the management socket documentation for more
+ information about this function.
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+
+.. js:function:: Server.agent_disable(sv)
+
+ Disable agent check. See the management socket documentation for more
+ information about this function.
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+
+.. js:function:: Server.agent_force_up(sv)
+
+ Force agent check up. See the management socket documentation for more
+ information about this function.
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+
+.. js:function:: Server.agent_force_down(sv)
+
+ Force agent check down. See the management socket documentation for more
+ information about this function.
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+
+.. js:function:: Server.tracking(sv)
+
+ Check if the current server is tracking another server.
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+ :returns: A :ref:`server_class` which indicates the tracked server or nil if
+ the server doesn't track another one.
+
+.. js:function:: Server.get_trackers(sv)
+
+ Check if the current server is being tracked by other servers.
+
+ :param class_server sv: A :ref:`server_class` which indicates the manipulated
+ server.
+ :returns: An array of :ref:`server_class` which indicates the tracking
+ servers (might be empty)
+
+.. js:function:: Server.event_sub(sv, event_types, func)
+
+ Register a function that will be called on specific server events.
+ It works exactly like :js:func:`core.event_sub()` except that the subscription
+ will be performed within the server dedicated subscription list instead of the
+ global one.
+ (Your callback function will only be called for server events affecting sv)
+
+ See :js:func:`core.event_sub()` for function usage.
+
+ A key advantage to using :js:func:`Server.event_sub()` over
+ :js:func:`core.event_sub()` for servers is that :js:func:`Server.event_sub()`
+ allows you to be notified for servers events of a single server only.
+ It removes the needs for extra filtering in your callback function if you only
+ care about a single server, and also prevents useless wakeups.
+
+ For instance, if you want to be notified for UP/DOWN events on a given set of
+ servers, it is recommended to perform multiple per-server subscriptions since
+ it will be more efficient that doing a single global subscription that will
+ filter the received events.
+ Unless you really want to be notified for servers events of ALL servers of
+ course, which could make sense given you setup but should be avoided if you
+ have an important number of servers as it will add a significant load on your
+ haproxy process in case of multiple servers state change in a short amount of
+ time.
+
+ .. Note::
+ You may also combine :js:func:`core.event_sub()` with
+ :js:func:`Server.event_sub()`.
+
+ Also, don't forget that you can use :js:func:`core.register_task()` from
+ your callback function if needed. (ie: parallel work)
+
+ Here is a working example combining :js:func:`core.event_sub()` with
+ :js:func:`Server.event_sub()` and :js:func:`core.register_task()`
+ (This only serves as a demo, this is not necessarily useful to do so)
+
+.. code-block:: lua
+
+ core.event_sub({"SERVER_ADD"}, function(event, data, sub)
+ -- in the global event handler
+ if data["reference"] ~= nil then
+ print("Tracking new server: ", data["name"])
+ data["reference"]:event_sub({"SERVER_UP", "SERVER_DOWN"}, function(event, data, sub)
+ -- in the per-server event handler
+ if data["reference"] ~= nil then
+ core.register_task(function(server)
+ -- subtask to perform some async work (e.g.: HTTP API calls, sending emails...)
+ print("ASYNC: SERVER ", server:get_name(), " is ", event == "SERVER_UP" and "UP" or "DOWN")
+ end, data["reference"])
+ end
+ end)
+ end
+ end)
+
+..
+
+ In this example, we will first track global server addition events.
+ For each newly added server ("add server" on the cli), we will register a
+ UP/DOWN server subscription.
+ Then, the callback function will schedule the event handling in an async
+ subtask which will receive the server reference as an argument.
+
+.. _listener_class:
+
+Listener class
+==============
+
+.. js:function:: Listener.get_stats(ls)
+
+ Returns server statistics.
+
+ :param class_listener ls: A :ref:`listener_class` which indicates the
+ manipulated listener.
+ :returns: a key/value table containing stats
+
+.. _event_sub_class:
+
+EventSub class
+==============
+
+.. js:function:: EventSub.unsub()
+
+ End the subscription, the callback function will not be called again.
+
+.. _server_event_class:
+
+ServerEvent class
+=================
+
+.. js:class:: ServerEvent
+
+This class is provided with every **SERVER** events.
+
+See :js:func:`core.event_sub()` for more info.
+
+.. js:attribute:: ServerEvent.name
+
+ Contains the name of the server.
+
+.. js:attribute:: ServerEvent.puid
+
+ Contains the proxy-unique uid of the server
+
+.. js:attribute:: ServerEvent.rid
+
+ Contains the revision ID of the server
+
+.. js:attribute:: ServerEvent.proxy_name
+
+ Contains the name of the proxy to which the server belongs
+
+.. js:attribute:: ServerEvent.proxy_uuid
+
+ Contains the uuid of the proxy to which the server belongs
+
+.. js:attribute:: ServerEvent.reference
+
+ Reference to the live server (A :ref:`server_class`).
+
+ .. Warning::
+ Not available if the server was removed in the meantime.
+ (Will never be set for SERVER_DEL event since the server does not exist
+ anymore)
+
+.. js:attribute:: ServerEvent.state
+
+ A :ref:`server_event_state_class`
+
+ .. Note::
+ Only available for SERVER_STATE event
+
+.. js:attribute:: ServerEvent.admin
+
+ A :ref:`server_event_admin_class`
+
+ .. Note::
+ Only available for SERVER_ADMIN event
+
+.. js:attribute:: ServerEvent.check
+
+ A :ref:`server_event_checkres_class`
+
+ .. Note::
+ Only available for SERVER_CHECK event
+
+.. _server_event_checkres_class:
+
+ServerEventCheckRes class
+=========================
+
+.. js:class:: ServerEventCheckRes
+
+This class describes the result of a server's check.
+
+.. js:attribute:: ServerEventCheckRes.result
+
+ Effective check result.
+
+ Check result is a string and will be set to one of the following values:
+ - "FAILED": the check failed
+ - "PASSED": the check succeeded
+ - "CONDPASS": the check conditionally passed
+
+.. js:attribute:: ServerEventCheckRes.agent
+
+ Boolean set to true if the check is an agent check.
+ Else it is a health check.
+
+.. js:attribute:: ServerEventCheckRes.duration
+
+ Check's duration in milliseconds
+
+.. js:attribute:: ServerEventCheckRes.reason
+
+ Check's status. An array containing three fields:
+ - **short**: a string representing check status short name
+ - **desc**: a string representing check status description
+ - **code**: an integer, this extra information is provided for checks
+ that went through the data analysis stage (>= layer 5)
+
+.. js:attribute:: ServerEventCheckRes.health
+
+ An array containing values about check's health (integers):
+ - **cur**: current health counter:
+ - 0 to (**rise** - 1) = BAD
+ - **rise** to (**rise** + **fall** - 1) = GOOD
+ - **rise**: server will be considered as operational after **rise**
+ consecutive successful checks
+ - **fall**: server will be considered as dead after **fall** consecutive
+ unsuccessful checks
+
+.. _server_event_state_class:
+
+ServerEventState class
+======================
+
+.. js:class:: ServerEventState
+
+This class contains additional info related to **SERVER_STATE** event.
+
+.. js:attribute:: ServerEventState.admin
+
+ Boolean set to true if the server state change is due to an administrative
+ change. Else it is an operational change.
+
+.. js:attribute:: ServerEventState.check
+
+ A :ref:`server_event_checkres_class`, provided if the state change is
+ due to a server check (must be an operational change).
+
+.. js:attribute:: ServerEventState.cause
+
+ Printable state change cause. Might be empty.
+
+.. js:attribute:: ServerEventState.new_state
+
+ New server state due to operational or admin change.
+
+ It is a string that can be any of the following values:
+ - "STOPPED": The server is down
+ - "STOPPING": The server is up but soft-stopping
+ - "STARTING": The server is warming up
+ - "RUNNING": The server is fully up
+
+.. js:attribute:: ServerEventState.old_state
+
+ Previous server state prior to the operational or admin change.
+
+ Can be any value described in **new_state**, but they should differ.
+
+.. js:attribute:: ServerEventState.requeued
+
+ Number of connections that were requeued due to the server state change.
+
+ For a server going DOWN: it is the number of pending server connections
+ that are requeued to the backend (such connections will be redispatched
+ to any server that is suitable according to the configured load balancing
+ algorithm).
+
+ For a server doing UP: it is the number of pending connections on the
+ backend that may be redispatched to the server according to the load
+ balancing algorithm that is in use.
+
+.. _server_event_admin_class:
+
+ServerEventAdmin class
+======================
+
+.. js:class:: ServerEventAdmin
+
+This class contains additional info related to **SERVER_ADMIN** event.
+
+.. js:attribute:: ServerEventAdmin.cause
+
+ Printable admin state change cause. Might be empty.
+
+.. js:attribute:: ServerEventAdmin.new_admin
+
+ New server admin state due to the admin change.
+
+ It is an array of string containing a composition of following values:
+ - "**MAINT**": server is in maintenance mode
+ - "FMAINT": server is in forced maintenance mode (MAINT is also set)
+ - "IMAINT": server is in inherited maintenance mode (MAINT is also set)
+ - "RMAINT": server is in resolve maintenance mode (MAINT is also set)
+ - "CMAINT": server is in config maintenance mode (MAINT is also set)
+ - "**DRAIN**": server is in drain mode
+ - "FDRAIN": server is in forced drain mode (DRAIN is also set)
+ - "IDRAIN": server is in inherited drain mode (DRAIN is also set)
+
+.. js:attribute:: ServerEventAdmin.old_admin
+
+ Previous server admin state prior to the admin change.
+
+ Values are presented as in **new_admin**, but they should differ.
+ (Comparing old and new helps to find out the change(s))
+
+.. js:attribute:: ServerEventAdmin.requeued
+
+ Same as :js:attr:`ServerEventState.requeued` but when the requeue is due to
+ the server administrative state change.
+
+.. _queue_class:
+
+Queue class
+===========
+
+.. js:class:: Queue
+
+ This class provides a generic FIFO storage mechanism that may be shared
+ between multiple lua contexts to easily pass data between them, as stock
+ Lua doesn't provide easy methods for passing data between multiple coroutines.
+
+ inter-task example:
+
+.. code-block:: lua
+
+ -- script wide shared queue
+ local queue = core.queue()
+
+ -- master task
+ core.register_task(function()
+ -- send the date every second
+ while true do
+ queue:push(os.date("%c", core.now().sec))
+ core.sleep(1)
+ end
+ end)
+
+ -- worker task
+ core.register_task(function()
+ while true do
+ -- print the date sent by master
+ print(queue:pop_wait())
+ end
+ end)
+..
+
+ Of course, queue may also be used as a local storage mechanism.
+
+ Use :js:func:`core.queue` to get a new Queue object.
+
+.. js:function:: Queue.size(queue)
+
+ This function returns the number of items within the Queue.
+
+ :param class_queue queue: A :ref:`queue_class` to the current queue
+
+.. js:function:: Queue.push(queue, item)
+
+ This function pushes the item (may be of any type) to the queue.
+ Pushed item cannot be nil or invalid, or an error will be thrown.
+
+ :param class_queue queue: A :ref:`queue_class` to the current queue
+ :returns: boolean true for success and false for error
+
+.. js:function:: Queue.pop(queue)
+
+ This function immediately tries to pop an item from the queue.
+ It returns nil of no item is available at the time of the call.
+
+ :param class_queue queue: A :ref:`queue_class` to the current queue
+ :returns: the item at the top of the stack (any type) or nil if no items
+
+.. js:function:: Queue.pop_wait(queue)
+
+ **context**: task
+
+ This is an alternative to pop() that may be used within task contexts.
+
+ The call waits for data if no item is currently available. This may be
+ useful when used in a while loop to prevent cpu waste.
+
+ Note that this requires yielding, thus it is only available within contexts
+ that support yielding (mainly task context).
+
+ :param class_queue queue: A :ref:`queue_class` to the current queue
+ :returns: the item at the top of the stack (any type) or nil in case of error
+
+.. _concat_class:
+
+Concat class
+============
+
+.. js:class:: Concat
+
+ This class provides a fast way for string concatenation. The way using native
+ Lua concatenation like the code below is slow for some reasons.
+
+.. code-block:: lua
+
+ str = "string1"
+ str = str .. ", string2"
+ str = str .. ", string3"
+..
+
+ For each concatenation, Lua:
+ - allocates memory for the result,
+ - catenates the two string copying the strings in the new memory block,
+ - frees the old memory block containing the string which is no longer used.
+
+ This process does many memory move, allocation and free. In addition, the
+ memory is not really freed, it is just marked as unused and waits for the
+ garbage collector.
+
+ The Concat class provides an alternative way to concatenate strings. It uses
+ the internal Lua mechanism (it does not allocate memory), but it doesn't copy
+ the data more than once.
+
+ On my computer, the following loops spends 0.2s for the Concat method and
+ 18.5s for the pure Lua implementation. So, the Concat class is about 1000x
+ faster than the embedded solution.
+
+.. code-block:: lua
+
+ for j = 1, 100 do
+ c = core.concat()
+ for i = 1, 20000 do
+ c:add("#####")
+ end
+ end
+..
+
+.. code-block:: lua
+
+ for j = 1, 100 do
+ c = ""
+ for i = 1, 20000 do
+ c = c .. "#####"
+ end
+ end
+..
+
+.. js:function:: Concat.add(concat, string)
+
+ This function adds a string to the current concatenated string.
+
+ :param class_concat concat: A :ref:`concat_class` which contains the currently
+ built string.
+ :param string string: A new string to concatenate to the current built
+ string.
+
+.. js:function:: Concat.dump(concat)
+
+ This function returns the concatenated string.
+
+ :param class_concat concat: A :ref:`concat_class` which contains the currently
+ built string.
+ :returns: the concatenated string
+
+.. _fetches_class:
+
+Fetches class
+=============
+
+.. js:class:: Fetches
+
+ This class contains a lot of internal HAProxy sample fetches. See the
+ HAProxy "configuration.txt" documentation for more information.
+ (chapters 7.3.2 to 7.3.6)
+
+ .. warning::
+ some sample fetches are not available in some context. These limitations
+ are specified in this documentation when they're useful.
+
+ :see: :js:attr:`TXN.f`
+ :see: :js:attr:`TXN.sf`
+
+ Fetches are useful to:
+
+ * get system time,
+ * get environment variable,
+ * get random numbers,
+ * know backend status like the number of users in queue or the number of
+ connections established,
+ * get client information like ip source or destination,
+ * deal with stick tables,
+ * fetch established SSL information,
+ * fetch HTTP information like headers or method.
+
+.. code-block:: lua
+
+ function action(txn)
+ -- Get source IP
+ local clientip = txn.f:src()
+ end
+..
+
+.. _converters_class:
+
+Converters class
+================
+
+.. js:class:: Converters
+
+ This class contains a lot of internal HAProxy sample converters. See the
+ HAProxy documentation "configuration.txt" for more information about her
+ usage. Its the chapter 7.3.1.
+
+ :see: :js:attr:`TXN.c`
+ :see: :js:attr:`TXN.sc`
+
+ Converters provides stateful transformation. They are useful to:
+
+ * convert input to base64,
+ * apply hash on input string (djb2, crc32, sdbm, wt6),
+ * format date,
+ * json escape,
+ * extract preferred language comparing two lists,
+ * turn to lower or upper chars,
+ * deal with stick tables.
+
+.. _channel_class:
+
+Channel class
+=============
+
+.. js:class:: Channel
+
+ **context**: action, sample-fetch, convert, filter
+
+ HAProxy uses two buffers for the processing of the requests. The first one is
+ used with the request data (from the client to the server) and the second is
+ used for the response data (from the server to the client).
+
+ Each buffer contains two types of data. The first type is the incoming data
+ waiting for a processing. The second part is the outgoing data already
+ processed. Usually, the incoming data is processed, after it is tagged as
+ outgoing data, and finally it is sent. The following functions provides tools
+ for manipulating these data in a buffer.
+
+ The following diagram shows where the channel class function are applied.
+
+ .. image:: _static/channel.png
+
+ .. warning::
+ It is not possible to read from the response in request action, and it is
+ not possible to read from the request channel in response action.
+
+ .. warning::
+ It is forbidden to alter the Channels buffer from HTTP contexts. So only
+ :js:func:`Channel.input`, :js:func:`Channel.output`,
+ :js:func:`Channel.may_recv`, :js:func:`Channel.is_full` and
+ :js:func:`Channel.is_resp` can be called from a HTTP context.
+
+ All the functions provided by this class are available in the
+ **sample-fetches**, **actions** and **filters** contexts. For **filters**,
+ incoming data (offset and length) are relative to the filter. Some functions
+ may yield, but only for **actions**. Yield is not possible for
+ **sample-fetches**, **converters** and **filters**.
+
+.. js:function:: Channel.append(channel, string)
+
+ This function copies the string **string** at the end of incoming data of the
+ channel buffer. The function returns the copied length on success or -1 if
+ data cannot be copied.
+
+ Same that :js:func:`Channel.insert(channel, string, channel:input())`.
+
+ :param class_channel channel: The manipulated Channel.
+ :param string string: The data to copy at the end of incoming data.
+ :returns: an integer containing the amount of bytes copied or -1.
+
+.. js:function:: Channel.data(channel [, offset [, length]])
+
+ This function returns **length** bytes of incoming data from the channel
+ buffer, starting at the offset **offset**. The data are not removed from the
+ buffer.
+
+ By default, if no length is provided, all incoming data found, starting at the
+ given offset, are returned. If **length** is set to -1, the function tries to
+ retrieve a maximum of data and, if called by an action, it yields if
+ necessary. It also waits for more data if the requested length exceeds the
+ available amount of incoming data. Not providing an offset is the same as
+ setting it to 0. A positive offset is relative to the beginning of incoming
+ data of the channel buffer while negative offset is relative to the end.
+
+ If there is no incoming data and the channel can't receive more data, a 'nil'
+ value is returned.
+
+ :param class_channel channel: The manipulated Channel.
+ :param integer offset: *optional* The offset in incoming data to start to get
+ data. 0 by default. May be negative to be relative to the end of incoming
+ data.
+ :param integer length: *optional* The expected length of data to retrieve. All
+ incoming data by default. May be set to -1 to get a maximum of data.
+ :returns: a string containing the data found or nil.
+
+.. js:function:: Channel.forward(channel, length)
+
+ This function forwards **length** bytes of data from the channel buffer. If
+ the requested length exceeds the available amount of incoming data, and if
+ called by an action, the function yields, waiting for more data to forward. It
+ returns the amount of data forwarded.
+
+ :param class_channel channel: The manipulated Channel.
+ :param integer int: The amount of data to forward.
+
+.. js:function:: Channel.input(channel)
+
+ This function returns the length of incoming data in the channel buffer. When
+ called by a filter, this value is relative to the filter.
+
+ :param class_channel channel: The manipulated Channel.
+ :returns: an integer containing the amount of available bytes.
+
+.. js:function:: Channel.insert(channel, string [, offset])
+
+ This function copies the string **string** at the offset **offset** in
+ incoming data of the channel buffer. The function returns the copied length on
+ success or -1 if data cannot be copied.
+
+ By default, if no offset is provided, the string is copied in front of
+ incoming data. A positive offset is relative to the beginning of incoming data
+ of the channel buffer while negative offset is relative to their end.
+
+ :param class_channel channel: The manipulated Channel.
+ :param string string: The data to copy into incoming data.
+ :param integer offset: *optional* The offset in incoming data where to copy
+ data. 0 by default. May be negative to be relative to the end of incoming
+ data.
+ :returns: an integer containing the amount of bytes copied or -1.
+
+.. js:function:: Channel.is_full(channel)
+
+ This function returns true if the channel buffer is full.
+
+ :param class_channel channel: The manipulated Channel.
+ :returns: a boolean
+
+.. js:function:: Channel.is_resp(channel)
+
+ This function returns true if the channel is the response one.
+
+ :param class_channel channel: The manipulated Channel.
+ :returns: a boolean
+
+.. js:function:: Channel.line(channel [, offset [, length]])
+
+ This function parses **length** bytes of incoming data of the channel buffer,
+ starting at offset **offset**, and returns the first line found, including the
+ '\\n'. The data are not removed from the buffer. If no line is found, all
+ data are returned.
+
+ By default, if no length is provided, all incoming data, starting at the given
+ offset, are evaluated. If **length** is set to -1, the function tries to
+ retrieve a maximum of data and, if called by an action, yields if
+ necessary. It also waits for more data if the requested length exceeds the
+ available amount of incoming data. Not providing an offset is the same as
+ setting it to 0. A positive offset is relative to the beginning of incoming
+ data of the channel buffer while negative offset is relative to the end.
+
+ If there is no incoming data and the channel can't receive more data, a 'nil'
+ value is returned.
+
+ :param class_channel channel: The manipulated Channel.
+ :param integer offset: *optional* The offset in incoming data to start to
+ parse data. 0 by default. May be negative to be relative to the end of
+ incoming data.
+ :param integer length: *optional* The length of data to parse. All incoming
+ data by default. May be set to -1 to get a maximum of data.
+ :returns: a string containing the line found or nil.
+
+.. js:function:: Channel.may_recv(channel)
+
+ This function returns true if the channel may still receive data.
+
+ :param class_channel channel: The manipulated Channel.
+ :returns: a boolean
+
+.. js:function:: Channel.output(channel)
+
+ This function returns the length of outgoing data of the channel buffer. When
+ called by a filter, this value is relative to the filter.
+
+ :param class_channel channel: The manipulated Channel.
+ :returns: an integer containing the amount of available bytes.
+
+.. js:function:: Channel.prepend(channel, string)
+
+ This function copies the string **string** in front of incoming data of the
+ channel buffer. The function returns the copied length on success or -1 if
+ data cannot be copied.
+
+ Same that :js:func:`Channel.insert(channel, string, 0)`.
+
+ :param class_channel channel: The manipulated Channel.
+ :param string string: The data to copy in front of incoming data.
+ :returns: an integer containing the amount of bytes copied or -1.
+
+.. js:function:: Channel.remove(channel [, offset [, length]])
+
+ This function removes **length** bytes of incoming data of the channel buffer,
+ starting at offset **offset**. This function returns number of bytes removed
+ on success.
+
+ By default, if no length is provided, all incoming data, starting at the given
+ offset, are removed. Not providing an offset is the same as setting it
+ to 0. A positive offset is relative to the beginning of incoming data of the
+ channel buffer while negative offset is relative to the end.
+
+ :param class_channel channel: The manipulated Channel.
+ :param integer offset: *optional* The offset in incoming data where to start
+ to remove data. 0 by default. May be negative to be relative to the end of
+ incoming data.
+ :param integer length: *optional* The length of data to remove. All incoming
+ data by default.
+ :returns: an integer containing the amount of bytes removed.
+
+.. js:function:: Channel.send(channel, string)
+
+ This function requires immediate send of the string **string**. It means the
+ string is copied at the beginning of incoming data of the channel buffer and
+ immediately forwarded. Unless if the connection is close, and if called by an
+ action, this function yields to copy and forward all the string.
+
+ :param class_channel channel: The manipulated Channel.
+ :param string string: The data to send.
+ :returns: an integer containing the amount of bytes copied or -1.
+
+.. js:function:: Channel.set(channel, string [, offset [, length]])
+
+ This function replaces **length** bytes of incoming data of the channel
+ buffer, starting at offset **offset**, by the string **string**. The function
+ returns the copied length on success or -1 if data cannot be copied.
+
+ By default, if no length is provided, all incoming data, starting at the given
+ offset, are replaced. Not providing an offset is the same as setting it
+ to 0. A positive offset is relative to the beginning of incoming data of the
+ channel buffer while negative offset is relative to the end.
+
+ :param class_channel channel: The manipulated Channel.
+ :param string string: The data to copy into incoming data.
+ :param integer offset: *optional* The offset in incoming data where to start
+ the data replacement. 0 by default. May be negative to be relative to the
+ end of incoming data.
+ :param integer length: *optional* The length of data to replace. All incoming
+ data by default.
+ :returns: an integer containing the amount of bytes copied or -1.
+
+.. js:function:: Channel.dup(channel)
+
+ **DEPRECATED**
+
+ This function returns all incoming data found in the channel buffer. The data
+ are not removed from the buffer and can be reprocessed later.
+
+ If there is no incoming data and the channel can't receive more data, a 'nil'
+ value is returned.
+
+ :param class_channel channel: The manipulated Channel.
+ :returns: a string containing all data found or nil.
+
+ .. warning::
+ This function is deprecated. :js:func:`Channel.data()` must be used
+ instead.
+
+.. js:function:: Channel.get(channel)
+
+ **DEPRECATED**
+
+ This function returns all incoming data found in the channel buffer and remove
+ them from the buffer.
+
+ If there is no incoming data and the channel can't receive more data, a 'nil'
+ value is returned.
+
+ :param class_channel channel: The manipulated Channel.
+ :returns: a string containing all the data found or nil.
+
+ .. warning::
+ This function is deprecated. :js:func:`Channel.data()` must be used to
+ retrieve data followed by a call to :js:func:`Channel:remove()` to remove
+ data.
+
+ .. code-block:: lua
+
+ local data = chn:data()
+ chn:remove(0, data:len())
+
+ ..
+
+.. js:function:: Channel.getline(channel)
+
+ **DEPRECATED**
+
+ This function returns the first line found in incoming data of the channel
+ buffer, including the '\\n'. The returned data are removed from the buffer. If
+ no line is found, and if called by an action, this function yields to wait for
+ more data, except if the channel can't receive more data. In this case all
+ data are returned.
+
+ If there is no incoming data and the channel can't receive more data, a 'nil'
+ value is returned.
+
+ :param class_channel channel: The manipulated Channel.
+ :returns: a string containing the line found or nil.
+
+ .. warning::
+ This function is deprecated. :js:func:`Channel.line()` must be used to
+ retrieve a line followed by a call to :js:func:`Channel:remove()` to remove
+ data.
+
+ .. code-block:: lua
+
+ local line = chn:line(0, -1)
+ chn:remove(0, line:len())
+
+ ..
+
+.. js:function:: Channel.get_in_len(channel)
+
+ **DEPRECATED**
+
+ This function returns the length of the input part of the buffer. When called
+ by a filter, this value is relative to the filter.
+
+ :param class_channel channel: The manipulated Channel.
+ :returns: an integer containing the amount of available bytes.
+
+ .. warning::
+ This function is deprecated. :js:func:`Channel.input()` must be used
+ instead.
+
+.. js:function:: Channel.get_out_len(channel)
+
+ **DEPRECATED**
+
+ This function returns the length of the output part of the buffer. When called
+ by a filter, this value is relative to the filter.
+
+ :param class_channel channel: The manipulated Channel.
+ :returns: an integer containing the amount of available bytes.
+
+ .. warning::
+ This function is deprecated. :js:func:`Channel.output()` must be used
+ instead.
+
+.. _http_class:
+
+HTTP class
+==========
+
+.. js:class:: HTTP
+
+ This class contain all the HTTP manipulation functions.
+
+.. js:function:: HTTP.req_get_headers(http)
+
+ Returns a table containing all the request headers.
+
+ :param class_http http: The related http object.
+ :returns: table of headers.
+ :see: :js:func:`HTTP.res_get_headers`
+
+ This is the form of the returned table:
+
+.. code-block:: lua
+
+ HTTP:req_get_headers()['<header-name>'][<header-index>] = "<header-value>"
+
+ local hdr = HTTP:req_get_headers()
+ hdr["host"][0] = "www.test.com"
+ hdr["accept"][0] = "audio/basic q=1"
+ hdr["accept"][1] = "audio/*, q=0.2"
+ hdr["accept"][2] = "*/*, q=0.1"
+..
+
+.. js:function:: HTTP.res_get_headers(http)
+
+ Returns a table containing all the response headers.
+
+ :param class_http http: The related http object.
+ :returns: table of headers.
+ :see: :js:func:`HTTP.req_get_headers`
+
+ This is the form of the returned table:
+
+.. code-block:: lua
+
+ HTTP:res_get_headers()['<header-name>'][<header-index>] = "<header-value>"
+
+ local hdr = HTTP:req_get_headers()
+ hdr["host"][0] = "www.test.com"
+ hdr["accept"][0] = "audio/basic q=1"
+ hdr["accept"][1] = "audio/*, q=0.2"
+ hdr["accept"][2] = "*.*, q=0.1"
+..
+
+.. js:function:: HTTP.req_add_header(http, name, value)
+
+ Appends a HTTP header field in the request whose name is
+ specified in "name" and whose value is defined in "value".
+
+ :param class_http http: The related http object.
+ :param string name: The header name.
+ :param string value: The header value.
+ :see: :js:func:`HTTP.res_add_header`
+
+.. js:function:: HTTP.res_add_header(http, name, value)
+
+ Appends a HTTP header field in the response whose name is
+ specified in "name" and whose value is defined in "value".
+
+ :param class_http http: The related http object.
+ :param string name: The header name.
+ :param string value: The header value.
+ :see: :js:func:`HTTP.req_add_header`
+
+.. js:function:: HTTP.req_del_header(http, name)
+
+ Removes all HTTP header fields in the request whose name is
+ specified in "name".
+
+ :param class_http http: The related http object.
+ :param string name: The header name.
+ :see: :js:func:`HTTP.res_del_header`
+
+.. js:function:: HTTP.res_del_header(http, name)
+
+ Removes all HTTP header fields in the response whose name is
+ specified in "name".
+
+ :param class_http http: The related http object.
+ :param string name: The header name.
+ :see: :js:func:`HTTP.req_del_header`
+
+.. js:function:: HTTP.req_set_header(http, name, value)
+
+ This variable replace all occurrence of all header "name", by only
+ one containing the "value".
+
+ :param class_http http: The related http object.
+ :param string name: The header name.
+ :param string value: The header value.
+ :see: :js:func:`HTTP.res_set_header`
+
+ This function does the same work as the following code:
+
+.. code-block:: lua
+
+ function fcn(txn)
+ TXN.http:req_del_header("header")
+ TXN.http:req_add_header("header", "value")
+ end
+..
+
+.. js:function:: HTTP.res_set_header(http, name, value)
+
+ This function replaces all occurrence of all header "name", by only
+ one containing the "value".
+
+ :param class_http http: The related http object.
+ :param string name: The header name.
+ :param string value: The header value.
+ :see: :js:func:`HTTP.req_rep_header()`
+
+.. js:function:: HTTP.req_rep_header(http, name, regex, replace)
+
+ Matches the regular expression in all occurrences of header field "name"
+ according to "regex", and replaces them with the "replace" argument. The
+ replacement value can contain back references like \1, \2, ... This
+ function works with the request.
+
+ :param class_http http: The related http object.
+ :param string name: The header name.
+ :param string regex: The match regular expression.
+ :param string replace: The replacement value.
+ :see: :js:func:`HTTP.res_rep_header()`
+
+.. js:function:: HTTP.res_rep_header(http, name, regex, string)
+
+ Matches the regular expression in all occurrences of header field "name"
+ according to "regex", and replaces them with the "replace" argument. The
+ replacement value can contain back references like \1, \2, ... This
+ function works with the request.
+
+ :param class_http http: The related http object.
+ :param string name: The header name.
+ :param string regex: The match regular expression.
+ :param string replace: The replacement value.
+ :see: :js:func:`HTTP.req_rep_header()`
+
+.. js:function:: HTTP.req_set_method(http, method)
+
+ Rewrites the request method with the parameter "method".
+
+ :param class_http http: The related http object.
+ :param string method: The new method.
+
+.. js:function:: HTTP.req_set_path(http, path)
+
+ Rewrites the request path with the "path" parameter.
+
+ :param class_http http: The related http object.
+ :param string path: The new path.
+
+.. js:function:: HTTP.req_set_query(http, query)
+
+ Rewrites the request's query string which appears after the first question
+ mark ("?") with the parameter "query".
+
+ :param class_http http: The related http object.
+ :param string query: The new query.
+
+.. js:function:: HTTP.req_set_uri(http, uri)
+
+ Rewrites the request URI with the parameter "uri".
+
+ :param class_http http: The related http object.
+ :param string uri: The new uri.
+
+.. js:function:: HTTP.res_set_status(http, status [, reason])
+
+ Rewrites the response status code with the parameter "code".
+
+ If no custom reason is provided, it will be generated from the status.
+
+ :param class_http http: The related http object.
+ :param integer status: The new response status code.
+ :param string reason: The new response reason (optional).
+
+.. _httpclient_class:
+
+HTTPClient class
+================
+
+.. js:class:: HTTPClient
+
+ The httpclient class allows issue of outbound HTTP requests through a simple
+ API without the knowledge of HAProxy internals.
+
+.. js:function:: HTTPClient.get(httpclient, request)
+.. js:function:: HTTPClient.head(httpclient, request)
+.. js:function:: HTTPClient.put(httpclient, request)
+.. js:function:: HTTPClient.post(httpclient, request)
+.. js:function:: HTTPClient.delete(httpclient, request)
+
+ Send a HTTP request and wait for a response. GET, HEAD PUT, POST and DELETE
+ methods can be used.
+ The HTTPClient will send asynchronously the data and is able to send and
+ receive more than HAProxy bufsize.
+
+ The HTTPClient interface is not able to decompress responses, it is not
+ recommended to send an Accept-Encoding in the request so the response is
+ received uncompressed.
+
+ :param class httpclient: Is the manipulated HTTPClient.
+ :param table request: Is a table containing the parameters of the request
+ that will be send.
+ :param string request.url: Is a mandatory parameter for the request that
+ contains the URL.
+ :param string request.body: Is an optional parameter for the request that
+ contains the body to send.
+ :param table request.headers: Is an optional parameter for the request that
+ contains the headers to send.
+ :param string request.dst: Is an optional parameter for the destination in
+ haproxy address format.
+ :param integer request.timeout: Optional timeout parameter, set a
+ "timeout server" on the connections.
+ :returns: Lua table containing the response
+
+
+.. code-block:: lua
+
+ local httpclient = core.httpclient()
+ local response = httpclient:post{url="http://127.0.0.1", body=body, dst="unix@/var/run/http.sock"}
+
+..
+
+.. code-block:: lua
+
+ response = {
+ status = 400,
+ reason = "Bad request",
+ headers = {
+ ["content-type"] = { "text/html" },
+ ["cache-control"] = { "no-cache", "no-store" },
+ },
+ body = "<html><body><h1>invalid request<h1></body></html>",
+ }
+..
+
+
+.. _txn_class:
+
+TXN class
+=========
+
+.. js:class:: TXN
+
+ The txn class contain all the functions relative to the http or tcp
+ transaction (Note than a tcp stream is the same than a tcp transaction, but
+ a HTTP transaction is not the same than a tcp stream).
+
+ The usage of this class permits to retrieve data from the requests, alter it
+ and forward it.
+
+ All the functions provided by this class are available in the context
+ **sample-fetches**, **actions** and **filters**.
+
+.. js:attribute:: TXN.c
+
+ :returns: An :ref:`converters_class`.
+
+ This attribute contains a Converters class object.
+
+.. js:attribute:: TXN.sc
+
+ :returns: An :ref:`converters_class`.
+
+ This attribute contains a Converters class object. The functions of
+ this object returns always a string.
+
+.. js:attribute:: TXN.f
+
+ :returns: An :ref:`fetches_class`.
+
+ This attribute contains a Fetches class object.
+
+.. js:attribute:: TXN.sf
+
+ :returns: An :ref:`fetches_class`.
+
+ This attribute contains a Fetches class object. The functions of
+ this object returns always a string.
+
+.. js:attribute:: TXN.req
+
+ :returns: An :ref:`channel_class`.
+
+ This attribute contains a channel class object for the request buffer.
+
+.. js:attribute:: TXN.res
+
+ :returns: An :ref:`channel_class`.
+
+ This attribute contains a channel class object for the response buffer.
+
+.. js:attribute:: TXN.http
+
+ :returns: An :ref:`http_class`.
+
+ This attribute contains a HTTP class object. It is available only if the
+ proxy has the "mode http" enabled.
+
+.. js:attribute:: TXN.http_req
+
+ :returns: An :ref:`httpmessage_class`.
+
+ This attribute contains the request HTTPMessage class object. It is available
+ only if the proxy has the "mode http" enabled and only in the **filters**
+ context.
+
+.. js:attribute:: TXN.http_res
+
+ :returns: An :ref:`httpmessage_class`.
+
+ This attribute contains the response HTTPMessage class object. It is available
+ only if the proxy has the "mode http" enabled and only in the **filters**
+ context.
+
+.. js:function:: TXN.log(TXN, loglevel, msg)
+
+ This function sends a log. The log is sent, according with the HAProxy
+ configuration file, to the loggers relevant to the current context and/or
+ to stderr if it is allowed.
+
+ The exact behaviour depends on tune.lua.log.loggers and tune.lua.log.stderr.
+
+ :param class_txn txn: The class txn object containing the data.
+ :param integer loglevel: Is the log level associated with the message. It is
+ a number between 0 and 7.
+ :param string msg: The log content.
+ :see: :js:attr:`core.emerg`, :js:attr:`core.alert`, :js:attr:`core.crit`,
+ :js:attr:`core.err`, :js:attr:`core.warning`, :js:attr:`core.notice`,
+ :js:attr:`core.info`, :js:attr:`core.debug` (log level definitions)
+ :see: :js:func:`TXN.deflog`
+ :see: :js:func:`TXN.Debug`
+ :see: :js:func:`TXN.Info`
+ :see: :js:func:`TXN.Warning`
+ :see: :js:func:`TXN.Alert`
+
+.. js:function:: TXN.deflog(TXN, msg)
+
+ Sends a log line with the default loglevel for the proxy associated with the
+ transaction.
+
+ :param class_txn txn: The class txn object containing the data.
+ :param string msg: The log content.
+ :see: :js:func:`TXN.log`
+
+.. js:function:: TXN.Debug(txn, msg)
+
+ :param class_txn txn: The class txn object containing the data.
+ :param string msg: The log content.
+ :see: :js:func:`TXN.log`
+
+ Does the same job as:
+
+.. code-block:: lua
+
+ function Debug(txn, msg)
+ TXN.log(txn, core.debug, msg)
+ end
+..
+
+.. js:function:: TXN.Info(txn, msg)
+
+ :param class_txn txn: The class txn object containing the data.
+ :param string msg: The log content.
+ :see: :js:func:`TXN.log`
+
+ Does the same job as:
+
+.. code-block:: lua
+
+ function Info(txn, msg)
+ TXN.log(txn, core.info, msg)
+ end
+..
+
+.. js:function:: TXN.Warning(txn, msg)
+
+ :param class_txn txn: The class txn object containing the data.
+ :param string msg: The log content.
+ :see: :js:func:`TXN.log`
+
+ Does the same job as:
+
+.. code-block:: lua
+
+ function Warning(txn, msg)
+ TXN.log(txn, core.warning, msg)
+ end
+..
+
+.. js:function:: TXN.Alert(txn, msg)
+
+ :param class_txn txn: The class txn object containing the data.
+ :param string msg: The log content.
+ :see: :js:func:`TXN.log`
+
+ Does the same job as:
+
+.. code-block:: lua
+
+ function Alert(txn, msg)
+ TXN.log(txn, core.alert, msg)
+ end
+..
+
+.. js:function:: TXN.get_priv(txn)
+
+ Return Lua data stored in the current transaction (with the `TXN.set_priv()`)
+ function. If no data are stored, it returns a nil value.
+
+ :param class_txn txn: The class txn object containing the data.
+ :returns: the opaque data previously stored, or nil if nothing is
+ available.
+
+.. js:function:: TXN.set_priv(txn, data)
+
+ Store any data in the current HAProxy transaction. This action replaces the
+ old stored data.
+
+ :param class_txn txn: The class txn object containing the data.
+ :param opaque data: The data which is stored in the transaction.
+
+.. js:function:: TXN.set_var(TXN, var, value[, ifexist])
+
+ Converts a Lua type in a HAProxy type and store it in a variable <var>.
+
+ :param class_txn txn: The class txn object containing the data.
+ :param string var: The variable name according with the HAProxy variable
+ syntax.
+ :param type value: The value associated to the variable. The type can be
+ string or integer.
+ :param boolean ifexist: If this parameter is set to true the variable will
+ only be set if it was defined elsewhere (i.e. used within the configuration).
+ For global variables (using the "proc" scope), they will only be updated and
+ never created. It is highly recommended to always set this to true.
+
+.. js:function:: TXN.unset_var(TXN, var)
+
+ Unset the variable <var>.
+
+ :param class_txn txn: The class txn object containing the data.
+ :param string var: The variable name according with the HAProxy variable
+ syntax.
+
+.. js:function:: TXN.get_var(TXN, var)
+
+ Returns data stored in the variable <var> converter in Lua type.
+
+ :param class_txn txn: The class txn object containing the data.
+ :param string var: The variable name according with the HAProxy variable
+ syntax.
+
+.. js:function:: TXN.reply([reply])
+
+ Return a new reply object
+
+ :param table reply: A table containing info to initialize the reply fields.
+ :returns: A :ref:`reply_class` object.
+
+ The table used to initialized the reply object may contain following entries :
+
+ * status : The reply status code. the code 200 is used by default.
+ * reason : The reply reason. The reason corresponding to the status code is
+ used by default.
+ * headers : A list of headers, indexed by header name. Empty by default. For
+ a given name, multiple values are possible, stored in an ordered list.
+ * body : The reply body, empty by default.
+
+.. code-block:: lua
+
+ local reply = txn:reply{
+ status = 400,
+ reason = "Bad request",
+ headers = {
+ ["content-type"] = { "text/html" },
+ ["cache-control"] = {"no-cache", "no-store" }
+ },
+ body = "<html><body><h1>invalid request<h1></body></html>"
+ }
+..
+ :see: :js:class:`Reply`
+
+.. js:function:: TXN.done(txn[, reply])
+
+ This function terminates processing of the transaction and the associated
+ session and optionally reply to the client for HTTP sessions.
+
+ :param class_txn txn: The class txn object containing the data.
+ :param class_reply reply: The class reply object to return to the client.
+
+ This functions can be used when a critical error is detected or to terminate
+ processing after some data have been returned to the client (eg: a redirect).
+ To do so, a reply may be provided. This object is optional and may contain a
+ status code, a reason, a header list and a body. All these fields are
+ optional. When not provided, the default values are used. By default, with an
+ empty reply object, an empty HTTP 200 response is returned to the client. If
+ no reply object is provided, the transaction is terminated without any
+ reply. If a reply object is provided, it must not exceed the buffer size once
+ converted into the internal HTTP representation. Because for now there is no
+ easy way to be sure it fits, it is probably better to keep it reasonably
+ small.
+
+ The reply object may be fully created in lua or the class Reply may be used to
+ create it.
+
+.. code-block:: lua
+
+ local reply = txn:reply()
+ reply:set_status(400, "Bad request")
+ reply:add_header("content-type", "text/html")
+ reply:add_header("cache-control", "no-cache")
+ reply:add_header("cache-control", "no-store")
+ reply:set_body("<html><body><h1>invalid request<h1></body></html>")
+ txn:done(reply)
+..
+
+.. code-block:: lua
+
+ txn:done{
+ status = 400,
+ reason = "Bad request",
+ headers = {
+ ["content-type"] = { "text/html" },
+ ["cache-control"] = { "no-cache", "no-store" },
+ },
+ body = "<html><body><h1>invalid request<h1></body></html>"
+ }
+..
+
+ .. warning::
+ It does not make sense to call this function from sample-fetches. In this
+ case the behavior is the same than core.done(): it finishes the Lua
+ execution. The transaction is really aborted only from an action registered
+ function.
+
+ :see: :js:func:`TXN.reply`, :js:class:`Reply`
+
+.. js:function:: TXN.set_loglevel(txn, loglevel)
+
+ Is used to change the log level of the current request. The "loglevel" must
+ be an integer between 0 and 7.
+
+ :param class_txn txn: The class txn object containing the data.
+ :param integer loglevel: The required log level. This variable can be one of
+ :see: :js:attr:`core.emerg`, :js:attr:`core.alert`, :js:attr:`core.crit`,
+ :js:attr:`core.err`, :js:attr:`core.warning`, :js:attr:`core.notice`,
+ :js:attr:`core.info`, :js:attr:`core.debug` (log level definitions)
+
+.. js:function:: TXN.set_tos(txn, tos)
+
+ Is used to set the TOS or DSCP field value of packets sent to the client to
+ the value passed in "tos" on platforms which support this.
+
+ :param class_txn txn: The class txn object containing the data.
+ :param integer tos: The new TOS os DSCP.
+
+.. js:function:: TXN.set_mark(txn, mark)
+
+ Is used to set the Netfilter MARK on all packets sent to the client to the
+ value passed in "mark" on platforms which support it.
+
+ :param class_txn txn: The class txn object containing the data.
+ :param integer mark: The mark value.
+
+.. js:function:: TXN.set_priority_class(txn, prio)
+
+ This function adjusts the priority class of the transaction. The value should
+ be within the range -2047..2047. Values outside this range will be
+ truncated.
+
+ See the HAProxy configuration.txt file keyword "http-request" action
+ "set-priority-class" for details.
+
+.. js:function:: TXN.set_priority_offset(txn, prio)
+
+ This function adjusts the priority offset of the transaction. The value
+ should be within the range -524287..524287. Values outside this range will be
+ truncated.
+
+ See the HAProxy configuration.txt file keyword "http-request" action
+ "set-priority-offset" for details.
+
+.. _reply_class:
+
+Reply class
+============
+
+.. js:class:: Reply
+
+ **context**: action
+
+ This class represents a HTTP response message. It provides some methods to
+ enrich it. Once converted into the internal HTTP representation, the response
+ message must not exceed the buffer size. Because for now there is no
+ easy way to be sure it fits, it is probably better to keep it reasonably
+ small.
+
+ See tune.bufsize in the configuration manual for details.
+
+.. code-block:: lua
+
+ local reply = txn:reply({status = 400}) -- default HTTP 400 reason-phase used
+ reply:add_header("content-type", "text/html")
+ reply:add_header("cache-control", "no-cache")
+ reply:add_header("cache-control", "no-store")
+ reply:set_body("<html><body><h1>invalid request<h1></body></html>")
+..
+
+ :see: :js:func:`TXN.reply`
+
+.. js:attribute:: Reply.status
+
+ The reply status code. By default, the status code is set to 200.
+
+ :returns: integer
+
+.. js:attribute:: Reply.reason
+
+ The reason string describing the status code.
+
+ :returns: string
+
+.. js:attribute:: Reply.headers
+
+ A table indexing all reply headers by name. To each name is associated an
+ ordered list of values.
+
+ :returns: Lua table
+
+.. code-block:: lua
+
+ {
+ ["content-type"] = { "text/html" },
+ ["cache-control"] = {"no-cache", "no-store" },
+ x_header_name = { "value1", "value2", ... }
+ ...
+ }
+..
+
+.. js:attribute:: Reply.body
+
+ The reply payload.
+
+ :returns: string
+
+.. js:function:: Reply.set_status(REPLY, status[, reason])
+
+ Set the reply status code and optionally the reason-phrase. If the reason is
+ not provided, the default reason corresponding to the status code is used.
+
+ :param class_reply reply: The related Reply object.
+ :param integer status: The reply status code.
+ :param string reason: The reply status reason (optional).
+
+.. js:function:: Reply.add_header(REPLY, name, value)
+
+ Add a header to the reply object. If the header does not already exist, a new
+ entry is created with its name as index and a one-element list containing its
+ value as value. Otherwise, the header value is appended to the ordered list of
+ values associated to the header name.
+
+ :param class_reply reply: The related Reply object.
+ :param string name: The header field name.
+ :param string value: The header field value.
+
+.. js:function:: Reply.del_header(REPLY, name)
+
+ Remove all occurrences of a header name from the reply object.
+
+ :param class_reply reply: The related Reply object.
+ :param string name: The header field name.
+
+.. js:function:: Reply.set_body(REPLY, body)
+
+ Set the reply payload.
+
+ :param class_reply reply: The related Reply object.
+ :param string body: The reply payload.
+
+.. _socket_class:
+
+Socket class
+============
+
+.. js:class:: Socket
+
+ This class must be compatible with the Lua Socket class. Only the 'client'
+ functions are available. See the Lua Socket documentation:
+
+ `http://w3.impa.br/~diego/software/luasocket/tcp.html
+ <http://w3.impa.br/~diego/software/luasocket/tcp.html>`_
+
+.. js:function:: Socket.close(socket)
+
+ Closes a TCP object. The internal socket used by the object is closed and the
+ local address to which the object was bound is made available to other
+ applications. No further operations (except for further calls to the close
+ method) are allowed on a closed Socket.
+
+ :param class_socket socket: Is the manipulated Socket.
+
+ Note: It is important to close all used sockets once they are not needed,
+ since, in many systems, each socket uses a file descriptor, which are limited
+ system resources. Garbage-collected objects are automatically closed before
+ destruction, though.
+
+.. js:function:: Socket.connect(socket, address[, port])
+
+ Attempts to connect a socket object to a remote host.
+
+
+ In case of error, the method returns nil followed by a string describing the
+ error. In case of success, the method returns 1.
+
+ :param class_socket socket: Is the manipulated Socket.
+ :param string address: can be an IP address or a host name. See below for more
+ information.
+ :param integer port: must be an integer number in the range [1..64K].
+ :returns: 1 or nil.
+
+ An address field extension permits to use the connect() function to connect to
+ other stream than TCP. The syntax containing a simpleipv4 or ipv6 address is
+ the basically expected format. This format requires the port.
+
+ Other format accepted are a socket path like "/socket/path", it permits to
+ connect to a socket. Abstract namespaces are supported with the prefix
+ "abns@", and finally a file descriptor can be passed with the prefix "fd@".
+ The prefix "ipv4@", "ipv6@" and "unix@" are also supported. The port can be
+ passed int the string. The syntax "127.0.0.1:1234" is valid. In this case, the
+ parameter *port* must not be set.
+
+.. js:function:: Socket.connect_ssl(socket, address, port)
+
+ Same behavior than the function socket:connect, but uses SSL.
+
+ :param class_socket socket: Is the manipulated Socket.
+ :returns: 1 or nil.
+
+.. js:function:: Socket.getpeername(socket)
+
+ Returns information about the remote side of a connected client object.
+
+ Returns a string with the IP address of the peer, followed by the port number
+ that peer is using for the connection. In case of error, the method returns
+ nil.
+
+ :param class_socket socket: Is the manipulated Socket.
+ :returns: a string containing the server information.
+
+.. js:function:: Socket.getsockname(socket)
+
+ Returns the local address information associated to the object.
+
+ The method returns a string with local IP address and a number with the port.
+ In case of error, the method returns nil.
+
+ :param class_socket socket: Is the manipulated Socket.
+ :returns: a string containing the client information.
+
+.. js:function:: Socket.receive(socket, [pattern [, prefix]])
+
+ Reads data from a client object, according to the specified read pattern.
+ Patterns follow the Lua file I/O format, and the difference in performance
+ between all patterns is negligible.
+
+ :param class_socket socket: Is the manipulated Socket.
+ :param string|integer pattern: Describe what is required (see below).
+ :param string prefix: A string which will be prefix the returned data.
+ :returns: a string containing the required data or nil.
+
+ Pattern can be any of the following:
+
+ * **`*a`**: reads from the socket until the connection is closed. No
+ end-of-line translation is performed;
+
+ * **`*l`**: reads a line of text from the Socket. The line is terminated by a
+ LF character (ASCII 10), optionally preceded by a CR character
+ (ASCII 13). The CR and LF characters are not included in the
+ returned line. In fact, all CR characters are ignored by the
+ pattern. This is the default pattern.
+
+ * **number**: causes the method to read a specified number of bytes from the
+ Socket. Prefix is an optional string to be concatenated to the
+ beginning of any received data before return.
+
+ * **empty**: If the pattern is left empty, the default option is `*l`.
+
+ If successful, the method returns the received pattern. In case of error, the
+ method returns nil followed by an error message which can be the string
+ 'closed' in case the connection was closed before the transmission was
+ completed or the string 'timeout' in case there was a timeout during the
+ operation. Also, after the error message, the function returns the partial
+ result of the transmission.
+
+ Important note: This function was changed severely. It used to support
+ multiple patterns (but I have never seen this feature used) and now it
+ doesn't anymore. Partial results used to be returned in the same way as
+ successful results. This last feature violated the idea that all functions
+ should return nil on error. Thus it was changed too.
+
+.. js:function:: Socket.send(socket, data [, start [, end ]])
+
+ Sends data through client object.
+
+ :param class_socket socket: Is the manipulated Socket.
+ :param string data: The data that will be sent.
+ :param integer start: The start position in the buffer of the data which will
+ be sent.
+ :param integer end: The end position in the buffer of the data which will
+ be sent.
+ :returns: see below.
+
+ Data is the string to be sent. The optional arguments i and j work exactly
+ like the standard string.sub Lua function to allow the selection of a
+ substring to be sent.
+
+ If successful, the method returns the index of the last byte within [start,
+ end] that has been sent. Notice that, if start is 1 or absent, this is
+ effectively the total number of bytes sent. In case of error, the method
+ returns nil, followed by an error message, followed by the index of the last
+ byte within [start, end] that has been sent. You might want to try again from
+ the byte following that. The error message can be 'closed' in case the
+ connection was closed before the transmission was completed or the string
+ 'timeout' in case there was a timeout during the operation.
+
+ Note: Output is not buffered. For small strings, it is always better to
+ concatenate them in Lua (with the '..' operator) and send the result in one
+ call instead of calling the method several times.
+
+.. js:function:: Socket.setoption(socket, option [, value])
+
+ Just implemented for compatibility, this cal does nothing.
+
+.. js:function:: Socket.settimeout(socket, value [, mode])
+
+ Changes the timeout values for the object. All I/O operations are blocking.
+ That is, any call to the methods send, receive, and accept will block
+ indefinitely, until the operation completes. The settimeout method defines a
+ limit on the amount of time the I/O methods can block. When a timeout time
+ has elapsed, the affected methods give up and fail with an error code.
+
+ The amount of time to wait is specified as the value parameter, in seconds.
+
+ The timeout modes are not implemented, the only settable timeout is the
+ inactivity time waiting for complete the internal buffer send or waiting for
+ receive data.
+
+ :param class_socket socket: Is the manipulated Socket.
+ :param float value: The timeout value. Use floating point to specify
+ milliseconds.
+
+.. _regex_class:
+
+Regex class
+===========
+
+.. js:class:: Regex
+
+ This class allows the usage of HAProxy regexes because classic lua doesn't
+ provides regexes. This class inherits the HAProxy compilation options, so the
+ regexes can be libc regex, pcre regex or pcre JIT regex.
+
+ The expression matching number is limited to 20 per regex. The only available
+ option is case sensitive.
+
+ Because regexes compilation is a heavy process, it is better to define all
+ your regexes in the **body context** and use it during the runtime.
+
+.. code-block:: lua
+
+ -- Create the regex
+ st, regex = Regex.new("needle (..) (...)", true);
+
+ -- Check compilation errors
+ if st == false then
+ print "error: " .. regex
+ end
+
+ -- Match the regexes
+ print(regex:exec("Looking for a needle in the haystack")) -- true
+ print(regex:exec("Lokking for a cat in the haystack")) -- false
+
+ -- Extract words
+ st, list = regex:match("Looking for a needle in the haystack")
+ print(st) -- true
+ print(list[1]) -- needle in the
+ print(list[2]) -- in
+ print(list[3]) -- the
+
+.. js:function:: Regex.new(regex, case_sensitive)
+
+ Create and compile a regex.
+
+ :param string regex: The regular expression according with the libc or pcre
+ standard
+ :param boolean case_sensitive: Match is case sensitive or not.
+ :returns: boolean status and :ref:`regex_class` or string containing fail
+ reason.
+
+.. js:function:: Regex.exec(regex, str)
+
+ Execute the regex.
+
+ :param class_regex regex: A :ref:`regex_class` object.
+ :param string str: The input string will be compared with the compiled regex.
+ :returns: a boolean status according with the match result.
+
+.. js:function:: Regex.match(regex, str)
+
+ Execute the regex and return matched expressions.
+
+ :param class_map map: A :ref:`regex_class` object.
+ :param string str: The input string will be compared with the compiled regex.
+ :returns: a boolean status according with the match result, and
+ a table containing all the string matched in order of declaration.
+
+.. _map_class:
+
+Map class
+=========
+
+.. js:class:: Map
+
+ This class permits to do some lookups in HAProxy maps. The declared maps can
+ be modified during the runtime through the HAProxy management socket.
+
+.. code-block:: lua
+
+ default = "usa"
+
+ -- Create and load map
+ geo = Map.new("geo.map", Map._ip);
+
+ -- Create new fetch that returns the user country
+ core.register_fetches("country", function(txn)
+ local src;
+ local loc;
+
+ src = txn.f:fhdr("x-forwarded-for");
+ if (src == nil) then
+ src = txn.f:src()
+ if (src == nil) then
+ return default;
+ end
+ end
+
+ -- Perform lookup
+ loc = geo:lookup(src);
+
+ if (loc == nil) then
+ return default;
+ end
+
+ return loc;
+ end);
+
+.. js:attribute:: Map._int
+
+ See the HAProxy configuration.txt file, chapter "Using ACLs and fetching
+ samples" and subchapter "ACL basics" to understand this pattern matching
+ method.
+
+ Note that :js:attr:`Map.int` is also available for compatibility.
+
+.. js:attribute:: Map._ip
+
+ See the HAProxy configuration.txt file, chapter "Using ACLs and fetching
+ samples" and subchapter "ACL basics" to understand this pattern matching
+ method.
+
+ Note that :js:attr:`Map.ip` is also available for compatibility.
+
+.. js:attribute:: Map._str
+
+ See the HAProxy configuration.txt file, chapter "Using ACLs and fetching
+ samples" and subchapter "ACL basics" to understand this pattern matching
+ method.
+
+ Note that :js:attr:`Map.str` is also available for compatibility.
+
+.. js:attribute:: Map._beg
+
+ See the HAProxy configuration.txt file, chapter "Using ACLs and fetching
+ samples" and subchapter "ACL basics" to understand this pattern matching
+ method.
+
+ Note that :js:attr:`Map.beg` is also available for compatibility.
+
+.. js:attribute:: Map._sub
+
+ See the HAProxy configuration.txt file, chapter "Using ACLs and fetching
+ samples" and subchapter "ACL basics" to understand this pattern matching
+ method.
+
+ Note that :js:attr:`Map.sub` is also available for compatibility.
+
+.. js:attribute:: Map._dir
+
+ See the HAProxy configuration.txt file, chapter "Using ACLs and fetching
+ samples" and subchapter "ACL basics" to understand this pattern matching
+ method.
+
+ Note that :js:attr:`Map.dir` is also available for compatibility.
+
+.. js:attribute:: Map._dom
+
+ See the HAProxy configuration.txt file, chapter "Using ACLs and fetching
+ samples" and subchapter "ACL basics" to understand this pattern matching
+ method.
+
+ Note that :js:attr:`Map.dom` is also available for compatibility.
+
+.. js:attribute:: Map._end
+
+ See the HAProxy configuration.txt file, chapter "Using ACLs and fetching
+ samples" and subchapter "ACL basics" to understand this pattern matching
+ method.
+
+.. js:attribute:: Map._reg
+
+ See the HAProxy configuration.txt file, chapter "Using ACLs and fetching
+ samples" and subchapter "ACL basics" to understand this pattern matching
+ method.
+
+ Note that :js:attr:`Map.reg` is also available for compatibility.
+
+
+.. js:function:: Map.new(file, method)
+
+ Creates and load a map.
+
+ :param string file: Is the file containing the map.
+ :param integer method: Is the map pattern matching method. See the attributes
+ of the Map class.
+ :returns: a class Map object.
+ :see: The Map attributes: :js:attr:`Map._int`, :js:attr:`Map._ip`,
+ :js:attr:`Map._str`, :js:attr:`Map._beg`, :js:attr:`Map._sub`,
+ :js:attr:`Map._dir`, :js:attr:`Map._dom`, :js:attr:`Map._end` and
+ :js:attr:`Map._reg`.
+
+.. js:function:: Map.lookup(map, str)
+
+ Perform a lookup in a map.
+
+ :param class_map map: Is the class Map object.
+ :param string str: Is the string used as key.
+ :returns: a string containing the result or nil if no match.
+
+.. js:function:: Map.slookup(map, str)
+
+ Perform a lookup in a map.
+
+ :param class_map map: Is the class Map object.
+ :param string str: Is the string used as key.
+ :returns: a string containing the result or empty string if no match.
+
+.. _applethttp_class:
+
+AppletHTTP class
+================
+
+.. js:class:: AppletHTTP
+
+ This class is used with applets that requires the 'http' mode. The http applet
+ can be registered with the *core.register_service()* function. They are used
+ for processing an http request like a server in back of HAProxy.
+
+ This is an hello world sample code:
+
+.. code-block:: lua
+
+ core.register_service("hello-world", "http", function(applet)
+ local response = "Hello World !"
+ applet:set_status(200)
+ applet:add_header("content-length", string.len(response))
+ applet:add_header("content-type", "text/plain")
+ applet:start_response()
+ applet:send(response)
+ end)
+
+.. js:attribute:: AppletHTTP.c
+
+ :returns: A :ref:`converters_class`
+
+ This attribute contains a Converters class object.
+
+.. js:attribute:: AppletHTTP.sc
+
+ :returns: A :ref:`converters_class`
+
+ This attribute contains a Converters class object. The
+ functions of this object always return a string.
+
+.. js:attribute:: AppletHTTP.f
+
+ :returns: A :ref:`fetches_class`
+
+ This attribute contains a Fetches class object. Note that the
+ applet execution place cannot access to a valid HAProxy core HTTP
+ transaction, so some sample fetches related to the HTTP dependent
+ values (hdr, path, ...) are not available.
+
+.. js:attribute:: AppletHTTP.sf
+
+ :returns: A :ref:`fetches_class`
+
+ This attribute contains a Fetches class object. The functions of
+ this object always return a string. Note that the applet
+ execution place cannot access to a valid HAProxy core HTTP
+ transaction, so some sample fetches related to the HTTP dependent
+ values (hdr, path, ...) are not available.
+
+.. js:attribute:: AppletHTTP.method
+
+ :returns: string
+
+ The attribute method returns a string containing the HTTP
+ method.
+
+.. js:attribute:: AppletHTTP.version
+
+ :returns: string
+
+ The attribute version, returns a string containing the HTTP
+ request version.
+
+.. js:attribute:: AppletHTTP.path
+
+ :returns: string
+
+ The attribute path returns a string containing the HTTP
+ request path.
+
+.. js:attribute:: AppletHTTP.qs
+
+ :returns: string
+
+ The attribute qs returns a string containing the HTTP
+ request query string.
+
+.. js:attribute:: AppletHTTP.length
+
+ :returns: integer
+
+ The attribute length returns an integer containing the HTTP
+ body length.
+
+.. js:attribute:: AppletHTTP.headers
+
+ :returns: table
+
+ The attribute headers returns a table containing the HTTP
+ headers. The header names are always in lower case. As the header name can be
+ encountered more than once in each request, the value is indexed with 0 as
+ first index value. The table has this form:
+
+.. code-block:: lua
+
+ AppletHTTP.headers['<header-name>'][<header-index>] = "<header-value>"
+
+ AppletHTTP.headers["host"][0] = "www.test.com"
+ AppletHTTP.headers["accept"][0] = "audio/basic q=1"
+ AppletHTTP.headers["accept"][1] = "audio/*, q=0.2"
+ AppletHTTP.headers["accept"][2] = "*/*, q=0.1"
+..
+
+.. js:function:: AppletHTTP.set_status(applet, code [, reason])
+
+ This function sets the HTTP status code for the response. The allowed code are
+ from 100 to 599.
+
+ :param class_AppletHTTP applet: An :ref:`applethttp_class`
+ :param integer code: the status code returned to the client.
+ :param string reason: the status reason returned to the client (optional).
+
+.. js:function:: AppletHTTP.add_header(applet, name, value)
+
+ This function adds a header in the response. Duplicated headers are not
+ collapsed. The special header *content-length* is used to determinate the
+ response length. If it does not exist, a *transfer-encoding: chunked* is set,
+ and all the write from the function *AppletHTTP:send()* become a chunk.
+
+ :param class_AppletHTTP applet: An :ref:`applethttp_class`
+ :param string name: the header name
+ :param string value: the header value
+
+.. js:function:: AppletHTTP.start_response(applet)
+
+ This function indicates to the HTTP engine that it can process and send the
+ response headers. After this called we cannot add headers to the response; We
+ cannot use the *AppletHTTP:send()* function if the
+ *AppletHTTP:start_response()* is not called.
+
+ :param class_AppletHTTP applet: An :ref:`applethttp_class`
+
+.. js:function:: AppletHTTP.getline(applet)
+
+ This function returns a string containing one line from the http body. If the
+ data returned doesn't contains a final '\\n' its assumed than its the last
+ available data before the end of stream.
+
+ :param class_AppletHTTP applet: An :ref:`applethttp_class`
+ :returns: a string. The string can be empty if we reach the end of the stream.
+
+.. js:function:: AppletHTTP.receive(applet, [size])
+
+ Reads data from the HTTP body, according to the specified read *size*. If the
+ *size* is missing, the function tries to read all the content of the stream
+ until the end. If the *size* is bigger than the http body, it returns the
+ amount of data available.
+
+ :param class_AppletHTTP applet: An :ref:`applethttp_class`
+ :param integer size: the required read size.
+ :returns: always return a string,the string can be empty is the connection is
+ closed.
+
+.. js:function:: AppletHTTP.send(applet, msg)
+
+ Send the message *msg* on the http request body.
+
+ :param class_AppletHTTP applet: An :ref:`applethttp_class`
+ :param string msg: the message to send.
+
+.. js:function:: AppletHTTP.get_priv(applet)
+
+ Return Lua data stored in the current transaction. If no data are stored,
+ it returns a nil value.
+
+ :param class_AppletHTTP applet: An :ref:`applethttp_class`
+ :returns: the opaque data previously stored, or nil if nothing is
+ available.
+ :see: :js:func:`AppletHTTP.set_priv`
+
+.. js:function:: AppletHTTP.set_priv(applet, data)
+
+ Store any data in the current HAProxy transaction. This action replaces the
+ old stored data.
+
+ :param class_AppletHTTP applet: An :ref:`applethttp_class`
+ :param opaque data: The data which is stored in the transaction.
+ :see: :js:func:`AppletHTTP.get_priv`
+
+.. js:function:: AppletHTTP.set_var(applet, var, value[, ifexist])
+
+ Converts a Lua type in a HAProxy type and store it in a variable <var>.
+
+ :param class_AppletHTTP applet: An :ref:`applethttp_class`
+ :param string var: The variable name according with the HAProxy variable
+ syntax.
+ :param type value: The value associated to the variable. The type ca be string
+ or integer.
+ :param boolean ifexist: If this parameter is set to true the variable will
+ only be set if it was defined elsewhere (i.e. used within the configuration).
+ For global variables (using the "proc" scope), they will only be updated and
+ never created. It is highly recommended to always set this to true.
+
+ :see: :js:func:`AppletHTTP.unset_var`
+ :see: :js:func:`AppletHTTP.get_var`
+
+.. js:function:: AppletHTTP.unset_var(applet, var)
+
+ Unset the variable <var>.
+
+ :param class_AppletHTTP applet: An :ref:`applethttp_class`
+ :param string var: The variable name according with the HAProxy variable
+ syntax.
+ :see: :js:func:`AppletHTTP.set_var`
+ :see: :js:func:`AppletHTTP.get_var`
+
+.. js:function:: AppletHTTP.get_var(applet, var)
+
+ Returns data stored in the variable <var> converter in Lua type.
+
+ :param class_AppletHTTP applet: An :ref:`applethttp_class`
+ :param string var: The variable name according with the HAProxy variable
+ syntax.
+ :see: :js:func:`AppletHTTP.set_var`
+ :see: :js:func:`AppletHTTP.unset_var`
+
+.. _applettcp_class:
+
+AppletTCP class
+===============
+
+.. js:class:: AppletTCP
+
+ This class is used with applets that requires the 'tcp' mode. The tcp applet
+ can be registered with the *core.register_service()* function. They are used
+ for processing a tcp stream like a server in back of HAProxy.
+
+.. js:attribute:: AppletTCP.c
+
+ :returns: A :ref:`converters_class`
+
+ This attribute contains a Converters class object.
+
+.. js:attribute:: AppletTCP.sc
+
+ :returns: A :ref:`converters_class`
+
+ This attribute contains a Converters class object. The
+ functions of this object always return a string.
+
+.. js:attribute:: AppletTCP.f
+
+ :returns: A :ref:`fetches_class`
+
+ This attribute contains a Fetches class object.
+
+.. js:attribute:: AppletTCP.sf
+
+ :returns: A :ref:`fetches_class`
+
+ This attribute contains a Fetches class object.
+
+.. js:function:: AppletTCP.getline(applet)
+
+ This function returns a string containing one line from the stream. If the
+ data returned doesn't contains a final '\\n' its assumed than its the last
+ available data before the end of stream.
+
+ :param class_AppletTCP applet: An :ref:`applettcp_class`
+ :returns: a string. The string can be empty if we reach the end of the stream.
+
+.. js:function:: AppletTCP.receive(applet, [size])
+
+ Reads data from the TCP stream, according to the specified read *size*. If the
+ *size* is missing, the function tries to read all the content of the stream
+ until the end.
+
+ :param class_AppletTCP applet: An :ref:`applettcp_class`
+ :param integer size: the required read size.
+ :returns: always return a string, the string can be empty if the connection is
+ closed.
+
+.. js:function:: AppletTCP.send(appletmsg)
+
+ Send the message on the stream.
+
+ :param class_AppletTCP applet: An :ref:`applettcp_class`
+ :param string msg: the message to send.
+
+.. js:function:: AppletTCP.get_priv(applet)
+
+ Return Lua data stored in the current transaction. If no data are stored,
+ it returns a nil value.
+
+ :param class_AppletTCP applet: An :ref:`applettcp_class`
+ :returns: the opaque data previously stored, or nil if nothing is
+ available.
+ :see: :js:func:`AppletTCP.set_priv`
+
+.. js:function:: AppletTCP.set_priv(applet, data)
+
+ Store any data in the current HAProxy transaction. This action replaces the
+ old stored data.
+
+ :param class_AppletTCP applet: An :ref:`applettcp_class`
+ :param opaque data: The data which is stored in the transaction.
+ :see: :js:func:`AppletTCP.get_priv`
+
+.. js:function:: AppletTCP.set_var(applet, var, value[, ifexist])
+
+ Converts a Lua type in a HAProxy type and stores it in a variable <var>.
+
+ :param class_AppletTCP applet: An :ref:`applettcp_class`
+ :param string var: The variable name according with the HAProxy variable
+ syntax.
+ :param type value: The value associated to the variable. The type can be
+ string or integer.
+ :param boolean ifexist: If this parameter is set to true the variable will
+ only be set if it was defined elsewhere (i.e. used within the configuration).
+ For global variables (using the "proc" scope), they will only be updated and
+ never created. It is highly recommended to always set this to true.
+
+ :see: :js:func:`AppletTCP.unset_var`
+ :see: :js:func:`AppletTCP.get_var`
+
+.. js:function:: AppletTCP.unset_var(applet, var)
+
+ Unsets the variable <var>.
+
+ :param class_AppletTCP applet: An :ref:`applettcp_class`
+ :param string var: The variable name according with the HAProxy variable
+ syntax.
+ :see: :js:func:`AppletTCP.unset_var`
+ :see: :js:func:`AppletTCP.set_var`
+
+.. js:function:: AppletTCP.get_var(applet, var)
+
+ Returns data stored in the variable <var> converter in Lua type.
+
+ :param class_AppletTCP applet: An :ref:`applettcp_class`
+ :param string var: The variable name according with the HAProxy variable
+ syntax.
+ :see: :js:func:`AppletTCP.unset_var`
+ :see: :js:func:`AppletTCP.set_var`
+
+.. _sticktable_class:
+
+StickTable class
+================
+
+.. js:class:: StickTable
+
+ **context**: task, action, sample-fetch
+
+ This class can be used to access the HAProxy stick tables from Lua.
+
+.. js:function:: StickTable.info()
+
+ Returns stick table attributes as a Lua table. See HAProxy documentation for
+ "stick-table" for canonical info, or check out example below.
+
+ :returns: Lua table
+
+ Assume our table has IPv4 key and gpc0 and conn_rate "columns":
+
+.. code-block:: lua
+
+ {
+ expire=<int>, # Value in ms
+ size=<int>, # Maximum table size
+ used=<int>, # Actual number of entries in table
+ data={ # Data columns, with types as key, and periods as values
+ (-1 if type is not rate counter)
+ conn_rate=<int>,
+ gpc0=-1
+ },
+ length=<int>, # max string length for string table keys, key length
+ # otherwise
+ nopurge=<boolean>, # purge oldest entries when table is full
+ type="ip" # can be "ip", "ipv6", "integer", "string", "binary"
+ }
+
+.. js:function:: StickTable.lookup(key)
+
+ Returns stick table entry for given <key>
+
+ :param string key: Stick table key (IP addresses and strings are supported)
+ :returns: Lua table
+
+.. js:function:: StickTable.dump([filter])
+
+ Returns all entries in stick table. An optional filter can be used
+ to extract entries with specific data values. Filter is a table with valid
+ comparison operators as keys followed by data type name and value pairs.
+ Check out the HAProxy docs for "show table" for more details. For the
+ reference, the supported operators are:
+
+ "eq", "ne", "le", "lt", "ge", "gt"
+
+ For large tables, execution of this function can take a long time (for
+ HAProxy standards). That's also true when filter is used, so take care and
+ measure the impact.
+
+ :param table filter: Stick table filter
+ :returns: Stick table entries (table)
+
+ See below for example filter, which contains 4 entries (or comparisons).
+ (Maximum number of filter entries is 4, defined in the source code)
+
+.. code-block:: lua
+
+ local filter = {
+ {"gpc0", "gt", 30}, {"gpc1", "gt", 20}}, {"conn_rate", "le", 10}
+ }
+
+.. _action_class:
+
+Action class
+=============
+
+.. js:class:: Act
+
+ **context**: action
+
+ This class contains all return codes an action may return. It is the lua
+ equivalent to HAProxy "ACT_RET_*" code.
+
+.. code-block:: lua
+
+ core.register_action("deny", { "http-req" }, function (txn)
+ return act.DENY
+ end)
+..
+.. js:attribute:: act.CONTINUE
+
+ This attribute is an integer (0). It instructs HAProxy to continue the
+ current ruleset processing on the message. It is the default return code
+ for a lua action.
+
+ :returns: integer
+
+.. js:attribute:: act.STOP
+
+ This attribute is an integer (1). It instructs HAProxy to stop the current
+ ruleset processing on the message.
+
+.. js:attribute:: act.YIELD
+
+ This attribute is an integer (2). It instructs HAProxy to temporarily pause
+ the message processing. It will be resumed later on the same rule. The
+ corresponding lua script is re-executed for the start.
+
+.. js:attribute:: act.ERROR
+
+ This attribute is an integer (3). It triggers an internal errors The message
+ processing is stopped and the transaction is terminated. For HTTP streams, an
+ HTTP 500 error is returned to the client.
+
+ :returns: integer
+
+.. js:attribute:: act.DONE
+
+ This attribute is an integer (4). It instructs HAProxy to stop the message
+ processing.
+
+ :returns: integer
+
+.. js:attribute:: act.DENY
+
+ This attribute is an integer (5). It denies the current message. The message
+ processing is stopped and the transaction is terminated. For HTTP streams, an
+ HTTP 403 error is returned to the client if the deny is returned during the
+ request analysis. During the response analysis, a HTTP 502 error is returned
+ and the server response is discarded.
+
+ :returns: integer
+
+.. js:attribute:: act.ABORT
+
+ This attribute is an integer (6). It aborts the current message. The message
+ processing is stopped and the transaction is terminated. For HTTP streams,
+ HAProxy assumes a response was already sent to the client. From the Lua
+ actions point of view, when this code is used, the transaction is terminated
+ with no reply.
+
+ :returns: integer
+
+.. js:attribute:: act.INVALID
+
+ This attribute is an integer (7). It triggers an internal errors. The message
+ processing is stopped and the transaction is terminated. For HTTP streams, an
+ HTTP 400 error is returned to the client if the error is returned during the
+ request analysis. During the response analysis, a HTTP 502 error is returned
+ and the server response is discarded.
+
+ :returns: integer
+
+.. js:function:: act:wake_time(milliseconds)
+
+ **context**: action
+
+ Set the script pause timeout to the specified time, defined in
+ milliseconds.
+
+ :param integer milliseconds: the required milliseconds.
+
+ This function may be used when a lua action returns `act.YIELD`, to force its
+ wake-up at most after the specified number of milliseconds.
+
+.. _filter_class:
+
+Filter class
+=============
+
+.. js:class:: filter
+
+ **context**: filter
+
+ This class contains return codes some filter callback functions may return. It
+ also contains configuration flags and some helper functions. To understand how
+ the filter API works, see `doc/internal/filters.txt` documentation.
+
+.. js:attribute:: filter.CONTINUE
+
+ This attribute is an integer (1). It may be returned by some filter callback
+ functions to instruct this filtering step is finished for this filter.
+
+.. js:attribute:: filter.WAIT
+
+ This attribute is an integer (0). It may be returned by some filter callback
+ functions to instruct the filtering must be paused, waiting for more data or
+ for an external event depending on this filter.
+
+.. js:attribute:: filter.ERROR
+
+ This attribute is an integer (-1). It may be returned by some filter callback
+ functions to trigger an error.
+
+.. js:attribute:: filter.FLT_CFG_FL_HTX
+
+ This attribute is a flag corresponding to the filter flag FLT_CFG_FL_HTX. When
+ it is set for a filter, it means the filter is able to filter HTTP streams.
+
+.. js:function:: filter.register_data_filter(chn)
+
+ **context**: filter
+
+ Enable the data filtering on the channel **chn** for the current filter. It
+ may be called at any time from any callback functions proceeding the data
+ analysis.
+
+ :param class_Channel chn: A :ref:`channel_class`.
+
+.. js:function:: filter.unregister_data_filter(chn)
+
+ **context**: filter
+
+ Disable the data filtering on the channel **chn** for the current filter. It
+ may be called at any time from any callback functions.
+
+ :param class_Channel chn: A :ref:`channel_class`.
+
+.. js:function:: filter.wake_time(milliseconds)
+
+ **context**: filter
+
+ Set the script pause timeout to the specified time, defined in
+ milliseconds.
+
+ :param integer milliseconds: the required milliseconds.
+
+ This function may be used from any lua filter callback function to force its
+ wake-up at most after the specified number of milliseconds. Especially, when
+ `filter.CONTINUE` is returned.
+
+
+A filters is declared using :js:func:`core.register_filter()` function. The
+provided class will be used to instantiate filters. It may define following
+attributes:
+
+* id: The filter identifier. It is a string that identifies the filter and is
+ optional.
+
+* flags: The filter flags. Only :js:attr:`filter.FLT_CFG_FL_HTX` may be set
+ for now.
+
+Such filter class must also define all required callback functions in the
+following list. Note that :js:func:`Filter.new()` must be defined otherwise the
+filter is ignored. Others are optional.
+
+* .. js:function:: FILTER.new()
+
+ Called to instantiate a new filter. This function must be defined.
+
+ :returns: a Lua object that will be used as filter instance for the current
+ stream.
+
+* .. js:function:: FILTER.start_analyze(flt, txn, chn)
+
+ Called when the analysis starts on the channel **chn**.
+
+* .. js:function:: FILTER.end_analyze(flt, txn, chn)
+
+ Called when the analysis ends on the channel **chn**.
+
+* .. js:function:: FILTER.http_headers(flt, txn, http_msg)
+
+ Called just before the HTTP payload analysis and after any processing on the
+ HTTP message **http_msg**. This callback functions is only called for HTTP
+ streams.
+
+* .. js:function:: FILTER.http_payload(flt, txn, http_msg)
+
+ Called during the HTTP payload analysis on the HTTP message **http_msg**. This
+ callback functions is only called for HTTP streams.
+
+* .. js:function:: FILTER.http_end(flt, txn, http_msg)
+
+ Called after the HTTP payload analysis on the HTTP message **http_msg**. This
+ callback functions is only called for HTTP streams.
+
+* .. js:function:: FILTER.tcp_payload(flt, txn, chn)
+
+ Called during the TCP payload analysis on the channel **chn**.
+
+Here is a full example:
+
+.. code-block:: lua
+
+ Trace = {}
+ Trace.id = "Lua trace filter"
+ Trace.flags = filter.FLT_CFG_FL_HTX;
+ Trace.__index = Trace
+
+ function Trace:new()
+ local trace = {}
+ setmetatable(trace, Trace)
+ trace.req_len = 0
+ trace.res_len = 0
+ return trace
+ end
+
+ function Trace:start_analyze(txn, chn)
+ if chn:is_resp() then
+ print("Start response analysis")
+ else
+ print("Start request analysis")
+ end
+ filter.register_data_filter(self, chn)
+ end
+
+ function Trace:end_analyze(txn, chn)
+ if chn:is_resp() then
+ print("End response analysis: "..self.res_len.." bytes filtered")
+ else
+ print("End request analysis: "..self.req_len.." bytes filtered")
+ end
+ end
+
+ function Trace:http_headers(txn, http_msg)
+ stline = http_msg:get_stline()
+ if http_msg.channel:is_resp() then
+ print("response:")
+ print(stline.version.." "..stline.code.." "..stline.reason)
+ else
+ print("request:")
+ print(stline.method.." "..stline.uri.." "..stline.version)
+ end
+
+ for n, hdrs in pairs(http_msg:get_headers()) do
+ for i,v in pairs(hdrs) do
+ print(n..": "..v)
+ end
+ end
+ return filter.CONTINUE
+ end
+
+ function Trace:http_payload(txn, http_msg)
+ body = http_msg:body(-20000)
+ if http_msg.channel:is_resp() then
+ self.res_len = self.res_len + body:len()
+ else
+ self.req_len = self.req_len + body:len()
+ end
+ end
+
+ core.register_filter("trace", Trace, function(trace, args)
+ return trace
+ end)
+
+..
+
+.. _httpmessage_class:
+
+HTTPMessage class
+===================
+
+.. js:class:: HTTPMessage
+
+ **context**: filter
+
+ This class contains all functions to manipulate a HTTP message. For now, this
+ class is only available from a filter context.
+
+.. js:function:: HTTPMessage.add_header(http_msg, name, value)
+
+ Appends a HTTP header field in the HTTP message **http_msg** whose name is
+ specified in **name** and whose value is defined in **value**.
+
+ :param class_httpmessage http_msg: The manipulated HTTP message.
+ :param string name: The header name.
+ :param string value: The header value.
+
+.. js:function:: HTTPMessage.append(http_msg, string)
+
+ This function copies the string **string** at the end of incoming data of the
+ HTTP message **http_msg**. The function returns the copied length on success
+ or -1 if data cannot be copied.
+
+ Same that :js:func:`HTTPMessage.insert(http_msg, string, http_msg:input())`.
+
+ :param class_httpmessage http_msg: The manipulated HTTP message.
+ :param string string: The data to copy at the end of incoming data.
+ :returns: an integer containing the amount of bytes copied or -1.
+
+.. js:function:: HTTPMessage.body(http_msgl[, offset[, length]])
+
+ This function returns **length** bytes of incoming data from the HTTP message
+ **http_msg**, starting at the offset **offset**. The data are not removed from
+ the buffer.
+
+ By default, if no length is provided, all incoming data found, starting at the
+ given offset, are returned. If **length** is set to -1, the function tries to
+ retrieve a maximum of data. Because it is called in the filter context, it
+ never yield. Not providing an offset is the same as setting it to 0. A
+ positive offset is relative to the beginning of incoming data of the
+ http_message buffer while negative offset is relative to their end.
+
+ If there is no incoming data and the HTTP message can't receive more data,
+ a 'nil' value is returned.
+
+ :param class_httpmessage http_msg: The manipulated HTTP message.
+ :param integer offset: *optional* The offset in incoming data to start to get
+ data. 0 by default. May be negative to be relative to the end of incoming
+ data.
+ :param integer length: *optional* The expected length of data to retrieve.
+ All incoming data by default. May be set to -1 to get a maximum of data.
+ :returns: a string containing the data found or nil.
+
+.. js:function:: HTTPMessage.eom(http_msg)
+
+ This function returns true if the end of message is reached for the HTTP
+ message **http_msg**.
+
+ :param class_httpmessage http_msg: The manipulated HTTP message.
+ :returns: an integer containing the amount of available bytes.
+
+.. js:function:: HTTPMessage.del_header(http_msg, name)
+
+ Removes all HTTP header fields in the HTTP message **http_msg** whose name is
+ specified in **name**.
+
+ :param class_httpmessage http_msg: The manipulated http message.
+ :param string name: The header name.
+
+.. js:function:: HTTPMessage.get_headers(http_msg)
+
+ Returns a table containing all the headers of the HTTP message **http_msg**.
+
+ :param class_httpmessage http_msg: The manipulated http message.
+ :returns: table of headers.
+
+ This is the form of the returned table:
+
+.. code-block:: lua
+
+ http_msg:get_headers()['<header-name>'][<header-index>] = "<header-value>"
+
+ local hdr = http_msg:get_headers()
+ hdr["host"][0] = "www.test.com"
+ hdr["accept"][0] = "audio/basic q=1"
+ hdr["accept"][1] = "audio/*, q=0.2"
+ hdr["accept"][2] = "*.*, q=0.1"
+..
+
+.. js:function:: HTTPMessage.get_stline(http_msg)
+
+ Returns a table containing the start-line of the HTTP message **http_msg**.
+
+ :param class_httpmessage http_msg: The manipulated http message.
+ :returns: the start-line.
+
+ This is the form of the returned table:
+
+.. code-block:: lua
+
+ -- for the request :
+ {"method" = string, "uri" = string, "version" = string}
+
+ -- for the response:
+ {"version" = string, "code" = string, "reason" = string}
+..
+
+.. js:function:: HTTPMessage.forward(http_msg, length)
+
+ This function forwards **length** bytes of data from the HTTP message
+ **http_msg**. Because it is called in the filter context, it never yields. Only
+ available incoming data may be forwarded, event if the requested length
+ exceeds the available amount of incoming data. It returns the amount of data
+ forwarded.
+
+ :param class_httpmessage http_msg: The manipulated HTTP message.
+ :param integer int: The amount of data to forward.
+
+.. js:function:: HTTPMessage.input(http_msg)
+
+ This function returns the length of incoming data in the HTTP message
+ **http_msg** from the filter point of view.
+
+ :param class_httpmessage http_msg: The manipulated HTTP message.
+ :returns: an integer containing the amount of available bytes.
+
+.. js:function:: HTTPMessage.insert(http_msg, string[, offset])
+
+ This function copies the string **string** at the offset **offset** in
+ incoming data of the HTTP message **http_msg**. The function returns the
+ copied length on success or -1 if data cannot be copied.
+
+ By default, if no offset is provided, the string is copied in front of
+ incoming data. A positive offset is relative to the beginning of incoming data
+ of the HTTP message while negative offset is relative to their end.
+
+ :param class_httpmessage http_msg: The manipulated HTTP message.
+ :param string string: The data to copy into incoming data.
+ :param integer offset: *optional* The offset in incoming data where to copy
+ data. 0 by default. May be negative to be relative to the end of incoming
+ data.
+ :returns: an integer containing the amount of bytes copied or -1.
+
+.. js:function:: HTTPMessage.is_full(http_msg)
+
+ This function returns true if the HTTP message **http_msg** is full.
+
+ :param class_httpmessage http_msg: The manipulated HTTP message.
+ :returns: a boolean
+
+.. js:function:: HTTPMessage.is_resp(http_msg)
+
+ This function returns true if the HTTP message **http_msg** is the response
+ one.
+
+ :param class_httpmessage http_msg: The manipulated HTTP message.
+ :returns: a boolean
+
+.. js:function:: HTTPMessage.may_recv(http_msg)
+
+ This function returns true if the HTTP message **http_msg** may still receive
+ data.
+
+ :param class_httpmessage http_msg: The manipulated HTTP message.
+ :returns: a boolean
+
+.. js:function:: HTTPMessage.output(http_msg)
+
+ This function returns the length of outgoing data of the HTTP message
+ **http_msg**.
+
+ :param class_httpmessage http_msg: The manipulated HTTP message.
+ :returns: an integer containing the amount of available bytes.
+
+.. js:function:: HTTPMessage.prepend(http_msg, string)
+
+ This function copies the string **string** in front of incoming data of the
+ HTTP message **http_msg**. The function returns the copied length on success
+ or -1 if data cannot be copied.
+
+ Same that :js:func:`HTTPMessage.insert(http_msg, string, 0)`.
+
+ :param class_httpmessage http_msg: The manipulated HTTP message.
+ :param string string: The data to copy in front of incoming data.
+ :returns: an integer containing the amount of bytes copied or -1.
+
+.. js:function:: HTTPMessage.remove(http_msg[, offset[, length]])
+
+ This function removes **length** bytes of incoming data of the HTTP message
+ **http_msg**, starting at offset **offset**. This function returns number of
+ bytes removed on success.
+
+ By default, if no length is provided, all incoming data, starting at the given
+ offset, are removed. Not providing an offset is the same that setting it
+ to 0. A positive offset is relative to the beginning of incoming data of the
+ HTTP message while negative offset is relative to the end.
+
+ :param class_httpmessage http_msg: The manipulated HTTP message.
+ :param integer offset: *optional* The offset in incoming data where to start
+ to remove data. 0 by default. May be negative to be relative to the end of
+ incoming data.
+ :param integer length: *optional* The length of data to remove. All incoming
+ data by default.
+ :returns: an integer containing the amount of bytes removed.
+
+.. js:function:: HTTPMessage.rep_header(http_msg, name, regex, replace)
+
+ Matches the regular expression in all occurrences of header field **name**
+ according to regex **regex**, and replaces them with the string **replace**.
+ The replacement value can contain back references like \1, \2, ... This
+ function acts on whole header lines, regardless of the number of values they
+ may contain.
+
+ :param class_httpmessage http_msg: The manipulated HTTP message.
+ :param string name: The header name.
+ :param string regex: The match regular expression.
+ :param string replace: The replacement value.
+
+.. js:function:: HTTPMessage.rep_value(http_msg, name, regex, replace)
+
+ Matches the regular expression on every comma-delimited value of header field
+ **name** according to regex **regex**, and replaces them with the string
+ **replace**. The replacement value can contain back references like \1, \2,
+ ...
+
+ :param class_httpmessage http_msg: The manipulated HTTP message.
+ :param string name: The header name.
+ :param string regex: The match regular expression.
+ :param string replace: The replacement value.
+
+.. js:function:: HTTPMessage.send(http_msg, string)
+
+ This function requires immediate send of the string **string**. It means the
+ string is copied at the beginning of incoming data of the HTTP message
+ **http_msg** and immediately forwarded. Because it is called in the filter
+ context, it never yields.
+
+ :param class_httpmessage http_msg: The manipulated HTTP message.
+ :param string string: The data to send.
+ :returns: an integer containing the amount of bytes copied or -1.
+
+.. js:function:: HTTPMessage.set(http_msg, string[, offset[, length]])
+
+ This function replaces **length** bytes of incoming data of the HTTP message
+ **http_msg**, starting at offset **offset**, by the string **string**. The
+ function returns the copied length on success or -1 if data cannot be copied.
+
+ By default, if no length is provided, all incoming data, starting at the given
+ offset, are replaced. Not providing an offset is the same as setting it
+ to 0. A positive offset is relative to the beginning of incoming data of the
+ HTTP message while negative offset is relative to the end.
+
+ :param class_httpmessage http_msg: The manipulated HTTP message.
+ :param string string: The data to copy into incoming data.
+ :param integer offset: *optional* The offset in incoming data where to start
+ the data replacement. 0 by default. May be negative to be relative to the
+ end of incoming data.
+ :param integer length: *optional* The length of data to replace. All incoming
+ data by default.
+ :returns: an integer containing the amount of bytes copied or -1.
+
+.. js:function:: HTTPMessage.set_eom(http_msg)
+
+ This function set the end of message for the HTTP message **http_msg**.
+
+ :param class_httpmessage http_msg: The manipulated HTTP message.
+
+.. js:function:: HTTPMessage.set_header(http_msg, name, value)
+
+ This variable replace all occurrence of all header matching the name **name**,
+ by only one containing the value **value**.
+
+ :param class_httpmessage http_msg: The manipulated HTTP message.
+ :param string name: The header name.
+ :param string value: The header value.
+
+ This function does the same work as the following code:
+
+.. code-block:: lua
+
+ http_msg:del_header("header")
+ http_msg:add_header("header", "value")
+..
+
+.. js:function:: HTTPMessage.set_method(http_msg, method)
+
+ Rewrites the request method with the string **method**. The HTTP message
+ **http_msg** must be the request.
+
+ :param class_httpmessage http_msg: The manipulated HTTP message.
+ :param string method: The new method.
+
+.. js:function:: HTTPMessage.set_path(http_msg, path)
+
+ Rewrites the request path with the string **path**. The HTTP message
+ **http_msg** must be the request.
+
+ :param class_httpmessage http_msg: The manipulated HTTP message.
+ :param string method: The new method.
+
+.. js:function:: HTTPMessage.set_query(http_msg, query)
+
+ Rewrites the request's query string which appears after the first question
+ mark ("?") with the string **query**. The HTTP message **http_msg** must be
+ the request.
+
+ :param class_httpmessage http_msg: The manipulated HTTP message.
+ :param string query: The new query.
+
+.. js:function:: HTTPMessage.set_status(http_msg, status[, reason])
+
+ Rewrites the response status code with the integer **code** and optional the
+ reason **reason**. If no custom reason is provided, it will be generated from
+ the status. The HTTP message **http_msg** must be the response.
+
+ :param class_httpmessage http_msg: The manipulated HTTP message.
+ :param integer status: The new response status code.
+ :param string reason: The new response reason (optional).
+
+.. js:function:: HTTPMessage.set_uri(http_msg, uri)
+
+ Rewrites the request URI with the string **uri**. The HTTP message
+ **http_msg** must be the request.
+
+ :param class_httpmessage http_msg: The manipulated HTTP message.
+ :param string uri: The new uri.
+
+.. js:function:: HTTPMessage.unset_eom(http_msg)
+
+ This function remove the end of message for the HTTP message **http_msg**.
+
+ :param class_httpmessage http_msg: The manipulated HTTP message.
+
+.. _CertCache_class:
+
+CertCache class
+================
+
+.. js:class:: CertCache
+
+ This class allows to update an SSL certificate file in the memory of the
+ current HAProxy process. It will do the same as "set ssl cert" + "commit ssl
+ cert" over the HAProxy CLI.
+
+.. js:function:: CertCache.set(certificate)
+
+ This function updates a certificate in memory.
+
+ :param table certificate: A table containing the fields to update.
+ :param string certificate.filename: The mandatory filename of the certificate
+ to update, it must already exist in memory.
+ :param string certificate.crt: A certificate in the PEM format. It can also
+ contain a private key.
+ :param string certificate.key: A private key in the PEM format.
+ :param string certificate.ocsp: An OCSP response in base64. (cf management.txt)
+ :param string certificate.issuer: The certificate of the OCSP issuer.
+ :param string certificate.sctl: An SCTL file.
+
+.. code-block:: lua
+
+ CertCache.set{filename="certs/localhost9994.pem.rsa", crt=crt}
+
+
+External Lua libraries
+======================
+
+A lot of useful lua libraries can be found here:
+
+* Lua toolbox has been superseded by
+ `https://luarocks.org/ <https://luarocks.org/>`_
+
+ The old lua toolbox source code is still available here
+ `https://github.com/catwell/lua-toolbox <https://github.com/catwell/lua-toolbox>`_ (DEPRECATED)
+
+Redis client library:
+
+* `https://github.com/nrk/redis-lua <https://github.com/nrk/redis-lua>`_
+
+This is an example about the usage of the Redis library within HAProxy.
+Note that each call to any function of this library can throw an error if
+the socket connection fails.
+
+.. code-block:: lua
+
+ -- load the redis library
+ local redis = require("redis");
+
+ function do_something(txn)
+
+ -- create and connect new tcp socket
+ local tcp = core.tcp();
+ tcp:settimeout(1);
+ tcp:connect("127.0.0.1", 6379);
+
+ -- use the redis library with this new socket
+ local client = redis.connect({socket=tcp});
+ client:ping();
+
+ end
+
+OpenSSL:
+
+* `http://mkottman.github.io/luacrypto/index.html
+ <http://mkottman.github.io/luacrypto/index.html>`_
+
+* `https://github.com/brunoos/luasec/wiki
+ <https://github.com/brunoos/luasec/wiki>`_
diff --git a/doc/lua.txt b/doc/lua.txt
new file mode 100644
index 0000000..5d41a30
--- /dev/null
+++ b/doc/lua.txt
@@ -0,0 +1,972 @@
+ Lua: Architecture and first steps
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ version 2.9
+
+ author: Thierry FOURNIER
+ contact: tfournier at arpalert dot org
+
+
+
+HAProxy is a powerful load balancer. It embeds many options and many
+configuration styles in order to give a solution to many load balancing
+problems. However, HAProxy is not universal and some special or specific
+problems do not have solution with the native software.
+
+This text is not a full explanation of the Lua syntax.
+
+This text is not a replacement of the HAProxy Lua API documentation. The API
+documentation can be found at the project root, in the documentation directory.
+The goal of this text is to discover how Lua is implemented in HAProxy and using
+it efficiently.
+
+However, this can be read by Lua beginners. Some examples are detailed.
+
+Why a scripting language in HAProxy
+===================================
+
+HAProxy 1.5 makes at possible to do many things using samples, but some people
+want to more combining results of samples fetches, programming conditions and
+loops which is not possible. Sometimes people implement these functionalities
+in patches which have no meaning outside their network. These people must
+maintain these patches, or worse we must integrate them in the HAProxy
+mainstream.
+
+Their need is to have an embedded programming language in order to no longer
+modify the HAProxy source code, but to write their own control code. Lua is
+encountered very often in the software industry, and in some open source
+projects. It is easy to understand, efficient, light without external
+dependencies, and leaves the resource control to the implementation. Its design
+is close to the HAProxy philosophy which uses components for what they do
+perfectly.
+
+The HAProxy control block allows one to take a decision based on the comparison
+between samples and patterns. The samples are extracted using fetch functions
+easily extensible, and are used by actions which are also extensible. It seems
+natural to allow Lua to give samples, modify them, and to be an action target.
+So, Lua uses the same entities as the configuration language. This is the most
+natural and reliable way for the Lua integration. So, the Lua engine allows one
+to add new sample fetch functions, new converter functions and new actions.
+These new entities can access the existing samples fetches and converters
+allowing to extend them without rewriting them.
+
+The writing of the first Lua functions shows that implementing complex concepts
+like protocol analysers is easy and can be extended to full services. It appears
+that these services are not easy to implement with the HAProxy configuration
+model which is based on four steps: fetch, convert, compare and action. HAProxy
+is extended with a notion of services which are a formalisation of the existing
+services like stats, cli and peers. The service is an autonomous entity with a
+behaviour pattern close to that of an external client or server. The Lua engine
+inherits from this new service and offers new possibilities for writing
+services.
+
+This scripting language is useful for testing new features as proof of concept.
+Later, if there is general interest, the proof of concept could be integrated
+with C language in the HAProxy core.
+
+The HAProxy Lua integration also provides a simple way for distributing Lua
+packages. The final user needs only to install the Lua file, load it in HAProxy
+and follow the attached documentation.
+
+Design and technical things
+===========================
+
+Lua is integrated into the HAProxy event driven core. We want to preserve the
+fast processing of HAProxy. To ensure this, we implement some technical concepts
+between HAProxy and the Lua library.
+
+The following paragraph also describes the interactions between Lua and HAProxy
+from a technical point of view.
+
+Prerequisite
+-----------
+
+Reading the following documentation links is required to understand the
+current paragraph:
+
+ HAProxy doc: http://docs.haproxy.org/
+ Lua API: http://www.lua.org/manual/5.3/
+ HAProxy API: http://www.arpalert.org/src/haproxy-lua-api/2.6/index.html
+ Lua guide: http://www.lua.org/pil/
+
+more about Lua choice
+---------------------
+
+Lua language is very simple to extend. It is easy to add new functions written
+in C in the core language. It is not required to embed very intrusive libraries,
+and we do not change compilation processes.
+
+The amount of memory consumed can be controlled, and the issues due to lack of
+memory are perfectly caught. The maximum amount of memory allowed for the Lua
+processes is configurable. If some memory is missing, the current Lua action
+fails, and the HAProxy processing flow continues.
+
+Lua provides a way for implementing event driven design. When the Lua code
+wants to do a blocking action, the action is started, it executes non blocking
+operations, and returns control to the HAProxy scheduler when it needs to wait
+for some external event.
+
+The Lua process can be interrupted after a number of instructions executed. The
+Lua execution will resume later. This is a useful way for controlling the
+execution time. This system also keeps HAProxy responsive. When the Lua
+execution is interrupted, HAProxy accepts some connections or transfers pending
+data. The Lua execution does not block the main HAProxy processing, except in
+some cases which we will see later.
+
+Lua function integration
+------------------------
+
+The Lua actions, sample fetches, converters and services are integrated in
+HAProxy with "register_*" functions. The register system is a choice for
+providing HAProxy Lua packages easily. The register system adds new sample
+fetches, converters, actions or services usable in the HAProxy configuration
+file.
+
+The register system is defined in the "core" functions collection. This
+collection is provided by HAProxy and is always available. Below, the list of
+these functions:
+
+ - core.register_action()
+ - core.register_converters()
+ - core.register_fetches()
+ - core.register_init()
+ - core.register_service()
+ - core.register_task()
+
+These functions are the execution entry points.
+
+HTTP action must be used for manipulating HTTP request headers. This action
+can not manipulates HTTP content. It is dangerous to use the channel
+manipulation object with an HTTP request in an HTTP action. The channel
+manipulation can transform a valid request in an invalid request. In this case,
+the action will never resume and the processing will be frozen. HAProxy
+discards the request after the reception timeout.
+
+Non blocking design
+-------------------
+
+HAProxy is an event driven software, so blocking system calls are absolutely
+forbidden. However, the Lua allows to do blocking actions. When an action
+blocks, HAProxy is waiting and do nothing, so the basic functionalities like
+accepting connections or forwarding data are blocked while the end of the system
+call. In this case HAProxy will be less responsive.
+
+This is very insidious because when the developer tries to execute its Lua code
+with only one stream, HAProxy seems to run fine. When the code is used with
+production stream, HAProxy encounters some slow processing, and it cannot
+hold the load.
+
+However, during the initialisation state, you can obviously using blocking
+functions. There are typically used for loading files.
+
+The list of prohibited standard Lua functions during the runtime contains all
+that do filesystem access:
+
+ - os.remove()
+ - os.rename()
+ - os.tmpname()
+ - package.*()
+ - io.*()
+ - file.*()
+
+Some other functions are prohibited:
+
+ - os.execute(), waits for the end of the required execution blocking HAProxy.
+
+ - os.exit(), is not really dangerous for the process, but it's not the good way
+ for exiting the HAProxy process.
+
+ - print(), writes data on stdout. In some cases these writes are blocking, the
+ best practice is reserving this call for debugging. We must prefer
+ to use core.log() or TXN.log() for sending messages.
+
+Some HAProxy functions have a blocking behaviour pattern in the Lua code, but
+there are compatible with the non blocking design. These functions are:
+
+ - All the socket class
+ - core.sleep()
+
+Responsive design
+-----------------
+
+HAProxy must process connections accept, forwarding data and processing timeouts
+as soon as possible. The first thing is to believe that a Lua script with a long
+execution time should impact the expected responsive behaviour.
+
+It is not the case, the Lua script execution are regularly interrupted, and
+HAProxy can process other things. These interruptions are exprimed in number of
+Lua instructions. The number of interruptions between two interrupts is
+configured with the following "tune" option:
+
+ tune.lua.forced-yield <nb>
+
+The default value is 10 000. For determining it, I ran benchmark on my laptop.
+I executed a Lua loop between 10 seconds with different values for the
+"tune.lua.forced-yield" option, and I noted the results:
+
+ configured | Number of
+ instructions | loops executed
+ between two | in millions
+ forced yields |
+ ---------------+---------------
+ 10 | 160
+ 500 | 670
+ 1000 | 680
+ 5000 | 700
+ 7000 | 700
+ 8000 | 700
+ 9000 | 710 <- ceil
+ 10000 | 710
+ 100000 | 710
+ 1000000 | 710
+
+The result showed that from 9000 instructions between two interrupt, we reached
+a ceil, so the default parameter is 10 000.
+
+When HAProxy interrupts the Lua processing, we have two states possible:
+
+ - Lua is resumable, and it returns control to the HAProxy scheduler,
+ - Lua is not resumable, and we just check the execution timeout.
+
+The second case occurs if it is required by the HAProxy core. This state is
+forced if the Lua is processed in a non resumable HAProxy part, like sample
+fetches or converters.
+
+It occurs also if the Lua is non resumable. For example, if some code is
+executed through the Lua pcall() function, the execution is not resumable. This
+is explained later.
+
+So, the Lua code must be fast and simple when is executed as sample fetches and
+converters, it could be slow and complex when is executed as actions and
+services.
+
+Execution time
+--------------
+
+The Lua execution time is measured and limited. Each group of functions has its
+own timeout configured. The time measured is the real Lua execution time, and
+not the difference between the end time and the start time. The groups are:
+
+ - main code and init are not submitted to the timeout,
+ - fetches, converters and action have a default timeout of 4s,
+ - task, by default does not have timeout,
+ - service have a default timeout of 4s.
+
+The corresponding tune options are:
+
+ - tune.lua.session-timeout (action, filter, cli)
+ - tune.lua.task-timeout (task)
+ - tune.lua.service-timeout (services)
+ - tune.lua.burst-timeout (max time between two lua yields)
+
+The task does not have a timeout because it runs in background along the
+HAProxy process life.
+
+For example, if an Lua script is executed during 1.1s and the script executes a
+sleep of 1 second, the effective measured running time is 0.1s.
+
+This timeout is useful for preventing infinite loops. During the runtime, it
+should be never triggered.
+
+The stack and the coprocess
+---------------------------
+
+The Lua execution is organized around a stack. Each Lua action, even out of the
+effective execution, affects the stack. HAProxy integration uses one main stack,
+which is common for all the process, and a secondary one used as coprocess.
+After the initialization, the main stack is no longer used by HAProxy, except
+for global storage. The second type of stack is used by all the Lua functions
+called from different Lua actions declared in HAProxy. The main stack permits
+to store coroutines pointers, and some global variables.
+
+Do you want to see an example of how seems Lua C development around a stack ?
+Some examples follows. This first one, is a simple addition:
+
+ lua_pushnumber(L, 1)
+ lua_pushnumber(L, 2)
+ lua_arith(L, LUA_OPADD)
+
+It's easy, we push 1 on the stack, after, we push 2, and finally, we perform an
+addition. The two top entries of the stack are added, popped, and the result is
+pushed. It is a classic way with a stack.
+
+Now an example for constructing array and objects. It's a little bit more
+complicated. The difficult consist to keep in mind the state of the stack while
+we write the code. The goal is to create the entity described below. Note that
+the notation "*1" is a metatable reference. The metatable will be explained
+later.
+
+ name*1 = {
+ [0] = <userdata>,
+ }
+
+ *1 = {
+ "__index" = {
+ "method1" = <function>,
+ "method2" = <function>
+ }
+ "__gc" = <function>
+ }
+
+Let's go:
+
+ lua_newtable() // The "name" table
+ lua_newtable() // The metatable *1
+ lua_pushstring("__index")
+ lua_newtable() // The "__index" table
+ lua_pushstring("method1")
+ lua_pushfunction(function)
+ lua_settable(-3) // -3 is an index in the stack. insert method1
+ lua_pushstring("method2")
+ lua_pushfunction(function)
+ lua_settable(-3) // insert method2
+ lua_settable(-3) // insert "__index"
+ lua_pushstring("__gc")
+ lua_pushfunction(function)
+ lua_settable() // insert "__gc"
+ lua_setmetatable(-1) // attach metatable to "name"
+ lua_pushnumber(0)
+ lua_pushuserdata(userdata)
+ lua_settable(-3)
+ lua_setglobal("name")
+
+So, coding for Lua in C, is not complex, but it needs some mental gymnastic.
+
+The object concept and the HAProxy format
+-----------------------------------------
+
+The object seems to be not a native concept. An Lua object is a table. We can
+note that the table notation accept three forms:
+
+ 1. mytable["entry"](mytable, "param")
+ 2. mytable.entry(mytable, "param")
+ 3. mytable:entry("param")
+
+These three notation have the same behaviour pattern: a function is executed
+with the table itself as first parameter and string "param" as second parameter
+The notation with [] is commonly used for storing data in a hash table, and the
+dotted notation is used for objects. The notation with ":" indicates that the
+first parameter is the element at the left of the symbol ":".
+
+So, an object is a table and each entry of the table is a variable. A variable
+can be a function. These are the first concepts of the object notation in the
+Lua, but it is not the end.
+
+With the objects, we usually expect classes and inheritance. This is the role of
+the metable. A metable is a table with predefined entries. These entries modify
+the default behaviour of the table. The simplest example is the "__index" entry.
+If this entry exists, it is called when a value is requested in the table. The
+behaviour is the following:
+
+ 1 - looks in the table if the entry exists, and if it the case, return it
+
+ 2 - looks if a metatable exists, and if the "__index" entry exists
+
+ 3 - if "__index" is a function, execute it with the key as parameter, and
+ returns the result of the function.
+
+ 4 - if "__index" is a table, looks if the requested entry exists, and if
+ exists, return it.
+
+ 5 - if not exists, return to step 2
+
+The behaviour of the point 5 represents the inheritance.
+
+In HAProxy all the provided objects are tables, the entry "[0]" contains private
+data, there are often userdata or lightuserdata. The metatable is registered in
+the global part of the main Lua stack, and it is called with the case sensitive
+class name. A great part of these class must not be used directly because it
+requires an initialisation using the HAProxy internal structs.
+
+The HAProxy objects use unified conventions. An Lua object is always a table.
+In most cases, an HAProxy Lua object needs some private data. These are always
+set in the index [0] of the array. The metatable entry "__tostring" returns the
+object name.
+
+The Lua developer can add entries to the HAProxy object. They just work carefully
+and prevent to modify the index [0].
+
+Common HAProxy objects are:
+
+ - TXN : manipulates the transaction between the client and the server
+ - Channel : manipulates proxified data between the client and the server
+ - HTTP : manipulates HTTP between the client and the server
+ - Map : manipulates HAProxy maps.
+ - Fetches : access to all HAProxy sample fetches
+ - Converters : access to all HAProxy sample converters
+ - AppletTCP : process client request like a TCP server
+ - AppletHTTP : process client request like an HTTP server
+ - Socket : establish tcp connection to a server (ipv4/ipv6/socket/ssl/...)
+
+The garbage collector and the memory allocation
+-----------------------------------------------
+
+Lua doesn't really have a global memory limit, but HAProxy implements it. This
+permits to control the amount of memory dedicated to the Lua processes. It is
+specially useful with embedded environments.
+
+When the memory limit is reached, HAProxy refuses to give more memory to the Lua
+scripts. The current Lua execution is terminated with an error and HAProxy
+continues its processing.
+
+The max amount of memory is configured with the option:
+
+ tune.lua.maxmem
+
+As many other script languages, Lua uses a garbage collector for reusing its
+memory. The Lua developer can work without memory preoccupation. Usually, the
+garbage collector is controlled by the Lua core, but sometimes it will be useful
+to run when the user/developer requires. So the garbage collector can be called
+from C part or Lua part.
+
+Sometimes, objects using lightuserdata or userdata requires to free some memory
+block or close filedescriptor not controlled by the Lua. A dedicated garbage
+collection function is provided through the metatable. It is referenced with the
+special entry "__gc".
+
+Generally, in HAProxy, the garbage collector does this job without any
+intervention. However some objects use a great amount of memory, and we want to
+release as quickly as possible. The problem is that only the GC knows if the
+object is in use or not. The reason is simple variable containing objects can be
+shared between coroutines and the main thread, so an object can be used
+everywhere in HAProxy.
+
+The only one example is the HAProxy sockets. These are explained later, just for
+understanding the GC issues, a quick overview of the socket follows. The HAProxy
+socket uses an internal session and stream, the session uses resources like
+memory and file descriptor and in some cases keeps a socket open while it is no
+longer used by Lua.
+
+If the HAProxy socket is used, we forcing a garbage collector cycle after the
+end of each function using HAProxy socket. The reason is simple: if the socket
+is no longer used, we want to close the connection quickly.
+
+A special flag is used in HAProxy indicating that a HAProxy socket is created.
+If this flag is set, a full GC cycle is started after each Lua action. This is
+not free, we loose about 10% of performances, but it is the only way for closing
+sockets quickly.
+
+The yield concept / longjmp issues
+----------------------------------
+
+The "yield" is an action which does some Lua processing in pause and give back
+the hand to the HAProxy core. This action is do when the Lua needs to wait about
+data or other things. The most basically example is the sleep() function. In an
+event driven software the code must not process blocking systems call, so the
+sleep blocks the software between a lot of time. In HAProxy, an Lua sleep does a
+yield, and ask to the scheduler to be woken up in a required sleep time.
+Meanwhile, the HAProxy scheduler does other things, like accepting new
+connection or forwarding data.
+
+A yield is also executed regularly, after a lot of Lua instructions processed.
+This yield permits to control the effective execution time, and also give back
+the hand to the HAProxy core. When HAProxy finishes to process the pending jobs,
+the Lua execution continues.
+
+This special "yield" uses the Lua "debug" functions. Lua provides a debug method
+called "lua_sethook()" which permits to interrupt the execution after some
+configured condition and call a function. This condition used in HAProxy is
+a number of instructions processed and when a function returns. The function
+called controls the effective execution time, and if it is possible to send a
+"yield".
+
+The yield system is based on a couple setjmp/longjmp. In brief, the setjmp()
+stores a stack state, and the longjmp restores the stack in its state which had
+before the last Lua execution.
+
+Lua can immediately stop its execution if an error occurs. This system uses also
+the longjmp system. In HAProxy, we try to use this system only for unrecoverable
+errors. Maybe some trivial errors target an exception, but we try to remove it.
+
+It seems that Lua uses the longjmp system for having a behaviour like the java
+try / catch. We can use the function pcall() to execute some code. The function
+pcall() run a setjmp(). So, if any error occurs while the Lua code execution,
+the flow immediately returns from the pcall() with an error.
+
+The big issue of this behaviour is that we cannot do a yield. So if some Lua code
+executes a library using pcall for catching errors, HAProxy must be wait for the
+end of execution without processing any accept or any stream. The cause is the
+yield must be jump to the root of execution. The intermediate setjmp() avoids
+this behaviour.
+
+
+ HAProxy start Lua execution
+ + Lua puts a setjmp()
+ + Lua executes code
+ + Some code is executed in a pcall()
+ + pcall() puts a setjmp()
+ + Lua executes code
+ + A yield is require for a sleep function
+ it cannot be jumps to the Lua root execution.
+
+
+Another issue with the processing of strong errors is the manipulation of the
+Lua stack outside of an Lua processing. If one of the functions called occurs a
+strong error, the default behaviour is an abort(). It is not acceptable when
+HAProxy is in runtime mode. The Lua documentation propose to use another
+setjmp/longjmp to avoid the abort(). The goal is to put a setjmp between
+manipulating the Lua stack and using an alternative "panic" function which jumps
+to the setjmp() in error case.
+
+All of these behaviours are very dangerous for the stability, and the internal
+HAProxy code must be modified with many precautions.
+
+For preserving a good behaviour of HAProxy, the yield is mandatory.
+Unfortunately, some HAProxy parts are not adapted for resuming an execution
+after a yield. These parts are the sample fetches and the sample converters. So,
+the Lua code written in these parts of HAProxy must be quickly executed, and can
+not do actions which require yield like TCP connection or simple sleep.
+
+HAProxy socket object
+---------------------
+
+The HAProxy design is optimized for the data transfers between a client and a
+server, and processing the many errors which can occurs during these exchanges.
+HAProxy is not designed for having a third connection established to a third
+party server.
+
+The solution consist to put the main stream in pause waiting for the end of the
+exchanges with the third connection. This is completed by a signal between
+internal tasks. The following graph shows the HAProxy Lua socket:
+
+
+ +--------------------+
+ | Lua processing |
+ ------------------\ | creates socket | ------------------\
+ incoming request > | and puts the | Outgoing request >
+ ------------------/ | current processing | ------------------/
+    | in pause waiting |
+ | for TCP applet |
+ +-----------------+--+
+ ^ |
+ | |
+ | signal | read / write
+ | | data
+ | |
+ +-------------+---------+ v
+ | HAProxy internal +----------------+
+ | applet send signals | |
+ | when data is received | | -------------------\
+ | or some room is | Attached I/O | Client TCP stream >
+ | available | Buffers | -------------------/
+ +--------------------+--+ |
+ | |
+ +-------------------+
+
+
+A more detailed graph is available in the "doc/internals" directory.
+
+The HAProxy Lua socket uses a full HAProxy session / stream for establishing the
+connection. This mechanism provides all the facilities and HAProxy features,
+like the SSL stack, many socket type, and support for namespaces.
+Technically it supports the proxy protocol, but there are no way to enable it.
+
+How compiling HAProxy with Lua
+==============================
+
+HAProxy 1.6 requires Lua 5.3. Lua 5.3 offers some features which make easy the
+integration. Lua 5.3 is young, and some distros do not distribute it. Luckily,
+Lua is a great product because it does not require exotic dependencies, and its
+build process is really easy.
+
+The compilation process for linux is easy:
+
+ - download the source tarball
+ wget http://www.lua.org/ftp/lua-5.3.1.tar.gz
+
+ - untar it
+ tar xf lua-5.3.1.tar.gz
+
+ - enter the directory
+ cd lua-5.3.1
+
+ - build the library for linux
+ make linux
+
+ - install it:
+ sudo make INSTALL_TOP=/opt/lua-5.3.1 install
+
+HAProxy builds with your favourite options, plus the following options for
+embedding the Lua script language:
+
+ - download the source tarball
+ wget http://www.haproxy.org/download/1.6/src/haproxy-1.6.2.tar.gz
+
+ - untar it
+ tar xf haproxy-1.6.2.tar.gz
+
+ - enter the directory
+ cd haproxy-1.6.2
+
+ - build HAProxy:
+ make TARGET=linux-glibc \
+ USE_LUA=1 \
+ LUA_LIB=/opt/lua-5.3.1/lib \
+ LUA_INC=/opt/lua-5.3.1/include
+
+ - install it:
+ sudo make PREFIX=/opt/haproxy-1.6.2 install
+
+First steps with Lua
+====================
+
+Now, it's time to use Lua in HAProxy.
+
+Start point
+-----------
+
+The HAProxy global directive "lua-load <file>" allows to load an Lua file. This
+is the entry point. This load become during the configuration parsing, and the
+Lua file is immediately executed.
+
+All the register_*() functions must be called at this time because they are used
+just after the processing of the global section, in the frontend/backend/listen
+sections.
+
+The most simple "Hello world !" is the following line a loaded Lua file:
+
+ core.Alert("Hello World !");
+
+It displays a log during the HAProxy startup:
+
+ [alert] 285/083533 (14465) : Hello World !
+
+Note: By default, logs originating from a LUA script are sent to the loggers
+applicable to the current context, if any. If none are configured for use,
+logs are instead sent to stderr. See tune.lua.log.loggers and tune.lua.log.stderr
+for more information.
+
+Default path and libraries
+--------------------------
+
+Lua can embed some libraries. These libraries can be included from different
+paths. It seems that Lua doesn't like subdirectories. In the following example,
+I try to load a compiled library, so the first line is Lua code, the second line
+is an 'strace' extract proving that the library was opened. The next lines are
+the associated error.
+
+ require("luac/concat")
+
+ open("./luac/concat.so", O_RDONLY|O_CLOEXEC) = 4
+
+ [ALERT] (22806) : parsing [commonstats.conf:15] : lua runtime
+ error: error loading module 'luac/concat' from file './luac/concat.so':
+ ./luac/concat.so: undefined symbol: luaopen_luac/concat
+
+Lua tries to load the C symbol 'luaopen_luac/concat'. When Lua tries to open a
+library, it tries to execute the function associated to the symbol
+"luaopen_<libname>".
+
+The variable "<libname>" is defined using the content of the variable
+"package.cpath" and/or "package.path". The default definition of the
+"package.cpath" (on my computer is ) variable is:
+
+ /usr/local/lib/lua/5.3/?.so;/usr/local/lib/lua/5.3/loadall.so;./?.so
+
+The "<libname>" is the content which replaces the symbol "<?>". In the previous
+example, its "luac/concat", and obviously the Lua core try to load the function
+associated with the symbol "luaopen_luac/concat".
+
+My conclusion is that Lua doesn't support subdirectories. So, for loading
+libraries in subdirectory, it must fill the variable with the name of this
+subdirectory. The extension .so must disappear, otherwise Lua try to execute the
+function associated with the symbol "luaopen_concat.so". The following syntax is
+correct:
+
+ package.cpath = package.cpath .. ";./luac/?.so"
+ require("concat")
+
+First useful example
+--------------------
+
+ core.register_fetches("my-hash", function(txn, salt)
+ return txn.sc:sdbm(salt .. txn.sf:req_fhdr("host") .. txn.sf:path() .. txn.sf:src(), 1)
+ end)
+
+You will see that these 3 lines can generate a lot of explanations :)
+
+Core.register_fetches() is executed during the processing of the global section
+by the HAProxy configuration parser. A new sample fetch is declared with name
+"my-hash", this name is always prefixed by "lua.". So this new declared
+sample fetch will be used calling "lua.my-hash" in the HAProxy configuration
+file.
+
+The second parameter is an inline declared anonymous function. Note the closed
+parenthesis after the keyword "end" which ends the function. The first parameter
+of this anonymous function is "txn". It is an object of class TXN. It provides
+access functions. The second parameter is an arbitrary value provided by the
+HAProxy configuration file. This parameter is optional, the developer must
+check if it is present.
+
+The anonymous function registration is executed when the HAProxy backend or
+frontend configuration references the sample fetch "lua.my-hash".
+
+This example can be written with another style, like below:
+
+ function my_hash(txn, salt)
+ return txn.sc:sdbm(salt .. txn.sf:req_fhdr("host") .. txn.sf:path() .. txn.sf:src(), 1)
+ end
+
+ core.register_fetches("my-hash", my_hash)
+
+This second form is clearer, but the first one is compact.
+
+The operator ".." is a string concatenation. If one of the two operands is not a
+string, an error occurs and the execution is immediately stopped. This is
+important to keep in mind for the following things.
+
+Now I write the example on more than one line. Its an easiest way for commenting
+the code:
+
+ 1. function my_hash(txn, salt)
+ 2. local str = ""
+ 3. str = str .. salt
+ 4. str = str .. txn.sf:req_fhdr("host")
+ 5. str = str .. txn.sf:path()
+ 6. str = str .. txn.sf:src()
+ 7. local result = txn.sc:sdbm(str, 1)
+ 8. return result
+ 9. end
+ 10.
+ 11. core.register_fetches("my-hash", my_hash)
+
+local
+~~~~~
+
+The first keyword is "local". This is a really important keyword. You must
+understand that the function "my_hash" will be called for each HAProxy request
+using the declared sample fetch. So, this function can be executed many times in
+parallel.
+
+By default, Lua uses global variables. So in this example, if the variable "str"
+is declared without the keyword "local", it will be shared by all the parallel
+executions of the function and obviously, the content of the requests will be
+shared.
+
+This warning is very important. I tried to write useful Lua code like a rewrite
+of the statistics page, and it is very hard thing to declare each variable as
+"local".
+
+I guess that this behaviour will be the cause of many troubles on the mailing
+list.
+
+str = str ..
+~~~~~~~~~~~~
+
+Now a parenthesis about the form "str = str ..". This form allows to do string
+concatenations. Remember that Lua uses a garbage collector, so what happens when
+we do "str = str .. 'another string'" ?
+
+ str = str .. "another string"
+ ^ ^ ^ ^
+ 1 2 3 4
+
+Lua executes first the concatenation operator (3), it allocates memory for the
+resulting string and fill this memory with the concatenation of the operands 2
+and 4. Next, it frees the variable 1, now the old content of 1 can be garbage
+collected. And finally, the new content of 1 is the concatenation.
+
+what the matter ? when we do this operation many times, we consume a lot of
+memory, and the string data is duplicated and move many times. So, this practice
+is expensive in execution time and memory consumption.
+
+There are easy ways to prevent this behaviour. I guess that a C binding for
+concatenation with chunks will be available ASAP (it is already written). I do
+some benchmarks. I compare the execution time of 1 000 times, 1 000
+concatenation of 10 bytes written in pure Lua and with a C library. The result is
+10 times faster in C (1s in Lua, and 0.1s in C).
+
+txn
+~~~
+
+txn is an HAProxy object of class TXN. The documentation is available in the
+HAProxy Lua API reference. This class allow the access to the native HAProxy
+sample fetches and converters. The object txn contains 2 members dedicated to
+the sample fetches and 2 members dedicated to the converters.
+
+The sample fetches members are "f" (as sample-Fetch) and "sf" (as String
+sample-Fetch). These two members contain exactly the same functions. All the
+HAProxy native sample fetches are available, obviously, the Lua registered sample
+fetches are not available. Unfortunately, HAProxy sample fetches names are not
+compatible with the Lua function names, and they are renamed. The rename
+convention is simple, we replace all the '.', '+' and '-' by '_'. The '.' is the
+object member separator, and the "-" and "+" is math operator.
+
+Now, that I'm writing this article, I know the Lua better than I wrote the
+sample-fetches wrapper. The original HAProxy sample-fetches name should be used
+using alternative manner to call an object member, so the sample-fetch
+"req.fhdr" (actually renamed req_fhdr") should be used like this:
+
+ txn.f["req.fhdr"](txn.f, ...)
+
+However, I think that this form is not elegant.
+
+The "s" collection return a data with a type near to the original returned type.
+A string returns an Lua string, an integer returns an Lua integer and an IP
+address returns an Lua string. Sometime the data is not or not yet available, in
+this case it returns the Lua nil value.
+
+The "sf" collection guarantees that a string will be always returned. If the data
+is not available, an empty string is returned. The main usage of these collection
+is to concatenate the returned sample-fetches without testing each function.
+
+The parameters of the sample-fetches are according with the HAProxy
+documentation.
+
+The converters run exactly with the same manner as the sample fetches. The
+only one difference is that the first parameter is the converter entry element.
+The "c" collection returns a precise result, and the "sc" collection returns
+always a string.
+
+The sample-fetches used in the example function are "txn.sf:req_fhdr()",
+"txn.sf:path()" and "txn.sf:src()". The converter is "txn.sc:sdbm()". The same
+function with the "s" collection of sample-fetches and the "c" collection of
+converter should be written like this:
+
+ 1. function my_hash(txn, salt)
+ 2. local str = ""
+ 3. str = str .. salt
+ 4. str = str .. tostring(txn.f:req_fhdr("host"))
+ 5. str = str .. tostring(txn.f:path())
+ 6. str = str .. tostring(txn.f:src())
+ 7. local result = tostring(txn.c:sdbm(str, 1))
+ 8. return result
+ 9. end
+ 10.
+ 11. core.register_fetches("my-hash", my_hash)
+
+tostring
+~~~~~~~~
+
+The function tostring ensures that its parameter is returned as a string. If the
+parameter is a table or a thread or anything that will not have any sense as a
+string, a form like the typename followed by a pointer is returned. For example:
+
+ t = {}
+ print(tostring(t))
+
+returns:
+
+ table: 0x15facc0
+
+For objects, if the special function __tostring() is registered in the attached
+metatable, it will be called with the table itself as first argument. The
+HAProxy object returns its own type.
+
+About the converters entry point
+--------------------------------
+
+In HAProxy, a converter is a stateless function that takes a data as entry and
+returns a transformation of this data as output. In Lua it is exactly the same
+behaviour.
+
+So, the registered Lua function doesn't have any special parameters, just a
+variable as input which contains the value to convert, and it must return data.
+
+The data required as input by the Lua converter is a string. So HAProxy will
+always provide a string as input. If the native sample fetch is not a string it
+will be converted in best effort.
+
+The returned value will have anything type, it will be converted as sample of
+the near HAProxy type. The conversion rules from Lua variables to HAProxy
+samples are:
+
+ Lua | HAProxy sample types
+ -----------+---------------------
+ "number" | "sint"
+ "boolean" | "bool"
+ "string" | "str"
+ "userdata" | "bool" (false)
+ "nil" | "bool" (false)
+ "table" | "bool" (false)
+ "function" | "bool" (false)
+ "thread" | "bool" (false)
+
+The function used for registering a converter is:
+
+ core.register_converters()
+
+The task entry point
+--------------------
+
+The function "core.register_task(fcn)" executes once the function "fcn" when the
+scheduler starts. This way is used for executing background task. For example,
+you can use this functionality for periodically checking the health of another
+service, and giving the result to each proxy needing it.
+
+The task is started once, if you want periodic actions, you can use the
+"core.sleep()" or "core.msleep()" for waiting the next runtime.
+
+Storing Lua variable between function in the same session
+---------------------------------------------------------
+
+All the functions registered as action or sample fetch can share an Lua context.
+This context is a memory zone in the stack. sample fetch and action use the
+same stack, so both can access to the context.
+
+The context is accessible via the function get_priv and set_priv provided by an
+object of class TXN. The value given to set_priv replaces the current stored
+value. This value can be a table, it is useful if a lot of data can be shared.
+
+If the value stored is a table, you can add or remove entries from the table
+without storing again the new table. Maybe an example will be clearer:
+
+ local t = {}
+ txn:set_priv(t)
+
+ t["entry1"] = "foo"
+ t["entry2"] = "bar"
+
+ -- this will display "foo"
+ print(txn:get_priv()["entry1"])
+
+HTTP actions
+============
+
+ ... coming soon ...
+
+Lua is fast, but my service require more execution speed
+========================================================
+
+We can write C modules for Lua. These modules must run with HAProxy while they
+are compliant with the HAProxy Lua version. A simple example is the "concat"
+module.
+
+It is very easy to write and compile a C Lua library, however, I don't see
+documentation about this process. So the current chapter is a quick howto.
+
+The entry point
+---------------
+
+The entry point is called "luaopen_<name>", where <name> is the name of the ".so"
+file. An hello world is like this:
+
+ #include <stdio.h>
+ #include <lua.h>
+ #include <lauxlib.h>
+
+ int luaopen_mymod(lua_State *L)
+ {
+ printf("Hello world\n");
+ return 0;
+ }
+
+The build
+---------
+
+The compilation of the source file requires the Lua "include" directory. The
+compilation and the link of the object file requires the -fPIC option. That's
+all.
+
+ cc -I/opt/lua/include -fPIC -shared -o mymod.so mymod.c
+
+Usage
+-----
+
+You can load this module with the following Lua syntax:
+
+ require("mymod")
+
+When you start HAProxy, this module just print "Hello world" when it is loaded.
+Please, remember that HAProxy doesn't allow blocking method, so if you write a
+function doing filesystem access or synchronous network access, all the HAProxy
+process will fail.
diff --git a/doc/management.txt b/doc/management.txt
new file mode 100644
index 0000000..b1789db
--- /dev/null
+++ b/doc/management.txt
@@ -0,0 +1,4521 @@
+ ------------------------
+ HAProxy Management Guide
+ ------------------------
+ version 2.9
+
+
+This document describes how to start, stop, manage, and troubleshoot HAProxy,
+as well as some known limitations and traps to avoid. It does not describe how
+to configure it (for this please read configuration.txt).
+
+Note to documentation contributors :
+ This document is formatted with 80 columns per line, with even number of
+ spaces for indentation and without tabs. Please follow these rules strictly
+ so that it remains easily printable everywhere. If you add sections, please
+ update the summary below for easier searching.
+
+
+Summary
+-------
+
+1. Prerequisites
+2. Quick reminder about HAProxy's architecture
+3. Starting HAProxy
+4. Stopping and restarting HAProxy
+5. File-descriptor limitations
+6. Memory management
+7. CPU usage
+8. Logging
+9. Statistics and monitoring
+9.1. CSV format
+9.2. Typed output format
+9.3. Unix Socket commands
+9.4. Master CLI
+9.4.1. Master CLI commands
+10. Tricks for easier configuration management
+11. Well-known traps to avoid
+12. Debugging and performance issues
+13. Security considerations
+
+
+1. Prerequisites
+----------------
+
+In this document it is assumed that the reader has sufficient administration
+skills on a UNIX-like operating system, uses the shell on a daily basis and is
+familiar with troubleshooting utilities such as strace and tcpdump.
+
+
+2. Quick reminder about HAProxy's architecture
+----------------------------------------------
+
+HAProxy is a multi-threaded, event-driven, non-blocking daemon. This means is
+uses event multiplexing to schedule all of its activities instead of relying on
+the system to schedule between multiple activities. Most of the time it runs as
+a single process, so the output of "ps aux" on a system will report only one
+"haproxy" process, unless a soft reload is in progress and an older process is
+finishing its job in parallel to the new one. It is thus always easy to trace
+its activity using the strace utility. In order to scale with the number of
+available processors, by default haproxy will start one worker thread per
+processor it is allowed to run on. Unless explicitly configured differently,
+the incoming traffic is spread over all these threads, all running the same
+event loop. A great care is taken to limit inter-thread dependencies to the
+strict minimum, so as to try to achieve near-linear scalability. This has some
+impacts such as the fact that a given connection is served by a single thread.
+Thus in order to use all available processing capacity, it is needed to have at
+least as many connections as there are threads, which is almost always granted.
+
+HAProxy is designed to isolate itself into a chroot jail during startup, where
+it cannot perform any file-system access at all. This is also true for the
+libraries it depends on (eg: libc, libssl, etc). The immediate effect is that
+a running process will not be able to reload a configuration file to apply
+changes, instead a new process will be started using the updated configuration
+file. Some other less obvious effects are that some timezone files or resolver
+files the libc might attempt to access at run time will not be found, though
+this should generally not happen as they're not needed after startup. A nice
+consequence of this principle is that the HAProxy process is totally stateless,
+and no cleanup is needed after it's killed, so any killing method that works
+will do the right thing.
+
+HAProxy doesn't write log files, but it relies on the standard syslog protocol
+to send logs to a remote server (which is often located on the same system).
+
+HAProxy uses its internal clock to enforce timeouts, that is derived from the
+system's time but where unexpected drift is corrected. This is done by limiting
+the time spent waiting in poll() for an event, and measuring the time it really
+took. In practice it never waits more than one second. This explains why, when
+running strace over a completely idle process, periodic calls to poll() (or any
+of its variants) surrounded by two gettimeofday() calls are noticed. They are
+normal, completely harmless and so cheap that the load they imply is totally
+undetectable at the system scale, so there's nothing abnormal there. Example :
+
+ 16:35:40.002320 gettimeofday({1442759740, 2605}, NULL) = 0
+ 16:35:40.002942 epoll_wait(0, {}, 200, 1000) = 0
+ 16:35:41.007542 gettimeofday({1442759741, 7641}, NULL) = 0
+ 16:35:41.007998 gettimeofday({1442759741, 8114}, NULL) = 0
+ 16:35:41.008391 epoll_wait(0, {}, 200, 1000) = 0
+ 16:35:42.011313 gettimeofday({1442759742, 11411}, NULL) = 0
+
+HAProxy is a TCP proxy, not a router. It deals with established connections that
+have been validated by the kernel, and not with packets of any form nor with
+sockets in other states (eg: no SYN_RECV nor TIME_WAIT), though their existence
+may prevent it from binding a port. It relies on the system to accept incoming
+connections and to initiate outgoing connections. An immediate effect of this is
+that there is no relation between packets observed on the two sides of a
+forwarded connection, which can be of different size, numbers and even family.
+Since a connection may only be accepted from a socket in LISTEN state, all the
+sockets it is listening to are necessarily visible using the "netstat" utility
+to show listening sockets. Example :
+
+ # netstat -ltnp
+ Active Internet connections (only servers)
+ Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
+ tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1629/sshd
+ tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 2847/haproxy
+ tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 2847/haproxy
+
+
+3. Starting HAProxy
+-------------------
+
+HAProxy is started by invoking the "haproxy" program with a number of arguments
+passed on the command line. The actual syntax is :
+
+ $ haproxy [<options>]*
+
+where [<options>]* is any number of options. An option always starts with '-'
+followed by one of more letters, and possibly followed by one or multiple extra
+arguments. Without any option, HAProxy displays the help page with a reminder
+about supported options. Available options may vary slightly based on the
+operating system. A fair number of these options overlap with an equivalent one
+if the "global" section. In this case, the command line always has precedence
+over the configuration file, so that the command line can be used to quickly
+enforce some settings without touching the configuration files. The current
+list of options is :
+
+ -- <cfgfile>* : all the arguments following "--" are paths to configuration
+ file/directory to be loaded and processed in the declaration order. It is
+ mostly useful when relying on the shell to load many files that are
+ numerically ordered. See also "-f". The difference between "--" and "-f" is
+ that one "-f" must be placed before each file name, while a single "--" is
+ needed before all file names. Both options can be used together, the
+ command line ordering still applies. When more than one file is specified,
+ each file must start on a section boundary, so the first keyword of each
+ file must be one of "global", "defaults", "peers", "listen", "frontend",
+ "backend", and so on. A file cannot contain just a server list for example.
+
+ -f <cfgfile|cfgdir> : adds <cfgfile> to the list of configuration files to be
+ loaded. If <cfgdir> is a directory, all the files (and only files) it
+ contains are added in lexical order (using LC_COLLATE=C) to the list of
+ configuration files to be loaded ; only files with ".cfg" extension are
+ added, only non hidden files (not prefixed with ".") are added.
+ Configuration files are loaded and processed in their declaration order.
+ This option may be specified multiple times to load multiple files. See
+ also "--". The difference between "--" and "-f" is that one "-f" must be
+ placed before each file name, while a single "--" is needed before all file
+ names. Both options can be used together, the command line ordering still
+ applies. When more than one file is specified, each file must start on a
+ section boundary, so the first keyword of each file must be one of
+ "global", "defaults", "peers", "listen", "frontend", "backend", and so on.
+ A file cannot contain just a server list for example.
+
+ -C <dir> : changes to directory <dir> before loading configuration
+ files. This is useful when using relative paths. Warning when using
+ wildcards after "--" which are in fact replaced by the shell before
+ starting haproxy.
+
+ -D : start as a daemon. The process detaches from the current terminal after
+ forking, and errors are not reported anymore in the terminal. It is
+ equivalent to the "daemon" keyword in the "global" section of the
+ configuration. It is recommended to always force it in any init script so
+ that a faulty configuration doesn't prevent the system from booting.
+
+ -L <name> : change the local peer name to <name>, which defaults to the local
+ hostname. This is used only with peers replication. You can use the
+ variable $HAPROXY_LOCALPEER in the configuration file to reference the
+ peer name.
+
+ -N <limit> : sets the default per-proxy maxconn to <limit> instead of the
+ builtin default value (usually 2000). Only useful for debugging.
+
+ -V : enable verbose mode (disables quiet mode). Reverts the effect of "-q" or
+ "quiet".
+
+ -W : master-worker mode. It is equivalent to the "master-worker" keyword in
+ the "global" section of the configuration. This mode will launch a "master"
+ which will monitor the "workers". Using this mode, you can reload HAProxy
+ directly by sending a SIGUSR2 signal to the master. The master-worker mode
+ is compatible either with the foreground or daemon mode. It is
+ recommended to use this mode with multiprocess and systemd.
+
+ -Ws : master-worker mode with support of `notify` type of systemd service.
+ This option is only available when HAProxy was built with `USE_SYSTEMD`
+ build option enabled.
+
+ -c : only performs a check of the configuration files and exits before trying
+ to bind. The exit status is zero if everything is OK, or non-zero if an
+ error is encountered. Presence of warnings will be reported if any.
+
+ -cc : evaluates a condition as used within a conditional block of the
+ configuration. The exit status is zero if the condition is true, 1 if the
+ condition is false or 2 if an error is encountered.
+
+ -d : enable debug mode. This disables daemon mode, forces the process to stay
+ in foreground and to show incoming and outgoing events. It must never be
+ used in an init script.
+
+ -dC[key] : dump the configuration file. It is performed after the lines are
+ tokenized, so comments are stripped and indenting is forced. If a non-zero
+ key is specified, lines are truncated before sensitive/confidential fields,
+ and identifiers and addresses are emitted hashed with this key using the
+ same algorithm as the one used by the anonymized mode on the CLI. This
+ means that the output may safely be shared with a developer who needs it
+ to figure what's happening in a dump that was anonymized using the same
+ key. Please also see the CLI's "set anon" command.
+
+ -dD : enable diagnostic mode. This mode will output extra warnings about
+ suspicious configuration statements. This will never prevent startup even in
+ "zero-warning" mode nor change the exit status code.
+
+ -dF : disable data fast-forward. It is a mechanism to optimize the data
+ forwarding by passing data directly from a side to the other one without
+ waking the stream up. Thanks to this directive, it is possible to disable
+ this optimization. Note it also disable any kernel tcp splicing. This
+ command is not meant for regular use, it will generally only be suggested by
+ developers along complex debugging sessions.
+
+ -dG : disable use of getaddrinfo() to resolve host names into addresses. It
+ can be used when suspecting that getaddrinfo() doesn't work as expected.
+ This option was made available because many bogus implementations of
+ getaddrinfo() exist on various systems and cause anomalies that are
+ difficult to troubleshoot.
+
+ -dK<class[,class]*> : dumps the list of registered keywords in each class.
+ The list of classes is available with "-dKhelp". All classes may be dumped
+ using "-dKall", otherwise a selection of those shown in the help can be
+ specified as a comma-delimited list. The output format will vary depending
+ on what class of keywords is being dumped (e.g. "cfg" will show the known
+ configuration keywords in a format resembling the config file format while
+ "smp" will show sample fetch functions prefixed with a compatibility matrix
+ with each rule set). These may rarely be used as-is by humans but can be of
+ great help for external tools that try to detect the appearance of new
+ keywords at certain places to automatically update some documentation,
+ syntax highlighting files, configuration parsers, API etc. The output
+ format may evolve a bit over time so it is really recommended to use this
+ output mostly to detect differences with previous archives. Note that not
+ all keywords are listed because many keywords have existed long before the
+ different keyword registration subsystems were created, and they do not
+ appear there. However since new keywords are only added via the modern
+ mechanisms, it's reasonably safe to assume that this output may be used to
+ detect language additions with a good accuracy. The keywords are only
+ dumped after the configuration is fully parsed, so that even dynamically
+ created keywords can be dumped. A good way to dump and exit is to run a
+ silent config check on an existing configuration:
+
+ ./haproxy -dKall -q -c -f foo.cfg
+
+ If no configuration file is available, using "-f /dev/null" will work as
+ well to dump all default keywords, but then the return status will not be
+ zero since there will be no listener, and will have to be ignored.
+
+ -dL : dumps the list of dynamic shared libraries that are loaded at the end
+ of the config processing. This will generally also include deep dependencies
+ such as anything loaded from Lua code for example, as well as the executable
+ itself. The list is printed in a format that ought to be easy enough to
+ sanitize to directly produce a tarball of all dependencies. Since it doesn't
+ stop the program's startup, it is recommended to only use it in combination
+ with "-c" and "-q" where only the list of loaded objects will be displayed
+ (or nothing in case of error). In addition, keep in mind that when providing
+ such a package to help with a core file analysis, most libraries are in fact
+ symbolic links that need to be dereferenced when creating the archive:
+
+ ./haproxy -W -q -c -dL -f foo.cfg | tar -T - -hzcf archive.tgz
+
+ When started in verbose mode (-V) the shared libraries' address ranges are
+ also enumerated, unless the quiet mode is in use (-q).
+
+ -dM[<byte>[,]][help|options,...] : forces memory poisoning, and/or changes
+ memory other debugging options. Memory poisonning means that each and every
+ memory region allocated with malloc() or pool_alloc() will be filled with
+ <byte> before being passed to the caller. When <byte> is not specified, it
+ defaults to 0x50 ('P'). While this slightly slows down operations, it is
+ useful to reliably trigger issues resulting from missing initializations in
+ the code that cause random crashes. Note that -dM0 has the effect of
+ turning any malloc() into a calloc(). In any case if a bug appears or
+ disappears when using this option it means there is a bug in haproxy, so
+ please report it. A number of other options are available either alone or
+ after a comma following the byte. The special option "help" will list the
+ currently supported options and their current value. Each debugging option
+ may be forced on or off. The most optimal options are usually chosen at
+ build time based on the operating system and do not need to be adjusted,
+ unless suggested by a developer. Supported debugging options include
+ (set/clear):
+ - fail / no-fail:
+ This enables randomly failing memory allocations, in conjunction with
+ the global "tune.fail-alloc" setting. This is used to detect missing
+ error checks in the code. Setting the option presets the ratio to 1%
+ failure rate.
+
+ - no-merge / merge:
+ By default, pools of very similar sizes are merged, resulting in more
+ efficiency, but this complicates the analysis of certain memory dumps.
+ This option allows to disable this mechanism, and may slightly increase
+ the memory usage.
+
+ - cold-first / hot-first:
+ In order to optimize the CPU cache hit ratio, by default the most
+ recently released objects ("hot") are recycled for new allocations.
+ But doing so also complicates analysis of memory dumps and may hide
+ use-after-free bugs. This option allows to instead pick the coldest
+ objects first, which may result in a slight increase of CPU usage.
+
+ - integrity / no-integrity:
+ When this option is enabled, memory integrity checks are enabled on
+ the allocated area to verify that it hasn't been modified since it was
+ last released. This works best with "no-merge", "cold-first" and "tag".
+ Enabling this option will slightly increase the CPU usage.
+
+ - no-global / global:
+ Depending on the operating system, a process-wide global memory cache
+ may be enabled if it is estimated that the standard allocator is too
+ slow or inefficient with threads. This option allows to forcefully
+ disable it or enable it. Disabling it may result in a CPU usage
+ increase with inefficient allocators. Enabling it may result in a
+ higher memory usage with efficient allocators.
+
+ - no-cache / cache:
+ Each thread uses a very fast local object cache for allocations, which
+ is always enabled by default. This option allows to disable it. Since
+ the global cache also passes via the local caches, this will
+ effectively result in disabling all caches and allocating directly from
+ the default allocator. This may result in a significant increase of CPU
+ usage, but may also result in small memory savings on tiny systems.
+
+ - caller / no-caller:
+ Enabling this option reserves some extra space in each allocated object
+ to store the address of the last caller that allocated or released it.
+ This helps developers go back in time when analysing memory dumps and
+ to guess how something unexpected happened.
+
+ - tag / no-tag:
+ Enabling this option reserves some extra space in each allocated object
+ to store a tag that allows to detect bugs such as double-free, freeing
+ an invalid object, and buffer overflows. It offers much stronger
+ reliability guarantees at the expense of 4 or 8 extra bytes per
+ allocation. It usually is the first step to detect memory corruption.
+
+ - poison / no-poison:
+ Enabling this option will fill allocated objects with a fixed pattern
+ that will make sure that some accidental values such as 0 will not be
+ present if a newly added field was mistakenly forgotten in an
+ initialization routine. Such bugs tend to rarely reproduce, especially
+ when pools are not merged. This is normally enabled by directly passing
+ the byte's value to -dM but using this option allows to disable/enable
+ use of a previously set value.
+
+ -dS : disable use of the splice() system call. It is equivalent to the
+ "global" section's "nosplice" keyword. This may be used when splice() is
+ suspected to behave improperly or to cause performance issues, or when
+ using strace to see the forwarded data (which do not appear when using
+ splice()).
+
+ -dV : disable SSL verify on the server side. It is equivalent to having
+ "ssl-server-verify none" in the "global" section. This is useful when
+ trying to reproduce production issues out of the production
+ environment. Never use this in an init script as it degrades SSL security
+ to the servers.
+
+ -dW : if set, haproxy will refuse to start if any warning was emitted while
+ processing the configuration. This helps detect subtle mistakes and keep the
+ configuration clean and portable across versions. It is recommended to set
+ this option in service scripts when configurations are managed by humans,
+ but it is recommended not to use it with generated configurations, which
+ tend to emit more warnings. It may be combined with "-c" to cause warnings
+ in checked configurations to fail. This is equivalent to global option
+ "zero-warning".
+
+ -db : disable background mode and multi-process mode. The process remains in
+ foreground. It is mainly used during development or during small tests, as
+ Ctrl-C is enough to stop the process. Never use it in an init script.
+
+ -de : disable the use of the "epoll" poller. It is equivalent to the "global"
+ section's keyword "noepoll". It is mostly useful when suspecting a bug
+ related to this poller. On systems supporting epoll, the fallback will
+ generally be the "poll" poller.
+
+ -dk : disable the use of the "kqueue" poller. It is equivalent to the
+ "global" section's keyword "nokqueue". It is mostly useful when suspecting
+ a bug related to this poller. On systems supporting kqueue, the fallback
+ will generally be the "poll" poller.
+
+ -dp : disable the use of the "poll" poller. It is equivalent to the "global"
+ section's keyword "nopoll". It is mostly useful when suspecting a bug
+ related to this poller. On systems supporting poll, the fallback will
+ generally be the "select" poller, which cannot be disabled and is limited
+ to 1024 file descriptors.
+
+ -dr : ignore server address resolution failures. It is very common when
+ validating a configuration out of production not to have access to the same
+ resolvers and to fail on server address resolution, making it difficult to
+ test a configuration. This option simply appends the "none" method to the
+ list of address resolution methods for all servers, ensuring that even if
+ the libc fails to resolve an address, the startup sequence is not
+ interrupted.
+
+ -dt [<trace_desc>,...] : activates traces on stderr. Without argument, this
+ enables all trace sources on error level. This can notably be useful to
+ detect protocol violations from clients or servers. An optional argument
+ can be used to specify a list of various trace configurations using ',' as
+ separator. Each element activates one or all trace sources. Additionally,
+ level and verbosity can be optionally specified on each element using ':' as
+ inner separator with trace name.
+
+ -m <limit> : limit the total allocatable memory to <limit> megabytes across
+ all processes. This may cause some connection refusals or some slowdowns
+ depending on the amount of memory needed for normal operations. This is
+ mostly used to force the processes to work in a constrained resource usage
+ scenario. It is important to note that the memory is not shared between
+ processes, so in a multi-process scenario, this value is first divided by
+ global.nbproc before forking.
+
+ -n <limit> : limits the per-process connection limit to <limit>. This is
+ equivalent to the global section's keyword "maxconn". It has precedence
+ over this keyword. This may be used to quickly force lower limits to avoid
+ a service outage on systems where resource limits are too low.
+
+ -p <file> : write all processes' pids into <file> during startup. This is
+ equivalent to the "global" section's keyword "pidfile". The file is opened
+ before entering the chroot jail, and after doing the chdir() implied by
+ "-C". Each pid appears on its own line.
+
+ -q : set "quiet" mode. This disables the output messages. It can be used in
+ combination with "-c" to just check if a configuration file is valid or not.
+
+ -S <bind>[,bind_options...]: in master-worker mode, bind a master CLI, which
+ allows the access to every processes, running or leaving ones.
+ For security reasons, it is recommended to bind the master CLI to a local
+ UNIX socket. The bind options are the same as the keyword "bind" in
+ the configuration file with words separated by commas instead of spaces.
+
+ Note that this socket can't be used to retrieve the listening sockets from
+ an old process during a seamless reload.
+
+ -sf <pid>* : send the "finish" signal (SIGUSR1) to older processes after boot
+ completion to ask them to finish what they are doing and to leave. <pid>
+ is a list of pids to signal (one per argument). The list ends on any
+ option starting with a "-". It is not a problem if the list of pids is
+ empty, so that it can be built on the fly based on the result of a command
+ like "pidof" or "pgrep".
+
+ -st <pid>* : send the "terminate" signal (SIGTERM) to older processes after
+ boot completion to terminate them immediately without finishing what they
+ were doing. <pid> is a list of pids to signal (one per argument). The list
+ is ends on any option starting with a "-". It is not a problem if the list
+ of pids is empty, so that it can be built on the fly based on the result of
+ a command like "pidof" or "pgrep".
+
+ -v : report the version and build date.
+
+ -vv : display the version, build options, libraries versions and usable
+ pollers. This output is systematically requested when filing a bug report.
+
+ -x <unix_socket> : connect to the specified socket and try to retrieve any
+ listening sockets from the old process, and use them instead of trying to
+ bind new ones. This is useful to avoid missing any new connection when
+ reloading the configuration on Linux. The capability must be enable on the
+ stats socket using "expose-fd listeners" in your configuration.
+ In master-worker mode, the master will use this option upon a reload with
+ the "sockpair@" syntax, which allows the master to connect directly to a
+ worker without using stats socket declared in the configuration.
+
+A safe way to start HAProxy from an init file consists in forcing the daemon
+mode, storing existing pids to a pid file and using this pid file to notify
+older processes to finish before leaving :
+
+ haproxy -f /etc/haproxy.cfg \
+ -D -p /var/run/haproxy.pid -sf $(cat /var/run/haproxy.pid)
+
+When the configuration is split into a few specific files (eg: tcp vs http),
+it is recommended to use the "-f" option :
+
+ haproxy -f /etc/haproxy/global.cfg -f /etc/haproxy/stats.cfg \
+ -f /etc/haproxy/default-tcp.cfg -f /etc/haproxy/tcp.cfg \
+ -f /etc/haproxy/default-http.cfg -f /etc/haproxy/http.cfg \
+ -D -p /var/run/haproxy.pid -sf $(cat /var/run/haproxy.pid)
+
+When an unknown number of files is expected, such as customer-specific files,
+it is recommended to assign them a name starting with a fixed-size sequence
+number and to use "--" to load them, possibly after loading some defaults :
+
+ haproxy -f /etc/haproxy/global.cfg -f /etc/haproxy/stats.cfg \
+ -f /etc/haproxy/default-tcp.cfg -f /etc/haproxy/tcp.cfg \
+ -f /etc/haproxy/default-http.cfg -f /etc/haproxy/http.cfg \
+ -D -p /var/run/haproxy.pid -sf $(cat /var/run/haproxy.pid) \
+ -f /etc/haproxy/default-customers.cfg -- /etc/haproxy/customers/*
+
+Sometimes a failure to start may happen for whatever reason. Then it is
+important to verify if the version of HAProxy you are invoking is the expected
+version and if it supports the features you are expecting (eg: SSL, PCRE,
+compression, Lua, etc). This can be verified using "haproxy -vv". Some
+important information such as certain build options, the target system and
+the versions of the libraries being used are reported there. It is also what
+you will systematically be asked for when posting a bug report :
+
+ $ haproxy -vv
+ HAProxy version 1.6-dev7-a088d3-4 2015/10/08
+ Copyright 2000-2015 Willy Tarreau <willy@haproxy.org>
+
+ Build options :
+ TARGET = linux2628
+ CPU = generic
+ CC = gcc
+ CFLAGS = -pg -O0 -g -fno-strict-aliasing -Wdeclaration-after-statement \
+ -DBUFSIZE=8030 -DMAXREWRITE=1030 -DSO_MARK=36 -DTCP_REPAIR=19
+ OPTIONS = USE_ZLIB=1 USE_DLMALLOC=1 USE_OPENSSL=1 USE_LUA=1 USE_PCRE=1
+
+ Default settings :
+ maxconn = 2000, bufsize = 8030, maxrewrite = 1030, maxpollevents = 200
+
+ Encrypted password support via crypt(3): yes
+ Built with zlib version : 1.2.6
+ Compression algorithms supported : identity("identity"), deflate("deflate"), \
+ raw-deflate("deflate"), gzip("gzip")
+ Built with OpenSSL version : OpenSSL 1.0.1o 12 Jun 2015
+ Running on OpenSSL version : OpenSSL 1.0.1o 12 Jun 2015
+ OpenSSL library supports TLS extensions : yes
+ OpenSSL library supports SNI : yes
+ OpenSSL library supports prefer-server-ciphers : yes
+ Built with PCRE version : 8.12 2011-01-15
+ PCRE library supports JIT : no (USE_PCRE_JIT not set)
+ Built with Lua version : Lua 5.3.1
+ Built with transparent proxy support using: IP_TRANSPARENT IP_FREEBIND
+
+ Available polling systems :
+ epoll : pref=300, test result OK
+ poll : pref=200, test result OK
+ select : pref=150, test result OK
+ Total: 3 (3 usable), will use epoll.
+
+The relevant information that many non-developer users can verify here are :
+ - the version : 1.6-dev7-a088d3-4 above means the code is currently at commit
+ ID "a088d3" which is the 4th one after after official version "1.6-dev7".
+ Version 1.6-dev7 would show as "1.6-dev7-8c1ad7". What matters here is in
+ fact "1.6-dev7". This is the 7th development version of what will become
+ version 1.6 in the future. A development version not suitable for use in
+ production (unless you know exactly what you are doing). A stable version
+ will show as a 3-numbers version, such as "1.5.14-16f863", indicating the
+ 14th level of fix on top of version 1.5. This is a production-ready version.
+
+ - the release date : 2015/10/08. It is represented in the universal
+ year/month/day format. Here this means August 8th, 2015. Given that stable
+ releases are issued every few months (1-2 months at the beginning, sometimes
+ 6 months once the product becomes very stable), if you're seeing an old date
+ here, it means you're probably affected by a number of bugs or security
+ issues that have since been fixed and that it might be worth checking on the
+ official site.
+
+ - build options : they are relevant to people who build their packages
+ themselves, they can explain why things are not behaving as expected. For
+ example the development version above was built for Linux 2.6.28 or later,
+ targeting a generic CPU (no CPU-specific optimizations), and lacks any
+ code optimization (-O0) so it will perform poorly in terms of performance.
+
+ - libraries versions : zlib version is reported as found in the library
+ itself. In general zlib is considered a very stable product and upgrades
+ are almost never needed. OpenSSL reports two versions, the version used at
+ build time and the one being used, as found on the system. These ones may
+ differ by the last letter but never by the numbers. The build date is also
+ reported because most OpenSSL bugs are security issues and need to be taken
+ seriously, so this library absolutely needs to be kept up to date. Seeing a
+ 4-months old version here is highly suspicious and indeed an update was
+ missed. PCRE provides very fast regular expressions and is highly
+ recommended. Certain of its extensions such as JIT are not present in all
+ versions and still young so some people prefer not to build with them,
+ which is why the build status is reported as well. Regarding the Lua
+ scripting language, HAProxy expects version 5.3 which is very young since
+ it was released a little time before HAProxy 1.6. It is important to check
+ on the Lua web site if some fixes are proposed for this branch.
+
+ - Available polling systems will affect the process's scalability when
+ dealing with more than about one thousand of concurrent connections. These
+ ones are only available when the correct system was indicated in the TARGET
+ variable during the build. The "epoll" mechanism is highly recommended on
+ Linux, and the kqueue mechanism is highly recommended on BSD. Lacking them
+ will result in poll() or even select() being used, causing a high CPU usage
+ when dealing with a lot of connections.
+
+
+4. Stopping and restarting HAProxy
+----------------------------------
+
+HAProxy supports a graceful and a hard stop. The hard stop is simple, when the
+SIGTERM signal is sent to the haproxy process, it immediately quits and all
+established connections are closed. The graceful stop is triggered when the
+SIGUSR1 signal is sent to the haproxy process. It consists in only unbinding
+from listening ports, but continue to process existing connections until they
+close. Once the last connection is closed, the process leaves.
+
+The hard stop method is used for the "stop" or "restart" actions of the service
+management script. The graceful stop is used for the "reload" action which
+tries to seamlessly reload a new configuration in a new process.
+
+Both of these signals may be sent by the new haproxy process itself during a
+reload or restart, so that they are sent at the latest possible moment and only
+if absolutely required. This is what is performed by the "-st" (hard) and "-sf"
+(graceful) options respectively.
+
+In master-worker mode, it is not needed to start a new haproxy process in
+order to reload the configuration. The master process reacts to the SIGUSR2
+signal by reexecuting itself with the -sf parameter followed by the PIDs of
+the workers. The master will then parse the configuration file and fork new
+workers.
+
+To understand better how these signals are used, it is important to understand
+the whole restart mechanism.
+
+First, an existing haproxy process is running. The administrator uses a system
+specific command such as "/etc/init.d/haproxy reload" to indicate they want to
+take the new configuration file into effect. What happens then is the following.
+First, the service script (/etc/init.d/haproxy or equivalent) will verify that
+the configuration file parses correctly using "haproxy -c". After that it will
+try to start haproxy with this configuration file, using "-st" or "-sf".
+
+Then HAProxy tries to bind to all listening ports. If some fatal errors happen
+(eg: address not present on the system, permission denied), the process quits
+with an error. If a socket binding fails because a port is already in use, then
+the process will first send a SIGTTOU signal to all the pids specified in the
+"-st" or "-sf" pid list. This is what is called the "pause" signal. It instructs
+all existing haproxy processes to temporarily stop listening to their ports so
+that the new process can try to bind again. During this time, the old process
+continues to process existing connections. If the binding still fails (because
+for example a port is shared with another daemon), then the new process sends a
+SIGTTIN signal to the old processes to instruct them to resume operations just
+as if nothing happened. The old processes will then restart listening to the
+ports and continue to accept connections. Note that this mechanism is system
+dependent and some operating systems may not support it in multi-process mode.
+
+If the new process manages to bind correctly to all ports, then it sends either
+the SIGTERM (hard stop in case of "-st") or the SIGUSR1 (graceful stop in case
+of "-sf") to all processes to notify them that it is now in charge of operations
+and that the old processes will have to leave, either immediately or once they
+have finished their job.
+
+It is important to note that during this timeframe, there are two small windows
+of a few milliseconds each where it is possible that a few connection failures
+will be noticed during high loads. Typically observed failure rates are around
+1 failure during a reload operation every 10000 new connections per second,
+which means that a heavily loaded site running at 30000 new connections per
+second may see about 3 failed connection upon every reload. The two situations
+where this happens are :
+
+ - if the new process fails to bind due to the presence of the old process,
+ it will first have to go through the SIGTTOU+SIGTTIN sequence, which
+ typically lasts about one millisecond for a few tens of frontends, and
+ during which some ports will not be bound to the old process and not yet
+ bound to the new one. HAProxy works around this on systems that support the
+ SO_REUSEPORT socket options, as it allows the new process to bind without
+ first asking the old one to unbind. Most BSD systems have been supporting
+ this almost forever. Linux has been supporting this in version 2.0 and
+ dropped it around 2.2, but some patches were floating around by then. It
+ was reintroduced in kernel 3.9, so if you are observing a connection
+ failure rate above the one mentioned above, please ensure that your kernel
+ is 3.9 or newer, or that relevant patches were backported to your kernel
+ (less likely).
+
+ - when the old processes close the listening ports, the kernel may not always
+ redistribute any pending connection that was remaining in the socket's
+ backlog. Under high loads, a SYN packet may happen just before the socket
+ is closed, and will lead to an RST packet being sent to the client. In some
+ critical environments where even one drop is not acceptable, these ones are
+ sometimes dealt with using firewall rules to block SYN packets during the
+ reload, forcing the client to retransmit. This is totally system-dependent,
+ as some systems might be able to visit other listening queues and avoid
+ this RST. A second case concerns the ACK from the client on a local socket
+ that was in SYN_RECV state just before the close. This ACK will lead to an
+ RST packet while the haproxy process is still not aware of it. This one is
+ harder to get rid of, though the firewall filtering rules mentioned above
+ will work well if applied one second or so before restarting the process.
+
+For the vast majority of users, such drops will never ever happen since they
+don't have enough load to trigger the race conditions. And for most high traffic
+users, the failure rate is still fairly within the noise margin provided that at
+least SO_REUSEPORT is properly supported on their systems.
+
+5. File-descriptor limitations
+------------------------------
+
+In order to ensure that all incoming connections will successfully be served,
+HAProxy computes at load time the total number of file descriptors that will be
+needed during the process's life. A regular Unix process is generally granted
+1024 file descriptors by default, and a privileged process can raise this limit
+itself. This is one reason for starting HAProxy as root and letting it adjust
+the limit. The default limit of 1024 file descriptors roughly allow about 500
+concurrent connections to be processed. The computation is based on the global
+maxconn parameter which limits the total number of connections per process, the
+number of listeners, the number of servers which have a health check enabled,
+the agent checks, the peers, the loggers and possibly a few other technical
+requirements. A simple rough estimate of this number consists in simply
+doubling the maxconn value and adding a few tens to get the approximate number
+of file descriptors needed.
+
+Originally HAProxy did not know how to compute this value, and it was necessary
+to pass the value using the "ulimit-n" setting in the global section. This
+explains why even today a lot of configurations are seen with this setting
+present. Unfortunately it was often miscalculated resulting in connection
+failures when approaching maxconn instead of throttling incoming connection
+while waiting for the needed resources. For this reason it is important to
+remove any vestigial "ulimit-n" setting that can remain from very old versions.
+
+Raising the number of file descriptors to accept even moderate loads is
+mandatory but comes with some OS-specific adjustments. First, the select()
+polling system is limited to 1024 file descriptors. In fact on Linux it used
+to be capable of handling more but since certain OS ship with excessively
+restrictive SELinux policies forbidding the use of select() with more than
+1024 file descriptors, HAProxy now refuses to start in this case in order to
+avoid any issue at run time. On all supported operating systems, poll() is
+available and will not suffer from this limitation. It is automatically picked
+so there is nothing to do to get a working configuration. But poll's becomes
+very slow when the number of file descriptors increases. While HAProxy does its
+best to limit this performance impact (eg: via the use of the internal file
+descriptor cache and batched processing), a good rule of thumb is that using
+poll() with more than a thousand concurrent connections will use a lot of CPU.
+
+For Linux systems base on kernels 2.6 and above, the epoll() system call will
+be used. It's a much more scalable mechanism relying on callbacks in the kernel
+that guarantee a constant wake up time regardless of the number of registered
+monitored file descriptors. It is automatically used where detected, provided
+that HAProxy had been built for one of the Linux flavors. Its presence and
+support can be verified using "haproxy -vv".
+
+For BSD systems which support it, kqueue() is available as an alternative. It
+is much faster than poll() and even slightly faster than epoll() thanks to its
+batched handling of changes. At least FreeBSD and OpenBSD support it. Just like
+with Linux's epoll(), its support and availability are reported in the output
+of "haproxy -vv".
+
+Having a good poller is one thing, but it is mandatory that the process can
+reach the limits. When HAProxy starts, it immediately sets the new process's
+file descriptor limits and verifies if it succeeds. In case of failure, it
+reports it before forking so that the administrator can see the problem. As
+long as the process is started by as root, there should be no reason for this
+setting to fail. However, it can fail if the process is started by an
+unprivileged user. If there is a compelling reason for *not* starting haproxy
+as root (eg: started by end users, or by a per-application account), then the
+file descriptor limit can be raised by the system administrator for this
+specific user. The effectiveness of the setting can be verified by issuing
+"ulimit -n" from the user's command line. It should reflect the new limit.
+
+Warning: when an unprivileged user's limits are changed in this user's account,
+it is fairly common that these values are only considered when the user logs in
+and not at all in some scripts run at system boot time nor in crontabs. This is
+totally dependent on the operating system, keep in mind to check "ulimit -n"
+before starting haproxy when running this way. The general advice is never to
+start haproxy as an unprivileged user for production purposes. Another good
+reason is that it prevents haproxy from enabling some security protections.
+
+Once it is certain that the system will allow the haproxy process to use the
+requested number of file descriptors, two new system-specific limits may be
+encountered. The first one is the system-wide file descriptor limit, which is
+the total number of file descriptors opened on the system, covering all
+processes. When this limit is reached, accept() or socket() will typically
+return ENFILE. The second one is the per-process hard limit on the number of
+file descriptors, it prevents setrlimit() from being set higher. Both are very
+dependent on the operating system. On Linux, the system limit is set at boot
+based on the amount of memory. It can be changed with the "fs.file-max" sysctl.
+And the per-process hard limit is set to 1048576 by default, but it can be
+changed using the "fs.nr_open" sysctl.
+
+File descriptor limitations may be observed on a running process when they are
+set too low. The strace utility will report that accept() and socket() return
+"-1 EMFILE" when the process's limits have been reached. In this case, simply
+raising the "ulimit-n" value (or removing it) will solve the problem. If these
+system calls return "-1 ENFILE" then it means that the kernel's limits have
+been reached and that something must be done on a system-wide parameter. These
+trouble must absolutely be addressed, as they result in high CPU usage (when
+accept() fails) and failed connections that are generally visible to the user.
+One solution also consists in lowering the global maxconn value to enforce
+serialization, and possibly to disable HTTP keep-alive to force connections
+to be released and reused faster.
+
+
+6. Memory management
+--------------------
+
+HAProxy uses a simple and fast pool-based memory management. Since it relies on
+a small number of different object types, it's much more efficient to pick new
+objects from a pool which already contains objects of the appropriate size than
+to call malloc() for each different size. The pools are organized as a stack or
+LIFO, so that newly allocated objects are taken from recently released objects
+still hot in the CPU caches. Pools of similar sizes are merged together, in
+order to limit memory fragmentation.
+
+By default, since the focus is set on performance, each released object is put
+back into the pool it came from, and allocated objects are never freed since
+they are expected to be reused very soon.
+
+On the CLI, it is possible to check how memory is being used in pools thanks to
+the "show pools" command :
+
+ > show pools
+ Dumping pools usage. Use SIGQUIT to flush them.
+ - Pool cache_st (16 bytes) : 0 allocated (0 bytes), 0 used, 0 failures, 1 users, @0x9ccc40=03 [SHARED]
+ - Pool pipe (32 bytes) : 5 allocated (160 bytes), 5 used, 0 failures, 2 users, @0x9ccac0=00 [SHARED]
+ - Pool comp_state (48 bytes) : 3 allocated (144 bytes), 3 used, 0 failures, 5 users, @0x9cccc0=04 [SHARED]
+ - Pool filter (64 bytes) : 0 allocated (0 bytes), 0 used, 0 failures, 3 users, @0x9ccbc0=02 [SHARED]
+ - Pool vars (80 bytes) : 0 allocated (0 bytes), 0 used, 0 failures, 2 users, @0x9ccb40=01 [SHARED]
+ - Pool uniqueid (128 bytes) : 0 allocated (0 bytes), 0 used, 0 failures, 2 users, @0x9cd240=15 [SHARED]
+ - Pool task (144 bytes) : 55 allocated (7920 bytes), 55 used, 0 failures, 1 users, @0x9cd040=11 [SHARED]
+ - Pool session (160 bytes) : 1 allocated (160 bytes), 1 used, 0 failures, 1 users, @0x9cd140=13 [SHARED]
+ - Pool h2s (208 bytes) : 0 allocated (0 bytes), 0 used, 0 failures, 2 users, @0x9ccec0=08 [SHARED]
+ - Pool h2c (288 bytes) : 0 allocated (0 bytes), 0 used, 0 failures, 1 users, @0x9cce40=07 [SHARED]
+ - Pool spoe_ctx (304 bytes) : 0 allocated (0 bytes), 0 used, 0 failures, 2 users, @0x9ccf40=09 [SHARED]
+ - Pool connection (400 bytes) : 2 allocated (800 bytes), 2 used, 0 failures, 1 users, @0x9cd1c0=14 [SHARED]
+ - Pool hdr_idx (416 bytes) : 0 allocated (0 bytes), 0 used, 0 failures, 1 users, @0x9cd340=17 [SHARED]
+ - Pool dns_resolut (480 bytes) : 0 allocated (0 bytes), 0 used, 0 failures, 1 users, @0x9ccdc0=06 [SHARED]
+ - Pool dns_answer_ (576 bytes) : 0 allocated (0 bytes), 0 used, 0 failures, 1 users, @0x9ccd40=05 [SHARED]
+ - Pool stream (960 bytes) : 1 allocated (960 bytes), 1 used, 0 failures, 1 users, @0x9cd0c0=12 [SHARED]
+ - Pool requri (1024 bytes) : 0 allocated (0 bytes), 0 used, 0 failures, 1 users, @0x9cd2c0=16 [SHARED]
+ - Pool buffer (8030 bytes) : 3 allocated (24090 bytes), 2 used, 0 failures, 1 users, @0x9cd3c0=18 [SHARED]
+ - Pool trash (8062 bytes) : 1 allocated (8062 bytes), 1 used, 0 failures, 1 users, @0x9cd440=19
+ Total: 19 pools, 42296 bytes allocated, 34266 used.
+
+The pool name is only indicative, it's the name of the first object type using
+this pool. The size in parenthesis is the object size for objects in this pool.
+Object sizes are always rounded up to the closest multiple of 16 bytes. The
+number of objects currently allocated and the equivalent number of bytes is
+reported so that it is easy to know which pool is responsible for the highest
+memory usage. The number of objects currently in use is reported as well in the
+"used" field. The difference between "allocated" and "used" corresponds to the
+objects that have been freed and are available for immediate use. The address
+at the end of the line is the pool's address, and the following number is the
+pool index when it exists, or is reported as -1 if no index was assigned.
+
+It is possible to limit the amount of memory allocated per process using the
+"-m" command line option, followed by a number of megabytes. It covers all of
+the process's addressable space, so that includes memory used by some libraries
+as well as the stack, but it is a reliable limit when building a resource
+constrained system. It works the same way as "ulimit -v" on systems which have
+it, or "ulimit -d" for the other ones.
+
+If a memory allocation fails due to the memory limit being reached or because
+the system doesn't have any enough memory, then haproxy will first start to
+free all available objects from all pools before attempting to allocate memory
+again. This mechanism of releasing unused memory can be triggered by sending
+the signal SIGQUIT to the haproxy process. When doing so, the pools state prior
+to the flush will also be reported to stderr when the process runs in
+foreground.
+
+During a reload operation, the process switched to the graceful stop state also
+automatically performs some flushes after releasing any connection so that all
+possible memory is released to save it for the new process.
+
+
+7. CPU usage
+------------
+
+HAProxy normally spends most of its time in the system and a smaller part in
+userland. A finely tuned 3.5 GHz CPU can sustain a rate about 80000 end-to-end
+connection setups and closes per second at 100% CPU on a single core. When one
+core is saturated, typical figures are :
+ - 95% system, 5% user for long TCP connections or large HTTP objects
+ - 85% system and 15% user for short TCP connections or small HTTP objects in
+ close mode
+ - 70% system and 30% user for small HTTP objects in keep-alive mode
+
+The amount of rules processing and regular expressions will increase the user
+land part. The presence of firewall rules, connection tracking, complex routing
+tables in the system will instead increase the system part.
+
+On most systems, the CPU time observed during network transfers can be cut in 4
+parts :
+ - the interrupt part, which concerns all the processing performed upon I/O
+ receipt, before the target process is even known. Typically Rx packets are
+ accounted for in interrupt. On some systems such as Linux where interrupt
+ processing may be deferred to a dedicated thread, it can appear as softirq,
+ and the thread is called ksoftirqd/0 (for CPU 0). The CPU taking care of
+ this load is generally defined by the hardware settings, though in the case
+ of softirq it is often possible to remap the processing to another CPU.
+ This interrupt part will often be perceived as parasitic since it's not
+ associated with any process, but it actually is some processing being done
+ to prepare the work for the process.
+
+ - the system part, which concerns all the processing done using kernel code
+ called from userland. System calls are accounted as system for example. All
+ synchronously delivered Tx packets will be accounted for as system time. If
+ some packets have to be deferred due to queues filling up, they may then be
+ processed in interrupt context later (eg: upon receipt of an ACK opening a
+ TCP window).
+
+ - the user part, which exclusively runs application code in userland. HAProxy
+ runs exclusively in this part, though it makes heavy use of system calls.
+ Rules processing, regular expressions, compression, encryption all add to
+ the user portion of CPU consumption.
+
+ - the idle part, which is what the CPU does when there is nothing to do. For
+ example HAProxy waits for an incoming connection, or waits for some data to
+ leave, meaning the system is waiting for an ACK from the client to push
+ these data.
+
+In practice regarding HAProxy's activity, it is in general reasonably accurate
+(but totally inexact) to consider that interrupt/softirq are caused by Rx
+processing in kernel drivers, that user-land is caused by layer 7 processing
+in HAProxy, and that system time is caused by network processing on the Tx
+path.
+
+Since HAProxy runs around an event loop, it waits for new events using poll()
+(or any alternative) and processes all these events as fast as possible before
+going back to poll() waiting for new events. It measures the time spent waiting
+in poll() compared to the time spent doing processing events. The ratio of
+polling time vs total time is called the "idle" time, it's the amount of time
+spent waiting for something to happen. This ratio is reported in the stats page
+on the "idle" line, or "Idle_pct" on the CLI. When it's close to 100%, it means
+the load is extremely low. When it's close to 0%, it means that there is
+constantly some activity. While it cannot be very accurate on an overloaded
+system due to other processes possibly preempting the CPU from the haproxy
+process, it still provides a good estimate about how HAProxy considers it is
+working : if the load is low and the idle ratio is low as well, it may indicate
+that HAProxy has a lot of work to do, possibly due to very expensive rules that
+have to be processed. Conversely, if HAProxy indicates the idle is close to
+100% while things are slow, it means that it cannot do anything to speed things
+up because it is already waiting for incoming data to process. In the example
+below, haproxy is completely idle :
+
+ $ echo "show info" | socat - /var/run/haproxy.sock | grep ^Idle
+ Idle_pct: 100
+
+When the idle ratio starts to become very low, it is important to tune the
+system and place processes and interrupts correctly to save the most possible
+CPU resources for all tasks. If a firewall is present, it may be worth trying
+to disable it or to tune it to ensure it is not responsible for a large part
+of the performance limitation. It's worth noting that unloading a stateful
+firewall generally reduces both the amount of interrupt/softirq and of system
+usage since such firewalls act both on the Rx and the Tx paths. On Linux,
+unloading the nf_conntrack and ip_conntrack modules will show whether there is
+anything to gain. If so, then the module runs with default settings and you'll
+have to figure how to tune it for better performance. In general this consists
+in considerably increasing the hash table size. On FreeBSD, "pfctl -d" will
+disable the "pf" firewall and its stateful engine at the same time.
+
+If it is observed that a lot of time is spent in interrupt/softirq, it is
+important to ensure that they don't run on the same CPU. Most systems tend to
+pin the tasks on the CPU where they receive the network traffic because for
+certain workloads it improves things. But with heavily network-bound workloads
+it is the opposite as the haproxy process will have to fight against its kernel
+counterpart. Pinning haproxy to one CPU core and the interrupts to another one,
+all sharing the same L3 cache tends to sensibly increase network performance
+because in practice the amount of work for haproxy and the network stack are
+quite close, so they can almost fill an entire CPU each. On Linux this is done
+using taskset (for haproxy) or using cpu-map (from the haproxy config), and the
+interrupts are assigned under /proc/irq. Many network interfaces support
+multiple queues and multiple interrupts. In general it helps to spread them
+across a small number of CPU cores provided they all share the same L3 cache.
+Please always stop irq_balance which always does the worst possible thing on
+such workloads.
+
+For CPU-bound workloads consisting in a lot of SSL traffic or a lot of
+compression, it may be worth using multiple processes dedicated to certain
+tasks, though there is no universal rule here and experimentation will have to
+be performed.
+
+In order to increase the CPU capacity, it is possible to make HAProxy run as
+several processes, using the "nbproc" directive in the global section. There
+are some limitations though :
+ - health checks are run per process, so the target servers will get as many
+ checks as there are running processes ;
+ - maxconn values and queues are per-process so the correct value must be set
+ to avoid overloading the servers ;
+ - outgoing connections should avoid using port ranges to avoid conflicts
+ - stick-tables are per process and are not shared between processes ;
+ - each peers section may only run on a single process at a time ;
+ - the CLI operations will only act on a single process at a time.
+
+With this in mind, it appears that the easiest setup often consists in having
+one first layer running on multiple processes and in charge for the heavy
+processing, passing the traffic to a second layer running in a single process.
+This mechanism is suited to SSL and compression which are the two CPU-heavy
+features. Instances can easily be chained over UNIX sockets (which are cheaper
+than TCP sockets and which do not waste ports), and the proxy protocol which is
+useful to pass client information to the next stage. When doing so, it is
+generally a good idea to bind all the single-process tasks to process number 1
+and extra tasks to next processes, as this will make it easier to generate
+similar configurations for different machines.
+
+On Linux versions 3.9 and above, running HAProxy in multi-process mode is much
+more efficient when each process uses a distinct listening socket on the same
+IP:port ; this will make the kernel evenly distribute the load across all
+processes instead of waking them all up. Please check the "process" option of
+the "bind" keyword lines in the configuration manual for more information.
+
+
+8. Logging
+----------
+
+For logging, HAProxy always relies on a syslog server since it does not perform
+any file-system access. The standard way of using it is to send logs over UDP
+to the log server (by default on port 514). Very commonly this is configured to
+127.0.0.1 where the local syslog daemon is running, but it's also used over the
+network to log to a central server. The central server provides additional
+benefits especially in active-active scenarios where it is desirable to keep
+the logs merged in arrival order. HAProxy may also make use of a UNIX socket to
+send its logs to the local syslog daemon, but it is not recommended at all,
+because if the syslog server is restarted while haproxy runs, the socket will
+be replaced and new logs will be lost. Since HAProxy will be isolated inside a
+chroot jail, it will not have the ability to reconnect to the new socket. It
+has also been observed in field that the log buffers in use on UNIX sockets are
+very small and lead to lost messages even at very light loads. But this can be
+fine for testing however.
+
+It is recommended to add the following directive to the "global" section to
+make HAProxy log to the local daemon using facility "local0" :
+
+ log 127.0.0.1:514 local0
+
+and then to add the following one to each "defaults" section or to each frontend
+and backend section :
+
+ log global
+
+This way, all logs will be centralized through the global definition of where
+the log server is.
+
+Some syslog daemons do not listen to UDP traffic by default, so depending on
+the daemon being used, the syntax to enable this will vary :
+
+ - on sysklogd, you need to pass argument "-r" on the daemon's command line
+ so that it listens to a UDP socket for "remote" logs ; note that there is
+ no way to limit it to address 127.0.0.1 so it will also receive logs from
+ remote systems ;
+
+ - on rsyslogd, the following lines must be added to the configuration file :
+
+ $ModLoad imudp
+ $UDPServerAddress *
+ $UDPServerRun 514
+
+ - on syslog-ng, a new source can be created the following way, it then needs
+ to be added as a valid source in one of the "log" directives :
+
+ source s_udp {
+ udp(ip(127.0.0.1) port(514));
+ };
+
+Please consult your syslog daemon's manual for more information. If no logs are
+seen in the system's log files, please consider the following tests :
+
+ - restart haproxy. Each frontend and backend logs one line indicating it's
+ starting. If these logs are received, it means logs are working.
+
+ - run "strace -tt -s100 -etrace=sendmsg -p <haproxy's pid>" and perform some
+ activity that you expect to be logged. You should see the log messages
+ being sent using sendmsg() there. If they don't appear, restart using
+ strace on top of haproxy. If you still see no logs, it definitely means
+ that something is wrong in your configuration.
+
+ - run tcpdump to watch for port 514, for example on the loopback interface if
+ the traffic is being sent locally : "tcpdump -As0 -ni lo port 514". If the
+ packets are seen there, it's the proof they're sent then the syslogd daemon
+ needs to be troubleshooted.
+
+While traffic logs are sent from the frontends (where the incoming connections
+are accepted), backends also need to be able to send logs in order to report a
+server state change consecutive to a health check. Please consult HAProxy's
+configuration manual for more information regarding all possible log settings.
+
+It is convenient to chose a facility that is not used by other daemons. HAProxy
+examples often suggest "local0" for traffic logs and "local1" for admin logs
+because they're never seen in field. A single facility would be enough as well.
+Having separate logs is convenient for log analysis, but it's also important to
+remember that logs may sometimes convey confidential information, and as such
+they must not be mixed with other logs that may accidentally be handed out to
+unauthorized people.
+
+For in-field troubleshooting without impacting the server's capacity too much,
+it is recommended to make use of the "halog" utility provided with HAProxy.
+This is sort of a grep-like utility designed to process HAProxy log files at
+a very fast data rate. Typical figures range between 1 and 2 GB of logs per
+second. It is capable of extracting only certain logs (eg: search for some
+classes of HTTP status codes, connection termination status, search by response
+time ranges, look for errors only), count lines, limit the output to a number
+of lines, and perform some more advanced statistics such as sorting servers
+by response time or error counts, sorting URLs by time or count, sorting client
+addresses by access count, and so on. It is pretty convenient to quickly spot
+anomalies such as a bot looping on the site, and block them.
+
+
+9. Statistics and monitoring
+----------------------------
+
+It is possible to query HAProxy about its status. The most commonly used
+mechanism is the HTTP statistics page. This page also exposes an alternative
+CSV output format for monitoring tools. The same format is provided on the
+Unix socket.
+
+Statistics are regroup in categories labelled as domains, corresponding to the
+multiple components of HAProxy. There are two domains available: proxy and dns.
+If not specified, the proxy domain is selected. Note that only the proxy
+statistics are printed on the HTTP page.
+
+9.1. CSV format
+---------------
+
+The statistics may be consulted either from the unix socket or from the HTTP
+page. Both means provide a CSV format whose fields follow. The first line
+begins with a sharp ('#') and has one word per comma-delimited field which
+represents the title of the column. All other lines starting at the second one
+use a classical CSV format using a comma as the delimiter, and the double quote
+('"') as an optional text delimiter, but only if the enclosed text is ambiguous
+(if it contains a quote or a comma). The double-quote character ('"') in the
+text is doubled ('""'), which is the format that most tools recognize. Please
+do not insert any column before these ones in order not to break tools which
+use hard-coded column positions.
+
+For proxy statistics, after each field name, the types which may have a value
+for that field are specified in brackets. The types are L (Listeners), F
+(Frontends), B (Backends), and S (Servers). There is a fixed set of static
+fields that are always available in the same order. A column containing the
+character '-' delimits the end of the static fields, after which presence or
+order of the fields are not guaranteed.
+
+Here is the list of static fields using the proxy statistics domain:
+ 0. pxname [LFBS]: proxy name
+ 1. svname [LFBS]: service name (FRONTEND for frontend, BACKEND for backend,
+ any name for server/listener)
+ 2. qcur [..BS]: current queued requests. For the backend this reports the
+ number queued without a server assigned.
+ 3. qmax [..BS]: max value of qcur
+ 4. scur [LFBS]: current sessions
+ 5. smax [LFBS]: max sessions
+ 6. slim [LFBS]: configured session limit
+ 7. stot [LFBS]: cumulative number of sessions
+ 8. bin [LFBS]: bytes in
+ 9. bout [LFBS]: bytes out
+ 10. dreq [LFB.]: requests denied because of security concerns.
+ - For tcp this is because of a matched tcp-request content rule.
+ - For http this is because of a matched http-request or tarpit rule.
+ 11. dresp [LFBS]: responses denied because of security concerns.
+ - For http this is because of a matched http-request rule, or
+ "option checkcache".
+ 12. ereq [LF..]: request errors. Some of the possible causes are:
+ - early termination from the client, before the request has been sent.
+ - read error from the client
+ - client timeout
+ - client closed connection
+ - various bad requests from the client.
+ - request was tarpitted.
+ 13. econ [..BS]: number of requests that encountered an error trying to
+ connect to a backend server. The backend stat is the sum of the stat
+ for all servers of that backend, plus any connection errors not
+ associated with a particular server (such as the backend having no
+ active servers).
+ 14. eresp [..BS]: response errors. srv_abrt will be counted here also.
+ Some other errors are:
+ - write error on the client socket (won't be counted for the server stat)
+ - failure applying filters to the response.
+ 15. wretr [..BS]: number of times a connection to a server was retried.
+ 16. wredis [..BS]: number of times a request was redispatched to another
+ server. The server value counts the number of times that server was
+ switched away from.
+ 17. status [LFBS]: status (UP/DOWN/NOLB/MAINT/MAINT(via)/MAINT(resolution)...)
+ 18. weight [..BS]: total effective weight (backend), effective weight (server)
+ 19. act [..BS]: number of active servers (backend), server is active (server)
+ 20. bck [..BS]: number of backup servers (backend), server is backup (server)
+ 21. chkfail [...S]: number of failed checks. (Only counts checks failed when
+ the server is up.)
+ 22. chkdown [..BS]: number of UP->DOWN transitions. The backend counter counts
+ transitions to the whole backend being down, rather than the sum of the
+ counters for each server.
+ 23. lastchg [..BS]: number of seconds since the last UP<->DOWN transition
+ 24. downtime [..BS]: total downtime (in seconds). The value for the backend
+ is the downtime for the whole backend, not the sum of the server downtime.
+ 25. qlimit [...S]: configured maxqueue for the server, or nothing in the
+ value is 0 (default, meaning no limit)
+ 26. pid [LFBS]: process id (0 for first instance, 1 for second, ...)
+ 27. iid [LFBS]: unique proxy id
+ 28. sid [L..S]: server id (unique inside a proxy)
+ 29. throttle [...S]: current throttle percentage for the server, when
+ slowstart is active, or no value if not in slowstart.
+ 30. lbtot [..BS]: total number of times a server was selected, either for new
+ sessions, or when re-dispatching. The server counter is the number
+ of times that server was selected.
+ 31. tracked [...S]: id of proxy/server if tracking is enabled.
+ 32. type [LFBS]: (0=frontend, 1=backend, 2=server, 3=socket/listener)
+ 33. rate [.FBS]: number of sessions per second over last elapsed second
+ 34. rate_lim [.F..]: configured limit on new sessions per second
+ 35. rate_max [.FBS]: max number of new sessions per second
+ 36. check_status [...S]: status of last health check, one of:
+ UNK -> unknown
+ INI -> initializing
+ SOCKERR -> socket error
+ L4OK -> check passed on layer 4, no upper layers testing enabled
+ L4TOUT -> layer 1-4 timeout
+ L4CON -> layer 1-4 connection problem, for example
+ "Connection refused" (tcp rst) or "No route to host" (icmp)
+ L6OK -> check passed on layer 6
+ L6TOUT -> layer 6 (SSL) timeout
+ L6RSP -> layer 6 invalid response - protocol error
+ L7OK -> check passed on layer 7
+ L7OKC -> check conditionally passed on layer 7, for example 404 with
+ disable-on-404
+ L7TOUT -> layer 7 (HTTP/SMTP) timeout
+ L7RSP -> layer 7 invalid response - protocol error
+ L7STS -> layer 7 response error, for example HTTP 5xx
+ Notice: If a check is currently running, the last known status will be
+ reported, prefixed with "* ". e. g. "* L7OK".
+ 37. check_code [...S]: layer5-7 code, if available
+ 38. check_duration [...S]: time in ms took to finish last health check
+ 39. hrsp_1xx [.FBS]: http responses with 1xx code
+ 40. hrsp_2xx [.FBS]: http responses with 2xx code
+ 41. hrsp_3xx [.FBS]: http responses with 3xx code
+ 42. hrsp_4xx [.FBS]: http responses with 4xx code
+ 43. hrsp_5xx [.FBS]: http responses with 5xx code
+ 44. hrsp_other [.FBS]: http responses with other codes (protocol error)
+ 45. hanafail [...S]: failed health checks details
+ 46. req_rate [.F..]: HTTP requests per second over last elapsed second
+ 47. req_rate_max [.F..]: max number of HTTP requests per second observed
+ 48. req_tot [.FB.]: total number of HTTP requests received
+ 49. cli_abrt [..BS]: number of data transfers aborted by the client
+ 50. srv_abrt [..BS]: number of data transfers aborted by the server
+ (inc. in eresp)
+ 51. comp_in [.FB.]: number of HTTP response bytes fed to the compressor
+ 52. comp_out [.FB.]: number of HTTP response bytes emitted by the compressor
+ 53. comp_byp [.FB.]: number of bytes that bypassed the HTTP compressor
+ (CPU/BW limit)
+ 54. comp_rsp [.FB.]: number of HTTP responses that were compressed
+ 55. lastsess [..BS]: number of seconds since last session assigned to
+ server/backend
+ 56. last_chk [...S]: last health check contents or textual error
+ 57. last_agt [...S]: last agent check contents or textual error
+ 58. qtime [..BS]: the average queue time in ms over the 1024 last requests
+ 59. ctime [..BS]: the average connect time in ms over the 1024 last requests
+ 60. rtime [..BS]: the average response time in ms over the 1024 last requests
+ (0 for TCP)
+ 61. ttime [..BS]: the average total session time in ms over the 1024 last
+ requests
+ 62. agent_status [...S]: status of last agent check, one of:
+ UNK -> unknown
+ INI -> initializing
+ SOCKERR -> socket error
+ L4OK -> check passed on layer 4, no upper layers testing enabled
+ L4TOUT -> layer 1-4 timeout
+ L4CON -> layer 1-4 connection problem, for example
+ "Connection refused" (tcp rst) or "No route to host" (icmp)
+ L7OK -> agent reported "up"
+ L7STS -> agent reported "fail", "stop", or "down"
+ 63. agent_code [...S]: numeric code reported by agent if any (unused for now)
+ 64. agent_duration [...S]: time in ms taken to finish last check
+ 65. check_desc [...S]: short human-readable description of check_status
+ 66. agent_desc [...S]: short human-readable description of agent_status
+ 67. check_rise [...S]: server's "rise" parameter used by checks
+ 68. check_fall [...S]: server's "fall" parameter used by checks
+ 69. check_health [...S]: server's health check value between 0 and rise+fall-1
+ 70. agent_rise [...S]: agent's "rise" parameter, normally 1
+ 71. agent_fall [...S]: agent's "fall" parameter, normally 1
+ 72. agent_health [...S]: agent's health parameter, between 0 and rise+fall-1
+ 73. addr [L..S]: address:port or "unix". IPv6 has brackets around the address.
+ 74: cookie [..BS]: server's cookie value or backend's cookie name
+ 75: mode [LFBS]: proxy mode (tcp, http, health, unknown)
+ 76: algo [..B.]: load balancing algorithm
+ 77: conn_rate [.F..]: number of connections over the last elapsed second
+ 78: conn_rate_max [.F..]: highest known conn_rate
+ 79: conn_tot [.F..]: cumulative number of connections
+ 80: intercepted [.FB.]: cum. number of intercepted requests (monitor, stats)
+ 81: dcon [LF..]: requests denied by "tcp-request connection" rules
+ 82: dses [LF..]: requests denied by "tcp-request session" rules
+ 83: wrew [LFBS]: cumulative number of failed header rewriting warnings
+ 84: connect [..BS]: cumulative number of connection establishment attempts
+ 85: reuse [..BS]: cumulative number of connection reuses
+ 86: cache_lookups [.FB.]: cumulative number of cache lookups
+ 87: cache_hits [.FB.]: cumulative number of cache hits
+ 88: srv_icur [...S]: current number of idle connections available for reuse
+ 89: src_ilim [...S]: limit on the number of available idle connections
+ 90. qtime_max [..BS]: the maximum observed queue time in ms
+ 91. ctime_max [..BS]: the maximum observed connect time in ms
+ 92. rtime_max [..BS]: the maximum observed response time in ms (0 for TCP)
+ 93. ttime_max [..BS]: the maximum observed total session time in ms
+ 94. eint [LFBS]: cumulative number of internal errors
+ 95. idle_conn_cur [...S]: current number of unsafe idle connections
+ 96. safe_conn_cur [...S]: current number of safe idle connections
+ 97. used_conn_cur [...S]: current number of connections in use
+ 98. need_conn_est [...S]: estimated needed number of connections
+ 99. uweight [..BS]: total user weight (backend), server user weight (server)
+
+For all other statistics domains, the presence or the order of the fields are
+not guaranteed. In this case, the header line should always be used to parse
+the CSV data.
+
+9.2. Typed output format
+------------------------
+
+Both "show info" and "show stat" support a mode where each output value comes
+with its type and sufficient information to know how the value is supposed to
+be aggregated between processes and how it evolves.
+
+In all cases, the output consists in having a single value per line with all
+the information split into fields delimited by colons (':').
+
+The first column designates the object or metric being dumped. Its format is
+specific to the command producing this output and will not be described in this
+section. Usually it will consist in a series of identifiers and field names.
+
+The second column contains 3 characters respectively indicating the origin, the
+nature and the scope of the value being reported. The first character (the
+origin) indicates where the value was extracted from. Possible characters are :
+
+ M The value is a metric. It is valid at one instant any may change depending
+ on its nature .
+
+ S The value is a status. It represents a discrete value which by definition
+ cannot be aggregated. It may be the status of a server ("UP" or "DOWN"),
+ the PID of the process, etc.
+
+ K The value is a sorting key. It represents an identifier which may be used
+ to group some values together because it is unique among its class. All
+ internal identifiers are keys. Some names can be listed as keys if they
+ are unique (eg: a frontend name is unique). In general keys come from the
+ configuration, even though some of them may automatically be assigned. For
+ most purposes keys may be considered as equivalent to configuration.
+
+ C The value comes from the configuration. Certain configuration values make
+ sense on the output, for example a concurrent connection limit or a cookie
+ name. By definition these values are the same in all processes started
+ from the same configuration file.
+
+ P The value comes from the product itself. There are very few such values,
+ most common use is to report the product name, version and release date.
+ These elements are also the same between all processes.
+
+The second character (the nature) indicates the nature of the information
+carried by the field in order to let an aggregator decide on what operation to
+use to aggregate multiple values. Possible characters are :
+
+ A The value represents an age since a last event. This is a bit different
+ from the duration in that an age is automatically computed based on the
+ current date. A typical example is how long ago did the last session
+ happen on a server. Ages are generally aggregated by taking the minimum
+ value and do not need to be stored.
+
+ a The value represents an already averaged value. The average response times
+ and server weights are of this nature. Averages can typically be averaged
+ between processes.
+
+ C The value represents a cumulative counter. Such measures perpetually
+ increase until they wrap around. Some monitoring protocols need to tell
+ the difference between a counter and a gauge to report a different type.
+ In general counters may simply be summed since they represent events or
+ volumes. Examples of metrics of this nature are connection counts or byte
+ counts.
+
+ D The value represents a duration for a status. There are a few usages of
+ this, most of them include the time taken by the last health check and
+ the time a server has spent down. Durations are generally not summed,
+ most of the time the maximum will be retained to compute an SLA.
+
+ G The value represents a gauge. It's a measure at one instant. The memory
+ usage or the current number of active connections are of this nature.
+ Metrics of this type are typically summed during aggregation.
+
+ L The value represents a limit (generally a configured one). By nature,
+ limits are harder to aggregate since they are specific to the point where
+ they were retrieved. In certain situations they may be summed or be kept
+ separate.
+
+ M The value represents a maximum. In general it will apply to a gauge and
+ keep the highest known value. An example of such a metric could be the
+ maximum amount of concurrent connections that was encountered in the
+ product's life time. To correctly aggregate maxima, you are supposed to
+ output a range going from the maximum of all maxima and the sum of all
+ of them. There is indeed no way to know if they were encountered
+ simultaneously or not.
+
+ m The value represents a minimum. In general it will apply to a gauge and
+ keep the lowest known value. An example of such a metric could be the
+ minimum amount of free memory pools that was encountered in the product's
+ life time. To correctly aggregate minima, you are supposed to output a
+ range going from the minimum of all minima and the sum of all of them.
+ There is indeed no way to know if they were encountered simultaneously
+ or not.
+
+ N The value represents a name, so it is a string. It is used to report
+ proxy names, server names and cookie names. Names have configuration or
+ keys as their origin and are supposed to be the same among all processes.
+
+ O The value represents a free text output. Outputs from various commands,
+ returns from health checks, node descriptions are of such nature.
+
+ R The value represents an event rate. It's a measure at one instant. It is
+ quite similar to a gauge except that the recipient knows that this measure
+ moves slowly and may decide not to keep all values. An example of such a
+ metric is the measured amount of connections per second. Metrics of this
+ type are typically summed during aggregation.
+
+ T The value represents a date or time. A field emitting the current date
+ would be of this type. The method to aggregate such information is left
+ as an implementation choice. For now no field uses this type.
+
+The third character (the scope) indicates what extent the value reflects. Some
+elements may be per process while others may be per configuration or per system.
+The distinction is important to know whether or not a single value should be
+kept during aggregation or if values have to be aggregated. The following
+characters are currently supported :
+
+ C The value is valid for a whole cluster of nodes, which is the set of nodes
+ communicating over the peers protocol. An example could be the amount of
+ entries present in a stick table that is replicated with other peers. At
+ the moment no metric use this scope.
+
+ P The value is valid only for the process reporting it. Most metrics use
+ this scope.
+
+ S The value is valid for the whole service, which is the set of processes
+ started together from the same configuration file. All metrics originating
+ from the configuration use this scope. Some other metrics may use it as
+ well for some shared resources (eg: shared SSL cache statistics).
+
+ s The value is valid for the whole system, such as the system's hostname,
+ current date or resource usage. At the moment this scope is not used by
+ any metric.
+
+Consumers of these information will generally have enough of these 3 characters
+to determine how to accurately report aggregated information across multiple
+processes.
+
+After this column, the third column indicates the type of the field, among "s32"
+(signed 32-bit integer), "s64" (signed 64-bit integer), "u32" (unsigned 32-bit
+integer), "u64" (unsigned 64-bit integer), "str" (string). It is important to
+know the type before parsing the value in order to properly read it. For example
+a string containing only digits is still a string an not an integer (eg: an
+error code extracted by a check).
+
+Then the fourth column is the value itself, encoded according to its type.
+Strings are dumped as-is immediately after the colon without any leading space.
+If a string contains a colon, it will appear normally. This means that the
+output should not be exclusively split around colons or some check outputs
+or server addresses might be truncated.
+
+
+9.3. Unix Socket commands
+-------------------------
+
+The stats socket is not enabled by default. In order to enable it, it is
+necessary to add one line in the global section of the haproxy configuration.
+A second line is recommended to set a larger timeout, always appreciated when
+issuing commands by hand :
+
+ global
+ stats socket /var/run/haproxy.sock mode 600 level admin
+ stats timeout 2m
+
+It is also possible to add multiple instances of the stats socket by repeating
+the line, and make them listen to a TCP port instead of a UNIX socket. This is
+never done by default because this is dangerous, but can be handy in some
+situations :
+
+ global
+ stats socket /var/run/haproxy.sock mode 600 level admin
+ stats socket ipv4@192.168.0.1:9999 level admin
+ stats timeout 2m
+
+To access the socket, an external utility such as "socat" is required. Socat is
+a swiss-army knife to connect anything to anything. We use it to connect
+terminals to the socket, or a couple of stdin/stdout pipes to it for scripts.
+The two main syntaxes we'll use are the following :
+
+ # socat /var/run/haproxy.sock stdio
+ # socat /var/run/haproxy.sock readline
+
+The first one is used with scripts. It is possible to send the output of a
+script to haproxy, and pass haproxy's output to another script. That's useful
+for retrieving counters or attack traces for example.
+
+The second one is only useful for issuing commands by hand. It has the benefit
+that the terminal is handled by the readline library which supports line
+editing and history, which is very convenient when issuing repeated commands
+(eg: watch a counter).
+
+The socket supports two operation modes :
+ - interactive
+ - non-interactive
+
+The non-interactive mode is the default when socat connects to the socket. In
+this mode, a single line may be sent. It is processed as a whole, responses are
+sent back, and the connection closes after the end of the response. This is the
+mode that scripts and monitoring tools use. It is possible to send multiple
+commands in this mode, they need to be delimited by a semi-colon (';'). For
+example :
+
+ # echo "show info;show stat;show table" | socat /var/run/haproxy stdio
+
+If a command needs to use a semi-colon or a backslash (eg: in a value), it
+must be preceded by a backslash ('\').
+
+The interactive mode displays a prompt ('>') and waits for commands to be
+entered on the line, then processes them, and displays the prompt again to wait
+for a new command. This mode is entered via the "prompt" command which must be
+sent on the first line in non-interactive mode. The mode is a flip switch, if
+"prompt" is sent in interactive mode, it is disabled and the connection closes
+after processing the last command of the same line.
+
+For this reason, when debugging by hand, it's quite common to start with the
+"prompt" command :
+
+ # socat /var/run/haproxy readline
+ prompt
+ > show info
+ ...
+ >
+
+Optionally the process' uptime may be displayed in the prompt. In order to
+enable this, the "prompt timed" command will enable the prompt and toggle the
+displaying of the time. The uptime is displayed in format "d:hh:mm:ss" where
+"d" is the number of days, and "hh", "mm", "ss" are respectively the number
+of hours, minutes and seconds on two digits each:
+
+ # socat /var/run/haproxy readline
+ prompt timed
+
+ [23:03:34:39]> show version
+ 2.8-dev9-e5e622-18
+
+ [23:03:34:41]> quit
+
+When the timed prompt is set on the master CLI, the prompt will display the
+currently selected process' uptime, so this will work for the master, current
+worker or an older worker:
+
+ master> prompt timed
+ [0:00:00:50] master> show proc
+ (...)
+ [0:00:00:58] master> @!11955 <-- master, switch to current worker
+ [0:00:01:03] 11955> @!11942 <-- current worker, switch to older worker
+ [0:00:02:17] 11942> @ <-- older worker, switch back to master
+ [0:00:01:10] master>
+
+Since multiple commands may be issued at once, haproxy uses the empty line as a
+delimiter to mark an end of output for each command, and takes care of ensuring
+that no command can emit an empty line on output. A script can thus easily
+parse the output even when multiple commands were pipelined on a single line.
+
+Some commands may take an optional payload. To add one to a command, the first
+line needs to end with the "<<\n" pattern. The next lines will be treated as
+the payload and can contain as many lines as needed. To validate a command with
+a payload, it needs to end with an empty line.
+
+The payload pattern can be customized in order to change the way the payload
+ends. In order to end a payload with something else than an empty line, a
+customized pattern can be set between '<<' and '\n'. Only 7 characters can be
+used in addiction to '<<', otherwise this won't be considered a payload.
+For example, to use a PEM file that contains empty lines and comments:
+
+ # echo -e "set ssl cert common.pem <<%EOF%\n$(cat common.pem)\n%EOF%\n" | \
+ socat /var/run/haproxy.stat -
+
+Limitations do exist: the length of the whole buffer passed to the CLI must
+not be greater than tune.bfsize and the pattern "<<" must not be glued to the
+last word of the line.
+
+When entering a paylod while in interactive mode, the prompt will change from
+"> " to "+ ".
+
+It is important to understand that when multiple haproxy processes are started
+on the same sockets, any process may pick up the request and will output its
+own stats.
+
+The list of commands currently supported on the stats socket is provided below.
+If an unknown command is sent, haproxy displays the usage message which reminds
+all supported commands. Some commands support a more complex syntax, generally
+it will explain what part of the command is invalid when this happens.
+
+Some commands require a higher level of privilege to work. If you do not have
+enough privilege, you will get an error "Permission denied". Please check
+the "level" option of the "bind" keyword lines in the configuration manual
+for more information.
+
+abort ssl ca-file <cafile>
+ Abort and destroy a temporary CA file update transaction.
+
+ See also "set ssl ca-file" and "commit ssl ca-file".
+
+abort ssl cert <filename>
+ Abort and destroy a temporary SSL certificate update transaction.
+
+ See also "set ssl cert" and "commit ssl cert".
+
+abort ssl crl-file <crlfile>
+ Abort and destroy a temporary CRL file update transaction.
+
+ See also "set ssl crl-file" and "commit ssl crl-file".
+
+add acl [@<ver>] <acl> <pattern>
+ Add an entry into the acl <acl>. <acl> is the #<id> or the <file> returned by
+ "show acl". This command does not verify if the entry already exists. Entries
+ are added to the current version of the ACL, unless a specific version is
+ specified with "@<ver>". This version number must have preliminary been
+ allocated by "prepare acl", and it will be comprised between the versions
+ reported in "curr_ver" and "next_ver" on the output of "show acl". Entries
+ added with a specific version number will not match until a "commit acl"
+ operation is performed on them. They may however be consulted using the
+ "show acl @<ver>" command, and cleared using a "clear acl @<ver>" command.
+ This command cannot be used if the reference <acl> is a file also used with
+ a map. In this case, the "add map" command must be used instead.
+
+add map [@<ver>] <map> <key> <value>
+add map [@<ver>] <map> <payload>
+ Add an entry into the map <map> to associate the value <value> to the key
+ <key>. This command does not verify if the entry already exists. It is
+ mainly used to fill a map after a "clear" or "prepare" operation. Entries
+ are added to the current version of the ACL, unless a specific version is
+ specified with "@<ver>". This version number must have preliminary been
+ allocated by "prepare acl", and it will be comprised between the versions
+ reported in "curr_ver" and "next_ver" on the output of "show acl". Entries
+ added with a specific version number will not match until a "commit map"
+ operation is performed on them. They may however be consulted using the
+ "show map @<ver>" command, and cleared using a "clear acl @<ver>" command.
+ If the designated map is also used as an ACL, the ACL will only match the
+ <key> part and will ignore the <value> part. Using the payload syntax it is
+ possible to add multiple key/value pairs by entering them on separate lines.
+ On each new line, the first word is the key and the rest of the line is
+ considered to be the value which can even contains spaces.
+
+ Example:
+
+ # socat /tmp/sock1 -
+ prompt
+
+ > add map #-1 <<
+ + key1 value1
+ + key2 value2 with spaces
+ + key3 value3 also with spaces
+ + key4 value4
+
+ >
+
+add server <backend>/<server> [args]*
+ Instantiate a new server attached to the backend <backend>.
+
+ The <server> name must not be already used in the backend. A special
+ restriction is put on the backend which must used a dynamic load-balancing
+ algorithm. A subset of keywords from the server config file statement can be
+ used to configure the server behavior. Also note that no settings will be
+ reused from an hypothetical 'default-server' statement in the same backend.
+
+ Currently a dynamic server is statically initialized with the "none"
+ init-addr method. This means that no resolution will be undertaken if a FQDN
+ is specified as an address, even if the server creation will be validated.
+
+ To support the reload operations, it is expected that the server created via
+ the CLI is also manually inserted in the relevant haproxy configuration file.
+ A dynamic server not present in the configuration won't be restored after a
+ reload operation.
+
+ A dynamic server may use the "track" keyword to follow the check status of
+ another server from the configuration. However, it is not possible to track
+ another dynamic server. This is to ensure that the tracking chain is kept
+ consistent even in the case of dynamic servers deletion.
+
+ Use the "check" keyword to enable health-check support. Note that the
+ health-check is disabled by default and must be enabled independently from
+ the server using the "enable health" command. For agent checks, use the
+ "agent-check" keyword and the "enable agent" command. Note that in this case
+ the server may be activated via the agent depending on the status reported,
+ without an explicit "enable server" command. This also means that extra care
+ is required when removing a dynamic server with agent check. The agent should
+ be first deactivated via "disable agent" to be able to put the server in the
+ required maintenance mode before removal.
+
+ It may be possible to reach the fd limit when using a large number of dynamic
+ servers. Please refer to the "u-limit" global keyword documentation in this
+ case.
+
+ Here is the list of the currently supported keywords :
+
+ - agent-addr
+ - agent-check
+ - agent-inter
+ - agent-port
+ - agent-send
+ - allow-0rtt
+ - alpn
+ - addr
+ - backup
+ - ca-file
+ - check
+ - check-alpn
+ - check-proto
+ - check-send-proxy
+ - check-sni
+ - check-ssl
+ - check-via-socks4
+ - ciphers
+ - ciphersuites
+ - crl-file
+ - crt
+ - disabled
+ - downinter
+ - enabled
+ - error-limit
+ - fall
+ - fastinter
+ - force-sslv3/tlsv10/tlsv11/tlsv12/tlsv13
+ - id
+ - inter
+ - maxconn
+ - maxqueue
+ - minconn
+ - no-ssl-reuse
+ - no-sslv3/tlsv10/tlsv11/tlsv12/tlsv13
+ - no-tls-tickets
+ - npn
+ - observe
+ - on-error
+ - on-marked-down
+ - on-marked-up
+ - pool-low-conn
+ - pool-max-conn
+ - pool-purge-delay
+ - port
+ - proto
+ - proxy-v2-options
+ - rise
+ - send-proxy
+ - send-proxy-v2
+ - send-proxy-v2-ssl
+ - send-proxy-v2-ssl-cn
+ - slowstart
+ - sni
+ - source
+ - ssl
+ - ssl-max-ver
+ - ssl-min-ver
+ - tfo
+ - tls-tickets
+ - track
+ - usesrc
+ - verify
+ - verifyhost
+ - weight
+ - ws
+
+ Their syntax is similar to the server line from the configuration file,
+ please refer to their individual documentation for details.
+
+add ssl ca-file <cafile> <payload>
+ Add a new certificate to a ca-file. This command is useful when you reached
+ the buffer size limit on the CLI and want to add multiple certificates.
+ Instead of doing a "set" with all the certificates you are able to add each
+ certificate individually. A "set ssl ca-file" will reset the ca-file.
+
+ Example:
+ echo -e "set ssl ca-file cafile.pem <<\n$(cat rootCA.crt)\n" | \
+ socat /var/run/haproxy.stat -
+ echo -e "add ssl ca-file cafile.pem <<\n$(cat intermediate1.crt)\n" | \
+ socat /var/run/haproxy.stat -
+ echo -e "add ssl ca-file cafile.pem <<\n$(cat intermediate2.crt)\n" | \
+ socat /var/run/haproxy.stat -
+ echo "commit ssl ca-file cafile.pem" | socat /var/run/haproxy.stat -
+
+add ssl crt-list <crtlist> <certificate>
+add ssl crt-list <crtlist> <payload>
+ Add an certificate in a crt-list. It can also be used for directories since
+ directories are now loaded the same way as the crt-lists. This command allow
+ you to use a certificate name in parameter, to use SSL options or filters a
+ crt-list line must sent as a payload instead. Only one crt-list line is
+ supported in the payload. This command will load the certificate for every
+ bind lines using the crt-list. To push a new certificate to HAProxy the
+ commands "new ssl cert" and "set ssl cert" must be used.
+
+ Example:
+ $ echo "new ssl cert foobar.pem" | socat /tmp/sock1 -
+ $ echo -e "set ssl cert foobar.pem <<\n$(cat foobar.pem)\n" | socat
+ /tmp/sock1 -
+ $ echo "commit ssl cert foobar.pem" | socat /tmp/sock1 -
+ $ echo "add ssl crt-list certlist1 foobar.pem" | socat /tmp/sock1 -
+
+ $ echo -e 'add ssl crt-list certlist1 <<\nfoobar.pem [allow-0rtt] foo.bar.com
+ !test1.com\n' | socat /tmp/sock1 -
+
+clear counters
+ Clear the max values of the statistics counters in each proxy (frontend &
+ backend) and in each server. The accumulated counters are not affected. The
+ internal activity counters reported by "show activity" are also reset. This
+ can be used to get clean counters after an incident, without having to
+ restart nor to clear traffic counters. This command is restricted and can
+ only be issued on sockets configured for levels "operator" or "admin".
+
+clear counters all
+ Clear all statistics counters in each proxy (frontend & backend) and in each
+ server. This has the same effect as restarting. This command is restricted
+ and can only be issued on sockets configured for level "admin".
+
+clear acl [@<ver>] <acl>
+ Remove all entries from the acl <acl>. <acl> is the #<id> or the <file>
+ returned by "show acl". Note that if the reference <acl> is a file and is
+ shared with a map, this map will be also cleared. By default only the current
+ version of the ACL is cleared (the one being matched against). However it is
+ possible to specify another version using '@' followed by this version.
+
+clear map [@<ver>] <map>
+ Remove all entries from the map <map>. <map> is the #<id> or the <file>
+ returned by "show map". Note that if the reference <map> is a file and is
+ shared with a acl, this acl will be also cleared. By default only the current
+ version of the map is cleared (the one being matched against). However it is
+ possible to specify another version using '@' followed by this version.
+
+clear table <table> [ data.<type> <operator> <value> ] | [ key <key> ]
+ Remove entries from the stick-table <table>.
+
+ This is typically used to unblock some users complaining they have been
+ abusively denied access to a service, but this can also be used to clear some
+ stickiness entries matching a server that is going to be replaced (see "show
+ table" below for details). Note that sometimes, removal of an entry will be
+ refused because it is currently tracked by a session. Retrying a few seconds
+ later after the session ends is usual enough.
+
+ In the case where no options arguments are given all entries will be removed.
+
+ When the "data." form is used entries matching a filter applied using the
+ stored data (see "stick-table" in section 4.2) are removed. A stored data
+ type must be specified in <type>, and this data type must be stored in the
+ table otherwise an error is reported. The data is compared according to
+ <operator> with the 64-bit integer <value>. Operators are the same as with
+ the ACLs :
+
+ - eq : match entries whose data is equal to this value
+ - ne : match entries whose data is not equal to this value
+ - le : match entries whose data is less than or equal to this value
+ - ge : match entries whose data is greater than or equal to this value
+ - lt : match entries whose data is less than this value
+ - gt : match entries whose data is greater than this value
+
+ When the key form is used the entry <key> is removed. The key must be of the
+ same type as the table, which currently is limited to IPv4, IPv6, integer and
+ string.
+
+ Example :
+ $ echo "show table http_proxy" | socat stdio /tmp/sock1
+ >>> # table: http_proxy, type: ip, size:204800, used:2
+ >>> 0x80e6a4c: key=127.0.0.1 use=0 exp=3594729 gpc0=0 conn_rate(30000)=1 \
+ bytes_out_rate(60000)=187
+ >>> 0x80e6a80: key=127.0.0.2 use=0 exp=3594740 gpc0=1 conn_rate(30000)=10 \
+ bytes_out_rate(60000)=191
+
+ $ echo "clear table http_proxy key 127.0.0.1" | socat stdio /tmp/sock1
+
+ $ echo "show table http_proxy" | socat stdio /tmp/sock1
+ >>> # table: http_proxy, type: ip, size:204800, used:1
+ >>> 0x80e6a80: key=127.0.0.2 use=0 exp=3594740 gpc0=1 conn_rate(30000)=10 \
+ bytes_out_rate(60000)=191
+ $ echo "clear table http_proxy data.gpc0 eq 1" | socat stdio /tmp/sock1
+ $ echo "show table http_proxy" | socat stdio /tmp/sock1
+ >>> # table: http_proxy, type: ip, size:204800, used:1
+
+commit acl @<ver> <acl>
+ Commit all changes made to version <ver> of ACL <acl>, and deletes all past
+ versions. <acl> is the #<id> or the <file> returned by "show acl". The
+ version number must be between "curr_ver"+1 and "next_ver" as reported in
+ "show acl". The contents to be committed to the ACL can be consulted with
+ "show acl @<ver> <acl>" if desired. The specified version number has normally
+ been created with the "prepare acl" command. The replacement is atomic. It
+ consists in atomically updating the current version to the specified version,
+ which will instantly cause all entries in other versions to become invisible,
+ and all entries in the new version to become visible. It is also possible to
+ use this command to perform an atomic removal of all visible entries of an
+ ACL by calling "prepare acl" first then committing without adding any
+ entries. This command cannot be used if the reference <acl> is a file also
+ used as a map. In this case, the "commit map" command must be used instead.
+
+commit map @<ver> <map>
+ Commit all changes made to version <ver> of map <map>, and deletes all past
+ versions. <map> is the #<id> or the <file> returned by "show map". The
+ version number must be between "curr_ver"+1 and "next_ver" as reported in
+ "show map". The contents to be committed to the map can be consulted with
+ "show map @<ver> <map>" if desired. The specified version number has normally
+ been created with the "prepare map" command. The replacement is atomic. It
+ consists in atomically updating the current version to the specified version,
+ which will instantly cause all entries in other versions to become invisible,
+ and all entries in the new version to become visible. It is also possible to
+ use this command to perform an atomic removal of all visible entries of an
+ map by calling "prepare map" first then committing without adding any
+ entries.
+
+commit ssl ca-file <cafile>
+ Commit a temporary SSL CA file update transaction.
+
+ In the case of an existing CA file (in a "Used" state in "show ssl ca-file"),
+ the new CA file tree entry is inserted in the CA file tree and every instance
+ that used the CA file entry is rebuilt, along with the SSL contexts it needs.
+ All the contexts previously used by the rebuilt instances are removed.
+ Upon success, the previous CA file entry is removed from the tree.
+ Upon failure, nothing is removed or deleted, and all the original SSL
+ contexts are kept and used.
+ Once the temporary transaction is committed, it is destroyed.
+
+ In the case of a new CA file (after a "new ssl ca-file" and in a "Unused"
+ state in "show ssl ca-file"), the CA file will be inserted in the CA file
+ tree but it won't be used anywhere in HAProxy. To use it and generate SSL
+ contexts that use it, you will need to add it to a crt-list with "add ssl
+ crt-list".
+
+ See also "new ssl ca-file", "set ssl ca-file", "add ssl ca-file",
+ "abort ssl ca-file" and "add ssl crt-list".
+
+commit ssl cert <filename>
+ Commit a temporary SSL certificate update transaction.
+
+ In the case of an existing certificate (in a "Used" state in "show ssl
+ cert"), generate every SSL contextes and SNIs it need, insert them, and
+ remove the previous ones. Replace in memory the previous SSL certificates
+ everywhere the <filename> was used in the configuration. Upon failure it
+ doesn't remove or insert anything. Once the temporary transaction is
+ committed, it is destroyed.
+
+ In the case of a new certificate (after a "new ssl cert" and in a "Unused"
+ state in "show ssl cert"), the certificate will be committed in a certificate
+ storage, but it won't be used anywhere in haproxy. To use it and generate
+ its SNIs you will need to add it to a crt-list or a directory with "add ssl
+ crt-list".
+
+ See also "new ssl cert", "set ssl cert", "abort ssl cert" and
+ "add ssl crt-list".
+
+commit ssl crl-file <crlfile>
+ Commit a temporary SSL CRL file update transaction.
+
+ In the case of an existing CRL file (in a "Used" state in "show ssl
+ crl-file"), the new CRL file entry is inserted in the CA file tree (which
+ holds both the CA files and the CRL files) and every instance that used the
+ CRL file entry is rebuilt, along with the SSL contexts it needs.
+ All the contexts previously used by the rebuilt instances are removed.
+ Upon success, the previous CRL file entry is removed from the tree.
+ Upon failure, nothing is removed or deleted, and all the original SSL
+ contexts are kept and used.
+ Once the temporary transaction is committed, it is destroyed.
+
+ In the case of a new CRL file (after a "new ssl crl-file" and in a "Unused"
+ state in "show ssl crl-file"), the CRL file will be inserted in the CRL file
+ tree but it won't be used anywhere in HAProxy. To use it and generate SSL
+ contexts that use it, you will need to add it to a crt-list with "add ssl
+ crt-list".
+
+ See also "new ssl crl-file", "set ssl crl-file", "abort ssl crl-file" and
+ "add ssl crt-list".
+
+debug dev <command> [args]*
+ Call a developer-specific command. Only supported on a CLI connection running
+ in expert mode (see "expert-mode on"). Such commands are extremely dangerous
+ and not forgiving, any misuse may result in a crash of the process. They are
+ intended for experts only, and must really not be used unless told to do so.
+ Some of them are only available when haproxy is built with DEBUG_DEV defined
+ because they may have security implications. All of these commands require
+ admin privileges, and are purposely not documented to avoid encouraging their
+ use by people who are not at ease with the source code.
+
+del acl <acl> [<key>|#<ref>]
+ Delete all the acl entries from the acl <acl> corresponding to the key <key>.
+ <acl> is the #<id> or the <file> returned by "show acl". If the <ref> is used,
+ this command delete only the listed reference. The reference can be found with
+ listing the content of the acl. Note that if the reference <acl> is a file and
+ is shared with a map, the entry will be also deleted in the map.
+
+del map <map> [<key>|#<ref>]
+ Delete all the map entries from the map <map> corresponding to the key <key>.
+ <map> is the #<id> or the <file> returned by "show map". If the <ref> is used,
+ this command delete only the listed reference. The reference can be found with
+ listing the content of the map. Note that if the reference <map> is a file and
+ is shared with a acl, the entry will be also deleted in the map.
+
+del ssl ca-file <cafile>
+ Delete a CA file tree entry from HAProxy. The CA file must be unused and
+ removed from any crt-list. "show ssl ca-file" displays the status of the CA
+ files. The deletion doesn't work with a certificate referenced directly with
+ the "ca-file" or "ca-verify-file" directives in the configuration.
+
+del ssl cert <certfile>
+ Delete a certificate store from HAProxy. The certificate must be unused and
+ removed from any crt-list or directory. "show ssl cert" displays the status
+ of the certificate. The deletion doesn't work with a certificate referenced
+ directly with the "crt" directive in the configuration.
+
+del ssl crl-file <crlfile>
+ Delete a CRL file tree entry from HAProxy. The CRL file must be unused and
+ removed from any crt-list. "show ssl crl-file" displays the status of the CRL
+ files. The deletion doesn't work with a certificate referenced directly with
+ the "crl-file" directive in the configuration.
+
+del ssl crt-list <filename> <certfile[:line]>
+ Delete an entry in a crt-list. This will delete every SNIs used for this
+ entry in the frontends. If a certificate is used several time in a crt-list,
+ you will need to provide which line you want to delete. To display the line
+ numbers, use "show ssl crt-list -n <crtlist>".
+
+del server <backend>/<server>
+ Remove a server attached to the backend <backend>. All servers are eligible,
+ except servers which are referenced by other configuration elements. The
+ server must be put in maintenance mode prior to its deletion. The operation
+ is cancelled if the serveur still has active or idle connection or its
+ connection queue is not empty.
+
+disable agent <backend>/<server>
+ Mark the auxiliary agent check as temporarily stopped.
+
+ In the case where an agent check is being run as a auxiliary check, due
+ to the agent-check parameter of a server directive, new checks are only
+ initialized when the agent is in the enabled. Thus, disable agent will
+ prevent any new agent checks from begin initiated until the agent
+ re-enabled using enable agent.
+
+ When an agent is disabled the processing of an auxiliary agent check that
+ was initiated while the agent was set as enabled is as follows: All
+ results that would alter the weight, specifically "drain" or a weight
+ returned by the agent, are ignored. The processing of agent check is
+ otherwise unchanged.
+
+ The motivation for this feature is to allow the weight changing effects
+ of the agent checks to be paused to allow the weight of a server to be
+ configured using set weight without being overridden by the agent.
+
+ This command is restricted and can only be issued on sockets configured for
+ level "admin".
+
+disable dynamic-cookie backend <backend>
+ Disable the generation of dynamic cookies for the backend <backend>
+
+disable frontend <frontend>
+ Mark the frontend as temporarily stopped. This corresponds to the mode which
+ is used during a soft restart : the frontend releases the port but can be
+ enabled again if needed. This should be used with care as some non-Linux OSes
+ are unable to enable it back. This is intended to be used in environments
+ where stopping a proxy is not even imaginable but a misconfigured proxy must
+ be fixed. That way it's possible to release the port and bind it into another
+ process to restore operations. The frontend will appear with status "STOP"
+ on the stats page.
+
+ The frontend may be specified either by its name or by its numeric ID,
+ prefixed with a sharp ('#').
+
+ This command is restricted and can only be issued on sockets configured for
+ level "admin".
+
+disable health <backend>/<server>
+ Mark the primary health check as temporarily stopped. This will disable
+ sending of health checks, and the last health check result will be ignored.
+ The server will be in unchecked state and considered UP unless an auxiliary
+ agent check forces it down.
+
+ This command is restricted and can only be issued on sockets configured for
+ level "admin".
+
+disable server <backend>/<server>
+ Mark the server DOWN for maintenance. In this mode, no more checks will be
+ performed on the server until it leaves maintenance.
+ If the server is tracked by other servers, those servers will be set to DOWN
+ during the maintenance.
+
+ In the statistics page, a server DOWN for maintenance will appear with a
+ "MAINT" status, its tracking servers with the "MAINT(via)" one.
+
+ Both the backend and the server may be specified either by their name or by
+ their numeric ID, prefixed with a sharp ('#').
+
+ This command is restricted and can only be issued on sockets configured for
+ level "admin".
+
+enable agent <backend>/<server>
+ Resume auxiliary agent check that was temporarily stopped.
+
+ See "disable agent" for details of the effect of temporarily starting
+ and stopping an auxiliary agent.
+
+ This command is restricted and can only be issued on sockets configured for
+ level "admin".
+
+enable dynamic-cookie backend <backend>
+ Enable the generation of dynamic cookies for the backend <backend>.
+ A secret key must also be provided.
+
+enable frontend <frontend>
+ Resume a frontend which was temporarily stopped. It is possible that some of
+ the listening ports won't be able to bind anymore (eg: if another process
+ took them since the 'disable frontend' operation). If this happens, an error
+ is displayed. Some operating systems might not be able to resume a frontend
+ which was disabled.
+
+ The frontend may be specified either by its name or by its numeric ID,
+ prefixed with a sharp ('#').
+
+ This command is restricted and can only be issued on sockets configured for
+ level "admin".
+
+enable health <backend>/<server>
+ Resume a primary health check that was temporarily stopped. This will enable
+ sending of health checks again. Please see "disable health" for details.
+
+ This command is restricted and can only be issued on sockets configured for
+ level "admin".
+
+enable server <backend>/<server>
+ If the server was previously marked as DOWN for maintenance, this marks the
+ server UP and checks are re-enabled.
+
+ Both the backend and the server may be specified either by their name or by
+ their numeric ID, prefixed with a sharp ('#').
+
+ This command is restricted and can only be issued on sockets configured for
+ level "admin".
+
+experimental-mode [on|off]
+ Without options, this indicates whether the experimental mode is enabled or
+ disabled on the current connection. When passed "on", it turns the
+ experimental mode on for the current CLI connection only. With "off" it turns
+ it off.
+
+ The experimental mode is used to access to extra features still in
+ development. These features are currently not stable and should be used with
+ care. They may be subject to breaking changes across versions.
+
+ When used from the master CLI, this command shouldn't be prefixed, as it will
+ set the mode for any worker when connecting to its CLI.
+
+ Example:
+ echo "@1; experimental-mode on; <experimental_cmd>..." | socat /var/run/haproxy.master -
+ echo "experimental-mode on; @1 <experimental_cmd>..." | socat /var/run/haproxy.master -
+
+expert-mode [on|off]
+ This command is similar to experimental-mode but is used to toggle the
+ expert mode.
+
+ The expert mode enables displaying of expert commands that can be extremely
+ dangerous for the process and which may occasionally help developers collect
+ important information about complex bugs. Any misuse of these features will
+ likely lead to a process crash. Do not use this option without being invited
+ to do so. Note that this command is purposely not listed in the help message.
+ This command is only accessible in admin level. Changing to another level
+ automatically resets the expert mode.
+
+ When used from the master CLI, this command shouldn't be prefixed, as it will
+ set the mode for any worker when connecting to its CLI.
+
+ Example:
+ echo "@1; expert-mode on; debug dev exit 1" | socat /var/run/haproxy.master -
+ echo "expert-mode on; @1 debug dev exit 1" | socat /var/run/haproxy.master -
+
+get map <map> <value>
+get acl <acl> <value>
+ Lookup the value <value> in the map <map> or in the ACL <acl>. <map> or <acl>
+ are the #<id> or the <file> returned by "show map" or "show acl". This command
+ returns all the matching patterns associated with this map. This is useful for
+ debugging maps and ACLs. The output format is composed by one line par
+ matching type. Each line is composed by space-delimited series of words.
+
+ The first two words are:
+
+ <match method>: The match method applied. It can be "found", "bool",
+ "int", "ip", "bin", "len", "str", "beg", "sub", "dir",
+ "dom", "end" or "reg".
+
+ <match result>: The result. Can be "match" or "no-match".
+
+ The following words are returned only if the pattern matches an entry.
+
+ <index type>: "tree" or "list". The internal lookup algorithm.
+
+ <case>: "case-insensitive" or "case-sensitive". The
+ interpretation of the case.
+
+ <entry matched>: match="<entry>". Return the matched pattern. It is
+ useful with regular expressions.
+
+ The two last word are used to show the returned value and its type. With the
+ "acl" case, the pattern doesn't exist.
+
+ return=nothing: No return because there are no "map".
+ return="<value>": The value returned in the string format.
+ return=cannot-display: The value cannot be converted as string.
+
+ type="<type>": The type of the returned sample.
+
+get var <name>
+ Show the existence, type and contents of the process-wide variable 'name'.
+ Only process-wide variables are readable, so the name must begin with
+ 'proc.' otherwise no variable will be found. This command requires levels
+ "operator" or "admin".
+
+get weight <backend>/<server>
+ Report the current weight and the initial weight of server <server> in
+ backend <backend> or an error if either doesn't exist. The initial weight is
+ the one that appears in the configuration file. Both are normally equal
+ unless the current weight has been changed. Both the backend and the server
+ may be specified either by their name or by their numeric ID, prefixed with a
+ sharp ('#').
+
+help [<command>]
+ Print the list of known keywords and their basic usage, or commands matching
+ the requested one. The same help screen is also displayed for unknown
+ commands.
+
+httpclient <method> <URI>
+ Launch an HTTP client request and print the response on the CLI. Only
+ supported on a CLI connection running in expert mode (see "expert-mode on").
+ It's only meant for debugging. The httpclient is able to resolve a server
+ name in the URL using the "default" resolvers section, which is populated
+ with the DNS servers of your /etc/resolv.conf by default. However it won't be
+ able to resolve an host from /etc/hosts if you don't use a local dns daemon
+ which can resolve those.
+
+new ssl ca-file <cafile>
+ Create a new empty CA file tree entry to be filled with a set of CA
+ certificates and added to a crt-list. This command should be used in
+ combination with "set ssl ca-file", "add ssl ca-file" and "add ssl crt-list".
+
+new ssl cert <filename>
+ Create a new empty SSL certificate store to be filled with a certificate and
+ added to a directory or a crt-list. This command should be used in
+ combination with "set ssl cert" and "add ssl crt-list".
+
+new ssl crl-file <crlfile>
+ Create a new empty CRL file tree entry to be filled with a set of CRLs
+ and added to a crt-list. This command should be used in combination with "set
+ ssl crl-file" and "add ssl crt-list".
+
+prepare acl <acl>
+ Allocate a new version number in ACL <acl> for atomic replacement. <acl> is
+ the #<id> or the <file> returned by "show acl". The new version number is
+ shown in response after "New version created:". This number will then be
+ usable to prepare additions of new entries into the ACL which will then
+ atomically replace the current ones once committed. It is reported as
+ "next_ver" in "show acl". There is no impact of allocating new versions, as
+ unused versions will automatically be removed once a more recent version is
+ committed. Version numbers are unsigned 32-bit values which wrap at the end,
+ so care must be taken when comparing them in an external program. This
+ command cannot be used if the reference <acl> is a file also used as a map.
+ In this case, the "prepare map" command must be used instead.
+
+prepare map <map>
+ Allocate a new version number in map <map> for atomic replacement. <map> is
+ the #<id> or the <file> returned by "show map". The new version number is
+ shown in response after "New version created:". This number will then be
+ usable to prepare additions of new entries into the map which will then
+ atomically replace the current ones once committed. It is reported as
+ "next_ver" in "show map". There is no impact of allocating new versions, as
+ unused versions will automatically be removed once a more recent version is
+ committed. Version numbers are unsigned 32-bit values which wrap at the end,
+ so care must be taken when comparing them in an external program.
+
+prompt
+ Toggle the prompt at the beginning of the line and enter or leave interactive
+ mode. In interactive mode, the connection is not closed after a command
+ completes. Instead, the prompt will appear again, indicating the user that
+ the interpreter is waiting for a new command. The prompt consists in a right
+ angle bracket followed by a space "> ". This mode is particularly convenient
+ when one wants to periodically check information such as stats or errors.
+ It is also a good idea to enter interactive mode before issuing a "help"
+ command.
+
+quit
+ Close the connection when in interactive mode.
+
+set anon [on|off] [<key>]
+ This command enables or disables the "anonymized mode" for the current CLI
+ session, which replaces certain fields considered sensitive or confidential
+ in command outputs with hashes that preserve sufficient consistency between
+ elements to help developers identify relations between elements when trying
+ to spot bugs, but a low enough bit count (24) to make them non-reversible due
+ to the high number of possible matches. When turned on, if no key is
+ specified, the global key will be used (either specified in the configuration
+ file by "anonkey" or set via the CLI command "set anon global-key"). If no such
+ key was set, a random one will be generated. Otherwise it's possible to
+ specify the 32-bit key to be used for the current session, for example, to
+ reuse the key that was used in a previous dump to help compare outputs.
+ Developers will never need this key and it's recommended never to share it as
+ it could allow to confirm/infirm some guesses about what certain hashes could
+ be hiding.
+
+set dynamic-cookie-key backend <backend> <value>
+ Modify the secret key used to generate the dynamic persistent cookies.
+ This will break the existing sessions.
+
+set anon global-key <key>
+ This sets the global anonymizing key to <key>, which must be a 32-bit
+ integer between 0 and 4294967295 (0 disables the global key). This command
+ requires admin privilege.
+
+set map <map> [<key>|#<ref>] <value>
+ Modify the value corresponding to each key <key> in a map <map>. <map> is the
+ #<id> or <file> returned by "show map". If the <ref> is used in place of
+ <key>, only the entry pointed by <ref> is changed. The new value is <value>.
+
+set maxconn frontend <frontend> <value>
+ Dynamically change the specified frontend's maxconn setting. Any positive
+ value is allowed including zero, but setting values larger than the global
+ maxconn does not make much sense. If the limit is increased and connections
+ were pending, they will immediately be accepted. If it is lowered to a value
+ below the current number of connections, new connections acceptation will be
+ delayed until the threshold is reached. The frontend might be specified by
+ either its name or its numeric ID prefixed with a sharp ('#').
+
+set maxconn server <backend/server> <value>
+ Dynamically change the specified server's maxconn setting. Any positive
+ value is allowed including zero, but setting values larger than the global
+ maxconn does not make much sense.
+
+set maxconn global <maxconn>
+ Dynamically change the global maxconn setting within the range defined by the
+ initial global maxconn setting. If it is increased and connections were
+ pending, they will immediately be accepted. If it is lowered to a value below
+ the current number of connections, new connections acceptation will be
+ delayed until the threshold is reached. A value of zero restores the initial
+ setting.
+
+set profiling { tasks | memory } { auto | on | off }
+ Enables or disables CPU or memory profiling for the indicated subsystem. This
+ is equivalent to setting or clearing the "profiling" settings in the "global"
+ section of the configuration file. Please also see "show profiling". Note
+ that manually setting the tasks profiling to "on" automatically resets the
+ scheduler statistics, thus allows to check activity over a given interval.
+ The memory profiling is limited to certain operating systems (known to work
+ on the linux-glibc target), and requires USE_MEMORY_PROFILING to be set at
+ compile time.
+
+set rate-limit connections global <value>
+ Change the process-wide connection rate limit, which is set by the global
+ 'maxconnrate' setting. A value of zero disables the limitation. This limit
+ applies to all frontends and the change has an immediate effect. The value
+ is passed in number of connections per second.
+
+set rate-limit http-compression global <value>
+ Change the maximum input compression rate, which is set by the global
+ 'maxcomprate' setting. A value of zero disables the limitation. The value is
+ passed in number of kilobytes per second. The value is available in the "show
+ info" on the line "CompressBpsRateLim" in bytes.
+
+set rate-limit sessions global <value>
+ Change the process-wide session rate limit, which is set by the global
+ 'maxsessrate' setting. A value of zero disables the limitation. This limit
+ applies to all frontends and the change has an immediate effect. The value
+ is passed in number of sessions per second.
+
+set rate-limit ssl-sessions global <value>
+ Change the process-wide SSL session rate limit, which is set by the global
+ 'maxsslrate' setting. A value of zero disables the limitation. This limit
+ applies to all frontends and the change has an immediate effect. The value
+ is passed in number of sessions per second sent to the SSL stack. It applies
+ before the handshake in order to protect the stack against handshake abuses.
+
+set server <backend>/<server> addr <ip4 or ip6 address> [port <port>]
+ Replace the current IP address of a server by the one provided.
+ Optionally, the port can be changed using the 'port' parameter.
+ Note that changing the port also support switching from/to port mapping
+ (notation with +X or -Y), only if a port is configured for the health check.
+
+set server <backend>/<server> agent [ up | down ]
+ Force a server's agent to a new state. This can be useful to immediately
+ switch a server's state regardless of some slow agent checks for example.
+ Note that the change is propagated to tracking servers if any.
+
+set server <backend>/<server> agent-addr <addr> [port <port>]
+ Change addr for servers agent checks. Allows to migrate agent-checks to
+ another address at runtime. You can specify both IP and hostname, it will be
+ resolved.
+ Optionally, change the port agent.
+
+set server <backend>/<server> agent-port <port>
+ Change the port used for agent checks.
+
+set server <backend>/<server> agent-send <value>
+ Change agent string sent to agent check target. Allows to update string while
+ changing server address to keep those two matching.
+
+set server <backend>/<server> health [ up | stopping | down ]
+ Force a server's health to a new state. This can be useful to immediately
+ switch a server's state regardless of some slow health checks for example.
+ Note that the change is propagated to tracking servers if any.
+
+set server <backend>/<server> check-addr <ip4 | ip6> [port <port>]
+ Change the IP address used for server health checks.
+ Optionally, change the port used for server health checks.
+
+set server <backend>/<server> check-port <port>
+ Change the port used for health checking to <port>
+
+set server <backend>/<server> state [ ready | drain | maint ]
+ Force a server's administrative state to a new state. This can be useful to
+ disable load balancing and/or any traffic to a server. Setting the state to
+ "ready" puts the server in normal mode, and the command is the equivalent of
+ the "enable server" command. Setting the state to "maint" disables any traffic
+ to the server as well as any health checks. This is the equivalent of the
+ "disable server" command. Setting the mode to "drain" only removes the server
+ from load balancing but still allows it to be checked and to accept new
+ persistent connections. Changes are propagated to tracking servers if any.
+
+set server <backend>/<server> weight <weight>[%]
+ Change a server's weight to the value passed in argument. This is the exact
+ equivalent of the "set weight" command below.
+
+set server <backend>/<server> fqdn <FQDN>
+ Change a server's FQDN to the value passed in argument. This requires the
+ internal run-time DNS resolver to be configured and enabled for this server.
+
+set server <backend>/<server> ssl [ on | off ] (deprecated)
+ This option configures SSL ciphering on outgoing connections to the server.
+ When switch off, all traffic becomes plain text; health check path is not
+ changed.
+
+ This command is deprecated, create a new server dynamically with or without
+ SSL instead, using the "add server" command.
+
+set severity-output [ none | number | string ]
+ Change the severity output format of the stats socket connected to for the
+ duration of the current session.
+
+set ssl ca-file <cafile> <payload>
+ this command is part of a transaction system, the "commit ssl ca-file" and
+ "abort ssl ca-file" commands could be required.
+ if there is no on-going transaction, it will create a ca file tree entry into
+ which the certificates contained in the payload will be stored. the ca file
+ entry will not be stored in the ca file tree and will only be kept in a
+ temporary transaction. if a transaction with the same filename already exists,
+ the previous ca file entry will be deleted and replaced by the new one.
+ once the modifications are done, you have to commit the transaction through
+ a "commit ssl ca-file" call. If you want to add multiple certificates
+ separately, you can use the "add ssl ca-file" command
+
+ Example:
+ echo -e "set ssl ca-file cafile.pem <<\n$(cat rootCA.crt)\n" | \
+ socat /var/run/haproxy.stat -
+ echo "commit ssl ca-file cafile.pem" | socat /var/run/haproxy.stat -
+
+set ssl cert <filename> <payload>
+ This command is part of a transaction system, the "commit ssl cert" and
+ "abort ssl cert" commands could be required.
+ This whole transaction system works on any certificate displayed by the
+ "show ssl cert" command, so on any frontend or backend certificate.
+ If there is no on-going transaction, it will duplicate the certificate
+ <filename> in memory to a temporary transaction, then update this
+ transaction with the PEM file in the payload. If a transaction exists with
+ the same filename, it will update this transaction. It's also possible to
+ update the files linked to a certificate (.issuer, .sctl, .oscp etc.)
+ Once the modification are done, you have to "commit ssl cert" the
+ transaction.
+
+ Injection of files over the CLI must be done with caution since an empty line
+ is used to notify the end of the payload. It is recommended to inject a PEM
+ file which has been sanitized. A simple method would be to remove every empty
+ line and only leave what are in the PEM sections. It could be achieved with a
+ sed command.
+
+ Example:
+
+ # With some simple sanitizing
+ echo -e "set ssl cert localhost.pem <<\n$(sed -n '/^$/d;/-BEGIN/,/-END/p' 127.0.0.1.pem)\n" | \
+ socat /var/run/haproxy.stat -
+
+ # Complete example with commit
+ echo -e "set ssl cert localhost.pem <<\n$(cat 127.0.0.1.pem)\n" | \
+ socat /var/run/haproxy.stat -
+ echo -e \
+ "set ssl cert localhost.pem.issuer <<\n $(cat 127.0.0.1.pem.issuer)\n" | \
+ socat /var/run/haproxy.stat -
+ echo -e \
+ "set ssl cert localhost.pem.ocsp <<\n$(base64 -w 1000 127.0.0.1.pem.ocsp)\n" | \
+ socat /var/run/haproxy.stat -
+ echo "commit ssl cert localhost.pem" | socat /var/run/haproxy.stat -
+
+set ssl crl-file <crlfile> <payload>
+ This command is part of a transaction system, the "commit ssl crl-file" and
+ "abort ssl crl-file" commands could be required.
+ If there is no on-going transaction, it will create a CRL file tree entry into
+ which the Revocation Lists contained in the payload will be stored. The CRL
+ file entry will not be stored in the CRL file tree and will only be kept in a
+ temporary transaction. If a transaction with the same filename already exists,
+ the previous CRL file entry will be deleted and replaced by the new one.
+ Once the modifications are done, you have to commit the transaction through
+ a "commit ssl crl-file" call.
+
+ Example:
+ echo -e "set ssl crl-file crlfile.pem <<\n$(cat rootCRL.pem)\n" | \
+ socat /var/run/haproxy.stat -
+ echo "commit ssl crl-file crlfile.pem" | socat /var/run/haproxy.stat -
+
+set ssl ocsp-response <response | payload>
+ This command is used to update an OCSP Response for a certificate (see "crt"
+ on "bind" lines). Same controls are performed as during the initial loading of
+ the response. The <response> must be passed as a base64 encoded string of the
+ DER encoded response from the OCSP server. This command is not supported with
+ BoringSSL.
+
+ Example:
+ openssl ocsp -issuer issuer.pem -cert server.pem \
+ -host ocsp.issuer.com:80 -respout resp.der
+ echo "set ssl ocsp-response $(base64 -w 10000 resp.der)" | \
+ socat stdio /var/run/haproxy.stat
+
+ using the payload syntax:
+ echo -e "set ssl ocsp-response <<\n$(base64 resp.der)\n" | \
+ socat stdio /var/run/haproxy.stat
+
+set ssl tls-key <id> <tlskey>
+ Set the next TLS key for the <id> listener to <tlskey>. This key becomes the
+ ultimate key, while the penultimate one is used for encryption (others just
+ decrypt). The oldest TLS key present is overwritten. <id> is either a numeric
+ #<id> or <file> returned by "show tls-keys". <tlskey> is a base64 encoded 48
+ or 80 bits TLS ticket key (ex. openssl rand 80 | openssl base64 -A).
+
+set table <table> key <key> [data.<data_type> <value>]*
+ Create or update a stick-table entry in the table. If the key is not present,
+ an entry is inserted. See stick-table in section 4.2 to find all possible
+ values for <data_type>. The most likely use consists in dynamically entering
+ entries for source IP addresses, with a flag in gpc0 to dynamically block an
+ IP address or affect its quality of service. It is possible to pass multiple
+ data_types in a single call.
+
+set timeout cli <delay>
+ Change the CLI interface timeout for current connection. This can be useful
+ during long debugging sessions where the user needs to constantly inspect
+ some indicators without being disconnected. The delay is passed in seconds.
+
+set var <name> <expression>
+set var <name> expr <expression>
+set var <name> fmt <format>
+ Allows to set or overwrite the process-wide variable 'name' with the result
+ of expression <expression> or format string <format>. Only process-wide
+ variables may be used, so the name must begin with 'proc.' otherwise no
+ variable will be set. The <expression> and <format> may only involve
+ "internal" sample fetch keywords and converters even though the most likely
+ useful ones will be str('something'), int(), simple strings or references to
+ other variables. Note that the command line parser doesn't know about quotes,
+ so any space in the expression must be preceded by a backslash. This command
+ requires levels "operator" or "admin". This command is only supported on a
+ CLI connection running in experimental mode (see "experimental-mode on").
+
+set weight <backend>/<server> <weight>[%]
+ Change a server's weight to the value passed in argument. If the value ends
+ with the '%' sign, then the new weight will be relative to the initially
+ configured weight. Absolute weights are permitted between 0 and 256.
+ Relative weights must be positive with the resulting absolute weight is
+ capped at 256. Servers which are part of a farm running a static
+ load-balancing algorithm have stricter limitations because the weight
+ cannot change once set. Thus for these servers, the only accepted values
+ are 0 and 100% (or 0 and the initial weight). Changes take effect
+ immediately, though certain LB algorithms require a certain amount of
+ requests to consider changes. A typical usage of this command is to
+ disable a server during an update by setting its weight to zero, then to
+ enable it again after the update by setting it back to 100%. This command
+ is restricted and can only be issued on sockets configured for level
+ "admin". Both the backend and the server may be specified either by their
+ name or by their numeric ID, prefixed with a sharp ('#').
+
+show acl [[@<ver>] <acl>]
+ Dump info about acl converters. Without argument, the list of all available
+ acls is returned. If a <acl> is specified, its contents are dumped. <acl> is
+ the #<id> or <file>. By default the current version of the ACL is shown (the
+ version currently being matched against and reported as 'curr_ver' in the ACL
+ list). It is possible to instead dump other versions by prepending '@<ver>'
+ before the ACL's identifier. The version works as a filter and non-existing
+ versions will simply report no result. The dump format is the same as for the
+ maps even for the sample values. The data returned are not a list of
+ available ACL, but are the list of all patterns composing any ACL. Many of
+ these patterns can be shared with maps. The 'entry_cnt' value represents the
+ count of all the ACL entries, not just the active ones, which means that it
+ also includes entries currently being added.
+
+show anon
+ Display the current state of the anonymized mode (enabled or disabled) and
+ the current session's key.
+
+show backend
+ Dump the list of backends available in the running process
+
+show cli level
+ Display the CLI level of the current CLI session. The result could be
+ 'admin', 'operator' or 'user'. See also the 'operator' and 'user' commands.
+
+ Example :
+
+ $ socat /tmp/sock1 readline
+ prompt
+ > operator
+ > show cli level
+ operator
+ > user
+ > show cli level
+ user
+ > operator
+ Permission denied
+
+operator
+ Decrease the CLI level of the current CLI session to operator. It can't be
+ increased. It also drops expert and experimental mode. See also "show cli
+ level".
+
+user
+ Decrease the CLI level of the current CLI session to user. It can't be
+ increased. It also drops expert and experimental mode. See also "show cli
+ level".
+
+show activity [-1 | 0 | thread_num]
+ Reports some counters about internal events that will help developers and
+ more generally people who know haproxy well enough to narrow down the causes
+ of reports of abnormal behaviours. A typical example would be a properly
+ running process never sleeping and eating 100% of the CPU. The output fields
+ will be made of one line per metric, and per-thread counters on the same
+ line. These counters are 32-bit and will wrap during the process's life, which
+ is not a problem since calls to this command will typically be performed
+ twice. The fields are purposely not documented so that their exact meaning is
+ verified in the code where the counters are fed. These values are also reset
+ by the "clear counters" command. On multi-threaded deployments, the first
+ column will indicate the total (or average depending on the nature of the
+ metric) for all threads, and the list of all threads' values will be
+ represented between square brackets in the thread order. Optionally the
+ thread number to be dumped may be specified in argument. The special value
+ "0" will report the aggregated value (first column), and "-1", which is the
+ default, will display all the columns. Note that just like in single-threaded
+ mode, there will be no brackets when a single column is requested.
+
+show cli sockets
+ List CLI sockets. The output format is composed of 3 fields separated by
+ spaces. The first field is the socket address, it can be a unix socket, a
+ ipv4 address:port couple or a ipv6 one. Socket of other types won't be dump.
+ The second field describe the level of the socket: 'admin', 'user' or
+ 'operator'. The last field list the processes on which the socket is bound,
+ separated by commas, it can be numbers or 'all'.
+
+ Example :
+
+ $ echo 'show cli sockets' | socat stdio /tmp/sock1
+ # socket lvl processes
+ /tmp/sock1 admin all
+ 127.0.0.1:9999 user 2,3,4
+ 127.0.0.2:9969 user 2
+ [::1]:9999 operator 2
+
+show cache
+ List the configured caches and the objects stored in each cache tree.
+
+ $ echo 'show cache' | socat stdio /tmp/sock1
+ 0x7f6ac6c5b03a: foobar (shctx:0x7f6ac6c5b000, available blocks:3918)
+ 1 2 3 4
+
+ 1. pointer to the cache structure
+ 2. cache name
+ 3. pointer to the mmap area (shctx)
+ 4. number of blocks available for reuse in the shctx
+
+ 0x7f6ac6c5b4cc hash:286881868 vary:0x0011223344556677 size:39114 (39 blocks), refcount:9, expire:237
+ 1 2 3 4 5 6 7
+
+ 1. pointer to the cache entry
+ 2. first 32 bits of the hash
+ 3. secondary hash of the entry in case of vary
+ 4. size of the object in bytes
+ 5. number of blocks used for the object
+ 6. number of transactions using the entry
+ 7. expiration time, can be negative if already expired
+
+show dev
+ This command is meant to centralize some information that HAProxy developers
+ might need to better understand the causes of a given problem. It generally
+ does not provide useful information for the user, but these information allow
+ developers to eliminate certain hypothesis. The format is roughly a series of
+ sections containing indented lines with one element per line, such as the OS
+ type and version, the CPU type or the boot-time FD limits for example. Some
+ fields will be omitted to avoid repetition or output pollution when they do
+ not add value (e.g. unlimited values). More fields may appear in the future,
+ and some may change. This output is not meant for being parsed by scripts, and
+ should not be considered with a high degree of reliability, it's essentially
+ aimed at saving time for those who can read it.
+
+ Technically speaking, such information are taken as-is out of an internal
+ structure that stores them together at boot time so that they can also be
+ found in a core file after a crash. As such, it may happen that developers
+ ask for an early output on a well behaving process to compare with what is
+ found in a core dump, or to compare between several reloads (e.g. some limits
+ might change). If anonymizing is enabled, any possibly sensitive value will
+ be anonymized as well (e.g. the node name).
+
+ Example of output:
+ $ socat stdio /tmp/sock1 <<< "show dev"
+ Platform info
+ machine vendor: To be filled by O.E.M
+ machine family: Altra
+ cpu model: Impl 0x41 Arch 8 Part 0xd0c r3p1
+ virtual machine: no
+ container: no
+ OS name: Linux
+ OS release: 6.2.0-36-generic
+ OS version: #37~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Oct 9 18:01:07 UTC 2
+ OS architecture: aarch64
+ node name: 489aaf
+ Process info
+ pid: 1735846
+ boot uid: 509
+ boot gid: 1002
+ fd limit (soft): 1024
+ fd limit (hard): 1048576
+
+show env [<name>]
+ Dump one or all environment variables known by the process. Without any
+ argument, all variables are dumped. With an argument, only the specified
+ variable is dumped if it exists. Otherwise "Variable not found" is emitted.
+ Variables are dumped in the same format as they are stored or returned by the
+ "env" utility, that is, "<name>=<value>". This can be handy when debugging
+ certain configuration files making heavy use of environment variables to
+ ensure that they contain the expected values. This command is restricted and
+ can only be issued on sockets configured for levels "operator" or "admin".
+
+show errors [<iid>|<proxy>] [request|response]
+ Dump last known request and response errors collected by frontends and
+ backends. If <iid> is specified, the limit the dump to errors concerning
+ either frontend or backend whose ID is <iid>. Proxy ID "-1" will cause
+ all instances to be dumped. If a proxy name is specified instead, its ID
+ will be used as the filter. If "request" or "response" is added after the
+ proxy name or ID, only request or response errors will be dumped. This
+ command is restricted and can only be issued on sockets configured for
+ levels "operator" or "admin".
+
+ The errors which may be collected are the last request and response errors
+ caused by protocol violations, often due to invalid characters in header
+ names. The report precisely indicates what exact character violated the
+ protocol. Other important information such as the exact date the error was
+ detected, frontend and backend names, the server name (when known), the
+ internal transaction ID and the source address which has initiated the
+ session are reported too.
+
+ All characters are returned, and non-printable characters are encoded. The
+ most common ones (\t = 9, \n = 10, \r = 13 and \e = 27) are encoded as one
+ letter following a backslash. The backslash itself is encoded as '\\' to
+ avoid confusion. Other non-printable characters are encoded '\xNN' where
+ NN is the two-digits hexadecimal representation of the character's ASCII
+ code.
+
+ Lines are prefixed with the position of their first character, starting at 0
+ for the beginning of the buffer. At most one input line is printed per line,
+ and large lines will be broken into multiple consecutive output lines so that
+ the output never goes beyond 79 characters wide. It is easy to detect if a
+ line was broken, because it will not end with '\n' and the next line's offset
+ will be followed by a '+' sign, indicating it is a continuation of previous
+ line.
+
+ Example :
+ $ echo "show errors -1 response" | socat stdio /tmp/sock1
+ >>> [04/Mar/2009:15:46:56.081] backend http-in (#2) : invalid response
+ src 127.0.0.1, session #54, frontend fe-eth0 (#1), server s2 (#1)
+ response length 213 bytes, error at position 23:
+
+ 00000 HTTP/1.0 200 OK\r\n
+ 00017 header/bizarre:blah\r\n
+ 00038 Location: blah\r\n
+ 00054 Long-line: this is a very long line which should b
+ 00104+ e broken into multiple lines on the output buffer,
+ 00154+ otherwise it would be too large to print in a ter
+ 00204+ minal\r\n
+ 00211 \r\n
+
+ In the example above, we see that the backend "http-in" which has internal
+ ID 2 has blocked an invalid response from its server s2 which has internal
+ ID 1. The request was on transaction 54 (called "session" here) initiated
+ by source 127.0.0.1 and received by frontend fe-eth0 whose ID is 1. The
+ total response length was 213 bytes when the error was detected, and the
+ error was at byte 23. This is the slash ('/') in header name
+ "header/bizarre", which is not a valid HTTP character for a header name.
+
+show events [<sink>] [-w] [-n]
+ With no option, this lists all known event sinks and their types. With an
+ option, it will dump all available events in the designated sink if it is of
+ type buffer. If option "-w" is passed after the sink name, then once the end
+ of the buffer is reached, the command will wait for new events and display
+ them. It is possible to stop the operation by entering any input (which will
+ be discarded) or by closing the session. Finally, option "-n" is used to
+ directly seek to the end of the buffer, which is often convenient when
+ combined with "-w" to only report new events. For convenience, "-wn" or "-nw"
+ may be used to enable both options at once.
+
+show fd [-!plcfbsd]* [<fd>]
+ Dump the list of either all open file descriptors or just the one number <fd>
+ if specified. A set of flags may optionally be passed to restrict the dump
+ only to certain FD types or to omit certain FD types. When '-' or '!' are
+ encountered, the selection is inverted for the following characters in the
+ same argument. The inversion is reset before each argument word delimited by
+ white spaces. Selectable FD types include 'p' for pipes, 'l' for listeners,
+ 'c' for connections (any type), 'f' for frontend connections, 'b' for backend
+ connections (any type), 's' for connections to servers, 'd' for connections
+ to the "dispatch" address or the backend's transparent address. With this,
+ 'b' is a shortcut for 'sd' and 'c' for 'fb' or 'fsd'. 'c!f' is equivalent to
+ 'b' ("any connections except frontend connections" are indeed backend
+ connections). This is only aimed at developers who need to observe internal
+ states in order to debug complex issues such as abnormal CPU usages. One fd
+ is reported per lines, and for each of them, its state in the poller using
+ upper case letters for enabled flags and lower case for disabled flags, using
+ "P" for "polled", "R" for "ready", "A" for "active", the events status using
+ "H" for "hangup", "E" for "error", "O" for "output", "P" for "priority" and
+ "I" for "input", a few other flags like "N" for "new" (just added into the fd
+ cache), "U" for "updated" (received an update in the fd cache), "L" for
+ "linger_risk", "C" for "cloned", then the cached entry position, the pointer
+ to the internal owner, the pointer to the I/O callback and its name when
+ known. When the owner is a connection, the connection flags, and the target
+ are reported (frontend, proxy or server). When the owner is a listener, the
+ listener's state and its frontend are reported. There is no point in using
+ this command without a good knowledge of the internals. It's worth noting
+ that the output format may evolve over time so this output must not be parsed
+ by tools designed to be durable. Some internal structure states may look
+ suspicious to the function listing them, in this case the output line will be
+ suffixed with an exclamation mark ('!'). This may help find a starting point
+ when trying to diagnose an incident.
+
+show info [typed|json] [desc] [float]
+ Dump info about haproxy status on current process. If "typed" is passed as an
+ optional argument, field numbers, names and types are emitted as well so that
+ external monitoring products can easily retrieve, possibly aggregate, then
+ report information found in fields they don't know. Each field is dumped on
+ its own line. If "json" is passed as an optional argument then
+ information provided by "typed" output is provided in JSON format as a
+ list of JSON objects. By default, the format contains only two columns
+ delimited by a colon (':'). The left one is the field name and the right
+ one is the value. It is very important to note that in typed output
+ format, the dump for a single object is contiguous so that there is no
+ need for a consumer to store everything at once. If "float" is passed as an
+ optional argument, some fields usually emitted as integers may switch to
+ floats for higher accuracy. It is purposely unspecified which ones are
+ concerned as this might evolve over time. Using this option implies that the
+ consumer is able to process floats. The output format used is sprintf("%f").
+
+ When using the typed output format, each line is made of 4 columns delimited
+ by colons (':'). The first column is a dot-delimited series of 3 elements. The
+ first element is the numeric position of the field in the list (starting at
+ zero). This position shall not change over time, but holes are to be expected,
+ depending on build options or if some fields are deleted in the future. The
+ second element is the field name as it appears in the default "show info"
+ output. The third element is the relative process number starting at 1.
+
+ The rest of the line starting after the first colon follows the "typed output
+ format" described in the section above. In short, the second column (after the
+ first ':') indicates the origin, nature and scope of the variable. The third
+ column indicates the type of the field, among "s32", "s64", "u32", "u64" and
+ "str". Then the fourth column is the value itself, which the consumer knows
+ how to parse thanks to column 3 and how to process thanks to column 2.
+
+ Thus the overall line format in typed mode is :
+
+ <field_pos>.<field_name>.<process_num>:<tags>:<type>:<value>
+
+ When "desc" is appended to the command, one extra colon followed by a quoted
+ string is appended with a description for the metric. At the time of writing,
+ this is only supported for the "typed" and default output formats.
+
+ Example :
+
+ > show info
+ Name: HAProxy
+ Version: 1.7-dev1-de52ea-146
+ Release_date: 2016/03/11
+ Nbproc: 1
+ Process_num: 1
+ Pid: 28105
+ Uptime: 0d 0h00m04s
+ Uptime_sec: 4
+ Memmax_MB: 0
+ PoolAlloc_MB: 0
+ PoolUsed_MB: 0
+ PoolFailed: 0
+ (...)
+
+ > show info typed
+ 0.Name.1:POS:str:HAProxy
+ 1.Version.1:POS:str:1.7-dev1-de52ea-146
+ 2.Release_date.1:POS:str:2016/03/11
+ 3.Nbproc.1:CGS:u32:1
+ 4.Process_num.1:KGP:u32:1
+ 5.Pid.1:SGP:u32:28105
+ 6.Uptime.1:MDP:str:0d 0h00m08s
+ 7.Uptime_sec.1:MDP:u32:8
+ 8.Memmax_MB.1:CLP:u32:0
+ 9.PoolAlloc_MB.1:MGP:u32:0
+ 10.PoolUsed_MB.1:MGP:u32:0
+ 11.PoolFailed.1:MCP:u32:0
+ (...)
+
+ In the typed format, the presence of the process ID at the end of the
+ first column makes it very easy to visually aggregate outputs from
+ multiple processes.
+ Example :
+
+ $ ( echo show info typed | socat /var/run/haproxy.sock1 ; \
+ echo show info typed | socat /var/run/haproxy.sock2 ) | \
+ sort -t . -k 1,1n -k 2,2 -k 3,3n
+ 0.Name.1:POS:str:HAProxy
+ 0.Name.2:POS:str:HAProxy
+ 1.Version.1:POS:str:1.7-dev1-868ab3-148
+ 1.Version.2:POS:str:1.7-dev1-868ab3-148
+ 2.Release_date.1:POS:str:2016/03/11
+ 2.Release_date.2:POS:str:2016/03/11
+ 3.Nbproc.1:CGS:u32:2
+ 3.Nbproc.2:CGS:u32:2
+ 4.Process_num.1:KGP:u32:1
+ 4.Process_num.2:KGP:u32:2
+ 5.Pid.1:SGP:u32:30120
+ 5.Pid.2:SGP:u32:30121
+ 6.Uptime.1:MDP:str:0d 0h01m28s
+ 6.Uptime.2:MDP:str:0d 0h01m28s
+ (...)
+
+ The format of JSON output is described in a schema which may be output
+ using "show schema json".
+
+ The JSON output contains no extra whitespace in order to reduce the
+ volume of output. For human consumption passing the output through a
+ pretty printer may be helpful. Example :
+
+ $ echo "show info json" | socat /var/run/haproxy.sock stdio | \
+ python -m json.tool
+
+ The JSON output contains no extra whitespace in order to reduce the
+ volume of output. For human consumption passing the output through a
+ pretty printer may be helpful. Example :
+
+ $ echo "show info json" | socat /var/run/haproxy.sock stdio | \
+ python -m json.tool
+
+show libs
+ Dump the list of loaded shared dynamic libraries and object files, on systems
+ that support it. When available, for each shared object the range of virtual
+ addresses will be indicated, the size and the path to the object. This can be
+ used for example to try to estimate what library provides a function that
+ appears in a dump. Note that on many systems, addresses will change upon each
+ restart (address space randomization), so that this list would need to be
+ retrieved upon startup if it is expected to be used to analyse a core file.
+ This command may only be issued on sockets configured for levels "operator"
+ or "admin". Note that the output format may vary between operating systems,
+ architectures and even haproxy versions, and ought not to be relied on in
+ scripts.
+
+show map [[@<ver>] <map>]
+ Dump info about map converters. Without argument, the list of all available
+ maps is returned. If a <map> is specified, its contents are dumped. <map> is
+ the #<id> or <file>. By default the current version of the map is shown (the
+ version currently being matched against and reported as 'curr_ver' in the map
+ list). It is possible to instead dump other versions by prepending '@<ver>'
+ before the map's identifier. The version works as a filter and non-existing
+ versions will simply report no result. The 'entry_cnt' value represents the
+ count of all the map entries, not just the active ones, which means that it
+ also includes entries currently being added.
+
+ In the output, the first column is a unique entry identifier, which is usable
+ as a reference for operations "del map" and "set map". The second column is
+ the pattern and the third column is the sample if available. The data returned
+ are not directly a list of available maps, but are the list of all patterns
+ composing any map. Many of these patterns can be shared with ACL.
+
+show peers [dict|-] [<peers section>]
+ Dump info about the peers configured in "peers" sections. Without argument,
+ the list of the peers belonging to all the "peers" sections are listed. If
+ <peers section> is specified, only the information about the peers belonging
+ to this "peers" section are dumped. When "dict" is specified before the peers
+ section name, the entire Tx/Rx dictionary caches will also be dumped (very
+ large). Passing "-" may be required to dump a peers section called "dict".
+
+ Here are two examples of outputs where hostA, hostB and hostC peers belong to
+ "sharedlb" peers sections. Only hostA and hostB are connected. Only hostA has
+ sent data to hostB.
+
+ $ echo "show peers" | socat - /tmp/hostA
+ 0x55deb0224320: [15/Apr/2019:11:28:01] id=sharedlb state=0 flags=0x3 \
+ resync_timeout=<PAST> task_calls=45122
+ 0x55deb022b540: id=hostC(remote) addr=127.0.0.12:10002 status=CONN \
+ reconnect=4s confirm=0
+ flags=0x0
+ 0x55deb022a440: id=hostA(local) addr=127.0.0.10:10000 status=NONE \
+ reconnect=<NEVER> confirm=0
+ flags=0x0
+ 0x55deb0227d70: id=hostB(remote) addr=127.0.0.11:10001 status=ESTA
+ reconnect=2s confirm=0
+ flags=0x20000200 appctx:0x55deb028fba0 st0=7 st1=0 task_calls=14456 \
+ state=EST
+ xprt=RAW src=127.0.0.1:37257 addr=127.0.0.10:10000
+ remote_table:0x55deb0224a10 id=stkt local_id=1 remote_id=1
+ last_local_table:0x55deb0224a10 id=stkt local_id=1 remote_id=1
+ shared tables:
+ 0x55deb0224a10 local_id=1 remote_id=1 flags=0x0 remote_data=0x65
+ last_acked=0 last_pushed=3 last_get=0 teaching_origin=0 update=3
+ table:0x55deb022d6a0 id=stkt update=3 localupdate=3 \
+ commitupdate=3 syncing=0
+
+ $ echo "show peers" | socat - /tmp/hostB
+ 0x55871b5ab320: [15/Apr/2019:11:28:03] id=sharedlb state=0 flags=0x3 \
+ resync_timeout=<PAST> task_calls=3
+ 0x55871b5b2540: id=hostC(remote) addr=127.0.0.12:10002 status=CONN \
+ reconnect=3s confirm=0
+ flags=0x0
+ 0x55871b5b1440: id=hostB(local) addr=127.0.0.11:10001 status=NONE \
+ reconnect=<NEVER> confirm=0
+ flags=0x0
+ 0x55871b5aed70: id=hostA(remote) addr=127.0.0.10:10000 status=ESTA \
+ reconnect=2s confirm=0
+ flags=0x20000200 appctx:0x7fa46800ee00 st0=7 st1=0 task_calls=62356 \
+ state=EST
+ remote_table:0x55871b5ab960 id=stkt local_id=1 remote_id=1
+ last_local_table:0x55871b5ab960 id=stkt local_id=1 remote_id=1
+ shared tables:
+ 0x55871b5ab960 local_id=1 remote_id=1 flags=0x0 remote_data=0x65
+ last_acked=3 last_pushed=0 last_get=3 teaching_origin=0 update=0
+ table:0x55871b5b46a0 id=stkt update=1 localupdate=0 \
+ commitupdate=0 syncing=0
+
+show pools [byname|bysize|byusage] [match <pfx>] [<nb>]
+ Dump the status of internal memory pools. This is useful to track memory
+ usage when suspecting a memory leak for example. It does exactly the same
+ as the SIGQUIT when running in foreground except that it does not flush the
+ pools. The output is not sorted by default. If "byname" is specified, it is
+ sorted by pool name; if "bysize" is specified, it is sorted by item size in
+ reverse order; if "byusage" is specified, it is sorted by total usage in
+ reverse order, and only used entries are shown. It is also possible to limit
+ the output to the <nb> first entries (e.g. when sorting by usage). Finally,
+ if "match" followed by a prefix is specified, then only pools whose name
+ starts with this prefix will be shown. The reported total only concerns pools
+ matching the filtering criteria. Example:
+
+ $ socat - /tmp/haproxy.sock <<< "show pools match quic byusage"
+ Dumping pools usage. Use SIGQUIT to flush them.
+ - Pool quic_conn_r (65560 bytes) : 1337 allocated (87653720 bytes), ...
+ - Pool quic_crypto (1048 bytes) : 6685 allocated (7005880 bytes), ...
+ - Pool quic_conn (4056 bytes) : 1337 allocated (5422872 bytes), ...
+ - Pool quic_rxbuf (262168 bytes) : 8 allocated (2097344 bytes), ...
+ - Pool quic_conne (184 bytes) : 9359 allocated (1722056 bytes), ...
+ - Pool quic_frame (184 bytes) : 7938 allocated (1460592 bytes), ...
+ - Pool quic_tx_pac (152 bytes) : 6454 allocated (981008 bytes), ...
+ - Pool quic_tls_ke (56 bytes) : 12033 allocated (673848 bytes), ...
+ - Pool quic_rx_pac (408 bytes) : 1596 allocated (651168 bytes), ...
+ - Pool quic_tls_se (88 bytes) : 6685 allocated (588280 bytes), ...
+ - Pool quic_cstrea (88 bytes) : 4011 allocated (352968 bytes), ...
+ - Pool quic_tls_iv (24 bytes) : 12033 allocated (288792 bytes), ...
+ - Pool quic_dgram (344 bytes) : 732 allocated (251808 bytes), ...
+ - Pool quic_arng (56 bytes) : 4011 allocated (224616 bytes), ...
+ - Pool quic_conn_c (152 bytes) : 1337 allocated (203224 bytes), ...
+ Total: 15 pools, 109578176 bytes allocated, 109578176 used ...
+
+show profiling [{all | status | tasks | memory}] [byaddr|bytime|aggr|<max_lines>]*
+ Dumps the current profiling settings, one per line, as well as the command
+ needed to change them. When tasks profiling is enabled, some per-function
+ statistics collected by the scheduler will also be emitted, with a summary
+ covering the number of calls, total/avg CPU time and total/avg latency. When
+ memory profiling is enabled, some information such as the number of
+ allocations/releases and their sizes will be reported. It is possible to
+ limit the dump to only the profiling status, the tasks, or the memory
+ profiling by specifying the respective keywords; by default all profiling
+ information are dumped. It is also possible to limit the number of lines
+ of output of each category by specifying a numeric limit. If is possible to
+ request that the output is sorted by address or by total execution time
+ instead of usage, e.g. to ease comparisons between subsequent calls or to
+ check what needs to be optimized, and to aggregate task activity by called
+ function instead of seeing the details. Please note that profiling is
+ essentially aimed at developers since it gives hints about where CPU cycles
+ or memory are wasted in the code. There is nothing useful to monitor there.
+
+show resolvers [<resolvers section id>]
+ Dump statistics for the given resolvers section, or all resolvers sections
+ if no section is supplied.
+
+ For each name server, the following counters are reported:
+ sent: number of DNS requests sent to this server
+ valid: number of DNS valid responses received from this server
+ update: number of DNS responses used to update the server's IP address
+ cname: number of CNAME responses
+ cname_error: CNAME errors encountered with this server
+ any_err: number of empty response (IE: server does not support ANY type)
+ nx: non existent domain response received from this server
+ timeout: how many time this server did not answer in time
+ refused: number of requests refused by this server
+ other: any other DNS errors
+ invalid: invalid DNS response (from a protocol point of view)
+ too_big: too big response
+ outdated: number of response arrived too late (after another name server)
+
+show quic [oneline|full] [all]
+ Dump information on all active QUIC frontend connections. This command is
+ restricted and can only be issued on sockets configured for levels "operator"
+ or "admin". An optional format can be specified as first argument to control
+ the verbosity. Currently supported values are "oneline" which is the default
+ if format is unspecified or "full". By default, connections on closing or
+ draining state are not displayed. Use the extra argument "all" to include
+ them in the output.
+
+show servers conn [<backend>]
+ Dump the current and idle connections state of the servers belonging to the
+ designated backend (or all backends if none specified). A backend name or
+ identifier may be used.
+
+ The output consists in a header line showing the fields titles, then one
+ server per line with for each, the backend name and ID, server name and ID,
+ the address, port and a series or values. The number of fields varies
+ depending on thread count.
+
+ Given the threaded nature of idle connections, it's important to understand
+ that some values may change once read, and that as such, consistency within a
+ line isn't granted. This output is mostly provided as a debugging tool and is
+ not relevant to be routinely monitored nor graphed.
+
+show servers state [<backend>]
+ Dump the state of the servers found in the running configuration. A backend
+ name or identifier may be provided to limit the output to this backend only.
+
+ The dump has the following format:
+ - first line contains the format version (1 in this specification);
+ - second line contains the column headers, prefixed by a sharp ('#');
+ - third line and next ones contain data;
+ - each line starting by a sharp ('#') is considered as a comment.
+
+ Since multiple versions of the output may co-exist, below is the list of
+ fields and their order per file format version :
+ 1:
+ be_id: Backend unique id.
+ be_name: Backend label.
+ srv_id: Server unique id (in the backend).
+ srv_name: Server label.
+ srv_addr: Server IP address.
+ srv_op_state: Server operational state (UP/DOWN/...).
+ 0 = SRV_ST_STOPPED
+ The server is down.
+ 1 = SRV_ST_STARTING
+ The server is warming up (up but
+ throttled).
+ 2 = SRV_ST_RUNNING
+ The server is fully up.
+ 3 = SRV_ST_STOPPING
+ The server is up but soft-stopping
+ (eg: 404).
+ srv_admin_state: Server administrative state (MAINT/DRAIN/...).
+ The state is actually a mask of values :
+ 0x01 = SRV_ADMF_FMAINT
+ The server was explicitly forced into
+ maintenance.
+ 0x02 = SRV_ADMF_IMAINT
+ The server has inherited the maintenance
+ status from a tracked server.
+ 0x04 = SRV_ADMF_CMAINT
+ The server is in maintenance because of
+ the configuration.
+ 0x08 = SRV_ADMF_FDRAIN
+ The server was explicitly forced into
+ drain state.
+ 0x10 = SRV_ADMF_IDRAIN
+ The server has inherited the drain status
+ from a tracked server.
+ 0x20 = SRV_ADMF_RMAINT
+ The server is in maintenance because of an
+ IP address resolution failure.
+ 0x40 = SRV_ADMF_HMAINT
+ The server FQDN was set from stats socket.
+
+ srv_uweight: User visible server's weight.
+ srv_iweight: Server's initial weight.
+ srv_time_since_last_change: Time since last operational change.
+ srv_check_status: Last health check status.
+ srv_check_result: Last check result (FAILED/PASSED/...).
+ 0 = CHK_RES_UNKNOWN
+ Initialized to this by default.
+ 1 = CHK_RES_NEUTRAL
+ Valid check but no status information.
+ 2 = CHK_RES_FAILED
+ Check failed.
+ 3 = CHK_RES_PASSED
+ Check succeeded and server is fully up
+ again.
+ 4 = CHK_RES_CONDPASS
+ Check reports the server doesn't want new
+ sessions.
+ srv_check_health: Checks rise / fall current counter.
+ srv_check_state: State of the check (ENABLED/PAUSED/...).
+ The state is actually a mask of values :
+ 0x01 = CHK_ST_INPROGRESS
+ A check is currently running.
+ 0x02 = CHK_ST_CONFIGURED
+ This check is configured and may be
+ enabled.
+ 0x04 = CHK_ST_ENABLED
+ This check is currently administratively
+ enabled.
+ 0x08 = CHK_ST_PAUSED
+ Checks are paused because of maintenance
+ (health only).
+ srv_agent_state: State of the agent check (ENABLED/PAUSED/...).
+ This state uses the same mask values as
+ "srv_check_state", adding this specific one :
+ 0x10 = CHK_ST_AGENT
+ Check is an agent check (otherwise it's a
+ health check).
+ bk_f_forced_id: Flag to know if the backend ID is forced by
+ configuration.
+ srv_f_forced_id: Flag to know if the server's ID is forced by
+ configuration.
+ srv_fqdn: Server FQDN.
+ srv_port: Server port.
+ srvrecord: DNS SRV record associated to this SRV.
+ srv_use_ssl: use ssl for server connections.
+ srv_check_port: Server health check port.
+ srv_check_addr: Server health check address.
+ srv_agent_addr: Server health agent address.
+ srv_agent_port: Server health agent port.
+
+show sess
+ Dump all known active streams (formerly called "sessions"). Avoid doing this
+ on slow connections as this can be huge. This command is restricted and can
+ only be issued on sockets configured for levels "operator" or "admin". Note
+ that on machines with quickly recycled connections, it is possible that this
+ output reports less entries than really exist because it will dump all
+ existing streams up to the last one that was created before the command was
+ entered; those which die in the mean time will not appear.
+
+show sess <id> | older <age> | susp | all
+ Display a lot of internal information about the matching streams. In the
+ first form, only the stream matching the specified stream identifier will
+ be shown. This identifier is the first field at the beginning of the lines in
+ the dumps of "show sess" (it corresponds to the stream pointer). In the
+ second form, only streams older than <age> (in seconds by default) will be
+ shown. Passing "susp" instead will only report entries that are considered as
+ suspicious by the developers based on criteria that may in time or vary along
+ versions. If "all" is used instead, then all streams will be dumped. Dumping
+ many streams can produce a huge output, take a lot of time and be CPU
+ intensive, so it's always better to only dump the minimum needed. Those
+ information are useless to most users but may be used by haproxy developers
+ to troubleshoot a complex bug. The output format is intentionally not
+ documented so that it can freely evolve depending on demands. This output
+ is meant to be interpreted while checking function strm_dump_to_buffer() in
+ src/stream.c to figure the exact meaning of certain fields.
+
+show stat [domain <dns|proxy>] [{<iid>|<proxy>} <type> <sid>] [typed|json] \
+ [desc] [up|no-maint]
+ Dump statistics. The domain is used to select which statistics to print; dns
+ and proxy are available for now. By default, the CSV format is used; you can
+ activate the extended typed output format described in the section above if
+ "typed" is passed after the other arguments; or in JSON if "json" is passed
+ after the other arguments. By passing <id>, <type> and <sid>, it is possible
+ to dump only selected items :
+ - <iid> is a proxy ID, -1 to dump everything. Alternatively, a proxy name
+ <proxy> may be specified. In this case, this proxy's ID will be used as
+ the ID selector.
+ - <type> selects the type of dumpable objects : 1 for frontends, 2 for
+ backends, 4 for servers, -1 for everything. These values can be ORed,
+ for example:
+ 1 + 2 = 3 -> frontend + backend.
+ 1 + 2 + 4 = 7 -> frontend + backend + server.
+ - <sid> is a server ID, -1 to dump everything from the selected proxy.
+
+ Example :
+ $ echo "show info;show stat" | socat stdio unix-connect:/tmp/sock1
+ >>> Name: HAProxy
+ Version: 1.4-dev2-49
+ Release_date: 2009/09/23
+ Nbproc: 1
+ Process_num: 1
+ (...)
+
+ # pxname,svname,qcur,qmax,scur,smax,slim,stot,bin,bout,dreq, (...)
+ stats,FRONTEND,,,0,0,1000,0,0,0,0,0,0,,,,,OPEN,,,,,,,,,1,1,0, (...)
+ stats,BACKEND,0,0,0,0,1000,0,0,0,0,0,,0,0,0,0,UP,0,0,0,,0,250,(...)
+ (...)
+ www1,BACKEND,0,0,0,0,1000,0,0,0,0,0,,0,0,0,0,UP,1,1,0,,0,250, (...)
+
+ $
+
+ In this example, two commands have been issued at once. That way it's easy to
+ find which process the stats apply to in multi-process mode. This is not
+ needed in the typed output format as the process number is reported on each
+ line. Notice the empty line after the information output which marks the end
+ of the first block. A similar empty line appears at the end of the second
+ block (stats) so that the reader knows the output has not been truncated.
+
+ When "typed" is specified, the output format is more suitable to monitoring
+ tools because it provides numeric positions and indicates the type of each
+ output field. Each value stands on its own line with process number, element
+ number, nature, origin and scope. This same format is available via the HTTP
+ stats by passing ";typed" after the URI. It is very important to note that in
+ typed output format, the dump for a single object is contiguous so that there
+ is no need for a consumer to store everything at once.
+
+ The "up" modifier will result in listing only servers which reportedly up or
+ not checked. Those down, unresolved, or in maintenance will not be listed.
+ This is analogous to the ";up" option on the HTTP stats. Similarly, the
+ "no-maint" modifier will act like the ";no-maint" HTTP modifier and will
+ result in disabled servers not to be listed. The difference is that those
+ which are enabled but down will not be evicted.
+
+ When using the typed output format, each line is made of 4 columns delimited
+ by colons (':'). The first column is a dot-delimited series of 5 elements. The
+ first element is a letter indicating the type of the object being described.
+ At the moment the following object types are known : 'F' for a frontend, 'B'
+ for a backend, 'L' for a listener, and 'S' for a server. The second element
+ The second element is a positive integer representing the unique identifier of
+ the proxy the object belongs to. It is equivalent to the "iid" column of the
+ CSV output and matches the value in front of the optional "id" directive found
+ in the frontend or backend section. The third element is a positive integer
+ containing the unique object identifier inside the proxy, and corresponds to
+ the "sid" column of the CSV output. ID 0 is reported when dumping a frontend
+ or a backend. For a listener or a server, this corresponds to their respective
+ ID inside the proxy. The fourth element is the numeric position of the field
+ in the list (starting at zero). This position shall not change over time, but
+ holes are to be expected, depending on build options or if some fields are
+ deleted in the future. The fifth element is the field name as it appears in
+ the CSV output. The sixth element is a positive integer and is the relative
+ process number starting at 1.
+
+ The rest of the line starting after the first colon follows the "typed output
+ format" described in the section above. In short, the second column (after the
+ first ':') indicates the origin, nature and scope of the variable. The third
+ column indicates the field type, among "s32", "s64", "u32", "u64", "flt' and
+ "str". Then the fourth column is the value itself, which the consumer knows
+ how to parse thanks to column 3 and how to process thanks to column 2.
+
+ When "desc" is appended to the command, one extra colon followed by a quoted
+ string is appended with a description for the metric. At the time of writing,
+ this is only supported for the "typed" output format.
+
+ Thus the overall line format in typed mode is :
+
+ <obj>.<px_id>.<id>.<fpos>.<fname>.<process_num>:<tags>:<type>:<value>
+
+ Here's an example of typed output format :
+
+ $ echo "show stat typed" | socat stdio unix-connect:/tmp/sock1
+ F.2.0.0.pxname.1:MGP:str:private-frontend
+ F.2.0.1.svname.1:MGP:str:FRONTEND
+ F.2.0.8.bin.1:MGP:u64:0
+ F.2.0.9.bout.1:MGP:u64:0
+ F.2.0.40.hrsp_2xx.1:MGP:u64:0
+ L.2.1.0.pxname.1:MGP:str:private-frontend
+ L.2.1.1.svname.1:MGP:str:sock-1
+ L.2.1.17.status.1:MGP:str:OPEN
+ L.2.1.73.addr.1:MGP:str:0.0.0.0:8001
+ S.3.13.60.rtime.1:MCP:u32:0
+ S.3.13.61.ttime.1:MCP:u32:0
+ S.3.13.62.agent_status.1:MGP:str:L4TOUT
+ S.3.13.64.agent_duration.1:MGP:u64:2001
+ S.3.13.65.check_desc.1:MCP:str:Layer4 timeout
+ S.3.13.66.agent_desc.1:MCP:str:Layer4 timeout
+ S.3.13.67.check_rise.1:MCP:u32:2
+ S.3.13.68.check_fall.1:MCP:u32:3
+ S.3.13.69.check_health.1:SGP:u32:0
+ S.3.13.70.agent_rise.1:MaP:u32:1
+ S.3.13.71.agent_fall.1:SGP:u32:1
+ S.3.13.72.agent_health.1:SGP:u32:1
+ S.3.13.73.addr.1:MCP:str:1.255.255.255:8888
+ S.3.13.75.mode.1:MAP:str:http
+ B.3.0.0.pxname.1:MGP:str:private-backend
+ B.3.0.1.svname.1:MGP:str:BACKEND
+ B.3.0.2.qcur.1:MGP:u32:0
+ B.3.0.3.qmax.1:MGP:u32:0
+ B.3.0.4.scur.1:MGP:u32:0
+ B.3.0.5.smax.1:MGP:u32:0
+ B.3.0.6.slim.1:MGP:u32:1000
+ B.3.0.55.lastsess.1:MMP:s32:-1
+ (...)
+
+ In the typed format, the presence of the process ID at the end of the
+ first column makes it very easy to visually aggregate outputs from
+ multiple processes, as show in the example below where each line appears
+ for each process :
+
+ $ ( echo show stat typed | socat /var/run/haproxy.sock1 - ; \
+ echo show stat typed | socat /var/run/haproxy.sock2 - ) | \
+ sort -t . -k 1,1 -k 2,2n -k 3,3n -k 4,4n -k 5,5 -k 6,6n
+ B.3.0.0.pxname.1:MGP:str:private-backend
+ B.3.0.0.pxname.2:MGP:str:private-backend
+ B.3.0.1.svname.1:MGP:str:BACKEND
+ B.3.0.1.svname.2:MGP:str:BACKEND
+ B.3.0.2.qcur.1:MGP:u32:0
+ B.3.0.2.qcur.2:MGP:u32:0
+ B.3.0.3.qmax.1:MGP:u32:0
+ B.3.0.3.qmax.2:MGP:u32:0
+ B.3.0.4.scur.1:MGP:u32:0
+ B.3.0.4.scur.2:MGP:u32:0
+ B.3.0.5.smax.1:MGP:u32:0
+ B.3.0.5.smax.2:MGP:u32:0
+ B.3.0.6.slim.1:MGP:u32:1000
+ B.3.0.6.slim.2:MGP:u32:1000
+ (...)
+
+ The format of JSON output is described in a schema which may be output
+ using "show schema json".
+
+ The JSON output contains no extra whitespace in order to reduce the
+ volume of output. For human consumption passing the output through a
+ pretty printer may be helpful. Example :
+
+ $ echo "show stat json" | socat /var/run/haproxy.sock stdio | \
+ python -m json.tool
+
+ The JSON output contains no extra whitespace in order to reduce the
+ volume of output. For human consumption passing the output through a
+ pretty printer may be helpful. Example :
+
+ $ echo "show stat json" | socat /var/run/haproxy.sock stdio | \
+ python -m json.tool
+
+show ssl ca-file [<cafile>[:<index>]]
+ Display the list of CA files loaded into the process and their respective
+ certificate counts. The certificates are not used by any frontend or backend
+ until their status is "Used".
+ A "@system-ca" entry can appear in the list, it is loaded by the httpclient
+ by default. It contains the list of trusted CA of your system returned by
+ OpenSSL.
+ If a filename is prefixed by an asterisk, it is a transaction which
+ is not committed yet. If a <cafile> is specified without <index>, it will show
+ the status of the CA file ("Used"/"Unused") followed by details about all the
+ certificates contained in the CA file. The details displayed for every
+ certificate are the same as the ones displayed by a "show ssl cert" command.
+ If a <cafile> is specified followed by an <index>, it will only display the
+ details of the certificate having the specified index. Indexes start from 1.
+ If the index is invalid (too big for instance), nothing will be displayed.
+ This command can be useful to check if a CA file was properly updated.
+ You can also display the details of an ongoing transaction by prefixing the
+ filename by an asterisk.
+
+ Example :
+
+ $ echo "show ssl ca-file" | socat /var/run/haproxy.master -
+ # transaction
+ *cafile.crt - 2 certificate(s)
+ # filename
+ cafile.crt - 1 certificate(s)
+
+ $ echo "show ssl ca-file cafile.crt" | socat /var/run/haproxy.master -
+ Filename: /home/tricot/work/haproxy/reg-tests/ssl/set_cafile_ca2.crt
+ Status: Used
+
+ Certificate #1:
+ Serial: 11A4D2200DC84376E7D233CAFF39DF44BF8D1211
+ notBefore: Apr 1 07:40:53 2021 GMT
+ notAfter: Aug 17 07:40:53 2048 GMT
+ Subject Alternative Name:
+ Algorithm: RSA4096
+ SHA1 FingerPrint: A111EF0FEFCDE11D47FE3F33ADCA8435EBEA4864
+ Subject: /C=FR/ST=Some-State/O=HAProxy Technologies/CN=HAProxy Technologies CA
+ Issuer: /C=FR/ST=Some-State/O=HAProxy Technologies/CN=HAProxy Technologies CA
+
+ $ echo "show ssl ca-file *cafile.crt:2" | socat /var/run/haproxy.master -
+ Filename: */home/tricot/work/haproxy/reg-tests/ssl/set_cafile_ca2.crt
+ Status: Unused
+
+ Certificate #2:
+ Serial: 587A1CE5ED855040A0C82BF255FF300ADB7C8136
+ [...]
+
+show ssl cert [<filename>]
+ Display the list of certificates loaded into the process. They are not used
+ by any frontend or backend until their status is "Used".
+ If a filename is prefixed by an asterisk, it is a transaction which is not
+ committed yet. If a filename is specified, it will show details about the
+ certificate. This command can be useful to check if a certificate was well
+ updated. You can also display details on a transaction by prefixing the
+ filename by an asterisk.
+ This command can also be used to display the details of a certificate's OCSP
+ response by suffixing the filename with a ".ocsp" extension. It works for
+ committed certificates as well as for ongoing transactions. On a committed
+ certificate, this command is equivalent to calling "show ssl ocsp-response"
+ with the certificate's corresponding OCSP response ID.
+
+ Example :
+
+ $ echo "@1 show ssl cert" | socat /var/run/haproxy.master -
+ # transaction
+ *test.local.pem
+ # filename
+ test.local.pem
+
+ $ echo "@1 show ssl cert test.local.pem" | socat /var/run/haproxy.master -
+ Filename: test.local.pem
+ Status: Used
+ Serial: 03ECC19BA54B25E85ABA46EE561B9A10D26F
+ notBefore: Sep 13 21:20:24 2019 GMT
+ notAfter: Dec 12 21:20:24 2019 GMT
+ Issuer: /C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
+ Subject: /CN=test.local
+ Subject Alternative Name: DNS:test.local, DNS:imap.test.local
+ Algorithm: RSA2048
+ SHA1 FingerPrint: 417A11CAE25F607B24F638B4A8AEE51D1E211477
+
+ $ echo "@1 show ssl cert *test.local.pem" | socat /var/run/haproxy.master -
+ Filename: *test.local.pem
+ Status: Unused
+ [...]
+
+show ssl crl-file [<crlfile>[:<index>]]
+ Display the list of CRL files loaded into the process. They are not used
+ by any frontend or backend until their status is "Used".
+ If a filename is prefixed by an asterisk, it is a transaction which is not
+ committed yet. If a <crlfile> is specified without <index>, it will show the
+ status of the CRL file ("Used"/"Unused") followed by details about all the
+ Revocation Lists contained in the CRL file. The details displayed for every
+ list are based on the output of "openssl crl -text -noout -in <file>".
+ If a <crlfile> is specified followed by an <index>, it will only display the
+ details of the list having the specified index. Indexes start from 1.
+ If the index is invalid (too big for instance), nothing will be displayed.
+ This command can be useful to check if a CRL file was properly updated.
+ You can also display the details of an ongoing transaction by prefixing the
+ filename by an asterisk.
+
+ Example :
+
+ $ echo "show ssl crl-file" | socat /var/run/haproxy.master -
+ # transaction
+ *crlfile.pem
+ # filename
+ crlfile.pem
+
+ $ echo "show ssl crl-file crlfile.pem" | socat /var/run/haproxy.master -
+ Filename: /home/tricot/work/haproxy/reg-tests/ssl/crlfile.pem
+ Status: Used
+
+ Certificate Revocation List #1:
+ Version 1
+ Signature Algorithm: sha256WithRSAEncryption
+ Issuer: /C=FR/O=HAProxy Technologies/CN=Intermediate CA2
+ Last Update: Apr 23 14:45:39 2021 GMT
+ Next Update: Sep 8 14:45:39 2048 GMT
+ Revoked Certificates:
+ Serial Number: 1008
+ Revocation Date: Apr 23 14:45:36 2021 GMT
+
+ Certificate Revocation List #2:
+ Version 1
+ Signature Algorithm: sha256WithRSAEncryption
+ Issuer: /C=FR/O=HAProxy Technologies/CN=Root CA
+ Last Update: Apr 23 14:30:44 2021 GMT
+ Next Update: Sep 8 14:30:44 2048 GMT
+ No Revoked Certificates.
+
+show ssl crt-list [-n] [<filename>]
+ Display the list of crt-list and directories used in the HAProxy
+ configuration. If a filename is specified, dump the content of a crt-list or
+ a directory. Once dumped the output can be used as a crt-list file.
+ The '-n' option can be used to display the line number, which is useful when
+ combined with the 'del ssl crt-list' option when a entry is duplicated. The
+ output with the '-n' option is not compatible with the crt-list format and
+ not loadable by haproxy.
+
+ Example:
+ echo "show ssl crt-list -n localhost.crt-list" | socat /tmp/sock1 -
+ # localhost.crt-list
+ common.pem:1 !not.test1.com *.test1.com !localhost
+ common.pem:2
+ ecdsa.pem:3 [verify none allow-0rtt ssl-min-ver TLSv1.0 ssl-max-ver TLSv1.3] localhost !www.test1.com
+ ecdsa.pem:4 [verify none allow-0rtt ssl-min-ver TLSv1.0 ssl-max-ver TLSv1.3]
+
+show ssl ocsp-response [[text|base64] <id|path>]
+ Display the IDs of the OCSP tree entries corresponding to all the OCSP
+ responses used in HAProxy, as well as the corresponding frontend
+ certificate's path, the issuer's name and key hash and the serial number of
+ the certificate for which the OCSP response was built.
+ If a valid <id> or the <path> of a valid frontend certificate is provided,
+ display the contents of the corresponding OCSP response. When an <id> is
+ provided, it it possible to define the format in which the data is dumped.
+ The 'text' option is the default one and it allows to display detailed
+ information about the OCSP response the same way as in an "openssl ocsp
+ -respin <ocsp-response> -text" call. The 'base64' format allows to dump the
+ contents of an OCSP response in base64.
+
+ Example :
+
+ $ echo "show ssl ocsp-response" | socat /var/run/haproxy.master -
+ # Certificate IDs
+ Certificate ID key : 303b300906052b0e03021a050004148a83e0060faff709ca7e9b95522a2e81635fda0a0414f652b0e435d5ea923851508f0adbe92d85de007a0202100a
+ Certificate path : /path_to_cert/foo.pem
+ Certificate ID:
+ Issuer Name Hash: 8A83E0060FAFF709CA7E9B95522A2E81635FDA0A
+ Issuer Key Hash: F652B0E435D5EA923851508F0ADBE92D85DE007A
+ Serial Number: 100A
+
+ $ echo "show ssl ocsp-response 303b300906052b0e03021a050004148a83e0060faff709ca7e9b95522a2e81635fda0a0414f652b0e435d5ea923851508f0adbe92d85de007a0202100a" | socat /var/run/haproxy.master -
+ OCSP Response Data:
+ OCSP Response Status: successful (0x0)
+ Response Type: Basic OCSP Response
+ Version: 1 (0x0)
+ Responder Id: C = FR, O = HAProxy Technologies, CN = ocsp.haproxy.com
+ Produced At: May 27 15:43:38 2021 GMT
+ Responses:
+ Certificate ID:
+ Hash Algorithm: sha1
+ Issuer Name Hash: 8A83E0060FAFF709CA7E9B95522A2E81635FDA0A
+ Issuer Key Hash: F652B0E435D5EA923851508F0ADBE92D85DE007A
+ Serial Number: 100A
+ Cert Status: good
+ This Update: May 27 15:43:38 2021 GMT
+ Next Update: Oct 12 15:43:38 2048 GMT
+ [...]
+
+ $ echo "show ssl ocsp-response base64 /path_to_cert/foo.pem" | socat /var/run/haproxy.sock -
+ MIIB8woBAKCCAewwggHoBgkrBgEFBQcwAQEEggHZMIIB1TCBvqE[...]
+
+show ssl ocsp-updates
+ Display information about the entries concerned by the OCSP update mechanism.
+ The command will output one line per OCSP response and will contain the
+ expected update time of the response as well as the time of the last
+ successful update and counters of successful and failed updates. It will also
+ give the status of the last update (successful or not) in numerical form as
+ well as text form. See below for a full list of possible errors. The lines
+ will be sorted by ascending 'Next Update' time. The lines will also contain a
+ path to the first frontend certificate that uses the OCSP response.
+ See "show ssl ocsp-response" command and "ocsp-update" option for more
+ information on the OCSP auto update.
+
+ The update error codes and error strings can be the following:
+
+ +----+-------------------------------------+
+ | ID | message |
+ +----+-------------------------------------+
+ | 0 | "Unknown" |
+ | 1 | "Update successful" |
+ | 2 | "HTTP error" |
+ | 3 | "Missing \"ocsp-response\" header" |
+ | 4 | "OCSP response check failure" |
+ | 5 | "Error during insertion" |
+ +----+-------------------------------------+
+
+ Example :
+ $ echo "show ssl ocsp-updates" | socat /tmp/haproxy.sock -
+ OCSP Certid | Path | Next Update | Last Update | Successes | Failures | Last Update Status | Last Update Status (str)
+ 303b300906052b0e03021a050004148a83e0060faff709ca7e9b95522a2e81635fda0a0414f652b0e435d5ea923851508f0adbe92d85de007a02021015 | /path_to_cert/cert.pem | 30/Jan/2023:00:08:09 +0000 | - | 0 | 1 | 2 | HTTP error
+ 304b300906052b0e03021a0500041448dac9a0fb2bd32d4ff0de68d2f567b735f9b3c40414142eb317b75856cbae500940e61faf9d8b14c2c6021203e16a7aa01542f291237b454a627fdea9c1 | /path_to_cert/other_cert.pem | 30/Jan/2023:01:07:09 +0000 | 30/Jan/2023:00:07:09 +0000 | 1 | 0 | 1 | Update successful
+
+show ssl providers
+ Display the names of the providers loaded by OpenSSL during init. Provider
+ loading can indeed be configured via the OpenSSL configuration file and this
+ option allows to check that the right providers were loaded. This command is
+ only available with OpenSSL v3.
+
+ Example :
+ $ echo "show ssl providers" | socat /var/run/haproxy.master -
+ Loaded providers :
+ - fips
+ - base
+
+show startup-logs
+ Dump all messages emitted during the startup of the current haproxy process,
+ each startup-logs buffer is unique to its haproxy worker.
+
+ This keyword also exists on the master CLI, which shows the latest startup or
+ reload tentative.
+
+show table
+ Dump general information on all known stick-tables. Their name is returned
+ (the name of the proxy which holds them), their type (currently zero, always
+ IP), their size in maximum possible number of entries, and the number of
+ entries currently in use.
+
+ Example :
+ $ echo "show table" | socat stdio /tmp/sock1
+ >>> # table: front_pub, type: ip, size:204800, used:171454
+ >>> # table: back_rdp, type: ip, size:204800, used:0
+
+show table <name> [ data.<type> <operator> <value> [data.<type> ...]] | [ key <key> ]
+ Dump contents of stick-table <name>. In this mode, a first line of generic
+ information about the table is reported as with "show table", then all
+ entries are dumped. Since this can be quite heavy, it is possible to specify
+ a filter in order to specify what entries to display.
+
+ When the "data." form is used the filter applies to the stored data (see
+ "stick-table" in section 4.2). A stored data type must be specified
+ in <type>, and this data type must be stored in the table otherwise an
+ error is reported. The data is compared according to <operator> with the
+ 64-bit integer <value>. Operators are the same as with the ACLs :
+
+ - eq : match entries whose data is equal to this value
+ - ne : match entries whose data is not equal to this value
+ - le : match entries whose data is less than or equal to this value
+ - ge : match entries whose data is greater than or equal to this value
+ - lt : match entries whose data is less than this value
+ - gt : match entries whose data is greater than this value
+
+ In this form, you can use multiple data filter entries, up to a maximum
+ defined during build time (4 by default).
+
+ When the key form is used the entry <key> is shown. The key must be of the
+ same type as the table, which currently is limited to IPv4, IPv6, integer,
+ and string.
+
+ Example :
+ $ echo "show table http_proxy" | socat stdio /tmp/sock1
+ >>> # table: http_proxy, type: ip, size:204800, used:2
+ >>> 0x80e6a4c: key=127.0.0.1 use=0 exp=3594729 gpc0=0 conn_rate(30000)=1 \
+ bytes_out_rate(60000)=187
+ >>> 0x80e6a80: key=127.0.0.2 use=0 exp=3594740 gpc0=1 conn_rate(30000)=10 \
+ bytes_out_rate(60000)=191
+
+ $ echo "show table http_proxy data.gpc0 gt 0" | socat stdio /tmp/sock1
+ >>> # table: http_proxy, type: ip, size:204800, used:2
+ >>> 0x80e6a80: key=127.0.0.2 use=0 exp=3594740 gpc0=1 conn_rate(30000)=10 \
+ bytes_out_rate(60000)=191
+
+ $ echo "show table http_proxy data.conn_rate gt 5" | \
+ socat stdio /tmp/sock1
+ >>> # table: http_proxy, type: ip, size:204800, used:2
+ >>> 0x80e6a80: key=127.0.0.2 use=0 exp=3594740 gpc0=1 conn_rate(30000)=10 \
+ bytes_out_rate(60000)=191
+
+ $ echo "show table http_proxy key 127.0.0.2" | \
+ socat stdio /tmp/sock1
+ >>> # table: http_proxy, type: ip, size:204800, used:2
+ >>> 0x80e6a80: key=127.0.0.2 use=0 exp=3594740 gpc0=1 conn_rate(30000)=10 \
+ bytes_out_rate(60000)=191
+
+ When the data criterion applies to a dynamic value dependent on time such as
+ a bytes rate, the value is dynamically computed during the evaluation of the
+ entry in order to decide whether it has to be dumped or not. This means that
+ such a filter could match for some time then not match anymore because as
+ time goes, the average event rate drops.
+
+ It is possible to use this to extract lists of IP addresses abusing the
+ service, in order to monitor them or even blacklist them in a firewall.
+ Example :
+ $ echo "show table http_proxy data.gpc0 gt 0" \
+ | socat stdio /tmp/sock1 \
+ | fgrep 'key=' | cut -d' ' -f2 | cut -d= -f2 > abusers-ip.txt
+ ( or | awk '/key/{ print a[split($2,a,"=")]; }' )
+
+ When the stick-table is synchronized to a peers section supporting sharding,
+ the shard number will be displayed for each key (otherwise '0' is reported).
+ This allows to know which peers will receive this key.
+ Example:
+ $ echo "show table http_proxy" | socat stdio /tmp/sock1 | fgrep shard=
+ 0x7f23b0c822a8: key=10.0.0.2 use=0 exp=296398 shard=9 gpc0=0
+ 0x7f23a063f948: key=10.0.0.6 use=0 exp=296075 shard=12 gpc0=0
+ 0x7f23b03920b8: key=10.0.0.8 use=0 exp=296766 shard=1 gpc0=0
+ 0x7f23a43c09e8: key=10.0.0.12 use=0 exp=295368 shard=8 gpc0=0
+
+show tasks
+ Dumps the number of tasks currently in the run queue, with the number of
+ occurrences for each function, and their average latency when it's known
+ (for pure tasks with task profiling enabled). The dump is a snapshot of the
+ instant it's done, and there may be variations depending on what tasks are
+ left in the queue at the moment it happens, especially in mono-thread mode
+ as there's less chance that I/Os can refill the queue (unless the queue is
+ full). This command takes exclusive access to the process and can cause
+ minor but measurable latencies when issued on a highly loaded process, so
+ it must not be abused by monitoring bots.
+
+show threads
+ Dumps some internal states and structures for each thread, that may be useful
+ to help developers understand a problem. The output tries to be readable by
+ showing one block per thread. When haproxy is built with USE_THREAD_DUMP=1,
+ an advanced dump mechanism involving thread signals is used so that each
+ thread can dump its own state in turn. Without this option, the thread
+ processing the command shows all its details but the other ones are less
+ detailed. A star ('*') is displayed in front of the thread handling the
+ command. A right angle bracket ('>') may also be displayed in front of
+ threads which didn't make any progress since last invocation of this command,
+ indicating a bug in the code which must absolutely be reported. When this
+ happens between two threads it usually indicates a deadlock. If a thread is
+ alone, it's a different bug like a corrupted list. In all cases the process
+ needs is not fully functional anymore and needs to be restarted.
+
+ The output format is purposely not documented so that it can easily evolve as
+ new needs are identified, without having to maintain any form of backwards
+ compatibility, and just like with "show activity", the values are meaningless
+ without the code at hand.
+
+show tls-keys [id|*]
+ Dump all loaded TLS ticket keys references. The TLS ticket key reference ID
+ and the file from which the keys have been loaded is shown. Both of those
+ can be used to update the TLS keys using "set ssl tls-key". If an ID is
+ specified as parameter, it will dump the tickets, using * it will dump every
+ keys from every references.
+
+show schema json
+ Dump the schema used for the output of "show info json" and "show stat json".
+
+ The contains no extra whitespace in order to reduce the volume of output.
+ For human consumption passing the output through a pretty printer may be
+ helpful. Example :
+
+ $ echo "show schema json" | socat /var/run/haproxy.sock stdio | \
+ python -m json.tool
+
+ The schema follows "JSON Schema" (json-schema.org) and accordingly
+ verifiers may be used to verify the output of "show info json" and "show
+ stat json" against the schema.
+
+show trace [<source>]
+ Show the current trace status. For each source a line is displayed with a
+ single-character status indicating if the trace is stopped, waiting, or
+ running. The output sink used by the trace is indicated (or "none" if none
+ was set), as well as the number of dropped events in this sink, followed by a
+ brief description of the source. If a source name is specified, a detailed
+ list of all events supported by the source, and their status for each action
+ (report, start, pause, stop), indicated by a "+" if they are enabled, or a
+ "-" otherwise. All these events are independent and an event might trigger
+ a start without being reported and conversely.
+
+show version
+ Show the version of the current HAProxy process. This is available from
+ master and workers CLI.
+ Example:
+
+ $ echo "show version" | socat /var/run/haproxy.sock stdio
+ 2.4.9
+
+ $ echo "show version" | socat /var/run/haproxy-master.sock stdio
+ 2.5.0
+
+shutdown frontend <frontend>
+ Completely delete the specified frontend. All the ports it was bound to will
+ be released. It will not be possible to enable the frontend anymore after
+ this operation. This is intended to be used in environments where stopping a
+ proxy is not even imaginable but a misconfigured proxy must be fixed. That
+ way it's possible to release the port and bind it into another process to
+ restore operations. The frontend will not appear at all on the stats page
+ once it is terminated.
+
+ The frontend may be specified either by its name or by its numeric ID,
+ prefixed with a sharp ('#').
+
+ This command is restricted and can only be issued on sockets configured for
+ level "admin".
+
+shutdown session <id>
+ Immediately terminate the stream matching the specified stream identifier.
+ This identifier is the first field at the beginning of the lines in the dumps
+ of "show sess" (it corresponds to the stream pointer). This can be used to
+ terminate a long-running stream without waiting for a timeout or when an
+ endless transfer is ongoing. Such terminated streams are reported with a 'K'
+ flag in the logs.
+
+shutdown sessions server <backend>/<server>
+ Immediately terminate all the streams attached to the specified server. This
+ can be used to terminate long-running streams after a server is put into
+ maintenance mode, for instance. Such terminated streams are reported with a
+ 'K' flag in the logs.
+
+trace
+ The "trace" command alone lists the trace sources, their current status, and
+ their brief descriptions. It is only meant as a menu to enter next levels,
+ see other "trace" commands below.
+
+trace 0
+ Immediately stops all traces. This is made to be used as a quick solution
+ to terminate a debugging session or as an emergency action to be used in case
+ complex traces were enabled on multiple sources and impact the service.
+
+trace <source> event [ [+|-|!]<name> ]
+ Without argument, this will list all the events supported by the designated
+ source. They are prefixed with a "-" if they are not enabled, or a "+" if
+ they are enabled. It is important to note that a single trace may be labelled
+ with multiple events, and as long as any of the enabled events matches one of
+ the events labelled on the trace, the event will be passed to the trace
+ subsystem. For example, receiving an HTTP/2 frame of type HEADERS may trigger
+ a frame event and a stream event since the frame creates a new stream. If
+ either the frame event or the stream event are enabled for this source, the
+ frame will be passed to the trace framework.
+
+ With an argument, it is possible to toggle the state of each event and
+ individually enable or disable them. Two special keywords are supported,
+ "none", which matches no event, and is used to disable all events at once,
+ and "any" which matches all events, and is used to enable all events at
+ once. Other events are specific to the event source. It is possible to
+ enable one event by specifying its name, optionally prefixed with '+' for
+ better readability. It is possible to disable one event by specifying its
+ name prefixed by a '-' or a '!'.
+
+ One way to completely disable a trace source is to pass "event none", and
+ this source will instantly be totally ignored.
+
+trace <source> level [<level>]
+ Without argument, this will list all trace levels for this source, and the
+ current one will be indicated by a star ('*') prepended in front of it. With
+ an argument, this will change the trace level to the specified level. Detail
+ levels are a form of filters that are applied before reporting the events.
+ These filters are used to selectively include or exclude events depending on
+ their level of importance. For example a developer might need to know
+ precisely where in the code an HTTP header was considered invalid while the
+ end user may not even care about this header's validity at all. There are
+ currently 5 distinct levels for a trace :
+
+ user this will report information that are suitable for use by a
+ regular haproxy user who wants to observe his traffic.
+ Typically some HTTP requests and responses will be reported
+ without much detail. Most sources will set this as the
+ default level to ease operations.
+
+ proto in addition to what is reported at the "user" level, it also
+ displays protocol-level updates. This can for example be the
+ frame types or HTTP headers after decoding.
+
+ state in addition to what is reported at the "proto" level, it
+ will also display state transitions (or failed transitions)
+ which happen in parsers, so this will show attempts to
+ perform an operation while the "proto" level only shows
+ the final operation.
+
+ data in addition to what is reported at the "state" level, it
+ will also include data transfers between the various layers.
+
+ developer it reports everything available, which can include advanced
+ information such as "breaking out of this loop" that are
+ only relevant to a developer trying to understand a bug that
+ only happens once in a while in field. Function names are
+ only reported at this level.
+
+ It is highly recommended to always use the "user" level only and switch to
+ other levels only if instructed to do so by a developer. Also it is a good
+ idea to first configure the events before switching to higher levels, as it
+ may save from dumping many lines if no filter is applied.
+
+trace <source> lock [criterion]
+ Without argument, this will list all the criteria supported by this source
+ for lock-on processing, and display the current choice by a star ('*') in
+ front of it. Lock-on means that the source will focus on the first matching
+ event and only stick to the criterion which triggered this event, and ignore
+ all other ones until the trace stops. This allows for example to take a trace
+ on a single connection or on a single stream. The following criteria are
+ supported by some traces, though not necessarily all, since some of them
+ might not be available to the source :
+
+ backend lock on the backend that started the trace
+ connection lock on the connection that started the trace
+ frontend lock on the frontend that started the trace
+ listener lock on the listener that started the trace
+ nothing do not lock on anything
+ server lock on the server that started the trace
+ session lock on the session that started the trace
+ thread lock on the thread that started the trace
+
+ In addition to this, each source may provide up to 4 specific criteria such
+ as internal states or connection IDs. For example in HTTP/2 it is possible
+ to lock on the H2 stream and ignore other streams once a strace starts.
+
+ When a criterion is passed in argument, this one is used instead of the
+ other ones and any existing tracking is immediately terminated so that it can
+ restart with the new criterion. The special keyword "nothing" is supported by
+ all sources to permanently disable tracking.
+
+trace <source> { pause | start | stop } [ [+|-|!]event]
+ Without argument, this will list the events enabled to automatically pause,
+ start, or stop a trace for this source. These events are specific to each
+ trace source. With an argument, this will either enable the event for the
+ specified action (if optionally prefixed by a '+') or disable it (if
+ prefixed by a '-' or '!'). The special keyword "now" is not an event and
+ requests to take the action immediately. The keywords "none" and "any" are
+ supported just like in "trace event".
+
+ The 3 supported actions are respectively "pause", "start" and "stop". The
+ "pause" action enumerates events which will cause a running trace to stop and
+ wait for a new start event to restart it. The "start" action enumerates the
+ events which switch the trace into the waiting mode until one of the start
+ events appears. And the "stop" action enumerates the events which definitely
+ stop the trace until it is manually enabled again. In practice it makes sense
+ to manually start a trace using "start now" without caring about events, and
+ to stop it using "stop now". In order to capture more subtle event sequences,
+ setting "start" to a normal event (like receiving an HTTP request) and "stop"
+ to a very rare event like emitting a certain error, will ensure that the last
+ captured events will match the desired criteria. And the pause event is
+ useful to detect the end of a sequence, disable the lock-on and wait for
+ another opportunity to take a capture. In this case it can make sense to
+ enable lock-on to spot only one specific criterion (e.g. a stream), and have
+ "start" set to anything that starts this criterion (e.g. all events which
+ create a stream), "stop" set to the expected anomaly, and "pause" to anything
+ that ends that criterion (e.g. any end of stream event). In this case the
+ trace log will contain complete sequences of perfectly clean series affecting
+ a single object, until the last sequence containing everything from the
+ beginning to the anomaly.
+
+trace <source> sink [<sink>]
+ Without argument, this will list all event sinks available for this source,
+ and the currently configured one will have a star ('*') prepended in front
+ of it. Sink "none" is always available and means that all events are simply
+ dropped, though their processing is not ignored (e.g. lock-on does occur).
+ Other sinks are available depending on configuration and build options, but
+ typically "stdout" and "stderr" will be usable in debug mode, and in-memory
+ ring buffers should be available as well. When a name is specified, the sink
+ instantly changes for the specified source. Events are not changed during a
+ sink change. In the worst case some may be lost if an invalid sink is used
+ (or "none"), but operations do continue to a different destination.
+
+trace <source> verbosity [<level>]
+ Without argument, this will list all verbosity levels for this source, and the
+ current one will be indicated by a star ('*') prepended in front of it. With
+ an argument, this will change the verbosity level to the specified one.
+
+ Verbosity levels indicate how far the trace decoder should go to provide
+ detailed information. It depends on the trace source, since some sources will
+ not even provide a specific decoder. Level "quiet" is always available and
+ disables any decoding. It can be useful when trying to figure what's
+ happening before trying to understand the details, since it will have a very
+ low impact on performance and trace size. When no verbosity levels are
+ declared by a source, level "default" is available and will cause a decoder
+ to be called when specified in the traces. It is an opportunistic decoding.
+ When the source declares some verbosity levels, these ones are listed with
+ a description of what they correspond to. In this case the trace decoder
+ provided by the source will be as accurate as possible based on the
+ information available at the trace point. The first level above "quiet" is
+ set by default.
+
+update ssl ocsp-response <certfile>
+ Create an OCSP request for the specified <certfile> and send it to the OCSP
+ responder whose URI should be specified in the "Authority Information Access"
+ section of the certificate. Only the first URI is taken into account. The
+ OCSP response that we should receive in return is then checked and inserted
+ in the local OCSP response tree. This command will only work for certificates
+ that already had a stored OCSP response, either because it was provided
+ during init or if it was previously set through the "set ssl cert" or "set
+ ssl ocsp-response" commands.
+ If the received OCSP response is valid and was properly inserted into the
+ local tree, its contents will be displayed on the standard output. The format
+ is the same as the one described in "show ssl ocsp-response".
+
+
+9.4. Master CLI
+---------------
+
+The master CLI is a socket bound to the master process in master-worker mode.
+This CLI gives access to the unix socket commands in every running or leaving
+processes and allows a basic supervision of those processes.
+
+The master CLI is configurable only from the haproxy program arguments with
+the -S option. This option also takes bind options separated by commas.
+
+Example:
+
+ # haproxy -W -S 127.0.0.1:1234 -f test1.cfg
+ # haproxy -Ws -S /tmp/master-socket,uid,1000,gid,1000,mode,600 -f test1.cfg
+ # haproxy -W -S /tmp/master-socket,level,user -f test1.cfg
+
+
+9.4.1. Master CLI commands
+--------------------------
+
+@<[!]pid>
+ The master CLI uses a special prefix notation to access the multiple
+ processes. This notation is easily identifiable as it begins by a @.
+
+ A @ prefix can be followed by a relative process number or by an exclamation
+ point and a PID. (e.g. @1 or @!1271). A @ alone could be use to specify the
+ master. Leaving processes are only accessible with the PID as relative process
+ number are only usable with the current processes.
+
+ Examples:
+
+ $ socat /var/run/haproxy-master.sock readline
+ prompt
+ master> @1 show info; @2 show info
+ [...]
+ Process_num: 1
+ Pid: 1271
+ [...]
+ Process_num: 2
+ Pid: 1272
+ [...]
+ master>
+
+ $ echo '@!1271 show info; @!1272 show info' | socat /var/run/haproxy-master.sock -
+ [...]
+
+ A prefix could be use as a command, which will send every next commands to
+ the specified process.
+
+ Examples:
+
+ $ socat /var/run/haproxy-master.sock readline
+ prompt
+ master> @1
+ 1271> show info
+ [...]
+ 1271> show stat
+ [...]
+ 1271> @
+ master>
+
+ $ echo '@1; show info; show stat; @2; show info; show stat' | socat /var/run/haproxy-master.sock -
+ [...]
+
+expert-mode [on|off]
+ This command activates the "expert-mode" for every worker accessed from the
+ master CLI. Combined with "mcli-debug-mode" it also activates the command on
+ the master. Display the flag "e" in the master CLI prompt.
+
+ See also "expert-mode" in Section 9.3 and "mcli-debug-mode" in 9.4.1.
+
+experimental-mode [on|off]
+ This command activates the "experimental-mode" for every worker accessed from
+ the master CLI. Combined with "mcli-debug-mode" it also activates the command on
+ the master. Display the flag "x" in the master CLI prompt.
+
+ See also "experimental-mode" in Section 9.3 and "mcli-debug-mode" in 9.4.1.
+
+hard-reload
+ This command does the same as the "reload" command over the master CLI with
+ the exception that it does a hard-stop (-st) instead of a stop-stop (-sf) of
+ the previous process. This means the previous process does not wait to
+ achieve anything before exiting, so all connections will be closed.
+
+ See also the "reload" command.
+
+mcli-debug-mode [on|off]
+ This keyword allows a special mode in the master CLI which enables every
+ keywords that were meant for a worker CLI on the master CLI, allowing to debug
+ the master process. Once activated, you list the new available keywords with
+ "help". Combined with "experimental-mode" or "expert-mode" it enables even
+ more keywords. Display the flag "d" in the master CLI prompt.
+
+prompt
+ When the prompt is enabled (via the "prompt" command), the context the CLI is
+ working on is displayed in the prompt. The master is identified by the "master"
+ string, and other processes are identified with their PID. In case the last
+ reload failed, the master prompt will be changed to "master[ReloadFailed]>" so
+ that it becomes visible that the process is still running on the previous
+ configuration and that the new configuration is not operational.
+
+ The prompt of the master CLI is able to display several flags which are the
+ enable modes. "d" for mcli-debug-mode, "e" for expert-mode, "x" for
+ experimental-mode.
+
+ Example:
+ $ socat /var/run/haproxy-master.sock -
+ prompt
+ master> expert-mode on
+ master(e)> experimental-mode on
+ master(xe)> mcli-debug-mode on
+ master(xed)> @1
+ 95191(xed)>
+
+reload
+ You can also reload the HAProxy master process with the "reload" command which
+ does the same as a `kill -USR2` on the master process, provided that the user
+ has at least "operator" or "admin" privileges.
+
+ This command allows you to perform a synchronous reload, the command will
+ return a reload status, once the reload was performed. Be careful with the
+ timeout if a tool is used to parse it, it is only returned once the
+ configuration is parsed and the new worker is forked. The "socat" command uses
+ a timeout of 0.5s by default so it will quits before showing the message if
+ the reload is too long. "ncat" does not have a timeout by default.
+ When compiled with USE_SHM_OPEN=1, the reload command is also able to dump
+ the startup-logs of the master.
+
+ Example:
+
+ $ echo "reload" | socat -t300 /var/run/haproxy-master.sock stdin
+ Success=1
+ --
+ [NOTICE] (482713) : haproxy version is 2.7-dev7-4827fb-69
+ [NOTICE] (482713) : path to executable is ./haproxy
+ [WARNING] (482713) : config : 'http-request' rules ignored for proxy 'frt1' as they require HTTP mode.
+ [NOTICE] (482713) : New worker (482720) forked
+ [NOTICE] (482713) : Loading success.
+
+ $ echo "reload" | socat -t300 /var/run/haproxy-master.sock stdin
+ Success=0
+ --
+ [NOTICE] (482886) : haproxy version is 2.7-dev7-4827fb-69
+ [NOTICE] (482886) : path to executable is ./haproxy
+ [ALERT] (482886) : config : parsing [test3.cfg:1]: unknown keyword 'Aglobal' out of section.
+ [ALERT] (482886) : config : Fatal errors found in configuration.
+ [WARNING] (482886) : Loading failure!
+
+ $
+
+ The reload command is the last executed on the master CLI, every other
+ command after it are ignored. Once the reload command returns its status, it
+ will close the connection to the CLI.
+
+ Note that a reload will close all connections to the master CLI.
+ See also the "hard-reload" command.
+
+show proc
+ The master CLI introduces a 'show proc' command to surpervise the
+ processe.
+
+ Example:
+
+ $ echo 'show proc' | socat /var/run/haproxy-master.sock -
+ #<PID> <type> <reloads> <uptime> <version>
+ 1162 master 5 [failed: 0] 0d00h02m07s 2.5-dev13
+ # workers
+ 1271 worker 1 0d00h00m00s 2.5-dev13
+ # old workers
+ 1233 worker 3 0d00h00m43s 2.0-dev3-6019f6-289
+ # programs
+ 1244 foo 0 0d00h00m00s -
+ 1255 bar 0 0d00h00m00s -
+
+ In this example, the master has been reloaded 5 times but one of the old
+ worker is still running and survived 3 reloads. You could access the CLI of
+ this worker to understand what's going on.
+
+show startup-logs
+ HAProxy needs to be compiled with USE_SHM_OPEN=1 to be used correctly on the
+ master CLI or all messages won't be visible.
+
+ Like its counterpart on the stats socket, this command is able to show the
+ startup messages of HAProxy. However it does not dump the startup messages
+ of the current worker, but the startup messages of the latest startup or
+ reload, which means it is able to dump the parsing messages of a failed
+ reload.
+
+ Those messages are also dumped with the "reload" command.
+
+10. Tricks for easier configuration management
+----------------------------------------------
+
+It is very common that two HAProxy nodes constituting a cluster share exactly
+the same configuration modulo a few addresses. Instead of having to maintain a
+duplicate configuration for each node, which will inevitably diverge, it is
+possible to include environment variables in the configuration. Thus multiple
+configuration may share the exact same file with only a few different system
+wide environment variables. This started in version 1.5 where only addresses
+were allowed to include environment variables, and 1.6 goes further by
+supporting environment variables everywhere. The syntax is the same as in the
+UNIX shell, a variable starts with a dollar sign ('$'), followed by an opening
+curly brace ('{'), then the variable name followed by the closing brace ('}').
+Except for addresses, environment variables are only interpreted in arguments
+surrounded with double quotes (this was necessary not to break existing setups
+using regular expressions involving the dollar symbol).
+
+Environment variables also make it convenient to write configurations which are
+expected to work on various sites where only the address changes. It can also
+permit to remove passwords from some configs. Example below where the the file
+"site1.env" file is sourced by the init script upon startup :
+
+ $ cat site1.env
+ LISTEN=192.168.1.1
+ CACHE_PFX=192.168.11
+ SERVER_PFX=192.168.22
+ LOGGER=192.168.33.1
+ STATSLP=admin:pa$$w0rd
+ ABUSERS=/etc/haproxy/abuse.lst
+ TIMEOUT=10s
+
+ $ cat haproxy.cfg
+ global
+ log "${LOGGER}:514" local0
+
+ defaults
+ mode http
+ timeout client "${TIMEOUT}"
+ timeout server "${TIMEOUT}"
+ timeout connect 5s
+
+ frontend public
+ bind "${LISTEN}:80"
+ http-request reject if { src -f "${ABUSERS}" }
+ stats uri /stats
+ stats auth "${STATSLP}"
+ use_backend cache if { path_end .jpg .css .ico }
+ default_backend server
+
+ backend cache
+ server cache1 "${CACHE_PFX}.1:18080" check
+ server cache2 "${CACHE_PFX}.2:18080" check
+
+ backend server
+ server cache1 "${SERVER_PFX}.1:8080" check
+ server cache2 "${SERVER_PFX}.2:8080" check
+
+
+11. Well-known traps to avoid
+-----------------------------
+
+Once in a while, someone reports that after a system reboot, the haproxy
+service wasn't started, and that once they start it by hand it works. Most
+often, these people are running a clustered IP address mechanism such as
+keepalived, to assign the service IP address to the master node only, and while
+it used to work when they used to bind haproxy to address 0.0.0.0, it stopped
+working after they bound it to the virtual IP address. What happens here is
+that when the service starts, the virtual IP address is not yet owned by the
+local node, so when HAProxy wants to bind to it, the system rejects this
+because it is not a local IP address. The fix doesn't consist in delaying the
+haproxy service startup (since it wouldn't stand a restart), but instead to
+properly configure the system to allow binding to non-local addresses. This is
+easily done on Linux by setting the net.ipv4.ip_nonlocal_bind sysctl to 1. This
+is also needed in order to transparently intercept the IP traffic that passes
+through HAProxy for a specific target address.
+
+Multi-process configurations involving source port ranges may apparently seem
+to work but they will cause some random failures under high loads because more
+than one process may try to use the same source port to connect to the same
+server, which is not possible. The system will report an error and a retry will
+happen, picking another port. A high value in the "retries" parameter may hide
+the effect to a certain extent but this also comes with increased CPU usage and
+processing time. Logs will also report a certain number of retries. For this
+reason, port ranges should be avoided in multi-process configurations.
+
+Since HAProxy uses SO_REUSEPORT and supports having multiple independent
+processes bound to the same IP:port, during troubleshooting it can happen that
+an old process was not stopped before a new one was started. This provides
+absurd test results which tend to indicate that any change to the configuration
+is ignored. The reason is that in fact even the new process is restarted with a
+new configuration, the old one also gets some incoming connections and
+processes them, returning unexpected results. When in doubt, just stop the new
+process and try again. If it still works, it very likely means that an old
+process remains alive and has to be stopped. Linux's "netstat -lntp" is of good
+help here.
+
+When adding entries to an ACL from the command line (eg: when blacklisting a
+source address), it is important to keep in mind that these entries are not
+synchronized to the file and that if someone reloads the configuration, these
+updates will be lost. While this is often the desired effect (for blacklisting)
+it may not necessarily match expectations when the change was made as a fix for
+a problem. See the "add acl" action of the CLI interface.
+
+
+12. Debugging and performance issues
+------------------------------------
+
+When HAProxy is started with the "-d" option, it will stay in the foreground
+and will print one line per event, such as an incoming connection, the end of a
+connection, and for each request or response header line seen. This debug
+output is emitted before the contents are processed, so they don't consider the
+local modifications. The main use is to show the request and response without
+having to run a network sniffer. The output is less readable when multiple
+connections are handled in parallel, though the "debug2ansi" and "debug2html"
+scripts found in the examples/ directory definitely help here by coloring the
+output.
+
+If a request or response is rejected because HAProxy finds it is malformed, the
+best thing to do is to connect to the CLI and issue "show errors", which will
+report the last captured faulty request and response for each frontend and
+backend, with all the necessary information to indicate precisely the first
+character of the input stream that was rejected. This is sometimes needed to
+prove to customers or to developers that a bug is present in their code. In
+this case it is often possible to relax the checks (but still keep the
+captures) using "option accept-invalid-http-request" or its equivalent for
+responses coming from the server "option accept-invalid-http-response". Please
+see the configuration manual for more details.
+
+Example :
+
+ > show errors
+ Total events captured on [13/Oct/2015:13:43:47.169] : 1
+
+ [13/Oct/2015:13:43:40.918] frontend HAProxyLocalStats (#2): invalid request
+ backend <NONE> (#-1), server <NONE> (#-1), event #0
+ src 127.0.0.1:51981, session #0, session flags 0x00000080
+ HTTP msg state 26, msg flags 0x00000000, tx flags 0x00000000
+ HTTP chunk len 0 bytes, HTTP body len 0 bytes
+ buffer flags 0x00808002, out 0 bytes, total 31 bytes
+ pending 31 bytes, wrapping at 8040, error at position 13:
+
+ 00000 GET /invalid request HTTP/1.1\r\n
+
+
+The output of "show info" on the CLI provides a number of useful information
+regarding the maximum connection rate ever reached, maximum SSL key rate ever
+reached, and in general all information which can help to explain temporary
+issues regarding CPU or memory usage. Example :
+
+ > show info
+ Name: HAProxy
+ Version: 1.6-dev7-e32d18-17
+ Release_date: 2015/10/12
+ Nbproc: 1
+ Process_num: 1
+ Pid: 7949
+ Uptime: 0d 0h02m39s
+ Uptime_sec: 159
+ Memmax_MB: 0
+ Ulimit-n: 120032
+ Maxsock: 120032
+ Maxconn: 60000
+ Hard_maxconn: 60000
+ CurrConns: 0
+ CumConns: 3
+ CumReq: 3
+ MaxSslConns: 0
+ CurrSslConns: 0
+ CumSslConns: 0
+ Maxpipes: 0
+ PipesUsed: 0
+ PipesFree: 0
+ ConnRate: 0
+ ConnRateLimit: 0
+ MaxConnRate: 1
+ SessRate: 0
+ SessRateLimit: 0
+ MaxSessRate: 1
+ SslRate: 0
+ SslRateLimit: 0
+ MaxSslRate: 0
+ SslFrontendKeyRate: 0
+ SslFrontendMaxKeyRate: 0
+ SslFrontendSessionReuse_pct: 0
+ SslBackendKeyRate: 0
+ SslBackendMaxKeyRate: 0
+ SslCacheLookups: 0
+ SslCacheMisses: 0
+ CompressBpsIn: 0
+ CompressBpsOut: 0
+ CompressBpsRateLim: 0
+ ZlibMemUsage: 0
+ MaxZlibMemUsage: 0
+ Tasks: 5
+ Run_queue: 1
+ Idle_pct: 100
+ node: wtap
+ description:
+
+When an issue seems to randomly appear on a new version of HAProxy (eg: every
+second request is aborted, occasional crash, etc), it is worth trying to enable
+memory poisoning so that each call to malloc() is immediately followed by the
+filling of the memory area with a configurable byte. By default this byte is
+0x50 (ASCII for 'P'), but any other byte can be used, including zero (which
+will have the same effect as a calloc() and which may make issues disappear).
+Memory poisoning is enabled on the command line using the "-dM" option. It
+slightly hurts performance and is not recommended for use in production. If
+an issue happens all the time with it or never happens when poisoning uses
+byte zero, it clearly means you've found a bug and you definitely need to
+report it. Otherwise if there's no clear change, the problem it is not related.
+
+When debugging some latency issues, it is important to use both strace and
+tcpdump on the local machine, and another tcpdump on the remote system. The
+reason for this is that there are delays everywhere in the processing chain and
+it is important to know which one is causing latency to know where to act. In
+practice, the local tcpdump will indicate when the input data come in. Strace
+will indicate when haproxy receives these data (using recv/recvfrom). Warning,
+openssl uses read()/write() syscalls instead of recv()/send(). Strace will also
+show when haproxy sends the data, and tcpdump will show when the system sends
+these data to the interface. Then the external tcpdump will show when the data
+sent are really received (since the local one only shows when the packets are
+queued). The benefit of sniffing on the local system is that strace and tcpdump
+will use the same reference clock. Strace should be used with "-tts200" to get
+complete timestamps and report large enough chunks of data to read them.
+Tcpdump should be used with "-nvvttSs0" to report full packets, real sequence
+numbers and complete timestamps.
+
+In practice, received data are almost always immediately received by haproxy
+(unless the machine has a saturated CPU or these data are invalid and not
+delivered). If these data are received but not sent, it generally is because
+the output buffer is saturated (ie: recipient doesn't consume the data fast
+enough). This can be confirmed by seeing that the polling doesn't notify of
+the ability to write on the output file descriptor for some time (it's often
+easier to spot in the strace output when the data finally leave and then roll
+back to see when the write event was notified). It generally matches an ACK
+received from the recipient, and detected by tcpdump. Once the data are sent,
+they may spend some time in the system doing nothing. Here again, the TCP
+congestion window may be limited and not allow these data to leave, waiting for
+an ACK to open the window. If the traffic is idle and the data take 40 ms or
+200 ms to leave, it's a different issue (which is not an issue), it's the fact
+that the Nagle algorithm prevents empty packets from leaving immediately, in
+hope that they will be merged with subsequent data. HAProxy automatically
+disables Nagle in pure TCP mode and in tunnels. However it definitely remains
+enabled when forwarding an HTTP body (and this contributes to the performance
+improvement there by reducing the number of packets). Some HTTP non-compliant
+applications may be sensitive to the latency when delivering incomplete HTTP
+response messages. In this case you will have to enable "option http-no-delay"
+to disable Nagle in order to work around their design, keeping in mind that any
+other proxy in the chain may similarly be impacted. If tcpdump reports that data
+leave immediately but the other end doesn't see them quickly, it can mean there
+is a congested WAN link, a congested LAN with flow control enabled and
+preventing the data from leaving, or more commonly that HAProxy is in fact
+running in a virtual machine and that for whatever reason the hypervisor has
+decided that the data didn't need to be sent immediately. In virtualized
+environments, latency issues are almost always caused by the virtualization
+layer, so in order to save time, it's worth first comparing tcpdump in the VM
+and on the external components. Any difference has to be credited to the
+hypervisor and its accompanying drivers.
+
+When some TCP SACK segments are seen in tcpdump traces (using -vv), it always
+means that the side sending them has got the proof of a lost packet. While not
+seeing them doesn't mean there are no losses, seeing them definitely means the
+network is lossy. Losses are normal on a network, but at a rate where SACKs are
+not noticeable at the naked eye. If they appear a lot in the traces, it is
+worth investigating exactly what happens and where the packets are lost. HTTP
+doesn't cope well with TCP losses, which introduce huge latencies.
+
+The "netstat -i" command will report statistics per interface. An interface
+where the Rx-Ovr counter grows indicates that the system doesn't have enough
+resources to receive all incoming packets and that they're lost before being
+processed by the network driver. Rx-Drp indicates that some received packets
+were lost in the network stack because the application doesn't process them
+fast enough. This can happen during some attacks as well. Tx-Drp means that
+the output queues were full and packets had to be dropped. When using TCP it
+should be very rare, but will possibly indicate a saturated outgoing link.
+
+
+13. Security considerations
+---------------------------
+
+HAProxy is designed to run with very limited privileges. The standard way to
+use it is to isolate it into a chroot jail and to drop its privileges to a
+non-root user without any permissions inside this jail so that if any future
+vulnerability were to be discovered, its compromise would not affect the rest
+of the system.
+
+In order to perform a chroot, it first needs to be started as a root user. It is
+pointless to build hand-made chroots to start the process there, these ones are
+painful to build, are never properly maintained and always contain way more
+bugs than the main file-system. And in case of compromise, the intruder can use
+the purposely built file-system. Unfortunately many administrators confuse
+"start as root" and "run as root", resulting in the uid change to be done prior
+to starting haproxy, and reducing the effective security restrictions.
+
+HAProxy will need to be started as root in order to :
+ - adjust the file descriptor limits
+ - bind to privileged port numbers
+ - bind to a specific network interface
+ - transparently listen to a foreign address
+ - isolate itself inside the chroot jail
+ - drop to another non-privileged UID
+
+HAProxy may require to be run as root in order to :
+ - bind to an interface for outgoing connections
+ - bind to privileged source ports for outgoing connections
+ - transparently bind to a foreign address for outgoing connections
+
+Most users will never need the "run as root" case. But the "start as root"
+covers most usages.
+
+A safe configuration will have :
+
+ - a chroot statement pointing to an empty location without any access
+ permissions. This can be prepared this way on the UNIX command line :
+
+ # mkdir /var/empty && chmod 0 /var/empty || echo "Failed"
+
+ and referenced like this in the HAProxy configuration's global section :
+
+ chroot /var/empty
+
+ - both a uid/user and gid/group statements in the global section :
+
+ user haproxy
+ group haproxy
+
+ - a stats socket whose mode, uid and gid are set to match the user and/or
+ group allowed to access the CLI so that nobody may access it :
+
+ stats socket /var/run/haproxy.stat uid hatop gid hatop mode 600
+
diff --git a/doc/netscaler-client-ip-insertion-protocol.txt b/doc/netscaler-client-ip-insertion-protocol.txt
new file mode 100644
index 0000000..dc64327
--- /dev/null
+++ b/doc/netscaler-client-ip-insertion-protocol.txt
@@ -0,0 +1,55 @@
+When NetScaler application switch is used as L3+ switch, information
+regarding the original IP and TCP headers are lost as a new TCP
+connection is created between the NetScaler and the backend server.
+
+NetScaler provides a feature to insert in the TCP data the original data
+that can then be consumed by the backend server.
+
+Specifications and documentations from NetScaler:
+ https://support.citrix.com/article/CTX205670
+ https://www.citrix.com/blogs/2016/04/25/how-to-enable-client-ip-in-tcpip-option-of-netscaler/
+
+When CIP is enabled on the NetScaler, then a TCP packet is inserted just after
+the TCP handshake. Two versions of the CIP extension exist.
+
+Legacy (NetScaler < 10.5)
+
+ - CIP magic number : 4 bytes
+ Both sender and receiver have to agree on a magic number so that
+ they both handle the incoming data as a NetScaler Client IP insertion
+ packet.
+
+ - Header length : 4 bytes
+ Defines the length on the remaining data.
+
+ - IP header : >= 20 bytes if IPv4, 40 bytes if IPv6
+ Contains the header of the last IP packet sent by the client during TCP
+ handshake.
+
+ - TCP header : >= 20 bytes
+ Contains the header of the last TCP packet sent by the client during TCP
+ handshake.
+
+Standard (NetScaler >= 10.5)
+
+ - CIP magic number : 4 bytes
+ Both sender and receiver have to agree on a magic number so that
+ they both handle the incoming data as a NetScaler Client IP insertion
+ packet.
+
+ - CIP length : 4 bytes
+ Defines the total length on the CIP header.
+
+ - CIP type: 2 bytes
+ Always set to 1.
+
+ - Header length : 2 bytes
+ Defines the length on the remaining data.
+
+ - IP header : >= 20 bytes if IPv4, 40 bytes if IPv6
+ Contains the header of the last IP packet sent by the client during TCP
+ handshake.
+
+ - TCP header : >= 20 bytes
+ Contains the header of the last TCP packet sent by the client during TCP
+ handshake.
diff --git a/doc/network-namespaces.txt b/doc/network-namespaces.txt
new file mode 100644
index 0000000..9448f43
--- /dev/null
+++ b/doc/network-namespaces.txt
@@ -0,0 +1,106 @@
+Linux network namespace support for HAProxy
+===========================================
+
+HAProxy supports proxying between Linux network namespaces. This
+feature can be used, for example, in a multi-tenant networking
+environment to proxy between different networks. HAProxy can also act
+as a front-end proxy for non namespace-aware services.
+
+The proxy protocol has been extended to support transferring the
+namespace information, so the originating namespace information can be
+kept. This is useful when chaining multiple proxies and services.
+
+To enable Linux namespace support, compile HAProxy with the `USE_NS=1`
+make option.
+
+
+## Setting up namespaces on Linux
+
+To create network namespaces, use the 'ip netns' command. See the
+manual page ip-netns(8) for details.
+
+Make sure that the file descriptors representing the network namespace
+are located under `/var/run/netns`.
+
+For example, you can create a network namespace and assign one of the
+networking interfaces to the new namespace:
+
+```
+$ ip netns add netns1
+$ ip link set eth7 netns netns1
+```
+
+
+## Listing namespaces in the configuration file
+
+HAProxy uses namespaces explicitly listed in its configuration file.
+If you are not using namespace information received through the proxy
+protocol, this usually means that you must specify namespaces for
+listeners and servers in the configuration file with the 'namespace'
+keyword.
+
+However, if you're using the namespace information received through
+the proxy protocol to determine the namespace of servers (see
+'namespace * below'), you have to explicitly list all allowed
+namespaces in the namespace_list section of your configuration file:
+
+```
+namespace_list
+ namespace netns1
+ namespace netns2
+```
+
+
+## Namespace information flow
+
+The haproxy process always runs in the namespace it was started on.
+This is the default namespace.
+
+The bind addresses of listeners can have their namespace specified in
+the configuration file. Unless specified, sockets associated with
+listener bind addresses are created in the default namespace. For
+example, this creates a listener in the netns2 namespace:
+
+```
+frontend f_example
+ bind 192.168.1.1:80 namespace netns2
+ default_backend http
+```
+
+Each client connection is associated with its source namespace. By
+default, this is the namespace of the bind socket it arrived on, but
+can be overridden by information received through the proxy protocol.
+Proxy protocol v2 supports transferring namespace information, so if
+it is enabled for the listener, it can override the associated
+namespace of the connection.
+
+Servers can have their namespaces specified in the configuration file
+with the 'namespace' keyword:
+
+```
+backend b_example
+ server s1 192.168.1.100:80 namespace netns2
+```
+
+If no namespace is set for a server, it is assumed that it is in the
+default namespace. When specified, outbound sockets to the server are
+created in the network namespace configured. To create the outbound
+(server) connection in the namespace associated with the client, use
+the '*' namespace. This is especially useful when using the
+destination address and namespace received from the proxy protocol.
+
+```
+frontend f_example
+ bind 192.168.1.1:9990 accept-proxy
+ default_backend b_example
+
+backend b_example
+ mode tcp
+ source 0.0.0.0 usesrc clientip
+ server snodes * namespace *
+```
+
+If HAProxy is configured to send proxy protocol v2 headers to the
+server, the outgoing header will always contain the namespace
+associated with the client connection, not the namespace configured
+for the server.
diff --git a/doc/peers-v2.0.txt b/doc/peers-v2.0.txt
new file mode 100644
index 0000000..711c949
--- /dev/null
+++ b/doc/peers-v2.0.txt
@@ -0,0 +1,294 @@
+ HAProxy's peers v2.0 protocol 08/18/2016
+
+Author: Emeric Brun ebrun@haproxy.com
+
+
+I) Encoded Integer and Bitfield.
+
+
+ 0 <= X < 240 : 1 byte (7.875 bits) [ XXXX XXXX ]
+ 240 <= X < 2288 : 2 bytes (11 bits) [ 1111 XXXX ] [ 0XXX XXXX ]
+ 2288 <= X < 264432 : 3 bytes (18 bits) [ 1111 XXXX ] [ 1XXX XXXX ] [ 0XXX XXXX ]
+ 264432 <= X < 33818864 : 4 bytes (25 bits) [ 1111 XXXX ] [ 1XXX XXXX ]*2 [ 0XXX XXXX ]
+ 33818864 <= X < 4328786160 : 5 bytes (32 bits) [ 1111 XXXX ] [ 1XXX XXXX ]*3 [ 0XXX XXXX ]
+
+
+
+
+I) Handshake
+
+Each peer try to connect to each others, and each peer is listening
+for a connect from other peers.
+
+
+Client Server
+ Hello Message
+ ------------------------>
+
+ Status Message
+ <------------------------
+
+1) Hello Message
+
+Hello message is composed of 3 lines:
+
+<protocol> <version>
+<remotepeerid>
+<localpeerid> <processpid> <relativepid>
+
+protocol: current value is "HAProxyS"
+version: current value is "2.0"
+remotepeerid: is the name of the target peer as defined in the configuration peers section.
+localpeerid: is the name of the local peer as defined on cmdline or using hostname.
+processid: is the system process id of the local process.
+relativepid: is the haproxy's relative pid (0 if nbproc == 1)
+
+2) Status Message
+
+Status message is a code followed by a LF.
+
+200: Handshake succeeded
+300: Try again later
+501: Protocol error
+502: Bad version
+503: Local peer name mismatch
+504: Remote peer name mismatch
+
+
+IV) Messages
+
+Messages:
+
+0 - - - - - - - 8 - - - - - - - 16
+ Message Class| Message Type
+
+if Message Type >= 128
+
+0 - - - - - - - 8 - - - - - - - 16 .....
+ Message Class| Message Type | encoded data length | data
+
+Message Classes:
+0: control
+1: error
+10: related to stick table updates
+255: reserved
+
+
+1) Control Messages Class
+
+Available message Types for class control:
+0: resync request
+1: resync finished
+2: resync partial
+3: resync confirm
+
+
+a) Resync Request Message
+
+This message is used to request a full resync from a peer
+
+b) Resync Finished Message
+
+This message is used to signal remote peer that locally known updates have been pushed, and local peer was considered up to date.
+
+c) Resync Partial Message
+
+This message is used to signal remote peer that locally known updates have been pushed, and but the local peer is not considered up to date.
+
+d) Resync Confirm Message
+
+This message is an ack for Resync Partial or Finished Messages.
+
+It's allow the remote peer to go back to "on the fly" update process.
+
+
+2) Messages Class
+
+Available message Types for this class are:
+0: protocol error
+1: size limit reached
+
+a) Protocol Message
+
+To signal that a protocol error occurred. Connection will be shutdown just after sending this message.
+
+b) Size Limit Error Message
+
+To signal that a message is outsized and can not be correctly handled. Connection will be broken.
+
+
+
+3) Stick Table Updates Messages Class
+
+Available message Types for this class are:
+0: Entry update
+1: Incremental entry update
+2: table definition
+3: table switch
+4: updates ack message.
+
+
+a) Update Message
+
+0 - - - - - - - 8 - - - - - - - 16 .....
+ Message class | Message Type | encoded data length | data
+
+
+data is composed like this
+
+0 - - - - - - - 32 .............................
+Local Update ID | Key value | data values ....
+
+Update ID in a 32bits identifier of the local update.
+
+Key value format depends of the table key type:
+
+- for keytype string
+
+0 .................................
+encoded string length | string value
+
+- for keytype integer
+0 - - - - - - - - - - 32
+encoded integer value |
+
+- for other key type
+
+The value length is announced in table definition message
+0 ....................
+value
+
+
+b) Incremental Update Message
+
+Same format than update message except the Update ID is not present, the receiver should
+consider that the update ID is an increment of 1 of the previous considered update message (partial or not)
+
+
+c) Table Definition Message
+
+This message is used by the receiver to identify the stick table concerned by next update messages and
+to know which data is pushed in these updates.
+
+0 - - - - - - - 8 - - - - - - - 16 .....
+ Message class | Message Type | encoded data length | data
+
+
+data is composed like this
+
+0 ...................................................................
+Encoded Sender Table Id | Encoded Local Table Name Length | Table Name | Encoded Table Type | Encoded Table Keylen | Encoded Table Data Types Bitfield
+
+
+Encoded Sender Table ID present a the id numerical ID affected to that table by the sender
+It will be used by "Updates Aknowlegement Messages" and "Table Switch Messages".
+
+Encoded Local Table Name Length present the length to read the table name.
+
+"Table Name" is the shared identifier of the table (name of the current table in the configuration)
+It permits the receiver to identify the concerned table. The receiver should keep in memory the matching
+between the "Sender Table ID" to identify it directly in case of "Table Switch Message".
+
+Table Type present the numeric type of key used to store stick table entries:
+integer
+ 2: signed integer
+ 4: IPv4 address
+ 5: IPv6 address
+ 6: string
+ 7: binary
+
+Table Keylen present the key length or max length in case of strings or binary (padded with 0).
+
+Data Types Bitfield present the types of data linearly pushed in next updates message (they will be linearly pushed in the update message)
+Known types are
+bit
+ 0: server id
+ 1: gpt0
+ 2: gpc0
+ 3: gpc0 rate
+ 4: connections counter
+ 5: connection rate
+ 6: number of current connections
+ 7: sessions counter
+ 8: session rate
+ 9: http requests counter
+ 10: http requests rate
+ 11: errors counter
+ 12: errors rate
+ 13: bytes in counter
+ 14: bytes in rate
+ 15: bytes out rate
+ 16: bytes out rate
+ 17: gpc1
+ 18: gpc1 rate
+ 19: server key
+ 20: http fail counter
+ 21: http fail rate
+ 22: gpt array
+ 23: gpc array
+ 24: gpc rate array
+
+d) Table Switch Message
+
+After a Table Message Define, this message can be used by the receiver to identify the stick table concerned by next update messages.
+
+0 - - - - - - - 8 - - - - - - - 16 .....
+ Message class | Message Type | encoded data length | data
+
+
+data is composed like this
+
+
+0 .....................
+encoded Sender Table Id
+
+c) Update Ack Message
+
+0 - - - - - - - 8 - - - - - - - 16 .....
+ Message class | Message Type | encoded data length | data
+
+data is composed like this
+
+0 ....................... - - - - - - - - 32
+Encoded Remote Table Id | Update Id
+
+
+Remote Table Id is the numeric identifier of the table on the remote side.
+Update Id is the id of the last update locally committed.
+
+If a re-connection occurred, the sender should know they will have to restart the push of updates from this point.
+
+III) Initial full resync process.
+
+
+a) Resync from local old process
+
+An old soft-stopped process will close all established sessions with remote peers and will try to connect to a new
+local process to push all known ending with a Resync Finished Message or a Resync Partial Message (if it it does not consider itself as full updated).
+
+A new process will wait for a an incoming connection from a local process during 5 seconds. It will learn the updates from this
+process until it receives a Resync Finished Message or a Resync Partial Message. If it receive a Resync Finished Message it will consider itself
+as fully updated and stops to ask for resync. If it receive a Resync Partial Message it will wait once again for 5 seconds for an other incoming connection from a local process.
+Same thing if the session was broken before receiving any "Resync Partial Message" or "Resync Finished Message".
+
+If one of these 5 seconds timeout expire, the process will try to request resync from a remote connected peer (see b). The process will wait until 5seconds
+if no available remote peers are found.
+
+If the timeout expire, the process will consider itself ass fully updated
+
+b) Resync from remote peers
+
+The process will randomly choose a remote connected peer and ask for a full resync using a Resync Request Message. The process will wait until 5seconds
+if no available remote peers are found.
+
+The chosen remote peer will push its all known data ending with a Resync Finished Message or a Resync Partial Message (if it it does not consider itself as full updated).
+
+If it receives a Resync Finished Message it will consider itself as fully updated and stops to ask for resync.
+
+If it receives a Resync Partial Message, the current peer will be flagged to anymore be requested and any other connected peer will be randomly chosen for a resync request (5s).
+
+If the session is broken before receiving any of these messages any other connected peer will be randomly chosen for a resync request (5s).
+
+If the timeout expire, the process will consider itself as fully updated
+
+
diff --git a/doc/peers.txt b/doc/peers.txt
new file mode 100644
index 0000000..7ce2fcb
--- /dev/null
+++ b/doc/peers.txt
@@ -0,0 +1,491 @@
+ +--------------------+
+ | Peers protocol 2.1 |
+ +--------------------+
+
+
+ Peers protocol has been implemented over TCP. Its aim is to transmit
+ stick-table entries information between several haproxy processes.
+
+ This protocol is symmetrical. This means that at any time, each peer
+ may connect to other peers they have been configured for, to send
+ their last stick-table updates. There is no role of client or server in this
+ protocol. As peers may connect to each others at the same time, the protocol
+ ensures that only one peer session may stay opened between a couple of peers
+ before they start sending their stick-table information, possibly in both
+ directions (or not).
+
+
+ Handshake
+ +++++++++
+
+ Just after having connected to another one, a peer must identify itself
+ and identify the remote peer, sending a "hello" message. The remote peer
+ replies with a "status" message.
+
+ A "hello" message is made of three lines terminated by a line feed character
+ as follows:
+
+ <protocol identifier> <version>\n
+ <remote peer identifier>\n
+ <local peer identifier> <process ID> <relative process ID>\n
+
+ protocol identifier : HAProxyS
+ version : 2.1
+ remote peer identifier: the peer name this "hello" message is sent to.
+ local peer identifier : the name of the peer which sends this "hello" message.
+ process ID : the ID of the process handling this peer session.
+ relative process ID : the haproxy's relative process ID (0 if nbproc == 1).
+
+ The "status" message is made of a unique line terminated by a line feed
+ character as follows:
+
+ <status code>\n
+
+ with these values as status code (a three-digit number):
+
+ +-------------+---------------------------------+
+ | status code | signification |
+ +-------------+---------------------------------+
+ | 200 | Handshake succeeded |
+ +-------------+---------------------------------+
+ | 300 | Try again later |
+ +-------------+---------------------------------+
+ | 501 | Protocol error |
+ +-------------+---------------------------------+
+ | 502 | Bad version |
+ +-------------+---------------------------------+
+ | 503 | Local peer identifier mismatch |
+ +-------------+---------------------------------+
+ | 504 | Remote peer identifier mismatch |
+ +-------------+---------------------------------+
+
+ As the protocol is symmetrical, some peers may connect to each other at the
+ same time. For efficiency reasons, the protocol ensures there may be only
+ one TCP session opened after the handshake succeeded and before transmitting
+ any stick-table data information. In fact, for each couple of peers, this is
+ the last connected peer which wins. Each time a peer A receives a "hello"
+ message from a peer B, peer A checks if it already managed to open a peer
+ session with peer B, so with a successful handshake. If it is the case,
+ peer A closes its peer session. So, this is the peer session opened by B
+ which stays opened.
+
+
+ Peer A Peer B
+ hello
+ ---------------------->
+ status 200
+ <----------------------
+ hello
+ <++++++++++++++++++++++
+ TCP/FIN-ACK
+ ---------------------->
+ TCP/FIN-ACK
+ <----------------------
+ status 200
+ ++++++++++++++++++++++>
+ data
+ <++++++++++++++++++++++
+ data
+ ++++++++++++++++++++++>
+ data
+ ++++++++++++++++++++++>
+ data
+ <++++++++++++++++++++++
+ .
+ .
+ .
+
+ As it is still possible that a couple of peers decide to close both their
+ peer sessions at the same time, the protocol ensures peers will not reconnect
+ at the same time, adding a random delay (50 up to 2050 ms) before any
+ reconnection.
+
+
+ Encoding
+ ++++++++
+
+ As some TCP data may be corrupted, for integrity reason, some data fields
+ are encoded at peer session level.
+
+ The following algorithms explain how to encode/decode the data.
+
+ encode:
+ input : val (64bits integer)
+ output: bitf (variable-length bitfield)
+
+ if val has no bit set above bit 4 (or if val is less than 0xf0)
+ set the next byte of bitf to the value of val
+ return bitf
+
+ set the next byte of bitf to the value of val OR'ed with 0xf0
+ subtract 0xf0 from val
+ right shift val by 4
+
+ while val bit 7 is set (or if val is greater or equal to 0x80):
+ set the next byte of bitf to the value of the byte made of the last
+ 7 bits of val OR'ed with 0x80
+ subtract 0x80 from val
+ right shift val by 7
+
+ set the next byte of bitf to the value of val
+ return bitf
+
+ decode:
+ input : bitf (variable-length bitfield)
+ output: val (64bits integer)
+
+ set val to the value of the first byte of bitf
+ if bit 4 up to 7 of val are not set
+ return val
+
+ set loop to 0
+ do
+ add to val the value of the next byte of bitf left shifted by (4 + 7*loop)
+ set loop to (loop + 1)
+ while the bit 7 of the next byte of bitf is set
+ return val
+
+ Example:
+
+ let's say that we must encode 0x1234.
+
+ "set the next byte of bitf to the value of val OR'ed with 0xf0"
+ => bitf[0] = (0x1234 | 0xf0) & 0xff = 0xf4
+
+ "subtract 0xf0 from val"
+ => val = 0x1144
+
+ right shift val by 4
+ => val = 0x114
+
+ "set the next byte of bitf to the value of the byte made of the last
+ 7 bits of val OR'ed with 0x80"
+ => bitf[1] = (0x114 | 0x80) & 0xff = 0x94
+
+ "subtract 0x80 from val"
+ => val= 0x94
+
+ "right shift val by 7"
+ => val = 0x1
+
+ => bitf[2] = 0x1
+
+ So, the encoded value of 0x1234 is 0xf49401.
+
+ To decode this value:
+
+ "set val to the value of the first byte of bitf"
+ => val = 0xf4
+
+ "add to val the value of the next byte of bitf left shifted by 4"
+ => val = 0xf4 + (0x94 << 4) = 0xf4 + 0x940 = 0xa34
+
+ "add to val the value of the next byte of bitf left shifted by (4 + 7)"
+ => val = 0xa34 + (0x01 << 11) = 0xa34 + 0x800 = 0x1234
+
+
+ Messages
+ ++++++++
+
+ *** General ***
+
+ After the handshake has successfully completed, peers are authorized to send
+ some messages to each others, possibly in both direction.
+
+ All the messages are made at least of a two bytes length header.
+
+ The first byte of this header identifies the class of the message. The next
+ byte identifies the type of message in the class.
+
+ Some of these messages are variable-length. Others have a fixed size.
+ Variable-length messages are identified by the value of the message type
+ byte. For such messages, it is greater than or equal to 128.
+
+ All variable-length message headers must be followed by the encoded length
+ of the remaining bytes (so the encoded length of the message minus 2 bytes
+ for the header and minus the length of the encoded length).
+
+ There exist four classes of messages:
+
+ +------------+---------------------+--------------+
+ | class byte | signification | message size |
+ +------------+---------------------+--------------+
+ | 0 | control | fixed (2) |
+ +------------+---------------------+--------------|
+ | 1 | error | fixed (2) |
+ +------------+---------------------+--------------|
+ | 10 | stick-table updates | variable |
+ +------------+---------------------+--------------|
+ | 255 | reserved | |
+ +------------+---------------------+--------------+
+
+ At this time of this writing, only control and error messages have a fixed
+ size of two bytes (header only). The stick-table updates messages are all
+ variable-length (their message type bytes are greater than 128).
+
+
+ *** Control message class ***
+
+ At this time of writing, control messages are fixed-length messages used
+ only to control the synchronizations between local and/or remote processes
+ and to emit heartbeat messages.
+
+ There exists five types of such control messages:
+
+ +------------+--------------------------------------------------------+
+ | type byte | signification |
+ +------------+--------------------------------------------------------+
+ | 0 | synchronisation request: ask a remote peer for a full |
+ | | synchronization |
+ +------------+--------------------------------------------------------+
+ | 1 | synchronization finished: signal a remote peer that |
+ | | local updates have been pushed and local is considered |
+ | | up to date. |
+ +------------+--------------------------------------------------------+
+ | 2 | synchronization partial: signal a remote peer that |
+ | | local updates have been pushed and local is not |
+ | | considered up to date. |
+ +------------+--------------------------------------------------------+
+ | 3 | synchronization confirmed: acknowledge a finished or |
+ | | partial synchronization message. |
+ +------------+--------------------------------------------------------+
+ | 4 | Heartbeat message. |
+ +------------+--------------------------------------------------------+
+
+ About heartbeat messages: a peer sends heartbeat messages to peers it is
+ connected to after periods of 3s of inactivity (i.e. when there is no
+ stick-table to synchronize for 3s). After a successful peer protocol
+ handshake between two peers, if one of them does not send any other peer
+ protocol messages (i.e. no heartbeat and no stick-table update messages)
+ during a 5s period, it is considered as no more alive by its remote peer
+ which closes the session and then tries to reconnect to the peer which
+ has just disappeared.
+
+ *** Error message class ***
+
+ There exits two types of such error messages:
+
+ +-----------+------------------+
+ | type byte | signification |
+ +-----------+------------------+
+ | 0 | protocol error |
+ +-----------+------------------+
+ | 1 | size limit error |
+ +-----------+------------------+
+
+
+ *** Stick-table update message class ***
+
+ This class is the more important one because it is in relation with the
+ stick-table entries handling between peers which is at the core of peers
+ protocol.
+
+ All the messages of this class are variable-length. Their type bytes are
+ all greater than or equal to 128.
+
+ There exits five types of such stick-table update messages:
+
+ +-----------+--------------------------------+
+ | type byte | signification |
+ +-----------+--------------------------------+
+ | 128 | Entry update |
+ +-----------+--------------------------------+
+ | 129 | Incremental entry update |
+ +-----------+--------------------------------+
+ | 130 | Stick-table definition |
+ +-----------+--------------------------------+
+ | 131 | Stick-table switch (unused) |
+ +-----------+--------------------------------+
+ | 133 | Update message acknowledgement |
+ +-----------+--------------------------------+
+
+ Note that entry update messages may be multiplexed. This means that different
+ entry update messages for different stick-tables may be sent over the same
+ peer session.
+
+ To do so, each time entry update messages have to sent, they must be preceded
+ by a stick-table definition message. This remains true for incremental entry
+ update messages.
+
+ As its name indicate, "Update message acknowledgement" messages are used to
+ acknowledge the entry update messages.
+
+ In this following paragraph, we give some information about the format of
+ each stick-table update messages. This very simple following legend will
+ contribute in understanding it. The unit used is the octet.
+
+ XX
+ +-----------+
+ | foo | Unique fixed sized "foo" field, made of XX octets.
+ +-----------+
+
+ +===========+
+ | foo | Variable-length "foo" field.
+ +===========+
+
+ +xxxxxxxxxxx+
+ | foo | Encoded variable-length "foo" field.
+ +xxxxxxxxxxx+
+
+ +###########+
+ | foo | hereunder described "foo" field.
+ +###########+
+
+
+ With this legend, all the stick-table update messages have such a header:
+
+ 1 1
+ +--------------------+------------------------+xxxxxxxxxxxxxxxx+
+ | Message Class (10) | Message type (128-133) | Message length |
+ +--------------------+------------------------+xxxxxxxxxxxxxxxx+
+
+ Note that to help in making communicate different versions of peers protocol,
+ such stick-table update messages may be extended adding non mandatory
+ fields at the end of such messages, announcing a total message length
+ which is greater than the message length of the previous versions of
+ peers protocol. After having parsed such messages, the remaining ones
+ will be skipped to parse the next message.
+
+ - Definition message format:
+
+ Before sending entry update messages, a peer must announce the configuration
+ of the stick-table in relation with these messages thanks to a
+ "Stick-table definition" message with such a following format:
+
+ +xxxxxxxxxxxxxxxx+xxxxxxxxxxxxxxxxxxxxxxxxx+==================+
+ | Stick-table ID | Stick-table name length | Stick-table name |
+ +xxxxxxxxxxxxxxxx+xxxxxxxxxxxxxxxxxxxxxxxxx+==================+
+
+ +xxxxxxxxxxxx+xxxxxxxxxxxxxx+xxxxxxxxxxxxxxxxxxxxxxx+xxxxxxxxx+
+ | Key type | Key length | Data types bitfield | Expiry |
+ +xxxxxxxxxxxx+xxxxxxxxxxxxxx+xxxxxxxxxxxxxxxxxxxxxxx+xxxxxxxxx+
+
+ +xxxxxxxxxxxxxxxxxxxxxxxxxxx+xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx+
+ | Frequency counter #1 | Frequency counter #1 period |
+ +xxxxxxxxxxxxxxxxxxxxxxxxxxx+xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx+
+
+ +xxxxxxxxxxxxxxxxxxxxxxxxxxx+xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx+
+ | Frequency counter #2 | Frequency counter #2 period |
+ +xxxxxxxxxxxxxxxxxxxxxxxxxxx+xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx+
+ .
+ .
+ .
+
+ Note that "Stick-table ID" field is an encoded integer which is used to
+ identify the stick-table without using its name (or "Stick-table name"
+ field). It is local to the process handling the stick-table. So we can have
+ two peers attached to processes which generate stick-table updates for
+ the same stick-table (same name) but with different stick-table IDs.
+
+ Also note that the list of "Frequency counter #X" and their associated
+ periods fields exists only if their underlying types are already defined
+ in "Data types bitfield" field.
+
+ "Expiry" field and the remaining ones are not used by all the existing
+ version of haproxy peers. But they are MANDATORY, so that to make a
+ stick-table aggregator peer be able to autoconfigure itself.
+
+
+ - Entry update message format:
+ 4
+ +-----------------+###########+############+
+ | Local update ID | Key | Data |
+ +-----------------+###########+############+
+
+ with "Key" described as follows:
+
+ +xxxxxxxxxxx+=======+
+ | length | value | if key type is (non null terminated) "string",
+ +xxxxxxxxxxx+=======+
+
+ 4
+ +-------+
+ | value | if key type is "integer",
+ +-------+
+
+ +=======+
+ | value | for other key types: the size is announced in
+ +=======+ the previous stick-table definition message.
+
+ "Data" field is basically a list of encoded values for each type announced
+ by the "Data types bitfield" field of the previous "Stick-table definition"
+ message:
+
+ +xxxxxxxxxxxxxxxxxxxx+xxxxxxxxxxxxxxxxxxxx+ +xxxxxxxxxxxxxxxxxxxx+
+ | Data type #1 value | Data type #2 value | .... | Data type #n value |
+ +xxxxxxxxxxxxxxxxxxxx+xxxxxxxxxxxxxxxxxxxx+ +xxxxxxxxxxxxxxxxxxxx+
+
+
+ Most of these fields are internally stored as uint32_t (see STD_T_SINT,
+ STD_T_UINT, STD_T_ULL C enumerations) or structures made of several uint32_t
+ (see STD_T_FRQP C enumeration). The remaining one STD_T_DICT is internally
+ used to store entries of LRU caches for others literal dictionary entries
+ (couples of IDs associated to strings). It is used to transmit these cache
+ entries as follows:
+
+ +xxxxxxxxxxx+xxxx+xxxxxxxxxxxxxxx+========+
+ | length | ID | string length | string |
+ +xxxxxxxxxxx+xxxx+xxxxxxxxxxxxxxx+========+
+
+ "length" is the length in bytes of the remaining data after this "length" field.
+ "string length" is the length of "string" field which follows.
+
+ Here the cache is used so that not to have to send again and again an already
+ sent string. Indeed, the second time we have to send the same dictionary entry,
+ if still cached, a peer sends only its ID:
+
+ +xxxxxxxxxxx+xxxx+
+ | length | ID |
+ +xxxxxxxxxxx+xxxx+
+
+ - Update message acknowledgement format:
+
+ These messages are responses to "Entry update" messages only.
+
+ Its format is very basic for efficiency reasons:
+
+ 4
+ +xxxxxxxxxxxxxxxx+-----------+
+ | Stick-table ID | Update ID |
+ +xxxxxxxxxxxxxxxx+-----------+
+
+
+ Note that the "Stick-table ID" field value is in relation with the one which
+ has been previously announce by a "Stick-table definition" message.
+
+ The following schema may help in understanding how to handle a stream of
+ stick-table update messages. The handshake step is not represented.
+ Stick-table IDs are preceded by a '#' character.
+
+
+ Peer A Peer B
+
+ stkt def. #1
+ ---------------------->
+ updates (1-5)
+ ---------------------->
+ stkt def. #3
+ ---------------------->
+ updates (1000-1005)
+ ---------------------->
+
+ stkt def. #2
+ <----------------------
+ updates (10-15)
+ <----------------------
+ ack 5 for #1
+ <----------------------
+ ack 1005 for #3
+ <----------------------
+ stkt def. #4
+ <----------------------
+ updates (100-105)
+ <----------------------
+
+ ack 10 for #2
+ ---------------------->
+ ack 105 for #4
+ ---------------------->
+ (from here, on both sides, all stick-table updates
+ are considered as received)
+
diff --git a/doc/proxy-protocol.txt b/doc/proxy-protocol.txt
new file mode 100644
index 0000000..fac0331
--- /dev/null
+++ b/doc/proxy-protocol.txt
@@ -0,0 +1,1051 @@
+2020/03/05 Willy Tarreau
+ HAProxy Technologies
+ The PROXY protocol
+ Versions 1 & 2
+
+Abstract
+
+ The PROXY protocol provides a convenient way to safely transport connection
+ information such as a client's address across multiple layers of NAT or TCP
+ proxies. It is designed to require little changes to existing components and
+ to limit the performance impact caused by the processing of the transported
+ information.
+
+
+Revision history
+
+ 2010/10/29 - first version
+ 2011/03/20 - update: implementation and security considerations
+ 2012/06/21 - add support for binary format
+ 2012/11/19 - final review and fixes
+ 2014/05/18 - modify and extend PROXY protocol version 2
+ 2014/06/11 - fix example code to consider ver+cmd merge
+ 2014/06/14 - fix v2 header check in example code, and update Forwarded spec
+ 2014/07/12 - update list of implementations (add Squid)
+ 2015/05/02 - update list of implementations and format of the TLV add-ons
+ 2017/03/10 - added the checksum, noop and more SSL-related TLV types,
+ reserved TLV type ranges, added TLV documentation, clarified
+ string encoding. With contributions from Andriy Palamarchuk
+ (Amazon.com).
+ 2020/03/05 - added the unique ID TLV type (Tim DĂĽsterhus)
+
+
+1. Background
+
+Relaying TCP connections through proxies generally involves a loss of the
+original TCP connection parameters such as source and destination addresses,
+ports, and so on. Some protocols make it a little bit easier to transfer such
+information. For SMTP, Postfix authors have proposed the XCLIENT protocol [1]
+which received broad adoption and is particularly suited to mail exchanges.
+For HTTP, there is the "Forwarded" extension [2], which aims at replacing the
+omnipresent "X-Forwarded-For" header which carries information about the
+original source address, and the less common X-Original-To which carries
+information about the destination address.
+
+However, both mechanisms require a knowledge of the underlying protocol to be
+implemented in intermediaries.
+
+Then comes a new class of products which we'll call "dumb proxies", not because
+they don't do anything, but because they're processing protocol-agnostic data.
+Both Stunnel[3] and Stud[4] are examples of such "dumb proxies". They talk raw
+TCP on one side, and raw SSL on the other one, and do that reliably, without
+any knowledge of what protocol is transported on top of the connection. HAProxy
+running in pure TCP mode obviously falls into that category as well.
+
+The problem with such a proxy when it is combined with another one such as
+haproxy, is to adapt it to talk the higher level protocol. A patch is available
+for Stunnel to make it capable of inserting an X-Forwarded-For header in the
+first HTTP request of each incoming connection. HAProxy is able not to add
+another one when the connection comes from Stunnel, so that it's possible to
+hide it from the servers.
+
+The typical architecture becomes the following one :
+
+
+ +--------+ HTTP :80 +----------+
+ | client | --------------------------------> | |
+ | | | haproxy, |
+ +--------+ +---------+ | 1 or 2 |
+ / / HTTPS | stunnel | HTTP :81 | listening|
+ <________/ ---------> | (server | ---------> | ports |
+ | mode) | | |
+ +---------+ +----------+
+
+
+The problem appears when haproxy runs with keep-alive on the side towards the
+client. The Stunnel patch will only add the X-Forwarded-For header to the first
+request of each connection and all subsequent requests will not have it. One
+solution could be to improve the patch to make it support keep-alive and parse
+all forwarded data, whether they're announced with a Content-Length or with a
+Transfer-Encoding, taking care of special methods such as HEAD which announce
+data without transferring them, etc... In fact, it would require implementing a
+full HTTP stack in Stunnel. It would then become a lot more complex, a lot less
+reliable and would not anymore be the "dumb proxy" that fits every purposes.
+
+In practice, we don't need to add a header for each request because we'll emit
+the exact same information every time : the information related to the client
+side connection. We could then cache that information in haproxy and use it for
+every other request. But that becomes dangerous and is still limited to HTTP
+only.
+
+Another approach consists in prepending each connection with a header reporting
+the characteristics of the other side's connection. This method is simpler to
+implement, does not require any protocol-specific knowledge on either side, and
+completely fits the purpose since what is desired precisely is to know the
+other side's connection endpoints. It is easy to perform for the sender (just
+send a short header once the connection is established) and to parse for the
+receiver (simply perform one read() on the incoming connection to fill in
+addresses after an accept). The protocol used to carry connection information
+across proxies was thus called the PROXY protocol.
+
+
+2. The PROXY protocol header
+
+This document uses a few terms that are worth explaining here :
+ - "connection initiator" is the party requesting a new connection
+ - "connection target" is the party accepting a connection request
+ - "client" is the party for which a connection was requested
+ - "server" is the party to which the client desired to connect
+ - "proxy" is the party intercepting and relaying the connection
+ from the client to the server.
+ - "sender" is the party sending data over a connection.
+ - "receiver" is the party receiving data from the sender.
+ - "header" or "PROXY protocol header" is the block of connection information
+ the connection initiator prepends at the beginning of a connection, which
+ makes it the sender from the protocol point of view.
+
+The PROXY protocol's goal is to fill the server's internal structures with the
+information collected by the proxy that the server would have been able to get
+by itself if the client was connecting directly to the server instead of via a
+proxy. The information carried by the protocol are the ones the server would
+get using getsockname() and getpeername() :
+ - address family (AF_INET for IPv4, AF_INET6 for IPv6, AF_UNIX)
+ - socket protocol (SOCK_STREAM for TCP, SOCK_DGRAM for UDP)
+ - layer 3 source and destination addresses
+ - layer 4 source and destination ports if any
+
+Unlike the XCLIENT protocol, the PROXY protocol was designed with limited
+extensibility in order to help the receiver parse it very fast. Version 1 was
+focused on keeping it human-readable for better debugging possibilities, which
+is always desirable for early adoption when few implementations exist. Version
+2 adds support for a binary encoding of the header which is much more efficient
+to produce and to parse, especially when dealing with IPv6 addresses that are
+expensive to emit in ASCII form and to parse.
+
+In both cases, the protocol simply consists in an easily parsable header placed
+by the connection initiator at the beginning of each connection. The protocol
+is intentionally stateless in that it does not expect the sender to wait for
+the receiver before sending the header, nor the receiver to send anything back.
+
+This specification supports two header formats, a human-readable format which
+is the only format supported in version 1 of the protocol, and a binary format
+which is only supported in version 2. Both formats were designed to ensure that
+the header cannot be confused with common higher level protocols such as HTTP,
+SSL/TLS, FTP or SMTP, and that both formats are easily distinguishable one from
+each other for the receiver.
+
+Version 1 senders MAY only produce the human-readable header format. Version 2
+senders MAY only produce the binary header format. Version 1 receivers MUST at
+least implement the human-readable header format. Version 2 receivers MUST at
+least implement the binary header format, and it is recommended that they also
+implement the human-readable header format for better interoperability and ease
+of upgrade when facing version 1 senders.
+
+Both formats are designed to fit in the smallest TCP segment that any TCP/IP
+host is required to support (576 - 40 = 536 bytes). This ensures that the whole
+header will always be delivered at once when the socket buffers are still empty
+at the beginning of a connection. The sender must always ensure that the header
+is sent at once, so that the transport layer maintains atomicity along the path
+to the receiver. The receiver may be tolerant to partial headers or may simply
+drop the connection when receiving a partial header. Recommendation is to be
+tolerant, but implementation constraints may not always easily permit this. It
+is important to note that nothing forces any intermediary to forward the whole
+header at once, because TCP is a streaming protocol which may be processed one
+byte at a time if desired, causing the header to be fragmented when reaching
+the receiver. But due to the places where such a protocol is used, the above
+simplification generally is acceptable because the risk of crossing such a
+device handling one byte at a time is close to zero.
+
+The receiver MUST NOT start processing the connection before it receives a
+complete and valid PROXY protocol header. This is particularly important for
+protocols where the receiver is expected to speak first (eg: SMTP, FTP or SSH).
+The receiver may apply a short timeout and decide to abort the connection if
+the protocol header is not seen within a few seconds (at least 3 seconds to
+cover a TCP retransmit).
+
+The receiver MUST be configured to only receive the protocol described in this
+specification and MUST not try to guess whether the protocol header is present
+or not. This means that the protocol explicitly prevents port sharing between
+public and private access. Otherwise it would open a major security breach by
+allowing untrusted parties to spoof their connection addresses. The receiver
+SHOULD ensure proper access filtering so that only trusted proxies are allowed
+to use this protocol.
+
+Some proxies are smart enough to understand transported protocols and to reuse
+idle server connections for multiple messages. This typically happens in HTTP
+where requests from multiple clients may be sent over the same connection. Such
+proxies MUST NOT implement this protocol on multiplexed connections because the
+receiver would use the address advertised in the PROXY header as the address of
+all forwarded requests's senders. In fact, such proxies are not dumb proxies,
+and since they do have a complete understanding of the transported protocol,
+they MUST use the facilities provided by this protocol to present the client's
+address.
+
+
+2.1. Human-readable header format (Version 1)
+
+This is the format specified in version 1 of the protocol. It consists in one
+line of US-ASCII text matching exactly the following block, sent immediately
+and at once upon the connection establishment and prepended before any data
+flowing from the sender to the receiver :
+
+ - a string identifying the protocol : "PROXY" ( \x50 \x52 \x4F \x58 \x59 )
+ Seeing this string indicates that this is version 1 of the protocol.
+
+ - exactly one space : " " ( \x20 )
+
+ - a string indicating the proxied INET protocol and family. As of version 1,
+ only "TCP4" ( \x54 \x43 \x50 \x34 ) for TCP over IPv4, and "TCP6"
+ ( \x54 \x43 \x50 \x36 ) for TCP over IPv6 are allowed. Other, unsupported,
+ or unknown protocols must be reported with the name "UNKNOWN" ( \x55 \x4E
+ \x4B \x4E \x4F \x57 \x4E ). For "UNKNOWN", the rest of the line before the
+ CRLF may be omitted by the sender, and the receiver must ignore anything
+ presented before the CRLF is found. Note that an earlier version of this
+ specification suggested to use this when sending health checks, but this
+ causes issues with servers that reject the "UNKNOWN" keyword. Thus is it
+ now recommended not to send "UNKNOWN" when the connection is expected to
+ be accepted, but only when it is not possible to correctly fill the PROXY
+ line.
+
+ - exactly one space : " " ( \x20 )
+
+ - the layer 3 source address in its canonical format. IPv4 addresses must be
+ indicated as a series of exactly 4 integers in the range [0..255] inclusive
+ written in decimal representation separated by exactly one dot between each
+ other. Heading zeroes are not permitted in front of numbers in order to
+ avoid any possible confusion with octal numbers. IPv6 addresses must be
+ indicated as series of sets of 4 hexadecimal digits (upper or lower case)
+ delimited by colons between each other, with the acceptance of one double
+ colon sequence to replace the largest acceptable range of consecutive
+ zeroes. The total number of decoded bits must exactly be 128. The
+ advertised protocol family dictates what format to use.
+
+ - exactly one space : " " ( \x20 )
+
+ - the layer 3 destination address in its canonical format. It is the same
+ format as the layer 3 source address and matches the same family.
+
+ - exactly one space : " " ( \x20 )
+
+ - the TCP source port represented as a decimal integer in the range
+ [0..65535] inclusive. Heading zeroes are not permitted in front of numbers
+ in order to avoid any possible confusion with octal numbers.
+
+ - exactly one space : " " ( \x20 )
+
+ - the TCP destination port represented as a decimal integer in the range
+ [0..65535] inclusive. Heading zeroes are not permitted in front of numbers
+ in order to avoid any possible confusion with octal numbers.
+
+ - the CRLF sequence ( \x0D \x0A )
+
+
+The maximum line lengths the receiver must support including the CRLF are :
+ - TCP/IPv4 :
+ "PROXY TCP4 255.255.255.255 255.255.255.255 65535 65535\r\n"
+ => 5 + 1 + 4 + 1 + 15 + 1 + 15 + 1 + 5 + 1 + 5 + 2 = 56 chars
+
+ - TCP/IPv6 :
+ "PROXY TCP6 ffff:f...f:ffff ffff:f...f:ffff 65535 65535\r\n"
+ => 5 + 1 + 4 + 1 + 39 + 1 + 39 + 1 + 5 + 1 + 5 + 2 = 104 chars
+
+ - unknown connection (short form) :
+ "PROXY UNKNOWN\r\n"
+ => 5 + 1 + 7 + 2 = 15 chars
+
+ - worst case (optional fields set to 0xff) :
+ "PROXY UNKNOWN ffff:f...f:ffff ffff:f...f:ffff 65535 65535\r\n"
+ => 5 + 1 + 7 + 1 + 39 + 1 + 39 + 1 + 5 + 1 + 5 + 2 = 107 chars
+
+So a 108-byte buffer is always enough to store all the line and a trailing zero
+for string processing.
+
+The receiver must wait for the CRLF sequence before starting to decode the
+addresses in order to ensure they are complete and properly parsed. If the CRLF
+sequence is not found in the first 107 characters, the receiver should declare
+the line invalid. A receiver may reject an incomplete line which does not
+contain the CRLF sequence in the first atomic read operation. The receiver must
+not tolerate a single CR or LF character to end the line when a complete CRLF
+sequence is expected.
+
+Any sequence which does not exactly match the protocol must be discarded and
+cause the receiver to abort the connection. It is recommended to abort the
+connection as soon as possible so that the sender gets a chance to notice the
+anomaly and log it.
+
+If the announced transport protocol is "UNKNOWN", then the receiver knows that
+the sender speaks the correct PROXY protocol with the appropriate version, and
+SHOULD accept the connection and use the real connection's parameters as if
+there were no PROXY protocol header on the wire. However, senders SHOULD not
+use the "UNKNOWN" protocol when they are the initiators of outgoing connections
+because some receivers may reject them. When a load balancing proxy has to send
+health checks to a server, it SHOULD build a valid PROXY line which it will
+fill with a getsockname()/getpeername() pair indicating the addresses used. It
+is important to understand that doing so is not appropriate when some source
+address translation is performed between the sender and the receiver.
+
+An example of such a line before an HTTP request would look like this (CR
+marked as "\r" and LF marked as "\n") :
+
+ PROXY TCP4 192.168.0.1 192.168.0.11 56324 443\r\n
+ GET / HTTP/1.1\r\n
+ Host: 192.168.0.11\r\n
+ \r\n
+
+For the sender, the header line is easy to put into the output buffers once the
+connection is established. Note that since the line is always shorter than an
+MSS, the sender is guaranteed to always be able to emit it at once and should
+not even bother handling partial sends. For the receiver, once the header is
+parsed, it is easy to skip it from the input buffers. Please consult section 9
+for implementation suggestions.
+
+
+2.2. Binary header format (version 2)
+
+Producing human-readable IPv6 addresses and parsing them is very inefficient,
+due to the multiple possible representation formats and the handling of compact
+address format. It was also not possible to specify address families outside
+IPv4/IPv6 nor non-TCP protocols. Another drawback of the human-readable format
+is the fact that implementations need to parse all characters to find the
+trailing CRLF, which makes it harder to read only the exact bytes count. Last,
+the UNKNOWN address type has not always been accepted by servers as a valid
+protocol because of its imprecise meaning.
+
+Version 2 of the protocol thus introduces a new binary format which remains
+distinguishable from version 1 and from other commonly used protocols. It was
+specially designed in order to be incompatible with a wide range of protocols
+and to be rejected by a number of common implementations of these protocols
+when unexpectedly presented (please see section 7). Also for better processing
+efficiency, IPv4 and IPv6 addresses are respectively aligned on 4 and 16 bytes
+boundaries.
+
+The binary header format starts with a constant 12 bytes block containing the
+protocol signature :
+
+ \x0D \x0A \x0D \x0A \x00 \x0D \x0A \x51 \x55 \x49 \x54 \x0A
+
+Note that this block contains a null byte at the 5th position, so it must not
+be handled as a null-terminated string.
+
+The next byte (the 13th one) is the protocol version and command.
+
+The highest four bits contains the version. As of this specification, it must
+always be sent as \x2 and the receiver must only accept this value.
+
+The lowest four bits represents the command :
+ - \x0 : LOCAL : the connection was established on purpose by the proxy
+ without being relayed. The connection endpoints are the sender and the
+ receiver. Such connections exist when the proxy sends health-checks to the
+ server. The receiver must accept this connection as valid and must use the
+ real connection endpoints and discard the protocol block including the
+ family which is ignored.
+
+ - \x1 : PROXY : the connection was established on behalf of another node,
+ and reflects the original connection endpoints. The receiver must then use
+ the information provided in the protocol block to get original the address.
+
+ - other values are unassigned and must not be emitted by senders. Receivers
+ must drop connections presenting unexpected values here.
+
+The 14th byte contains the transport protocol and address family. The highest 4
+bits contain the address family, the lowest 4 bits contain the protocol.
+
+The address family maps to the original socket family without necessarily
+matching the values internally used by the system. It may be one of :
+
+ - 0x0 : AF_UNSPEC : the connection is forwarded for an unknown, unspecified
+ or unsupported protocol. The sender should use this family when sending
+ LOCAL commands or when dealing with unsupported protocol families. The
+ receiver is free to accept the connection anyway and use the real endpoint
+ addresses or to reject it. The receiver should ignore address information.
+
+ - 0x1 : AF_INET : the forwarded connection uses the AF_INET address family
+ (IPv4). The addresses are exactly 4 bytes each in network byte order,
+ followed by transport protocol information (typically ports).
+
+ - 0x2 : AF_INET6 : the forwarded connection uses the AF_INET6 address family
+ (IPv6). The addresses are exactly 16 bytes each in network byte order,
+ followed by transport protocol information (typically ports).
+
+ - 0x3 : AF_UNIX : the forwarded connection uses the AF_UNIX address family
+ (UNIX). The addresses are exactly 108 bytes each.
+
+ - other values are unspecified and must not be emitted in version 2 of this
+ protocol and must be rejected as invalid by receivers.
+
+The transport protocol is specified in the lowest 4 bits of the 14th byte :
+
+ - 0x0 : UNSPEC : the connection is forwarded for an unknown, unspecified
+ or unsupported protocol. The sender should use this family when sending
+ LOCAL commands or when dealing with unsupported protocol families. The
+ receiver is free to accept the connection anyway and use the real endpoint
+ addresses or to reject it. The receiver should ignore address information.
+
+ - 0x1 : STREAM : the forwarded connection uses a SOCK_STREAM protocol (eg:
+ TCP or UNIX_STREAM). When used with AF_INET/AF_INET6 (TCP), the addresses
+ are followed by the source and destination ports represented on 2 bytes
+ each in network byte order.
+
+ - 0x2 : DGRAM : the forwarded connection uses a SOCK_DGRAM protocol (eg:
+ UDP or UNIX_DGRAM). When used with AF_INET/AF_INET6 (UDP), the addresses
+ are followed by the source and destination ports represented on 2 bytes
+ each in network byte order.
+
+ - other values are unspecified and must not be emitted in version 2 of this
+ protocol and must be rejected as invalid by receivers.
+
+In practice, the following protocol bytes are expected :
+
+ - \x00 : UNSPEC : the connection is forwarded for an unknown, unspecified
+ or unsupported protocol. The sender should use this family when sending
+ LOCAL commands or when dealing with unsupported protocol families. When
+ used with a LOCAL command, the receiver must accept the connection and
+ ignore any address information. For other commands, the receiver is free
+ to accept the connection anyway and use the real endpoints addresses or to
+ reject the connection. The receiver should ignore address information.
+
+ - \x11 : TCP over IPv4 : the forwarded connection uses TCP over the AF_INET
+ protocol family. Address length is 2*4 + 2*2 = 12 bytes.
+
+ - \x12 : UDP over IPv4 : the forwarded connection uses UDP over the AF_INET
+ protocol family. Address length is 2*4 + 2*2 = 12 bytes.
+
+ - \x21 : TCP over IPv6 : the forwarded connection uses TCP over the AF_INET6
+ protocol family. Address length is 2*16 + 2*2 = 36 bytes.
+
+ - \x22 : UDP over IPv6 : the forwarded connection uses UDP over the AF_INET6
+ protocol family. Address length is 2*16 + 2*2 = 36 bytes.
+
+ - \x31 : UNIX stream : the forwarded connection uses SOCK_STREAM over the
+ AF_UNIX protocol family. Address length is 2*108 = 216 bytes.
+
+ - \x32 : UNIX datagram : the forwarded connection uses SOCK_DGRAM over the
+ AF_UNIX protocol family. Address length is 2*108 = 216 bytes.
+
+
+Only the UNSPEC protocol byte (\x00) is mandatory to implement on the receiver.
+A receiver is not required to implement other ones, provided that it
+automatically falls back to the UNSPEC mode for the valid combinations above
+that it does not support.
+
+The 15th and 16th bytes is the address length in bytes in network endian order.
+It is used so that the receiver knows how many address bytes to skip even when
+it does not implement the presented protocol. Thus the length of the protocol
+header in bytes is always exactly 16 + this value. When a sender presents a
+LOCAL connection, it should not present any address so it sets this field to
+zero. Receivers MUST always consider this field to skip the appropriate number
+of bytes and must not assume zero is presented for LOCAL connections. When a
+receiver accepts an incoming connection showing an UNSPEC address family or
+protocol, it may or may not decide to log the address information if present.
+
+So the 16-byte version 2 header can be described this way :
+
+ struct proxy_hdr_v2 {
+ uint8_t sig[12]; /* hex 0D 0A 0D 0A 00 0D 0A 51 55 49 54 0A */
+ uint8_t ver_cmd; /* protocol version and command */
+ uint8_t fam; /* protocol family and address */
+ uint16_t len; /* number of following bytes part of the header */
+ };
+
+Starting from the 17th byte, addresses are presented in network byte order.
+The address order is always the same :
+ - source layer 3 address in network byte order
+ - destination layer 3 address in network byte order
+ - source layer 4 address if any, in network byte order (port)
+ - destination layer 4 address if any, in network byte order (port)
+
+The address block may directly be sent from or received into the following
+union which makes it easy to cast from/to the relevant socket native structs
+depending on the address type :
+
+ union proxy_addr {
+ struct { /* for TCP/UDP over IPv4, len = 12 */
+ uint32_t src_addr;
+ uint32_t dst_addr;
+ uint16_t src_port;
+ uint16_t dst_port;
+ } ipv4_addr;
+ struct { /* for TCP/UDP over IPv6, len = 36 */
+ uint8_t src_addr[16];
+ uint8_t dst_addr[16];
+ uint16_t src_port;
+ uint16_t dst_port;
+ } ipv6_addr;
+ struct { /* for AF_UNIX sockets, len = 216 */
+ uint8_t src_addr[108];
+ uint8_t dst_addr[108];
+ } unix_addr;
+ };
+
+The sender must ensure that all the protocol header is sent at once. This block
+is always smaller than an MSS, so there is no reason for it to be segmented at
+the beginning of the connection. The receiver should also process the header
+at once. The receiver must not start to parse an address before the whole
+address block is received. The receiver must also reject incoming connections
+containing partial protocol headers.
+
+A receiver may be configured to support both version 1 and version 2 of the
+protocol. Identifying the protocol version is easy :
+
+ - if the incoming byte count is 16 or above and the 13 first bytes match
+ the protocol signature block followed by the protocol version 2 :
+
+ \x0D\x0A\x0D\x0A\x00\x0D\x0A\x51\x55\x49\x54\x0A\x20
+
+ - otherwise, if the incoming byte count is 8 or above, and the 5 first
+ characters match the US-ASCII representation of "PROXY" then the protocol
+ must be parsed as version 1 :
+
+ \x50\x52\x4F\x58\x59
+
+ - otherwise the protocol is not covered by this specification and the
+ connection must be dropped.
+
+If the length specified in the PROXY protocol header indicates that additional
+bytes are part of the header beyond the address information, a receiver may
+choose to skip over and ignore those bytes, or attempt to interpret those
+bytes.
+
+The information in those bytes will be arranged in Type-Length-Value (TLV
+vectors) in the following format. The first byte is the Type of the vector.
+The second two bytes represent the length in bytes of the value (not included
+the Type and Length bytes), and following the length field is the number of
+bytes specified by the length.
+
+ struct pp2_tlv {
+ uint8_t type;
+ uint8_t length_hi;
+ uint8_t length_lo;
+ uint8_t value[0];
+ };
+
+A receiver may choose to skip over and ignore the TLVs it is not interested in
+or it does not understand. Senders can generate the TLVs only for
+the information they choose to publish.
+
+The following types have already been registered for the <type> field :
+
+ #define PP2_TYPE_ALPN 0x01
+ #define PP2_TYPE_AUTHORITY 0x02
+ #define PP2_TYPE_CRC32C 0x03
+ #define PP2_TYPE_NOOP 0x04
+ #define PP2_TYPE_UNIQUE_ID 0x05
+ #define PP2_TYPE_SSL 0x20
+ #define PP2_SUBTYPE_SSL_VERSION 0x21
+ #define PP2_SUBTYPE_SSL_CN 0x22
+ #define PP2_SUBTYPE_SSL_CIPHER 0x23
+ #define PP2_SUBTYPE_SSL_SIG_ALG 0x24
+ #define PP2_SUBTYPE_SSL_KEY_ALG 0x25
+ #define PP2_TYPE_NETNS 0x30
+
+
+2.2.1 PP2_TYPE_ALPN
+
+Application-Layer Protocol Negotiation (ALPN). It is a byte sequence defining
+the upper layer protocol in use over the connection. The most common use case
+will be to pass the exact copy of the ALPN extension of the Transport Layer
+Security (TLS) protocol as defined by RFC7301 [9].
+
+
+2.2.2 PP2_TYPE_AUTHORITY
+
+Contains the host name value passed by the client, as an UTF8-encoded string.
+In case of TLS being used on the client connection, this is the exact copy of
+the "server_name" extension as defined by RFC3546 [10], section 3.1, often
+referred to as "SNI". There are probably other situations where an authority
+can be mentioned on a connection without TLS being involved at all.
+
+
+2.2.3. PP2_TYPE_CRC32C
+
+The value of the type PP2_TYPE_CRC32C is a 32-bit number storing the CRC32c
+checksum of the PROXY protocol header.
+
+When the checksum is supported by the sender after constructing the header
+the sender MUST:
+
+ - initialize the checksum field to '0's.
+
+ - calculate the CRC32c checksum of the PROXY header as described in RFC4960,
+ Appendix B [8].
+
+ - put the resultant value into the checksum field, and leave the rest of
+ the bits unchanged.
+
+If the checksum is provided as part of the PROXY header and the checksum
+functionality is supported by the receiver, the receiver MUST:
+
+ - store the received CRC32c checksum value aside.
+
+ - replace the 32 bits of the checksum field in the received PROXY header with
+ all '0's and calculate a CRC32c checksum value of the whole PROXY header.
+
+ - verify that the calculated CRC32c checksum is the same as the received
+ CRC32c checksum. If it is not, the receiver MUST treat the TCP connection
+ providing the header as invalid.
+
+The default procedure for handling an invalid TCP connection is to abort it.
+
+
+2.2.4. PP2_TYPE_NOOP
+
+The TLV of this type should be ignored when parsed. The value is zero or more
+bytes. Can be used for data padding or alignment. Note that it can be used
+to align only by 3 or more bytes because a TLV can not be smaller than that.
+
+
+2.2.5. PP2_TYPE_UNIQUE_ID
+
+The value of the type PP2_TYPE_UNIQUE_ID is an opaque byte sequence of up to
+128 bytes generated by the upstream proxy that uniquely identifies the
+connection.
+
+The unique ID can be used to easily correlate connections across multiple
+layers of proxies, without needing to look up IP addresses and port numbers.
+
+
+2.2.6. The PP2_TYPE_SSL type and subtypes
+
+For the type PP2_TYPE_SSL, the value is itself a defined like this :
+
+ struct pp2_tlv_ssl {
+ uint8_t client;
+ uint32_t verify;
+ struct pp2_tlv sub_tlv[0];
+ };
+
+The <verify> field will be zero if the client presented a certificate
+and it was successfully verified, and non-zero otherwise.
+
+The <client> field is made of a bit field from the following values,
+indicating which element is present :
+
+ #define PP2_CLIENT_SSL 0x01
+ #define PP2_CLIENT_CERT_CONN 0x02
+ #define PP2_CLIENT_CERT_SESS 0x04
+
+Note, that each of these elements may lead to extra data being appended to
+this TLV using a second level of TLV encapsulation. It is thus possible to
+find multiple TLV values after this field. The total length of the pp2_tlv_ssl
+TLV will reflect this.
+
+The PP2_CLIENT_SSL flag indicates that the client connected over SSL/TLS. When
+this field is present, the US-ASCII string representation of the TLS version is
+appended at the end of the field in the TLV format using the type
+PP2_SUBTYPE_SSL_VERSION.
+
+PP2_CLIENT_CERT_CONN indicates that the client provided a certificate over the
+current connection. PP2_CLIENT_CERT_SESS indicates that the client provided a
+certificate at least once over the TLS session this connection belongs to.
+
+The second level TLV PP2_SUBTYPE_SSL_CIPHER provides the US-ASCII string name
+of the used cipher, for example "ECDHE-RSA-AES128-GCM-SHA256".
+
+The second level TLV PP2_SUBTYPE_SSL_SIG_ALG provides the US-ASCII string name
+of the algorithm used to sign the certificate presented by the frontend when
+the incoming connection was made over an SSL/TLS transport layer, for example
+"SHA256".
+
+The second level TLV PP2_SUBTYPE_SSL_KEY_ALG provides the US-ASCII string name
+of the algorithm used to generate the key of the certificate presented by the
+frontend when the incoming connection was made over an SSL/TLS transport layer,
+for example "RSA2048".
+
+In all cases, the string representation (in UTF8) of the Common Name field
+(OID: 2.5.4.3) of the client certificate's Distinguished Name, is appended
+using the TLV format and the type PP2_SUBTYPE_SSL_CN. E.g. "example.com".
+
+
+2.2.7. The PP2_TYPE_NETNS type
+
+The type PP2_TYPE_NETNS defines the value as the US-ASCII string representation
+of the namespace's name.
+
+
+2.2.8. Reserved type ranges
+
+The following range of 16 type values is reserved for application-specific
+data and will be never used by the PROXY Protocol. If you need more values
+consider extending the range with a type field in your TLVs.
+
+ #define PP2_TYPE_MIN_CUSTOM 0xE0
+ #define PP2_TYPE_MAX_CUSTOM 0xEF
+
+This range of 8 values is reserved for temporary experimental use by
+application developers and protocol designers. The values from the range will
+never be used by the PROXY protocol and should not be used by production
+functionality.
+
+ #define PP2_TYPE_MIN_EXPERIMENT 0xF0
+ #define PP2_TYPE_MAX_EXPERIMENT 0xF7
+
+The following range of 8 values is reserved for future use, potentially to
+extend the protocol with multibyte type values.
+
+ #define PP2_TYPE_MIN_FUTURE 0xF8
+ #define PP2_TYPE_MAX_FUTURE 0xFF
+
+
+3. Implementations
+
+HAProxy 1.5 implements version 1 of the PROXY protocol on both sides :
+ - the listening sockets accept the protocol when the "accept-proxy" setting
+ is passed to the "bind" keyword. Connections accepted on such listeners
+ will behave just as if the source really was the one advertised in the
+ protocol. This is true for logging, ACLs, content filtering, transparent
+ proxying, etc...
+
+ - the protocol may be used to connect to servers if the "send-proxy" setting
+ is present on the "server" line. It is enabled on a per-server basis, so it
+ is possible to have it enabled for remote servers only and still have local
+ ones behave differently. If the incoming connection was accepted with the
+ "accept-proxy", then the relayed information is the one advertised in this
+ connection's PROXY line.
+
+ - HAProxy 1.5 also implements version 2 of the PROXY protocol as a sender. In
+ addition, a TLV with limited, optional, SSL information has been added.
+
+Stunnel added support for version 1 of the protocol for outgoing connections in
+version 4.45.
+
+Stud added support for version 1 of the protocol for outgoing connections on
+2011/06/29.
+
+Postfix added support for version 1 of the protocol for incoming connections
+in smtpd and postscreen in version 2.10.
+
+A patch is available for Stud[5] to implement version 1 of the protocol on
+incoming connections.
+
+Support for versions 1 and 2 of the protocol was added to Varnish 4.1 [6].
+
+Exim added support for version 1 and version 2 of the protocol for incoming
+connections on 2014/05/13, and will be released as part of version 4.83.
+
+Squid added support for versions 1 and 2 of the protocol in version 3.5 [7].
+
+Jetty 9.3.0 supports protocol version 1.
+
+lighttpd added support for versions 1 and 2 of the protocol for incoming
+connections in version 1.4.46 [11].
+
+The protocol is simple enough that it is expected that other implementations
+will appear, especially in environments such as SMTP, IMAP, FTP, RDP where the
+client's address is an important piece of information for the server and some
+intermediaries. In fact, several proprietary deployments have already done so
+on FTP and SMTP servers.
+
+Proxy developers are encouraged to implement this protocol, because it will
+make their products much more transparent in complex infrastructures, and will
+get rid of a number of issues related to logging and access control.
+
+
+4. Architectural benefits
+4.1. Multiple layers
+
+Using the PROXY protocol instead of transparent proxy provides several benefits
+in multiple-layer infrastructures. The first immediate benefit is that it
+becomes possible to chain multiple layers of proxies and always present the
+original IP address. for instance, let's consider the following 2-layer proxy
+architecture :
+
+ Internet
+ ,---. | client to PX1:
+ ( X ) | native protocol
+ `---' |
+ | V
+ +--+--+ +-----+
+ | FW1 |------| PX1 |
+ +--+--+ +-----+ | PX1 to PX2: PROXY + native
+ | V
+ +--+--+ +-----+
+ | FW2 |------| PX2 |
+ +--+--+ +-----+ | PX2 to SRV: PROXY + native
+ | V
+ +--+--+
+ | SRV |
+ +-----+
+
+Firewall FW1 receives traffic from internet-based clients and forwards it to
+reverse-proxy PX1. PX1 adds a PROXY header then forwards to PX2 via FW2. PX2
+is configured to read the PROXY header and to emit it on output. It then joins
+the origin server SRV and presents the original client's address there. Since
+all TCP connections endpoints are real machines and are not spoofed, there is
+no issue for the return traffic to pass via the firewalls and reverse proxies.
+Using transparent proxy, this would be quite difficult because the firewalls
+would have to deal with the client's address coming from the proxies in the DMZ
+and would have to correctly route the return traffic there instead of using the
+default route.
+
+
+4.2. IPv4 and IPv6 integration
+
+The protocol also eases IPv4 and IPv6 integration : if only the first layer
+(FW1 and PX1) is IPv6-capable, it is still possible to present the original
+client's IPv6 address to the target server even though the whole chain is only
+connected via IPv4.
+
+
+4.3. Multiple return paths
+
+When transparent proxy is used, it is not possible to run multiple proxies
+because the return traffic would follow the default route instead of finding
+the proper proxy. Some tricks are sometimes possible using multiple server
+addresses and policy routing but these are very limited.
+
+Using the PROXY protocol, this problem disappears as the servers don't need
+to route to the client, just to the proxy that forwarded the connection. So
+it is perfectly possible to run a proxy farm in front of a very large server
+farm and have it working effortless, even when dealing with multiple sites.
+
+This is particularly important in Cloud-like environments where there is little
+choice of binding to random addresses and where the lower processing power per
+node generally requires multiple front nodes.
+
+The example below illustrates the following case : virtualized infrastructures
+are deployed in 3 datacenters (DC1..DC3). Each DC uses its own VIP which is
+handled by the hosting provider's layer 3 load balancer. This load balancer
+routes the traffic to a farm of layer 7 SSL/cache offloaders which load balance
+among their local servers. The VIPs are advertised by geolocalised DNS so that
+clients generally stick to a given DC. Since clients are not guaranteed to
+stick to one DC, the L7 load balancing proxies have to know the other DCs'
+servers that may be reached via the hosting provider's LAN or via the internet.
+The L7 proxies use the PROXY protocol to join the servers behind them, so that
+even inter-DC traffic can forward the original client's address and the return
+path is unambiguous. This would not be possible using transparent proxy because
+most often the L7 proxies would not be able to spoof an address, and this would
+never work between datacenters.
+
+ Internet
+
+ DC1 DC2 DC3
+ ,---. ,---. ,---.
+ ( X ) ( X ) ( X )
+ `---' `---' `---'
+ | +-------+ | +-------+ | +-------+
+ +----| L3 LB | +----| L3 LB | +----| L3 LB |
+ | +-------+ | +-------+ | +-------+
+ ------+------- ~ ~ ~ ------+------- ~ ~ ~ ------+-------
+ ||||| |||| ||||| |||| ||||| ||||
+ 50 SRV 4 PX 50 SRV 4 PX 50 SRV 4 PX
+
+
+5. Security considerations
+
+Version 1 of the protocol header (the human-readable format) was designed so as
+to be distinguishable from HTTP. It will not parse as a valid HTTP request and
+an HTTP request will not parse as a valid proxy request. Version 2 add to use a
+non-parsable binary signature to make many products fail on this block. The
+signature was designed to cause immediate failure on HTTP, SSL/TLS, SMTP, FTP,
+and POP. It also causes aborts on LDAP and RDP servers (see section 6). That
+makes it easier to enforce its use under certain connections and at the same
+time, it ensures that improperly configured servers are quickly detected.
+
+Implementers should be very careful about not trying to automatically detect
+whether they have to decode the header or not, but rather they must only rely
+on a configuration parameter. Indeed, if the opportunity is left to a normal
+client to use the protocol, it will be able to hide its activities or make them
+appear as coming from somewhere else. However, accepting the header only from a
+number of known sources should be safe.
+
+
+6. Validation
+
+The version 2 protocol signature has been sent to a wide variety of protocols
+and implementations including old ones. The following protocol and products
+have been tested to ensure the best possible behavior when the signature was
+presented, even with minimal implementations :
+
+ - HTTP :
+ - Apache 1.3.33 : connection abort => pass/optimal
+ - Nginx 0.7.69 : 400 Bad Request + abort => pass/optimal
+ - lighttpd 1.4.20 : 400 Bad Request + abort => pass/optimal
+ - thttpd 2.20c : 400 Bad Request + abort => pass/optimal
+ - mini-httpd-1.19 : 400 Bad Request + abort => pass/optimal
+ - haproxy 1.4.21 : 400 Bad Request + abort => pass/optimal
+ - Squid 3 : 400 Bad Request + abort => pass/optimal
+ - SSL :
+ - stud 0.3.47 : connection abort => pass/optimal
+ - stunnel 4.45 : connection abort => pass/optimal
+ - nginx 0.7.69 : 400 Bad Request + abort => pass/optimal
+ - FTP :
+ - Pure-ftpd 1.0.20 : 3*500 then 221 Goodbye => pass/optimal
+ - vsftpd 2.0.1 : 3*530 then 221 Goodbye => pass/optimal
+ - SMTP :
+ - postfix 2.3 : 3*500 + 221 Bye => pass/optimal
+ - exim 4.69 : 554 + connection abort => pass/optimal
+ - POP :
+ - dovecot 1.0.10 : 3*ERR + Logout => pass/optimal
+ - IMAP :
+ - dovecot 1.0.10 : 5*ERR + hang => pass/non-optimal
+ - LDAP :
+ - openldap 2.3 : abort => pass/optimal
+ - SSH :
+ - openssh 3.9p1 : abort => pass/optimal
+ - RDP :
+ - Windows XP SP3 : abort => pass/optimal
+
+This means that most protocols and implementations will not be confused by an
+incoming connection exhibiting the protocol signature, which avoids issues when
+facing misconfigurations.
+
+
+7. Future developments
+
+It is possible that the protocol may slightly evolve to present other
+information such as the incoming network interface, or the origin addresses in
+case of network address translation happening before the first proxy, but this
+is not identified as a requirement right now. Some deep thinking has been spent
+on this and it appears that trying to add a few more information open a Pandora
+box with many information from MAC addresses to SSL client certificates, which
+would make the protocol much more complex. So at this point it is not planned.
+Suggestions on improvements are welcome.
+
+
+8. Contacts and links
+
+Please use w@1wt.eu to send any comments to the author.
+
+The following links were referenced in the document.
+
+[1] http://www.postfix.org/XCLIENT_README.html
+[2] http://tools.ietf.org/html/rfc7239
+[3] http://www.stunnel.org/
+[4] https://github.com/bumptech/stud
+[5] https://github.com/bumptech/stud/pull/81
+[6] https://www.varnish-cache.org/docs/trunk/phk/ssl_again.html
+[7] http://wiki.squid-cache.org/Squid-3.5
+[8] https://tools.ietf.org/html/rfc4960#appendix-B
+[9] https://tools.ietf.org/rfc/rfc7301.txt
+[10] https://www.ietf.org/rfc/rfc3546.txt
+[11] https://redmine.lighttpd.net/issues/2804
+
+9. Sample code
+
+The code below is an example of how a receiver may deal with both versions of
+the protocol header for TCP over IPv4 or IPv6. The function is supposed to be
+called upon a read event. Addresses may be directly copied into their final
+memory location since they're transported in network byte order. The sending
+side is even simpler and can easily be deduced from this sample code.
+
+ struct sockaddr_storage from; /* already filled by accept() */
+ struct sockaddr_storage to; /* already filled by getsockname() */
+ const char v2sig[12] = "\x0D\x0A\x0D\x0A\x00\x0D\x0A\x51\x55\x49\x54\x0A";
+
+ /* returns 0 if needs to poll, <0 upon error or >0 if it did the job */
+ int read_evt(int fd)
+ {
+ union {
+ struct {
+ char line[108];
+ } v1;
+ struct {
+ uint8_t sig[12];
+ uint8_t ver_cmd;
+ uint8_t fam;
+ uint16_t len;
+ union {
+ struct { /* for TCP/UDP over IPv4, len = 12 */
+ uint32_t src_addr;
+ uint32_t dst_addr;
+ uint16_t src_port;
+ uint16_t dst_port;
+ } ip4;
+ struct { /* for TCP/UDP over IPv6, len = 36 */
+ uint8_t src_addr[16];
+ uint8_t dst_addr[16];
+ uint16_t src_port;
+ uint16_t dst_port;
+ } ip6;
+ struct { /* for AF_UNIX sockets, len = 216 */
+ uint8_t src_addr[108];
+ uint8_t dst_addr[108];
+ } unx;
+ } addr;
+ } v2;
+ } hdr;
+
+ int size, ret;
+
+ do {
+ ret = recv(fd, &hdr, sizeof(hdr), MSG_PEEK);
+ } while (ret == -1 && errno == EINTR);
+
+ if (ret == -1)
+ return (errno == EAGAIN) ? 0 : -1;
+
+ if (ret >= 16 && memcmp(&hdr.v2, v2sig, 12) == 0 &&
+ (hdr.v2.ver_cmd & 0xF0) == 0x20) {
+ size = 16 + ntohs(hdr.v2.len);
+ if (ret < size)
+ return -1; /* truncated or too large header */
+
+ switch (hdr.v2.ver_cmd & 0xF) {
+ case 0x01: /* PROXY command */
+ switch (hdr.v2.fam) {
+ case 0x11: /* TCPv4 */
+ ((struct sockaddr_in *)&from)->sin_family = AF_INET;
+ ((struct sockaddr_in *)&from)->sin_addr.s_addr =
+ hdr.v2.addr.ip4.src_addr;
+ ((struct sockaddr_in *)&from)->sin_port =
+ hdr.v2.addr.ip4.src_port;
+ ((struct sockaddr_in *)&to)->sin_family = AF_INET;
+ ((struct sockaddr_in *)&to)->sin_addr.s_addr =
+ hdr.v2.addr.ip4.dst_addr;
+ ((struct sockaddr_in *)&to)->sin_port =
+ hdr.v2.addr.ip4.dst_port;
+ goto done;
+ case 0x21: /* TCPv6 */
+ ((struct sockaddr_in6 *)&from)->sin6_family = AF_INET6;
+ memcpy(&((struct sockaddr_in6 *)&from)->sin6_addr,
+ hdr.v2.addr.ip6.src_addr, 16);
+ ((struct sockaddr_in6 *)&from)->sin6_port =
+ hdr.v2.addr.ip6.src_port;
+ ((struct sockaddr_in6 *)&to)->sin6_family = AF_INET6;
+ memcpy(&((struct sockaddr_in6 *)&to)->sin6_addr,
+ hdr.v2.addr.ip6.dst_addr, 16);
+ ((struct sockaddr_in6 *)&to)->sin6_port =
+ hdr.v2.addr.ip6.dst_port;
+ goto done;
+ }
+ /* unsupported protocol, keep local connection address */
+ break;
+ case 0x00: /* LOCAL command */
+ /* keep local connection address for LOCAL */
+ break;
+ default:
+ return -1; /* not a supported command */
+ }
+ }
+ else if (ret >= 8 && memcmp(hdr.v1.line, "PROXY", 5) == 0) {
+ char *end = memchr(hdr.v1.line, '\r', ret - 1);
+ if (!end || end[1] != '\n')
+ return -1; /* partial or invalid header */
+ *end = '\0'; /* terminate the string to ease parsing */
+ size = end + 2 - hdr.v1.line; /* skip header + CRLF */
+ /* parse the V1 header using favorite address parsers like inet_pton.
+ * return -1 upon error, or simply fall through to accept.
+ */
+ }
+ else {
+ /* Wrong protocol */
+ return -1;
+ }
+
+ done:
+ /* we need to consume the appropriate amount of data from the socket */
+ do {
+ ret = recv(fd, &hdr, size, 0);
+ } while (ret == -1 && errno == EINTR);
+ return (ret >= 0) ? 1 : -1;
+ }
diff --git a/doc/queuing.fig b/doc/queuing.fig
new file mode 100644
index 0000000..8d57504
--- /dev/null
+++ b/doc/queuing.fig
@@ -0,0 +1,192 @@
+#FIG 3.2
+Portrait
+Center
+Metric
+A4
+100.00
+Single
+-2
+1200 2
+6 900 4770 1575 5220
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 3
+ 900 4770 1125 4995 1125 5220
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 3
+ 1575 4770 1350 4995 1350 5220
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 1170 4995 1170 5220
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 1215 4995 1215 5220
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 1260 4995 1260 5220
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 1305 4995 1305 5220
+2 3 0 1 7 7 52 -1 20 0.000 2 0 -1 0 0 7
+ 900 4770 1125 4995 1125 5220 1350 5220 1350 4995 1575 4770
+ 900 4770
+-6
+6 2250 4770 2925 5220
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 3
+ 2250 4770 2475 4995 2475 5220
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 3
+ 2925 4770 2700 4995 2700 5220
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 2520 4995 2520 5220
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 2565 4995 2565 5220
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 2610 4995 2610 5220
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 2655 4995 2655 5220
+2 3 0 1 7 7 52 -1 20 0.000 2 0 -1 0 0 7
+ 2250 4770 2475 4995 2475 5220 2700 5220 2700 4995 2925 4770
+ 2250 4770
+-6
+6 1710 3420 2115 3870
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 1710 3780 2115 3780
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 1710 3825 2115 3825
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 1710 3735 2115 3735
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 1710 3690 2115 3690
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 1710 3645 2115 3645
+2 1 0 1 0 6 51 -1 20 0.000 0 0 -1 0 0 4
+ 1710 3420 1710 3870 2115 3870 2115 3420
+-6
+1 2 0 1 0 7 51 -1 20 0.000 1 0.0000 1935 2182 450 113 1485 2182 2385 2182
+1 2 0 1 0 7 51 -1 20 0.000 0 0.0000 2790 3082 450 113 2340 3082 3240 3082
+1 2 0 1 0 7 51 -1 20 0.000 1 0.0000 1935 1367 450 113 1485 1367 2385 1367
+1 2 0 1 0 7 51 -1 20 0.000 1 0.0000 1035 3082 450 113 585 3082 1485 3082
+2 1 2 1 0 2 53 -1 -1 3.000 0 0 -1 0 0 2
+ 2745 3870 3015 3870
+2 1 2 1 0 2 53 -1 -1 3.000 0 0 -1 0 0 2
+ 2745 4320 3015 4320
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 60.00
+ 2970 5085 2745 5085
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 60.00
+ 2205 5085 2430 5085
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 60.00
+ 1620 5085 1395 5085
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 60.00
+ 855 5085 1080 5085
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 3
+ 0 0 1.00 60.00 60.00
+ 1890 3870 1440 4320 1440 4770
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 3
+ 0 0 1.00 60.00 60.00
+ 1935 3870 2385 4320 2385 4770
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 0 0 1.00 60.00 60.00
+ 2610 4320 2610 4770
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 0 0 1.00 60.00 60.00
+ 2835 3195 2835 4770
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 3
+ 0 0 1.00 60.00 60.00
+ 2745 3195 2610 3330 2610 3870
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 0 0 1.00 60.00 60.00
+ 1935 2295 1935 3420
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 3
+ 0 0 1.00 60.00 60.00
+ 1080 3195 1215 3330 1215 3870
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 3
+ 0 0 1.00 60.00 60.00
+ 1890 2295 1035 2745 1035 2970
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 3
+ 0 0 1.00 60.00 60.00
+ 1980 2295 2790 2745 2790 2970
+2 1 0 1 0 2 50 -1 -1 0.000 0 0 -1 1 0 2
+ 0 0 1.00 60.00 60.00
+ 1935 1485 1935 2070
+2 1 1 1 0 2 50 -1 -1 4.000 0 0 -1 1 0 5
+ 0 0 1.00 60.00 60.00
+ 810 5220 450 5220 450 2160 1080 2160 1485 2160
+2 1 1 1 0 2 50 -1 -1 4.000 0 0 -1 1 0 5
+ 0 0 1.00 60.00 60.00
+ 3060 5220 3375 5220 3375 2160 2655 2160 2385 2160
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 0 0 1.00 60.00 60.00
+ 1215 4320 1215 4770
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 0 0 1.00 60.00 60.00
+ 990 3195 990 4770
+2 1 0 1 0 2 50 -1 -1 0.000 0 0 -1 1 0 2
+ 0 0 1.00 60.00 60.00
+ 1935 855 1935 1260
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 3
+ 0 0 1.00 60.00 60.00
+ 1620 1440 900 2025 900 2970
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 3
+ 0 0 1.00 60.00 60.00
+ 2205 1440 2925 2025 2925 2970
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 1125 4230 1350 4230
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 1125 4275 1350 4275
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 1125 4185 1350 4185
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 1125 4140 1350 4140
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 1125 4095 1350 4095
+2 1 0 1 0 3 51 -1 20 0.000 0 0 -1 0 0 4
+ 1125 3870 1125 4320 1350 4320 1350 3870
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 2700 4230 2475 4230
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 2700 4275 2475 4275
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 2700 4185 2475 4185
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 2700 4140 2475 4140
+2 1 0 1 0 3 51 -1 20 0.000 0 0 -1 0 0 4
+ 2700 3870 2700 4320 2475 4320 2475 3870
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 1125 4050 1350 4050
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 1125 4005 1350 4005
+2 1 0 1 0 2 53 -1 -1 0.000 0 0 -1 1 1 2
+ 0 0 1.00 60.00 60.00
+ 0 0 1.00 60.00 60.00
+ 900 3870 900 4320
+2 1 2 1 0 2 53 -1 -1 3.000 0 0 -1 0 0 2
+ 855 3870 1125 3870
+2 1 2 1 0 2 53 -1 -1 3.000 0 0 -1 0 0 2
+ 855 4320 1125 4320
+2 1 0 1 0 2 53 -1 -1 0.000 0 0 -1 1 1 2
+ 0 0 1.00 60.00 60.00
+ 0 0 1.00 60.00 60.00
+ 2970 3870 2970 4320
+4 0 0 53 -1 16 7 0.0000 4 75 195 1260 3510 Yes\001
+4 2 0 53 -1 16 7 0.0000 4 75 135 945 3510 No\001
+4 2 0 53 -1 16 7 0.0000 4 75 195 2565 3510 Yes\001
+4 0 0 53 -1 16 7 0.0000 4 75 135 2880 3510 No\001
+4 1 0 50 -1 16 6 0.0000 4 75 210 1935 4140 global\001
+4 1 0 50 -1 16 6 0.0000 4 60 225 1935 4230 queue\001
+4 1 0 50 -1 16 8 1.5708 4 120 1005 405 4680 Redispatch on error\001
+4 1 0 53 -1 14 6 1.5708 4 60 480 2205 3645 maxqueue\001
+4 1 0 50 -1 18 8 0.0000 4 90 165 1935 2205 LB\001
+4 1 0 53 -1 16 7 1.5708 4 90 870 2070 2880 server, all are full.\001
+4 1 0 50 -1 18 8 0.0000 4 90 360 1935 1395 cookie\001
+4 1 0 50 -1 16 10 0.0000 4 135 1200 1935 765 Incoming request\001
+4 1 0 53 -1 16 7 1.5708 4 75 480 1890 1755 no cookie\001
+4 1 0 53 -1 16 7 1.5708 4 75 600 1890 2880 no available\001
+4 0 0 53 -1 16 7 5.6200 4 75 735 2340 1530 SRV2 selected\001
+4 1 0 50 -1 16 10 0.0000 4 105 405 1260 5445 SRV1\001
+4 1 0 50 -1 16 10 0.0000 4 105 405 2610 5445 SRV2\001
+4 2 0 53 -1 16 7 0.4712 4 75 735 1665 2385 SRV1 selected\001
+4 0 0 53 -1 16 7 5.8119 4 75 735 2205 2385 SRV2 selected\001
+4 2 0 53 -1 16 7 0.6632 4 75 735 1485 1530 SRV1 selected\001
+4 0 0 53 -1 14 6 0.0000 4 45 420 2880 5040 maxconn\001
+4 1 0 50 -1 16 8 1.5708 4 120 1005 3510 4680 Redispatch on error\001
+4 1 0 50 -1 18 8 0.0000 4 90 615 2790 3105 SRV2 full ?\001
+4 1 0 50 -1 18 8 0.0000 4 90 615 1035 3105 SRV1 full ?\001
+4 1 0 53 -1 14 6 1.5708 4 60 480 855 4095 maxqueue\001
+4 1 0 53 -1 14 6 1.5708 4 60 480 3105 4095 maxqueue\001
diff --git a/doc/regression-testing.txt b/doc/regression-testing.txt
new file mode 100644
index 0000000..5b070a4
--- /dev/null
+++ b/doc/regression-testing.txt
@@ -0,0 +1,706 @@
+ +---------------------------------------+
+ | HAProxy regression testing with vtest |
+ +---------------------------------------+
+
+
+The information found in this file are a short starting guide to help you to
+write VTC (Varnish Test Case) scripts (or VTC files) for haproxy regression testing.
+Such VTC files are currently used to test Varnish cache application developed by
+Poul-Henning Kamp. A very big thanks you to him for having helped you to add
+our haproxy C modules to vtest tool. Note that vtest was formally developed for
+varnish cache reg testing and was named varnishtest. vtest is an haproxy specific
+version of varnishtest program which reuses the non varnish cache specific code.
+
+A lot of general information about how to write VTC files may be found in 'man/vtc.7'
+manual of varnish cache sources directory or directly on the web here:
+
+ https://varnish-cache.org/docs/trunk/reference/vtc.html
+
+It is *highly* recommended to read this manual before asking to haproxy ML. This
+documentation only deals with the vtest support for haproxy.
+
+
+vtest installation
+------------------------
+
+To use vtest you will have to download and compile the recent vtest
+sources found at https://github.com/vtest/VTest.
+
+To compile vtest:
+
+ $ cd VTest
+ $ make vtest
+
+Note that varnishtest may be also compiled but not without the varnish cache
+sources already compiled:
+
+ $ VARNISH_SRC=<...> make varnishtest
+
+After having compiled these sources, the vtest executable location is at the
+root of the vtest sources directory.
+
+
+vtest execution
+---------------------
+
+vtest is able to search for the haproxy executable file it is supposed to
+launch thanks to the PATH environment variable. To force the executable to be used by
+vtest, the HAPROXY_PROGRAM environment variable for vtest may be
+typically set as follows:
+
+ $ HAPROXY_PROGRAM=~/srcs/haproxy/haproxy vtest ...
+
+vtest program comes with interesting options. The most interesting are:
+
+ -t Timeout in seconds to abort the test if some launched program
+ -v By default, vtest does not dump the outputs of process it launched
+ when the test passes. With this option the outputs are dumped even
+ when the test passes.
+ -L to always keep the temporary VTC directories.
+ -l to keep the temporary VTC directories only when the test fails.
+
+About haproxy, when launched by vtest, -d option is enabled by default.
+
+
+How to write VTC files
+----------------------
+
+A VTC file must start with a "varnishtest" or "vtest" command line followed by a
+descriptive line enclosed by double quotes. This is not specific to the VTC files
+for haproxy.
+
+The VTC files for haproxy must also contain a "feature ignore_unknown_macro" line
+if any macro is used for haproxy in this file. This is due to the fact that
+vtest parser code for haproxy commands generates macros the vtest
+parser code for varnish cache has no knowledge of. This line prevents vtest from
+failing in such cases. As a "cli" macro automatically generated, this
+"feature ignore_unknown_macro" is mandatory for each VTC file for haproxy.
+
+To make vtest capable of testing haproxy, two new VTC commands have been
+implemented: "haproxy" and "syslog". "haproxy" is used to start haproxy process.
+"syslog" is used to start syslog servers (at this time, only used by haproxy).
+
+As haproxy cannot work without configuration file, a VTC file for haproxy must
+embed the configuration files contents for the haproxy instances it declares.
+This may be done using the following intuitive syntax construction: -conf {...}.
+Here -conf is an argument of "haproxy" VTC command to declare the configuration
+file of the haproxy instances it also declares (see "Basic HAProxy test" VTC file
+below).
+
+As for varnish VTC files, the parser of VTC files for haproxy automatically
+generates macros for the declared frontends to be reused by the clients later
+in the script, so after having written the "haproxy" command sections.
+The syntax "fd@${my_frontend_fd_name}" must be used to bind the frontend
+listeners to localhost address and random ports (see "Environment variables"
+section of haproxy documentation). This is mandatory.
+
+Each time the haproxy command parser finds a "fd@${xyz}" string in a 'ABC'
+"haproxy" command section, it generates three macros: 'ABC_xyz_addr', 'ABC_xyz_port'
+and 'ABC_xyz_sock', with 'ABC_xyz_sock' being resolved as 'ABC_xyz_addr
+ABC_xyz_port' typically used by clients -connect parameter.
+
+Each haproxy instance works in their own temporary working directories located
+at '/tmp/vtc.<varnitest PID>.XXXXXXXX/<haproxy_instance_name>' (with XXXXXXXX
+a random 8 digits hexadecimal integer. This is in this temporary directory that
+the configuration file is temporarily written.
+
+A 'stats.sock' UNIX socket is also created in this directory. There is no need
+to declare such stats sockets in the -conf {...} section. The name of the parent
+directory of the haproxy instances working directories is stored in 'tmpdir'. In
+fact this is the working directory of the current vtest process.
+
+There also exists 'testdir' macro which is the parent directory of the VTC file.
+It may be useful to use other files located in the same directory than the current
+VTC file.
+
+
+
+VTC file examples
+-----------------
+
+The following first VTC file is a real regression test case file for a bug which has
+been fixed by 84c844e commit. We declare a basic configuration for a 'h1' haproxy
+instance.
+
+ varnishtest "SPOE bug: missing configuration file"
+
+ #commit 84c844eb12b250aa86f2aadaff77c42dfc3cb619
+ #Author: Christopher Faulet <cfaulet@haproxy.com>
+ #Date: Fri Mar 23 14:37:14 2018 +0100
+
+ # BUG/MINOR: spoe: Initialize variables used during conf parsing before any check
+
+ # Some initializations must be done at the beginning of parse_spoe_flt to avoid
+ # segmentation fault when first errors are caught, when the "filter spoe" line is
+ # parsed.
+
+ haproxy h1 -conf-BAD {} {
+ defaults
+ timeout connect 5000ms
+ timeout client 50000ms
+ timeout server 50000ms
+
+ frontend my-front
+ filter spoe
+ }
+
+
+-conf-BAD haproxy command argument is used. Its role it to launch haproxy with
+-c option (configuration file checking) and check that 'h1' exit(3) with 1 as
+status. Here is the output when running this VTC file:
+
+
+ **** top 0.0 extmacro def pwd=/home/fred/src/haproxy
+ **** top 0.0 extmacro def localhost=127.0.0.1
+ **** top 0.0 extmacro def bad_backend=127.0.0.1 39564
+ **** top 0.0 extmacro def bad_ip=192.0.2.255
+ **** top 0.0 macro def testdir=//home/fred/src/varnish-cache-haproxy
+ **** top 0.0 macro def tmpdir=/tmp/vtc.6377.64329194
+ * top 0.0 TEST /home/fred/src/varnish-cache-haproxy/spoe_bug.vtc starting
+ ** top 0.0 === varnishtest "SPOE bug: missing configuration file"
+ * top 0.0 TEST SPOE bug: missing configuration file
+ ** top 0.0 === haproxy h1 -conf-BAD {} {
+ **** h1 0.0 conf| global
+ **** h1 0.0 conf|\tstats socket /tmp/vtc.6377.64329194/h1/stats.sock level admin mode 600
+ **** h1 0.0 conf|
+ **** h1 0.0 conf|\tdefaults
+ **** h1 0.0 conf| timeout connect 5000ms
+ **** h1 0.0 conf| timeout client 50000ms
+ **** h1 0.0 conf| timeout server 50000ms
+ **** h1 0.0 conf|
+ **** h1 0.0 conf|\tfrontend my-front
+ **** h1 0.0 conf|\t\tfilter spoe
+ **** h1 0.0 conf|
+ ** h1 0.0 haproxy_start
+ **** h1 0.0 opt_worker 0 opt_daemon 0 opt_check_mode 1
+ **** h1 0.0 argv|exec /home/fred/src/haproxy/haproxy -c -f /tmp/vtc.6377.64329194/h1/cfg
+ **** h1 0.0 XXX 5 @277
+ *** h1 0.0 PID: 6395
+ **** h1 0.0 macro def h1_pid=6395
+ **** h1 0.0 macro def h1_name=/tmp/vtc.6377.64329194/h1
+ ** h1 0.0 Wait
+ ** h1 0.0 Stop HAProxy pid=6395
+ **** h1 0.0 STDOUT poll 0x10
+ ** h1 0.0 WAIT4 pid=6395 status=0x008b (user 0.000000 sys 0.000000)
+ * h1 0.0 Expected exit: 0x1 signal: 0 core: 0
+ ---- h1 0.0 Bad exit status: 0x008b exit 0x0 signal 11 core 128
+ * top 0.0 RESETTING after /home/fred/src/varnish-cache-haproxy/spoe_bug.vtc
+ ** h1 0.0 Reset and free h1 haproxy 6395
+ ** h1 0.0 Wait
+ ---- h1 0.0 Assert error in haproxy_wait(), vtc_haproxy.c line 326: Condition(*(&h->fds[1]) >= 0) not true.
+
+ * top 0.0 failure during reset
+ # top TEST /home/fred/src/varnish-cache-haproxy/spoe_bug.vtc FAILED (0.008) exit=2
+
+
+'h1' exited with (128 + 11) status and a core file was produced in
+/tmp/vtc.6377.64329194/h1 directory.
+With the patch provided by 84c844e commit, varnishtest makes this VTC file pass
+as expected (verbose mode execution):
+
+ **** top 0.0 extmacro def pwd=/home/fred/src/haproxy
+ **** top 0.0 extmacro def localhost=127.0.0.1
+ **** top 0.0 extmacro def bad_backend=127.0.0.1 42264
+ **** top 0.0 extmacro def bad_ip=192.0.2.255
+ **** top 0.0 macro def testdir=//home/fred/src/varnish-cache-haproxy
+ **** top 0.0 macro def tmpdir=/tmp/vtc.25540.59b6ec5d
+ * top 0.0 TEST /home/fred/src/varnish-cache-haproxy/spoe_bug.vtc starting
+ ** top 0.0 === varnishtest "SPOE bug: missing configuration file"
+ * top 0.0 TEST SPOE bug: missing configuration file
+ ** top 0.0 === haproxy h1 -conf-BAD {} {
+ **** h1 0.0 conf| global
+ **** h1 0.0 conf|\tstats socket /tmp/vtc.25540.59b6ec5d/h1/stats.sock level admin mode 600
+ **** h1 0.0 conf|
+ **** h1 0.0 conf|\tdefaults
+ **** h1 0.0 conf| timeout connect 5000ms
+ **** h1 0.0 conf| timeout client 50000ms
+ **** h1 0.0 conf| timeout server 50000ms
+ **** h1 0.0 conf|
+ **** h1 0.0 conf|\tfrontend my-front
+ **** h1 0.0 conf|\t\tfilter spoe
+ **** h1 0.0 conf|
+ ** h1 0.0 haproxy_start
+ **** h1 0.0 opt_worker 0 opt_daemon 0 opt_check_mode 1
+ **** h1 0.0 argv|exec /home/fred/src/haproxy/haproxy -c -f /tmp/vtc.25540.59b6ec5d/h1/cfg
+ **** h1 0.0 XXX 5 @277
+ *** h1 0.0 PID: 25558
+ **** h1 0.0 macro def h1_pid=25558
+ **** h1 0.0 macro def h1_name=/tmp/vtc.25540.59b6ec5d/h1
+ ** h1 0.0 Wait
+ ** h1 0.0 Stop HAProxy pid=25558
+ *** h1 0.0 debug|[ALERT] (25558) : parsing [/tmp/vtc.25540.59b6ec5d/h1/cfg:10] : 'filter' : ''spoe' : missing config file'
+ *** h1 0.0 debug|[ALERT] (25558) : Error(s) found in configuration file : /tmp/vtc.25540.59b6ec5d/h1/cfg
+ *** h1 0.0 debug|[ALERT] (25558) : Fatal errors found in configuration.
+ **** h1 0.0 STDOUT poll 0x10
+ ** h1 0.0 WAIT4 pid=25558 status=0x0100 (user 0.000000 sys 0.000000)
+ ** h1 0.0 Found expected ''
+ * top 0.0 RESETTING after /home/fred/src/varnish-cache-haproxy/spoe_bug.vtc
+ ** h1 0.0 Reset and free h1 haproxy -1
+ * top 0.0 TEST /home/fred/src/varnish-cache-haproxy/spoe_bug.vtc completed
+ # top TEST /home/fred/src/varnish-cache-haproxy/spoe_bug.vtc passed (0.004)
+
+
+The following VTC file does almost nothing except running a shell to list
+the contents of 'tmpdir' directory after having launched a haproxy instance
+and 's1' HTTP server. This shell also prints the content of 'cfg' 'h1' configuration
+file.
+
+ varnishtest "List the contents of 'tmpdir'"
+ feature ignore_unknown_macro
+
+ server s1 {
+ } -start
+
+ haproxy h1 -conf {
+ defaults
+ mode http
+ timeout connect 5s
+ timeout server 30s
+ timeout client 30s
+
+ backend be1
+ server srv1 ${s1_addr}:${s1_port}
+
+ frontend http1
+ use_backend be1
+ bind "fd@${my_frontend_fd}"
+ } -start
+
+ shell {
+ echo "${tmpdir} working directory content:"
+ ls -lR ${tmpdir}
+ cat ${tmpdir}/h1/cfg
+ }
+
+We give only the output of the shell to illustrate this example:
+
+ .
+ .
+ .
+ ** top 0.0 === shell {
+ **** top 0.0 shell_cmd|exec 2>&1 ;
+ **** top 0.0 shell_cmd| echo "tmpdir: /tmp/vtc.32092.479d521e"
+ **** top 0.0 shell_cmd| ls -lR /tmp/vtc.32092.479d521e
+ **** top 0.0 shell_cmd| cat /tmp/vtc.32092.479d521e/h1/cfg
+ .
+ .
+ .
+ **** top 0.0 shell_out|/tmp/vtc.3808.448cbfe0 working directory content:
+ **** top 0.0 shell_out|/tmp/vtc.32092.479d521e:
+ **** top 0.0 shell_out|total 8
+ **** top 0.0 shell_out|drwxr-xr-x 2 users 4096 Jun 7 11:09 h1
+ **** top 0.0 shell_out|-rw-r--r-- 1 me users 84 Jun 7 11:09 INFO
+ **** top 0.0 shell_out|
+ **** top 0.0 shell_out|/tmp/vtc.32092.479d521e/h1:
+ **** top 0.0 shell_out|total 4
+ **** top 0.0 shell_out|-rw-r----- 1 fred users 339 Jun 7 11:09 cfg
+ **** top 0.0 shell_out|srw------- 1 fred users 0 Jun 7 11:09 stats.sock
+ **** top 0.0 shell_out| global
+ **** top 0.0 shell_out|\tstats socket /tmp/vtc.32092.479d521e/h1/stats.sock level admin mode 600
+ **** top 0.0 shell_out|
+ **** top 0.0 shell_out| defaults
+ **** top 0.0 shell_out| mode http
+ **** top 0.0 shell_out| timeout connect 5s
+ **** top 0.0 shell_out| timeout server 30s
+ **** top 0.0 shell_out| timeout client 30s
+ **** top 0.0 shell_out|
+ **** top 0.0 shell_out| backend be1
+ **** top 0.0 shell_out| server srv1 127.0.0.1:36984
+ **** top 0.0 shell_out|
+ **** top 0.0 shell_out| frontend http1
+ **** top 0.0 shell_out| use_backend be1
+ **** top 0.0 shell_out| bind "fd@${my_frontend_fd}"
+ **** top 0.0 shell_status = 0x0000
+
+
+The following example illustrate how to run a basic HTTP transaction between 'c1'
+client and 's1' server with 'http1' as haproxy frontend. This frontend listen
+on TCP socket with 'my_frontend_fd' as file descriptor.
+
+ # Mandatory line
+ varnishtest "Basic HAProxy test"
+
+ # As some macros for haproxy are used in this file, this line is mandatory.
+ feature ignore_unknown_macro
+
+ server s1 {
+ rxreq
+ txresp -body "s1 >>> Hello world!"
+ } -start
+
+ haproxy h1 -conf {
+ # Configuration file of 'h1' haproxy instance.
+ defaults
+ mode http
+ timeout connect 5s
+ timeout server 30s
+ timeout client 30s
+
+ backend be1
+ # declare 'srv1' server to point to 's1' server instance declare above.
+ server srv1 ${s1_addr}:${s1_port}
+
+ frontend http1
+ use_backend be1
+ bind "fd@${my_frontend_fd}"
+ } -start
+
+ client c1 -connect ${h1_my_frontend_fd_sock} {
+ txreq -url "/"
+ rxresp
+ expect resp.status == 200
+ expect resp.body == "s1 >>> Hello world!"
+ } -run
+
+
+It is possible to shorten the previous VTC file haproxy command section as follows:
+
+ haproxy h1 -conf {
+ # Configuration file of 'h1' haproxy instance.
+ defaults
+ mode http
+ timeout connect 5s
+ timeout server 30s
+ timeout client 30s
+ }
+
+In this latter example, "backend" and "frontend" sections are automatically
+generated depending on the declarations of server instances.
+
+
+Another interesting real regression test case is the following: we declare one
+server 's1', a syslog server 'Slg_1' and a basic haproxy configuration for 'h1'
+haproxy instance. Here we want to check that the syslog message are correctly
+formatted thanks to "expect" "syslog" command (see syslog Slg_1 {...} command)
+below.
+
+ varnishtest "Wrong ip/port logging"
+ feature ignore_unknown_macro
+
+ #commit d02286d6c866e5c0a7eb6fbb127fa57f3becaf16
+ #Author: Willy Tarreau <w@1wt.eu>
+ #Date: Fri Jun 23 11:23:43 2017 +0200
+ #
+ # BUG/MINOR: log: pin the front connection when front ip/ports are logged
+ #
+ # Mathias Weiersmueller reported an interesting issue with logs which Lukas
+ # diagnosed as dating back from commit 9b061e332 (1.5-dev9). When front
+ # connection information (ip, port) are logged in TCP mode and the log is
+ # emitted at the end of the connection (eg: because %B or any log tag
+ # requiring LW_BYTES is set), the log is emitted after the connection is
+ # closed, so the address and ports cannot be retrieved anymore.
+ #
+ # It could be argued that we'd make a special case of these to immediately
+ # retrieve the source and destination addresses from the connection, but it
+ # seems cleaner to simply pin the front connection, marking it "tracked" by
+ # adding the LW_XPRT flag to mention that we'll need some of these elements
+ # at the last moment. Only LW_FRTIP and LW_CLIP are affected. Note that after
+ # this change, LW_FRTIP could simply be removed as it's not used anywhere.
+
+ # Note that the problem doesn't happen when using %[src] or %[dst] since
+ # all sample expressions set LW_XPRT.
+
+
+ server s1 {
+ rxreq
+ txresp
+ } -start
+
+ syslog Slg_1 -level notice {
+ recv
+ recv
+ recv info
+ expect ~ \"dip\":\"${h1_fe_1_addr}\",\"dport\":\"${h1_fe_1_port}.*\"ts\":\"cD\",\"
+ } -start
+
+ haproxy h1 -conf {
+ global
+ log ${Slg_1_addr}:${Slg_1_port} local0
+
+ defaults
+ log global
+ timeout connect 3000
+ timeout client 5
+ timeout server 10000
+
+ frontend fe1
+ bind "fd@${fe_1}"
+ mode tcp
+ log-format {\"dip\":\"%fi\",\"dport\":\"%fp\",\"c_ip\":\"%ci\",\"c_port\":\"%cp\",\"fe_name\":\"%ft\",\"be_name\":\"%b\",\"s_name\":\"%s\",\"ts\":\"%ts\",\"bytes_read\":\"%B\"}
+ default_backend be_app
+
+ backend be_app
+ server app1 ${s1_addr}:${s1_port} check
+ } -start
+
+ client c1 -connect ${h1_fe_1_sock} {
+ txreq -url "/"
+ delay 0.02
+ } -run
+
+ syslog Slg_1 -wait
+
+
+Here is the output produced by varnishtest with the latter VTC file:
+
+ **** top 0.0 extmacro def pwd=/home/fred/src/haproxy
+ **** top 0.0 extmacro def localhost=127.0.0.1
+ **** top 0.0 extmacro def bad_backend=127.0.0.1 40386
+ **** top 0.0 extmacro def bad_ip=192.0.2.255
+ **** top 0.0 macro def testdir=//home/fred/src/varnish-cache-haproxy
+ **** top 0.0 macro def tmpdir=/tmp/vtc.15752.560ca66b
+ * top 0.0 TEST /home/fred/src/varnish-cache-haproxy/d02286d.vtc starting
+ ** top 0.0 === varnishtest "HAPEE bug 2788"
+ * top 0.0 TEST HAPEE bug 2788
+ ** top 0.0 === feature ignore_unknown_macro
+ ** top 0.0 === server s1 {
+ ** s1 0.0 Starting server
+ **** s1 0.0 macro def s1_addr=127.0.0.1
+ **** s1 0.0 macro def s1_port=35564
+ **** s1 0.0 macro def s1_sock=127.0.0.1 35564
+ * s1 0.0 Listen on 127.0.0.1 35564
+ ** top 0.0 === syslog Slg_1 -level notice {
+ ** Slg_1 0.0 Starting syslog server
+ ** s1 0.0 Started on 127.0.0.1 35564
+ **** Slg_1 0.0 macro def Slg_1_addr=127.0.0.1
+ **** Slg_1 0.0 macro def Slg_1_port=33012
+ **** Slg_1 0.0 macro def Slg_1_sock=127.0.0.1 33012
+ * Slg_1 0.0 Bound on 127.0.0.1 33012
+ ** top 0.0 === haproxy h1 -conf {
+ ** Slg_1 0.0 Started on 127.0.0.1 33012 (level: 5)
+ ** Slg_1 0.0 === recv
+ **** h1 0.0 macro def h1_fe_1_sock=::1 51782
+ **** h1 0.0 macro def h1_fe_1_addr=::1
+ **** h1 0.0 macro def h1_fe_1_port=51782
+ **** h1 0.0 setenv(fe_1, 7)
+ **** h1 0.0 conf| global
+ **** h1 0.0 conf|\tstats socket /tmp/vtc.15752.560ca66b/h1/stats.sock level admin mode 600
+ **** h1 0.0 conf|
+ **** h1 0.0 conf| global
+ **** h1 0.0 conf| log 127.0.0.1:33012 local0
+ **** h1 0.0 conf|
+ **** h1 0.0 conf| defaults
+ **** h1 0.0 conf| log global
+ **** h1 0.0 conf| timeout connect 3000
+ **** h1 0.0 conf| timeout client 5
+ **** h1 0.0 conf| timeout server 10000
+ **** h1 0.0 conf|
+ **** h1 0.0 conf| frontend fe1
+ **** h1 0.0 conf| bind "fd@${fe_1}"
+ **** h1 0.0 conf| mode tcp
+ **** h1 0.0 conf| log-format {\"dip\":\"%fi\",\"dport\":\"%fp\",\"c_ip\":\"%ci\",\"c_port\":\"%cp\",\"fe_name\":\"%ft\",\"be_name\":\"%b\",\"s_name\":\"%s\",\"ts\":\"%ts\",\"bytes_read\":\"%B\"}
+ **** h1 0.0 conf| default_backend be_app
+ **** h1 0.0 conf|
+ **** h1 0.0 conf| backend be_app
+ **** h1 0.0 conf| server app1 127.0.0.1:35564 check
+ ** h1 0.0 haproxy_start
+ **** h1 0.0 opt_worker 0 opt_daemon 0 opt_check_mode 0
+ **** h1 0.0 argv|exec /home/fred/src/haproxy/haproxy -d -f /tmp/vtc.15752.560ca66b/h1/cfg
+ **** h1 0.0 XXX 9 @277
+ *** h1 0.0 PID: 15787
+ **** h1 0.0 macro def h1_pid=15787
+ **** h1 0.0 macro def h1_name=/tmp/vtc.15752.560ca66b/h1
+ ** top 0.0 === client c1 -connect ${h1_fe_1_sock} {
+ ** c1 0.0 Starting client
+ ** c1 0.0 Waiting for client
+ *** c1 0.0 Connect to ::1 51782
+ *** c1 0.0 connected fd 8 from ::1 46962 to ::1 51782
+ ** c1 0.0 === txreq -url "/"
+ **** c1 0.0 txreq|GET / HTTP/1.1\r
+ **** c1 0.0 txreq|Host: 127.0.0.1\r
+ **** c1 0.0 txreq|\r
+ ** c1 0.0 === delay 0.02
+ *** c1 0.0 delaying 0.02 second(s)
+ *** h1 0.0 debug|Note: setting global.maxconn to 2000.
+ *** h1 0.0 debug|Available polling systems :
+ *** h1 0.0 debug| epoll :
+ *** h1 0.0 debug|pref=300,
+ *** h1 0.0 debug| test result OK
+ *** h1 0.0 debug| poll : pref=200, test result OK
+ *** h1 0.0 debug| select :
+ *** h1 0.0 debug|pref=150, test result FAILED
+ *** h1 0.0 debug|Total: 3 (2 usable), will use epoll.
+ *** h1 0.0 debug|
+ *** h1 0.0 debug|Available filters :
+ *** h1 0.0 debug|\t[SPOE] spoe
+ *** h1 0.0 debug|\t[COMP] compression
+ *** h1 0.0 debug|\t[TRACE] trace
+ **** Slg_1 0.0 syslog|<133>Jun 7 14:12:51 haproxy[15787]: Proxy fe1 started.
+ ** Slg_1 0.0 === recv
+ **** Slg_1 0.0 syslog|<133>Jun 7 14:12:51 haproxy[15787]: Proxy be_app started.
+ ** Slg_1 0.0 === recv info
+ *** h1 0.0 debug|00000000:fe1.accept(0007)=000a from [::1:46962]
+ *** s1 0.0 accepted fd 6 127.0.0.1 56770
+ ** s1 0.0 === rxreq
+ **** s1 0.0 rxhdr|GET / HTTP/1.1\r
+ **** s1 0.0 rxhdr|Host: 127.0.0.1\r
+ **** s1 0.0 rxhdr|\r
+ **** s1 0.0 rxhdrlen = 35
+ **** s1 0.0 http[ 0] |GET
+ **** s1 0.0 http[ 1] |/
+ **** s1 0.0 http[ 2] |HTTP/1.1
+ **** s1 0.0 http[ 3] |Host: 127.0.0.1
+ **** s1 0.0 bodylen = 0
+ ** s1 0.0 === txresp
+ **** s1 0.0 txresp|HTTP/1.1 200 OK\r
+ **** s1 0.0 txresp|Content-Length: 0\r
+ **** s1 0.0 txresp|\r
+ *** s1 0.0 shutting fd 6
+ ** s1 0.0 Ending
+ *** h1 0.0 debug|00000000:be_app.srvcls[000a:000c]
+ *** h1 0.0 debug|00000000:be_app.clicls[000a:000c]
+ *** h1 0.0 debug|00000000:be_app.closed[000a:000c]
+ **** Slg_1 0.0 syslog|<134>Jun 7 14:12:51 haproxy[15787]: {"dip":"","dport":"0","c_ip":"::1","c_port":"46962","fe_name":"fe1","be_name":"be_app","s_name":"app1","ts":"cD","bytes_read":"38"}
+ ** Slg_1 0.0 === expect ~ \"dip\":\"${h1_fe_1_addr}\",\"dport\":\"${h1_fe_1_p...
+ ---- Slg_1 0.0 EXPECT FAILED ~ "\"dip\":\"::1\",\"dport\":\"51782.*\"ts\":\"cD\",\""
+ *** c1 0.0 closing fd 8
+ ** c1 0.0 Ending
+ * top 0.0 RESETTING after /home/fred/src/varnish-cache-haproxy/d02286d.vtc
+ ** h1 0.0 Reset and free h1 haproxy 15787
+ ** h1 0.0 Wait
+ ** h1 0.0 Stop HAProxy pid=15787
+ **** h1 0.0 Kill(2)=0: Success
+ **** h1 0.0 STDOUT poll 0x10
+ ** h1 0.1 WAIT4 pid=15787 status=0x0002 (user 0.000000 sys 0.004000)
+ ** s1 0.1 Waiting for server (4/-1)
+ ** Slg_1 0.1 Waiting for syslog server (5)
+ * top 0.1 TEST /home/fred/src/varnish-cache-haproxy/d02286d.vtc FAILED
+ # top TEST /home/fred/src/varnish-cache-haproxy/d02286d.vtc FAILED (0.131) exit=2
+
+This test does not pass without the bug fix of d02286d commit. Indeed the third syslog
+message received by 'Slg_1' syslog server does not match the regular expression
+of the "syslog" "expect" command:
+
+ expect ~ \"dip\":\"${h1_fe_1_addr}\",\"dport\":\"${h1_fe_1_port}.*\"ts\":\"cD\",\"
+
+(the IP address and port are missing), contrary to what happens with the correct bug fix:
+
+ **** top 0.0 extmacro def pwd=/home/fred/src/haproxy
+ **** top 0.0 extmacro def localhost=127.0.0.1
+ **** top 0.0 extmacro def bad_backend=127.0.0.1 37284
+ **** top 0.0 extmacro def bad_ip=192.0.2.255
+ **** top 0.0 macro def testdir=//home/fred/src/varnish-cache-haproxy
+ **** top 0.0 macro def tmpdir=/tmp/vtc.12696.186b28b0
+ * top 0.0 TEST /home/fred/src/varnish-cache-haproxy/d02286d.vtc starting
+ ** top 0.0 === varnishtest "HAPEE bug 2788"
+ * top 0.0 TEST HAPEE bug 2788
+ ** top 0.0 === feature ignore_unknown_macro
+ ** top 0.0 === server s1 {
+ ** s1 0.0 Starting server
+ **** s1 0.0 macro def s1_addr=127.0.0.1
+ **** s1 0.0 macro def s1_port=53384
+ **** s1 0.0 macro def s1_sock=127.0.0.1 53384
+ * s1 0.0 Listen on 127.0.0.1 53384
+ ** top 0.0 === syslog Slg_1 -level notice {
+ ** Slg_1 0.0 Starting syslog server
+ **** Slg_1 0.0 macro def Slg_1_addr=127.0.0.1
+ ** s1 0.0 Started on 127.0.0.1 53384
+ **** Slg_1 0.0 macro def Slg_1_port=36195
+ **** Slg_1 0.0 macro def Slg_1_sock=127.0.0.1 36195
+ * Slg_1 0.0 Bound on 127.0.0.1 36195
+ ** top 0.0 === haproxy h1 -conf {
+ ** Slg_1 0.0 Started on 127.0.0.1 36195 (level: 5)
+ ** Slg_1 0.0 === recv
+ **** h1 0.0 macro def h1_fe_1_sock=::1 39264
+ **** h1 0.0 macro def h1_fe_1_addr=::1
+ **** h1 0.0 macro def h1_fe_1_port=39264
+ **** h1 0.0 setenv(fe_1, 7)
+ **** h1 0.0 conf| global
+ **** h1 0.0 conf|\tstats socket /tmp/vtc.12696.186b28b0/h1/stats.sock level admin mode 600
+ **** h1 0.0 conf|
+ **** h1 0.0 conf| global
+ **** h1 0.0 conf| log 127.0.0.1:36195 local0
+ **** h1 0.0 conf|
+ **** h1 0.0 conf| defaults
+ **** h1 0.0 conf| log global
+ **** h1 0.0 conf| timeout connect 3000
+ **** h1 0.0 conf| timeout client 5
+ **** h1 0.0 conf| timeout server 10000
+ **** h1 0.0 conf|
+ **** h1 0.0 conf| frontend fe1
+ **** h1 0.0 conf| bind "fd@${fe_1}"
+ **** h1 0.0 conf| mode tcp
+ **** h1 0.0 conf| log-format {\"dip\":\"%fi\",\"dport\":\"%fp\",\"c_ip\":\"%ci\",\"c_port\":\"%cp\",\"fe_name\":\"%ft\",\"be_name\":\"%b\",\"s_name\":\"%s\",\"ts\":\"%ts\",\"bytes_read\":\"%B\"}
+ **** h1 0.0 conf| default_backend be_app
+ **** h1 0.0 conf|
+ **** h1 0.0 conf| backend be_app
+ **** h1 0.0 conf| server app1 127.0.0.1:53384 check
+ ** h1 0.0 haproxy_start
+ **** h1 0.0 opt_worker 0 opt_daemon 0 opt_check_mode 0
+ **** h1 0.0 argv|exec /home/fred/src/haproxy/haproxy -d -f /tmp/vtc.12696.186b28b0/h1/cfg
+ **** h1 0.0 XXX 9 @277
+ *** h1 0.0 PID: 12728
+ **** h1 0.0 macro def h1_pid=12728
+ **** h1 0.0 macro def h1_name=/tmp/vtc.12696.186b28b0/h1
+ ** top 0.0 === client c1 -connect ${h1_fe_1_sock} {
+ ** c1 0.0 Starting client
+ ** c1 0.0 Waiting for client
+ *** c1 0.0 Connect to ::1 39264
+ *** c1 0.0 connected fd 8 from ::1 41245 to ::1 39264
+ ** c1 0.0 === txreq -url "/"
+ **** c1 0.0 txreq|GET / HTTP/1.1\r
+ **** c1 0.0 txreq|Host: 127.0.0.1\r
+ **** c1 0.0 txreq|\r
+ ** c1 0.0 === delay 0.02
+ *** c1 0.0 delaying 0.02 second(s)
+ *** h1 0.0 debug|Note: setting global.maxconn to 2000.
+ *** h1 0.0 debug|Available polling systems :
+ *** h1 0.0 debug| epoll : pref=300,
+ *** h1 0.0 debug| test result OK
+ *** h1 0.0 debug| poll : pref=200, test result OK
+ *** h1 0.0 debug| select : pref=150, test result FAILED
+ *** h1 0.0 debug|Total: 3 (2 usable), will use epoll.
+ *** h1 0.0 debug|
+ *** h1 0.0 debug|Available filters :
+ *** h1 0.0 debug|\t[SPOE] spoe
+ *** h1 0.0 debug|\t[COMP] compression
+ *** h1 0.0 debug|\t[TRACE] trace
+ *** h1 0.0 debug|Using epoll() as the polling mechanism.
+ **** Slg_1 0.0 syslog|<133>Jun 7 14:10:18 haproxy[12728]: Proxy fe1 started.
+ ** Slg_1 0.0 === recv
+ **** Slg_1 0.0 syslog|<133>Jun 7 14:10:18 haproxy[12728]: Proxy be_app started.
+ ** Slg_1 0.0 === recv info
+ *** h1 0.0 debug|00000000:fe1.accept(0007)=000c from [::1:41245] ALPN=<none>
+ *** s1 0.0 accepted fd 6 127.0.0.1 49946
+ ** s1 0.0 === rxreq
+ **** s1 0.0 rxhdr|GET / HTTP/1.1\r
+ **** s1 0.0 rxhdr|Host: 127.0.0.1\r
+ **** s1 0.0 rxhdr|\r
+ **** s1 0.0 rxhdrlen = 35
+ **** s1 0.0 http[ 0] |GET
+ **** s1 0.0 http[ 1] |/
+ **** s1 0.0 http[ 2] |HTTP/1.1
+ **** s1 0.0 http[ 3] |Host: 127.0.0.1
+ **** s1 0.0 bodylen = 0
+ ** s1 0.0 === txresp
+ **** s1 0.0 txresp|HTTP/1.1 200 OK\r
+ **** s1 0.0 txresp|Content-Length: 0\r
+ **** s1 0.0 txresp|\r
+ *** s1 0.0 shutting fd 6
+ ** s1 0.0 Ending
+ *** h1 0.0 debug|00000000:be_app.srvcls[000c:adfd]
+ *** h1 0.0 debug|00000000:be_app.clicls[000c:adfd]
+ *** h1 0.0 debug|00000000:be_app.closed[000c:adfd]
+ **** Slg_1 0.0 syslog|<134>Jun 7 14:10:18 haproxy[12728]: {"dip":"::1","dport":"39264","c_ip":"::1","c_port":"41245","fe_name":"fe1","be_name":"be_app","s_name":"app1","ts":"cD","bytes_read":"38"}
+ ** Slg_1 0.0 === expect ~ \"dip\":\"${h1_fe_1_addr}\",\"dport\":\"${h1_fe_1_p...
+ **** Slg_1 0.0 EXPECT MATCH ~ "\"dip\":\"::1\",\"dport\":\"39264.*\"ts\":\"cD\",\""
+ *** Slg_1 0.0 shutting fd 5
+ ** Slg_1 0.0 Ending
+ *** c1 0.0 closing fd 8
+ ** c1 0.0 Ending
+ ** top 0.0 === syslog Slg_1 -wait
+ ** Slg_1 0.0 Waiting for syslog server (-1)
+ * top 0.0 RESETTING after /home/fred/src/varnish-cache-haproxy/d02286d.vtc
+ ** h1 0.0 Reset and free h1 haproxy 12728
+ ** h1 0.0 Wait
+ ** h1 0.0 Stop HAProxy pid=12728
+ **** h1 0.0 Kill(2)=0: Success
+ **** h1 0.0 STDOUT poll 0x10
+ ** h1 0.1 WAIT4 pid=12728 status=0x0002 (user 0.000000 sys 0.004000)
+ ** s1 0.1 Waiting for server (4/-1)
+ * top 0.1 TEST /home/fred/src/varnish-cache-haproxy/d02286d.vtc completed
+ # top TEST /home/fred/src/varnish-cache-haproxy/d02286d.vtc passed (0.128)
+
+In this latter execution the third syslog message is correct:
+
+ **** Slg_1 0.0 syslog|<134>Jun 7 14:10:18 haproxy[12728]: {"dip":"::1","dport":"39264","c_ip":"::1","c_port":"41245","fe_name":"fe1","be_name":"be_app","s_name":"app1","ts":"cD","bytes_read":"38"}
diff --git a/doc/seamless_reload.txt b/doc/seamless_reload.txt
new file mode 100644
index 0000000..94df1bd
--- /dev/null
+++ b/doc/seamless_reload.txt
@@ -0,0 +1,62 @@
+Reloading HAProxy without impacting server states
+=================================================
+
+Of course, to fully understand below information please consult
+doc/configuration.txt to understand how each HAProxy directive works.
+
+In the mean line, we update HAProxy's configuration to tell it where to
+retrieve the last know trustable servers state.
+Then, before reloading HAProxy, we simply dump servers state from running
+process into the locations we pointed into the configuration.
+And voilĂ  :)
+
+
+Using one file for all backends
+-------------------------------
+
+HAProxy configuration
+*********************
+
+ global
+ [...]
+ stats socket /var/run/haproxy/socket
+ server-state-file global
+ server-state-base /var/state/haproxy/
+
+ defaults
+ [...]
+ load-server-state-from-file global
+
+HAProxy init script
+*******************
+
+Run the following command BEFORE reloading:
+
+ socat /var/run/haproxy/socket - <<< "show servers state" > /var/state/haproxy/global
+
+
+Using one state file per backend
+--------------------------------
+
+HAProxy configuration
+*********************
+
+ global
+ [...]
+ stats socket /var/run/haproxy/socket
+ server-state-base /var/state/haproxy/
+
+ defaults
+ [...]
+ load-server-state-from-file local
+
+HAProxy init script
+*******************
+
+Run the following command BEFORE reloading:
+
+ for b in $(socat /var/run/haproxy/socket - <<< "show backend" | fgrep -v '#')
+ do
+ socat /var/run/haproxy/socket - <<< "show servers state $b" > /var/state/haproxy/$b
+ done
+