summaryrefslogtreecommitdiffstats
path: root/doc/internals
diff options
context:
space:
mode:
authorDaniel Baumann <daniel.baumann@progress-linux.org>2024-04-13 12:18:05 +0000
committerDaniel Baumann <daniel.baumann@progress-linux.org>2024-04-13 12:18:05 +0000
commitb46aad6df449445a9fc4aa7b32bd40005438e3f7 (patch)
tree751aa858ca01f35de800164516b298887382919d /doc/internals
parentInitial commit. (diff)
downloadhaproxy-b46aad6df449445a9fc4aa7b32bd40005438e3f7.tar.xz
haproxy-b46aad6df449445a9fc4aa7b32bd40005438e3f7.zip
Adding upstream version 2.9.5.upstream/2.9.5
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'doc/internals')
-rw-r--r--doc/internals/acl.txt82
-rw-r--r--doc/internals/api/appctx.txt142
-rw-r--r--doc/internals/api/buffer-api.txt653
-rw-r--r--doc/internals/api/event_hdl.txt1015
-rw-r--r--doc/internals/api/filters.txt1188
-rw-r--r--doc/internals/api/htx-api.txt570
-rw-r--r--doc/internals/api/initcalls.txt366
-rw-r--r--doc/internals/api/ist.txt167
-rw-r--r--doc/internals/api/layers.txt190
-rw-r--r--doc/internals/api/list.txt195
-rw-r--r--doc/internals/api/pools.txt585
-rw-r--r--doc/internals/api/scheduler.txt228
-rw-r--r--doc/internals/body-parsing.txt165
-rw-r--r--doc/internals/connect-status.txt28
-rw-r--r--doc/internals/connection-header.txt196
-rw-r--r--doc/internals/connection-scale.txt44
-rw-r--r--doc/internals/fd-migration.txt138
-rw-r--r--doc/internals/hashing.txt83
-rw-r--r--doc/internals/list.fig599
-rw-r--r--doc/internals/list.pngbin0 -> 33618 bytes
-rw-r--r--doc/internals/listener-states.fig150
-rw-r--r--doc/internals/listener-states.pngbin0 -> 36142 bytes
-rw-r--r--doc/internals/lua_socket.fig113
-rw-r--r--doc/internals/lua_socket.pdfbin0 -> 14969 bytes
-rw-r--r--doc/internals/muxes.fig401
-rw-r--r--doc/internals/muxes.pdfbin0 -> 12984 bytes
-rw-r--r--doc/internals/muxes.pngbin0 -> 32209 bytes
-rw-r--r--doc/internals/muxes.svg911
-rw-r--r--doc/internals/notes-layers.txt330
-rw-r--r--doc/internals/notes-poll-connect.txt93
-rw-r--r--doc/internals/notes-pollhup.txt281
-rw-r--r--doc/internals/notes-polling.txt192
-rw-r--r--doc/internals/pattern.diabin0 -> 5631 bytes
-rw-r--r--doc/internals/pattern.pdfbin0 -> 37269 bytes
-rw-r--r--doc/internals/polling-states.fig59
-rw-r--r--doc/internals/sched.fig748
-rw-r--r--doc/internals/sched.pdfbin0 -> 25334 bytes
-rw-r--r--doc/internals/sched.pngbin0 -> 150457 bytes
-rw-r--r--doc/internals/sched.svg1204
-rw-r--r--doc/internals/ssl_cert.diabin0 -> 6700 bytes
-rw-r--r--doc/internals/stats-v2.txt8
-rw-r--r--doc/internals/stconn-close.txt74
-rw-r--r--doc/internals/stream-sock-states.fig535
43 files changed, 11733 insertions, 0 deletions
diff --git a/doc/internals/acl.txt b/doc/internals/acl.txt
new file mode 100644
index 0000000..0379331
--- /dev/null
+++ b/doc/internals/acl.txt
@@ -0,0 +1,82 @@
+2011/12/16 - How ACLs work internally in haproxy - w@1wt.eu
+
+An ACL is declared by the keyword "acl" followed by a name, followed by a
+matching method, followed by one or multiple pattern values :
+
+ acl internal src 127.0.0.0/8 10.0.0.0/8 192.168.0.0/16
+
+In the statement above, "internal" is the ACL's name (acl->name), "src" is the
+ACL keyword defining the matching method (acl_expr->kw) and the IP addresses
+are patterns of type acl_pattern to match against the source address.
+
+The acl_pattern struct may define one single pattern, a range of values or a
+tree of values to match against. The type of the patterns is implied by the
+ACL keyword. For instance, the "src" keyword implies IPv4 patterns.
+
+The line above constitutes an ACL expression (acl_expr). ACL expressions are
+formed of a keyword, an optional argument for the keyword, and a list of
+patterns (in fact, both a list and a root tree).
+
+Dynamic values are extracted according to a fetch function defined by the ACL
+keyword. This fetch function fills or updates a struct acl_test with all the
+extracted information so that a match function can compare it against all the
+patterns. The fetch function is called iteratively by the ACL engine until it
+reports no more value. This makes sense for instance when checking IP addresses
+found in HTTP headers, which can appear multiple times. The acl_test is kept
+intact between calls and even holds a context so that the fetch function knows
+where to start from for subsequent calls. The match function may also use the
+context even though it was not designed for that purpose.
+
+An ACL is defined only by its name and can be a series of ACL expressions. The
+ACL is deemed true when any of its expressions is true. They are evaluated in
+the declared order and can involve multiple matching methods.
+
+So in summary :
+
+ - an ACL is a series of tests to perform on a stream, any of which is enough
+ to validate the result.
+
+ - each test is defined by an expression associating a keyword and a series of
+ patterns.
+
+ - a keyword implies several things at once :
+ - the type of the patterns and how to parse them
+ - the method to fetch the required information from the stream
+ - the method to match the fetched information against the patterns
+
+ - a fetch function fills an acl_test struct which is passed to the match
+ function defined by the keyword
+
+ - the match function tries to match the value in the acl_test against the
+ pattern list declared in the expression which involved its acl_keyword.
+
+
+ACLs are used by conditional processing rules. A rule generally uses an "if" or
+"unless" keyword followed by an ACL condition (acl_cond). This condition is a
+series of term suites which are ORed together. Each term suite is a series of
+terms which are ANDed together. Terms may be negated before being evaluated in
+a suite. A term simply is a pointer to an ACL.
+
+We could then represent a rule by the following BNF :
+
+ rule = if-cond
+ | unless-cond
+
+ if-cond (struct acl_cond with ->pol = ACL_COND_IF)
+ = "if" condition
+
+ unless-cond (struct acl_cond with ->pol = ACL_COND_UNLESS)
+ = "unless" condition
+
+ condition
+ = term-suite
+ | term-suite "||" term-suite
+ | term-suite "or" term-suite
+
+ term-suite (struct acl_term_suite)
+ = term
+ | term term
+
+ term = acl
+ | "!" acl
+
diff --git a/doc/internals/api/appctx.txt b/doc/internals/api/appctx.txt
new file mode 100644
index 0000000..137ec7b
--- /dev/null
+++ b/doc/internals/api/appctx.txt
@@ -0,0 +1,142 @@
+Instantiation of applet contexts (appctx) in 2.6.
+
+
+1. Background
+
+Most applets are in fact simplified services that are called by the CLI when a
+registered keyword is matched. Some of them only have a ->parse() function
+which immediately returns with a final result, while others will return zero
+asking for the->io_handler() one to be called till the end. For these ones, a
+context is generally needed between calls to know where to restart from.
+
+Other applets are completely autonomous applets with their init function and
+an I/O handler, and these ones also need a persistent context between calls to
+the I/O handler. These ones are typically instantiated by "use-service" or by
+other means.
+
+Originally a few integers were provided to keep a trivial state (st0, st1, st2)
+and these ones progressively proved insufficient, leading to a "ctx.cli" sub-
+context that was allowed to use extra fields of various types. Other applets
+preferred to use their own context definition.
+
+All this resulted in the appctx->ctx to contain a myriad of definitions of
+various service contexts, and in some services abusing other services'
+definitions by laziness, and others being extended to use their own definition
+after having run for a long time on the generic types, some of which were not
+noticed and mistakenly used the same storage locations by accident. A massive
+cleanup was needed.
+
+
+2. New approach in 2.6
+
+In 2.6, there's an "svcctx" pointer that's initialized to NULL before any
+instantiation of an applet or of a CLI keyword's function. Applets and keyword
+handlers are free to make it point wherever they want, and to find it unaltered
+between subsequent calls, including up to the ->release() call. The "st2" state
+that was totally abused with random enums is not used anymore and was marked as
+deprecated. It's still initialized to zero before the first call though.
+
+One special area, "svc.storage[]", is large enough to contain any of the
+contexts that used to be present under "appctx->ctx". The "svcctx" may be set
+to point to this area so that a small structure can be allocated for free and
+without requiring error checking. In order to make this easier, a specially
+purposed function is provided: "applet_reserve_svcctx()". This function will
+require the caller to indicate how large an area it needs, and will return a
+pointer to this area after checking that it fits. If it does not, haproxy will
+crash. This is purposely done so that it's known during development that if a
+small structure doesn't fit, a different approach is required.
+
+As such, for the vast majority of commands, the process is the following one:
+
+ struct foo_ctx {
+ int myfield1;
+ int myfield2;
+ char *myfield3;
+ };
+
+ int io_handler(struct appctx *appctx)
+ {
+ struct foo_ctx *ctx = applet_reserve_svcctx(appctx, sizeof(*ctx));
+
+ if (!ctx->myfield1) {
+ /* first call */
+ ctx->myfield1++;
+ }
+ ...
+ }
+
+The pointer may be directly accessed from the I/O handler if it's known that it
+was already reserved by the init handler or parsing function. Otherwise it's
+guaranteed to be NULL so that can also serve as a test for a first call:
+
+ int parse_handler(struct appctx *appctx)
+ {
+ struct foo_ctx *ctx = applet_reserve_svcctx(appctx, sizeof(*ctx));
+ ctx->myfield1 = 12;
+ return 0;
+ }
+
+ int io_handler(struct appctx *appctx)
+ {
+ struct foo_ctx *ctx = appctx->svcctx;
+
+ for (; !ctx->myfield1; ctx->myfield1--) {
+ do_something();
+ }
+ ...
+ }
+
+There is no need to free anything because that space is not allocated but just
+points to a reserved area.
+
+If it is too small (its size is APPLET_MAX_SVCCTX bytes), it is preferable to
+use it with dynamically allocated structures (pools, malloc, etc). For example:
+
+ int io_handler(struct appctx *appctx)
+ {
+ struct foo_ctx *ctx = appctx->svcctx;
+
+ if (!ctx) {
+ /* first call */
+ ctx = pool_alloc(pool_foo_ctx);
+ if (!ctx)
+ return 1;
+ }
+ ...
+ }
+
+ void io_release(struct appctx *appctx)
+ {
+ pool_free(pool_foo_ctx, appctx->svcctx);
+ }
+
+The CLI code itself uses this mechanism for the cli_print_*() functions. Since
+these functions are terminal (i.e. not meant to be used in the middle of an I/O
+handler as they share the same contextual space), they always reset the svcctx
+pointer to place it to the "cli_print_ctx" mapped in ->svc.storage.
+
+
+3. Transition for old code
+
+A lot of care was taken to make the transition as smooth as possible for
+out-of-tree code since that's an API change. A dummy "ctx.cli" struct still
+exists in the appctx struct, and it happens to map perfectly to the one set by
+cli_print_*, so that if some code uses a mix of both, it will still work.
+However, it will build with "deprecated" warnings allowing to spot the
+remaining places. It's a good exercise to rename "ctx.cli" in "appctx" and see
+if the code still compiles.
+
+Regarding the "st2" sub-state, it will disappear as well after 2.6, but is
+still provided and initialized so that code relying on it will still work even
+if it builds with deprecation warnings. The correct approach is to move this
+state into the newly defined applet's context, and to stop using the stats
+enums STAT_ST_* that often barely match the needs and result in code that is
+more complicated than desired (the STAT_ST_* enum values have also been marked
+as deprecated).
+
+The code dealing with "show fd", "show sess" and the peers applet show good
+examples of how to convert a registered keyword or an applet.
+
+All this transition code requires complex layouts that will be removed during
+2.7-dev so there is no other long-term option but to update the code (or better
+get it merged if it can be useful to other users).
diff --git a/doc/internals/api/buffer-api.txt b/doc/internals/api/buffer-api.txt
new file mode 100644
index 0000000..ac35300
--- /dev/null
+++ b/doc/internals/api/buffer-api.txt
@@ -0,0 +1,653 @@
+2018-07-13 - HAProxy Internal Buffer API
+
+
+1. Background
+
+HAProxy uses a "struct buffer" internally to store data received from external
+agents, as well as data to be sent to external agents. These buffers are also
+used during data transformation such as compression, header insertion or
+defragmentation, and are used to carry intermediary representations between the
+various internal layers. They support wrapping at the end, and they carry their
+own size information so that in theory it would be possible to use different
+buffer sizes in parallel even though this is not currently implemented.
+
+The format of this structure has evolved over time, to reach a point where it
+is convenient and versatile enough to have permitted to make several internal
+types converge into a single one (specifically the struct chunk disappeared).
+
+
+2. Representation as of 1.9-dev1
+
+The current buffer representation consists in a linear storage area of known
+size, with a head position indicating the oldest data, and a total data count
+expressed in bytes. The head position, data count and size are expressed as
+integers and are positive or null. By convention, the head position is strictly
+smaller than the buffer size and the data count is smaller than or equal to the
+size, so that wrapping can be resolved with a single subtract. A buffer not
+respecting these rules is said to be degenerate. Unless specified otherwise,
+the various API functions will adopt an undefined behaviour when passed such a
+degenerate buffer.
+
+ Buffer declaration :
+
+ struct buffer {
+ size_t size; // size of the storage area (wrapping point)
+ char *area; // start of the storage area
+ size_t data; // contents length after head
+ size_t head; // start offset of remaining data relative to area
+ };
+
+
+ Linear buffer representation :
+
+ area
+ |
+ V<--------------------------------------------------------->| size
+ +-----------+---------------------------------+-------------+
+ | |/////////////////////////////////| |
+ +-----------+---------------------------------+-------------+
+ |<--------->|<------------------------------->|
+ head data ^
+ |
+ tail
+
+
+ Wrapping buffer representation :
+
+ area
+ |
+ V<--------------------------------------------------------->| size
+ +---------------+------------------------+------------------+
+ |///////////////| |//////////////////|
+ +---------------+------------------------+------------------+
+ |<-------------------------------------->| head
+ |-------------->| ...data data...|<-----------------|
+ ^
+ |
+ tail
+
+
+3. Terminology
+
+Manipulating a buffer just based on a head and a wrapping data count is not
+very convenient, so we define a certain number of terms for important elements
+characterizing a buffer :
+
+ - origin : pointer to relative position 0 in the storage area. Undefined
+ when the buffer is not allocated.
+
+ - size : the allocated size of the storage area starting at the origin,
+ expressed in bytes. A buffer whose size is zero is said not to
+ be allocated, and its origin in this case is undefined.
+
+ - data : the amount of data the buffer contains, in bytes. It is always
+ lower than or equal to the buffer's size, hence it is always 0
+ for an unallocated buffer.
+
+ - emptiness : a buffer is said to be empty when it contains no data, hence
+ data == 0. It is possible for such buffers not to be allocated
+ and to have size == 0 as well.
+
+ - room : the available space in the buffer. This is its size minus data.
+
+ - head : position relative to origin where the oldest data byte is found
+ (it typically is what send() uses to pick outgoing data). The
+ head is strictly smaller than the size.
+
+ - tail : position relative to origin where the first spare byte is found
+ (it typically is what recv() uses to store incoming data). It
+ is always equal to the buffer's data added to its head modulo
+ the buffer's size.
+
+ - wrapping : the byte following the last one of the storage area loops back
+ to position 0. This is called wrapping. The wrapping point is
+ the first position relative to origin which doesn't belong to
+ the storage area. There is no wrapping when a buffer is not
+ allocated. Wrapping requires special care and means that the
+ regular string manipulation functions are not usable on most
+ buffers, unless it is known that no wrapping happens. Free
+ space may wrap as well if the buffer only contains data in the
+ middle.
+
+ - alignment : a buffer is said to be aligned if its data do not wrap. That
+ is, its head is strictly before the tail, or the buffer is
+ empty and the head is null. Aligning a buffer may be required
+ to use regular string manipulation functions which have no
+ support for wrapping.
+
+
+A buffer may be in three different states :
+ - unallocated : size == 0, area == 0 (b_is_null() is true)
+ - waiting : size == 0, area != 0
+ - allocated : size > 0, area > 0
+
+It is not permitted to have area == 0 with a non-null size. In addition, the
+waiting state may also be used to indicate a read-only buffer which does not
+wrap and which must not be freed (e.g. for use with error messages).
+
+The basic API only covers allocated buffers. Switching to/from the other states
+is covered by the management API since it requires specific allocation and free
+calls.
+
+
+4. Using buffers
+
+Buffers are defined in a few files :
+ - include/common/buf.h : structure definition, and manipulation functions
+ - include/common/buffer.h : resource management (alloc/free/wait lists)
+ - include/common/istbuf.h : advanced string manipulation
+
+
+4.1. Basic API
+
+The basic API is made of the functions which abstract accesses to the buffers
+and which help calculating their state, free space or used space.
+
+====================+==================+=======================================
+Function | Arguments/Return | Description
+--------------------+------------------+---------------------------------------
+b_is_null() | const buffer *buf| returns true if (and only if) the
+ | ret: int | buffer is not yet allocated and thus
+ | | points to a NULL area
+--------------------+------------------+---------------------------------------
+b_orig() | const buffer *buf| returns the pointer to the origin of
+ | ret: char * | the storage, which is the location of
+ | | byte at offset zero. This is mostly
+ | | used by functions which handle the
+ | | wrapping by themselves
+--------------------+------------------+---------------------------------------
+b_size() | const buffer *buf| returns the size of the buffer
+ | ret: size_t |
+--------------------+------------------+---------------------------------------
+b_wrap() | const buffer *buf| returns the pointer to the wrapping
+ | ret: char * | position of the buffer area, which is
+ | | by definition the first byte not part
+ | | of the buffer
+--------------------+------------------+---------------------------------------
+b_data() | const buffer *buf| returns the number of bytes present in
+ | ret: size_t | the buffer
+--------------------+------------------+---------------------------------------
+b_room() | const buffer *buf| returns the amount of room left in the
+ | ret: size_t | buffer
+--------------------+------------------+---------------------------------------
+b_full() | const buffer *buf| returns true if the buffer is full
+ | ret: int |
+--------------------+------------------+---------------------------------------
+__b_stop() | const buffer *buf| returns a pointer to the byte
+ | ret: char * | following the end of the buffer, which
+ | | may be out of the buffer if the buffer
+ | | ends on the last byte of the area. It
+ | | is the caller's responsibility to
+ | | either know that the buffer does not
+ | | wrap or to check that the result does
+ | | not wrap
+--------------------+------------------+---------------------------------------
+__b_stop_ofs() | const buffer *buf| returns an origin-relative offset
+ | ret: size_t | pointing to the byte following the end
+ | | of the buffer, which may be out of the
+ | | buffer if the buffer ends on the last
+ | | byte of the area. It's the caller's
+ | | responsibility to either know that the
+ | | buffer does not wrap or to check that
+ | | the result does not wrap
+--------------------+------------------+---------------------------------------
+b_stop() | const buffer *buf| returns the pointer to the byte
+ | ret: char * | following the end of the buffer, which
+ | | may be out of the buffer if the buffer
+ | | ends on the last byte of the area
+--------------------+------------------+---------------------------------------
+b_stop_ofs() | const buffer *buf| returns an origin-relative offset
+ | ret: size_t | pointing to the byte following the end
+ | | of the buffer, which may be out of the
+ | | buffer if the buffer ends on the last
+ | | byte of the area
+--------------------+------------------+---------------------------------------
+__b_peek() | const buffer *buf| returns a pointer to the data at
+ | size_t ofs | position <ofs> relative to the head of
+ | ret: char * | the buffer. Will typically point to
+ | | input data if called with the amount
+ | | of output data. It's the caller's
+ | | responsibility to either know that the
+ | | buffer does not wrap or to check that
+ | | the result does not wrap
+--------------------+------------------+---------------------------------------
+__b_peek_ofs() | const buffer *buf| returns an origin-relative offset
+ | size_t ofs | pointing to the data at position <ofs>
+ | ret: size_t | relative to the head of the
+ | | buffer. Will typically point to input
+ | | data if called with the amount of
+ | | output data. It's the caller's
+ | | responsibility to either know that the
+ | | buffer does not wrap or to check that
+ | | the result does not wrap
+--------------------+------------------+---------------------------------------
+b_peek() | const buffer *buf| returns a pointer to the data at
+ | size_t ofs | position <ofs> relative to the head of
+ | ret: char * | the buffer. Will typically point to
+ | | input data if called with the amount
+ | | of output data. If applying <ofs> to
+ | | the buffers' head results in a
+ | | position between <size> and 2*>size>-1
+ | | included, a wrapping compensation is
+ | | applied to the result
+--------------------+------------------+---------------------------------------
+b_peek_ofs() | const buffer *buf| returns an origin-relative offset
+ | size_t ofs | pointing to the data at position <ofs>
+ | ret: size_t | relative to the head of the
+ | | buffer. Will typically point to input
+ | | data if called with the amount of
+ | | output data. If applying <ofs> to the
+ | | buffers' head results in a position
+ | | between <size> and 2*>size>-1
+ | | included, a wrapping compensation is
+ | | applied to the result
+--------------------+------------------+---------------------------------------
+__b_head() | const buffer *buf| returns the pointer to the buffer's
+ | ret: char * | head, which is the location of the
+ | | next byte to be dequeued. The result
+ | | is undefined for unallocated buffers
+--------------------+------------------+---------------------------------------
+__b_head_ofs() | const buffer *buf| returns an origin-relative offset
+ | ret: size_t | pointing to the buffer's head, which
+ | | is the location of the next byte to be
+ | | dequeued. The result is undefined for
+ | | unallocated buffers
+--------------------+------------------+---------------------------------------
+b_head() | const buffer *buf| returns the pointer to the buffer's
+ | ret: char * | head, which is the location of the
+ | | next byte to be dequeued. The result
+ | | is undefined for unallocated
+ | | buffers. If applying <ofs> to the
+ | | buffers' head results in a position
+ | | between <size> and 2*>size>-1
+ | | included, a wrapping compensation is
+ | | applied to the result
+--------------------+------------------+---------------------------------------
+b_head_ofs() | const buffer *buf| returns an origin-relative offset
+ | ret: size_t | pointing to the buffer's head, which
+ | | is the location of the next byte to be
+ | | dequeued. The result is undefined for
+ | | unallocated buffers. If applying
+ | | <ofs> to the buffers' head results in
+ | | a position between <size> and
+ | | 2*>size>-1 included, a wrapping
+ | | compensation is applied to the result
+--------------------+------------------+---------------------------------------
+__b_tail() | const buffer *buf| returns the pointer to the tail of the
+ | ret: char * | buffer, which is the location of the
+ | | first byte where it is possible to
+ | | enqueue new data. The result is
+ | | undefined for unallocated buffers
+--------------------+------------------+---------------------------------------
+__b_tail_ofs() | const buffer *buf| returns an origin-relative offset
+ | ret: size_t | pointing to the tail of the buffer,
+ | | which is the location of the first
+ | | byte where it is possible to enqueue
+ | | new data. The result is undefined for
+ | | unallocated buffers
+--------------------+------------------+---------------------------------------
+b_tail() | const buffer *buf| returns the pointer to the tail of the
+ | ret: char * | buffer, which is the location of the
+ | | first byte where it is possible to
+ | | enqueue new data. The result is
+ | | undefined for unallocated buffers
+--------------------+------------------+---------------------------------------
+b_tail_ofs() | const buffer *buf| returns an origin-relative offset
+ | ret: size_t | pointing to the tail of the buffer,
+ | | which is the location of the first
+ | | byte where it is possible to enqueue
+ | | new data. The result is undefined for
+ | | unallocated buffers
+--------------------+------------------+---------------------------------------
+b_next() | const buffer *buf| for an absolute pointer <p> pointing
+ | const char *p | to a valid location within buffer <b>,
+ | ret: char * | returns the absolute pointer to the
+ | | next byte, which usually is at (p + 1)
+ | | unless p reaches the wrapping point
+ | | and wrapping is needed
+--------------------+------------------+---------------------------------------
+b_next_ofs() | const buffer *buf| for an origin-relative offset <o>
+ | size_t o | pointing to a valid location within
+ | ret: size_t | buffer <b>, returns either the
+ | | relative offset pointing to the next
+ | | byte, which usually is at (o + 1)
+ | | unless o reaches the wrapping point
+ | | and wrapping is needed
+--------------------+------------------+---------------------------------------
+b_dist() | const buffer *buf| returns the distance between two
+ | const char *from | pointers, taking into account the
+ | const char *to | ability to wrap around the buffer's
+ | ret: size_t | end. The operation is not defined if
+ | | either of the pointers does not belong
+ | | to the buffer or if their distance is
+ | | greater than the buffer's size
+--------------------+------------------+---------------------------------------
+b_almost_full() | const buffer *buf| returns 1 if the buffer uses at least
+ | ret: int | 3/4 of its capacity, otherwise
+ | | zero. Buffers of size zero are
+ | | considered full
+--------------------+------------------+---------------------------------------
+b_space_wraps() | const buffer *buf| returns non-zero only if the buffer's
+ | ret: int | free space wraps, which means that the
+ | | buffer contains data that are not
+ | | touching at least one edge
+--------------------+------------------+---------------------------------------
+b_contig_data() | const buffer *buf| returns the amount of data that can
+ | size_t start | contiguously be read at once starting
+ | ret: size_t | from a relative offset <start> (which
+ | | allows to easily pre-compute blocks
+ | | for memcpy). The start point will
+ | | typically contain the amount of past
+ | | data already returned by a previous
+ | | call to this function
+--------------------+------------------+---------------------------------------
+b_contig_space() | const buffer *buf| returns the amount of bytes that can
+ | ret: size_t | be appended to the buffer at once
+--------------------+------------------+---------------------------------------
+b_getblk() | const buffer *buf| gets one full block of data at once
+ | char *blk | from a buffer, starting from offset
+ | size_t len | <offset> after the buffer's head, and
+ | size_t offset | limited to no more than <len> bytes.
+ | ret: size_t | The caller is responsible for ensuring
+ | | that neither <offset> nor <offset> +
+ | | <len> exceed the total number of bytes
+ | | available in the buffer. Return zero
+ | | if not enough data was available, in
+ | | which case blk is left undefined, or
+ | | the number of bytes read which is
+ | | equal to the requested size
+--------------------+------------------+---------------------------------------
+b_getblk_nc() | const buffer *buf| gets one or two blocks of data at once
+ | const char **blk1| from a buffer, starting from offset
+ | size_t *len1 | <ofs> after the beginning of its
+ | const char **blk2| output, and limited to no more than
+ | size_t *len2 | <max> bytes. The caller is responsible
+ | size_t ofs | for ensuring that neither <ofs> nor
+ | size_t max | <ofs>+<max> exceed the total number of
+ | ret: int | bytes available in the buffer. Returns
+ | | 0 if not enough data were available,
+ | | or the number of blocks filled (1 or
+ | | 2). <blk1> is always filled before
+ | | <blk2>. The unused blocks are left
+ | | undefined, and the buffer is left
+ | | unaffected. Unused buffers are left in
+ | | an undefined state
+--------------------+------------------+---------------------------------------
+b_reset() | buffer *buf | resets a buffer. The size is not
+ | ret: void | touched. In practice it resets the
+ | | head and the data length
+--------------------+------------------+---------------------------------------
+b_sub() | buffer *buf | decreases the buffer length by <count>
+ | size_t count | without touching the head position
+ | ret: void | (only the tail moves). this may mostly
+ | | be used to trim pending data before
+ | | reusing a buffer. The caller is
+ | | responsible for not removing more than
+ | | the available data
+--------------------+------------------+---------------------------------------
+b_add() | buffer *buf | increase the buffer length by <count>
+ | size_t count | without touching the head position
+ | ret: void | (only the tail moves). This is used
+ | | when adding data at the tail of a
+ | | buffer. The caller is responsible for
+ | | not adding more than the available
+ | | room
+--------------------+------------------+---------------------------------------
+b_set_data() | buffer *buf | sets the buffer's length, by adjusting
+ | size_t len | the buffer's tail only. The caller is
+ | ret: void | responsible for passing a valid length
+--------------------+------------------+---------------------------------------
+b_del() | buffer *buf | deletes <del> bytes at the head of
+ | size_t del | buffer <b> and updates the head. The
+ | ret: void | caller is responsible for not removing
+ | | more than the available data. This is
+ | | used after sending data from the
+ | | buffer
+--------------------+------------------+---------------------------------------
+b_realign_if_empty()| buffer *buf | realigns a buffer if it's empty, does
+ | ret: void | nothing otherwise. This is mostly used
+ | | after b_del() to make an empty
+ | | buffer's free space contiguous
+--------------------+------------------+---------------------------------------
+b_slow_realign() | buffer *buf | realigns a possibly wrapping buffer so
+ | size_t output | that the part remaining to be parsed
+ | ret: void | is contiguous and starts at the
+ | | beginning of the buffer and the
+ | | already parsed output part ends at the
+ | | end of the buffer. This provides the
+ | | best conditions since it allows the
+ | | largest inputs to be processed at once
+ | | and ensures that once the output data
+ | | leaves, the whole buffer is available
+ | | at once. The number of output bytes
+ | | supposedly present at the beginning of
+ | | the buffer and which need to be moved
+ | | to the end must be passed in <output>.
+ | | It will effectively make this offset
+ | | the new wrapping point. A temporary
+ | | swap area at least as large as b->size
+ | | must be provided in <swap>. It's up
+ | | to the caller to ensure <output> is no
+ | | larger than the difference between the
+ | | whole buffer's length and its input
+--------------------+------------------+---------------------------------------
+b_putchar() | buffer *buf | tries to append char <c> at the end of
+ | char c | buffer <b>. Supports wrapping. New
+ | ret: void | data are silently discarded if the
+ | | buffer is already full
+--------------------+------------------+---------------------------------------
+b_putblk() | buffer *buf | tries to append block <blk> at the end
+ | const char *blk | of buffer <b>. Supports wrapping. Data
+ | size_t len | are truncated if the buffer is too
+ | ret: size_t | short or if not enough space is
+ | | available. It returns the number of
+ | | bytes really copied
+--------------------+------------------+---------------------------------------
+b_move() | buffer *buf | moves block (src,len) left or right
+ | size_t src | by <shift> bytes, supporting wrapping
+ | size_t len | and overlapping.
+ | size_t shift |
+--------------------+------------------+---------------------------------------
+b_rep_blk() | buffer *buf | writes the block <blk> at position
+ | char *pos | <pos> which must be in buffer <b>, and
+ | char *end | moves the part between <end> and the
+ | const char *blk | buffer's tail just after the end of
+ | size_t len | the copy of <blk>. This effectively
+ | ret: int | replaces the part located between
+ | | <pos> and <end> with a copy of <blk>
+ | | of length <len>. The buffer's length
+ | | is automatically updated. This is used
+ | | to replace a block with another one
+ | | inside a buffer. The shift value
+ | | (positive or negative) is returned. If
+ | | there's no space left, the move is not
+ | | done. If <len> is null, the <blk>
+ | | pointer is allowed to be null, in
+ | | order to erase a block
+--------------------+------------------+---------------------------------------
+b_xfer() | buffer *src | transfers at most <count> bytes from
+ | buffer *dst | buffer <src> to buffer <dst> and
+ | size_t cout | returns the number of bytes copied.
+ | ret: size_t | The bytes are removed from <src> and
+ | | added to <dst>. The caller guarantees
+ | | that <count> is <= b_room(dst)
+====================+==================+=======================================
+
+
+4.2. String API
+
+The string API aims at providing both convenient and efficient ways to read and
+write to/from buffers using indirect strings (ist). These strings and some
+associated functions are defined in ist.h.
+
+====================+==================+=======================================
+Function | Arguments/Return | Description
+--------------------+------------------+---------------------------------------
+b_isteq() | const buffer *b | b_isteq() : returns > 0 if the first
+ | size_t o | <n> characters of buffer <b> starting
+ | size_t n | at offset <o> relative to the buffer's
+ | const ist ist | head match <ist>. (empty strings do
+ | ret: int | match). It is designed to be used with
+ | | reasonably small strings (it matches a
+ | | single byte per loop iteration). It is
+ | | expected to be used with an offset to
+ | | skip old data. Return value number of
+ | | matching bytes if >0, not enough bytes
+ | | or empty string if 0, or non-matching
+ | | byte found if <0.
+--------------------+------------------+---------------------------------------
+b_isteat | struct buffer *b | b_isteat() : "eats" string <ist> from
+ | const ist ist | the head of buffer <b>. Wrapping data
+ | ret: ssize_t | is explicitly supported. It matches a
+ | | single byte per iteration so strings
+ | | should remain reasonably small.
+ | | Returns the number of bytes matched
+ | | and eaten if >0, not enough bytes or
+ | | matched empty string if 0, or non
+ | | matching byte found if <0.
+--------------------+------------------+---------------------------------------
+b_istput | struct buffer *b | b_istput() : injects string <ist> at
+ | const ist ist | the tail of output buffer <b> provided
+ | ret: ssize_t | that it fits. Wrapping is supported.
+ | | It's designed for small strings as it
+ | | only writes a single byte per
+ | | iteration. Returns the number of
+ | | characters copied (ist.len), 0 if it
+ | | temporarily does not fit, or -1 if it
+ | | will never fit. It will only modify
+ | | the buffer upon success. In all cases,
+ | | the contents are copied prior to
+ | | reporting an error, so that the
+ | | destination at least contains a valid
+ | | but truncated string.
+--------------------+------------------+---------------------------------------
+b_putist | struct buffer *b | b_putist() : tries to copy as much as
+ | const ist ist | possible of string <ist> into buffer
+ | ret: size_t | <b> and returns the number of bytes
+ | | copied (truncation is possible). It
+ | | uses b_putblk() and is suitable for
+ | | large blocks.
+====================+==================+=======================================
+
+
+4.3. Management API
+
+The management API makes a distinction between an empty buffer, which by
+definition is not allocated but is ready to be allocated at any time, and a
+buffer which failed an allocation and is waiting for an available area to be
+offered. The functions allow to register on a list to be notified about buffer
+availability, to notify others of a number of buffers just released, and to be
+and to be notified of buffer availability. All allocations are made through the
+standard buffer pools.
+
+====================+==================+=======================================
+Function | Arguments/Return | Description
+--------------------+------------------+---------------------------------------
+buffer_almost_full | const buffer *buf| returns true if the buffer is not null
+ | ret: int | and at least 3/4 of the buffer's space
+ | | are used. A waiting buffer will match.
+--------------------+------------------+---------------------------------------
+b_alloc | buffer *buf | ensures that <buf> is allocated or
+ | ret: buffer * | allocates a buffer and assigns it to
+ | | *buf. If no memory is available, (1)
+ | | is assigned instead with a zero size.
+ | | The allocated buffer is returned, or
+ | | NULL in case no memory is available
+--------------------+------------------+---------------------------------------
+__b_free | buffer *buf | releases <buf> which must be allocated
+ | ret: void | and marks it empty
+--------------------+------------------+---------------------------------------
+b_free | buffer *buf | releases <buf> only if it is allocated
+ | ret: void | and marks it empty
+--------------------+------------------+---------------------------------------
+offer_buffers() | void *from | offer a buffer currently belonging to
+ | uint threshold | target <from> to whoever needs
+ | ret: void | one. Any pointer is valid for <from>,
+ | | including NULL. Its purpose is to
+ | | avoid passing a buffer to oneself in
+ | | case of failed allocations (e.g. need
+ | | two buffers, get one, fail, release it
+ | | and wake up self again). In case of
+ | | normal buffer release where it is
+ | | expected that the caller is not
+ | | waiting for a buffer, NULL is fine
+====================+==================+=======================================
+
+
+5. Porting code from older versions
+
+The previous buffer API introduced in 1.5-dev9 (May 2012) used to look like the
+following (with the struct renamed to old_buffer here to avoid confusion during
+quick lookups at the doc). It's worth noting that the "data" field used to be
+part of the struct but with a different type and meaning. It's important to be
+careful about potential code making use of &b->data as it will silently compile
+but fail.
+
+ Previous buffer declaration :
+
+ struct old_buffer {
+ char *p; /* buffer's start pointer, separates in and out data */
+ unsigned int size; /* buffer size in bytes */
+ unsigned int i; /* number of input bytes pending for analysis in the buffer */
+ unsigned int o; /* number of out bytes the sender can consume from this buffer */
+ char data[0]; /* <size> bytes */
+ };
+
+ Previous linear buffer representation :
+
+ data p
+ | |
+ V V
+ +-----------+--------------------+------------+-------------+
+ | |////////////////////|////////////| |
+ +-----------+--------------------+------------+-------------+
+ <---------------------------------------------------------> size
+ <------------------> <---------->
+ o i
+
+There is this correspondence between old and new fields (some will involve a
+knowledge of a channel when the output byte count is required) :
+
+ Old | New
+ --------+----------------------------------------------------
+ p | data + head + co_data(channel) // ci_head(channel)
+ size | size
+ i | data - co_data(channel) // ci_data(channel)
+ o | co_data(channel) // channel->output
+ data | area
+ --------+-----------------------------------------------------
+
+Then some common expressions can be mapped like this :
+
+ Old | New
+ -----------------------+---------------------------------------
+ b->data | b_orig(b)
+ &b->data | b_orig(b)
+ bi_ptr(b) | ci_head(channel)
+ bi_end(b) | b_tail(b)
+ bo_ptr(b) | b_head(b)
+ bo_end(b) | co_tail(channel)
+ bi_putblk(b,s,l) | b_putblk(b,s,l)
+ bo_getblk(b,s,l,o) | b_getblk(b,s,l,o)
+ bo_getblk_nc(b,s,l,o) | b_getblk_nc(b,s,l,o,0,co_data(channel))
+ b->i + b->o | b_data(b)
+ b->data + b->size | b_wrap(b)
+ b->i += len | b_add(b, len)
+ b->i -= len | b_sub(b, len)
+ b->i = len | b_set_data(b, co_data(channel) + len)
+ b->o += len | b_add(b, len); channel->output += len
+ b->o -= len | b_del(b, len); channel->output -= len
+ -----------------------+---------------------------------------
+
+The buffer modification functions are less straightforward and depend a lot on
+the context where they are used. It is strongly advised to figure in the list
+of functions above what is available based on what is attempted to be done in
+the existing code.
+
+Note that it is very likely that any out-of-tree code relying on buffers will
+not use both ->i and ->o but instead will use exclusively ->i on the side
+producing data and use exclusively ->o on the side consuming data (such as in a
+mux or in an applet). In both cases, it should be assumed that the other side
+is always zero and that either ->i or ->o is replaced with ->data, making the
+remaining code much simpler (no more code duplication based on the data
+direction).
diff --git a/doc/internals/api/event_hdl.txt b/doc/internals/api/event_hdl.txt
new file mode 100644
index 0000000..72eeff8
--- /dev/null
+++ b/doc/internals/api/event_hdl.txt
@@ -0,0 +1,1015 @@
+ -----------------------------------------
+ event_hdl Guide - version 2.8
+ ( Last update: 2022-11-14 )
+ ------------------------------------------
+
+ABSTRACT
+--------
+
+The event_hdl support is a new feature of HAProxy 2.7. It is a way to easily
+handle general events in a simple to maintain fashion, while keeping core code
+impact to the bare minimum.
+
+This document first describes how to use already supported events,
+then how to add support for your very own events.
+
+This feature is quite new for now. The API is not frozen and will be
+updated/modified/improved/extended as needed.
+
+SUMMARY
+-------
+
+ 1. event_hdl introduction
+ 2. How to handle existing events
+ 2.1 SYNC mode
+ 2.2 ASYNC mode
+ 2.2.1 normal version
+ 2.2.2 task version
+ 2.3 Advanced features
+ 2.3.1 sub_mgmt
+ 2.3.2 subscription external lookups
+ 2.3.3 subscription ptr
+ 2.3.4 private_free
+ 3. How to add support for new events
+ 3.1 Declaring a new event data structure
+ 3.2 Publishing an event
+ 4. Subscription lists
+ 5. misc/helper functions
+
+
+1. EVENT_HDL INTRODUCTION
+-----------------------
+
+EVENT_HDL provides two complementary APIs, both are implemented
+in src/event_hdl.c and include/haproxy/event_hdl(-t).h:
+
+One API targeting developers that want to register event
+handlers that will be notified when specific events occur in the process.
+(See section 2.)
+
+One API targeting developers that want to notify registered handlers about
+an event that is happening in the process.
+(See section 3.)
+
+2. HOW TO HANDLE EXISTING EVENTS
+---------------------
+
+To handle existing events, you must first decide which events you're
+interested in.
+
+event types are defined as follow:
+
+```
+ /* type for storing event subscription type */
+ typedef struct event_hdl_sub_type
+ {
+ /* up to 256 families, non cumulative, adjust if needed */
+ uint8_t family;
+ /* up to 16 sub types using bitmasks, adjust if needed */
+ uint16_t subtype;
+ } event_hdl_sub_type;
+```
+
+For an up to date list of already supported events,
+please refer to include/haproxy/event_hdl-t.h
+At the end of the file you will find existing event types.
+
+Each event family provides an unique data structure that will
+be provided to the event handler (registered to one or more
+event subtypes) when such events occur.
+
+An event handler can subscribe to a single event family type at a time, but
+within the family type it can subscribe to multiple event subtypes.
+
+ For example, let's consider the SERVER family type.
+
+ Let's assume it provides the event_hdl_cb_data_server data structure.
+
+ We can register a handler that will be notified for
+ every SERVER event types using:
+ EVENT_HDL_SUB_SERVER
+
+ This will include EVENT_HDL_SUB_SERVER_ADD,
+ EVENT_HDL_SUB_SERVER_DEL [...]
+
+ But we can also subscribe to a specific subtype only,
+ for example server deletion:
+ EVENT_HDL_SUB_SERVER_DEL
+
+ You can even combine multiple SERVER subtypes using
+ event_hdl_sub_type_add function helper:
+ event_hdl_sub_type_add(EVENT_HDL_SUB_SERVER_DEL,
+ EVENT_HDL_SUB_SERVER_ADD)
+
+ (will refer to server deletion as well as server addition)
+
+Registering a handler comes into multiple flavors:
+
+ SYNC mode:
+ handler is called in a blocking manner directly from the
+ thread that publishes the event.
+ This mode should be used with precaution because it could
+ slow the caller or cause deadlocks if used improperly.
+
+ Sync mode is useful when you directly depend on data or
+ state consistency from the caller.
+
+ Sync mode gives you access to unsafe elements in the data structure
+ provided by the caller (again, see event_hdl-t.h for more details).
+ The data structure may provide lock hints in the unsafe section
+ so that you know which locks are already held within the
+ calling context, hopefully preventing you from relocking
+ an already locked element and preventing deadlocks.
+
+ ASYNC mode:
+ handler is called in a non-blocking manner
+ (in a dedicated tasklet),
+ thus, the caller (that published the event) is not affected
+ by the handler. (time wise and data wise)
+
+ This is the safest way to handle events,
+ but it also comes with a limitation:
+
+ unsafe elements in the data structure provided by
+ the caller SHOULD be used under NO circumstances.
+ Indeed, only safe elements are meant to be used
+ when handling the event in async mode.
+
+ ASYNC mode is declined in 2 different versions:
+ normal:
+ handler is simply a function pointer
+ (same prototype as sync mode),
+ that is called asynchronously with relevant data
+ when the event is published. Only difference with
+ sync mode here is that 'unsafe' data provided
+ by the data structure may not be used.
+ task:
+ handler is a user defined task(let) that uses an event
+ queue to consume pending events.
+ This mode is interesting when you need to perform
+ advanced operations or you need to handle the event
+ in an already existing task context.
+ It is a bit more complicated to setup, but really
+ nothing to worry about, some examples will be
+ provided later in this document.
+
+event subscription is performed using the function:
+
+ event_hdl_subscribe(list, event, hdl);
+
+ The function returns 1 in case of success,
+ and 0 in case of failure (bad arguments, or memory error)
+
+ The function may BUG_ON if used improperly (invalid arguments)
+
+ <list> is either user specified list used to store the
+ new subscription, or NULL if you want to store the subscription
+ in the process global list.
+
+ <list> is also asked when publishing an event,
+ so specifying list could be useful, if, for example,
+ you only want to subscribe to a specific subscription list
+ (see this as a scope for example, NULL being full scope,
+ and specific list being limited scope)
+
+ We will use server events as an example:
+
+ You could register to events for ALL servers by using the
+ global list (NULL), or only to a specific server events
+ by using the subscription list dedicated to a single server.
+
+ <event> are the events (family.subtypes) you're subscribing to
+
+ <hdl> contains required handler options, it must be provided using
+ EVENT_HDL_(TASK_)(A)SYNC() and EVENT_HDL_ID_(TASK_)(A)SYNC()
+ helper macros.
+
+ See include/haproxy/event_hdl.h or below to know which macro
+ best suits your needs.
+
+ When registering a handler, you have the ability to provide an
+ unique ID (using EVENT_HDL_ID_ macro family) that could be used
+ later to perform lookups on the subscription.
+ ID is stored as an uint64_t hash that is expected to be computed using
+ general purpose event_hdl_id inline function provided by event_hdl.h.
+ Not providing an ID (using EVENT_HDL_ macro family)
+ results in the subscription being considered as anonymous.
+ As the name implies, anonymous subscriptions don't support lookups.
+
+2.1 SYNC MODE
+---------------------
+
+Example, you want to register a sync handler that will be called when
+a new server is added.
+
+Here is what the handler function will look like:
+```
+void my_sync_handler(const struct event_hdl_cb *cb, void *private)
+{
+ const struct event_hdl_cb_data_server *server = cb->e_data;
+
+ /* using EVENT_HDL_ASSERT_SYNC is a good practice to ensure
+ * that the function breaks if used in async mode
+ * (because we will access unsafe data in this function that
+ * is sync mode only)
+ */
+ EVENT_HDL_ASSERT_SYNC(cb);
+ printf("I've been called for '%s', private = %p\n",
+ event_hdl_sub_type_to_string(cb->e_type), private);
+ printf("server name is '%s'\n", server->safe.name);
+
+ /* here it is safe to use unsafe data */
+ printf("server ptr is '%p'\n", server->unsafe.ptr);
+
+ /* from here you have the possibility to manage the subscription
+ * cb->sub_mgmt->unsub(cb->sub_mgmt);
+ * // hdl will be removed from the subscription list
+ */
+}
+```
+
+Here is how you perform the subscription:
+
+anonymous subscription:
+```
+ int private = 10;
+
+ event_hdl_subscribe(NULL, EVENT_HDL_SUB_SERVER_ADD,
+ EVENT_HDL_SYNC(my_sync_handler, &private, NULL));
+```
+
+identified subscription:
+```
+ int private = 10;
+ uint64_t id = event_hdl_id("test", "sync");
+
+ event_hdl_subscribe(NULL, EVENT_HDL_SUB_SERVER_ADD,
+ EVENT_HDL_ID_SYNC(id,
+ my_sync_handler,
+ &private,
+ NULL));
+
+```
+
+identified subscription where freeing private is required when subscription ends:
+(also works for anonymous)
+(more on this feature in 2.3.4)
+```
+ int *private = malloc(sizeof(*private));
+ uint64_t id = event_hdl_id("test", "sync_free");
+
+ BUG_ON(!private);
+ *private = 10;
+
+ /* passing free as 'private_free' function so that
+ * private can be freed when unregistering is performed
+ */
+ event_hdl_subscribe(NULL, EVENT_HDL_SUB_SERVER_ADD,
+ EVENT_HDL_ID_SYNC(id,
+ my_sync_handler,
+ private,
+ free));
+
+
+ /* ... */
+
+ // unregistering the identified hdl
+ if (event_hdl_lookup_unsubscribe(NULL, id)) {
+ printf("private will automatically be freed!\n");
+ }
+```
+
+2.2 ASYNC MODE
+---------------------
+
+As mentioned before, async mode comes in 2 flavors, normal and task.
+
+2.2.1 NORMAL VERSION
+---------------------
+
+Normal is meant to be really easy to use, and highly compatible with sync mode.
+
+(Handler can easily be converted or copy pasted from async to sync mode
+and vice versa)
+
+Quick warning about sync to async handler conversion:
+
+please always use EVENT_HDL_ASSERT_SYNC whenever you develop a
+sync handler that performs unsafe data access.
+
+This way, if the handler were to be converted or copy pasted as is to
+async mode without removing unsafe data accesses,
+the handler will forcefully fail to indicate an error so that you
+know something has to be fixed in your handler code.
+
+Back to our async handler, let's say you want to declare an
+async handler that will be called when a new server is added.
+
+Here is what the handler function will look like:
+```
+void my_async_handler(const struct event_hdl_cb *cb, void *private)
+{
+ const struct event_hdl_cb_data_server *server = cb->e_data;
+
+ printf("I've been called for '%s', private = %p\n",
+ event_hdl_sub_type_to_string(cb->e_type), private);
+ printf("server name is '%s'\n", server->safe.name);
+
+ /* here it is not safe to use unsafe data */
+
+ /* from here you have the possibility to manage the subscription
+ * cb->sub_mgmt->unsub(cb->sub_mgmt);
+ * // hdl will be removed from the subscription list
+ */
+}
+```
+
+Note that it is pretty similar to sync handler, except
+for unsafe data access.
+
+Here is how you declare the subscription:
+
+anonymous subscription:
+```
+ int private = 10;
+
+ event_hdl_subscribe(NULL, EVENT_HDL_SUB_SERVER_ADD,
+ EVENT_HDL_ASYNC(my_async_handler, &private, NULL));
+```
+
+identified subscription:
+```
+ int private = 10;
+ uint64_t id = event_hdl_id("test", "async");
+
+ event_hdl_subscribe(NULL, EVENT_HDL_SUB_SERVER_ADD,
+ EVENT_HDL_ID_ASYNC(id,
+ my_async_handler,
+ &private,
+ NULL));
+
+```
+
+identified subscription where freeing private is required when subscription ends:
+(also works for anonymous)
+```
+ int *private = malloc(sizeof(*private));
+ uint64_t id = event_hdl_id("test", "async_free");
+
+ BUG_ON(!private);
+ *private = 10;
+
+ /* passing free as 'private_free' function so that
+ * private can be freed when unregistering is performed
+ */
+ event_hdl_subscribe(NULL, EVENT_HDL_SUB_SERVER_ADD,
+ EVENT_HDL_ID_ASYNC(id,
+ my_async_handler,
+ private,
+ free));
+
+ /* ... */
+
+ // unregistering the identified hdl
+ if (event_hdl_lookup_unsubscribe(NULL, id)) {
+ printf("private will automatically be freed when "
+ "all pending events referencing private "
+ "are consumed!\n");
+ }
+```
+
+2.2.2 TASK VERSION
+---------------------
+
+task version requires a bit more setup, but it's pretty
+straightforward actually.
+
+
+First, you need to initialize an event queue that will be used
+by event_hdl facility to push you events according to your subscription:
+
+```
+ event_hdl_async_equeue my_q;
+
+ event_hdl_async_equeue_init(&my_q);
+```
+
+
+Then, you need to declare a task(let) (or reuse existing task(let))
+
+It is your responsibility to make sure that the task(let) still exists
+(is not freed) when calling the subscribe function
+(and that the task remains valid as long as the subscription is).
+
+When a subscription referencing your task is over
+(either ended because of list purge, external code or from the handler itself),
+you will receive the EVENT_HDL_SUB_END event.
+When you receive this event, you must free it as usual and you can safely
+assume that the related subscription won't be sending you any more events.
+
+Here is what your task will look like (involving a single event queue):
+
+```
+struct task *event_hdl_async_task_my(struct task *task,
+ void *ctx, unsigned int state)
+{
+ struct tasklet *tl = (struct tasklet *)task;
+ event_hdl_async_equeue *queue = ctx;
+ struct event_hdl_async_event *event;
+ struct event_hdl_cb_data_server *srv;
+ uint8_t done = 0;
+
+ while ((event = event_hdl_async_equeue_pop(queue)))
+ {
+ if (event_hdl_sub_type_equal(event->type, EVENT_HDL_SUB_END)) {
+ done = 1;
+ event_hdl_async_free_event(event);
+ printf("no more events to come, "
+ "subscription is over\n");
+ break;
+ }
+
+ srv = event->data;
+
+ printf("task event %s, %d (name = %s)\n",
+ event_hdl_sub_type_to_string(event->type),
+ *((int *)event->private), srv->safe.name);
+ event_hdl_async_free_event(event);
+ }
+
+ if (done) {
+ /* our job is done, subscription is over:
+ * no more events to come
+ */
+ tasklet_free(tl);
+ return NULL;
+ }
+ return task;
+}
+
+```
+
+Here is how we would initialize the task event_hdl_async_task_my:
+```
+ struct tasklet *my_task;
+
+ my_task = tasklet_new();
+ BUG_ON(!my_task);
+ my_task->context = &my_q; // we declared my_q previously in this example
+ /* we declared event_hdl_async_task_my previously
+ * in this example
+ */
+ my_task->process = event_hdl_async_task_my;
+
+```
+
+Given our task and our previously initialized event queue, here is how
+to perform the subscription:
+```
+ int test_val = 11;
+ uint64_t id = event_hdl_id("test", "my_task");
+
+ /* anonymous variant */
+ event_hdl_subscribe(NULL, EVENT_HDL_SUB_SERVER_ADD,
+ EVENT_HDL_ASYNC_TASK(&my_q,
+ my_task,
+ &test_val,
+ NULL));
+ /* identified variant */
+ event_hdl_subscribe(NULL, EVENT_HDL_SUB_SERVER_ADD,
+ EVENT_HDL_ID_ASYNC_TASK(id,
+ &my_q,
+ my_task,
+ &test_val,
+ NULL));
+```
+
+Note: it is not recommended to perform multiple subscriptions
+ that share the same event queue or same task(let) (or both)
+
+ That is, having more than one subscription waking a task(let)
+ and/or feeding the same event queue.
+
+ No check is performed on this when registering, so the API
+ won't prevent you from doing it.
+
+ If you are going to do this anyway despite this warning:
+
+ In the case you need to stop the task prematurely
+ (if this is not going to happen please skip this paragraph):
+ You are responsible for acknowledging the end of every
+ active subscriptions that refer to your task or
+ your event queue(s).
+ And you really don't want a subscription associated with
+ your task or event queue to keep going when the task
+ is not active anymore because:
+ 1: there will be memory leak
+ (event queue might continue to receive new events)
+ 2: there is a 100% chance of process crash in case of event
+ because we will try to wake a task (your task)
+ that might already be freed. Thus UAF will occur.
+
+2.3 ADVANCED FEATURES
+-----------------------
+
+We've already covered some of these features in the previous examples.
+Here is a documented recap.
+
+
+2.3.1 SUB MGMT
+-----------------------
+
+From an event handler context, either sync or async mode:
+ You have the ability to directly manage the subscription
+ that provided the event.
+
+As of today, these actions are supported:
+ - Consulting the subscription.
+ - Modifying the subscription (resubscribing within same family)
+ - Unregistering the subscription (unsubscribing).
+
+To do this, consider the following structure:
+```
+ struct event_hdl_sub_mgmt
+ {
+ /* manage subscriptions from event
+ * this must not be used directly because
+ * locking might be required
+ */
+ struct event_hdl_sub *this;
+ /* safe functions than can be used from
+ * event context (sync and async mode)
+ */
+ struct event_hdl_sub_type (*getsub)(const struct event_hdl_sub_mgmt *);
+ int (*resub)(const struct event_hdl_sub_mgmt *, struct event_hdl_sub_type);
+ void (*unsub)(const struct event_hdl_sub_mgmt *);
+ };
+
+```
+A reference to this structure is provided in every handler mode.
+
+Sync mode and normal async mode (directly from the callback data pointer):
+```
+ const struct event_hdl_cb *cb;
+ // cb->sub_mgmt
+ // cb->sub_mgmt->getsub(cb->sub_mgmt);
+ // cb->sub_mgmt->unsub(cb->sub_mgmt);
+```
+
+task and notify async modes (from the event):
+```
+ struct event_hdl_async_event *event;
+ // event->sub_mgmt
+ // event->sub_mgmt.getsub(&event->sub_mgmt);
+ // event->sub_mgmt.unsub(&event->sub_mgmt);
+```
+
+2.3.2 SUBSCRIPTION EXTERNAL LOOKUPS
+-----------------------
+
+As you've seen in 2.3.1, managing the subscription directly
+from the handler is a possibility.
+
+But for identified subscriptions, you also have the ability to
+perform lookups and management operations on specific subscriptions
+within a list based on their ID, anywhere in the code.
+
+/!\ This feature is not available for anonymous subscriptions /!\
+
+Here are the actions already supported:
+
+ - unregistering a subscription (unsubscribing)
+ - updating a subscription (resubscribing within same family)
+ - getting a ptr/reference to the subscription
+
+Those functions are documented in event_hdl.h
+(search for EVENT_HDL_LOOKUP section).
+
+To select a specific subscription, you must provide
+the unique identifier (uint64_t hash) that was provided when subscribing.
+(using event_hdl_id(scope, name) function)
+
+Notes:
+ "id" is only unique within a given subscription list.
+
+ When using event_hdl_id to provide the id:
+ It is your responsibility to make sure that you "own"
+ the scope if you rely on name to be "free".
+
+ As ID computation is backed by xxhash hash API,
+ you should be aware that hash collisions could occur,
+ but are extremely rare and are thus considered safe
+ enough for this usage.
+ (see event_hdl.h for implementation details)
+
+ Please consider ptr based subscription management if
+ these limitations don't fit your requirements.
+
+Here are some examples:
+
+unsubscribing:
+```
+ /* registering "scope":"name" subscription */
+ event_hdl_subscribe(NULL, EVENT_HDL_SUB_SERVER_ADD,
+ EVENT_HDL_ID_SYNC(event_hdl_id("scope", "name"),
+ my_sync_handler,
+ NULL,
+ NULL));
+ /* unregistering "scope":"name" subscription */
+ event_hdl_lookup_unsubscribe(NULL, event_hdl_id("scope", "name"));
+```
+
+2.3.3 SUBSCRIPTION PTR
+-----------------------
+
+To manage existing subscriptions from external code,
+we already talked about identified subscriptions that
+allow lookups within list.
+
+But there is another way to accomplish this.
+
+When subscribing, you can use the event_hdl_subscribe_ptr() function
+variant (same arguments as event_hdl_subscribe()).
+
+What this function does, is instead of returning 1 in case of
+success and 0 in case of failure: it returns a valid subscription ptr
+for success and NULL for failure.
+
+Returned ptr is guaranteed to remain valid even if subscription
+is ended meanwhile because the ptr is internally guarded with a refcount.
+
+Thus, as long as you don't explicitly unregister the subscription with
+event_hdl_unsubscribe() or drop the reference using event_hdl_drop(),
+subscription ptr won't be freed.
+
+This ptr will allow you to use the following subscription
+management functions from external code:
+
+ - event_hdl_take() to increment subscription ptr refcount
+ (automatically incremented when using event_hdl_subscribe_ptr)
+ - event_hdl_drop() to decrement subscription ptr refcount
+ - event_hdl_resubscribe() to modify subscription subtype
+ - event_hdl_unsubscribe() to end the subscription
+ (refcount will be automatically decremented)
+
+Here is an example:
+```
+ struct event_hdl_sub *sub_ptr;
+
+ /* registering a subscription with subscribe_ptr */
+ sub_ptr = event_hdl_subscribe_ptr(NULL, EVENT_HDL_SUB_SERVER_ADD,
+ EVENT_HDL_SYNC(my_sync_handler,
+ NULL,
+ NULL));
+
+ /* ... */
+
+ /* unregistering the subscription */
+ event_hdl_unsubscribe(sub_ptr);
+```
+
+Regarding identified subscriptions that were registered using the non ptr
+subscribe function:
+
+You still have the ability to get a reference to the related subscription
+(if it still exists), by using event_hdl_lookup_take(list, id) function.
+event_hdl_lookup_take will return a subscription ptr in case of success
+and NULL in case of failure.
+Returned ptr reference is automatically incremented, so it is safe to use.
+
+Please don't forget to drop the reference
+when holding the ptr is no longer needed.
+
+Example:
+```
+ struct event_hdl_sub *sub_ptr = NULL;
+
+ /* registering subscription id "test":"ptr" with normal subscribe */
+ if (event_hdl_subscribe(NULL, EVENT_HDL_SUB_SERVER_ADD,
+ EVENT_HDL_ID_SYNC(event_hdl_id("test", "ptr"),
+ my_sync_handler,
+ NULL,
+ NULL))) {
+ /* fetch ref to subscription "test":"ptr" */
+ sub_ptr = event_hdl_lookup_take(NULL,
+ event_hdl_id("test", "ptr"));
+
+ /* unregister the subscription using lookup */
+ event_hdl_lookup_unsubscribe(NULL,
+ event_hdl_id("test", "ptr"));
+ }
+
+ /* ... */
+
+ /* unregistering the subscription with ptr
+ * will do nothing because subscription was
+ * already ended by lookup_unsubscribe, but
+ * here the catch is that sub_ptr is still
+ * valid so this won't crash the program
+ */
+ if (sub_ptr) {
+ event_hdl_unsubscribe(sub_ptr);
+ /* unsubscribe will also result in subscription
+ * reference drop, thus subscription will be freed here
+ * because sub_ptr was the last active reference.
+ * You must not use sub_ptr anymore past this point
+ * or UAF could occur
+ */
+ }
+
+```
+
+2.3.4 PRIVATE FREE
+-----------------------
+
+Upon handler subscription, you have the ability to provide
+a private data pointer that will be passed to the handler
+when subscribed events occur.
+
+Sometimes this private data pointer will rely on dynamically allocated memory.
+And in such cases, you have no way of knowing when
+freeing this pointer can be done safely.
+
+You could be tempted to think that freeing right after performing
+the unsubscription could be safe.
+But this is not the case, remember we could be dealing with async handlers
+that might still consume pending events even though unsubscription
+has been performed from external code.
+
+To deal with this, you may want to provide the private_free
+function pointer upon subscription.
+This way, private_free function will automatically be called
+(with private as argument) when private is no longer be used.
+
+Example:
+First we declare our private free function:
+```
+void my_private_free(void *my_private_data) {
+ /* here we only call free,
+ * but you could do more sophisticated stuff
+ */
+ free(my_private_data);
+}
+```
+Then:
+```
+ char *my_private_data = strdup("this string needs to be freed");
+
+ BUG_ON(!my_private_data);
+
+ event_hdl_subscribe(NULL, EVENT_HDL_SUB_SERVER_DEL,
+ EVENT_HDL_ID_ASYNC(event_hdl_id("test", "private"),
+ my_async_handler,
+ my_private_data,
+ my_private_free));
+
+ /* freeing my_private_data is not required anymore,
+ * it will be automatically freed by our private free
+ * function when subscription ends
+ */
+
+ /* unregistering "test":"private" subscription */
+ event_hdl_lookup_unsubscribe(NULL, event_hdl_id("test", "private"));
+
+ /* my_private_free will be automatically summoned when my_private_data
+ * is not referenced anymore
+ */
+```
+
+3 HOW TO ADD SUPPORT FOR NEW EVENTS
+-----------------------
+
+Adding support for a new event is pretty straightforward.
+
+First, you need to declare a new event subtype in event_hdl-t.h file
+(bottom of the file).
+
+You might want to declare a whole new event family, in which case
+you declare both the new family and the associated subtypes (if any).
+
+```
+ #define EVENT_HDL_SUB_NEW_FAMILY EVENT_HDL_SUB_FAMILY(4)
+ #define EVENT_HDL_SUB_NEW_FAMILY_SUBTYPE_1 EVENT_HDL_SUB_TYPE(4,0)
+```
+
+Then, you need to update the event_hdl_sub_type_map map,
+defined in src/event_hdl.c file (top of the file)
+to add string to event type and event type to string conversion support.
+You just need to add the missing entries corresponding to
+the event family / subtypes you've defined.
+
+Please follow this procedure:
+ You only added a new subtype to existing family: go to section 3.2
+ You added a new family: go to section 3.1
+
+3.1 DECLARING A NEW EVENT DATA STRUCTURE
+-----------------------
+
+You have the ability to provide additional data for a given
+event family when such events occur.
+
+Note that it is not mandatory: you could simply declare a new event family
+that does not provide any data.
+If this is your case, you can skip this section and go to 3.2 section.
+
+Now, take a look at this event data structure template
+(also defined at the top of event_hdl-t.h file):
+```
+ /* event data struct are defined as followed */
+ struct event_hdl_cb_data_template {
+ struct {
+ /* safe data can be safely used from both
+ * sync and async functions
+ * data consistency is guaranteed
+ */
+ } safe;
+ struct {
+ /* unsafe data may only be used from sync functions:
+ * in async mode, data consistency cannot be guaranteed
+ * and unsafe data may already be stale, thus using
+ * it is highly discouraged because it
+ * could lead to undefined behavior
+ * (UAF, null dereference...)
+ */
+ } unsafe;
+ };
+```
+
+This structure template allows you to easily create a new event
+data structure that can be provided with your new event family.
+
+You should name it after 'struct event_hdl_cb_data_new_family' so that it is
+easy to guess the event family it relates to.
+
+Indeed, each event data structure is to be associated with an
+unique event family type.
+For each subtypes within a family type, the associated data structure
+should be provided when publishing the event.
+
+The event data struct declaration should not be performed
+directly under event_hdl-t.h file:
+
+ It should be done in the header files of the corresponding
+ facility that will publish/provide this event.
+
+ Example: struct event_hdl_cb_data_server, provided for the
+ EVENT_HDL_SUB_SERVER event family, is going to be declared in
+ include/haproxy/server-t.h file.
+
+ However, in event_hdl-t.h, where you declare event family/subtypes,
+ you should add comments or links to the file containing the relevant
+ data struct declaration. This way we make sure all events related
+ information is centralized in event_hdl-t.h while keeping it clean
+ and not depending on any additional includes (you are free to
+ depend on specific data types within your custom event data structure).
+
+Please make sure that EVENT_HDL_ASYNC_EVENT_DATA (defined in event_hdl-t.h)
+is greater than sizeof(event_hdl_cb_data_new_family).
+
+It is required for async handlers to properly consume event data.
+
+You are free to adjust EVENT_HDL_ASYNC_EVENT_DATA size if needed.
+
+If EVENT_HDL_ASYNC_EVENT_DATA is not big enough to store your new
+event family struct, a compilation assert triggered by EVENT_HDL_CB_DATA
+will occur. In addition to this, an extra runtime BUG_ON will make
+sure the condition is met when publishing the event.
+The goal here is to force haproxy to fail explicitly so you know that
+something must be done on your side.
+
+3.1 PUBLISHING AN EVENT
+-----------------------
+
+Publishing an event is really simple.
+It relies on the event_hdl_publish function.
+
+The function is defined as follow:
+```
+ int event_hdl_publish(event_hdl_sub_list *sub_list,
+ event_hdl_sub_type e_type,
+ const struct event_hdl_cb_data *data);
+```
+
+We will ignore sub_list argument for now.
+In the examples below, we will use sub_list = NULL.
+Go to section 4 for a full picture about this feature.
+
+<e_type>: the event type that should be published.
+ All subscriptions referring to this event within
+ a subscription list context will be notified about the event.
+<data>: data provided for the event family of <e_type>
+ If <e_type>.family does not provide additional data,
+ data should be set to NULL.
+ If <e_type>.family does provide additional data, data should be set
+ using EVENT_HDL_CB_DATA macro.
+ (see the example below)
+
+The function returns 1 in case of SUCCESS (handlers successfully notified)
+and 0 in case of FAILURE (no handlers notified, because of memory error).
+
+Event publishing can be performed from anywhere in the code.
+(this example does not compile)
+```
+ struct event_hdl_cb_data_new_family event_data;
+
+ /* first we need to prepare event data
+ * that will be provided to event handlers
+ */
+
+ /* safe data, available from both sync and async contexts */
+ event_data.safe.my_custom_data = x;
+
+ /* unsafe data, only available from sync contexts */
+ event_data.unsafe.my_unsafe_data = y;
+
+ /* once data is prepared, we can publish the event */
+ event_hdl_publish(NULL,
+ EVENT_HDL_SUB_NEW_FAMILY_SUBTYPE_1,
+ EVENT_HDL_CB_DATA(&event_data));
+
+ /* EVENT_HDL_SUB_NEW_FAMILY_SUBTYPE_1 event was
+ * successfully published in global subscription list
+ */
+```
+
+--------------------------------------------------------------------------------
+|You should know that there is currently a limitation about publish function: |
+|The function should not be used from critical places |
+|(where the calling frequency is high |
+|or where timing sensitivity is high). |
+| |
+|Because in current implementation, subscription list lookups are not |
+|optimized for such uses cases. |
+--------------------------------------------------------------------------------
+
+4 SUBSCRIPTION LISTS
+-----------------------
+
+As you may already know, EVENT_HDL API main functions rely on
+subscription lists.
+Providing NULL where subscription list argument is required
+allows to use the implicit global subscription list.
+
+But you can also provide a specific subscription list, example:
+ subscription list associated with a single entity so that you only
+ subscribe to events of this single entity
+
+A subscription list is of type event_hdl_sub_list.
+It is defined in event_hdl-t.h
+
+To make use of this feature, you should know about these 2 functions:
+
+event_hdl_sub_list_init(list): use this fcn to initialize
+ a new subscription list.
+
+Example:
+```
+ event_hdl_sub_list my_custom_list;
+
+ event_hdl_sub_list_init(&my_custom_list);
+```
+
+event_hdl_sub_list_destroy(list): use this fcn to destroy
+ an existing subscription list.
+
+Example:
+```
+ event_hdl_sub_list_init(&my_custom_list);
+```
+
+ Using this function will cause all the existing subscriptions
+ within the provided sub_list to be properly unregistered
+ and deleted according to their types.
+
+Now we'll take another quick look at event_hdl_publish() function:
+
+Remember that the function is defined as follow:
+```
+ int event_hdl_publish(event_hdl_sub_list *sub_list,
+ event_hdl_sub_type e_type,
+ const struct event_hdl_cb_data *data);
+```
+
+In the previous examples, we used sub_list = NULL.
+
+if sub_list is NULL:
+ event will be published in in global list
+else
+ event will be published in user specified sub_list
+
+5 MISC/HELPER FUNCTIONS
+-----------------------
+
+Don't forget to take a look at MISC/HELPER FUNCTIONS in
+include/haproxy/event_hdl.h (end of the file) for a
+complete list of helper functions / macros.
+
+We've already used some, if not the vast majority
+in the examples shown in this document.
+
+This includes, to name a few:
+ - event types manipulation
+ - event types comparison
+ - lookup id computing
+ - subscriber list management (covered in section 4)
+ - sync/async handler helpers
diff --git a/doc/internals/api/filters.txt b/doc/internals/api/filters.txt
new file mode 100644
index 0000000..f1d2f34
--- /dev/null
+++ b/doc/internals/api/filters.txt
@@ -0,0 +1,1188 @@
+ -----------------------------------------
+ Filters Guide - version 2.9
+ ( Last update: 2021-02-24 )
+ ------------------------------------------
+ Author : Christopher Faulet
+ Contact : christopher dot faulet at capflam dot org
+
+
+ABSTRACT
+--------
+
+The filters support is a new feature of HAProxy 1.7. It is a way to extend
+HAProxy without touching its core code and, in certain extent, without knowing
+its internals. This feature will ease contributions, reducing impact of
+changes. Another advantage will be to simplify HAProxy by replacing some parts
+by filters. As we will see, and as an example, the HTTP compression is the first
+feature moved in a filter.
+
+This document describes how to write a filter and what to keep in mind to do
+so. It also talks about the known limits and the pitfalls to avoid.
+
+As said, filters are quite new for now. The API is not freezed and will be
+updated/modified/improved/extended as needed.
+
+
+
+SUMMARY
+-------
+
+ 1. Filters introduction
+ 2. How to use filters
+ 3. How to write a new filter
+ 3.1. API Overview
+ 3.2. Defining the filter name and its configuration
+ 3.3. Managing the filter lifecycle
+ 3.3.1. Dealing with threads
+ 3.4. Handling the streams activity
+ 3.5. Analyzing the channels activity
+ 3.6. Filtering the data exchanged
+ 4. FAQ
+
+
+
+1. FILTERS INTRODUCTION
+-----------------------
+
+First of all, to fully understand how filters work and how to create one, it is
+best to know, at least from a distance, what is a proxy (frontend/backend), a
+stream and a channel in HAProxy and how these entities are linked to each other.
+In doc/internals/api/layers.txt is a good overview of the different layers in
+HAProxy and in doc/internals/muxes.pdf is described the flow between the
+different muxes.
+
+Then, to support filters, many callbacks has been added to HAProxy at different
+places, mainly around channel analyzers. Their purpose is to allow filters to
+be involved in the data processing, from the stream creation/destruction to
+the data forwarding. Depending of what it should do, a filter can implement all
+or part of these callbacks. For now, existing callbacks are focused on
+streams. But future improvements could enlarge filters scope. For instance, it
+could be useful to handle events at the connection level.
+
+In HAProxy configuration file, a filter is declared in a proxy section, except
+default. So the configuration corresponding to a filter declaration is attached
+to a specific proxy, and will be shared by all its instances. it is opaque from
+the HAProxy point of view, this is the filter responsibility to manage it. For
+each filter declaration matches a uniq configuration. Several declarations of
+the same filter in the same proxy will be handle as different filters by
+HAProxy.
+
+A filter instance is represented by a partially opaque context (or a state)
+attached to a stream and passed as arguments to callbacks. Through this context,
+filter instances are stateful. Depending the filter is declared in a frontend or
+a backend section, its instances will be created, respectively, when a stream is
+created or when a backend is selected. Their behaviors will also be
+different. Only instances of filters declared in a frontend section will be
+aware of the creation and the destruction of the stream, and will take part in
+the channels analyzing before the backend is defined.
+
+It is important to remember the configuration of a filter is shared by all its
+instances, while the context of an instance is owned by a uniq stream.
+
+Filters are designed to be chained. It is possible to declare several filters in
+the same proxy section. The declaration order is important because filters will
+be called one after the other respecting this order. Frontend and backend
+filters are also chained, frontend ones called first. Even if the filters
+processing is serialized, each filter will bahave as it was alone (unless it was
+developed to be aware of other filters). For all that, some constraints are
+imposed to filters, especially when data exchanged between the client and the
+server are processed. We will discuss again these constraints when we will tackle
+the subject of writing a filter.
+
+
+
+2. HOW TO USE FILTERS
+---------------------
+
+To use a filter, the parameter 'filter' should be used, followed by the filter
+name and, optionally, its configuration in the desired listen, frontend or
+backend section. For instance :
+
+ listen test
+ ...
+ filter trace name TST
+ ...
+
+
+See doc/configuration.txt for a formal definition of the parameter 'filter'.
+Note that additional parameters on the filter line must be parsed by the filter
+itself.
+
+The list of available filters is reported by 'haproxy -vv' :
+
+ $> haproxy -vv
+ HAProxy version 1.7-dev2-3a1d4a-33 2016/03/21
+ Copyright 2000-2016 Willy Tarreau <willy@haproxy.org>
+
+ [...]
+
+ Available filters :
+ [COMP] compression
+ [TRACE] trace
+
+
+Multiple filter lines can be used in a proxy section to chain filters. Filters
+will be called in the declaration order.
+
+Some filters can support implicit declarations in certain circumstances
+(without the filter line). This is not recommended for new features but are
+useful for existing ones moved in a filter, for backward compatibility
+reasons. Implicit declarations are supported when there is only one filter used
+on a proxy. When several filters are used, explicit declarations are mandatory.
+The HTTP compression filter is one of these filters. Alone, using 'compression'
+keywords is enough to use it. But when at least a second filter is used, a
+filter line must be added.
+
+ # filter line is optional
+ listen t1
+ bind *:80
+ compression algo gzip
+ compression offload
+ server srv x.x.x.x:80
+
+ # filter line is mandatory for the compression filter
+ listen t2
+ bind *:81
+ filter trace name T2
+ filter compression
+ compression algo gzip
+ compression offload
+ server srv x.x.x.x:80
+
+
+
+
+3. HOW TO WRITE A NEW FILTER
+----------------------------
+
+To write a filter, there are 2 header files to explore :
+
+ * include/haproxy/filters-t.h : This is the main header file, containing all
+ important structures to use. It represents the
+ filter API.
+
+ * include/haproxy/filters.h : This header file contains helper functions that
+ may be used. It also contains the internal API
+ used by HAProxy to handle filters.
+
+To ease the filters integration, it is better to follow some conventions :
+
+ * Use 'flt_' prefix to name the filter (e.g flt_http_comp or flt_trace).
+
+ * Keep everything related to the filter in a same file.
+
+The filter 'trace' can be used as a template to write new filter. It is a good
+start to see how filters really work.
+
+3.1 API OVERVIEW
+----------------
+
+Writing a filter can be summarized to write functions and attach them to the
+existing callbacks. Available callbacks are listed in the following structure :
+
+ struct flt_ops {
+ /*
+ * Callbacks to manage the filter lifecycle
+ */
+ int (*init) (struct proxy *p, struct flt_conf *fconf);
+ void (*deinit) (struct proxy *p, struct flt_conf *fconf);
+ int (*check) (struct proxy *p, struct flt_conf *fconf);
+ int (*init_per_thread) (struct proxy *p, struct flt_conf *fconf);
+ void (*deinit_per_thread)(struct proxy *p, struct flt_conf *fconf);
+
+ /*
+ * Stream callbacks
+ */
+ int (*attach) (struct stream *s, struct filter *f);
+ int (*stream_start) (struct stream *s, struct filter *f);
+ int (*stream_set_backend)(struct stream *s, struct filter *f, struct proxy *be);
+ void (*stream_stop) (struct stream *s, struct filter *f);
+ void (*detach) (struct stream *s, struct filter *f);
+ void (*check_timeouts) (struct stream *s, struct filter *f);
+
+ /*
+ * Channel callbacks
+ */
+ int (*channel_start_analyze)(struct stream *s, struct filter *f,
+ struct channel *chn);
+ int (*channel_pre_analyze) (struct stream *s, struct filter *f,
+ struct channel *chn,
+ unsigned int an_bit);
+ int (*channel_post_analyze) (struct stream *s, struct filter *f,
+ struct channel *chn,
+ unsigned int an_bit);
+ int (*channel_end_analyze) (struct stream *s, struct filter *f,
+ struct channel *chn);
+
+ /*
+ * HTTP callbacks
+ */
+ int (*http_headers) (struct stream *s, struct filter *f,
+ struct http_msg *msg);
+ int (*http_payload) (struct stream *s, struct filter *f,
+ struct http_msg *msg, unsigned int offset,
+ unsigned int len);
+ int (*http_end) (struct stream *s, struct filter *f,
+ struct http_msg *msg);
+
+ void (*http_reset) (struct stream *s, struct filter *f,
+ struct http_msg *msg);
+ void (*http_reply) (struct stream *s, struct filter *f,
+ short status,
+ const struct buffer *msg);
+
+ /*
+ * TCP callbacks
+ */
+ int (*tcp_payload) (struct stream *s, struct filter *f,
+ struct channel *chn, unsigned int offset,
+ unsigned int len);
+ };
+
+
+We will explain in following parts when these callbacks are called and what they
+should do.
+
+Filters are declared in proxy sections. So each proxy have an ordered list of
+filters, possibly empty if no filter is used. When the configuration of a proxy
+is parsed, each filter line represents an entry in this list. In the structure
+'proxy', the filters configurations are stored in the field 'filter_configs',
+each one of type 'struct flt_conf *' :
+
+ /*
+ * Structure representing the filter configuration, attached to a proxy and
+ * accessible from a filter when instantiated in a stream
+ */
+ struct flt_conf {
+ const char *id; /* The filter id */
+ struct flt_ops *ops; /* The filter callbacks */
+ void *conf; /* The filter configuration */
+ struct list list; /* Next filter for the same proxy */
+ unsigned int flags; /* FLT_CFG_FL_* */
+ };
+
+ * 'flt_conf.id' is an identifier, defined by the filter. It can be
+ NULL. HAProxy does not use this field. Filters can use it in log messages or
+ as a uniq identifier to check multiple declarations. It is the filter
+ responsibility to free it, if necessary.
+
+ * 'flt_conf.conf' is opaque. It is the internal configuration of a filter,
+ generally allocated and filled by its parsing function (See § 3.2). It is
+ the filter responsibility to free it.
+
+ * 'flt_conf.ops' references the callbacks implemented by the filter. This
+ field must be set during the parsing phase (See § 3.2) and can be refine
+ during the initialization phase (See § 3.3). If it is dynamically allocated,
+ it is the filter responsibility to free it.
+
+ * 'flt_conf.flags' is a bitfield to specify the filter capabilities. For now,
+ only FLT_CFG_FL_HTX may be set when a filter is able to process HTX
+ streams. If not set, the filter is excluded from the HTTP filtering.
+
+
+The filter configuration is global and shared by all its instances. A filter
+instance is created in the context of a stream and attached to this stream. in
+the structure 'stream', the field 'strm_flt' is the state of all filter
+instances attached to a stream :
+
+ /*
+ * Structure representing the "global" state of filters attached to a
+ * stream.
+ */
+ struct strm_flt {
+ struct list filters; /* List of filters attached to a stream */
+ struct filter *current[2]; /* From which filter resume processing, for a specific channel.
+ * This is used for resumable callbacks only,
+ * If NULL, we start from the first filter.
+ * 0: request channel, 1: response channel */
+ unsigned short flags; /* STRM_FL_* */
+ unsigned char nb_req_data_filters; /* Number of data filters registered on the request channel */
+ unsigned char nb_rsp_data_filters; /* Number of data filters registered on the response channel */
+ unsigned long long offset[2]; /* gloal offset of input data already filtered for a specific channel
+ * 0: request channel, 1: response channel */
+ };
+
+
+Filter instances attached to a stream are stored in the field
+'strm_flt.filters', each instance is of type 'struct filter *' :
+
+ /*
+ * Structure representing a filter instance attached to a stream
+ *
+ * 2D-Array fields are used to store info per channel. The first index
+ * stands for the request channel, and the second one for the response
+ * channel. Especially, <next> and <fwd> are offsets representing amount of
+ * data that the filter are, respectively, parsed and forwarded on a
+ * channel. Filters can access these values using FLT_NXT and FLT_FWD
+ * macros.
+ */
+ struct filter {
+ struct flt_conf *config; /* the filter's configuration */
+ void *ctx; /* The filter context (opaque) */
+ unsigned short flags; /* FLT_FL_* */
+ unsigned long long offset[2]; /* Offset of input data already filtered for a specific channel
+ * 0: request channel, 1: response channel */
+ unsigned int pre_analyzers; /* bit field indicating analyzers to
+ * pre-process */
+ unsigned int post_analyzers; /* bit field indicating analyzers to
+ * post-process */
+ struct list list; /* Next filter for the same proxy/stream */
+ };
+
+ * 'filter.config' is the filter configuration previously described. All
+ instances of a filter share it.
+
+ * 'filter.ctx' is an opaque context. It is managed by the filter, so it is its
+ responsibility to free it.
+
+ * 'filter.pre_analyzers and 'filter.post_analyzers will be described later
+ (See § 3.5).
+
+ * 'filter.offset' will be described later (See § 3.6).
+
+
+3.2. DEFINING THE FILTER NAME AND ITS CONFIGURATION
+---------------------------------------------------
+
+During the filter development, the first thing to do is to add it in the
+supported filters. To do so, its name must be registered as a valid keyword on
+the filter line :
+
+ /* Declare the filter parser for "my_filter" keyword */
+ static struct flt_kw_list flt_kws = { "MY_FILTER_SCOPE", { }, {
+ { "my_filter", parse_my_filter_cfg, NULL /* private data */ },
+ { NULL, NULL, NULL },
+ }
+ };
+ INITCALL1(STG_REGISTER, flt_register_keywords, &flt_kws);
+
+
+Then the filter internal configuration must be defined. For instance :
+
+ struct my_filter_config {
+ struct proxy *proxy;
+ char *name;
+ /* ... */
+ };
+
+
+All callbacks implemented by the filter must then be declared. Here, a global
+variable is used :
+
+ struct flt_ops my_filter_ops {
+ .init = my_filter_init,
+ .deinit = my_filter_deinit,
+ .check = my_filter_config_check,
+
+ /* ... */
+ };
+
+
+Finally, the function to parse the filter configuration must be written, here
+'parse_my_filter_cfg'. This function must parse all remaining keywords on the
+filter line :
+
+ /* Return -1 on error, else 0 */
+ static int
+ parse_my_filter_cfg(char **args, int *cur_arg, struct proxy *px,
+ struct flt_conf *flt_conf, char **err, void *private)
+ {
+ struct my_filter_config *my_conf;
+ int pos = *cur_arg;
+
+ /* Allocate the internal configuration used by the filter */
+ my_conf = calloc(1, sizeof(*my_conf));
+ if (!my_conf) {
+ memprintf(err, "%s : out of memory", args[*cur_arg]);
+ return -1;
+ }
+ my_conf->proxy = px;
+
+ /* ... */
+
+ /* Parse all keywords supported by the filter and fill the internal
+ * configuration */
+ pos++; /* Skip the filter name */
+ while (*args[pos]) {
+ if (!strcmp(args[pos], "name")) {
+ if (!*args[pos + 1]) {
+ memprintf(err, "'%s' : '%s' option without value",
+ args[*cur_arg], args[pos]);
+ goto error;
+ }
+ my_conf->name = strdup(args[pos + 1]);
+ if (!my_conf->name) {
+ memprintf(err, "%s : out of memory", args[*cur_arg]);
+ goto error;
+ }
+ pos += 2;
+ }
+
+ /* ... parse other keywords ... */
+ }
+ *cur_arg = pos;
+
+ /* Set callbacks supported by the filter */
+ flt_conf->ops = &my_filter_ops;
+
+ /* Last, save the internal configuration */
+ flt_conf->conf = my_conf;
+ return 0;
+
+ error:
+ if (my_conf->name)
+ free(my_conf->name);
+ free(my_conf);
+ return -1;
+ }
+
+
+WARNING : In this parsing function, 'flt_conf->ops' must be initialized. All
+ arguments of the filter line must also be parsed. This is mandatory.
+
+In the previous example, the filter lne should be read as follows :
+
+ filter my_filter name MY_NAME ...
+
+
+Optionally, by implementing the 'flt_ops.check' callback, an extra set is added
+to check the internal configuration of the filter after the parsing phase, when
+the HAProxy configuration is fully defined. For instance :
+
+ /* Check configuration of a trace filter for a specified proxy.
+ * Return 1 on error, else 0. */
+ static int
+ my_filter_config_check(struct proxy *px, struct flt_conf *my_conf)
+ {
+ if (px->mode != PR_MODE_HTTP) {
+ Alert("The filter 'my_filter' cannot be used in non-HTTP mode.\n");
+ return 1;
+ }
+
+ /* ... */
+
+ return 0;
+ }
+
+
+
+3.3. MANAGING THE FILTER LIFECYCLE
+----------------------------------
+
+Once the configuration parsed and checked, filters are ready to by used. There
+are two main callbacks to manage the filter lifecycle :
+
+ * 'flt_ops.init' : It initializes the filter for a proxy. This callback may be
+ defined to finish the filter configuration.
+
+ * 'flt_ops.deinit' : It cleans up what the parsing function and the init
+ callback have done. This callback is useful to release
+ memory allocated for the filter configuration.
+
+Here is an example :
+
+ /* Initialize the filter. Returns -1 on error, else 0. */
+ static int
+ my_filter_init(struct proxy *px, struct flt_conf *fconf)
+ {
+ struct my_filter_config *my_conf = fconf->conf;
+
+ /* ... */
+
+ return 0;
+ }
+
+ /* Free resources allocated by the trace filter. */
+ static void
+ my_filter_deinit(struct proxy *px, struct flt_conf *fconf)
+ {
+ struct my_filter_config *my_conf = fconf->conf;
+
+ if (my_conf) {
+ free(my_conf->name);
+ /* ... */
+ free(my_conf);
+ }
+ fconf->conf = NULL;
+ }
+
+
+3.3.1 DEALING WITH THREADS
+--------------------------
+
+When HAProxy is compiled with the threads support and started with more that one
+thread (global.nbthread > 1), then it is possible to manage the filter per
+thread with following callbacks :
+
+ * 'flt_ops.init_per_thread': It initializes the filter for each thread. It
+ works the same way than 'flt_ops.init' but in the
+ context of a thread. This callback is called
+ after the thread creation.
+
+ * 'flt_ops.deinit_per_thread': It cleans up what the init_per_thread callback
+ have done. It is called in the context of a
+ thread, before exiting it.
+
+It is the filter responsibility to deal with concurrency. check, init and deinit
+callbacks are called on the main thread. All others are called on a "worker"
+thread (not always the same). It is also the filter responsibility to know if
+HAProxy is started with more than one thread. If it is started with one thread
+(or compiled without the threads support), these callbacks will be silently
+ignored (in this case, global.nbthread will be always equal to one).
+
+
+3.4. HANDLING THE STREAMS ACTIVITY
+-----------------------------------
+
+It may be interesting to handle streams activity. For now, there is three
+callbacks that should define to do so :
+
+ * 'flt_ops.stream_start' : It is called when a stream is started. This
+ callback can fail by returning a negative value. It
+ will be considered as a critical error by HAProxy
+ which disabled the listener for a short time.
+
+ * 'flt_ops.stream_set_backend' : It is called when a backend is set for a
+ stream. This callbacks will be called for all
+ filters attached to a stream (frontend and
+ backend). Note this callback is not called if
+ the frontend and the backend are the same.
+
+ * 'flt_ops.stream_stop' : It is called when a stream is stopped. This callback
+ always succeed. Anyway, it is too late to return an
+ error.
+
+For instance :
+
+ /* Called when a stream is created. Returns -1 on error, else 0. */
+ static int
+ my_filter_stream_start(struct stream *s, struct filter *filter)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+
+ /* ... */
+
+ return 0;
+ }
+
+ /* Called when a backend is set for a stream */
+ static int
+ my_filter_stream_set_backend(struct stream *s, struct filter *filter,
+ struct proxy *be)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+
+ /* ... */
+
+ return 0;
+ }
+
+ /* Called when a stream is destroyed */
+ static void
+ my_filter_stream_stop(struct stream *s, struct filter *filter)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+
+ /* ... */
+ }
+
+
+WARNING : Handling the streams creation and destruction is only possible for
+ filters defined on proxies with the frontend capability.
+
+In addition, it is possible to handle creation and destruction of filter
+instances using following callbacks:
+
+ * 'flt_ops.attach' : It is called after a filter instance creation, when it is
+ attached to a stream. This happens when the stream is
+ started for filters defined on the stream's frontend and
+ when the backend is set for filters declared on the
+ stream's backend. It is possible to ignore the filter, if
+ needed, by returning 0. This could be useful to have
+ conditional filtering.
+
+ * 'flt_ops.detach' : It is called when a filter instance is detached from a
+ stream, before its destruction. This happens when the
+ stream is stopped for filters defined on the stream's
+ frontend and when the analyze ends for filters defined on
+ the stream's backend.
+
+For instance :
+
+ /* Called when a filter instance is created and attach to a stream */
+ static int
+ my_filter_attach(struct stream *s, struct filter *filter)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+
+ if (/* ... */)
+ return 0; /* Ignore the filter here */
+ return 1;
+ }
+
+ /* Called when a filter instance is detach from a stream, just before its
+ * destruction */
+ static void
+ my_filter_detach(struct stream *s, struct filter *filter)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+
+ /* ... */
+ }
+
+Finally, it may be interesting to notify the filter when the stream is woken up
+because of an expired timer. This could let a chance to check some internal
+timeouts, if any. To do so the following callback must be used :
+
+ * 'flt_opt.check_timeouts' : It is called when a stream is woken up because of
+ an expired timer.
+
+For instance :
+
+ /* Called when a stream is woken up because of an expired timer */
+ static void
+ my_filter_check_timeouts(struct stream *s, struct filter *filter)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+
+ /* ... */
+ }
+
+
+3.5. ANALYZING THE CHANNELS ACTIVITY
+------------------------------------
+
+The main purpose of filters is to take part in the channels analyzing. To do so,
+there is 2 callbacks, 'flt_ops.channel_pre_analyze' and
+'flt_ops.channel_post_analyze', called respectively before and after each
+analyzer attached to a channel, except analyzers responsible for the data
+forwarding (TCP or HTTP). Concretely, on the request channel, these callbacks
+could be called before following analyzers :
+
+ * tcp_inspect_request (AN_REQ_INSPECT_FE and AN_REQ_INSPECT_BE)
+ * http_wait_for_request (AN_REQ_WAIT_HTTP)
+ * http_wait_for_request_body (AN_REQ_HTTP_BODY)
+ * http_process_req_common (AN_REQ_HTTP_PROCESS_FE)
+ * process_switching_rules (AN_REQ_SWITCHING_RULES)
+ * http_process_req_ common (AN_REQ_HTTP_PROCESS_BE)
+ * http_process_tarpit (AN_REQ_HTTP_TARPIT)
+ * process_server_rules (AN_REQ_SRV_RULES)
+ * http_process_request (AN_REQ_HTTP_INNER)
+ * tcp_persist_rdp_cookie (AN_REQ_PRST_RDP_COOKIE)
+ * process_sticking_rules (AN_REQ_STICKING_RULES)
+
+And on the response channel :
+
+ * tcp_inspect_response (AN_RES_INSPECT)
+ * http_wait_for_response (AN_RES_WAIT_HTTP)
+ * process_store_rules (AN_RES_STORE_RULES)
+ * http_process_res_common (AN_RES_HTTP_PROCESS_BE)
+
+Unlike the other callbacks previously seen before, 'flt_ops.channel_pre_analyze'
+can interrupt the stream processing. So a filter can decide to not execute the
+analyzer that follows and wait the next iteration. If there are more than one
+filter, following ones are skipped. On the next iteration, the filtering resumes
+where it was stopped, i.e. on the filter that has previously stopped the
+processing. So it is possible for a filter to stop the stream processing on a
+specific analyzer for a while before continuing. Moreover, this callback can be
+called many times for the same analyzer, until it finishes its processing. For
+instance :
+
+ /* Called before a processing happens on a given channel.
+ * Returns a negative value if an error occurs, 0 if it needs to wait,
+ * any other value otherwise. */
+ static int
+ my_filter_chn_pre_analyze(struct stream *s, struct filter *filter,
+ struct channel *chn, unsigned an_bit)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+
+ switch (an_bit) {
+ case AN_REQ_WAIT_HTTP:
+ if (/* wait that a condition is verified before continuing */)
+ return 0;
+ break;
+ /* ... * /
+ }
+ return 1;
+ }
+
+ * 'an_bit' is the analyzer id. All analyzers are listed in
+ 'include/haproxy/channels-t.h'.
+
+ * 'chn' is the channel on which the analyzing is done. It is possible to
+ determine if it is the request or the response channel by testing if
+ CF_ISRESP flag is set :
+
+ │ ((chn->flags & CF_ISRESP) == CF_ISRESP)
+
+
+In previous example, the stream processing is blocked before receipt of the HTTP
+request until a condition is verified.
+
+'flt_ops.channel_post_analyze', for its part, is not resumable. It returns a
+negative value if an error occurs, any other value otherwise. It is called when
+a filterable analyzer finishes its processing, so once for the same analyzer.
+For instance :
+
+ /* Called after a processing happens on a given channel.
+ * Returns a negative value if an error occurs, any other
+ * value otherwise. */
+ static int
+ my_filter_chn_post_analyze(struct stream *s, struct filter *filter,
+ struct channel *chn, unsigned an_bit)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+ struct http_msg *msg;
+
+ switch (an_bit) {
+ case AN_REQ_WAIT_HTTP:
+ if (/* A test on received headers before any other treatment */) {
+ msg = ((chn->flags & CF_ISRESP) ? &s->txn->rsp : &s->txn->req);
+ txn->status = 400;
+ msg->msg_state = HTTP_MSG_ERROR;
+ http_reply_and_close(s, s->txn->status, http_error_message(s));
+ return -1; /* This is an error ! */
+ }
+ break;
+ /* ... * /
+ }
+ return 1;
+ }
+
+
+Pre and post analyzer callbacks of a filter are not automatically called. They
+must be regiesterd explicitly on analyzers, updating the value of
+'filter.pre_analyzers' and 'filter.post_analyzers' bit fields. All analyzer bits
+are listed in 'include/types/channels.h'. Here is an example :
+
+ static int
+ my_filter_stream_start(struct stream *s, struct filter *filter)
+ {
+ /* ... * /
+
+ /* Register the pre analyzer callback on all request and response
+ * analyzers */
+ filter->pre_analyzers |= (AN_REQ_ALL | AN_RES_ALL)
+
+ /* Register the post analyzer callback of only on AN_REQ_WAIT_HTTP and
+ * AN_RES_WAIT_HTTP analyzers */
+ filter->post_analyzers |= (AN_REQ_WAIT_HTTP | AN_RES_WAIT_HTTP)
+
+ /* ... * /
+ return 0;
+ }
+
+
+To surround activity of a filter during the channel analyzing, two new analyzers
+has been added :
+
+ * 'flt_start_analyze' (AN_REQ/RES_FLT_START_FE/AN_REQ_RES_FLT_START_BE) : For
+ a specific filter, this analyzer is called before any call to the
+ 'channel_analyze' callback. From the filter point of view, it calls the
+ 'flt_ops.channel_start_analyze' callback.
+
+ * 'flt_end_analyze' (AN_REQ/RES_FLT_END) : For a specific filter, this
+ analyzer is called when all other analyzers have finished their
+ processing. From the filter point of view, it calls the
+ 'flt_ops.channel_end_analyze' callback.
+
+These analyzers are called only once per streams.
+
+'flt_ops.channel_start_analyze' and 'flt_ops.channel_end_analyze' callbacks can
+interrupt the stream processing, as 'flt_ops.channel_analyze'. Here is an
+example :
+
+ /* Called when analyze starts for a given channel
+ * Returns a negative value if an error occurs, 0 if it needs to wait,
+ * any other value otherwise. */
+ static int
+ my_filter_chn_start_analyze(struct stream *s, struct filter *filter,
+ struct channel *chn)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+
+ /* ... TODO ... */
+
+ return 1;
+ }
+
+ /* Called when analyze ends for a given channel
+ * Returns a negative value if an error occurs, 0 if it needs to wait,
+ * any other value otherwise. */
+ static int
+ my_filter_chn_end_analyze(struct stream *s, struct filter *filter,
+ struct channel *chn)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+
+ /* ... TODO ... */
+
+ return 1;
+ }
+
+
+Workflow on channels can be summarized as following :
+
+ FE: Called for filters defined on the stream's frontend
+ BE: Called for filters defined on the stream's backend
+
+ +------->---------+
+ | | |
+ +----------------------+ | +----------------------+
+ | flt_ops.attach (FE) | | | flt_ops.attach (BE) |
+ +----------------------+ | +----------------------+
+ | | |
+ V | V
+ +--------------------------+ | +------------------------------------+
+ | flt_ops.stream_start (FE)| | | flt_ops.stream_set_backend (FE+BE) |
+ +--------------------------+ | +------------------------------------+
+ | | |
+ ... | ...
+ | | |
+ | ^ |
+ | --+ | | --+
+ +------<----------+ | | +--------<--------+ |
+ | | | | | | |
+ V | | | V | |
++-------------------------------+ | | | +-------------------------------+ | |
+| flt_start_analyze (FE) +-+ | | | flt_start_analyze (BE) +-+ |
+|(flt_ops.channel_start_analyze)| | F | |(flt_ops.channel_start_analyze)| |
++---------------+---------------+ | R | +-------------------------------+ |
+ | | O | | |
+ +------<---------+ | N ^ +--------<-------+ | B
+ | | | T | | | | A
++---------------|------------+ | | E | +---------------|------------+ | | C
+|+--------------V-------------+ | | N | |+--------------V-------------+ | | K
+||+----------------------------+ | | D | ||+----------------------------+ | | E
+|||flt_ops.channel_pre_analyze | | | | |||flt_ops.channel_pre_analyze | | | N
+||| V | | | | ||| V | | | D
+||| analyzer (FE) +-+ | | ||| analyzer (FE+BE) +-+ |
++|| V | | | +|| V | |
+ +|flt_ops.channel_post_analyze| | | +|flt_ops.channel_post_analyze| |
+ +----------------------------+ | | +----------------------------+ |
+ | --+ | | |
+ +------------>------------+ ... |
+ | |
+ [ data filtering (see below) ] |
+ | |
+ ... |
+ | |
+ +--------<--------+ |
+ | | |
+ V | |
+ +-------------------------------+ | |
+ | flt_end_analyze (FE+BE) +-+ |
+ | (flt_ops.channel_end_analyze) | |
+ +---------------+---------------+ |
+ | --+
+ V
+ +----------------------+
+ | flt_ops.detach (BE) |
+ +----------------------+
+ |
+ V
+ +--------------------------+
+ | flt_ops.stream_stop (FE) |
+ +--------------------------+
+ |
+ V
+ +----------------------+
+ | flt_ops.detach (FE) |
+ +----------------------+
+ |
+ V
+
+By zooming on an analyzer box we have:
+
+ ...
+ |
+ V
+ |
+ +-----------<-----------+
+ | |
+ +-----------------+--------------------+ |
+ | | | |
+ | +--------<---------+ | |
+ | | | | |
+ | V | | |
+ | flt_ops.channel_pre_analyze ->-+ | ^
+ | | | |
+ | | | |
+ | V | |
+ | analyzer --------->-----+--+
+ | | |
+ | | |
+ | V |
+ | flt_ops.channel_post_analyze |
+ | | |
+ | | |
+ +-----------------+--------------------+
+ |
+ V
+ ...
+
+
+ 3.6. FILTERING THE DATA EXCHANGED
+-----------------------------------
+
+WARNING : To fully understand this part, it is important to be aware on how the
+ buffers work in HAProxy. For the HTTP part, it is also important to
+ understand how data are parsed and structured, and how the internal
+ representation, called HTX, works. See doc/internals/buffer-api.txt
+ and doc/internals/htx-api.txt for details.
+
+An extended feature of the filters is the data filtering. By default a filter
+does not look into data exchanged between the client and the server because it
+is expensive. Indeed, instead of forwarding data without any processing, each
+byte need to be buffered.
+
+So, to enable the data filtering on a channel, at any time, in one of previous
+callbacks, 'register_data_filter' function must be called. And conversely, to
+disable it, 'unregister_data_filter' function must be called. For instance :
+
+ my_filter_http_headers(struct stream *s, struct filter *filter,
+ struct http_msg *msg)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+
+ /* 'chn' must be the request channel */
+ if (!(msg->chn->flags & CF_ISRESP)) {
+ struct htx *htx;
+ struct ist hdr;
+ struct http_hdr_ctx ctx;
+
+ htx = htxbuf(msg->chn->buf);
+
+ /* Enable the data filtering for the request if 'X-Filter' header
+ * is set to 'true'. */
+ hdr = ist("X-Filter);
+ ctx.blk = NULL;
+ if (http_find_header(htx, hdr, &ctx, 0) &&
+ ctx.value.len >= 4 && memcmp(ctx.value.ptr, "true", 4) == 0)
+ register_data_filter(s, chn, filter);
+ }
+
+ return 1;
+ }
+
+Here, the data filtering is enabled if the HTTP header 'X-Filter' is found and
+set to 'true'.
+
+If several filters are declared, the evaluation order remains the same,
+regardless the order of the registrations to the data filtering. Data
+registrations must be performed before the data forwarding step. However, a
+filter may be unregistered from the data filtering at any time.
+
+Depending on the stream type, TCP or HTTP, the way to handle data filtering is
+different. HTTP data are structured while TCP data are raw. And there are more
+callbacks for HTTP streams to fully handle all steps of an HTTP transaction. But
+the main part is the same. The data filtering is performed in one callback,
+called in loop on input data starting at a specific offset for a given
+length. Data analyzed by a filter are considered as forwarded from its point of
+view. Because filters are chained, a filter never analyzes more data than its
+predecessors. Thus only data analyzed by the last filter are effectively
+forwarded. This means, at any time, any filter may choose to not analyze all
+available data (available from its point of view), blocking the data forwarding.
+
+Internally, filters own 2 offsets representing the number of bytes already
+analyzed in the available input data, one per channel. There is also an offset
+couple at the stream level, in the strm_flt object, representing the total
+number of bytes already forwarded. These offsets may be retrieved and updated
+using following macros :
+
+ * FLT_OFF(flt, chn)
+
+ * FLT_STRM_OFF(s, chn)
+
+where 'flt' is the 'struct filter' passed as argument in all callbacks, 's' the
+filtered stream and 'chn' is the considered channel. However, there is no reason
+for a filter to use these macros or take care of these offsets.
+
+
+3.6.1 FILTERING DATA ON TCP STREAMS
+-----------------------------------
+
+The TCP data filtering for TCP streams is the easy case, because HAProxy do not
+parse these data. Data are stored in raw in the buffer. So there is only one
+callback to consider:
+
+ * 'flt_ops.tcp_payload : This callback is called when input data are
+ available. If not defined, all available data will be considered as analyzed
+ and forwarded from the filter point of view.
+
+This callback is called only if the filter is registered to analyze TCP
+data. Here is an example :
+
+ /* Returns a negative value if an error occurs, else the number of
+ * consumed bytes. */
+ static int
+ my_filter_tcp_payload(struct stream *s, struct filter *filter,
+ struct channel *chn, unsigned int offset,
+ unsigned int len)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+ int ret = len;
+
+ /* Do not parse more than 'my_conf->max_parse' bytes at a time */
+ if (my_conf->max_parse != 0 && ret > my_conf->max_parse)
+ ret = my_conf->max_parse;
+
+ /* if available data are not completely parsed, wake up the stream to
+ * be sure to not freeze it. The best is probably to set a
+ * chn->analyse_exp timer */
+ if (ret != len)
+ task_wakeup(s->task, TASK_WOKEN_MSG);
+ return ret;
+ }
+
+But it is important to note that tunnelled data of an HTTP stream may also be
+filtered via this callback. Tunnelled data are data exchange after an HTTP tunnel
+is established between the client and the server, via an HTTP CONNECT or via a
+protocol upgrade. In this case, the data are structured. Of course, to do so,
+the filter must be able to parse HTX data and must have the FLT_CFG_FL_HTX flag
+set. At any time, the IS_HTX_STRM() macros may be used on the stream to know if
+it is an HTX stream or a TCP stream.
+
+
+3.6.2 FILTERING DATA ON HTTP STREAMS
+------------------------------------
+
+The HTTP data filtering is a bit more complex because HAProxy data are
+structutred and represented to an internal format, called HTX. So basically
+there is the HTTP counterpart to the previous callback :
+
+ * 'flt_ops.http_payload' : This callback is called when input data are
+ available. If not defined, all available data will be considered as analyzed
+ and forwarded for the filter.
+
+But the prototype for this callbacks is slightly different. Instead of having
+the channel as parameter, we have the HTTP message (struct http_msg). This
+callback is called only if the filter is registered to analyze TCP data. Here is
+an example :
+
+ /* Returns a negative value if an error occurs, else the number of
+ * consumed bytes. */
+ static int
+ my_filter_http_payload(struct stream *s, struct filter *filter,
+ struct http_msg *msg, unsigned int offset,
+ unsigned int len)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+ struct htx *htx = htxbuf(&msg->chn->buf);
+ struct htx_ret htxret = htx_find_offset(htx, offset);
+ struct htx_blk *blk;
+
+ blk = htxret.blk;
+ offset = htxret.ret;
+ for (; blk; blk = htx_get_next_blk(blk, htx)) {
+ enum htx_blk_type type = htx_get_blk_type(blk);
+
+ if (type == HTX_BLK_UNUSED)
+ continue;
+ else if (type == HTX_BLK_DATA) {
+ /* filter data */
+ }
+ else
+ break;
+ }
+
+ return len;
+ }
+
+In addition, there are two others callbacks :
+
+ * 'flt_ops.http_headers' : This callback is called just before the HTTP body
+ forwarding and after any processing on the request/response HTTP
+ headers. When defined, this callback is always called for HTTP streams
+ (i.e. without needs of a registration on data filtering).
+ Here is an example :
+
+
+ /* Returns a negative value if an error occurs, 0 if it needs to wait,
+ * any other value otherwise. */
+ static int
+ my_filter_http_headers(struct stream *s, struct filter *filter,
+ struct http_msg *msg)
+ {
+ struct my_filter_config *my_conf = FLT_CONF(filter);
+ struct htx *htx = htxbuf(&msg->chn->buf);
+ struct htx_sl *sl = http_get_stline(htx);
+ int32_t pos;
+
+ for (pos = htx_get_first(htx); pos != -1; pos = htx_get_next(htx, pos)) {
+ struct htx_blk *blk = htx_get_blk(htx, pos);
+ enum htx_blk_type type = htx_get_blk_type(blk);
+ struct ist n, v;
+
+ if (type == HTX_BLK_EOH)
+ break;
+ if (type != HTX_BLK_HDR)
+ continue;
+
+ n = htx_get_blk_name(htx, blk);
+ v = htx_get_blk_value(htx, blk);
+ /* Do something on the header name/value */
+ }
+
+ return 1;
+ }
+
+ * 'flt_ops.http_end' : This callback is called when the whole HTTP message was
+ processed. It may interrupt the stream processing. So, it could be used to
+ synchronize the HTTP request with the HTTP response, for instance :
+
+ /* Returns a negative value if an error occurs, 0 if it needs to wait,
+ * any other value otherwise. */
+ static int
+ my_filter_http_end(struct stream *s, struct filter *filter,
+ struct http_msg *msg)
+ {
+ struct my_filter_ctx *my_ctx = filter->ctx;
+
+
+ if (!(msg->chn->flags & CF_ISRESP)) /* The request */
+ my_ctx->end_of_req = 1;
+ else /* The response */
+ my_ctx->end_of_rsp = 1;
+
+ /* Both the request and the response are finished */
+ if (my_ctx->end_of_req == 1 && my_ctx->end_of_rsp == 1)
+ return 1;
+
+ /* Wait */
+ return 0;
+ }
+
+Then, to finish, there are 2 informational callbacks :
+
+ * 'flt_ops.http_reset' : This callback is called when an HTTP message is
+ reset. This happens either when a 1xx informational response is received, or
+ if we're retrying to send the request to the server after it failed. It
+ could be useful to reset the filter context before receiving the true
+ response.
+ By checking s->txn->status, it is possible to know why this callback is
+ called. If it's a 1xx, we're called because of an informational
+ message. Otherwise, it is a L7 retry.
+
+ * 'flt_ops.http_reply' : This callback is called when, at any time, HAProxy
+ decides to stop the processing on a HTTP message and to send an internal
+ response to the client. This mainly happens when an error or a redirect
+ occurs.
+
+
+3.6.3 REWRITING DATA
+--------------------
+
+The last part, and the trickiest one about the data filtering, is about the data
+rewriting. For now, the filter API does not offer a lot of functions to handle
+it. There are only functions to notify HAProxy that the data size has changed to
+let it update internal state of filters. This is the developer responsibility to
+update data itself, i.e. the buffer offsets, using following function :
+
+ * 'flt_update_offsets()' : This function must be called when a filter alter
+ incoming data. It updates offsets of the stream and of all filters
+ preceding the calling one. Do not call this function when a filter change
+ the size of incoming data leads to an undefined behavior.
+
+A good example of filter changing the data size is the HTTP compression filter.
diff --git a/doc/internals/api/htx-api.txt b/doc/internals/api/htx-api.txt
new file mode 100644
index 0000000..62b3093
--- /dev/null
+++ b/doc/internals/api/htx-api.txt
@@ -0,0 +1,570 @@
+ -----------------------------------------------
+ HTX API
+ Version 1.1
+ ( Last update: 2021-02-24 )
+ -----------------------------------------------
+ Author : Christopher Faulet
+ Contact : cfaulet at haproxy dot com
+
+1. Background
+
+Historically, HAProxy stored HTTP messages in a raw fashion in buffers, keeping
+parsing information separately in a "struct http_msg" owned by the stream. It was
+optimized to the data transfer, but not so much for rewrites. It was also HTTP/1
+centered. While it was the only HTTP version supported, it was not a
+problem. But with the rise of HTTP/2, it starts to be hard to still use this
+representation.
+
+At the first age of the HTTP/2 in HAProxy, H2 messages were converted into
+H1. This was terribly unefficient because it required two parsing passes, a
+first one in H2 and a second one in H1, with a conversion in the middle. And of
+course, the same was also true in the opposite direction. outgoing H1 messages
+had to be converted back in H2 to be sent. Even worse, because the H2->H1
+conversion, only client H2 connections were supported.
+
+So, to address all these problems, we decided to replace the old raw
+representation by a version-agnostic and self-structured internal HTTP
+representation, the HTX. As an additional benefit, with this new representation,
+the message parsing and its processing are now separated, making all the HTTP
+analysis simpler and cleaner. The parsing of HTTP messages is now handled by
+the multiplexers (h1 or h2).
+
+
+2. The HTX message
+
+The HTX is a structure containing useful information about an HTTP message
+followed by a contiguous array with some parts of the message. These parts are
+called blocks. A block is composed of metadata (htx_blk) and an associated
+payload. Blocks' metadata are stored starting from the end of the array while
+their payload are stored at the beginning. Blocks' metadata are often simply
+called blocks. it is a misuse of language that's simplify explanations.
+
+Internally, this structure is "hidden" in a buffer. This way, there are few
+changes into intermediate layers (stream-interface and channels). They still
+manipulate buffers. Only the multiplexer and the stream have to know how data
+are really stored. From the HTX perspective, a buffer is just a memory
+area. When an HTX message is stored in a buffer, this one appears as full.
+
+ * General view of an HTX message :
+
+
+ buffer->area
+ |
+ |<------------ buffer->size == buffer->data ----------------------|
+ | |
+ | |<------------- Blocks array (htx->size) ------------------>|
+ V | |
+ +-----+-----------------+-------------------------+---------------+
+ | HTX | PAYLOADS ==> | | <== HTX_BLKs |
+ +-----+-----------------+-------------------------+---------------+
+ | | | |
+ |<-payloads part->|<----- free space ------>|<-blocks part->|
+ (htx->data)
+
+
+The blocks part remains linear and sorted. It may be see as an array with
+negative indexes. But, instead of using negative indexes, we use positive
+positions to identify a block. This position is then converted to an address
+relatively to the beginning of the blocks array.
+
+ tail head
+ | |
+ V V
+ .....--+----+-----------------------+------+------+
+ | Bn | ... | B1 | B0 |
+ .....--+----+-----------------------+------+------+
+ ^ ^ ^
+ Addr of the block Addr of the block Addr of the block
+ at the position N at the position 1 at the position 0
+
+
+In the HTX structure, 3 "special" positions are stored :
+
+ - tail : Position of the newest inserted block
+ - head : Position of the oldest inserted block
+ - first : Position of the first block to (re)start the analyse
+
+The blocks part never wrap. If we have no space to allocate a new block and if
+there is a hole at the beginning of the blocks part (so at the end of the blocks
+array), we move back all blocks.
+
+
+ tail head tail head
+ | | | |
+ V V V V
+ ...+--------------+---------+ blocks ...----------+--------------+
+ | X== HTX_BLKS | | defrag | <== HTX_BLKS |
+ ...+--------------+---------+ =====> ...----------+--------------+
+
+
+The payloads part is a raw space that may wrap. A block's payload must never be
+accessed directly. Instead a block must be selected to retrieve the address of
+its payload.
+
+
+ +------------------------( B0.addr )--------------------------+
+ | +-------------------( B1.addr )----------------------+ |
+ | | +-----------( B2.addr )----------------+ | |
+ V V V | | |
+ +-----+----+-------+----+--------+-------------+-------+----+----+----+
+ | HTX | P0 | P1 | P2 | ...==> | | <=... | B2 | B1 | B0 |
+ +-----+----+-------+----+--------+-------------+-------+----+----+----+
+
+
+Because the payloads part may wrap, there are 2 usable free spaces :
+
+ - The free space in front of the blocks part. This one is used if and only if
+ the other one was not used yet.
+
+ - The free space at the beginning of the message. Once this one is used, the
+ other one is never used again, until a message defragmentation.
+
+
+ * Linear payloads part :
+
+
+ head_addr end_addr tail_addr
+ | | |
+ V V V
+ +-----+--------------------+-------------+--------------------+-------...
+ | HTX | | PAYLOADS | | HTX_BLKs
+ +-----+--------------------+-------------+--------------------+-------...
+ |<-- free space 2 -->| |<-- free space 1 -->|
+ (used if the other is too small) (used in priority)
+
+
+ * Wrapping payloads part :
+
+
+ head_addr end_addr tail_addr
+ | | |
+ V V V
+ +-----+----+----------------+--------+----------------+-------+-------...
+ | HTX | | PAYLOADS part2 | | PAYLOADS part1 | | HTX_BLKs
+ +-----+----+----------------+--------+----------------+-------+-------...
+ |<-->| |<------>| |<----->|
+ unusable free space unusable
+ free space free space
+
+
+Finally, when the usable free space is not enough to store a new block, unusable
+parts may be get back with a full defragmentation. The payloads part is then
+realigned at the beginning of the blocks array and the free space becomes
+continuous again.
+
+
+3. The HTX blocks
+
+An HTX block can be as well a start-line as a header, a body part or a
+trailer. For all these types of block, a payload is attached to the block. It
+can also be a marker, the end-of-headers or end-of-trailers. For these blocks,
+there is no payload but it counts for a byte. It is important to not skip it
+when data are forwarded.
+
+As already said, a block is composed of metadata and a payload. Metadata are
+stored in the blocks part and are composed of 2 fields :
+
+ - info : It a 32 bits field containing the block's type on 4 bits followed
+ by the payload length. See below for details.
+
+ - addr : The payload's address, if any, relatively to the beginning the
+ array used to store part of the HTTP message itself.
+
+
+ * Block's info representation :
+
+ 0b 0000 0000 0000 0000 0000 0000 0000 0000
+ ---- ------------------------ ---------
+ type value (1 MB max) name length (header/trailer - 256B max)
+ ----------------------------------
+ data length (256 MB max)
+ (body, method, path, version, status, reason)
+
+
+Supported types are :
+
+ - 0000 (0) : The request start-line
+ - 0001 (1) : The response start-line
+ - 0010 (2) : A header block
+ - 0011 (3) : The end-of-headers marker
+ - 0100 (4) : A data block
+ - 0101 (5) : A trailer block
+ - 0110 (6) : The end-of-trailers marker
+ - 1111 (15) : An unused block
+
+Other types are unused for now and reserved for futur extensions.
+
+An HTX message is typically composed of following blocks, in this order :
+
+ - a start-line
+ - zero or more header blocks
+ - an end-of-headers marker
+ - zero or more data blocks
+ - zero or more trailer blocks (optional)
+ - an end-of-trailers marker (optional but always set if there is at least
+ one trailer block)
+
+Only one HTTP request at a time can be stored in an HTX message. For HTTP
+response, it is more complicated. Only one "final" response can be stored in an
+HTX message. It is a response with status-code 101 or greater or equal to
+200. But it may be preceded by several 1xx informational responses. Such
+responses are part of the same HTX message.
+
+When the end of the message is reached a special flag is set on the message
+(HTX_FL_EOM). It means no more data are expected for this message, except
+tunneled data. But tunneled data will never be mixed with message data to avoid
+ambiguities. Thus once the flag marking the end of the message is set, it is
+easy to know the message ends. The end is reached if the HTX message is empty or
+on the tail HTX block in the HTX message. Once all blocks of the HTX message are
+consumed, tunneled data, if any, may be transferred.
+
+
+3.1. The start-line
+
+Every HTX message starts with a start-line. Its payload is a "struct htx_sl". In
+addition to the parts of the HTTP start-line, this structure contains some
+information about the represented HTTP message, mainly in the form of flags
+(HTX_SL_F_*). For instance, if an HTTP message contains the header
+"conten-length", then the flag HTX_SL_F_CLEN is set.
+
+Each HTTP message has its own start-line. So an HTX request has one and only one
+start-line because it must contain only one HTTP request at a time. But an HTX
+response may have more than one start-line if the final HTTP response is
+precedeed by some 1xx informational responses.
+
+In HTTP/2, there is no start-line. So the H2 multiplexer must create one when it
+converts an H2 message to HTX :
+
+ - For the request, it uses the pseudo headers ":method", ":path" or
+ ":authority" depending on the method and the hardcoded version "HTTP/2.0".
+
+ - For the response, it used the hardcoded version "HTTP/2.0", the
+ pseudo-header ":status" and an empty reason.
+
+
+3.2. The headers and trailers
+
+HTX Headers and trailers are quite similar. Different types are used to simplify
+headers processing. But from the HTX point of view, there is no real difference,
+except their position in the HTX message. The header blocks always follow an HTX
+start-line while trailer blocks come after the data. If there is no data, they
+follow the end-of-headers marker.
+
+Headers and trailers are the only blocks containing a Key/Value payload. The
+corresponding end-of marker must always be placed after each group to mark, as
+it name suggests, the end.
+
+In HTTP/1, trailers are only present on chunked messages. But chunked messages
+do not always have trailers. In this case, the end-of-trailers block may or may
+not be present. Multiplexers must be able to handle both situations. In HTTP/2,
+trailers are only present if a HEADERS frame is sent after DATA frames.
+
+
+3.3. The data
+
+The payload body of an HTTP message is stored as DATA blocks in the HTX
+message. For HTTP/1 messages, it is the message body without the chunks
+formatting, if any. For HTTP/2, it is the payload of DATA frames.
+
+The DATA blocks are the only HTX blocks that may be partially processed (copied
+or removed). All other types of block must be entirely processed. This means
+DATA blocks can be resized.
+
+
+3.4. The end-of markers
+
+These blocks are used to delimit parts of an HTX message. It exists two
+markers :
+
+ - end-of-headers (EOH)
+ - end-of-trailers (EOT)
+
+EOH is always present in an HTX message. EOT is optional.
+
+
+4. The HTX API
+
+
+4.1. Get/set HTX message from/to the underlying buffer
+
+The first thing to do to process an HTX message is to get it from the underlying
+buffer. There are 2 functions to do so, the second one relying on the first :
+
+ - htxbuf() returns an HTX message from a buffer. It does not modify the
+ buffer. It only initialize the HTX message if the buffer is empty.
+
+ - htx_from_buf() uses htxbuf(). But it also updates the underlying buffer so
+ that it appears as full.
+
+Both functions return a "zero-sized" HTX message if the buffer is null. This
+way, the HTX message is always valid. The first function is the default function
+to use. The second one is only useful when some content will be added. For
+instance, it used by the HTX analyzers when HAProxy generates a response. Thus,
+the buffer is in a right state.
+
+Once the processing done, if the HTX message has been modified, the underlying
+buffer must be also updated, except htx_from_buf() was used _AND_ data was only
+added. For all other cases, the function htx_to_buf() must be called.
+
+Finally, the function htx_reset() may be called at any time to reset an HTX
+message. And the function buf_room_for_htx_data() may be called to know if a raw
+buffer is full from the HTX perspective. It is used during conversion from/to
+the HTX.
+
+
+4.2. Helpers to deal with free space in an HTX message
+
+Once with an HTX message, following functions may help to process it :
+
+ - htx_used_space() and htx_meta_space() return, respectively, the total
+ space used in an HTX message and the space used by block's metadata only.
+
+ - htx_free_space() and htx_free_data_space() return, respectively, the total
+ free space in an HTX message and the free space available for the payload
+ if a new HTX block is stored (so it is the total free space minus the size
+ of an HTX block).
+
+ - htx_is_empty() and htx_is_not_empty() are boolean functions to know if an
+ HTX message is empty or not.
+
+ - htx_get_max_blksz() returns the maximum size available for the payload,
+ not exceeding a maximum, metadata included.
+
+ - htx_almost_full() should be used to know if an HTX message uses at least
+ 3/4 of its capacity.
+
+
+4.3. HTX Blocks manipulations
+
+Once the available sapce in an HTX message is known, the next step is to add HTX
+blocks. First of all the function htx_nbblks() returns the number of blocks
+allocated in an HTX message. Then, there is an add function per block's type :
+
+ - htx_add_stline() adds a start-line. The type (request or response) and the
+ flags of the start-line must be provided, as well as its three parts
+ (method,uri,version or version,status-code,reason).
+
+ - htx_add_header() and htx_add_trailers() are similar. The name and the
+ value must be provided. The inserted HTX block is returned on success or
+ NULL if an error occurred.
+
+ - htx_add_endof() must be used to add any end-of marker. The block's type
+ (EOH or EOT) must be specified. The inserted HTX block is returned on
+ success or NULL if an error occurred.
+
+ - htx_add_all_headers() and htx_add_all_trailers() add, respectively, a list
+ of headers and a list of trailers, followed by the appropriate end-of
+ marker. On success, this marker is returned. Otherwise, NULL is
+ returned. Note there is no rollback on the HTX message when an error
+ occurred. Some headers or trailers may have been added. So it is the
+ caller responsibility to take care of that.
+
+ - htx_add_data() must be used to add a DATA block. Unlike previous
+ functions, this one returns the number of bytes copied or 0 if nothing was
+ copied. If possible, the data are appended to the tail block if it is a
+ DATA block. Only a part of the payload may be copied because this function
+ will try to limit the message defragmentation and the wrapping of blocks
+ as far as possible.
+
+ - htx_add_data_atonce() must be used if all data must be added or nothing.
+ It tries to insert all the payload, this function returns the inserted
+ block on success. Otherwise it returns NULL.
+
+When an HTX block is added, it is always the last one (the tail). But, if a
+block must be added at a specific place, it is not really handy. 2 functions may
+help (others could be added) :
+
+ - htx_add_last_data() adds a DATA block just after all other DATA blocks and
+ before any trailers and EOT marker. It relies on htx_add_data_atonce(), so
+ a defragmentation may be performed.
+
+ - htx_move_blk_before() moves a specific block just after another one. Both
+ blocks must already be in the HTX message and the block to move must
+ always be placed after the "pivot".
+
+Once added, there are three functions to update the block's payload :
+
+ - htx_replace_stline() updates a start-line. The HTX block must be passed as
+ argument. Only string parts of the start-line are updated by this
+ function. On success, it returns the new start-line. So it is pretty easy
+ to update its flags. NULL is returned if an error occurred.
+
+ - htx_replace_header() fully replaces a header (its name and its value) by a
+ new one. The HTX block must be passed a argument, as well as its new name
+ and its new value. The new header can be smaller or larger than the old
+ one. This function returns the new HTX block on success, or NULL is an
+ error occurred.
+
+ - htx_replace_blk_value() replaces a part of a block's payload or its
+ totality. It works for HEADERS, TRAILERS or DATA blocks. The HTX block
+ must be provided with the part to remove and the new one. The new part can
+ be smaller or larger than the old one. This function returns the new HTX
+ block on success, or NULL is an error occurred.
+
+ - htx_change_blk_value_len() changes the size of the value. It is the caller
+ responsibility to change the value itself, make sure there is enough space
+ and update allocated value. This function updates the HTX message
+ accordingly.
+
+ - htx_set_blk_value_len() changes the size of the value. It is the caller
+ responsibility to change the value itself, make sure there is enough space
+ and update allocated value. Unlike the function
+ htx_change_blk_value_len(), this one does not update the HTX message. So
+ it should be used with caution.
+
+ - htx_cut_data_blk() removes <n> bytes from the beginning of a DATA
+ block. The block's start address and its length are adjusted, and the
+ htx's total data count is updated. This is used to mark that part of some
+ data were transferred from a DATA block without removing this DATA
+ block. No sanity check is performed, the caller is responsible for doing
+ this exclusively on DATA blocks, and never removing more than the block's
+ size.
+
+ - htx_remove_blk() removes a block from an HTX message. It returns the
+ following block or NULL if it is the tail block.
+
+Finally, a block may be removed using the function htx_remove_blk(). This
+function returns the block following the one removed or NULL if it is the tail
+block.
+
+
+4.4. The HTX start-line
+
+Unlike other HTX blocks, the start-line is a bit special because its payload is
+a structure followed by its three parts :
+
+ +--------+-------+-------+-------+
+ | HTX_SL | PART1 | PART2 | PART3 |
+ +--------+-------+-------+-------+
+
+Some macros and functions may help to manipulate these parts :
+
+ - HTX_SL_P{N}_LEN() and HTX_SL_P{N}_PTR() are macros to get the length of a
+ part and a pointer on it. {N} should be 1, 2 or 3.
+
+ - HTX_SL_REQ_MLEN(), HTX_SL_REQ_ULEN(), HTX_SL_REQ_VLEN(),
+ HTX_SL_REQ_MPTR(), HTX_SL_REQ_UPTR() and HTX_SL_REQ_VPTR() are macros to
+ get info about a request start-line. These macros only wrap HTX_SL_P*
+ ones.
+
+ - HTX_SL_RES_VLEN(), HTX_SL_RES_CLEN(), HTX_SL_RES_RLEN(),
+ HTX_SL_RES_VPTR(), HTX_SL_RES_CPTR() and HTX_SL_RES_RPTR() are macros to
+ get info about a response start-line. These macros only wrap HTX_SL_P*
+ ones.
+
+ - htx_sl_p1(), htx_sl_p2() and htx_sl_p2() are functions to get the ist
+ corresponding to the right part of a start-line.
+
+ - htx_sl_req_meth(), htx_sl_req_uri() and htx_sl_req_vsn() get the ist
+ corresponding to the right part of a request start-line.
+
+ - htx_sl_res_vsn(), htx_sl_res_code() and htx_sl_res_reason() get the ist
+ corresponding to the right part of a response start-line.
+
+
+4.5. Iterate on the HTX message
+
+To iterate on an HTX message, the first thing to do is to get the HTX block to
+start the loop. There are three special blocks in an HTX message that may be
+good candidates to start a loop :
+
+ - the head block. It is the oldest inserted block. Multiplexers always start
+ to consume an HTX message from this block. The function htx_get_head()
+ returns its position and htx_get_head_blk() returns the blocks itself. In
+ addition, the function htx_get_head_type() returns its block's type.
+
+ - the tail block. It is the newest inserted block. The function
+ htx_get_tail() returns its position and htx_get_tail_blk() returns the
+ blocks itself. In addition, the function htx_get_tail_type() returns its
+ block's type.
+
+ - the first block. It is the block where to (re)start the analyse. It is
+ used as start point by HTX analyzers. The function htx_get_first() returns
+ its position and htx_get_first_blk() returns the blocks itself. In
+ addition, the function htx_get_first_type() returns its block's type.
+
+For all these functions, if the HTX message is empty, -1 is returned for the
+block's position, NULL instead of a block and HTX_BLK_UNUSED for its type.
+
+Then to iterate on blocks, foreword or backward :
+
+ - htx_get_prev() and htx_get_next() return, respectively, the position of
+ the previous block or the next block, given a specific position. Or -1 if
+ an edge is reached.
+
+ - htx_get_prev_blk() and htx_get_next_blk() return, respectively, the
+ previous block or the next one, given a specific block. Or NULL if an edge
+ is reached.
+
+4.6. Access block content and info
+
+Following functions may be used to retrieve information about a specific HTX
+block :
+
+ - htx_get_blk_pos() returns the position of a block. It must be in the HTX
+ message.
+
+ - htx_get_blk_ptr() returns a pointer on the payload of a block.
+
+ - htx_get_blk_type() returns the type of a block.
+
+ - htx_get_blksz() returns the payload size of a block
+
+ - htx_get_blk_name() returns the name of a block, only if it is a header or
+ a trailer. Otherwise, it returns an empty string.
+
+ - htx_get_blk_value() returns the value of a block, depending on its
+ type. For header and trailer blocks, it is the value field. For markers
+ (EOH or EOT), an empty string is returned. For other blocks an ist
+ pointing on the block payload is returned.
+
+ - htx_is_unique_blk() may be used to know if a block is the only one
+ remaining inside an HTX message, excluding unused blocks. This function is
+ pretty useful to determine the end of a HTX message, in conjunction with
+ HTX_FL_EOM flag.
+
+4.7. Advanced functions
+
+Some more advanced functions may be used to do complex processing on the HTX
+message. These functions are used by HTX analyzers or by multiplexers.
+
+ - htx_truncate() removes all blocks after the one containing a specific
+ offset relatively to the head block of the HTX message. If the offset is
+ inside a DATA block, it is truncated. For all other blocks, the removal
+ starts to the next block.
+
+ - htx_drain() tries to remove a specific amount of bytes of payload. If the
+ tail block is a DATA block, it may be truncated if necessary. All other
+ block are removed at once or kept. This function returns a mixed value,
+ with the first block not removed, or NULL if everything was removed, and
+ the amount of data drained.
+
+ - htx_xfer_blks() transfers HTX blocks from an HTX message to another,
+ stopping on the first block of a specified type or when a specific amount
+ of bytes, including meta-data, was moved. If the tail block is a DATA
+ block, it may be partially moved. All other block are transferred at once
+ or kept. This function returns a mixed value, with the last block moved,
+ or NULL if nothing was moved, and the amount of data transferred. When
+ HEADERS or TRAILERS blocks must be transferred, this function transfers
+ all of them. Otherwise, if it is not possible, it triggers an error. It is
+ the caller responsibility to transfer all headers or trailers at once.
+
+ - htx_append_msg() append an HTX message to another one. All the message is
+ copied or nothing. So, if an error occurred, a rollback is performed. This
+ function returns 1 on success and 0 on error.
+
+ - htx_reserve_max_data() Reserves the maximum possible size for an HTX data
+ block, by extending an existing one or by creating a new one. It returns a
+ compound result with the HTX block and the position where new data must be
+ inserted (0 for a new block). If an error occurs or if there is no space
+ left, NULL is returned instead of a pointer on an HTX block.
+
+ - htx_find_offset() looks for the HTX block containing a specific offset,
+ starting at the HTX message's head. The function returns the found HTX
+ block and the position inside this block where the offset is. If the
+ offset is outside of the HTX message, NULL is returned.
+
+ - htx_defrag() defragments an HTX message. It removes unused blocks and
+ unwraps the payloads part. A temporary buffer is used to do so. This
+ function never fails. A referenced block may be provided. If so, the
+ corresponding new block is returned. Otherwise, NULL is returned.
diff --git a/doc/internals/api/initcalls.txt b/doc/internals/api/initcalls.txt
new file mode 100644
index 0000000..a341edc
--- /dev/null
+++ b/doc/internals/api/initcalls.txt
@@ -0,0 +1,366 @@
+Initialization stages aka how to get your code initialized at the right moment
+
+
+1. Background
+
+Originally all subsystems were initialized via a dedicated function call
+from the huge main() function. Then some code started to become conditional
+or a bit more modular and the #ifdef placed there became a mess, resulting
+in init code being moved to function constructors in each subsystem's own
+file. Then pools of various things were introduced, starting to make the
+whole init sequence more complicated due to some forms of internal
+dependencies. Later epoll was introduced, requiring a post-fork callback,
+and finally threads arrived also requiring some post-thread init/deinit
+and allocation, marking the old architecture's last breath. Finally the
+whole thing resulted in lots of init code duplication and was simplified
+in 1.9 with the introduction of initcalls and initialization stages.
+
+
+2. New architecture
+
+The new architecture relies on two layers :
+ - the registration functions
+ - the INITCALL macros and initialization stages
+
+The first ones are mostly used to add a callback to a list. The second ones
+are used to specify when to call a function. Both are totally independent,
+however they are generally combined via another set consisting in the REGISTER
+macros which make some registration functions be called at some specific points
+during the init sequence.
+
+
+3. Registration functions
+
+Registration functions never fail. Or more precisely, if they fail it will only
+be on out-of-memory condition, and they will cause the process to immediately
+exit. As such they do not return any status and the caller doesn't have to care
+about their success.
+
+All available functions are described below in alphanumeric ordering. Please
+make sure to respect this ordering when adding new ones.
+
+- void hap_register_build_opts(const char *str, int must_free)
+
+ This appends the zero-terminated constant string <str> to the list of known
+ build options that will be reported on the output of "haproxy -vv". A line
+ feed character ('\n') will automatically be appended after the string when it
+ is displayed. The <must_free> argument must be zero, unless the string was
+ allocated by any malloc-compatible function such as malloc()/calloc()/
+ realloc()/strdup() or memprintf(), in which case it's better to pass a
+ non-null value so that the string is freed upon exit. Note that despite the
+ function's prototype taking a "const char *", the pointer will actually be
+ cast and freed. The const char* is here to leave more freedom to use consts
+ when making such options lists.
+
+- void hap_register_per_thread_alloc(int (*fct)())
+
+ This adds a call to function <fct> to the list of functions to be called when
+ threads are started, at the beginning of the polling loop. This is also valid
+ for the main thread and will be called even if threads are disabled, so that
+ it is guaranteed that this function will be called in any circumstance. Each
+ thread will first call all these functions exactly once when it starts. Calls
+ are serialized by the init_mutex, so that locking is not necessary in these
+ functions. There is no relation between the thread numbers and the callback
+ ordering. The function is expected to return non-zero on success, or zero on
+ failure. A failure will make the process emit a succinct error message and
+ immediately exit. See also hap_register_per_thread_free() for functions
+ called after these ones.
+
+- void hap_register_per_thread_deinit(void (*fct)());
+
+ This adds a call to function <fct> to the list of functions to be called when
+ threads are gracefully stopped, at the end of the polling loop. This is also
+ valid for the main thread and will be called even if threads are disabled, so
+ that it is guaranteed that this function will be called in any circumstance
+ if the process experiences a soft stop. Each thread will call this function
+ exactly once when it stops. However contrary to _alloc() and _init(), the
+ calls are made without any protection, thus if any shared resource if touched
+ by the function, the function is responsible for protecting it. The reason
+ behind this is that such resources are very likely to be still in use in one
+ other thread and that most of the time the functions will in fact only touch
+ a refcount or deinitialize their private resources. See also
+ hap_register_per_thread_free() for functions called after these ones.
+
+- void hap_register_per_thread_free(void (*fct)());
+
+ This adds a call to function <fct> to the list of functions to be called when
+ threads are gracefully stopped, at the end of the polling loop, after all calls
+ to _deinit() callbacks are done for this thread. This is also valid for the
+ main thread and will be called even if threads are disabled, so that it is
+ guaranteed that this function will be called in any circumstance if the
+ process experiences a soft stop. Each thread will call this function exactly
+ once when it stops. However contrary to _alloc() and _init(), the calls are
+ made without any protection, thus if any shared resource if touched by the
+ function, the function is responsible for protecting it. The reason behind
+ this is that such resources are very likely to be still in use in one other
+ thread and that most of the time the functions will in fact only touch a
+ refcount or deinitialize their private resources. See also
+ hap_register_per_thread_deinit() for functions called before these ones.
+
+- void hap_register_per_thread_init(int (*fct)())
+
+ This adds a call to function <fct> to the list of functions to be called when
+ threads are started, at the beginning of the polling loop, right after the
+ list of _alloc() functions. This is also valid for the main thread and will
+ be called even if threads are disabled, so that it is guaranteed that this
+ function will be called in any circumstance. Each thread will call this
+ function exactly once when it starts, and calls are serialized by the
+ init_mutex which is held over all _alloc() and _init() calls, so that locking
+ is not necessary in these functions. In other words for all threads but the
+ current one, the sequence of _alloc() and _init() calls will be atomic. There
+ is no relation between the thread numbers and the callback ordering. The
+ function is expected to return non-zero on success, or zero on failure. A
+ failure will make the process emit a succinct error message and immediately
+ exit. See also hap_register_per_thread_alloc() for functions called before
+ these ones.
+
+- void hap_register_pre_check(int (*fct)())
+
+ This adds a call to function <fct> to the list of functions to be called at
+ the step just before the configuration validity checks. This is useful when you
+ need to create things like it would have been done during the configuration
+ parsing and where the initialization should continue in the configuration
+ check.
+ It could be used for example to generate a proxy with multiple servers using
+ the configuration parser itself. At this step the final trash buffers are
+ allocated. Threads are not yet started so no protection is required. The
+ function is expected to return non-zero on success, or zero on failure. A
+ failure will make the process emit a succinct error message and immediately
+ exit.
+
+- void hap_register_post_check(int (*fct)())
+
+ This adds a call to function <fct> to the list of functions to be called at
+ the end of the configuration validity checks, just at the point where the
+ program either forks or exits depending whether it's called with "-c" or not.
+ Such calls are suited for memory allocation or internal table pre-computation
+ that would preferably not be done on the fly to avoid inducing extra time to
+ a pure configuration check. Threads are not yet started so no protection is
+ required. The function is expected to return non-zero on success, or zero on
+ failure. A failure will make the process emit a succinct error message and
+ immediately exit.
+
+- void hap_register_post_deinit(void (*fct)())
+
+ This adds a call to function <fct> to the list of functions to be called when
+ freeing the global sections at the end of deinit(), after everything is
+ stopped. The process is single-threaded at this point, thus these functions
+ are suitable for releasing configuration elements provided that no other
+ _deinit() function uses them, i.e. only close/release what is strictly
+ private to the subsystem. Since such functions are mostly only called during
+ soft stops (reloads) or failed startups, they tend to experience much less
+ test coverage than others despite being more exposed, and as such a lot of
+ care must be taken to test them especially when facing partial subsystem
+ initializations followed by errors.
+
+- void hap_register_post_proxy_check(int (*fct)(struct proxy *))
+
+ This adds a call to function <fct> to the list of functions to be called for
+ each proxy, after the calls to _post_server_check(). This can allow, for
+ example, to pre-configure default values for an option in a frontend based on
+ the "bind" lines or something in a backend based on the "server" lines. It's
+ worth being aware that such a function must be careful not to waste too much
+ time in order not to significantly slow down configurations with tens of
+ thousands of backends. The function is expected to return non-zero on
+ success, or zero on failure. A failure will make the process emit a succinct
+ error message and immediately exit.
+
+- void hap_register_post_server_check(int (*fct)(struct server *))
+
+ This adds a call to function <fct> to the list of functions to be called for
+ each server, after the call to check_config_validity(). This can allow, for
+ example, to preset a health state on a server or to allocate a protocol-
+ specific memory area. It's worth being aware that such a function must be
+ careful not to waste too much time in order not to significantly slow down
+ configurations with tens of thousands of servers. The function is expected
+ to return non-zero on success, or zero on failure. A failure will make the
+ process emit a succinct error message and immediately exit.
+
+- void hap_register_proxy_deinit(void (*fct)(struct proxy *))
+
+ This adds a call to function <fct> to the list of functions to be called when
+ freeing the resources during deinit(). These functions will be called as part
+ of the proxy's resource cleanup. Note that some of the proxy's fields will
+ already have been freed and others not, so such a function must not use any
+ information from the proxy that is subject to being released. In particular,
+ all servers have already been deleted. Since such functions are mostly only
+ called during soft stops (reloads) or failed startups, they tend to
+ experience much less test coverage than others despite being more exposed,
+ and as such a lot of care must be taken to test them especially when facing
+ partial subsystem initializations followed by errors. It's worth mentioning
+ that too slow functions could have a significant impact on the configuration
+ check or exit time especially on large configurations.
+
+- void hap_register_server_deinit(void (*fct)(struct server *))
+
+ This adds a call to function <fct> to the list of functions to be called when
+ freeing the resources during deinit(). These functions will be called as part
+ of the server's resource cleanup. Note that some of the server's fields will
+ already have been freed and others not, so such a function must not use any
+ information from the server that is subject to being released. Since such
+ functions are mostly only called during soft stops (reloads) or failed
+ startups, they tend to experience much less test coverage than others despite
+ being more exposed, and as such a lot of care must be taken to test them
+ especially when facing partial subsystem initializations followed by errors.
+ It's worth mentioning that too slow functions could have a significant impact
+ on the configuration check or exit time especially on large configurations.
+
+
+4. Initialization stages
+
+In order to offer some guarantees, the startup of the program is split into
+several stages. Some callbacks can be placed into each of these stages using
+an INITCALL macro, with 0 to 3 arguments, respectively called INITCALL0 to
+INITCALL3. These macros must be placed anywhere at the top level of a C file,
+preferably at the end so that the referenced symbols have already been met,
+but it may also be fine to place them right after the callbacks themselves.
+
+Such callbacks are referenced into small structures containing a pointer to the
+function and 3 arguments. NULL replaces unused arguments. The callbacks are
+cast to (void (*)(void *, void *, void *)) and the arguments to (void *).
+
+The first argument to the INITCALL macro is the initialization stage. The
+second one is the callback function, and others if any are the arguments.
+The init stage must be among the values of the "init_stage" enum, currently,
+and in this execution order:
+
+ - STG_PREPARE : used to preset variables, pre-initialize lookup tables and
+ pre-initialize list heads
+ - STG_LOCK : used to pre-initialize locks
+ - STG_REGISTER : used to register static lists such as keywords
+ - STG_ALLOC : used to allocate the required structures
+ - STG_POOL : used to create pools
+ - STG_INIT : used to initialize subsystems
+
+Each stage is guaranteed that previous stages have successfully completed. This
+means that an INITCALL placed at stage STG_INIT is guaranteed that all pools
+were already created and will be usable. Conversely, an INITCALL placed at
+stage STG_REGISTER must not rely on any field that requires preliminary
+allocation nor initialization. A callback cannot rely on other callbacks of the
+same stage, as the execution order within a stage is undefined and essentially
+depends on the linking order.
+
+The STG_REGISTER level is made for run-time linking of the various modules that
+compose the executable. Keywords, protocols and various other elements that are
+local known to each compilation unit can will be appended into common lists at
+boot time. This is why this call is placed just before STG_ALLOC.
+
+Note that trash is needed in various functions. Trash is a pool and is
+allocated during STG_POOL, so it's not permitted to use it before STG_INIT,
+where it will only use the default size, and may be reallocated later with a
+different size.
+
+Example: register a very early call to init_log() with no argument, and another
+ call to cli_register_kw(&cli_kws) much later:
+
+ INITCALL0(STG_PREPARE, init_log);
+ INITCALL1(STG_REGISTER, cli_register_kw, &cli_kws);
+
+Technically speaking, each call to such a macro adds a distinct local symbol
+whose dynamic name involves the line number. These symbols are placed into a
+separate section and the beginning and end section pointers are provided by the
+linker. When too old a linker is used, a fallback is applied consisting in
+placing them into a linked list which is built by a constructor function for
+each initcall (this takes more room).
+
+Due to the symbols internally using the line number, it is very important not
+to place more than one INITCALL per line in the source file.
+
+It is also strongly recommended that functions and referenced arguments are
+static symbols local to the source file, unless they are global registration
+functions like in the example above with cli_register_kw(), where only the
+argument is a local keywords table.
+
+INITCALLs do not expect the callback function to return anything and as such
+do not perform any error check. As such, they are very similar to constructors
+offered by the compiler except that they are segmented in stages. It is thus
+the responsibility of the called functions to perform their own error checking
+and to exit in case of error. This may change in the future.
+
+
+5. REGISTER family of macros
+
+The association of INITCALLs and registration functions allows to perform some
+early dynamic registration of functions to be used anywhere, as well as values
+to be added to existing lists without having to manipulate list elements. For
+the sake of simplification, these combinations are available as a set of
+REGISTER macros which register calls to certain functions at the appropriate
+init stage. Such macros must be used at the top level in a file, just like
+INITCALL macros. The following macros are currently supported. Please keep them
+alphanumerically ordered:
+
+- REGISTER_BUILD_OPTS(str)
+
+ Adds the constant string <str> to the list of build options. This is done by
+ registering a call to hap_register_build_opts(str, 0) at stage STG_REGISTER.
+ The string will not be freed.
+
+- REGISTER_CONFIG_POSTPARSER(name, parser)
+
+ Adds a call to function <parser> at the end of the config parsing. The
+ function is called at the very end of check_config_validity() and may be used
+ to initialize a subsystem based on global settings for example. This is done
+ by registering a call to cfg_register_postparser(name, parser) at stage
+ STG_REGISTER.
+
+- REGISTER_CONFIG_SECTION(name, parse, post)
+
+ Registers a new config section name <name> which will be parsed by function
+ <parse> (if not null), and with an optional call to function <post> at the
+ end of the section. Function <parse> must be of type (int (*parse)(const char
+ *file, int linenum, char **args, int inv)), and returns 0 on success or an
+ error code among the ERR_* set on failure. The <post> callback takes no
+ argument and returns a similar error code. This is achieved by registering a
+ call to cfg_register_section() with the three arguments at stage
+ STG_REGISTER.
+
+- REGISTER_PER_THREAD_ALLOC(fct)
+
+ Registers a call to register_per_thread_alloc(fct) at stage STG_REGISTER.
+
+- REGISTER_PER_THREAD_DEINIT(fct)
+
+ Registers a call to register_per_thread_deinit(fct) at stage STG_REGISTER.
+
+- REGISTER_PER_THREAD_FREE(fct)
+
+ Registers a call to register_per_thread_free(fct) at stage STG_REGISTER.
+
+- REGISTER_PER_THREAD_INIT(fct)
+
+ Registers a call to register_per_thread_init(fct) at stage STG_REGISTER.
+
+- REGISTER_POOL(ptr, name, size)
+
+ Used internally to declare a new pool. This is made by calling function
+ create_pool_callback() with these arguments at stage STG_POOL. Do not use it
+ directly, use either DECLARE_POOL() or DECLARE_STATIC_POOL() instead.
+
+- REGISTER_PRE_CHECK(fct)
+
+ Registers a call to register_pre_check(fct) at stage STG_REGISTER.
+
+- REGISTER_POST_CHECK(fct)
+
+ Registers a call to register_post_check(fct) at stage STG_REGISTER.
+
+- REGISTER_POST_DEINIT(fct)
+
+ Registers a call to register_post_deinit(fct) at stage STG_REGISTER.
+
+- REGISTER_POST_PROXY_CHECK(fct)
+
+ Registers a call to register_post_proxy_check(fct) at stage STG_REGISTER.
+
+- REGISTER_POST_SERVER_CHECK(fct)
+
+ Registers a call to register_post_server_check(fct) at stage STG_REGISTER.
+
+- REGISTER_PROXY_DEINIT(fct)
+
+ Registers a call to register_proxy_deinit(fct) at stage STG_REGISTER.
+
+- REGISTER_SERVER_DEINIT(fct)
+
+ Registers a call to register_server_deinit(fct) at stage STG_REGISTER.
+
diff --git a/doc/internals/api/ist.txt b/doc/internals/api/ist.txt
new file mode 100644
index 0000000..0f118d6
--- /dev/null
+++ b/doc/internals/api/ist.txt
@@ -0,0 +1,167 @@
+2021-11-08 - Indirect Strings (IST) API
+
+
+1. Background
+-------------
+
+When parsing traffic, most of the standard C string functions are unusable
+since they rely on a trailing zero. In addition, for the rare ones that support
+a length, we have to constantly maintain both the pointer and the length. But
+then, it's easy to come up with complex lengths and offsets calculations all
+over the place, rendering the code hard to read and bugs hard to avoid or spot.
+
+IST provides a solution to this by defining a structure made of exactly two
+word size elements, that most C ABIs know how to handle as a register when
+used as a function argument or a function's return value. The functions are
+inlined to leave a maximum set of opportunities to the compiler or optimization
+and expression reduction, and as a result they are often inexpensive to use. It
+is important however to keep in mind that all of these are designed for minimal
+code size when dealing with short strings (i.e. parsing tokens in protocols),
+and they are not optimal for processing large blocks.
+
+
+2. API description
+------------------
+
+IST are defined like this:
+
+ struct ist {
+ char *ptr; // pointer to the string's first byte
+ size_t len; // number of valid bytes starting from ptr
+ };
+
+A string is not set if its ->ptr member is NULL. In this case .len is undefined
+and is recommended to be zero.
+
+Declaring a function returning an IST:
+
+ struct ist produce_ist(int ok)
+ {
+ return ok ? IST("OK") : IST("KO");
+ }
+
+Declaring a function consuming an IST:
+
+ void say_ist(struct ist i)
+ {
+ write(1, istptr(i), istlen(i));
+ }
+
+Chaining the two:
+
+ void say_ok(int ok)
+ {
+ say_ist(produce_ist(ok));
+ }
+
+Notes:
+ - the arguments are passed as value, not reference, so there's no need for
+ any "const" in their declaration (except to catch coding mistakes).
+ Pointers to ist may benefit from being marked "const" however.
+
+ - similarly for the return value, there's no point is marking it "const" as
+ this would protect the pointer and length, not the data.
+
+ - use ist0() to append a trailing zero to a variable string for use with
+ printf()'s "%s" format, or for use with functions that work on NUL-
+ terminated strings, but beware of not doing this with constants.
+
+ - the API provides a starting pointer and current length, but does not
+ provide an allocated size. It remains up to the caller to know how large
+ the allocated area is when adding data, though most functions make this
+ easy.
+
+The following macros and functions are defined. Those whose name starts with
+underscores require special care and must not be used without being certain
+they are properly used (typically subject to buffer overflows if misused). Note
+that most functions were added over time depending on instant needs, and some
+are very close to each other. Many useful functions are still missing and would
+deserve being added.
+
+Below, arguments "i1","i2" are all of type "ist". Arguments "s" are
+NUL-terminated strings of type "char*", and "cs" are of type "const char *".
+Arguments "c" are of type "char", and "n" are of type size_t.
+
+ IST(cs):ist make constant IST from a NUL-terminated const string
+ IST_NULL:ist return an unset IST = ist2(NULL,0)
+ __istappend(i1,c):ist append character <c> at the end of ist <i1>
+ ist(s):ist return an IST from a nul-terminated string
+ ist0(i1):char* write a \0 at the end of an IST, return the string
+ ist2(cs,l):ist return a variable IST from a const string and length
+ ist2bin(s,i1):ist copy IST into a buffer, return the result
+ ist2bin_lc(s,i1):ist like ist2bin() but turning turning to lower case
+ ist2bin_uc(s,i1):ist like ist2bin() but turning turning to upper case
+ ist2str(s,i1):ist copy IST into a buffer, add NUL and return the result
+ ist2str_lc(s,i1):ist like ist2str() but turning turning to lower case
+ ist2str_uc(s,i1):ist like ist2str() but turning turning to upper case
+ ist_find(i1,c):ist return first occurrence of char <c> in <i1>
+ ist_find_ctl(i1):char* return pointer to first CTL char in <i1> or NULL
+ ist_skip(i1,c):ist return first occurrence of char not <c> in <i1>
+ istadv(i1,n):ist advance the string by <n> characters
+ istalloc(n):ist return allocated string of zero initial length
+ istcat(d,s,n):ssize_t copy <s> after <d> for <n> chars max, return len or -1
+ istchr(i1,c):char* return pointer to first occurrence of <c> in <i1>
+ istclear(i1*):size_t return previous size and set size to zero
+ istcpy(d,s,n):ssize_t copy <s> over <d> for <n> chars max, return len or -1
+ istdiff(i1,i2):int return the ordinal difference, like strcmp()
+ istdup(i1):ist allocate new ist and copy original one into it
+ istend(i1):char* return pointer to first character after the IST
+ isteq(i1,i2):int return non-zero if strings are equal
+ isteqi(i1,i2):int like isteq() but case-insensitive
+ istfree(i1*) free of allocated <i1>/IST_NULL and set it to IST_NULL
+ istissame(i1,i2):int return true if pointers and lengths are equal
+ istist(i1,i2):ist return first occurrence of <i2> in <i1>
+ istlen(i1):size_t return the length of the IST (number of characters)
+ istmatch(i1,i2):int return non-zero if i1 starts like i2 (empty OK)
+ istmatchi(i1,i2):int like istmatch() but case insensitive
+ istneq(i1,i2,n):int like isteq() but limited to the first <n> chars
+ istnext(i1):ist return the IST advanced by one character
+ istnmatch(i1,i2,n):int like istmatch() but limited to the first <n> chars
+ istpad(s,i1):ist copy IST into a buffer, add a NUL, return the result
+ istptr(i1):char* return the starting pointer of the IST
+ istscat(d,s,n):ssize_t same as istcat() but always place a NUL at the end
+ istscpy(d,s,n):ssize_t same as istcpy() but always place a NUL at the end
+ istshift(i1*):char return the first character and advance the IST by one
+ istsplit(i1*,c):ist return part before <c>, make ist start from <c>
+ iststop(i1,c):ist truncate ist before first occurrence of <c>
+ isttest(i1):int return true if ist is not NULL, false otherwise
+ isttrim(i1,n):ist return ist trimmed to no more than <n> characters
+ istzero(i1,n):ist trim to <n> chars, trailing zero included.
+
+
+3. Quick index by typical C construct or function
+-------------------------------------------------
+
+Some common C constructs may be adjusted to use ist instead. The mapping is not
+always one-to-one, but usually the computations on the length part tends to
+disappear in the refactoring, allowing to directly chain function calls. The
+entries below are hints to figure what function to look for in order to rewrite
+some common use cases.
+
+ char* IST equivalent
+
+ strchr() istchr(), ist_find(), iststop()
+ strstr() istist()
+ strcpy() istcpy()
+ strscpy() istscpy()
+ strlcpy() istscpy()
+ strcat() istcat()
+ strscat() istscat()
+ strlcat() istscat()
+ strcmp() istdiff()
+ strdup() istdup()
+ !strcmp() isteq()
+ !strncmp() istneq(), istmatch(), istnmatch()
+ !strcasecmp() isteqi()
+ !strncasecmp() istneqi(), istmatchi()
+ strtok() istsplit()
+ return NULL return IST_NULL
+ s = malloc() s = istalloc()
+ free(s); s = NULL istfree(&s)
+ p != NULL isttest(p)
+ c = *(p++) c = istshift(p)
+ *(p++) = c __istappend(p, c)
+ p += n istadv(p, n)
+ p + strlen(p) istend(p)
+ p[max] = 0 isttrim(p, max)
+ p[max+1] = 0 istzero(p, max)
diff --git a/doc/internals/api/layers.txt b/doc/internals/api/layers.txt
new file mode 100644
index 0000000..b5c35f4
--- /dev/null
+++ b/doc/internals/api/layers.txt
@@ -0,0 +1,190 @@
+2022-05-27 - Stream layers in HAProxy 2.6
+
+
+1. Background
+
+There are streams at plenty of levels in haproxy, essentially due to the
+introduction of multiplexed protocols which provide high-level streams on top
+of low-level streams, themselves either based on stream-oriented protocols or
+datagram-oriented protocols.
+
+The refactoring of the appctx and muxes that allowed to drop a lot of duplicate
+code between 2.5 and 2.6-dev6 raised another concern with some entities like
+"conn_stream" that were not specific to connections anymore, "endpoints" that
+became entities on their own, and "targets" whose life had been extended to
+last all along a connection.
+
+It was time to rename all such legacy entities introduced in 1.8 and which had
+turned particularly confusing over time as their roles evolved.
+
+
+2. Naming principles
+
+The global renaming of some entities between streams and connections was
+articulated around several principles:
+
+ - avoid the confusing use of "context" in shared places. For example, the
+ endpoint's connection is in "ctx" and nothing makes it obvious that the
+ endpoint's context is a connection, especially when an applet is there.
+
+ - reserve relative nouns for pointers and not for types. "endpoint", just
+ like "owner" or "peer" is relative, but when accessed from a different
+ layer it starts to make no sense at all, or to make one believe it's
+ something else, particularly with void*.
+
+ - avoid too generic terms that have multiple meanings, or words that are
+ synonyms in a same place (e.g. "peer" and "remote", or "endpoint" and
+ "target"). If two synonyms are needed to designate two distinct entities,
+ there's probably a problem elsewhere, or the problem is poorly defined.
+
+ - make it clearer that all that is manipulated is related to streams. This
+ particularly important in sample fetch functions for example, which tend
+ to require low-level access and could be mislead in trying to follow the
+ wrong chain when trying to get information about a connection.
+
+ - use easily spellable short names that abbreviate unambiguously when used
+ together in adjacent contexts
+
+
+3. Current state as of 2.6
+
+- when a name is required to designate the lower block that starts at the mux
+ stream or the appctx, it is spoken of as a "stream endpoint", and abbreviated
+ "se". It's okay because while "endpoint" itself is relative, "stream
+ endpoint" unequivocally designates one extremity of a stream. If a type is
+ needed for this in the future (e.g. via obj_type), then the type "stendp"
+ may be used. Before 2.6-dev6 there was no name for this, it was known as
+ conn_stream->ctx.
+
+- the 2.6-dev6 cs_endpoint which preserves the state of a mux stream or an
+ appctx and abstracts them in front of a conn_stream becomes a "stream
+ endpoint descriptor", of type "sedesc" and often abbreviated "sd", "sed"
+ or "ed". Its "target" pointer became "se" as per the rule above. Before
+ 2.6-dev6, these elements were mixed with others inside conn_stream. From
+ the appctx it's called "sedesc" (few occurrences hence long name OK).
+
+- the conn_stream which is always attached to either a stream or a health check
+ and that is used to reach a mux or an applet becomes a "stream connector" of
+ type "stconn", generally abbreviated "sc". Its "endp" pointer becomes
+ "sedesc" as per the rule above, and that one has a back pointer "sc". The
+ stream uses "scf" and "scb" as the respective front and back pointers to the
+ stconns. Prior to 2.6-dev6, these parts were split between conn_stream and
+ stream_interface.
+
+- the sedesc's "ctx" which is solely used to store the connection as of now, is
+ renamed "conn" to void any doubt in the context of applets or even muxes. In
+ the future the connection should be attached to the "se" instead and this
+ pointer should disappear (or be recycled for anything else).
+
+The new 2.6 model looks like this:
+
+ +------------------------+
+ | stream or health check |
+ +------------------------+
+ ^ \ scf, scb
+ / \
+ | |
+ \ /
+ app \ v
+ +----------+
+ | stconn |
+ +----------+
+ ^ \ sedesc
+ / \
+ . . . . | . . . | . . . . . split point (retries etc)
+ \ /
+ sc \ v
+ +----------+
+ flags <--| sedesc | : sedesc :
+ +----------+ ... +----------+
+ conn / ^ \ se ^ \
+ +------------+ / / \ | \
+ | connection |<--' | | ... OR ... | |
+ +------------+ \ / \ |
+ mux| ^ |ctx sd \ v : sedesc \ v
+ | | | +----------------------+ \ # +----------+ svcctx
+ | | | | mux stream or appctx | | # | appctx |--.
+ | | | +----------------------+ | # +----------+ |
+ | | | ^ | / private # : : |
+ v | | | v > to the # +----------+ |
+ mux_ops | | +----------------+ \ mux # | svcctx |<-'
+ | +---->| mux connection | ) # +----------+
+ +------ +----------------+ / #
+
+Stream descriptors may exist in the following modes:
+ - .conn = NULL, .se = NULL : backend, not connection attempt yet
+ - .conn = NULL, .se = <appctx> : frontend or backend, applet
+ - .conn = <conn>, .se = NULL : backend, connection in progress
+ - .conn = <conn>, .se = <muxs> : frontend or backend, connected
+
+Notes:
+ - for historical reasons (connect, forced protocol upgrades, etc), during a
+ connection setup or a rule-based protocol upgrade, the connection's "ctx"
+ may temporarily point to the stconn
+
+
+4. Invariants and cardinalities
+
+Usually a stream is created from an existing stconn from a mux or some applets,
+but may also be allocated first by other applets schedulers. After stream_new()
+a stream always has exactly one stconn per side (scf, scb), each of which has
+one ->sedesc. Each side is initialized with either one or no stream endpoint
+attached to the descriptor.
+
+Both applets and a mux stream always have a stream endpoint descriptor. AS SUCH
+IT IS NEVER NECESSARY TO TEST FOR THE EXISTENCE OF THE SEDESC FROM ANY SIDE, IT
+ALWAYS EXISTS. This explains why as much as possible it's preferable to use the
+sedesc to access flags and statuses from any side, rather than bouncing via the
+stconn.
+
+An applet's app layer is always a stream, which means that there are always
+channels accessible above, and there is always an opposite stream connector and
+a stream endpoint descriptor. As such, it's always safe for an applet to access
+the other side using sc_opposite().
+
+When an outgoing connection is in the process of being established, the backend
+side sedesc has its ->conn pointer pointing to the pending connection, and no
+->se. Once the connection is established and a mux is chosen, it's attached to
+the ->se. If an applet is used instead of a mux, the appctx is attached to the
+sedesc's ->se and ->conn remains NULL.
+
+If either side wants to detach from the other, it must allocate a new virgin
+sedesc to replace the existing one, and leave the existing one to the endpoint,
+since it continues to describe the stream endpoint. The stconn keeps its state
+(modulo the updates related to the disconnection). The previous sedesc points
+to a NULL stconn. For example, disconnecting from a backend mux will leave the
+entities like this:
+
+ +------------------------+
+ | stream or health check |
+ +------------------------+
+ ^ \ scf, scb
+ / \
+ | |
+ \ /
+ app \ v
+ +----------+
+ | stconn |
+ +----------+
+ ^ \ sedesc
+ / \
+ NULL | |
+ ^ \ /
+ sc | / sc \ v
+ +----------+ / +----------+
+ flags <--| sedesc1 | . . . . . | sedesc2 |--> flags
+ +----------+ / +----------+
+ conn / ^ \ se / conn / \ se
+ +------------+ / / \ | |
+ | connection |<--' | | v v
+ +------------+ \ / NULL NULL
+ mux| ^ |ctx sd \ v
+ | | | +----------------------+
+ | | | | mux stream or appctx |
+ | | | +----------------------+
+ | | | ^ |
+ v | | | v
+ mux_ops | | +----------------+
+ | +---->| mux connection |
+ +------ +----------------+
+
diff --git a/doc/internals/api/list.txt b/doc/internals/api/list.txt
new file mode 100644
index 0000000..d03cf03
--- /dev/null
+++ b/doc/internals/api/list.txt
@@ -0,0 +1,195 @@
+2021-11-09 - List API
+
+
+1. Background
+-------------
+
+HAProxy's lists are almost all doubly-linked and circular so that it is always
+possible to insert at the beginning, append at the end, scan them in any order
+and delete any element without having to scan to search the predecessor nor the
+successor.
+
+A list's head is just a regular list element, and an element always points to
+another list element. Such elements only have two pointers, the next and the
+previous elements. The object being pointed to is retrieved by subtracting the
+list element's offset in its structure from the list element's pointer. This
+way there is no need for any separate allocation for the list element, for a
+pointer to the object in the list, nor for a pointer to the list element from
+the object, as the list is embedded into the object.
+
+All basic operations are provided, as well as some iterators. Some iterators
+are safe for removal of the current element within the loop, others not. In any
+case a list cannot be freely modified while iterating over it (e.g. the current
+element's successor cannot not be freed if it's saved as the restart point).
+
+Extreme care is taken nowadays in HAProxy to make sure that no dangling
+pointers are left in elements, so it is important to always initialize list
+heads and list elements, as well as elements that are removed from a list if
+they are not immediately freed, so that their deletion is idempotent. A rule of
+thumb is that a list pointer's validity never has to be checked, it is always
+valid to dereference it. A lot of complex bugs have been caused in the past by
+incorrect list manipulation, such as an element being deleted twice, resulting
+in damaging previously adjacent elements' neighbours. This usually has serious
+consequences at locations that are totally different from the one of the bug,
+and that are only detected much later, so it is required to be particularly
+strict on using lists safely.
+
+The lists are not thread-safe, but mt_lists may be used instead.
+
+
+2. API description
+------------------
+
+A list is defined like this, both for the list's head, and for any other
+element:
+
+ struct list {
+ struct list *n; /* next */
+ struct list *p; /* prev */
+ };
+
+An empty list points to itself for both pointers. I.e. a list's head is both
+its own successor and its own predecessor. This guarantees that insertions
+and deletions can be done without any check and that deletion is idempotent.
+For this reason and by convention, a detached element ought to be represented
+like an empty head.
+
+Lists are manipulated using a set of macros which are used to initialize, add,
+remove, or iterate over elements. Most of these macros are extremely simple and
+are not even protected against multiple evaluation, so it is fundamentally
+important that the expressions used in the arguments are idempotent and that
+the result does not depend on the evaluation order of the arguments.
+
+Macro Description
+
+ILH
+ Initialized List Head : this is a non-NULL, non-empty list element used
+ to prevent the compiler from moving an empty list head declaration to
+ BSS, typically when it appears in an array of keywords Without this,
+ some older versions of gcc tend to trim all the array and cause
+ corruption.
+
+LIST_INIT(l)
+ Initialize the list as an empty list head
+
+LIST_HEAD_INIT(l)
+ Return a valid initialized empty list head pointing to this
+ element. Essentially used with assignments in declarations.
+
+LIST_INSERT(l, e)
+ Add an element at the beginning of a list and return it
+
+LIST_APPEND(l, e)
+ Add an element at the end of a list and return it
+
+LIST_SPLICE(n, o)
+ Add the contents of a list <o> at the beginning of another list <n>.
+ The old list head remains untouched.
+
+LIST_SPLICE_END_DETACHED(n, o)
+ Add the contents of a list whose first element is is <o> and last one
+ is <o->p> at the end of another list <n>. The old list DOES NOT have
+ any head here.
+
+LIST_DELETE(e)
+ Remove an element from a list and return it. Safe to call on
+ initialized elements, but will not change the element itself so it is
+ not idempotent. Consider using LIST_DEL_INIT() instead unless called
+ immediately after a free().
+
+LIST_DEL_INIT(e)
+ Remove an element from a list, initialize it and return it so that a
+ subsequent LIST_DELETE() is safe. This is faster than performing a
+ LIST_DELETE() followed by a LIST_INIT() as pointers are not reloaded.
+
+LIST_ELEM(l, t, m)
+ Return a pointer of type <t> to a structure containing a list head
+ member called <m> at address <l>. Note that <l> can be the result of a
+ function or macro since it's used only once.
+
+LIST_ISEMPTY(l)
+ Check if the list head <l> is empty (=initialized) or not, and return
+ non-zero only if so.
+
+LIST_INLIST(e)
+ Check if the list element <e> was added to a list or not, thus return
+ true unless the element was initialized.
+
+LIST_INLIST_ATOMIC(e)
+ Atomically check if the list element's next pointer points to anything
+ different from itself, implying the element should be part of a
+ list. This usually is similar to LIST_INLIST() except that while that
+ one might be instrumented using debugging code to perform further
+ consistency checks, the macro below guarantees to always perform a
+ single atomic test and is safe to use with barriers.
+
+LIST_NEXT(l, t, m)
+ Return a pointer of type <t> to a structure following the element which
+ contains list head <l>, which is known as member <m> in struct <t>.
+
+LIST_PREV(l, t, m)
+ Return a pointer of type <t> to a structure preceding the element which
+ contains list head <l>, which is known as member <m> in struct <t>.
+ Note that this macro is first undefined as it happened to already exist
+ on some old OSes.
+
+list_for_each_entry(i, l, m)
+ Iterate local variable <i> through a list of items of type "typeof(*i)"
+ which are linked via a "struct list" member named <m>. A pointer to the
+ head of the list is passed in <l>. No temporary variable is needed.
+ Note that <i> must not be modified during the loop.
+
+list_for_each_entry_from(i, l, m)
+ Same as list_for_each_entry() but starting from current value of <i>
+ instead of the list's head.
+
+list_for_each_entry_from_rev(i, l, m)
+ Same as list_for_each_entry_rev() but starting from current value of <i>
+ instead of the list's head.
+
+list_for_each_entry_rev(i, l, m)
+ Iterate backwards local variable <i> through a list of items of type
+ "typeof(*i)" which are linked via a "struct list" member named <m>. A
+ pointer to the head of the list is passed in <l>. No temporary variable
+ is needed. Note that <i> must not be modified during the loop.
+
+list_for_each_entry_safe(i, b, l, m)
+ Iterate variable <i> through a list of items of type "typeof(*i)" which
+ are linked via a "struct list" member named <m>. A pointer to the head
+ of the list is passed in <l>. A temporary backup variable <b> of same
+ type as <i> is needed so that <i> may safely be deleted if needed. Note
+ that it is only permitted to delete <i> and no other element during
+ this operation!
+
+list_for_each_entry_safe_from(i, b, l, m)
+ Same as list_for_each_entry_safe() but starting from current value of
+ <i> instead of the list's head.
+
+list_for_each_entry_safe_from_rev(i, b, l, m)
+ Same as list_for_each_entry_safe_rev() but starting from current value
+ of <i> instead of the list's head.
+
+list_for_each_entry_safe_rev(i, b, l, m)
+ Iterate backwards local variable <i> through a list of items of type
+ "typeof(*i)" which are linked via a "struct list" member named <m>. A
+ pointer to the head of the list is passed in <l>. A temporary variable
+ <b> of same type as <i> is needed so that <i> may safely be deleted if
+ needed. Note that it is only permitted to delete <i> and no other
+ element during this operation!
+
+3. Notes
+--------
+
+- This API is quite old and some macros are missing. For example there's still
+ no list_first() so it's common to use LIST_ELEM(head->n, ...) instead. Some
+ older parts of the code also used to rely on list_for_each() followed by a
+ break to stop on the first element.
+
+- Some parts were recently renamed because LIST_ADD() used to do what
+ LIST_INSERT() currently does and was often mistaken with LIST_ADDQ() which is
+ what LIST_APPEND() now is. As such it is not totally impossible that some
+ places use a LIST_INSERT() where a LIST_APPEND() would be desired.
+
+- The structure must not be modified at all (even to add debug info). Some
+ parts of the code assume that its layout is exactly this one, particularly
+ the parts ensuring the casting between MT lists and lists.
diff --git a/doc/internals/api/pools.txt b/doc/internals/api/pools.txt
new file mode 100644
index 0000000..d84fb9d
--- /dev/null
+++ b/doc/internals/api/pools.txt
@@ -0,0 +1,585 @@
+2022-02-24 - Pools structure and API
+
+1. Background
+-------------
+
+Memory allocation is a complex problem covered by a massive amount of
+literature. Memory allocators found in field cover a broad spectrum of
+capabilities, performance, fragmentation, efficiency etc.
+
+The main difficulty of memory allocation comes from finding the optimal chunks
+for arbitrary sized requests, that will still preserve a low fragmentation
+level. Doing this well is often expensive in CPU usage and/or memory usage.
+
+In programs like HAProxy that deal with a large number of fixed size objects,
+there is no point having to endure all this risk of fragmentation, and the
+associated costs (sometimes up to several milliseconds with certain minimalist
+allocators) are simply not acceptable. A better approach consists in grouping
+frequently used objects by size, knowing that due to the high repetitiveness of
+operations, a freed object will immediately be needed for another operation.
+
+This grouping of objects by size is what is called a pool. Pools are created
+for certain frequently allocated objects, are usually merged together when they
+are of the same size (or almost the same size), and significantly reduce the
+number of calls to the memory allocator.
+
+With the arrival of threads, pools started to become a bottleneck so they now
+implement an optional thread-local lockless cache. Finally with the arrival of
+really efficient memory allocator in modern operating systems, the shared part
+has also become optional so that it doesn't consume memory if it does not bring
+any value.
+
+In 2.6-dev2, a number of debugging options that used to be configured at build
+time only changed to boot-time and can be modified using keywords passed after
+"-dM" on the command line, which sets or clears bits in the pool_debugging
+variable. The build-time options still affect the default settings however.
+Default values may be consulted using "haproxy -dMhelp".
+
+
+2. Principles
+-------------
+
+The pools architecture is selected at build time. The main options are:
+
+ - thread-local caches and process-wide shared pool enabled (1)
+
+ This is the default situation on most operating systems. Each thread has
+ its own local cache, and when depleted it refills from the process-wide
+ pool that avoids calling the standard allocator too often. It is possible
+ to force this mode at build time by setting CONFIG_HAP_GLOBAL_POOLS or at
+ boot time with "-dMglobal".
+
+ - thread-local caches only are enabled (2)
+
+ This is the situation on operating systems where a fast and modern memory
+ allocator is detected and when it is estimated that the process-wide shared
+ pool will not bring any benefit. This detection is automatic at build time,
+ but may also be forced at build tmie by setting CONFIG_HAP_NO_GLOBAL_POOLS
+ or at boot time with "-dMno-global".
+
+ - pass-through to the standard allocator (3)
+
+ This is used when one absolutely wants to disable pools and rely on regular
+ malloc() and free() calls, essentially in order to trace memory allocations
+ by call points, either internally via DEBUG_MEM_STATS, or externally via
+ tools such as Valgrind. This mode of operation may be forced at build time
+ by setting DEBUG_NO_POOLS or at boot time with "-dMno-cache".
+
+ - pass-through to an mmap-based allocator for debugging (4)
+
+ This is used only during deep debugging when trying to detect various
+ conditions such as use-after-free. In this case each allocated object's
+ size is rounded up to a multiple of a page size (4096 bytes) and an
+ integral number of pages is allocated for each object using mmap(),
+ surrounded by two unaccessible holes that aim to detect some out-of-bounds
+ accesses. Released objects are instantly freed using munmap() so that any
+ immediate subsequent access to the memory area crashes the process if the
+ area had not been reallocated yet. This mode can be enabled at build time
+ by setting DEBUG_UAF, or at run time by disabling pools and enabling UAF
+ with "-dMuaf". It tends to consume a lot of memory and not to scale at all
+ with concurrent calls, that tends to make the system stall. The watchdog
+ may even trigger on some slow allocations.
+
+There are no more provisions for running with a shared pool but no thread-local
+cache: the shared pool's main goal is to compensate for the expensive calls to
+the memory allocator. This gain may be huge on tiny systems using basic
+allocators, but the thread-local cache will already achieve this. And on larger
+threaded systems, the shared pool's benefit is visible when the underlying
+allocator scales poorly, but in this case the shared pool would suffer from
+the same limitations without its thread-local cache and wouldn't provide any
+benefit.
+
+Summary of the various operation modes:
+
+ (1) (2) (3) (4)
+
+ User User User User
+ | | | |
+ pool_alloc() V V | |
+ +---------+ +---------+ | |
+ | Thread | | Thread | | |
+ | Local | | Local | | |
+ | Cache | | Cache | | |
+ +---------+ +---------+ | |
+ | | | |
+ pool_refill*() V | | |
+ +---------+ | | |
+ | Shared | | | |
+ | Pool | | | |
+ +---------+ | | |
+ | | | |
+ malloc() V V V |
+ +---------+ +---------+ +---------+ |
+ | Library | | Library | | Library | |
+ +---------+ +---------+ +---------+ |
+ | | | |
+ mmap() V V V V
+ +---------+ +---------+ +---------+ +---------+
+ | OS | | OS | | OS | | OS |
+ +---------+ +---------+ +---------+ +---------+
+
+One extra build define, DEBUG_FAIL_ALLOC, is used to enforce random allocation
+failure in pool_alloc() by randomly returning NULL, to test that callers
+properly handle allocation failures. It may also be enabled at boot time using
+"-dMfail". In this case the desired average rate of allocation failures can be
+fixed by global setting "tune.fail-alloc" expressed in percent.
+
+The thread-local caches contain the freshest objects. Its total size amounts to
+the number of bytes set in global.tune.pool_cache_size and that may be adjusted
+by the "tune.memory.hot-size" global option, which itself defaults to build
+time setting CONFIG_HAP_POOL_CACHE_SIZE, which was 1MB before 2.6 and 512kB
+after. The aim is to keep hot objects that still fit in the CPU core's private
+L2 cache. Once these objects do not fit into the cache anymore, there's no
+benefit keeping them local to the thread, so they'd rather be returned to the
+shared pool or the main allocator so that any other thread may make use of
+them. Under extreme thread contention the cost of accessing shared structures
+in the global cache or in malloc() may still be important and it may prove
+useful to increase the thread-local cache size.
+
+
+3. Storage in thread-local caches
+---------------------------------
+
+This section describes how objects are linked in thread local caches. This is
+not meant to be a concern for users of the pools API but it can be useful when
+inspecting post-mortem dumps or when trying to figure certain size constraints.
+
+Objects are stored in the local cache using a doubly-linked list. This ensures
+that they can be visited by freshness order like a stack, while at the same
+time being able to access them from oldest to newest when it is needed to
+evict coldest ones first:
+
+ - releasing an object to the cache always puts it on the top.
+
+ - allocating an object from the cache always takes the topmost one, hence the
+ freshest one.
+
+ - scanning for older objects to evict starts from the bottom, where the
+ oldest ones are located
+
+To that end, each thread-local cache keeps a list head in the "list" member of
+its "pool_cache_head" descriptor, that links all objects cast to type
+"pool_cache_item" via their "by_pool" member.
+
+Note that the mechanism described above only works for a single pool. When
+trying to limit the total cache size to a certain value, all pools included,
+there is also a need to arrange all objects from all pools together in the
+local caches. For this, each thread_ctx maintains a list head of recently
+released objects, all pools included, in its member "pool_lru_head". All items
+in a thread-local cache are linked there via their "by_lru" member.
+
+This means that releasing an object using pool_free() consists in inserting
+it at the beginning of two lists:
+ - the local pool_cache_head's "list" list head
+ - the thread context's "pool_lru_head" list head
+
+Allocating an object consists in picking the first entry from the pool's "list"
+and deleting its "by_pool" and "by_lru" links.
+
+Evicting an object consists in scanning the thread context's "pool_lru_head"
+backwards and deleting the object's "by_pool" and "by_lru" links.
+
+Given that entries are both inserted and removed synchronously, we have the
+guarantee that the oldest object in the thread's LRU list is always the oldest
+object in its pool, and that the next element is the cache's list head. This is
+what allows the LRU eviction mechanism to figure what pool an object belongs to
+when releasing it.
+
+Note:
+ | Since a pool_cache_item has two list entries, on 64-bit systems it will be
+ | 32-bytes long. This is the smallest size that a pool may be, and any smaller
+ | size will automatically be rounded up to this size.
+
+When build option DEBUG_POOL_INTEGRITY is set, or the boot-time option
+"-dMintegrity" is passed on the command line, the area of the object between
+the two list elements and the end according to pool->size will be filled with
+pseudo-random words during pool_put_to_cache(), and these words will be
+compared between each other during pool_get_from_cache(), and the process will
+crash in case any bit differs, as this would indicate that the memory area was
+modified after the free. The pseudo-random pattern is in fact incremented by
+(~0)/3 upon each free so that roughly half of the bits change each time and we
+maximize the likelihood of detecting a single bit flip in either direction. In
+order to avoid an immediate reuse and maximize the time the object spends in
+the cache, when this option is set, objects are picked from the cache from the
+oldest one instead of the freshest one. This way even late memory corruptions
+have a chance to be detected.
+
+When build option DEBUG_MEMORY_POOLS is set, or the boot-time option "-dMtag"
+is passed on the executable's command line, pool objects are allocated with
+one extra pointer compared to the requested size, so that the bytes that follow
+the memory area point to the pool descriptor itself as long as the object is
+allocated via pool_alloc(). Upon releasing via pool_free(), the pointer is
+compared and the code will crash in if it differs. This allows to detect both
+memory overflows and object released to the wrong pool (code bug resulting from
+a copy-paste error typically).
+
+Thus an object will look like this depending whether it's in the cache or is
+currently in use:
+
+ in cache in use
+ +------------+ +------------+
+ <--+ by_pool.p | | N bytes |
+ | by_pool.n +--> | |
+ +------------+ |N=16 min on |
+ <--+ by_lru.p | | 32-bit, |
+ | by_lru.n +--> | 32 min on |
+ +------------+ | 64-bit |
+ : : : :
+ | N bytes | | |
+ +------------+ +------------+ \ optional, only if
+ : (unused) : : pool ptr : > DEBUG_MEMORY_POOLS
+ +------------+ +------------+ / is set at build time
+ or -dMtag at boot time
+
+Right now no provisions are made to return objects aligned on larger boundaries
+than those currently covered by malloc() (i.e. two pointers). This need appears
+from time to time and the layout above might evolve a little bit if needed.
+
+
+4. Storage in the process-wide shared pool
+------------------------------------------
+
+In order for the shared pool not to be a contention point in a multi-threaded
+environment, objects are allocated from or released to shared pools by clusters
+of a few objects at once. The maximum number of objects that may be moved to or
+from a shared pool at once is defined by CONFIG_HAP_POOL_CLUSTER_SIZE at build
+time, and currently defaults to 8.
+
+In order to remain scalable, the shared pool has to make some tradeoffs to
+limit the number of atomic operations and the duration of any locked operation.
+As such, it's composed of a single-linked list of clusters, themselves made of
+a single-linked list of objects.
+
+Clusters and objects are of the same type "pool_item" and are accessed from the
+pool's "free_list" member. This member points to the latest pool_item inserted
+into the pool by a release operation. And the pool_item's "next" member points
+to the next pool_item, which was the one present in the pool's free_list just
+before the pool_item was inserted, and the last pool_item in the list simply
+has a NULL "next" field.
+
+The pool_item's "down" pointer points down to the next objects part of the same
+cluster, that will be released or allocated at the same time as the first one.
+Each of these items also has a NULL "next" field, and are chained by their
+respective "down" pointers until the last one is detected by a NULL value.
+
+This results in the following layout:
+
+ pool pool_item pool_item pool_item
+ +-----------+ +------+ +------+ +------+
+ | free_list +--> | next +--> | next +--> | NULL |
+ +-----------+ +------+ +------+ +------+
+ | down | | NULL | | down |
+ +--+---+ +------+ +--+---+
+ | |
+ V V
+ +------+ +------+
+ | NULL | | NULL |
+ +------+ +------+
+ | down | | NULL |
+ +--+---+ +------+
+ |
+ V
+ +------+
+ | NULL |
+ +------+
+ | NULL |
+ +------+
+
+Allocating an entry is only a matter of performing two atomic allocations on
+the free_list and reading the pool's "next" value:
+
+ - atomically mark the free_list as being updated by writing a "magic" pointer
+ - read the first pool_item's "next" field
+ - atomically replace the free_list with this value
+
+This results in a fast operation that instantly retrieves a cluster at once.
+Then outside of the critical section entries are walked over and inserted into
+the local cache one at a time. In order to keep the code simple and efficient,
+objects allocated from the shared pool are all placed into the local cache, and
+only then the first one is allocated from the cache. This operation is
+performed by the dedicated function pool_refill_local_from_shared() which is
+called from pool_get_from_cache() when the cache is empty. It means there is an
+overhead of two list insert/delete operations for the first object and that
+could be avoided at the expense of more complex code in the fast path, but this
+is negligible since it only concerns objects that need to be visited anyway.
+
+Freeing a group of objects consists in performing the operation the other way
+around:
+
+ - atomically mark the free_list as being updated by writing a "magic" pointer
+ - write the free_list value to the to-be-released item's "next" entry
+ - atomically replace the free_list with the pool_item's pointer
+
+The cluster will simply have to be prepared before being sent to the shared
+pool. The operation of releasing a cluster at once is performed by function
+pool_put_to_shared_cache() which is called from pool_evict_last_items() which
+itself is responsible for building the clusters.
+
+Due to the way objects are stored, it is important to try to group objects as
+much as possible when releasing them because this is what will condition their
+retrieval as groups as well. This is the reason why pool_evict_last_items()
+uses the LRU to find a first entry but tries to pick several items at once from
+a single cache. Tests have shown that CONFIG_HAP_POOL_CLUSTER_SIZE set to 8
+achieves up to 6-6.5 objects on average per operation, which effectively
+divides by as much the average time spent per object by each thread and pushes
+the contention point further.
+
+Also, grouping items in clusters is a property of the process-wide shared pool
+and not of the thread-local caches. This means that there is no grouped
+operation when not using the shared pool (mode "2" in the diagram above).
+
+
+5. API
+------
+
+The following functions are public and available for user code:
+
+struct pool_head *create_pool(char *name, uint size, uint flags)
+ Create a new pool named <name> for objects of size <size> bytes. Pool
+ names are truncated to their first 11 characters. Pools of very similar
+ size will usually be merged if both have set the flag MEM_F_SHARED in
+ <flags>. When DEBUG_DONT_SHARE_POOLS was set at build time, or
+ "-dMno-merge" is passed on the executable's command line, the pools
+ also need to have the exact same name to be merged. In addition, unless
+ MEM_F_EXACT is set in <flags>, the object size will usually be rounded
+ up to the size of pointers (16 or 32 bytes). The name that will appear
+ in the pool upon merging is the name of the first created pool. The
+ returned pointer is the new (or reused) pool head, or NULL upon error.
+ Pools created this way must be destroyed using pool_destroy().
+
+void *pool_destroy(struct pool_head *pool)
+ Destroy pool <pool>, that is, all of its unused objects are freed and
+ the structure is freed as well if the pool didn't have any used objects
+ anymore. In this case NULL is returned. If some objects remain in use,
+ the pool is preserved and its pointer is returned. This ought to be
+ used essentially on exit or in rare situations where some internal
+ entities that hold pools have to be destroyed.
+
+void pool_destroy_all(void)
+ Destroy all pools, without checking which ones still have used entries.
+ This is only meant for use on exit.
+
+void *__pool_alloc(struct pool_head *pool, uint flags)
+ Allocate an entry from the pool <pool>. The allocator will first look
+ for an object in the thread-local cache if enabled, then in the shared
+ pool if enabled, then will fall back to the operating system's default
+ allocator. NULL is returned if the object couldn't be allocated (due to
+ configured limits or lack of memory). Object allocated this way have to
+ be released using pool_free(). Like with malloc(), by default the
+ contents of the returned object are undefined. If memory poisonning is
+ enabled, the object will be filled with the poisonning byte. If the
+ global "pool.fail-alloc" setting is non-zero and DEBUG_FAIL_ALLOC is
+ enabled, a random number generator will be called to randomly return a
+ NULL. The allocator's behavior may be adjusted using a few flags passed
+ in <flags>:
+ - POOL_F_NO_POISON : when set, disables memory poisonning (e.g. when
+ pointless and expensive, like for buffers)
+ - POOL_F_MUST_ZERO : when set, the memory area will be zeroed before
+ being returned, similar to what calloc() does
+ - POOL_F_NO_FAIL : when set, disables the random allocation failure,
+ e.g. for use during early init code or critical sections.
+
+void *pool_alloc(struct pool_head *pool)
+ This is an exact equivalent of __pool_alloc(pool, 0). It is the regular
+ way to allocate entries from a pool.
+
+void *pool_alloc_nocache(struct pool_head *pool)
+ Allocate an entry from the pool <pool>, bypassing the cache. If shared
+ pools are enabled, they will be consulted first. Otherwise the object
+ is allocated using the operating system's default allocator. This is
+ essentially used during early boot to pre-allocate a number of objects
+ for pools which require a minimum number of entries to exist.
+
+void *pool_zalloc(struct pool_head *pool)
+ This is an exact equivalent of __pool_alloc(pool, POOL_F_MUST_ZERO).
+
+void pool_free(struct pool_head *pool, void *ptr)
+ Free an entry allocate from one of the pool_alloc() functions above
+ from pool <pool>. The object will be placed into the thread-local cache
+ if enabled, or in the shared pool if enabled, or will be released using
+ the operating system's default allocator. When a local cache is
+ enabled, if the local cache size becomes larger than 75% of the maximum
+ size configured at build time, some objects will be evicted to the
+ shared pool. Such objects are taken first from the same pool, but if
+ the total size is really huge, other pools might be checked as well.
+ Some extra checks enabled at build time may enforce extra checks so
+ that the process will immediately crash if the object was not allocated
+ from this pool or experienced an overflow or some memory corruption.
+
+void pool_flush(struct pool_head *pool)
+ Free all unused objects from shared pool <pool>. Thread-local caches
+ are not affected. This is essentially used when running low on memory
+ or when stopping, in order to release a maximum amount of memory for
+ the new process.
+
+void pool_gc(struct pool_head *pool)
+ Free all unused objects from all pools, but respecting the minimum
+ number of spare objects required for each of them. Then, for operating
+ systems which support it, indicate the system that all unused memory
+ can be released. Thread-local caches are not affected. This operation
+ differs from pool_flush() in that it is run locklessly, under thread
+ isolation, and on all pools in a row. It is called by the SIGQUIT
+ signal handler and upon exit. Note that the obsolete argument <pool> is
+ not used and the convention is to pass NULL there.
+
+void dump_pools_to_trash(void)
+ Dump the current status of all pools into the trash buffer. This is
+ essentially used by the "show pools" CLI command or the SIGQUIT signal
+ handler to dump them on stderr. The total report size may not exceed
+ the size of the trash buffer. If it does, some entries will be missing.
+
+void dump_pools(void)
+ Dump the current status of all pools to stderr. This just calls
+ dump_pools_to_trash() and writes the trash to stderr.
+
+int pool_total_failures(void)
+ Report the total number of failed allocations. This is solely used to
+ report the "PoolFailed" metrics of the "show info" output. The total
+ is calculated on the fly by summing the number of failures in all pools
+ and is only meant to be used as an indicator rather than a precise
+ measure.
+
+ullong pool_total_allocated(void)
+ Report the total number of bytes allocated in all pools, for reporting
+ in the "PoolAlloc_MB" field of the "show info" output. The total is
+ calculated on the fly by summing the number of allocated bytes in all
+ pools and is only meant to be used as an indicator rather than a
+ precise measure.
+
+ullong pool_total_used(void)
+ Report the total number of bytes used in all pools, for reporting in
+ the "PoolUsed_MB" field of the "show info" output. The total is
+ calculated on the fly by summing the number of used bytes in all pools
+ and is only meant to be used as an indicator rather than a precise
+ measure. Note that objects present in caches are accounted as used.
+
+Some other functions exist and are only used by the pools code itself. While
+not strictly forbidden to use outside of this code, it is generally recommended
+to avoid touching them in order not to create undesired dependencies that will
+complicate maintenance.
+
+A few macros exist to ease the declaration of pools:
+
+DECLARE_POOL(ptr, name, size)
+ Placed at the top level of a file, this declares a global memory pool
+ as variable <ptr>, name <name> and size <size> bytes per element. This
+ is made via a call to REGISTER_POOL() and by assigning the resulting
+ pointer to variable <ptr>. <ptr> will be created of type "struct
+ pool_head *". If the pool needs to be visible outside of the function
+ (which is likely), it will also need to be declared somewhere as
+ "extern struct pool_head *<ptr>;". It is recommended to place such
+ declarations very early in the source file so that the variable is
+ already known to all subsequent functions which may use it.
+
+DECLARE_STATIC_POOL(ptr, name, size)
+ Placed at the top level of a file, this declares a static memory pool
+ as variable <ptr>, name <name> and size <size> bytes per element. This
+ is made via a call to REGISTER_POOL() and by assigning the resulting
+ pointer to local variable <ptr>. <ptr> will be created of type "static
+ struct pool_head *". It is recommended to place such declarations very
+ early in the source file so that the variable is already known to all
+ subsequent functions which may use it.
+
+
+6. Build options
+----------------
+
+A number of build-time defines allow to tune the pools behavior. All of them
+have to be enabled using "-Dxxx" or "-Dxxx=yyy" in the makefile's DEBUG
+variable.
+
+DEBUG_NO_POOLS
+ When this is set, pools are entirely disabled, and allocations are made
+ using malloc() instead. This is not recommended for production but may
+ be useful for tracing allocations. It corresponds to "-dMno-cache" at
+ boot time.
+
+DEBUG_MEMORY_POOLS
+ When this is set, an extra pointer is allocated at the end of each
+ object to reference the pool the object was allocated from and detect
+ buffer overflows. Then, pool_free() will provoke a crash in case it
+ detects an anomaly (pointer at the end not matching the pool). It
+ corresponds to "-dMtag" at boot time.
+
+DEBUG_FAIL_ALLOC
+ When enabled, a global setting "tune.fail-alloc" may be set to a non-
+ zero value representing a percentage of memory allocations that will be
+ made to fail in order to stress the calling code. It corresponds to
+ "-dMfail" at boot time.
+
+DEBUG_DONT_SHARE_POOLS
+ When enabled, pools of similar sizes are not merged unless the have the
+ exact same name. It corresponds to "-dMno-merge" at boot time.
+
+DEBUG_UAF
+ When enabled, pools are disabled and all allocations and releases pass
+ through mmap() and munmap(). The memory usage significantly inflates
+ and the performance degrades, but this allows to detect a lot of
+ use-after-free conditions by crashing the program at the first abnormal
+ access. This should not be used in production. It corresponds to
+ boot-time options "-dMuaf". Caching is disabled but may be re-enabled
+ using "-dMcache".
+
+DEBUG_POOL_INTEGRITY
+ When enabled, objects picked from the cache are checked for corruption
+ by comparing their contents against a pattern that was placed when they
+ were inserted into the cache. Objects are also allocated in the reverse
+ order, from the oldest one to the most recent, so as to maximize the
+ ability to detect such a corruption. The goal is to detect writes after
+ free (or possibly hardware memory corruptions). Contrary to DEBUG_UAF
+ this cannot detect reads after free, but may possibly detect later
+ corruptions and will not consume extra memory. The CPU usage will
+ increase a bit due to the cost of filling/checking the area and for the
+ preference for cold cache instead of hot cache, though not as much as
+ with DEBUG_UAF. This option is meant to be usable in production. It
+ corresponds to boot-time options "-dMcold-first,integrity".
+
+DEBUG_POOL_TRACING
+ When enabled, the callers of pool_alloc() and pool_free() will be
+ recorded into an extra memory area placed after the end of the object.
+ This may only be required by developers who want to get a few more
+ hints about code paths involved in some crashes, but will serve no
+ purpose outside of this. It remains compatible (and completes well)
+ DEBUG_POOL_INTEGRITY above. Such information become meaningless once
+ the objects leave the thread-local cache. It corresponds to boot-time
+ option "-dMcaller".
+
+DEBUG_MEM_STATS
+ When enabled, all malloc/calloc/realloc/strdup/free calls are accounted
+ for per call place (file+line number), and may be displayed or reset on
+ the CLI using "debug dev memstats". This is essentially used to detect
+ potential leaks or abnormal usages. When pools are enabled (default),
+ such calls are rare and the output will mostly contain calls induced by
+ libraries. When pools are disabled, about all calls to pool_alloc() and
+ pool_free() will also appear since they will be remapped to standard
+ functions.
+
+CONFIG_HAP_GLOBAL_POOLS
+ When enabled, process-wide shared pools will be forcefully enabled even
+ if not considered useful on the platform. The default is to let haproxy
+ decide based on the OS and C library. It corresponds to boot-time
+ option "-dMglobal".
+
+CONFIG_HAP_NO_GLOBAL_POOLS
+ When enabled, process-wide shared pools will be forcefully disabled
+ even if considered useful on the platform. The default is to let
+ haproxy decide based on the OS and C library. It corresponds to
+ boot-time option "-dMno-global".
+
+CONFIG_HAP_POOL_CACHE_SIZE
+ This allows one to define the default size of the per-thread cache, in
+ bytes. The default value is 512 kB (524288). Smaller values will use
+ less memory at the expense of a possibly higher CPU usage when using
+ many threads. Higher values will give diminishing returns on
+ performance while using much more memory. Usually there is no benefit
+ in using more than a per-core L2 cache size. It would be better not to
+ set this value lower than a few times the size of a buffer (bufsize,
+ defaults to 16 kB). In addition, keep in mind that this option may be
+ changed at runtime using "tune.memory.hot-size".
+
+CONFIG_HAP_POOL_CLUSTER_SIZE
+ This allows one to define the maximum number of objects that will be
+ groupped together in an allocation from the shared pool. Values 4 to 8
+ have experimentally shown good results with 16 threads. On systems with
+ more cores or loosely coupled caches exhibiting slow atomic operations,
+ it could possibly make sense to slightly increase this value.
diff --git a/doc/internals/api/scheduler.txt b/doc/internals/api/scheduler.txt
new file mode 100644
index 0000000..dd1ad5f
--- /dev/null
+++ b/doc/internals/api/scheduler.txt
@@ -0,0 +1,228 @@
+2021-11-17 - Scheduler API
+
+
+1. Background
+-------------
+
+The scheduler relies on two major parts:
+ - the wait queue or timers queue, which contains an ordered tree of the next
+ timers to expire
+
+ - the run queue, which contains tasks that were already woken up and are
+ waiting for a CPU slot to execute.
+
+There are two types of schedulable objects in HAProxy:
+ - tasks: they contain one timer and can be in the run queue without leaving
+ their place in the timers queue.
+
+ - tasklets: they do not have the timers part and are either sleeping or
+ running.
+
+Both the timers queue and run queue in fact exist both shared between all
+threads and per-thread. A task or tasklet may only be queued in a single of
+each at a time. The thread-local queues are not thread-safe while the shared
+ones are. This means that it is only permitted to manipulate an object which
+is in the local queue or in a shared queue, but then after locking it. As such
+tasks and tasklets are usually pinned to threads and do not move, or only in
+very specific ways not detailed here.
+
+In case of doubt, keep in mind that it's not permitted to manipulate another
+thread's private task or tasklet, and that any task held by another thread
+might vanish while it's being looked at.
+
+Internally a large part of the task and tasklet struct is shared between
+the two types, which reduces code duplication and eases the preservation
+of fairness in the run queue by interleaving all of them. As such, some
+fields or flags may not always be relevant to tasklets and may be ignored.
+
+
+Tasklets do not use a thread mask but use a thread ID instead, to which they
+are bound. If the thread ID is negative, the tasklet is not bound but may only
+be run on the calling thread.
+
+
+2. API
+------
+
+There are few functions exposed by the scheduler. A few more ones are in fact
+accessible but if not documented there they'd rather be avoided or used only
+when absolutely certain they're suitable, as some have delicate corner cases.
+In doubt, checking the sched.pdf diagram may help.
+
+int total_run_queues()
+ Return the approximate number of tasks in run queues. This is racy
+ and a bit inaccurate as it iterates over all queues, but it is
+ sufficient for stats reporting.
+
+int task_in_rq(t)
+ Return non-zero if the designated task is in the run queue (i.e. it was
+ already woken up).
+
+int task_in_wq(t)
+ Return non-zero if the designated task is in the timers queue (i.e. it
+ has a valid timeout and will eventually expire).
+
+int thread_has_tasks()
+ Return non-zero if the current thread has some work to be done in the
+ run queue. This is used to decide whether or not to sleep in poll().
+
+void task_wakeup(t, f)
+ Will make sure task <t> will wake up, that is, will execute at least
+ once after the start of the function is called. The task flags <f> will
+ be ORed on the task's state, among TASK_WOKEN_* flags exclusively. In
+ multi-threaded environments it is safe to wake up another thread's task
+ and even if the thread is sleeping it will be woken up. Users have to
+ keep in mind that a task running on another thread might very well
+ finish and go back to sleep before the function returns. It is
+ permitted to wake the current task up, in which case it will be
+ scheduled to run another time after it returns to the scheduler.
+
+struct task *task_unlink_wq(t)
+ Remove the task from the timers queue if it was in it, and return it.
+ It may only be done for the local thread, or for a shared thread that
+ might be in the shared queue. It must not be done for another thread's
+ task.
+
+void task_queue(t)
+ Place or update task <t> into the timers queue, where it may already
+ be, scheduling it for an expiration at date t->expire. If t->expire is
+ infinite, nothing is done, so it's safe to call this function without
+ prior checking the expiration date. It is only valid to call this
+ function for local tasks or for shared tasks who have the calling
+ thread in their thread mask.
+
+void task_set_thread(t, id)
+ Change task <t>'s thread ID to new value <id>. This may only be
+ performed by the task itself while running. This is only used to let a
+ task voluntarily migrate to another thread. Thread id -1 is used to
+ indicate "any thread". It's ignored and replaced by zero when threads
+ are disabled.
+
+void tasklet_wakeup(tl)
+ Make sure that tasklet <tl> will wake up, that is, will execute at
+ least once. The tasklet will run on its assigned thread, or on any
+ thread if its TID is negative.
+
+void tasklet_wakeup_on(tl, thr)
+ Make sure that tasklet <tl> will wake up on thread <thr>, that is, will
+ execute at least once. The designated thread may only differ from the
+ calling one if the tasklet is already configured to run on another
+ thread, and it is not permitted to self-assign a tasklet if its tid is
+ negative, as it may already be scheduled to run somewhere else. Just in
+ case, only use tasklet_wakeup() which will pick the tasklet's assigned
+ thread ID.
+
+struct tasklet *tasklet_new()
+ Allocate a new tasklet and set it to run by default on the calling
+ thread. The caller may change its tid to another one before using it.
+ The new tasklet is returned.
+
+struct task *task_new_anywhere()
+ Allocate a new task to run on any thread, and return the task, or NULL
+ in case of allocation issue. Note that such tasks will be marked as
+ shared and will go through the locked queues, thus their activity will
+ be heavier than for other ones. See also task_new_here().
+
+struct task *task_new_here()
+ Allocate a new task to run on the calling thread, and return the task,
+ or NULL in case of allocation issue.
+
+struct task *task_new_on(t)
+ Allocate a new task to run on thread <t>, and return the task, or NULL
+ in case of allocation issue.
+
+void task_destroy(t)
+ Destroy this task. The task will be unlinked from any timers queue,
+ and either immediately freed, or asynchronously killed if currently
+ running. This may only be done by one of the threads this task is
+ allowed to run on. Developers must not forget that the task's memory
+ area is not always immediately freed, and that certain misuses could
+ only have effect later down the chain (e.g. use-after-free).
+
+void tasklet_free()
+ Free this tasklet, which must not be running, so that may only be
+ called by the thread responsible for the tasklet, typically the
+ tasklet's process() function itself.
+
+void task_schedule(t, d)
+ Schedule task <t> to run no later than date <d>. If the task is already
+ running, or scheduled for an earlier instant, nothing is done. If the
+ task was not in queued or was scheduled to run later, its timer entry
+ will be updated. This function assumes that it will never be called
+ with a timer in the past nor with TICK_ETERNITY. Only one of the
+ threads assigned to the task may call this function.
+
+The task's ->process() function receives the following arguments:
+
+ - struct task *t: a pointer to the task itself. It is always valid.
+
+ - void *ctx : a copy of the task's ->context pointer at the moment
+ the ->process() function was called by the scheduler. A
+ function must use this and not task->context, because
+ task->context might possibly be changed by another thread.
+ For instance, the muxes' takeover() function do this.
+
+ - uint state : a copy of the task's ->state field at the moment the
+ ->process() function was executed. A function must use
+ this and not task->state as the latter misses the wakeup
+ reasons and may constantly change during execution along
+ concurrent wakeups (threads or signals).
+
+The possible state flags to use during a call to task_wakeup() or seen by the
+task being called are the following; they're automatically cleaned from the
+state field before the call to ->process()
+
+ - TASK_WOKEN_INIT each creation of a task causes a first wakeup with this
+ flag set. Applications should not set it themselves.
+
+ - TASK_WOKEN_TIMER this indicates the task's expire date was reached in the
+ timers queue. Applications should not set it themselves.
+
+ - TASK_WOKEN_IO indicates the wake-up happened due to I/O activity. Now
+ that all low-level I/O processing happens on tasklets,
+ this notion of I/O is now application-defined (for
+ example stream-interfaces use it to notify the stream).
+
+ - TASK_WOKEN_SIGNAL indicates that a signal the task was subscribed to was
+ received. Applications should not set it themselves.
+
+ - TASK_WOKEN_MSG any application-defined wake-up reason, usually for
+ inter-task communication (e.g filters vs streams).
+
+ - TASK_WOKEN_RES a resource the task was waiting for was finally made
+ available, allowing the task to continue its work. This
+ is essentially used by buffers and queues. Applications
+ may carefully use it for their own purpose if they're
+ certain not to rely on existing ones.
+
+ - TASK_WOKEN_OTHER any other application-defined wake-up reason.
+
+
+In addition, a few persistent flags may be observed or manipulated by the
+application, both for tasks and tasklets:
+
+ - TASK_SELF_WAKING when set, indicates that this task was found waking
+ itself up, and its class will change to bulk processing.
+ If this behavior is under control temporarily expected,
+ and it is not expected to happen again, it may make
+ sense to reset this flag from the ->process() function
+ itself.
+
+ - TASK_HEAVY when set, indicates that this task does so heavy
+ processing that it will become mandatory to give back
+ control to I/Os otherwise big latencies might occur. It
+ may be set by an application that expects something
+ heavy to happen (tens to hundreds of microseconds), and
+ reset once finished. An example of user is the TLS stack
+ which sets it when an imminent crypto operation is
+ expected.
+
+ - TASK_F_USR1 This is the first application-defined persistent flag.
+ It is always zero unless the application changes it. An
+ example of use cases is the I/O handler for backend
+ connections, to mention whether the connection is safe
+ to use or might have recently been migrated.
+
+Finally, when built with -DDEBUG_TASK, an extra sub-structure "debug" is added
+to both tasks and tasklets to note the code locations of the last two calls to
+task_wakeup() and tasklet_wakeup().
diff --git a/doc/internals/body-parsing.txt b/doc/internals/body-parsing.txt
new file mode 100644
index 0000000..be209af
--- /dev/null
+++ b/doc/internals/body-parsing.txt
@@ -0,0 +1,165 @@
+2014/04/16 - Pointer assignments during processing of the HTTP body
+
+In HAProxy, a struct http_msg is a descriptor for an HTTP message, which stores
+the state of an HTTP parser at any given instant, relative to a buffer which
+contains part of the message being inspected.
+
+Currently, an http_msg holds a few pointers and offsets to some important
+locations in a message depending on the state the parser is in. Some of these
+pointers and offsets may move when data are inserted into or removed from the
+buffer, others won't move.
+
+An important point is that the state of the parser only translates what the
+parser is reading, and not at all what is being done on the message (eg:
+forwarding).
+
+For an HTTP message <msg> and a buffer <buf>, we have the following elements
+to work with :
+
+
+Buffer :
+--------
+
+buf.size : the allocated size of the buffer. A message cannot be larger than
+ this size. In general, a message will even be smaller because the
+ size is almost always reduced by global.maxrewrite bytes.
+
+buf.data : memory area containing the part of the message being worked on. This
+ area is exactly <buf.size> bytes long. It should be seen as a sliding
+ window over the message, but in terms of implementation, it's closer
+ to a wrapping window. For ease of processing, new messages (requests
+ or responses) are aligned to the beginning of the buffer so that they
+ never wrap and common string processing functions can be used.
+
+buf.p : memory pointer (char *) to the beginning of the buffer as the parser
+ understands it. It commonly refers to the first character of an HTTP
+ request or response, but during forwarding, it can point to other
+ locations. This pointer always points to a location in <buf.data>.
+
+buf.i : number of bytes after <buf.p> that are available in the buffer. If
+ <buf.p + buf.i> exceeds <buf.data + buf.size>, then the pending data
+ wrap at the end of the buffer and continue at <buf.data>.
+
+buf.o : number of bytes already processed before <buf.p> that are pending
+ for departure. These bytes may leave at any instant once a connection
+ is established. These ones may wrap before <buf.data> to start before
+ <buf.data + buf.size>.
+
+It's common to call the part between buf.p and buf.p+buf.i the input buffer, and
+the part between buf.p-buf.o and buf.p the output buffer. This design permits
+efficient forwarding without copies. As a result, forwarding one byte from the
+input buffer to the output buffer only consists in :
+ - incrementing buf.p
+ - incrementing buf.o
+ - decrementing buf.i
+
+
+Message :
+---------
+Unless stated otherwise, all values are relative to <buf.p>, and are always
+comprised between 0 and <buf.i>. These values are relative offsets and they do
+not need to take wrapping into account, they are used as if the buffer was an
+infinite length sliding window. The buffer management functions handle the
+wrapping automatically.
+
+msg.next : points to the next byte to inspect. This offset is automatically
+ adjusted when inserting/removing some headers. In data states, it is
+ automatically adjusted to the number of bytes already inspected.
+
+msg.sov : start of value. First character of the header's value in the header
+ states, start of the body in the data states. Strictly positive
+ values indicate that headers were not forwarded yet (<buf.p> is
+ before the start of the body), and null or negative values are seen
+ after headers are forwarded (<buf.p> is at or past the start of the
+ body). The value stops changing when data start to leave the buffer
+ (in order to avoid integer overflows). So the maximum possible range
+ is -<buf.size> to +<buf.size>. This offset is automatically adjusted
+ when inserting or removing some headers. It is useful to rewind the
+ request buffer to the beginning of the body at any phase. The
+ response buffer does not really use it since it is immediately
+ forwarded to the client.
+
+msg.sol : start of line. Points to the beginning of the current header line
+ while parsing headers. It is cleared to zero in the BODY state,
+ and contains exactly the number of bytes comprising the preceding
+ chunk size in the DATA state (which can be zero), so that the sum of
+ msg.sov + msg.sol always points to the beginning of data for all
+ states starting with DATA. For chunked encoded messages, this sum
+ always corresponds to the beginning of the current chunk of data as
+ it appears in the buffer, or to be more precise, it corresponds to
+ the first of the remaining bytes of chunked data to be inspected. In
+ TRAILERS state, it contains the length of the last parsed part of
+ the trailer headers.
+
+msg.eoh : end of headers. Points to the CRLF (or LF) preceding the body and
+ marking the end of headers. It is where new headers are appended.
+ This offset is automatically adjusted when inserting/removing some
+ headers. It always contains the size of the headers excluding the
+ trailing CRLF even after headers have been forwarded.
+
+msg.eol : end of line. Points to the CRLF or LF of the current header line
+ being inspected during the various header states. In data states, it
+ holds the trailing CRLF length (1 or 2) so that msg.eoh + msg.eol
+ always equals the exact header length. It is not affected during data
+ states nor by forwarding.
+
+The beginning of the message headers can always be found this way even after
+headers or data have been forwarded, provided that everything is still present
+in the buffer :
+
+ headers = buf.p + msg->sov - msg->eoh - msg->eol
+
+
+Message length :
+----------------
+msg.chunk_len : amount of bytes of the current chunk or total message body
+ remaining to be inspected after msg.next. It is automatically
+ incremented when parsing a chunk size, and decremented as data
+ are forwarded.
+
+msg.body_len : total message body length, for logging. Equals Content-Length
+ when used, otherwise is the sum of all correctly parsed chunks.
+
+
+Message state :
+---------------
+msg.msg_state contains the current parser state, one of HTTP_MSG_*. The state
+indicates what byte is expected at msg->next.
+
+HTTP_MSG_BODY : all headers have been parsed, parsing of body has not
+ started yet.
+
+HTTP_MSG_100_SENT : parsing of body has started. If a 100-Continue was needed
+ it has already been sent.
+
+HTTP_MSG_DATA : some bytes are remaining for either the whole body when
+ the message size is determined by Content-Length, or for
+ the current chunk in chunked-encoded mode.
+
+HTTP_MSG_CHUNK_CRLF : msg->next points to the CRLF after the current data chunk.
+
+HTTP_MSG_TRAILERS : msg->next points to the beginning of a possibly empty
+ trailer line after the final empty chunk.
+
+HTTP_MSG_DONE : all the Content-Length data has been inspected, or the
+ final CRLF after trailers has been met.
+
+
+Message forwarding :
+--------------------
+Forwarding part of a message consists in advancing buf.p up to the point where
+it points to the byte following the last one to be forwarded. This can be done
+inline if enough bytes are present in the buffer, or in multiple steps if more
+buffers need to be forwarded (possibly including splicing). Thus by definition,
+after a block has been scheduled for being forwarded, msg->next and msg->sov
+must be reset.
+
+The communication channel between the producer and the consumer holds a counter
+of extra bytes remaining to be forwarded directly without consulting analysers,
+after buf.p. This counter is called to_forward. It commonly holds the advertised
+chunk length or content-length that does not fit in the buffer. For example, if
+2000 bytes are to be forwarded, and 10 bytes are present after buf.p as reported
+by buf.i, then both buf.o and buf.p will advance by 10, buf.i will be reset, and
+to_forward will be set to 1990 so that in total, 2000 bytes will be forwarded.
+At the end of the forwarding, buf.p will point to the first byte to be inspected
+after the 2000 forwarded bytes.
diff --git a/doc/internals/connect-status.txt b/doc/internals/connect-status.txt
new file mode 100644
index 0000000..70bbcc5
--- /dev/null
+++ b/doc/internals/connect-status.txt
@@ -0,0 +1,28 @@
+Normally, we should use getsockopt(fd, SOL_SOCKET, SO_ERROR) on a pending
+connect() to detect whether the connection correctly established or not.
+
+Unfortunately, getsockopt() does not report the status of a pending connection,
+which means that it returns 0 if the connection is still pending. This has to
+be expected, because as the name implies it, it only returns errors.
+
+With the speculative I/O, a new problem was introduced : if we pretend the
+socket was indicated as ready and we go to the socket's write() function,
+a pending connection will then inevitably be identified as established.
+
+In fact, there are solutions to this issue :
+
+ - send() returns -EAGAIN if it cannot write, so that as long as there are
+ pending data in the buffer, we'll be informed about the status of the
+ connection
+
+ - connect() on an already pending connection will return -1 with errno set to
+ one of the following values :
+ - EALREADY : connection already in progress
+ - EISCONN : connection already established
+ - anything else will indicate an error.
+
+=> So instead of using getsockopt() on a pending connection with no data, we
+ will switch to connect(). This implies that the connection address must be
+ known within the socket's write() function.
+
+
diff --git a/doc/internals/connection-header.txt b/doc/internals/connection-header.txt
new file mode 100644
index 0000000..b74cea0
--- /dev/null
+++ b/doc/internals/connection-header.txt
@@ -0,0 +1,196 @@
+2010/01/16 - Connection header adjustments depending on the transaction mode.
+
+
+HTTP transactions supports 5 possible modes :
+
+ WANT_TUN : default, nothing changed
+ WANT_TUN + httpclose : headers set for close in both dirs
+ WANT_KAL : keep-alive desired in both dirs
+ WANT_SCL : want close with the server and KA with the client
+ WANT_CLO : want close on both sides.
+
+When only WANT_TUN is set, nothing is changed nor analysed, so for commodity
+below, we'll refer to WANT_TUN+httpclose as WANT_TUN.
+
+The mode is adjusted in 3 steps :
+ - configuration sets initial mode
+ - request headers set required request mode
+ - response headers set the final mode
+
+
+1) Adjusting the initial mode via the configuration
+
+ option httpclose => TUN
+ option http-keep-alive => KAL
+ option http-server-close => SCL
+ option forceclose => CLO
+
+Note that option httpclose combined with any other option is equivalent to
+forceclose.
+
+
+2) Adjusting the request mode once the request is parsed
+
+If we cannot determine the body length from the headers, we set the mode to CLO
+but later we'll switch to tunnel mode once forwarding the body. That way, all
+parties are informed of the correct mode.
+
+Depending on the request version and request Connection header, we may have to
+adjust the current transaction mode and to update the connection header.
+
+mode req_ver req_hdr new_mode hdr_change
+TUN 1.0 - TUN -
+TUN 1.0 ka TUN del_ka
+TUN 1.0 close TUN del_close
+TUN 1.0 both TUN del_ka, del_close
+
+TUN 1.1 - TUN add_close
+TUN 1.1 ka TUN del_ka, add_close
+TUN 1.1 close TUN -
+TUN 1.1 both TUN del_ka
+
+KAL 1.0 - CLO -
+KAL 1.0 ka KAL -
+KAL 1.0 close CLO del_close
+KAL 1.0 both CLO del_ka, del_close
+
+KAL 1.1 - KAL -
+KAL 1.1 ka KAL del_ka
+KAL 1.1 close CLO -
+KAL 1.1 both CLO del_ka
+
+SCL 1.0 - CLO -
+SCL 1.0 ka SCL del_ka
+SCL 1.0 close CLO del_close
+SCL 1.0 both CLO del_ka, del_close
+
+SCL 1.1 - SCL add_close
+SCL 1.1 ka SCL del_ka, add_close
+SCL 1.1 close CLO -
+SCL 1.1 both CLO del_ka
+
+CLO 1.0 - CLO -
+CLO 1.0 ka CLO del_ka
+CLO 1.0 close CLO del_close
+CLO 1.0 both CLO del_ka, del_close
+
+CLO 1.1 - CLO add_close
+CLO 1.1 ka CLO del_ka, add_close
+CLO 1.1 close CLO -
+CLO 1.1 both CLO del_ka
+
+=> Summary:
+ - KAL and SCL are only possible with the same requests :
+ - 1.0 + ka
+ - 1.1 + ka or nothing
+
+ - CLO is assumed for any non-TUN request which contains at least a close
+ header, as well as for any 1.0 request without a keep-alive header.
+
+ - del_ka is set whenever we want a CLO or SCL or TUN and req contains a KA,
+ or when the req is 1.1 and contains a KA.
+
+ - del_close is set whenever a 1.0 request contains a close.
+
+ - add_close is set whenever a 1.1 request must be switched to TUN, SCL, CLO
+ and did not have a close hdr.
+
+Note that the request processing is performed in two passes, one with the
+frontend's config and a second one with the backend's config. It is only
+possible to "raise" the mode between them, so during the second pass, we have
+no reason to re-add a header that we previously removed. As an exception, the
+TUN mode is converted to CLO once combined because in fact it's an httpclose
+option set on a TUN mode connection :
+
+ BE (2)
+ | TUN KAL SCL CLO
+ ----+----+----+----+----
+ TUN | TUN CLO CLO CLO
+ +
+ KAL | CLO KAL SCL CLO
+ FE +
+ (1) SCL | CLO SCL SCL CLO
+ +
+ CLO | CLO CLO CLO CLO
+
+
+3) Adjusting the final mode once the response is parsed
+
+This part becomes trickier. It is possible that the server responds with a
+version that the client does not necessarily understand. Obviously, 1.1 clients
+are asusmed to understand 1.0 responses. The problematic case is a 1.0 client
+receiving a 1.1 response without any Connection header. Some 1.0 clients might
+know that in 1.1 this means "keep-alive" while others might ignore the version
+and assume a "close". Since we know the version on both sides, we may have to
+adjust some responses to remove any ambiguous case. That's the reason why the
+following table considers both the request and the response version. If the
+response length cannot be determined, we switch to CLO mode.
+
+mode res_ver res_hdr req_ver new_mode hdr_change
+TUN 1.0 - any TUN -
+TUN 1.0 ka any TUN del_ka
+TUN 1.0 close any TUN del_close
+TUN 1.0 both any TUN del_ka, del_close
+
+TUN 1.1 - any TUN add_close
+TUN 1.1 ka any TUN del_ka, add_close
+TUN 1.1 close any TUN -
+TUN 1.1 both any TUN del_ka
+
+KAL 1.0 - any SCL add_ka
+KAL 1.0 ka any KAL -
+KAL 1.0 close any SCL del_close, add_ka
+KAL 1.0 both any SCL del_close
+
+KAL 1.1 - 1.0 KAL add_ka
+KAL 1.1 - 1.1 KAL -
+KAL 1.1 ka 1.0 KAL -
+KAL 1.1 ka 1.1 KAL del_ka
+KAL 1.1 close 1.0 SCL del_close, add_ka
+KAL 1.1 close 1.1 SCL del_close
+KAL 1.1 both 1.0 SCL del_close
+KAL 1.1 both 1.1 SCL del_ka, del_close
+
+SCL 1.0 - any SCL add_ka
+SCL 1.0 ka any SCL -
+SCL 1.0 close any SCL del_close, add_ka
+SCL 1.0 both any SCL del_close
+
+SCL 1.1 - 1.0 SCL add_ka
+SCL 1.1 - 1.1 SCL -
+SCL 1.1 ka 1.0 SCL -
+SCL 1.1 ka 1.1 SCL del_ka
+SCL 1.1 close 1.0 SCL del_close, add_ka
+SCL 1.1 close 1.1 SCL del_close
+SCL 1.1 both 1.0 SCL del_close
+SCL 1.1 both 1.1 SCL del_ka, del_close
+
+CLO 1.0 - any CLO -
+CLO 1.0 ka any CLO del_ka
+CLO 1.0 close any CLO del_close
+CLO 1.0 both any CLO del_ka, del_close
+
+CLO 1.1 - any CLO add_close
+CLO 1.1 ka any CLO del_ka, add_close
+CLO 1.1 close any CLO -
+CLO 1.1 both any CLO del_ka
+
+=> in summary :
+ - the header operations do not depend on the initial mode, they only depend
+ on versions and current connection header(s).
+
+ - both CLO and TUN modes work similarly, they need to set a close mode on the
+ response. A 1.1 response will exclusively need the close header, while a 1.0
+ response will have it removed. Any keep-alive header is always removed when
+ found.
+
+ - a KAL request where the server wants to close turns into an SCL response so
+ that we release the server but still maintain the connection to the client.
+
+ - the KAL and SCL modes work the same way as we need to set keep-alive on the
+ response. So a 1.0 response will only have the keep-alive header with any
+ close header removed. A 1.1 response will have the keep-alive header added
+ for 1.0 requests and the close header removed for all requests.
+
+Note that the SCL and CLO modes will automatically cause the server connection
+to be closed at the end of the data transfer.
diff --git a/doc/internals/connection-scale.txt b/doc/internals/connection-scale.txt
new file mode 100644
index 0000000..7c3d902
--- /dev/null
+++ b/doc/internals/connection-scale.txt
@@ -0,0 +1,44 @@
+Problème des connexions simultanées avec un backend
+
+Pour chaque serveur, 3 cas possibles :
+
+ - pas de limite (par défaut)
+ - limite statique (maxconn)
+ - limite dynamique (maxconn/(ratio de px->conn), avec minconn)
+
+On a donc besoin d'une limite sur le proxy dans le cas de la limite
+dynamique, afin de fixer un seuil et un ratio. Ce qui compte, c'est
+le point après lequel on passe d'un régime linéaire à un régime
+saturé.
+
+On a donc 3 phases :
+
+ - régime minimal (0..srv->minconn)
+ - régime linéaire (srv->minconn..srv->maxconn)
+ - régime saturé (srv->maxconn..)
+
+Le minconn pourrait aussi ressortir du serveur ?
+En pratique, on veut :
+ - un max par serveur
+ - un seuil global auquel les serveurs appliquent le max
+ - un seuil minimal en-dessous duquel le nb de conn est
+ maintenu. Cette limite a un sens par serveur (jamais moins de X conns)
+ mais aussi en global (pas la peine de faire du dynamique en dessous de
+ X conns à répartir). La difficulté en global, c'est de savoir comment
+ on calcule le nombre min associé à chaque serveur, vu que c'est un ratio
+ défini à partir du max.
+
+Ca revient à peu près à la même chose que de faire 2 états :
+
+ - régime linéaire avec un offset (srv->minconn..srv->maxconn)
+ - régime saturé (srv->maxconn..)
+
+Sauf que dans ce cas, le min et le max sont bien par serveur, et le seuil est
+global et correspond à la limite de connexions au-delà de laquel on veut
+tourner à plein régime sur l'ensemble des serveurs. On peut donc parler de
+passage en mode "full", "saturated", "optimal". On peut également parler de
+la fin de la partie "scalable", "dynamique".
+
+=> fullconn 1000 par exemple ?
+
+
diff --git a/doc/internals/fd-migration.txt b/doc/internals/fd-migration.txt
new file mode 100644
index 0000000..aaddad3
--- /dev/null
+++ b/doc/internals/fd-migration.txt
@@ -0,0 +1,138 @@
+2021-07-30 - File descriptor migration between threads
+
+An FD migration may happen on any idle connection that experiences a takeover()
+operation by another thread. In this case the acting thread becomes the owner
+of the connection (and FD) while previous one(s) need to forget about it.
+
+File descriptor migration between threads is a fairly complex operation because
+it is required to maintain a durable consistency between the pollers states and
+the haproxy's desired state. Indeed, very often the FD is registered within one
+thread's poller and that thread might be waiting in the system, so there is no
+way to synchronously update it. This is where thread_mask, polled_mask and per
+thread updates are used:
+
+ - a thread knows if it's allowed to manipulate an FD by looking at its bit in
+ the FD's thread_mask ;
+
+ - each thread knows if it was polling an FD by looking at its bit in the
+ polled_mask field ; a recent migration is usually indicated by a bit being
+ present in polled_mask and absent from thread_mask.
+
+ - other threads know whether it's safe to take over an FD by looking at the
+ running mask: if it contains any other thread's bit, then other threads are
+ using it and it's not safe to take it over.
+
+ - sleeping threads are notified about the need to update their polling via
+ local or global updates to the FD. Each thread has its own local update
+ list and its own bit in the update_mask to know whether there are pending
+ updates for it. This allows to reconverge polling with the desired state
+ at the last instant before polling.
+
+While the description above could be seen as "progressive" (it technically is)
+in that there is always a transition and convergence period in a migrated FD's
+life, functionally speaking it's perfectly atomic thanks to the running bit and
+to the per-thread idle connections lock: no takeover is permitted without
+holding the idle_conns lock, and takeover may only happen by atomically picking
+a connection from the list that is also protected by this lock. In practice, an
+FD is never taken over by itself, but always in the context of a connection,
+and by atomically removing a connection from an idle list, it is possible to
+guarantee that a connection will not be picked, hence that its FD will not be
+taken over.
+
+same thread as list!
+
+The possible entry points to a race to use a file descriptor are the following
+ones, with their respective sequences:
+
+ 1) takeover: requested by conn_backend_get() on behalf of connect_server()
+ - take the idle_conns_lock, protecting against a parallel access from the
+ I/O tasklet or timeout task
+ - pick the first connection from the list
+ - attempt an fd_takeover() on this connection's fd. Usually it works,
+ unless a late wakeup of the owning thread shows up in the FD's running
+ mask. The operation is performed in fd_takeover() using a DWCAS which
+ tries to switch both running and thread_mask to the caller's tid_bit. A
+ concurrent bit in running is enough to make it fail. This guarantees
+ another thread does not wakeup from I/O in the middle of the takeover.
+ In case of conflict, this FD is skipped and the attempt is tried again
+ with the next connection.
+ - resets the task/tasklet contexts to NULL, as a signal that they are not
+ allowed to run anymore. The tasks retrieve their execution context from
+ the scheduler in the arguments, but will check the tasks' context from
+ the structure under the lock to detect this possible change, and abort.
+ - at this point the takeover succeeded, the idle_conns_lock is released and
+ the connection and its FD are now owned by the caller
+
+ 2) poll report: happens on late rx, shutdown or error on idle conns
+ - fd_set_running() is called to atomically set the running_mask and check
+ that the caller's tid_bit is still present in the thread_mask. Upon
+ failure the caller arranges itself to stop reporting that FD (e.g. by
+ immediate removal or by an asynchronous update). Upon success, it's
+ guaranteed that any concurrent fd_takeover() will fail the DWCAS and that
+ another connection will need to be picked instead.
+ - FD's state is possibly updated
+ - the iocb is called if needed (almost always)
+ - if the iocb didn't kill the connection, release the bit from running_mask
+ making the connection possibly available to a subsequent fd_takeover().
+
+ 3) I/O tasklet, timeout task: timeout or subscribed wakeup
+ - start by taking the idle_conns_lock, ensuring no takeover() will pick the
+ same connection from this point.
+ - check the task/tasklet's context to verify that no recently completed
+ takeover() stole the connection. If it's NULL, the connection was lost,
+ the lock is released and the task/tasklet killed. Otherwise it is
+ guaranteed that no other thread may use that connection (current takeover
+ candidates are waiting on the lock, previous owners waking from poll()
+ lost their bit in the thread_mask and will not touch the FD).
+ - the connection is removed from the idle conns list. From this point on,
+ no other thread will even find it there nor even try fd_takeover() on it.
+ - the idle_conns_lock is now released, the connection is protected and its
+ FD is not reachable by other threads anymore.
+ - the task does what it has to do
+ - if the connection is still usable (i.e. not upon timeout), it's inserted
+ again into the idle conns list, meaning it may instantly be taken over
+ by a competing thread.
+
+ 4) wake() callback: happens on last user after xfers (may free() the conn)
+ - the connection is still owned by the caller, it's still subscribed to
+ polling but the connection is idle thus inactive. Errors or shutdowns
+ may be reported late, via sock_conn_iocb() and conn_notify_mux(), thus
+ the running bit is set (i.e. a concurrent fd_takeover() will fail).
+ - if the connection is in the list, the idle_conns_lock is grabbed, the
+ connection is removed from the list, and the lock is released.
+ - mux->wake() is called
+ - if the connection previously was in the list, it's reinserted under the
+ idle_conns_lock.
+
+
+With the DWCAS removal between running_mask & thread_mask:
+
+fd_takeover:
+ 1 if (!CAS(&running_mask, 0, tid_bit))
+ 2 return fail;
+ 3 atomic_store(&thread_mask, tid_bit);
+ 4 atomic_and(&running_mask, ~tid_bit);
+
+poller:
+ 1 do {
+ 2 /* read consistent running_mask & thread_mask */
+ 3 do {
+ 4 run = atomic_load(&running_mask);
+ 5 thr = atomic_load(&thread_mask);
+ 6 } while (run & ~thr);
+ 7
+ 8 if (!(thr & tid_bit)) {
+ 9 /* takeover has started */
+ 10 goto disable_fd;
+ 11 }
+ 12 } while (!CAS(&running_mask, run, run | tid_bit));
+
+fd_delete:
+ 1 atomic_or(&running_mask, tid_bit);
+ 2 atomic_store(&thread_mask, 0);
+ 3 atomic_and(&running_mask, ~tid_bit);
+
+The loop in poller:3-6 is used to make sure the thread_mask we read matches
+the last updated running_mask. If nobody can give up on fd_takeover(), it
+might even be possible to spin on thread_mask only. Late pollers will not
+set running anymore with this.
diff --git a/doc/internals/hashing.txt b/doc/internals/hashing.txt
new file mode 100644
index 0000000..260b6af
--- /dev/null
+++ b/doc/internals/hashing.txt
@@ -0,0 +1,83 @@
+2013/11/20 - How hashing works internally in haproxy - maddalab@gmail.com
+
+This document describes how HAProxy implements hashing both map-based and
+consistent hashing, both prior to versions 1.5 and the motivation and tests
+that were done when providing additional options starting in version 2.4
+
+A note on hashing in general, hash functions strive to have little
+correlation between input and output. The heart of a hash function is its
+mixing step. The behavior of the mixing step largely determines whether the
+hash function is collision-resistant. Hash functions that are collision
+resistant are more likely to have an even distribution of load.
+
+The purpose of the mixing function is to spread the effect of each message
+bit throughout all the bits of the internal state. Ideally every bit in the
+hash state is affected by every bit in the message. And we want to do that
+as quickly as possible simply for the sake of program performance. A
+function is said to satisfy the strict avalanche criterion if, whenever a
+single input bit is complemented (toggled between 0 and 1), each of the
+output bits should change with a probability of one half for an arbitrary
+selection of the remaining input bits.
+
+To guard against a combination of hash function and input that results in
+high rate of collisions, haproxy implements an avalanche algorithm on the
+result of the hashing function. In all versions 1.4 and prior avalanche is
+always applied when using the consistent hashing directive. It is intended
+to provide quite a good distribution for little input variations. The result
+is quite suited to fit over a 32-bit space with enough variations so that
+a randomly picked number falls equally before any server position, which is
+ideal for consistently hashed backends, a common use case for caches.
+
+In all versions 1.4 and prior HAProxy implements the SDBM hashing function.
+However tests show that alternatives to SDBM have a better cache
+distribution on different hashing criteria. Additional tests involving
+alternatives for hash input and an option to trigger avalanche, we found
+different algorithms perform better on different criteria. DJB2 performs
+well when hashing ascii text and is a good choice when hashing on host
+header. Other alternatives perform better on numbers and are a good choice
+when using source ip. The results also vary by use of the avalanche flag.
+
+The results of the testing can be found under the tests folder. Here is
+a summary of the discussion on the results on 1 input criteria and the
+methodology used to generate the results.
+
+A note of the setup when validating the results independently, one
+would want to avoid backend server counts that may skew the results. As
+an example with DJB2 avoid 33 servers. Please see the implementations of
+the hashing function, which can be found in the links under references.
+
+The following was the set up used
+
+(a) hash-type consistent/map-based
+(b) avalanche on/off
+(c) balanche host(hdr)
+(d) 3 criteria for inputs
+ - ~ 10K requests, including duplicates
+ - ~ 46K requests, unique requests from 1 MM requests were obtained
+ - ~ 250K requests, including duplicates
+(e) 17 servers in backend, all servers were assigned the same weight
+
+Result of the hashing were obtained across the server via monitoring log
+files for haproxy. Population Standard deviation was used to evaluate the
+efficacy of the hashing algorithm. Lower standard deviation, indicates
+a better distribution of load across the backends.
+
+On 10K requests, when using consistent hashing with avalanche on host
+headers, DJB2 significantly out performs SDBM. Std dev on SDBM was 48.95
+and DJB2 was 26.29. This relationship is inverted with avalanche disabled,
+however DJB2 with avalanche enabled out performs SDBM with avalanche
+disabled.
+
+On map-based hashing SDBM out performs DJB2 irrespective of the avalanche
+option. SDBM without avalanche is marginally better than with avalanche.
+DJB2 performs significantly worse with avalanche enabled.
+
+Summary: The results of the testing indicate that there isn't a hashing
+algorithm that can be applied across all input criteria. It is necessary
+to support alternatives to SDBM, which is generally the best option, with
+algorithms that are better for different inputs. Avalanche is not always
+applicable and may result in less smooth distribution.
+
+References:
+Mixing Functions/Avalanche: https://papa.bretmulvey.com/post/124027987928/hash-functions
+Hash Functions: http://www.cse.yorku.ca/~oz/hash.html
diff --git a/doc/internals/list.fig b/doc/internals/list.fig
new file mode 100644
index 0000000..aeb1f1d
--- /dev/null
+++ b/doc/internals/list.fig
@@ -0,0 +1,599 @@
+#FIG 3.2 Produced by xfig version 2.4
+Landscape
+Center
+Metric
+A4
+119.50
+Single
+-2
+1200 2
+6 3960 3420 4320 4230
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 4230 3860 4005 3860
+2 2 0 2 0 2 53 0 20 0.000 0 0 -1 0 0 5
+ 4005 3510 4230 3510 4230 4185 4005 4185 4005 3510
+4 1 0 50 0 14 10 0.0000 4 105 105 4120 4062 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 4118 3735 N\001
+-6
+6 4185 5580 4545 6390
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 4455 6020 4230 6020
+2 2 0 2 0 4 53 0 20 0.000 0 0 -1 0 0 5
+ 4230 5670 4455 5670 4455 6345 4230 6345 4230 5670
+4 1 0 50 0 14 10 0.0000 4 105 105 4345 6222 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 4343 5895 N\001
+-6
+6 4905 5445 5445 6525
+6 4905 5445 5445 6525
+6 5085 5580 5445 6390
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 5355 6020 5130 6020
+2 2 0 2 0 6 53 0 20 0.000 0 0 -1 0 0 5
+ 5130 5670 5355 5670 5355 6345 5130 6345 5130 5670
+4 1 0 50 0 14 10 0.0000 4 105 105 5245 6222 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 5243 5895 N\001
+-6
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 5355 5670 4905 5670
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 4905 6345 5355 6345
+2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 4905 5445 5355 5445 5355 6525 4905 6525 4905 5445
+4 1 0 50 0 14 12 0.0000 4 120 120 5040 6075 L\001
+-6
+-6
+6 5805 5445 6345 6525
+6 5805 5445 6345 6525
+6 5985 5580 6345 6390
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 6255 6020 6030 6020
+2 2 0 2 0 6 53 0 20 0.000 0 0 -1 0 0 5
+ 6030 5670 6255 5670 6255 6345 6030 6345 6030 5670
+4 1 0 50 0 14 10 0.0000 4 105 105 6145 6222 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 6143 5895 N\001
+-6
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 6255 5670 5805 5670
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 5805 6345 6255 6345
+2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 5805 5445 6255 5445 6255 6525 5805 6525 5805 5445
+4 1 0 50 0 14 12 0.0000 4 120 120 5940 6075 L\001
+-6
+-6
+6 6705 5445 7245 6525
+6 6705 5445 7245 6525
+6 6885 5580 7245 6390
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 7155 6020 6930 6020
+2 2 0 2 0 6 53 0 20 0.000 0 0 -1 0 0 5
+ 6930 5670 7155 5670 7155 6345 6930 6345 6930 5670
+4 1 0 50 0 14 10 0.0000 4 105 105 7045 6222 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 7043 5895 N\001
+-6
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 7155 5670 6705 5670
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 6705 6345 7155 6345
+2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 6705 5445 7155 5445 7155 6525 6705 6525 6705 5445
+4 1 0 50 0 14 12 0.0000 4 120 120 6840 6075 L\001
+-6
+-6
+6 450 5580 810 6390
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 720 6020 495 6020
+2 2 0 2 0 4 53 0 20 0.000 0 0 -1 0 0 5
+ 495 5670 720 5670 720 6345 495 6345 495 5670
+4 1 0 50 0 14 10 0.0000 4 105 105 610 6222 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 608 5895 N\001
+-6
+6 1170 5445 1710 6525
+6 1170 5445 1710 6525
+6 1350 5580 1710 6390
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 1620 6020 1395 6020
+2 2 0 2 0 6 53 0 20 0.000 0 0 -1 0 0 5
+ 1395 5670 1620 5670 1620 6345 1395 6345 1395 5670
+4 1 0 50 0 14 10 0.0000 4 105 105 1510 6222 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 1508 5895 N\001
+-6
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 1620 5670 1170 5670
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 1170 6345 1620 6345
+2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 1170 5445 1620 5445 1620 6525 1170 6525 1170 5445
+4 1 0 50 0 14 12 0.0000 4 120 120 1305 6075 L\001
+-6
+-6
+6 2070 5445 2610 6525
+6 2070 5445 2610 6525
+6 2250 5580 2610 6390
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 2520 6020 2295 6020
+2 2 0 2 0 6 53 0 20 0.000 0 0 -1 0 0 5
+ 2295 5670 2520 5670 2520 6345 2295 6345 2295 5670
+4 1 0 50 0 14 10 0.0000 4 105 105 2410 6222 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 2408 5895 N\001
+-6
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 2520 5670 2070 5670
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 2070 6345 2520 6345
+2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 2070 5445 2520 5445 2520 6525 2070 6525 2070 5445
+4 1 0 50 0 14 12 0.0000 4 120 120 2205 6075 L\001
+-6
+-6
+6 2970 5445 3510 6525
+6 2970 5445 3510 6525
+6 3150 5580 3510 6390
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 3420 6020 3195 6020
+2 2 0 2 0 6 53 0 20 0.000 0 0 -1 0 0 5
+ 3195 5670 3420 5670 3420 6345 3195 6345 3195 5670
+4 1 0 50 0 14 10 0.0000 4 105 105 3310 6222 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 3308 5895 N\001
+-6
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 3420 5670 2970 5670
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 2970 6345 3420 6345
+2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 2970 5445 3420 5445 3420 6525 2970 6525 2970 5445
+4 1 0 50 0 14 12 0.0000 4 120 120 3105 6075 L\001
+-6
+-6
+6 720 3420 1080 4230
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 990 3860 765 3860
+2 2 0 2 0 2 53 0 20 0.000 0 0 -1 0 0 5
+ 765 3510 990 3510 990 4185 765 4185 765 3510
+4 1 0 50 0 14 10 0.0000 4 105 105 880 4062 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 878 3735 N\001
+-6
+6 2700 3420 3060 4230
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 2970 3860 2745 3860
+2 2 0 2 0 6 53 0 20 0.000 0 0 -1 0 0 5
+ 2745 3510 2970 3510 2970 4185 2745 4185 2745 3510
+4 1 0 50 0 14 10 0.0000 4 105 105 2860 4062 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 2858 3735 N\001
+-6
+6 1620 3465 1935 4230
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 1890 3860 1665 3860
+2 2 0 2 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 1665 3510 1890 3510 1890 4185 1665 4185 1665 3510
+4 1 0 50 0 14 10 0.0000 4 105 105 1780 4062 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 1778 3735 N\001
+-6
+6 10485 3330 11025 4410
+6 10665 3465 11025 4275
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 10935 3905 10710 3905
+2 2 0 2 0 6 53 0 20 0.000 0 0 -1 0 0 5
+ 10710 3555 10935 3555 10935 4230 10710 4230 10710 3555
+4 1 0 50 0 14 10 0.0000 4 105 105 10825 4107 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 10823 3780 N\001
+-6
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 10935 3555 10485 3555
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 10485 4230 10935 4230
+2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 10485 3330 10935 3330 10935 4410 10485 4410 10485 3330
+4 1 0 50 0 14 12 0.0000 4 120 120 10620 3960 L\001
+-6
+6 7110 3105 7650 4185
+6 7110 3105 7650 4185
+6 7290 3240 7650 4050
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 7560 3680 7335 3680
+2 2 0 2 0 6 53 0 20 0.000 0 0 -1 0 0 5
+ 7335 3330 7560 3330 7560 4005 7335 4005 7335 3330
+4 1 0 50 0 14 10 0.0000 4 105 105 7450 3882 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 7448 3555 N\001
+-6
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 7560 3330 7110 3330
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 7110 4005 7560 4005
+2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 7110 3105 7560 3105 7560 4185 7110 4185 7110 3105
+4 1 0 50 0 14 12 0.0000 4 120 120 7245 3735 L\001
+-6
+-6
+6 8010 3105 8550 4185
+6 8010 3105 8550 4185
+6 8190 3240 8550 4050
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 8460 3680 8235 3680
+2 2 0 2 0 6 53 0 20 0.000 0 0 -1 0 0 5
+ 8235 3330 8460 3330 8460 4005 8235 4005 8235 3330
+4 1 0 50 0 14 10 0.0000 4 105 105 8350 3882 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 8348 3555 N\001
+-6
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 8460 3330 8010 3330
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 8010 4005 8460 4005
+2 2 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 8010 3105 8460 3105 8460 4185 8010 4185 8010 3105
+4 1 0 50 0 14 12 0.0000 4 120 120 8145 3735 L\001
+-6
+-6
+6 9315 990 12195 2160
+6 9675 1080 10035 1890
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 9945 1520 9720 1520
+2 2 0 2 0 2 53 0 20 0.000 0 0 -1 0 0 5
+ 9720 1170 9945 1170 9945 1845 9720 1845 9720 1170
+4 1 0 50 0 14 10 0.0000 4 105 105 9835 1722 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 9833 1395 N\001
+-6
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 10935 1520 10710 1520
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 11925 1520 11700 1520
+2 2 0 2 0 7 52 0 20 0.000 0 0 -1 0 0 5
+ 10710 1170 10935 1170 10935 1845 10710 1845 10710 1170
+2 2 0 2 0 6 52 0 20 0.000 0 0 -1 0 0 5
+ 11700 1170 11925 1170 11925 1845 11700 1845 11700 1170
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 9945 1350 10665 1350
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 10935 1350 11655 1350
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 8
+ 1 1 1.00 60.00 120.00
+ 11925 1350 12105 1350 12195 1350 12195 990 9315 990 9315 1350
+ 9495 1350 9675 1350
+ 0.000 1.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 8
+ 1 1 1.00 60.00 120.00
+ 9675 1710 9495 1710 9315 1710 9405 2160 12195 2160 12195 1710
+ 12105 1710 11925 1710
+ 0.000 1.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 11655 1710 10935 1710
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 10665 1710 9945 1710
+ 0.000 0.000
+4 1 0 50 0 14 10 0.0000 4 105 105 10825 1722 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 10823 1395 N\001
+4 1 0 50 0 14 10 0.0000 4 105 105 11815 1722 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 11813 1395 N\001
+-6
+6 6345 1080 6705 1890
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 6615 1520 6390 1520
+2 2 0 2 0 2 53 0 20 0.000 0 0 -1 0 0 5
+ 6390 1170 6615 1170 6615 1845 6390 1845 6390 1170
+4 1 0 50 0 14 10 0.0000 4 105 105 6505 1722 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 6503 1395 N\001
+-6
+6 7335 1080 7695 1890
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 7605 1520 7380 1520
+2 2 0 2 0 6 52 0 20 0.000 0 0 -1 0 0 5
+ 7380 1170 7605 1170 7605 1845 7380 1845 7380 1170
+4 1 0 50 0 14 10 0.0000 4 105 105 7495 1722 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 7493 1395 N\001
+-6
+6 8325 1080 8685 1890
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 8595 1520 8370 1520
+2 2 0 2 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 8370 1170 8595 1170 8595 1845 8370 1845 8370 1170
+4 1 0 50 0 14 10 0.0000 4 105 105 8485 1722 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 8483 1395 N\001
+-6
+6 3870 1215 4185 1980
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 4140 1610 3915 1610
+2 2 0 2 0 2 53 0 20 0.000 0 0 -1 0 0 5
+ 3915 1260 4140 1260 4140 1935 3915 1935 3915 1260
+4 1 0 50 0 14 10 0.0000 4 105 105 4030 1812 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 4028 1485 N\001
+-6
+6 4770 1215 5085 1980
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 5040 1610 4815 1610
+2 2 0 2 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 4815 1260 5040 1260 5040 1935 4815 1935 4815 1260
+4 1 0 50 0 14 10 0.0000 4 105 105 4930 1812 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 4928 1485 N\001
+-6
+6 2205 990 2925 2160
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 2655 1610 2430 1610
+2 2 0 2 0 2 53 0 20 0.000 0 0 -1 0 0 5
+ 2430 1260 2655 1260 2655 1935 2430 1935 2430 1260
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 6
+ 1 1 1.00 60.00 120.00
+ 2655 1440 2880 1440 2880 1035 2205 1035 2205 1440 2430 1440
+ 0.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 6
+ 1 1 1.00 60.00 120.00
+ 2655 1755 2880 1755 2880 2160 2205 2160 2205 1755 2430 1755
+ 0.000 1.000 1.000 1.000 1.000 0.000
+4 1 0 50 0 14 10 0.0000 4 105 105 2545 1812 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 2543 1485 N\001
+-6
+6 525 1350 1455 1830
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 540 1590 1440 1590
+2 2 0 2 0 7 50 0 -1 0.000 0 0 -1 0 0 5
+ 540 1365 1440 1365 1440 1815 540 1815 540 1365
+4 1 0 50 0 14 10 0.0000 4 105 735 990 1545 list *N\001
+4 1 0 50 0 14 10 0.0000 4 105 735 990 1770 list *P\001
+-6
+6 4815 3420 5175 4230
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 5085 3860 4860 3860
+2 2 0 2 0 7 53 0 20 0.000 0 0 -1 0 0 5
+ 4860 3510 5085 3510 5085 4185 4860 4185 4860 3510
+4 1 0 50 0 14 10 0.0000 4 105 105 4975 4062 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 4973 3735 N\001
+-6
+6 5715 3285 6390 4410
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2
+ 6165 3860 5940 3860
+2 2 0 2 0 6 53 0 20 0.000 0 0 -1 0 0 5
+ 5940 3510 6165 3510 6165 4185 5940 4185 5940 3510
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 6
+ 1 1 1.00 60.00 120.00
+ 6165 3690 6390 3690 6390 3285 5715 3285 5715 3690 5940 3690
+ 0.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 6
+ 1 1 1.00 60.00 120.00
+ 6165 4005 6390 4005 6390 4410 5715 4410 5715 4005 5940 4005
+ 0.000 1.000 1.000 1.000 1.000 0.000
+4 1 0 50 0 14 10 0.0000 4 105 105 6055 4062 P\001
+4 1 0 50 0 14 10 0.0000 4 105 105 6053 3735 N\001
+-6
+2 4 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 5
+ 4050 4725 7605 4725 7605 6840 4050 6840 4050 4725
+2 4 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 5
+ 315 4725 3870 4725 3870 6840 315 6840 315 4725
+2 4 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 5
+ 3150 4500 315 4500 315 2475 3150 2475 3150 4500
+2 4 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 5
+ 6660 2475 8910 2475 8910 4500 6660 4500 6660 2475
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 10035 3375 10485 3330
+2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 10080 3735 10485 3555
+2 4 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 5
+ 9135 2475 12285 2475 12285 4500 9135 4500 9135 2475
+2 4 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 5
+ 9270 270 12285 270 12285 2250 9270 2250 9270 270
+2 4 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 5
+ 5760 270 9045 270 9045 2250 5760 2250 5760 270
+2 4 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 5
+ 3465 270 5535 270 5535 2250 3465 2250 3465 270
+2 4 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 5
+ 1845 270 3240 270 3240 2250 1845 2250 1845 270
+2 4 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 5
+ 315 270 1620 270 1620 2250 315 2250 315 270
+2 4 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 5
+ 3330 2475 6435 2475 6435 4500 3330 4500 3330 2475
+2 4 0 1 0 7 50 0 -1 0.000 0 0 7 0 0 5
+ 12285 6840 12285 4725 7785 4725 7785 6840 12285 6840
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 4230 3690 4860 3690
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 4860 4050 4230 4050
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 7
+ 1 1 1.00 60.00 120.00
+ 3960 4050 3780 4050 3600 4050 3600 4410 5580 4410 5580 4050
+ 5130 4050
+ 0.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 6261 5805 6711 5670
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 4461 5805 4911 5670
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 5358 5805 5808 5670
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 6705 6210 6255 6210
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 5805 6210 5355 6210
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 4905 6210 4455 6210
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 7
+ 1 1 1.00 60.00 120.00
+ 4320 6345 4320 6525 4320 6750 7470 6750 7470 6480 7470 6210
+ 7155 6210
+ 0.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 8
+ 1 1 1.00 60.00 120.00
+ 7155 5850 7335 5850 7470 5850 7470 5355 7470 5085 4590 5085
+ 4590 5355 4860 5625
+ 0.000 1.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2526 5805 2976 5670
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 726 5805 1176 5670
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 1623 5805 2073 5670
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2970 6210 2520 6210
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2070 6210 1620 6210
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 1170 6210 720 6210
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 7
+ 1 1 1.00 60.00 120.00
+ 585 6345 585 6525 585 6750 3735 6750 3735 6480 3735 6210
+ 3420 6210
+ 0.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 8
+ 1 1 1.00 60.00 120.00
+ 3420 5850 3600 5850 3735 5850 3735 5355 3735 5085 585 5085
+ 585 5265 585 5670
+ 0.000 1.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 990 3690 1620 3690
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 1620 4050 990 4050
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 7
+ 1 1 1.00 60.00 120.00
+ 1890 3690 2340 3690 2340 3240 360 3240 360 3690 540 3690
+ 720 3690
+ 0.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 7
+ 1 1 1.00 60.00 120.00
+ 720 4050 540 4050 360 4050 360 4410 2340 4410 2340 4050
+ 1890 4050
+ 0.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 7560 3465 8010 3330
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 7560 3915 8010 3375
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 7
+ 1 1 1.00 60.00 120.00
+ 8460 3465 8775 3465 8820 3060 8730 2745 6750 2745 6705 3330
+ 7110 3330
+ 0.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 8
+ 1 1 1.00 60.00 120.00
+ 8460 3870 8820 3870 8820 4230 8640 4365 6930 4365 6750 4230
+ 6705 3510 7065 3375
+ 0.000 1.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 6615 1350 7335 1350
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 7605 1350 8325 1350
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 8
+ 1 1 1.00 60.00 120.00
+ 8595 1350 8775 1350 8865 1350 8865 990 5985 990 5985 1350
+ 6165 1350 6345 1350
+ 0.000 1.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 8
+ 1 1 1.00 60.00 120.00
+ 6345 1710 6165 1710 5985 1710 6075 2160 8865 2160 8865 1710
+ 8775 1710 8595 1710
+ 0.000 1.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 8325 1710 7605 1710
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 7335 1710 6615 1710
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 4140 1440 4770 1440
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 4770 1800 4140 1800
+ 0.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 7
+ 1 1 1.00 60.00 120.00
+ 5040 1440 5490 1440 5490 990 3510 990 3510 1440 3690 1440
+ 3870 1440
+ 0.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 7
+ 1 1 1.00 60.00 120.00
+ 3870 1800 3690 1800 3510 1800 3510 2160 5490 2160 5490 1800
+ 5040 1800
+ 0.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 0 -1 0.000 0 1 0 7
+ 1 1 1.00 60.00 120.00
+ 5130 3690 5580 3690 5580 3240 3600 3240 3600 3690 3780 3690
+ 3960 3690
+ 0.000 1.000 1.000 1.000 1.000 1.000 0.000
+4 1 0 50 0 14 10 0.0000 4 135 3780 5805 4950 Asymmetrical list starting at R(red)\001
+4 1 0 50 0 14 10 0.0000 4 135 4095 10215 4950 Symmetrical lists vs Asymmetrical lists\001
+4 1 0 50 0 12 10 0.0000 4 135 525 5130 5355 foo_0\001
+4 1 0 50 0 12 10 0.0000 4 135 525 6030 5355 foo_1\001
+4 1 0 50 0 12 10 0.0000 4 135 525 6930 5355 foo_2\001
+4 1 0 50 0 14 10 0.0000 4 135 3675 2070 4950 Symmetrical list starting at R(red)\001
+4 1 0 50 0 12 10 0.0000 4 135 525 3195 5355 foo_2\001
+4 1 0 50 0 12 10 0.0000 4 135 525 2295 5355 foo_1\001
+4 1 0 50 0 12 10 0.0000 4 135 525 1395 5355 foo_0\001
+4 1 0 50 0 12 10 0.0000 4 105 315 9855 3420 foo\001
+4 1 0 50 0 12 10 0.0000 4 105 105 9990 3825 E\001
+4 1 0 50 0 14 10 0.0000 4 135 1680 7785 2655 Linking elements\001
+4 1 0 50 0 12 10 0.0000 4 135 525 8235 3015 foo_1\001
+4 1 0 50 0 12 10 0.0000 4 135 525 7335 3015 foo_0\001
+4 1 0 50 0 14 10 0.0000 4 105 1470 2565 675 struct list *G\001
+4 1 0 50 0 14 10 0.0000 4 135 1470 2565 495 LIST_INIT(G):G\001
+4 1 0 50 0 14 10 0.0000 4 135 1890 4500 495 LIST_INSERT(G,W):W\001
+4 1 0 50 0 14 10 0.0000 4 135 3360 10665 2700 foo=LIST_ELEM(E, struct foo*, L)\001
+4 1 0 50 0 14 10 0.0000 4 135 1890 4500 675 LIST_APPEND(G,W):W\001
+4 1 0 50 0 14 10 0.0000 4 135 1890 7425 540 LIST_INSERT(G,Y):Y\001
+4 1 0 50 0 14 10 0.0000 4 135 1890 10755 540 LIST_APPEND(G,Y):Y\001
+4 1 0 50 0 14 10 0.0000 4 135 1680 1755 2790 LIST_DELETE(Y):Y\001
+4 1 0 50 0 14 10 0.0000 4 135 1890 4905 2745 LIST_DEL_INIT(Y):Y\001
+4 1 0 50 0 12 9 0.0000 4 120 2880 10665 2925 Returns a pointer to struct foo*\001
+4 1 0 50 0 12 9 0.0000 4 120 2790 10665 3105 containing header E as member L\001
+4 1 0 50 0 12 9 0.0000 4 120 2700 10755 810 adds Y at the queue (before G)\001
+4 1 0 50 0 12 9 0.0000 4 120 3060 7425 810 adds Y(yellow) just after G(green)\001
+4 1 0 50 0 12 9 0.0000 4 120 1170 4500 855 adds W(white)\001
+4 1 0 50 0 12 9 0.0000 4 105 1080 2565 900 Terminates G\001
+4 1 0 50 0 12 9 0.0000 4 90 540 990 855 N=next\001
+4 1 0 50 0 12 9 0.0000 4 105 540 990 1080 P=prev\001
+4 1 0 50 0 12 9 0.0000 4 120 2610 1755 3060 unlinks and returns Y(yellow)\001
+4 1 0 50 0 12 9 0.0000 4 120 2610 4905 3060 unlinks, inits, and returns Y\001
+4 0 0 50 0 12 8 0.0000 4 105 2175 7875 5265 - both are empty if R->P == R\001
+4 0 0 50 0 12 8 0.0000 4 90 2175 7875 5490 - last element has R->P == &L\001
+4 0 0 50 0 12 8 0.0000 4 105 3150 7875 5715 - FOREACH_ITEM(it, R, end, struct foo*, L)\001
+4 0 0 50 0 12 8 0.0000 4 105 3300 7875 5940 iterates <it> through foo{0,1,2} and stops\001
+4 0 0 50 0 12 8 0.0000 4 105 3900 7875 6165 - FOREACH_ITEM_SAFE(it, bck, R, end, struct foo*, L)\001
+4 0 0 50 0 12 8 0.0000 4 105 3750 7875 6390 does the same except that <bck> allows to delete\001
+4 0 0 50 0 12 8 0.0000 4 105 1950 7875 6570 any node, including <it>\001
+4 1 0 50 0 14 11 0.0000 4 135 1155 945 585 struct list\001
diff --git a/doc/internals/list.png b/doc/internals/list.png
new file mode 100644
index 0000000..ec41a6b
--- /dev/null
+++ b/doc/internals/list.png
Binary files differ
diff --git a/doc/internals/listener-states.fig b/doc/internals/listener-states.fig
new file mode 100644
index 0000000..863e7f5
--- /dev/null
+++ b/doc/internals/listener-states.fig
@@ -0,0 +1,150 @@
+#FIG 3.2 Produced by xfig version 2.3
+Portrait
+Center
+Metric
+A4
+300.00
+Single
+-2
+1200 2
+0 32 #ff60e0
+0 33 #ff8020
+0 34 #56c5ff
+0 35 #55d941
+0 36 #f8e010
+1 1 0 3 0 7 51 -1 20 0.000 1 0.0000 900 450 495 225 900 450 1395 450
+1 1 0 3 0 7 51 -1 20 0.000 1 0.0000 2700 450 495 225 2700 450 3195 450
+1 1 0 3 0 7 51 -1 20 0.000 1 0.0000 4500 450 495 225 4500 450 4995 450
+1 1 0 3 0 7 51 -1 20 0.000 1 0.0000 900 3465 495 225 900 3465 1395 3465
+1 1 0 3 0 7 51 -1 20 0.000 1 0.0000 2700 2475 495 225 2700 2475 3195 2475
+1 1 0 3 0 7 51 -1 20 0.000 1 0.0000 3645 1575 495 225 3645 1575 4140 1575
+1 1 0 3 0 7 51 -1 20 0.000 1 0.0000 4500 2475 495 225 4500 2475 4995 2475
+1 1 0 3 0 7 51 -1 20 0.000 1 0.0000 2700 3471 495 225 2700 3471 3195 3471
+2 1 1 3 1 7 52 -1 -1 8.000 1 0 -1 0 0 2
+ 270 1980 5355 1350
+2 2 0 2 32 32 52 -1 20 0.000 1 0 -1 0 0 5
+ 2070 3060 3330 3060 3330 3870 2070 3870 2070 3060
+2 3 0 1 33 33 53 -1 20 0.000 1 0 -1 0 0 5
+ 2070 990 5130 990 5130 2880 2070 2880 2070 990
+2 2 0 2 35 35 52 -1 20 0.000 1 0 -1 0 0 5
+ 270 90 5130 90 5130 855 270 855 270 90
+2 2 0 2 36 36 52 -1 20 0.000 1 0 -1 0 0 5
+ 270 3060 1530 3060 1530 3870 270 3870 270 3060
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 1395 450 2250 450
+ 0.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 3195 450 4050 450
+ 0.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 4095 1665 4455 2025 4500 2250
+ 0.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 3195 3510 3600 3465 4140 2655
+ 0.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 4410 2250 4365 2070 4050 1710
+ 0.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 945 3240 936 2142 2961 1917 3240 1710
+ 0.000 1.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 3195 1665 2835 1845 855 2115 855 3240
+ 0.000 1.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 5
+ 1 1 1.00 60.00 120.00
+ 990 3690 1035 3960 2880 4050 4365 3915 4410 2700
+ 0.000 1.000 1.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2700 2700 2700 3240
+ 0.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 4095 2610 3600 3375 3150 3420
+ 0.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 5
+ 1 1 1.00 60.00 120.00
+ 4500 2700 4455 4005 2655 4140 945 4005 900 3690
+ 0.000 1.000 1.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 2205 2520 1395 2745 1260 2970 1125 3240
+ 0.000 1.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 3510 1800 3330 2025 3330 2835 2970 3285
+ 0.000 1.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 1170 3285 1305 3015 1485 2790 2250 2610
+ 0.000 1.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 2205 3420 1710 3420 1395 3465
+ 0.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 1395 3510 1800 3510 2205 3465
+ 0.000 1.000 0.000
+3 0 0 3 0 7 51 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 2925 2295 3060 1980 3330 1755
+ 0.000 1.000 0.000
+3 0 0 3 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 4500 675 4455 990 3960 1395
+ 0.000 1.000 0.000
+4 1 0 50 -1 18 10 0.0000 4 120 375 900 450 NEW\001
+4 1 0 50 -1 18 10 0.0000 4 120 315 2700 450 INIT\001
+4 1 0 50 -1 18 10 0.0000 4 120 810 4500 450 ASSIGNED\001
+4 1 1 50 -1 16 10 0.0000 4 120 90 900 630 0\001
+4 1 1 50 -1 16 10 0.0000 4 120 90 2700 630 1\001
+4 1 1 50 -1 16 10 0.0000 4 120 90 4500 630 2\001
+4 1 0 50 -1 16 7 0.0000 4 120 420 1755 405 create()\001
+4 2 0 50 -1 16 7 0.0000 4 120 660 1215 2160 enable() &&\001
+4 2 0 50 -1 16 7 0.0000 4 90 540 1080 2295 !maxconn\001
+4 2 1 51 -1 16 7 1.5708 4 105 600 5355 1485 transitions\001
+4 0 1 51 -1 16 7 1.5708 4 105 600 5355 1260 transitions\001
+4 2 1 51 -1 16 7 1.5708 4 105 795 5265 1485 multi-threaded\001
+4 0 1 51 -1 16 7 1.5708 4 120 870 5265 1260 single-threaded\001
+4 0 0 52 -1 17 7 0.0000 4 90 345 315 765 no FD\001
+4 0 0 52 -1 17 7 0.0000 4 135 315 315 3825 polled\001
+4 1 0 50 -1 18 10 0.0000 4 120 555 900 3465 READY\001
+4 0 0 50 -1 16 7 0.0000 4 120 255 1170 3825 full()\001
+4 2 0 50 -1 16 7 0.0000 4 90 540 2205 3375 !maxconn\001
+4 2 0 50 -1 16 7 0.0000 4 105 675 2295 3240 resume() &&\001
+4 0 0 50 -1 16 7 0.0000 4 105 405 1395 3645 pause()\001
+4 0 0 52 -1 17 7 0.0000 4 135 585 2115 3825 shut(sock)\001
+4 2 0 50 -1 16 7 0.0000 4 120 480 4320 2205 disable()\001
+4 2 0 50 -1 16 7 0.0000 4 105 405 4005 2655 pause()\001
+4 0 0 50 -1 16 7 0.0000 4 105 465 4545 2835 resume()\001
+4 2 0 50 -1 16 7 0.0000 4 120 480 2925 2160 disable()\001
+4 0 0 50 -1 16 7 0.0000 4 105 405 3465 1980 pause()\001
+4 0 0 50 -1 16 7 0.0000 4 120 660 4230 1710 enable() &&\001
+4 0 0 50 -1 16 7 0.0000 4 75 510 4320 1845 maxconn\001
+4 2 0 50 -1 16 7 0.0000 4 105 405 2655 2835 pause()\001
+4 0 0 50 -1 16 7 0.0000 4 105 675 3375 3555 resume() &&\001
+4 0 0 50 -1 16 7 0.0000 4 75 510 3375 3645 maxconn\001
+4 0 0 50 -1 16 7 0.0000 4 120 480 1080 2655 disable()\001
+4 2 0 50 -1 16 7 0.0000 4 105 465 2160 2475 resume()\001
+4 1 0 50 -1 16 7 0.0000 4 120 330 3555 405 .add()\001
+4 0 0 50 -1 16 7 0.0000 4 120 375 4545 810 .bind()\001
+4 0 0 52 -1 17 7 0.0000 4 135 1080 2115 1125 FD ready, not polled\001
+4 0 0 50 -1 16 7 0.0000 4 120 315 1305 3240 limit()\001
+4 1 0 50 -1 18 10 0.0000 4 120 630 2700 2475 LIMITED\001
+4 1 0 50 -1 18 10 0.0000 4 120 555 3645 1575 LISTEN\001
+4 1 0 50 -1 18 10 0.0000 4 120 375 4500 2475 FULL\001
+4 1 0 50 -1 18 10 0.0000 4 120 630 2700 3465 PAUSED\001
+4 1 1 50 -1 16 10 0.0000 4 120 90 2700 3645 3\001
+4 1 1 50 -1 16 10 0.0000 4 120 90 2700 2655 7\001
+4 1 1 50 -1 16 10 0.0000 4 120 90 4500 2655 6\001
+4 1 1 50 -1 16 10 0.0000 4 120 90 900 3645 5\001
+4 1 1 50 -1 16 10 0.0000 4 120 90 3645 1755 4\001
diff --git a/doc/internals/listener-states.png b/doc/internals/listener-states.png
new file mode 100644
index 0000000..8757a12
--- /dev/null
+++ b/doc/internals/listener-states.png
Binary files differ
diff --git a/doc/internals/lua_socket.fig b/doc/internals/lua_socket.fig
new file mode 100644
index 0000000..7da3294
--- /dev/null
+++ b/doc/internals/lua_socket.fig
@@ -0,0 +1,113 @@
+#FIG 3.2 Produced by xfig version 1.8
+Landscape
+Center
+Metric
+A4
+100.00
+Single
+-2
+1200 2
+6 1125 2745 2565 3555
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 1125 2745 2565 2745 2565 3555 1125 3555 1125 2745
+4 0 0 50 -1 16 12 0.0000 4 180 1080 1215 3195 lua_State *T\001
+4 0 0 50 -1 18 12 0.0000 4 150 990 1215 2925 struct hlua\001
+4 0 0 50 -1 16 12 0.0000 4 195 1245 1215 3465 stop_list *stop\001
+-6
+6 7560 4365 10620 5265
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 7650 4635 10530 4635 10530 5175 7650 5175 7650 4635
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 7560 4365 10620 4365 10620 5265 7560 5265 7560 4365
+4 0 0 50 -1 18 12 0.0000 4 195 2565 7740 4815 struct stream_interface si[0]\001
+4 0 0 50 -1 16 12 0.0000 4 195 1725 7740 5085 enum obj_type *end\001
+4 0 0 50 -1 18 12 0.0000 4 150 1215 7650 4545 struct stream\001
+-6
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 225 4500 2745 4500
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 225 5040 2745 5040
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 225 4770 2745 4770
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 1935 5715 7740 6705
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2520 3420 3600 4095
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 225 4230 2745 4230 2745 7020 225 7020 225 4230
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 225 6300 2745 6300
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 225 6660 2745 6660
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 1035 2205 2655 2205 2655 3645 1035 3645 1035 2205
+2 1 1 4 4 7 500 -1 -1 4.000 0 0 -1 0 0 2
+ 4860 1935 4860 9225
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 7695 6435 5760 4410
+2 2 0 1 0 7 50 -1 20 0.000 0 0 -1 0 0 5
+ 3600 3915 6075 3915 6075 4410 3600 4410 3600 3915
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 9450 5040 9225 5670
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 4
+ 7740 6300 7695 6345 7695 6525 7740 6570
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 7560 5670 9765 5670 9765 7200 7560 7200 7560 5670
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 7650 5940 9675 5940 9675 7110 7650 7110 7650 5940
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 315 5310 2655 5310 2655 6165 315 6165 315 5310
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 7830 6840 2565 5580
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 7740 6705 9540 6705 9540 6930 7740 6930 7740 6705
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 405 5580 2565 5580 2565 5805 405 5805 405 5580
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 5
+ 1 1 1.00 60.00 120.00
+ 1215 3105 765 3330 720 3555 765 3915 810 4230
+ 0.000 1.000 1.000 1.000 0.000
+3 0 1 1 13 7 50 -1 -1 1.000 0 1 0 3
+ 5 1 1.00 60.00 120.00
+ 675 7020 675 7785 900 8104
+ 0.000 1.000 0.000
+3 0 1 1 13 7 50 -1 -1 1.000 0 1 0 2
+ 5 1 1.00 60.00 120.00
+ 7740 7200 7740 8100
+ 0.000 0.000
+3 0 1 1 13 7 50 -1 -1 1.000 0 1 0 3
+ 5 1 1.00 60.00 120.00
+ 7605 7200 7605 8865 7740 9000
+ 0.000 1.000 0.000
+4 0 0 50 -1 18 12 0.0000 4 150 885 315 4410 stack Lua\001
+4 0 0 50 -1 16 12 0.0000 4 195 1140 315 4680 stack entry 0\001
+4 0 0 50 -1 16 12 0.0000 4 195 1140 315 4950 stack entry 1\001
+4 0 0 50 -1 16 12 0.0000 4 195 1140 315 5220 stack entry 2\001
+4 0 0 50 -1 18 12 0.0000 4 195 1695 405 5490 struct hlua_socket\001
+4 0 0 50 -1 16 12 0.0000 4 195 1140 315 6570 stack entry 3\001
+4 0 0 50 -1 16 12 0.0000 4 195 1140 315 6930 stack entry 4\001
+4 1 12 50 -1 12 9 5.6723 4 135 540 3150 3735 (list)\001
+4 0 0 50 -1 18 12 0.0000 4 150 1305 1125 2430 struct session\001
+4 0 0 50 -1 16 12 0.0000 4 150 1440 1125 2655 struct task *task\001
+4 0 0 50 -1 12 12 0.0000 4 165 1560 990 8100 hlua_tcp_gc()\001
+4 0 0 50 -1 16 12 0.0000 4 195 2430 990 8295 Called just before the object\001
+4 0 0 50 -1 16 12 0.0000 4 195 840 990 8535 garbaging\001
+4 1 12 50 -1 12 9 5.5327 4 135 540 6390 4905 (list)\001
+4 0 0 50 -1 18 12 0.0000 4 195 2205 3690 4095 struct hlua_socket_com\001
+4 0 0 50 -1 16 12 0.0000 4 150 1440 3690 4320 struct task *task\001
+4 0 0 50 -1 18 12 0.0000 4 195 1200 7650 5850 struct appctx\001
+4 0 0 50 -1 18 12 0.0000 4 150 1110 7740 6120 struct <lua>\001
+4 0 0 50 -1 16 12 0.0000 4 195 1620 7740 6615 struct hlua_tcp *wr\001
+4 0 0 50 -1 16 12 0.0000 4 195 1590 7740 6390 struct hlua_tcp *rd\001
+4 0 0 50 -1 12 12 0.0000 4 165 2160 7875 9000 hlua_tcp_release()\001
+4 0 0 50 -1 16 12 0.0000 4 195 3150 7875 9195 Called when the applet is destroyed.\001
+4 0 0 50 -1 12 12 0.0000 4 165 2400 7875 8100 update_tcp_handler()\001
+4 0 0 50 -1 16 12 0.0000 4 195 2640 7875 8295 Called on each change on the \001
+4 0 0 50 -1 16 12 0.0000 4 195 1830 7875 8535 tcp connection state.\001
+4 0 0 50 -1 16 12 0.0000 4 150 1350 495 5760 struct xref *xref\001
+4 0 0 50 -1 16 12 0.0000 4 150 1350 7830 6885 struct xref *xref\001
diff --git a/doc/internals/lua_socket.pdf b/doc/internals/lua_socket.pdf
new file mode 100644
index 0000000..e3b80ee
--- /dev/null
+++ b/doc/internals/lua_socket.pdf
Binary files differ
diff --git a/doc/internals/muxes.fig b/doc/internals/muxes.fig
new file mode 100644
index 0000000..babdd55
--- /dev/null
+++ b/doc/internals/muxes.fig
@@ -0,0 +1,401 @@
+#FIG 3.2 Produced by xfig version 3.2.8b
+Landscape
+Center
+Inches
+Letter
+100.00
+Single
+-1
+1200 2
+0 32 #bbf2e2
+0 33 #a7ceb3
+0 34 #dae8fc
+0 35 #458dba
+0 36 #ffe6cc
+0 37 #e9b000
+0 38 #1a1a1a
+0 39 #8e8e8e
+0 40 #ffc1e7
+6 4200 8700 4800 9825
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 4261 9751 4261 8751 4761 8751 4761 9751
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 4761 9751 4761 8751 4261 8751 4261 9751 4761 9751
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 4261 8850 4761 8850
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 4261 8925 4761 8925
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 4261 9000 4761 9000
+-6
+6 1425 3525 2025 4650
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 1486 4576 1486 3576 1986 3576 1986 4576
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 1986 4576 1986 3576 1486 3576 1486 4576 1986 4576
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 1486 3675 1986 3675
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 1486 3750 1986 3750
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 1486 3825 1986 3825
+-6
+6 3225 3525 3825 4650
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 3286 4576 3286 3576 3786 3576 3786 4576
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 3786 4576 3786 3576 3286 3576 3286 4576 3786 4576
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 3286 3675 3786 3675
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 3286 3750 3786 3750
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 3286 3825 3786 3825
+-6
+6 5025 3525 5625 4650
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 5086 4576 5086 3576 5586 3576 5586 4576
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 5586 4576 5586 3576 5086 3576 5086 4576 5586 4576
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 5086 3675 5586 3675
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 5086 3750 5586 3750
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 5086 3825 5586 3825
+-6
+6 6900 3525 7500 4650
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 6961 4576 6961 3576 7461 3576 7461 4576
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 7461 4576 7461 3576 6961 3576 6961 4576 7461 4576
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 6961 3675 7461 3675
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 6961 3750 7461 3750
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 6961 3825 7461 3825
+-6
+6 11925 10725 13875 11475
+2 4 0 3 0 35 50 -1 20 0.000 1 0 7 0 0 5
+ 13800 11400 12000 11400 12000 10800 13800 10800 13800 11400
+4 1 0 49 -1 4 18 0.0000 4 285 1335 12900 11175 Transport\001
+-6
+6 6600 1200 10050 1800
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 6692 1261 9959 1261 9959 1761 6692 1761
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 6692 1761 9959 1761 9959 1261 6692 1261 6692 1761
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9750 1261 9750 1761
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9525 1261 9525 1761
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9300 1261 9300 1761
+4 1 0 46 -1 4 16 0.0000 4 210 1605 8025 1575 channel buf\001
+-6
+6 12375 8100 12900 8700
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 12600 8161 12600 8661
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 12425 8161 12825 8161 12825 8661 12425 8661
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 12425 8661 12825 8661 12825 8161 12425 8161 12425 8661
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 12675 8161 12675 8661
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 12750 8161 12750 8661
+-6
+6 11700 8100 12225 8700
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 11925 8161 11925 8661
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 11750 8161 12150 8161 12150 8661 11750 8661
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 11750 8661 12150 8661 12150 8161 11750 8161 11750 8661
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 12000 8161 12000 8661
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 12075 8161 12075 8661
+-6
+6 11025 8100 11550 8700
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 11250 8161 11250 8661
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 11075 8161 11475 8161 11475 8661 11075 8661
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 11075 8661 11475 8661 11475 8161 11075 8161 11075 8661
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 11325 8161 11325 8661
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 11400 8161 11400 8661
+-6
+6 10350 8100 10875 8700
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 10575 8161 10575 8661
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 10400 8161 10800 8161 10800 8661 10400 8661
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 10400 8661 10800 8661 10800 8161 10400 8161 10400 8661
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 10650 8161 10650 8661
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 10725 8161 10725 8661
+-6
+6 13050 8100 13575 8700
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 13275 8161 13275 8661
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 13100 8161 13500 8161 13500 8661 13100 8661
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 13100 8661 13500 8661 13500 8161 13100 8161 13100 8661
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 13350 8161 13350 8661
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 13425 8161 13425 8661
+-6
+6 13725 8100 14250 8700
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 13950 8161 13950 8661
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 13775 8161 14175 8161 14175 8661 13775 8661
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 13775 8661 14175 8661 14175 8161 13775 8161 13775 8661
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 14025 8161 14025 8661
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 14100 8161 14100 8661
+-6
+6 11100 11700 13050 12150
+1 1 0 4 20 40 49 -1 20 0.000 1 0.0000 11400 11925 225 150 11400 11925 11625 12075
+4 0 0 49 -1 4 12 0.0000 4 165 960 11850 12000 I/O tasklet\001
+-6
+6 11100 12300 11700 12600
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 11357 12331 11357 12581
+2 1 0 4 35 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 11157 12331 11614 12331 11614 12581 11157 12581
+2 3 0 0 -1 34 49 -1 20 0.000 0 0 -1 0 0 5
+ 11157 12581 11614 12581 11614 12331 11157 12331 11157 12581
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 11443 12331 11443 12581
+2 1 0 2 35 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 11529 12331 11529 12581
+-6
+1 3 0 3 0 0 49 -1 20 0.000 1 0.0000 10725 5700 75 75 10725 5700 10800 5700
+1 3 0 3 0 0 49 -1 20 0.000 1 0.0000 12750 5700 75 75 12750 5700 12825 5700
+1 3 0 3 0 0 49 -1 20 0.000 1 0.0000 13875 5700 75 75 13875 5700 13950 5700
+1 3 0 3 0 0 49 -1 20 0.000 1 0.0000 11700 5700 75 75 11700 5700 11775 5700
+1 3 0 3 0 0 49 -1 20 0.000 1 0.0000 2925 6750 75 75 2925 6750 3000 6750
+1 3 0 3 0 0 49 -1 20 0.000 1 0.0000 4950 6750 75 75 4950 6750 5025 6750
+1 3 0 3 0 0 49 -1 20 0.000 1 0.0000 6075 6750 75 75 6075 6750 6150 6750
+1 3 0 3 0 0 49 -1 20 0.000 1 0.0000 3900 6750 75 75 3900 6750 3975 6750
+1 1 0 4 37 36 49 -1 20 0.000 1 0.0000 9525 4140 583 250 9525 4140 10108 3890
+1 1 0 4 37 36 49 -1 20 0.000 1 0.0000 11341 4140 583 250 11341 4140 11924 3890
+1 1 0 4 37 36 49 -1 20 0.000 1 0.0000 13154 4140 583 250 13154 4140 13737 3890
+1 1 0 4 37 36 49 -1 20 0.000 1 0.0000 15033 4140 583 250 15033 4140 15616 3890
+1 1 0 4 37 36 49 -1 20 0.000 1 0.0000 7182 5173 583 250 7182 5173 7765 4923
+1 1 0 4 37 36 49 -1 20 0.000 1 0.0000 3507 5173 583 250 3507 5173 4090 4923
+1 1 0 4 37 36 49 -1 20 0.000 1 0.0000 1719 5173 583 250 1719 5173 2302 4923
+1 1 0 4 37 36 49 -1 20 0.000 1 0.0000 5325 5175 583 250 5325 5175 5908 4925
+1 1 0 4 10 11 45 -1 20 0.000 1 0.0000 4488 8082 612 250 4488 8082 5100 8082
+1 1 0 4 10 11 49 -1 20 0.000 1 0.0000 12333 7025 417 250 12333 7025 12750 7025
+1 1 0 4 20 40 49 -1 20 0.000 1 0.0000 12392 9240 808 210 12392 9240 13200 9240
+1 1 0 4 20 40 49 -1 20 0.000 1 0.0000 3167 9240 808 210 3167 9240 3975 9240
+1 1 0 4 37 36 49 -1 20 0.000 1 0.0000 1800 11925 225 150 1800 11925 2025 12075
+1 1 0 4 10 11 45 -1 20 0.000 1 0.0000 6600 11925 225 150 6600 11925 6825 12075
+1 1 0 4 20 40 49 -1 20 0.000 1 0.0000 8400 600 900 210 8400 600 9300 600
+2 1 1 1 0 7 49 -1 -1 4.000 1 0 -1 0 0 2
+ 2550 3300 2550 6150
+2 1 1 1 0 7 49 -1 -1 4.000 1 0 -1 0 0 2
+ 4500 3300 4500 6150
+2 1 1 1 0 7 49 -1 -1 4.000 1 0 -1 0 0 2
+ 6300 3300 6300 6150
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 90.00 180.00
+ 600 8025 600 12225
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 90.00 180.00
+ 600 3150 600 1800
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 90.00 180.00
+ 600 1500 600 150
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 1 1 4
+ 1 1 1.00 90.00 180.00
+ 1 1 1.00 90.00 180.00
+ 3000 3300 3000 1425 3675 600 7500 600
+2 3 0 4 33 32 50 -1 20 0.000 0 0 -1 0 0 5
+ 900 3300 900 9900 8100 9900 8100 3300 900 3300
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 1 0 4
+ 1 1 1.00 90.00 180.00
+ 3525 3525 3525 2625 4500 1500 6750 1500
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 1 0 3
+ 1 1 1.00 90.00 180.00
+ 11295 4425 11295 4725 11700 5625
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 1 0 3
+ 1 1 1.00 90.00 180.00
+ 9495 4425 9495 4725 10695 5700
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 1 0 3
+ 1 1 1.00 90.00 180.00
+ 13163 4425 13163 4725 12788 5625
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 1 0 3
+ 1 1 1.00 90.00 180.00
+ 15013 4427 15013 4725 13888 5702
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 9525 3525 9525 3825
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 13125 3525 13125 3825
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 15000 3525 15000 3825
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 5
+ 1 1 1.00 90.00 180.00
+ 12300 7275 12300 7725 9975 7725 9975 8400 10425 8400
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 1 0 3
+ 1 1 1.00 90.00 180.00
+ 11775 5850 12300 6450 12300 6825
+2 1 1 3 0 7 49 -1 -1 8.000 1 0 -1 0 0 3
+ 11475 6150 13200 6150 13200 6825
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 0 1 3
+ 1 1 1.00 90.00 180.00
+ 3975 6900 4500 7650 4500 7875
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 0 1 3
+ 1 1 1.00 90.00 180.00
+ 3495 5475 3495 5775 3900 6675
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 0 1 3
+ 1 1 1.00 90.00 180.00
+ 1695 5475 1695 5775 2895 6750
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 0 1 3
+ 1 1 1.00 90.00 180.00
+ 7213 5477 7213 5775 6088 6752
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 1725 4875 1725 4575
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 3525 4875 3525 4575
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 5325 4875 5325 4575
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 7200 4875 7200 4575
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 0 1 2
+ 1 1 1.00 90.00 180.00
+ 4500 8325 4500 8721
+2 1 1 3 0 7 49 -1 -1 8.000 1 0 -1 0 0 3
+ 3225 7875 3225 7350 4725 7350
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 1 1 2
+ 1 1 1.00 90.00 180.00
+ 1 1 1.00 90.00 180.00
+ 3900 10800 3225 9450
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 90.00 180.00
+ 4500 10800 4500 9750
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 1 1 3
+ 1 1 1.00 90.00 180.00
+ 1 1 1.00 90.00 180.00
+ 12375 10800 12375 9750 12375 9450
+2 1 1 1 0 7 49 -1 -1 4.000 1 0 -1 0 0 2
+ 12225 3300 12225 5025
+2 1 1 1 0 7 49 -1 -1 4.000 1 0 -1 0 0 2
+ 10425 3300 10425 5025
+2 1 1 1 0 7 49 -1 -1 4.000 1 0 -1 0 0 2
+ 14025 3300 14025 5025
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 4
+ 1 1 1.00 90.00 180.00
+ 9975 1500 10800 1500 11325 2100 11325 3825
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 1 4
+ 1 1 1.00 90.00 180.00
+ 1 1 1.00 90.00 180.00
+ 9300 600 11175 600 11775 1275 11775 3300
+2 3 0 4 33 32 50 -1 20 0.000 0 0 -1 0 0 5
+ 8700 3300 8700 9900 15900 9900 15900 3300 8700 3300
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 0 1 5
+ 1 1 1.00 90.00 180.00
+ 13200 10800 13200 10200 14625 9750 14625 8400 14175 8400
+2 1 0 3 0 7 49 -1 -1 0.000 1 0 -1 0 1 3
+ 1 1 1.00 90.00 180.00
+ 5325 5475 5325 5775 4950 6675
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 90.00 180.00
+ 600 5400 600 3300
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 0 0 2
+ 600 7800 600 5700
+2 4 0 3 0 35 50 -1 20 0.000 1 0 7 0 0 5
+ 5400 11400 3600 11400 3600 10800 5400 10800 5400 11400
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 12150 8400 12450 8400
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 11475 8400 11775 8400
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 10800 8400 11100 8400
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 12825 8400 13125 8400
+2 1 0 3 0 7 49 -1 -1 8.000 1 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 13500 8400 13800 8400
+2 4 0 3 0 35 50 -1 20 0.000 1 0 7 0 0 5
+ 2100 12600 1575 12600 1575 12300 2100 12300 2100 12600
+2 4 0 3 33 32 50 -1 20 0.000 1 0 7 0 0 5
+ 6900 12600 6375 12600 6375 12300 6900 12300 6900 12600
+4 1 0 49 -1 4 14 1.5708 4 225 1335 450 825 application\001
+4 0 0 49 -1 4 12 1.5708 4 180 2595 2850 3225 mux->subscribe(SUB_RECV)\001
+4 1 0 46 -1 4 16 1.5708 4 210 645 3600 4200 rxbuf\001
+4 1 0 46 -1 4 16 1.5708 4 210 615 4575 9375 dbuf\001
+4 1 0 49 -1 4 16 0.0000 4 210 600 12300 7125 MUX\001
+4 1 0 44 -1 4 16 0.0000 4 210 945 4500 8175 DEMUX\001
+4 2 0 49 -1 4 12 0.0000 4 150 915 3600 8100 Stream ID\001
+4 0 0 49 -1 4 12 0.0000 4 150 915 12825 7125 Stream ID\001
+4 2 0 49 -1 4 12 0.0000 4 180 1635 3300 10125 tasklet_wakeup()\001
+4 2 0 49 -1 4 12 0.0000 4 180 1635 12150 10125 tasklet_wakeup()\001
+4 2 0 49 -1 4 12 0.0000 4 180 1470 11175 3150 mux->snd_buf()\001
+4 0 0 49 -1 4 12 0.0000 4 180 1425 3675 3225 mux->rcv_buf()\001
+4 0 0 49 -1 4 12 0.0000 4 180 1920 13425 10575 xprt->snd_buf(mbuf)\001
+4 0 0 49 -1 4 12 0.0000 4 180 1830 4725 10500 xprt->rcv_buf(dbuf)\001
+4 1 0 49 -1 4 12 0.0000 4 150 3105 8400 2100 HTX contents when mode==HTTP\001
+4 2 0 49 -1 4 12 0.0000 4 180 1635 7500 450 tasklet_wakeup()\001
+4 0 0 49 -1 4 12 0.0000 4 180 1635 9300 450 tasklet_wakeup()\001
+4 1 38 48 -1 4 12 0.0000 4 150 750 9534 4200 encode\001
+4 1 38 48 -1 4 12 0.0000 4 150 750 11325 4200 encode\001
+4 1 38 48 -1 4 12 0.0000 4 150 750 13134 4200 encode\001
+4 1 38 48 -1 4 12 0.0000 4 150 750 15009 4200 encode\001
+4 1 38 48 -1 4 12 0.0000 4 150 765 1725 5250 decode\001
+4 1 38 48 -1 4 12 0.0000 4 150 765 3525 5250 decode\001
+4 1 38 48 -1 4 12 0.0000 4 150 765 5325 5250 decode\001
+4 1 38 48 -1 4 12 0.0000 4 150 765 7200 5250 decode\001
+4 1 38 48 -1 4 12 0.0000 4 180 1035 12375 9300 mux_io_cb\001
+4 0 0 49 -1 4 12 1.5708 4 180 2580 12075 3225 mux->subscribe(SUB_SEND)\001
+4 1 0 49 -1 4 14 1.5708 4 180 1425 450 4500 mux streams\001
+4 1 0 49 -1 4 14 1.5708 4 135 1980 450 6750 mux=conn->mux\001
+4 1 0 49 -1 4 18 0.0000 4 285 1335 4500 11175 Transport\001
+4 1 0 46 -1 4 16 0.0000 4 210 690 14625 8175 mbuf\001
+4 1 38 48 -1 4 12 0.0000 4 180 1035 3159 9300 mux_io_cb\001
+4 0 0 49 -1 4 12 0.0000 4 195 2805 2250 12000 encoding/decoding function\001
+4 0 0 49 -1 4 12 0.0000 4 180 1365 2250 12525 transport layer\001
+4 0 0 49 -1 4 12 0.0000 4 180 2445 7050 12525 multiplexer (MUX/DEMUX)\001
+4 0 0 49 -1 4 12 0.0000 4 195 2655 7050 12000 general processing function\001
+4 0 0 49 -1 4 12 0.0000 4 180 2820 11775 12525 stream buffer (byte-level FIFO)\001
+4 2 0 49 -1 4 12 0.0000 4 180 2550 3675 10725 xprt->subscribe(SUB_RECV)\001
+4 2 0 49 -1 4 12 0.0000 4 180 2535 12225 10725 xprt->subscribe(SUB_SEND)\001
+4 1 0 49 -1 4 14 1.5708 4 180 780 450 2550 stconn\001
+4 1 0 49 -1 4 12 1.5708 4 195 2010 900 1125 (eg: checks, streams)\001
+4 1 0 49 -1 4 14 1.5708 4 180 3720 450 10125 connection = sc->sedesc->conn\001
+4 0 0 49 -1 4 12 0.0000 4 150 600 12225 225 Notes:\001
+4 0 0 49 -1 4 12 0.0000 4 180 2220 12975 675 snd_buf() will move the\001
+4 0 0 49 -1 4 12 0.0000 4 180 2310 12975 975 buffer (zero-copy) when\001
+4 0 0 49 -1 4 12 0.0000 4 180 2310 12975 1275 the destination is empty.\001
+4 0 0 49 -1 4 12 0.0000 4 180 2220 12825 1650 - the application is also\001
+4 0 0 49 -1 4 12 0.0000 4 180 2700 12975 2250 is sc->app with sc->app_ops\001
+4 0 0 49 -1 4 12 0.0000 4 180 2490 12825 2550 - transport layers (xprt) are\001
+4 0 0 49 -1 4 12 0.0000 4 180 2250 12975 2775 stackable. conn->xprt is\001
+4 0 0 49 -1 4 12 0.0000 4 180 1635 12975 3000 the topmost one.\001
+4 0 0 49 -1 4 12 0.0000 4 180 2400 12975 1950 called the app layer and\001
+4 0 0 49 -1 4 12 0.0000 4 180 1995 12825 375 - mux->rcv_buf() and\001
+4 1 38 48 -1 4 12 0.0000 4 180 1440 8409 657 sc_conn_io_cb\001
diff --git a/doc/internals/muxes.pdf b/doc/internals/muxes.pdf
new file mode 100644
index 0000000..54f8cc7
--- /dev/null
+++ b/doc/internals/muxes.pdf
Binary files differ
diff --git a/doc/internals/muxes.png b/doc/internals/muxes.png
new file mode 100644
index 0000000..a58f42f
--- /dev/null
+++ b/doc/internals/muxes.png
Binary files differ
diff --git a/doc/internals/muxes.svg b/doc/internals/muxes.svg
new file mode 100644
index 0000000..3feaa4d
--- /dev/null
+++ b/doc/internals/muxes.svg
@@ -0,0 +1,911 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Creator: fig2dev Version 3.2.8b -->
+<!-- CreationDate: 2022-05-27 11:37:43 -->
+<!-- Magnification: 1 -->
+<svg xmlns="http://www.w3.org/2000/svg"
+ xmlns:xlink="http://www.w3.org/1999/xlink"
+ width="942pt" height="755pt"
+ viewBox="254 60 15690 12573">
+<g fill="none">
+<!-- Line -->
+<rect x="12000" y="10800" width="1800" height="600" rx="105" fill="#458dba"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Line -->
+<polygon points=" 900,3300 900,9900 8100,9900 8100,3300" fill="#bbf2e2"
+ stroke="#a7ceb3" stroke-width="45px"/>
+<!-- Line -->
+<polygon points=" 8700,3300 8700,9900 15900,9900 15900,3300" fill="#bbf2e2"
+ stroke="#a7ceb3" stroke-width="45px"/>
+<!-- Line -->
+<rect x="3600" y="10800" width="1800" height="600" rx="105" fill="#458dba"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Line -->
+<rect x="1575" y="12300" width="525" height="300" rx="105" fill="#458dba"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Line -->
+<rect x="6375" y="12300" width="525" height="300" rx="105" fill="#bbf2e2"
+ stroke="#a7ceb3" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Line -->
+<polygon points=" 4761,9751 4761,8751 4261,8751 4261,9751" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 1986,4576 1986,3576 1486,3576 1486,4576" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 3786,4576 3786,3576 3286,3576 3286,4576" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 5586,4576 5586,3576 5086,3576 5086,4576" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 7461,4576 7461,3576 6961,3576 6961,4576" fill="#dae8fc"/>
+<!-- Text -->
+<text xml:space="preserve" x="12900" y="11175" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="216" text-anchor="middle">Transport</text>
+<!-- Line -->
+<polygon points=" 6692,1761 9959,1761 9959,1261 6692,1261" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 12425,8661 12825,8661 12825,8161 12425,8161" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 11750,8661 12150,8661 12150,8161 11750,8161" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 11075,8661 11475,8661 11475,8161 11075,8161" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 10400,8661 10800,8661 10800,8161 10400,8161" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 13100,8661 13500,8661 13500,8161 13100,8161" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 13775,8661 14175,8661 14175,8161 13775,8161" fill="#dae8fc"/>
+<!-- Ellipse -->
+<ellipse cx="11400" cy="11925" rx="225" ry="150" fill="#ffc1e7"
+ stroke="#d10000" stroke-width="45px"/>
+<!-- Text -->
+<text xml:space="preserve" x="11850" y="12000" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">I/O tasklet</text>
+<!-- Line -->
+<polygon points=" 11157,12581 11614,12581 11614,12331 11157,12331" fill="#dae8fc"/>
+<!-- Circle -->
+<circle cx="10725" cy="5700" r="75" fill="#000000"
+ stroke="#000000" stroke-width="30px"/>
+<!-- Circle -->
+<circle cx="12750" cy="5700" r="75" fill="#000000"
+ stroke="#000000" stroke-width="30px"/>
+<!-- Circle -->
+<circle cx="13875" cy="5700" r="75" fill="#000000"
+ stroke="#000000" stroke-width="30px"/>
+<!-- Circle -->
+<circle cx="11700" cy="5700" r="75" fill="#000000"
+ stroke="#000000" stroke-width="30px"/>
+<!-- Circle -->
+<circle cx="2925" cy="6750" r="75" fill="#000000"
+ stroke="#000000" stroke-width="30px"/>
+<!-- Circle -->
+<circle cx="4950" cy="6750" r="75" fill="#000000"
+ stroke="#000000" stroke-width="30px"/>
+<!-- Circle -->
+<circle cx="6075" cy="6750" r="75" fill="#000000"
+ stroke="#000000" stroke-width="30px"/>
+<!-- Circle -->
+<circle cx="3900" cy="6750" r="75" fill="#000000"
+ stroke="#000000" stroke-width="30px"/>
+<!-- Ellipse -->
+<ellipse cx="9525" cy="4140" rx="583" ry="250" fill="#ffe6cc"
+ stroke="#e9b000" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="11341" cy="4140" rx="583" ry="250" fill="#ffe6cc"
+ stroke="#e9b000" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="13154" cy="4140" rx="583" ry="250" fill="#ffe6cc"
+ stroke="#e9b000" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="15033" cy="4140" rx="583" ry="250" fill="#ffe6cc"
+ stroke="#e9b000" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="7182" cy="5173" rx="583" ry="250" fill="#ffe6cc"
+ stroke="#e9b000" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="3507" cy="5173" rx="583" ry="250" fill="#ffe6cc"
+ stroke="#e9b000" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="1719" cy="5173" rx="583" ry="250" fill="#ffe6cc"
+ stroke="#e9b000" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="5325" cy="5175" rx="583" ry="250" fill="#ffe6cc"
+ stroke="#e9b000" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="12333" cy="7025" rx="417" ry="250" fill="#87cfff"
+ stroke="#0000d1" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="12392" cy="9240" rx="808" ry="210" fill="#ffc1e7"
+ stroke="#d10000" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="3167" cy="9240" rx="808" ry="210" fill="#ffc1e7"
+ stroke="#d10000" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="1800" cy="11925" rx="225" ry="150" fill="#ffe6cc"
+ stroke="#e9b000" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="8400" cy="600" rx="900" ry="210" fill="#ffc1e7"
+ stroke="#d10000" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 2550,3300 2550,6150"
+ stroke="#000000" stroke-width="8px" stroke-linejoin="round" stroke-dasharray="40 40"/>
+<!-- Line -->
+<polyline points=" 4500,3300 4500,6150"
+ stroke="#000000" stroke-width="8px" stroke-linejoin="round" stroke-dasharray="40 40"/>
+<!-- Line -->
+<polyline points=" 6300,3300 6300,6150"
+ stroke="#000000" stroke-width="8px" stroke-linejoin="round" stroke-dasharray="40 40"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp0">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 645,12029 555,12029 582,12243 618,12243z"/>
+</clipPath>
+</defs>
+<polyline points=" 600,8025 600,12225" clip-path="url(#cp0)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 600,12225 -->
+<polygon points=" 555,12029 600,12209 645,12029 555,12029"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp1">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 555,1996 645,1996 618,1782 582,1782z"/>
+</clipPath>
+</defs>
+<polyline points=" 600,3150 600,1800" clip-path="url(#cp1)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 600,1800 -->
+<polygon points=" 645,1996 600,1816 555,1996 645,1996"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp2">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 555,346 645,346 618,132 582,132z"/>
+</clipPath>
+</defs>
+<polyline points=" 600,1500 600,150" clip-path="url(#cp2)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 600,150 -->
+<polygon points=" 645,346 600,166 555,346 645,346"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp3">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 7304,555 7304,645 7518,618 7518,582z
+ M 3045,3104 2955,3104 2982,3318 3018,3318z"/>
+</clipPath>
+</defs>
+<polyline points=" 3000,3300 3000,1425 3675,600 7500,600" clip-path="url(#cp3)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 7500,600 -->
+<polygon points=" 7304,645 7484,600 7304,555 7304,645"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Backward arrow to point 3000,3300 -->
+<polygon points=" 2955,3104 3000,3284 3045,3104 2955,3104"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp4">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 6554,1455 6554,1545 6768,1518 6768,1482z
+ M 3045,3104 2955,3104 2982,3318 3018,3318z"/>
+</clipPath>
+</defs>
+<polyline points=" 3525,3525 3525,2625 4500,1500 6750,1500" clip-path="url(#cp4)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 6750,1500 -->
+<polygon points=" 6554,1545 6734,1500 6554,1455 6554,1545"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp5">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 11661,5428 11578,5465 11691,5649 11724,5634z
+ M 3045,3104 2955,3104 2982,3318 3018,3318z"/>
+</clipPath>
+</defs>
+<polyline points=" 11295,4425 11295,4725 11700,5625" clip-path="url(#cp5)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 11700,5625 -->
+<polygon points=" 11578,5465 11693,5610 11661,5428 11578,5465"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp6">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 10571,5541 10514,5611 10698,5725 10720,5697z
+ M 3045,3104 2955,3104 2982,3318 3018,3318z"/>
+</clipPath>
+</defs>
+<polyline points=" 9495,4425 9495,4725 10695,5700" clip-path="url(#cp6)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 10695,5700 -->
+<polygon points=" 10514,5611 10682,5690 10571,5541 10514,5611"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp7">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 12905,5461 12822,5427 12764,5635 12798,5649z
+ M 3045,3104 2955,3104 2982,3318 3018,3318z"/>
+</clipPath>
+</defs>
+<polyline points=" 13163,4425 13163,4725 12788,5625" clip-path="url(#cp7)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 12788,5625 -->
+<polygon points=" 12822,5427 12794,5610 12905,5461 12822,5427"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp8">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 14066,5607 14007,5539 13863,5700 13886,5727z
+ M 3045,3104 2955,3104 2982,3318 3018,3318z"/>
+</clipPath>
+</defs>
+<polyline points=" 15013,4427 15013,4725 13888,5702" clip-path="url(#cp8)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 13888,5702 -->
+<polygon points=" 14007,5539 13900,5691 14066,5607 14007,5539"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp9">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 9555,3689 9495,3689 9507,3843 9543,3843z
+ M 3045,3104 2955,3104 2982,3318 3018,3318z"/>
+</clipPath>
+</defs>
+<polyline points=" 9525,3525 9525,3825" clip-path="url(#cp9)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 9525,3825 -->
+<polygon points=" 9495,3689 9525,3809 9555,3689 9495,3689"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp10">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 13155,3689 13095,3689 13107,3843 13143,3843z
+ M 3045,3104 2955,3104 2982,3318 3018,3318z"/>
+</clipPath>
+</defs>
+<polyline points=" 13125,3525 13125,3825" clip-path="url(#cp10)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 13125,3825 -->
+<polygon points=" 13095,3689 13125,3809 13155,3689 13095,3689"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp11">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 15030,3689 14970,3689 14982,3843 15018,3843z
+ M 3045,3104 2955,3104 2982,3318 3018,3318z"/>
+</clipPath>
+</defs>
+<polyline points=" 15000,3525 15000,3825" clip-path="url(#cp11)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 15000,3825 -->
+<polygon points=" 14970,3689 15000,3809 15030,3689 14970,3689"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp12">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 10229,8355 10229,8445 10443,8418 10443,8382z
+ M 3045,3104 2955,3104 2982,3318 3018,3318z"/>
+</clipPath>
+</defs>
+<polyline points=" 12300,7275 12300,7725 9975,7725 9975,8400 10425,8400" clip-path="url(#cp12)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 10425,8400 -->
+<polygon points=" 10229,8445 10409,8400 10229,8355 10229,8445"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp13">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 12345,6629 12255,6629 12282,6843 12318,6843z
+ M 3045,3104 2955,3104 2982,3318 3018,3318z"/>
+</clipPath>
+</defs>
+<polyline points=" 11775,5850 12300,6450 12300,6825" clip-path="url(#cp13)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 12300,6825 -->
+<polygon points=" 12255,6629 12300,6809 12345,6629 12255,6629"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<polyline points=" 11475,6150 13200,6150 13200,6825"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round" stroke-dasharray="80 80"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp14">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 12345,6629 12255,6629 12282,6843 12318,6843z
+ M 4051,7087 4124,7035 3979,6875 3950,6896z"/>
+</clipPath>
+</defs>
+<polyline points=" 3975,6900 4500,7650 4500,7875" clip-path="url(#cp14)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Backward arrow to point 3975,6900 -->
+<polygon points=" 4124,7035 3984,6913 4051,7087 4124,7035"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp15">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 12345,6629 12255,6629 12282,6843 12318,6843z
+ M 3450,5671 3540,5671 3513,5457 3477,5457z"/>
+</clipPath>
+</defs>
+<polyline points=" 3495,5475 3495,5775 3900,6675" clip-path="url(#cp15)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Backward arrow to point 3495,5475 -->
+<polygon points=" 3540,5671 3495,5491 3450,5671 3540,5671"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp16">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 12345,6629 12255,6629 12282,6843 12318,6843z
+ M 1650,5671 1740,5671 1713,5457 1677,5457z"/>
+</clipPath>
+</defs>
+<polyline points=" 1695,5475 1695,5775 2895,6750" clip-path="url(#cp16)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Backward arrow to point 1695,5475 -->
+<polygon points=" 1740,5671 1695,5491 1650,5671 1740,5671"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp17">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 12345,6629 12255,6629 12282,6843 12318,6843z
+ M 7168,5673 7258,5673 7231,5459 7195,5459z"/>
+</clipPath>
+</defs>
+<polyline points=" 7213,5477 7213,5775 6088,6752" clip-path="url(#cp17)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Backward arrow to point 7213,5477 -->
+<polygon points=" 7258,5673 7213,5493 7168,5673 7258,5673"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp18">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 1695,4711 1755,4711 1743,4557 1707,4557z
+ M 7168,5673 7258,5673 7231,5459 7195,5459z"/>
+</clipPath>
+</defs>
+<polyline points=" 1725,4875 1725,4575" clip-path="url(#cp18)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 1725,4575 -->
+<polygon points=" 1755,4711 1725,4591 1695,4711 1755,4711"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp19">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 3495,4711 3555,4711 3543,4557 3507,4557z
+ M 7168,5673 7258,5673 7231,5459 7195,5459z"/>
+</clipPath>
+</defs>
+<polyline points=" 3525,4875 3525,4575" clip-path="url(#cp19)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 3525,4575 -->
+<polygon points=" 3555,4711 3525,4591 3495,4711 3555,4711"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp20">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 5295,4711 5355,4711 5343,4557 5307,4557z
+ M 7168,5673 7258,5673 7231,5459 7195,5459z"/>
+</clipPath>
+</defs>
+<polyline points=" 5325,4875 5325,4575" clip-path="url(#cp20)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 5325,4575 -->
+<polygon points=" 5355,4711 5325,4591 5295,4711 5355,4711"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp21">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 7170,4711 7230,4711 7218,4557 7182,4557z
+ M 7168,5673 7258,5673 7231,5459 7195,5459z"/>
+</clipPath>
+</defs>
+<polyline points=" 7200,4875 7200,4575" clip-path="url(#cp21)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 7200,4575 -->
+<polygon points=" 7230,4711 7200,4591 7170,4711 7230,4711"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp22">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 7170,4711 7230,4711 7218,4557 7182,4557z
+ M 4455,8521 4545,8521 4518,8307 4482,8307z"/>
+</clipPath>
+</defs>
+<polyline points=" 4500,8325 4500,8721" clip-path="url(#cp22)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Backward arrow to point 4500,8325 -->
+<polygon points=" 4545,8521 4500,8341 4455,8521 4545,8521"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<polyline points=" 3225,7875 3225,7350 4725,7350"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round" stroke-dasharray="80 80"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp23">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 3272,9646 3353,9605 3233,9426 3201,9442z
+ M 3853,10604 3772,10645 3892,10824 3924,10808z"/>
+</clipPath>
+</defs>
+<polyline points=" 3900,10800 3225,9450" clip-path="url(#cp23)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 3225,9450 -->
+<polygon points=" 3353,9605 3232,9464 3272,9646 3353,9605"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Backward arrow to point 3900,10800 -->
+<polygon points=" 3772,10645 3893,10786 3853,10604 3772,10645"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp24">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 4455,9946 4545,9946 4518,9732 4482,9732z
+ M 3853,10604 3772,10645 3892,10824 3924,10808z"/>
+</clipPath>
+</defs>
+<polyline points=" 4500,10800 4500,9750" clip-path="url(#cp24)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 4500,9750 -->
+<polygon points=" 4545,9946 4500,9766 4455,9946 4545,9946"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp25">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 12330,9646 12420,9646 12393,9432 12357,9432z
+ M 12420,10604 12330,10604 12357,10818 12393,10818z"/>
+</clipPath>
+</defs>
+<polyline points=" 12375,10800 12375,9750 12375,9450" clip-path="url(#cp25)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 12375,9450 -->
+<polygon points=" 12420,9646 12375,9466 12330,9646 12420,9646"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Backward arrow to point 12375,10800 -->
+<polygon points=" 12330,10604 12375,10784 12420,10604 12330,10604"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<polyline points=" 12225,3300 12225,5025"
+ stroke="#000000" stroke-width="8px" stroke-linejoin="round" stroke-dasharray="40 40"/>
+<!-- Line -->
+<polyline points=" 10425,3300 10425,5025"
+ stroke="#000000" stroke-width="8px" stroke-linejoin="round" stroke-dasharray="40 40"/>
+<!-- Line -->
+<polyline points=" 14025,3300 14025,5025"
+ stroke="#000000" stroke-width="8px" stroke-linejoin="round" stroke-dasharray="40 40"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp26">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 11370,3629 11280,3629 11307,3843 11343,3843z
+ M 12420,10604 12330,10604 12357,10818 12393,10818z"/>
+</clipPath>
+</defs>
+<polyline points=" 9975,1500 10800,1500 11325,2100 11325,3825" clip-path="url(#cp26)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 11325,3825 -->
+<polygon points=" 11280,3629 11325,3809 11370,3629 11280,3629"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp27">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 11820,3104 11730,3104 11757,3318 11793,3318z
+ M 9496,645 9496,555 9282,582 9282,618z"/>
+</clipPath>
+</defs>
+<polyline points=" 9300,600 11175,600 11775,1275 11775,3300" clip-path="url(#cp27)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 11775,3300 -->
+<polygon points=" 11730,3104 11775,3284 11820,3104 11730,3104"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Backward arrow to point 9300,600 -->
+<polygon points=" 9496,555 9316,600 9496,645 9496,555"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp28">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 11820,3104 11730,3104 11757,3318 11793,3318z
+ M 13245,10604 13155,10604 13182,10818 13218,10818z"/>
+</clipPath>
+</defs>
+<polyline points=" 13200,10800 13200,10200 14625,9750 14625,8400 14175,8400" clip-path="url(#cp28)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Backward arrow to point 13200,10800 -->
+<polygon points=" 13155,10604 13200,10784 13245,10604 13155,10604"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp29">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 11820,3104 11730,3104 11757,3318 11793,3318z
+ M 5280,5671 5370,5671 5343,5457 5307,5457z"/>
+</clipPath>
+</defs>
+<polyline points=" 5325,5475 5325,5775 4950,6675" clip-path="url(#cp29)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Backward arrow to point 5325,5475 -->
+<polygon points=" 5370,5671 5325,5491 5280,5671 5370,5671"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp30">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 555,3496 645,3496 618,3282 582,3282z
+ M 5280,5671 5370,5671 5343,5457 5307,5457z"/>
+</clipPath>
+</defs>
+<polyline points=" 600,5400 600,3300" clip-path="url(#cp30)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 600,3300 -->
+<polygon points=" 645,3496 600,3316 555,3496 645,3496"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<polyline points=" 600,7800 600,5700"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp31">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 12314,8370 12314,8430 12468,8418 12468,8382z
+ M 5280,5671 5370,5671 5343,5457 5307,5457z"/>
+</clipPath>
+</defs>
+<polyline points=" 12150,8400 12450,8400" clip-path="url(#cp31)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 12450,8400 -->
+<polygon points=" 12314,8430 12434,8400 12314,8370 12314,8430"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp32">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 11639,8370 11639,8430 11793,8418 11793,8382z
+ M 5280,5671 5370,5671 5343,5457 5307,5457z"/>
+</clipPath>
+</defs>
+<polyline points=" 11475,8400 11775,8400" clip-path="url(#cp32)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 11775,8400 -->
+<polygon points=" 11639,8430 11759,8400 11639,8370 11639,8430"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp33">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 10964,8370 10964,8430 11118,8418 11118,8382z
+ M 5280,5671 5370,5671 5343,5457 5307,5457z"/>
+</clipPath>
+</defs>
+<polyline points=" 10800,8400 11100,8400" clip-path="url(#cp33)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 11100,8400 -->
+<polygon points=" 10964,8430 11084,8400 10964,8370 10964,8430"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp34">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 12989,8370 12989,8430 13143,8418 13143,8382z
+ M 5280,5671 5370,5671 5343,5457 5307,5457z"/>
+</clipPath>
+</defs>
+<polyline points=" 12825,8400 13125,8400" clip-path="url(#cp34)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 13125,8400 -->
+<polygon points=" 12989,8430 13109,8400 12989,8370 12989,8430"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp35">
+ <path clip-rule="evenodd" d="M 254,60 H 15944 V 12633 H 254 z
+ M 13664,8370 13664,8430 13818,8418 13818,8382z
+ M 5280,5671 5370,5671 5343,5457 5307,5457z"/>
+</clipPath>
+</defs>
+<polyline points=" 13500,8400 13800,8400" clip-path="url(#cp35)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 13800,8400 -->
+<polygon points=" 13664,8430 13784,8400 13664,8370 13664,8430"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Text -->
+<g transform="translate(450,825) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="168" text-anchor="middle">application</text>
+</g><!-- Text -->
+<g transform="translate(2850,3225) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">mux-&gt;subscribe(SUB_RECV)</text>
+</g><!-- Text -->
+<text xml:space="preserve" x="12300" y="7125" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="192" text-anchor="middle">MUX</text>
+<!-- Text -->
+<text xml:space="preserve" x="3600" y="8100" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="end">Stream ID</text>
+<!-- Text -->
+<text xml:space="preserve" x="12825" y="7125" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">Stream ID</text>
+<!-- Text -->
+<text xml:space="preserve" x="3300" y="10125" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="end">tasklet_wakeup()</text>
+<!-- Text -->
+<text xml:space="preserve" x="12150" y="10125" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="end">tasklet_wakeup()</text>
+<!-- Text -->
+<text xml:space="preserve" x="11175" y="3150" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="end">mux-&gt;snd_buf()</text>
+<!-- Text -->
+<text xml:space="preserve" x="3675" y="3225" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">mux-&gt;rcv_buf()</text>
+<!-- Text -->
+<text xml:space="preserve" x="13425" y="10575" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">xprt-&gt;snd_buf(mbuf)</text>
+<!-- Text -->
+<text xml:space="preserve" x="4725" y="10500" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">xprt-&gt;rcv_buf(dbuf)</text>
+<!-- Text -->
+<text xml:space="preserve" x="8400" y="2100" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">HTX contents when mode==HTTP</text>
+<!-- Text -->
+<text xml:space="preserve" x="7500" y="450" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="end">tasklet_wakeup()</text>
+<!-- Text -->
+<text xml:space="preserve" x="9300" y="450" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">tasklet_wakeup()</text>
+<!-- Text -->
+<g transform="translate(12075,3225) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">mux-&gt;subscribe(SUB_SEND)</text>
+</g><!-- Text -->
+<g transform="translate(450,4500) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="168" text-anchor="middle">mux streams</text>
+</g><!-- Text -->
+<g transform="translate(450,6750) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="168" text-anchor="middle">mux=conn-&gt;mux</text>
+</g><!-- Text -->
+<text xml:space="preserve" x="4500" y="11175" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="216" text-anchor="middle">Transport</text>
+<!-- Text -->
+<text xml:space="preserve" x="2250" y="12000" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">encoding/decoding function</text>
+<!-- Text -->
+<text xml:space="preserve" x="2250" y="12525" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">transport layer</text>
+<!-- Text -->
+<text xml:space="preserve" x="7050" y="12525" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">multiplexer (MUX/DEMUX)</text>
+<!-- Text -->
+<text xml:space="preserve" x="7050" y="12000" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">general processing function</text>
+<!-- Text -->
+<text xml:space="preserve" x="11775" y="12525" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">stream buffer (byte-level FIFO)</text>
+<!-- Text -->
+<text xml:space="preserve" x="3675" y="10725" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="end">xprt-&gt;subscribe(SUB_RECV)</text>
+<!-- Text -->
+<text xml:space="preserve" x="12225" y="10725" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="end">xprt-&gt;subscribe(SUB_SEND)</text>
+<!-- Text -->
+<g transform="translate(450,2550) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="168" text-anchor="middle">stconn</text>
+</g><!-- Text -->
+<g transform="translate(900,1125) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">(eg: checks, streams)</text>
+</g><!-- Text -->
+<g transform="translate(450,10125) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="168" text-anchor="middle">connection = sc-&gt;sedesc-&gt;conn</text>
+</g><!-- Text -->
+<text xml:space="preserve" x="12225" y="225" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">Notes:</text>
+<!-- Text -->
+<text xml:space="preserve" x="12975" y="675" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">snd_buf() will move the</text>
+<!-- Text -->
+<text xml:space="preserve" x="12975" y="975" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">buffer (zero-copy) when</text>
+<!-- Text -->
+<text xml:space="preserve" x="12975" y="1275" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">the destination is empty.</text>
+<!-- Text -->
+<text xml:space="preserve" x="12825" y="1650" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">- the application is also</text>
+<!-- Text -->
+<text xml:space="preserve" x="12975" y="2250" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">is sc-&gt;app with sc-&gt;app_ops</text>
+<!-- Text -->
+<text xml:space="preserve" x="12825" y="2550" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">- transport layers (xprt) are</text>
+<!-- Text -->
+<text xml:space="preserve" x="12975" y="2775" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">stackable. conn-&gt;xprt is</text>
+<!-- Text -->
+<text xml:space="preserve" x="12975" y="3000" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">the topmost one.</text>
+<!-- Text -->
+<text xml:space="preserve" x="12975" y="1950" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">called the app layer and</text>
+<!-- Text -->
+<text xml:space="preserve" x="12825" y="375" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="start">- mux-&gt;rcv_buf() and</text>
+<!-- Line -->
+<polyline points=" 4261,9751 4261,8751 4761,8751 4761,9751"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 1486,4576 1486,3576 1986,3576 1986,4576"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 3286,4576 3286,3576 3786,3576 3786,4576"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 5086,4576 5086,3576 5586,3576 5586,4576"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 6961,4576 6961,3576 7461,3576 7461,4576"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 6692,1261 9959,1261 9959,1761 6692,1761"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 12425,8161 12825,8161 12825,8661 12425,8661"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 11750,8161 12150,8161 12150,8661 11750,8661"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 11075,8161 11475,8161 11475,8661 11075,8661"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 10400,8161 10800,8161 10800,8661 10400,8661"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 13100,8161 13500,8161 13500,8661 13100,8661"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 13775,8161 14175,8161 14175,8661 13775,8661"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 11157,12331 11614,12331 11614,12581 11157,12581"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Text -->
+<text xml:space="preserve" x="9534" y="4200" fill="#1a1a1a" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">encode</text>
+<!-- Text -->
+<text xml:space="preserve" x="11325" y="4200" fill="#1a1a1a" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">encode</text>
+<!-- Text -->
+<text xml:space="preserve" x="13134" y="4200" fill="#1a1a1a" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">encode</text>
+<!-- Text -->
+<text xml:space="preserve" x="15009" y="4200" fill="#1a1a1a" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">encode</text>
+<!-- Text -->
+<text xml:space="preserve" x="1725" y="5250" fill="#1a1a1a" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">decode</text>
+<!-- Text -->
+<text xml:space="preserve" x="3525" y="5250" fill="#1a1a1a" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">decode</text>
+<!-- Text -->
+<text xml:space="preserve" x="5325" y="5250" fill="#1a1a1a" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">decode</text>
+<!-- Text -->
+<text xml:space="preserve" x="7200" y="5250" fill="#1a1a1a" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">decode</text>
+<!-- Text -->
+<text xml:space="preserve" x="12375" y="9300" fill="#1a1a1a" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">mux_io_cb</text>
+<!-- Text -->
+<text xml:space="preserve" x="3159" y="9300" fill="#1a1a1a" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">mux_io_cb</text>
+<!-- Text -->
+<text xml:space="preserve" x="8409" y="657" fill="#1a1a1a" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="144" text-anchor="middle">sc_conn_io_cb</text>
+<!-- Line -->
+<polyline points=" 4261,8850 4761,8850"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 4261,8925 4761,8925"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 4261,9000 4761,9000"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 1486,3675 1986,3675"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 1486,3750 1986,3750"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 1486,3825 1986,3825"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 3286,3675 3786,3675"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 3286,3750 3786,3750"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 3286,3825 3786,3825"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 5086,3675 5586,3675"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 5086,3750 5586,3750"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 5086,3825 5586,3825"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 6961,3675 7461,3675"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 6961,3750 7461,3750"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 6961,3825 7461,3825"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9750,1261 9750,1761"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9525,1261 9525,1761"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9300,1261 9300,1761"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 12600,8161 12600,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 12675,8161 12675,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 12750,8161 12750,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 11925,8161 11925,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 12000,8161 12000,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 12075,8161 12075,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 11250,8161 11250,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 11325,8161 11325,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 11400,8161 11400,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 10575,8161 10575,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 10650,8161 10650,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 10725,8161 10725,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 13275,8161 13275,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 13350,8161 13350,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 13425,8161 13425,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 13950,8161 13950,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 14025,8161 14025,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 14100,8161 14100,8661"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 11357,12331 11357,12581"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 11443,12331 11443,12581"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 11529,12331 11529,12581"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Text -->
+<text xml:space="preserve" x="8025" y="1575" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="192" text-anchor="middle">channel buf</text>
+<!-- Text -->
+<g transform="translate(3600,4200) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="192" text-anchor="middle">rxbuf</text>
+</g><!-- Text -->
+<g transform="translate(4575,9375) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="192" text-anchor="middle">dbuf</text>
+</g><!-- Text -->
+<text xml:space="preserve" x="14625" y="8175" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="192" text-anchor="middle">mbuf</text>
+<!-- Ellipse -->
+<ellipse cx="4488" cy="8082" rx="612" ry="250" fill="#87cfff"
+ stroke="#0000d1" stroke-width="45px"/>
+<!-- Ellipse -->
+<ellipse cx="6600" cy="11925" rx="225" ry="150" fill="#87cfff"
+ stroke="#0000d1" stroke-width="45px"/>
+<!-- Text -->
+<text xml:space="preserve" x="4500" y="8175" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="192" text-anchor="middle">DEMUX</text>
+</g>
+</svg>
diff --git a/doc/internals/notes-layers.txt b/doc/internals/notes-layers.txt
new file mode 100644
index 0000000..541c125
--- /dev/null
+++ b/doc/internals/notes-layers.txt
@@ -0,0 +1,330 @@
+2018-02-21 - Layering in haproxy 1.9
+------------------------------------
+
+2 main zones :
+ - application : reads from conn_streams, writes to conn_streams, often uses
+ streams
+
+ - connection : receives data from the network, presented into buffers
+ available via conn_streams, sends data to the network
+
+
+The connection zone contains multiple layers which behave independently in each
+direction. The Rx direction is activated upon callbacks from the lower layers.
+The Tx direction is activated recursively from the upper layers. Between every
+two layers there may be a buffer, in each direction. When a buffer is full
+either in Tx or Rx direction, this direction is paused from the network layer
+and the location where the congestion is encountered. Upon end of congestion
+(cs_recv() from the upper layer, of sendto() at the lower layers), a
+tasklet_wakeup() is performed on the blocked layer so that suspended operations
+can be resumed. In this case, the Rx side restarts propagating data upwards
+from the lowest blocked level, while the Tx side restarts propagating data
+downwards from the highest blocked level. Proceeding like this ensures that
+information known to the producer may always be used to tailor the buffer sizes
+or decide of a strategy to best aggregate data. Additionally, each time a layer
+is crossed without transformation, it becomes possible to send without copying.
+
+The Rx side notifies the application of data readiness using a wakeup or a
+callback. The Tx side notifies the application of room availability once data
+have been moved resulting in the uppermost buffer having some free space.
+
+When crossing a mux downwards, it is possible that the sender is not allowed to
+access the buffer because it is not yet its turn. It is not a problem, the data
+remains in the conn_stream's buffer (or the stream one) and will be restarted
+once the mux is ready to consume these data.
+
+
+ cs_recv() -------. cs_send()
+ ^ +--------> |||||| -------------+ ^
+ | | -------' | | stream
+ --|----------|-------------------------------|-------|-------------------
+ | | V | connection
+ data .---. | | room
+ ready! |---| |---| available!
+ |---| |---|
+ |---| |---|
+ | | '---'
+ ^ +------------+-------+ |
+ | | ^ | /
+ / V | V /
+ / recvfrom() | sendto() |
+ -------------|----------------|--------------|---------------------------
+ | | poll! V kernel
+
+
+The cs_recv() function should act on pointers to buffer pointers, so that the
+callee may decide to pass its own buffer directly by simply swapping pointers.
+Similarly for cs_send() it is desirable to let the callee steal the buffer by
+swapping the pointers. This way it remains possible to implement zero-copy
+forwarding.
+
+Some operation flags will be needed on cs_recv() :
+ - RECV_ZERO_COPY : refuse to merge new data into the current buffer if it
+ will result in a data copy (ie the buffer is not empty), unless no more
+ than XXX bytes have to be copied (eg: copying 2 cache lines may be cheaper
+ than waiting and playing with pointers)
+
+ - RECV_AT_ONCE : only perform the operation if it will result in the source
+ buffer to become empty at the end of the operation so that no two buffers
+ remain allocated at the end. It will most of the time result in either a
+ small read or a zero-copy operation.
+
+ - RECV_PEEK : retrieve a copy of pending data without removing these data
+ from the source buffer. Maybe an alternate solution could consist in
+ finding the pointer to the source buffer and accessing these data directly,
+ except that it might be less interesting for the long term, thread-wise.
+
+ - RECV_MIN : receive minimum X bytes (or less with a shutdown), or fail.
+ This should help various protocol parsers which need to receive a complete
+ frame before proceeding.
+
+ - RECV_ENOUGH : no more data expected after this read if it's of the
+ requested size, thus no need to re-enable receiving on the lower layers.
+
+ - RECV_ONE_SHOT : perform a single read without re-enabling reading on the
+ lower layers, like we currently do when receiving an HTTP/1 request. Like
+ RECV_ENOUGH where any size is enough. Probably that the two could be merged
+ (eg: by having a MIN argument like RECV_MIN).
+
+
+Some operation flags will be needed on cs_send() :
+ - SEND_ZERO_COPY : refuse to merge the presented data with existing data and
+ prefer to wait for current data to leave and try again, unless the consumer
+ considers the amount of data acceptable for a copy.
+
+ - SEND_AT_ONCE : only perform the operation if it will result in the source
+ buffer to become empty at the end of the operation so that no two buffers
+ remain allocated at the end. It will most of the time result in either a
+ small write or a zero-copy operation.
+
+
+Both operations should return a composite status :
+ - number of bytes transferred
+ - status flags (shutr, shutw, reset, empty, full, ...)
+
+
+2018-07-23 - Update after merging rxbuf
+---------------------------------------
+
+It becomes visible that the mux will not always be welcome to decode incoming
+data because it will sometimes imply extra memory copies and/or usage for no
+benefit.
+
+Ideally, when when a stream is instantiated based on incoming data, these
+incoming data should be passed and the upper layers called, but it should then
+be up these upper layers to peek more data in certain circumstances. Typically
+if the pending connection data are larger than what is expected to be passed
+above, it means some data may cause head-of-line blocking (HOL) to other
+streams, and needs to be pushed up through the layers to let other streams
+continue to work. Similarly very large H2 data frames after header frames
+should probably not be passed as they may require copies that could be avoided
+if passed later. However if the decoded frame fits into the conn_stream's
+buffer, there is an opportunity to use a single buffer for the conn_stream
+and the channel. The H2 demux could set a blocking flag indicating it's waiting
+for the upper stream to take over demuxing. This flag would be purged once the
+upper stream would start reading, or when extra data come and change the
+conditions.
+
+Forcing structured headers and raw data to coexist within a single buffer is
+quite challenging for many code parts. For example it's perfectly possible to
+see a fragmented buffer containing series of headers, then a small data chunk
+that was received at the same time, then a few other headers added by request
+processing, then another data block received afterwards, then possibly yet
+another header added by option http-send-name-header, and yet another data
+block. This causes some pain for compression which still needs to know where
+compressed and uncompressed data start/stop. It also makes it very difficult
+to account the exact bytes to pass through the various layers.
+
+One solution consists in thinking about buffers using 3 representations :
+
+ - a structured message, which is used for the internal HTTP representation.
+ This message may only be atomically processed. It has no clear byte count,
+ it's a message.
+
+ - a raw stream, consisting in sequences of bytes. That's typically what
+ happens in data sequences or in tunnel.
+
+ - a pipe, which contains data to be forwarded, and that haproxy cannot have
+ access to.
+
+The processing efficiency decreases with the higher complexity above, but the
+capabilities increase. The structured message can contain anything including
+serialized data blocks to be processed or forwarded. The raw stream contains
+data blocks to be processed or forwarded. The pipe only contains data blocks
+to be forwarded. The the latter ones are only an optimization of the former
+ones.
+
+Thus ideally a channel should have access to all such 3 storage areas at once,
+depending on the use case :
+ (1) a structured message,
+ (2) a raw stream,
+ (3) a pipe
+
+Right now a channel only has (2) and (3) but after the native HTTP rework, it
+will only have (1) and (3). Placing a raw stream exclusively in (1) comes with
+some performance drawbacks which are not easily recovered, and with some quite
+difficult management still involving the reserve to ensure that a data block
+doesn't prevent headers from being appended. But during header processing, the
+payload may be necessary so we cannot decide to drop this option.
+
+A long-term approach would consist in ensuring that a single channel may have
+access to all 3 representations at once, and to enumerate priority rules to
+define how they interact together. That's exactly what is currently being done
+with the pipe and the raw buffer right now. Doing so would also save the need
+for storing payload in the structured message and void the requirement for the
+reserve. But it would cost more memory to process POST data and server
+responses. Thus an intermediary step consists in keeping this model in mind but
+not implementing everything yet.
+
+Short term proposal : a channel has access to a buffer and a pipe. A non-empty
+buffer is either in structured message format OR raw stream format. Only the
+channel knows. However a structured buffer MAY contain raw data in a properly
+formatted way (using the envelope defined by the structured message format).
+
+By default, when a demux writes to a CS rxbuf, it will try to use the lowest
+possible level for what is being done (i.e. splice if possible, otherwise raw
+stream, otherwise structured message). If the buffer already contains a
+structured message, then this format is exclusive. From this point the MUX has
+two options : either encode the incoming data to match the structured message
+format, or refrain from receiving into the CS's rxbuf and wait until the upper
+layer request those data.
+
+This opens a simplified option which could be suited even for the long term :
+ - cs_recv() will take one or two flags to indicate if a buffer already
+ contains a structured message or not ; the upper layer knows it.
+
+ - cs_recv() will take two flags to indicate what the upper layer is willing
+ to take :
+ - structured message only
+ - raw stream only
+ - any of them
+
+ From this point the mux can decide to either pass anything or refrain from
+ doing so.
+
+ - the demux stores the knowledge it has from the contents into some CS flags
+ to indicate whether or not some structured message are still available, and
+ whether or not some raw data are still available. Thus the caller knows
+ whether or not extra data are available.
+
+ - when the demux works on its own, it refrains from passing structured data
+ to a non-empty buffer, unless these data are causing trouble to other
+ streams (HOL).
+
+ - when a demux has to encapsulate raw data into a structured message, it will
+ always have to respect a configured reserve so that extra header processing
+ can be done on the structured message inside the buffer, regardless of the
+ supposed available room. In addition, the upper layer may indicate using an
+ extra recv() flag whether it wants the demux to defragment serialized data
+ (for example by moving trailing headers apart) or if it's not necessary.
+ This flag will be set by the stream interface if compression is required or
+ if the http-buffer-request option is set for example. Probably that using
+ to_forward==0 is a stronger indication that the reserve must be respected.
+
+ - cs_recv() and cs_send() when fed with a message, should not return byte
+ counts but message counts (i.e. 0 or 1). This implies that a single call to
+ either of these functions cannot mix raw data and structured messages at
+ the same time.
+
+At this point it looks like the conn_stream will have some encapsulation work
+to do for the payload if it needs to be encapsulated into a message. This
+further magnifies the importance of *not* decoding DATA frames into the CS's
+rxbuf until really needed.
+
+The CS will probably need to hold indication of what is available at the mux
+level, not only in the CS. Eg: we know that payload is still available.
+
+Using these elements, it should be possible to ensure that full header frames
+may be received without enforcing any reserve, that too large frames that do
+not fit will be detected because they return 0 message and indicate that such
+a message is still pending, and that data availability is correctly detected
+(later we may expect that the stream-interface allocates a larger or second
+buffer to place the payload).
+
+Regarding the ability for the channel to forward data, it looks like having a
+new function "cs_xfer(src_cs, dst_cs, count)" could be very productive in
+optimizing the forwarding to make use of splicing when available. It is not yet
+totally clear whether it will split into "cs_xfer_in(src_cs, pipe, count)"
+followed by "cs_xfer_out(dst_cs, pipe, count)" or anything different, and it
+still needs to be studied. The general idea seems to be that the receiver might
+have to call the sender directly once they agree on how to transfer data (pipe
+or buffer). If the transfer is incomplete, the cs_xfer() return value and/or
+flags will indicate the current situation (src empty, dst full, etc) so that
+the caller may register for notifications on the appropriate event and wait to
+be called again to continue.
+
+Short term implementation :
+ 1) add new CS flags to qualify what the buffer contains and what we expect
+ to read into it;
+
+ 2) set these flags to pretend we have a structured message when receiving
+ headers (after all, H1 is an atomic header as well) and see what it
+ implies for the code; for H1 it's unclear whether it makes sense to try
+ to set it without the H1 mux.
+
+ 3) use these flags to refrain from sending DATA frames after HEADERS frames
+ in H2.
+
+ 4) flush the flags at the stream interface layer when performing a cs_send().
+
+ 5) use the flags to enforce receipt of data only when necessary
+
+We should be able to end up with sequential receipt in H2 modelling what is
+needed for other protocols without interfering with the native H1 devs.
+
+
+2018-08-17 - Considerations after killing cs_recv()
+---------------------------------------------------
+
+With the ongoing reorganisation of the I/O layers, it's visible that cs_recv()
+will have to transfer data between the cs' rxbuf and the channel's buffer while
+not being aware of the data format. Moreover, in case there's no data there, it
+needs to recursively call the mux's rcv_buf() to trigger a decoding, while this
+function is sometimes replaced with cs_recv(). All this shows that cs_recv() is
+in fact needed while data are pushed upstream from the lower layers, and is not
+suitable for the "pull" mode. Thus it was decided to remove this function and
+put its code back into h2_rcv_buf(). The H1 mux's rcv_buf() already couldn't be
+replaced with cs_recv() since it is the only one knowing about the buffer's
+format.
+
+This opportunity simplified something : if the cs's rxbuf is only read by the
+mux's rcv_buf() method, then it doesn't need to be located into the CS and is
+well placed into the mux's representation of the stream. This has an important
+impact for H2 as it offers more freedom to the mux to allocate/free/reallocate
+this buffer, and it ensures the mux always has access to it.
+
+Furthermore, the conn_stream's txbuf experienced the same fate. Indeed, the H1
+mux has already uncovered the difficulty related to the channel shutting down
+on output, with data stuck into the CS's txbuf. Since the CS is tightly coupled
+to the stream and the stream can close immediately once its buffers are empty,
+it required a way to support orphaned CS with pending data in their txbuf. This
+is something that the H2 mux already has to deal with, by carefully leaving the
+data in the channel's buffer. But due to the snd_buf() call being top-down, it
+is always possible to push the stream's data via the mux's snd_buf() call
+without requiring a CS txbuf anymore. Thus the txbuf (when needed) is only
+implemented in the mux and attached to the mux's representation of the stream,
+and doing so allows to immediately release the channel once the data are safe
+in the mux's buffer.
+
+This is an important change which clarifies the roles and responsibilities of
+each layer in the chain : when receiving data from a mux, it's the mux's
+responsibility to make sure it can correctly decode the incoming data and to
+buffer the possible excess of data it cannot pass to the requester. This means
+that decoding an H2 frame, which is not retryable since it has an impact on the
+HPACK decompression context, and which cannot be reordered for the same reason,
+simply needs to be performed to the H2 stream's rxbuf which will then be passed
+to the stream when this one calls h2_rcv_buf(), even if it reads one byte at a
+time. Similarly when calling h2_snd_buf(), it's the mux's responsibility to
+read as much as it needs to be able to restart later, possibly by buffering
+some data into a local buffer. And it's only once all the output data has been
+consumed by snd_buf() that the stream is free to disappear.
+
+This model presents the nice benefit of being infinitely stackable and solving
+the last identified showstoppers to move towards a structured message internal
+representation, as it will give full power to the rcv_buf() and snd_buf() to
+process what they need.
+
+For now the conn_stream's flags indicating whether a shutdown has been seen in
+any direction or if an end of stream was seen will remain in the conn_stream,
+though it's likely that some of them will move to the mux's representation of
+the stream after structured messages are implemented.
diff --git a/doc/internals/notes-poll-connect.txt b/doc/internals/notes-poll-connect.txt
new file mode 100644
index 0000000..5cb0885
--- /dev/null
+++ b/doc/internals/notes-poll-connect.txt
@@ -0,0 +1,93 @@
+2022-11-17 - Tests involving poll() return states upon a pending connect().
+
+- connect() to a closed port returns OUT and HUP:
+
+ $ dev/poll/poll -v -l clo -c pol
+ #### BEGIN ####
+ cmd #1 stp #1: clo(l=3): ret=0
+ cmd #2 stp #0: con(c=4): ret=-1 (Connection refused)
+ cmd #2 stp #1: pol(c=4): ret=1 ev=0x14 (OUT HUP)
+ #### END ####
+
+=> with HUP we *know* the connection failed, since we never asked for a
+ SHUTW before connecting. It is indeed an error as can be seen with
+ connect() returning -1 ECONNREFUSED.
+
+- connect() to a port that does close(accept()) does return IN and RDHUP:
+
+ $ dev/poll/poll -v -s clo -c pol
+ #### BEGIN ####
+ cmd #1 stp #0: con(c=4): ret=0
+ cmd #1 stp #0: acc(l=3): ret=5
+ cmd #1 stp #1: clo(s=5): ret=0
+ cmd #2 stp #1: pol(c=4): ret=1 ev=0x2005 (IN OUT RDHUP)
+ #### END ####
+
+=> here there's no HUP, only RDHUP because the FIN is pending in the
+ socket buffers, waiting to be read.
+
+- for a HUP to happen after a connect() to a valid port, one would have to
+ perform a shutw() on the client, which is normally not the case, indicating
+ that HUP is reliable here:
+
+ $ dev/poll/poll -v -s clo -c shw,pol
+ #### BEGIN ####
+ cmd #1 stp #0: con(c=4): ret=0
+ cmd #1 stp #0: acc(l=3): ret=5
+ cmd #1 stp #1: clo(s=5): ret=0
+ cmd #2 stp #1: shw(c=4): ret=0
+ cmd #2 stp #2: pol(c=4): ret=1 ev=0x2015 (IN OUT HUP RDHUP)
+ #### END ####
+
+- one case that may happen is when sending a request and immediately shutw()
+ (which leaves a TIME_WAIT so not recommended):
+
+ $ dev/poll/poll -v -c snd,shw -s clo -c pol,rcv,pol
+ #### BEGIN ####
+ cmd #1 stp #0: con(c=4): ret=0
+ cmd #1 stp #1: snd(c=4): ret=3
+ cmd #1 stp #2: shw(c=4): ret=0
+ cmd #2 stp #0: acc(l=3): ret=5
+ cmd #2 stp #1: clo(s=5): ret=0
+ cmd #3 stp #1: pol(c=4): ret=1 ev=0x201d (IN OUT ERR HUP RDHUP)
+ cmd #3 stp #2: rcv(c=4): ret=-1 (Connection reset by peer)
+ cmd #3 stp #3: pol(c=4): ret=1 ev=0x2015 (IN OUT HUP RDHUP)
+ #### END ####
+
+=> here it's impossible to know from the client whether the server consumed the
+ data or not, which is normal since a close on the server causes an RST to be
+ emitted for the data in flight, hence the ERR here. It's also worth noting
+ that once POLL_ERR is consumed by recv() it disappears.
+
+- for the server, sending a shutw() before closing here delivers an ACK in time
+ that prevents the RST from being sent, thus connect() is not notified (but if
+ the server has too much to send, it will truncate and emit an RST):
+
+ $ dev/poll/poll -v -c snd,shw -s shw,clo -c pol,rcv,pol
+ #### BEGIN ####
+ cmd #1 stp #0: con(c=4): ret=0
+ cmd #1 stp #1: snd(c=4): ret=3
+ cmd #1 stp #2: shw(c=4): ret=0
+ cmd #2 stp #0: acc(l=3): ret=5
+ cmd #2 stp #1: shw(s=5): ret=0
+ cmd #2 stp #2: clo(s=5): ret=0
+ cmd #3 stp #1: pol(c=4): ret=1 ev=0x2015 (IN OUT HUP RDHUP)
+ cmd #3 stp #2: rcv(c=4): ret=0
+ cmd #3 stp #3: pol(c=4): ret=1 ev=0x2015 (IN OUT HUP RDHUP)
+ #### END ####
+
+- if the server sends a response, disables lingering and closes with RST, it is
+ possible to get HUP and ERR at the same time during the connect() phase, and
+ recv() can still receive the pending response:
+
+ $ dev/poll/poll -v -s snd,lin,clo -c pol,rcv,pol
+ #### BEGIN ####
+ cmd #1 stp #0: con(c=4): ret=0
+ cmd #1 stp #0: acc(l=3): ret=5
+ cmd #1 stp #1: snd(s=5): ret=3
+ cmd #1 stp #2: lin(s=5): ret=0
+ cmd #1 stp #3: clo(s=5): ret=0
+ cmd #2 stp #1: pol(c=4): ret=1 ev=0x201d (IN OUT ERR HUP RDHUP)
+ cmd #2 stp #2: rcv(c=4): ret=3
+ cmd #2 stp #3: pol(c=4): ret=1 ev=0x201d (IN OUT ERR HUP RDHUP)
+ #### END ####
diff --git a/doc/internals/notes-pollhup.txt b/doc/internals/notes-pollhup.txt
new file mode 100644
index 0000000..ced332b
--- /dev/null
+++ b/doc/internals/notes-pollhup.txt
@@ -0,0 +1,281 @@
+tcp mode 8001->8008
+
+
+Remote test:
+============
+
+willy@up1:~$ echo bar | ncat -lp8008
+willy@wtap:haproxy$ echo foo | ncat 127.1 8001
+
+17:09:53.663154 epoll_wait(3, [{EPOLLIN, {u32=5, u64=5}}], 200, 1000) = 1
+17:09:54.582146 accept4(5, {sa_family=AF_INET, sin_port=htons(33378), sin_addr=inet_addr("127.0.0.1")}, [128->16], SOCK_NONBLOCK) = 8
+17:09:54.582299 setsockopt(8, SOL_TCP, TCP_NODELAY, [1], 4) = 0
+17:09:54.582527 accept4(5, 0x7ffc4a8bf330, [128], SOCK_NONBLOCK) = -1 EAGAIN (Resource temporarily unavailable)
+17:09:54.582655 recvfrom(8, "foo\n", 15360, 0, NULL, NULL) = 4
+17:09:54.582727 recvfrom(8, "", 15356, 0, NULL, NULL) = 0
+17:09:54.582827 socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 9
+17:09:54.582878 setsockopt(9, SOL_TCP, TCP_NODELAY, [1], 4) = 0
+17:09:54.582897 connect(9, {sa_family=AF_INET, sin_port=htons(8008), sin_addr=inet_addr("10.0.3.82")}, 16) = -1 EINPROGRESS (Operation now in progress)
+17:09:54.582941 sendto(9, "foo\n", 4, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = -1 EAGAIN (Resource temporarily unavailable)
+17:09:54.582968 epoll_ctl(3, EPOLL_CTL_ADD, 9, {EPOLLOUT, {u32=9, u64=9}}) = 0
+17:09:54.582997 epoll_wait(3, [{EPOLLOUT, {u32=9, u64=9}}], 200, 1000) = 1
+17:09:54.583686 connect(9, {sa_family=AF_INET, sin_port=htons(8008), sin_addr=inet_addr("10.0.3.82")}, 16) = 0
+17:09:54.583706 sendto(9, "foo\n", 4, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 4
+17:09:54.583733 recvfrom(9, 0x19c2300, 15360, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
+17:09:54.583755 shutdown(9, SHUT_WR) = 0
+17:09:54.583775 epoll_ctl(3, EPOLL_CTL_MOD, 9, {EPOLLIN|EPOLLRDHUP, {u32=9, u64=9}}) = 0
+17:09:54.583802 epoll_wait(3, [{EPOLLIN, {u32=9, u64=9}}], 200, 1000) = 1
+17:09:54.584672 recvfrom(9, "bar\n", 16384, 0, NULL, NULL) = 4
+17:09:54.584713 recvfrom(9, "", 16380, 0, NULL, NULL) = 0
+17:09:54.584743 sendto(8, "bar\n", 4, MSG_DONTWAIT|MSG_NOSIGNAL|MSG_MORE, NULL, 0) = 4
+17:09:54.584819 epoll_wait(3, [], 200, 0) = 0
+17:09:54.584901 epoll_wait(3, [], 200, 1000) = 0
+
+
+Notes:
+ - we had data available to try the connect() (see first attempt), despite
+ this during the retry we sent the connect again!
+
+ - why do we wait before sending the shutw to the server if we already know
+ it's needed ? missing CF_SHUTW_NOW ? Missing request forwarding ? Missing
+ auto-close ?
+
+ - response didn't feature HUP nor RDHUP
+
+
+Local:
+
+17:15:43.010786 accept4(5, {sa_family=AF_INET, sin_port=htons(33506), sin_addr=inet_addr("127.0.0.1")}, [128->16], SOCK_NONBLOCK) = 8
+17:15:43.011013 setsockopt(8, SOL_TCP, TCP_NODELAY, [1], 4) = 0
+17:15:43.011181 accept4(5, 0x7ffcd9092cd0, [128], SOCK_NONBLOCK) = -1 EAGAIN (Resource temporarily unavailable)
+17:15:43.011231 recvfrom(8, "foo\n", 15360, 0, NULL, NULL) = 4
+17:15:43.011296 recvfrom(8, "", 15356, 0, NULL, NULL) = 0
+17:15:43.011318 socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 9
+17:15:43.011340 setsockopt(9, SOL_TCP, TCP_NODELAY, [1], 4) = 0
+17:15:43.011353 connect(9, {sa_family=AF_INET, sin_port=htons(8008), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EINPROGRESS (Operation now in progress)
+17:15:43.011395 sendto(9, "foo\n", 4, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 4
+17:15:43.011425 shutdown(9, SHUT_WR) = 0
+17:15:43.011459 recvfrom(9, "bar\n", 16384, 0, NULL, NULL) = 4
+17:15:43.011491 recvfrom(9, "", 16380, 0, NULL, NULL) = 0
+17:15:43.011525 sendto(8, "bar\n", 4, MSG_DONTWAIT|MSG_NOSIGNAL|MSG_MORE, NULL, 0) = 4
+17:15:43.011584 epoll_wait(3, [], 200, 0) = 0
+
+Notes:
+ - the shutdown() was properly done right after the sendto(), proving that
+ CF_SHUTW_NOW and auto-close were present. Maybe difference is sync vs async
+ send.
+
+
+Local with delay before closing client:
+
+17:18:17.155349 epoll_wait(3, [{EPOLLIN, {u32=5, u64=5}}], 200, 1000) = 1
+17:18:17.727327 accept4(5, {sa_family=AF_INET, sin_port=htons(33568), sin_addr=inet_addr("127.0.0.1")}, [128->16], SOCK_NONBLOCK) = 8
+17:18:17.727553 setsockopt(8, SOL_TCP, TCP_NODELAY, [1], 4) = 0
+17:18:17.727661 accept4(5, 0x7fff4eb9a0b0, [128], SOCK_NONBLOCK) = -1 EAGAIN (Resource temporarily unavailable)
+17:18:17.727798 recvfrom(8, 0xbda300, 15360, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
+17:18:17.727830 socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 9
+17:18:17.727858 setsockopt(9, SOL_TCP, TCP_NODELAY, [1], 4) = 0
+17:18:17.727877 connect(9, {sa_family=AF_INET, sin_port=htons(8008), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EINPROGRESS (Operation now in progress)
+17:18:17.727923 epoll_ctl(3, EPOLL_CTL_ADD, 8, {EPOLLIN|EPOLLRDHUP, {u32=8, u64=8}}) = 0
+17:18:17.727945 epoll_ctl(3, EPOLL_CTL_ADD, 9, {EPOLLOUT, {u32=9, u64=9}}) = 0
+17:18:17.727989 epoll_wait(3, [{EPOLLOUT, {u32=9, u64=9}}], 200, 1000) = 1
+17:18:17.728010 connect(9, {sa_family=AF_INET, sin_port=htons(8008), sin_addr=inet_addr("127.0.0.1")}, 16) = 0
+17:18:17.728027 recvfrom(9, "bar\n", 15360, 0, NULL, NULL) = 4
+17:18:17.728055 recvfrom(9, 0xbd62f4, 15356, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
+17:18:17.728073 sendto(8, "bar\n", 4, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 4
+17:18:17.728104 epoll_ctl(3, EPOLL_CTL_MOD, 9, {EPOLLIN|EPOLLRDHUP, {u32=9, u64=9}}) = 0
+17:18:17.728127 epoll_wait(3, [], 200, 1000) = 0
+17:18:18.729411 epoll_wait(3, [], 200, 1000) = 0
+17:18:19.730654 epoll_wait(3, [{EPOLLIN|EPOLLRDHUP, {u32=8, u64=8}}], 200, 1000) = 1
+17:18:20.299268 recvfrom(8, "", 16384, 0, NULL, NULL) = 0
+17:18:20.299336 epoll_ctl(3, EPOLL_CTL_DEL, 8, 0x7ff3a969f7d0) = 0
+17:18:20.299379 epoll_wait(3, [], 200, 0) = 0
+17:18:20.299401 shutdown(9, SHUT_WR) = 0
+17:18:20.299523 epoll_wait(3, [{EPOLLIN|EPOLLHUP|EPOLLRDHUP, {u32=9, u64=9}}], 200, 1000) = 1
+17:18:20.299678 recvfrom(9, "", 16384, 0, NULL, NULL) = 0
+17:18:20.299761 epoll_wait(3, [], 200, 0) = 0
+
+Notes: server sent the response in two parts ("bar" then EOF) just due to
+netcat's implementation. The second epoll_wait() caught it.
+
+Here we clearly see that :
+ - read0 alone returns EPOLLIN|EPOLLRDHUP
+ - read0 after shutw returns EPOLLIN|EPOLLRDHUP|EPOLLHUP
+ => difference indeed is "cannot write"
+
+
+Local with a delay before closing the server:
+
+17:30:32.527157 epoll_wait(3, [{EPOLLIN, {u32=5, u64=5}}], 200, 1000) = 1
+17:30:33.216827 accept4(5, {sa_family=AF_INET, sin_port=htons(33908), sin_addr=inet_addr("127.0.0.1")}, [128->16], SOCK_NONBLOCK) = 8
+17:30:33.216957 setsockopt(8, SOL_TCP, TCP_NODELAY, [1], 4) = 0
+17:30:33.216984 accept4(5, 0x7ffc1a1fb0c0, [128], SOCK_NONBLOCK) = -1 EAGAIN (Resource temporarily unavailable)
+17:30:33.217071 recvfrom(8, "GET / HTTP/1.0\r\n\r\n\n", 15360, 0, NULL, NULL) = 19
+17:30:33.217115 recvfrom(8, "", 15341, 0, NULL, NULL) = 0
+17:30:33.217135 socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 9
+17:30:33.217176 setsockopt(9, SOL_TCP, TCP_NODELAY, [1], 4) = 0
+17:30:33.217190 connect(9, {sa_family=AF_INET, sin_port=htons(8000), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EINPROGRESS (Operation now in progress)
+17:30:33.217233 sendto(9, "GET / HTTP/1.0\r\n\r\n\n", 19, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 19
+17:30:33.217272 shutdown(9, SHUT_WR) = 0
+17:30:33.217318 recvfrom(9, 0x109b2f0, 16384, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
+17:30:33.217332 epoll_ctl(3, EPOLL_CTL_ADD, 9, {EPOLLIN|EPOLLRDHUP, {u32=9, u64=9}}) = 0
+17:30:33.217355 epoll_wait(3, [{EPOLLIN|EPOLLHUP|EPOLLRDHUP, {u32=9, u64=9}}], 200, 1000) = 1
+17:30:33.217377 recvfrom(9, "HTTP/1.0 200\r\nContent-length: 0\r\nX-req: size=19, time=0 ms\r\nX-rsp: id=dummy, code=200, cache=1, size=0, time=0 ms (0 real)\r\n\r\n", 16384, 0, NULL, NULL) = 126
+17:30:33.217395 close(9) = 0
+17:30:33.217411 sendto(8, "HTTP/1.0 200\r\nContent-length: 0\r\nX-req: size=19, time=0 ms\r\nX-rsp: id=dummy, code=200, cache=1, size=0, time=0 ms (0 real)\r\n\r\n", 126, MSG_DONTWAIT|MSG_NOSIGNAL|MSG_MORE, NULL, 0) = 126
+17:30:33.217464 close(8) = 0
+17:30:33.217496 epoll_wait(3, [], 200, 0) = 0
+
+
+Notes:
+ - RDHUP is properly present while some data remain pending.
+ - HUP is present since RDHUP + shutw
+
+It could be concluded that HUP indicates RDHUP+shutw and in no way indicates
+the ability to avoid reading.
+
+Below HUP|ERR|OUT are reported on connection failures, thus WITHOUT read:
+
+accept4(5, {sa_family=AF_INET, sin_port=htons(39080), sin_addr=inet_addr("127.0.0.1")}, [128->16], SOCK_NONBLOCK) = 8
+setsockopt(8, SOL_TCP, TCP_NODELAY, [1], 4) = 0
+accept4(5, 0x7ffffba55730, [128], SOCK_NONBLOCK) = -1 EAGAIN (Resource temporarily unavailable)
+recvfrom(8, "foo\n", 15360, 0, NULL, NULL) = 4
+recvfrom(8, 0x7f634dcfeff4, 15356, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
+socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 9
+fcntl(9, F_SETFL, O_RDONLY|O_NONBLOCK) = 0
+setsockopt(9, SOL_TCP, TCP_NODELAY, [1], 4) = 0
+connect(9, {sa_family=AF_INET, sin_port=htons(8008), sin_addr=inet_addr("10.0.3.82")}, 16) = -1 EINPROGRESS (Operation now in progress)
+sendto(9, "foo\n", 4, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = -1 EAGAIN (Resource temporarily unavailable)
+epoll_ctl(3, EPOLL_CTL_ADD, 8, {EPOLLIN|EPOLLRDHUP, {u32=8, u64=8}}) = 0
+epoll_ctl(3, EPOLL_CTL_ADD, 9, {EPOLLOUT, {u32=9, u64=9}}) = 0
+epoll_wait(3, [{EPOLLOUT|EPOLLERR|EPOLLHUP, {u32=9, u64=9}}], 200, 1000) = 1
+getsockopt(9, SOL_SOCKET, SO_ERROR, [111], [4]) = 0
+recvfrom(9, "", 15360, 0, NULL, NULL) = 0
+close(9) = 0
+
+
+On a failed connect attempt immediately followed by a failed recv (all flags
+set), we can see this:
+
+socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 8
+fcntl(8, F_SETFL, O_RDONLY|O_NONBLOCK) = 0
+setsockopt(8, SOL_TCP, TCP_NODELAY, [1], 4) = 0
+connect(8, {sa_family=AF_INET, sin_port=htons(8008), sin_addr=inet_addr("10.0.3.82")}, 16) = -1 EINPROGRESS (Operation now in progress)
+recvfrom(8, 0x1084a20, 16384, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
+epoll_ctl(3, EPOLL_CTL_ADD, 8, {EPOLLIN|EPOLLOUT|EPOLLRDHUP, {u32=8, u64=8}}) = 0
+epoll_wait(3, [{EPOLLIN|EPOLLOUT|EPOLLERR|EPOLLHUP|EPOLLRDHUP, {u32=8, u64=8}}], 200, 1000) = 1
+connect(8, {sa_family=AF_INET, sin_port=htons(8008), sin_addr=inet_addr("10.0.3.82")}, 16) = -1 ECONNREFUSED (Connection refused)
+close(8) = 0
+
+=> all flags are reported in case of error.
+
+It's also interesting to note that POLLOUT is still reported after a shutw,
+and no send error is ever reported after shutw:
+
+ shutdown(4, SHUT_WR) = 0
+ poll([{fd=4, events=POLLIN|POLLOUT|POLLRDHUP}], 1, 0) = 1 ([{fd=4, revents=POLLOUT}])
+
+and:
+ shutdown(4, SHUT_WR) = 0
+ sendto(5, "foo", 3, MSG_NOSIGNAL, NULL, 0) = 3
+ poll([{fd=4, events=POLLIN|POLLOUT|POLLRDHUP}], 1, 0) = 1 ([{fd=4, revents=POLLIN|POLLOUT}])
+
+and:
+ shutdown(4, SHUT_WR) = 0
+ sendto(4, "bar", 3, MSG_NOSIGNAL, NULL, 0) = -1 EPIPE (Broken pipe)
+ poll([{fd=4, events=POLLIN|POLLOUT|POLLRDHUP}], 1, 0) = 1 ([{fd=4, revents=POLLOUT}])
+
+
+POLLOUT is still reported after a SHUTWR:
+
+socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
+setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
+bind(3, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("0.0.0.0")}, 16) = 0
+listen(3, 1000) = 0
+getsockname(3, {sa_family=AF_INET, sin_port=htons(34729), sin_addr=inet_addr("0.0.0.0")}, [16]) = 0
+socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 4
+connect(4, {sa_family=AF_INET, sin_port=htons(34729), sin_addr=inet_addr("0.0.0.0")}, 16) = 0
+accept(3, 0x7ffcd6a68300, [0->16]) = 5
+fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 3), ...}) = 0
+brk(NULL) = 0xc4e000
+brk(0xc6f000) = 0xc6f000
+write(1, "\n", 1
+) = 1
+shutdown(4, SHUT_WR) = 0
+poll([{fd=4, events=POLLIN|POLLOUT|POLLRDHUP}], 1, 0) = 1 ([{fd=4, revents=POLLOUT}])
+write(1, "ret=1 ev={fd:4 ev:4}\n", 21ret=1 ev={fd:4 ev:4}
+) = 21
+close(5) = 0
+close(4) = 0
+close(3) = 0
+
+Performing a write() on it reports a SIGPIPE:
+
+shutdown(4, SHUT_WR) = 0
+sendto(4, "bar", 3, MSG_NOSIGNAL, NULL, 0) = -1 EPIPE (Broken pipe)
+poll([{fd=4, events=POLLIN|POLLOUT|POLLRDHUP}], 1, 0) = 1 ([{fd=4, revents=POLLOUT}])
+
+
+On SHUT_RD we see POLLIN|POLLOUT|POLLRDHUP (there's no data pending here) :
+shutdown(4, SHUT_RD) = 0
+poll([{fd=4, events=POLLIN|POLLOUT|POLLRDHUP}], 1, 0) = 1 ([{fd=4, revents=POLLIN|POLLOUT|POLLRDHUP}])
+
+
+What is observed in the end :
+ - POLLOUT is always reported for anything SHUT_WR even if it would cause a broken pipe, including listeners if they're also SHUT_RD
+ - POLLHUP is always reported for anything SHUT_WR + having a SHUT_RD pending with or without anything to read, including listeners
+ - POLLIN is always reported for anything to read or a pending zero
+ - POLLIN is NOT reported for SHUT_RD listeners, even with pending connections, only OUT+HUP are reported
+ - POLLIN and POLLRDHUP are always reported after a SHUTR
+ - POLLERR also enables IN,OUT,HUP,RHUP
+
+
+
+
+
+
+
+Currently there's a bit of an issue with connect() being too impatient to read:
+
+16:26:06.818521 connect(9, {sa_family=AF_INET, sin_port=htons(8000), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EINPROGRESS (Operation now in progress)
+16:26:06.818558 recvfrom(9, 0x1db9400, 16320, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
+16:26:06.818571 epoll_ctl(3, EPOLL_CTL_ADD, 9, {EPOLLIN|EPOLLOUT|EPOLLRDHUP, {u32=9, u64=9}}) = 0
+16:26:06.818588 epoll_wait(3, [{EPOLLOUT, {u32=9, u64=9}}], 200, 1000) = 1
+16:26:06.818603 connect(9, {sa_family=AF_INET, sin_port=htons(8000), sin_addr=inet_addr("127.0.0.1")}, 16) = 0
+16:26:06.818617 sendto(9, "GET /?s=10k HTTP/1.1\r\nhost: 127.0.0.1:4445\r\nuser-agent: curl/7.54.1\r\naccept: */*\r\n\r\n", 84, MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 84
+16:26:06.818660 epoll_ctl(3, EPOLL_CTL_MOD, 9, {EPOLLIN|EPOLLRDHUP, {u32=9, u64=9}}) = 0
+16:26:06.818696 epoll_wait(3, [{EPOLLIN, {u32=9, u64=9}}], 200, 1000) = 1
+16:26:06.818711 recvfrom(9, "HTTP/1.1 200\r\nContent-length: 10240\r\nX-req: size=84, time=0 ms\r\nX-rsp: id=dummy, code=200, cache=1, size=10240, time=0 ms (0 real)\r\n\r\n89.123456789.12345678\n.123456789.123456789.123456789.123456789.123"..., 16320, 0, NULL, NULL) = 10374
+16:26:06.818735 recvfrom(9, 0x1dd75f6, 5946, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)
+16:26:06.818790 epoll_ctl(3, EPOLL_CTL_DEL, 9, 0x7ffa818fd7d0) = 0
+16:26:06.818804 epoll_wait(3, [], 200, 0) = 0
+
+
+
+
+This one shows that the error is not definitive, it disappears once it's
+been signaled, then only shut remains! Also it's a proof that an error
+may well be reported after a shutw, so the r/w error may not be merged
+with a shutw since it may appear after an deliberate shutw.
+
+$ ./contrib/debug/poll -v -c snd,shw -s pol,rcv,pol,rcv,pol,snd,lin,clo -c pol,rcv,pol,rcv,pol,rcv,pol
+#### BEGIN ####
+cmd #1 stp #1: do_snd(4): ret=3
+cmd #1 stp #2: do_shw(4): ret=0
+cmd #2 stp #0: do_acc(3): ret=5
+cmd #2 stp #1: do_pol(5): ret=1 ev=0x2005 (IN OUT RDHUP)
+cmd #2 stp #2: do_rcv(5): ret=3
+cmd #2 stp #3: do_pol(5): ret=1 ev=0x2005 (IN OUT RDHUP)
+cmd #2 stp #4: do_rcv(5): ret=0
+cmd #2 stp #5: do_pol(5): ret=1 ev=0x2005 (IN OUT RDHUP)
+cmd #2 stp #6: do_snd(5): ret=3
+cmd #2 stp #7: do_lin(5): ret=0
+cmd #2 stp #8: do_clo(5): ret=0
+cmd #3 stp #1: do_pol(4): ret=1 ev=0x201d (IN OUT ERR HUP RDHUP)
+cmd #3 stp #2: do_rcv(4): ret=3
+cmd #3 stp #3: do_pol(4): ret=1 ev=0x201d (IN OUT ERR HUP RDHUP)
+cmd #3 stp #4: do_rcv(4): ret=-1 (Connection reset by peer)
+cmd #3 stp #5: do_pol(4): ret=1 ev=0x2015 (IN OUT HUP RDHUP)
+cmd #3 stp #6: do_rcv(4): ret=0
+cmd #3 stp #7: do_pol(4): ret=1 ev=0x2015 (IN OUT HUP RDHUP)
+#### END ####
diff --git a/doc/internals/notes-polling.txt b/doc/internals/notes-polling.txt
new file mode 100644
index 0000000..e7741a6
--- /dev/null
+++ b/doc/internals/notes-polling.txt
@@ -0,0 +1,192 @@
+2019-09-03
+
+u8 fd.state;
+u8 fd.ev;
+
+
+ev = one of :
+ #define FD_POLL_IN 0x01
+ #define FD_POLL_PRI 0x02
+ #define FD_POLL_OUT 0x04
+ #define FD_POLL_ERR 0x08
+ #define FD_POLL_HUP 0x10
+
+Could we instead have :
+
+ FD_WAIT_IN 0x01
+ FD_WAIT_OUT 0x02
+ FD_WAIT_PRI 0x04
+ FD_SEEN_HUP 0x08
+ FD_SEEN_HUP 0x10
+ FD_WAIT_CON 0x20 <<= shouldn't this be in the connection itself in fact ?
+
+=> not needed, covered by the state instead.
+
+What is missing though is :
+ - FD_DATA_PENDING -- overlaps with READY_R, OK if passed by pollers only
+ - FD_EOI_PENDING
+ - FD_ERR_PENDING
+ - FD_EOI
+ - FD_SHW
+ - FD_ERR
+
+fd_update_events() could do that :
+
+ if ((fd_data_pending|fd_eoi_pending|fd_err_pending) && !(fd_err|fd_eoi))
+ may_recv()
+
+ if (fd_send_ok && !(fd_err|fd_shw))
+ may_send()
+
+ if (fd_err)
+ wake()
+
+the poller could do that :
+ HUP+OUT => always indicates a failed connect(), it should not lack ERR. Is this err_pending ?
+
+ ERR HUP OUT IN
+ 0 0 0 0 => nothing
+ 0 0 0 1 => FD_DATA_PENDING
+ 0 0 1 0 => FD_SEND_OK
+ 0 0 1 1 => FD_DATA_PENDING|FD_SEND_OK
+ 0 1 0 0 => FD_EOI (|FD_SHW)
+ 0 1 0 1 => FD_DATA_PENDING|FD_EOI_PENDING (|FD_SHW)
+ 0 1 1 0 => FD_EOI |FD_ERR (|FD_SHW)
+ 0 1 1 1 => FD_EOI_PENDING (|FD_ERR_PENDING) |FD_DATA_PENDING (|FD_SHW)
+ 1 X 0 0 => FD_ERR | FD_EOI (|FD_SHW)
+ 1 X X 1 => FD_ERR_PENDING | FD_EOI_PENDING | FD_DATA_PENDING (|FD_SHW)
+ 1 X 1 0 => FD_ERR | FD_EOI (|FD_SHW)
+
+ OUT+HUP,OUT+HUP+ERR => FD_ERR
+
+This reorders to:
+
+ IN ERR HUP OUT
+ 0 0 0 0 => nothing
+ 0 0 0 1 => FD_SEND_OK
+ 0 0 1 0 => FD_EOI (|FD_SHW)
+
+ 0 X 1 1 => FD_ERR | FD_EOI (|FD_SHW)
+ 0 1 X 0 => FD_ERR | FD_EOI (|FD_SHW)
+ 0 1 X 1 => FD_ERR | FD_EOI (|FD_SHW)
+
+ 1 0 0 0 => FD_DATA_PENDING
+ 1 0 0 1 => FD_DATA_PENDING|FD_SEND_OK
+ 1 0 1 0 => FD_DATA_PENDING|FD_EOI_PENDING (|FD_SHW)
+ 1 0 1 1 => FD_EOI_PENDING (|FD_ERR_PENDING) |FD_DATA_PENDING (|FD_SHW)
+ 1 1 X X => FD_ERR_PENDING | FD_EOI_PENDING | FD_DATA_PENDING (|FD_SHW)
+
+Regarding "|SHW", it's normally useless since it will already have been done,
+except on connect() error where this indicates there's no need for SHW.
+
+FD_EOI and FD_SHW could be part of the state (FD_EV_SHUT_R, FD_EV_SHUT_W).
+Then all states having these bit and another one would be transient and need
+to resync. We could then have "fd_shut_recv" and "fd_shut_send" to turn these
+states.
+
+The FD's ev then only needs to update EOI_PENDING, ERR_PENDING, ERR, DATA_PENDING.
+With this said, these are not exactly polling states either, as err/eoi/shw are
+orthogonal to the other states and are required to update them so that the polling
+state really is DISABLED in the end. So we need more of an operational status for
+the FD containing EOI_PENDING, EOI, ERR_PENDING, ERR, SHW, CLO?. These could be
+classified in 3 categories: read:(OPEN, EOI_PENDING, EOI); write:(OPEN,SHW),
+ctrl:(OPEN,ERR_PENDING,ERR,CLO). That would be 2 bits for R, 1 for W, 2 for ctrl
+or total 5 vs 6 for individual ones, but would be harder to manipulate.
+
+Proposal:
+ - rename fdtab[].state to "polling_state"
+ - rename fdtab[].ev to "status"
+
+Note: POLLHUP is also reported is a listen() socket has gone in shutdown()
+TEMPORARILY! Thus we may not always consider this as a final error.
+
+
+Work hypothesis:
+
+SHUT RDY ACT
+ 0 0 0 => disabled
+ 0 0 1 => active
+ 0 1 0 => stopped
+ 0 1 1 => ready
+ 1 0 0 => final shut
+ 1 0 1 => shut pending without data
+ 1 1 0 => shut pending, stopped
+ 1 1 1 => shut pending
+
+PB: we can land into final shut if one thread disables the FD while another
+ one that was waiting on it reports it as shut. Theorically it should be
+ implicitly ready though, since reported. But if no data is reported, it
+ will be reportedly shut only. And no event will be reported then. This
+ might still make sense since it's not active, thus we don't want events.
+ But it will not be enabled later either in this case so the shut really
+ risks not to be properly reported. The issue is that there's no difference
+ between a shut coming from the bottom and a shut coming from the top, and
+ we need an event to report activity here. Or we may consider that a poller
+ never leaves a final shut by itself (100) and always reports it as
+ shut+stop (thus ready) if it was not active. Alternately, if active is
+ disabled, shut should possibly be ignored, then a poller cannot report
+ shut. But shut+stopped seems the most suitable as it corresponds to
+ disabled->stopped transition.
+
+Now let's add ERR. ERR necessarily implies SHUT as there doesn't seem to be a
+valid case of ERR pending without shut pending.
+
+ERR SHUT RDY ACT
+ 0 0 0 0 => disabled
+ 0 0 0 1 => active
+ 0 0 1 0 => stopped
+ 0 0 1 1 => ready
+
+ 0 1 0 0 => final shut, no error
+ 0 1 0 1 => shut pending without data
+ 0 1 1 0 => shut pending, stopped
+ 0 1 1 1 => shut pending
+
+ 1 0 X X => invalid
+
+ 1 1 0 0 => final shut, error encountered
+ 1 1 0 1 => error pending without data
+ 1 1 1 0 => error pending after data, stopped
+ 1 1 1 1 => error pending
+
+So the algorithm for the poller is:
+ - if (shutdown_pending or error) reported and ACT==0,
+ report SHUT|RDY or SHUT|ERR|RDY
+
+For read handlers :
+ - if (!(flags & (RDY|ACT)))
+ return
+ - if (ready)
+ try_to_read
+ - if (err)
+ report error
+ - if (shut)
+ read0
+
+For write handlers:
+ - if (!(flags & (RDY|ACT)))
+ return
+ - if (err||shut)
+ report error
+ - if (ready)
+ try_to_write
+
+For listeners:
+ - if (!(flags & (RDY|ACT)))
+ return
+ - if (err||shut)
+ pause
+ - if (ready)
+ try_to_accept
+
+Kqueue reports events differently, it says EV_EOF() on READ or WRITE, that
+we currently map to FD_POLL_HUP and FD_POLL_ERR. Thus kqueue reports only
+POLLRDHUP and not POLLHUP, so for now a direct mapping of POLLHUP to
+FD_POLL_HUP does NOT imply write closed with kqueue while it does for others.
+
+Other approach, use the {RD,WR}_{ERR,SHUT,RDY} flags to build a composite
+status in each poller and pass this to fd_update_events(). We normally
+have enough to be precise, and this latter will rework the events.
+
+FIXME: Normally on KQUEUE we're supposed to look at kev[].fflags to get the error
+on EV_EOF() on read or write.
diff --git a/doc/internals/pattern.dia b/doc/internals/pattern.dia
new file mode 100644
index 0000000..3d13215
--- /dev/null
+++ b/doc/internals/pattern.dia
Binary files differ
diff --git a/doc/internals/pattern.pdf b/doc/internals/pattern.pdf
new file mode 100644
index 0000000..a8d8bc9
--- /dev/null
+++ b/doc/internals/pattern.pdf
Binary files differ
diff --git a/doc/internals/polling-states.fig b/doc/internals/polling-states.fig
new file mode 100644
index 0000000..3b2c782
--- /dev/null
+++ b/doc/internals/polling-states.fig
@@ -0,0 +1,59 @@
+#FIG 3.2 Produced by xfig version 2.3
+Portrait
+Center
+Metric
+A4
+100.00
+Single
+-2
+1200 2
+2 1 0 1 0 7 50 -1 -1 0.000 1 0 -1 1 0 2
+ 1 1 1.00 90.00 180.00
+ 1125 1350 1125 1800
+2 1 0 1 0 7 50 -1 -1 0.000 1 0 -1 1 0 2
+ 1 1 1.00 90.00 180.00
+ 1125 2250 1125 2700
+2 1 0 1 0 7 50 -1 -1 0.000 1 0 -1 1 0 2
+ 1 1 1.00 90.00 180.00
+ 1125 3150 1125 3600
+2 1 0 1 0 7 50 -1 -1 0.000 1 0 -1 1 0 2
+ 1 1 1.00 90.00 180.00
+ 1575 1800 1575 1350
+2 1 0 1 0 7 50 -1 -1 0.000 1 0 -1 1 0 2
+ 1 1 1.00 90.00 180.00
+ 1575 3600 1575 3150
+2 4 0 1 0 7 51 -1 20 0.000 0 0 7 0 0 5
+ 1800 1350 900 1350 900 900 1800 900 1800 1350
+2 4 0 1 0 7 51 -1 20 0.000 0 0 7 0 0 5
+ 1800 2250 900 2250 900 1800 1800 1800 1800 2250
+2 4 0 1 0 7 51 -1 20 0.000 0 0 7 0 0 5
+ 1800 4050 900 4050 900 3600 1800 3600 1800 4050
+2 4 0 1 0 7 51 -1 20 0.000 0 0 7 0 0 5
+ 1800 3150 900 3150 900 2700 1800 2700 1800 3150
+2 1 0 1 0 7 50 -1 -1 0.000 1 0 -1 1 0 2
+ 1 1 1.00 90.00 180.00
+ 1350 450 1350 900
+2 1 0 1 0 7 50 -1 -1 0.000 1 0 -1 1 0 2
+ 1 1 1.00 90.00 180.00
+ 1575 2700 1575 2250
+4 2 0 50 -1 16 8 0.0000 4 105 270 1080 1485 want\001
+4 2 0 50 -1 16 8 0.0000 4 120 255 1035 3285 stop\001
+4 0 0 50 -1 16 8 0.0000 4 105 270 1665 3510 want\001
+4 1 0 50 -1 16 10 0.0000 4 120 735 1350 1080 STOPPED\001
+4 1 0 50 -1 16 10 0.0000 4 120 795 1350 3780 DISABLED\001
+4 1 0 50 -1 16 10 0.0000 4 120 555 1350 2880 ACTIVE\001
+4 1 0 50 -1 16 10 0.0000 4 120 540 1350 1980 READY\001
+4 0 0 50 -1 16 8 0.0000 4 90 210 1665 2565 may\001
+4 2 0 50 -1 16 8 0.0000 4 105 240 1035 2430 cant\001
+4 1 0 50 -1 16 8 0.0000 4 120 240 1350 1260 R,!A\001
+4 1 0 50 -1 16 8 0.0000 4 120 210 1350 2160 R,A\001
+4 1 0 50 -1 16 8 0.0000 4 120 240 1350 3060 !R,A\001
+4 1 0 50 -1 16 8 0.0000 4 120 270 1350 3960 !R,!A\001
+4 0 0 50 -1 16 8 0.0000 4 120 255 1665 1710 stop\001
+4 0 0 50 -1 16 10 0.0000 4 150 855 2520 1125 R=ready flag\001
+4 0 0 50 -1 16 10 0.0000 4 150 885 2520 1290 A=active flag\001
+4 0 0 50 -1 16 10 0.0000 4 150 1365 2520 2475 fd_want sets A flag\001
+4 0 0 50 -1 16 10 0.0000 4 150 1440 2520 2640 fd_stop clears A flag\001
+4 0 0 50 -1 16 10 0.0000 4 150 1905 2520 3300 update() updates the poller.\001
+4 0 0 50 -1 16 10 0.0000 4 150 2190 2520 2970 fd_cant clears R flag (EAGAIN)\001
+4 0 0 50 -1 16 10 0.0000 4 150 2115 2520 3135 fd_rdy sets R flag (poll return)\001
diff --git a/doc/internals/sched.fig b/doc/internals/sched.fig
new file mode 100644
index 0000000..4134420
--- /dev/null
+++ b/doc/internals/sched.fig
@@ -0,0 +1,748 @@
+#FIG 3.2 Produced by xfig version 2.4
+Landscape
+Center
+Metric
+A4
+150.00
+Single
+-2
+1200 2
+0 32 #c5ebe1
+0 33 #86c8a2
+0 34 #ffebac
+0 35 #cbb366
+0 36 #c7b696
+0 37 #effbff
+0 38 #dfcba6
+0 39 #414141
+0 40 #aeaaae
+0 41 #595559
+0 42 #414141
+0 43 #868286
+0 44 #bec3be
+0 45 #868286
+0 46 #bec3be
+0 47 #dfe3df
+0 48 #8e8e8e
+0 49 #8e8e8e
+0 50 #414141
+0 51 #868286
+0 52 #bec3be
+0 53 #dfe3df
+0 54 #414141
+0 55 #868286
+0 56 #bec3be
+0 57 #dfe3df
+0 58 #868286
+0 59 #bec3be
+0 60 #dfe3df
+0 61 #c7b696
+0 62 #effbff
+0 63 #dfcba6
+0 64 #c7b696
+0 65 #effbff
+0 66 #dfcba6
+0 67 #aeaaae
+0 68 #595559
+0 69 #8e8e8e
+0 70 #414141
+0 71 #868286
+0 72 #bec3be
+0 73 #dfe3df
+0 74 #414141
+0 75 #868286
+0 76 #bec3be
+0 77 #dfe3df
+0 78 #868286
+0 79 #bec3be
+0 80 #dfe3df
+0 81 #414141
+0 82 #868286
+0 83 #bec3be
+0 84 #414141
+0 85 #bec3be
+0 86 #dfe3df
+0 87 #414141
+0 88 #868286
+0 89 #bec3be
+0 90 #8e8e8e
+0 91 #414141
+0 92 #868286
+0 93 #bec3be
+0 94 #dfe3df
+0 95 #414141
+0 96 #868286
+0 97 #bec3be
+0 98 #dfe3df
+0 99 #bebebe
+0 100 #515151
+0 101 #e7e3e7
+0 102 #000049
+0 103 #797979
+0 104 #303430
+0 105 #414541
+0 106 #414141
+0 107 #868286
+0 108 #bec3be
+0 109 #dfe3df
+0 110 #cfcfcf
+0 111 #cfcfcf
+0 112 #cfcfcf
+0 113 #cfcfcf
+0 114 #cfcfcf
+0 115 #cfcfcf
+0 116 #cfcfcf
+0 117 #cfcfcf
+0 118 #cfcfcf
+0 119 #cfcfcf
+0 120 #cfcfcf
+0 121 #cfcfcf
+0 122 #cfcfcf
+0 123 #cfcfcf
+0 124 #cfcfcf
+0 125 #cfcfcf
+0 126 #cfcfcf
+0 127 #cfcfcf
+0 128 #cfcfcf
+0 129 #cfcfcf
+0 130 #cfcfcf
+0 131 #cfcfcf
+0 132 #cfcfcf
+0 133 #cfcfcf
+0 134 #cfcfcf
+0 135 #cfcfcf
+0 136 #cfcfcf
+0 137 #cfcfcf
+0 138 #cfcfcf
+0 139 #cfcfcf
+0 140 #cfcfcf
+0 141 #cfcfcf
+0 142 #cfcfcf
+0 143 #cfcfcf
+0 144 #cfcfcf
+0 145 #cfcfcf
+0 146 #cfcfcf
+0 147 #cfcfcf
+0 148 #cfcfcf
+0 149 #cfcfcf
+0 150 #c7c3c7
+0 151 #868286
+0 152 #bec3be
+0 153 #dfe3df
+0 154 #8e8e8e
+0 155 #8e8e8e
+0 156 #494549
+0 157 #868686
+0 158 #c7c7c7
+0 159 #e7e7e7
+0 160 #f7f7f7
+0 161 #9e9e9e
+0 162 #717571
+0 163 #aeaaae
+0 164 #494549
+0 165 #aeaaae
+0 166 #595559
+0 167 #bec3be
+0 168 #dfe3df
+0 169 #494549
+0 170 #616561
+0 171 #494549
+0 172 #868286
+0 173 #bec3be
+0 174 #dfe3df
+0 175 #bec3be
+0 176 #dfe3df
+0 177 #c7b696
+0 178 #effbff
+0 179 #dfcba6
+0 180 #414141
+0 181 #868286
+0 182 #bec3be
+0 183 #dfe3df
+0 184 #8e8e8e
+0 185 #aeaaae
+0 186 #595559
+0 187 #414141
+0 188 #868286
+0 189 #bec3be
+0 190 #868286
+0 191 #bec3be
+0 192 #dfe3df
+0 193 #8e8e8e
+0 194 #8e8e8e
+0 195 #414141
+0 196 #868286
+0 197 #bec3be
+0 198 #dfe3df
+0 199 #414141
+0 200 #868286
+0 201 #bec3be
+0 202 #dfe3df
+0 203 #868286
+0 204 #bec3be
+0 205 #dfe3df
+0 206 #c7b696
+0 207 #effbff
+0 208 #dfcba6
+0 209 #c7b696
+0 210 #effbff
+0 211 #dfcba6
+0 212 #aeaaae
+0 213 #595559
+0 214 #8e8e8e
+0 215 #414141
+0 216 #868286
+0 217 #bec3be
+0 218 #dfe3df
+0 219 #414141
+0 220 #868286
+0 221 #bec3be
+0 222 #dfe3df
+0 223 #868286
+0 224 #bec3be
+0 225 #dfe3df
+0 226 #414141
+0 227 #868286
+0 228 #bec3be
+0 229 #414141
+0 230 #bec3be
+0 231 #dfe3df
+0 232 #414141
+0 233 #868286
+0 234 #bec3be
+0 235 #8e8e8e
+0 236 #414141
+0 237 #868286
+0 238 #bec3be
+0 239 #dfe3df
+0 240 #414141
+0 241 #868286
+0 242 #bec3be
+0 243 #dfe3df
+0 244 #414141
+0 245 #868286
+0 246 #bec3be
+0 247 #dfe3df
+0 248 #868286
+0 249 #bec3be
+0 250 #dfe3df
+0 251 #8e8e8e
+0 252 #8e8e8e
+0 253 #494549
+0 254 #aeaaae
+0 255 #494549
+0 256 #aeaaae
+0 257 #595559
+0 258 #bec3be
+0 259 #dfe3df
+0 260 #494549
+0 261 #616561
+0 262 #494549
+0 263 #868286
+0 264 #bec3be
+0 265 #dfe3df
+0 266 #bec3be
+0 267 #dfe3df
+0 268 #dfe3ef
+0 269 #96969e
+0 270 #d7dbd7
+0 271 #9ea2b6
+0 272 #9e0000
+0 273 #efefef
+0 274 #86aeff
+0 275 #7171ff
+0 276 #bbf2e2
+0 277 #a7ceb3
+0 278 #dae8fc
+0 279 #458dba
+0 280 #ffe6cc
+0 281 #e9b000
+0 282 #1a1a1a
+0 283 #ffc1e7
+0 284 #009ed7
+0 285 #006d9e
+0 286 #00719e
+0 287 #9e9a9e
+0 288 #000000
+0 289 #595959
+0 290 #006596
+0 291 #00a6d7
+0 292 #b6b6b6
+0 293 #8edbef
+0 294 #00699e
+0 295 #595d59
+0 296 #69d3e7
+0 297 #a6e3ef
+0 298 #9ec7d7
+0 299 #aeb2ae
+0 300 #00b6df
+0 301 #00aed7
+0 302 #797d79
+0 303 #00a2d7
+0 304 #303030
+0 305 #006996
+0 306 #086d9e
+0 307 #86b6cf
+0 308 #f7fbf7
+0 309 #9ec3d7
+0 310 #ffff96
+0 311 #ff600a
+5 1 0 2 0 7 50 -1 -1 0.000 0 0 1 0 11301.000 3060.000 11205 3825 10530 3060 11205 2295
+ 0 0 1.00 60.00 120.00
+5 1 0 2 0 7 50 -1 -1 0.000 0 1 0 1 11289.000 3060.000 11385 3825 12060 3060 11385 2295
+ 0 0 1.00 60.00 120.00
+5 1 0 2 0 7 50 -1 -1 0.000 0 1 0 1 11293.750 3060.000 10890 3105 11700 3060 10890 3015
+ 2 1 1.00 60.00 120.00
+5 1 0 2 0 7 50 -1 -1 0.000 0 0 1 0 7611.000 3060.000 7515 3825 6840 3060 7515 2295
+ 0 0 1.00 60.00 120.00
+5 1 0 2 0 7 50 -1 -1 0.000 0 1 0 1 7599.000 3060.000 7695 3825 8370 3060 7695 2295
+ 0 0 1.00 60.00 120.00
+5 1 0 2 0 7 50 -1 -1 0.000 0 1 0 1 7603.750 3060.000 7200 3105 8010 3060 7200 3015
+ 2 1 1.00 60.00 120.00
+5 1 0 2 0 7 50 -1 -1 0.000 0 0 1 0 4956.000 3060.000 4860 3825 4185 3060 4860 2295
+ 0 0 1.00 60.00 120.00
+5 1 0 2 0 7 50 -1 -1 0.000 0 1 0 1 4944.000 3060.000 5040 3825 5715 3060 5040 2295
+ 0 0 1.00 60.00 120.00
+5 1 0 2 0 7 50 -1 -1 0.000 0 1 0 1 4948.750 3060.000 4545 3105 5355 3060 4545 3015
+ 2 1 1.00 60.00 120.00
+5 1 0 2 0 7 50 -1 -1 0.000 0 0 1 0 1266.000 3060.000 1170 3825 495 3060 1170 2295
+ 0 0 1.00 60.00 120.00
+5 1 0 2 0 7 50 -1 -1 0.000 0 1 0 1 1254.000 3060.000 1350 3825 2025 3060 1350 2295
+ 0 0 1.00 60.00 120.00
+5 1 0 2 0 7 50 -1 -1 0.000 0 1 0 1 1258.750 3060.000 855 3105 1665 3060 855 3015
+ 2 1 1.00 60.00 120.00
+6 10606 2371 11985 3749
+2 3 0 2 0 31 50 -1 20 0.000 0 0 -1 0 0 15
+ 11768 3060 11970 3060 11967 3119 11959 3177 11946 3234 11929 3291
+ 11907 3345 11879 3397 11704 3296 11723 3259 11738 3222 11751 3182
+ 11760 3142 11765 3101 11768 3060
+2 3 0 2 0 31 50 -1 20 0.000 0 0 -1 0 0 15
+ 11704 3296 11879 3397 11848 3447 11812 3494 11772 3537 11729 3577
+ 11682 3613 11633 3644 11531 3469 11566 3447 11599 3422 11628 3393
+ 11657 3364 11682 3331 11704 3296
+2 3 0 2 0 31 50 -1 20 0.000 0 0 -1 0 0 15
+ 11531 3469 11633 3644 11580 3672 11526 3694 11469 3711 11412 3724
+ 11354 3732 11295 3734 11295 3532 11336 3530 11377 3525 11417 3516
+ 11457 3503 11494 3488 11531 3469
+2 3 0 2 0 31 50 -1 20 0.000 0 0 -1 0 0 15
+ 11295 3532 11295 3734 11236 3732 11178 3724 11121 3711 11064 3694
+ 11010 3672 10958 3644 11059 3469 11096 3488 11133 3503 11173 3516
+ 11213 3525 11254 3530 11295 3532
+2 3 0 2 0 31 50 -1 20 0.000 0 0 -1 0 0 15
+ 11059 3469 10958 3644 10908 3613 10861 3577 10818 3537 10778 3494
+ 10742 3447 10711 3398 10886 3296 10908 3331 10933 3364 10962 3393
+ 10991 3422 11024 3447 11059 3469
+2 3 0 2 0 31 50 -1 20 0.000 0 0 -1 0 0 15
+ 10886 3296 10711 3398 10683 3345 10661 3291 10644 3234 10631 3177
+ 10623 3119 10621 3060 10823 3060 10825 3101 10830 3142 10839 3182
+ 10852 3222 10867 3259 10886 3296
+2 3 0 2 0 31 50 -1 20 0.000 0 0 -1 0 0 15
+ 10823 3060 10621 3060 10623 3001 10631 2943 10644 2886 10661 2829
+ 10683 2775 10711 2723 10886 2824 10867 2861 10852 2898 10839 2938
+ 10830 2978 10825 3019 10823 3060
+2 3 0 2 0 31 50 -1 20 0.000 0 0 -1 0 0 15
+ 10886 2824 10711 2723 10742 2673 10778 2626 10818 2583 10861 2543
+ 10908 2507 10958 2476 11059 2651 11024 2673 10991 2698 10962 2727
+ 10933 2756 10908 2789 10886 2824
+2 3 0 2 0 31 50 -1 20 0.000 0 0 -1 0 0 15
+ 11059 2651 10958 2476 11010 2448 11064 2426 11121 2409 11178 2396
+ 11236 2388 11295 2386 11295 2588 11254 2590 11213 2595 11173 2604
+ 11133 2617 11096 2632 11059 2651
+2 3 0 2 0 31 50 -1 20 0.000 0 0 -1 0 0 15
+ 11295 2588 11295 2386 11354 2388 11412 2396 11469 2409 11526 2426
+ 11580 2448 11632 2476 11531 2651 11494 2632 11457 2617 11417 2604
+ 11377 2595 11336 2590 11295 2588
+2 3 0 2 0 31 50 -1 20 0.000 0 0 -1 0 0 15
+ 11531 2651 11632 2476 11682 2507 11729 2543 11772 2583 11812 2626
+ 11848 2673 11879 2723 11704 2824 11682 2789 11657 2756 11628 2727
+ 11599 2698 11566 2673 11531 2651
+2 3 0 2 0 31 50 -1 20 0.000 0 0 -1 0 0 15
+ 11704 2824 11879 2723 11907 2775 11929 2829 11946 2886 11959 2943
+ 11967 3001 11969 3060 11767 3060 11765 3019 11760 2978 11751 2938
+ 11738 2898 11723 2861 11704 2824
+-6
+6 4261 2371 5640 3749
+2 3 0 2 0 13 50 -1 20 0.000 0 0 -1 0 0 15
+ 5423 3060 5625 3060 5622 3119 5614 3177 5601 3234 5584 3291
+ 5562 3345 5534 3397 5359 3296 5378 3259 5393 3222 5406 3182
+ 5415 3142 5420 3101 5423 3060
+2 3 0 2 0 13 50 -1 20 0.000 0 0 -1 0 0 15
+ 5359 3296 5534 3397 5503 3447 5467 3494 5427 3537 5384 3577
+ 5337 3613 5288 3644 5186 3469 5221 3447 5254 3422 5283 3393
+ 5312 3364 5337 3331 5359 3296
+2 3 0 2 0 13 50 -1 20 0.000 0 0 -1 0 0 15
+ 5186 3469 5288 3644 5235 3672 5181 3694 5124 3711 5067 3724
+ 5009 3732 4950 3734 4950 3532 4991 3530 5032 3525 5072 3516
+ 5112 3503 5149 3488 5186 3469
+2 3 0 2 0 13 50 -1 20 0.000 0 0 -1 0 0 15
+ 4950 3532 4950 3734 4891 3732 4833 3724 4776 3711 4719 3694
+ 4665 3672 4613 3644 4714 3469 4751 3488 4788 3503 4828 3516
+ 4868 3525 4909 3530 4950 3532
+2 3 0 2 0 13 50 -1 20 0.000 0 0 -1 0 0 15
+ 4714 3469 4613 3644 4563 3613 4516 3577 4473 3537 4433 3494
+ 4397 3447 4366 3398 4541 3296 4563 3331 4588 3364 4617 3393
+ 4646 3422 4679 3447 4714 3469
+2 3 0 2 0 13 50 -1 20 0.000 0 0 -1 0 0 15
+ 4541 3296 4366 3398 4338 3345 4316 3291 4299 3234 4286 3177
+ 4278 3119 4276 3060 4478 3060 4480 3101 4485 3142 4494 3182
+ 4507 3222 4522 3259 4541 3296
+2 3 0 2 0 13 50 -1 20 0.000 0 0 -1 0 0 15
+ 4478 3060 4276 3060 4278 3001 4286 2943 4299 2886 4316 2829
+ 4338 2775 4366 2723 4541 2824 4522 2861 4507 2898 4494 2938
+ 4485 2978 4480 3019 4478 3060
+2 3 0 2 0 13 50 -1 20 0.000 0 0 -1 0 0 15
+ 4541 2824 4366 2723 4397 2673 4433 2626 4473 2583 4516 2543
+ 4563 2507 4613 2476 4714 2651 4679 2673 4646 2698 4617 2727
+ 4588 2756 4563 2789 4541 2824
+2 3 0 2 0 13 50 -1 20 0.000 0 0 -1 0 0 15
+ 4714 2651 4613 2476 4665 2448 4719 2426 4776 2409 4833 2396
+ 4891 2388 4950 2386 4950 2588 4909 2590 4868 2595 4828 2604
+ 4788 2617 4751 2632 4714 2651
+2 3 0 2 0 13 50 -1 20 0.000 0 0 -1 0 0 15
+ 4950 2588 4950 2386 5009 2388 5067 2396 5124 2409 5181 2426
+ 5235 2448 5287 2476 5186 2651 5149 2632 5112 2617 5072 2604
+ 5032 2595 4991 2590 4950 2588
+2 3 0 2 0 13 50 -1 20 0.000 0 0 -1 0 0 15
+ 5186 2651 5287 2476 5337 2507 5384 2543 5427 2583 5467 2626
+ 5503 2673 5534 2723 5359 2824 5337 2789 5312 2756 5283 2727
+ 5254 2698 5221 2673 5186 2651
+2 3 0 2 0 13 50 -1 20 0.000 0 0 -1 0 0 15
+ 5359 2824 5534 2723 5562 2775 5584 2829 5601 2886 5614 2943
+ 5622 3001 5624 3060 5422 3060 5420 3019 5415 2978 5406 2938
+ 5393 2898 5378 2861 5359 2824
+-6
+6 2250 4815 3960 5265
+1 1 0 3 8 11 52 -1 20 0.000 1 0.0000 3105 5049 810 171 3105 5049 3915 5049
+4 1 0 50 -1 6 10 0.0000 4 150 1125 3105 5130 Most Urgent\001
+-6
+6 6916 2371 8295 3749
+2 3 0 2 0 31 50 -1 43 0.000 0 0 -1 0 0 15
+ 8078 3060 8280 3060 8277 3119 8269 3177 8256 3234 8239 3291
+ 8217 3345 8189 3397 8014 3296 8033 3259 8048 3222 8061 3182
+ 8070 3142 8075 3101 8078 3060
+2 3 0 2 0 31 50 -1 43 0.000 0 0 -1 0 0 15
+ 8014 3296 8189 3397 8158 3447 8122 3494 8082 3537 8039 3577
+ 7992 3613 7943 3644 7841 3469 7876 3447 7909 3422 7938 3393
+ 7967 3364 7992 3331 8014 3296
+2 3 0 2 0 31 50 -1 43 0.000 0 0 -1 0 0 15
+ 7841 3469 7943 3644 7890 3672 7836 3694 7779 3711 7722 3724
+ 7664 3732 7605 3734 7605 3532 7646 3530 7687 3525 7727 3516
+ 7767 3503 7804 3488 7841 3469
+2 3 0 2 0 31 50 -1 43 0.000 0 0 -1 0 0 15
+ 7605 3532 7605 3734 7546 3732 7488 3724 7431 3711 7374 3694
+ 7320 3672 7268 3644 7369 3469 7406 3488 7443 3503 7483 3516
+ 7523 3525 7564 3530 7605 3532
+2 3 0 2 0 31 50 -1 43 0.000 0 0 -1 0 0 15
+ 7369 3469 7268 3644 7218 3613 7171 3577 7128 3537 7088 3494
+ 7052 3447 7021 3398 7196 3296 7218 3331 7243 3364 7272 3393
+ 7301 3422 7334 3447 7369 3469
+2 3 0 2 0 31 50 -1 43 0.000 0 0 -1 0 0 15
+ 7196 3296 7021 3398 6993 3345 6971 3291 6954 3234 6941 3177
+ 6933 3119 6931 3060 7133 3060 7135 3101 7140 3142 7149 3182
+ 7162 3222 7177 3259 7196 3296
+2 3 0 2 0 31 50 -1 43 0.000 0 0 -1 0 0 15
+ 7133 3060 6931 3060 6933 3001 6941 2943 6954 2886 6971 2829
+ 6993 2775 7021 2723 7196 2824 7177 2861 7162 2898 7149 2938
+ 7140 2978 7135 3019 7133 3060
+2 3 0 2 0 31 50 -1 43 0.000 0 0 -1 0 0 15
+ 7196 2824 7021 2723 7052 2673 7088 2626 7128 2583 7171 2543
+ 7218 2507 7268 2476 7369 2651 7334 2673 7301 2698 7272 2727
+ 7243 2756 7218 2789 7196 2824
+2 3 0 2 0 31 50 -1 43 0.000 0 0 -1 0 0 15
+ 7369 2651 7268 2476 7320 2448 7374 2426 7431 2409 7488 2396
+ 7546 2388 7605 2386 7605 2588 7564 2590 7523 2595 7483 2604
+ 7443 2617 7406 2632 7369 2651
+2 3 0 2 0 31 50 -1 43 0.000 0 0 -1 0 0 15
+ 7605 2588 7605 2386 7664 2388 7722 2396 7779 2409 7836 2426
+ 7890 2448 7942 2476 7841 2651 7804 2632 7767 2617 7727 2604
+ 7687 2595 7646 2590 7605 2588
+2 3 0 2 0 31 50 -1 43 0.000 0 0 -1 0 0 15
+ 7841 2651 7942 2476 7992 2507 8039 2543 8082 2583 8122 2626
+ 8158 2673 8189 2723 8014 2824 7992 2789 7967 2756 7938 2727
+ 7909 2698 7876 2673 7841 2651
+2 3 0 2 0 31 50 -1 43 0.000 0 0 -1 0 0 15
+ 8014 2824 8189 2723 8217 2775 8239 2829 8256 2886 8269 2943
+ 8277 3001 8279 3060 8077 3060 8075 3019 8070 2978 8061 2938
+ 8048 2898 8033 2861 8014 2824
+-6
+6 571 2371 1950 3749
+2 3 0 2 0 13 50 -1 43 0.000 0 0 -1 0 0 15
+ 1733 3060 1935 3060 1932 3119 1924 3177 1911 3234 1894 3291
+ 1872 3345 1844 3397 1669 3296 1688 3259 1703 3222 1716 3182
+ 1725 3142 1730 3101 1733 3060
+2 3 0 2 0 13 50 -1 43 0.000 0 0 -1 0 0 15
+ 1669 3296 1844 3397 1813 3447 1777 3494 1737 3537 1694 3577
+ 1647 3613 1598 3644 1496 3469 1531 3447 1564 3422 1593 3393
+ 1622 3364 1647 3331 1669 3296
+2 3 0 2 0 13 50 -1 43 0.000 0 0 -1 0 0 15
+ 1496 3469 1598 3644 1545 3672 1491 3694 1434 3711 1377 3724
+ 1319 3732 1260 3734 1260 3532 1301 3530 1342 3525 1382 3516
+ 1422 3503 1459 3488 1496 3469
+2 3 0 2 0 13 50 -1 43 0.000 0 0 -1 0 0 15
+ 1260 3532 1260 3734 1201 3732 1143 3724 1086 3711 1029 3694
+ 975 3672 923 3644 1024 3469 1061 3488 1098 3503 1138 3516
+ 1178 3525 1219 3530 1260 3532
+2 3 0 2 0 13 50 -1 43 0.000 0 0 -1 0 0 15
+ 1024 3469 923 3644 873 3613 826 3577 783 3537 743 3494
+ 707 3447 676 3398 851 3296 873 3331 898 3364 927 3393
+ 956 3422 989 3447 1024 3469
+2 3 0 2 0 13 50 -1 43 0.000 0 0 -1 0 0 15
+ 851 3296 676 3398 648 3345 626 3291 609 3234 596 3177
+ 588 3119 586 3060 788 3060 790 3101 795 3142 804 3182
+ 817 3222 832 3259 851 3296
+2 3 0 2 0 13 50 -1 43 0.000 0 0 -1 0 0 15
+ 788 3060 586 3060 588 3001 596 2943 609 2886 626 2829
+ 648 2775 676 2723 851 2824 832 2861 817 2898 804 2938
+ 795 2978 790 3019 788 3060
+2 3 0 2 0 13 50 -1 43 0.000 0 0 -1 0 0 15
+ 851 2824 676 2723 707 2673 743 2626 783 2583 826 2543
+ 873 2507 923 2476 1024 2651 989 2673 956 2698 927 2727
+ 898 2756 873 2789 851 2824
+2 3 0 2 0 13 50 -1 43 0.000 0 0 -1 0 0 15
+ 1024 2651 923 2476 975 2448 1029 2426 1086 2409 1143 2396
+ 1201 2388 1260 2386 1260 2588 1219 2590 1178 2595 1138 2604
+ 1098 2617 1061 2632 1024 2651
+2 3 0 2 0 13 50 -1 43 0.000 0 0 -1 0 0 15
+ 1260 2588 1260 2386 1319 2388 1377 2396 1434 2409 1491 2426
+ 1545 2448 1597 2476 1496 2651 1459 2632 1422 2617 1382 2604
+ 1342 2595 1301 2590 1260 2588
+2 3 0 2 0 13 50 -1 43 0.000 0 0 -1 0 0 15
+ 1496 2651 1597 2476 1647 2507 1694 2543 1737 2583 1777 2626
+ 1813 2673 1844 2723 1669 2824 1647 2789 1622 2756 1593 2727
+ 1564 2698 1531 2673 1496 2651
+2 3 0 2 0 13 50 -1 43 0.000 0 0 -1 0 0 15
+ 1669 2824 1844 2723 1872 2775 1894 2829 1911 2886 1924 2943
+ 1932 3001 1934 3060 1732 3060 1730 3019 1725 2978 1716 2938
+ 1703 2898 1688 2861 1669 2824
+-6
+6 1800 1845 2520 2385
+4 1 0 50 -1 6 10 0.0000 4 120 570 2160 1980 Global\001
+4 1 0 50 -1 6 10 0.0000 4 120 495 2160 2160 tasks\001
+4 1 0 50 -1 6 10 0.0000 4 150 705 2160 2340 (locked)\001
+-6
+6 3960 1935 4500 2250
+4 1 0 50 -1 6 10 0.0000 4 120 495 4230 2250 tasks\001
+4 1 0 50 -1 6 10 0.0000 4 120 465 4230 2070 Local\001
+-6
+6 8190 1845 8910 2385
+4 1 0 50 -1 6 10 0.0000 4 150 705 8550 2340 (locked)\001
+4 1 0 50 -1 6 10 0.0000 4 120 585 8550 2160 timers\001
+4 1 0 50 -1 6 10 0.0000 4 120 570 8550 1980 Global\001
+-6
+6 10215 1935 10845 2250
+4 1 0 50 -1 6 10 0.0000 4 120 585 10530 2250 timers\001
+4 1 0 50 -1 6 10 0.0000 4 120 465 10530 2070 Local\001
+-6
+6 2430 945 3735 1530
+1 1 0 3 20 29 52 -1 20 0.000 1 0.0000 3083 1180 607 170 3083 1180 3690 1350
+4 1 0 50 -1 6 10 0.0000 4 120 615 3105 1260 Local ?\001
+4 0 0 50 -1 6 9 0.0000 4 105 315 3375 1530 Yes\001
+4 2 0 50 -1 6 9 0.0000 4 105 225 2790 1530 No\001
+-6
+6 8775 945 10080 1530
+1 1 0 3 20 29 52 -1 20 0.000 1 0.0000 9428 1180 607 170 9428 1180 10035 1350
+4 1 0 50 -1 6 10 0.0000 4 120 615 9450 1260 Local ?\001
+4 0 0 50 -1 6 9 0.0000 4 105 315 9720 1530 Yes\001
+4 2 0 50 -1 6 9 0.0000 4 105 225 9135 1530 No\001
+-6
+6 7200 6345 9810 6885
+2 1 0 4 279 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 7234 6398 9776 6398 9776 6838 7234 6838
+2 3 0 0 -1 278 49 -1 20 0.000 0 0 -1 0 0 5
+ 7234 6838 9776 6838 9776 6398 7234 6398 7234 6838
+2 1 0 2 279 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9613 6398 9613 6838
+2 1 0 2 279 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9438 6398 9438 6838
+2 1 0 2 279 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9264 6398 9264 6838
+4 1 0 46 -1 4 16 0.0000 4 210 1620 8460 6705 TL_URGENT\001
+-6
+6 4140 7830 4545 9045
+1 1 0 3 20 29 52 -1 20 0.000 1 1.5708 4330 8437 607 170 4330 8437 4500 7830
+4 1 0 50 -1 6 10 1.5708 4 120 585 4410 8415 Class?\001
+-6
+1 1 0 3 8 11 52 -1 20 0.000 1 0.0000 9450 5049 540 171 9450 5049 9990 5049
+1 1 0 3 20 29 52 -1 20 0.000 1 1.5708 2440 7672 607 170 2440 7672 2610 7065
+1 1 0 3 8 11 52 -1 20 0.000 1 1.5708 10755 7695 810 171 10755 7695 10755 6885
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 4
+ 2 1 1.00 90.00 180.00
+ 7605 3870 7605 4185 9270 4545 9270 4905
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 4
+ 2 1 1.00 90.00 180.00
+ 11301 3870 11301 4185 9636 4545 9636 4905
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 4
+ 2 1 1.00 90.00 180.00
+ 9630 1395 9626 1591 11291 1800 11295 2295
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 4
+ 2 1 1.00 90.00 180.00
+ 9270 1395 9270 1575 7605 1800 7605 2295
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 2
+ 2 1 1.00 90.00 180.00
+ 9450 360 9450 1035
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 4
+ 2 1 1.00 90.00 180.00
+ 1260 3870 1260 4185 2925 4545 2925 4905
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 4
+ 2 1 1.00 90.00 180.00
+ 4956 3870 4956 4185 3291 4545 3291 4905
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 2
+ 2 1 1.00 90.00 180.00
+ 3105 360 3105 1035
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 4
+ 2 1 1.00 90.00 180.00
+ 3285 1395 3285 1575 4950 1845 4950 2385
+2 1 0 3 22 7 54 -1 -1 0.000 1 0 -1 0 0 2
+ 9180 5535 9000 5805
+2 1 0 5 13 7 54 -1 -1 0.000 1 0 -1 1 0 4
+ 2 1 1.00 120.00 240.00
+ 3105 5220 3105 5850 3105 7200 7200 7200
+2 1 0 5 22 7 54 -1 -1 0.000 1 0 -1 1 0 5
+ 2 1 1.00 120.00 240.00
+ 9450 5220 9450 5670 6300 5670 6300 1215 3690 1170
+2 1 0 3 13 7 54 -1 -1 0.000 1 0 -1 0 0 2
+ 3195 5535 3015 5805
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 4
+ 2 1 1.00 90.00 180.00
+ 2925 1395 2925 1575 1260 1845 1260 2385
+2 2 0 3 35 34 100 -1 20 0.000 1 0 -1 0 0 5
+ 6570 720 12330 720 12330 5400 6570 5400 6570 720
+2 2 0 3 33 32 100 -1 20 0.000 1 0 -1 0 0 5
+ 270 720 6030 720 6030 5400 270 5400 270 720
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 2
+ 2 1 1.00 90.00 180.00
+ 315 7650 2250 7650
+2 1 0 5 4 7 54 -1 -1 0.000 1 0 -1 1 0 2
+ 2 1 1.00 120.00 240.00
+ 10890 7695 12285 7695
+2 1 0 3 0 7 54 -1 -1 0.000 1 0 -1 1 0 3
+ 2 1 1.00 90.00 180.00
+ 4455 8775 4725 8910 7200 8910
+2 1 0 4 279 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 7234 7118 9776 7118 9776 7558 7234 7558
+2 3 0 0 -1 278 49 -1 20 0.000 0 0 -1 0 0 5
+ 7234 7558 9776 7558 9776 7118 7234 7118 7234 7558
+2 1 0 2 279 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9613 7118 9613 7558
+2 1 0 2 279 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9438 7118 9438 7558
+2 1 0 2 279 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9264 7118 9264 7558
+2 3 0 0 -1 278 49 -1 20 0.000 0 0 -1 0 0 5
+ 7234 8278 9776 8278 9776 7838 7234 7838 7234 8278
+2 1 0 2 279 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9613 7838 9613 8278
+2 1 0 2 279 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9438 7838 9438 8278
+2 1 0 2 279 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9264 7838 9264 8278
+2 1 0 4 279 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 7234 8558 9776 8558 9776 8998 7234 8998
+2 3 0 0 -1 278 49 -1 20 0.000 0 0 -1 0 0 5
+ 7234 8998 9776 8998 9776 8558 7234 8558 7234 8998
+2 1 0 2 279 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9613 8558 9613 8998
+2 1 0 2 279 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9438 8558 9438 8998
+2 1 0 2 279 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 9264 8558 9264 8998
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 2
+ 2 1 1.00 90.00 180.00
+ 6075 6480 7200 6480
+2 1 0 3 0 7 50 -1 -1 0.000 1 0 -1 1 0 3
+ 2 1 1.00 90.00 180.00
+ 2610 7830 3195 8415 4140 8415
+2 1 0 4 45 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 4166 6398 6094 6398 6094 6838 4166 6838
+2 1 0 2 45 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 5923 6398 5923 6838
+2 1 0 2 45 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 5748 6398 5748 6838
+2 1 0 2 45 -1 47 -1 -1 0.000 0 0 -1 0 0 2
+ 5574 6398 5574 6838
+2 1 0 3 0 7 54 -1 -1 0.000 1 0 -1 1 0 4
+ 2 1 1.00 90.00 180.00
+ 2610 7515 2925 6660 3645 6660 4140 6660
+2 3 0 0 277 276 49 -1 43 0.000 0 0 -1 0 0 5
+ 4166 6838 6094 6838 6094 6398 4166 6398 4166 6838
+2 1 0 3 0 7 54 -1 -1 0.000 1 0 -1 1 0 3
+ 2 1 1.00 90.00 180.00
+ 9765 8775 10350 8775 10665 8280
+2 1 0 3 0 7 54 -1 -1 0.000 1 0 -1 1 0 3
+ 2 1 1.00 90.00 180.00
+ 9765 8055 10305 8055 10620 7875
+2 1 0 3 0 7 54 -1 -1 0.000 1 0 -1 1 0 3
+ 2 1 1.00 90.00 180.00
+ 9806 6605 10350 6615 10665 7155
+2 1 0 3 0 7 54 -1 -1 0.000 1 0 -1 1 0 3
+ 2 1 1.00 90.00 180.00
+ 9720 7335 10350 7335 10620 7560
+2 1 1 5 4 7 57 -1 -1 12.000 1 0 -1 1 0 2
+ 2 1 1.00 120.00 240.00
+ 9900 6165 9900 9450
+2 1 0 2 0 7 54 -1 -1 0.000 1 0 -1 0 0 2
+ 10080 7245 9990 7425
+2 1 0 2 0 7 54 -1 -1 0.000 1 0 -1 0 0 2
+ 10080 7965 9990 8145
+2 1 0 2 0 7 54 -1 -1 0.000 1 0 -1 0 0 2
+ 10080 8685 9990 8865
+2 1 0 3 0 7 54 -1 -1 0.000 1 0 -1 1 0 4
+ 2 1 1.00 90.00 180.00
+ 4500 8550 6255 8550 6705 8190 7200 8190
+2 1 0 3 0 7 54 -1 -1 0.000 1 0 -1 1 0 5
+ 2 1 1.00 90.00 180.00
+ 4500 8280 4725 8100 6435 8100 6750 7470 7200 7470
+2 1 0 3 0 7 54 -1 -1 0.000 1 0 -1 1 0 5
+ 2 1 1.00 90.00 180.00
+ 4455 8055 4635 7740 6390 7740 6750 6750 7200 6750
+2 1 0 4 279 -1 48 -1 -1 0.000 0 0 -1 0 0 4
+ 7234 7838 9776 7838 9776 8278 7234 8278
+2 1 0 2 0 7 54 -1 -1 0.000 1 0 -1 0 0 2
+ 10080 6525 9990 6705
+2 2 0 3 43 47 100 -1 20 0.000 1 0 -1 0 0 5
+ 1935 5985 11070 5985 11070 9585 1935 9585 1935 5985
+4 1 0 50 -1 4 9 1.5708 4 135 315 12240 3060 past\001
+4 1 0 50 -1 4 9 1.5708 4 120 465 10440 3060 future\001
+4 1 0 50 -1 4 9 1.5708 4 135 315 8550 3060 past\001
+4 1 0 50 -1 4 9 1.5708 4 120 465 6750 3060 future\001
+4 1 0 50 -1 6 10 0.0000 4 120 600 9450 5130 Oldest\001
+4 1 0 50 -1 4 9 1.5708 4 105 540 405 3060 newest\001
+4 1 0 50 -1 4 9 1.5708 4 120 450 2205 3060 oldest\001
+4 1 0 50 -1 4 9 1.5708 4 105 540 4095 3060 newest\001
+4 1 0 50 -1 4 9 1.5708 4 120 450 5895 3060 oldest\001
+4 0 0 50 -1 14 10 0.0000 4 135 1470 9135 5850 runqueue-depth\001
+4 0 0 50 -1 14 10 0.0000 4 135 1470 3195 5715 runqueue-depth\001
+4 1 0 50 -1 6 12 0.0000 4 165 1320 9450 3600 Time-based\001
+4 1 0 50 -1 6 12 0.0000 4 195 1395 9450 3780 Wait queues\001
+4 0 0 50 -1 6 12 0.0000 4 195 1050 9000 4005 - 1 global\001
+4 0 0 50 -1 6 12 0.0000 4 195 1605 9000 4185 - 1 per thread\001
+4 1 0 50 -1 6 12 0.0000 4 195 1650 3105 3600 Priority-based\001
+4 1 0 50 -1 6 12 0.0000 4 180 1365 3105 3780 Run queues\001
+4 0 0 50 -1 6 12 0.0000 4 195 1050 2655 4005 - 1 global\001
+4 0 0 50 -1 6 12 0.0000 4 195 1605 2655 4185 - 1 per thread\001
+4 0 0 50 -1 14 10 0.0000 4 135 1365 3240 585 task_wakeup()\001
+4 0 0 50 -1 14 10 0.0000 4 135 1575 9585 630 task_schedule()\001
+4 0 0 50 -1 14 10 0.0000 4 135 1260 9585 450 task_queue()\001
+4 0 0 50 -1 14 10 0.0000 4 135 1680 315 7560 tasklet_wakeup()\001
+4 2 0 50 -1 14 10 0.0000 4 135 1260 12285 7515 t->process()\001
+4 2 4 50 -1 6 12 0.0000 4 150 525 12285 7335 Run!\001
+4 1 0 46 -1 4 16 0.0000 4 210 1695 8460 7425 TL_NORMAL\001
+4 1 0 46 -1 4 16 0.0000 4 210 1200 8460 8145 TL_BULK\001
+4 1 0 46 -1 4 16 0.0000 4 210 1425 8460 8865 TL_HEAVY\001
+4 1 0 46 -1 4 16 0.0000 4 195 1095 4950 6705 SHARED\001
+4 0 0 50 -1 6 9 0.0000 4 105 345 10035 7515 37%\001
+4 0 0 50 -1 6 9 0.0000 4 105 210 10080 8955 =1\001
+4 1 0 50 -1 4 10 0.0000 4 150 2280 5085 6255 (accessed using atomic ops)\001
+4 0 0 50 -1 6 9 0.0000 4 105 345 10035 6795 50%\001
+4 0 0 50 -1 6 9 0.0000 4 105 345 10035 8235 13%\001
+4 2 0 50 -1 6 9 1.5708 4 105 315 2745 8100 Yes\001
+4 1 0 50 -1 6 10 1.5708 4 120 615 2520 7650 Local ?\001
+4 0 0 50 -1 6 9 1.5708 4 105 225 2700 7110 No\001
+4 0 0 50 -1 14 10 0.0000 4 135 1680 4725 8460 TASK_SELF_WAKING\001
+4 0 0 50 -1 14 10 0.0000 4 135 1050 4725 8820 TASK_HEAVY\001
+4 0 0 50 -1 4 10 0.0000 4 165 675 4725 8010 (default)\001
+4 0 0 50 -1 4 10 0.0000 4 150 1290 4725 7650 In I/O or signals\001
+4 1 0 50 -1 6 10 1.5708 4 150 1125 10815 7695 Most Urgent\001
+4 0 4 50 -1 6 10 0.0000 4 120 480 9990 6480 order\001
+4 0 4 50 -1 6 10 0.0000 4 120 420 9990 6300 Scan\001
+4 1 0 50 -1 6 12 0.0000 4 195 9075 6030 9450 5 class-based tasklet queues per thread (one accessible from remote threads)\001
diff --git a/doc/internals/sched.pdf b/doc/internals/sched.pdf
new file mode 100644
index 0000000..d1ce3de
--- /dev/null
+++ b/doc/internals/sched.pdf
Binary files differ
diff --git a/doc/internals/sched.png b/doc/internals/sched.png
new file mode 100644
index 0000000..65c97a1
--- /dev/null
+++ b/doc/internals/sched.png
Binary files differ
diff --git a/doc/internals/sched.svg b/doc/internals/sched.svg
new file mode 100644
index 0000000..0fa329a
--- /dev/null
+++ b/doc/internals/sched.svg
@@ -0,0 +1,1204 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Creator: fig2dev Version 3.2.7b -->
+<!-- CreationDate: 2021-02-26 17:49:00 -->
+<!-- Magnification: 1.57 -->
+<svg xmlns="http://www.w3.org/2000/svg"
+ xmlns:xlink="http://www.w3.org/1999/xlink"
+ width="1146pt" height="878pt"
+ viewBox="237 327 12126 9291">
+<g fill="none">
+<!-- Line -->
+<rect x="6570" y="720" width="5760" height="4680" fill="#ffebac"
+ stroke="#cbb366" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Line -->
+<rect x="270" y="720" width="5760" height="4680" fill="#c5ebe1"
+ stroke="#86c8a2" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Line -->
+<rect x="1935" y="5985" width="9135" height="3600" fill="#dfe3df"
+ stroke="#868286" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp0">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 9960,9130 9900,9190 9840,9130 9867,9483 9933,9483z"/>
+</clipPath>
+</defs>
+<polyline points=" 9900,6165 9900,9450" clip-path="url(#cp0)"
+ stroke="#ff0000" stroke-width="60px" stroke-linejoin="round" stroke-dasharray="120 120"/>
+<!-- Forward arrow to point 9900,9450 -->
+<polygon points=" 9840,9130 9900,9430 9960,9130 9900,9190 9840,9130"
+ stroke="#ff0000" stroke-width="8px" stroke-miterlimit="8" fill="#ff0000"/>
+<!-- Line -->
+<polyline points=" 9180,5535 9000,5805"
+ stroke="#b000b0" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp1">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 6880,7140 6940,7200 6880,7260 7233,7233 7233,7167z"/>
+</clipPath>
+</defs>
+<polyline points=" 3105,5220 3105,5850 3105,7200 7200,7200" clip-path="url(#cp1)"
+ stroke="#00b000" stroke-width="60px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 7200,7200 -->
+<polygon points=" 6880,7260 7180,7200 6880,7140 6940,7200 6880,7260"
+ stroke="#00b000" stroke-width="8px" stroke-miterlimit="8" fill="#00b000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp2">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 4009,1236 3950,1174 4011,1116 3658,1136 3656,1202z"/>
+</clipPath>
+</defs>
+<polyline points=" 9450,5220 9450,5670 6300,5670 6300,1215 3690,1170" clip-path="url(#cp2)"
+ stroke="#b000b0" stroke-width="60px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 3690,1170 -->
+<polygon points=" 4011,1116 3710,1170 4009,1236 3950,1174 4011,1116"
+ stroke="#b000b0" stroke-width="8px" stroke-miterlimit="8" fill="#b000b0"/>
+<!-- Line -->
+<polyline points=" 3195,5535 3015,5805"
+ stroke="#00b000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp3">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 11965,7635 12025,7695 11965,7755 12318,7728 12318,7662z"/>
+</clipPath>
+</defs>
+<polyline points=" 10890,7695 12285,7695" clip-path="url(#cp3)"
+ stroke="#ff0000" stroke-width="60px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 12285,7695 -->
+<polygon points=" 11965,7755 12265,7695 11965,7635 12025,7695 11965,7755"
+ stroke="#ff0000" stroke-width="8px" stroke-miterlimit="8" fill="#ff0000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp4">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 6955,8865 7000,8910 6955,8955 7218,8928 7218,8892z"/>
+</clipPath>
+</defs>
+<polyline points=" 4455,8775 4725,8910 7200,8910" clip-path="url(#cp4)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 7200,8910 -->
+<polygon points=" 6955,8955 7180,8910 6955,8865 7000,8910 6955,8955"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp5">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 3895,6615 3940,6660 3895,6705 4158,6678 4158,6642z"/>
+</clipPath>
+</defs>
+<polyline points=" 2610,7515 2925,6660 3645,6660 4140,6660" clip-path="url(#cp5)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 4140,6660 -->
+<polygon points=" 3895,6705 4120,6660 3895,6615 3940,6660 3895,6705"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp6">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 10496,8462 10558,8449 10572,8511 10690,8274 10659,8255z"/>
+</clipPath>
+</defs>
+<polyline points=" 9765,8775 10350,8775 10665,8280" clip-path="url(#cp6)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 10665,8280 -->
+<polygon points=" 10572,8511 10654,8297 10496,8462 10558,8449 10572,8511"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp7">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 10385,7957 10446,7974 10430,8036 10645,7882 10627,7850z"/>
+</clipPath>
+</defs>
+<polyline points=" 9765,8055 10305,8055 10620,7875" clip-path="url(#cp7)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 10620,7875 -->
+<polygon points=" 10430,8036 10603,7885 10385,7957 10446,7974 10430,8036"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp8">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 10580,6921 10564,6982 10503,6966 10659,7180 10690,7161z"/>
+</clipPath>
+</defs>
+<polyline points=" 9806,6605 10350,6615 10665,7155" clip-path="url(#cp8)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 10665,7155 -->
+<polygon points=" 10503,6966 10655,7138 10580,6921 10564,6982 10503,6966"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp9">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 10461,7369 10466,7432 10403,7438 10622,7585 10645,7558z"/>
+</clipPath>
+</defs>
+<polyline points=" 9720,7335 10350,7335 10620,7560" clip-path="url(#cp9)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 10620,7560 -->
+<polygon points=" 10403,7438 10605,7547 10461,7369 10466,7432 10403,7438"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<polyline points=" 10080,7245 9990,7425"
+ stroke="#000000" stroke-width="15px" stroke-linejoin="round"/>
+<!-- Line -->
+<polyline points=" 10080,7965 9990,8145"
+ stroke="#000000" stroke-width="15px" stroke-linejoin="round"/>
+<!-- Line -->
+<polyline points=" 10080,8685 9990,8865"
+ stroke="#000000" stroke-width="15px" stroke-linejoin="round"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp10">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 6955,8145 7000,8190 6955,8235 7218,8208 7218,8172z"/>
+</clipPath>
+</defs>
+<polyline points=" 4500,8550 6255,8550 6705,8190 7200,8190" clip-path="url(#cp10)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 7200,8190 -->
+<polygon points=" 6955,8235 7180,8190 6955,8145 7000,8190 6955,8235"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp11">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 6955,7425 7000,7470 6955,7515 7218,7488 7218,7452z"/>
+</clipPath>
+</defs>
+<polyline points=" 4500,8280 4725,8100 6435,8100 6750,7470 7200,7470" clip-path="url(#cp11)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 7200,7470 -->
+<polygon points=" 6955,7515 7180,7470 6955,7425 7000,7470 6955,7515"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp12">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 6955,6705 7000,6750 6955,6795 7218,6768 7218,6732z"/>
+</clipPath>
+</defs>
+<polyline points=" 4455,8055 4635,7740 6390,7740 6750,6750 7200,6750" clip-path="url(#cp12)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 7200,6750 -->
+<polygon points=" 6955,6795 7180,6750 6955,6705 7000,6750 6955,6795"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<polyline points=" 10080,6525 9990,6705"
+ stroke="#000000" stroke-width="15px" stroke-linejoin="round"/>
+<!-- Ellipse -->
+<ellipse cx="3105" cy="5049" rx="810" ry="171" fill="#87cfff"
+ stroke="#00008f" stroke-width="30px"/>
+<!-- Ellipse -->
+<ellipse cx="3083" cy="1180" rx="607" ry="170" fill="#ffbfbf"
+ stroke="#d10000" stroke-width="30px"/>
+<!-- Ellipse -->
+<ellipse cx="9428" cy="1180" rx="607" ry="170" fill="#ffbfbf"
+ stroke="#d10000" stroke-width="30px"/>
+<!-- Ellipse -->
+<ellipse transform="translate(4330,8437) rotate(-90)" rx="607" ry="170" fill="#ffbfbf"
+ stroke="#d10000" stroke-width="30px"/>
+<!-- Ellipse -->
+<ellipse cx="9450" cy="5049" rx="540" ry="171" fill="#87cfff"
+ stroke="#00008f" stroke-width="30px"/>
+<!-- Ellipse -->
+<ellipse transform="translate(2440,7672) rotate(-90)" rx="607" ry="170" fill="#ffbfbf"
+ stroke="#d10000" stroke-width="30px"/>
+<!-- Ellipse -->
+<ellipse transform="translate(10755,7695) rotate(-90)" rx="810" ry="171" fill="#87cfff"
+ stroke="#00008f" stroke-width="30px"/>
+<!-- Line -->
+<polygon points=" 11768,3060 11970,3060 11967,3119 11959,3177 11946,3234 11929,3291 11907,3345
+ 11879,3397 11704,3296 11723,3259 11738,3222 11751,3182 11760,3142 11765,3101
+" fill="#ffd600"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 11704,3296 11879,3397 11848,3447 11812,3494 11772,3537 11729,3577 11682,3613
+ 11633,3644 11531,3469 11566,3447 11599,3422 11628,3393 11657,3364 11682,3331
+" fill="#ffd600"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 11531,3469 11633,3644 11580,3672 11526,3694 11469,3711 11412,3724 11354,3732
+ 11295,3734 11295,3532 11336,3530 11377,3525 11417,3516 11457,3503 11494,3488
+" fill="#ffd600"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 11295,3532 11295,3734 11236,3732 11178,3724 11121,3711 11064,3694 11010,3672
+ 10958,3644 11059,3469 11096,3488 11133,3503 11173,3516 11213,3525 11254,3530
+" fill="#ffd600"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 11059,3469 10958,3644 10908,3613 10861,3577 10818,3537 10778,3494 10742,3447
+ 10711,3398 10886,3296 10908,3331 10933,3364 10962,3393 10991,3422 11024,3447
+" fill="#ffd600"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 10886,3296 10711,3398 10683,3345 10661,3291 10644,3234 10631,3177 10623,3119
+ 10621,3060 10823,3060 10825,3101 10830,3142 10839,3182 10852,3222 10867,3259
+" fill="#ffd600"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 10823,3060 10621,3060 10623,3001 10631,2943 10644,2886 10661,2829 10683,2775
+ 10711,2723 10886,2824 10867,2861 10852,2898 10839,2938 10830,2978 10825,3019
+" fill="#ffd600"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 10886,2824 10711,2723 10742,2673 10778,2626 10818,2583 10861,2543 10908,2507
+ 10958,2476 11059,2651 11024,2673 10991,2698 10962,2727 10933,2756 10908,2789
+" fill="#ffd600"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 11059,2651 10958,2476 11010,2448 11064,2426 11121,2409 11178,2396 11236,2388
+ 11295,2386 11295,2588 11254,2590 11213,2595 11173,2604 11133,2617 11096,2632
+" fill="#ffd600"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 11295,2588 11295,2386 11354,2388 11412,2396 11469,2409 11526,2426 11580,2448
+ 11632,2476 11531,2651 11494,2632 11457,2617 11417,2604 11377,2595 11336,2590
+" fill="#ffd600"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 11531,2651 11632,2476 11682,2507 11729,2543 11772,2583 11812,2626 11848,2673
+ 11879,2723 11704,2824 11682,2789 11657,2756 11628,2727 11599,2698 11566,2673
+" fill="#ffd600"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 11704,2824 11879,2723 11907,2775 11929,2829 11946,2886 11959,2943 11967,3001
+ 11969,3060 11767,3060 11765,3019 11760,2978 11751,2938 11738,2898 11723,2861
+" fill="#ffd600"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 5423,3060 5625,3060 5622,3119 5614,3177 5601,3234 5584,3291 5562,3345 5534,3397
+ 5359,3296 5378,3259 5393,3222 5406,3182 5415,3142 5420,3101" fill="#00b000"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 5359,3296 5534,3397 5503,3447 5467,3494 5427,3537 5384,3577 5337,3613 5288,3644
+ 5186,3469 5221,3447 5254,3422 5283,3393 5312,3364 5337,3331" fill="#00b000"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 5186,3469 5288,3644 5235,3672 5181,3694 5124,3711 5067,3724 5009,3732 4950,3734
+ 4950,3532 4991,3530 5032,3525 5072,3516 5112,3503 5149,3488" fill="#00b000"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 4950,3532 4950,3734 4891,3732 4833,3724 4776,3711 4719,3694 4665,3672 4613,3644
+ 4714,3469 4751,3488 4788,3503 4828,3516 4868,3525 4909,3530" fill="#00b000"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 4714,3469 4613,3644 4563,3613 4516,3577 4473,3537 4433,3494 4397,3447 4366,3398
+ 4541,3296 4563,3331 4588,3364 4617,3393 4646,3422 4679,3447" fill="#00b000"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 4541,3296 4366,3398 4338,3345 4316,3291 4299,3234 4286,3177 4278,3119 4276,3060
+ 4478,3060 4480,3101 4485,3142 4494,3182 4507,3222 4522,3259" fill="#00b000"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 4478,3060 4276,3060 4278,3001 4286,2943 4299,2886 4316,2829 4338,2775 4366,2723
+ 4541,2824 4522,2861 4507,2898 4494,2938 4485,2978 4480,3019" fill="#00b000"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 4541,2824 4366,2723 4397,2673 4433,2626 4473,2583 4516,2543 4563,2507 4613,2476
+ 4714,2651 4679,2673 4646,2698 4617,2727 4588,2756 4563,2789" fill="#00b000"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 4714,2651 4613,2476 4665,2448 4719,2426 4776,2409 4833,2396 4891,2388 4950,2386
+ 4950,2588 4909,2590 4868,2595 4828,2604 4788,2617 4751,2632" fill="#00b000"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 4950,2588 4950,2386 5009,2388 5067,2396 5124,2409 5181,2426 5235,2448 5287,2476
+ 5186,2651 5149,2632 5112,2617 5072,2604 5032,2595 4991,2590" fill="#00b000"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 5186,2651 5287,2476 5337,2507 5384,2543 5427,2583 5467,2626 5503,2673 5534,2723
+ 5359,2824 5337,2789 5312,2756 5283,2727 5254,2698 5221,2673" fill="#00b000"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<polygon points=" 5359,2824 5534,2723 5562,2775 5584,2829 5601,2886 5614,2943 5622,3001 5624,3060
+ 5422,3060 5420,3019 5415,2978 5406,2938 5393,2898 5378,2861" fill="#00b000"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Text -->
+<text xml:space="preserve" x="3105" y="5130" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">Most Urgent</text>
+<!-- Line -->
+<defs>
+<polygon points=" 8078,3060 8280,3060 8277,3119 8269,3177 8256,3234 8239,3291 8217,3345 8189,3397
+ 8014,3296 8033,3259 8048,3222 8061,3182 8070,3142 8075,3101" id="p0"/>
+<pattern id="tile0" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p0" fill="#ffd600"/>
+<use xlink:href="#p0" fill="url(#tile0)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 8014,3296 8189,3397 8158,3447 8122,3494 8082,3537 8039,3577 7992,3613 7943,3644
+ 7841,3469 7876,3447 7909,3422 7938,3393 7967,3364 7992,3331" id="p1"/>
+<pattern id="tile1" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p1" fill="#ffd600"/>
+<use xlink:href="#p1" fill="url(#tile1)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 7841,3469 7943,3644 7890,3672 7836,3694 7779,3711 7722,3724 7664,3732 7605,3734
+ 7605,3532 7646,3530 7687,3525 7727,3516 7767,3503 7804,3488" id="p2"/>
+<pattern id="tile2" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p2" fill="#ffd600"/>
+<use xlink:href="#p2" fill="url(#tile2)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 7605,3532 7605,3734 7546,3732 7488,3724 7431,3711 7374,3694 7320,3672 7268,3644
+ 7369,3469 7406,3488 7443,3503 7483,3516 7523,3525 7564,3530" id="p3"/>
+<pattern id="tile3" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p3" fill="#ffd600"/>
+<use xlink:href="#p3" fill="url(#tile3)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 7369,3469 7268,3644 7218,3613 7171,3577 7128,3537 7088,3494 7052,3447 7021,3398
+ 7196,3296 7218,3331 7243,3364 7272,3393 7301,3422 7334,3447" id="p4"/>
+<pattern id="tile4" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p4" fill="#ffd600"/>
+<use xlink:href="#p4" fill="url(#tile4)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 7196,3296 7021,3398 6993,3345 6971,3291 6954,3234 6941,3177 6933,3119 6931,3060
+ 7133,3060 7135,3101 7140,3142 7149,3182 7162,3222 7177,3259" id="p5"/>
+<pattern id="tile5" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p5" fill="#ffd600"/>
+<use xlink:href="#p5" fill="url(#tile5)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 7133,3060 6931,3060 6933,3001 6941,2943 6954,2886 6971,2829 6993,2775 7021,2723
+ 7196,2824 7177,2861 7162,2898 7149,2938 7140,2978 7135,3019" id="p6"/>
+<pattern id="tile6" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p6" fill="#ffd600"/>
+<use xlink:href="#p6" fill="url(#tile6)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 7196,2824 7021,2723 7052,2673 7088,2626 7128,2583 7171,2543 7218,2507 7268,2476
+ 7369,2651 7334,2673 7301,2698 7272,2727 7243,2756 7218,2789" id="p7"/>
+<pattern id="tile7" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p7" fill="#ffd600"/>
+<use xlink:href="#p7" fill="url(#tile7)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 7369,2651 7268,2476 7320,2448 7374,2426 7431,2409 7488,2396 7546,2388 7605,2386
+ 7605,2588 7564,2590 7523,2595 7483,2604 7443,2617 7406,2632" id="p8"/>
+<pattern id="tile8" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p8" fill="#ffd600"/>
+<use xlink:href="#p8" fill="url(#tile8)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 7605,2588 7605,2386 7664,2388 7722,2396 7779,2409 7836,2426 7890,2448 7942,2476
+ 7841,2651 7804,2632 7767,2617 7727,2604 7687,2595 7646,2590" id="p9"/>
+<pattern id="tile9" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p9" fill="#ffd600"/>
+<use xlink:href="#p9" fill="url(#tile9)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 7841,2651 7942,2476 7992,2507 8039,2543 8082,2583 8122,2626 8158,2673 8189,2723
+ 8014,2824 7992,2789 7967,2756 7938,2727 7909,2698 7876,2673" id="p10"/>
+<pattern id="tile10" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p10" fill="#ffd600"/>
+<use xlink:href="#p10" fill="url(#tile10)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 8014,2824 8189,2723 8217,2775 8239,2829 8256,2886 8269,2943 8277,3001 8279,3060
+ 8077,3060 8075,3019 8070,2978 8061,2938 8048,2898 8033,2861" id="p11"/>
+<pattern id="tile11" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p11" fill="#ffd600"/>
+<use xlink:href="#p11" fill="url(#tile11)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 1733,3060 1935,3060 1932,3119 1924,3177 1911,3234 1894,3291 1872,3345 1844,3397
+ 1669,3296 1688,3259 1703,3222 1716,3182 1725,3142 1730,3101" id="p12"/>
+<pattern id="tile12" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p12" fill="#00b000"/>
+<use xlink:href="#p12" fill="url(#tile12)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 1669,3296 1844,3397 1813,3447 1777,3494 1737,3537 1694,3577 1647,3613 1598,3644
+ 1496,3469 1531,3447 1564,3422 1593,3393 1622,3364 1647,3331" id="p13"/>
+<pattern id="tile13" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p13" fill="#00b000"/>
+<use xlink:href="#p13" fill="url(#tile13)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 1496,3469 1598,3644 1545,3672 1491,3694 1434,3711 1377,3724 1319,3732 1260,3734
+ 1260,3532 1301,3530 1342,3525 1382,3516 1422,3503 1459,3488" id="p14"/>
+<pattern id="tile14" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p14" fill="#00b000"/>
+<use xlink:href="#p14" fill="url(#tile14)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 1260,3532 1260,3734 1201,3732 1143,3724 1086,3711 1029,3694 975,3672 923,3644
+ 1024,3469 1061,3488 1098,3503 1138,3516 1178,3525 1219,3530" id="p15"/>
+<pattern id="tile15" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p15" fill="#00b000"/>
+<use xlink:href="#p15" fill="url(#tile15)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 1024,3469 923,3644 873,3613 826,3577 783,3537 743,3494 707,3447 676,3398 851,3296
+ 873,3331 898,3364 927,3393 956,3422 989,3447" id="p16"/>
+<pattern id="tile16" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p16" fill="#00b000"/>
+<use xlink:href="#p16" fill="url(#tile16)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 851,3296 676,3398 648,3345 626,3291 609,3234 596,3177 588,3119 586,3060 788,3060
+ 790,3101 795,3142 804,3182 817,3222 832,3259" id="p17"/>
+<pattern id="tile17" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p17" fill="#00b000"/>
+<use xlink:href="#p17" fill="url(#tile17)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 788,3060 586,3060 588,3001 596,2943 609,2886 626,2829 648,2775 676,2723 851,2824
+ 832,2861 817,2898 804,2938 795,2978 790,3019" id="p18"/>
+<pattern id="tile18" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p18" fill="#00b000"/>
+<use xlink:href="#p18" fill="url(#tile18)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 851,2824 676,2723 707,2673 743,2626 783,2583 826,2543 873,2507 923,2476 1024,2651
+ 989,2673 956,2698 927,2727 898,2756 873,2789" id="p19"/>
+<pattern id="tile19" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p19" fill="#00b000"/>
+<use xlink:href="#p19" fill="url(#tile19)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 1024,2651 923,2476 975,2448 1029,2426 1086,2409 1143,2396 1201,2388 1260,2386
+ 1260,2588 1219,2590 1178,2595 1138,2604 1098,2617 1061,2632" id="p20"/>
+<pattern id="tile20" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p20" fill="#00b000"/>
+<use xlink:href="#p20" fill="url(#tile20)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 1260,2588 1260,2386 1319,2388 1377,2396 1434,2409 1491,2426 1545,2448 1597,2476
+ 1496,2651 1459,2632 1422,2617 1382,2604 1342,2595 1301,2590" id="p21"/>
+<pattern id="tile21" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p21" fill="#00b000"/>
+<use xlink:href="#p21" fill="url(#tile21)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 1496,2651 1597,2476 1647,2507 1694,2543 1737,2583 1777,2626 1813,2673 1844,2723
+ 1669,2824 1647,2789 1622,2756 1593,2727 1564,2698 1531,2673" id="p22"/>
+<pattern id="tile22" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p22" fill="#00b000"/>
+<use xlink:href="#p22" fill="url(#tile22)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Line -->
+<defs>
+<polygon points=" 1669,2824 1844,2723 1872,2775 1894,2829 1911,2886 1924,2943 1932,3001 1934,3060
+ 1732,3060 1730,3019 1725,2978 1716,2938 1703,2898 1688,2861" id="p23"/>
+<pattern id="tile23" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#000000" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p23" fill="#00b000"/>
+<use xlink:href="#p23" fill="url(#tile23)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Text -->
+<text xml:space="preserve" x="2160" y="1980" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">Global</text>
+<!-- Text -->
+<text xml:space="preserve" x="2160" y="2160" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">tasks</text>
+<!-- Text -->
+<text xml:space="preserve" x="2160" y="2340" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">(locked)</text>
+<!-- Text -->
+<text xml:space="preserve" x="4230" y="2250" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">tasks</text>
+<!-- Text -->
+<text xml:space="preserve" x="4230" y="2070" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">Local</text>
+<!-- Text -->
+<text xml:space="preserve" x="8550" y="2340" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">(locked)</text>
+<!-- Text -->
+<text xml:space="preserve" x="8550" y="2160" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">timers</text>
+<!-- Text -->
+<text xml:space="preserve" x="8550" y="1980" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">Global</text>
+<!-- Text -->
+<text xml:space="preserve" x="10530" y="2250" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">timers</text>
+<!-- Text -->
+<text xml:space="preserve" x="10530" y="2070" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">Local</text>
+<!-- Text -->
+<text xml:space="preserve" x="3105" y="1260" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">Local ?</text>
+<!-- Text -->
+<text xml:space="preserve" x="3375" y="1530" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="108" text-anchor="start">Yes</text>
+<!-- Text -->
+<text xml:space="preserve" x="2790" y="1530" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="108" text-anchor="end">No</text>
+<!-- Text -->
+<text xml:space="preserve" x="9450" y="1260" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">Local ?</text>
+<!-- Text -->
+<text xml:space="preserve" x="9720" y="1530" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="108" text-anchor="start">Yes</text>
+<!-- Text -->
+<text xml:space="preserve" x="9135" y="1530" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="108" text-anchor="end">No</text>
+<!-- Text -->
+<g transform="translate(4410,8415) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">Class?</text>
+</g><!-- Arc -->
+<defs>
+<clipPath id="cp13">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 11065,2294 11189,2298 11078,2353 11217,2303 11213,2283z"/>
+</clipPath>
+</defs>
+<path d="M 11205,3825 A 771 771 0 0 1 11205 2295" clip-path="url(#cp13)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Forward arrow to point 11205,2295 -->
+<polyline points=" 11065,2294 11189,2298 11078,2353"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8"/>
+<!-- Arc -->
+<defs>
+<clipPath id="cp14">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 11065,2294 11189,2298 11078,2353 11217,2303 11213,2283z
+ M 11525,3826 11401,3822 11512,3767 11373,3817 11377,3837z"/>
+</clipPath>
+</defs>
+<path d="M 11385,3825 A 771 771 0 0 0 11385 2295" clip-path="url(#cp14)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Backward arrow to point 11385,3825 -->
+<polyline points=" 11525,3826 11401,3822 11512,3767"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8"/>
+<!-- Arc -->
+<defs>
+<clipPath id="cp15">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 11065,2294 11189,2298 11078,2353 11217,2303 11213,2283z
+ M 10908,3277 10928,3240 10965,3260 10897,3092 10877,3098z"/>
+</clipPath>
+</defs>
+<path d="M 10890,3105 A 406 406 0 1 0 10890 3015" clip-path="url(#cp15)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Backward arrow to point 10890,3105 -->
+<polygon points=" 10965,3260 10895,3124 10908,3277 10928,3240 10965,3260"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Arc -->
+<defs>
+<clipPath id="cp16">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 7375,2294 7499,2298 7388,2353 7527,2303 7523,2283z
+ M 10908,3277 10928,3240 10965,3260 10897,3092 10877,3098z"/>
+</clipPath>
+</defs>
+<path d="M 7515,3825 A 771 771 0 0 1 7515 2295" clip-path="url(#cp16)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Forward arrow to point 7515,2295 -->
+<polyline points=" 7375,2294 7499,2298 7388,2353"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8"/>
+<!-- Arc -->
+<defs>
+<clipPath id="cp17">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 7375,2294 7499,2298 7388,2353 7527,2303 7523,2283z
+ M 7835,3826 7711,3822 7822,3767 7683,3817 7687,3837z"/>
+</clipPath>
+</defs>
+<path d="M 7695,3825 A 771 771 0 0 0 7695 2295" clip-path="url(#cp17)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Backward arrow to point 7695,3825 -->
+<polyline points=" 7835,3826 7711,3822 7822,3767"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8"/>
+<!-- Arc -->
+<defs>
+<clipPath id="cp18">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 7375,2294 7499,2298 7388,2353 7527,2303 7523,2283z
+ M 7218,3277 7238,3240 7275,3260 7207,3092 7187,3098z"/>
+</clipPath>
+</defs>
+<path d="M 7200,3105 A 406 406 0 1 0 7200 3015" clip-path="url(#cp18)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Backward arrow to point 7200,3105 -->
+<polygon points=" 7275,3260 7205,3124 7218,3277 7238,3240 7275,3260"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Arc -->
+<defs>
+<clipPath id="cp19">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 4720,2294 4844,2298 4733,2353 4872,2303 4868,2283z
+ M 7218,3277 7238,3240 7275,3260 7207,3092 7187,3098z"/>
+</clipPath>
+</defs>
+<path d="M 4860,3825 A 771 771 0 0 1 4860 2295" clip-path="url(#cp19)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Forward arrow to point 4860,2295 -->
+<polyline points=" 4720,2294 4844,2298 4733,2353"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8"/>
+<!-- Arc -->
+<defs>
+<clipPath id="cp20">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 4720,2294 4844,2298 4733,2353 4872,2303 4868,2283z
+ M 5180,3826 5056,3822 5167,3767 5028,3817 5032,3837z"/>
+</clipPath>
+</defs>
+<path d="M 5040,3825 A 771 771 0 0 0 5040 2295" clip-path="url(#cp20)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Backward arrow to point 5040,3825 -->
+<polyline points=" 5180,3826 5056,3822 5167,3767"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8"/>
+<!-- Arc -->
+<defs>
+<clipPath id="cp21">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 4720,2294 4844,2298 4733,2353 4872,2303 4868,2283z
+ M 4563,3277 4583,3240 4620,3260 4552,3092 4532,3098z"/>
+</clipPath>
+</defs>
+<path d="M 4545,3105 A 406 406 0 1 0 4545 3015" clip-path="url(#cp21)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Backward arrow to point 4545,3105 -->
+<polygon points=" 4620,3260 4550,3124 4563,3277 4583,3240 4620,3260"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Arc -->
+<defs>
+<clipPath id="cp22">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 1030,2294 1154,2298 1043,2353 1182,2303 1178,2283z
+ M 4563,3277 4583,3240 4620,3260 4552,3092 4532,3098z"/>
+</clipPath>
+</defs>
+<path d="M 1170,3825 A 771 771 0 0 1 1170 2295" clip-path="url(#cp22)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Forward arrow to point 1170,2295 -->
+<polyline points=" 1030,2294 1154,2298 1043,2353"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8"/>
+<!-- Arc -->
+<defs>
+<clipPath id="cp23">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 1030,2294 1154,2298 1043,2353 1182,2303 1178,2283z
+ M 1490,3826 1366,3822 1477,3767 1338,3817 1342,3837z"/>
+</clipPath>
+</defs>
+<path d="M 1350,3825 A 771 771 0 0 0 1350 2295" clip-path="url(#cp23)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Backward arrow to point 1350,3825 -->
+<polyline points=" 1490,3826 1366,3822 1477,3767"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8"/>
+<!-- Arc -->
+<defs>
+<clipPath id="cp24">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 1030,2294 1154,2298 1043,2353 1182,2303 1178,2283z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<path d="M 855,3105 A 406 406 0 1 0 855 3015" clip-path="url(#cp24)"
+ stroke="#000000" stroke-width="15px"/>
+<!-- Backward arrow to point 855,3105 -->
+<polygon points=" 930,3260 860,3124 873,3277 893,3240 930,3260"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp25">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 9315,4660 9270,4705 9225,4660 9252,4923 9288,4923z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 7605,3870 7605,4185 9270,4545 9270,4905" clip-path="url(#cp25)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 9270,4905 -->
+<polygon points=" 9225,4660 9270,4885 9315,4660 9270,4705 9225,4660"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp26">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 9681,4660 9636,4705 9591,4660 9618,4923 9654,4923z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 11301,3870 11301,4185 9636,4545 9636,4905" clip-path="url(#cp26)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 9636,4905 -->
+<polygon points=" 9591,4660 9636,4885 9681,4660 9636,4705 9591,4660"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp27">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 11338,2050 11293,2095 11248,2051 11277,2313 11313,2313z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 9630,1395 9626,1591 11291,1800 11295,2295" clip-path="url(#cp27)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 11295,2295 -->
+<polygon points=" 11248,2051 11295,2275 11338,2050 11293,2095 11248,2051"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp28">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 7650,2050 7605,2095 7560,2050 7587,2313 7623,2313z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 9270,1395 9270,1575 7605,1800 7605,2295" clip-path="url(#cp28)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 7605,2295 -->
+<polygon points=" 7560,2050 7605,2275 7650,2050 7605,2095 7560,2050"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp29">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 9495,790 9450,835 9405,790 9432,1053 9468,1053z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 9450,360 9450,1035" clip-path="url(#cp29)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 9450,1035 -->
+<polygon points=" 9405,790 9450,1015 9495,790 9450,835 9405,790"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp30">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 2970,4660 2925,4705 2880,4660 2907,4923 2943,4923z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 1260,3870 1260,4185 2925,4545 2925,4905" clip-path="url(#cp30)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 2925,4905 -->
+<polygon points=" 2880,4660 2925,4885 2970,4660 2925,4705 2880,4660"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp31">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 3336,4660 3291,4705 3246,4660 3273,4923 3309,4923z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 4956,3870 4956,4185 3291,4545 3291,4905" clip-path="url(#cp31)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 3291,4905 -->
+<polygon points=" 3246,4660 3291,4885 3336,4660 3291,4705 3246,4660"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp32">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 3150,790 3105,835 3060,790 3087,1053 3123,1053z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 3105,360 3105,1035" clip-path="url(#cp32)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 3105,1035 -->
+<polygon points=" 3060,790 3105,1015 3150,790 3105,835 3060,790"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp33">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 4995,2140 4950,2185 4905,2140 4932,2403 4968,2403z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 3285,1395 3285,1575 4950,1845 4950,2385" clip-path="url(#cp33)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 4950,2385 -->
+<polygon points=" 4905,2140 4950,2365 4995,2140 4950,2185 4905,2140"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp34">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 1305,2140 1260,2185 1215,2140 1242,2403 1278,2403z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 2925,1395 2925,1575 1260,1845 1260,2385" clip-path="url(#cp34)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 1260,2385 -->
+<polygon points=" 1215,2140 1260,2365 1305,2140 1260,2185 1215,2140"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp35">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 2005,7605 2050,7650 2005,7695 2268,7668 2268,7632z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 315,7650 2250,7650" clip-path="url(#cp35)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 2250,7650 -->
+<polygon points=" 2005,7695 2230,7650 2005,7605 2050,7650 2005,7695"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp36">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 6955,6435 7000,6480 6955,6525 7218,6498 7218,6462z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 6075,6480 7200,6480" clip-path="url(#cp36)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 7200,6480 -->
+<polygon points=" 6955,6525 7180,6480 6955,6435 7000,6480 6955,6525"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Line -->
+<defs>
+<clipPath id="cp37">
+ <path clip-rule="evenodd" d="M 237,327 H 12363 V 9618 H 237 z
+ M 3895,8370 3940,8415 3895,8460 4158,8433 4158,8397z
+ M 873,3277 893,3240 930,3260 862,3092 842,3098z"/>
+</clipPath>
+</defs>
+<polyline points=" 2610,7830 3195,8415 4140,8415" clip-path="url(#cp37)"
+ stroke="#000000" stroke-width="30px" stroke-linejoin="round"/>
+<!-- Forward arrow to point 4140,8415 -->
+<polygon points=" 3895,8460 4120,8415 3895,8370 3940,8415 3895,8460"
+ stroke="#000000" stroke-width="8px" stroke-miterlimit="8" fill="#000000"/>
+<!-- Text -->
+<g transform="translate(12240,3060) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="108" text-anchor="middle">past</text>
+</g><!-- Text -->
+<g transform="translate(10440,3060) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="108" text-anchor="middle">future</text>
+</g><!-- Text -->
+<g transform="translate(8550,3060) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="108" text-anchor="middle">past</text>
+</g><!-- Text -->
+<g transform="translate(6750,3060) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="108" text-anchor="middle">future</text>
+</g><!-- Text -->
+<text xml:space="preserve" x="9450" y="5130" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">Oldest</text>
+<!-- Text -->
+<g transform="translate(405,3060) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="108" text-anchor="middle">newest</text>
+</g><!-- Text -->
+<g transform="translate(2205,3060) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="108" text-anchor="middle">oldest</text>
+</g><!-- Text -->
+<g transform="translate(4095,3060) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="108" text-anchor="middle">newest</text>
+</g><!-- Text -->
+<g transform="translate(5895,3060) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="108" text-anchor="middle">oldest</text>
+</g><!-- Text -->
+<text xml:space="preserve" x="9135" y="5850" fill="#000000" font-family="Courier" font-style="normal" font-weight="bold" font-size="120" text-anchor="start">runqueue-depth</text>
+<!-- Text -->
+<text xml:space="preserve" x="3195" y="5715" fill="#000000" font-family="Courier" font-style="normal" font-weight="bold" font-size="120" text-anchor="start">runqueue-depth</text>
+<!-- Text -->
+<text xml:space="preserve" x="9450" y="3600" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="144" text-anchor="middle">Time-based</text>
+<!-- Text -->
+<text xml:space="preserve" x="9450" y="3780" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="144" text-anchor="middle">Wait queues</text>
+<!-- Text -->
+<text xml:space="preserve" x="9000" y="4005" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="144" text-anchor="start">- 1 global</text>
+<!-- Text -->
+<text xml:space="preserve" x="9000" y="4185" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="144" text-anchor="start">- 1 per thread</text>
+<!-- Text -->
+<text xml:space="preserve" x="3105" y="3600" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="144" text-anchor="middle">Priority-based</text>
+<!-- Text -->
+<text xml:space="preserve" x="3105" y="3780" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="144" text-anchor="middle">Run queues</text>
+<!-- Text -->
+<text xml:space="preserve" x="2655" y="4005" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="144" text-anchor="start">- 1 global</text>
+<!-- Text -->
+<text xml:space="preserve" x="2655" y="4185" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="144" text-anchor="start">- 1 per thread</text>
+<!-- Text -->
+<text xml:space="preserve" x="3240" y="585" fill="#000000" font-family="Courier" font-style="normal" font-weight="bold" font-size="120" text-anchor="start">task_wakeup()</text>
+<!-- Text -->
+<text xml:space="preserve" x="9585" y="630" fill="#000000" font-family="Courier" font-style="normal" font-weight="bold" font-size="120" text-anchor="start">task_schedule()</text>
+<!-- Text -->
+<text xml:space="preserve" x="9585" y="450" fill="#000000" font-family="Courier" font-style="normal" font-weight="bold" font-size="120" text-anchor="start">task_queue()</text>
+<!-- Text -->
+<text xml:space="preserve" x="315" y="7560" fill="#000000" font-family="Courier" font-style="normal" font-weight="bold" font-size="120" text-anchor="start">tasklet_wakeup()</text>
+<!-- Text -->
+<text xml:space="preserve" x="12285" y="7515" fill="#000000" font-family="Courier" font-style="normal" font-weight="bold" font-size="120" text-anchor="end">t-&gt;process()</text>
+<!-- Text -->
+<text xml:space="preserve" x="12285" y="7335" fill="#ff0000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="144" text-anchor="end">Run!</text>
+<!-- Text -->
+<text xml:space="preserve" x="10035" y="7515" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="108" text-anchor="start">37%</text>
+<!-- Text -->
+<text xml:space="preserve" x="10080" y="8955" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="108" text-anchor="start">=1</text>
+<!-- Text -->
+<text xml:space="preserve" x="5085" y="6255" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="120" text-anchor="middle">(accessed using atomic ops)</text>
+<!-- Text -->
+<text xml:space="preserve" x="10035" y="6795" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="108" text-anchor="start">50%</text>
+<!-- Text -->
+<text xml:space="preserve" x="10035" y="8235" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="108" text-anchor="start">13%</text>
+<!-- Text -->
+<g transform="translate(2745,8100) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="108" text-anchor="end">Yes</text>
+</g><!-- Text -->
+<g transform="translate(2520,7650) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">Local ?</text>
+</g><!-- Text -->
+<g transform="translate(2700,7110) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="108" text-anchor="start">No</text>
+</g><!-- Text -->
+<text xml:space="preserve" x="4725" y="8460" fill="#000000" font-family="Courier" font-style="normal" font-weight="bold" font-size="120" text-anchor="start">TASK_SELF_WAKING</text>
+<!-- Text -->
+<text xml:space="preserve" x="4725" y="8820" fill="#000000" font-family="Courier" font-style="normal" font-weight="bold" font-size="120" text-anchor="start">TASK_HEAVY</text>
+<!-- Text -->
+<text xml:space="preserve" x="4725" y="8010" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="120" text-anchor="start">(default)</text>
+<!-- Text -->
+<text xml:space="preserve" x="4725" y="7650" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="120" text-anchor="start">In I/O or signals</text>
+<!-- Text -->
+<g transform="translate(10815,7695) rotate(-90)" >
+<text xml:space="preserve" x="0" y="0" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="middle">Most Urgent</text>
+</g><!-- Text -->
+<text xml:space="preserve" x="9990" y="6480" fill="#ff0000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="start">order</text>
+<!-- Text -->
+<text xml:space="preserve" x="9990" y="6300" fill="#ff0000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="120" text-anchor="start">Scan</text>
+<!-- Text -->
+<text xml:space="preserve" x="6030" y="9450" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="bold" font-size="144" text-anchor="middle">5 class-based tasklet queues per thread (one accessible from remote threads)</text>
+<!-- Line -->
+<polygon points=" 7234,6838 9776,6838 9776,6398 7234,6398" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 7234,7558 9776,7558 9776,7118 7234,7118" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 7234,8278 9776,8278 9776,7838 7234,7838" fill="#dae8fc"/>
+<!-- Line -->
+<polygon points=" 7234,8998 9776,8998 9776,8558 7234,8558" fill="#dae8fc"/>
+<!-- Line -->
+<defs>
+<polygon points=" 4166,6838 6094,6838 6094,6398 4166,6398" id="p24"/>
+<pattern id="tile24" patternUnits="userSpaceOnUse"
+ x="0" y="0" width="134" height="67">
+<g stroke-width="7.5" stroke="#a7ceb3" fill="none">
+<path d="M -7,30 73,70 M 61,-3 141,37 M -7,37 73,-3 M 61,70 141,30"/>
+</g>
+</pattern>
+</defs>
+<use xlink:href="#p24" fill="#bbf2e2"/>
+<use xlink:href="#p24" fill="url(#tile24)"/>
+<!-- Line -->
+<polyline points=" 7234,6398 9776,6398 9776,6838 7234,6838"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 7234,7118 9776,7118 9776,7558 7234,7558"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 7234,8558 9776,8558 9776,8998 7234,8998"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 4166,6398 6094,6398 6094,6838 4166,6838"
+ stroke="#868286" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 7234,7838 9776,7838 9776,8278 7234,8278"
+ stroke="#458dba" stroke-width="45px"/>
+<!-- Line -->
+<polyline points=" 9613,6398 9613,6838"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9438,6398 9438,6838"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9264,6398 9264,6838"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9613,7118 9613,7558"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9438,7118 9438,7558"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9264,7118 9264,7558"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9613,7838 9613,8278"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9438,7838 9438,8278"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9264,7838 9264,8278"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9613,8558 9613,8998"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9438,8558 9438,8998"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 9264,8558 9264,8998"
+ stroke="#458dba" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 5923,6398 5923,6838"
+ stroke="#868286" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 5748,6398 5748,6838"
+ stroke="#868286" stroke-width="15px"/>
+<!-- Line -->
+<polyline points=" 5574,6398 5574,6838"
+ stroke="#868286" stroke-width="15px"/>
+<!-- Text -->
+<text xml:space="preserve" x="8460" y="6705" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="192" text-anchor="middle">TL_URGENT</text>
+<!-- Text -->
+<text xml:space="preserve" x="8460" y="7425" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="192" text-anchor="middle">TL_NORMAL</text>
+<!-- Text -->
+<text xml:space="preserve" x="8460" y="8145" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="192" text-anchor="middle">TL_BULK</text>
+<!-- Text -->
+<text xml:space="preserve" x="8460" y="8865" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="192" text-anchor="middle">TL_HEAVY</text>
+<!-- Text -->
+<text xml:space="preserve" x="4950" y="6705" fill="#000000" font-family="AvantGarde" font-style="normal" font-weight="normal" font-size="192" text-anchor="middle">SHARED</text>
+</g>
+</svg>
diff --git a/doc/internals/ssl_cert.dia b/doc/internals/ssl_cert.dia
new file mode 100644
index 0000000..52496a1
--- /dev/null
+++ b/doc/internals/ssl_cert.dia
Binary files differ
diff --git a/doc/internals/stats-v2.txt b/doc/internals/stats-v2.txt
new file mode 100644
index 0000000..7d2ae76
--- /dev/null
+++ b/doc/internals/stats-v2.txt
@@ -0,0 +1,8 @@
+
+ Qcur Qmax Scur Smax Slim Scum Fin Fout Bin Bout Ereq Econ Ersp Sts Wght Act Bck EChk Down
+Frontend - - X maxX Y totX I O I O Q - - - - - - - -
+Server X maxX X maxX Y totX I O I O - C R S W A B E D
+Server X maxX X maxX Y totX I O I O - C R S W A B E D
+Server X maxX X maxX Y totX I O I O - C R S W A B E D
+Backend X maxX X maxX Y totX I O I O - C R S totW totA totB totE totD
+
diff --git a/doc/internals/stconn-close.txt b/doc/internals/stconn-close.txt
new file mode 100644
index 0000000..fe1ddca
--- /dev/null
+++ b/doc/internals/stconn-close.txt
@@ -0,0 +1,74 @@
+2023-05-23 - closing states on the stream endpoint descriptor
+
+This document deals with the current flags on the SE desc:
+
+ - SE_FL_ERR_PENDING: an error was met while sending, but some incoming data
+ might still be pending. This flag will be promoted to SE_FL_ERROR when the
+ SE_FL_EOI or SE_FL_EOS flags are set via the standard API (se_fl_set()).
+
+ - SE_FL_ERROR ("ERR"): an error was met, last data were received if any, and no
+ more progress will happen.
+
+ - SE_FL_EOI ("EOI"): the end of the input message was seen, without implying
+ an end of the connection nor the end of event reporting for this stream. For
+ example and end of HTTP request or response will set EOI, after which it's
+ still possible (in case of a request) to bring an abort or error. Said
+ differently, the expected end of the message was seen.
+
+ - SE_FL_EOS ("EOS"): the definitive end of the input data was detected. It may
+ result from an error, an abort, a connection shutdown, and no more receive
+ events will be reported.
+
+The different muxes (H1,H2,H3) can face slightly different situations due to
+the nature, properties, and limitations of their underlying protocols, and will
+set these 3 flags to best translate the lower layer's situation and report it
+to the upper layer:
+
+ +-----------+-----------------------------------------------------------------
+ |ERR EOS EOI| Description per mux
+ +-----------+-----------------------------------------------------------------
+ | 0 0 0 | all: transfer still in progress
+ +-----------+-----------------------------------------------------------------
+ | 0 0 1 | H1: end of message reached.
+ | | H2: "ES" flag seen on a frame.
+ | | H3: not set
+ +-----------+-----------------------------------------------------------------
+ | 0 1 0 | H1: not set (*1)
+ | | H2: not set (*2)
+ | | H3: RST received before FIN (client stops uploading)
+ +-----------+-----------------------------------------------------------------
+ | 0 1 1 | H1: end of message + read0, such as close response or aborted
+ | | request
+ | | H2: not set (*2)
+ | | H3: end of message reached (any subsequent RSTs are ignored)
+ +-----------+-----------------------------------------------------------------
+ | 1 0 0 | all: could be used to report a protocol error (ex: invalid chunk
+ | | encoding, forbidden response header seen from a server).
+ +-----------+-----------------------------------------------------------------
+ | 1 0 1 | all: could be used to report an internal error or a downstream
+ | | protocol error, such as a forbidden header in an HTX block
+ | | coming from the stream layer or the impossibility to encode
+ | | a message. Seems unused right now.
+ +-----------+-----------------------------------------------------------------
+ | 1 1 0 | H1: truncated client input data before response, or truncated
+ | | response from the server
+ | | H2: RST or read0 received before end of input message
+ | | H3: RST + STOP_SENDING before FIN
+ +-----------+-----------------------------------------------------------------
+ | 1 1 1 | H1: error face while sending after end of input message
+ | | H2: RST or read0 received after end of input message
+ | | H3: STOP_SENDING received after a frame with FIN
+ +-----------+-----------------------------------------------------------------
+
+*1: EOS alone is currently not set by H1, however this situation could best
+ describe an H1 upload that was interrupted by the client while receiving
+ an early response, a reused persistent server connection that delivered a
+ read0 immediately after the request was sent, or a truncated server
+ response (or possibly one in close mode when no C-L was advertised). Right
+ now these situations are always accompanied with an ERR flag in addition to
+ the EOS one.
+
+*2: H2 doesn't set EOS without ERR because currently the only ways to close a
+ stream in H2 are by resetting the stream (which conveys an error) or
+ closing the connection (which renders it unusable in both directions and
+ prevents from sending as well).
diff --git a/doc/internals/stream-sock-states.fig b/doc/internals/stream-sock-states.fig
new file mode 100644
index 0000000..79131e5
--- /dev/null
+++ b/doc/internals/stream-sock-states.fig
@@ -0,0 +1,535 @@
+#FIG 3.2 Produced by xfig version 2.0
+Portrait
+Center
+Metric
+A4
+100.00
+Single
+-2
+1200 2
+0 32 #8e8e8e
+6 2295 1260 2430 1395
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 2363 1328 68 68 2430 1328 2295 1328
+4 1 0 50 -1 18 5 0.0000 4 60 60 2363 1361 1\001
+-6
+6 1845 2295 1980 2430
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 1913 2363 68 68 1980 2363 1845 2363
+4 1 0 50 -1 18 5 0.0000 4 60 60 1913 2396 2\001
+-6
+6 2475 2340 2610 2475
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 2543 2408 68 68 2610 2408 2475 2408
+4 1 0 50 -1 18 5 0.0000 4 60 60 2543 2441 9\001
+-6
+6 2835 2610 2970 2745
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 2903 2678 68 68 2970 2678 2835 2678
+4 1 0 50 -1 18 5 0.0000 4 60 60 2903 2711 7\001
+-6
+6 3195 2025 3330 2160
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 3263 2093 68 68 3330 2093 3195 2093
+4 1 0 50 -1 18 5 0.0000 4 60 60 3263 2126 8\001
+-6
+6 2745 2160 2880 2295
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 2813 2228 68 68 2880 2228 2745 2228
+4 1 0 50 -1 18 5 0.0000 4 60 60 2813 2261 6\001
+-6
+6 990 2700 1125 2835
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 1058 2768 68 68 1125 2768 990 2768
+4 1 0 50 -1 18 5 0.0000 4 60 120 1058 2801 13\001
+-6
+6 1305 2970 1440 3105
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 1373 3038 68 68 1440 3038 1305 3038
+4 1 0 50 -1 18 5 0.0000 4 60 120 1373 3071 12\001
+-6
+6 3105 1710 3240 1845
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 3173 1778 68 68 3240 1778 3105 1778
+4 1 0 50 -1 18 5 0.0000 4 60 120 3173 1811 15\001
+-6
+6 4275 1260 4410 1395
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 1328 68 68 4410 1328 4275 1328
+4 1 0 50 -1 18 5 0.0000 4 60 60 4343 1361 1\001
+-6
+6 4275 1440 4410 1575
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 1508 68 68 4410 1508 4275 1508
+4 1 0 50 -1 18 5 0.0000 4 60 60 4343 1541 2\001
+-6
+6 4275 1620 4410 1755
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 1688 68 68 4410 1688 4275 1688
+4 1 0 50 -1 18 5 0.0000 4 60 60 4343 1721 3\001
+-6
+6 4275 1800 4410 1935
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 1868 68 68 4410 1868 4275 1868
+4 1 0 50 -1 18 5 0.0000 4 60 60 4343 1901 4\001
+-6
+6 3240 2835 3375 2970
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 3308 2903 68 68 3375 2903 3240 2903
+4 1 0 50 -1 18 5 0.0000 4 60 120 3308 2936 16\001
+-6
+6 2835 3015 2970 3150
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 2903 3083 68 68 2970 3083 2835 3083
+4 1 0 50 -1 18 5 0.0000 4 60 120 2903 3116 17\001
+-6
+6 2295 3195 2430 3330
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 2363 3263 68 68 2430 3263 2295 3263
+4 1 0 50 -1 18 5 0.0000 4 60 60 2363 3296 3\001
+-6
+6 1440 4815 1620 4995
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 1508 4883 68 68 1575 4883 1440 4883
+4 1 0 50 -1 18 5 0.0000 4 60 120 1508 4916 19\001
+-6
+6 1800 3960 1980 4140
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 1868 4028 68 68 1935 4028 1800 4028
+4 1 0 50 -1 18 5 0.0000 4 60 120 1868 4061 18\001
+-6
+6 4275 1980 4410 2115
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 2048 68 68 4410 2048 4275 2048
+4 1 0 50 -1 18 5 0.0000 4 60 60 4343 2081 5\001
+-6
+6 4275 2340 4410 2475
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 2408 68 68 4410 2408 4275 2408
+4 1 0 50 -1 18 5 0.0000 4 60 60 4343 2441 6\001
+-6
+6 4275 2520 4410 2655
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 2588 68 68 4410 2588 4275 2588
+4 1 0 50 -1 18 5 0.0000 4 60 60 4343 2621 7\001
+-6
+6 4275 2700 4410 2835
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 2768 68 68 4410 2768 4275 2768
+4 1 0 50 -1 18 5 0.0000 4 60 60 4343 2801 8\001
+-6
+6 4275 2880 4410 3015
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 2948 68 68 4410 2948 4275 2948
+4 1 0 50 -1 18 5 0.0000 4 60 60 4343 2981 9\001
+-6
+6 4275 3060 4410 3195
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 3128 68 68 4410 3128 4275 3128
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 3161 10\001
+-6
+6 4275 3240 4410 3375
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 3308 68 68 4410 3308 4275 3308
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 3341 11\001
+-6
+6 4275 3420 4410 3555
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 3488 68 68 4410 3488 4275 3488
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 3521 12\001
+-6
+6 4275 3600 4410 3735
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 3668 68 68 4410 3668 4275 3668
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 3701 13\001
+-6
+6 4275 3960 4410 4095
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 4028 68 68 4410 4028 4275 4028
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 4061 15\001
+-6
+6 4275 4140 4410 4275
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 4208 68 68 4410 4208 4275 4208
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 4241 16\001
+-6
+6 4275 4320 4410 4455
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 4388 68 68 4410 4388 4275 4388
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 4421 17\001
+-6
+6 4275 3780 4455 3960
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 3848 68 68 4410 3848 4275 3848
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 3881 14\001
+-6
+6 4275 4590 4455 4770
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 4658 68 68 4410 4658 4275 4658
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 4691 18\001
+-6
+6 4275 4770 4455 4950
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 4838 68 68 4410 4838 4275 4838
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 4871 19\001
+-6
+6 4275 4950 4455 5130
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 5018 68 68 4410 5018 4275 5018
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 5051 20\001
+-6
+6 1170 3690 1350 3870
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 1238 3758 68 68 1305 3758 1170 3758
+4 1 0 50 -1 18 5 0.0000 4 60 120 1238 3791 11\001
+-6
+6 1530 3555 1710 3735
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 1598 3623 68 68 1665 3623 1530 3623
+4 1 0 50 -1 18 5 0.0000 4 60 120 1598 3656 10\001
+-6
+6 720 4095 900 4275
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 788 4163 68 68 855 4163 720 4163
+4 1 0 50 -1 18 5 0.0000 4 60 120 788 4196 14\001
+-6
+6 855 3645 1035 3825
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 923 3713 68 68 990 3713 855 3713
+4 1 0 50 -1 18 5 0.0000 4 60 120 923 3746 21\001
+-6
+6 4275 5130 4455 5310
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 5198 68 68 4410 5198 4275 5198
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 5231 21\001
+-6
+6 2295 4140 2430 4275
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 2363 4208 68 68 2430 4208 2295 4208
+4 1 0 50 -1 18 5 0.0000 4 60 60 2363 4241 4\001
+-6
+6 2475 3870 2655 4050
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 2543 3938 68 68 2610 3938 2475 3938
+4 1 0 50 -1 18 5 0.0000 4 60 120 2543 3971 22\001
+-6
+6 4275 5310 4455 5490
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 4343 5378 68 68 4410 5378 4275 5378
+4 1 0 50 -1 18 5 0.0000 4 60 120 4343 5411 22\001
+-6
+6 2295 5625 2430 5760
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 2363 5693 68 68 2430 5693 2295 5693
+4 1 0 50 -1 18 5 0.0000 4 60 60 2363 5726 5\001
+-6
+6 2295 6480 2475 6660
+1 4 0 1 0 7 50 -1 -1 0.000 1 0.0000 2363 6548 68 68 2430 6548 2295 6548
+4 1 0 50 -1 18 5 0.0000 4 60 120 2363 6581 20\001
+-6
+1 2 0 1 0 6 50 -1 20 0.000 1 0.0000 1350 4612 225 112 1125 4612 1575 4612
+1 2 0 1 0 6 50 -1 20 0.000 1 0.0000 2250 1912 225 112 2025 1912 2475 1912
+1 2 0 1 0 7 50 -1 20 0.000 1 0.0000 1125 3487 225 112 900 3487 1350 3487
+1 2 0 1 0 7 50 -1 20 0.000 1 0.0000 2250 3712 225 112 2025 3712 2475 3712
+1 2 0 1 0 6 50 -1 20 0.000 1 0.0000 2250 2812 225 112 2025 2812 2475 2812
+1 2 0 1 0 7 50 -1 20 0.000 1 0.0000 3375 2362 225 112 3150 2362 3600 2362
+1 2 0 1 0 7 50 -1 20 0.000 1 0.0000 2250 1012 225 112 2025 1012 2475 1012
+1 2 0 1 0 6 50 -1 20 0.000 1 0.0000 2250 6232 225 112 2025 6232 2475 6232
+1 2 0 1 0 7 50 -1 20 0.000 1 0.0000 2250 5422 225 112 2025 5422 2475 5422
+1 2 0 1 0 7 50 -1 20 0.000 1 0.0000 2250 6997 225 112 2025 6997 2475 6997
+1 2 0 1 0 6 50 -1 20 0.000 1 0.0000 2250 4587 225 112 2025 4587 2475 4587
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2250 1125 2250 1800
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8910 5805 4500 5805
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 3
+ 6885 5900 6930 5990 6975 5810
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 3
+ 6885 6570 6930 6660 6975 6480
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 5310 5589 5310 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 5670 5589 5670 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 6030 5589 6030 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 6390 5589 6390 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 6750 5589 6750 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 7110 5589 7110 6921
+2 1 0 2 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 4950 5589 4950 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8910 6705 4500 6705
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2250 5535 2250 6120
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2250 6345 2250 6885
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 4500 5580 8910 5580 8910 6930 4500 6930 4500 5580
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 4500 5580 8910 5580 8910 6930 4500 6930 4500 5580
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8910 6030 4500 6030
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8910 6255 4500 6255
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8910 6480 4500 6480
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 5310 5589 5310 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 5670 5589 5670 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 6030 5589 6030 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 6390 5589 6390 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 6750 5589 6750 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 7110 5589 7110 6921
+2 1 0 2 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 4950 5589 4950 6921
+2 2 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 5
+ 4500 5580 8910 5580 8910 6930 4500 6930 4500 5580
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8865 5805 4500 5805
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8865 6030 4500 6030
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8865 6255 4500 6255
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8865 6480 4500 6480
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 5310 5589 5310 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 5670 5589 5670 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 6030 5589 6030 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 6390 5589 6390 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 6750 5589 6750 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 7110 5589 7110 6921
+2 1 0 2 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 4950 5589 4950 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8865 6705 4500 6705
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8865 5805 4500 5805
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8865 6030 4500 6030
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8865 6255 4500 6255
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8865 6480 4500 6480
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 5310 5589 5310 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 5670 5589 5670 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 6030 5589 6030 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 6390 5589 6390 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 6750 5589 6750 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 7110 5589 7110 6921
+2 1 0 2 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 4950 5589 4950 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8865 6705 4500 6705
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 3
+ 7605 5890 7650 5980 7695 5800
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 3
+ 7605 6570 7650 6660 7695 6480
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 7470 5589 7470 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 3
+ 7965 5890 8010 5980 8055 5800
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 3
+ 7965 6570 8010 6660 8055 6480
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 7830 5589 7830 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 3
+ 8325 6570 8370 6660 8415 6480
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8190 5589 8190 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8550 5589 8550 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8190 5589 8190 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8550 5589 8550 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8190 5589 8190 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8550 5589 8550 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8190 5589 8190 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 2
+ 8550 5589 8550 6921
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 1
+ 4500 5805
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 1
+ 4500 6030
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 1
+ 4500 6255
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 1
+ 4500 6480
+2 1 0 1 0 7 50 -1 -1 0.000 0 0 -1 0 0 1
+ 4500 6705
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 2250 2700 2475 2475 2475 2250 2250 2025
+ 0.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 3375 2250 2925 2025 2475 1935
+ 0.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 3375 2475 3375 2700 2475 2835
+ 0.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 3420 2475 3420 4320 3150 5850 2475 6165
+ 0.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 1125 3375 1125 2925 2025 2790
+ 0.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 1125 3375 1125 2250 2025 1935
+ 0.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 6
+ 1 1 1.00 60.00 120.00
+ 2475 1890 3825 1800 3825 2520 3825 4500 3150 6075 2475 6210
+ 0.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 2250 2025 2025 2250 2025 2475 2250 2700
+ 0.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2250 3825 2250 4500
+ 0.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 2475 1980 2880 2115 3150 2340
+ 0.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2250 2925 2250 3600
+ 0.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 2205 3825 2070 4140 1622 4221 1440 4500
+ 0.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 1350 4725 1350 4950 1485 5760 2025 6165
+ 0.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 7
+ 1 1 1.00 60.00 120.00
+ 1125 4590 720 4455 675 4050 675 3600 675 2250 1350 1800
+ 2025 1935
+ 0.000 1.000 1.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 3
+ 1 1 1.00 60.00 120.00
+ 1260 4500 1125 4320 1125 3600
+ 0.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 1350 4500 1440 3645 1575 3330 2070 2880
+ 0.000 1.000 1.000 0.000
+3 0 0 1 32 7 51 -1 -1 0.000 0 1 0 5
+ 1 1 1.00 60.00 120.00
+ 1035 3600 990 4365 990 5040 1395 5895 2025 6210
+ 0.000 1.000 1.000 1.000 0.000
+3 0 0 1 32 7 51 -1 -1 0.000 0 1 0 5
+ 1 1 1.00 60.00 120.00
+ 2340 3825 2385 4005 2925 4275 2655 4815 2295 5310
+ 0.000 1.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 4
+ 1 1 1.00 60.00 120.00
+ 2475 2835 3150 3375 3150 5625 2475 6120
+ 0.000 1.000 1.000 0.000
+3 0 0 1 0 7 50 -1 -1 0.000 0 1 0 2
+ 1 1 1.00 60.00 120.00
+ 2250 4725 2250 5310
+ 0.000 0.000
+4 0 0 50 -1 14 6 0.0000 4 105 2880 4500 1710 ASS-CON: ssui(): connect_server() == SN_ERR_NONE\001
+4 0 0 50 -1 14 6 0.0000 4 90 540 4500 1350 INI-REQ: \001
+4 0 0 50 -1 14 6 0.0000 4 120 3720 4500 1530 REQ-ASS: prepare_conn_request(): srv_redispatch_connect() == 0\001
+4 0 4 50 -1 14 10 0.0000 4 105 105 2475 2700 4\001
+4 0 4 50 -1 14 10 0.0000 4 105 105 1620 4500 6\001
+4 0 0 50 -1 14 6 0.0000 4 120 3360 4500 1890 CON-EST: sess_update_st_con_tcp(): !timeout && !conn_err\001
+4 0 0 50 -1 14 6 0.0000 4 105 2460 4500 3510 TAR-ASS: ssui(): SI_FL_EXP && SN_ASSIGNED\001
+4 0 0 50 -1 14 6 0.0000 4 105 3420 4500 2970 ASS-REQ: connect_server: conn_retries == 0 && PR_O_REDISP\001
+4 0 0 50 -1 14 6 0.0000 4 120 2460 4500 2610 QUE-REQ: ssui(): !pend_pos && SN_ASSIGNED\001
+4 0 0 50 -1 14 6 0.0000 4 120 2520 4500 2790 QUE-REQ: ssui(): !pend_pos && !SN_ASSIGNED\001
+4 0 0 50 -1 14 6 0.0000 4 120 3300 4500 4230 QUE-CLO: ssui(): pend_pos && (SI_FL_EXP || req_aborted)\001
+4 0 0 50 -1 14 6 0.0000 4 105 2520 4500 3690 TAR-REQ: ssui(): SI_FL_EXP && !SN_ASSIGNED\001
+4 0 0 50 -1 14 6 0.0000 4 120 3960 4500 4545 ASS-CLO: PR_O_REDISP && SN_REDIRECTABLE && perform_http_redirect()\001
+4 0 0 50 -1 14 6 0.0000 4 120 4440 4500 2430 REQ-QUE: prepare_conn_request(): srv_redispatch_connect() != 0 (SI_ST_QUE)\001
+4 0 0 50 -1 14 6 0.0000 4 120 4200 4500 4050 REQ-CLO: prepare_conn_request(): srv_redispatch_connect() != 0 (error)\001
+4 0 0 50 -1 14 6 0.0000 4 105 4320 4500 4410 ASS-CLO: ssui(): connect_server() == SN_ERR_INTERNAL || conn_retries < 0\001
+4 0 0 50 -1 14 6 0.0000 4 120 3120 4500 4680 CON-CER: sess_update_st_con_tcp(): timeout/SI_FL_ERR\001
+4 0 0 50 -1 14 6 0.0000 4 120 3600 4500 4860 CER-CLO: sess_update_st_cer(): (ERR/EXP) && conn_retries < 0\001
+4 0 0 50 -1 14 6 0.0000 4 120 4200 4500 3870 CER-REQ: sess_update_st_cer(): timeout && !conn_retries && PR_O_REDISP\001
+4 0 0 50 -1 14 6 0.0000 4 120 3600 4500 3330 CER-TAR: sess_update_st_cer(): conn_err && conn_retries >= 0\001
+4 0 0 50 -1 14 6 0.0000 4 120 4620 4500 3150 CER-ASS: sess_update_st_cer(): timeout && (conn_retries >= 0 || !PR_O_REDISP)\001
+4 0 4 50 -1 14 10 0.0000 4 105 105 1305 3375 3\001
+4 0 4 50 -1 14 10 0.0000 4 105 105 2430 3600 5\001
+4 0 4 50 -1 14 10 0.0000 4 105 105 3555 2250 2\001
+4 0 4 50 -1 14 10 0.0000 4 105 105 2430 1800 1\001
+4 0 4 50 -1 14 10 0.0000 4 105 105 2430 900 0\001
+4 0 0 50 -1 14 6 0.0000 4 105 3000 4500 2070 EST-DIS: stream_sock_read/write/shutr/shutw: close\001
+4 0 0 50 -1 14 6 0.0000 4 120 1980 4500 2250 EST-DIS: process_session(): error\001
+4 0 0 50 -1 14 6 0.0000 4 120 2100 4500 5040 DIS-CLO: process_session(): cleanup\001
+4 1 0 50 -1 14 10 0.0000 4 105 315 1350 4680 CER\001
+4 1 0 50 -1 14 10 0.0000 4 135 315 2250 1980 REQ\001
+4 1 0 50 -1 14 10 0.0000 4 105 315 1125 3555 TAR\001
+4 1 0 50 -1 14 10 0.0000 4 105 315 2250 2880 ASS\001
+4 1 0 50 -1 14 10 0.0000 4 135 315 3375 2430 QUE\001
+4 1 0 50 -1 14 10 0.0000 4 105 315 2250 3780 CON\001
+4 1 0 50 -1 14 10 0.0000 4 105 315 2250 1080 INI\001
+4 0 0 50 -1 14 6 0.0000 4 120 2820 4500 5220 TAR-CLO: sess_update_stream_int(): client abort\001
+4 0 0 50 -1 14 6 0.0000 4 120 2820 4500 5400 CON-DIS: sess_update_st_con_tcp(): client abort\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 5130 5985 -\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 5490 5985 -\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 5850 5985 -\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 6210 5985 -\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 6570 5985 -\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 7290 5985 -\001
+4 1 0 50 -1 16 7 0.0000 4 105 120 4725 5985 fd\001
+4 1 0 50 -1 14 8 0.0000 4 90 225 5130 5760 INI\001
+4 1 0 50 -1 16 7 0.0000 4 105 270 4725 5760 state\001
+4 1 0 50 -1 14 8 0.0000 4 120 225 5490 5760 REQ\001
+4 1 0 50 -1 14 8 0.0000 4 120 225 5850 5760 QUE\001
+4 1 0 50 -1 14 8 0.0000 4 90 225 6210 5760 TAR\001
+4 1 0 50 -1 14 8 0.0000 4 90 225 6570 5760 ASS\001
+4 1 0 50 -1 14 8 0.0000 4 90 225 6930 5760 CON\001
+4 1 0 50 -1 14 8 0.0000 4 90 225 7290 5760 CER\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 5850 6210 0\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 5130 6210 0\001
+4 1 0 50 -1 16 7 0.0000 4 90 270 4725 6210 ERR\001
+4 1 0 50 -1 16 7 0.0000 4 90 270 4725 6435 EXP\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 5490 6210 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 6210 6210 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 6570 6210 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 6570 6435 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 5490 6435 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 5130 6435 0\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 5850 6435 0\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 6210 6435 0\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 7290 6435 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 6930 6435 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 7290 6210 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 6930 6210 X\001
+4 1 0 50 -1 16 7 0.0000 4 75 240 4725 6660 sess\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 5130 6660 -\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 5490 6660 -\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 5850 6660 -\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 6210 6660 -\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 6570 6660 -\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 7290 6660 -\001
+4 0 0 50 -1 16 6 0.0000 4 120 5970 675 7335 Note: states painted yellow above are transient ; process_session() will never leave a stream interface in any of those upon return.\001
+4 1 0 50 -1 16 7 0.0000 4 90 330 4725 6840 SHUT\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 7290 6840 -\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 6930 6840 0\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 6570 6840 0\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 6210 6840 0\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 5850 6840 0\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 5490 6840 0\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 5130 6840 0\001
+4 1 0 50 -1 14 10 0.0000 4 105 315 2250 6300 DIS\001
+4 1 0 50 -1 14 10 0.0000 4 105 315 2250 5490 EST\001
+4 1 0 50 -1 14 10 0.0000 4 105 315 2250 7065 CLO\001
+4 1 0 50 -1 14 10 0.0000 4 105 315 2250 4635 RDY\001
+4 0 4 50 -1 14 10 0.0000 4 105 105 2430 4455 7\001
+4 0 4 50 -1 14 10 0.0000 4 105 105 2430 5310 8\001
+4 0 4 50 -1 14 10 0.0000 4 105 105 2385 6120 9\001
+4 0 4 50 -1 14 10 0.0000 4 105 210 2385 6840 10\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 7650 6210 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 7650 6435 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 7650 6840 0\001
+4 1 0 50 -1 14 8 0.0000 4 90 225 8010 5760 EST\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 8010 6210 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 8010 6435 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 8010 6840 0\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 8370 5985 -\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 8370 6210 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 8370 6435 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 225 8730 5760 CLO\001
+4 1 0 50 -1 14 8 0.0000 4 90 225 8370 5760 DIS\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 8730 5985 -\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 8730 6210 X\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 8730 6435 X\001
+4 1 0 50 -1 14 8 0.0000 4 15 75 8730 6660 -\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 8370 6840 1\001
+4 1 0 50 -1 14 8 0.0000 4 90 75 8730 6840 1\001
+4 1 0 50 -1 14 8 0.0000 4 90 225 7650 5760 RDY\001