diff options
Diffstat (limited to 'doc/design-thoughts')
| -rw-r--r-- | doc/design-thoughts/backends-v0.txt | 27 | ||||
| -rw-r--r-- | doc/design-thoughts/backends.txt | 125 | ||||
| -rw-r--r-- | doc/design-thoughts/be-fe-changes.txt | 74 | ||||
| -rw-r--r-- | doc/design-thoughts/binding-possibilities.txt | 167 | ||||
| -rw-r--r-- | doc/design-thoughts/config-language.txt | 262 | ||||
| -rw-r--r-- | doc/design-thoughts/connection-reuse.txt | 224 | ||||
| -rw-r--r-- | doc/design-thoughts/connection-sharing.txt | 31 | ||||
| -rw-r--r-- | doc/design-thoughts/dynamic-buffers.txt | 41 | ||||
| -rw-r--r-- | doc/design-thoughts/entities-v2.txt | 276 | ||||
| -rw-r--r-- | doc/design-thoughts/how-it-works.txt | 60 | ||||
| -rw-r--r-- | doc/design-thoughts/http2.txt | 277 | ||||
| -rw-r--r-- | doc/design-thoughts/http_load_time.url | 5 | ||||
| -rw-r--r-- | doc/design-thoughts/pool-debugging.txt | 243 | ||||
| -rw-r--r-- | doc/design-thoughts/rate-shaping.txt | 90 | ||||
| -rw-r--r-- | doc/design-thoughts/sess_par_sec.txt | 13 | ||||
| -rw-r--r-- | doc/design-thoughts/thread-group.txt | 528 |
16 files changed, 2443 insertions, 0 deletions
diff --git a/doc/design-thoughts/backends-v0.txt b/doc/design-thoughts/backends-v0.txt new file mode 100644 index 0000000..d350e22 --- /dev/null +++ b/doc/design-thoughts/backends-v0.txt @@ -0,0 +1,27 @@ +1 type générique "entité", avec les attributs suivants : + + - frontend *f + - l7switch *s + - backend *b + +des types spécifiques sont simplement des entités avec certains +de ces champs remplis et pas forcément tous : + + listen = f [s] b + frontend = f [s] + l7switch = s + backend = [s] b + +Ensuite, les traitements sont évalués dans l'ordre : + - listen -> s'il a des règles de l7, on les évalue, et potentiellement on branche vers d'autres listen, l7 ou back, ou on travaille avec le back local. + - frontend -> s'il a des règles de l7, on les évalue, et potentiellement on branche vers d'autres listen, l7 ou back + - l7switch -> on évalue ses règles, potentiellement on branche vers d'autres listen, l7 ou backends + - backend -> s'il a des règles l7, on les évalue (quitte à changer encore de backend) puis on traite. + +Les requêtes sont traitées dans l'ordre des chaînages f->s*->b, et les réponses doivent être +traitées dans l'ordre inverse b->s*->f. Penser aux réécritures de champs Host à l'aller et +Location en retour. + +D'autre part, prévoir des "profils" plutôt que des blocs de nouveaux paramètres par défaut. +Ca permettra d'avoir plein de jeux de paramètres par défaut à utiliser dans chacun de ces +types. diff --git a/doc/design-thoughts/backends.txt b/doc/design-thoughts/backends.txt new file mode 100644 index 0000000..d2474ce --- /dev/null +++ b/doc/design-thoughts/backends.txt @@ -0,0 +1,125 @@ +There has been a lot of confusion during the development because of the +backends and frontends. + +What we want : + +- being able to still use a listener as it has always been working + +- being able to write a rule stating that we will *change* the backend when we + match some pattern. Only one jump is allowed. + +- being able to write a "use_filters_from XXX" line stating that we will ignore + any filter in the current listener, and that those from XXX will be borrowed + instead. A warning would be welcome for options which will silently get + masked. This is used to factor configuration. + +- being able to write a "use_backend_from XXX" line stating that we will ignore + any server and timeout config in the current listener, and that those from + XXX will be borrowed instead. A warning would be welcome for options which + will silently get masked. This is used to factor configuration. + + + +Example : +--------- + + | # frontend HTTP/80 + | listen fe_http 1.1.1.1:80 + | use_filters_from default_http + | use_backend_from appli1 + | + | # frontend HTTPS/443 + | listen fe_https 1.1.1.1:443 + | use_filters_from default_https + | use_backend_from appli1 + | + | # frontend HTTP/8080 + | listen fe_http-dev 1.1.1.1:8080 + | reqadd "X-proto: http" + | reqisetbe "^Host: www1" appli1 + | reqisetbe "^Host: www2" appli2 + | reqisetbe "^Host: www3" appli-dev + | use_backend_from appli1 + | + | + | # filters default_http + | listen default_http + | reqadd "X-proto: http" + | reqisetbe "^Host: www1" appli1 + | reqisetbe "^Host: www2" appli2 + | + | # filters default_https + | listen default_https + | reqadd "X-proto: https" + | reqisetbe "^Host: www1" appli1 + | reqisetbe "^Host: www2" appli2 + | + | + | # backend appli1 + | listen appli1 + | reqidel "^X-appli1:.*" + | reqadd "Via: appli1" + | balance roundrobin + | cookie app1 + | server srv1 + | server srv2 + | + | # backend appli2 + | listen appli2 + | reqidel "^X-appli2:.*" + | reqadd "Via: appli2" + | balance roundrobin + | cookie app2 + | server srv1 + | server srv2 + | + | # backend appli-dev + | listen appli-dev + | reqadd "Via: appli-dev" + | use_backend_from appli2 + | + | + + +Now we clearly see multiple things : +------------------------------------ + + - a frontend can EITHER have filters OR reference a use_filter + + - a backend can EITHER have servers OR reference a use_backend + + - we want the evaluation to cross TWO levels per request. When a request is + being processed, it keeps track of its "frontend" side (where it came + from), and of its "backend" side (where the server-side parameters have + been found). + + - the use_{filters|backend} have nothing to do with how the request is + decomposed. + + +Conclusion : +------------ + + - a proxy is always its own frontend. It also has 2 parameters : + - "fi_prm" : pointer to the proxy holding the filters (itself by default) + - "be_prm" : pointer to the proxy holding the servers (itself by default) + + - a request has a frontend (fe) and a backend (be). By default, the backend + is initialized to the frontend. Everything related to the client side is + accessed through ->fe. Everything related to the server side is accessed + through ->be. + + - request filters are first called from ->fe then ->be. Since only the + filters can change ->be, it is possible to iterate the filters on ->be + only and stop when ->be does not change anymore. + + - response filters are first called from ->be then ->fe IF (fe != be). + + +When we parse the configuration, we immediately configure ->fi and ->be for +all proxies. + +Upon session creation, s->fe and s->be are initialized to the proxy. Filters +are executed via s->fe->fi_prm and s->be->fi_prm. Servers are found in +s->be->be_prm. + diff --git a/doc/design-thoughts/be-fe-changes.txt b/doc/design-thoughts/be-fe-changes.txt new file mode 100644 index 0000000..f242f8a --- /dev/null +++ b/doc/design-thoughts/be-fe-changes.txt @@ -0,0 +1,74 @@ +- PR_O_TRANSP => FE !!! devra peut-être changer vu que c'est un complément du mode dispatch. +- PR_O_NULLNOLOG => FE +- PR_O_HTTP_CLOSE => FE. !!! mettre BE aussi !!! +- PR_O_TCP_CLI_KA => FE + +- PR_O_FWDFOR => BE. FE aussi ? +- PR_O_FORCE_CLO => BE +- PR_O_PERSIST => BE +- PR_O_COOK_RW, PR_O_COOK_INS, PR_O_COOK_PFX, PR_O_COOK_POST => BE +- PR_O_COOK_NOC, PR_O_COOK_IND => BE +- PR_O_ABRT_CLOSE => BE +- PR_O_REDISP => BE +- PR_O_BALANCE, PR_O_BALANCE_RR, PR_O_BALANCE_SH => BE +- PR_O_CHK_CACHE => BE +- PR_O_TCP_SRV_KA => BE +- PR_O_BIND_SRC => BE +- PR_O_TPXY_MASK => BE + + +- PR_MODE_TCP : BE côté serveur, FE côté client + +- nbconn -> fe->nbconn, be->nbconn. + Pb: rendre impossible le fait que (fe == be) avant de faire ça, + sinon on va compter les connexions en double. Ce ne sera possible + que lorsque les FE et BE seront des entités distinctes. On va donc + commencer par laisser uniquement fe->nbconn (vu que le fe ne change + pas), et modifier ceci plus tard, ne serait-ce que pour prendre en + compte correctement les minconn/maxconn. + => solution : avoir beconn et feconn dans chaque proxy. + +- failed_conns, failed_secu (réponses bloquées), failed_resp... : be + Attention: voir les cas de ERR_SRVCL, il semble que parfois on + indique ça alors qu'il y a un write error côté client (ex: ligne + 2044 dans proto_http). + + => be et pas be->beprm + +- logs du backup : ->be (idem) + +- queue : be + +- logs/debug : srv toujours associé à be (ex: proxy->id:srv->id). Rien + pour le client pour le moment. D'une manière générale, les erreurs + provoquées côté serveur vont sur BE et celles côté client vont sur + FE. +- logswait & LW_BYTES : FE (puisqu'on veut savoir si on logue tout de suite) + +- messages d'erreurs personnalisés (errmsg, ...) -> fe + +- monitor_uri -> fe +- uri_auth -> (fe->firpm puis be->fiprm). Utilisation de ->be + +- req_add, req_exp => fe->fiprm, puis be->fiprm +- req_cap, rsp_cap -> fe->fiprm +- rsp_add, rsp_exp => be->fiprm, devrait être fait ensuite aussi sur fe->fiprm +- capture_name, capture_namelen : fe->fiprm + + Ce n'est pas la solution idéale, mais au moins la capture et configurable + par les filtres du FE et ne bouge pas lorsque le BE est réassigné. Cela + résoud aussi un pb d'allocation mémoire. + + +- persistance (appsessions, cookiename, ...) -> be +- stats:scope "." = fe (celui par lequel on arrive) + !!!ERREUR!!! => utiliser be pour avoir celui qui a été validé par + l'uri_auth. + + +--------- corrections à effectuer --------- + +- remplacement de headers : parser le header et éventuellement le supprimer puis le(les) rajouter. +- session->proto.{l4state,l7state,l7substate} pour CLI et SRV +- errorloc : si définie dans backend, la prendre, sinon dans front. +- logs : faire be sinon fe. diff --git a/doc/design-thoughts/binding-possibilities.txt b/doc/design-thoughts/binding-possibilities.txt new file mode 100644 index 0000000..3f5e432 --- /dev/null +++ b/doc/design-thoughts/binding-possibilities.txt @@ -0,0 +1,167 @@ +2013/10/10 - possibilities for setting source and destination addresses + + +When establishing a connection to a remote device, this device is designated +as a target, which designates an entity defined in the configuration. A same +target appears only once in a configuration, and multiple targets may share +the same settings if needed. + +The following types of targets are currently supported : + + - listener : all connections with this type of target come from clients ; + - server : connections to such targets are for "server" lines ; + - peer : connections to such target address "peer" lines in "peers" + sections ; + - proxy : these targets are used by "dispatch", "option transparent" + or "option http_proxy" statements. + +A connection might not be reused between two different targets, even if all +parameters seem similar. One of the reason is that some parameters are specific +to the target and are not easy or not cheap to compare (eg: bind to interface, +mss, ...). + +A number of source and destination addresses may be set for a given target. + + - listener : + - the "from" address:port is set by accept() + + - the "to" address:port is set if conn_get_to_addr() is called + + - peer : + - the "from" address:port is not set + + - the "to" address:port is static and dependent only on the peer + + - server : + - the "from" address may be set alone when "source" is used with + a forced IP address, or when "usesrc clientip" is used. + + - the "from" port may be set only combined with the address when + "source" is used with IP:port, IP:port-range or "usesrc client" is + used. Note that in this case, both the address and the port may be + 0, meaning that the kernel will pick the address or port and that + the final value might not match the one explicitly set (eg: + important for logging). + + - the "from" address may be forced from a header which implies it + may change between two consecutive requests on the same connection. + + - the "to" address and port are set together when connecting to a + regular server, or by copying the client's IP address when + "server 0.0.0.0" is used. Note that the destination port may be + an offset applied to the original destination port. + + - proxy : + - the "from" address may be set alone when "source" is used with a + forced IP address or when "usesrc clientip" is used. + + - the "from" port may be set only combined with the address when + "source" is used with IP:port or with "usesrc client". There is + no ip:port range for a proxy as of now. Same comment applies as + above when port and/or address are 0. + + - the "from" address may be forced from a header which implies it + may change between two consecutive requests on the same connection. + + - the "to" address and port are set together, either by configuration + when "dispatch" is used, or dynamically when "transparent" is used + (1:1 with client connection) or "option http_proxy" is used, where + each client request may lead to a different destination address. + + +At the moment, there are some limits in what might happen between multiple +concurrent requests to a same target. + + - peers parameter do not change, so no problem. + + - server parameters may change in this way : + - a connection may require a source bound to an IP address found in a + header, which will fall back to the "source" settings if the address + is not found in this header. This means that the source address may + switch between a dynamically forced IP address and another forced + IP and/or port range. + + - if the element is not found (eg: header), the remaining "forced" + source address might very well be empty (unset), so the connection + reuse is acceptable when switching in that direction. + + - it is not possible to switch between client and clientip or any of + these and hdr_ip() because they're exclusive. + + - using a source address/port belonging to a port range is compatible + with connection reuse because there is a single range per target, so + switching from a range to another range means we remain in the same + range. + + - destination address may currently not change since the only possible + case for dynamic destination address setting is the transparent mode, + reproducing the client's destination address. + + - proxy parameters may change in this way : + - a connection may require a source bound to an IP address found in a + header, which will fall back to the "source" settings if the address + is not found in this header. This means that the source address may + switch between a dynamically forced IP address and another forced + IP and/or port range. + + - if the element is not found (eg: header), the remaining "forced" + source address might very well be empty (unset), so the connection + reuse is acceptable when switching in that direction. + + - it is not possible to switch between client and clientip or any of + these and hdr_ip() because they're exclusive. + + - proxies do not support port ranges at the moment. + + - destination address might change in the case where "option http_proxy" + is used. + +So, for each source element (IP, port), we want to know : + - if the element was assigned by static configuration (eg: ":80") + - if the element was assigned from a connection-specific value (eg: usesrc clientip) + - if the element was assigned from a configuration-specific range (eg: 1024-65535) + - if the element was assigned from a request-specific value (eg: hdr_ip(xff)) + - if the element was not assigned at all + +For the destination, we want to know : + - if the element was assigned by static configuration (eg: ":80") + - if the element was assigned from a connection-specific value (eg: transparent) + - if the element was assigned from a request-specific value (eg: http_proxy) + +We don't need to store the information about the origin of the dynamic value +since we have the value itself. So in practice we have : + - default value, unknown (not yet checked with getsockname/getpeername) + - default value, known (check done) + - forced value (known) + - forced range (known) + +We can't do that on an ip:port basis because the port may be fixed regardless +of the address and conversely. + +So that means : + + enum { + CO_ADDR_NONE = 0, /* not set, unknown value */ + CO_ADDR_KNOWN = 1, /* not set, known value */ + CO_ADDR_FIXED = 2, /* fixed value, known */ + CO_ADDR_RANGE = 3, /* from assigned range, known */ + } conn_addr_values; + + unsigned int new_l3_src_status:2; + unsigned int new_l4_src_status:2; + unsigned int new_l3_dst_status:2; + unsigned int new_l4_dst_status:2; + + unsigned int cur_l3_src_status:2; + unsigned int cur_l4_src_status:2; + unsigned int cur_l3_dsp_status:2; + unsigned int cur_l4_dst_status:2; + + unsigned int new_family:2; + unsigned int cur_family:2; + +Note: this obsoletes CO_FL_ADDR_FROM_SET and CO_FL_ADDR_TO_SET. These flags +must be changed to individual l3+l4 checks ORed between old and new values, +or better, set to cur only which will inherit new. + +In the connection, these values may be merged in the same word as err_code. diff --git a/doc/design-thoughts/config-language.txt b/doc/design-thoughts/config-language.txt new file mode 100644 index 0000000..20c4fbd --- /dev/null +++ b/doc/design-thoughts/config-language.txt @@ -0,0 +1,262 @@ +Prévoir des commandes en plusieurs mots clés. +Par exemple : + + timeout connection XXX + connection scale XXX + +On doit aussi accepter les préfixes : + + tim co XXX + co sca XXX + +Prévoir de ranger les combinaisons dans un tableau. On doit même +pouvoir effectuer un mapping simplifiant le parseur. + + +Pour les filtres : + + + <direction> <where> <what> <operator> <pattern> <action> [ <args>* ] + + <direction> = [ req | rsp ] + <where> = [ in | out ] + <what> = [ line | LINE | METH | URI | h(hdr) | H(hdr) | c(cookie) | C(cookie) ] + <operator> = [ == | =~ | =* | =^ | =/ | != | !~ | !* | !^ | !/ ] + <pattern> = "<string>" + <action> = [ allow | permit | deny | delete | replace | switch | add | set | redir ] + <args> = optional action args + + examples: + + req in URI =^ "/images" switch images + req in h(host) =* ".mydomain.com" switch mydomain + req in h(host) =~ "localhost(.*)" replace "www\1" + + alternative : + + <direction> <where> <action> [not] <what> [<operator> <pattern> [ <args>* ]] + + req in switch URI =^ "/images" images + req in switch h(host) =* ".mydomain.com" mydomain + req in replace h(host) =~ "localhost(.*)" "www\1" + req in delete h(Connection) + req in deny not line =~ "((GET|HEAD|POST|OPTIONS) /)|(OPTIONS *)" + req out set h(Connection) "close" + req out add line "Server: truc" + + + <direction> <action> <where> [not] <what> [<operator> <pattern> [ <args>* ]] ';' <action2> <what2> + + req in switch URI =^ "/images/" images ; replace "/" + req in switch h(host) =* ".mydomain.com" mydomain + req in replace h(host) =~ "localhost(.*)" "www\1" + req in delete h(Connection) + req in deny not line =~ "((GET|HEAD|POST|OPTIONS) /)|(OPTIONS *)" + req out set h(Connection) "close" + req out add line == "Server: truc" + + +Extension avec des ACL : + + req in acl(meth_valid) METH =~ "(GET|POST|HEAD|OPTIONS)" + req in acl(meth_options) METH == "OPTIONS" + req in acl(uri_slash) URI =^ "/" + req in acl(uri_star) URI == "*" + + req in deny acl !(meth_options && uri_star || meth_valid && uri_slash) + +Peut-être plus simplement : + + acl meth_valid METH =~ "(GET|POST|HEAD|OPTIONS)" + acl meth_options METH == "OPTIONS" + acl uri_slash URI =^ "/" + acl uri_star URI == "*" + + req in deny not acl(meth_options uri_star, meth_valid uri_slash) + + req in switch URI =^ "/images/" images ; replace "/" + req in switch h(host) =* ".mydomain.com" mydomain + req in replace h(host) =~ "localhost(.*)" "www\1" + req in delete h(Connection) + req in deny not line =~ "((GET|HEAD|POST|OPTIONS) /)|(OPTIONS *)" + req out set h(Connection) "close" + req out add line == "Server: truc" + +Prévoir le cas du "if" pour exécuter plusieurs actions : + + req in if URI =^ "/images/" then replace "/" ; switch images + +Utiliser les noms en majuscules/minuscules pour indiquer si on veut prendre +en compte la casse ou non : + + if uri =^ "/watch/" setbe watch rebase "/watch/" "/" + if uri =* ".jpg" setbe images + if uri =~ ".*dll.*" deny + if HOST =* ".mydomain.com" setbe mydomain + etc... + +Another solution would be to have a dedicated keyword to URI remapping. It +would both rewrite the URI and optionally switch to another backend. + + uriremap "/watch/" "/" watch + uriremap "/chat/" "/" chat + uriremap "/event/" "/event/" event + +Or better : + + uriremap "/watch/" watch "/" + uriremap "/chat/" chat "/" + uriremap "/event/" event + +For the URI, using a regex is sometimes useful (eg: providing a set of possible prefixes. + + +Sinon, peut-être que le "switch" peut prendre un paramètre de mapping pour la partie matchée : + + req in switch URI =^ "/images/" images:"/" + + +2007/03/31 - Besoins plus précis. + +1) aucune extension de branchement ou autre dans les "listen", c'est trop complexe. + +Distinguer les données entrantes (in) et sortantes (out). + +Le frontend ne voit que les requetes entrantes et les réponses sortantes. +Le backend voir les requêtes in/out et les réponses in/out. +Le frontend permet les branchements d'ensembles de filtres de requêtes vers +d'autres. Le frontend et les ensembles de filtres de requêtes peuvent brancher +vers un backend. + +-----------+--------+----------+----------+---------+----------+ + \ Where | | | | | | + \______ | Listen | Frontend | ReqRules | Backend | RspRules | + \| | | | | | +Capability | | | | | | +-----------+--------+----------+----------+---------+----------+ +Frontend | X | X | | | | +-----------+--------+----------+----------+---------+----------+ +FiltReqIn | X | X | X | X | | +-----------+--------+----------+----------+---------+----------+ +JumpFiltReq| X | X | X | | | \ +-----------+--------+----------+----------+---------+----------+ > = ReqJump +SetBackend | X | X | X | | | / +-----------+--------+----------+----------+---------+----------+ +FiltReqOut | | | | X | | +-----------+--------+----------+----------+---------+----------+ +FiltRspIn | X | | | X | X | +-----------+--------+----------+----------+---------+----------+ +JumpFiltRsp| | | | X | X | +-----------+--------+----------+----------+---------+----------+ +FiltRspOut | | X | | X | X | +-----------+--------+----------+----------+---------+----------+ +Backend | X | | | X | | +-----------+--------+----------+----------+---------+----------+ + +En conclusion +------------- + +Il y a au moins besoin de distinguer 8 fonctionnalités de base : + - capacité à recevoir des connexions (frontend) + - capacité à filtrer les requêtes entrantes + - capacité à brancher vers un backend ou un ensemble de règles de requêtes + - capacité à filtrer les requêtes sortantes + - capacité à filtrer les réponses entrantes + - capacité à brancher vers un autre ensemble de règles de réponses + - capacité à filtrer la réponse sortante + - capacité à gérer des serveurs (backend) + +Remarque +-------- + - on a souvent besoin de pouvoir appliquer un petit traitement sur un ensemble + host/uri/autre. Le petit traitement peut consister en quelques filtres ainsi + qu'une réécriture du couple (host,uri). + + +Proposition : ACL + +Syntaxe : +--------- + + acl <name> <what> <operator> <value> ... + +Ceci créera une acl référencée sous le nom <name> qui sera validée si +l'application d'au moins une des valeurs <value> avec l'opérateur <operator> +sur le sujet <what> est validée. + +Opérateurs : +------------ + +Toujours 2 caractères : + + [=!][~=*^%/.] + +Premier caractère : + '=' : OK si test valide + '!' : OK si test échoué. + +Second caractère : + '~' : compare avec une regex + '=' : compare chaîne à chaîne + '*' : compare la fin de la chaîne (ex: =* ".mydomain.com") + '^' : compare le début de la chaîne (ex: =^ "/images/") + '%' : recherche une sous-chaîne + '/' : compare avec un mot entier en acceptant le '/' comme délimiteur + '.' : compare avec un mot entier en acceptant el '.' comme délimiteur + +Ensuite on exécute une action de manière conditionnelle si l'ensemble des ACLs +mentionnées sont validées (ou invalidées pour celles précédées d'un "!") : + + <what> <where> <action> on [!]<aclname> ... + + +Exemple : +--------- + + acl www_pub host =. www www01 dev preprod + acl imghost host =. images + acl imgdir uri =/ img + acl imagedir uri =/ images + acl msie h(user-agent) =% "MSIE" + + set_host "images" on www_pub imgdir + remap_uri "/img" "/" on www_pub imgdir + remap_uri "/images" "/" on www_pub imagedir + setbe images on imghost + reqdel "Cookie" on all + + + +Actions possibles : + + req {in|out} {append|delete|rem|add|set|rep|mapuri|rewrite|reqline|deny|allow|setbe|tarpit} + resp {in|out} {append|delete|rem|add|set|rep|maploc|rewrite|stsline|deny|allow} + + req in append <line> + req in delete <line_regex> + req in rem <header> + req in add <header> <new_value> + req in set <header> <new_value> + req in rep <header> <old_value> <new_value> + req in mapuri <old_uri_prefix> <new_uri_prefix> + req in rewrite <old_uri_regex> <new_uri> + req in reqline <old_req_regex> <new_req> + req in deny + req in allow + req in tarpit + req in setbe <backend> + + resp out maploc <old_location_prefix> <new_loc_prefix> + resp out stsline <old_sts_regex> <new_sts_regex> + +Les chaînes doivent être délimitées par un même caractère au début et à la fin, +qui doit être échappé s'il est présent dans la chaîne. Tout ce qui se trouve +entre le caractère de fin et les premiers espace est considéré comme des +options passées au traitement. Par exemple : + + req in rep host /www/i /www/ + req in rep connection /keep-alive/i "close" + +Il serait pratique de pouvoir effectuer un remap en même temps qu'un setbe. + +Captures: les séparer en in/out. Les rendre conditionnelles ? diff --git a/doc/design-thoughts/connection-reuse.txt b/doc/design-thoughts/connection-reuse.txt new file mode 100644 index 0000000..4eb22f6 --- /dev/null +++ b/doc/design-thoughts/connection-reuse.txt @@ -0,0 +1,224 @@ +2015/08/06 - server connection sharing + +Improvements on the connection sharing strategies +------------------------------------------------- + +4 strategies are currently supported : + - never + - safe + - aggressive + - always + +The "aggressive" and "always" strategies take into account the fact that the +connection has already been reused at least once or not. The principle is that +second requests can be used to safely "validate" connection reuse on newly +added connections, and that such validated connections may be used even by +first requests from other sessions. A validated connection is a connection +which has already been reused, hence proving that it definitely supports +multiple requests. Such connections are easy to verify : after processing the +response, if the txn already had the TX_NOT_FIRST flag, then it was not the +first request over that connection, and it is validated as safe for reuse. +Validated connections are put into a distinct list : server->safe_conns. + +Incoming requests with TX_NOT_FIRST first pick from the regular idle_conns +list so that any new idle connection is validated as soon as possible. + +Incoming requests without TX_NOT_FIRST only pick from the safe_conns list for +strategy "aggressive", guaranteeing that the server properly supports connection +reuse, or first from the safe_conns list, then from the idle_conns list for +strategy "always". + +Connections are always stacked into the list (LIFO) so that there are higher +changes to convert recent connections and to use them. This will first optimize +the likeliness that the connection works, and will avoid TCP metrics from being +lost due to an idle state, and/or the congestion window to drop and the +connection going to slow start mode. + + +Handling connections in pools +----------------------------- + +A per-server "pool-max" setting should be added to permit disposing unused idle +connections not attached anymore to a session for use by future requests. The +principle will be that attached connections are queued from the front of the +list while the detached connections will be queued from the tail of the list. + +This way, most reused connections will be fairly recent and detached connections +will most often be ignored. The number of detached idle connections in the lists +should be accounted for (pool_used) and limited (pool_max). + +After some time, a part of these detached idle connections should be killed. +For this, the list is walked from tail to head and connections without an owner +may be evicted. It may be useful to have a per-server pool_min setting +indicating how many idle connections should remain in the pool, ready for use +by new requests. Conversely, a pool_low metric should be kept between eviction +runs, to indicate the lowest amount of detached connections that were found in +the pool. + +For eviction, the principle of a half-life is appealing. The principle is +simple : over a period of time, half of the connections between pool_min and +pool_low should be gone. Since pool_low indicates how many connections were +remaining unused over a period, it makes sense to kill some of them. + +In order to avoid killing thousands of connections in one run, the purge +interval should be split into smaller batches. Let's call N the ratio of the +half-life interval and the effective interval. + +The algorithm consists in walking over them from the end every interval and +killing ((pool_low - pool_min) + 2 * N - 1) / (2 * N). It ensures that half +of the unused connections are killed over the half-life period, in N batches +of population/2N entries at most. + +Unsafe connections should be evicted first. There should be quite few of them +since most of them are probed and become safe. Since detached connections are +quickly recycled and attached to a new session, there should not be too many +detached connections in the pool, and those present there may be killed really +quickly. + +Another interesting point of pools is that when a pool-max is not null, then it +makes sense to automatically enable pretend-keep-alive on non-private connections +going to the server in order to be able to feed them back into the pool. With +the "aggressive" or "always" strategies, it can allow clients making a single +request over their connection to share persistent connections to the servers. + + + +2013/10/17 - server connection management and reuse + +Current state +------------- + +At the moment, a connection entity is needed to carry any address +information. This means in the following situations, we need a server +connection : + +- server is elected and the server's destination address is set + +- transparent mode is elected and the destination address is set from + the incoming connection + +- proxy mode is enabled, and the destination's address is set during + the parsing of the HTTP request + +- connection to the server fails and must be retried on the same + server using the same parameters, especially the destination + address (SN_ADDR_SET not removed) + + +On the accepting side, we have further requirements : + +- allocate a clean connection without a stream interface + +- incrementally set the accepted connection's parameters without + clearing it, and keep track of what is set (eg: getsockname). + +- initialize a stream interface in established mode + +- attach the accepted connection to a stream interface + + +This means several things : + +- the connection has to be allocated on the fly the first time it is + needed to store the source or destination address ; + +- the connection has to be attached to the stream interface at this + moment ; + +- it must be possible to incrementally set some settings on the + connection's addresses regardless of the connection's current state + +- the connection must not be released across connection retries ; + +- it must be possible to clear a connection's parameters for a + redispatch without having to detach/attach the connection ; + +- we need to allocate a connection without an existing stream interface + +So on the accept() side, it looks like this : + + fd = accept(); + conn = new_conn(); + get_some_addr_info(&conn->addr); + ... + si = new_si(); + si_attach_conn(si, conn); + si_set_state(si, SI_ST_EST); + ... + get_more_addr_info(&conn->addr); + +On the connect() side, it looks like this : + + si = new_si(); + while (!properly_connected) { + if (!(conn = si->end)) { + conn = new_conn(); + conn_clear(conn); + si_attach_conn(si, conn); + } + else { + if (connected) { + f = conn->flags & CO_FL_XPRT_TRACKED; + conn->flags &= ~CO_FL_XPRT_TRACKED; + conn_close(conn); + conn->flags |= f; + } + if (!correct_dest) + conn_clear(conn); + } + set_some_addr_info(&conn->addr); + si_set_state(si, SI_ST_CON); + ... + set_more_addr_info(&conn->addr); + conn->connect(); + if (must_retry) { + close_conn(conn); + } + } + +Note: we need to be able to set the control and transport protocols. +On outgoing connections, this is set once we know the destination address. +On incoming connections, this is set the earliest possible (once we know +the source address). + +The problem analysed below was solved on 2013/10/22 + +| ==> the real requirement is to know whether a connection is still valid or not +| before deciding to close it. CO_FL_CONNECTED could be enough, though it +| will not indicate connections that are still waiting for a connect to occur. +| This combined with CO_FL_WAIT_L4_CONN and CO_FL_WAIT_L6_CONN should be OK. +| +| Alternatively, conn->xprt could be used for this, but needs some careful checks +| (it's used by conn_full_close at least). +| +| Right now, conn_xprt_close() checks conn->xprt and sets it to NULL. +| conn_full_close() also checks conn->xprt and sets it to NULL, except +| that the check on ctrl is performed within xprt. So conn_xprt_close() +| followed by conn_full_close() will not close the file descriptor. +| Note that conn_xprt_close() is never called, maybe we should kill it ? +| +| Note: at the moment, it's problematic to leave conn->xprt to NULL before doing +| xprt_init() because we might end up with a pending file descriptor. Or at +| least with some transport not de-initialized. We might thus need +| conn_xprt_close() when conn_xprt_init() fails. +| +| The fd should be conditioned by ->ctrl only, and the transport layer by ->xprt. +| +| - conn_prepare_ctrl(conn, ctrl) +| - conn_prepare_xprt(conn, xprt) +| - conn_prepare_data(conn, data) +| +| Note: conn_xprt_init() needs conn->xprt so it's not a problem to set it early. +| +| One problem might be with conn_xprt_close() not being able to know if xprt_init() +| was called or not. That's where it might make sense to only set ->xprt during init. +| Except that it does not fly with outgoing connections (xprt_init is called after +| connect()). +| +| => currently conn_xprt_close() is only used by ssl_sock.c and decides whether +| to do something based on ->xprt_ctx which is set by ->init() from xprt_init(). +| So there is nothing to worry about. We just need to restore conn_xprt_close() +| and rely on ->ctrl to close the fd instead of ->xprt. +| +| => we have the same issue with conn_ctrl_close() : when is the fd supposed to be +| valid ? On outgoing connections, the control is set much before the fd... diff --git a/doc/design-thoughts/connection-sharing.txt b/doc/design-thoughts/connection-sharing.txt new file mode 100644 index 0000000..99be1cd --- /dev/null +++ b/doc/design-thoughts/connection-sharing.txt @@ -0,0 +1,31 @@ +2014/10/28 - Server connection sharing + +For HTTP/2 we'll have to use multiplexed connections to the servers and to +share them between multiple streams. We'll also have to do this for H/1, but +with some variations since H1 doesn't offer connection status verification. + +In order to validate that an idle connection is still usable, it is desirable +to periodically send health checks over it. Normally, idle connections are +meant to be heavily used, so there is no reason for having them idle for a long +time. Thus we have two possibilities : + + - either we time them out after some inactivity, this saves server resources ; + - or we check them after some inactivity. For this we can send the server- + side HTTP health check (only when the server uses HTTP checks), and avoid + using that to mark the server down, and instead consider the connection as + dead. + +For HTTP/2 we'll have to send pings periodically over these connections, so +it's worth considering a per-connection task to validate that the channel still +works. + +In the current model, a connection necessarily belongs to a session, so it's +not really possible to share them, at best they can be exchanged, but that +doesn't make much sense as it means that it could disturb parallel traffic. + +Thus we need to have a per-server list of idle connections and a max-idle-conn +setting to kill them when there are too many. In the case of H/1 it is also +advisable to consider that if a connection was created to pass a first non- +idempotent request while other idle connections were still existing, then a +connection will have to be killed in order not to exceed the limit. + diff --git a/doc/design-thoughts/dynamic-buffers.txt b/doc/design-thoughts/dynamic-buffers.txt new file mode 100644 index 0000000..564d868 --- /dev/null +++ b/doc/design-thoughts/dynamic-buffers.txt @@ -0,0 +1,41 @@ +2014/10/30 - dynamic buffer management + +Since HTTP/2 processing will significantly increase the need for buffering, it +becomes mandatory to be able to support dynamic buffer allocation. This also +means that at any moment some buffer allocation will fail and that a task or an +I/O operation will have to be paused for the time needed to allocate a buffer. + +There are 3 places where buffers are needed : + + - receive side of a stream interface. A connection notifies about a pending + recv() and the SI calls the receive function to put the data into a buffer. + Here the buffer will have to be picked from a pool first, and if the + allocation fails, the I/O will have to temporarily be disabled, the + connection will have to subscribe for buffer release notification to be + woken up once a buffer is available again. It's important to keep in mind + that buffer availability doesn't necessarily mean a desire to enable recv + again, just that recv is not paused anymore for resource reasons. + + - receive side of a stream interface when the other end point is an applet. + The applet wants to write into the buffer and for this the buffer needs to + be allocated as well. It is the same as above except that it is the applet + which is put to a pause. Since the applet might be at the core of the task + itself, it could become tricky to handle the situation correctly. Stats and + peers are in this situation. + + - Tx of a task : some tasks perform spontaneous writes to a buffer. Checks + are an example of this. The checks will have to be able to sleep while a + buffer is being awaited. + +One important point is that such pauses must not prevent the task from timing +out. There it becomes difficult because in the case of a time out, we could +want to emit a timeout error message and for this, require a buffer. So it is +important to keep the ability not to send messages upon error processing, and +to be able to give up and stop waiting for buffers. + +The refill mechanism needs to be designed in a thread-safe way because this +will become one of the rare cases of inter-task activity. Thus it is important +to ensure that checking the state of the task and passing of the freshly +released buffer are performed atomically, and that in case the task doesn't +want it anymore, it is responsible for passing it to the next one. + diff --git a/doc/design-thoughts/entities-v2.txt b/doc/design-thoughts/entities-v2.txt new file mode 100644 index 0000000..91c4fa9 --- /dev/null +++ b/doc/design-thoughts/entities-v2.txt @@ -0,0 +1,276 @@ +2012/07/05 - Connection layering and sequencing + + +An FD has a state : + - CLOSED + - READY + - ERROR (?) + - LISTEN (?) + +A connection has a state : + - CLOSED + - ACCEPTED + - CONNECTING + - ESTABLISHED + - ERROR + +A stream interface has a state : + - INI, REQ, QUE, TAR, ASS, CON, CER, EST, DIS, CLO + +Note that CON and CER might be replaced by EST if the connection state is used +instead. CON might even be more suited than EST to indicate that a connection +is known. + + +si_shutw() must do : + + data_shutw() + if (shutr) { + data_close() + ctrl_shutw() + ctrl_close() + } + +si_shutr() must do : + data_shutr() + if (shutw) { + data_close() + ctrl_shutr() + ctrl_close() + } + +Each of these steps may fail, in which case the step must be retained and the +operations postponed in an asynchronous task. + +The first asynchronous data_shut() might already fail so it is mandatory to +save the other side's status with the connection in order to let the async task +know whether the 3 next steps must be performed. + +The connection (or perhaps the FD) needs to know : + - the desired close operations : DSHR, DSHW, CSHR, CSHW + - the completed close operations : DSHR, DSHW, CSHR, CSHW + + +On the accept() side, we probably need to know : + - if a header is expected (eg: accept-proxy) + - if this header is still being waited for + => maybe both info might be combined into one bit + + - if a data-layer accept() is expected + - if a data-layer accept() has been started + - if a data-layer accept() has been performed + => possibly 2 bits, to indicate the need to free() + +On the connect() side, we need to know : + - the desire to send a header (eg: send-proxy) + - if this header has been sent + => maybe both info might be combined + + - if a data-layer connect() is expected + - if a data-layer connect() has been started + - if a data-layer connect() has been completed + => possibly 2 bits, to indicate the need to free() + +On the response side, we also need to know : + - the desire to send a header (eg: health check response for monitor-net) + - if this header was sent + => might be the same as sending a header over a new connection + +Note: monitor-net has precedence over proxy proto and data layers. Same for + health mode. + +For multi-step operations, use 2 bits : + 00 = operation not desired, not performed + 10 = operation desired, not started + 11 = operation desired, started but not completed + 01 = operation desired, started and completed + + => X != 00 ==> operation desired + X & 01 ==> operation at least started + X & 10 ==> operation not completed + +Note: no way to store status information for error reporting. + +Note2: it would be nice if "tcp-request connection" rules could work at the +connection level, just after headers ! This means support for tracking stick +tables, possibly not too much complicated. + + +Proposal for incoming connection sequence : + +- accept() +- if monitor-net matches or if mode health => try to send response +- if accept-proxy, wait for proxy request +- if tcp-request connection, process tcp rules and possibly keep the + pointer to stick-table +- if SSL is enabled, switch to SSL handshake +- then switch to DATA state and instantiate a session + +We just need a map of handshake handlers on the connection. They all manage the +FD status themselves and set the callbacks themselves. If their work succeeds, +they remove themselves from the list. If it fails, they remain subscribed and +enable the required polling until they are woken up again or the timeout strikes. + +Identified handshake handlers for incoming connections : + - HH_HEALTH (tries to send OK and dies) + - HH_MONITOR_IN (matches src IP and adds/removes HH_SEND_OK/HH_SEND_HTTP_OK) + - HH_SEND_OK (tries to send "OK" and dies) + - HH_SEND_HTTP_OK (tries to send "HTTP/1.0 200 OK" and dies) + - HH_ACCEPT_PROXY (waits for PROXY line and parses it) + - HH_TCP_RULES (processes TCP rules) + - HH_SSL_HS (starts SSL handshake) + - HH_ACCEPT_SESSION (instantiates a session) + +Identified handshake handlers for outgoing connections : + - HH_SEND_PROXY (tries to build and send the PROXY line) + - HH_SSL_HS (starts SSL handshake) + +For the pollers, we could check that handshake handlers are not 0 and decide to +call a generic connection handshake handler instead of usual callbacks. Problem +is that pollers don't know connections, they know fds. So entities which manage +handlers should update change the FD callbacks accordingly. + +With a bit of care, we could have : + - HH_SEND_LAST_CHUNK (sends the chunk pointed to by a pointer and dies) + => merges HEALTH, SEND_OK and SEND_HTTP_OK + +It sounds like the ctrl vs data state for the connection are per-direction +(eg: support an async ctrl shutw while still reading data). + +Also support shutr/shutw status at L4/L7. + +In practice, what we really need is : + +shutdown(conn) = + conn.data.shut() + conn.ctrl.shut() + conn.fd.shut() + +close(conn) = + conn.data.close() + conn.ctrl.close() + conn.fd.close() + +With SSL over Remote TCP (RTCP + RSSL) to reach the server, we would have : + + HTTP -> RTCP+RSSL connection <-> RTCP+RRAW connection -> TCP+SSL connection + +The connection has to be closed at 3 places after a successful response : + - DATA (RSSL over RTCP) + - CTRL (RTCP to close connection to server) + - SOCK (FD to close connection to second process) + +Externally, the connection is seen with very few flags : + - SHR + - SHW + - ERR + +We don't need a CLOSED flag as a connection must always be detached when it's closed. + +The internal status doesn't need to be exposed : + - FD allocated (Y/N) + - CTRL initialized (Y/N) + - CTRL connected (Y/N) + - CTRL handlers done (Y/N) + - CTRL failed (Y/N) + - CTRL shutr (Y/N) + - CTRL shutw (Y/N) + - DATA initialized (Y/N) + - DATA connected (Y/N) + - DATA handlers done (Y/N) + - DATA failed (Y/N) + - DATA shutr (Y/N) + - DATA shutw (Y/N) + +(note that having flags for operations needing to be completed might be easier) +-------------- + +Maybe we need to be able to call conn->fdset() and conn->fdclr() but it sounds +very unlikely since the only functions manipulating this are in the code of +the data/ctrl handlers. + +FDSET/FDCLR cannot be directly controlled by the stream interface since it also +depends on the DATA layer (WANT_READ/WANT_WRITE). + +But FDSET/FDCLR is probably controlled by who owns the connection (eg: DATA). + +Example: an SSL conn relies on an FD. The buffer is full, and wants the conn to +stop reading. It must not stop the FD itself. It is the read function which +should notice that it has nothing to do with a read wake-up, which needs to +disable reading. + +Conversely, when calling conn->chk_rcv(), the reader might get a WANT_READ or +even WANT_WRITE and adjust the FDs accordingly. + +------------------------ + +OK, the problem is simple : we don't manipulate the FD at the right level. +We should have : + ->connect(), ->chk_snd(), ->chk_rcv(), ->shutw(), ->shutr() which are + called from the upper layer (buffer) + ->recv(), ->send(), called from the lower layer + +Note that the SHR is *reported* by lower layer but can be forced by upper +layer. In this case it's like a delayed abort. The difficulty consists in +knowing the output data were correctly read. Probably we'd need to drain +incoming data past the active shutr(). + +The only four purposes of the top-down shutr() call are : + - acknowledge a shut read report : could probably be done better + - read timeout => disable reading : it's a delayed abort. We want to + report that the buffer is SHR, maybe even the connection, but the + FD clearly isn't. + - read abort due to error on the other side or desire to close (eg: + http-server-close) : delayed abort + - complete abort + +The active shutr() is problematic as we can't disable reading if we expect some +exchanges for data acknowledgement. We probably need to drain data only until +the shutw() has been performed and ACKed. + +A connection shut down for read would behave like this : + + 1) bidir exchanges + + 2) shutr() => read_abort_pending=1 + + 3) drain input, still send output + + 4) shutw() + + 5) drain input, wait for read0 or ack(shutw) + + 6) close() + +--------------------- 2012/07/05 ------------------- + +Communications must be performed this way : + + connection <-> channel <-> connection + +A channel is composed of flags and stats, and may store data in either a buffer +or a pipe. We need low-layer operations between sockets and buffers or pipes. +Right now we only support sockets, but later we might support remote sockets +and maybe pipes or shared memory segments. + +So we need : + + - raw_sock_to_buf() => receive raw data from socket into buffer + - raw_sock_to_pipe => receive raw data from socket into pipe (splice in) + - raw_sock_from_buf() => send raw data from buffer to socket + - raw_sock_from_pipe => send raw data from pipe to socket (splice out) + + - ssl_sock_to_buf() => receive ssl data from socket into buffer + - ssl_sock_to_pipe => receive ssl data from socket into a pipe (NULL) + - ssl_sock_from_buf() => send ssl data from buffer to socket + - ssl_sock_from_pipe => send ssl data from pipe to socket (NULL) + +These functions should set such status flags : + +#define ERR_IN 0x01 +#define ERR_OUT 0x02 +#define SHUT_IN 0x04 +#define SHUT_OUT 0x08 +#define EMPTY_IN 0x10 +#define FULL_OUT 0x20 + diff --git a/doc/design-thoughts/how-it-works.txt b/doc/design-thoughts/how-it-works.txt new file mode 100644 index 0000000..2d1cb89 --- /dev/null +++ b/doc/design-thoughts/how-it-works.txt @@ -0,0 +1,60 @@ +How it works ? (unfinished and inexact) + +For TCP and HTTP : + +- listeners create listening sockets with a READ callback pointing to the + protocol-specific accept() function. + +- the protocol-specific accept() function then accept()'s the connection and + instantiates a "server TCP socket" (which is dedicated to the client side), + and configures it (non_block, get_original_dst, ...). + +For TCP : +- in case of pure TCP, a request buffer is created, as well as a "client TCP + socket", which tries to connect to the server. + +- once the connection is established, the response buffer is allocated and + connected to both ends. + +- both sockets are set to "autonomous mode" so that they only wake up their + supervising session when they encounter a special condition (error or close). + + +For HTTP : +- in case of HTTP, a request buffer is created with the "HOLD" flag set and + a read limit to support header rewriting (may be this one will be removed + eventually because it's better to limit only to the buffer size and report + an error when rewritten data overflows) + +- a "flow analyzer" is attached to the buffer (or possibly multiple flow + analyzers). For the request, the flow analyzer is "http_lb_req". The flow + analyzer is a function which gets called when new data is present and + blocked. It has a timeout (request timeout). It can also be bypassed on + demand. + +- when the "http_lb_req" has received the whole request, it creates a client + socket with all the parameters needed to try to connect to the server. When + the connection establishes, the response buffer is allocated on the fly, + put to HOLD mode, and a an "http_lb_resp" flow analyzer is attached to the + buffer. + + +For client-side HTTPS : + +- the accept() function must completely instantiate a TCP socket + an SSL + reader. It is when the SSL session is complete that we call the + protocol-specific accept(), and create its buffer. + + + + +Conclusions +----------- + +- we need a generic TCP accept() function with a lot of flags set by the + listener, to tell it what info we need to get at the accept() time, and + what flags will have to be set on the socket. + +- once the TCP accept() function ends, it wakes up the protocol supervisor + which is in charge of creating the buffers, etc, switch states, etc... + diff --git a/doc/design-thoughts/http2.txt b/doc/design-thoughts/http2.txt new file mode 100644 index 0000000..c21ac10 --- /dev/null +++ b/doc/design-thoughts/http2.txt @@ -0,0 +1,277 @@ +2014/10/23 - design thoughts for HTTP/2 + +- connections : HTTP/2 depends a lot more on a connection than HTTP/1 because a + connection holds a compression context (headers table, etc...). We probably + need to have an h2_conn struct. + +- multiple transactions will be handled in parallel for a given h2_conn. They + are called streams in HTTP/2 terminology. + +- multiplexing : for a given client-side h2 connection, we can have multiple + server-side h2 connections. And for a server-side h2 connection, we can have + multiple client-side h2 connections. Streams circulate in N-to-N fashion. + +- flow control : flow control will be applied between multiple streams. Special + care must be taken so that an H2 client cannot block some H2 servers by + sending requests spread over multiple servers to the point where one server + response is blocked and prevents other responses from the same server from + reaching their clients. H2 connection buffers must always be empty or nearly + empty. The per-stream flow control needs to be respected as well as the + connection's buffers. It is important to implement some fairness between all + the streams so that it's not always the same which gets the bandwidth when + the connection is congested. + +- some clients can be H1 with an H2 server (is this really needed ?). Most of + the initial use case will be H2 clients to H1 servers. It is important to keep + in mind that H1 servers do not do flow control and that we don't want them to + block transfers (eg: post upload). + +- internal tasks : some H2 clients will be internal tasks (eg: health checks). + Some H2 servers will be internal tasks (eg: stats, cache). The model must be + compatible with this use case. + +- header indexing : headers are transported compressed, with a reference to a + static or a dynamic header, or a literal, possibly huffman-encoded. Indexing + is specific to the H2 connection. This means there is no way any binary data + can flow between both sides, headers will have to be decoded according to the + incoming connection's context and re-encoded according to the outgoing + connection's context, which can significantly differ. In order to avoid the + parsing trouble we currently face, headers will have to be clearly split + between name and value. It is worth noting that neither the incoming nor the + outgoing connections' contexts will be of any use while processing the + headers. At best we can have some shortcuts for well-known names that map + well to the static ones (eg: use the first static entry with same name), and + maybe have a few special cases for static name+value as well. Probably we can + classify headers in such categories : + + - static name + value + - static name + other value + - dynamic name + other value + + This will allow for better processing in some specific cases. Headers + supporting a single value (:method, :status, :path, ...) should probably + be stored in a single location with a direct access. That would allow us + to retrieve a method using hdr[METHOD]. All such indexing must be performed + while parsing. That also means that HTTP/1 will have to be converted to this + representation very early in the parser and possibly converted back to H/1 + after processing. + + Header names/values will have to be placed in a small memory area that will + inevitably get fragmented as headers are rewritten. An automatic packing + mechanism must be implemented so that when there's no more room, headers are + simply defragmented/packet to a new table and the old one is released. Just + like for the static chunks, we need to have a few such tables pre-allocated + and ready to be swapped at any moment. Repacking must not change any index + nor affect the way headers are compressed so that it can happen late after a + retry (send-name-header for example). + +- header processing : can still happen on a (header, value) basis. Reqrep/ + rsprep completely disappear and will have to be replaced with something else + to support renaming headers and rewriting url/path/... + +- push_promise : servers can push dummy requests+responses. They advertise + the stream ID in the push_promise frame indicating the associated stream ID. + This means that it is possible to initiate a client-server stream from the + information coming from the server and make the data flow as if the client + had made it. It's likely that we'll have to support two types of server + connections: those which support push and those which do not. That way client + streams will be distributed to existing server connections based on their + capabilities. It's important to keep in mind that PUSH will not be rewritten + in responses. + +- stream ID mapping : since the stream ID is per H2 connection, stream IDs will + have to be mapped. Thus a given stream is an entity with two IDs (one per + side). Or more precisely a stream has two end points, each one carrying an ID + when it ends on an HTTP2 connection. Also, for each stream ID we need to + quickly find the associated transaction in progress. Using a small quick + unique tree seems indicated considering the wide range of valid values. + +- frame sizes : frame have to be remapped between both sides as multiplexed + connections won't always have the same characteristics. Thus some frames + might be spliced and others will be sliced. + +- error processing : care must be taken to never break a connection unless it + is dead or corrupt at the protocol level. Stats counter must exist to observe + the causes. Timeouts are a great problem because silent connections might + die out of inactivity. Ping frames should probably be scheduled a few seconds + before the connection timeout so that an unused connection is verified before + being killed. Abnormal requests must be dealt with using RST_STREAM. + +- ALPN : ALPN must be observed on the client side, and transmitted to the server + side. + +- proxy protocol : proxy protocol makes little to no sense in a multiplexed + protocol. A per-stream equivalent will surely be needed if implementations + do not quickly generalize the use of Forward. + +- simplified protocol for local devices (eg: haproxy->varnish in clear and + without handshake, and possibly even with splicing if the connection's + settings are shared) + +- logging : logging must report a number of extra information such as the + stream ID, and whether the transaction was initiated by the client or by the + server (which can be deduced from the stream ID's parity). In case of push, + the number of the associated stream must also be reported. + +- memory usage : H2 increases memory usage by mandating use of 16384 bytes + frame size minimum. That means slightly more than 16kB of buffer in each + direction to process any frame. It will definitely have an impact on the + deployed maxconn setting in places using less than this (4..8kB are common). + Also, the header list is persistent per connection, so if we reach the same + size as the request, that's another 16kB in each direction, resulting in + about 48kB of memory where 8 were previously used. A more careful encoder + can work with a much smaller set even if that implies evicting entries + between multiple headers of the same message. + +- HTTP/1.0 should very carefully be transported over H2. Since there's no way + to pass version information in the protocol, the server could use some + features of HTTP/1.1 that are unsafe in HTTP/1.0 (compression, trailers, + ...). + +- host / :authority : ":authority" is the norm, and "host" will be absent when + H2 clients generate :authority. This probably means that a dummy Host header + will have to be produced internally from :authority and removed when passing + to H2 behind. This can cause some trouble when passing H2 requests to H1 + proxies, because there's no way to know if the request should contain scheme + and authority in H1 or not based on the H2 request. Thus a "proxy" option + will have to be explicitly mentioned on HTTP/1 server lines. One of the + problem that it creates is that it's not longer possible to pass H/1 requests + to H/1 proxies without an explicit configuration. Maybe a table of the + various combinations is needed. + + :scheme :authority host + HTTP/2 request present present absent + HTTP/1 server req absent absent present + HTTP/1 proxy req present present present + + So in the end the issue is only with H/2 requests passed to H/1 proxies. + +- ping frames : they don't indicate any stream ID so by definition they cannot + be forwarded to any server. The H2 connection should deal with them only. + +There's a layering problem with H2. The framing layer has to be aware of the +upper layer semantics. We can't simply re-encode HTTP/1 to HTTP/2 then pass +it over a framing layer to mux the streams, the frame type must be passed below +so that frames are properly arranged. Header encoding is connection-based and +all streams using the same connection will interact in the way their headers +are encoded. Thus the encoder *has* to be placed in the h2_conn entity, and +this entity has to know for each stream what its headers are. + +Probably that we should remove *all* headers from transported data and move +them on the fly to a parallel structure that can be shared between H1 and H2 +and consumed at the appropriate level. That means buffers only transport data. +Trailers have to be dealt with differently. + +So if we consider an H1 request being forwarded between a client and a server, +it would look approximately like this : + + - request header + body land into a stream's receive buffer + - headers are indexed and stripped out so that only the body and whatever + follows remain in the buffer + - both the header index and the buffer with the body stay attached to the + stream + - the sender can rebuild the whole headers. Since they're found in a table + supposed to be stable, it can rebuild them as many times as desired and + will always get the same result, so it's safe to build them into the trash + buffer for immediate sending, just as we do for the PROXY protocol. + - the upper protocol should probably provide a build_hdr() callback which + when called by the socket layer, builds this header block based on the + current stream's header list, ready to be sent. + - the socket layer has to know how many bytes from the headers are left to be + forwarded prior to processing the body. + - the socket layer needs to consume only the acceptable part of the body and + must not release the buffer if any data remains in it (eg: pipelining over + H1). This is already handled by channel->o and channel->to_forward. + - we could possibly have another optional callback to send a preamble before + data, that could be used to send chunk sizes in H1. The danger is that it + absolutely needs to be stable if it has to be retried. But it could + considerably simplify de-chunking. + +When the request is sent to an H2 server, an H2 stream request must be made +to the server, we find an existing connection whose settings are compatible +with our needs (eg: tls/clear, push/no-push), and with a spare stream ID. If +none is found, a new connection must be established, unless maxconn is reached. + +Servers must have a maxstream setting just like they have a maxconn. The same +queue may be used for that. + +The "tcp-request content" ruleset must apply to the TCP layer. But with HTTP/2 +that becomes impossible (and useless). We still need something like the +"tcp-request session" hook to apply just after the SSL handshake is done. + +It is impossible to defragment the body on the fly in HTTP/2. Since multiple +messages are interleaved, we cannot wait for all of them and block the head of +line. Thus if body analysis is required, it will have to use the stream's +buffer, which necessarily implies a copy. That means that with each H2 end we +necessarily have at least one copy. Sometimes we might be able to "splice" some +bytes from one side to the other without copying into the stream buffer (same +rules as for TCP splicing). + +In theory, only data should flow through the channel buffer, so each side's +connector is responsible for encoding data (H1: linear/chunks, H2: frames). +Maybe the same mechanism could be extrapolated to tunnels / TCP. + +Since we'd use buffers only for data (and for receipt of headers), we need to +have dynamic buffer allocation. + +Thus : +- Tx buffers do not exist. We allocate a buffer on the fly when we're ready to + send something that we need to build and that needs to be persistent in case + of partial send. H1 headers are built on the fly from the header table to a + temporary buffer that is immediately sent and whose amount of sent bytes is + the only information kept (like for PROXY protocol). H2 headers are more + complex since the encoding depends on what was successfully sent. Thus we + need to build them and put them into a temporary buffer that remains + persistent in case send() fails. It is possible to have a limited pool of + Tx buffers and refrain from sending if there is no more buffer available in + the pool. In that case we need a wake-up mechanism once a buffer is + available. Once the data are sent, the Tx buffer is then immediately recycled + in its pool. Note that no tx buffer being used (eg: for hdr or control) means + that we have to be able to serialize access to the connection and retry with + the same stream. It also means that a stream that times out while waiting for + the connector to read the second half of its request has to stay there, or at + least needs to be handled gracefully. However if the connector cannot read + the data to be sent, it means that the buffer is congested and the connection + is dead, so that probably means it can be killed. + +- Rx buffers have to be pre-allocated just before calling recv(). A connection + will first try to pick a buffer and disable reception if it fails, then + subscribe to the list of tasks waiting for an Rx buffer. + +- full Rx buffers might sometimes be moved around to the next buffer instead of + experiencing a copy. That means that channels and connectors must use the + same format of buffer, and that only the channel will have to see its + pointers adjusted. + +- Tx of data should be made as much as possible without copying. That possibly + means by directly looking into the connection buffer on the other side if + the local Tx buffer does not exist and the stream buffer is not allocated, or + even performing a splice() call between the two sides. One of the problem in + doing this is that it requires proper ordering of the operations (eg: when + multiple readers are attached to a same buffer). If the splitting occurs upon + receipt, there's no problem. If we expect to retrieve data directly from the + original buffer, it's harder since it contains various things in an order + which does not even indicate what belongs to whom. Thus possibly the only + mechanism to implement is the buffer permutation which guarantees zero-copy + and only in the 100% safe case. Also it's atomic and does not cause HOL + blocking. + +It makes sense to chose the frontend_accept() function right after the +handshake ended. It is then possible to check the ALPN, the SNI, the ciphers +and to accept to switch to the h2_conn_accept handler only if everything is OK. +The h2_conn_accept handler will have to deal with the connection setup, +initialization of the header table, exchange of the settings frames and +preparing whatever is needed to fire new streams upon receipt of unknown +stream IDs. Note: most of the time it will not be possible to splice() because +we need to know in advance the amount of bytes to write the header, and here it +will not be possible. + +H2 health checks must be seen as regular transactions/streams. The check runs a +normal client which seeks an available stream from a server. The server then +finds one on an existing connection or initiates a new H2 connection. The H2 +checks will have to be configurable for sharing streams or not. Another option +could be to specify how many requests can be made over existing connections +before insisting on getting a separate connection. Note that such separate +connections might end up stacking up once released. So probably that they need +to be recycled very quickly (eg: fix how many unused ones can exist max). + diff --git a/doc/design-thoughts/http_load_time.url b/doc/design-thoughts/http_load_time.url new file mode 100644 index 0000000..f178e46 --- /dev/null +++ b/doc/design-thoughts/http_load_time.url @@ -0,0 +1,5 @@ +Excellent paper about page load time for keepalive on/off, pipelining, +multiple host names, etc... + +http://www.die.net/musings/page_load_time/ + diff --git a/doc/design-thoughts/pool-debugging.txt b/doc/design-thoughts/pool-debugging.txt new file mode 100644 index 0000000..106e41c --- /dev/null +++ b/doc/design-thoughts/pool-debugging.txt @@ -0,0 +1,243 @@ +2022-02-22 - debugging options with pools + +Two goals: + - help developers spot bugs as early as possible + + - make the process more reliable in field, by killing sick ones as soon as + possible instead of letting them corrupt data, cause trouble, or even be + exploited. + +An allocated object may exist in 5 forms: + - in use: currently referenced and used by haproxy, 100% of its size are + dedicated to the application which can do absolutely anything with it, + but it may never touch anything before nor after that area. + + - in cache: the object is neither referenced nor used anymore, but it sits + in a thread's cache. The application may not touch it at all anymore, and + some parts of it could even be unmapped. Only the current thread may safely + reach it, though others might find/release it when under thread isolation. + The thread cache needs some LRU linking that may be stored anywhere, either + inside the area, or outside. The parts surrounding the <size> parts remain + invisible to the application layer, and can serve as a protection. + + - in shared cache: the object is neither referenced nor used anymore, but it + may be reached by any thread. Some parts of it could be unmapped. Any + thread may pick it but only one may find it, hence once grabbed, it is + guaranteed no other one will find it. The shared cache needs to set up a + linked list and a single pointer needs to be stored anywhere, either inside + or outside the area. The parts surrounding the <size> parts remain + invisible to the application layer, and can serve as a protection. + + - in the system's memory allocator: the object is not known anymore from + haproxy. It may be reassigned in parts or totally to other pools or other + subsystems (e.g. crypto library). Some or all of it may be unmapped. The + areas surrounding the <size> parts are also part of the object from the + library's point of view and may be delivered to other areas. Tampering + with these may cause any other part to malfunction in dirty ways. + + - in the OS only: the memory allocator gave it back to the OS. + +The following options need to be configurable: + - detect improper initialization: this is done by poisonning objects before + delivering them to the application. + + - help figure where an object was allocated when in use: a pointer to the + call place will help. Pointing to the last pool_free() as well for the + same reasons when dealing with a UAF. + + - detection of wrong pointer/pool when in use: a pointer to the pool before + or after the area will definitely help. + + - detection of overflows when in use: a canary at the end of the area + (closest possible to <size>) will definitely help. The pool above can do + that job. Ideally, we should fill some data at the end so that even + unaligned sizes can be checked (e.g. a buffer that gets a zero appended). + If we just align on 2 pointers, writing the same pointer twice at the end + may do the job, but we won't necessarily have our bytes. Thus a particular + end-of-string pattern would be useful (e.g. ff55aa01) to fill it. + + - detection of double free when in cache: similar to detection of wrong + pointer/pool when in use: the pointer at the end may simply be changed so + that it cannot match the pool anymore. By using a pointer to the caller of + the previous free() operation, we have the guarantee to see different + pointers, and this pointer can be inspected to figure where the object was + previously freed. An extra check may even distinguish a perfect double-free + (same caller) from just a wrong free (pointer differs from pool). + + - detection of late corruption when in cache: keeping a copy of the + checksum of the whole area upon free() will do the job, but requires one + extra storage area for the checksum. Filling the area with a pattern also + does the job and doesn't require extra storage, but it loses the contents + and can be a bit slower. Sometimes losing the contents can be a feature, + especially when trying to detect late reads. Probably that both need to + be implemented. Note that if contents are not strictly needed, storing a + checksum inside the area does the job. + + - preserve total contents in cache for debugging: losing some precious + information can be a problem. + + - pattern filling of the area helps detect use-after-free in read-only mode. + + - allocate cold first helps with both cases above. + +Uncovered: + - overflow/underflow when in cache/shared/libc: it belongs to use-after-free + pattern and such an error during regular use ought to be caught while the + object was still in use. + + - integrity when in libc: not under our control anymore, this is a libc + problem. + +Arbitrable: + - integrity when in shared cache: unlikely to happen only then if it could + have happened in the local cache. Shared cache not often used anymore, thus + probably not worth the effort + + - protection against double-free when in shared cache/libc: might be done for + a cheap price, probably worth being able to quickly tell that such an + object left the local cache (e.g. the mark points to the caller, but could + possibly just be incremented, hence still point to the same code location+1 + byte when released. Calls are 4 bytes min on RISC, 5 on x86 so we do have + some margin by having a caller's location be +0,+1,+2 or +3. + + - underflow when in use: hasn't been really needed over time but may change. + + - detection of late corruption when in shared cache: checksum or area filling + are possible, but is this as relevant as it used to considering the less + common use of the shared cache ? + +Design considerations: + - object allocation when in use must remain minimal + + - when in cache, there are 2 lists which the compiler expect to be at least + aligned each (e.g. if/when we start to use DWCAS). + + - the original "pool debugging" feature covers both pool tracking, double- + free detection, overflow detection and caller info at the cost of a single + pointer placed immediately after the area. + + - preserving the contents might be done by placing the cache links and the + shared cache's list outside of the area (either before or after). Placing + it before has the merit that the allocated object preserves the 4-ptr + alignment. But when a larger alignment is desired this often does not work + anymore. Placing it after requires some dynamic adjustment depending on the + object's size. If any protection is installed, this protection must be + placed before the links so that the list doesn't get randomly corrupted and + corrupts adjacent elements. Note that if protection is desired, the extra + waste is probably less critical. + + - a link to the last caller might have to be stored somewhere. Without + preservation the free() caller may be placed anywhere while the alloc() + caller may only be placed outside. With preservation, again the free() + caller may be placed either before the object or after the mark at the end. + There is no particular need that both share the same location though it may + help. Note that when debugging is enabled, the free() caller doesn't need + to be duplicated and can continue to serve as the double-free detection. + Thus maybe in the end we only need to store the caller to the last alloc() + but not the free() since if we want it it's available via the pool debug. + + - use-after-free detection: contents may be erased on free() and checked on + alloc(), but they can also be checksummed on free() and rechecked on + alloc(). In the latter case we need to store a checksum somewhere. Note + that with pure checksum we don't know what part was modified, but seeing + previous contents can be useful. + +Possibilities: + +1) Linked lists inside the area: + + V size alloc + ---+------------------------------+-----------------+-- + in use |##############################| (Pool) (Tracer) | + ---+------------------------------+-----------------+-- + + ---+--+--+------------------------+-----------------+-- + in cache |L1|L2|########################| (Caller) (Sum) | + ---+--+--+------------------------+-----------------+-- +or: + ---+--+--+------------------------+-----------------+-- + in cache |L1|L2|###################(sum)| (Caller) | + ---+--+--+------------------------+-----------------+-- + + ---+-+----------------------------+-----------------+-- + in global |N|XXXX########################| (Caller) | + ---+-+----------------------------+-----------------+-- + + +2) Linked lists before the the area leave room for tracer and pool before + the area, but the canary must remain at the end, however the area will + be more difficult to keep aligned: + + V head size alloc + ----+-+-+------------------------------+-----------------+-- + in use |T|P|##############################| (canary) | + ----+-+-+------------------------------+-----------------+-- + + --+-----+------------------------------+-----------------+-- + in cache |L1|L2|##############################| (Caller) (Sum) | + --+-----+------------------------------+-----------------+-- + + ------+-+------------------------------+-----------------+-- + in global |N|##############################| (Caller) | + ------+-+------------------------------+-----------------+-- + + +3) Linked lists at the end of the area, might be shared with extra data + depending on the state: + + V size alloc + ---+------------------------------+-----------------+-- + in use |##############################| (Pool) (Tracer) | + ---+------------------------------+-----------------+-- + + ---+------------------------------+--+--+-----------+-- + in cache |##############################|L1|L2| (Caller) (Sum) + ---+------------------------------+--+--+-----------+-- + + ---+------------------------------+-+---------------+-- + in global |##############################|N| (Caller) | + ---+------------------------------+-+---------------+-- + +This model requires a little bit of alignment at the end of the area, which is +not incompatible with pattern filling and/or checksumming: + - preserving the area for post-mortem analysis means nothing may be placed + inside. In this case it could make sense to always store the last releaser. + - detecting late corruption may be done either with filling or checksumming, + but the simple fact of assuming a risk of corruption that needs to be + chased means we must not store the lists nor caller inside the area. + +Some models imply dedicating some place when in cache: + - preserving contents forces the lists to be prefixed or appended, which + leaves unused places when in use. Thus we could systematically place the + pool pointer and the caller in this case. + + - if preserving contents is not desired, almost everything can be stored + inside when not in use. Then each situation's size should be calculated + so that the allocated size is known, and entries are filled from the + beginning while not in use, or after the size when in use. + + - if poisonning is requested, late corruption might be detected but then we + don't want the list to be stored inside at the risk of being corrupted. + +Maybe just implement a few models: + - compact/optimal: put l1/l2 inside + - detect late corruption: fill/sum, put l1/l2 out + - preserve contents: put l1/l2 out + - corruption+preserve: do not fill, sum out + - poisonning: not needed on free if pattern filling is done. + +try2: + - poison on alloc to detect missing initialization: yes/no + (note: nothing to do if filling done) + - poison on free to detect use-after-free: yes/no + (note: nothing to do if filling done) + - check on alloc for corruption-after-free: yes/no + If content-preserving => sum, otherwise pattern filling; in + any case, move L1/L2 out. + - check for overflows: yes/no: use a canary after the area. The + canary can be the pointer to the pool. + - check for alloc caller: yes/no => always after the area + - content preservation: yes/no + (disables filling, moves lists out) + - improved caller tracking: used to detect double-free, may benefit + from content-preserving but not only. diff --git a/doc/design-thoughts/rate-shaping.txt b/doc/design-thoughts/rate-shaping.txt new file mode 100644 index 0000000..ca09408 --- /dev/null +++ b/doc/design-thoughts/rate-shaping.txt @@ -0,0 +1,90 @@ +2010/01/24 - Design of multi-criteria request rate shaping. + +We want to be able to rate-shape traffic on multiple cirteria. For instance, we +may want to support shaping of per-host header requests, as well as per source. + +In order to achieve this, we will use checkpoints, one per criterion. Each of +these checkpoints will consist in a test, a rate counter and a queue. + +A request reaches the checkpoint and checks the counter. If the counter is +below the limit, it is updated and the request continues. If the limit is +reached, the request attaches itself into the queue and sleeps. The sleep time +is computed from the queue status, and updates the queue status. + +A task is dedicated to each queue. Its sole purpose is to be woken up when the +next task may wake up, to check the frequency counter, wake as many requests as +possible and update the counter. All the woken up requests are detached from +the queue. Maybe the task dedicated to the queue can be avoided and replaced +with all queued tasks's sleep counters, though this looks tricky. Or maybe it's +just the first request in the queue that should be responsible for waking up +other tasks, and not to forget to pass on this responsibility to next tasks if +it leaves the queue. + +The woken up request then goes on evaluating other criteria and possibly sleeps +again on another one. In the end, the task will have waited the amount of time +required to pass all checkpoints, and all checkpoints will be able to maintain +a permanent load of exactly their limit if enough streams flow through them. + +Since a request can only sleep in one queue at a time, it makes sense to use a +linked list element in each session to attach it to any queue. It could very +well be shared with the pendconn hooks which could then be part of the session. + +This mechanism could be used to rate-shape sessions and requests per backend +and per server. + +When rate-shaping on dynamic criteria, such as the source IP address, we have +to first extract the data pattern, then look it up in a table very similar to +the stickiness tables, but with a frequency counter. At the checkpoint, the +pattern is looked up, the entry created or refreshed, and the frequency counter +updated and checked. Then the request either goes on or sleeps as described +above, but if it sleeps, it's still in the checkpoint's queue, but with a date +computed from the criterion's status. + +This means that we need 3 distinct features : + + - optional pattern extraction + - per-pattern or per-queue frequency counter + - time-ordered queue with a task + +Based on past experiences with frequency counters, it does not appear very easy +to exactly compute sleep delays in advance for multiple requests. So most +likely we'll have to run per-criterion queues too, with only the head of the +queue holding a wake-up timeout. + +This finally leads us to the following : + + - optional pattern extraction + - per-pattern or per-queue frequency counter + - per-frequency counter queue + - head of the queue serves as a global queue timer. + +This brings us to a very flexible architecture : + - 1 list of rule-based checkpoints per frontend + - 1 list of rule-based checkpoints per backend + - 1 list of rule-based checkpoints per server + +Each of these lists have a lot of rules conditioned by ACLs, just like the +use-backend rules, except that all rules are evaluated in turn. + +Since we might sometimes just want to enable that without setting any limit and +just for enabling control in ACLs (or logging ?), we should probably try to +find a flexible way of declaring just a counter without a queue. + +These checkpoints could be of two types : + - rate-limit (described here) + - concurrency-limit (very similar with the counter and no timer). This + feature would require to keep track of all accounted criteria in a + request so that they can be released upon request completion. + +It should be possible to define a max of requests in the queue, above which a +503 is returned. The same applies for the max delay in the queue. We could have +it per-task (currently it's the connection timeout) and abort tasks with a 503 +when the delay is exceeded. + +Per-server connection concurrency could be converted to use this mechanism +which is very similar. + +The construct should be flexible enough so that the counters may be checked +from ACLs. That would allow to reject connections or switch to an alternate +backend when some limits are reached. + diff --git a/doc/design-thoughts/sess_par_sec.txt b/doc/design-thoughts/sess_par_sec.txt new file mode 100644 index 0000000..e936374 --- /dev/null +++ b/doc/design-thoughts/sess_par_sec.txt @@ -0,0 +1,13 @@ +Graphe des nombres de traitements par seconde unité de temps avec + - un algo linéaire et très peu coûteux unitairement (0.01 ut) + - un algo en log(2) et 5 fois plus coûteux (0.05 ut) + +set yrange [0:1] +plot [0:1000] 1/(1+0.01*x), 1/(1+0.05*log(x+1)/log(2)) + +Graphe de la latence induite par ces traitements en unités de temps : + +set yrange [0:1000] +plot [0:1000] x/(1+0.01*x), x/(1+0.05*log(x+1)/log(2)) + + diff --git a/doc/design-thoughts/thread-group.txt b/doc/design-thoughts/thread-group.txt new file mode 100644 index 0000000..f1aad7b --- /dev/null +++ b/doc/design-thoughts/thread-group.txt @@ -0,0 +1,528 @@ +Thread groups +############# + +2021-07-13 - first draft +========== + +Objective +--------- +- support multi-socket systems with limited cache-line bouncing between + physical CPUs and/or L3 caches + +- overcome the 64-thread limitation + +- Support a reasonable number of groups. I.e. if modern CPUs arrive with + core complexes made of 8 cores, with 8 CC per chip and 2 chips in a + system, it makes sense to support 16 groups. + + +Non-objective +------------- +- no need to optimize to the last possible cycle. I.e. some algos like + leastconn will remain shared across all threads, servers will keep a + single queue, etc. Global information remains global. + +- no stubborn enforcement of FD sharing. Per-server idle connection lists + can become per-group; listeners can (and should probably) be per-group. + Other mechanisms (like SO_REUSEADDR) can already overcome this. + +- no need to go beyond 64 threads per group. + + +Identified tasks +================ + +General +------- +Everywhere tid_bit is used we absolutely need to find a complement using +either the current group or a specific one. Thread debugging will need to +be extended as masks are extensively used. + + +Scheduler +--------- +The global run queue and global wait queue must become per-group. This +means that a task may only be queued into one of them at a time. It +sounds like tasks may only belong to a given group, but doing so would +bring back the original issue that it's impossible to perform remote wake +ups. + +We could probably ignore the group if we don't need to set the thread mask +in the task. the task's thread_mask is never manipulated using atomics so +it's safe to complement it with a group. + +The sleeping_thread_mask should become per-group. Thus possibly that a +wakeup may only be performed on the assigned group, meaning that either +a task is not assigned, in which case it be self-assigned (like today), +otherwise the tg to be woken up will be retrieved from the task itself. + +Task creation currently takes a thread mask of either tid_bit, a specific +mask, or MAX_THREADS_MASK. How to create a task able to run anywhere +(checks, Lua, ...) ? + +Profiling +--------- +There should be one task_profiling_mask per thread group. Enabling or +disabling profiling should be made per group (possibly by iterating). + +Thread isolation +---------------- +Thread isolation is difficult as we solely rely on atomic ops to figure +who can complete. Such operation is rare, maybe we could have a global +read_mostly flag containing a mask of the groups that require isolation. +Then the threads_want_rdv_mask etc can become per-group. However setting +and clearing the bits will become problematic as this will happen in two +steps hence will require careful ordering. + +FD +-- +Tidbit is used in a number of atomic ops on the running_mask. If we have +one fdtab[] per group, the mask implies that it's within the group. +Theoretically we should never face a situation where an FD is reported nor +manipulated for a remote group. + +There will still be one poller per thread, except that this time all +operations will be related to the current thread_group. No fd may appear +in two thread_groups at once, but we can probably not prevent that (e.g. +delayed close and reopen). Should we instead have a single shared fdtab[] +(less memory usage also) ? Maybe adding the group in the fdtab entry would +work, but when does a thread know it can leave it ? Currently this is +solved by running_mask and by update_mask. Having two tables could help +with this (each table sees the FD in a different group with a different +mask) but this looks overkill. + +There's polled_mask[] which needs to be decided upon. Probably that it +should be doubled as well. Note, polled_mask left fdtab[] for cacheline +alignment reasons in commit cb92f5cae4. + +If we have one fdtab[] per group, what *really* prevents from using the +same FD in multiple groups ? _fd_delete_orphan() and fd_update_events() +need to check for no-thread usage before closing the FD. This could be +a limiting factor. Enabling could require to wake every poller. + +Shouldn't we remerge fdinfo[] with fdtab[] (one pointer + one int/short, +used only during creation and close) ? + +Other problem, if we have one fdtab[] per TG, disabling/enabling an FD +(e.g. pause/resume on listener) can become a problem if it's not necessarily +on the current TG. We'll then need a way to figure that one. It sounds like +FDs from listeners and receivers are very specific and suffer from problems +all other ones under high load do not suffer from. Maybe something specific +ought to be done for them, if we can guarantee there is no risk of accidental +reuse (e.g. locate the TG info in the receiver and have a "MT" bit in the +FD's flags). The risk is always that a close() can result in instant pop-up +of the same FD on any other thread of the same process. + +Observations: right now fdtab[].thread_mask more or less corresponds to a +declaration of interest, it's very close to meaning "active per thread". It is +in fact located in the FD while it ought to do nothing there, as it should be +where the FD is used as it rules accesses to a shared resource that is not +the FD but what uses it. Indeed, if neither polled_mask nor running_mask have +a thread's bit, the FD is unknown to that thread and the element using it may +only be reached from above and not from the FD. As such we ought to have a +thread_mask on a listener and another one on connections. These ones will +indicate who uses them. A takeover could then be simplified (atomically set +exclusivity on the FD's running_mask, upon success, takeover the connection, +clear the running mask). Probably that the change ought to be performed on +the connection level first, not the FD level by the way. But running and +polled are the two relevant elements, one indicates userland knowledge, +the other one kernel knowledge. For listeners there's no exclusivity so it's +a bit different but the rule remains the same that we don't have to know +what threads are *interested* in the FD, only its holder. + +Not exact in fact, see FD notes below. + +activity +-------- +There should be one activity array per thread group. The dump should +simply scan them all since the cumuled values are not very important +anyway. + +applets +------- +They use tid_bit only for the task. It looks like the appctx's thread_mask +is never used (now removed). Furthermore, it looks like the argument is +*always* tid_bit. + +CPU binding +----------- +This is going to be tough. It will be needed to detect that threads overlap +and are not bound (i.e. all threads on same mask). In this case, if the number +of threads is higher than the number of threads per physical socket, one must +try hard to evenly spread them among physical sockets (e.g. one thread group +per physical socket) and start as many threads as needed on each, bound to +all threads/cores of each socket. If there is a single socket, the same job +may be done based on L3 caches. Maybe it could always be done based on L3 +caches. The difficulty behind this is the number of sockets to be bound: it +is not possible to bind several FDs per listener. Maybe with a new bind +keyword we can imagine to automatically duplicate listeners ? In any case, +the initially bound cpumap (via taskset) must always be respected, and +everything should probably start from there. + +Frontend binding +---------------- +We'll have to define a list of threads and thread-groups per frontend. +Probably that having a group mask and a same thread-mask for each group +would suffice. + +Threads should have two numbers: + - the per-process number (e.g. 1..256) + - the per-group number (1..64) + +The "bind-thread" lines ought to use the following syntax: + - bind 45 ## bind to process' thread 45 + - bind 1/45 ## bind to group 1's thread 45 + - bind all/45 ## bind to thread 45 in each group + - bind 1/all ## bind to all threads in group 1 + - bind all ## bind to all threads + - bind all/all ## bind to all threads in all groups (=all) + - bind 1/65 ## rejected + - bind 65 ## OK if there are enough + - bind 35-45 ## depends. Rejected if it crosses a group boundary. + +The global directive "nbthread 28" means 28 total threads for the process. The +number of groups will sub-divide this. E.g. 4 groups will very likely imply 7 +threads per group. At the beginning, the nbgroup should be manual since it +implies config adjustments to bind lines. + +There should be a trivial way to map a global thread to a group and local ID +and to do the opposite. + + +Panic handler + watchdog +------------------------ +Will probably depend on what's done for thread_isolate + +Per-thread arrays inside structures +----------------------------------- +- listeners have a thr_conn[] array, currently limited to MAX_THREADS. Should + we simply bump the limit ? +- same for servers with idle connections. +=> doesn't seem very practical. +- another solution might be to point to dynamically allocated arrays of + arrays (e.g. nbthread * nbgroup) or a first level per group and a second + per thread. +=> dynamic allocation based on the global number + +Other +----- +- what about dynamic thread start/stop (e.g. for containers/VMs) ? + E.g. if we decide to start $MANY threads in 4 groups, and only use + one, in the end it will not be possible to use less than one thread + per group, and at most 64 will be present in each group. + + +FD Notes +-------- + - updt_fd_polling() uses thread_mask to figure where to send the update, + the local list or a shared list, and which bits to set in update_mask. + This could be changed so that it takes the update mask in argument. The + call from the poller's fork would just have to broadcast everywhere. + + - pollers use it to figure whether they're concerned or not by the activity + update. This looks important as otherwise we could re-enable polling on + an FD that changed to another thread. + + - thread_mask being a per-thread active mask looks more exact and is + precisely used this way by _update_fd(). In this case using it instead + of running_mask to gauge a change or temporarily lock it during a + removal could make sense. + + - running should be conditioned by thread. Polled not (since deferred + or migrated). In this case testing thread_mask can be enough most of + the time, but this requires synchronization that will have to be + extended to tgid.. But migration seems a different beast that we shouldn't + care about here: if first performed at the higher level it ought to + be safe. + +In practice the update_mask can be dropped to zero by the first fd_delete() +as the only authority allowed to fd_delete() is *the* owner, and as soon as +all running_mask are gone, the FD will be closed, hence removed from all +pollers. This will be the only way to make sure that update_mask always +refers to the current tgid. + +However, it may happen that a takeover within the same group causes a thread +to read the update_mask late, while the FD is being wiped by another thread. +That other thread may close it, causing another thread in another group to +catch it, and change the tgid and start to update the update_mask. This means +that it would be possible for a thread entering do_poll() to see the correct +tgid, then the fd would be closed, reopened and reassigned to another tgid, +and the thread would see its bit in the update_mask, being confused. Right +now this should already happen when the update_mask is not cleared, except +that upon wakeup a migration would be detected and that would be all. + +Thus we might need to set the running bit to prevent the FD from migrating +before reading update_mask, which also implies closing on fd_clr_running() == 0 :-( + +Also even fd_update_events() leaves a risk of updating update_mask after +clearing running, thus affecting the wrong one. Probably that update_mask +should be updated before clearing running_mask there. Also, how about not +creating an update on a close ? Not trivial if done before running, unless +thread_mask==0. + +########################################################### + +Current state: + + +Mux / takeover / fd_delete() code ||| poller code +-------------------------------------------------|||--------------------------------------------------- + \|/ +mux_takeover(): | fd_set_running(): + if (fd_takeover()<0) | old = {running, thread}; + return fail; | new = {tid_bit, tid_bit}; + ... | +fd_takeover(): | do { + atomic_or(running, tid_bit); | if (!(old.thread & tid_bit)) + old = {running, thread}; | return -1; + new = {tid_bit, tid_bit}; | new = { running | tid_bit, old.thread } + if (owner != expected) { | } while (!dwcas({running, thread}, &old, &new)); + atomic_and(runnning, ~tid_bit); | + return -1; // fail | fd_clr_running(): + } | return atomic_and_fetch(running, ~tid_bit); + | + while (old == {tid_bit, !=0 }) | poll(): + if (dwcas({running, thread}, &old, &new)) { | if (!owner) + atomic_and(runnning, ~tid_bit); | continue; + return 0; // success | + } | if (!(thread_mask & tid_bit)) { + } | epoll_ctl_del(); + | continue; + atomic_and(runnning, ~tid_bit); | } + return -1; // fail | + | // via fd_update_events() +fd_delete(): | if (fd_set_running() != -1) { + atomic_or(running, tid_bit); | iocb(); + atomic_store(thread, 0); | if (fd_clr_running() == 0 && !thread_mask) + if (fd_clr_running(fd) = 0) | fd_delete_orphan(); + fd_delete_orphan(); | } + + +The idle_conns_lock prevents the connection from being *picked* and released +while someone else is reading it. What it does is guarantee that on idle +connections, the caller of the IOCB will not dereference the task's context +while the connection is still in the idle list, since it might be picked then +freed at the same instant by another thread. As soon as the IOCB manages to +get that lock, it removes the connection from the list so that it cannot be +taken over anymore. Conversely, the mux's takeover() code runs under that +lock so that if it frees the connection and task, this will appear atomic +to the IOCB. The timeout task (which is another entry point for connection +deletion) does the same. Thus, when coming from the low-level (I/O or timeout): + - task always exists, but ctx checked under lock validates; conn removal + from list prevents takeover(). + - t->context is stable, except during changes under takeover lock. So + h2_timeout_task may well run on a different thread than h2_io_cb(). + +Coming from the top: + - takeover() done under lock() clears task's ctx and possibly closes the FD + (unless some running remains present). + +Unlikely but currently possible situations: + - multiple pollers (up to N) may have an idle connection's FD being + polled, if the connection was passed from thread to thread. The first + event on the connection would wake all of them. Most of them would + see fdtab[].owner set (the late ones might miss it). All but one would + see that their bit is missing from fdtab[].thread_mask and give up. + However, just after this test, others might take over the connection, + so in practice if terribly unlucky, all but 1 could see their bit in + thread_mask just before it gets removed, all of them set their bit + in running_mask, and all of them call iocb() (sock_conn_iocb()). + Thus all of them dereference the connection and touch the subscriber + with no protection, then end up in conn_notify_mux() that will call + the mux's wake(). + + - multiple pollers (up to N-1) might still be in fd_update_events() + manipulating fdtab[].state. The cause is that the "locked" variable + is determined by atleast2(thread_mask) but that thread_mask is read + at a random instant (i.e. it may be stolen by another one during a + takeover) since we don't yet hold running to prevent this from being + done. Thus we can arrive here with thread_mask==something_else (1bit), + locked==0 and fdtab[].state assigned non-atomically. + + - it looks like nothing prevents h2_release() from being called on a + thread (e.g. from the top or task timeout) while sock_conn_iocb() + dereferences the connection on another thread. Those killing the + connection don't yet consider the fact that it's an FD that others + might currently be waking up on. + +################### + +pb with counter: + +users count doesn't say who's using the FD and two users can do the same +close in turn. The thread_mask should define who's responsible for closing +the FD, and all those with a bit in it ought to do it. + + +2021-08-25 - update with minimal locking on tgid value +========== + + - tgid + refcount at once using CAS + - idle_conns lock during updates + - update: + if tgid differs => close happened, thus drop update + otherwise normal stuff. Lock tgid until running if needed. + - poll report: + if tgid differs => closed + if thread differs => stop polling (migrated) + keep tgid lock until running + - test on thread_id: + if (xadd(&tgid,65536) != my_tgid) { + // was closed + sub(&tgid, 65536) + return -1 + } + if !(thread_id & tidbit) => migrated/closed + set_running() + sub(tgid,65536) + - note: either fd_insert() or the final close() ought to set + polled and update to 0. + +2021-09-13 - tid / tgroups etc. +========== + + - tid currently is the thread's global ID. It's essentially used as an index + for arrays. It must be clearly stated that it works this way. + + - tid_bit makes no sense process-wide, so it must be redefined to represent + the thread's tid within its group. The name is not much welcome though, but + there are 286 of it that are not going to be changed that fast. + + - just like "ti" is the thread_info, we need to have "tg" pointing to the + thread_group. + + - other less commonly used elements should be retrieved from ti->xxx. E.g. + the thread's local ID. + + - lock debugging must reproduce tgid + + - an offset might be placed in the tgroup so that even with 64 threads max + we could have completely separate tid_bits over several groups. + +2021-09-15 - bind + listen() + rx +========== + + - thread_mask (in bind_conf->rx_settings) should become an array of + MAX_TGROUP longs. + - when parsing "thread 123" or "thread 2/37", the proper bit is set, + assuming the array is either a contiguous bitfield or a tgroup array. + An option RX_O_THR_PER_GRP or RX_O_THR_PER_PROC is set depending on + how the thread num was parsed, so that we reject mixes. + - end of parsing: entries translated to the cleanest form (to be determined) + - binding: for each socket()/bind()/listen()... just perform one extra dup() + for each tgroup and store the multiple FDs into an FD array indexed on + MAX_TGROUP. => allows to use one FD per tgroup for the same socket, hence + to have multiple entries in all tgroup pollers without requiring the user + to duplicate the bind line. + +2021-09-15 - global thread masks +========== + +Some global variables currently expect to know about thread IDs and it's +uncertain what must be done with them: + - global_tasks_mask /* Mask of threads with tasks in the global runqueue */ + => touched under the rq lock. Change it per-group ? What exact use is made ? + + - sleeping_thread_mask /* Threads that are about to sleep in poll() */ + => seems that it can be made per group + + - all_threads_mask: a bit complicated, derived from nbthread and used with + masks and with my_ffsl() to wake threads up. Should probably be per-group + but we might miss something for global. + + - stopping_thread_mask: used in combination with all_threads_mask, should + move per-group. + + - threads_harmless_mask: indicates all threads that are currently harmless in + that they promise not to access a shared resource. Must be made per-group + but then we'll likely need a second stage to have the harmless groups mask. + threads_idle_mask, threads_sync_mask, threads_want_rdv_mask go with the one + above. Maybe the right approach will be to request harmless on a group mask + so that we can detect collisions and arbiter them like today, but on top of + this it becomes possible to request harmless only on the local group if + desired. The subtlety is that requesting harmless at the group level does + not mean it's achieved since the requester cannot vouch for the other ones + in the same group. + +In addition, some variables are related to the global runqueue: + __decl_aligned_spinlock(rq_lock); /* spin lock related to run queue */ + struct eb_root rqueue; /* tree constituting the global run queue, accessed under rq_lock */ + unsigned int grq_total; /* total number of entries in the global run queue, atomic */ + static unsigned int global_rqueue_ticks; /* insertion count in the grq, use rq_lock */ + +And others to the global wait queue: + struct eb_root timers; /* sorted timers tree, global, accessed under wq_lock */ + __decl_aligned_rwlock(wq_lock); /* RW lock related to the wait queue */ + struct eb_root timers; /* sorted timers tree, global, accessed under wq_lock */ + + +2021-09-29 - group designation and masks +========== + +Neither FDs nor tasks will belong to incomplete subsets of threads spanning +over multiple thread groups. In addition there may be a difference between +configuration and operation (for FDs). This allows to fix the following rules: + + group mask description + 0 0 bind_conf: groups & thread not set. bind to any/all + task: it would be nice to mean "run on the same as the caller". + + 0 xxx bind_conf: thread set but not group: thread IDs are global + FD/task: group 0, mask xxx + + G>0 0 bind_conf: only group is set: bind to all threads of group G + FD/task: mask 0 not permitted (= not owned). May be used to + mention "any thread of this group", though already covered by + G/xxx like today. + + G>0 xxx bind_conf: Bind to these threads of this group + FD/task: group G, mask xxx + +It looks like keeping groups starting at zero internally complicates everything +though. But forcing it to start at 1 might also require that we rescan all tasks +to replace 0 with 1 upon startup. This would also allow group 0 to be special and +be used as the default group for any new thread creation, so that group0.count +would keep the number of unassigned threads. Let's try: + + group mask description + 0 0 bind_conf: groups & thread not set. bind to any/all + task: "run on the same group & thread as the caller". + + 0 xxx bind_conf: thread set but not group: thread IDs are global + FD/task: invalid. Or maybe for a task we could use this to + mean "run on current group, thread XXX", which would cover + the need for health checks (g/t 0/0 while sleeping, 0/xxx + while running) and have wake_expired_tasks() detect 0/0 and + wake them up to a random group. + + G>0 0 bind_conf: only group is set: bind to all threads of group G + FD/task: mask 0 not permitted (= not owned). May be used to + mention "any thread of this group", though already covered by + G/xxx like today. + + G>0 xxx bind_conf: Bind to these threads of this group + FD/task: group G, mask xxx + +With a single group declared in the config, group 0 would implicitly find the +first one. + + +The problem with the approach above is that a task queued in one group+thread's +wait queue could very well receive a signal from another thread and/or group, +and that there is no indication about where the task is queued, nor how to +dequeue it. Thus it seems that it's up to the application itself to unbind/ +rebind a task. This contradicts the principle of leaving a task waiting in a +wait queue and waking it anywhere. + +Another possibility might be to decide that a task having a defined group but +a mask of zero is shared and will always be queued into its group's wait queue. +However, upon expiry, the scheduler would notice the thread-mask 0 and would +broadcast it to any group. + +Right now in the code we have: + - 18 calls of task_new(tid_bit) + - 18 calls of task_new(MAX_THREADS_MASK) + - 2 calls with a single bit + +Thus it looks like "task_new_anywhere()", "task_new_on()" and +"task_new_here()" would be sufficient. |