diff options
Diffstat (limited to 'doc/design-thoughts/entities-v2.txt')
| -rw-r--r-- | doc/design-thoughts/entities-v2.txt | 276 |
1 files changed, 276 insertions, 0 deletions
diff --git a/doc/design-thoughts/entities-v2.txt b/doc/design-thoughts/entities-v2.txt new file mode 100644 index 0000000..91c4fa9 --- /dev/null +++ b/doc/design-thoughts/entities-v2.txt @@ -0,0 +1,276 @@ +2012/07/05 - Connection layering and sequencing + + +An FD has a state : + - CLOSED + - READY + - ERROR (?) + - LISTEN (?) + +A connection has a state : + - CLOSED + - ACCEPTED + - CONNECTING + - ESTABLISHED + - ERROR + +A stream interface has a state : + - INI, REQ, QUE, TAR, ASS, CON, CER, EST, DIS, CLO + +Note that CON and CER might be replaced by EST if the connection state is used +instead. CON might even be more suited than EST to indicate that a connection +is known. + + +si_shutw() must do : + + data_shutw() + if (shutr) { + data_close() + ctrl_shutw() + ctrl_close() + } + +si_shutr() must do : + data_shutr() + if (shutw) { + data_close() + ctrl_shutr() + ctrl_close() + } + +Each of these steps may fail, in which case the step must be retained and the +operations postponed in an asynchronous task. + +The first asynchronous data_shut() might already fail so it is mandatory to +save the other side's status with the connection in order to let the async task +know whether the 3 next steps must be performed. + +The connection (or perhaps the FD) needs to know : + - the desired close operations : DSHR, DSHW, CSHR, CSHW + - the completed close operations : DSHR, DSHW, CSHR, CSHW + + +On the accept() side, we probably need to know : + - if a header is expected (eg: accept-proxy) + - if this header is still being waited for + => maybe both info might be combined into one bit + + - if a data-layer accept() is expected + - if a data-layer accept() has been started + - if a data-layer accept() has been performed + => possibly 2 bits, to indicate the need to free() + +On the connect() side, we need to know : + - the desire to send a header (eg: send-proxy) + - if this header has been sent + => maybe both info might be combined + + - if a data-layer connect() is expected + - if a data-layer connect() has been started + - if a data-layer connect() has been completed + => possibly 2 bits, to indicate the need to free() + +On the response side, we also need to know : + - the desire to send a header (eg: health check response for monitor-net) + - if this header was sent + => might be the same as sending a header over a new connection + +Note: monitor-net has precedence over proxy proto and data layers. Same for + health mode. + +For multi-step operations, use 2 bits : + 00 = operation not desired, not performed + 10 = operation desired, not started + 11 = operation desired, started but not completed + 01 = operation desired, started and completed + + => X != 00 ==> operation desired + X & 01 ==> operation at least started + X & 10 ==> operation not completed + +Note: no way to store status information for error reporting. + +Note2: it would be nice if "tcp-request connection" rules could work at the +connection level, just after headers ! This means support for tracking stick +tables, possibly not too much complicated. + + +Proposal for incoming connection sequence : + +- accept() +- if monitor-net matches or if mode health => try to send response +- if accept-proxy, wait for proxy request +- if tcp-request connection, process tcp rules and possibly keep the + pointer to stick-table +- if SSL is enabled, switch to SSL handshake +- then switch to DATA state and instantiate a session + +We just need a map of handshake handlers on the connection. They all manage the +FD status themselves and set the callbacks themselves. If their work succeeds, +they remove themselves from the list. If it fails, they remain subscribed and +enable the required polling until they are woken up again or the timeout strikes. + +Identified handshake handlers for incoming connections : + - HH_HEALTH (tries to send OK and dies) + - HH_MONITOR_IN (matches src IP and adds/removes HH_SEND_OK/HH_SEND_HTTP_OK) + - HH_SEND_OK (tries to send "OK" and dies) + - HH_SEND_HTTP_OK (tries to send "HTTP/1.0 200 OK" and dies) + - HH_ACCEPT_PROXY (waits for PROXY line and parses it) + - HH_TCP_RULES (processes TCP rules) + - HH_SSL_HS (starts SSL handshake) + - HH_ACCEPT_SESSION (instantiates a session) + +Identified handshake handlers for outgoing connections : + - HH_SEND_PROXY (tries to build and send the PROXY line) + - HH_SSL_HS (starts SSL handshake) + +For the pollers, we could check that handshake handlers are not 0 and decide to +call a generic connection handshake handler instead of usual callbacks. Problem +is that pollers don't know connections, they know fds. So entities which manage +handlers should update change the FD callbacks accordingly. + +With a bit of care, we could have : + - HH_SEND_LAST_CHUNK (sends the chunk pointed to by a pointer and dies) + => merges HEALTH, SEND_OK and SEND_HTTP_OK + +It sounds like the ctrl vs data state for the connection are per-direction +(eg: support an async ctrl shutw while still reading data). + +Also support shutr/shutw status at L4/L7. + +In practice, what we really need is : + +shutdown(conn) = + conn.data.shut() + conn.ctrl.shut() + conn.fd.shut() + +close(conn) = + conn.data.close() + conn.ctrl.close() + conn.fd.close() + +With SSL over Remote TCP (RTCP + RSSL) to reach the server, we would have : + + HTTP -> RTCP+RSSL connection <-> RTCP+RRAW connection -> TCP+SSL connection + +The connection has to be closed at 3 places after a successful response : + - DATA (RSSL over RTCP) + - CTRL (RTCP to close connection to server) + - SOCK (FD to close connection to second process) + +Externally, the connection is seen with very few flags : + - SHR + - SHW + - ERR + +We don't need a CLOSED flag as a connection must always be detached when it's closed. + +The internal status doesn't need to be exposed : + - FD allocated (Y/N) + - CTRL initialized (Y/N) + - CTRL connected (Y/N) + - CTRL handlers done (Y/N) + - CTRL failed (Y/N) + - CTRL shutr (Y/N) + - CTRL shutw (Y/N) + - DATA initialized (Y/N) + - DATA connected (Y/N) + - DATA handlers done (Y/N) + - DATA failed (Y/N) + - DATA shutr (Y/N) + - DATA shutw (Y/N) + +(note that having flags for operations needing to be completed might be easier) +-------------- + +Maybe we need to be able to call conn->fdset() and conn->fdclr() but it sounds +very unlikely since the only functions manipulating this are in the code of +the data/ctrl handlers. + +FDSET/FDCLR cannot be directly controlled by the stream interface since it also +depends on the DATA layer (WANT_READ/WANT_WRITE). + +But FDSET/FDCLR is probably controlled by who owns the connection (eg: DATA). + +Example: an SSL conn relies on an FD. The buffer is full, and wants the conn to +stop reading. It must not stop the FD itself. It is the read function which +should notice that it has nothing to do with a read wake-up, which needs to +disable reading. + +Conversely, when calling conn->chk_rcv(), the reader might get a WANT_READ or +even WANT_WRITE and adjust the FDs accordingly. + +------------------------ + +OK, the problem is simple : we don't manipulate the FD at the right level. +We should have : + ->connect(), ->chk_snd(), ->chk_rcv(), ->shutw(), ->shutr() which are + called from the upper layer (buffer) + ->recv(), ->send(), called from the lower layer + +Note that the SHR is *reported* by lower layer but can be forced by upper +layer. In this case it's like a delayed abort. The difficulty consists in +knowing the output data were correctly read. Probably we'd need to drain +incoming data past the active shutr(). + +The only four purposes of the top-down shutr() call are : + - acknowledge a shut read report : could probably be done better + - read timeout => disable reading : it's a delayed abort. We want to + report that the buffer is SHR, maybe even the connection, but the + FD clearly isn't. + - read abort due to error on the other side or desire to close (eg: + http-server-close) : delayed abort + - complete abort + +The active shutr() is problematic as we can't disable reading if we expect some +exchanges for data acknowledgement. We probably need to drain data only until +the shutw() has been performed and ACKed. + +A connection shut down for read would behave like this : + + 1) bidir exchanges + + 2) shutr() => read_abort_pending=1 + + 3) drain input, still send output + + 4) shutw() + + 5) drain input, wait for read0 or ack(shutw) + + 6) close() + +--------------------- 2012/07/05 ------------------- + +Communications must be performed this way : + + connection <-> channel <-> connection + +A channel is composed of flags and stats, and may store data in either a buffer +or a pipe. We need low-layer operations between sockets and buffers or pipes. +Right now we only support sockets, but later we might support remote sockets +and maybe pipes or shared memory segments. + +So we need : + + - raw_sock_to_buf() => receive raw data from socket into buffer + - raw_sock_to_pipe => receive raw data from socket into pipe (splice in) + - raw_sock_from_buf() => send raw data from buffer to socket + - raw_sock_from_pipe => send raw data from pipe to socket (splice out) + + - ssl_sock_to_buf() => receive ssl data from socket into buffer + - ssl_sock_to_pipe => receive ssl data from socket into a pipe (NULL) + - ssl_sock_from_buf() => send ssl data from buffer to socket + - ssl_sock_from_pipe => send ssl data from pipe to socket (NULL) + +These functions should set such status flags : + +#define ERR_IN 0x01 +#define ERR_OUT 0x02 +#define SHUT_IN 0x04 +#define SHUT_OUT 0x08 +#define EMPTY_IN 0x10 +#define FULL_OUT 0x20 + |