2015/08/06 - server connection sharing Improvements on the connection sharing strategies ------------------------------------------------- 4 strategies are currently supported : - never - safe - aggressive - always The "aggressive" and "always" strategies take into account the fact that the connection has already been reused at least once or not. The principle is that second requests can be used to safely "validate" connection reuse on newly added connections, and that such validated connections may be used even by first requests from other sessions. A validated connection is a connection which has already been reused, hence proving that it definitely supports multiple requests. Such connections are easy to verify : after processing the response, if the txn already had the TX_NOT_FIRST flag, then it was not the first request over that connection, and it is validated as safe for reuse. Validated connections are put into a distinct list : server->safe_conns. Incoming requests with TX_NOT_FIRST first pick from the regular idle_conns list so that any new idle connection is validated as soon as possible. Incoming requests without TX_NOT_FIRST only pick from the safe_conns list for strategy "aggressive", guaranteeing that the server properly supports connection reuse, or first from the safe_conns list, then from the idle_conns list for strategy "always". Connections are always stacked into the list (LIFO) so that there are higher changes to convert recent connections and to use them. This will first optimize the likeliness that the connection works, and will avoid TCP metrics from being lost due to an idle state, and/or the congestion window to drop and the connection going to slow start mode. Handling connections in pools ----------------------------- A per-server "pool-max" setting should be added to permit disposing unused idle connections not attached anymore to a session for use by future requests. The principle will be that attached connections are queued from the front of the list while the detached connections will be queued from the tail of the list. This way, most reused connections will be fairly recent and detached connections will most often be ignored. The number of detached idle connections in the lists should be accounted for (pool_used) and limited (pool_max). After some time, a part of these detached idle connections should be killed. For this, the list is walked from tail to head and connections without an owner may be evicted. It may be useful to have a per-server pool_min setting indicating how many idle connections should remain in the pool, ready for use by new requests. Conversely, a pool_low metric should be kept between eviction runs, to indicate the lowest amount of detached connections that were found in the pool. For eviction, the principle of a half-life is appealing. The principle is simple : over a period of time, half of the connections between pool_min and pool_low should be gone. Since pool_low indicates how many connections were remaining unused over a period, it makes sense to kill some of them. In order to avoid killing thousands of connections in one run, the purge interval should be split into smaller batches. Let's call N the ratio of the half-life interval and the effective interval. The algorithm consists in walking over them from the end every interval and killing ((pool_low - pool_min) + 2 * N - 1) / (2 * N). It ensures that half of the unused connections are killed over the half-life period, in N batches of population/2N entries at most. Unsafe connections should be evicted first. There should be quite few of them since most of them are probed and become safe. Since detached connections are quickly recycled and attached to a new session, there should not be too many detached connections in the pool, and those present there may be killed really quickly. Another interesting point of pools is that when a pool-max is not null, then it makes sense to automatically enable pretend-keep-alive on non-private connections going to the server in order to be able to feed them back into the pool. With the "aggressive" or "always" strategies, it can allow clients making a single request over their connection to share persistent connections to the servers. 2013/10/17 - server connection management and reuse Current state ------------- At the moment, a connection entity is needed to carry any address information. This means in the following situations, we need a server connection : - server is elected and the server's destination address is set - transparent mode is elected and the destination address is set from the incoming connection - proxy mode is enabled, and the destination's address is set during the parsing of the HTTP request - connection to the server fails and must be retried on the same server using the same parameters, especially the destination address (SN_ADDR_SET not removed) On the accepting side, we have further requirements : - allocate a clean connection without a stream interface - incrementally set the accepted connection's parameters without clearing it, and keep track of what is set (eg: getsockname). - initialize a stream interface in established mode - attach the accepted connection to a stream interface This means several things : - the connection has to be allocated on the fly the first time it is needed to store the source or destination address ; - the connection has to be attached to the stream interface at this moment ; - it must be possible to incrementally set some settings on the connection's addresses regardless of the connection's current state - the connection must not be released across connection retries ; - it must be possible to clear a connection's parameters for a redispatch without having to detach/attach the connection ; - we need to allocate a connection without an existing stream interface So on the accept() side, it looks like this : fd = accept(); conn = new_conn(); get_some_addr_info(&conn->addr); ... si = new_si(); si_attach_conn(si, conn); si_set_state(si, SI_ST_EST); ... get_more_addr_info(&conn->addr); On the connect() side, it looks like this : si = new_si(); while (!properly_connected) { if (!(conn = si->end)) { conn = new_conn(); conn_clear(conn); si_attach_conn(si, conn); } else { if (connected) { f = conn->flags & CO_FL_XPRT_TRACKED; conn->flags &= ~CO_FL_XPRT_TRACKED; conn_close(conn); conn->flags |= f; } if (!correct_dest) conn_clear(conn); } set_some_addr_info(&conn->addr); si_set_state(si, SI_ST_CON); ... set_more_addr_info(&conn->addr); conn->connect(); if (must_retry) { close_conn(conn); } } Note: we need to be able to set the control and transport protocols. On outgoing connections, this is set once we know the destination address. On incoming connections, this is set the earliest possible (once we know the source address). The problem analysed below was solved on 2013/10/22 | ==> the real requirement is to know whether a connection is still valid or not | before deciding to close it. CO_FL_CONNECTED could be enough, though it | will not indicate connections that are still waiting for a connect to occur. | This combined with CO_FL_WAIT_L4_CONN and CO_FL_WAIT_L6_CONN should be OK. | | Alternatively, conn->xprt could be used for this, but needs some careful checks | (it's used by conn_full_close at least). | | Right now, conn_xprt_close() checks conn->xprt and sets it to NULL. | conn_full_close() also checks conn->xprt and sets it to NULL, except | that the check on ctrl is performed within xprt. So conn_xprt_close() | followed by conn_full_close() will not close the file descriptor. | Note that conn_xprt_close() is never called, maybe we should kill it ? | | Note: at the moment, it's problematic to leave conn->xprt to NULL before doing | xprt_init() because we might end up with a pending file descriptor. Or at | least with some transport not de-initialized. We might thus need | conn_xprt_close() when conn_xprt_init() fails. | | The fd should be conditioned by ->ctrl only, and the transport layer by ->xprt. | | - conn_prepare_ctrl(conn, ctrl) | - conn_prepare_xprt(conn, xprt) | - conn_prepare_data(conn, data) | | Note: conn_xprt_init() needs conn->xprt so it's not a problem to set it early. | | One problem might be with conn_xprt_close() not being able to know if xprt_init() | was called or not. That's where it might make sense to only set ->xprt during init. | Except that it does not fly with outgoing connections (xprt_init is called after | connect()). | | => currently conn_xprt_close() is only used by ssl_sock.c and decides whether | to do something based on ->xprt_ctx which is set by ->init() from xprt_init(). | So there is nothing to worry about. We just need to restore conn_xprt_close() | and rely on ->ctrl to close the fd instead of ->xprt. | | => we have the same issue with conn_ctrl_close() : when is the fd supposed to be | valid ? On outgoing connections, the control is set much before the fd...