summaryrefslogtreecommitdiffstats
path: root/doc/programmers-guide.rst
diff options
context:
space:
mode:
authorDaniel Baumann <daniel.baumann@progress-linux.org>2024-04-28 07:34:15 +0000
committerDaniel Baumann <daniel.baumann@progress-linux.org>2024-04-28 07:34:15 +0000
commitafea5f9539cbf1eeaa85ec77d79eb2f59724f470 (patch)
tree2a4eba394a6bc60d2eaa8304d91168a07225d51e /doc/programmers-guide.rst
parentInitial commit. (diff)
downloadnghttp2-afea5f9539cbf1eeaa85ec77d79eb2f59724f470.tar.xz
nghttp2-afea5f9539cbf1eeaa85ec77d79eb2f59724f470.zip
Adding upstream version 1.52.0.upstream/1.52.0upstream
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to '')
-rw-r--r--doc/programmers-guide.rst526
1 files changed, 526 insertions, 0 deletions
diff --git a/doc/programmers-guide.rst b/doc/programmers-guide.rst
new file mode 100644
index 0000000..820cd20
--- /dev/null
+++ b/doc/programmers-guide.rst
@@ -0,0 +1,526 @@
+Programmers' Guide
+==================
+
+Architecture
+------------
+
+The most notable point in nghttp2 library architecture is it does not
+perform any I/O. nghttp2 only performs HTTP/2 protocol stuff based on
+input byte strings. It will call callback functions set by
+applications while processing input. The output of nghttp2 is just
+byte string. An application is responsible to send these output to
+the remote peer. The callback functions may be called while producing
+output.
+
+Not doing I/O makes embedding nghttp2 library in the existing code
+base very easy. Usually, the existing applications have its own I/O
+event loops. It is very hard to use nghttp2 in that situation if
+nghttp2 does its own I/O. It also makes light weight language wrapper
+for nghttp2 easy with the same reason. The down side is that an
+application author has to write more code to write complete
+application using nghttp2. This is especially true for simple "toy"
+application. For the real applications, however, this is not the
+case. This is because you probably want to support HTTP/1 which
+nghttp2 does not provide, and to do that, you will need to write your
+own HTTP/1 stack or use existing third-party library, and bind them
+together with nghttp2 and I/O event loop. In this point, not
+performing I/O in nghttp2 has more point than doing it.
+
+The primary object that an application uses is :type:`nghttp2_session`
+object, which is opaque struct and its details are hidden in order to
+ensure the upgrading its internal architecture without breaking the
+backward compatibility. An application can set callbacks to
+:type:`nghttp2_session` object through the dedicated object and
+functions, and it also interacts with it via many API function calls.
+
+An application can create as many :type:`nghttp2_session` object as it
+wants. But single :type:`nghttp2_session` object must be used by a
+single thread at the same time. This is not so hard to enforce since
+most event-based architecture applications use is single thread per
+core, and handling one connection I/O is done by single thread.
+
+To feed input to :type:`nghttp2_session` object, one can use
+`nghttp2_session_recv()` or `nghttp2_session_mem_recv()` functions.
+They behave similarly, and the difference is that
+`nghttp2_session_recv()` will use :type:`nghttp2_read_callback` to get
+input. On the other hand, `nghttp2_session_mem_recv()` will take
+input as its parameter. If in doubt, use `nghttp2_session_mem_recv()`
+since it is simpler, and could be faster since it avoids calling
+callback function.
+
+To get output from :type:`nghttp2_session` object, one can use
+`nghttp2_session_send()` or `nghttp2_session_mem_send()`. The
+difference between them is that the former uses
+:type:`nghttp2_send_callback` to pass output to an application. On
+the other hand, the latter returns the output to the caller. If in
+doubt, use `nghttp2_session_mem_send()` since it is simpler. But
+`nghttp2_session_send()` might be easier to use if the output buffer
+an application has is fixed sized.
+
+In general, an application should call `nghttp2_session_mem_send()`
+when it gets input from underlying connection. Since there is great
+chance to get something pushed into transmission queue while the call
+of `nghttp2_session_mem_send()`, it is recommended to call
+`nghttp2_session_mem_recv()` after `nghttp2_session_mem_send()`.
+
+There is a question when we are safe to close HTTP/2 session without
+waiting for the closure of underlying connection. We offer 2 API
+calls for this: `nghttp2_session_want_read()` and
+`nghttp2_session_want_write()`. If they both return 0, application
+can destroy :type:`nghttp2_session`, and then close the underlying
+connection. But make sure that the buffered output has been
+transmitted to the peer before closing the connection when
+`nghttp2_session_mem_send()` is used, since
+`nghttp2_session_want_write()` does not take into account the
+transmission of the buffered data outside of :type:`nghttp2_session`.
+
+Includes
+--------
+
+To use the public APIs, include ``nghttp2/nghttp2.h``::
+
+ #include <nghttp2/nghttp2.h>
+
+The header files are also available online: :doc:`nghttp2.h` and
+:doc:`nghttp2ver.h`.
+
+Remarks
+-------
+
+Do not call `nghttp2_session_send()`, `nghttp2_session_mem_send()`,
+`nghttp2_session_recv()` or `nghttp2_session_mem_recv()` from the
+nghttp2 callback functions directly or indirectly. It will lead to the
+crash. You can submit requests or frames in the callbacks then call
+these functions outside the callbacks.
+
+`nghttp2_session_send()` and `nghttp2_session_mem_send()` send first
+24 bytes of client magic string (MAGIC)
+(:macro:`NGHTTP2_CLIENT_MAGIC`) on client configuration. The
+applications are responsible to send SETTINGS frame as part of
+connection preface using `nghttp2_submit_settings()`. Similarly,
+`nghttp2_session_recv()` and `nghttp2_session_mem_recv()` consume
+MAGIC on server configuration unless
+`nghttp2_option_set_no_recv_client_magic()` is used with nonzero
+option value.
+
+.. _http-messaging:
+
+HTTP Messaging
+--------------
+
+By default, nghttp2 library checks HTTP messaging rules described in
+`HTTP/2 specification, section 8
+<https://tools.ietf.org/html/rfc7540#section-8>`_. Everything
+described in that section is not validated however. We briefly
+describe what the library does in this area. In the following
+description, without loss of generality we omit CONTINUATION frame
+since they must follow HEADERS frame and are processed atomically. In
+other words, they are just one big HEADERS frame. To disable these
+validations, use `nghttp2_option_set_no_http_messaging()`. Please
+note that disabling this feature does not change the fundamental
+client and server model of HTTP. That is, even if the validation is
+disabled, only client can send requests.
+
+For HTTP request, including those carried by PUSH_PROMISE, HTTP
+message starts with one HEADERS frame containing request headers. It
+is followed by zero or more DATA frames containing request body, which
+is followed by zero or one HEADERS containing trailer headers. The
+request headers must include ":scheme", ":method" and ":path" pseudo
+header fields unless ":method" is not "CONNECT". ":authority" is
+optional, but nghttp2 requires either ":authority" or "Host" header
+field must be present. If ":method" is "CONNECT", the request headers
+must include ":method" and ":authority" and must omit ":scheme" and
+":path".
+
+For HTTP response, HTTP message starts with zero or more HEADERS
+frames containing non-final response (status code 1xx). They are
+followed by one HEADERS frame containing final response headers
+(non-1xx). It is followed by zero or more DATA frames containing
+response body, which is followed by zero or one HEADERS containing
+trailer headers. The non-final and final response headers must
+contain ":status" pseudo header field containing 3 digits only.
+
+All request and response headers must include exactly one valid value
+for each pseudo header field. Additionally nghttp2 requires all
+request headers must not include more than one "Host" header field.
+
+HTTP/2 prohibits connection-specific header fields. The following
+header fields must not appear: "Connection", "Keep-Alive",
+"Proxy-Connection", "Transfer-Encoding" and "Upgrade". Additionally,
+"TE" header field must not include any value other than "trailers".
+
+Each header field name and value must obey the field-name and
+field-value production rules described in `RFC 7230, section
+3.2. <https://tools.ietf.org/html/rfc7230#section-3.2>`_.
+Additionally, all field name must be lower cased. The invalid header
+fields are treated as stream error, and that stream is reset. If
+application wants to treat these headers in their own way, use
+`nghttp2_on_invalid_header_callback
+<https://nghttp2.org/documentation/types.html#c.nghttp2_on_invalid_header_callback>`_.
+
+For "http" or "https" URIs, ":path" pseudo header fields must start
+with "/". The only exception is OPTIONS request, in that case, "*" is
+allowed in ":path" pseudo header field to represent system-wide
+OPTIONS request.
+
+With the above validations, nghttp2 library guarantees that header
+field name passed to `nghttp2_on_header_callback()` is not empty.
+Also required pseudo headers are all present and not empty.
+
+nghttp2 enforces "Content-Length" validation as well. All request or
+response headers must not contain more than one "Content-Length"
+header field. If "Content-Length" header field is present, it must be
+parsed as 64 bit signed integer. The sum of data length in the
+following DATA frames must match with the number in "Content-Length"
+header field if it is present (this does not include padding bytes).
+
+RFC 7230 says that server must not send "Content-Length" in any
+response with 1xx, and 204 status code. It also says that
+"Content-Length" is not allowed in any response with 200 status code
+to a CONNECT request. nghttp2 enforces them as well.
+
+Any deviation results in stream error of type PROTOCOL_ERROR. If
+error is found in PUSH_PROMISE frame, stream error is raised against
+promised stream.
+
+The order of transmission of the HTTP/2 frames
+----------------------------------------------
+
+This section describes the internals of libnghttp2 about the
+scheduling of transmission of HTTP/2 frames. This is pretty much
+internal stuff, so the details could change in the future versions of
+the library.
+
+libnghttp2 categorizes HTTP/2 frames into 4 categories: urgent,
+regular, syn_stream, and data in the order of higher priority.
+
+The urgent category includes PING and SETTINGS. They are sent with
+highest priority. The order inside the category is FIFO.
+
+The regular category includes frames other than PING, SETTINGS, DATA,
+and HEADERS which does not create stream (which counts toward
+concurrent stream limit). The order inside the category is FIFO.
+
+The syn_stream category includes HEADERS frame which creates stream,
+that counts toward the concurrent stream limit.
+
+The data category includes DATA frame, and the scheduling among DATA
+frames are determined by HTTP/2 dependency tree.
+
+If the application wants to send frames in the specific order, and the
+default transmission order does not fit, it has to schedule frames by
+itself using the callbacks (e.g.,
+:type:`nghttp2_on_frame_send_callback`).
+
+RST_STREAM has special side effect when it is submitted by
+`nghttp2_submit_rst_stream()`. It cancels all pending HEADERS and
+DATA frames whose stream ID matches the one in the RST_STREAM frame.
+This may cause unexpected behaviour for the application in some cases.
+For example, suppose that application wants to send RST_STREAM after
+sending response HEADERS and DATA. Because of the reason we mentioned
+above, the following code does not work:
+
+.. code-block:: c
+
+ nghttp2_submit_response(...)
+ nghttp2_submit_rst_stream(...)
+
+RST_STREAM cancels HEADERS (and DATA), and just RST_STREAM is sent.
+The correct way is use :type:`nghttp2_on_frame_send_callback`, and
+after HEADERS and DATA frames are sent, issue
+`nghttp2_submit_rst_stream()`. FYI,
+:type:`nghttp2_on_frame_not_send_callback` tells you why frames are
+not sent.
+
+Implement user defined HTTP/2 non-critical extensions
+-----------------------------------------------------
+
+As of nghttp2 v1.8.0, we have added HTTP/2 non-critical extension
+framework, which lets application send and receive user defined custom
+HTTP/2 non-critical extension frames. nghttp2 also offers built-in
+functionality to send and receive official HTTP/2 extension frames
+(e.g., ALTSVC frame). For these built-in handler, refer to the next
+section.
+
+To send extension frame, use `nghttp2_submit_extension()`, and
+implement :type:`nghttp2_pack_extension_callback`. The callback
+implements how to encode data into wire format. The callback must be
+set to :type:`nghttp2_session_callbacks` using
+`nghttp2_session_callbacks_set_pack_extension_callback()`.
+
+For example, we will illustrate how to send `ALTSVC
+<https://tools.ietf.org/html/rfc7838>`_ frame.
+
+.. code-block:: c
+
+ typedef struct {
+ const char *origin;
+ const char *field;
+ } alt_svc;
+
+ ssize_t pack_extension_callback(nghttp2_session *session, uint8_t *buf,
+ size_t len, const nghttp2_frame *frame,
+ void *user_data) {
+ const alt_svc *altsvc = (const alt_svc *)frame->ext.payload;
+ size_t originlen = strlen(altsvc->origin);
+ size_t fieldlen = strlen(altsvc->field);
+
+ uint8_t *p;
+
+ if (len < 2 + originlen + fieldlen || originlen > 0xffff) {
+ return NGHTTP2_ERR_CANCEL;
+ }
+
+ p = buf;
+ *p++ = originlen >> 8;
+ *p++ = originlen & 0xff;
+ memcpy(p, altsvc->origin, originlen);
+ p += originlen;
+ memcpy(p, altsvc->field, fieldlen);
+ p += fieldlen;
+
+ return p - buf;
+ }
+
+This implements :type:`nghttp2_pack_extension_callback`. We have to
+set this callback to :type:`nghttp2_session_callbacks`:
+
+.. code-block:: c
+
+ nghttp2_session_callbacks_set_pack_extension_callback(
+ callbacks, pack_extension_callback);
+
+To send ALTSVC frame, call `nghttp2_submit_extension()`:
+
+.. code-block:: c
+
+ static const alt_svc altsvc = {"example.com", "h2=\":8000\""};
+
+ nghttp2_submit_extension(session, 0xa, NGHTTP2_FLAG_NONE, 0,
+ (void *)&altsvc);
+
+Notice that ALTSVC is use frame type ``0xa``.
+
+To receive extension frames, implement 2 callbacks:
+:type:`nghttp2_unpack_extension_callback` and
+:type:`nghttp2_on_extension_chunk_recv_callback`.
+:type:`nghttp2_unpack_extension_callback` implements the way how to
+decode wire format. :type:`nghttp2_on_extension_chunk_recv_callback`
+implements how to buffer the incoming extension payload. These
+callbacks must be set using
+`nghttp2_session_callbacks_set_unpack_extension_callback()` and
+`nghttp2_session_callbacks_set_on_extension_chunk_recv_callback()`
+respectively. The application also must tell the library which
+extension frame type it is willing to receive using
+`nghttp2_option_set_user_recv_extension_type()`. Note that the
+application has to create :type:`nghttp2_option` object for that
+purpose, and initialize session with it.
+
+We use ALTSVC again to illustrate how to receive extension frames. We
+use different ``alt_svc`` struct than the previous one.
+
+First implement 2 callbacks. We store incoming ALTSVC payload to
+global variable ``altsvc_buffer``. Don't do this in production code
+since this is not thread safe:
+
+.. code-block:: c
+
+ typedef struct {
+ const uint8_t *origin;
+ size_t originlen;
+ const uint8_t *field;
+ size_t fieldlen;
+ } alt_svc;
+
+ /* buffers incoming ALTSVC payload */
+ uint8_t altsvc_buffer[4096];
+ /* The length of byte written to altsvc_buffer */
+ size_t altsvc_bufferlen = 0;
+
+ int on_extension_chunk_recv_callback(nghttp2_session *session,
+ const nghttp2_frame_hd *hd,
+ const uint8_t *data, size_t len,
+ void *user_data) {
+ if (sizeof(altsvc_buffer) < altsvc_bufferlen + len) {
+ altsvc_bufferlen = 0;
+ return NGHTTP2_ERR_CANCEL;
+ }
+
+ memcpy(altsvc_buffer + altsvc_bufferlen, data, len);
+ altsvc_bufferlen += len;
+
+ return 0;
+ }
+
+ int unpack_extension_callback(nghttp2_session *session, void **payload,
+ const nghttp2_frame_hd *hd, void *user_data) {
+ uint8_t *origin, *field;
+ size_t originlen, fieldlen;
+ uint8_t *p, *end;
+ alt_svc *altsvc;
+
+ if (altsvc_bufferlen < 2) {
+ altsvc_bufferlen = 0;
+ return NGHTTP2_ERR_CANCEL;
+ }
+
+ p = altsvc_buffer;
+ end = altsvc_buffer + altsvc_bufferlen;
+
+ originlen = ((*p) << 8) + *(p + 1);
+ p += 2;
+
+ if (p + originlen > end) {
+ altsvc_bufferlen = 0;
+ return NGHTTP2_ERR_CANCEL;
+ }
+
+ origin = p;
+ field = p + originlen;
+ fieldlen = end - field;
+
+ altsvc = (alt_svc *)malloc(sizeof(alt_svc));
+ altsvc->origin = origin;
+ altsvc->originlen = originlen;
+ altsvc->field = field;
+ altsvc->fieldlen = fieldlen;
+
+ *payload = altsvc;
+
+ altsvc_bufferlen = 0;
+
+ return 0;
+ }
+
+Set these callbacks to :type:`nghttp2_session_callbacks`:
+
+.. code-block:: c
+
+ nghttp2_session_callbacks_set_on_extension_chunk_recv_callback(
+ callbacks, on_extension_chunk_recv_callback);
+
+ nghttp2_session_callbacks_set_unpack_extension_callback(
+ callbacks, unpack_extension_callback);
+
+
+In ``unpack_extension_callback`` above, we set unpacked ``alt_svc``
+object to ``*payload``. nghttp2 library then, calls
+:type:`nghttp2_on_frame_recv_callback`, and ``*payload`` will be
+available as ``frame->ext.payload``:
+
+.. code-block:: c
+
+ int on_frame_recv_callback(nghttp2_session *session,
+ const nghttp2_frame *frame, void *user_data) {
+
+ switch (frame->hd.type) {
+ ...
+ case 0xa: {
+ alt_svc *altsvc = (alt_svc *)frame->ext.payload;
+ fprintf(stderr, "ALTSVC frame received\n");
+ fprintf(stderr, " origin: %.*s\n", (int)altsvc->originlen, altsvc->origin);
+ fprintf(stderr, " field : %.*s\n", (int)altsvc->fieldlen, altsvc->field);
+ free(altsvc);
+ break;
+ }
+ }
+
+ return 0;
+ }
+
+Finally, application should set the extension frame types it is
+willing to receive:
+
+.. code-block:: c
+
+ nghttp2_option_set_user_recv_extension_type(option, 0xa);
+
+The :type:`nghttp2_option` must be set to :type:`nghttp2_session` on
+its creation:
+
+.. code-block:: c
+
+ nghttp2_session_client_new2(&session, callbacks, user_data, option);
+
+How to use built-in HTTP/2 extension frame handlers
+---------------------------------------------------
+
+In the previous section, we talked about the user defined HTTP/2
+extension frames. In this section, we talk about HTTP/2 extension
+frame support built into nghttp2 library.
+
+As of this writing, nghttp2 supports ALTSVC extension frame. To send
+ALTSVC frame, use `nghttp2_submit_altsvc()` function.
+
+To receive ALTSVC frame through built-in functionality, application
+has to use `nghttp2_option_set_builtin_recv_extension_type()` to
+indicate the willingness of receiving ALTSVC frame:
+
+.. code-block:: c
+
+ nghttp2_option_set_builtin_recv_extension_type(option, NGHTTP2_ALTSVC);
+
+This is very similar to the case when we used to receive user defined
+frames.
+
+If the same frame type is set using
+`nghttp2_option_set_builtin_recv_extension_type()` and
+`nghttp2_option_set_user_recv_extension_type()`, the latter takes
+precedence. Application can implement its own frame handler rather
+than using built-in handler.
+
+The :type:`nghttp2_option` must be set to :type:`nghttp2_session` on
+its creation, like so:
+
+.. code-block:: c
+
+ nghttp2_session_client_new2(&session, callbacks, user_data, option);
+
+When ALTSVC is received, :type:`nghttp2_on_frame_recv_callback` will
+be called as usual.
+
+Stream priorities
+-----------------
+
+By default, the stream prioritization scheme described in :rfc:`7540`
+is used. This scheme has been formally deprecated by :rfc:`9113`. In
+order to disable it, send
+:enum:`nghttp2_settings_id.NGHTTP2_SETTINGS_NO_RFC7540_PRIORITIES` of
+value of 1 via `nghttp2_submit_settings()`. This settings ID is
+defined by :rfc:`9218`. The sender of this settings value disables
+RFC 7540 priorities, and instead it enables RFC 9218 Extensible
+Prioritization Scheme. This new prioritization scheme has 2 methods
+to convey the stream priorities to a remote endpoint: Priority header
+field and PRIORITY_UPDATE frame. nghttp2 supports both methods. In
+order to receive and process PRIORITY_UPDATE frame, server has to call
+``nghttp2_option_set_builtin_recv_extension_type(option,
+NGHTTP2_PRIORITY_UPDATE)`` (see the above section), and pass the
+option to `nghttp2_session_server_new2()` or
+`nghttp2_session_server_new3()` to create a server session. Client
+can send Priority header field via `nghttp2_submit_request()`. It can
+also send PRIORITY_UPDATE frame via
+`nghttp2_submit_priority_update()`. Server processes Priority header
+field in a request header field and updates the stream priority unless
+HTTP messaging rule enforcement is disabled (see
+`nghttp2_option_set_no_http_messaging()`).
+
+For the purpose of smooth migration from RFC 7540 priorities, client
+is advised to send
+:enum:`nghttp2_settings_id.NGHTTP2_SETTINGS_NO_RFC7540_PRIORITIES` of
+value of 1. Until it receives the first server SETTINGS frame, it can
+send both RFC 7540 and RFC 9128 priority signals. If client receives
+SETTINGS_NO_RFC7540_PRIORITIES of value of 0, or it is omitted ,
+client stops sending PRIORITY_UPDATE frame. Priority header field
+will be sent in anyway since it is an end-to-end signal. If
+SETTINGS_NO_RFC7540_PRIORITIES of value of 1 is received, client stops
+sending RFC 7540 priority signals. This is the advice described in
+:rfc:`9218#section-2.1.1`.
+
+Server has an optional mechanism to fallback to RFC 7540 priorities.
+By default, if server sends SETTINGS_NO_RFC7540_PRIORITIES of value of
+1, it completely disables RFC 7540 priorities and no fallback. By
+setting nonzero value to
+`nghttp2_option_set_server_fallback_rfc7540_priorities()`, server
+falls back to RFC 7540 priorities if it sends
+SETTINGS_NO_RFC7540_PRIORITIES value of value of 1, and client omits
+SETTINGS_NO_RFC7540_PRIORITIES in its SETTINGS frame.