diff options
Diffstat (limited to '')
| -rw-r--r-- | doc/sources/building-android-binary.rst | 127 | ||||
| -rw-r--r-- | doc/sources/contribute.rst | 56 | ||||
| -rw-r--r-- | doc/sources/h2load-howto.rst | 142 | ||||
| -rw-r--r-- | doc/sources/index.rst | 50 | ||||
| -rw-r--r-- | doc/sources/nghttpx-howto.rst | 642 | ||||
| -rw-r--r-- | doc/sources/security.rst | 38 | ||||
| -rw-r--r-- | doc/sources/tutorial-client.rst | 498 | ||||
| -rw-r--r-- | doc/sources/tutorial-hpack.rst | 126 | ||||
| -rw-r--r-- | doc/sources/tutorial-server.rst | 625 |
9 files changed, 2304 insertions, 0 deletions
diff --git a/doc/sources/building-android-binary.rst b/doc/sources/building-android-binary.rst new file mode 100644 index 0000000..339ac44 --- /dev/null +++ b/doc/sources/building-android-binary.rst @@ -0,0 +1,127 @@ +Building Android binary +======================= + +In this article, we briefly describe how to build Android binary using +`Android NDK <https://developer.android.com/ndk>`_ cross-compiler on +Debian Linux. + +The easiest way to build android binary is use Dockerfile.android. +See Dockerfile.android for more details. If you cannot use +Dockerfile.android for whatever reason, continue to read the rest of +this article. + +We offer ``android-config`` script to make the build easier. To make +the script work, NDK directory must be set to ``NDK`` environment +variable. NDK directory is the directory where NDK is unpacked: + +.. code-block:: text + + $ unzip android-ndk-$NDK_VERSION-linux.zip + $ cd android-ndk-$NDK_VERSION + $ export NDK=$PWD + +The dependent libraries, such as OpenSSL, libev, and c-ares should be +built with the same NDK toolchain and installed under +``$NDK/usr/local``. We recommend to build these libraries as static +library to make the deployment easier. libxml2 support is currently +disabled. + +Although zlib comes with Android NDK, it seems not to be a part of +public API, so we have to built it for our own. That also provides us +proper .pc file as a bonus. + +Before running ``android-config``, ``NDK`` environment variable must +be set to point to the correct path. + +You need to set ``NGHTTP2`` environment variable to the absolute path +to the source directory of nghttp2. + +To configure OpenSSL, use the following script: + +.. code-block:: sh + + #!/bin/sh + + . $NGHTTP2/android-env + + export ANDROID_NDK_HOME=$NDK + export PATH=$TOOLCHAIN/bin:$PATH + + ./Configure no-shared --prefix=$PREFIX android-arm64 + +And run the following script to build and install without +documentation: + +.. code-block:: sh + + #!/bin/sh + + . $NGHTTP2/android-env + + export PATH=$TOOLCHAIN/bin:$PATH + + make install_sw + +To configure libev, use the following script: + +.. code-block:: sh + + #!/bin/sh + + . $NGHTTP2/android-env + + ./configure \ + --host=$TARGET \ + --build=`dpkg-architecture -qDEB_BUILD_GNU_TYPE` \ + --prefix=$PREFIX \ + --disable-shared \ + --enable-static \ + CPPFLAGS=-I$PREFIX/include \ + LDFLAGS=-L$PREFIX/lib + +And run ``make install`` to build and install. + +To configure c-ares, use the following script: + +.. code-block:: sh + + #!/bin/sh -e + + . $NGHTTP2/android-env + + ./configure \ + --host=$TARGET \ + --build=`dpkg-architecture -qDEB_BUILD_GNU_TYPE` \ + --prefix=$PREFIX \ + --disable-shared + +And run ``make install`` to build and install. + +To configure zlib, use the following script: + +.. code-block:: sh + + #!/bin/sh -e + + . $NGHTTP2/android-env + + export HOST=$TARGET + + ./configure \ + --prefix=$PREFIX \ + --libdir=$PREFIX/lib \ + --includedir=$PREFIX/include \ + --static + +And run ``make install`` to build and install. + +After prerequisite libraries are prepared, run ``android-config`` and +then ``make`` to compile nghttp2 source files. + +If all went well, application binaries, such as nghttpx, are created +under src directory. Strip debugging information from the binary +using the following command: + +.. code-block:: text + + $ $NDK/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-strip src/nghttpx diff --git a/doc/sources/contribute.rst b/doc/sources/contribute.rst new file mode 100644 index 0000000..32f43c8 --- /dev/null +++ b/doc/sources/contribute.rst @@ -0,0 +1,56 @@ +Contribution Guidelines +======================= + +[This text was composed based on 1.2. License section of curl/libcurl +project.] + +When contributing with code, you agree to put your changes and new +code under the same license nghttp2 is already using unless stated and +agreed otherwise. + +When changing existing source code, you do not alter the copyright of +the original file(s). The copyright will still be owned by the +original creator(s) or those who have been assigned copyright by the +original author(s). + +By submitting a patch to the nghttp2 project, you are assumed to have +the right to the code and to be allowed by your employer or whatever +to hand over that patch/code to us. We will credit you for your +changes as far as possible, to give credit but also to keep a trace +back to who made what changes. Please always provide us with your +full real name when contributing! + +Coding style +------------ + +We use clang-format to format source code consistently. The +clang-format configuration file .clang-format is located at the root +directory. Since clang-format produces slightly different results +between versions, we currently use clang-format-14. + +To detect any violation to the coding style, we recommend to setup git +pre-commit hook to check coding style of the changes you introduced. +The pre-commit file is located at the root directory. Copy it under +.git/hooks and make sure that it is executable. The pre-commit script +uses clang-format-diff.py to detect any style errors. If it is not in +your PATH or it exists under different name (e.g., +clang-format-diff-14 in debian), either add it to PATH variable or add +git option ``clangformatdiff.binary`` to point to the script. + +For emacs users, integrating clang-format to emacs is very easy. +clang-format.el should come with clang distribution. If it is not +found, download it from `here +<https://github.com/llvm/llvm-project/blob/main/clang/tools/clang-format/clang-format.el>`_. +And add these lines to your .emacs file: + +.. code-block:: lisp + + ;; From + ;; https://code.google.com/p/chromium/wiki/Emacs#Use_Google's_C++_style! + (load "/<path/to>/clang-format.el") + (add-hook 'c-mode-common-hook + (function (lambda () (local-set-key (kbd "TAB") + 'clang-format-region)))) + +You can find other editor integration in +http://clang.llvm.org/docs/ClangFormat.html. diff --git a/doc/sources/h2load-howto.rst b/doc/sources/h2load-howto.rst new file mode 100644 index 0000000..2ee2754 --- /dev/null +++ b/doc/sources/h2load-howto.rst @@ -0,0 +1,142 @@ +.. program:: h2load + +h2load - HTTP/2 benchmarking tool - HOW-TO +========================================== + +:doc:`h2load.1` is benchmarking tool for HTTP/2 and HTTP/1.1. It +supports SSL/TLS and clear text for all supported protocols. + +Compiling from source +--------------------- + +h2load is compiled alongside nghttp2 and requires that the +``--enable-app`` flag is passed to ``./configure`` and `required +dependencies <https://github.com/nghttp2/nghttp2#requirements>`_ are +available during compilation. For details on compiling, see `nghttp2: +Building from Git +<https://github.com/nghttp2/nghttp2#building-from-git>`_. + +Basic Usage +----------- + +In order to set benchmark settings, specify following 3 options. + +:option:`-n` + The number of total requests. Default: 1 + +:option:`-c` + The number of concurrent clients. Default: 1 + +:option:`-m` + The max concurrent streams to issue per client. Default: 1 + +For SSL/TLS connection, the protocol will be negotiated via ALPN/NPN. +You can set specific protocols in :option:`--npn-list` option. For +cleartext connection, the default protocol is HTTP/2. To change the +protocol in cleartext connection, use :option:`--no-tls-proto` option. +For convenience, :option:`--h1` option forces HTTP/1.1 for both +cleartext and SSL/TLS connections. + +Here is a command-line to perform benchmark to URI \https://localhost +using total 100000 requests, 100 concurrent clients and 10 max +concurrent streams: + +.. code-block:: text + + $ h2load -n100000 -c100 -m10 https://localhost + +The benchmarking result looks like this: + +.. code-block:: text + + finished in 7.08s, 141164.80 req/s, 555.33MB/s + requests: 1000000 total, 1000000 started, 1000000 done, 1000000 succeeded, 0 failed, 0 errored, 0 timeout + status codes: 1000000 2xx, 0 3xx, 0 4xx, 0 5xx + traffic: 4125025824 bytes total, 11023424 bytes headers (space savings 93.07%), 4096000000 bytes data + min max mean sd +/- sd + time for request: 15.31ms 146.85ms 69.78ms 9.26ms 92.43% + time for connect: 1.08ms 25.04ms 10.71ms 9.80ms 64.00% + time to 1st byte: 25.36ms 184.96ms 79.11ms 53.97ms 78.00% + req/s (client) : 1412.04 1447.84 1426.52 10.57 63.00% + +See the h2load manual page :ref:`h2load-1-output` section for the +explanation of the above numbers. + +Timing-based load-testing +------------------------- + +As of v1.26.0, h2load supports timing-based load-testing. This method +performs load-testing in terms of a given duration instead of a +pre-defined number of requests. The new option :option:`--duration` +specifies how long the load-testing takes. For example, +``--duration=10`` makes h2load perform load-testing against a server +for 10 seconds. You can also specify a “warming-up” period with +:option:`--warm-up-time`. If :option:`--duration` is used, +:option:`-n` option is ignored. + +The following command performs load-testing for 10 seconds after 5 +seconds warming up period: + +.. code-block:: text + + $ h2load -c100 -m100 --duration=10 --warm-up-time=5 https://localhost + +Flow Control +------------ + +HTTP/2 has flow control and it may affect benchmarking results. By +default, h2load uses large enough flow control window, which +effectively disables flow control. To adjust receiver flow control +window size, there are following options: + +:option:`-w` + Sets the stream level initial window size to + (2**<N>)-1. + +:option:`-W` + Sets the connection level initial window size to + (2**<N>)-1. + +Multi-Threading +--------------- + +Sometimes benchmarking client itself becomes a bottleneck. To remedy +this situation, use :option:`-t` option to specify the number of native +thread to use. + +:option:`-t` + The number of native threads. Default: 1 + +Selecting protocol for clear text +--------------------------------- + +By default, if \http:// URI is given, HTTP/2 protocol is used. To +change the protocol to use for clear text, use :option:`-p` option. + +Multiple URIs +------------- + +If multiple URIs are specified, they are used in round robin manner. + +.. note:: + + Please note that h2load uses scheme, host and port in the first URI + and ignores those parts in the rest of the URIs. + +UNIX domain socket +------------------ + +To request against UNIX domain socket, use :option:`--base-uri`, and +specify ``unix:`` followed by the path to UNIX domain socket. For +example, if UNIX domain socket is ``/tmp/nghttpx.sock``, use +``--base-uri=unix:/tmp/nghttpx.sock``. h2load uses scheme, host and +port in the first URI in command-line or input file. + +HTTP/3 +------ + +h2load supports HTTP/3 if it is built with HTTP/3 enabled. HTTP/3 +support is experimental. + +In order to send HTTP/3 request, specify ``h3`` to +:option:`--npn-list`. diff --git a/doc/sources/index.rst b/doc/sources/index.rst new file mode 100644 index 0000000..b03c348 --- /dev/null +++ b/doc/sources/index.rst @@ -0,0 +1,50 @@ +.. nghttp2 documentation main file, created by + sphinx-quickstart on Sun Mar 11 22:57:49 2012. + You can adapt this file completely to your liking, but it should at least + contain the root `toctree` directive. + +nghttp2 - HTTP/2 C Library +============================ + +This is an implementation of Hypertext Transfer Protocol version 2. + +The project is hosted at `github.com/nghttp2/nghttp2 +<https://github.com/nghttp2/nghttp2>`_. + +Contents: + +.. toctree:: + :maxdepth: 2 + + package_README + contribute + security + building-android-binary + tutorial-client + tutorial-server + tutorial-hpack + nghttp.1 + nghttpd.1 + nghttpx.1 + h2load.1 + nghttpx-howto + h2load-howto + programmers-guide + apiref + nghttp2.h + nghttp2ver.h + Source <https://github.com/nghttp2/nghttp2> + Issues <https://github.com/nghttp2/nghttp2/issues> + nghttp2.org <https://nghttp2.org/> + +Released Versions +================= + +https://github.com/nghttp2/nghttp2/releases + +Resources +--------- + +* HTTP/2 https://tools.ietf.org/html/rfc7540 +* HPACK https://tools.ietf.org/html/rfc7541 +* HTTP Alternative Services https://tools.ietf.org/html/rfc7838 diff --git a/doc/sources/nghttpx-howto.rst b/doc/sources/nghttpx-howto.rst new file mode 100644 index 0000000..a979504 --- /dev/null +++ b/doc/sources/nghttpx-howto.rst @@ -0,0 +1,642 @@ +.. program:: nghttpx + +nghttpx - HTTP/2 proxy - HOW-TO +=============================== + +:doc:`nghttpx.1` is a proxy translating protocols between HTTP/2 and +other protocols (e.g., HTTP/1). It operates in several modes and each +mode may require additional programs to work with. This article +describes each operation mode and explains the intended use-cases. It +also covers some useful options later. + +Default mode +------------ + +If nghttpx is invoked without :option:`--http2-proxy`, it operates in +default mode. In this mode, it works as reverse proxy (gateway) for +HTTP/3, HTTP/2 and HTTP/1 clients to backend servers. This is also +known as "HTTP/2 router". + +By default, frontend connection is encrypted using SSL/TLS. So +server's private key and certificate must be supplied to the command +line (or through configuration file). In this case, the frontend +protocol selection will be done via ALPN or NPN. + +To turn off encryption on frontend connection, use ``no-tls`` keyword +in :option:`--frontend` option. HTTP/2 and HTTP/1 are available on +the frontend, and an HTTP/1 connection can be upgraded to HTTP/2 using +HTTP Upgrade. Starting HTTP/2 connection by sending HTTP/2 connection +preface is also supported. + +In order to receive HTTP/3 traffic, use ``quic`` parameter in +:option:`--frontend` option (.e.g, ``--frontend='*,443;quic'``) + +nghttpx can listen on multiple frontend addresses. This is achieved +by using multiple :option:`--frontend` options. For each frontend +address, TLS can be enabled or disabled. + +By default, backend connections are not encrypted. To enable TLS +encryption on backend connections, use ``tls`` keyword in +:option:`--backend` option. Using patterns and ``proto`` keyword in +:option:`--backend` option, backend application protocol can be +specified per host/request path pattern. It means that you can use +both HTTP/2 and HTTP/1 in backend connections at the same time. Note +that default backend protocol is HTTP/1.1. To use HTTP/2 in backend, +you have to specify ``h2`` in ``proto`` keyword in :option:`--backend` +explicitly. + +The backend is supposed to be a Web server. For example, to make +nghttpx listen to encrypted HTTP/2 requests at port 8443, and a +backend Web server is configured to listen to HTTP requests at port +8080 on the same host, run nghttpx command-line like this: + +.. code-block:: text + + $ nghttpx -f0.0.0.0,8443 -b127.0.0.1,8080 /path/to/server.key /path/to/server.crt + +Then an HTTP/2 enabled client can access the nghttpx server using HTTP/2. For +example, you can send a GET request using nghttp: + +.. code-block:: text + + $ nghttp -nv https://localhost:8443/ + +HTTP/2 proxy mode +----------------- + +If nghttpx is invoked with :option:`--http2-proxy` (or its shorthand +:option:`-s`) option, it operates in HTTP/2 proxy mode. The supported +protocols in frontend and backend connections are the same as in `default +mode`_. The difference is that this mode acts like a forward proxy and +assumes the backend is an HTTP proxy server (e.g., Squid, Apache Traffic +Server). HTTP/1 requests must include an absolute URI in request line. + +By default, the frontend connection is encrypted. So this mode is +also called secure proxy. + +To turn off encryption on the frontend connection, use ``no-tls`` keyword +in :option:`--frontend` option. + +The backend must be an HTTP proxy server. nghttpx supports multiple +backend server addresses. It translates incoming requests to HTTP +request to backend server. The backend server performs real proxy +work for each request, for example, dispatching requests to the origin +server and caching contents. + +The backend connection is not encrypted by default. To enable +encryption, use ``tls`` keyword in :option:`--backend` option. The +default backend protocol is HTTP/1.1. To use HTTP/2 in backend +connection, use :option:`--backend` option, and specify ``h2`` in +``proto`` keyword explicitly. + +For example, to make nghttpx listen to encrypted HTTP/2 requests at +port 8443, and a backend HTTP proxy server is configured to listen to +HTTP/1 requests at port 8080 on the same host, run nghttpx command-line +like this: + +.. code-block:: text + + $ nghttpx -s -f'*,8443' -b127.0.0.1,8080 /path/to/server.key /path/to/server.crt + +At the time of this writing, Firefox 41 and Chromium v46 can use +nghttpx as HTTP/2 proxy. + +To make Firefox or Chromium use nghttpx as HTTP/2 proxy, user has to +create proxy.pac script file like this: + +.. code-block:: javascript + + function FindProxyForURL(url, host) { + return "HTTPS SERVERADDR:PORT"; + } + +``SERVERADDR`` and ``PORT`` is the hostname/address and port of the +machine nghttpx is running. Please note that both Firefox and +Chromium require valid certificate for secure proxy. + +For Firefox, open Preference window and select Advanced then click +Network tab. Clicking Connection Settings button will show the +dialog. Select "Automatic proxy configuration URL" and enter the path +to proxy.pac file, something like this: + +.. code-block:: text + + file:///path/to/proxy.pac + +For Chromium, use following command-line: + +.. code-block:: text + + $ google-chrome --proxy-pac-url=file:///path/to/proxy.pac --use-npn + +As HTTP/1 proxy server, Squid may work as out-of-box. Traffic server +requires to be configured as forward proxy. Here is the minimum +configuration items to edit: + +.. code-block:: text + + CONFIG proxy.config.reverse_proxy.enabled INT 0 + CONFIG proxy.config.url_remap.remap_required INT 0 + +Consult Traffic server `documentation +<http://trafficserver.readthedocs.org/en/latest/admin-guide/configuration/transparent-forward-proxying.en.html>`_ +to know how to configure traffic server as forward proxy and its +security implications. + +ALPN support +------------ + +ALPN support requires OpenSSL >= 1.0.2. + +Disable frontend SSL/TLS +------------------------ + +The frontend connections are encrypted with SSL/TLS by default. To +turn off SSL/TLS, use ``no-tls`` keyword in :option:`--frontend` +option. If this option is used, the private key and certificate are +not required to run nghttpx. + +Enable backend SSL/TLS +---------------------- + +The backend connections are not encrypted by default. To enable +SSL/TLS encryption, use ``tls`` keyword in :option:`--backend` option. + +Enable SSL/TLS on memcached connection +-------------------------------------- + +By default, memcached connection is not encrypted. To enable +encryption, use ``tls`` keyword in +:option:`--tls-ticket-key-memcached` for TLS ticket key, and +:option:`--tls-session-cache-memcached` for TLS session cache. + +Specifying additional server certificates +----------------------------------------- + +nghttpx accepts additional server private key and certificate pairs +using :option:`--subcert` option. It can be used multiple times. + +Specifying additional CA certificate +------------------------------------ + +By default, nghttpx tries to read CA certificate from system. But +depending on the system you use, this may fail or is not supported. +To specify CA certificate manually, use :option:`--cacert` option. +The specified file must be PEM format and can contain multiple +certificates. + +By default, nghttpx validates server's certificate. If you want to +turn off this validation, knowing this is really insecure and what you +are doing, you can use :option:`--insecure` option to disable +certificate validation. + +Read/write rate limit +--------------------- + +nghttpx supports transfer rate limiting on frontend connections. You +can do rate limit per frontend connection for reading and writing +individually. + +To perform rate limit for reading, use :option:`--read-rate` and +:option:`--read-burst` options. For writing, use +:option:`--write-rate` and :option:`--write-burst`. + +Please note that rate limit is performed on top of TCP and nothing to +do with HTTP/2 flow control. + +Rewriting location header field +------------------------------- + +nghttpx automatically rewrites location response header field if the +following all conditions satisfy: + +* In the default mode (:option:`--http2-proxy` is not used) +* :option:`--no-location-rewrite` is not used +* URI in location header field is an absolute URI +* URI in location header field includes non empty host component. +* host (without port) in URI in location header field must match the + host appearing in ``:authority`` or ``host`` header field. + +When rewrite happens, URI scheme is replaced with the ones used in +frontend, and authority is replaced with which appears in +``:authority``, or ``host`` request header field. ``:authority`` +header field has precedence over ``host``. + +Hot swapping +------------ + +nghttpx supports hot swapping using signals. The hot swapping in +nghttpx is multi step process. First send USR2 signal to nghttpx +process. It will do fork and execute new executable, using same +command-line arguments and environment variables. + +As of nghttpx version 1.20.0, that is all you have to do. The new +main process sends QUIT signal to the original process, when it is +ready to serve requests, to shut it down gracefully. + +For earlier versions of nghttpx, you have to do one more thing. At +this point, both current and new processes can accept requests. To +gracefully shutdown current process, send QUIT signal to current +nghttpx process. When all existing frontend connections are done, the +current process will exit. At this point, only new nghttpx process +exists and serves incoming requests. + +If you want to just reload configuration file without executing new +binary, send SIGHUP to nghttpx main process. + +Re-opening log files +-------------------- + +When rotating log files, it is desirable to re-open log files after +log rotation daemon renamed existing log files. To tell nghttpx to +re-open log files, send USR1 signal to nghttpx process. It will +re-open files specified by :option:`--accesslog-file` and +:option:`--errorlog-file` options. + +Multiple frontend addresses +--------------------------- + +nghttpx can listen on multiple frontend addresses. To specify them, +just use :option:`--frontend` (or its shorthand :option:`-f`) option +repeatedly. TLS can be enabled or disabled per frontend address +basis. For example, to listen on port 443 with TLS enabled, and on +port 80 without TLS: + +.. code-block:: text + + frontend=*,443 + frontend=*,80;no-tls + + +Multiple backend addresses +-------------------------- + +nghttpx supports multiple backend addresses. To specify them, just +use :option:`--backend` (or its shorthand :option:`-b`) option +repeatedly. For example, to use ``192.168.0.10:8080`` and +``192.168.0.11:8080``, use command-line like this: +``-b192.168.0.10,8080 -b192.168.0.11,8080``. In configuration file, +this looks like: + +.. code-block:: text + + backend=192.168.0.10,8080 + backend=192.168.0.11,8008 + +nghttpx can route request to different backend according to request +host and path. For example, to route request destined to host +``doc.example.com`` to backend server ``docserv:3000``, you can write +like so: + +.. code-block:: text + + backend=docserv,3000;doc.example.com/ + +When you write this option in command-line, you should enclose +argument with single or double quotes, since the character ``;`` has a +special meaning in shell. + +To route, request to request path ``/foo`` to backend server +``[::1]:8080``, you can write like so: + +.. code-block:: text + + backend=::1,8080;/foo + +If the last character of path pattern is ``/``, all request paths +which start with that pattern match: + +.. code-block:: text + + backend=::1,8080;/bar/ + +The request path ``/bar/buzz`` matches the ``/bar/``. + +You can use ``*`` at the end of the path pattern to make it wildcard +pattern. ``*`` must match at least one character: + +.. code-block:: text + + backend=::1,8080;/sample* + +The request path ``/sample1/foo`` matches the ``/sample*`` pattern. + +Of course, you can specify both host and request path at the same +time: + +.. code-block:: text + + backend=192.168.0.10,8080;example.com/foo + +We can use ``*`` in the left most position of host to achieve wildcard +suffix match. If ``*`` is the left most character, then the remaining +string should match the request host suffix. ``*`` must match at +least one character. For example, ``*.example.com`` matches +``www.example.com`` and ``dev.example.com``, and does not match +``example.com`` and ``nghttp2.org``. The exact match (without ``*``) +always takes precedence over wildcard match. + +One important thing you have to remember is that we have to specify +default routing pattern for so called "catch all" pattern. To write +"catch all" pattern, just specify backend server address, without +pattern. + +Usually, host is the value of ``Host`` header field. In HTTP/2, the +value of ``:authority`` pseudo header field is used. + +When you write multiple backend addresses sharing the same routing +pattern, they are used as load balancing. For example, to use 2 +servers ``serv1:3000`` and ``serv2:3000`` for request host +``example.com`` and path ``/myservice``, you can write like so: + +.. code-block:: text + + backend=serv1,3000;example.com/myservice + backend=serv2,3000;example.com/myservice + +You can also specify backend application protocol in +:option:`--backend` option using ``proto`` keyword after pattern. +Utilizing this allows ngttpx to route certain request to HTTP/2, other +requests to HTTP/1. For example, to route requests to ``/ws/`` in +backend HTTP/1.1 connection, and use backend HTTP/2 for other +requests, do this: + +.. code-block:: text + + backend=serv1,3000;/;proto=h2 + backend=serv1,3000;/ws/;proto=http/1.1 + +The default backend protocol is HTTP/1.1. + +TLS can be enabled per pattern basis: + +.. code-block:: text + + backend=serv1,8443;/;proto=h2;tls + backend=serv2,8080;/ws/;proto=http/1.1 + +In the above case, connection to serv1 will be encrypted by TLS. On +the other hand, connection to serv2 will not be encrypted by TLS. + +Dynamic hostname lookup +----------------------- + +By default, nghttpx performs backend hostname lookup at start up, or +configuration reload, and keeps using them in its entire session. To +make nghttpx perform hostname lookup dynamically, use ``dns`` +parameter in :option:`--backend` option, like so: + +.. code-block:: text + + backend=foo.example.com,80;;dns + +nghttpx will cache resolved addresses for certain period of time. To +change this cache period, use :option:`--dns-cache-timeout`. + +Enable PROXY protocol +--------------------- + +PROXY protocol can be enabled per frontend. In order to enable PROXY +protocol, use ``proxyproto`` parameter in :option:`--frontend` option, +like so: + +.. code-block:: text + + frontend=*,443;proxyproto + +nghttpx supports both PROXY protocol v1 and v2. AF_UNIX in PROXY +protocol version 2 is ignored. + +Session affinity +---------------- + +Two kinds of session affinity are available: client IP, and HTTP +Cookie. + +To enable client IP based affinity, specify ``affinity=ip`` parameter +in :option:`--backend` option. If PROXY protocol is enabled, then an +address obtained from PROXY protocol is taken into consideration. + +To enable HTTP Cookie based affinity, specify ``affinity=cookie`` +parameter, and specify a name of cookie in ``affinity-cookie-name`` +parameter. Optionally, a Path attribute can be specified in +``affinity-cookie-path`` parameter: + +.. code-block:: text + + backend=127.0.0.1,3000;;affinity=cookie;affinity-cookie-name=nghttpxlb;affinity-cookie-path=/ + +Secure attribute of cookie is set if client connection is protected by +TLS. ``affinity-cookie-stickiness`` specifies the stickiness of this +affinity. If ``loose`` is given, which is the default, removing or +adding a backend server might break affinity. While ``strict`` is +given, removing the designated backend server breaks affinity, but +adding new backend server does not cause breakage. + +PSK cipher suites +----------------- + +nghttpx supports pre-shared key (PSK) cipher suites for both frontend +and backend TLS connections. For frontend connection, use +:option:`--psk-secrets` option to specify a file which contains PSK +identity and secrets. The format of the file is +``<identity>:<hex-secret>``, where ``<identity>`` is PSK identity, and +``<hex-secret>`` is PSK secret in hex, like so: + +.. code-block:: text + + client1:9567800e065e078085c241d54a01c6c3f24b3bab71a606600f4c6ad2c134f3b9 + client2:b1376c3f8f6dcf7c886c5bdcceecd1e6f1d708622b6ddd21bda26ebd0c0bca99 + +nghttpx server accepts any of the identity and secret pairs in the +file. The default cipher suite list does not contain PSK cipher +suites. In order to use PSK, PSK cipher suite must be enabled by +using :option:`--ciphers` option. The desired PSK cipher suite may be +listed in `HTTP/2 cipher block list +<https://tools.ietf.org/html/rfc7540#appendix-A>`_. In order to use +such PSK cipher suite with HTTP/2, disable HTTP/2 cipher block list by +using :option:`--no-http2-cipher-block-list` option. But you should +understand its implications. + +At the time of writing, even if only PSK cipher suites are specified +in :option:`--ciphers` option, certificate and private key are still +required. + +For backend connection, use :option:`--client-psk-secrets` option to +specify a file which contains single PSK identity and secret. The +format is the same as the file used by :option:`--psk-secrets` +described above, but only first identity and secret pair is solely +used, like so: + +.. code-block:: text + + client2:b1376c3f8f6dcf7c886c5bdcceecd1e6f1d708622b6ddd21bda26ebd0c0bca99 + +The default cipher suite list does not contain PSK cipher suites. In +order to use PSK, PSK cipher suite must be enabled by using +:option:`--client-ciphers` option. The desired PSK cipher suite may +be listed in `HTTP/2 cipher block list +<https://tools.ietf.org/html/rfc7540#appendix-A>`_. In order to use +such PSK cipher suite with HTTP/2, disable HTTP/2 cipher block list by +using :option:`--client-no-http2-cipher-block-list` option. But you +should understand its implications. + +TLSv1.3 +------- + +As of nghttpx v1.34.0, if it is built with OpenSSL 1.1.1 or later, it +supports TLSv1.3. 0-RTT data is supported, but by default its +processing is postponed until TLS handshake completes to mitigate +replay attack. This costs extra round trip and reduces effectiveness +of 0-RTT data. :option:`--tls-no-postpone-early-data` makes nghttpx +not wait for handshake to complete before forwarding request included +in 0-RTT to get full potential of 0-RTT data. In this case, nghttpx +adds ``Early-Data: 1`` header field when forwarding a request to a +backend server. All backend servers should recognize this header +field and understand that there is a risk for replay attack. See `RFC +8470 <https://tools.ietf.org/html/rfc8470>`_ for ``Early-Data`` header +field. + +nghttpx disables anti replay protection provided by OpenSSL. The anti +replay protection of OpenSSL requires that a resumed request must hit +the same server which generates the session ticket. Therefore it +might not work nicely in a deployment where there are multiple nghttpx +instances sharing ticket encryption keys via memcached. + +Because TLSv1.3 completely changes the semantics of cipher suite +naming scheme and structure, nghttpx provides the new option +:option:`--tls13-ciphers` and :option:`--tls13-client-ciphers` to +change preferred cipher list for TLSv1.3. + +WebSockets over HTTP/2 +---------------------- + +nghttpx supports `RFC 8441 <https://tools.ietf.org/html/rfc8441>`_ +Bootstrapping WebSockets with HTTP/2 for both frontend and backend +connections. This feature is enabled by default and no configuration +is required. + +WebSockets over HTTP/3 is also supported. + +HTTP/3 +------ + +nghttpx supports HTTP/3 if it is built with HTTP/3 support enabled. +HTTP/3 support is experimental. + +In order to listen UDP port to receive HTTP/3 traffic, +:option:`--frontend` option must have ``quic`` parameter: + +.. code-block:: text + + frontend=*,443;quic + +The above example makes nghttpx receive HTTP/3 traffic on UDP +port 443. + +nghttpx does not support HTTP/3 on backend connection. + +Hot swapping (SIGUSR2) or configuration reload (SIGHUP) require eBPF +program. Without eBPF, old worker processes keep getting HTTP/3 +traffic and do not work as intended. The QUIC keying material to +encrypt Connection ID must be set with +:option:`--frontend-quic-secret-file` and must provide the existing +keys in order to keep the existing connections alive during reload. + +The construction of Connection ID closely follows Block Cipher CID +Algorithm described in `QUIC-LB draft +<https://datatracker.ietf.org/doc/html/draft-ietf-quic-load-balancers>`_. +A Connection ID that nghttpx generates is always 20 bytes long. It +uses first 2 bits as a configuration ID. The remaining bits in the +first byte are reserved and random. The next 4 bytes are server ID. +The next 4 bytes are used to route UDP datagram to a correct +``SO_REUSEPORT`` socket. The remaining bytes are randomly generated. +The server ID and the next 12 bytes are encrypted with AES-ECB. The +key is derived from the keying materials stored in a file specified by +:option:`--frontend-quic-secret-file`. The first 2 bits of keying +material in the file is used as a configuration ID. The remaining +bits and following 3 bytes are reserved and unused. The next 32 bytes +are used as an initial secret. The remaining 32 bytes are used as a +salt. The encryption key is generated by `HKDF +<https://datatracker.ietf.org/doc/html/rfc5869>`_ with SHA256 and +these keying materials and ``connection id encryption key`` as info. + +In order announce that HTTP/3 endpoint is available, you should +specify alt-svc header field. For example, the following options send +alt-svc header field in HTTP/1.1 and HTTP/2 response: + +.. code-block:: text + + altsvc=h3,443,,,ma=3600 + http2-altsvc=h3,443,,,ma=3600 + +Migration from nghttpx v1.18.x or earlier +----------------------------------------- + +As of nghttpx v1.19.0, :option:`--ciphers` option only changes cipher +list for frontend TLS connection. In order to change cipher list for +backend connection, use :option:`--client-ciphers` option. + +Similarly, :option:`--no-http2-cipher-block-list` option only disables +HTTP/2 cipher block list for frontend connection. In order to disable +HTTP/2 cipher block list for backend connection, use +:option:`--client-no-http2-cipher-block-list` option. + +``--accept-proxy-protocol`` option was deprecated. Instead, use +``proxyproto`` parameter in :option:`--frontend` option to enable +PROXY protocol support per frontend. + +Migration from nghttpx v1.8.0 or earlier +---------------------------------------- + +As of nghttpx 1.9.0, ``--frontend-no-tls`` and ``--backend-no-tls`` +have been removed. + +To disable encryption on frontend connection, use ``no-tls`` keyword +in :option:`--frontend` potion: + +.. code-block:: text + + frontend=*,3000;no-tls + +The TLS encryption is now disabled on backend connection in all modes +by default. To enable encryption on backend connection, use ``tls`` +keyword in :option:`--backend` option: + +.. code-block:: text + + backend=127.0.0.1,8080;tls + +As of nghttpx 1.9.0, ``--http2-bridge``, ``--client`` and +``--client-proxy`` options have been removed. These functionality can +be used using combinations of options. + +Use following option instead of ``--http2-bridge``: + +.. code-block:: text + + backend=<ADDR>,<PORT>;;proto=h2;tls + +Use following options instead of ``--client``: + +.. code-block:: text + + frontend=<ADDR>,<PORT>;no-tls + backend=<ADDR>,<PORT>;;proto=h2;tls + +Use following options instead of ``--client-proxy``: + +.. code-block:: text + + http2-proxy=yes + frontend=<ADDR>,<PORT>;no-tls + backend=<ADDR>,<PORT>;;proto=h2;tls + +We also removed ``--backend-http2-connections-per-worker`` option. It +was present because previously the number of backend h2 connection was +statically configured, and defaulted to 1. Now the number of backend +h2 connection is increased on demand. We know the maximum number of +concurrent streams per connection. When we push as many request as +the maximum concurrency to the one connection, we create another new +connection so that we can distribute load and avoid delay the request +processing. This is done automatically without any configuration. diff --git a/doc/sources/security.rst b/doc/sources/security.rst new file mode 100644 index 0000000..ab27eab --- /dev/null +++ b/doc/sources/security.rst @@ -0,0 +1,38 @@ +Security Process +================ + +If you find a vulnerability in our software, please send the email to +"tatsuhiro.t at gmail dot com" about its details instead of submitting +issues on github issue page. It is a standard practice not to +disclose vulnerability information publicly until a fixed version is +released, or mitigation is worked out. In the future, we may setup a +dedicated mail address for this purpose. + +If we identify that the reported issue is really a vulnerability, we +open a new security advisory draft using `GitHub security feature +<https://github.com/nghttp2/nghttp2/security>`_ and discuss the +mitigation and bug fixes there. The fixes are committed to the +private repository. + +We write the security advisory and get CVE number from GitHub +privately. We also discuss the disclosure date to the public. + +We make a new release with the fix at the same time when the +vulnerability is disclosed to public. + +At least 7 days before the public disclosure date, we will post +security advisory (which includes all the details of the vulnerability +and the possible mitigation strategies) and the patches to fix the +issue to `distros@openwall +<https://oss-security.openwall.org/wiki/mailing-lists/distros>`_ +mailing list. We also open a new issue on `nghttp2 issue tracker +<https://github.com/nghttp2/nghttp2/issues>`_ which notifies that the +upcoming release will have a security fix. The ``SECURITY`` label is +attached to this kind of issue. + +Before few hours of new release, we merge the fixes to the master +branch (and/or a release branch if necessary) and make a new release. +Security advisory is disclosed on GitHub. We also post the +vulnerability information to `oss-security +<https://oss-security.openwall.org/wiki/mailing-lists/oss-security>`_ +mailing list. diff --git a/doc/sources/tutorial-client.rst b/doc/sources/tutorial-client.rst new file mode 100644 index 0000000..7c086a8 --- /dev/null +++ b/doc/sources/tutorial-client.rst @@ -0,0 +1,498 @@ +Tutorial: HTTP/2 client +========================= + +In this tutorial, we are going to write a very primitive HTTP/2 +client. The complete source code, `libevent-client.c`_, is attached at +the end of this page. It also resides in the examples directory in +the archive or repository. + +This simple client takes a single HTTPS URI and retrieves the resource +at the URI. The synopsis is: + +.. code-block:: text + + $ libevent-client HTTPS_URI + +We use libevent in this tutorial to handle networking I/O. Please +note that nghttp2 itself does not depend on libevent. + +The client starts with some libevent and OpenSSL setup in the +``main()`` and ``run()`` functions. This setup isn't specific to +nghttp2, but one thing you should look at is setup of the NPN +callback. The NPN callback is used by the client to select the next +application protocol over TLS. In this tutorial, we use the +`nghttp2_select_next_protocol()` helper function to select the HTTP/2 +protocol the library supports:: + + static int select_next_proto_cb(SSL *ssl _U_, unsigned char **out, + unsigned char *outlen, const unsigned char *in, + unsigned int inlen, void *arg _U_) { + if (nghttp2_select_next_protocol(out, outlen, in, inlen) <= 0) { + errx(1, "Server did not advertise " NGHTTP2_PROTO_VERSION_ID); + } + return SSL_TLSEXT_ERR_OK; + } + +If you are following TLS related RFC, you know that NPN is not the +standardized way to negotiate HTTP/2. NPN itself is not event +published as RFC. The standard way to negotiate HTTP/2 is ALPN, +Application-Layer Protocol Negotiation Extension, defined in `RFC 7301 +<https://tools.ietf.org/html/rfc7301>`_. The one caveat of ALPN is +that OpenSSL >= 1.0.2 is required. We use macro to enable/disable +ALPN support depending on OpenSSL version. OpenSSL's ALPN +implementation does not require callback function like the above. But +we have to instruct OpenSSL SSL_CTX to use ALPN, which we'll talk +about soon. + +The callback is added to the SSL_CTX object using +``SSL_CTX_set_next_proto_select_cb()``:: + + static SSL_CTX *create_ssl_ctx(void) { + SSL_CTX *ssl_ctx; + ssl_ctx = SSL_CTX_new(SSLv23_client_method()); + if (!ssl_ctx) { + errx(1, "Could not create SSL/TLS context: %s", + ERR_error_string(ERR_get_error(), NULL)); + } + SSL_CTX_set_options(ssl_ctx, + SSL_OP_ALL | SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3 | + SSL_OP_NO_COMPRESSION | + SSL_OP_NO_SESSION_RESUMPTION_ON_RENEGOTIATION); + SSL_CTX_set_next_proto_select_cb(ssl_ctx, select_next_proto_cb, NULL); + + #if OPENSSL_VERSION_NUMBER >= 0x10002000L + SSL_CTX_set_alpn_protos(ssl_ctx, (const unsigned char *)"\x02h2", 3); + #endif // OPENSSL_VERSION_NUMBER >= 0x10002000L + + return ssl_ctx; + } + +Here we see ``SSL_CTX_get_alpn_protos()`` function call. We instructs +OpenSSL to notify the server that we support h2, ALPN identifier for +HTTP/2. + +The example client defines a couple of structs: + +We define and use a ``http2_session_data`` structure to store data +related to the HTTP/2 session:: + + typedef struct { + nghttp2_session *session; + struct evdns_base *dnsbase; + struct bufferevent *bev; + http2_stream_data *stream_data; + } http2_session_data; + +Since this program only handles one URI, it uses only one stream. We +store the single stream's data in a ``http2_stream_data`` structure +and the ``stream_data`` points to it. The ``http2_stream_data`` +structure is defined as follows:: + + typedef struct { + /* The NULL-terminated URI string to retrieve. */ + const char *uri; + /* Parsed result of the |uri| */ + struct http_parser_url *u; + /* The authority portion of the |uri|, not NULL-terminated */ + char *authority; + /* The path portion of the |uri|, including query, not + NULL-terminated */ + char *path; + /* The length of the |authority| */ + size_t authoritylen; + /* The length of the |path| */ + size_t pathlen; + /* The stream ID of this stream */ + int32_t stream_id; + } http2_stream_data; + +We create and initialize these structures in +``create_http2_session_data()`` and ``create_http2_stream_data()`` +respectively. + +``initiate_connection()`` is called to start the connection to the +remote server. It's defined as:: + + static void initiate_connection(struct event_base *evbase, SSL_CTX *ssl_ctx, + const char *host, uint16_t port, + http2_session_data *session_data) { + int rv; + struct bufferevent *bev; + SSL *ssl; + + ssl = create_ssl(ssl_ctx); + bev = bufferevent_openssl_socket_new( + evbase, -1, ssl, BUFFEREVENT_SSL_CONNECTING, + BEV_OPT_DEFER_CALLBACKS | BEV_OPT_CLOSE_ON_FREE); + bufferevent_enable(bev, EV_READ | EV_WRITE); + bufferevent_setcb(bev, readcb, writecb, eventcb, session_data); + rv = bufferevent_socket_connect_hostname(bev, session_data->dnsbase, + AF_UNSPEC, host, port); + + if (rv != 0) { + errx(1, "Could not connect to the remote host %s", host); + } + session_data->bev = bev; + } + +``initiate_connection()`` creates a bufferevent for the connection and +sets up three callbacks: ``readcb``, ``writecb``, and ``eventcb``. + +The ``eventcb()`` is invoked by the libevent event loop when an event +(e.g. connection has been established, timeout, etc.) occurs on the +underlying network socket:: + + static void eventcb(struct bufferevent *bev, short events, void *ptr) { + http2_session_data *session_data = (http2_session_data *)ptr; + if (events & BEV_EVENT_CONNECTED) { + int fd = bufferevent_getfd(bev); + int val = 1; + const unsigned char *alpn = NULL; + unsigned int alpnlen = 0; + SSL *ssl; + + fprintf(stderr, "Connected\n"); + + ssl = bufferevent_openssl_get_ssl(session_data->bev); + + SSL_get0_next_proto_negotiated(ssl, &alpn, &alpnlen); + #if OPENSSL_VERSION_NUMBER >= 0x10002000L + if (alpn == NULL) { + SSL_get0_alpn_selected(ssl, &alpn, &alpnlen); + } + #endif // OPENSSL_VERSION_NUMBER >= 0x10002000L + + if (alpn == NULL || alpnlen != 2 || memcmp("h2", alpn, 2) != 0) { + fprintf(stderr, "h2 is not negotiated\n"); + delete_http2_session_data(session_data); + return; + } + + setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, (char *)&val, sizeof(val)); + initialize_nghttp2_session(session_data); + send_client_connection_header(session_data); + submit_request(session_data); + if (session_send(session_data) != 0) { + delete_http2_session_data(session_data); + } + return; + } + if (events & BEV_EVENT_EOF) { + warnx("Disconnected from the remote host"); + } else if (events & BEV_EVENT_ERROR) { + warnx("Network error"); + } else if (events & BEV_EVENT_TIMEOUT) { + warnx("Timeout"); + } + delete_http2_session_data(session_data); + } + +Here we validate that HTTP/2 is negotiated, and if not, drop +connection. + +For ``BEV_EVENT_EOF``, ``BEV_EVENT_ERROR``, and ``BEV_EVENT_TIMEOUT`` +events, we just simply tear down the connection. + +The ``BEV_EVENT_CONNECTED`` event is invoked when the SSL/TLS +handshake has completed successfully. After this we're ready to begin +communicating via HTTP/2. + +The ``initialize_nghttp2_session()`` function initializes the nghttp2 +session object and several callbacks:: + + static void initialize_nghttp2_session(http2_session_data *session_data) { + nghttp2_session_callbacks *callbacks; + + nghttp2_session_callbacks_new(&callbacks); + + nghttp2_session_callbacks_set_send_callback(callbacks, send_callback); + + nghttp2_session_callbacks_set_on_frame_recv_callback(callbacks, + on_frame_recv_callback); + + nghttp2_session_callbacks_set_on_data_chunk_recv_callback( + callbacks, on_data_chunk_recv_callback); + + nghttp2_session_callbacks_set_on_stream_close_callback( + callbacks, on_stream_close_callback); + + nghttp2_session_callbacks_set_on_header_callback(callbacks, + on_header_callback); + + nghttp2_session_callbacks_set_on_begin_headers_callback( + callbacks, on_begin_headers_callback); + + nghttp2_session_client_new(&session_data->session, callbacks, session_data); + + nghttp2_session_callbacks_del(callbacks); + } + +Since we are creating a client, we use `nghttp2_session_client_new()` +to initialize the nghttp2 session object. The callbacks setup are +explained later. + +The `delete_http2_session_data()` function destroys ``session_data`` +and frees its bufferevent, so the underlying connection is closed. It +also calls `nghttp2_session_del()` to delete the nghttp2 session +object. + +A HTTP/2 connection begins by sending the client connection preface, +which is a 24 byte magic byte string (:macro:`NGHTTP2_CLIENT_MAGIC`), +followed by a SETTINGS frame. The 24 byte magic string is sent +automatically by nghttp2. We send the SETTINGS frame in +``send_client_connection_header()``:: + + static void send_client_connection_header(http2_session_data *session_data) { + nghttp2_settings_entry iv[1] = { + {NGHTTP2_SETTINGS_MAX_CONCURRENT_STREAMS, 100}}; + int rv; + + /* client 24 bytes magic string will be sent by nghttp2 library */ + rv = nghttp2_submit_settings(session_data->session, NGHTTP2_FLAG_NONE, iv, + ARRLEN(iv)); + if (rv != 0) { + errx(1, "Could not submit SETTINGS: %s", nghttp2_strerror(rv)); + } + } + +Here we specify SETTINGS_MAX_CONCURRENT_STREAMS as 100. This is not +needed for this tiny example program, it just demonstrates use of the +SETTINGS frame. To queue the SETTINGS frame for transmission, we call +`nghttp2_submit_settings()`. Note that `nghttp2_submit_settings()` +only queues the frame for transmission, and doesn't actually send it. +All ``nghttp2_submit_*()`` family functions have this property. To +actually send the frame, `nghttp2_session_send()` has to be called, +which is described (and called) later. + +After the transmission of the client connection header, we enqueue the +HTTP request in the ``submit_request()`` function:: + + static void submit_request(http2_session_data *session_data) { + int32_t stream_id; + http2_stream_data *stream_data = session_data->stream_data; + const char *uri = stream_data->uri; + const struct http_parser_url *u = stream_data->u; + nghttp2_nv hdrs[] = { + MAKE_NV2(":method", "GET"), + MAKE_NV(":scheme", &uri[u->field_data[UF_SCHEMA].off], + u->field_data[UF_SCHEMA].len), + MAKE_NV(":authority", stream_data->authority, stream_data->authoritylen), + MAKE_NV(":path", stream_data->path, stream_data->pathlen)}; + fprintf(stderr, "Request headers:\n"); + print_headers(stderr, hdrs, ARRLEN(hdrs)); + stream_id = nghttp2_submit_request(session_data->session, NULL, hdrs, + ARRLEN(hdrs), NULL, stream_data); + if (stream_id < 0) { + errx(1, "Could not submit HTTP request: %s", nghttp2_strerror(stream_id)); + } + + stream_data->stream_id = stream_id; + } + +We build the HTTP request header fields in ``hdrs``, which is an array +of :type:`nghttp2_nv`. There are four header fields to be sent: +``:method``, ``:scheme``, ``:authority``, and ``:path``. To queue the +HTTP request, we call `nghttp2_submit_request()`. The ``stream_data`` +is passed via the *stream_user_data* parameter, which is helpfully +later passed back to callback functions. + +`nghttp2_submit_request()` returns the newly assigned stream ID for +the request. + +The next bufferevent callback is ``readcb()``, which is invoked when +data is available to read from the bufferevent input buffer:: + + static void readcb(struct bufferevent *bev, void *ptr) { + http2_session_data *session_data = (http2_session_data *)ptr; + ssize_t readlen; + struct evbuffer *input = bufferevent_get_input(bev); + size_t datalen = evbuffer_get_length(input); + unsigned char *data = evbuffer_pullup(input, -1); + + readlen = nghttp2_session_mem_recv(session_data->session, data, datalen); + if (readlen < 0) { + warnx("Fatal error: %s", nghttp2_strerror((int)readlen)); + delete_http2_session_data(session_data); + return; + } + if (evbuffer_drain(input, (size_t)readlen) != 0) { + warnx("Fatal error: evbuffer_drain failed"); + delete_http2_session_data(session_data); + return; + } + if (session_send(session_data) != 0) { + delete_http2_session_data(session_data); + return; + } + } + +In this function we feed all unprocessed, received data to the nghttp2 +session object using the `nghttp2_session_mem_recv()` function. +`nghttp2_session_mem_recv()` processes the received data and may +invoke nghttp2 callbacks and queue frames for transmission. Since +there may be pending frames for transmission, we call immediately +``session_send()`` to send them. ``session_send()`` is defined as +follows:: + + static int session_send(http2_session_data *session_data) { + int rv; + + rv = nghttp2_session_send(session_data->session); + if (rv != 0) { + warnx("Fatal error: %s", nghttp2_strerror(rv)); + return -1; + } + return 0; + } + +The `nghttp2_session_send()` function serializes pending frames into +wire format and calls the ``send_callback()`` function to send them. +``send_callback()`` has type :type:`nghttp2_send_callback` and is +defined as:: + + static ssize_t send_callback(nghttp2_session *session _U_, const uint8_t *data, + size_t length, int flags _U_, void *user_data) { + http2_session_data *session_data = (http2_session_data *)user_data; + struct bufferevent *bev = session_data->bev; + bufferevent_write(bev, data, length); + return (ssize_t)length; + } + +Since we use bufferevent to abstract network I/O, we just write the +data to the bufferevent object. Note that `nghttp2_session_send()` +continues to write all frames queued so far. If we were writing the +data to the non-blocking socket directly using the ``write()`` system +call, we'd soon receive an ``EAGAIN`` or ``EWOULDBLOCK`` error, since +sockets have a limited send buffer. If that happens, it's possible to +return :macro:`NGHTTP2_ERR_WOULDBLOCK` to signal the nghttp2 library +to stop sending further data. When writing to a bufferevent, you +should regulate the amount of data written, to avoid possible huge +memory consumption. In this example client however we don't implement +a limit. To see how to regulate the amount of buffered data, see the +``send_callback()`` in the server tutorial. + +The third bufferevent callback is ``writecb()``, which is invoked when +all data written in the bufferevent output buffer has been sent:: + + static void writecb(struct bufferevent *bev _U_, void *ptr) { + http2_session_data *session_data = (http2_session_data *)ptr; + if (nghttp2_session_want_read(session_data->session) == 0 && + nghttp2_session_want_write(session_data->session) == 0 && + evbuffer_get_length(bufferevent_get_output(session_data->bev)) == 0) { + delete_http2_session_data(session_data); + } + } + +As described earlier, we just write off all data in `send_callback()`, +so there is no data to write in this function. All we have to do is +check if the connection should be dropped or not. The nghttp2 session +object keeps track of reception and transmission of GOAWAY frames and +other error conditions. Using this information, the nghttp2 session +object can state whether the connection should be dropped or not. +More specifically, when both `nghttp2_session_want_read()` and +`nghttp2_session_want_write()` return 0, the connection is no-longer +required and can be closed. Since we're using bufferevent and its +deferred callback option, the bufferevent output buffer may still +contain pending data when the ``writecb()`` is called. To handle this +situation, we also check whether the output buffer is empty or not. If +all of these conditions are met, then we drop the connection. + +Now let's look at the remaining nghttp2 callbacks setup in the +``initialize_nghttp2_setup()`` function. + +A server responds to the request by first sending a HEADERS frame. +The HEADERS frame consists of response header name/value pairs, and +the ``on_header_callback()`` is called for each name/value pair:: + + static int on_header_callback(nghttp2_session *session _U_, + const nghttp2_frame *frame, const uint8_t *name, + size_t namelen, const uint8_t *value, + size_t valuelen, uint8_t flags _U_, + void *user_data) { + http2_session_data *session_data = (http2_session_data *)user_data; + switch (frame->hd.type) { + case NGHTTP2_HEADERS: + if (frame->headers.cat == NGHTTP2_HCAT_RESPONSE && + session_data->stream_data->stream_id == frame->hd.stream_id) { + /* Print response headers for the initiated request. */ + print_header(stderr, name, namelen, value, valuelen); + break; + } + } + return 0; + } + +In this tutorial, we just print the name/value pairs on stderr. + +After the HEADERS frame has been fully received (and thus all response +header name/value pairs have been received), the +``on_frame_recv_callback()`` function is called:: + + static int on_frame_recv_callback(nghttp2_session *session _U_, + const nghttp2_frame *frame, void *user_data) { + http2_session_data *session_data = (http2_session_data *)user_data; + switch (frame->hd.type) { + case NGHTTP2_HEADERS: + if (frame->headers.cat == NGHTTP2_HCAT_RESPONSE && + session_data->stream_data->stream_id == frame->hd.stream_id) { + fprintf(stderr, "All headers received\n"); + } + break; + } + return 0; + } + +``on_frame_recv_callback()`` is called for other frame types too. + +In this tutorial, we are just interested in the HTTP response HEADERS +frame. We check the frame type and its category (it should be +:macro:`NGHTTP2_HCAT_RESPONSE` for HTTP response HEADERS). We also +check its stream ID. + +Next, zero or more DATA frames can be received. The +``on_data_chunk_recv_callback()`` function is invoked when a chunk of +data is received from the remote peer:: + + static int on_data_chunk_recv_callback(nghttp2_session *session _U_, + uint8_t flags _U_, int32_t stream_id, + const uint8_t *data, size_t len, + void *user_data) { + http2_session_data *session_data = (http2_session_data *)user_data; + if (session_data->stream_data->stream_id == stream_id) { + fwrite(data, len, 1, stdout); + } + return 0; + } + +In our case, a chunk of data is HTTP response body. After checking the +stream ID, we just write the received data to stdout. Note the output +in the terminal may be corrupted if the response body contains some +binary data. + +The ``on_stream_close_callback()`` function is invoked when the stream +is about to close:: + + static int on_stream_close_callback(nghttp2_session *session, int32_t stream_id, + nghttp2_error_code error_code, + void *user_data) { + http2_session_data *session_data = (http2_session_data *)user_data; + int rv; + + if (session_data->stream_data->stream_id == stream_id) { + fprintf(stderr, "Stream %d closed with error_code=%d\n", stream_id, + error_code); + rv = nghttp2_session_terminate_session(session, NGHTTP2_NO_ERROR); + if (rv != 0) { + return NGHTTP2_ERR_CALLBACK_FAILURE; + } + } + return 0; + } + +If the stream ID matches the one we initiated, it means that its +stream is going to be closed. Since we have finished receiving +resource we wanted (or the stream was reset by RST_STREAM from the +remote peer), we call `nghttp2_session_terminate_session()` to +commence closure of the HTTP/2 session gracefully. If you have +some data associated for the stream to be closed, you may delete it +here. diff --git a/doc/sources/tutorial-hpack.rst b/doc/sources/tutorial-hpack.rst new file mode 100644 index 0000000..36e82d9 --- /dev/null +++ b/doc/sources/tutorial-hpack.rst @@ -0,0 +1,126 @@ +Tutorial: HPACK API +=================== + +In this tutorial, we describe basic use of nghttp2's HPACK API. We +briefly describe the APIs for deflating and inflating header fields. +The full example of using these APIs, `deflate.c`_, is attached at the +end of this page. It also resides in the examples directory in the +archive or repository. + +Deflating (encoding) headers +---------------------------- + +First we need to initialize a :type:`nghttp2_hd_deflater` object using +the `nghttp2_hd_deflate_new()` function:: + + int nghttp2_hd_deflate_new(nghttp2_hd_deflater **deflater_ptr, + size_t deflate_hd_table_bufsize_max); + +This function allocates a :type:`nghttp2_hd_deflater` object, +initializes it, and assigns its pointer to ``*deflater_ptr``. The +*deflate_hd_table_bufsize_max* is the upper bound of header table size +the deflater will use. This will limit the memory usage by the +deflater object for the dynamic header table. If in doubt, just +specify 4096 here, which is the default upper bound of dynamic header +table buffer size. + +To encode header fields, use the `nghttp2_hd_deflate_hd()` function:: + + ssize_t nghttp2_hd_deflate_hd(nghttp2_hd_deflater *deflater, + uint8_t *buf, size_t buflen, + const nghttp2_nv *nva, size_t nvlen); + +The *deflater* is the deflater object initialized by +`nghttp2_hd_deflate_new()` described above. The encoded byte string is +written to the buffer *buf*, which has length *buflen*. The *nva* is +a pointer to an array of headers fields, each of type +:type:`nghttp2_nv`. *nvlen* is the number of header fields which +*nva* contains. + +It is important to initialize and assign all members of +:type:`nghttp2_nv`. For security sensitive header fields (such as +cookies), set the :macro:`NGHTTP2_NV_FLAG_NO_INDEX` flag in +:member:`nghttp2_nv.flags`. Setting this flag prevents recovery of +sensitive header fields by compression based attacks: This is achieved +by not inserting the header field into the dynamic header table. + +`nghttp2_hd_deflate_hd()` processes all headers given in *nva*. The +*nva* must include all request or response header fields to be sent in +one HEADERS (or optionally following (multiple) CONTINUATION +frame(s)). The *buf* must have enough space to store the encoded +result, otherwise the function will fail. To estimate the upper bound +of the encoded result length, use `nghttp2_hd_deflate_bound()`:: + + size_t nghttp2_hd_deflate_bound(nghttp2_hd_deflater *deflater, + const nghttp2_nv *nva, size_t nvlen); + +Pass this function the same parameters (*deflater*, *nva*, and +*nvlen*) which will be passed to `nghttp2_hd_deflate_hd()`. + +Subsequent calls to `nghttp2_hd_deflate_hd()` will use the current +encoder state and perform differential encoding, which yields HPAC's +fundamental compression gain. + +If `nghttp2_hd_deflate_hd()` fails, the failure is fatal and any +further calls with the same deflater object will fail. Thus it's very +important to use `nghttp2_hd_deflate_bound()` to determine the +required size of the output buffer. + +To delete a :type:`nghttp2_hd_deflater` object, use the +`nghttp2_hd_deflate_del()` function. + +Inflating (decoding) headers +---------------------------- + +A :type:`nghttp2_hd_inflater` object is used to inflate compressed +header data. To initialize the object, use +`nghttp2_hd_inflate_new()`:: + + int nghttp2_hd_inflate_new(nghttp2_hd_inflater **inflater_ptr); + +To inflate header data, use `nghttp2_hd_inflate_hd2()`:: + + ssize_t nghttp2_hd_inflate_hd2(nghttp2_hd_inflater *inflater, + nghttp2_nv *nv_out, int *inflate_flags, + const uint8_t *in, size_t inlen, + int in_final); + +`nghttp2_hd_inflate_hd2()` reads a stream of bytes and outputs a +single header field at a time. Multiple calls are normally required to +read a full stream of bytes and output all of the header fields. + +The *inflater* is the inflater object initialized above. The *nv_out* +is a pointer to a :type:`nghttp2_nv` into which one header field may +be stored. The *in* is a pointer to input data, and *inlen* is its +length. The caller is not required to specify the whole deflated +header data via *in* at once: Instead it can call this function +multiple times as additional data bytes become available. If +*in_final* is nonzero, it tells the function that the passed data is +the final sequence of deflated header data. + +The *inflate_flags* is an output parameter; on success the function +sets it to a bitset of flags. It will be described later. + +This function returns when each header field is inflated. When this +happens, the function sets the :macro:`NGHTTP2_HD_INFLATE_EMIT` flag +in *inflate_flags*, and a header field is stored in *nv_out*. The +return value indicates the number of bytes read from *in* processed so +far, which may be less than *inlen*. The caller should call the +function repeatedly until all bytes are processed. Processed bytes +should be removed from *in*, and *inlen* should be adjusted +appropriately. + +If *in_final* is nonzero and all given data was processed, the +function sets the :macro:`NGHTTP2_HD_INFLATE_FINAL` flag in +*inflate_flags*. When you see this flag set, call the +`nghttp2_hd_inflate_end_headers()` function. + +If *in_final* is zero and the :macro:`NGHTTP2_HD_INFLATE_EMIT` flag is +not set, it indicates that all given data was processed. The caller +is required to pass additional data. + +Example usage of `nghttp2_hd_inflate_hd2()` is shown in the +`inflate_header_block()` function in `deflate.c`_. + +Finally, to delete a :type:`nghttp2_hd_inflater` object, use +`nghttp2_hd_inflate_del()`. diff --git a/doc/sources/tutorial-server.rst b/doc/sources/tutorial-server.rst new file mode 100644 index 0000000..3142837 --- /dev/null +++ b/doc/sources/tutorial-server.rst @@ -0,0 +1,625 @@ +Tutorial: HTTP/2 server +========================= + +In this tutorial, we are going to write a single-threaded, event-based +HTTP/2 web server, which supports HTTPS only. It can handle concurrent +multiple requests, but only the GET method is supported. The complete +source code, `libevent-server.c`_, is attached at the end of this +page. The source also resides in the examples directory in the +archive or repository. + +This simple server takes 3 arguments: The port number to listen on, +the path to your SSL/TLS private key file, and the path to your +certificate file. The synopsis is: + +.. code-block:: text + + $ libevent-server PORT /path/to/server.key /path/to/server.crt + +We use libevent in this tutorial to handle networking I/O. Please +note that nghttp2 itself does not depend on libevent. + +The server starts with some libevent and OpenSSL setup in the +``main()`` and ``run()`` functions. This setup isn't specific to +nghttp2, but one thing you should look at is setup of the NPN +callback. The NPN callback is used by the server to advertise which +application protocols the server supports to a client. In this +example program, when creating the ``SSL_CTX`` object, we store the +application protocol name in the wire format of NPN in a statically +allocated buffer. This is safe because we only create one ``SSL_CTX`` +object in the program's entire lifetime. + +If you are following TLS related RFC, you know that NPN is not the +standardized way to negotiate HTTP/2. NPN itself is not even +published as RFC. The standard way to negotiate HTTP/2 is ALPN, +Application-Layer Protocol Negotiation Extension, defined in `RFC 7301 +<https://tools.ietf.org/html/rfc7301>`_. The one caveat of ALPN is +that OpenSSL >= 1.0.2 is required. We use macro to enable/disable +ALPN support depending on OpenSSL version. In ALPN, client sends the +list of supported application protocols, and server selects one of +them. We provide the callback for it:: + + static unsigned char next_proto_list[256]; + static size_t next_proto_list_len; + + static int next_proto_cb(SSL *s _U_, const unsigned char **data, + unsigned int *len, void *arg _U_) { + *data = next_proto_list; + *len = (unsigned int)next_proto_list_len; + return SSL_TLSEXT_ERR_OK; + } + + #if OPENSSL_VERSION_NUMBER >= 0x10002000L + static int alpn_select_proto_cb(SSL *ssl _U_, const unsigned char **out, + unsigned char *outlen, const unsigned char *in, + unsigned int inlen, void *arg _U_) { + int rv; + + rv = nghttp2_select_next_protocol((unsigned char **)out, outlen, in, inlen); + + if (rv != 1) { + return SSL_TLSEXT_ERR_NOACK; + } + + return SSL_TLSEXT_ERR_OK; + } + #endif // OPENSSL_VERSION_NUMBER >= 0x10002000L + + static SSL_CTX *create_ssl_ctx(const char *key_file, const char *cert_file) { + SSL_CTX *ssl_ctx; + EC_KEY *ecdh; + + ssl_ctx = SSL_CTX_new(SSLv23_server_method()); + + ... + + next_proto_list[0] = NGHTTP2_PROTO_VERSION_ID_LEN; + memcpy(&next_proto_list[1], NGHTTP2_PROTO_VERSION_ID, + NGHTTP2_PROTO_VERSION_ID_LEN); + next_proto_list_len = 1 + NGHTTP2_PROTO_VERSION_ID_LEN; + + SSL_CTX_set_next_protos_advertised_cb(ssl_ctx, next_proto_cb, NULL); + + #if OPENSSL_VERSION_NUMBER >= 0x10002000L + SSL_CTX_set_alpn_select_cb(ssl_ctx, alpn_select_proto_cb, NULL); + #endif // OPENSSL_VERSION_NUMBER >= 0x10002000L + + return ssl_ctx; + } + +The wire format of NPN is a sequence of length prefixed strings, with +exactly one byte used to specify the length of each protocol +identifier. In this tutorial, we advertise the specific HTTP/2 +protocol version the current nghttp2 library supports, which is +exported in the identifier :macro:`NGHTTP2_PROTO_VERSION_ID`. The +``next_proto_cb()`` function is the server-side NPN callback. In the +OpenSSL implementation, we just assign the pointer to the NPN buffers +we filled in earlier. The NPN callback function is set to the +``SSL_CTX`` object using ``SSL_CTX_set_next_protos_advertised_cb()``. + +In ``alpn_select_proto_cb()``, we use `nghttp2_select_next_protocol()` +to select application protocol. The `nghttp2_select_next_protocol()` +returns 1 only if it selected h2 (ALPN identifier for HTTP/2), and out +parameters were assigned accordingly. + +Next, let's take a look at the main structures used by the example +application: + +We use the ``app_context`` structure to store application-wide data:: + + struct app_context { + SSL_CTX *ssl_ctx; + struct event_base *evbase; + }; + +We use the ``http2_session_data`` structure to store session-level +(which corresponds to one HTTP/2 connection) data:: + + typedef struct http2_session_data { + struct http2_stream_data root; + struct bufferevent *bev; + app_context *app_ctx; + nghttp2_session *session; + char *client_addr; + } http2_session_data; + +We use the ``http2_stream_data`` structure to store stream-level data:: + + typedef struct http2_stream_data { + struct http2_stream_data *prev, *next; + char *request_path; + int32_t stream_id; + int fd; + } http2_stream_data; + +A single HTTP/2 session can have multiple streams. To manage them, we +use a doubly linked list: The first element of this list is pointed +to by the ``root->next`` in ``http2_session_data``. Initially, +``root->next`` is ``NULL``. + +libevent's bufferevent structure is used to perform network I/O, with +the pointer to the bufferevent stored in the ``http2_session_data`` +structure. Note that the bufferevent object is kept in +``http2_session_data`` and not in ``http2_stream_data``. This is +because ``http2_stream_data`` is just a logical stream multiplexed +over the single connection managed by the bufferevent in +``http2_session_data``. + +We first create a listener object to accept incoming connections. +libevent's ``struct evconnlistener`` is used for this purpose:: + + static void start_listen(struct event_base *evbase, const char *service, + app_context *app_ctx) { + int rv; + struct addrinfo hints; + struct addrinfo *res, *rp; + + memset(&hints, 0, sizeof(hints)); + hints.ai_family = AF_UNSPEC; + hints.ai_socktype = SOCK_STREAM; + hints.ai_flags = AI_PASSIVE; + #ifdef AI_ADDRCONFIG + hints.ai_flags |= AI_ADDRCONFIG; + #endif /* AI_ADDRCONFIG */ + + rv = getaddrinfo(NULL, service, &hints, &res); + if (rv != 0) { + errx(1, NULL); + } + for (rp = res; rp; rp = rp->ai_next) { + struct evconnlistener *listener; + listener = evconnlistener_new_bind( + evbase, acceptcb, app_ctx, LEV_OPT_CLOSE_ON_FREE | LEV_OPT_REUSEABLE, + 16, rp->ai_addr, (int)rp->ai_addrlen); + if (listener) { + freeaddrinfo(res); + + return; + } + } + errx(1, "Could not start listener"); + } + +We specify the ``acceptcb`` callback, which is called when a new connection is +accepted:: + + static void acceptcb(struct evconnlistener *listener _U_, int fd, + struct sockaddr *addr, int addrlen, void *arg) { + app_context *app_ctx = (app_context *)arg; + http2_session_data *session_data; + + session_data = create_http2_session_data(app_ctx, fd, addr, addrlen); + + bufferevent_setcb(session_data->bev, readcb, writecb, eventcb, session_data); + } + +Here we create the ``http2_session_data`` object. The connection's +bufferevent is initialized at the same time. We specify three +callbacks for the bufferevent: ``readcb``, ``writecb``, and +``eventcb``. + +The ``eventcb()`` callback is invoked by the libevent event loop when an event +(e.g. connection has been established, timeout, etc.) occurs on the +underlying network socket:: + + static void eventcb(struct bufferevent *bev _U_, short events, void *ptr) { + http2_session_data *session_data = (http2_session_data *)ptr; + if (events & BEV_EVENT_CONNECTED) { + const unsigned char *alpn = NULL; + unsigned int alpnlen = 0; + SSL *ssl; + + fprintf(stderr, "%s connected\n", session_data->client_addr); + + ssl = bufferevent_openssl_get_ssl(session_data->bev); + + SSL_get0_next_proto_negotiated(ssl, &alpn, &alpnlen); + #if OPENSSL_VERSION_NUMBER >= 0x10002000L + if (alpn == NULL) { + SSL_get0_alpn_selected(ssl, &alpn, &alpnlen); + } + #endif // OPENSSL_VERSION_NUMBER >= 0x10002000L + + if (alpn == NULL || alpnlen != 2 || memcmp("h2", alpn, 2) != 0) { + fprintf(stderr, "%s h2 is not negotiated\n", session_data->client_addr); + delete_http2_session_data(session_data); + return; + } + + initialize_nghttp2_session(session_data); + + if (send_server_connection_header(session_data) != 0 || + session_send(session_data) != 0) { + delete_http2_session_data(session_data); + return; + } + + return; + } + if (events & BEV_EVENT_EOF) { + fprintf(stderr, "%s EOF\n", session_data->client_addr); + } else if (events & BEV_EVENT_ERROR) { + fprintf(stderr, "%s network error\n", session_data->client_addr); + } else if (events & BEV_EVENT_TIMEOUT) { + fprintf(stderr, "%s timeout\n", session_data->client_addr); + } + delete_http2_session_data(session_data); + } + +Here we validate that HTTP/2 is negotiated, and if not, drop +connection. + +For the ``BEV_EVENT_EOF``, ``BEV_EVENT_ERROR``, and +``BEV_EVENT_TIMEOUT`` events, we just simply tear down the connection. +The ``delete_http2_session_data()`` function destroys the +``http2_session_data`` object and its associated bufferevent member. +As a result, the underlying connection is closed. + +The +``BEV_EVENT_CONNECTED`` event is invoked when SSL/TLS handshake has +completed successfully. After this we are ready to begin communicating +via HTTP/2. + +The ``initialize_nghttp2_session()`` function initializes the nghttp2 +session object and several callbacks:: + + static void initialize_nghttp2_session(http2_session_data *session_data) { + nghttp2_session_callbacks *callbacks; + + nghttp2_session_callbacks_new(&callbacks); + + nghttp2_session_callbacks_set_send_callback(callbacks, send_callback); + + nghttp2_session_callbacks_set_on_frame_recv_callback(callbacks, + on_frame_recv_callback); + + nghttp2_session_callbacks_set_on_stream_close_callback( + callbacks, on_stream_close_callback); + + nghttp2_session_callbacks_set_on_header_callback(callbacks, + on_header_callback); + + nghttp2_session_callbacks_set_on_begin_headers_callback( + callbacks, on_begin_headers_callback); + + nghttp2_session_server_new(&session_data->session, callbacks, session_data); + + nghttp2_session_callbacks_del(callbacks); + } + +Since we are creating a server, we use `nghttp2_session_server_new()` +to initialize the nghttp2 session object. We also setup 5 callbacks +for the nghttp2 session, these are explained later. + +The server now begins by sending the server connection preface, which +always consists of a SETTINGS frame. +``send_server_connection_header()`` configures and submits it:: + + static int send_server_connection_header(http2_session_data *session_data) { + nghttp2_settings_entry iv[1] = { + {NGHTTP2_SETTINGS_MAX_CONCURRENT_STREAMS, 100}}; + int rv; + + rv = nghttp2_submit_settings(session_data->session, NGHTTP2_FLAG_NONE, iv, + ARRLEN(iv)); + if (rv != 0) { + warnx("Fatal error: %s", nghttp2_strerror(rv)); + return -1; + } + return 0; + } + +In the example SETTINGS frame we've set +SETTINGS_MAX_CONCURRENT_STREAMS to 100. `nghttp2_submit_settings()` +is used to queue the frame for transmission, but note it only queues +the frame for transmission, and doesn't actually send it. All +functions in the ``nghttp2_submit_*()`` family have this property. To +actually send the frame, `nghttp2_session_send()` should be used, as +described later. + +Since bufferevent may buffer more than the first 24 bytes from the client, we +have to process them here since libevent won't invoke callback functions for +this pending data. To process the received data, we call the +``session_recv()`` function:: + + static int session_recv(http2_session_data *session_data) { + ssize_t readlen; + struct evbuffer *input = bufferevent_get_input(session_data->bev); + size_t datalen = evbuffer_get_length(input); + unsigned char *data = evbuffer_pullup(input, -1); + + readlen = nghttp2_session_mem_recv(session_data->session, data, datalen); + if (readlen < 0) { + warnx("Fatal error: %s", nghttp2_strerror((int)readlen)); + return -1; + } + if (evbuffer_drain(input, (size_t)readlen) != 0) { + warnx("Fatal error: evbuffer_drain failed"); + return -1; + } + if (session_send(session_data) != 0) { + return -1; + } + return 0; + } + +In this function, we feed all unprocessed but already received data to +the nghttp2 session object using the `nghttp2_session_mem_recv()` +function. The `nghttp2_session_mem_recv()` function processes the data +and may both invoke the previously setup callbacks and also queue +outgoing frames. To send any pending outgoing frames, we immediately +call ``session_send()``. + +The ``session_send()`` function is defined as follows:: + + static int session_send(http2_session_data *session_data) { + int rv; + rv = nghttp2_session_send(session_data->session); + if (rv != 0) { + warnx("Fatal error: %s", nghttp2_strerror(rv)); + return -1; + } + return 0; + } + +The `nghttp2_session_send()` function serializes the frame into wire +format and calls the ``send_callback()``, which is of type +:type:`nghttp2_send_callback`. The ``send_callback()`` is defined as +follows:: + + static ssize_t send_callback(nghttp2_session *session _U_, const uint8_t *data, + size_t length, int flags _U_, void *user_data) { + http2_session_data *session_data = (http2_session_data *)user_data; + struct bufferevent *bev = session_data->bev; + /* Avoid excessive buffering in server side. */ + if (evbuffer_get_length(bufferevent_get_output(session_data->bev)) >= + OUTPUT_WOULDBLOCK_THRESHOLD) { + return NGHTTP2_ERR_WOULDBLOCK; + } + bufferevent_write(bev, data, length); + return (ssize_t)length; + } + +Since we use bufferevent to abstract network I/O, we just write the +data to the bufferevent object. Note that `nghttp2_session_send()` +continues to write all frames queued so far. If we were writing the +data to a non-blocking socket directly using the ``write()`` system +call in the ``send_callback()``, we'd soon receive an ``EAGAIN`` or +``EWOULDBLOCK`` error since sockets have a limited send buffer. If +that happens, it's possible to return :macro:`NGHTTP2_ERR_WOULDBLOCK` +to signal the nghttp2 library to stop sending further data. But here, +when writing to the bufferevent, we have to regulate the amount data +to buffered ourselves to avoid using huge amounts of memory. To +achieve this, we check the size of the output buffer and if it reaches +more than or equal to ``OUTPUT_WOULDBLOCK_THRESHOLD`` bytes, we stop +writing data and return :macro:`NGHTTP2_ERR_WOULDBLOCK`. + +The next bufferevent callback is ``readcb()``, which is invoked when +data is available to read in the bufferevent input buffer:: + + static void readcb(struct bufferevent *bev _U_, void *ptr) { + http2_session_data *session_data = (http2_session_data *)ptr; + if (session_recv(session_data) != 0) { + delete_http2_session_data(session_data); + return; + } + } + +In this function, we just call ``session_recv()`` to process incoming +data. + +The third bufferevent callback is ``writecb()``, which is invoked when all +data in the bufferevent output buffer has been sent:: + + static void writecb(struct bufferevent *bev, void *ptr) { + http2_session_data *session_data = (http2_session_data *)ptr; + if (evbuffer_get_length(bufferevent_get_output(bev)) > 0) { + return; + } + if (nghttp2_session_want_read(session_data->session) == 0 && + nghttp2_session_want_write(session_data->session) == 0) { + delete_http2_session_data(session_data); + return; + } + if (session_send(session_data) != 0) { + delete_http2_session_data(session_data); + return; + } + } + +First we check whether we should drop the connection or not. The +nghttp2 session object keeps track of reception and transmission of +GOAWAY frames and other error conditions as well. Using this +information, the nghttp2 session object can state whether the +connection should be dropped or not. More specifically, if both +`nghttp2_session_want_read()` and `nghttp2_session_want_write()` +return 0, the connection is no-longer required and can be closed. +Since we are using bufferevent and its deferred callback option, the +bufferevent output buffer may still contain pending data when the +``writecb()`` is called. To handle this, we check whether the output +buffer is empty or not. If all of these conditions are met, we drop +connection. + +Otherwise, we call ``session_send()`` to process the pending output +data. Remember that in ``send_callback()``, we must not write all data to +bufferevent to avoid excessive buffering. We continue processing pending data +when the output buffer becomes empty. + +We have already described the nghttp2 callback ``send_callback()``. Let's +learn about the remaining nghttp2 callbacks setup in +``initialize_nghttp2_setup()`` function. + +The ``on_begin_headers_callback()`` function is invoked when the reception of +a header block in HEADERS or PUSH_PROMISE frame is started:: + + static int on_begin_headers_callback(nghttp2_session *session, + const nghttp2_frame *frame, + void *user_data) { + http2_session_data *session_data = (http2_session_data *)user_data; + http2_stream_data *stream_data; + + if (frame->hd.type != NGHTTP2_HEADERS || + frame->headers.cat != NGHTTP2_HCAT_REQUEST) { + return 0; + } + stream_data = create_http2_stream_data(session_data, frame->hd.stream_id); + nghttp2_session_set_stream_user_data(session, frame->hd.stream_id, + stream_data); + return 0; + } + +We are only interested in the HEADERS frame in this function. Since +the HEADERS frame has several roles in the HTTP/2 protocol, we check +that it is a request HEADERS, which opens new stream. If the frame is +a request HEADERS, we create a ``http2_stream_data`` object to store +the stream related data. We associate the created +``http2_stream_data`` object with the stream in the nghttp2 session +object using `nghttp2_set_stream_user_data()`. The +``http2_stream_data`` object can later be easily retrieved from the +stream, without searching through the doubly linked list. + +In this example server, we want to serve files relative to the current working +directory in which the program was invoked. Each header name/value pair is +emitted via ``on_header_callback`` function, which is called after +``on_begin_headers_callback()``:: + + static int on_header_callback(nghttp2_session *session, + const nghttp2_frame *frame, const uint8_t *name, + size_t namelen, const uint8_t *value, + size_t valuelen, uint8_t flags _U_, + void *user_data _U_) { + http2_stream_data *stream_data; + const char PATH[] = ":path"; + switch (frame->hd.type) { + case NGHTTP2_HEADERS: + if (frame->headers.cat != NGHTTP2_HCAT_REQUEST) { + break; + } + stream_data = + nghttp2_session_get_stream_user_data(session, frame->hd.stream_id); + if (!stream_data || stream_data->request_path) { + break; + } + if (namelen == sizeof(PATH) - 1 && memcmp(PATH, name, namelen) == 0) { + size_t j; + for (j = 0; j < valuelen && value[j] != '?'; ++j) + ; + stream_data->request_path = percent_decode(value, j); + } + break; + } + return 0; + } + +We search for the ``:path`` header field among the request headers and +store the requested path in the ``http2_stream_data`` object. In this +example program, we ignore the ``:method`` header field and always +treat the request as a GET request. + +The ``on_frame_recv_callback()`` function is invoked when a frame is +fully received:: + + static int on_frame_recv_callback(nghttp2_session *session, + const nghttp2_frame *frame, void *user_data) { + http2_session_data *session_data = (http2_session_data *)user_data; + http2_stream_data *stream_data; + switch (frame->hd.type) { + case NGHTTP2_DATA: + case NGHTTP2_HEADERS: + /* Check that the client request has finished */ + if (frame->hd.flags & NGHTTP2_FLAG_END_STREAM) { + stream_data = + nghttp2_session_get_stream_user_data(session, frame->hd.stream_id); + /* For DATA and HEADERS frame, this callback may be called after + on_stream_close_callback. Check that stream still alive. */ + if (!stream_data) { + return 0; + } + return on_request_recv(session, session_data, stream_data); + } + break; + default: + break; + } + return 0; + } + +First we retrieve the ``http2_stream_data`` object associated with the +stream in ``on_begin_headers_callback()`` using +`nghttp2_session_get_stream_user_data()`. If the requested path +cannot be served for some reason (e.g. file is not found), we send a +404 response using ``error_reply()``. Otherwise, we open +the requested file and send its content. We send the header field +``:status`` as a single response header. + +Sending the file content is performed by the ``send_response()`` function:: + + static int send_response(nghttp2_session *session, int32_t stream_id, + nghttp2_nv *nva, size_t nvlen, int fd) { + int rv; + nghttp2_data_provider data_prd; + data_prd.source.fd = fd; + data_prd.read_callback = file_read_callback; + + rv = nghttp2_submit_response(session, stream_id, nva, nvlen, &data_prd); + if (rv != 0) { + warnx("Fatal error: %s", nghttp2_strerror(rv)); + return -1; + } + return 0; + } + +nghttp2 uses the :type:`nghttp2_data_provider` structure to send the +entity body to the remote peer. The ``source`` member of this +structure is a union, which can be either a void pointer or an int +(which is intended to be used as file descriptor). In this example +server, we use it as a file descriptor. We also set the +``file_read_callback()`` callback function to read the contents of the +file:: + + static ssize_t file_read_callback(nghttp2_session *session _U_, + int32_t stream_id _U_, uint8_t *buf, + size_t length, uint32_t *data_flags, + nghttp2_data_source *source, + void *user_data _U_) { + int fd = source->fd; + ssize_t r; + while ((r = read(fd, buf, length)) == -1 && errno == EINTR) + ; + if (r == -1) { + return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE; + } + if (r == 0) { + *data_flags |= NGHTTP2_DATA_FLAG_EOF; + } + return r; + } + +If an error occurs while reading the file, we return +:macro:`NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE`. This tells the +library to send RST_STREAM to the stream. When all data has been +read, the :macro:`NGHTTP2_DATA_FLAG_EOF` flag is set to signal nghttp2 +that we have finished reading the file. + +The `nghttp2_submit_response()` function is used to send the response to the +remote peer. + +The ``on_stream_close_callback()`` function is invoked when the stream +is about to close:: + + static int on_stream_close_callback(nghttp2_session *session, int32_t stream_id, + uint32_t error_code _U_, void *user_data) { + http2_session_data *session_data = (http2_session_data *)user_data; + http2_stream_data *stream_data; + + stream_data = nghttp2_session_get_stream_user_data(session, stream_id); + if (!stream_data) { + return 0; + } + remove_stream(session_data, stream_data); + delete_http2_stream_data(stream_data); + return 0; + } + +Lastly, we destroy the ``http2_stream_data`` object in this function, +since the stream is about to close and we no longer need the object. |