diff options
Diffstat (limited to 'examples/tests/README.rst')
-rw-r--r-- | examples/tests/README.rst | 60 |
1 files changed, 60 insertions, 0 deletions
diff --git a/examples/tests/README.rst b/examples/tests/README.rst new file mode 100644 index 0000000..f583ecf --- /dev/null +++ b/examples/tests/README.rst @@ -0,0 +1,60 @@ +Examples Tests +============== + +This is a ``pytest`` suite intended to verify interoperability between +the different example clients and servers built. + +You run it by executing ``pytest`` on top level project dir or in +the examples/tests directory. + +.. code-block:: text + + examples/test> pytest + ngtcp2-examples: [0.9.0-DEV, crypto_libs=['openssl', 'wolfssl']] + ... + +Requirements +------------ + +You need a Python3 (3.8 is probably sufficient), ``pytest`` and the +Python ``cryptography`` module installed. + +Usage +----- + +If you run ``pytest`` without arguments, it will print the test suite +and a ``.`` for every test case passed. Add ``-v`` and all test cases +will be listed in the full name. Adding several ``v`` will increase the +logging level on failure output. + +The name of test cases include the crypto libs of the server and client +used. For example: + +.. code-block:: text + + test_01_handshake.py::TestHandshake::test_01_01_get[openssl-openssl] PASSED [ 16%] + test_01_handshake.py::TestHandshake::test_01_01_get[openssl-wolfssl] PASSED + +Here, ``test_01_01`` is run first with the OpenSSL server and client and then +with the OpenSSL server and wolfSSL client. By default, the test suite runs +all combinations of servers and clients that have been configured in the project. + +To track down problems, you can restrict the test cases that are run by +matching patterns: + +.. code-block:: text + + # only tests with wolfSSL example server + > pytest -v -k 'wolfssl-' + # only tests with wolfSSL example client + > pytest -v -k 'test and -wolfssl' + # tests with a specific combination + > pytest -v -k 'openssl-wolfssl' + + +Analysing +--------- + +To make analysis of a broken test case easier, you best run only that +test case. Use ``pytest -vv`` (or more) to get more verbose logging. +Inspect server and client log files in ``examples/tests/gen``. |