summaryrefslogtreecommitdiffstats
path: root/examples/tests/README.rst
diff options
context:
space:
mode:
Diffstat (limited to 'examples/tests/README.rst')
-rw-r--r--examples/tests/README.rst60
1 files changed, 60 insertions, 0 deletions
diff --git a/examples/tests/README.rst b/examples/tests/README.rst
new file mode 100644
index 0000000..f583ecf
--- /dev/null
+++ b/examples/tests/README.rst
@@ -0,0 +1,60 @@
+Examples Tests
+==============
+
+This is a ``pytest`` suite intended to verify interoperability between
+the different example clients and servers built.
+
+You run it by executing ``pytest`` on top level project dir or in
+the examples/tests directory.
+
+.. code-block:: text
+
+ examples/test> pytest
+ ngtcp2-examples: [0.9.0-DEV, crypto_libs=['openssl', 'wolfssl']]
+ ...
+
+Requirements
+------------
+
+You need a Python3 (3.8 is probably sufficient), ``pytest`` and the
+Python ``cryptography`` module installed.
+
+Usage
+-----
+
+If you run ``pytest`` without arguments, it will print the test suite
+and a ``.`` for every test case passed. Add ``-v`` and all test cases
+will be listed in the full name. Adding several ``v`` will increase the
+logging level on failure output.
+
+The name of test cases include the crypto libs of the server and client
+used. For example:
+
+.. code-block:: text
+
+ test_01_handshake.py::TestHandshake::test_01_01_get[openssl-openssl] PASSED [ 16%]
+ test_01_handshake.py::TestHandshake::test_01_01_get[openssl-wolfssl] PASSED
+
+Here, ``test_01_01`` is run first with the OpenSSL server and client and then
+with the OpenSSL server and wolfSSL client. By default, the test suite runs
+all combinations of servers and clients that have been configured in the project.
+
+To track down problems, you can restrict the test cases that are run by
+matching patterns:
+
+.. code-block:: text
+
+ # only tests with wolfSSL example server
+ > pytest -v -k 'wolfssl-'
+ # only tests with wolfSSL example client
+ > pytest -v -k 'test and -wolfssl'
+ # tests with a specific combination
+ > pytest -v -k 'openssl-wolfssl'
+
+
+Analysing
+---------
+
+To make analysis of a broken test case easier, you best run only that
+test case. Use ``pytest -vv`` (or more) to get more verbose logging.
+Inspect server and client log files in ``examples/tests/gen``.