diff options
author | Daniel Baumann <daniel.baumann@progress-linux.org> | 2024-04-29 04:20:41 +0000 |
---|---|---|
committer | Daniel Baumann <daniel.baumann@progress-linux.org> | 2024-04-29 04:20:41 +0000 |
commit | b49f1524e250764592ff132af8fb0d39182620f7 (patch) | |
tree | a2c4da0c1bfc3be79c9b80180d8958804e91a07d /docs/test_suite.rst | |
parent | Initial commit. (diff) | |
download | python-build-upstream.tar.xz python-build-upstream.zip |
Adding upstream version 0.9.0.upstream/0.9.0upstream
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'docs/test_suite.rst')
-rw-r--r-- | docs/test_suite.rst | 51 |
1 files changed, 51 insertions, 0 deletions
diff --git a/docs/test_suite.rst b/docs/test_suite.rst new file mode 100644 index 0000000..d486106 --- /dev/null +++ b/docs/test_suite.rst @@ -0,0 +1,51 @@ +********** +Test Suite +********** + +Due to its nature, ``build`` has a somewhat complex test suite, which we will +try to go through in this document. + +Firstly, there are two set of tests, unit tests and integration tests. In unit +tests, we test the actual code implementation. In integration tests, we test +``build`` on a few real world projects; this is mostly a sanity test. + +Integration tests take a long time to run, and are not very helpful tracking +down issues, so they are **disabled by default**. They can be enabled by passing +either ``--run-integration`` or ``--only-integration`` arguments to pytest, +where the latter will disable the unit tests and only run the integration ones. +Even though these tests are disabled by default, they will be run in CI, +where test suite run durations are not a big issue. + +To run the test suite we use ``tox``, which automates running the test suite on +different environments: + + +.. code-block:: console + + tox + + +You can find out more about how to run ``tox`` and its arguments in the +`tox documentation`_. + +We have a fairly large environment matrix. We run tests for all supported Python +versions and implementations, and with the module being invoked from path, +sdist install, or wheel install. Additionally, we have an environment for type +checking, and one to produce the documentation. There are some other extra +environments, like checking the code with the minimum version of each +dependency. + +Some examples commands for this project: + - Run type checking: ``tox -e type`` + - Only run unit tests against Python 3.9: ``tox -e py39`` + - Run both unit and integration tests: ``tox -- --run-integration`` + - Only run integration tests: ``tox -- --only-integration`` + - Only run integration tests with parallel tasks: ``tox -- -n auto --only-integration`` + - Only run unit tests against Python 3.9 with the module installed via wheel: ``tox -e py39-wheel`` + + +We have CI testing, where we the test suite across all supported operating +systems, and have test coverage reports. + + +.. _tox documentation: https://tox.readthedocs.io/ |