summaryrefslogtreecommitdiffstats
path: root/doc/testing.txt
diff options
context:
space:
mode:
Diffstat (limited to 'doc/testing.txt')
-rw-r--r--doc/testing.txt112
1 files changed, 112 insertions, 0 deletions
diff --git a/doc/testing.txt b/doc/testing.txt
new file mode 100644
index 0000000..3ec7c97
--- /dev/null
+++ b/doc/testing.txt
@@ -0,0 +1,112 @@
+Automated testing
+=================
+
+Introduction
+------------
+The bash-completion package contains an automated test suite. Running the
+tests should help verifying that bash-completion works as expected. The tests
+are also very helpful in uncovering software regressions at an early stage.
+
+The test suite is written in Python, using https://pytest.org/[pytest]
+and https://pexpect.readthedocs.io/[pexpect].
+
+
+Coding Style Guide
+------------------
+
+For the Python part, all of it is formatted using
+https://github.com/ambv/black[Black], and we also run
+https://flake8.pycqa.org/[Flake8] on it.
+
+
+Installing dependencies
+-----------------------
+
+Installing dependencies should be easy using your local package manager or
+`pip`. Python 3.4 or newer is required, and the rest of the Python package
+dependencies are specified in the `test/requirements.txt` file. If using `pip`,
+this file can be fed directly to it, e.g. like:
+------------------------------------
+pip install -r test/requirements.txt
+------------------------------------
+
+
+Debian/Ubuntu
+~~~~~~~~~~~~~
+
+On Debian/Ubuntu you can use `apt-get`:
+-------------
+sudo apt-get install python3-pytest python3-pexpect
+-------------
+This should also install the necessary dependencies. Only Debian testing
+(buster) and Ubuntu 18.10 (cosmic) and later have an appropriate version
+of pytest in the repositories.
+
+Fedora/RHEL/CentOS
+~~~~~~~~~~~~~~~~~~
+
+On Fedora and RHEL/CentOS (with EPEL) you can try `yum` or `dnf`:
+-------------
+sudo yum install python3-pytest python3-pexpect
+-------------
+This should also install the necessary dependencies. At time of writing, only
+Fedora 29 comes with recent enough pytest.
+
+
+
+Structure
+---------
+
+Tests are in the `t/` subdirectory, with `t/test_\*.py` being completion
+tests, and `t/unit/test_unit_\*.py` unit tests.
+
+
+Running the tests
+-----------------
+
+Tests are run by calling `pytest` on the desired test directories or
+individual files, for example in the project root directory:
+-----------------------
+pytest test/t
+-----------------------
+
+See `test/docker/docker-script.sh` for how and what we run and test in CI.
+
+
+Specifying bash binary
+~~~~~~~~~~~~~~~~~~~~~~
+
+The test suite standard uses `bash` as found in PATH. Export the
+`bashcomp_bash` environment variable with a path to another bash executable if
+you want to test against something else.
+
+
+Maintenance
+-----------
+
+
+Adding a completion test
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+You can run `cd test && ./generate cmd` to add a test for the `cmd`
+command. Additional arguments will be passed to the first generated test case.
+This will add the `test/t/test_cmd.py` file with a very basic test, and add it
+to `test/t/Makefile.am`. Add additional tests to the generated file.
+
+Rationale
+---------
+
+
+Naming conventions
+~~~~~~~~~~~~~~~~~~
+
+Test suite or testsuite
+^^^^^^^^^^^^^^^^^^^^^^^
+The primary Wikipedia page is called
+https://en.wikipedia.org/wiki/Test_suite[test suite] and not testsuite, so
+that's what this document sticks to.
+
+script/generate
+^^^^^^^^^^^^^^^
+The name and location of this code generation script come from Ruby on Rails'
+https://en.wikibooks.org/wiki/Ruby_on_Rails/Tools/Generators[script/generate].