diff options
Diffstat (limited to 'doc/testing.txt')
| -rw-r--r-- | doc/testing.txt | 112 |
1 files changed, 112 insertions, 0 deletions
diff --git a/doc/testing.txt b/doc/testing.txt new file mode 100644 index 0000000..3ec7c97 --- /dev/null +++ b/doc/testing.txt @@ -0,0 +1,112 @@ +Automated testing +================= + +Introduction +------------ +The bash-completion package contains an automated test suite. Running the +tests should help verifying that bash-completion works as expected. The tests +are also very helpful in uncovering software regressions at an early stage. + +The test suite is written in Python, using https://pytest.org/[pytest] +and https://pexpect.readthedocs.io/[pexpect]. + + +Coding Style Guide +------------------ + +For the Python part, all of it is formatted using +https://github.com/ambv/black[Black], and we also run +https://flake8.pycqa.org/[Flake8] on it. + + +Installing dependencies +----------------------- + +Installing dependencies should be easy using your local package manager or +`pip`. Python 3.4 or newer is required, and the rest of the Python package +dependencies are specified in the `test/requirements.txt` file. If using `pip`, +this file can be fed directly to it, e.g. like: +------------------------------------ +pip install -r test/requirements.txt +------------------------------------ + + +Debian/Ubuntu +~~~~~~~~~~~~~ + +On Debian/Ubuntu you can use `apt-get`: +------------- +sudo apt-get install python3-pytest python3-pexpect +------------- +This should also install the necessary dependencies. Only Debian testing +(buster) and Ubuntu 18.10 (cosmic) and later have an appropriate version +of pytest in the repositories. + +Fedora/RHEL/CentOS +~~~~~~~~~~~~~~~~~~ + +On Fedora and RHEL/CentOS (with EPEL) you can try `yum` or `dnf`: +------------- +sudo yum install python3-pytest python3-pexpect +------------- +This should also install the necessary dependencies. At time of writing, only +Fedora 29 comes with recent enough pytest. + + + +Structure +--------- + +Tests are in the `t/` subdirectory, with `t/test_\*.py` being completion +tests, and `t/unit/test_unit_\*.py` unit tests. + + +Running the tests +----------------- + +Tests are run by calling `pytest` on the desired test directories or +individual files, for example in the project root directory: +----------------------- +pytest test/t +----------------------- + +See `test/docker/docker-script.sh` for how and what we run and test in CI. + + +Specifying bash binary +~~~~~~~~~~~~~~~~~~~~~~ + +The test suite standard uses `bash` as found in PATH. Export the +`bashcomp_bash` environment variable with a path to another bash executable if +you want to test against something else. + + +Maintenance +----------- + + +Adding a completion test +~~~~~~~~~~~~~~~~~~~~~~~~ + +You can run `cd test && ./generate cmd` to add a test for the `cmd` +command. Additional arguments will be passed to the first generated test case. +This will add the `test/t/test_cmd.py` file with a very basic test, and add it +to `test/t/Makefile.am`. Add additional tests to the generated file. + +Rationale +--------- + + +Naming conventions +~~~~~~~~~~~~~~~~~~ + +Test suite or testsuite +^^^^^^^^^^^^^^^^^^^^^^^ +The primary Wikipedia page is called +https://en.wikipedia.org/wiki/Test_suite[test suite] and not testsuite, so +that's what this document sticks to. + +script/generate +^^^^^^^^^^^^^^^ +The name and location of this code generation script come from Ruby on Rails' +https://en.wikibooks.org/wiki/Ruby_on_Rails/Tools/Generators[script/generate]. |