summaryrefslogtreecommitdiffstats
path: root/docbook/wsdg_src/wsdg_tests.adoc
diff options
context:
space:
mode:
authorDaniel Baumann <daniel.baumann@progress-linux.org>2024-09-19 04:14:26 +0000
committerDaniel Baumann <daniel.baumann@progress-linux.org>2024-09-19 04:14:26 +0000
commitc4e8a3222648fcf22ca207f1815ebbf7cd144eeb (patch)
tree93d5c6aa93d9987680dd1adad5685e2ad698f223 /docbook/wsdg_src/wsdg_tests.adoc
parentAdding upstream version 4.2.6. (diff)
downloadwireshark-upstream.tar.xz
wireshark-upstream.zip
Adding upstream version 4.4.0.upstream/4.4.0upstream
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'docbook/wsdg_src/wsdg_tests.adoc')
-rw-r--r--docbook/wsdg_src/wsdg_tests.adoc298
1 files changed, 0 insertions, 298 deletions
diff --git a/docbook/wsdg_src/wsdg_tests.adoc b/docbook/wsdg_src/wsdg_tests.adoc
deleted file mode 100644
index a9435ca7..00000000
--- a/docbook/wsdg_src/wsdg_tests.adoc
+++ /dev/null
@@ -1,298 +0,0 @@
-// WSDG Chapter Tests
-
-[#ChapterTests]
-== Wireshark Tests
-
-The Wireshark sources include a collection of Python scripts that test
-the features of Wireshark, TShark, Dumpcap, and other programs that
-accompany Wireshark. These are located in the `test` directory of the
-Wireshark source tree.
-
-The command line options of Wireshark and its companion command line
-tools are numerous. These tests help to ensure that we don't introduce
-bugs as Wireshark grows and evolves.
-
-[#TestsQuickStart]
-=== Quick Start
-
-The recommended steps to prepare for and run tests with a UN*X toolchain:
-
-* Install two Python packages, pytest: `pip install pytest pytest-xdist`
-* Build programs (“wireshark”, “tshark”, etc.): `ninja`
-* Build additional programs for the “unittests” suite: `ninja test-programs`
-* Run tests in the build directory: `pytest`
-
-Replace `ninja` by `make` as needed.
-
-If building with <<#ChWindowsBuild,Microsoft Visual Studio>> the analogous steps are:
-
-* Install pytest Python packages: `python -m pip install pytest pytest-xdist`
-* Build programs: `msbuild /m /p:Configuration=RelWithDebInfo Wireshark.sln`
-* Build test-programs: `msbuild /m /p:Configuration=RelWithDebInfo test-programs.vcxproj`
-* Run tests: `python -m pytest`
-
-TIP: Depending on your PATH, you may need to run the pytest module as a
-script from your Python interpreter, e.g, `python -m pytest` or
-`python3 -m pytest` instead of `pytest`.
-
-The test suite will attempt to test as much as possible and skip tests
-when its dependencies are not satisfied. For example, packet capture
-tests require a Loopback interface and capture privileges. To avoid
-capture tests, pass the `--disable-capture` option.
-
-List available tests with `pytest --collectonly`. Enable verbose output
-with `pytest --verbose`. For more details, see <<ChTestsRun>>.
-
-You can also run the "ninja test" target instead of invoking pytest
-directly. This will automatically build the test programs dependency,
-so it may be preferred for that reason.
-
-[#ChTestsStructure]
-=== Test suite structure
-
-The following sections describes how the test suite is organized.
-
-[#TestCoverage]
-==== Test Coverage And Availability
-
-The testing framework can run programs and check their stdout, stderr,
-and exit codes. It cannot interact with the Wireshark UI. Tests cover
-capture, command line options, decryption, file format support and
-conversion, Lua scripting, and other functionality.
-
-Available tests depend on the libraries with which Wireshark was built.
-For example, some decryption tests depend on a minimum version of
-Libgcrypt and Lua tests depend on Lua.
-
-Capture tests depend on the permissions of the user running the test
-script. We assume that the test user has capture permissions on Windows
-and macOS and capture tests are enabled by default on those platforms.
-
-TIP: Build the "test-capture" target on Linux (using sudo) to set dumpcap
-permissions and enable capture tests.
-
-If a feature is unavailable, the test will be skipped. For example, if
-an old version of Libgcrypt is in use, then some decryption tests will
-be skipped while other tests can still run to completion.
-
-[#TestsLayout]
-==== Suites, Cases, and Tests
-
-The test suite uses pytest as a test runner. Tests are organized according to
-suites, cases, and individual tests. Suites correspond to Python modules
-that match the pattern “suite_*.py”. Cases correspond to one or more
-classes in each module, and case class methods matching the pattern
-”test_*” correspond to individual tests. For example, the invalid
-capture filter test in the TShark capture command line options test case
-in the command line options suite has the ID
-“suite_clopts.py::TestTsharkCaptureClopts::test_tshark_invalid_capfilter”.
-
-[#TestsPytest]
-==== pytest fixtures
-
-A test has typically additional dependencies, like the path to an
-executable, the path to a capture file, a configuration directory, the
-availability of an optional library, and so on.
-
-https://pytest.org/[pytest] is a test framework which has full
-parallelization support (test-level instead of just suite-level),
-provides nice test reports, and allows
-https://docs.pytest.org/en/latest/fixture.html[modular fixtures].
-
-A fixture is a function decorated with `@pytest.fixture` and can
-either call `pytest.skip("reason")` to skip tests that depend on the
-fixture, or return/yield a value.
-Test functions (and other fixture functions) can receive the fixture
-value by using the name of the fixture function as function parameters.
-Common fixtures are available in `fixtures_ws.py` and includes
-`cmd_tshark` for the path to the `tshark` executable and `capture_file`
-for a factory function that produces the path to a capture file.
-
-[#ChTestsRun]
-=== Listing And Running Tests
-
-Tests are run with https://pytest.org/[pytest]. Pytest features versus the
-"unittest" standard library module include finer
-test selection, full parallelism, nicer test execution summaries, better output
-in case of failures (containing the contents of variables) and the ability to
-open the PDB debugger on failing tests.
-
-To get started, install pytest 3.0 or newer and
-https://pypi.org/project/pytest-xdist/[pytest-xdist]:
-
-[source,sh]
-----
-# Install required packages on Ubuntu 18.04 or Debian jessie-backports
-$ sudo apt install python3-pytest python3-pytest-xdist
-
-# Install required packages on other systems
-$ pip install pytest pytest-xdist
-----
-
-Run `pytest` in the Wireshark build directory, Wireshark binaries are assumed to
-be present in the `run` subdirectory (or `run\RelWithDebInfo` on Windows).
-
-[source,sh]
-----
-# Run all tests
-$ cd /path/to/wireshark/build
-$ pytest
-
-# Run all except capture tests
-$ pytest --disable-capture
-
-# Run all tests with "decryption" in its name
-$ pytest -k decryption
-
-# Run all tests with an explicit path to the Wireshark executables
-$ pytest --program-path /path/to/wireshark/build/run
-----
-
-To list tests without actually executing them, use the `--collect-only` option:
-
-[source,sh]
-----
-# List all tests
-$ pytest --collect-only
-
-# List only tests containing both "dfilter" and "tvb"
-$ pytest --collect-only -k "dfilter and tvb"
-----
-
-The test suite will fail tests when programs are missing. When only a
-subset of programs are built or when some programs are disabled, then
-the test suite can be instructed to skip instead of fail tests:
-
-[source,sh]
-----
-# Run tests when libpcap support is disabled (-DENABLE_PCAP=OFF)
-$ pytest --skip-missing-programs dumpcap,rawshark
-
-# Run tests and ignore all tests with missing program dependencies
-$ pytest --skip-missing-programs all
-----
-
-To open a Python debugger (PDB) on failing tests, use the `--pdb` option and
-disable parallelism with the `-n0` option:
-
-[source,sh]
-----
-# Run decryption tests sequentially and open a debugger on failing tests
-$ pytest -n0 --pdb -k decryption
-----
-
-[#ChTestsDevelop]
-=== Adding Or Modifying Tests
-
-Tests must be in a Python module whose name matches “suite_*.py”. The
-module must contain one or more subclasses with a name starting with
-"Test" something, for example "class TestDissectionHttp2:". Each test case
-method whose name starts with “test_” constitutes an individual test.
-
-Success or failure conditions are signalled using regular assertions
-with the "assert" Python keyword.
-
-Test dependencies (such as programs, directories, or the environment
-variables) are injected through method parameters. Commonly used
-fixtures include `cmd_tshark` and `capture_file`.
-
-Processes (tshark, capinfos, etc.) are run using the "subprocess" Python module,
-or the Wireshark `subprocesstest` module with some convenience functions.
-Possible functions include `subprocesstest.run()`, `subprocesstest.check_run()`
-or creating `subprocess.Popen` object if the utility functions are not sufficient for some reason.
-Usually this is only required if two-way communication is performed with
-the child process. `subprocesstest.check_run()` is exactly the same as
-calling `subprocesstest.run()` with `check=True` as an argument, only
-a bit more expressive.
-
-Check the documentation for the Python subprocess module for a full description
-of the arguments available to the `subprocesstest.run()` convenience wrapper
-and the `subprocess.Popen` object.
-
-All of the current tests run one or more of Wireshark's suite of
-executables and either check their return code or their output. A
-simple example is “suite_clopts.py::TestBasicClopts::test_existing_file”,
-which reads a capture file using TShark and checks its exit code.
-
-[source,python]
-----
-import subprocesstest
-import pytest
-
-class TestBasicClopts:
- def test_existing_file(self, cmd_tshark, capture_file, test_env):
- subprocess.check_run((cmd_tshark, '-r', capture_file('dhcp.pcap')), env=test_env)
-----
-
-Output can be checked using `assert subprocesstest.grep_output()`,
-`assert subprocesstest.count_output()` or any other `assert` statement.
-`subprocesstest.check_run()` also asserts that the child process returns
-the value 0 as exit code.
-
-[source,python]
-----
-import subprocesstest
-import pytest
-
-class TestDecrypt80211:
- def test_80211_wpa_psk(self, cmd_tshark, capture_file, test_env):
- tshark_proc = subprocesstest.run((cmd_tshark,
- '-o', 'wlan.enable_decryption: TRUE',
- '-Tfields',
- '-e', 'http.request.uri',
- '-r', capture_file('wpa-Induction.pcap.gz'),
- '-Y', 'http',
- ), capture_output=True, env=test_env)
- assert 'favicon.ico' in tshark_proc.stdout
-----
-
-Tests can be run in parallel. This means that any files you create must
-be unique for each test. Filenames based on the current test name are
-generated using fixtures such as "capture_file" and "result_file". By default
-pytest generates paths in the system's temporary directory and the last three
-pytest runs are kept. Temporary files from older runs are automatically deleted.
-
-[#ChTestsExternal]
-=== External Tests
-
-You can create your own Python test files outside of the Wireshark source tree.
-To include your tests when running the Wireshark test suite, simply add the
-directory containing your test files to the `pytest` command line. Note that
-filenames must match the same conventions as discussed above.
-
-In order for your tests to have access to the Wireshark test fixtures, you will
-need this line in each test file:
-
-[source,python]
-----
-from fixtures_ws import *
-----
-
-[#ChTestsExtFixtures]
-==== Custom Fixtures
-
-You may wish to define your own test fixtures -- for example, a fixture similar
-to `capture_file` but which gives the path to a file in your external test
-directory. Here is an example Python file containing such a fixture. It presumes
-a subdirectory named `extra_captures` which exists in the same directory, and
-which contains your extra capture files.
-
-[source,python]
-----
-# my_fixtures.py
-# To use in your own tests, import like so:
-# from my_fixtures import *
-
-from pathlib import Path
-import pytest
-
-@pytest.fixture(scope='session')
-def extra_file():
- def resolver(filename):
- return Path(__file__).parent.joinpath("extra_captures", filename)
- return resolver
-----
-
-NOTE: If you give your fixture the same name as an existing Wireshark fixture,
-any tests using your fixture library will lose access to the Wireshark fixture
-of the same name. This can lead to confusing behavior and is not recommended.