Merging upstream version 4.4.0.

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

1 files changed, 298 insertions, 0 deletions

diff --git a/doc/wsdg_src/wsdg_tests.adoc b/doc/wsdg_src/wsdg_tests.adoc
new file mode 100644
index 00000000..a9435ca7
--- /dev/null
+++ b/doc/wsdg_src/wsdg_tests.adoc
@@ -0,0 +1,298 @@
+// WSDG Chapter Tests
+
+[#ChapterTests]
+== Wireshark Tests
+
+The Wireshark sources include a collection of Python scripts that test
+the features of Wireshark, TShark, Dumpcap, and other programs that
+accompany Wireshark. These are located in the `test` directory of the
+Wireshark source tree.
+
+The command line options of Wireshark and its companion command line
+tools are numerous. These tests help to ensure that we don't introduce
+bugs as Wireshark grows and evolves.
+
+[#TestsQuickStart]
+=== Quick Start
+
+The recommended steps to prepare for and run tests with a UN*X toolchain:
+
+* Install two Python packages, pytest: `pip install pytest pytest-xdist`
+* Build programs (“wireshark”, “tshark”, etc.): `ninja`
+* Build additional programs for the “unittests” suite: `ninja test-programs`
+* Run tests in the build directory: `pytest`
+
+Replace `ninja` by `make` as needed.
+
+If building with <<#ChWindowsBuild,Microsoft Visual Studio>> the analogous steps are:
+
+* Install pytest Python packages: `python -m pip install pytest pytest-xdist`
+* Build programs: `msbuild /m /p:Configuration=RelWithDebInfo Wireshark.sln`
+* Build test-programs: `msbuild /m /p:Configuration=RelWithDebInfo test-programs.vcxproj`
+* Run tests: `python -m pytest`
+
+TIP: Depending on your PATH, you may need to run the pytest module as a
+script from your Python interpreter, e.g, `python -m pytest` or
+`python3 -m pytest` instead of `pytest`.
+
+The test suite will attempt to test as much as possible and skip tests
+when its dependencies are not satisfied. For example, packet capture
+tests require a Loopback interface and capture privileges. To avoid
+capture tests, pass the `--disable-capture` option.
+
+List available tests with `pytest --collectonly`. Enable verbose output
+with `pytest --verbose`. For more details, see <<ChTestsRun>>.
+
+You can also run the "ninja test" target instead of invoking pytest
+directly. This will automatically build the test programs dependency,
+so it may be preferred for that reason.
+
+[#ChTestsStructure]
+=== Test suite structure
+
+The following sections describes how the test suite is organized.
+
+[#TestCoverage]
+==== Test Coverage And Availability
+
+The testing framework can run programs and check their stdout, stderr,
+and exit codes. It cannot interact with the Wireshark UI. Tests cover
+capture, command line options, decryption, file format support and
+conversion, Lua scripting, and other functionality.
+
+Available tests depend on the libraries with which Wireshark was built.
+For example, some decryption tests depend on a minimum version of
+Libgcrypt and Lua tests depend on Lua.
+
+Capture tests depend on the permissions of the user running the test
+script. We assume that the test user has capture permissions on Windows
+and macOS and capture tests are enabled by default on those platforms.
+
+TIP: Build the "test-capture" target on Linux (using sudo) to set dumpcap
+permissions and enable capture tests.
+
+If a feature is unavailable, the test will be skipped. For example, if
+an old version of Libgcrypt is in use, then some decryption tests will
+be skipped while other tests can still run to completion.
+
+[#TestsLayout]
+==== Suites, Cases, and Tests
+
+The test suite uses pytest as a test runner. Tests are organized according to
+suites, cases, and individual tests. Suites correspond to Python modules
+that match the pattern “suite_*.py”. Cases correspond to one or more
+classes in each module, and case class methods matching the pattern
+”test_*” correspond to individual tests. For example, the invalid
+capture filter test in the TShark capture command line options test case
+in the command line options suite has the ID
+“suite_clopts.py::TestTsharkCaptureClopts::test_tshark_invalid_capfilter”.
+
+[#TestsPytest]
+==== pytest fixtures
+
+A test has typically additional dependencies, like the path to an
+executable, the path to a capture file, a configuration directory, the
+availability of an optional library, and so on.
+
+https://pytest.org/[pytest] is a test framework which has full
+parallelization support (test-level instead of just suite-level),
+provides nice test reports, and allows
+https://docs.pytest.org/en/latest/fixture.html[modular fixtures].
+
+A fixture is a function decorated with `@pytest.fixture` and can
+either call `pytest.skip("reason")` to skip tests that depend on the
+fixture, or return/yield a value.
+Test functions (and other fixture functions) can receive the fixture
+value by using the name of the fixture function as function parameters.
+Common fixtures are available in `fixtures_ws.py` and includes
+`cmd_tshark` for the path to the `tshark` executable and `capture_file`
+for a factory function that produces the path to a capture file.
+
+[#ChTestsRun]
+=== Listing And Running Tests
+
+Tests are run with https://pytest.org/[pytest]. Pytest features versus the
+"unittest" standard library module include finer
+test selection, full parallelism, nicer test execution summaries, better output
+in case of failures (containing the contents of variables) and the ability to
+open the PDB debugger on failing tests.
+
+To get started, install pytest 3.0 or newer and
+https://pypi.org/project/pytest-xdist/[pytest-xdist]:
+
+[source,sh]
+----
+# Install required packages on Ubuntu 18.04 or Debian jessie-backports
+$ sudo apt install python3-pytest python3-pytest-xdist
+
+# Install required packages on other systems
+$ pip install pytest pytest-xdist
+----
+
+Run `pytest` in the Wireshark build directory, Wireshark binaries are assumed to
+be present in the `run` subdirectory (or `run\RelWithDebInfo` on Windows).
+
+[source,sh]
+----
+# Run all tests
+$ cd /path/to/wireshark/build
+$ pytest
+
+# Run all except capture tests
+$ pytest --disable-capture
+
+# Run all tests with "decryption" in its name
+$ pytest -k decryption
+
+# Run all tests with an explicit path to the Wireshark executables
+$ pytest --program-path /path/to/wireshark/build/run
+----
+
+To list tests without actually executing them, use the `--collect-only` option:
+
+[source,sh]
+----
+# List all tests
+$ pytest --collect-only
+
+# List only tests containing both "dfilter" and "tvb"
+$ pytest --collect-only -k "dfilter and tvb"
+----
+
+The test suite will fail tests when programs are missing. When only a
+subset of programs are built or when some programs are disabled, then
+the test suite can be instructed to skip instead of fail tests:
+
+[source,sh]
+----
+# Run tests when libpcap support is disabled (-DENABLE_PCAP=OFF)
+$ pytest --skip-missing-programs dumpcap,rawshark
+
+# Run tests and ignore all tests with missing program dependencies
+$ pytest --skip-missing-programs all
+----
+
+To open a Python debugger (PDB) on failing tests, use the `--pdb` option and
+disable parallelism with the `-n0` option:
+
+[source,sh]
+----
+# Run decryption tests sequentially and open a debugger on failing tests
+$ pytest -n0 --pdb -k decryption
+----
+
+[#ChTestsDevelop]
+=== Adding Or Modifying Tests
+
+Tests must be in a Python module whose name matches “suite_*.py”. The
+module must contain one or more subclasses with a name starting with
+"Test" something, for example "class TestDissectionHttp2:". Each test case
+method whose name starts with “test_” constitutes an individual test.
+
+Success or failure conditions are signalled using regular assertions
+with the "assert" Python keyword.
+
+Test dependencies (such as programs, directories, or the environment
+variables) are injected through method parameters. Commonly used
+fixtures include `cmd_tshark` and `capture_file`.
+
+Processes (tshark, capinfos, etc.) are run using the "subprocess" Python module,
+or the Wireshark `subprocesstest` module with some convenience functions.
+Possible functions include `subprocesstest.run()`, `subprocesstest.check_run()`
+or creating `subprocess.Popen` object if the utility functions are not sufficient for some reason.
+Usually this is only required if two-way communication is performed with
+the child process. `subprocesstest.check_run()` is exactly the same as
+calling `subprocesstest.run()` with `check=True` as an argument, only
+a bit more expressive.
+
+Check the documentation for the Python subprocess module for a full description
+of the arguments available to the `subprocesstest.run()` convenience wrapper
+and the `subprocess.Popen` object.
+
+All of the current tests run one or more of Wireshark's suite of
+executables and either check their return code or their output. A
+simple example is “suite_clopts.py::TestBasicClopts::test_existing_file”,
+which reads a capture file using TShark and checks its exit code.
+
+[source,python]
+----
+import subprocesstest
+import pytest
+
+class TestBasicClopts:
+    def test_existing_file(self, cmd_tshark, capture_file, test_env):
+        subprocess.check_run((cmd_tshark, '-r', capture_file('dhcp.pcap')), env=test_env)
+----
+
+Output can be checked using `assert subprocesstest.grep_output()`,
+`assert subprocesstest.count_output()` or any other `assert` statement.
+`subprocesstest.check_run()` also asserts that the child process returns
+the value 0 as exit code.
+
+[source,python]
+----
+import subprocesstest
+import pytest
+
+class TestDecrypt80211:
+    def test_80211_wpa_psk(self, cmd_tshark, capture_file, test_env):
+        tshark_proc = subprocesstest.run((cmd_tshark,
+                '-o', 'wlan.enable_decryption: TRUE',
+                '-Tfields',
+                '-e', 'http.request.uri',
+                '-r', capture_file('wpa-Induction.pcap.gz'),
+                '-Y', 'http',
+            ), capture_output=True, env=test_env)
+        assert 'favicon.ico' in tshark_proc.stdout
+----
+
+Tests can be run in parallel. This means that any files you create must
+be unique for each test. Filenames based on the current test name are
+generated using fixtures such as "capture_file" and "result_file". By default
+pytest generates paths in the system's temporary directory and the last three
+pytest runs are kept. Temporary files from older runs are automatically deleted.
+
+[#ChTestsExternal]
+=== External Tests
+
+You can create your own Python test files outside of the Wireshark source tree.
+To include your tests when running the Wireshark test suite, simply add the
+directory containing your test files to the `pytest` command line. Note that
+filenames must match the same conventions as discussed above.
+
+In order for your tests to have access to the Wireshark test fixtures, you will
+need this line in each test file:
+
+[source,python]
+----
+from fixtures_ws import *
+----
+
+[#ChTestsExtFixtures]
+==== Custom Fixtures
+
+You may wish to define your own test fixtures -- for example, a fixture similar
+to `capture_file` but which gives the path to a file in your external test
+directory. Here is an example Python file containing such a fixture. It presumes
+a subdirectory named `extra_captures` which exists in the same directory, and
+which contains your extra capture files.
+
+[source,python]
+----
+# my_fixtures.py
+# To use in your own tests, import like so:
+#   from my_fixtures import *
+
+from pathlib import Path
+import pytest
+
+@pytest.fixture(scope='session')
+def extra_file():
+    def resolver(filename):
+        return Path(__file__).parent.joinpath("extra_captures", filename)
+    return resolver
+----
+
+NOTE: If you give your fixture the same name as an existing Wireshark fixture,
+any tests using your fixture library will lose access to the Wireshark fixture
+of the same name. This can lead to confusing behavior and is not recommended.