summaryrefslogtreecommitdiffstats
path: root/tools/testing/selftests/tc-testing/README
diff options
context:
space:
mode:
Diffstat (limited to '')
-rw-r--r--tools/testing/selftests/tc-testing/README257
1 files changed, 257 insertions, 0 deletions
diff --git a/tools/testing/selftests/tc-testing/README b/tools/testing/selftests/tc-testing/README
new file mode 100644
index 000000000..b0954c873
--- /dev/null
+++ b/tools/testing/selftests/tc-testing/README
@@ -0,0 +1,257 @@
+tdc - Linux Traffic Control (tc) unit testing suite
+
+Author: Lucas Bates - lucasb@mojatatu.com
+
+tdc is a Python script to load tc unit tests from a separate JSON file and
+execute them inside a network namespace dedicated to the task.
+
+
+REQUIREMENTS
+------------
+
+* Minimum Python version of 3.4. Earlier 3.X versions may work but are not
+ guaranteed.
+
+* The kernel must have network namespace support if using nsPlugin
+
+* The kernel must have veth support available, as a veth pair is created
+ prior to running the tests when using nsPlugin.
+
+* The kernel must have the appropriate infrastructure enabled to run all tdc
+ unit tests. See the config file in this directory for minimum required
+ features. As new tests will be added, config options list will be updated.
+
+* All tc-related features being tested must be built in or available as
+ modules. To check what is required in current setup run:
+ ./tdc.py -c
+
+ Note:
+ In the current release, tdc run will abort due to a failure in setup or
+ teardown commands - which includes not being able to run a test simply
+ because the kernel did not support a specific feature. (This will be
+ handled in a future version - the current workaround is to run the tests
+ on specific test categories that your kernel supports)
+
+
+BEFORE YOU RUN
+--------------
+
+The path to the tc executable that will be most commonly tested can be defined
+in the tdc_config.py file. Find the 'TC' entry in the NAMES dictionary and
+define the path.
+
+If you need to test a different tc executable on the fly, you can do so by
+using the -p option when running tdc:
+ ./tdc.py -p /path/to/tc
+
+
+RUNNING TDC
+-----------
+
+To use tdc, root privileges are required. This is because the
+commands being tested must be run as root. The code that enforces
+execution by root uid has been moved into a plugin (see PLUGIN
+ARCHITECTURE, below).
+
+Tests that use a network device should have nsPlugin.py listed as a
+requirement for that test. nsPlugin executes all commands within a
+network namespace and creates a veth pair which may be used in those test
+cases. To disable execution within the namespace, pass the -N option
+to tdc when starting a test run; the veth pair will still be created
+by the plugin.
+
+Running tdc without any arguments will run all tests. Refer to the section
+on command line arguments for more information, or run:
+ ./tdc.py -h
+
+tdc will list the test names as they are being run, and print a summary in
+TAP (Test Anything Protocol) format when they are done. If tests fail,
+output captured from the failing test will be printed immediately following
+the failed test in the TAP output.
+
+
+OVERVIEW OF TDC EXECUTION
+-------------------------
+
+One run of tests is considered a "test suite" (this will be refined in the
+future). A test suite has one or more test cases in it.
+
+A test case has four stages:
+
+ - setup
+ - execute
+ - verify
+ - teardown
+
+The setup and teardown stages can run zero or more commands. The setup
+stage does some setup if the test needs it. The teardown stage undoes
+the setup and returns the system to a "neutral" state so any other test
+can be run next. These two stages require any commands run to return
+success, but do not otherwise verify the results.
+
+The execute and verify stages each run one command. The execute stage
+tests the return code against one or more acceptable values. The
+verify stage checks the return code for success, and also compares
+the stdout with a regular expression.
+
+Each of the commands in any stage will run in a shell instance.
+
+
+USER-DEFINED CONSTANTS
+----------------------
+
+The tdc_config.py file contains multiple values that can be altered to suit
+your needs. Any value in the NAMES dictionary can be altered without affecting
+the tests to be run. These values are used in the tc commands that will be
+executed as part of the test. More will be added as test cases require.
+
+Example:
+ $TC qdisc add dev $DEV1 ingress
+
+The NAMES values are used to substitute into the commands in the test cases.
+
+
+COMMAND LINE ARGUMENTS
+----------------------
+
+Run tdc.py -h to see the full list of available arguments.
+
+usage: tdc.py [-h] [-p PATH] [-D DIR [DIR ...]] [-f FILE [FILE ...]]
+ [-c [CATG [CATG ...]]] [-e ID [ID ...]] [-l] [-s] [-i] [-v] [-N]
+ [-d DEVICE] [-P] [-n] [-V]
+
+Linux TC unit tests
+
+optional arguments:
+ -h, --help show this help message and exit
+ -p PATH, --path PATH The full path to the tc executable to use
+ -v, --verbose Show the commands that are being run
+ -N, --notap Suppress tap results for command under test
+ -d DEVICE, --device DEVICE
+ Execute test cases that use a physical device, where
+ DEVICE is its name. (If not defined, tests that require
+ a physical device will be skipped)
+ -P, --pause Pause execution just before post-suite stage
+
+selection:
+ select which test cases: files plus directories; filtered by categories
+ plus testids
+
+ -D DIR [DIR ...], --directory DIR [DIR ...]
+ Collect tests from the specified directory(ies)
+ (default [tc-tests])
+ -f FILE [FILE ...], --file FILE [FILE ...]
+ Run tests from the specified file(s)
+ -c [CATG [CATG ...]], --category [CATG [CATG ...]]
+ Run tests only from the specified category/ies, or if
+ no category/ies is/are specified, list known
+ categories.
+ -e ID [ID ...], --execute ID [ID ...]
+ Execute the specified test cases with specified IDs
+
+action:
+ select action to perform on selected test cases
+
+ -l, --list List all test cases, or those only within the
+ specified category
+ -s, --show Display the selected test cases
+ -i, --id Generate ID numbers for new test cases
+
+netns:
+ options for nsPlugin (run commands in net namespace)
+
+ -N, --no-namespace
+ Do not run commands in a network namespace.
+
+valgrind:
+ options for valgrindPlugin (run command under test under Valgrind)
+
+ -V, --valgrind Run commands under valgrind
+
+
+PLUGIN ARCHITECTURE
+-------------------
+
+There is now a plugin architecture, and some of the functionality that
+was in the tdc.py script has been moved into the plugins.
+
+The plugins are in the directory plugin-lib. The are executed from
+directory plugins. Put symbolic links from plugins to plugin-lib,
+and name them according to the order you want them to run. This is not
+necessary if a test case being run requires a specific plugin to work.
+
+Example:
+
+bjb@bee:~/work/tc-testing$ ls -l plugins
+total 4
+lrwxrwxrwx 1 bjb bjb 27 Oct 4 16:12 10-rootPlugin.py -> ../plugin-lib/rootPlugin.py
+lrwxrwxrwx 1 bjb bjb 25 Oct 12 17:55 20-nsPlugin.py -> ../plugin-lib/nsPlugin.py
+-rwxr-xr-x 1 bjb bjb 0 Sep 29 15:56 __init__.py
+
+The plugins are a subclass of TdcPlugin, defined in TdcPlugin.py and
+must be called "SubPlugin" so tdc can find them. They are
+distinguished from each other in the python program by their module
+name.
+
+This base class supplies "hooks" to run extra functions. These hooks are as follows:
+
+pre- and post-suite
+pre- and post-case
+pre- and post-execute stage
+adjust-command (runs in all stages and receives the stage name)
+
+The pre-suite hook receives the number of tests and an array of test ids.
+This allows you to dump out the list of skipped tests in the event of a
+failure during setup or teardown stage.
+
+The pre-case hook receives the ordinal number and test id of the current test.
+
+The adjust-command hook receives the stage id (see list below) and the
+full command to be executed. This allows for last-minute adjustment
+of the command.
+
+The stages are identified by the following strings:
+
+ - pre (pre-suite)
+ - setup
+ - command
+ - verify
+ - teardown
+ - post (post-suite)
+
+
+To write a plugin, you need to inherit from TdcPlugin in
+TdcPlugin.py. To use the plugin, you have to put the
+implementation file in plugin-lib, and add a symbolic link to it from
+plugins. It will be detected at run time and invoked at the
+appropriate times. There are a few examples in the plugin-lib
+directory:
+
+ - rootPlugin.py:
+ implements the enforcement of running as root
+ - nsPlugin.py:
+ sets up a network namespace and runs all commands in that namespace,
+ while also setting up dummy devices to be used in testing.
+ - valgrindPlugin.py
+ runs each command in the execute stage under valgrind,
+ and checks for leaks.
+ This plugin will output an extra test for each test in the test file,
+ one is the existing output as to whether the test passed or failed,
+ and the other is a test whether the command leaked memory or not.
+ (This one is a preliminary version, it may not work quite right yet,
+ but the overall template is there and it should only need tweaks.)
+ - buildebpfPlugin.py:
+ builds all programs in $EBPFDIR.
+
+
+ACKNOWLEDGEMENTS
+----------------
+
+Thanks to:
+
+Jamal Hadi Salim, for providing valuable test cases
+Keara Leibovitz, who wrote the CLI test driver that I used as a base for the
+ first version of the tc testing suite. This work was presented at
+ Netdev 1.2 Tokyo in October 2016.
+Samir Hussain, for providing help while I dove into Python for the first time
+ and being a second eye for this code.