Adding upstream version 5.10.209.upstream/5.10.209upstream

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

1 files changed, 105 insertions, 0 deletions

diff --git a/tools/testing/selftests/tc-testing/creating-testcases/AddingTestCases.txt b/tools/testing/selftests/tc-testing/creating-testcases/AddingTestCases.txt
new file mode 100644
index 000000000..a28571aff
--- /dev/null
+++ b/tools/testing/selftests/tc-testing/creating-testcases/AddingTestCases.txt
@@ -0,0 +1,105 @@
+tdc - Adding test cases for tdc
+
+Author: Lucas Bates - lucasb@mojatatu.com
+
+ADDING TEST CASES
+-----------------
+
+User-defined tests should be added by defining a separate JSON file.  This
+will help prevent conflicts when updating the repository. Refer to
+template.json for the required JSON format for test cases.
+
+Include the 'id' field, but do not assign a value. Running tdc with the -i
+option will generate a unique ID for that test case.
+
+tdc will recursively search the 'tc-tests' subdirectory (or the
+directories named with the -D option) for .json files.  Any test case
+files you create in these directories will automatically be included.
+If you wish to store your custom test cases elsewhere, be sure to run
+tdc with the -f argument and the path to your file, or the -D argument
+and the path to your directory(ies).
+
+Be aware of required escape characters in the JSON data - particularly
+when defining the match pattern. Refer to the supplied json test files
+for examples when in doubt.  The match pattern is written in json, and
+will be used by python.  So the match pattern will be a python regular
+expression, but should be written using json syntax.
+
+
+TEST CASE STRUCTURE
+-------------------
+
+Each test case has required data:
+
+id:           A unique alphanumeric value to identify a particular test case
+name:         Descriptive name that explains the command under test
+skip:         A completely optional key, if the corresponding value is "yes"
+              then tdc will not execute the test case in question. However,
+              this test case will still appear in the results output but
+              marked as skipped. This key can be placed anywhere inside the
+              test case at the top level.
+category:     A list of single-word descriptions covering what the command
+              under test is testing. Example: filter, actions, u32, gact, etc.
+setup:        The list of commands required to ensure the command under test
+              succeeds. For example: if testing a filter, the command to create
+              the qdisc would appear here.
+	      This list can be empty.
+	      Each command can be a string to be executed, or a list consisting
+	      of a string which is a command to be executed, followed by 1 or
+	      more acceptable exit codes for this command.
+	      If only a string is given for the command, then an exit code of 0
+	      will be expected.
+cmdUnderTest: The tc command being tested itself.
+expExitCode:  The code returned by the command under test upon its termination.
+              tdc will compare this value against the actual returned value.
+verifyCmd:    The tc command to be run to verify successful execution.
+              For example: if the command under test creates a gact action,
+              verifyCmd should be "$TC actions show action gact"
+matchPattern: A regular expression to be applied against the output of the
+              verifyCmd to prove the command under test succeeded. This pattern
+              should be as specific as possible so that a false positive is not
+              matched.
+matchCount:   How many times the regex in matchPattern should match. A value
+              of 0 is acceptable.
+teardown:     The list of commands to clean up after the test is completed.
+              The environment should be returned to the same state as when
+              this test was started: qdiscs deleted, actions flushed, etc.
+	      This list can be empty.
+	      Each command can be a string to be executed, or a list consisting
+	      of a string which is a command to be executed, followed by 1 or
+	      more acceptable exit codes for this command.
+	      If only a string is given for the command, then an exit code of 0
+	      will be expected.
+
+
+SETUP/TEARDOWN ERRORS
+---------------------
+
+If an error is detected during the setup/teardown process, execution of the
+tests will immediately stop with an error message and the namespace in which
+the tests are run will be destroyed. This is to prevent inaccurate results
+in the test cases.  tdc will output a series of TAP results for the skipped
+tests.
+
+Repeated failures of the setup/teardown may indicate a problem with the test
+case, or possibly even a bug in one of the commands that are not being tested.
+
+It's possible to include acceptable exit codes with the setup/teardown command
+so that it doesn't halt the script for an error that doesn't matter. Turn the
+individual command into a list, with the command being first, followed by all
+acceptable exit codes for the command.
+
+Example:
+
+A pair of setup commands.  The first can have exit code 0, 1 or 255, the
+second must have exit code 0.
+
+        "setup": [
+            [
+                "$TC actions flush action gact",
+                0,
+                1,
+                255
+            ],
+            "$TC actions add action reclassify index 65536"
+        ],