diff options
Diffstat (limited to '')
-rw-r--r-- | mysql-test/collections/README | 38 | ||||
-rw-r--r-- | mysql-test/collections/README.experimental | 36 |
2 files changed, 74 insertions, 0 deletions
diff --git a/mysql-test/collections/README b/mysql-test/collections/README new file mode 100644 index 00000000..f64c089e --- /dev/null +++ b/mysql-test/collections/README @@ -0,0 +1,38 @@ +This directory contains collections of test runs that we run during our +integration and release testing. Each file contains zero or more lines, +with one invocation of mysql-test-run.pl on each. These invocations are +written so that, with the assumption that perl is in your search path, +any collection can run as a shell script or a batch file, with the parent +mysql-test directory being the current working directory. + +During integration testing, we choose the collection to run by following +these steps: + +1) We choose the extension to look for, based on these rules: + - If we're running a per-push test, we choose ".push" as the extension. + - If we're running a daily test, we choose ".daily" as the extension. + - If we're running a weekly test, we choose ".weekly" as the extension. + +2) If there is a collection that has the same name as the branch we're + testing plus the extension as determined in step 1, we choose that + collection. + +3) If the branch is unknown or we have removed all characters from it + and still not found a matching collection, we choose the name "default" + plus the extension determined in step 1. If there is no such file, + we give up and don't test anything at all. + +4) If we haven't found a collection yet, we remove the last character from + the branch name and go back to step 2. + +5) The commands from the collection are run line by line via execv() or + similar system calls. They are not run as a shell script. Shell + expansions are not guaranteed to work and most likely won't. + +The directory may contain collections that are "super sets" of others, +identified by a file name suffix ".in". These files may contain lines +"#include <filename>", or lines with mysql-test-run.pl invocations. +CMake will create a new file without the .in suffix where +the include lines are replaced with the contents of the referred +file. Filename is local to the collections directory, and includes do +not nest. diff --git a/mysql-test/collections/README.experimental b/mysql-test/collections/README.experimental new file mode 100644 index 00000000..924e062b --- /dev/null +++ b/mysql-test/collections/README.experimental @@ -0,0 +1,36 @@ +The .experimental files in this directory contain names of test cases that +are still in development and whose failures should be considered expected, +instead of regressions. + +These files are to be used with the --experimental option of +mysql-test-run.pl. Please look at its help screen for usage information. + +The syntax is as follows: + +1) One line per test case. + +2) Empty lines and lines starting with a hash (#) are ignored. + +3) If any other line contains a blank followed by a hash (#), the hash + and any subsequent characters are ignored. + +4) The full test case name including the suite and execution mode + may be specified, for example: + main.alias 'row' # bug#00000 + +4b) Now, combinations will also be covered if only the test name is + specified, for example: + rpl.rpl_ps # Covers 'row', 'mix' and 'stmt' + +5) As an exception to item 4, the last character of the test case + specification may be an asterisk (*). In that case, all test cases that + start with the same characters up to the last letter before the asterisk + are considered experimental: + main.a* # get rid of main.alias, main.alibaba and main.agliolio + +6) Optionally, the test case may be followed by one or more platform + qualifiers beginning with @ or @!. The test will then be considered + experimental only/except on that platform. Basic OS names as + reported by $^O in Perl, or 'windows' are supported, this includes + solaris, linux, windows, aix, darwin, ... Example: + main.alias @aix @windows # Fails on those |