summaryrefslogtreecommitdiffstats
path: root/mysql-test/collections/README
diff options
context:
space:
mode:
Diffstat (limited to '')
-rw-r--r--mysql-test/collections/README38
-rw-r--r--mysql-test/collections/README.experimental36
2 files changed, 74 insertions, 0 deletions
diff --git a/mysql-test/collections/README b/mysql-test/collections/README
new file mode 100644
index 00000000..f64c089e
--- /dev/null
+++ b/mysql-test/collections/README
@@ -0,0 +1,38 @@
+This directory contains collections of test runs that we run during our
+integration and release testing. Each file contains zero or more lines,
+with one invocation of mysql-test-run.pl on each. These invocations are
+written so that, with the assumption that perl is in your search path,
+any collection can run as a shell script or a batch file, with the parent
+mysql-test directory being the current working directory.
+
+During integration testing, we choose the collection to run by following
+these steps:
+
+1) We choose the extension to look for, based on these rules:
+ - If we're running a per-push test, we choose ".push" as the extension.
+ - If we're running a daily test, we choose ".daily" as the extension.
+ - If we're running a weekly test, we choose ".weekly" as the extension.
+
+2) If there is a collection that has the same name as the branch we're
+ testing plus the extension as determined in step 1, we choose that
+ collection.
+
+3) If the branch is unknown or we have removed all characters from it
+ and still not found a matching collection, we choose the name "default"
+ plus the extension determined in step 1. If there is no such file,
+ we give up and don't test anything at all.
+
+4) If we haven't found a collection yet, we remove the last character from
+ the branch name and go back to step 2.
+
+5) The commands from the collection are run line by line via execv() or
+ similar system calls. They are not run as a shell script. Shell
+ expansions are not guaranteed to work and most likely won't.
+
+The directory may contain collections that are "super sets" of others,
+identified by a file name suffix ".in". These files may contain lines
+"#include <filename>", or lines with mysql-test-run.pl invocations.
+CMake will create a new file without the .in suffix where
+the include lines are replaced with the contents of the referred
+file. Filename is local to the collections directory, and includes do
+not nest.
diff --git a/mysql-test/collections/README.experimental b/mysql-test/collections/README.experimental
new file mode 100644
index 00000000..924e062b
--- /dev/null
+++ b/mysql-test/collections/README.experimental
@@ -0,0 +1,36 @@
+The .experimental files in this directory contain names of test cases that
+are still in development and whose failures should be considered expected,
+instead of regressions.
+
+These files are to be used with the --experimental option of
+mysql-test-run.pl. Please look at its help screen for usage information.
+
+The syntax is as follows:
+
+1) One line per test case.
+
+2) Empty lines and lines starting with a hash (#) are ignored.
+
+3) If any other line contains a blank followed by a hash (#), the hash
+ and any subsequent characters are ignored.
+
+4) The full test case name including the suite and execution mode
+ may be specified, for example:
+ main.alias 'row' # bug#00000
+
+4b) Now, combinations will also be covered if only the test name is
+ specified, for example:
+ rpl.rpl_ps # Covers 'row', 'mix' and 'stmt'
+
+5) As an exception to item 4, the last character of the test case
+ specification may be an asterisk (*). In that case, all test cases that
+ start with the same characters up to the last letter before the asterisk
+ are considered experimental:
+ main.a* # get rid of main.alias, main.alibaba and main.agliolio
+
+6) Optionally, the test case may be followed by one or more platform
+ qualifiers beginning with @ or @!. The test will then be considered
+ experimental only/except on that platform. Basic OS names as
+ reported by $^O in Perl, or 'windows' are supported, this includes
+ solaris, linux, windows, aix, darwin, ... Example:
+ main.alias @aix @windows # Fails on those