diff options
Diffstat (limited to '')
| -rw-r--r-- | mysql-test/README | 100 | ||||
| -rw-r--r-- | mysql-test/README-gcov | 8 | ||||
| -rw-r--r-- | mysql-test/README.stress | 120 |
3 files changed, 228 insertions, 0 deletions
diff --git a/mysql-test/README b/mysql-test/README new file mode 100644 index 00000000..ae1d6d56 --- /dev/null +++ b/mysql-test/README @@ -0,0 +1,100 @@ +This directory contains test suites for the MariaDB server. To run +currently existing test cases, execute ./mysql-test-run in this directory. + +Some tests are known to fail on some platforms or be otherwise unreliable. +In the file collections/smoke_test there is a list of tests that are +expected to be stable. + +In general you do not have to have to do "make install", and you can have +a co-existing MariaDB installation, the tests will not conflict with it. +To run the tests in a source directory, you must do "make" first. + +In Red Hat distributions, you should run the script as user "mysql". +The user is created with nologin shell, so the best bet is something like + # su - + # cd /usr/share/mysql-test + # su -s /bin/bash mysql -c ./mysql-test-run + +This will use the installed MariaDB executables, but will run a private +copy of the server process (using data files within /usr/share/mysql-test), +so you need not start the mysqld service beforehand. + +You can omit --skip-test-list option if you want to check whether +the listed failures occur for you. + +To clean up afterwards, remove the created "var" subdirectory, e.g. + # su -s /bin/bash - mysql -c "rm -rf /usr/share/mysql-test/var" + +If tests fail on your system, please read the following manual section +for instructions on how to report the problem: + +https://mariadb.com/kb/en/reporting-bugs + +If you want to use an already running MySQL server for specific tests, +use the --extern option to mysql-test-run. Please note that in this mode, +you are expected to provide names of the tests to run. + +For example, here is the command to run the "alias" and "analyze" tests +with an external server: + + # mysql-test-run --extern socket=/tmp/mysql.sock alias analyze + +To match your setup, you might need to provide other relevant options. + +With no test names on the command line, mysql-test-run will attempt +to execute the default set of tests, which will certainly fail, because +many tests cannot run with an external server (they need to control the +options with which the server is started, restart the server during +execution, etc.) + +You can create your own test cases. To create a test case, create a new +file in the main subdirectory using a text editor. The file should have a .test +extension. For example: + + # xemacs t/test_case_name.test + +In the file, put a set of SQL statements that create some tables, +load test data, and run some queries to manipulate it. + +Your test should begin by dropping the tables you are going to create and +end by dropping them again. This ensures that you can run the test over +and over again. + +If you are using mysqltest commands in your test case, you should create +the result file as follows: + + # mysql-test-run --record test_case_name + + or + + # mysqltest --record < t/test_case_name.test + +If you only have a simple test case consisting of SQL statements and +comments, you can create the result file in one of the following ways: + + # mysql-test-run --record test_case_name + + # mysql test < t/test_case_name.test > r/test_case_name.result + + # mysqltest --record --database test --result-file=r/test_case_name.result < t/test_case_name.test + +When this is done, take a look at r/test_case_name.result. +If the result is incorrect, you have found a bug. In this case, you should +edit the test result to the correct results so that we can verify that +the bug is corrected in future releases. + +If you want to submit your test case you can send it +to maria-developers@lists.launchpad.net or attach it to a bug report on +https://mariadb.org/jira/. + +If the test case is really big or if it contains 'not public' data, +then put your .test file and .result file(s) into a tar.gz archive, +add a README that explains the problem, ftp the archive to +ftp://ftp.mariadb.org/private and submit a report to +https://mariadb.org/jira about it. + +The latest information about mysql-test-run can be found at: +https://mariadb.com/kb/en/mariadb/mysqltest/ + +If you want to create .rdiff files, check +https://mariadb.com/kb/en/mariadb/mysql-test-auxiliary-files/ diff --git a/mysql-test/README-gcov b/mysql-test/README-gcov new file mode 100644 index 00000000..64a97ce4 --- /dev/null +++ b/mysql-test/README-gcov @@ -0,0 +1,8 @@ +To be able to see the level of coverage with the current test suite, +do the following: + + - Make sure gcov is installed + - Compile the MariaDB distribution with -DENABLE_GCOV=1 + - In the mysql-test directory, run this command: `./mysql-test-run --gcov` + - see var/last_changes.dgcov for the coverage of uncommitted code + - see `./dgcov -h` for more options diff --git a/mysql-test/README.stress b/mysql-test/README.stress new file mode 100644 index 00000000..6be4e9a0 --- /dev/null +++ b/mysql-test/README.stress @@ -0,0 +1,120 @@ + +Overview +-------- + +The stress script is designed to perform testing of the MySQL server in +a multi-threaded environment. + +All functionality regarding stress testing is implemented in the +mysql-stress-test.pl script. + +The stress script allows: + + - To stress test the mysqltest binary test engine. + - To stress test the regular test suite and any additional test suites + (such as mysql-test-extra-5.0). + - To specify files with lists of tests both for initialization of + stress db and for further testing itself. + - To define the number of threads to be concurrently used in testing. + - To define limitations for the test run. such as the number of tests or + loops for execution or duration of testing, delay between test + executions, and so forth. + - To get a readable log file that can be used for identification of + errors that occur during testing. + +There are two ways to run the mysql-stress-test.pl script: + + - For most cases, it is enough to use the options below for starting + the stress test from the mysql-test-run wrapper. In this case, the + server is run automatically, all preparation steps are performed, + and after that the stress test is started. + + - In advanced case, you can run the mysql-stress-test.pl script directly. + But this requires that you perform some preparation steps and to specify + a bunch of options as well, so this invocation method may be a bit + complicated. + +Usage +----- + +The following mysql-test-run options are specific to stress-testing: + +--stress + Enable stress mode + +--stress-suite=<suite name> + Test suite name to use in stress testing. We assume that all suites + are located in the mysql-test/suite directory. + There is one special suite name - <main|default> that corresponds + to the regular test suite located in the mysql-test directory. + +--stress-threads=<number of threads> + The number of threads to use in stress testing. + +--stress-tests-file=<filename with list of tests> + The file that contains the list of tests (without .test suffix) to use in + stress testing. The default filename is stress_tests.txt and the default + location of this file is suite/<suite name>/stress_tests.txt + +--stress-init-file=<filename with list of tests> + The file that contains list of tests (without .test suffix) to use in + stress testing for initialization of the stress db. These tests will be + executed only once before starting the test itself. The default filename + is stress_init.txt and the default location of this file is + suite/<suite name>/stress_init.txt + +--stress-mode=<method which will be used for choosing tests from the list> + Possible values are: random(default), seq + + There are two possible modes that affect the order of test selection + from the list: + - In random mode, tests are selected in random order + - In seq mode, each thread executes tests in a loop one by one in + the order specified in the list file. + +--stress-test-count=<number> + Total number of tests that will be executed concurrently by all threads + +--stress-loop-count=<number> + Total number of loops in seq mode that will be executed concurrently + by all threads + +--stress-test-duration=<number> + Duration of stress testing in seconds + +Examples +-------- + +1. Example of a simple command line to start a stress test: + + mysql-test-run --stress alias + +Runs a stress test with default values for number of threads and number +of tests, with test 'alias' from suite 'main'. + +2. Using in stress testing tests from other suites: + + - mysql-test-run --stress --stress-threads=10 --stress-test-count=1000 \ + --stress-suite=example --stress-tests-file=testslist.txt + + Runs a stress test with 10 threads, executes 1000 tests by all + threads, tests are used from suite 'example', the list of tests is + taken from file 'testslist.txt' + + - mysql-test-run --stress --stress-threads=10 --stress-test-count=1000 \ + --stress-suite=example sum_distinct + + Runs stress test with 10 threads, executes 1000 tests by all + threads, tests are used from suite 'example', the list of tests + contains only one test 'sum_distinct' + +3. Debugging of issues found with stress test + + Right now, the stress test is not fully integrated in mysql-test-run + and does not support the --gdb option. To debug issues found with the + stress test, you must start the MySQL server separately under a debugger + and then run the stress test like this: + + - mysql-test-run --extern --stress --stress-threads=10 \ + --stress-test-count=1000 --stress-suite=example \ + sum_distinct |