diff options
Diffstat (limited to 'selftest/README')
-rw-r--r-- | selftest/README | 120 |
1 files changed, 120 insertions, 0 deletions
diff --git a/selftest/README b/selftest/README new file mode 100644 index 0000000..c898c3c --- /dev/null +++ b/selftest/README @@ -0,0 +1,120 @@ +# vim: ft=rst + +This directory contains test scripts that are useful for running a +bunch of tests all at once. + +There are two parts to this: + + * The test runner (selftest/selftest.pl) + * The test formatter + +selftest.pl simply outputs subunit, which can then be formatted or analyzed +by tools that understand the subunit protocol. One of these tools is +format-subunit, which is used by default as part of "make test". + +Available testsuites +==================== +The available testsuites are obtained from a script, usually +source{3,4}/selftest/tests.py. This script should for each testsuite output +the name of the test, the command to run and the environment that should be +provided. Use the included "plantest" function to generate the required output. + +Testsuite behaviour +=================== + +Exit code +------------ +The testsuites should exit with a non-zero exit code if at least one +test failed. Skipped tests should not influence the exit code. + +Output format +------------- +Testsuites can simply use the exit code to indicate whether all of their +tests have succeeded or one or more have failed. It is also possible to +provide more granular information using the Subunit protocol. + +This protocol works by writing simple messages to standard output. Any +messages that can not be interpreted by this protocol are considered comments +for the last announced test. + +For a full description of the subunit protocol, see the README file in the subunit +repository at http://github.com/testing-cabal/subunit. + +The following commands are Samba extensions to Subunit: + +start-testsuite +~~~~~~~~~~~~~~~ +start-testsuite: name + +The testsuite name is used as prefix for all containing tests. + +skip-testsuite +~~~~~~~~~~~~~~ +skip-testsuite: name + +Mark the testsuite with the specified name as skipped. + +testsuite-success +~~~~~~~~~~~~~~~~~ +testsuite-success: name + +Indicate that the testsuite has succeeded successfully. + +testsuite-fail +~~~~~~~~~~~~~~ +testsuite-fail: name + +Indicate that a testsuite has failed. + +Environments +============ +Tests often need to run against a server with particular things set up, +a "environment". This environment is provided by the test "target": Samba 3, +Samba 4 or Windows. + +The environments are currently available include + + - none: No server set up, no variables set. + - dc,s3dc: Domain controller set up. The following environment variables will + be set: + + * USERNAME: Administrator user name + * PASSWORD: Administrator password + * DOMAIN: Domain name + * REALM: Realm name + * SERVER: DC host name + * SERVER_IP: DC IPv4 address + * SERVER_IPV6: DC IPv6 address + * NETBIOSNAME: DC NetBIOS name + * NETIOSALIAS: DC NetBIOS alias + + - member,s4member,s3member: Domain controller and member server that is joined to it set up. The + following environment variables will be set: + + * USERNAME: Domain administrator user name + * PASSWORD: Domain administrator password + * DOMAIN: Domain name + * REALM: Realm name + * SERVER: Name of the member server + +See Samba.pm, Samba3.pm and Samba4.pm for the full list. + +Running tests +============= + +To run all the tests use:: + + make test + +To run a quicker subset run:: + + make quicktest + +To run a specific test, use this syntax:: + + make test TESTS=testname + +for example:: + + make test TESTS=samba4.BASE-DELETE + |