diff options
Diffstat (limited to 'test/README.testsuite')
-rw-r--r-- | test/README.testsuite | 131 |
1 files changed, 131 insertions, 0 deletions
diff --git a/test/README.testsuite b/test/README.testsuite new file mode 100644 index 0000000..5d9e498 --- /dev/null +++ b/test/README.testsuite @@ -0,0 +1,131 @@ +The extended testsuite only works with UID=0. It contains of several +subdirectories named "test/TEST-??-*", which are run one by one. + +To run the extended testsuite do the following: + +$ ninja -C build # Avoid building anything as root later +$ sudo test/run-integration-tests.sh +ninja: Entering directory `/home/zbyszek/src/systemd/build' +ninja: no work to do. +--x-- Running TEST-01-BASIC --x-- ++ make -C TEST-01-BASIC BUILD_DIR=/home/zbyszek/src/systemd/build clean setup run +make: Entering directory '/home/zbyszek/src/systemd/test/TEST-01-BASIC' +TEST-01-BASIC CLEANUP: Basic systemd setup +TEST-01-BASIC SETUP: Basic systemd setup +... +TEST-01-BASIC RUN: Basic systemd setup [OK] +make: Leaving directory '/home/zbyszek/src/systemd/test/TEST-01-BASIC' +--x-- Result of TEST-01-BASIC: 0 --x-- +--x-- Running TEST-02-CRYPTSETUP --x-- ++ make -C TEST-02-CRYPTSETUP BUILD_DIR=/home/zbyszek/src/systemd/build clean setup run + +If one of the tests fails, then $subdir/test.log contains the log file of +the test. + +To run just one of the cases: + +$ sudo make -C test/TEST-01-BASIC clean setup run + +Specifying the build directory +============================== + +If the build directory is not detected automatically, it can be specified +with BUILD_DIR=: + +$ sudo BUILD_DIR=some-other-build/ test/run-integration-tests + +or + +$ sudo make -C test/TEST-01-BASIC BUILD_DIR=../../some-other-build/ ... + +Note that in the second case, the path is relative to the test case directory. +An absolute path may also be used in both cases. + +Configuration variables +======================= + +TEST_NO_QEMU=1 + Don't run tests under QEMU + +TEST_NO_NSPAWN=1 + Don't run tests under systemd-nspawn + +TEST_NO_KVM=1 + Disable QEMU KVM auto-detection (may be necessary when you're trying to run the + *vanilla* QEMU and have both qemu and qemu-kvm installed) + +TEST_NESTED_KVM=1 + Allow tests to run with nested KVM. By default, the testsuite disables + nested KVM if the host machine already runs under KVM. Setting this + variable disables such checks + +QEMU_MEM=512M + Configure amount of memory for QEMU VMs (defaults to 512M) + +QEMU_SMP=1 + Configure number of CPUs for QEMU VMs (defaults to 1) + +KERNEL_APPEND='...' + Append additional parameters to the kernel command line + +NSPAWN_ARGUMENTS='...' + Specify additional arguments for systemd-nspawn + +QEMU_TIMEOUT=infinity + Set a timeout for tests under QEMU (defaults to infinity) + +NSPAWN_TIMEOUT=infinity + Set a timeout for tests under systemd-nspawn (defaults to infinity) + +INTERACTIVE_DEBUG=1 + Configure the machine to be more *user-friendly* for interactive debuggung + (e.g. by setting a usable default terminal, suppressing the shutdown after + the test, etc.) + +The kernel and initramfs can be specified with $KERNEL_BIN and $INITRD. +(Fedora's or Debian's default kernel path and initramfs are used by default) + +A script will try to find your QEMU binary. If you want to specify a different +one with $QEMU_BIN. + +Debugging the qemu image +======================== + +If you want to log in the testsuite virtual machine, you can specify additional +kernel command line parameter with $KERNEL_APPEND and then log in as root. + +$ sudo make -C test/TEST-01-BASIC KERNEL_APPEND="systemd.unit=multi-user.target" run + +Root password is empty. + +Ubuntu CI +========= + +New PR submitted to the project are run through regression tests, and one set +of those is the 'autopkgtest' runs for several different architectures, called +'Ubuntu CI'. Part of that testing is to run all these tests. Sometimes these +tests are temporarily deny-listed from running in the 'autopkgtest' tests while +debugging a flaky test; that is done by creating a file in the test directory +named 'deny-list-ubuntu-ci', for example to prevent the TEST-01-BASIC test from +running in the 'autopkgtest' runs, create the file +'TEST-01-BASIC/deny-list-ubuntu-ci'. + +The tests may be disabled only for specific archs, by creating a deny-list file +with the arch name at the end, e.g. +'TEST-01-BASIC/deny-list-ubuntu-ci-arm64' to disable the TEST-01-BASIC test +only on test runs for the 'arm64' architecture. + +Note the arch naming is not from 'uname -m', it is Debian arch names: +https://wiki.debian.org/ArchitectureSpecificsMemo + +For PRs that fix a currently deny-listed test, the PR should include removal +of the deny-list file. + +In case a test fails, the full set of artifacts, including the journal of the +failed run, can be downloaded from the artifacts.tar.gz archive which will be +reachable in the same URL parent directory as the logs.gz that gets linked on +the Github CI status. + +To add new dependencies or new binaries to the packages used during the tests, +a merge request can be sent to: https://salsa.debian.org/systemd-team/systemd +targeting the 'upstream-ci' branch. |