summaryrefslogtreecommitdiffstats
path: root/Documentation/howto-tests.txt
diff options
context:
space:
mode:
Diffstat (limited to 'Documentation/howto-tests.txt')
-rw-r--r--Documentation/howto-tests.txt152
1 files changed, 152 insertions, 0 deletions
diff --git a/Documentation/howto-tests.txt b/Documentation/howto-tests.txt
new file mode 100644
index 0000000..15d0880
--- /dev/null
+++ b/Documentation/howto-tests.txt
@@ -0,0 +1,152 @@
+
+ util-linux regression tests
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ It's expected that for each invasive change or important bugfix you will
+ include a test to your patch.
+
+ Compile binaries, libs, extra test programs and run the basic tests:
+
+ $ make check
+
+ Compile extra test programs only:
+
+ $ make check-programs
+
+ Note that the configure option --disable-static disables many of libmount and
+ libblkid unit tests.
+
+ Run all tests including tests that require root permissions:
+
+ # cd tests
+ # ./run.sh [options, see --help]
+
+ Alternatively using sudo and make:
+
+ $ make check-programs
+ $ sudo -E make check TS_OPTS="--parallel=1"
+
+ note that as root you have to manually remove output and diff directories
+
+ # rm -rf output diff
+
+ or run 'make clean' as root.
+
+
+ Run subset of tests:
+
+ $ cd tests
+ $ ./run.sh <test_directory-name>
+
+ for example:
+
+ $ ./run.sh blkid
+ $ ./run.sh libmount
+
+ or individual test script, for example:
+
+ $ ./ts/cal/year
+
+ The tests is possible to exclude by ./run.sh --exclude=<list> where the
+ <list> is blank separated test names in format "testdir/testname", for example:
+
+ $ ./run.sh --exclude="mount/move"
+
+ The --exclude is evaluated by the ./run.sh script only. See below
+
+ TS_OPT_testdir_[testscript_]fake=
+
+ environment variable which provides more powerful functionality to skip tests.
+
+
+ *** WARNING for root users ***
+
+ The tests touch your /etc/fstab, initialize loop devices or scsi_debug devices
+ if executed with root permissions.
+
+ Please, be careful and use these tests only for development and never on
+ production system.
+
+fuzz targets
+------------
+
+The fuzz targets can be built and run along with the other tests (after installing
+clang):
+
+ $ ./tools/config-gen fuzz
+ $ make check
+
+environment variables
+---------------------
+
+TS_COMMAND
+
+ Evaluated by "make check" to override the default command (run.sh).
+ Example:
+ - build all test dependencies, but skip the actual test
+ $ make check TS_COMMAND="true"
+
+TS_OPTS
+
+ Evaluated by "make check" to pass options.to run.sh (see ./run.sh --help).
+ Examples:
+ - run utmp tests only
+ $ make check TS_OPTS="--parallel=1 utmp"
+
+TS_OPT_testdir_[testscript_]fake="<yes|no>"
+
+ Evaluated by any test script to skip certain tests.
+ Examples:
+ - skip all the tests within "fdisk" test-directory:
+ $ make check TS_OPT_fdisk_fake="yes"
+
+ - skip only "fdisk/bsd" test:
+ $ make check TS_OPT_fdisk_bsd_fake="yes"
+
+ - skip all "fdisk" tests except fdisk/bsd:
+ $ make check TS_OPT_fdisk_fake="yes" TS_OPT_fdisk_bsd_fake="no"
+
+TS_OPT_testdir_[testscript_]known_fail="<yes|no>"
+
+ Similar usage like TS_OPT_*_fake above. "known_fail" means that the given
+ test will run but (negative) results will be ignored. The build log and test
+ diffs will still remind you about the issue.
+
+TS_OPT_testdir_[testscript_]verbose="<yes|no>"
+
+ Set verbosity for certain tests. Similar usage like TS_OPT_*_fake above.
+
+TS_OPT_testdir_[testscript_]memcheck="<yes|no>"
+
+ Run certain tests with valgrind. Similar usage like TS_OPT_*_fake above.
+
+
+External services
+-----------------
+
+Coveralls coverage reports
+
+ URL: https://coveralls.io/github/util-linux/util-linux
+
+lgtm CI - automatically executed security code analysis
+
+ URL: https://lgtm.com/projects/g/util-linux/util-linux/
+
+Coverity Scan
+
+ URL: https://scan.coverity.com/projects/karelzak-util-linux
+
+Fossies codespell report
+
+ URL: https://fossies.org/linux/test/util-linux-master.tar.gz/codespell.html
+
+OSS-Fuzz
+
+ URL: https://google.github.io/oss-fuzz/
+ URL: https://oss-fuzz-build-logs.storage.googleapis.com/index.html#util-linux
+ URL: https://oss-fuzz.com/coverage-report/job/libfuzzer_asan_util-linux/latest
+
+CIFuzz
+
+ URL: https://google.github.io/oss-fuzz/getting-started/continuous-integration/
+ URL: https://github.com/util-linux/util-linux/actions?query=workflow%3ACIFuzz