diff options
Diffstat (limited to 'Documentation/howto-tests.txt')
| -rw-r--r-- | Documentation/howto-tests.txt | 152 |
1 files changed, 152 insertions, 0 deletions
diff --git a/Documentation/howto-tests.txt b/Documentation/howto-tests.txt new file mode 100644 index 0000000..15d0880 --- /dev/null +++ b/Documentation/howto-tests.txt @@ -0,0 +1,152 @@ + + util-linux regression tests + ~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + It's expected that for each invasive change or important bugfix you will + include a test to your patch. + + Compile binaries, libs, extra test programs and run the basic tests: + + $ make check + + Compile extra test programs only: + + $ make check-programs + + Note that the configure option --disable-static disables many of libmount and + libblkid unit tests. + + Run all tests including tests that require root permissions: + + # cd tests + # ./run.sh [options, see --help] + + Alternatively using sudo and make: + + $ make check-programs + $ sudo -E make check TS_OPTS="--parallel=1" + + note that as root you have to manually remove output and diff directories + + # rm -rf output diff + + or run 'make clean' as root. + + + Run subset of tests: + + $ cd tests + $ ./run.sh <test_directory-name> + + for example: + + $ ./run.sh blkid + $ ./run.sh libmount + + or individual test script, for example: + + $ ./ts/cal/year + + The tests is possible to exclude by ./run.sh --exclude=<list> where the + <list> is blank separated test names in format "testdir/testname", for example: + + $ ./run.sh --exclude="mount/move" + + The --exclude is evaluated by the ./run.sh script only. See below + + TS_OPT_testdir_[testscript_]fake= + + environment variable which provides more powerful functionality to skip tests. + + + *** WARNING for root users *** + + The tests touch your /etc/fstab, initialize loop devices or scsi_debug devices + if executed with root permissions. + + Please, be careful and use these tests only for development and never on + production system. + +fuzz targets +------------ + +The fuzz targets can be built and run along with the other tests (after installing +clang): + + $ ./tools/config-gen fuzz + $ make check + +environment variables +--------------------- + +TS_COMMAND + + Evaluated by "make check" to override the default command (run.sh). + Example: + - build all test dependencies, but skip the actual test + $ make check TS_COMMAND="true" + +TS_OPTS + + Evaluated by "make check" to pass options.to run.sh (see ./run.sh --help). + Examples: + - run utmp tests only + $ make check TS_OPTS="--parallel=1 utmp" + +TS_OPT_testdir_[testscript_]fake="<yes|no>" + + Evaluated by any test script to skip certain tests. + Examples: + - skip all the tests within "fdisk" test-directory: + $ make check TS_OPT_fdisk_fake="yes" + + - skip only "fdisk/bsd" test: + $ make check TS_OPT_fdisk_bsd_fake="yes" + + - skip all "fdisk" tests except fdisk/bsd: + $ make check TS_OPT_fdisk_fake="yes" TS_OPT_fdisk_bsd_fake="no" + +TS_OPT_testdir_[testscript_]known_fail="<yes|no>" + + Similar usage like TS_OPT_*_fake above. "known_fail" means that the given + test will run but (negative) results will be ignored. The build log and test + diffs will still remind you about the issue. + +TS_OPT_testdir_[testscript_]verbose="<yes|no>" + + Set verbosity for certain tests. Similar usage like TS_OPT_*_fake above. + +TS_OPT_testdir_[testscript_]memcheck="<yes|no>" + + Run certain tests with valgrind. Similar usage like TS_OPT_*_fake above. + + +External services +----------------- + +Coveralls coverage reports + + URL: https://coveralls.io/github/util-linux/util-linux + +lgtm CI - automatically executed security code analysis + + URL: https://lgtm.com/projects/g/util-linux/util-linux/ + +Coverity Scan + + URL: https://scan.coverity.com/projects/karelzak-util-linux + +Fossies codespell report + + URL: https://fossies.org/linux/test/util-linux-master.tar.gz/codespell.html + +OSS-Fuzz + + URL: https://google.github.io/oss-fuzz/ + URL: https://oss-fuzz-build-logs.storage.googleapis.com/index.html#util-linux + URL: https://oss-fuzz.com/coverage-report/job/libfuzzer_asan_util-linux/latest + +CIFuzz + + URL: https://google.github.io/oss-fuzz/getting-started/continuous-integration/ + URL: https://github.com/util-linux/util-linux/actions?query=workflow%3ACIFuzz |