summaryrefslogtreecommitdiffstats
path: root/Documentation/howto-tests.txt
blob: 71c7f85c1612719c82954fc136387c71e45ddfbd (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
 util-linux regression tests
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~

 It's expected that for each invasive change or important bugfix you will
 include a test to your patch.

 Compile tests & run basic tests:

	$ make check

 Note that the configure option --disable-static disables many of libmount and
 libblkid unit tests.

 Run all tests including tests that require root permissions:

	# cd tests
	# ./run.sh [--verbose] [--memcheck]

	Alternatively using sudo and make:

	$ make check TS_COMMAND="true"
	$ sudo -E make check TS_OPTS="--parallel=1"

 note that as root you have to manually remove output and diff directories

	# rm -rf output diff

 or run 'make clean' as root.


 Run subset of tests:

	$ ./run.sh <test_directory-name>

        for example:

              $ ./run.sh blkid
              $ ./run.sh libmount

 The tests is possible to exclude by ./run.sh --exclude=<list>  where the
 <list> is blank separated test names in format "testdir/testname", for example:

	$ ./run.sh --exclude="mount/move"

 The --exclude is evaluated by the ./run.sh script only. See below

	TS_OPT_testdir_[testscript_]fake=

 environment variable which provides more powerful functionality to skip tests.


 *** WARNING for root users ***

 The tests touch your /etc/fstab, initialize loop devices or scsi_debug devices
 if executed with root permissions.

 Please, be careful and use these tests only for development and never on
 production system.


environment variables
---------------------

TS_COMMAND

    Evaluated by "make check" to override the default command (run.sh).
    Example:
      - build all test dependencies, but skip the actual test
        $ make check TS_COMMAND="true"

TS_OPTS

    Evaluated by "make check" to pass options.to run.sh (see ./run.sh --help).
    Examples:
      - run utmp tests only
        $ make check TS_OPTS="--parallel=1 utmp"

TS_OPT_testdir_[testscript_]fake="<yes|no>"

    Evaluated by any test script to skip certain tests.
    Examples:
      - skip all the tests within "fdisk" test-directory:
        $ make check TS_OPT_fdisk_fake="yes"

      - skip only "fdisk/bsd" test:
        $ make check TS_OPT_fdisk_bsd_fake="yes"

      - skip all "fdisk" tests except fdisk/bsd:
        $ make check TS_OPT_fdisk_fake="yes" TS_OPT_fdisk_bsd_fake="no"

TS_OPT_testdir_[testscript_]known_fail="<yes|no>"

    Similar usage like TS_OPT_*_fake above. "known_fail" means that the given
    test will run but (negative) results will be ignored. The build log and test
    diffs will still remind you about the issue.

TS_OPT_testdir_[testscript_]verbose="<yes|no>"

    Set verbosity for certain tests. Similar usage like TS_OPT_*_fake above.

TS_OPT_testdir_[testscript_]memcheck="<yes|no>"

    Run certain tests with valgrind. Similar usage like TS_OPT_*_fake above.


External services 
-----------------

Travis CI - automatically executed for all github commits.

    URL: https://travis-ci.org/karelzak/util-linux/

    See .travis.yml for more details.

    We require "sudo" to install additional stuff and to run the tests with
    UID=0, it means that tests are executed on travis "legacy infrastructure".


Drone.io - automatically executed for all github commits.

    URL: https://drone.io/github.com/karelzak/util-linux

    The drone.io does not use any in-tree config file (like travis-ci), the
    currently used configuration (maintained by Drone.io web UI):

	MAKE_CHECK="root"
	MAKE_CHECK_OPTS="--skip-loopdevs --exclude=mount/move"
	source ./.travis-functions.sh
	travis_install_script || exit
	travis_before_script || exit
	ret=0
	travis_script || ret=$?
	travis_after_script
	exit $ret  
	
    yes, it shares the setup functions with travis-ci.