1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
|
Running NSPR tests
==================
NSPR has a test suite in the ``mozilla/nsprpub/pr/tests`` directory.
By default, we don't build the test programs. Running ``gmake`` in the
top-level directory (``mozilla/nsprpub``) only builds the NSPR
libraries. To build the test programs, you need to change directory to
``mozilla/nsprpub/pr/tests`` and run ``gmake``. Refer to :ref:`NSPR build
instructions` for details.
To run the test suite, run the shell script
``mozilla/nsprpub/pr/tests/runtests.sh`` in the directory where the test
program binaries reside, for example,
.. code::
cvs -q co -r NSPR_4_6_6_RTM mozilla/nsprpub
mkdir linux.debug
cd linux.debug
../mozilla/nsprpub/configure
gmake
cd pr/tests
gmake
../../../mozilla/nsprpub/pr/tests/runtests.sh
The output of the test suite looks like this:
.. code::
NSPR Test Results - tests
BEGIN Mon Mar 12 11:44:41 PDT 2007
NSPR_TEST_LOGFILE /dev/null
Test Result
accept Passed
acceptread Passed
acceptreademu Passed
affinity Passed
alarm Passed
anonfm Passed
atomic Passed
attach Passed
bigfile Passed
cleanup Passed
cltsrv Passed
concur Passed
cvar Passed
cvar2 Passed
...
sprintf FAILED
...
timetest Passed
tpd Passed
udpsrv Passed
vercheck Passed
version Passed
writev Passed
xnotify Passed
zerolen Passed
END Mon Mar 12 11:55:47 PDT 2007
.. _How_to_determine_if_the_test_suite_passed:
How to determine if the test suite passed
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If all the tests reported **Passed** as the results, the test suite
passed.
What if some of the tests crashed or reported **FAILED** as the results?
It doesn't necessarily mean the test suite failed because some of the
test programs are known to fail. Until the test failures are fixed, you
should run NSPR tests against **a known good version of NSPR on the same
platform**, and save the test results as the benchmark. Then you can
detect regressions of the new version by comparing its test results with
the benchmark.
.. _Known_issues:
Known issues
~~~~~~~~~~~~
Other issues with the NSPR test suite are:
#. Some of the test programs test the accuracy of the timeout of NSPR
functions. Since none of our operating systems is a real-time OS,
such test programs may fail when the test machine is heavily loaded.
#. Some tests, such as ``pipepong`` and ``sockpong``, should not be run
directly. They will be invoked by their companion test programs
(e.g., ``pipeping`` and ``sockping``). This is not an issue if you
run ``runtests.sh`` because ``runtests.sh`` knows not to run such
test programs directly.
|
