summaryrefslogtreecommitdiffstats
path: root/js/src/jit-test/README
blob: b8b01af4b589b90b45b71c959f7fed75f10f47b9 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
JS Internal Test Suite

* PURPOSE

This is a test suite primarily for testing the SpiderMonkey JIT, GC, and any
other internal mechanisms that are not visible to test262. All tests are run in
the JS shell.

In the future, we intend to migrate the "non262" jstests over to this framework.

* CONTINUOUS INTEGRATION

In CI, these tests will be run as part of the "SM(...)" set of jobs. They will
also be packaged up and then run via mozharness as separate test jobs on some
platforms. These will appear on treeherder as jobs starting with "Jit", eg
"Jit", "Jit3", "Jit-1proc3", etc.

Unlike jstests, we do not run jit-tests in the browser. All tests may assume
they are running in the JS shell environment.

* REQUIREMENTS

Python 3.6. This is already a standard requirement for building our tree.

* RUNNING THE TESTS

Basic usage:

    ./mach jit-test

from the top of the checkout. Directly invoking

    python jit_test.py <path-to-js-shell>

will also work. The progress bar shows [#tests passed, #tests failed, #tests
run] at the left. If all tests pass, the output is 'PASSED ALL'. The test suite
can be interrupted at any time with Ctrl+C and partial results will be printed.

To run only the basic tests, not including the slow tests:

    ./mach jit-test <path-to-js-shell> basic

For more options:

    ./mach jit-test -- -h

or

    python jit_test.py -h

for the jit-test harness options, or

    ./mach jit-test -h

for the mach driver's options (eg --cgc).

* CREATING NEW TESTS

Simply create a JS file under the 'tests/' directory. Most tests should go in
'tests/basic/'.

All tests are run with 'lib/prologue.js' included first on the command line. The
command line also creates a global variable 'libdir' that is set to the path
of the 'lib' directory. To include a file 'foo.js' from the lib directory in a
test case:

    load(libdir + 'foo.js')

* TEST METALINES

The first line of a test case can contain a special comment controlling how the
test is run. For example:

    // |jit-test| allow-oom; --no-threads

The general format in EBNF is:

    metaline  ::= cookie { item ";" }
    cookie    ::= "|jit-test|"
    item      ::= flag | attribute

    flag      ::= "slow" | "allow-oom" | "valgrind" | "tz-pacific" | "debug" |
                  "--" switch

    attribute ::= name ":" value
    name      ::= "error" | "exitstatus"
    value     ::= <string>
    switch    ::= <string>

The metaline may appear anywhere in the first line of the file: this allows it
to be placed inside any kind of comment.

The meaning of the items:

    slow         Test runs slowly. Do not run if the --no-slow option is given.
    allow-oom    If the test runs out of memory, it counts as passing.
    valgrind     Run test under valgrind.
    tz-pacific   Always run test with the Pacific time zone (TZ=PST8PDT).

    error        The test should be considered to pass iff it throws the
                 given JS exception.
    exitstatus   The test should exit with the given status value (an integer).

    debug        Run js with -d, whether --jitflags says to or not
    --SWITCH     Pass --SWITCH through to js

* END