1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
|
Adding Context to ``manifestparser`` based Manifests
----------------------------------------------------
Suites that use ``manifestparser``, like Mochitest and XPCShell, have test
manifests that denote whether a given test should be skipped or not based
on a set of context.
Gecko builds generate a ``target.mozinfo.json`` with metadata about the build.
An example ``target.mozinfo.json`` might look like :download:`this
<target.mozinfo.json>`. These keys can then be used ``skip-if`` in test manifests:
.. code-block::
skip-if = e10s && os == 'win'
In this case, ``e10s`` is a boolean.
The test will download the build's ``target.mozinfo.json``, then update the
mozinfo dictionary with additional runtime information based on the task or
runtime environment. This logic lives in `mozinfo
<https://hg.mozilla.org/mozilla-central/file/default/testing/mozbase/mozinfo/mozinfo/mozinfo.py>`__.
How to Add a Keyword
~~~~~~~~~~~~~~~~~~~~
Where to add the new key depends on what type of information it is.
1. If the key is a property of the build, you'll need to patch `this file
<https://searchfox.org/mozilla-central/source/python/mozbuild/mozbuild/mozinfo.py>`_.
2. If the key is a property of the test environment, you'll need to patch
`mozinfo <https://firefox-source-docs.mozilla.org/mozbase/mozinfo.html>`_.
3. If the key is a runtime configuration, for example based on a pref that is
passed in via mach or the task configuration, then you'll need to update the
individual test harnesses. For example, `this location
<https://searchfox.org/mozilla-central/rev/a7e33b7f61e7729e2b1051d2a7a27799f11a5de6/testing/mochitest/runtests.py#3341>`_
for Mochitest. Currently there is no shared location to set runtime keys
across test harnesses.
Adding a Context to Reftest Style Manifests
-------------------------------------------
Reftests and Crashtests use a different kind of manifest, but the general idea
is the same.
As before, Gecko builds generate a ``target.mozinfo.json`` with metadata about
the build. An example ``target.mozinfo.json`` might look like :download:`this
<target.mozinfo.json>`. This is consumed in the Reftest harness and translated
into keywords that can be used like:
.. code-block::
fuzzyIf(cocoaWidget&&isDebugBuild,1-1,85-88)
In this case, ``cocoaWidget`` and ``isDebugbuild`` are booleans.
The test will download the build's ``target.mozinfo.json``, then in addition to
the mozinfo, will query runtime info from the browser to build a sandbox of
keywords. This logic lives in `manifest.sys.mjs
<https://searchfox.org/mozilla-central/source/layout/tools/reftest/manifest.sys.mjs#439>`__.
How to Add a Keyword
~~~~~~~~~~~~~~~~~~~~
Where to add the new key depends on what type of information it is.
1. If the key is a property of the build, you'll need to patch `this file
<https://searchfox.org/mozilla-central/source/python/mozbuild/mozbuild/mozinfo.py>`_.
2. If the key is a property of the test environment or a runtime configuration,
then you'll need need to update manifest sandbox.
For example, for Apple Silicon, we can add an ``apple_silicon`` keyword with a
patch like this:
.. code-block:: diff
--- a/layout/tools/reftest/manifest.sys.mjs
+++ b/layout/tools/reftest/manifest.sys.mjs
@@ -572,16 +572,18 @@ function BuildConditionSandbox(aURL) {
// Set OSX to be the Mac OS X version, as an integer, or undefined
// for other platforms. The integer is formed by 100 times the
// major version plus the minor version, so 1006 for 10.6, 1010 for
// 10.10, etc.
var osxmatch = /Mac OS X (\d+).(\d+)$/.exec(hh.oscpu);
sandbox.OSX = osxmatch ? parseInt(osxmatch[1]) * 100 + parseInt(osxmatch[2]) : undefined;
+ sandbox.apple_silicon = sandbox.cocoaWidget && sandbox.OSX>=11;
+
// Plugins are no longer supported. Don't try to use TestPlugin.
sandbox.haveTestPlugin = false;
// Set a flag on sandbox if the windows default theme is active
sandbox.windowsDefaultTheme = g.containingWindow.matchMedia("(-moz-windows-default-theme)").matches;
try {
sandbox.nativeThemePref = !prefs.getBoolPref("widget.disable-native-theme-for-content");
Then to use this:
.. code-block::
fuzzy-if(apple_silicon,1-1,281-281) == frame_above_rules_none.html frame_above_rules_none_ref.html
|
