Adding upstream version 86.0.1.upstream/86.0.1upstream

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

1 files changed, 226 insertions, 0 deletions

diff --git a/build/docs/test_manifests.rst b/build/docs/test_manifests.rst
new file mode 100644
index 0000000000..89a7ded2aa
--- /dev/null
+++ b/build/docs/test_manifests.rst
@@ -0,0 +1,226 @@
+.. _test_manifests:
+
+==============
+Test Manifests
+==============
+
+Many test suites have their test metadata defined in files called
+**test manifests**.
+
+Test manifests are divided into two flavors: :ref:`manifestparser_manifests`
+and :ref:`reftest_manifests`.
+
+Naming Convention
+=================
+
+The build system does not enforce file naming for test manifest files.
+However, the following convention is used.
+
+mochitest.ini
+   For the *plain* flavor of mochitests.
+
+chrome.ini
+   For the *chrome* flavor of mochitests.
+
+browser.ini
+   For the *browser chrome* flavor of mochitests.
+
+a11y.ini
+   For the *a11y* flavor of mochitests.
+
+xpcshell.ini
+   For *xpcshell* tests.
+
+.. _manifestparser_manifests:
+
+ManifestParser Manifests
+==========================
+
+ManifestParser manifests are essentially ini files that conform to a basic
+set of assumptions.
+
+The :doc:`reference documentation </mozbase/manifestparser>`
+for manifestparser manifests describes the basic format of test manifests.
+
+In summary, manifests are ini files with section names describing test files::
+
+    [test_foo.js]
+    [test_bar.js]
+
+Keys under sections can hold metadata about each test::
+
+    [test_foo.js]
+    skip-if = os == "win"
+    [test_foo.js]
+    skip-if = os == "linux" && debug
+    [test_baz.js]
+    fail-if = os == "mac" || os == "android"
+
+There is a special **DEFAULT** section whose keys/metadata apply to all
+sections/tests::
+
+    [DEFAULT]
+    property = value
+
+    [test_foo.js]
+
+In the above example, **test_foo.js** inherits the metadata **property = value**
+from the **DEFAULT** section.
+
+Recognized Metadata
+-------------------
+
+Test manifests can define some common keys/metadata to influence behavior.
+Those keys are as follows:
+
+head
+   List of files that will be executed before the test file. (Used in
+   xpcshell tests.)
+
+tail
+   List of files that will be executed after the test file. (Used in
+   xpcshell tests.)
+
+support-files
+   List of additional files required to run tests. This is typically
+   defined in the **DEFAULT** section.
+
+   Unlike other file lists, *support-files* supports a globbing mechanism
+   to facilitate pulling in many files with minimal typing. This globbing
+   mechanism is activated if an entry in this value contains a ``*``
+   character. A single ``*`` will wildcard match all files in a directory.
+   A double ``**`` will descend into child directories. For example,
+   ``data/*`` will match ``data/foo`` but not ``data/subdir/bar`` where
+   ``data/**`` will match ``data/foo`` and ``data/subdir/bar``.
+
+   Support files starting with ``/`` are placed in a root directory, rather
+   than a location determined by the manifest location. For mochitests,
+   this allows for the placement of files at the server root. The source
+   file is selected from the base name (e.g., ``foo`` for ``/path/foo``).
+   Files starting with ``/`` cannot be selected using globbing.
+
+   Some support files are used by tests across multiple directories. In
+   this case, a test depending on a support file from another directory
+   must note that dependency with the path to the required support file
+   in its own **support-files** entry. These use a syntax where paths
+   starting with ``!/`` will indicate the beginning of the path to a
+   shared support file starting from the root of the srcdir. For example,
+   if a manifest at ``dom/base/test/mochitest.ini`` has a support file,
+   ``dom/base/test/server-script.sjs``, and a mochitest in
+   ``dom/workers/test`` depends on that support file, the test manifest
+   at ``dom/workers/test/mochitest.ini`` must include
+   ``!/dom/base/test/server-script.sjs`` in its **support-files** entry.
+
+generated-files
+   List of files that are generated as part of the build and don't exist in
+   the source tree.
+
+   The build system assumes that each manifest file, test file, and file
+   listed in **head**, **tail**, and **support-files** is static and
+   provided by the source tree (and not automatically generated as part
+   of the build). This variable tells the build system not to make this
+   assumption.
+
+   This variable will likely go away sometime once all generated files are
+   accounted for in the build config.
+
+   If a generated file is not listed in this key, a clobber build will
+   likely fail.
+
+dupe-manifest
+   Record that this manifest duplicates another manifest.
+
+   The common scenario is two manifest files will include a shared
+   manifest file via the ``[include:file]`` special section. The build
+   system enforces that each test file is only provided by a single
+   manifest. Having this key present bypasses that check.
+
+   The value of this key is ignored.
+
+skip-if
+   Skip this test if the specified condition is true.
+   See :ref:`manifest_filter_language`.
+
+   Conditions can be specified on multiple lines, where each line is implicitly
+   joined by a logical OR (``||``). This makes it easier to add comments to
+   distinct failures. For example:
+
+   .. parsed-literal::
+
+      [test_foo.js]
+      skip-if =
+          os == "mac" && fission  # bug 123 - fails on fission
+          os == "windows" && debug  # bug 456 - hits an assertion
+
+fail-if
+   Expect test failure if the specified condition is true.
+   See :ref:`manifest_filter_language`.
+
+   Conditions can be specified on multiple lines (see ``skip-if``).
+
+run-sequentially
+   If present, the test should not be run in parallel with other tests.
+
+   Some test harnesses support parallel test execution on separate processes
+   and/or threads (behavior varies by test harness). If this key is present,
+   the test harness should not attempt to run this test in parallel with any
+   other test.
+
+   By convention, the value of this key is a string describing why the test
+   can't be run in parallel.
+
+scheme
+   Changes the scheme and domain from which the test runs. (Only used in mochitest suites)
+   
+   There are two possible values:
+      - ``http`` (default): The test will run from http://mochi.test:8888
+      - ``https``: The test will run from https://example.com:443
+
+.. _manifest_filter_language:
+
+Manifest Filter Language
+------------------------
+
+Some manifest keys accept a special filter syntax as their values. These
+values are essentially boolean expressions that are evaluated at test
+execution time.
+
+The expressions can reference a well-defined set of variables, such as
+``os`` and ``debug``. These variables are populated from the
+``mozinfo.json`` file. For the full list of available variables, see
+the :ref:`mozinfo documentation <mozinfo_attributes>`.
+
+See
+`the source <https://hg.mozilla.org/mozilla-central/file/default/testing/mozbase/manifestparser/manifestparser/manifestparser.py>`_ for the full documentation of the
+expression syntax until it is documented here.
+
+.. todo::
+
+   Document manifest filter language.
+
+.. _manifest_file_installation:
+
+File Installation
+-----------------
+
+Files referenced by manifests are automatically installed into the object
+directory into paths defined in
+:py:func:`mozbuild.frontend.emitter.TreeMetadataEmitter._process_test_manifest`.
+
+Relative paths resolving to parent directory (e.g.
+``support-files = ../foo.txt`` have special behavior.
+
+For ``support-files``, the file will be installed to the default destination
+for that manifest. Only the file's base name is used to construct the final
+path: directories are irrelevant.  Files starting with ``/`` are an exception,
+these are installed relative to the root of the destination; the base name is
+instead used to select the file..
+
+For all other entry types, the file installation is skipped.
+
+.. _reftest_manifests:
+
+Reftest Manifests
+=================
+
+See `MDN <https://developer.mozilla.org/en-US/docs/Creating_reftest-based_unit_tests>`_.