Adding upstream version 124.0.1.upstream/124.0.1

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

5 files changed, 139 insertions, 0 deletions

diff --git a/taskcluster/docs/transforms/index.rst b/taskcluster/docs/transforms/index.rst
new file mode 100644
index 0000000000..641f1abbc4
--- /dev/null
+++ b/taskcluster/docs/transforms/index.rst
@@ -0,0 +1,22 @@
+Transforms
+==========
+
+`Transforms`_ are Python functions that can modify (or sometimes add and
+remove) tasks from the graph. This section describes some of the transform
+functions that are available in ``mozilla-central``.
+
+.. toctree::
+
+   signing
+   test
+   job
+   task
+
+Other Transforms
+----------------
+
+There are tons of transforms that are not documented here. Consult the source
+files under `taskcluster/gecko_taskgraph/transforms`_ which provide lots of
+additional detail, both in the code itself and in comments and docstrings.
+
+.. _taskcluster/gecko_taskgraph/transforms: https://searchfox.org/mozilla-central/source/taskcluster/gecko_taskgraph/transforms
diff --git a/taskcluster/docs/transforms/job.rst b/taskcluster/docs/transforms/job.rst
new file mode 100644
index 0000000000..160402dd36
--- /dev/null
+++ b/taskcluster/docs/transforms/job.rst
@@ -0,0 +1,42 @@
+Job Transforms
+==============
+
+.. note::
+
+   These transforms are currently duplicated by standalone Taskgraph
+   and will likely be refactored / removed at a later date.
+
+A job description says what to run in the task.  It is a combination of a
+``run`` section and all of the fields from a task description.  The run section
+has a ``using`` property that defines how this task should be run; for example,
+``mozharness`` to run a mozharness script, or ``mach`` to run a mach command.
+The remainder of the run section is specific to the run-using implementation.
+
+The effect of a job description is to say "run this thing on this worker".  The
+job description must contain enough information about the worker to identify
+the workerType and the implementation (docker-worker, generic-worker, etc.).
+Alternatively, job descriptions can specify the ``platforms`` field in
+conjunction with the  ``by-platform`` key to specify multiple workerTypes and
+implementations. Any other task-description information is passed along
+verbatim, although it is augmented by the run-using implementation.
+
+The run-using implementations are all located in
+``taskcluster/gecko_taskgraph/transforms/job``, along with the schemas for their
+implementations.  Those well-commented source files are the canonical
+documentation for what constitutes a job description, and should be considered
+part of the documentation.
+
+following ``run-using`` are available
+
+  * ``hazard``
+  * ``mach``
+  * ``mozharness``
+  * ``mozharness-test``
+  * ``run-task``
+  * ``spidermonkey`` or ``spidermonkey-package``
+  * ``debian-package``
+  * ``ubuntu-package``
+  * ``toolchain-script``
+  * ``always-optimized``
+  * ``fetch-url``
+  * ``python-test``
diff --git a/taskcluster/docs/transforms/signing.rst b/taskcluster/docs/transforms/signing.rst
new file mode 100644
index 0000000000..52b13765bf
--- /dev/null
+++ b/taskcluster/docs/transforms/signing.rst
@@ -0,0 +1,15 @@
+Signing Transforms
+==================
+
+Signing kinds are passed a single dependent job (from its kind dependency) to act
+on.
+
+The transforms in ``taskcluster/gecko_taskgraph/transforms/signing.py`` implement
+this common functionality.  They expect a "signing description", and produce a
+task definition.  The schema for a signing description is defined at the top of
+``signing.py``, with copious comments.
+
+In particular you define a set of upstream artifact urls (that point at the
+dependent task) and can optionally provide a dependent name (defaults to build)
+for use in ``task-reference``/``artifact-reference``. You also need to provide
+the signing formats to use.
diff --git a/taskcluster/docs/transforms/task.rst b/taskcluster/docs/transforms/task.rst
new file mode 100644
index 0000000000..f19b4a7e15
--- /dev/null
+++ b/taskcluster/docs/transforms/task.rst
@@ -0,0 +1,45 @@
+Task Transforms
+===============
+
+.. note::
+
+   These transforms are currently duplicated by standalone Taskgraph
+   and will likely be refactored / removed at a later date.
+
+Every kind needs to create tasks, and all of those tasks have some things in
+common.  They all run on one of a small set of worker implementations, each
+with their own idiosyncrasies.  And they all report to TreeHerder in a similar
+way.
+
+The transforms in ``taskcluster/gecko_taskgraph/transforms/task.py`` implement
+this common functionality.  They expect a "task description", and produce a
+task definition.  The schema for a task description is defined at the top of
+``task.py``, with copious comments.  Go forth and read it now!
+
+In general, the task-description transforms handle functionality that is common
+to all Gecko tasks.  While the schema is the definitive reference, the
+functionality includes:
+
+* TreeHerder metadata
+
+* Build index routes
+
+* Information about the projects on which this task should run
+
+* Optimizations
+
+* Defaults for ``expires-after`` and and ``deadline-after``, based on project
+
+* Worker configuration
+
+The parts of the task description that are specific to a worker implementation
+are isolated in a ``task_description['worker']`` object which has an
+``implementation`` property naming the worker implementation.  Each worker
+implementation has its own section of the schema describing the fields it
+expects.  Thus the transforms that produce a task description must be aware of
+the worker implementation to be used, but need not be aware of the details of
+its payload format.
+
+The ``task.py`` file also contains a dictionary mapping treeherder groups to
+group names using an internal list of group names.  Feel free to add additional
+groups to this list as necessary.
diff --git a/taskcluster/docs/transforms/test.rst b/taskcluster/docs/transforms/test.rst
new file mode 100644
index 0000000000..901aaa7cc1
--- /dev/null
+++ b/taskcluster/docs/transforms/test.rst
@@ -0,0 +1,15 @@
+Test Transforms
+===============
+
+Test descriptions specify how to run a unittest or talos run.  They aim to
+describe this abstractly, although in many cases the unique nature of
+invocation on different platforms leaves a lot of specific behavior in the test
+description, divided by ``by-test-platform``.
+
+Test descriptions are validated to conform to the schema in
+``taskcluster/gecko_taskgraph/transforms/test/__init__.py``.  This schema is
+extensively documented and is a the primary reference for anyone modifying
+tests.
+
+The output of ``tests.py`` is a task description.  Test dependencies are
+produced in the form of a dictionary mapping dependency name to task label.