summaryrefslogtreecommitdiffstats
path: root/testing/web-platform/tests/tools/ci/tc/README.md
diff options
context:
space:
mode:
authorDaniel Baumann <daniel.baumann@progress-linux.org>2024-04-07 19:33:14 +0000
committerDaniel Baumann <daniel.baumann@progress-linux.org>2024-04-07 19:33:14 +0000
commit36d22d82aa202bb199967e9512281e9a53db42c9 (patch)
tree105e8c98ddea1c1e4784a60a5a6410fa416be2de /testing/web-platform/tests/tools/ci/tc/README.md
parentInitial commit. (diff)
downloadfirefox-esr-36d22d82aa202bb199967e9512281e9a53db42c9.tar.xz
firefox-esr-36d22d82aa202bb199967e9512281e9a53db42c9.zip
Adding upstream version 115.7.0esr.upstream/115.7.0esr
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'testing/web-platform/tests/tools/ci/tc/README.md')
-rw-r--r--testing/web-platform/tests/tools/ci/tc/README.md243
1 files changed, 243 insertions, 0 deletions
diff --git a/testing/web-platform/tests/tools/ci/tc/README.md b/testing/web-platform/tests/tools/ci/tc/README.md
new file mode 100644
index 0000000000..785c82cca3
--- /dev/null
+++ b/testing/web-platform/tests/tools/ci/tc/README.md
@@ -0,0 +1,243 @@
+# Taskgraph Setup
+
+The taskgraph is built from a YAML file. This file has two top-level
+properties: `components` and `tasks`. The full list of tasks is
+defined by the `tasks` object; each task is an object with a single
+property representing the task with the corresponding value an object
+representing the task properties. Each task requires the following
+top-level properties:
+
+* `provisionerId`: String. Name of Taskcluster provisioner
+* `schedulerId`: String. Name of Taskcluster scheduler
+* `deadline`: String. Time until the task expires
+* `image`: String. Name of docker image to use for task
+* `maxRunTime`: Number. Maximum time in seconds for which the task can
+ run.
+* `artifacts`: Object. List of artifacts and directories to upload; see
+ Taskcluster documentation.
+* `command`: String. Command to run. This is automatically wrapped in a
+ run_tc command
+* `options`: Optional Object. Options to pass into run_tc
+ - xvfb: Boolean. Enable Xvfb for run
+ - oom-killer: Boolean. Enable xvfb for run
+ - hosts: Boolean. Update hosts file with wpt hosts before run
+ - install-certificates: Boolean. Install wpt certs into OS
+ certificate store for run
+ - browser: List. List of browser names for run
+ - channel: String. Browser channel for run
+* `trigger`: Object. Conditions on which to consider task. One or more
+ of following properties:
+ - branch: List. List of branch names on which to trigger.
+ - pull-request: No value. Trigger for pull request actions
+* `schedule-if`: Optional Object. Conditions on which task should be
+ scheduled given it meets the trigger conditions.
+ - `run-job`: List. Job names for which this task should be considered,
+ matching the output from `./wpt test-jobs`
+* `env`: Optional Object. Environment variables to set when running task.
+* `depends-on`: Optional list. List of task names that must be complete
+ before the current task is scheduled.
+* `description`: String. Task description.
+* `name`: Optional String. Name to use for the task overriding the
+ property name. This is useful in combination with substitutions
+ described below.
+* `download-artifacts`: Optional Object. An artifact to download from
+ a task that this task depends on. This has the following properties:
+ - `task` - Name of the task producing the artifact
+ - `glob` - A glob pattern for the filename of the artifact
+ - `dest` - A directory reltive to the home directory in which to place
+ the artifact
+ - `extract` - Optional. A boolean indicating whether an archive artifact
+ should be extracted in-place.
+
+## Task Expansions
+
+Using the above syntax it's possble to describe each task
+directly. But typically in a taskgraph there are many common
+properties between tasks so it's tedious and error prone to repeat
+information that's common to multiple tasks. Therefore the taskgraph
+format provides several mechanisms to reuse partial task definitions
+across multiple tasks.
+
+### Components
+
+The other top-level property in the taskgraph format is
+`components`. The value of this property is an object containing named
+partial task definitions. Each task definition may contain a property called
+`use` which is a list of components to use as the basis for the task
+definition. The components list is evaluated in order. If a property
+is not previously defined in the output it is added to the output. If
+it was previously defined, the value is updated according to the type:
+ * Strings and numbers are replaced with a new value
+ * Lists are extended with the additional values
+ * Objects are updated recursively following the above rules
+This means that types must always match between components and the
+final value.
+
+For example
+```
+components:
+ example-1:
+ list_prop:
+ - first
+ - second
+ object_prop:
+ key1: value1
+ key2: base_value
+ example-2:
+ list_prop:
+ - third
+ - fourth
+ object_prop:
+ key3:
+ - value3-1
+
+tasks:
+ - example-task:
+ use:
+ - example-1
+ - example-2
+ object_prop:
+ key2: value2
+ key3:
+ - value3-2
+```
+
+will evaluate to the following task:
+
+```
+example-task:
+ list_prop:
+ - first
+ - second
+ - third
+ - fourth
+ object_prop:
+ key1: value1
+ key2: value2
+ key3:
+ - value3-1
+ - value3-2
+```
+
+Note that components cannot currently define `use` properties of their own.
+
+## Substitutions
+
+Components and tasks can define a property `vars` that holds variables
+which are later substituted into the task definition using the syntax
+`${vars.property-name}`. For example:
+
+```
+components:
+ generic-component:
+ prop: ${vars.value}
+
+tasks:
+ - first:
+ use:
+ - generic-component
+ vars:
+ value: value1
+ - second:
+ use:
+ - generic-component
+ vars:
+ value: value2
+```
+
+Results in the following tasks:
+
+```
+first:
+ prop: value1
+second:
+ prop: value2
+```
+
+## Maps
+
+Instead of defining a task directly, an item in the tasks property may
+be an object with a single property `$map`. This object itself has two
+child properties; `for` and `do`. The value of `for` is a list of
+objects, and the value of `do` is either an object or a list of
+objects. For each object in the `for` property, a set of tasks is
+created by taking a copy of that object for each task in the `do`
+property, updating the object with the properties from the
+corresponding `do` object, using the same rules as for components
+above, and then processing as for a normal task. `$map` rules can also
+be nested.
+
+Note: Although `$map` shares a name with the `$map` used in json-e
+(used. in `.taskcluster.yml`), the semantics are different.
+
+For example
+
+```
+components: {}
+tasks:
+ $map:
+ for:
+ - vars:
+ example: value1
+ - vars:
+ example: value2
+ do:
+ example-${vars.example}
+ prop: ${vars.example}
+```
+
+Results in the tasks
+
+```
+example-value1:
+ prop: value1
+example-value2:
+ prop: value2
+```
+
+Note that in combination with `$map`, variable substitutions are
+applied *twice*; once after the `$map` is evaluated and once after the
+`use` statements are evaluated.
+
+## Chunks
+
+A common requirements for tasks is that they are "chunked" into N
+partial tasks. This is handled specially in the syntax. A top level
+property `chunks` can be used to define the number of individual
+chunks to create for a specific task. Each chunked task is created
+with a `chunks` property set to an object containing an `id` property
+containing the one-based index of the chunk an a `total` property
+containing the total number of chunks. These can be substituted into
+the task definition using the same syntax as for `vars` above
+e.g. `${chunks.id}`. Note that because task names must be unique, it's
+common to specify a `name` property on the task that will override the
+property name e.g.
+
+```
+components: {}
+tasks:
+ - chunked-task:
+ chunks:2
+ command: "task-run --chunk=${chunks.id} --totalChunks=${chunks.total}"
+ name: task-chunk-${chunks.id}
+```
+
+creates tasks:
+
+```
+task-chunk-1:
+ command: "task-run --chunk=1 --totalChunks=2"
+task-chunk-2:
+ command: "task-run --chunk=2 --totalChunks=2"
+```
+
+# Overall processing model
+
+The overall processing model for tasks is as follows:
+ * Evaluate maps
+ * Perform subsitutions
+ * Evaluate use statements
+ * Expand chunks
+ * Perform subsitutions
+
+At each point after maps are evaluated tasks must have a unique name.