diff options
Diffstat (limited to '')
| -rw-r--r-- | taskcluster/docs/kinds.rst | 796 | ||||
| -rw-r--r-- | taskcluster/docs/kinds/test.rst | 161 |
2 files changed, 957 insertions, 0 deletions
diff --git a/taskcluster/docs/kinds.rst b/taskcluster/docs/kinds.rst new file mode 100644 index 0000000000..b88ff40d75 --- /dev/null +++ b/taskcluster/docs/kinds.rst @@ -0,0 +1,796 @@ +Task Kinds +========== + +This section lists and documents the available task kinds. + +build +----- + +Builds are tasks that produce an installer or other output that can be run by +users or automated tests. This is more restrictive than most definitions of +"build" in a Mozilla context: it does not include tasks that run build-like +actions for static analysis or to produce instrumented artifacts. + +build-fat-aar +------------- + +Build architecture-independent GeckoView AAR (Android ARchive) files. This build-like tasks is an +artifact build (ARMv7, but this is arbitrary) that itself depends on arch-specific Android build +jobs. It fetches arch-specific AAR files, extracts arch-specific libraries and preference files, +and then assembles a multi-architecture "fat AAR". Downstream consumers are expected to use +per-ABI feature splits to produce arch-specific APKs. + +If you want to run this task locally, you need to specify these environment variable: + - MOZ_ANDROID_FAT_AAR_ARCHITECTURES: must be a comma-separated list of architecture. + Eg: "armeabi-v7a,arm64-v8a,x86,x86_64". + - each of MOZ_ANDROID_FAT_AAR_ARM64_V8A, MOZ_ANDROID_FAT_AAR_ARMEABI_V7A, + MOZ_ANDROID_FAT_AAR_X86, MOZ_ANDROID_FAT_AAR_X86_64 must be a path relative to + MOZ_FETCHES_DIR. + +build-signing +------------- + +Many builds must be signed. The build-signing task takes the unsigned `build` +kind artifacts and passes them through signingscriptworker to a signing server +and returns signed results. + +build-mac-signing +----------------- + +Mac signing without notarization. + +Uses a self-signed certificate on level 1 environments. + +Shippable downstream tasks should use artifacts from build-mac-notarization. + +build-mac-notarization +---------------------- + +Mac notarization on signingscript (linux) using rcodesign. + +Only available in production environments, as Apple doesn't offer a test +endpoint for notarizing apps. + +Downstream tasks switch to build-mac-signing in non-shippable builds or level 1 +environments. + +artifact-build +-------------- + +This kind performs an artifact build: one based on precompiled binaries +discovered via the TaskCluster index. This task verifies that such builds +continue to work correctly. + +bootstrap +--------- + +This kind performs a standalone bootstrap of a Firefox build from various system environments. + +hazard +------ + +Hazard builds are similar to "regular' builds, but use a compiler extension to +extract a bunch of data from the build and then analyze that data looking for +hazardous behaviors. + +l10n +---- + +The l10n kind takes the last published nightly build, and generates localized builds +from it. You can read more about how to trigger these on the `wiki +<https://wiki.mozilla.org/ReleaseEngineering/TryServer#Desktop_l10n_jobs_.28on_Taskcluster.29>`_. + +shippable-l10n +-------------- + +The nightly l10n kind repacks a specific nightly build (from the same source code) +in order to provide localized versions of the same source. + +shippable-l10n-signing +---------------------- + +The shippable l10n signing kind takes artifacts from the shippable-l10n kind and +passes them to signing servers to have their contents signed appropriately, based +on an appropriate signing format. One signing job is created for each shippable-l10n +job (usually chunked). + +shippable-l10n-mac-signing +-------------------------- + +Mac signing without notarization. + +Uses a self-signed certificate on level 1 environments. + +Shippable downstream tasks should use artifacts from build-mac-notarization. + +shippable-l10n-mac-notarization +------------------------------- + +Mac notarization on signingscript (linux) using rcodesign. + +Only available in production environments, as Apple doesn't offer a test +endpoint for notarizing apps. + +Downstream tasks switch to build-mac-signing in non-shippable builds or level 1 +environments. + +source-test +----------- + +Source-tests are tasks that run directly from the Gecko source. This can include linting, +unit tests, source-code analysis, or measurement work. While source-test tasks run from +a source checkout, it is still possible for them to depend on a build artifact, though +often they do not. + +code-review +----------- + +Publish issues found by source-test tasks on Phabricator. +This is a part of Release Management code review Bot. + +upload-symbols +-------------- + +Upload-symbols tasks run after builds and upload the symbols files generated by +build tasks to Socorro for later use in crash analysis. + +upload-symbols-dummy +-------------------- + +Upload-symbols-dummy ensures both x64 and macosx64 tasks run for nightlies and releases. + +upload-generated-sources +------------------------ + +Upload-generated-sources tasks run after builds and upload source files that were generated as part of the build process to an s3 bucket for later use in links from crash reports or when debugging shipped builds. + +upload-generated-sources-dummy +------------------------------ + +Upload-generated-sources-dummy ensures both x64 and macosx64 tasks run for nightlies and releases. + +valgrind +-------- + +Valgrind tasks produce builds instrumented by valgrind. + +searchfox +--------- + +Searchfox builds generate C++ index data for Searchfox. + +static-analysis-autotest +------------------------ + +Static analysis autotest utility in order to be sure that there is no regression +when upgrading utilities that impact static-analysis. + +toolchain +--------- + +Toolchain builds create the compiler toolchains used to build Firefox. These +will eventually be dependencies of the builds themselves, but for the moment +are run manually via try pushes and the results uploaded to tooltool. + +spidermonkey +------------ + +Spidermonkey tasks check out the full gecko source tree, then compile only the +spidermonkey portion. Each task runs specific tests after the build. + +test +---- + +See the :doc:`test kind documentation <kinds/test>` for more info. + +.. toctree:: + :hidden: + + kinds/test + + +docker-image +------------ + +Tasks of the ``docker-image`` kind build the Docker images in which other +Docker tasks run. + +The tasks to generate each docker image have predictable labels: +``docker-image-<name>``. + +Docker images are built from subdirectories of ``taskcluster/docker``, using +``docker build``. There is currently no capability for one Docker image to +depend on another in-tree docker image, without uploading the latter to a +Docker repository. + +balrog +------ + +Balrog tasks are responsible for submitting metadata to our update server (Balrog). +They are typically downstream of a beetmover job that moves signed MARs somewhere +(eg: beetmover and beetmover-l10n for releases, beetmover-repackage for nightlies). + +beetmover +--------- + +Beetmover, takes specific artifacts, "Beets", and pushes them to a location outside +of Taskcluster's task artifacts, (archive.mozilla.org as one place) and in the +process determines the final location and a "pretty" name (versioned product name) + +beetmover-l10n +-------------- + +Beetmover L10n, takes specific artifacts, "Beets", and pushes them to a location outside +of Taskcluster's task artifacts, (archive.mozilla.org as one place) and in the +process determines the final location and a "pretty" name (versioned product name) +This separate kind uses logic specific to localized artifacts, such as including +the language in the final artifact names. + +beetmover-repackage +------------------- + +Beetmover-repackage is beetmover but for tasks that need an intermediate step +between signing and packaging, such as OSX. For more details see the definitions +of the Beetmover kind above and the repackage kind below. + +release-beetmover-push-to-release +--------------------------------- + +release-beetmover-push-to-release publishes promoted releases from the +candidates directory to the release directory. This is part of release +promotion. + +beetmover-snap +-------------- +Beetmover-source publishes Ubuntu's snap. This is part of release promotion. + +beetmover-source +---------------- +Beetmover-source publishes release source. This is part of release promotion. + +beetmover-geckoview +------------------- +Beetmover-geckoview publishes the Android library called "geckoview". + +beetmover-apt +------------------- +Beetmover-apt publishes Linux .deb packages from the Mozilla archive to our APT repositories. + +condprof +-------- +condprof creates and updates realistic profiles. + +release-source-checksums-signing +-------------------------------- +release-source-checksums-signing take as input the checksums file generated by +source-related beetmover task and sign it via the signing scriptworkers. +Returns the same file signed and additional detached signature. + +beetmover-checksums +------------------- +Beetmover, takes specific artifact checksums and pushes it to a location outside +of Taskcluster's task artifacts (archive.mozilla.org as one place) and in the +process determines the final location and "pretty" names it (version product name) + +release-beetmover-source-checksums +---------------------------------- +Beetmover, takes source specific artifact checksums and pushes it to a location outside +of Taskcluster's task artifacts (archive.mozilla.org as one place) and in the +process determines the final location and "pretty" names it (version product name) + +perftest +-------- +Runs performance tests using mozperftest. + +release-balrog-submit-toplevel +------------------------------ +Toplevel tasks are responsible for submitting metadata to Balrog that is not specific to any +particular platform+locale. For example: fileUrl templates, versions, and platform aliases. + +Toplevel tasks are also responsible for updating test channel rules to point at the Release +being generated. + +release-secondary-balrog-submit-toplevel +---------------------------------------- +Performs the same function as `release-balrog-submit-toplevel`, but against the beta channel +during RC builds. + +release-balrog-scheduling +------------------------- +Schedules a Release for shipping in Balrog. If a `release_eta` was provided when starting the Release, +it will be scheduled to go live at that day and time. + +release-secondary-balrog-scheduling +----------------------------------- +Performs the same function as `release-balrog-scheduling`, except for the beta channel as part of RC +Releases. + +release-binary-transparency +--------------------------- +Binary transparency creates a publicly verifiable log of binary shas for downstream +release auditing. https://wiki.mozilla.org/Security/Binary_Transparency + +release-snap-repackage +---------------------- +Generate an installer using Ubuntu's Snap format. + +release-flatpak-repackage +------------------------- +Generate an installer using Flathub's Flatpak format. + +release-flatpak-push +-------------------- +Pushes Flatpak repackage on Flathub + +release-secondary-flatpak-push +------------------------------ +Performs the same function as `release-flatpak-push`, except for the beta channel as part of RC +Releases. + +release-notify-av-announce +-------------------------- +Notify anti-virus vendors when a release is likely shipping. + +release-notify-push +------------------- +Notify when a release has been pushed to CDNs. + +release-notify-ship +------------------- +Notify when a release has been shipped. + +release-secondary-notify-ship +----------------------------- +Notify when an RC release has been shipped to the beta channel. + +release-notify-promote +---------------------- +Notify when a release has been promoted. + +release-notify-started +---------------------- +Notify when a release has been started. + +release-bouncer-sub +------------------- +Submits bouncer information for releases. + +release-mark-as-shipped +----------------------- +Marks releases as shipped in Ship-It v1 + +release-bouncer-aliases +----------------------- +Update Bouncer's (download.mozilla.org) "latest" aliases. + +cron-bouncer-check +------------------ +Checks Bouncer (download.mozilla.org) uptake. + +bouncer-locations +----------------- +Updates nightly bouncer locations for version bump. + +release-bouncer-check +--------------------- +Checks Bouncer (download.mozilla.org) uptake as part of the release tasks. + +release-generate-checksums +-------------------------- +Generate the per-release checksums along with the summaries + +release-generate-checksums-signing +---------------------------------- +Sign the pre-release checksums produced by the above task + +release-generate-checksums-beetmover +------------------------------------ +Submit to S3 the artifacts produced by the release-checksums task and its signing counterpart. + +release-final-verify +-------------------- +Verifies the contents and package of release update MARs. + +release-secondary-final-verify +------------------------------ +Verifies the contents and package of release update MARs for RC releases. + +release-push-langpacks +------------------------------- +Publishes language packs onto addons.mozilla.org. + +release-beetmover-signed-langpacks +---------------------------------- +Publishes signed langpacks to archive.mozilla.org + +release-beetmover-signed-langpacks-checksums +-------------------------------------------- +Publishes signed langpacks to archive.mozilla.org + +release-update-verify +--------------------- +Verifies the contents and package of release update MARs. +release-secondary-update-verify +------------------------------- +Verifies the contents and package of release update MARs. + +release-update-verify-next +-------------------------- +Verifies the contents and package of release and updare MARs from the previous ESR release. + +release-update-verify-config +---------------------------- +Creates configs for release-update-verify tasks + +release-secondary-update-verify-config +-------------------------------------- +Creates configs for release-secondary-update-verify tasks + +release-update-verify-config-next +--------------------------------- +Creates configs for release-update-verify-next tasks + +release-updates-builder +----------------------- +Top level Balrog blob submission & patcher/update verify config updates. + +release-version-bump +-------------------- +Bumps to the next version. + +release-source +-------------- +Generates source for the release + +release-source-signing +---------------------- +Signs source for the release + +release-partner-repack +---------------------- +Generates customized versions of releases for partners. + +release-partner-attribution +--------------------------- +Generates attributed versions of releases for partners. + +release-partner-repack-chunking-dummy +------------------------------------- +Chunks the partner repacks by locale. + +release-partner-repack-signing +------------------------------ +Internal signing of partner repacks. + +release-partner-repack-mac-signing +---------------------------------- + +Mac signing without notarization. + +Uses a self-signed certificate on level 1 environments. + +Shippable downstream tasks should use artifacts from build-mac-notarization. + +release-partner-repack-mac-notarization +--------------------------------------- + +Mac notarization on signingscript (linux) using rcodesign. + +Only available in production environments, as Apple doesn't offer a test +endpoint for notarizing apps. + +Downstream tasks switch to build-mac-signing in non-shippable builds or level 1 +environments. + +release-partner-repack-repackage +-------------------------------- +Repackaging of partner repacks. + +release-partner-repack-repackage-signing +---------------------------------------- +External signing of partner repacks. + +release-partner-repack-beetmover +-------------------------------- +Moves the partner repacks to S3 buckets. + +release-partner-attribution-beetmover +------------------------------------- +Moves the partner attributions to S3 buckets. + +release-partner-repack-bouncer-sub +---------------------------------- +Sets up bouncer products for partners. + +release-early-tagging +--------------------- +Utilises treescript to perform tagging that should happen near the start of a release. + +release-eme-free-repack +----------------------- +Generates customized versions of releases for eme-free repacks. + +release-eme-free-repack-signing +------------------------------- +Internal signing of eme-free repacks + +release-eme-free-repack-repackage +--------------------------------- +Repackaging of eme-free repacks. + +release-eme-free-repack-repackage-signing +----------------------------------------- +External signing of eme-free repacks. + +release-eme-free-repack-beetmover +--------------------------------- +Moves the eme-free repacks to S3 buckets. + +release-eme-free-repack-beetmover-checksums +------------------------------------------- +Moves the beetmover checksum for eme-free repacks to S3 buckets. + +release-eme-free-repack-mac-signing +----------------------------------- + +Mac signing without notarization. + +Uses a self-signed certificate on level 1 environments. + +Shippable downstream tasks should use artifacts from build-mac-notarization. + +release-eme-free-repack-mac-notarization +---------------------------------------- + +Mac notarization on signingscript (linux) using rcodesign. + +Only available in production environments, as Apple doesn't offer a test +endpoint for notarizing apps. + +Downstream tasks switch to build-mac-signing in non-shippable builds or level 1 +environments. + +repackage +--------- +Repackage tasks take a signed output and package them up into something suitable +for shipping to our users. For example, on OSX we return a tarball as the signed output +and this task would package that up as an Apple Disk Image (.dmg) + +repackage-l10n +-------------- +Repackage-L10n is a ```Repackage``` task split up to be suitable for use after l10n repacks. + +repackage-deb +---------------- +These repackage tasks take signed Firefox Linux binaries and puts them in Debian packages. + +repackage-deb-l10n +------------------ +These repackage tasks take the signed langpacks (.xpi) binaries and puts them in Debian packages. + +repackage-signing +----------------- +Repackage-signing take the repackaged installers (windows) and signs them. + +repackage-signing-l10n +---------------------- +Repackage-signing-l10n take the repackaged installers (windows) and signs them for localized versions. + +mar-signing +----------- +Mar-signing takes the complete update MARs and signs them. + +mar-signing-l10n +---------------- +Mar-signing-l10n takes the complete update MARs and signs them for localized versions. + +mar-signing-autograph-stage +--------------------------- +These tasks are only to test autograph-stage, when the autograph team asks for their staging environment to be tested. + +repackage-msi +------------- +Repackage-msi takes the signed full installer and produces an msi installer (that wraps the full installer) +Using the ```./mach repackage``` command + +repackage-signing-msi +--------------------- +Repackage-signing-msi takes the repackaged msi installers and signs them. + +repackage-msix +-------------- +Repackage-msix takes a (possibly unsigned) package and produces a Windows MSIX package containing no langpacks using the +```./mach repackage``` command. + +These tasks are supposed intended for rapid iteration in ```try```. + +repackage-shippable-l10n-msix +----------------------------- +Repackage-msix takes a signed package and a list of signed langpacks and produces a Windows MSIX package using the +```./mach repackage``` command. + +The signed langpacks are produced on Linux, since langpacks are platform agnostic. + +These tasks are for releases; they are complete, and therefore slower, and not intended for rapid iteration in +```try```. + +repackage-signing-msix +---------------------- +Repackage-signing-msix takes Windows MSIX packages produced in ```repackage-msix``` and signs them. + +repackage-signing-shippable-l10n-msix +------------------------------------- +Repackage-signing-shippable-l10n-msix takes Windows MSIX packages produced in +```repackage-signing-shippable-l10n-msix``` and signs them. + +release-msix-push +-------------------- +Pushes msix repackage to the Microsoft Store. + +repo-update +----------- +Repo-Update tasks are tasks that perform some action on the project repo itself, +in order to update its state in some way. + +partials +-------- +Partials takes the complete.mar files produced in previous tasks and generates partial +updates between previous nightly releases and the new one. Requires a release_history +in the parameters. See ``mach release-history`` if doing this manually. + +partials-signing +---------------- +Partials-signing takes the partial updates produced in Partials and signs them. + +post-balrog-dummy +----------------- +Dummy tasks to consolidate balrog dependencies to avoid taskcluster limits on number of dependencies per task. + +post-beetmover-dummy +-------------------- +Dummy tasks to consolidate beetmover dependencies to avoid taskcluster limits on number of dependencies per task. + +post-beetmover-checksums-dummy +------------------------------ +Dummy tasks to consolidate beetmover-checksums dependencies to avoid taskcluster limits on number of dependencies per task. + +post-langpack-dummy +------------------- +Dummy tasks to consolidate language pack beetmover dependencies to avoid taskcluster limits on number of dependencies per task. + +post-update-verify-dummy +------------------------ +Dummy tasks to consolidate update verify dependencies to avoid taskcluster limits on number of dependencies per task. + +fetch +----- +Tasks that obtain something from a remote service and re-expose it as a +task artifact. These tasks are used to effectively cache and re-host +remote content so it is reliably and deterministically available. + +packages +-------- +Tasks used to build packages for use in docker images. + +diffoscope +---------- +Tasks used to compare pairs of Firefox builds using https://diffoscope.org/. +As of writing, this is mainly meant to be used in try builds, by editing +taskcluster/ci/diffoscope/kind.yml for your needs. + +addon +----- +Tasks used to build/package add-ons. + +openh264-plugin +--------------- +Tasks used to build the openh264 plugin. + +openh264-signing +---------------- +Signing for the openh264 plugin. + +webrender +--------- +Tasks used to do testing of WebRender standalone (without gecko). The +WebRender code lives in gfx/wr and has its own testing infrastructure. + +github-sync +------------ +Tasks used to do synchronize parts of Gecko that have downstream GitHub +repositories. + +instrumented-build +------------------ +Tasks that generate builds with PGO instrumentation enabled. This is an +intermediate build that can be used to generate profiling information for a +final PGO build. This is the 1st stage of the full 3-step PGO process. + +generate-profile +---------------- +Tasks that take a build configured for PGO and run the binary against a sample +set to generate profile data. This is the 2nd stage of the full 3-step PGO +process. + +geckodriver-signing +------------------- +Signing for geckodriver binary. + +geckodriver-mac-notarization +---------------------------- +Apple notarization for mac geckodriver binary. + +maybe-release +------------- +A shipitscript task that does the following: + +1. Checks if automated releases are disabled +2. Checks if the changes between the current revision and the previous releases + revision are considered "worthwhile" for a new release. +3. Triggers the release via ship-it, which will then create an action task. + +l10n-bump +--------- +Cron-driven tasks that bump l10n-changesets files in-tree, using data from the l10n dashboard. + +merge-automation +---------------- +Hook-driven tasks that automate "Merge Day" tasks during the release cycle. + +sentry +------ +Interact with Sentry, such as by publishing new project releases. + +system-symbols +-------------- +Generate missing macOS and windows system symbols from crash reports. + +system-symbols-upload +--------------------- +Upload macOS and windows system symbols to tecken. + +system-symbols-reprocess +------------------------ +Call Crash-Stats API to reprocess after symbols upload. + +scriptworker-canary +------------------- +Push tasks to try to test new scriptworker deployments. + +updatebot +------------------ +Check for updates to (supported) third party libraries, and manage their lifecycle. + +fuzzing +------- + +Performs fuzzing smoke tests + +startup-test +------------ + +Runs Firefox for a short period of time to see if it crashes + +l10n-cross-channel +------------------ + +Compiles a set of en-US strings from all shipping release trains and pushes to +the quarantine strings repo. + +fxrecord +-------- + +Visual metrics computation of desktop Firefox startup. The performance team +monitors this task to watch for regressions in Firefox startup performance. + +are-we-esmified-yet +--------------------- +Collects data about the transition to ECMAScript Modules from JSMs. + +attribution +----------- +Injects attribution information into en-US installers. + +attribution-l10n +---------------- +Injects attribution information into localized installers. diff --git a/taskcluster/docs/kinds/test.rst b/taskcluster/docs/kinds/test.rst new file mode 100644 index 0000000000..20d3635c5c --- /dev/null +++ b/taskcluster/docs/kinds/test.rst @@ -0,0 +1,161 @@ +Test Kind +========= + +The ``test`` kind defines both desktop and mobile tests for builds. Each YAML +file referenced in ``kind.yml`` defines the full set of tests for the +associated suite. + +The process of generating tests goes like this, based on a set of YAML files +named in ``kind.yml``: + + * For each build task, determine the related test platforms based on the build + platform. For example, a Windows 2010 build might be tested on Windows 7 + and Windows 10. Each test platform specifies "test sets" indicating which + tests to run. This is configured in the file named + ``test-platforms.yml``. + + * Each test set is expanded to a list of tests to run. This is configured in + the file named by ``test-sets.yml``. A platform may specify several test + sets, in which case the union of those sets is used. + + * Each named test is looked up in the file named by ``tests.yml`` to find a + test description. This test description indicates what the test does, how + it is reported to treeherder, and how to perform the test, all in a + platform-independent fashion. + + * Each test description is converted into one or more tasks. This is + performed by a sequence of transforms defined in the ``transforms`` key in + ``kind.yml``. See :ref:`transforms` for more information. + + * The resulting tasks become a part of the task graph. + +.. important:: + + This process generates *all* test jobs, regardless of tree or try syntax. + It is up to a later stages of the task-graph generation (the target set and + optimization) to select the tests that will actually be performed. + +Variants +-------- + +Sometimes we want to run the same tests under a different Firefox context, +usually this means with a pref set. The concept of ``variants`` was invented to +handle this use case. A variant is a stanza of configuration that can be merged +into each test definition. Variants are defined in the `variants.yml`_ file. +See this file for an up to date list of active variants and the pref(s) they +set. + +Each variant must conform to the +:py:data:`~gecko_taskgraph.transforms.test.variant_description_schema`: + +* **description** (required) - A description explaining what the variant is for. +* **component** (required) - The name of the component that owns the variant. It + should be formatted as ``PRODUCT``::``COMPONENT``. +* **expiration** (required) - The date when the variant will be expired (maximum + 6 months). +* **suffix** (required) - A suffix to apply to the task label and treeherder symbol. +* **when** - A `json-e`_ expression that must evaluate to ``true`` for the variant + to be applied. The ``task`` definition is passed in as context. +* **replace** - A dictionary that will overwrite keys in the task definition. +* **merge** - A dictionary that will be merged into the task definition using + the :py:func:`~gecko_taskgraph.util.templates.merge` function. + +.. note:: + + Exceptions can be requested to have a variant without expiration (using + "never") if this is a shipped mode we support. Teams should contact the CI + team to discuss this before submitting a patch if they think their variant + qualifies. All exceptions will require director approval. + + +Defining Variants +~~~~~~~~~~~~~~~~~ + +Variants can be defined in the test YAML files using the ``variants`` key. E.g: + +.. code-block:: yaml + + example-suite: + variants: + - foo + - bar + +This will split the task into three. The original task, the task with the +config from the variant named 'foo' merged in and the task with the config from +the variant named 'bar' merged in. + + +Composite Variants +~~~~~~~~~~~~~~~~~~ + +Sometimes we want to run tasks with multiple variants enabled at once. This can +be achieved with "composite variants". Composite variants are simply two or +more variant names joined with the ``+`` sign. Using the previous example, if +we wanted to run both the ``foo`` and ``bar`` variants together, we could do: + +.. code-block:: yaml + + example-suite: + variants: + - foo+bar + +This will first merge or replace the config of ``foo`` into the task, followed +by the config of ``bar``. Care should be taken if both variants are replacing +the same keys. The last variant's configuration will be the one that gets used. + + +Expired Variants +~~~~~~~~~~~~~~~~ +Ideally, when a variant is not needed anymore, it should be dropped (even if it +has not expired). If you need to extend the expiration date, you can submit a +patch to modify the expiration date in the `variants.yml`_ file. Variants will +not be scheduled to run after the expiration date. + +If an expired variant is not dropped, the triage owner of the component will be +notified. If the expired variant persists for an extended period, the autonag +bot will escalate to notify the manager and director of the triage owner. Once +at that point, we will submit a patch to remove the variant from Taskcluster and +manifest conditions pending the triage owner / manager to review. + +Please subscribe to alerts from `firefox-ci <https://groups.google.com/a/mozilla.com/g/firefox-ci>` +group in order to be aware of changes to the CI, scheduling, or the policy. + +.. _variants.yml: https://searchfox.org/mozilla-central/source/taskcluster/ci/test/variants.yml +.. _json-e: https://json-e.js.org/ + + +Setting +------- + +A test ``setting`` is the set of conditions under which a test is running. +Aside from the chunk number, a ``setting`` uniquely distinguishes a task from +another that is running the same set of tests. There are three types of inputs +that make up a ``setting``: + +1. Platform - Bits of information that describe the underlying platform the + test is running on. This includes things like the operating system and + version, CPU architecture, etc. + +2. Build - Bits of information that describe the build being tested. This + includes things like the build type and which build attributes (like + ``asan``, ``ccov``, etc) are enabled. + +3. Runtime - Bits of information that describe the configured state of Firefox. + This includes things like prefs and environment variables. Note that tasks + should only set runtime configuration via the variants system (see + `Variants`_). + +Test ``settings`` are available in the ``task.extra.test-setting`` object in +all test tasks. They are defined by the +:py:func:`~gecko_taskgraph.transforms.test.set_test_setting` transform +function. + +The full schema is defined in the +:py:data:`~gecko_taskgraph.transforms.test.test_setting_description_schema`. + +Setting Hash +~~~~~~~~~~~~ + +In addition to the three top-level objects, there is also a ``_hash`` key which +contains a hash of the rest of the setting object. This is a convenient way for +consumers to group or compare tasks that run under the same setting. |