diff options
Diffstat (limited to 'comm/taskcluster/docs')
| -rw-r--r-- | comm/taskcluster/docs/attributes.rst | 5 | ||||
| -rw-r--r-- | comm/taskcluster/docs/cron.rst | 56 | ||||
| -rw-r--r-- | comm/taskcluster/docs/index.rst | 22 | ||||
| -rw-r--r-- | comm/taskcluster/docs/kinds.rst | 65 | ||||
| -rw-r--r-- | comm/taskcluster/docs/loading.rst | 34 | ||||
| -rw-r--r-- | comm/taskcluster/docs/parameters.rst | 38 | ||||
| -rw-r--r-- | comm/taskcluster/docs/transforms/index.rst | 23 | ||||
| -rw-r--r-- | comm/taskcluster/docs/transforms/job.rst | 10 |
8 files changed, 253 insertions, 0 deletions
diff --git a/comm/taskcluster/docs/attributes.rst b/comm/taskcluster/docs/attributes.rst new file mode 100644 index 0000000000..cbe2dd556d --- /dev/null +++ b/comm/taskcluster/docs/attributes.rst @@ -0,0 +1,5 @@ +=============== +Task Attributes +=============== + +Additional task attributes for Thunderbird's taskgraph. diff --git a/comm/taskcluster/docs/cron.rst b/comm/taskcluster/docs/cron.rst new file mode 100644 index 0000000000..cd8ee94f08 --- /dev/null +++ b/comm/taskcluster/docs/cron.rst @@ -0,0 +1,56 @@ +Periodic Taskgraphs +=================== + +The cron functionality allows in-tree scheduling of task graphs that run +periodically, instead of on a push. + + +Cron.yml +-------- + +In the root of the Comm directory, you will find `.cron.yml`. This defines +the periodic tasks ("cron jobs") that run for Thunderbird. + +See `the Firefox CI cron documentation <https://firefox-source-docs.mozilla.org/taskcluster/cron.html>`_ +for a description of `.cron.yml`. + + +Disabling Cron Jobs +------------------- + +Sometimes due to build bustage, it's desirable to disable the automatic +Thunderbird Daily builds. The best way to do this is to change `.cron.yml` +so that the jobs are never scheduled. This is done by changing `when` to an +empty list. + +The Daily build is started by the `nightly-desktop` job. Additionally, there +is a `searchfox-index` job that should be disabled as well when disabling +Dailies. + +For both of these, comment out the `when` line that sets a time and uncomment +the next line which sets an empty list. + +.. code-block:: yaml + :linenos: + :emphasize-lines: 8,9 + + - name: nightly-desktop + job: + type: decision-task + treeherder-symbol: Nd + target-tasks-method: nightly_desktop + run-on-projects: + - comm-central + when: [{hour: 11, minute: 0}] + # when: [] + + +L10n-bump Cron Job +------------------ + +`l10n-bump` runs daily on comm-beta. For RC week (week 4) it needs to be +disabled by pushing a change directly to `comm-beta` setting ``when`` to +an empty list as above. + +**Do not make the change on comm-central first.** `l10n-bump` will be +re-enabled by merge day activities automatically. diff --git a/comm/taskcluster/docs/index.rst b/comm/taskcluster/docs/index.rst new file mode 100644 index 0000000000..3229405387 --- /dev/null +++ b/comm/taskcluster/docs/index.rst @@ -0,0 +1,22 @@ +.. taskcluster_index: + +TaskCluster Task-Graph Generation +================================= + +The ``comm/taskcluster`` directory contains support for defining the graph of tasks +that must be executed to build and test Thunderbird. + +As Thunderbird is built on top of Firefox's source, the +`Firefox Taskgraph documentation <https://firefox-source-docs.mozilla.org/taskcluster/index.html>`_ +is an invaluable resource. + +The documentation here describes Thunderbird specifics. + +.. toctree:: + + kinds + attributes + parameters + loading + transforms + cron diff --git a/comm/taskcluster/docs/kinds.rst b/comm/taskcluster/docs/kinds.rst new file mode 100644 index 0000000000..85618c43da --- /dev/null +++ b/comm/taskcluster/docs/kinds.rst @@ -0,0 +1,65 @@ +Task Kinds +========== + +This section lists and documents the additional task kinds that are specific +to Thunderbird and are implemented in it's source tree. + +shippable-l10n-pre +------------------ +Prepares a build artifact containing the translated strings from all locales. +The artifact is consumed by `shippable-l10n` to produce the localized +Thunderbird builds. + +Using +..... + +- kind-dependencies: + Must include `build` +- transforms: + Must include `comm_taskgraph.transforms.l10n_pre:transforms` +- only-for-attributes: + Must include `shippable` +- only-for-build-platforms: + This is set to `linux64-shippable/opt` so that it only runs for that + platform. All platforms will consume the build artifact from + `linux64-shippable/opt`. (It's just string data; nothing platform-specific + in there.) + +Parameters +.......... + +There are some task parameters specific to this job kind. + +- locale-list: + Points to either `shipped-locales` or `all-locales`. This file is used to + select the locales that are included in the build artifact. +- comm-locales-file: + This file contains the revision of the `comm-l10n` monorepo to checkout. +- browser-locales-file: + This file contains the revisions of the `l10n-central` repositories to checkout. + This is for toolkit and other strings used from mozilla-central. + +Other notes +........... + +The mozharness script reads its configuration from `thunderbird_split_l10n.py`. +In that file, `hg_l10n_base` refers to the `l10n-central` repository root. +This is used with `browser-locales-file` to get the strings from toolkit and +devtools that are needed. + +Also in the mozharness config file is `hg_comm_l10n_repo`, set to the URL of +the `comm-l10n` monorepo. + +The mozharness script will clone the necessary repositories from `l10n-central`, +and `comm-l10n`, merge them, and create a tar file. + + +shippable-l10n-pre-signing +-------------------------- + +Signing job for shippable-l10n-pre artifacts + +beetmover-strings-source +------------------------ + +Upload strings source files to FTP. diff --git a/comm/taskcluster/docs/loading.rst b/comm/taskcluster/docs/loading.rst new file mode 100644 index 0000000000..3c9c5b1468 --- /dev/null +++ b/comm/taskcluster/docs/loading.rst @@ -0,0 +1,34 @@ +Loading +======= + +Overview +-------- + +See the Firefox loading documentation. In addition to those loaders, in +``comm_taskgraph.loader`` there are loaders for Thunderbird. + + +comm_taskgraph.loader +--------------------- + +reference +......... + +Loads selected jobs from a different taskgraph hierarchy. + +The ``reference`` loader is used to import kinds from the Firefox Taskcluster +base path. + +``packages``, and ``fetch`` use this loader. + +merge +..... + +Loads jobs for a kind from two Taskcluster base paths. The results are "merged" +together into a single kind, allowing for using the Firefox defined jobs as +a base and adding additional jobs. + +First jobs are imported from Firefox's Taskcluster base path using the reference +loader. Then jobs are read from the kind directory using ``jobs-from``. + +``docker-image`` and ``toolchain`` use this loader. diff --git a/comm/taskcluster/docs/parameters.rst b/comm/taskcluster/docs/parameters.rst new file mode 100644 index 0000000000..fa5ac309aa --- /dev/null +++ b/comm/taskcluster/docs/parameters.rst @@ -0,0 +1,38 @@ +========== +Parameters +========== + +Overview +-------- + +See the gecko_taskgraph parameters documentation. + +Comm Push Information +--------------------- + +These parameters correspond to the repository and revision of the comm-central +repository to checkout. All parameters are required. + +``comm_base_repository`` + The repository from which to do an initial clone, utilizing any available + caching. In practice this is always set to ``https://hg.mozilla.org/comm-central``. + +``comm_base_rev`` + The previous revision before ``comm_head_rev`` got merged into. + +``comm_base_ref`` + Reference where ``comm_head_rev`` got merged into. It is usually a branch or a tag. + +``comm_head_repository`` + The repository containing the changeset to be built. This may differ from + ``comm_base_repository``. + +``comm_head_rev`` + The revision to check out; this can be a short revision string. + +``comm_head_ref`` + This is the same as ``head_rev``. + +``comm_src_path`` + This will effectively always be "comm/". It's used in `comm_taskgraph.files_changed.get_files_changed_extended` + to handle multiple VCS repositories without hardcoding parameters. diff --git a/comm/taskcluster/docs/transforms/index.rst b/comm/taskcluster/docs/transforms/index.rst new file mode 100644 index 0000000000..877c04d2e2 --- /dev/null +++ b/comm/taskcluster/docs/transforms/index.rst @@ -0,0 +1,23 @@ +Transforms +========== + +Overview +-------- + +Most transforms come from Firefox, and it's assumed that the reader is +familiar with those already. + +The transforms here are mostly used to work around assumptions made in +Firefox transforms. The general idea is to keep Thunderbird-specific exceptions +to those assumptions in this tree. + +Loaders +------- + +See :doc:`loading`. + + +Transforms +---------- + +TODO: Document the transforms found in comm_taskgraph.transforms diff --git a/comm/taskcluster/docs/transforms/job.rst b/comm/taskcluster/docs/transforms/job.rst new file mode 100644 index 0000000000..b626d20e78 --- /dev/null +++ b/comm/taskcluster/docs/transforms/job.rst @@ -0,0 +1,10 @@ +Job Transforms +============== + +Run-Using +--------- + +In order to build toolchains specific to Thunderbird, there is a ``run-using`` +implementation in `comm_taskgraph.transforms.job.toolchain`: + +* ``comm-toolchain-script`` |