Adding upstream version 124.0.1.upstream/124.0.1

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

17 files changed, 1007 insertions, 0 deletions

diff --git a/tools/tryselect/docs/configuration.rst b/tools/tryselect/docs/configuration.rst
new file mode 100644
index 0000000000..6743d5f385
--- /dev/null
+++ b/tools/tryselect/docs/configuration.rst
@@ -0,0 +1,30 @@
+Configuring Try
+===============
+
+
+Getting Level 1 Commit Access
+-----------------------------
+
+In order to push to try, `Level 1 Commit Access`_ is required. Please see `Becoming a Mozilla
+Committer`_ for more information on how to obtain this.
+
+
+Configuring Version Control
+---------------------------
+
+After you have level 1 access, you'll need to do a little bit of setup before you can push. Both
+``hg`` and ``git`` are supported, move on to the appropriate section below.
+
+
+Configuring Try with Mercurial / Git
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The recommended way to push to try is via the ``mach try`` command. This requires the
+``push-to-try`` extension which can be installed by running:
+
+.. code-block:: shell
+
+    $ mach vcs-setup
+
+.. _Level 1 Commit Access: https://www.mozilla.org/en-US/about/governance/policies/commit/access-policy/
+.. _Becoming a Mozilla Committer: https://www.mozilla.org/en-US/about/governance/policies/commit/
diff --git a/tools/tryselect/docs/img/add-new-jobs.png b/tools/tryselect/docs/img/add-new-jobs.png
new file mode 100644
index 0000000000..513565cb14
--- /dev/null
+++ b/tools/tryselect/docs/img/add-new-jobs.png
diff --git a/tools/tryselect/docs/img/phab-treeherder-link.png b/tools/tryselect/docs/img/phab-treeherder-link.png
new file mode 100644
index 0000000000..52b58b6231
--- /dev/null
+++ b/tools/tryselect/docs/img/phab-treeherder-link.png
diff --git a/tools/tryselect/docs/index.rst b/tools/tryselect/docs/index.rst
new file mode 100644
index 0000000000..8978a72eaf
--- /dev/null
+++ b/tools/tryselect/docs/index.rst
@@ -0,0 +1,92 @@
+Pushing to Try
+==============
+
+"Pushing to Try" allows developers to build and test their changes on Mozilla's automation servers
+without requiring their code to be reviewed and landed.
+
+First, :doc:`ensure that you can push to Try <configuration>`.
+Try knows how to run tasks that are defined in-tree,
+such as ``build-linux64/opt`` (build Firefox for Linux). To manually select some tasks for
+Try to process, run the following command:
+
+.. code-block:: shell
+
+    ./mach try fuzzy
+
+After submitting your requested tasks, you'll be given a link to your "push" in Treeherder.
+It may take a few minutes for your push to appear in Treeherder! Be patient, and it will automatically
+update when Try begins processing your work.
+
+Another very useful Try command is ``./mach try auto``, which will automatically select the tasks
+that are mostly likely to be affected by your changes.
+See the :doc:`selectors page <selectors/index>` to view all the other ways to select which tasks to push.
+
+It is possible to set environment variables, notably :doc:`MOZ_LOG </xpcom/logging>`, when pushing to
+try:
+
+.. code-block:: shell
+
+   ./mach try fuzzy --env MOZ_LOG=cubeb:4
+
+Resolving "<Try build> is damaged and can't be opened" error
+------------------------------------------------------------
+
+To run a try build on macOS, you need to get around Apple's restrictions on downloaded applications.
+
+These restrictions differ based on your hardware: Apple Silicon machines (M1 etc.) are much stricter.
+
+For Apple Silicon machines, you will need to download the target.dmg artifact from the
+"repackage-macosx64-shippable/opt" job.
+This is a universal build (i.e. it contains both x86_64 and arm64 code), and it is signed but not notarized.
+You can trigger this job using ``./mach try fuzzy --full``.
+
+On Intel Macs, you can run unsigned builds, once you get around the quarantining (see below),
+so you can just download the "target.dmg" from a regular opt build.
+
+Regardless of hardware, you need to make sure that there is no quarantining attribute on
+the downloaded dmg file before you attempt to run it:
+Apple automatically quarantines apps that are downloaded with a browser from an untrusted
+location. This "quarantine status" can be cleared by doing ``xattr -c <Try build>`` after
+downloading. You can avoid this "quarantine status" by downloading the build from the command
+line instead, such as by using ``curl``:
+
+.. code-block:: shell
+
+    curl -L <artifact-url> -o <file-name>
+
+.. _attach-job-review:
+
+Adding Try jobs to a Phabricator patch
+--------------------------------------
+
+For every patch submitted for review in Phabricator, a new Try run is automatically created.
+A link called ``Treeherder Jobs`` can be found in the ``Diff Detail`` section of the review in
+Phabricator.
+
+.. image:: img/phab-treeherder-link.png
+
+This run is created for static analysis, linting and other tasks. Attaching new jobs to the run is
+easy and doesn't require more actions from the developer.
+Click on the down-arrow to access the actions menu, select the relevant jobs
+and, click on ``Trigger X new jobs`` (located on the top of the job).
+
+.. image:: img/add-new-jobs.png
+
+Table of Contents
+-----------------
+
+.. toctree::
+  :maxdepth: 2
+
+  configuration
+  selectors/index
+  presets
+  tasks
+
+
+Indices and tables
+------------------
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`
diff --git a/tools/tryselect/docs/presets.rst b/tools/tryselect/docs/presets.rst
new file mode 100644
index 0000000000..a3368cf8b5
--- /dev/null
+++ b/tools/tryselect/docs/presets.rst
@@ -0,0 +1,85 @@
+Presets
+=======
+
+Some selectors, such as ``fuzzy`` and ``syntax``, allow saving and loading presets from a file. This is a
+good way to reuse a selection, either at a later date or by sharing with others. Look for a
+'preset' section in ``mach <selector> --help`` to determine whether the selector supports this
+functionality.
+
+Using Presets
+-------------
+
+To save a preset, run:
+
+.. code-block:: shell
+
+    $ mach try <selector> --save <name> <args>
+
+For example, to save a preset that selects all Windows mochitests:
+
+.. code-block:: shell
+
+    $ mach try fuzzy --save all-windows-mochitests --query "'win 'mochitest"
+    preset saved, run with: --preset=all-windows-mochitests
+
+Then run that saved preset like this:
+
+.. code-block:: shell
+
+    $ mach try --preset all-windows-mochitests
+
+To see a list of all available presets run:
+
+.. code-block:: shell
+
+    $ mach try --list-presets
+
+
+Editing and Sharing Presets
+---------------------------
+
+Presets can be defined in one of two places, in your home directory or in a file checked into
+mozilla-central.
+
+Local Presets
+~~~~~~~~~~~~~
+
+These are defined in your ``$MOZBUILD_STATE_DIR``, typically ``~/.mozbuild/try_presets.yml``.
+Presets defined here are your own personal collection of presets. You can modify them by running:
+
+.. code-block:: shell
+
+    $ ./mach try --edit-presets
+
+
+Shared Presets
+~~~~~~~~~~~~~~
+
+You can also check presets into mozilla-central in `tools/tryselect/try_presets.yml`_. These presets
+will be available to all users of ``mach try``, so please be mindful when editing this file. Make
+sure the name of the preset is scoped appropriately (i.e doesn't contain any team or module specific
+terminology). It is good practice to prefix the preset name with the name of the team or module that
+will get the most use out of it.
+
+
+Preset Format
+~~~~~~~~~~~~~
+
+Presets are simple key/value objects, with the name as the key and a metadata object as the value.
+For example, the preset saved above would look something like this in ``try_presets.yml``:
+
+.. code-block:: yaml
+
+    all-windows-mochitests:
+        selector: fuzzy
+        description: >-
+            Runs all windows mochitests.
+        query:
+            - "'win 'mochitest"
+
+The ``selector`` key (required) allows ``mach try`` to determine which subcommand to dispatch to.
+The ``description`` key (optional in user presets but required for shared presets) is a human
+readable string describing what the preset selects and when to use it. All other values in the
+preset are forwarded to the specified selector as is.
+
+.. _tools/tryselect/try_presets.yml: https://searchfox.org/mozilla-central/source/tools/tryselect/try_presets.yml
diff --git a/tools/tryselect/docs/selectors/again.rst b/tools/tryselect/docs/selectors/again.rst
new file mode 100644
index 0000000000..da592b7a34
--- /dev/null
+++ b/tools/tryselect/docs/selectors/again.rst
@@ -0,0 +1,36 @@
+Again Selector
+==============
+
+When you push to try, the computed ``try_task_config.json`` is saved in a
+history file under ``~/.mozbuild/srcdirs/<srcdir hash>/history`` (note: the
+``syntax`` selector does not use ``try_task_config.json`` yet so does not save
+any history). You can then use the ``again`` selector to re-push any of your
+previously generated task configs.
+
+In the simple case, you can re-run your last try push with:
+
+.. code-block:: shell
+
+    $ mach try again
+
+If you want to re-push a task config a little further down the history stack,
+first you need to figure out its index with:
+
+.. code-block:: shell
+
+    $ mach try again --list
+
+Then run:
+
+.. code-block:: shell
+
+    $ mach try again --index <index>
+
+Note that index ``0`` is always the most recent ``try_task_config.json`` in the
+history. You can clear your history with:
+
+.. code-block:: shell
+
+    $ mach try again --purge
+
+Only the 10 most recent pushes will be saved in your history.
diff --git a/tools/tryselect/docs/selectors/auto.rst b/tools/tryselect/docs/selectors/auto.rst
new file mode 100644
index 0000000000..c5f6d4df5e
--- /dev/null
+++ b/tools/tryselect/docs/selectors/auto.rst
@@ -0,0 +1,24 @@
+Auto Selector
+=============
+
+This selector automatically determines the most efficient set of tests and
+tasks to run against your push. It accomplishes this via a combination of
+machine learning and manual heuristics. The tasks selected here should match
+pretty closely to what would be scheduled if your patch were pushed to
+autoland.
+
+It is the officially recommended selector to use when you are unsure of which
+tasks should run on your push.
+
+To use:
+
+.. code-block:: bash
+
+   $ mach try auto
+
+Unlike other try selectors, tasks are not chosen locally. Rather they will be
+computed by the decision task.
+
+Like most other selectors, ``mach try auto`` supports many of the standard
+templates such as ``--artifact`` or ``--env``. See ``mach try auto --help`` for
+the full list of supported templates.
diff --git a/tools/tryselect/docs/selectors/chooser.rst b/tools/tryselect/docs/selectors/chooser.rst
new file mode 100644
index 0000000000..6c59d54009
--- /dev/null
+++ b/tools/tryselect/docs/selectors/chooser.rst
@@ -0,0 +1,32 @@
+Chooser Selector
+================
+
+When pushing to try, there are a very large amount of builds and tests to choose from. Often too
+many to remember, making it easy to forget a set of tasks which should otherwise have been run.
+
+This selector allows you to select tasks from a web interface that lists all the possible build and
+test tasks and allows you to select them from a list. It is similar in concept to the old `try
+syntax chooser`_ page, except that the values are dynamically generated using the :ref:`taskgraph<TaskCluster Task-Graph Generation>` as an
+input. This ensures that it will never be out of date.
+
+To use:
+
+.. code-block:: shell
+
+    $ mach try chooser
+
+This will spin up a local web server (using Flask) which serves the chooser app. After making your
+selection, simply press ``Push`` and the rest will be handled from there. No need to copy/paste any
+syntax strings or the like.
+
+You can run:
+
+.. code-block:: shell
+
+    $ mach try chooser --full
+
+To generate the interface using the full :ref:`taskgraph<TaskCluster Task-Graph Generation>` instead. This will include tasks that don't run
+on mozilla-central.
+
+
+.. _try syntax chooser: https://mozilla-releng.net/trychooser
diff --git a/tools/tryselect/docs/selectors/compare.rst b/tools/tryselect/docs/selectors/compare.rst
new file mode 100644
index 0000000000..a87b263030
--- /dev/null
+++ b/tools/tryselect/docs/selectors/compare.rst
@@ -0,0 +1,17 @@
+Compare Selector
+================
+
+When this command runs it pushes two identical try jobs to treeherder. The first
+job is on the current commit you are on, and the second one is a commit
+specified in the command line arguments. This selector is aimed at helping
+engineers test performance enhancements or resolve performance regressions.
+
+Currently the only way you can select jobs is through fuzzy but we are
+planning on expanding to other choosing frameworks also.
+
+You pass the commit you want to compare against as a commit hash as either
+``-cc`` or ``--compare-commit``, an example is show below
+
+.. code-block:: shell
+
+   $ mach try compare --compare-commit <commit-hash>
diff --git a/tools/tryselect/docs/selectors/empty.rst b/tools/tryselect/docs/selectors/empty.rst
new file mode 100644
index 0000000000..c3ea61b9ce
--- /dev/null
+++ b/tools/tryselect/docs/selectors/empty.rst
@@ -0,0 +1,21 @@
+Empty Selector
+==============
+
+The ``mach try empty`` subcommand is very simple, it won't schedule any additional tasks. You'll
+still see lint tasks and python-unittest tasks if applicable, this is due to a configuration option
+in taskcluster.
+
+Other than those, your try run will be empty. You can use treeherder's ``Add new jobs`` feature to
+selectively add additional tasks after the fact.
+
+.. note::
+
+    To use ``Add new jobs`` you'll need to be logged in and have commit access level 1, just as if
+    you were pushing to try.
+
+To do this:
+
+    1. Click the drop-down arrow at the top right of your commit.
+    2. Select ``Add new jobs`` (it may take a couple seconds to load).
+    3. Choose your desired tasks by clicking them one at a time.
+    4. At the top of your commit, select ``Trigger New Jobs``.
diff --git a/tools/tryselect/docs/selectors/fuzzy.rst b/tools/tryselect/docs/selectors/fuzzy.rst
new file mode 100644
index 0000000000..d50f801eb0
--- /dev/null
+++ b/tools/tryselect/docs/selectors/fuzzy.rst
@@ -0,0 +1,371 @@
+Fuzzy Selector
+==============
+
+The fuzzy selector uses a tool called `fzf`_. It allows you to filter down all of the task labels
+from a terminal based UI and an intelligent fuzzy finding algorithm. If the ``fzf`` binary is not
+installed, you'll be prompted to bootstrap it on first run.
+
+
+Understanding the Interface
+---------------------------
+
+When you run ``mach try fuzzy`` an interface similar to the one below will open. This is `fzf`_.
+
+.. image:: fzf.png
+
+There's a lot to unpack here, so let's examine each component a bit more closely.
+
+   A. The set of tasks that match the currently typed-out query. In the above image only tasks that
+   match the query ``'linux64 mochibrochr`` are displayed.
+
+   B. The set of selected tasks. These are the tasks that will be scheduled once you hit ``Enter``.
+   In other words, if the task you want does not appear here, *it won't be scheduled*.
+
+   C. Count information of the form ``x/y (z)``, where ``x`` is the number of tasks that match the
+   current query, ``y`` is the total number of tasks and ``z`` is the number of tasks you have
+   selected.
+
+   D. The input bar for entering queries. As you type you'll notice the list of tasks in ``A``
+   starts to update immediately. In the image above, the query ``'linux64 mochibrochr`` is entered.
+   Correspondingly only tasks matching that query are displayed.
+
+In general terms, you first find tasks on the left. Then you move them over to the right by
+selecting them. Once you are satisfied with your selection, press ``Enter`` to push to try.
+
+
+Selecting Tasks
+---------------
+
+There are few ways you can select tasks. If you are feeling a bit overwhelmed, it might be best to
+stick with the mouse to start:
+
+   1. Enter a query (e.g ``mochitest``) to reduce the task list a little.
+   2. Scroll up and look for the task(s) you want.
+   3. ``Right-Click`` as many tasks as desired to select them.
+   4. Optionally delete your query, go back to step 1) and repeat.
+   5. Press ``Enter`` to push (or ``Esc`` to cancel).
+
+.. note::
+
+   Dependencies are automatically filled in, so you can select a test task without needing
+   to select the build it depends on.
+
+As you ``Right-Click``, notice that a little arrow appears to the left of the task label. This
+indicates that the task is selected and exists in the preview pane to the right.
+
+Once you are a bit more comfortable with the interface, using the keyboard is much better at quickly
+selecting tasks. Here are the main shortcuts you'll need:
+
+.. code-block:: text
+
+    Ctrl-K / Up    => Move cursor up
+    Ctrl-J / Down  => Move cursor down
+    Tab            => Select task + move cursor down
+    Shift-Tab      => Select task + move cursor up
+    Ctrl-A         => Select all currently filtered tasks
+    Ctrl-T         => Toggle select all currently filtered tasks
+    Ctrl-D         => De-select all selected tasks (both filtered and not)
+    Alt-Bspace     => Clear query from input bar
+    Enter          => Accept selection and exit
+    Ctrl-C / Esc   => Cancel selection and exit
+    ?              => Toggle preview pane
+
+
+The process for selecting tasks is otherwise the same as for a mouse. A particularly fast and
+powerful way to select tasks is to:
+
+.. code-block:: text
+
+    Write a precise query => Ctrl-A => Alt-Bspace => Repeat
+
+As before, when you are satisfied with your selection press ``Enter`` and all the tasks in the
+preview pane will be pushed to try. If you change your mind you can press ``Esc`` or ``Ctrl-C`` to
+exit the interface without pushing anything.
+
+.. note::
+
+   Initially ``fzf`` will automatically select whichever task is under your cursor. This is a
+   convenience feature for the case where you are only selecting a single task. This feature will be
+   turned off as soon as you *lock in* a selection with ``Right-Click``, ``Tab`` or ``Ctrl-A``.
+
+
+Writing Queries
+---------------
+
+Queries are built from a series of terms, each separated by a space. Terms are logically joined by
+the AND operator. For example:
+
+.. code-block:: text
+
+    'windows 'mochitest
+
+This query has two terms, and is the equivalent of saying: Give me all the tasks that match both the
+term ``'windows'`` and the term ``'mochitest'``. In other words, this query matches all Windows
+mochitest tasks.
+
+The single quote prefix before each term tells ``fzf`` to use exact substring matches, so only tasks
+that contain both the literal string ``windows`` AND the literal string ``mochitest`` will be
+matched.
+
+Another thing to note is that the order of the terms makes no difference, so ``'windows 'mochitest``
+and ``'mochitest 'windows`` are equivalent.
+
+
+Fuzzy terms
+~~~~~~~~~~~
+
+If a term is *not* prefixed with a single quote, that makes it a fuzzy term. This means the
+characters in the term need to show up in order, but not in sequence. E.g the fuzzy term ``max``
+would match the string ``mozilla firefox`` (as first there is an ``m``, then an ``a`` and finally an
+``x``), but not the string ``firefox by mozilla`` (since the ``x`` is now out of order). Here's a
+less contrived example:
+
+.. code-block:: text
+
+    wndws mchtst
+
+Like the query above, this one would also select all Windows mochitest tasks. But it will
+additionally select:
+
+.. code-block:: text
+
+    test-macosx1014-64-shippable/opt-talos-sessionrestore-many-windows-e10s
+
+This is because both sequences of letters (``wndws`` and ``mchtst``) independently appear in order
+somewhere in this string (remember the order of the terms makes no difference).
+
+At first fuzzy terms may not seem very useful, but they are actually extremely powerful! Let's use
+the term from the interface image above, ``'linux64 mochibrochr``, as an example. First, just notice
+how in the image ``fzf`` highlights the characters that constitute the match in green. Next, notice
+how typing ``mochibrochr`` can quickly get us all mochitest browser-chrome tasks. The power of fuzzy
+terms is that you don't need to memorize the exact task labels you are looking for. Just start
+typing something you think is vaguely correct and chances are you'll see the task you're looking for.
+
+
+Term Modifiers
+~~~~~~~~~~~~~~
+
+The following modifiers can be applied to a search term:
+
+.. code-block:: text
+
+    'word    => exact match (line must contain the literal string "word")
+    ^word    => exact prefix match (line must start with literal "word")
+    word$    => exact suffix match (line must end with literal "word")
+    !word    => exact negation match (line must not contain literal "word")
+    'a | 'b  => OR operator (joins two exact match operators together)
+
+For example:
+
+.. code-block:: text
+
+    ^start 'exact | 'other !ignore fuzzy end$
+
+would match the string:
+
+.. code-block:: text
+
+    starting to bake isn't exactly fun, but pizza is yummy in the end
+
+.. note::
+
+    The best way to learn how to write queries is to run ``mach try fuzzy --no-push`` and play
+    around with all of these modifiers!
+
+
+Specifying Queries on the Command Line
+--------------------------------------
+
+Sometimes it's more convenient to skip the interactive interface and specify a query on the command
+line with ``-q/--query``. This is equivalent to opening the interface then typing:
+``<query><ctrl-a><enter>``.
+
+For example:
+
+.. code-block:: shell
+
+    # selects all mochitest tasks
+    $ mach try fuzzy --query "mochitest"
+
+You can pass in multiple queries at once and the results of each will be joined together:
+
+.. code-block:: shell
+
+    # selects all mochitest and reftest tasks
+    $ mach try fuzzy -q "mochitest" -q "reftest"
+
+If instead you want the intersection of queries, you can pass in ``-x/--and``:
+
+.. code-block:: shell
+
+    # selects all windows mochitest tasks
+    $ mach try fuzzy --and -q "mochitest" -q "windows"
+
+
+Modifying Presets
+~~~~~~~~~~~~~~~~~
+
+:doc:`Presets <../presets>` make it easy to run a pre-determined set of tasks. But sometimes you
+might not want to run that set exactly as is, you may only want to use the preset as a starting
+point then add or remove tasks as needed. This can be accomplished with ``-q/--query`` or
+``-i/--interactive``.
+
+Here are some examples of adding tasks to a preset:
+
+.. code-block:: shell
+
+    # selects all perf tasks plus all mochitest-chrome tasks
+    $ mach try fuzzy --preset perf -q "mochitest-chrome"
+
+    # adds tasks to the perf preset interactively
+    $ mach try fuzzy --preset perf -i
+
+Similarly, ``-x/--and`` can be used to filter down a preset by taking the intersection of the two
+sets:
+
+.. code-block:: shell
+
+    # limits perf tasks to windows only
+    $ mach try fuzzy --preset perf -xq "windows"
+
+    # limits perf tasks interactively
+    $ mach try fuzzy --preset perf -xi
+
+
+Shell Conflicts
+~~~~~~~~~~~~~~~
+
+Unfortunately ``fzf``'s query language uses some characters (namely ``'``, ``!`` and ``$``) that can
+interfere with your shell when using ``-q/--query``. Below are some tips for how to type out a query
+on the command line.
+
+The ``!`` character is typically used for history expansion. If you don't use this feature, the
+easiest way to specify queries on the command line is to disable it:
+
+.. code-block:: shell
+
+    # bash
+    $ set +H
+    $ ./mach try fuzzy -q "'foo !bar"
+
+    # zsh
+    $ setopt no_banghist
+    $ ./mach try fuzzy -q "'foo !bar"
+
+If using ``bash``, add ``set +H`` to your ``~/.bashrc``, ``~/.bash_profile`` or equivalent. If using
+``zsh``, add ``setopt no_banghist`` to your ``~/.zshrc`` or equivalent.
+
+If you don't want to disable history expansion, you can escape your queries like this:
+
+.. code-block:: shell
+
+    # bash
+    $ ./mach try fuzzy -q $'\'foo !bar'
+
+    # zsh
+    $ ./mach try fuzzy -q "'foo \!bar"
+
+
+The third option is to use ``-e/--exact`` which reverses the behaviour of the ``'`` character (see
+:ref:`additional-arguments` for more details). Using this flag means you won't need to escape the
+``'`` character as often and allows you to run your queries like this:
+
+.. code-block:: shell
+
+    # bash and zsh
+    $ ./mach try fuzzy -eq 'foo !bar'
+
+This method is only useful if you find you almost always prefix terms with ``'`` (and rarely use
+fuzzy terms). Otherwise as soon as you want to use a fuzzy match you'll run into the same problem as
+before.
+
+.. note:: All the examples in these three approaches will select the same set of tasks.
+
+If you use ``fish`` shell, you won't need to escape ``!``, however you will need to escape ``$``:
+
+.. code-block:: shell
+
+    # fish
+    $ ./mach try fuzzy -q "'foo !bar baz\$"
+
+
+Test Paths
+----------
+
+One or more paths to a file or directory may be specified as positional arguments. When
+specifying paths, the list of available tasks to choose from is filtered down such that
+only suites that have tests in a specified path can be selected. Notably, only the first
+chunk of each suite/platform appears. When the tasks are scheduled, only tests that live
+under one of the specified paths will be run.
+
+.. note::
+
+    When using paths, be aware that all tests under the specified paths will run in the
+    same chunk. This might produce a different ordering from what gets run on production
+    branches, and may yield different results.
+
+    For suites that restart the browser between each manifest (like mochitest), this
+    shouldn't be as big of a concern.
+
+Paths can be used with the interactive ``fzf`` window, or using the ``-q/--query`` argument.
+For example, running:
+
+.. code-block:: shell
+
+    $ mach try fuzzy layout/reftests/reftest-sanity -q "!pgo !cov !asan 'linux64"
+
+Would produce the following ``try_task_config.json``:
+
+.. code-block:: json
+
+    {
+      "env":{
+        "MOZHARNESS_TEST_PATHS":"{\"reftest\":\"layout/reftests/reftest-sanity\"}"
+      },
+      "tasks":[
+        "test-linux64-qr/debug-reftest-e10s-1",
+        "test-linux64-qr/opt-reftest-e10s-1",
+        "test-linux64/debug-reftest-e10s-1",
+        "test-linux64/debug-reftest-no-accel-e10s-1",
+        "test-linux64/opt-reftest-e10s-1",
+        "test-linux64/opt-reftest-no-accel-e10s-1",
+      ]
+    }
+
+Inside of these tasks, the reftest harness will only run tests that live under
+``layout/reftests/reftest-sanity``.
+
+
+.. _additional-arguments:
+
+Additional Arguments
+--------------------
+
+There are a few additional command line arguments you may wish to use:
+
+``-e/--exact``
+By default, ``fzf`` treats terms as a fuzzy match and prefixing a term with ``'`` turns it into an exact
+match. If passing in ``--exact``, this behaviour is reversed. Non-prefixed terms become exact, and a
+``'`` prefix makes a term fuzzy.
+
+``--full``
+By default, only target tasks (e.g tasks that would normally run on mozilla-central)
+are generated. Passing in ``--full`` allows you to select from all tasks. This is useful for
+things like nightly or release tasks.
+
+``-u/--update``
+Update the bootstrapped ``fzf`` binary to the latest version.
+
+For a full list of command line arguments, run:
+
+.. code-block:: shell
+
+    $ mach try fuzzy --help
+
+For more information on using ``fzf``, run:
+
+.. code-block:: shell
+
+    $ man fzf
+
+.. _fzf: https://github.com/junegunn/fzf
diff --git a/tools/tryselect/docs/selectors/fzf.png b/tools/tryselect/docs/selectors/fzf.png
new file mode 100644
index 0000000000..a64f4b04f3
--- /dev/null
+++ b/tools/tryselect/docs/selectors/fzf.png
diff --git a/tools/tryselect/docs/selectors/index.rst b/tools/tryselect/docs/selectors/index.rst
new file mode 100644
index 0000000000..be618202d6
--- /dev/null
+++ b/tools/tryselect/docs/selectors/index.rst
@@ -0,0 +1,44 @@
+Selectors
+=========
+
+These are the currently implemented try selectors:
+
+* :doc:`auto <auto>`: Have tasks chosen for you automatically.
+* :doc:`fuzzy <fuzzy>`: Select tasks using a fuzzy finding algorithm and
+  a terminal interface.
+* :doc:`chooser <chooser>`: Select tasks using a web interface.
+* :doc:`again <again>`: Re-run a previous ``try_task_config.json`` based
+  push.
+* :doc:`empty <empty>`: Don't select any tasks. Taskcluster will still run
+  some tasks automatically (like lint and python unittest tasks). Further tasks
+  can be chosen with treeherder's ``Add New Jobs`` feature.
+* :doc:`syntax <syntax>`: Select tasks using classic try syntax.
+* :doc:`release <release>`: Prepare a tree for doing a staging release.
+* :doc:`scriptworker <scriptworker>`: Run scriptworker tasks against a recent release.
+* :doc:`compare <compare>`: Push two identical try jobs, one on your current commit and another of your choice
+
+You can run them with:
+
+.. code-block:: shell
+
+    $ mach try <selector>
+
+See selector specific options by running:
+
+.. code-block:: shell
+
+    $ mach try <selector> --help
+
+.. toctree::
+  :caption: Available Selectors
+  :maxdepth: 1
+  :hidden:
+
+  Auto <auto>
+  Fuzzy <fuzzy>
+  Chooser <chooser>
+  Again <again>
+  Empty <empty>
+  Syntax <syntax>
+  Release <release>
+  Scriptworker <scriptworker>
diff --git a/tools/tryselect/docs/selectors/release.rst b/tools/tryselect/docs/selectors/release.rst
new file mode 100644
index 0000000000..946266249b
--- /dev/null
+++ b/tools/tryselect/docs/selectors/release.rst
@@ -0,0 +1,31 @@
+Release Selector
+================
+
+This command configures the tree in preparation for doing a staging release,
+and pushes the result to try. The changes that that are made include:
+
+- Updating the version number.
+- Applying the migrations that are done as part of merge day.
+- Disabling repacking most locales. (This can be disabled by passing ``--no-limit-locales``).
+
+For staging a beta release, run the following (with an appropriate version number):
+
+.. code-block:: shell
+
+   $ mach try release --version 64.0b5 --migration central-to-beta
+
+For staging a final release (rc or patch), run the following (with an appropriate version number)
+
+.. code-block:: shell
+
+   $ mach try release --version 64.0 --migration central-to-beta --migration beta-to-release
+
+Once the decision task is on the push is complete, you can start the release
+through `staging ship-it instance <https://shipit.staging.mozilla-releng.net/new>`_\ [#shipit]_.
+
+.. note::
+
+   If pushing from beta or release, the corresponding migration should not be
+   passed, as they have already been applied.
+
+.. [#shipit] This is only available to release engineering and release management (as of 2018-10-15).
diff --git a/tools/tryselect/docs/selectors/scriptworker.rst b/tools/tryselect/docs/selectors/scriptworker.rst
new file mode 100644
index 0000000000..a3cba08cbe
--- /dev/null
+++ b/tools/tryselect/docs/selectors/scriptworker.rst
@@ -0,0 +1,31 @@
+Scriptworker Selector
+=====================
+
+This command runs a selection of scriptworker tasks against builds from a
+recent release.  This is aimed at release engineering, to test changes to
+scriptworker implementations. It currently requires being connected to
+Mozilla's internal datacenter VPN with access to shipit\ [#shipit]_.
+
+There are a number of preset groups of tasks to run. To run a particular
+set of tasks, pass the name of the set to ``mach try scriptworker``:
+
+.. code-block:: shell
+
+   $ mach try scriptworker linux-signing
+
+To get the list of task sets, along with the list of tasks they will run:
+
+.. code-block:: shell
+
+   $ mach try scriptworker list
+
+The selector defaults to using tasks from the most recent beta, to use tasks
+from a different release, pass ``--release-type <release-type>``:
+
+.. code-block:: shell
+
+   $ mach try scriptworker --release-type release linux-signing
+
+
+.. [#shipit] The shipit API is not currently publicly available, and is used
+   to find the release graph to pull previous tasks from.
diff --git a/tools/tryselect/docs/selectors/syntax.rst b/tools/tryselect/docs/selectors/syntax.rst
new file mode 100644
index 0000000000..b64bb65ab7
--- /dev/null
+++ b/tools/tryselect/docs/selectors/syntax.rst
@@ -0,0 +1,41 @@
+Syntax Selector
+===============
+
+.. warning::
+
+    Try syntax is antiquated and hard to understand. If you aren't already
+    familiar with try syntax, you might want to use the ``fuzzy`` selector
+    instead.
+
+Try syntax is a command line string that goes into the commit message. Using
+``mach try syntax`` will automatically create a temporary commit with your
+chosen syntax and then delete it again after pushing to try.
+
+Try syntax can contain all kinds of different options parsed by various
+places in various repos, but the majority are parsed by `try_option_syntax.py`_.
+The most common arguments include:
+
+    * ``-b/--build`` - One of ``d``, ``o`` or ``do``. This is the build type,
+      either opt, debug or both (required).
+    * ``-p/--platform`` - The platforms you want to build and/or run tests on
+      (required).
+    * ``-u/--unittests`` - The test tasks you want to run (optional).
+    * ``-t/--talos`` - The talos tasks you want to run (optional).
+
+Here are some examples:
+
+.. code-block:: shell
+
+    $ mach try syntax -b do -p linux,macosx64 -u mochitest-e10s-1,crashtest -t none
+    $ mach try syntax -b d -p win64 -u all
+    $ mach try syntax -b o -p linux64
+
+Unfortunately, knowing the magic strings that make it work can be a bit of a
+guessing game. If you are unsure of what string will get your task to run, try
+using :doc:`mach try fuzzy <fuzzy>` instead.
+
+While using ``mach try syntax -b do -p all -u all -t all`` will work, heavy use
+of ``all`` is discouraged as it consumes a lot of unnecessary resources (some of
+which are hardware constrained).
+
+..  _try_option_syntax.py: https://searchfox.org/mozilla-central/source/taskcluster/gecko_taskgraph/try_option_syntax.py
diff --git a/tools/tryselect/docs/tasks.rst b/tools/tryselect/docs/tasks.rst
new file mode 100644
index 0000000000..61de9ec9ed
--- /dev/null
+++ b/tools/tryselect/docs/tasks.rst
@@ -0,0 +1,152 @@
+Task Generation
+===============
+
+Many selectors (including ``chooser``, ``coverage`` and ``fuzzy``) source their available tasks
+directly from the :ref:`taskgraph <TaskCluster Task-Graph Generation>` module by building the taskgraph
+locally. This means that the list of available tasks will never be stale. While this is very
+powerful, it comes with a large enough performance cost to get annoying (around twenty seconds).
+
+The result of the taskgraph generation will be cached, so this penalty will only be incurred
+whenever a file in the ``/taskcluster`` directory is modified. Unfortunately this directory changes
+pretty frequently, so developers can expect to rebuild the cache each time they pull in
+``mozilla-central``. Developers who regularly work on ``/taskcluster`` can expect to rebuild even
+more frequently.
+
+
+Configuring Watchman
+--------------------
+
+It's possible to bypass this penalty completely by using the file watching service `watchman`_. If
+you use the ``fsmonitor`` mercurial extension, you already have ``watchman`` installed.
+
+.. note::
+
+    If you aren't using `fsmonitor`_ but end up installng watchman anyway, you
+    might as well enable it for a faster Mercurial experience.
+
+Otherwise, `install watchman`_. If using Linux you'll likely run into the `inotify limits`_ outlined
+on that page due to the size of ``mozilla-central``. You can `read this page`_ for more information
+on how to bump the limits permanently.
+
+Next run the following commands:
+
+.. code-block:: shell
+
+    $ cd path/to/mozilla-central
+    $ watchman watch .
+    $ watchman -j < tools/tryselect/watchman.json
+
+You should see output like:
+
+.. code-block:: json
+
+    {
+        "triggerid": "rebuild-taskgraph-cache",
+        "disposition": "created",
+        "version": "20200920.192359.0"
+    }
+
+That's it. Now anytime a file under ``/taskcluster`` is modified (either by your editor, or by
+updating version control), the taskgraph cache will be rebuilt in the background, allowing you to
+skip the wait the next time you run ``mach try``.
+
+.. note::
+
+    Watchman triggers are persistent and don't need to be added more than once.
+    See `Managing Triggers`_ for how to remove a trigger.
+
+You can test that everything is working by running these commands:
+
+.. code-block:: shell
+
+    $ statedir=`mach python -c "from mach.util import get_state_dir; print(get_state_dir(specific_to_topsrcdir=True))"`
+    $ rm -rf $statedir/cache/taskgraph
+    $ touch taskcluster/mach_commands.py
+    # wait a minute for generation to trigger and finish
+    $ ls $statedir/cache/taskgraph
+
+If the ``target_task_set`` file exists, you are good to go. If not you can look at the ``watchman``
+log to see if there were any errors. This typically lives somewhere like
+``/usr/local/var/run/watchman/$USER-state/log``. In this case please file a bug under ``Firefox
+Build System :: Try`` and include the relevant portion of the log.
+
+
+Running Watchman on Startup
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Watchman is both a client and a service all in one. When running a ``watchman`` command, the client
+binary will start the service in the background if it isn't running. This means on reboot the
+service won't be running and you'll need to start the service each time by invoking the client
+binary (e.g by running ``watchman version``).
+
+If you'd like this to happen automatically, you can use your favourite platform specific way of
+running commands at startup (``crontab``, ``rc.local``, etc.). Watchman stores separate state for
+each user, so be sure you run the command as the user that set up the triggers.
+
+Setting up a systemd Service
+++++++++++++++++++++++++++++
+
+If ``systemd`` is an option you can create a service:
+
+.. code-block:: ini
+
+    [Unit]
+    Description=Watchman for %i
+    After=network.target
+
+    [Service]
+    Type=simple
+    User=%i
+    ExecStart=/usr/local/bin/watchman --log-level 1 watch-list -f
+    ExecStop=/usr/local/bin/watchman shutdown-server
+
+    [Install]
+    WantedBy=multi-user.target
+
+Save this to a file called ``/etc/systemd/system/watchman@.service``. Then run:
+
+.. code-block:: shell
+
+    $ sudo systemctl enable watchman@$USER.service
+    $ sudo systemctl start watchman@$USER.service
+
+The next time you reboot, the watchman service should start automatically.
+
+
+Managing Triggers
+~~~~~~~~~~~~~~~~~
+
+When adding a trigger watchman writes it to disk. Typically it'll be a path similar to
+``/usr/local/var/run/watchman/$USER-state/state``. While editing that file by hand would work, the
+watchman binary provides an interface for managing your triggers.
+
+To see all directories you are currently watching:
+
+.. code-block:: shell
+
+    $ watchman watch-list
+
+To view triggers that are active in a specified watch:
+
+.. code-block:: shell
+
+    $ watchman trigger-list <path>
+
+To delete a trigger from a specified watch:
+
+.. code-block:: shell
+
+    $ watchman trigger-del <path> <name>
+
+In the above two examples, replace ``<path>`` with the path of the watch, presumably
+``mozilla-central``. Using ``.`` works as well if that is already your working directory. For more
+information on managing triggers and a reference of other commands, see the `official docs`_.
+
+
+.. _watchman: https://facebook.github.io/watchman/
+.. _fsmonitor: https://www.mercurial-scm.org/wiki/FsMonitorExtension
+.. _install watchman: https://facebook.github.io/watchman/docs/install.html
+.. _inotify limits: https://facebook.github.io/watchman/docs/install.html#linux-inotify-limits
+.. _read this page: https://github.com/guard/listen/wiki/Increasing-the-amount-of-inotify-watchers
+.. _this hint: https://github.com/facebook/watchman/commit/2985377eaf8c8538b28fae9add061b67991a87c2
+.. _official docs: https://facebook.github.io/watchman/docs/cmd/trigger.html