diff options
Diffstat (limited to 'tools/tryselect/docs/tasks.rst')
| -rw-r--r-- | tools/tryselect/docs/tasks.rst | 152 |
1 files changed, 152 insertions, 0 deletions
diff --git a/tools/tryselect/docs/tasks.rst b/tools/tryselect/docs/tasks.rst new file mode 100644 index 0000000000..61de9ec9ed --- /dev/null +++ b/tools/tryselect/docs/tasks.rst @@ -0,0 +1,152 @@ +Task Generation +=============== + +Many selectors (including ``chooser``, ``coverage`` and ``fuzzy``) source their available tasks +directly from the :ref:`taskgraph <TaskCluster Task-Graph Generation>` module by building the taskgraph +locally. This means that the list of available tasks will never be stale. While this is very +powerful, it comes with a large enough performance cost to get annoying (around twenty seconds). + +The result of the taskgraph generation will be cached, so this penalty will only be incurred +whenever a file in the ``/taskcluster`` directory is modified. Unfortunately this directory changes +pretty frequently, so developers can expect to rebuild the cache each time they pull in +``mozilla-central``. Developers who regularly work on ``/taskcluster`` can expect to rebuild even +more frequently. + + +Configuring Watchman +-------------------- + +It's possible to bypass this penalty completely by using the file watching service `watchman`_. If +you use the ``fsmonitor`` mercurial extension, you already have ``watchman`` installed. + +.. note:: + + If you aren't using `fsmonitor`_ but end up installng watchman anyway, you + might as well enable it for a faster Mercurial experience. + +Otherwise, `install watchman`_. If using Linux you'll likely run into the `inotify limits`_ outlined +on that page due to the size of ``mozilla-central``. You can `read this page`_ for more information +on how to bump the limits permanently. + +Next run the following commands: + +.. code-block:: shell + + $ cd path/to/mozilla-central + $ watchman watch . + $ watchman -j < tools/tryselect/watchman.json + +You should see output like: + +.. code-block:: json + + { + "triggerid": "rebuild-taskgraph-cache", + "disposition": "created", + "version": "20200920.192359.0" + } + +That's it. Now anytime a file under ``/taskcluster`` is modified (either by your editor, or by +updating version control), the taskgraph cache will be rebuilt in the background, allowing you to +skip the wait the next time you run ``mach try``. + +.. note:: + + Watchman triggers are persistent and don't need to be added more than once. + See `Managing Triggers`_ for how to remove a trigger. + +You can test that everything is working by running these commands: + +.. code-block:: shell + + $ statedir=`mach python -c "from mach.util import get_state_dir; print(get_state_dir(specific_to_topsrcdir=True))"` + $ rm -rf $statedir/cache/taskgraph + $ touch taskcluster/mach_commands.py + # wait a minute for generation to trigger and finish + $ ls $statedir/cache/taskgraph + +If the ``target_task_set`` file exists, you are good to go. If not you can look at the ``watchman`` +log to see if there were any errors. This typically lives somewhere like +``/usr/local/var/run/watchman/$USER-state/log``. In this case please file a bug under ``Firefox +Build System :: Try`` and include the relevant portion of the log. + + +Running Watchman on Startup +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Watchman is both a client and a service all in one. When running a ``watchman`` command, the client +binary will start the service in the background if it isn't running. This means on reboot the +service won't be running and you'll need to start the service each time by invoking the client +binary (e.g by running ``watchman version``). + +If you'd like this to happen automatically, you can use your favourite platform specific way of +running commands at startup (``crontab``, ``rc.local``, etc.). Watchman stores separate state for +each user, so be sure you run the command as the user that set up the triggers. + +Setting up a systemd Service +++++++++++++++++++++++++++++ + +If ``systemd`` is an option you can create a service: + +.. code-block:: ini + + [Unit] + Description=Watchman for %i + After=network.target + + [Service] + Type=simple + User=%i + ExecStart=/usr/local/bin/watchman --log-level 1 watch-list -f + ExecStop=/usr/local/bin/watchman shutdown-server + + [Install] + WantedBy=multi-user.target + +Save this to a file called ``/etc/systemd/system/watchman@.service``. Then run: + +.. code-block:: shell + + $ sudo systemctl enable watchman@$USER.service + $ sudo systemctl start watchman@$USER.service + +The next time you reboot, the watchman service should start automatically. + + +Managing Triggers +~~~~~~~~~~~~~~~~~ + +When adding a trigger watchman writes it to disk. Typically it'll be a path similar to +``/usr/local/var/run/watchman/$USER-state/state``. While editing that file by hand would work, the +watchman binary provides an interface for managing your triggers. + +To see all directories you are currently watching: + +.. code-block:: shell + + $ watchman watch-list + +To view triggers that are active in a specified watch: + +.. code-block:: shell + + $ watchman trigger-list <path> + +To delete a trigger from a specified watch: + +.. code-block:: shell + + $ watchman trigger-del <path> <name> + +In the above two examples, replace ``<path>`` with the path of the watch, presumably +``mozilla-central``. Using ``.`` works as well if that is already your working directory. For more +information on managing triggers and a reference of other commands, see the `official docs`_. + + +.. _watchman: https://facebook.github.io/watchman/ +.. _fsmonitor: https://www.mercurial-scm.org/wiki/FsMonitorExtension +.. _install watchman: https://facebook.github.io/watchman/docs/install.html +.. _inotify limits: https://facebook.github.io/watchman/docs/install.html#linux-inotify-limits +.. _read this page: https://github.com/guard/listen/wiki/Increasing-the-amount-of-inotify-watchers +.. _this hint: https://github.com/facebook/watchman/commit/2985377eaf8c8538b28fae9add061b67991a87c2 +.. _official docs: https://facebook.github.io/watchman/docs/cmd/trigger.html |