Adding upstream version 1:115.7.0.upstream/1%115.7.0upstream

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

10 files changed, 1006 insertions, 0 deletions

diff --git a/python/mach/docs/commands.rst b/python/mach/docs/commands.rst
new file mode 100644
index 0000000000..7547193000
--- /dev/null
+++ b/python/mach/docs/commands.rst
@@ -0,0 +1,129 @@
+.. _mach_commands:
+
+=====================
+Implementing Commands
+=====================
+
+Mach commands are defined via Python decorators.
+
+All the relevant decorators are defined in the *mach.decorators* module.
+The important decorators are as follows:
+
+:py:func:`Command <mach.decorators.Command>`
+  A function decorator that denotes that the function should be called when
+  the specified command is requested. The decorator takes a command name
+  as its first argument and a number of additional arguments to
+  configure the behavior of the command. The decorated function must take a
+  ``command_context`` argument as its first.
+  ``command_context`` is a properly configured instance of a ``MozbuildObject``
+  subclass, meaning it can be used for accessing things like the current config
+  and running processes.
+
+:py:func:`CommandArgument <mach.decorators.CommandArgument>`
+  A function decorator that defines an argument to the command. Its
+  arguments are essentially proxied to ArgumentParser.add_argument()
+
+:py:func:`SubCommand <mach.decorators.SubCommand>`
+  A function decorator that denotes that the function should be a
+  sub-command to an existing ``@Command``. The decorator takes the
+  parent command name as its first argument and the sub-command name
+  as its second argument.
+
+  ``@CommandArgument`` can be used on ``@SubCommand`` instances just
+  like they can on ``@Command`` instances.
+
+
+Here is a complete example:
+
+.. code-block:: python
+
+   from mach.decorators import (
+       CommandArgument,
+       Command,
+   )
+
+    @Command('doit', help='Do ALL OF THE THINGS.')
+    @CommandArgument('--force', '-f', action='store_true',
+        help='Force doing it.')
+    def doit(command_context, force=False):
+        # Do stuff here.
+
+When the module is loaded, the decorators tell mach about all handlers.
+When mach runs, it takes the assembled metadata from these handlers and
+hooks it up to the command line driver. Under the hood, arguments passed
+to the decorators are being used to help mach parse command arguments,
+formulate arguments to the methods, etc. See the documentation in the
+:py:mod:`mach.base` module for more.
+
+The Python modules defining mach commands do not need to live inside the
+main mach source tree.
+
+Conditionally Filtering Commands
+================================
+
+Sometimes it might only make sense to run a command given a certain
+context. For example, running tests only makes sense if the product
+they are testing has been built, and said build is available. To make
+sure a command is only runnable from within a correct context, you can
+define a series of conditions on the
+:py:func:`Command <mach.decorators.Command>` decorator.
+
+A condition is simply a function that takes an instance of the
+:py:func:`mozbuild.base.MachCommandBase` class as an argument, and
+returns ``True`` or ``False``. If any of the conditions defined on a
+command return ``False``, the command will not be runnable. The
+docstring of a condition function is used in error messages, to explain
+why the command cannot currently be run.
+
+Here is an example:
+
+.. code-block:: python
+
+   from mach.decorators import (
+       Command,
+   )
+
+   def build_available(cls):
+       """The build needs to be available."""
+       return cls.build_path is not None
+
+   @Command('run_tests', conditions=[build_available])
+   def run_tests(command_context):
+       # Do stuff here.
+
+By default all commands without any conditions applied will be runnable,
+but it is possible to change this behaviour by setting
+``require_conditions`` to ``True``:
+
+.. code-block:: python
+
+   m = mach.main.Mach()
+   m.require_conditions = True
+
+Minimizing Code in Commands
+===========================
+
+Mach command modules, classes, and methods work best when they are
+minimal dispatchers. The reason is import bloat. Currently, the mach
+core needs to import every Python file potentially containing mach
+commands for every command invocation. If you have dozens of commands or
+commands in modules that import a lot of Python code, these imports
+could slow mach down and waste memory.
+
+It is thus recommended that mach modules, classes, and methods do as
+little work as possible. Ideally the module should only import from
+the :py:mod:`mach` package. If you need external modules, you should
+import them from within the command method.
+
+To keep code size small, the body of a command method should be limited
+to:
+
+1. Obtaining user input (parsing arguments, prompting, etc)
+2. Calling into some other Python package
+3. Formatting output
+
+Of course, these recommendations can be ignored if you want to risk
+slower performance.
+
+In the future, the mach driver may cache the dispatching information or
+have it intelligently loaded to facilitate lazy loading.
diff --git a/python/mach/docs/driver.rst b/python/mach/docs/driver.rst
new file mode 100644
index 0000000000..8a2a99a2f5
--- /dev/null
+++ b/python/mach/docs/driver.rst
@@ -0,0 +1,32 @@
+.. _mach_driver:
+
+=======
+Drivers
+=======
+
+Entry Points
+============
+
+It is possible to use setuptools' entry points to load commands
+directly from python packages. A mach entry point is a function which
+returns a list of files or directories containing mach command
+providers. e.g.:
+
+.. code-block:: python
+
+   def list_providers():
+       providers = []
+       here = os.path.abspath(os.path.dirname(__file__))
+       for p in os.listdir(here):
+           if p.endswith('.py'):
+               providers.append(os.path.join(here, p))
+       return providers
+
+See http://pythonhosted.org/setuptools/setuptools.html#dynamic-discovery-of-services-and-plugins
+for more information on creating an entry point. To search for entry
+point plugins, you can call
+:py:meth:`mach.main.Mach.load_commands_from_entry_point`. e.g.:
+
+.. code-block:: python
+
+   mach.load_commands_from_entry_point("mach.external.providers")
diff --git a/python/mach/docs/faq.rst b/python/mach/docs/faq.rst
new file mode 100644
index 0000000000..a640f83e87
--- /dev/null
+++ b/python/mach/docs/faq.rst
@@ -0,0 +1,152 @@
+.. _mach_faq:
+
+==========================
+Frequently Asked Questions
+==========================
+
+How do I report bugs?
+---------------------
+
+Bugs against the ``mach`` core can be filed in Bugzilla in the `Firefox
+Build System::Mach
+Core <https://bugzilla.mozilla.org/enter_bug.cgi?product=Firefox%20Build%20System&component=Mach%20Core>`__ component.
+
+.. note::
+
+   Most ``mach`` bugs are bugs in individual commands, not bugs in the core
+   ``mach`` code. Bugs for individual commands should be filed against the
+   component that command is related to. For example, bugs in the
+   *build* command should be filed against *Firefox Build System ::
+   General*. Bugs against testing commands should be filed somewhere in
+   the *Testing* product.
+
+How do I debug a command failing with a Python exception?
+---------------------------------------------------------
+
+You can run a command and break into ``pdb``, the Python debugger,
+when the command is invoked with:
+
+.. code-block:: shell
+
+   ./mach --debug-command FAILING-COMMAND ARGS ...
+
+How do I debug ``mach`` itself?
+-------------------------------
+
+If you are editing the mach code, or other Python modules you can
+open the terminal and start debugging with pdb with the following:
+
+.. code-block:: shell
+
+        python3 -m pdb ./mach <command>
+
+How do I debug ``pytest`` tests?
+--------------------------------
+
+First, before debugging, run ``./mach python-test`` once to ensure that
+the testing virtualenv is up-to-date:
+
+.. code-block:: shell
+
+    ./mach python-test path/to/test.py
+
+Then, using the testing virtualenv, debug the test file:
+
+.. code-block:: shell
+
+    <objdir>/_virtualenvs/python-test/bin/python -m pdb path/to/test.py
+
+How do I profile a slow command?
+--------------------------------
+
+To diagnose bottlenecks, you can collect a performance profile:
+
+.. code-block:: shell
+
+    ./mach --profile-command SLOW-COMMAND ARGS ...
+
+Then, you can visualize ``mach_profile_SLOW-COMMAND.cProfile`` using
+`snakeviz <https://jiffyclub.github.io/snakeviz/>`__:
+
+.. code-block:: shell
+
+    # If you don't have snakeviz installed yet:
+    python3 -m pip install snakeviz
+    python3 -m snakeviz mach_profile_SLOW-COMMAND.cProfile
+
+How do I profile ``mach`` itself?
+---------------------------------
+
+Since ``--profile-command`` only profiles commands, you'll need to invoke ``cProfile``
+directly to profile ``mach`` itself:
+
+.. code-block:: shell
+
+    python3 -m cProfile -o mach.cProfile ./mach ...
+    python3 -m snakeviz mach.cProfile
+
+Is ``mach`` a build system?
+---------------------------
+
+No. ``mach`` is just a generic command dispatching tool that happens to have
+a few commands that interact with the real build system. Historically,
+``mach`` *was* born to become a better interface to the build system.
+However, its potential beyond just build system interaction was quickly
+realized and ``mach`` grew to fit those needs.
+
+How do I add features to ``mach``?
+----------------------------------
+If you would like to add a new feature to ``mach`` that cannot be implemented as
+a ``mach`` command, the first step is to file a bug in the
+``Firefox Build System :: Mach Core`` component.
+
+Should I implement X as a ``mach`` command?
+-------------------------------------------
+
+There are no hard or fast rules. Generally speaking, if you have some
+piece of functionality or action that is useful to multiple people
+(especially if it results in productivity wins), then you should
+consider implementing a ``mach`` command for it.
+
+Some other cases where you should consider implementing something as a
+``mach`` command:
+
+-  When your tool is a random script in the tree. Random scripts are
+   hard to find and may not conform to coding conventions or best
+   practices. ``Mach`` provides a framework in which your tool can live that
+   will put it in a better position to succeed than if it were on its
+   own.
+-  When the alternative is a ``make`` target. The build team generally does
+   not like one-off ``make`` targets that aren't part of building (read:
+   compiling) the tree. This includes things related to testing and
+   packaging. These weigh down ``Makefiles`` and add to the burden of
+   maintaining the build system. Instead, you are encouraged to
+   implement ancillary functionality in Python. If you do implement something
+   in Python, hooking it up to ``mach`` is often trivial.
+
+How do I use 3rd-party Python packages in my ``mach`` command?
+--------------------------------------------------------------
+
+See :ref:`Using third-party Python packages`.
+
+How does ``mach`` fit into the modules system?
+----------------------------------------------
+
+Mozilla operates with a `modules governance
+system <https://www.mozilla.org/about/governance/policies/module-ownership/>`__ where
+there are different components with different owners. There is not
+currently a ``mach`` module. There may or may never be one; currently ``mach``
+is owned by the build team.
+
+Even if a ``mach`` module were established, ``mach`` command modules would
+likely never belong to it. Instead, ``mach`` command modules are owned by the
+team/module that owns the system they interact with. In other words, ``mach``
+is not a power play to consolidate authority for tooling. Instead, it aims to
+expose that tooling through a common, shared interface.
+
+
+Who do I contact for help or to report issues?
+----------------------------------------------
+
+You can ask questions in
+`#build <https://chat.mozilla.org/#/room/#build:mozilla.org>`__.
diff --git a/python/mach/docs/index.rst b/python/mach/docs/index.rst
new file mode 100644
index 0000000000..752fe93219
--- /dev/null
+++ b/python/mach/docs/index.rst
@@ -0,0 +1,89 @@
+====
+Mach
+====
+
+Mach (German for *do*) is a generic command dispatcher for the command
+line.
+
+To use mach, you install the mach core (a Python package), create an
+executable *driver* script (named whatever you want), and write mach
+commands. When the *driver* is executed, mach dispatches to the
+requested command handler automatically.
+
+.. raw:: html
+
+    <h2>Features</h2>
+
+----
+
+On a high level, mach is similar to using argparse with subparsers (for
+command handling). When you dig deeper, mach offers a number of
+additional features:
+
+Distributed command definitions
+  With optparse/argparse, you have to define your commands on a central
+  parser instance. With mach, you annotate your command methods with
+  decorators and mach finds and dispatches to them automatically.
+
+Command categories
+  Mach commands can be grouped into categories when displayed in help.
+  This is currently not possible with argparse.
+
+Logging management
+  Mach provides a facility for logging (both classical text and
+  structured) that is available to any command handler.
+
+Settings files
+  Mach provides a facility for reading settings from an ini-like file
+  format.
+
+.. raw:: html
+
+    <h2>Components</h2>
+
+----
+
+Mach is conceptually composed of the following components:
+
+core
+  The mach core is the core code powering mach. This is a Python package
+  that contains all the business logic that makes mach work. The mach
+  core is common to all mach deployments.
+
+commands
+  These are what mach dispatches to. Commands are simply Python methods
+  registered as command names. The set of commands is unique to the
+  environment mach is deployed in.
+
+driver
+  The *driver* is the entry-point to mach. It is simply an executable
+  script that loads the mach core, tells it where commands can be found,
+  then asks the mach core to handle the current request. The driver is
+  unique to the deployed environment. But, it's usually based on an
+  example from this source tree.
+
+.. raw:: html
+
+    <h2> Project State</h2>
+
+----
+
+mach was originally written as a command dispatching framework to aid
+Firefox development. While the code is mostly generic, there are still
+some pieces that closely tie it to Mozilla/Firefox. The goal is for
+these to eventually be removed and replaced with generic features so
+mach is suitable for anybody to use. Until then, mach may not be the
+best fit for you.
+
+.. toctree::
+   :maxdepth: 1
+   :hidden:
+
+   usage
+   commands
+   driver
+   logging
+   settings
+   telemetry
+   windows-usage-outside-mozillabuild
+   faq
diff --git a/python/mach/docs/logging.rst b/python/mach/docs/logging.rst
new file mode 100644
index 0000000000..ff245cf032
--- /dev/null
+++ b/python/mach/docs/logging.rst
@@ -0,0 +1,100 @@
+.. _mach_logging:
+
+=======
+Logging
+=======
+
+Mach configures a built-in logging facility so commands can easily log
+data.
+
+What sets the logging facility apart from most loggers you've seen is
+that it encourages structured logging. Instead of conventional logging
+where simple strings are logged, the internal logging mechanism logs all
+events with the following pieces of information:
+
+* A string *action*
+* A dict of log message fields
+* A formatting string
+
+Essentially, instead of assembling a human-readable string at
+logging-time, you create an object holding all the pieces of data that
+will constitute your logged event. For each unique type of logged event,
+you assign an *action* name.
+
+Depending on how logging is configured, your logged event could get
+written a couple of different ways.
+
+JSON Logging
+============
+
+Where machines are the intended target of the logging data, a JSON
+logger is configured. The JSON logger assembles an array consisting of
+the following elements:
+
+* Decimal wall clock time in seconds since UNIX epoch
+* String *action* of message
+* Object with structured message data
+
+The JSON-serialized array is written to a configured file handle.
+Consumers of this logging stream can just perform a readline() then feed
+that into a JSON deserializer to reconstruct the original logged
+message. They can key off the *action* element to determine how to
+process individual events. There is no need to invent a parser.
+Convenient, isn't it?
+
+Logging for Humans
+==================
+
+Where humans are the intended consumer of a log message, the structured
+log message are converted to more human-friendly form. This is done by
+utilizing the *formatting* string provided at log time. The logger
+simply calls the *format* method of the formatting string, passing the
+dict containing the message's fields.
+
+When *mach* is used in a terminal that supports it, the logging facility
+also supports terminal features such as colorization. This is done
+automatically in the logging layer - there is no need to control this at
+logging time.
+
+In addition, messages intended for humans typically prepends every line
+with the time passed since the application started.
+
+Logging HOWTO
+=============
+
+Structured logging piggybacks on top of Python's built-in logging
+infrastructure provided by the *logging* package. We accomplish this by
+taking advantage of *logging.Logger.log()*'s *extra* argument. To this
+argument, we pass a dict with the fields *action* and *params*. These
+are the string *action* and dict of message fields, respectively. The
+formatting string is passed as the *msg* argument, like normal.
+
+If you were logging to a logger directly, you would do something like:
+
+.. code-block:: python
+
+   logger.log(logging.INFO, 'My name is {name}',
+       extra={'action': 'my_name', 'params': {'name': 'Gregory'}})
+
+The JSON logging would produce something like::
+
+   [1339985554.306338, "my_name", {"name": "Gregory"}]
+
+Human logging would produce something like::
+
+   0.52 My name is Gregory
+
+Since there is a lot of complexity using logger.log directly, it is
+recommended to go through a wrapping layer that hides part of the
+complexity for you. The easiest way to do this is by utilizing the
+LoggingMixin:
+
+.. code-block:: python
+
+   import logging
+   from mach.mixin.logging import LoggingMixin
+
+   class MyClass(LoggingMixin):
+       def foo(self):
+            self.log(logging.INFO, 'foo_start', {'bar': True},
+                'Foo performed. Bar: {bar}')
diff --git a/python/mach/docs/metrics.md b/python/mach/docs/metrics.md
new file mode 100644
index 0000000000..8c826f54a9
--- /dev/null
+++ b/python/mach/docs/metrics.md
@@ -0,0 +1,55 @@
+<!-- AUTOGENERATED BY glean_parser.  DO NOT EDIT. -->
+
+# Metrics
+This document enumerates the metrics collected by this project using the [Glean SDK](https://mozilla.github.io/glean/book/index.html).
+This project may depend on other projects which also collect metrics.
+This means you might have to go searching through the dependency tree to get a full picture of everything collected by this project.
+
+# Pings
+
+ - [usage](#usage)
+
+
+## usage
+
+Sent when the mach invocation is completed (regardless of result). Contains information about the mach invocation that was made, its result, and some details about the current environment and hardware.
+
+
+This ping includes the [client id](https://mozilla.github.io/glean/book/user/pings/index.html#the-client_info-section).
+
+**Data reviews for this ping:**
+
+- <https://bugzilla.mozilla.org/show_bug.cgi?id=1291053#c34>
+
+**Bugs related to this ping:**
+
+- <https://bugzilla.mozilla.org/show_bug.cgi?id=1291053>
+
+The following metrics are added to the ping:
+
+| Name | Type | Description | Data reviews | Extras | Expiration | [Data Sensitivity](https://wiki.mozilla.org/Firefox/Data_Collection) |
+| --- | --- | --- | --- | --- | --- | --- |
+| mach.argv |[string_list](https://mozilla.github.io/glean/book/user/metrics/string_list.html) |Parameters provided to mach. Absolute paths are sanitized to be relative to one of a few key base paths, such as the "$topsrcdir", "$topobjdir", or "$HOME". For example: "/home/mozilla/dev/firefox/python/mozbuild" would be replaced with "$topsrcdir/python/mozbuild". If a valid replacement base path cannot be found, the path is replaced with "<path omitted>".  |[1](https://bugzilla.mozilla.org/show_bug.cgi?id=1291053#c34)||never | |
+| mach.command |[string](https://mozilla.github.io/glean/book/user/metrics/string.html) |The name of the mach command that was invoked, such as "build", "doc", or "try".  |[1](https://bugzilla.mozilla.org/show_bug.cgi?id=1291053#c34)||never | |
+| mach.duration |[timespan](https://mozilla.github.io/glean/book/user/metrics/timespan.html) |How long it took for the command to complete. |[1](https://bugzilla.mozilla.org/show_bug.cgi?id=1291053#c34)||never | |
+| mach.success |[boolean](https://mozilla.github.io/glean/book/user/metrics/boolean.html) |True if the mach invocation succeeded. |[1](https://bugzilla.mozilla.org/show_bug.cgi?id=1291053#c34)||never | |
+| mach.system.cpu_brand |[string](https://mozilla.github.io/glean/book/user/metrics/string.html) |CPU brand string from CPUID. |[1](https://bugzilla.mozilla.org/show_bug.cgi?id=1291053#c34)||never | |
+| mach.system.distro |[string](https://mozilla.github.io/glean/book/user/metrics/string.html) |The name of the operating system distribution. |[1](https://bugzilla.mozilla.org/show_bug.cgi?id=1655845#c3)||never | |
+| mach.system.distro_version |[string](https://mozilla.github.io/glean/book/user/metrics/string.html) |The high-level OS version. |[1](https://bugzilla.mozilla.org/show_bug.cgi?id=1655845#c3)||never | |
+| mach.system.logical_cores |[counter](https://mozilla.github.io/glean/book/user/metrics/counter.html) |Number of logical CPU cores present. |[1](https://bugzilla.mozilla.org/show_bug.cgi?id=1291053#c34)||never | |
+| mach.system.memory |[memory_distribution](https://mozilla.github.io/glean/book/user/metrics/memory_distribution.html) |Amount of system memory. |[1](https://bugzilla.mozilla.org/show_bug.cgi?id=1291053#c34)||never | |
+| mach.system.physical_cores |[counter](https://mozilla.github.io/glean/book/user/metrics/counter.html) |Number of physical CPU cores present. |[1](https://bugzilla.mozilla.org/show_bug.cgi?id=1291053#c34)||never | |
+| mozbuild.artifact |[boolean](https://mozilla.github.io/glean/book/user/metrics/boolean.html) |True if `--enable-artifact-builds`. |[1](https://bugzilla.mozilla.org/show_bug.cgi?id=1291053#c34)||never | |
+| mozbuild.ccache |[boolean](https://mozilla.github.io/glean/book/user/metrics/boolean.html) |True if `--with-ccache`. |[1](https://bugzilla.mozilla.org/show_bug.cgi?id=1291053#c34)||never | |
+| mozbuild.clobber |[boolean](https://mozilla.github.io/glean/book/user/metrics/boolean.html) |True if the build was a clobber/full build. |[1](https://bugzilla.mozilla.org/show_bug.cgi?id=1526072#c15)||never | |
+| mozbuild.compiler |[string](https://mozilla.github.io/glean/book/user/metrics/string.html) |The compiler type in use (CC_TYPE), such as "clang" or "gcc". |[1](https://bugzilla.mozilla.org/show_bug.cgi?id=1291053#c34)||never | |
+| mozbuild.debug |[boolean](https://mozilla.github.io/glean/book/user/metrics/boolean.html) |True if `--enable-debug`. |[1](https://bugzilla.mozilla.org/show_bug.cgi?id=1291053#c34)||never | |
+| mozbuild.icecream |[boolean](https://mozilla.github.io/glean/book/user/metrics/boolean.html) |True if icecream in use. |[1](https://bugzilla.mozilla.org/show_bug.cgi?id=1291053#c34)||never | |
+| mozbuild.opt |[boolean](https://mozilla.github.io/glean/book/user/metrics/boolean.html) |True if `--enable-optimize`. |[1](https://bugzilla.mozilla.org/show_bug.cgi?id=1291053#c34)||never | |
+| mozbuild.project |[string](https://mozilla.github.io/glean/book/user/metrics/string.html) |The project being built. |[1](https://bugzilla.mozilla.org/show_bug.cgi?id=1654084#c2)||never | |
+| mozbuild.sccache |[boolean](https://mozilla.github.io/glean/book/user/metrics/boolean.html) |True if ccache in use is sccache. |[1](https://bugzilla.mozilla.org/show_bug.cgi?id=1291053#c34)||never | |
+
+
+Data categories are [defined here](https://wiki.mozilla.org/Firefox/Data_Collection).
+
+<!-- AUTOGENERATED BY glean_parser.  DO NOT EDIT. -->
diff --git a/python/mach/docs/settings.rst b/python/mach/docs/settings.rst
new file mode 100644
index 0000000000..4daba37472
--- /dev/null
+++ b/python/mach/docs/settings.rst
@@ -0,0 +1,138 @@
+.. _mach_settings:
+
+========
+Settings
+========
+
+Mach can read settings in from a set of configuration files. These
+configuration files are either named ``machrc`` or ``.machrc`` and
+are specified by the bootstrap script. In mozilla-central, these files
+can live in ``~/.mozbuild`` and/or ``topsrcdir``.
+
+Settings can be specified anywhere, and used both by mach core or
+individual commands.
+
+
+Core Settings
+=============
+
+These settings are implemented by mach core.
+
+* alias - Create a command alias. This is useful if you want to alias a command to something else, optionally including some defaults. It can either be used to create an entire new command, or provide defaults for an existing one. For example:
+
+.. parsed-literal::
+
+    [alias]
+    mochitest = mochitest -f browser
+    browser-test = mochitest -f browser
+
+
+Defining Settings
+=================
+
+Settings need to be explicitly defined, along with their type,
+otherwise mach will throw when trying to access them.
+
+To define settings, use the :func:`~decorators.SettingsProvider`
+decorator in an existing mach command module. E.g:
+
+.. code-block:: python
+
+    from mach.decorators import SettingsProvider
+    from mozbuild.base import MachCommandBase
+
+    @SettingsProvider
+    class ArbitraryClassName(MachCommandBase):
+        config_settings = [
+            ('foo.bar', 'string', "A helpful description"),
+            ('foo.baz', 'int', "Another description", 0, {'choices': set([0,1,2])}),
+        ]
+
+``@SettingsProvider``'s must specify a variable called ``config_settings``
+that returns a list of tuples. Alternatively, it can specify a function
+called ``config_settings`` that returns a list of tuples.
+
+Each tuple is of the form:
+
+.. code-block:: python
+
+    ('<section>.<option>', '<type>', '<description>', default, extra)
+
+``type`` is a string and can be one of:
+string, boolean, int, pos_int, path
+
+``description`` is a string explaining how to define the settings and
+where they get used. Descriptions should ideally be multi-line paragraphs
+where the first line acts as a short description.
+
+``default`` is optional, and provides a default value in case none was
+specified by any of the configuration files.
+
+``extra`` is also optional and is a dict containing additional key/value
+pairs to add to the setting's metadata. The following keys may be specified
+in the ``extra`` dict:
+
+    * ``choices`` - A set of allowed values for the setting.
+
+Wildcards
+---------
+
+Sometimes a section should allow arbitrarily defined options from the user, such
+as the ``alias`` section mentioned above. To define a section like this, use ``*``
+as the option name. For example:
+
+.. parsed-literal::
+
+    ('foo.*', 'string', 'desc')
+
+This allows configuration files like this:
+
+.. parsed-literal::
+
+    [foo]
+    arbitrary1 = some string
+    arbitrary2 = some other string
+
+
+Finding Settings
+================
+
+You can see which settings are available as well as their description and
+expected values by running:
+
+.. parsed-literal::
+
+    ./mach settings  # or
+    ./mach settings --list
+
+
+Accessing Settings
+==================
+
+Now that the settings are defined and documented, they're accessible from
+individual mach commands from the mach command context.
+For example:
+
+.. code-block:: python
+
+    from mach.decorators import (
+        Command,
+        SettingsProvider,
+    )
+    from mozbuild.base import MachCommandBase
+
+    @SettingsProvider
+    class ExampleSettings(object):
+        config_settings = [
+            ('a.b', 'string', 'desc', 'default'),
+            ('foo.bar', 'string', 'desc',),
+            ('foo.baz', 'int', 'desc', 0, {'choices': set([0,1,2])}),
+        ]
+
+    @Command('command', category='misc',
+             description='Prints a setting')
+    def command(command_context):
+        settings = command_context._mach_context.settings
+        print(settings.a.b)
+        for option in settings.foo:
+            print(settings.foo[option])
diff --git a/python/mach/docs/telemetry.rst b/python/mach/docs/telemetry.rst
new file mode 100644
index 0000000000..2d185a970e
--- /dev/null
+++ b/python/mach/docs/telemetry.rst
@@ -0,0 +1,37 @@
+.. _mach_telemetry:
+
+==============
+Mach Telemetry
+==============
+
+`Glean <https://mozilla.github.io/glean/>`_ is used to collect telemetry, and uses the metrics
+defined in the ``metrics.yaml`` files in-tree.
+These files are all documented in a single :ref:`generated file here<metrics>`.
+
+.. toctree::
+   :maxdepth: 1
+
+   metrics
+
+Adding Metrics to a new Command
+===============================
+
+If you would like to submit telemetry metrics from your mach ``@Command``, you should take two steps:
+
+#. Parameterize your ``@Command`` annotation with ``metrics_path``.
+#. Use the ``command_context.metrics`` handle provided by ``MachCommandBase``
+
+For example::
+
+    METRICS_PATH = os.path.abspath(os.path.join(__file__, '..', '..', 'metrics.yaml'))
+
+    @Command('custom-command', metrics_path=METRICS_PATH)
+    def custom_command(command_context):
+        command_context.metrics.custom.foo.set('bar')
+
+Updating Generated Metrics Docs
+===============================
+
+When a ``metrics.yaml`` is added/changed/removed, :ref:`the metrics document<metrics>` will need to be updated::
+
+    ./mach doc mach-telemetry
diff --git a/python/mach/docs/usage.rst b/python/mach/docs/usage.rst
new file mode 100644
index 0000000000..a32b35395c
--- /dev/null
+++ b/python/mach/docs/usage.rst
@@ -0,0 +1,150 @@
+.. _mach_usage:
+
+==========
+User Guide
+==========
+
+Mach is the central entry point for most operations that can be performed in
+mozilla-central.
+
+
+Command Help
+------------
+
+To see an overview of all the available commands, run:
+
+.. code-block:: shell
+
+    $ ./mach help
+
+For more detailed information on a specific command, run:
+
+.. code-block:: shell
+
+    $ ./mach help <command>
+
+If a command has subcommands listed, you can see more details on the subcommand
+by running:
+
+.. code-block:: shell
+
+    $ ./mach help <command> <subcommand>
+
+Alternatively, you can pass ``-h/--help``. For example, all of the
+following are valid:
+
+.. code-block:: shell
+
+    $ ./mach help try
+    $ ./mach help try fuzzy
+    $ ./mach try -h
+    $ ./mach try fuzzy --help
+
+
+Tab Completion
+--------------
+
+There are commands built-in to ``mach`` that can generate a fast tab completion
+script for various shells. Supported shells are currently ``bash``, ``zsh`` and
+``fish``. These generated scripts will slowly become out of date over time, so
+you may want to create a cron task to periodically re-generate them.
+
+See below for installation instructions:
+
+Bash
+~~~~
+
+.. code-block:: shell
+
+    $ mach mach-completion bash -f _mach
+    $ sudo mv _mach /etc/bash_completion.d
+
+Bash (homebrew)
+~~~~~~~~~~~~~~~
+
+.. code-block:: shell
+
+    $ mach mach-completion bash -f $(brew --prefix)/etc/bash_completion.d/mach.bash-completion
+
+Zsh
+~~~
+
+.. code-block:: shell
+
+    $ mkdir ~/.zfunc
+    $ mach mach-completion zsh -f ~/.zfunc/_mach
+
+then edit ~/.zshrc and add:
+
+.. code-block:: shell
+
+    fpath+=~/.zfunc
+    autoload -U compinit && compinit
+
+You can use any directory of your choosing.
+
+Zsh (oh-my-zsh)
+~~~~~~~~~~~~~~~
+
+.. code-block:: shell
+
+    $ mkdir $ZSH/plugins/mach
+    $ mach mach-completion zsh -f $ZSH/plugins/mach/_mach
+
+then edit ~/.zshrc and add 'mach' to your enabled plugins:
+
+.. code-block:: shell
+
+    plugins(mach ...)
+
+Zsh (prezto)
+~~~~~~~~~~~~
+
+.. code-block:: shell
+
+    $ mach mach-completion zsh -f ~/.zprezto/modules/completion/external/src/_mach
+
+Fish
+~~~~
+
+.. code-block:: shell
+
+    $ ./mach mach-completion fish -f ~/.config/fish/completions/mach.fish
+
+Fish (homebrew)
+~~~~~~~~~~~~~~~
+
+.. code-block:: shell
+
+    $ ./mach mach-completion fish -f (brew --prefix)/share/fish/vendor_completions.d/mach.fish
+
+
+User Settings
+-------------
+
+Some mach commands can read configuration from a ``machrc`` file. The default
+location for this file is ``~/.mozbuild/machrc`` (you'll need to create it).
+This can also be set to a different location by setting the ``MACHRC``
+environment variable.
+
+For a list of all the available settings, run:
+
+.. code-block:: shell
+
+    $ ./mach settings
+
+The settings file follows the ``ini`` format, e.g:
+
+.. code-block:: ini
+
+    [alias]
+    eslint = lint -l eslint
+
+    [build]
+    telemetry = true
+
+    [try]
+    default = fuzzy
+
+
+.. _bash completion: https://searchfox.org/mozilla-central/source/python/mach/bash-completion.sh
diff --git a/python/mach/docs/windows-usage-outside-mozillabuild.rst b/python/mach/docs/windows-usage-outside-mozillabuild.rst
new file mode 100644
index 0000000000..6a034fd384
--- /dev/null
+++ b/python/mach/docs/windows-usage-outside-mozillabuild.rst
@@ -0,0 +1,124 @@
+==========================================
+Using Mach on Windows Outside MozillaBuild
+==========================================
+
+.. note::
+
+    These docs still require that you've followed the :ref:`Building Firefox On Windows` guide.
+
+`MozillaBuild <https://wiki.mozilla.org/MozillaBuild>`__ is required to build
+Firefox on Windows, because it provides necessary unix-y tools such as ``sh`` and ``awk``.
+
+Traditionally, to interact with Mach and the Firefox Build System, Windows
+developers would have to do so from within the MozillaBuild shell. This could be
+disadvantageous for two main reasons:
+
+1. The MozillaBuild environment is unix-y and based on ``bash``, which may be unfamiliar
+   for developers used to the Windows Command Prompt or Powershell.
+2. There have been long-standing stability issues with MozillaBuild - this is due to
+   the fragile interface point between the underlying "MSYS" tools and "native Windows"
+   binaries.
+
+It is now (experimentally!) possible to invoke Mach directly from other command line
+environments, such as Powershell, Command Prompt, or even a developer-managed MSYS2
+environment. Windows Terminal should work as well, for those on the "cutting edge".
+
+.. note::
+
+    If you're using a Cygwin-based environment such as MSYS2, it'll probably be
+    best to use the Windows-native version of Python (as described below) instead of a Python
+    distribution provided by the environment's package manager. Otherwise you'll likely run into
+    compatibility issues:
+
+    * Cygwin/MSYS Python will run into compatibility issues with Mach due to its unexpected Unix-y
+      conventions despite Mach assuming it's on a "Windows" platform. Additionally, there may
+      be performance issues.
+    * MinGW Python will encounter issues building native packages because they'll expect the
+      MSVC toolchain.
+
+.. warning::
+
+    This is only recommended for more advanced Windows developers: this work is experimental
+    and may run into unexpected failures!
+
+Following are steps for preparing Windows-native (Command Prompt/Powershell) usage of Mach:
+
+1. Install Python
+~~~~~~~~~~~~~~~~~
+
+Download Python from the `the official website <https://www.python.org/downloads/windows/>`__.
+
+.. note::
+
+    To avoid Mach compatibility issues with recent Python releases, it's recommended to install
+    the 2nd-most recent "major version". For example, at time of writing, the current modern Python
+    version is 3.10.1, so a safe version to install would be the most recent 3.9 release.
+
+You'll want to download the "Windows installer (64-bit)" associated with the release you've chosen.
+During installation, ensure that you check the "Add Python 3.x to PATH" option, otherwise you might
+`encounter issues running Mercurial <https://bz.mercurial-scm.org/show_bug.cgi?id=6635>`__.
+
+.. note::
+
+    Due to issues with Python DLL import failures with pip-installed binaries, it's not
+    recommended to use the Windows Store release of Python.
+
+2. Modify your PATH
+~~~~~~~~~~~~~~~~~~~
+
+The Python "user site-packages directory" needs to be added to your ``PATH`` so that packages
+installed via ``pip install --user`` (such as ``hg``) can be invoked from the command-line.
+
+1. From the Start menu, go to the Control Panel entry for "Edit environment variables
+   for your account".
+2. Double-click the ``Path`` row in the top list of variables. Click "New" to add a new item to
+   the list.
+3. In a Command Prompt window, resolve the Python directory with the command
+   ``python -c "import site; import os; print(os.path.abspath(os.path.join(site.getusersitepackages(), '..', 'Scripts')))"``.
+4. Paste the output into the new item entry in the "Edit environment variable" window.
+5. Click "New" again, and add the ``bin`` folder of MozillaBuild: probably ``C:\mozilla-build\bin``.
+6. Click "OK".
+
+3. Install Version Control System
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If you're using Mercurial, you'll need to install it to your Windows-native Python:
+
+.. code-block:: shell
+
+    pip3 install --user mercurial windows-curses
+
+If you're using Git with Cinnabar, follow its `setup instructions <https://github.com/glandium/git-cinnabar#setup>`__.
+
+4. Set Powershell Execution Policy
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If you're using Powershell, Windows will raise an error by default when you try to invoke
+``.\mach.ps1``:
+
+.. code::
+
+    .\mach : File <topsrcdir>\mach.ps1 cannot be loaded because running scripts is disabled on this system. For
+    more information, see about_Execution_Policies at https:/go.microsoft.com/fwlink/?LinkID=135170.
+    At line:1 char:1
+
+To work around this:
+
+1. From the Start menu, type in "Powershell", then right-click on the best match and click
+   "Run as administrator"
+2. Run the command ``Set-ExecutionPolicy RemoteSigned``
+3. Close the Administrator Powershell window, and open a regular Powershell window
+4. Go to your Firefox checkout (likely ``C:\mozilla-source\mozilla-unified``)
+5. Test the new execution policy by running ``.\mach bootstrap``. If it doesn't immediately fail
+   with the error about "Execution Policies", then the problem is resolved.
+
+Success!
+~~~~~~~~
+
+At this point, you should be able to invoke Mach and manage your version control system outside
+of MozillaBuild.
+
+.. tip::
+
+  `See here <https://crisal.io/words/2022/11/22/msys2-firefox-development.html>`__ for a detailed guide on
+  installing and customizing a development environment with MSYS2, zsh, and Windows Terminal.