Adding upstream version 110.0.1.upstream/110.0.1upstream

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

5 files changed, 667 insertions, 0 deletions

diff --git a/toolkit/components/normandy/docs/data-collection.rst b/toolkit/components/normandy/docs/data-collection.rst
new file mode 100644
index 0000000000..a2a2afb5a9
--- /dev/null
+++ b/toolkit/components/normandy/docs/data-collection.rst
@@ -0,0 +1,447 @@
+Data Collection
+===============
+This document describes the types of data that Normandy collects.
+
+Uptake
+------
+Normandy monitors the execution of recipes and reports to
+:ref:`telemetry` the amount of successful and failed runs. This data
+is reported using :ref:`telemetry/collection/uptake` under the
+``normandy`` namespace.
+
+Runner Status
+^^^^^^^^^^^^^
+Once per-fetch and execution of recipes, one of the following statuses is
+reported under the key ``normandy/runner``:
+
+.. data:: RUNNER_SUCCESS
+
+   :Telemetry value: success
+
+   The operation completed successfully. Individual failures with actions and
+   recipes may have been reported separately.
+
+.. data:: RUNNER_SERVER_ERROR
+
+   :Telemetry value: server_error
+
+   The data returned by the server when fetching the recipe is invalid in some
+   way.
+
+Action Status
+^^^^^^^^^^^^^
+For each action available from the Normandy service, one of the
+following statuses is reported under the key
+``normandy/action/<action name>``:
+
+.. data:: ACTION_NETWORK_ERROR
+
+   :Telemetry value: network_error
+
+   There was a network-related error while fetching actions
+
+.. data:: ACTION_PRE_EXECUTION_ERROR
+
+   :Telemetry value: custom_1_error
+
+   There was an error while running the pre-execution hook for the action.
+
+.. data:: ACTION_POST_EXECUTION_ERROR
+
+   :Telemetry value: custom_2_error
+
+   There was an error while running the post-execution hook for the action.
+
+.. data:: ACTION_SERVER_ERROR
+
+   :Telemetry value: server_error
+
+   The data returned by the server when fetching the action is invalid in some
+   way.
+
+.. data:: ACTION_SUCCESS
+
+   :Telemetry value: success
+
+   The operation completed successfully. Individual failures with recipes may
+   be reported separately.
+
+Recipe Status
+^^^^^^^^^^^^^
+For each recipe that is fetched and executed, one of the following statuses is
+reported under the key ``normandy/recipe/<recipe id>``:
+
+.. data:: RECIPE_ACTION_DISABLED
+
+   :Telemetry value: custom_1_error
+
+   The action for this recipe failed in some way and was disabled, so the recipe
+   could not be executed.
+
+.. data:: RECIPE_DIDNT_MATCH_FILTER
+
+   :Telemetry value: backoff
+
+   The recipe included a Jexl filter that the client did not match, so
+   the recipe was not executed.
+
+.. data:: RECIPE_EXECUTION_ERROR
+
+   :Telemetry value: apply_error
+
+   An error occurred while executing the recipe.
+
+.. data:: RECIPE_FILTER_BROKEN
+
+   :Telemetry value: content_error
+
+   An error occurred while evaluating the Jexl filter for this
+   recipe. Sometimes this represents a bug in the Jexl filter
+   parser/evaluator, such as in `bug 1477156
+   <https://bugzilla.mozilla.org/show_bug.cgi?id=1477156>`_, or it can
+   represent an error fetching some data that a Jexl recipe needs such
+   as `bug 1447804
+   <https://bugzilla.mozilla.org/show_bug.cgi?id=1447804>`_.
+
+.. data:: RECIPE_INVALID_ACTION
+
+   :Telemetry value: download_error
+
+   The action specified by the recipe was invalid and it could not be executed.
+
+.. data:: RECIPE_SUCCESS
+
+   :Telemetry value: success
+
+   The recipe was executed successfully.
+
+.. data:: RECIPE_SIGNATURE_INVALID
+
+   :Telemetry value: signature_error
+
+   Normandy failed to verify the signature of the recipe.
+
+
+Additionally, Normandy reports a :ref:`keyed scalar <Scalars>` to measure recipe
+freshness. This scalar is called ``normandy.recipe_freshness``, and it
+corresponds to the ``last_modified`` date of each recipe (using its ID
+as the key), reported as seconds since 1970 in UTC.
+
+
+Enrollment
+-----------
+Normandy records enrollment and unenrollment of users into studies, and
+records that data using :ref:`Telemetry Events <eventtelemetry>`. All data is stored in the
+``normandy`` category.
+
+
+Enrollment IDs
+^^^^^^^^^^^^^^
+
+Most Normandy telemetry carries an *enrollment ID*. These IDs are generated
+when Normandy enrolls the client in a change, be it a study, rollout, or
+something else. These enrollment IDs are used for the lifetime of that
+change, and are only used for that change (not shared between similar
+changes). Once a change ends (either via unenrollment, graduation, or another
+method) the enrollment ID should not be used again.
+
+When Telemetry upload is disabled, we must clear these enrollment IDs. This
+is done by replacing existing enrollment IDs with a filler value. New changes
+continue to receive a enrollment IDs as normal. The only thing that
+enrollment IDs are used for Telemetry, and so generated them while Telemetry
+is disabled is fine. They don't correlate to anything else, and won't be sent
+anywhere.
+
+Preference Studies
+^^^^^^^^^^^^^^^^^^
+Enrollment
+   method
+      The string ``"enroll"``
+   object
+      The string ``"preference_study"``
+   value
+      The name of the study (``recipe.arguments.slug``).
+   extra
+      branch
+         The name of the branch the user was assigned to (example:
+         ``"control"`` or ``"experiment"``).
+      experimentType
+         The type of preference experiment. Currently this can take
+         values "exp" and "exp-highpop", the latter being for
+         experiments targeting large numbers of users.
+      enrollmentId
+         A UUID that is unique to this users enrollment in this study. It
+         will be included in all future telemetry for this user in this
+         study.
+
+Enrollment Failed
+   method
+      The string ``"enrollFailed"``
+   object
+      The string ``"preference_study"``
+   value
+      The slug of the study (``recipe.arguments.slug``)
+   extra
+      reason
+         The reason for unenrollment. Possible values are:
+
+         * ``"invalid-branch"``: The recipe specifies an invalid preference
+           branch (not to be confused with the experiment branch). Valid values
+           are "default" and "user".
+      preferenceBranch
+         If the reason was ``"invalid-branch"``, the branch that was
+         specified, truncated to 80 characters.
+
+Unenrollment
+   method
+      The string ``"unenroll"``.
+   object
+      The string ``"preference_study"``.
+   value
+      The name of the study (``recipe.arguments.slug``).
+   extra
+      didResetValue
+         The string ``"true"`` if the preference was set back to its
+         original value, ``"false"`` if it was left as its current
+         value. This can happen when, for example, the user changes a
+         preference that was involved in a user-branch study.
+      reason
+         The reason for unenrollment. Possible values are:
+
+         * ``"recipe-not-seen"``: The recipe was no longer
+           applicable to this client This can be because the recipe
+           was disabled, or the user no longer matches the recipe's
+           filter.
+         * ``"unknown"``: A reason was not specified. This should be
+           considered a bug.
+      enrollmentId
+         The ID that was generated at enrollment.
+
+Unenroll Failed
+   method
+      The string ``"unenrollFailed"``.
+   object
+      The string ``"preference_study"``.
+   value
+      The name of the study (``recipe.arguments.slug``)
+   extra
+      enrollmentId
+         The ID that was generated at enrollment.
+      reason
+         A code describing the reason the unenroll failed. Possible values are:
+
+         * ``"does-not-exist"``: The system attempted to unenroll a study that
+           does not exist. This is a bug.
+         * ``"already-unenrolled"``: The system attempted to unenroll a study
+           that has already been unenrolled. This is likely a bug.
+      caller
+         On Nightly builds only, a string identifying the source of the requested stop.
+      originalReason
+         The code that would had been used for the unenrollment, had it not failed.
+
+Experimental Preference Changed
+   method
+      The string ``"expPrefChanged"``
+   object
+      The string ``"preference_study"``.
+   value
+      The name of the study (``recipe.arguments.slug``)
+   extra
+      enrollmentId
+         The ID that was generated at enrollment.
+      preferenceName
+         The name of the preference that changed. Note that the value of the
+         preference (old or new) is not given.
+      reason
+         A code describing the reason that Normandy detected the preference
+         change. Possible values are:
+
+         * ``"atEnroll"``: The preferences already had user value when the
+           experiment started.
+         * ``"sideload"``: A preference was changed while Normandy's observers
+           weren't active, likely while the browser was shut down.
+         * ``"observer"``: The preference was observed to change by Normandy at
+           runtime.
+
+Preference Rollouts
+^^^^^^^^^^^^^^^^^^^
+Enrollment
+   Sent when a user first enrolls in a rollout.
+
+   method
+      The string ``"enroll"``
+   object
+      The string ``"preference_rollout"``
+   value
+      The slug of the rollout (``recipe.arguments.slug``)
+   extra
+      enrollmentId
+         A UUID that is unique to this user's enrollment in this rollout. It
+         will be included in all future telemetry for this user in this
+         rollout.
+
+Enroll Failed
+   Sent when a user attempts to enroll in a rollout, but the enrollment process fails.
+
+   method
+      The string ``"enrollFailed"``
+   object
+      The string ``"preference_rollout"``
+   value
+      The slug of the rollout (``recipe.arguments.slug``)
+   extra
+      reason
+         A code describing the reason the unenroll failed. Possible values are:
+
+         * ``"invalid type"``: The preferences specified in the rollout do not
+           match the preferences built in to the browser. The represents a
+           misconfiguration of the preferences in the recipe on the server.
+         * ``"would-be-no-op"``: All of the preference specified in the rollout
+           already have the given values. This represents an error in targeting
+           on the server.
+         * ``"conflict"``: At least one of the preferences specified in the
+           rollout is already managed by another active rollout.
+      preference
+         For ``reason="invalid type"``, the first preference that was invalid.
+         For ``reason="conflict"``, the first preference that is conflicting.
+
+Update
+   Sent when the preferences specified in the recipe have changed, and the
+   client updates the preferences of the browser to match.
+
+   method
+      The string ``"update"``
+   object
+      The string ``"preference_rollout"``
+   value
+      The slug of the rollout (``recipe.arguments.slug``)
+   extra
+      previousState
+         The state the rollout was in before this update (such as ``"active"`` or ``"graduated"``).
+      enrollmentId
+         The ID that was generated at enrollment.
+
+Graduation
+   Sent when Normandy determines that further intervention is no longer
+   needed for this rollout. After this point, Normandy will stop making
+   changes to the browser for this rollout, unless the rollout recipe changes
+   to specify preferences different than the built-in.
+
+   method
+      The string ``"graduate"``
+   object
+      The string ``"preference_rollout"``
+   value
+      The slug of the rollout (``recipe.arguments.slug``)
+   extra
+      reason
+         A code describing the reason for the graduation. Possible values are:
+
+         * ``"all-prefs-match"``: All preferences specified in the rollout now
+           have built-in values that match the rollouts values.
+           ``"in-graduation-set"``: The browser has changed versions (usually
+           updated) to one that specifies this rollout no longer applies and
+           should be graduated regardless of the built-in preference values.
+           This behavior is controlled by the constant
+           ``PreferenceRollouts.GRADUATION_SET``.
+      enrollmentId
+         The ID that was generated at enrollment.
+
+Add-on Studies
+^^^^^^^^^^^^^^
+Enrollment
+   method
+      The string ``"enroll"``
+   object
+      The string ``"addon_study"``
+   value
+      The name of the study (``recipe.arguments.name``).
+   extra
+      addonId
+         The add-on's ID (example: ``"feature-study@shield.mozilla.com"``).
+      addonVersion
+         The add-on's version (example: ``"1.2.3"``).
+      enrollmentId
+         A UUID that is unique to this users enrollment in this study. It
+         will be included in all future telemetry for this user in this
+         study.
+
+Enroll Failure
+   method
+      The string ``"enrollFailed"``
+   object
+      The string ``"addon_study"``
+   value
+      The name of the study (``recipe.arguments.name``).
+   reason
+      A string containing the filename and line number of the code
+      that failed, and the name of the error thrown. This information
+      is purposely limited to avoid leaking personally identifiable
+      information. This should be considered a bug.
+
+Update
+   method
+      The string ``"update"``,
+   object
+      The string ``"addon_study"``,
+   value
+      The name of the study (``recipe.arguments.name``).
+   extra
+      addonId
+         The add-on's ID (example: ``"feature-study@shield.mozilla.com"``).
+      addonVersion
+         The add-on's version (example: ``"1.2.3"``).
+      enrollmentId
+         The ID that was generated at enrollment.
+
+Update Failure
+   method
+      The string ``"updateFailed"``
+   object
+      The string ``"addon_study"``
+   value
+      The name of the study (``recipe.arguments.name``).
+   extra
+      reason
+         A string containing the filename and line number of the code
+         that failed, and the name of the error thrown. This information
+         is purposely limited to avoid leaking personally identifiable
+         information. This should be considered a bug.
+      enrollmentId
+         The ID that was generated at enrollment.
+
+Unenrollment
+   method
+      The string ``"unenroll"``.
+   object
+      The string ``"addon_study"``.
+   value
+      The name of the study (``recipe.arguments.name``).
+   extra
+      addonId
+         The add-on's ID (example: ``"feature-study@shield.mozilla.com"``).
+      addonVersion
+         The add-on's version (example: ``"1.2.3"``).
+      reason
+         The reason for unenrollment. Possible values are:
+
+         * ``"install-failure"``: The add-on failed to install.
+         * ``"individual-opt-out"``: The user opted-out of this
+           particular study.
+         * ``"general-opt-out"``: The user opted-out of studies in
+           general.
+         * ``"recipe-not-seen"``: The recipe was no longer applicable
+           to this client. This can be because the recipe was
+           disabled, or the user no longer matches the recipe's
+           filter.
+         * ``"uninstalled"``: The study's add-on as uninstalled by some
+           mechanism. For example, this could be a user action or the
+           add-on self-uninstalling.
+         * ``"uninstalled-sideload"``: The study's add-on was
+           uninstalled while Normandy was inactive. This could be that
+           the add-on is no longer compatible, or was manually removed
+           from a profile.
+         * ``"unknown"``: A reason was not specified. This should be
+           considered a bug.
+      enrollmentId
+         The ID that was generated at enrollment.
diff --git a/toolkit/components/normandy/docs/execution-model.rst b/toolkit/components/normandy/docs/execution-model.rst
new file mode 100644
index 0000000000..6d5b16870c
--- /dev/null
+++ b/toolkit/components/normandy/docs/execution-model.rst
@@ -0,0 +1,95 @@
+Execution Model
+===============
+This document describes the execution model of the Normandy Client.
+
+The basic unit of instruction from the server is a *recipe*, which contains
+instructions for filtering, and arguments for a given action. See below for
+details.
+
+One iteration through all of these steps is called a *Normandy session*. This
+happens at least once every 6 hours, and possibly more often if Remote
+Settings syncs new changes.
+
+1. Fetching
+-----------
+A list of all active recipes is retrieved from Remote Settings, which has
+likely been syncing them in the background.
+
+2. Suitability
+--------------
+
+Once recipes have been retrieved, they go through several checks to determine
+their suitability for this client. Recipes contain information about which
+clients should execute the recipe. All recipes are processed by all clients,
+and all filtering happens in the client.
+
+For more information, see `the suitabilities docs <./suitabilities.html>`_.
+
+Signature
+~~~~~~~~~
+
+First, recipes are validated using a signature generated by Autograph_ that
+is included with the recipe. This signature validates both the contents of
+the recipe as well as its source.
+
+This signature is separate and distinct from the signing that happens on the
+Remote Settings collection. This provides additional assurance that this
+recipe is legitimate and intended to run on this client.
+
+.. _Autograph: https://github.com/mozilla-services/autograph
+
+Capabilities
+~~~~~~~~~~~~
+Next a recipe is checked for compatibility using *capabilities*.
+Capabilities are simple strings, such as ``"action:show-heartbeat"``. A
+recipe contains a list of required capabilities, and the Normandy Client has
+a list of capabilities that it supports. If any of the capabilities required
+by the recipe are not compatible with the client, then the recipe does not
+execute.
+
+Capabilities are used to avoid running recipes on a client that are so
+incompatible as to be harmful. For example, some changes to filter expression
+handling cannot be detected by filter expressions, and so older clients that
+receive filters using these new features would break.
+
+.. note::
+
+    Capabilities were first introduced in Firefox 70. Clients prior to this
+    do not check capabilities, and run all recipes provided. To accommodate
+    this, the server splits recipes into two Remote Settings collections,
+    ``normandy-recipes``, and ``normandy-recipes-capabilities``. Clients
+    prior to Firefox 70 use the former, whereas Firefox 70 and above use the
+    latter. Recipes that only require "baseline" capabilities are published
+    to both, and those that require advanced capabilities are only published
+    to the capabilities aware collection.
+
+Filter Expressions
+~~~~~~~~~~~~~~~~~~
+Finally the recipe's filter expression is checked. Filter expressions are
+written in an expression language named JEXL_ that is similar to JavaScript,
+but far simpler. It is intended to be as safe to evaluate as possible.
+
+.. _JEXL: https://github.com/mozilla/mozjexl
+
+Filters are evaluated in a context that contains details about the client
+including browser versions, installed add-ons, and Telemetry data. Filters
+have access to "transforms" which are simple functions that can do things like
+check preference values or parse strings into ``Date`` objects. Filters don't
+have access to change any state in the browser, and are generally
+idempotent. However, filters are *not* considered to be "pure functions",
+because they have access to state that may change, such as time and location.
+
+3. Execution
+------------
+After a recipe's suitability is determined, that recipe is executed. The
+recipe specifies an *action* by name, as well as arguments to pass to that
+action. The arguments are validated against an expected schema.
+
+All action have a pre- and post-step that runs once each Normandy session.
+The pre-step is run before any recipes are executed, and once the post-step
+is executed, no more recipes will be executed on that action in this session.
+
+Each recipe is passed to the action, along with its suitability. Individual
+actions have their own semantics about what to do with recipes. Many actions
+maintain their own life cycle of events for new recipes, existing recipes,
+and recipes that stop applying to this client.
diff --git a/toolkit/components/normandy/docs/index.rst b/toolkit/components/normandy/docs/index.rst
new file mode 100644
index 0000000000..eb77000ee3
--- /dev/null
+++ b/toolkit/components/normandy/docs/index.rst
@@ -0,0 +1,30 @@
+.. _components/normandy:
+
+====================
+Shield Recipe Client
+====================
+
+Normandy (aka the Shield Recipe Client) is a targeted change control
+system, allowing small changes to be made within a release of Firefox,
+such as studies.
+
+It downloads recipes and actions from :ref:`Remote Settings <services/remotesettings>`
+and then executes them.
+
+.. note::
+
+   Previously, the recipes were fetched from the `recipe server`_, but in `Bug 1513854`_
+   the source was changed to *Remote Settings*. The cryptographic signatures are verified
+   at the *Remote Settings* level (integrity) and at the *Normandy* level
+   (authenticity of publisher).
+
+.. _recipe server: https://github.com/mozilla/normandy/
+.. _Bug 1513854: https://bugzilla.mozilla.org/show_bug.cgi?id=1513854
+
+.. toctree::
+   :maxdepth: 1
+
+   data-collection
+   execution-model
+   suitabilities
+   services
diff --git a/toolkit/components/normandy/docs/services.rst b/toolkit/components/normandy/docs/services.rst
new file mode 100644
index 0000000000..6bec61a751
--- /dev/null
+++ b/toolkit/components/normandy/docs/services.rst
@@ -0,0 +1,22 @@
+Normandy Services
+=================
+The Normandy Client relies on several external services for correct operation.
+
+Normandy Server
+---------------
+This is the place where recipes are created, edited and approved. Normandy
+Client interacts with it to get an index of services, and to fetch extensions
+and their metadata for add-on studies and rollout.
+
+Remote Settings
+---------------
+This is the primary way that recipes are loaded from the internet by
+Normandy. It manages keeping the local list of recipes on the client up to
+date and notifying Normandy Client of changes.
+
+Classify Client
+---------------
+This is a service that helps Normandy with filtering. It determines the
+region a user is in, based on IP. It also includes an up-to-date time and
+date. This allows Normandy to perform location and time based filtering
+without having to rely on the local clock.
diff --git a/toolkit/components/normandy/docs/suitabilities.rst b/toolkit/components/normandy/docs/suitabilities.rst
new file mode 100644
index 0000000000..b199fc2938
--- /dev/null
+++ b/toolkit/components/normandy/docs/suitabilities.rst
@@ -0,0 +1,73 @@
+Suitabilities
+=============
+
+When Normandy's core passes a recipe to an action, it also passes a
+*suitability*, which is the result of evaluating a recipe's filters,
+compatibility, signatures, and other checks. This gives actions more
+information to make decisions with, which is especially important for
+experiments.
+
+Temporary errors
+----------------
+Some of the suitabilities below represent *temporary errors*. These could be
+caused by infrastructure problems that prevent the recipe from working
+correctly, or are otherwise not the fault of the recipe itself. These
+suitabilities should not immediately cause a change in state. If the problem
+persists, then eventually it should be considered permanent and state should
+be changed.
+
+In the case of a permanent failure, action such as unenrollment should happen
+immediately. For temporary failures, that action should be delayed until the
+failure persists longer than some threshold. It is up to individual actions
+to track and manage this transition.
+
+List of Suitabilities
+---------------------
+
+``FILTER_MATCH``
+~~~~~~~~~~~~~~~~
+All checks have passed, and the recipe is suitable to execute in this client.
+Experiments and Rollouts should enroll or update. Heartbeats should be shown
+to the user, etc.
+
+``SIGNATURE_ERROR``
+~~~~~~~~~~~~~~~~~~~
+The recipe's signature is not valid. If any action is taken this recipe
+should be treated with extreme suspicion.
+
+This should be considered a temporary error, because it may be related to
+server errors, local clocks, or other temporary problems.
+
+``CAPABILITIES_MISMATCH``
+~~~~~~~~~~~~~~~~~~~~~~~~~
+The recipe requires capabilities that this recipe runner does not have. Use
+caution when interacting with this recipe, as it may not match the expected
+schema.
+
+This should be considered a permanent error, because it is the result of a
+choice made on the server.
+
+``FILTER_MISMATCH``
+~~~~~~~~~~~~~~~~~~~
+This client does not match the recipe's filter, but it is otherwise a
+suitable recipe.
+
+This should be considered a permanent error, since the filter explicitly does
+not match the client.
+
+``FILTER_ERROR``
+~~~~~~~~~~~~~~~~
+There was an error while evaluating the filter. It is unknown if this client
+matches this filter. This may be temporary, due to network errors, or
+permanent due to syntax errors.
+
+This should be considered a temporary error, because it may be the result of
+infrastructure, such as `Classify Client <./services.html#classify-client>`_,
+temporarily failing.
+
+``ARGUMENTS_INVALID``
+~~~~~~~~~~~~~~~~~~~~~
+The arguments of the recipe do not match the expected schema for the named
+action.
+
+This should be considered a permanent error, since the arguments are generally validated by the server. This likely represents an unrecogonized compatibility error.