Adding upstream version 110.0.1.upstream/110.0.1upstream

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

7 files changed, 518 insertions, 0 deletions

diff --git a/toolkit/mozapps/extensions/docs/AMRemoteSettings-JSONSchema.json b/toolkit/mozapps/extensions/docs/AMRemoteSettings-JSONSchema.json
new file mode 100644
index 0000000000..20d905f090
--- /dev/null
+++ b/toolkit/mozapps/extensions/docs/AMRemoteSettings-JSONSchema.json
@@ -0,0 +1,41 @@
+{
+  "type": "object",
+  "required": [
+    "id"
+  ],
+  "properties": {
+    "id": {
+      "type": "string",
+      "default": "AddonManagerSettings",
+      "description": "The default id should NOT be changed, unless there is a specific need to create separate collection entries which target or exclude specific Firefox versions."
+    },
+    "installTriggerDeprecation": {
+      "$ref": "#/definitions/installTriggerDeprecation",
+      "optional": true
+    },
+    "filter_expression": {
+      "type": "string",
+      "description": "This is NOT directly used by AMRemoteSettings, but has special functionality in Remote Settings.\nSee https://remote-settings.readthedocs.io/en/latest/target-filters.html#how",
+      "optional": true
+    }
+  },
+  "definitions": {
+    "installTriggerDeprecation": {
+      "type": "object",
+      "properties": {
+        "extensions.InstallTrigger.enabled": {
+          "type": "boolean",
+          "default": false,
+          "description": "Show/Hide the InstallTrigger global completely (both the global and its methods will not be accessible anymore). IMPORTANT: The webcompat team should be consulted before turning this to false, because it may also potentially impact UA detection for some websites."
+        },
+        "extensions.InstallTriggerImpl.enabled": {
+          "type": "boolean",
+          "default": false,
+          "description": "Show/Hide the InstallTrigger methods. The InstallTrigger global will remain visible but set to null."
+        }
+      },
+      "description": "These settings control the visibility of the InstallTrigger global and its methods.",
+      "additionalProperties": false
+    }
+  }
+}
diff --git a/toolkit/mozapps/extensions/docs/AMRemoteSettings-UISchema.json b/toolkit/mozapps/extensions/docs/AMRemoteSettings-UISchema.json
new file mode 100644
index 0000000000..513449da92
--- /dev/null
+++ b/toolkit/mozapps/extensions/docs/AMRemoteSettings-UISchema.json
@@ -0,0 +1,10 @@
+{
+  "installTriggerDeprecation": {
+    "extensions.InstallTrigger.enabled": {
+      "ui:widget": "radio"
+    },
+    "extensions.InstallTriggerImpl.enabled": {
+      "ui:widget": "radio"
+    }
+  }
+}
diff --git a/toolkit/mozapps/extensions/docs/AMRemoteSettings-overview.rst b/toolkit/mozapps/extensions/docs/AMRemoteSettings-overview.rst
new file mode 100644
index 0000000000..e22c1fcc12
--- /dev/null
+++ b/toolkit/mozapps/extensions/docs/AMRemoteSettings-overview.rst
@@ -0,0 +1,162 @@
+AMRemoteSettings Overview
+=========================
+
+``AMRemoteSettings`` is a singleton that is responsible for fetching data from a RemoteSettings collection named
+``"main/addons-manager-settings"`` to remotely control a set of Add-ons related preferences (hardcoded inside
+the AMRemoteSettings definition part of a given Firefox release).
+
+``AMRemoteSettings`` will process the collection data when the RemoteSettings ``"sync"`` event is fired and on
+late browser startup if there is already some data stored locally in the collection (and so if the pref value is
+manually changed, then it would be set again to the value stored in the RemoteSettings collection data, either
+on the next browser startup or on the next time the ``"sync"`` event is fired again).
+
+.. warning::
+  Any about:config preference that AMRemoteSettings singleton sets while processing the RemoteSettings collection data
+  are **never cleared automatically** (e.g. if in a future Firefox version AMRemoteSettings does not handle a particular
+  settings anymore, or if the RemoteSettings entry is removed from the collection on the RemoteSettings service side).
+
+  In practice this makes this feature a good choice when the remotely controlled about:config preferences is related to:
+
+  * Firefox Add-ons features in the process of being deprecated and then fully removed in a future Firefox release
+
+  * or controlled settings that are never removed from the ``"main/addons-manager-settings"`` collection.
+
+  In general before changing controlled settings in the ``"main/addons-manager-settings"`` collection it is advisable to:
+
+  * make sure to default value for the related about:config preferences has been changed in mozilla-central first,
+    and to let it ride the release train and get some backing time on release;
+
+  * only after that consider changing the value controlled by ``"main/addons-manager-settings"`` collection,
+    to set the same value on older Firefox releases where the previous default value was set.
+
+The ``AMRemoteSettings`` singleton queries RemoteSettings and processes all the entries got from the
+``"main/addons-manager-settings"`` collection, besides entries that may be filtered out by RemoteSettings based on
+the ``"filter_expression"`` property (See https://remote-settings.readthedocs.io/en/latest/target-filters.html#how).
+
+Each of the entries created in the ``"main/addons-manager-settings"`` collection and is expected to match the JSONSchema
+described in the :ref:`JSON Schema section below<JSON Schema>`.
+
+For each entry found in the collection, only the properties that are explicitly included in the
+``AMRemoteSettings.RS_ENTRIES_MAP`` object are going to be processed (e.g. new Firefox versions may have introduced new
+ones that older Firefox version will just ignore):
+
+* Each of the ``AMRemoteSettings.RS_ENTRIES_MAP`` properties:
+
+  * represents a group of settings (e.g. the property named  ``"installTriggerDeprecation"``) is responsible of controlling
+    about:config preferences related to the InstallTrigger deprecation
+
+  * is set to an array of string, which lists the about:config preferences names that can actually be controlled by the
+    related group of settings (e.g. ``"installTriggerDeprecation"`` can only control two preferences,
+    ``"extensions.InstallTrigger.enabled"`` and ``"extensions.InstallTriggerImpl.enabled"``, that are controlling the
+    InstallTrigger and InstallTrigger's methods availability).
+
+.. warning::
+  Any other about:config preference names that are not listed explicitly in the ``AMRemoteSettings.RS_ENTRIES_MAP`` config
+  is expected to be ignored, even if allowed by a new version of the collection's JSONSchema (this is the expected behavior
+  and prevents the introduction of unexpected behaviors on older Firefox versions that may not be expecting new settings groups
+  that may be introduced in Firefox releases that followed it).
+
+  Any about:config preference with an unexpected value type is going to be ignored as well (look to the ``AMRemoteSettings.processEntries``
+  to confirm which preferences values types are already expected and handled accordingly).
+
+.. _Controlled Settings Groups:
+
+AMRemoteSettings - Controlled Settings Groups
+=============================================
+
+installTriggerDeprecation
+-------------------------
+
+Group of settings to control InstallTrigger deprecation (Bug 1754441)
+
+- **extensions.InstallTrigger.enabled** (boolean), controls the availability of the InstallTrigger global:
+
+  - Turning this to false will be hiding it completely
+
+    .. note::
+      Turning this off can potentially impact UA detection on website being using it to detect
+      Firefox. The WebCompat team should be consulted before setting this to `false` by default in
+      Nightly or across all Firefox >= 100 versions through the ``"addons-manager-settings"``
+      RemoteSettings collection).
+
+- **extensions.InstallTriggerImpl.enabled** (boolean): controls the availability of the InstallTrigger methods:
+
+  - Turning this to false will hide all the InstallTrigger implementation, preventing using it to
+    trigger the addon install flow, while the InstallTrigger global will still exists but be set to null.
+
+How to define new remotely controlled settings
+----------------------------------------------
+
+.. note::
+  Please update the :ref:`JSON Schema` and :ref:`UI Schema` in this documentation page updated when the ones used on the
+  RemoteSettings service side have been changed, and document new controlled settings in the section :ref:`Controlled Settings Groups`.
+
+* Confirm that the :ref:`JSON Schema` and :ref:`UI Schema` included in this page are in sync with the one actually used on the
+  RemoteSettings service side, and use it as the starting point to update it to include a new type on remotely controlled setting:
+
+  * choose a new unique string for the group of settings to be used in the ``definitions`` and ``properties``
+    objects (any that isn't already used in the existing JSON Schema ``definitions``), possibly choosing a name
+    that helps to understand what the purpose of the entry.
+
+  * add a new JSONSchema for the new group of settings in the ``definitions`` property
+
+    * each of the properties included in the new definition should be named after the name of the about:config pref
+      being controlled, their types should match the type expected by the pref (e.g. ``"boolean"`` for a boolean preference).
+
+    * make sure to add a description property to the definition and to each of the controlled preferences, which should
+      describe what is the settings group controlling and what is the expected behavior on the values allowed.
+
+* Add a new entry to ``"AMRemoteSettings.RS_ENTRIES_MAP"`` with the choosen ``"id"`` as its key and
+  the array of the about:config preferences names are its value.
+
+* If the value type of a controlled preference isn't yet supported, the method ``AMRemoteSettings.processEntries`` has to be
+  updated to handle the new value type (otherwise the preference value will be just ignored).
+
+* Add a new test to cover the expected behaviors on the new remotely controlled settings, the following RemoteSettings
+  documentation page provides some useful pointers:
+  * https://firefox-source-docs.mozilla.org/services/settings/index.html#unit-tests
+
+* Refer to the RemoteSettings docs for more details about managing the JSONSchema for the ``"main/addons-manager-settings"``
+  collection and how to test it interactively in a Firefox instance:
+  * https://remote-settings.readthedocs.io/en/latest/getting-started.html
+  * https://firefox-source-docs.mozilla.org/services/settings/index.html#create-new-remote-settings
+  * https://firefox-source-docs.mozilla.org/services/settings/index.html#remote-settings-dev-tools
+
+.. _JSON Schema:
+
+AMRemoteSettings - JSON Schema
+==============================
+
+The entries part of the ``"addons-manager-settings"`` collection are validated using a JSON Schema structured as follows:
+
+* The mandatory ``"id"`` property
+  * defaults to `"AddonManagerSettings"` (which enforces only one entry in the collection as the preferred use case)
+  * **should NOT be changed unless there is a specific need to create separate collection entries which target or exclude specific Firefox versions.**
+  * when changed and multiple entries are created in this collection, it is advisable to:
+
+    * set the id to a short string value that make easier to understand the purpose of the additional entry in the collection
+      (eg. `AddonManagerSettings-fx98-99` for an entry created that targets Firefox 98 and 99)
+    * make sure only one applied to each targeted Firefox version ranges, or at least that each entry is controlling a different settings group
+
+* Each supported group of controlled settings is described by its own property (e.g. ``"installTriggerDeprecation"``)
+
+  * JSON Schema for each group of settings is defined in an entry of the ``"definitions"`` property.
+
+  * Each group of settings is contained it its own entry in ``"properties"``, named as the entry added to the ``"definitions"``)
+    and referencing (using ``"$ref"``) the related definition
+
+.. literalinclude :: ./AMRemoteSettings-JSONSchema.json
+   :language: json
+
+UI Schema
+---------
+
+In addition to the JSON Schema, a separate json called ``"UI schema"`` is associated to the ``"addons-manager-settings"`` collection,
+and it can be used to customize the form auto-generated based on the JSONSchema data.
+
+.. note::
+  Extending this schema is only needed if it can help to make the RemoteSettings collection easier to manage
+  and less error prone.
+
+.. literalinclude :: ./AMRemoteSettings-UISchema.json
+   :language: json
diff --git a/toolkit/mozapps/extensions/docs/AMRemoteSettings.rst b/toolkit/mozapps/extensions/docs/AMRemoteSettings.rst
new file mode 100644
index 0000000000..a555f426cc
--- /dev/null
+++ b/toolkit/mozapps/extensions/docs/AMRemoteSettings.rst
@@ -0,0 +1,5 @@
+AMRemoteSettings Reference
+==========================
+
+.. js:autoclass:: AMRemoteSettings
+  :members:
diff --git a/toolkit/mozapps/extensions/docs/AddonManager.rst b/toolkit/mozapps/extensions/docs/AddonManager.rst
new file mode 100644
index 0000000000..9a5db82830
--- /dev/null
+++ b/toolkit/mozapps/extensions/docs/AddonManager.rst
@@ -0,0 +1,4 @@
+AddonManager Reference
+======================
+.. js:autoclass:: AddonManager
+  :members:
diff --git a/toolkit/mozapps/extensions/docs/SystemAddons.rst b/toolkit/mozapps/extensions/docs/SystemAddons.rst
new file mode 100644
index 0000000000..a8524a44b6
--- /dev/null
+++ b/toolkit/mozapps/extensions/docs/SystemAddons.rst
@@ -0,0 +1,275 @@
+System Add-ons Overview
+=======================
+
+System add-ons are a method for shipping extensions to Firefox that:
+
+* are hidden from the about:addons UI
+* cannot be user disabled
+* can be updated restartlessly based on criteria Mozilla sets
+
+Generally these are considered to be built-in features to Firefox, and the
+fact that they are extensions and can be updated restartlessly are implementation
+details as far as users are concerned.
+
+If you'd like to ship an add-on with Firefox or as an update (either to an existing
+feature or as a "hotfix" to patch critical problems in the wild) please contact the
+GoFaster team: https://mail.mozilla.org/listinfo/gofaster
+
+The add-ons themselves are either legacy Firefox add-ons or WebExtensions.
+They must be:
+
+* restartless
+* multi-process compatible
+
+Other than these restrictions there is nothing special or different about
+the extensions themselves.
+
+It is possible to override an installed system add-on by installing a different add-on
+with the same ID into a higher priority location.
+
+Available locations, starting from the highest priority include:
+
+1) temporary install (about:debugging)
+2) normal user install into profile (about:addons or AMO/TestPilot/etc.)
+3) system add-on updates
+4) built-in system add-ons
+
+This makes it possible for a developer or user to override a system add-on
+by installing an add-on with the same ID from AMO or TestPilot or as a temporary
+add-on.
+
+Default, built-in system add-ons
+--------------------------------
+
+The set of **default** system add-ons are checked into `mozilla-central` under
+`./browser/extensions`. These get placed into the `features` directory of the
+application directory at build time.
+
+System add-on updates
+---------------------
+
+System add-on **updates** are served via Mozilla's Automatic Update Service
+(AUS, aka `Balrog`_). These are installed into the users profile under the `features`
+directory.
+
+Updates must be specifically signed by Mozilla - the signature that addons.mozilla.org
+uses will not work for system add-ons.
+
+As noted above, these updates may override a built-in system add-on, or they may
+be a new install. Updates are always served as a set - if any add-on in the set
+fails to install or upgrade, then the whole set fails. This is to leave Firefox
+in a consistent state.
+
+System add-on updates are removed when the Firefox application version changes,
+to avoid compatibility problems - for instance a user downgrading to an earlier
+version of Firefox than the update supports will end up with a disabled update
+rather than falling back to the built-in version.
+
+Firefox System Add-on Update Protocol
+=====================================
+This section describes the protocol that Firefox uses when retrieving updates
+from AUS, and the expected behavior of Firefox based on the updater service's response.
+
+.. _Balrog: https://wiki.mozilla.org/Balrog
+
+Update Request
+--------------
+To determine what updates to install, Firefox makes an HTTP **GET** request to
+AUS once a day via a URL of the form::
+
+  https://aus5.mozilla.org/update/3/SystemAddons/%VERSION%/%BUILD_ID%/%BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/update.xml
+
+The path segments surrounded by ``%`` symbols are variable fields that Firefox
+fills in with information about itself and the environment it's running in:
+
+``VERSION``
+  Firefox version number
+``BUILD_ID``
+  Build ID
+``BUILD_TARGET``
+  Build target
+``LOCALE``
+  Build locale
+``CHANNEL``
+  Update channel
+``OS_VERSION``
+  OS Version
+``DISTRIBUTION``
+  Firefox Distribution
+``DISTRIBUTION_VERSION``
+  Firefox Distribution version
+
+Update Response
+---------------
+AUS should respond with an XML document that looks something like this:
+
+.. code-block:: xml
+
+  <?xml version="1.0"?>
+  <updates>
+    <addons>
+      <addon id="flyweb@mozilla.org" URL="https://ftp.mozilla.org/pub/system-addons/flyweb/flyweb@mozilla.org-1.0.xpi" hashFunction="sha512" hashValue="abcdef123" size="1234" version="1.0"/>
+      <addon id="pocket@mozilla.org" URL="https://ftp.mozilla.org/pub/system-addons/pocket/pocket@mozilla.org-1.0.xpi" hashFunction="sha512" hashValue="abcdef123" size="1234" version="1.0"/>
+    </addons>
+  </updates>
+
+* The root element is ``<updates>``, used for all updater responses.
+* The only child of ``<updates>`` is ``<addons>``, which represents a list of
+  system add-ons to update.
+* Within ``<addons>`` are several ``<addon>`` tags, each one corresponding to a
+  system add-on to update.
+
+``<addon>`` tags **must** have the following attributes:
+
+``id``
+  The extension ID
+``URL``
+  URL to a signed XPI of the specified add-on version to download
+``hashFunction``
+  Identifier of the hash function used to generate the hashValue attribute.
+``hashValue``
+  Hash of the XPI file linked from the URL attribute, calculated using the function specified in the hashValue attribute.
+``size``
+  Size (in bytes) of the XPI file linked from the URL attribute.
+``version``
+  Version number of the add-on
+
+Update Behavior
+---------------
+After receiving the update response, Firefox modifies the **update** add-ons
+according to the following algorithm:
+
+1. If the ``<addons>`` tag is empty (``<addons></addons>``) in the response,
+   **remove all system add-on updates**.
+2. If no add-ons were specified in the response (i.e. the ``<addons>`` tag
+   is not present), do nothing and finish.
+3. If the **update** add-on set is equal to the set of add-ons specified in the
+   update response, do nothing and finish.
+4. If the set of **default** add-ons is equal to the set of add-ons specified in
+   the update response, remove all the  **update** add-ons and finish.
+5. Download each add-on specified in the update response and store them in the
+   "downloaded add-on set". A failed download **must** abort the entire system
+   add-on update.
+6. Validate the downloaded add-ons. The following **must** be true for all
+   downloaded add-ons, or the update process is aborted:
+
+   a. The ID and version of the downloaded add-on must match the specified ID or
+      version in the update response.
+   b. The hash provided in the update response must match the downloaded add-on
+      file.
+   c. The downloaded add-on file size must match the size given in the update
+      response.
+   d. The add-on must be compatible with Firefox (i.e. it must not be for a
+      different application, such as Thunderbird).
+   e. The add-on must be packed (i.e. be an XPI file).
+   f. The add-on must be restartless.
+   g. The add-on must be signed by the system add-on root certificate.
+
+6. Once all downloaded add-ons are validated, install them into the profile
+   directory as part of the **update** set.
+
+Notes on the update process:
+
+* Add-ons are considered "equal" if they have the same ID and version number.
+
+Examples
+--------
+The follow section describes common situations that we have or expect to run
+into and how the protocol described above handles them.
+
+For simplicity, unless otherwise specified, all examples assume that there are
+two system add-ons in existence: **FlyWeb** and **Pocket**.
+
+Basic
+~~~~~
+A user has Firefox 45, which shipped with FlyWeb 1.0 and Pocket 1.0. We want to
+update users to FlyWeb 2.0. AUS sends out the following update response:
+
+.. code-block:: xml
+
+  <updates>
+    <addons>
+      <addon id="flyweb@mozilla.org" URL="https://ftp.mozilla.org/pub/system-addons/flyweb/flyweb@mozilla.org-2.0.xpi" hashFunction="sha512" hashValue="abcdef123" size="1234" version="2.0"/>
+      <addon id="pocket@mozilla.org" URL="https://ftp.mozilla.org/pub/system-addons/pocket/pocket@mozilla.org-1.0.xpi" hashFunction="sha512" hashValue="abcdef123" size="1234" version="1.0"/>
+    </addons>
+  </updates>
+
+Firefox will download FlyWeb 2.0 and Pocket 1.0 and store them in the profile directory.
+
+Missing Add-on
+~~~~~~~~~~~~~~
+A user has Firefox 45, which shipped with FlyWeb 1.0 and Pocket 1.0. We want to
+update users to FlyWeb 2.0, but accidentally forget to specify Pocket in the
+update response. AUS sends out the following:
+
+.. code-block:: xml
+
+  <updates>
+    <addons>
+      <addon id="flyweb@mozilla.org" URL="https://ftp.mozilla.org/pub/system-addons/flyweb/flyweb@mozilla.org-2.0.xpi" hashFunction="sha512" hashValue="abcdef123" size="1234" version="2.0"/>
+    </addons>
+  </updates>
+
+Firefox will download FlyWeb 2.0 and store it in the profile directory. Pocket
+1.0 from the **default** location will be used.
+
+Remove all system add-on updates
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+A response from AUS with an empty add-on set will *remove all system add-on
+updates*:
+
+.. code-block:: xml
+
+  <updates>
+    <addons></addons>
+  </updates>
+
+Rollout
+~~~~~~~
+A user has Firefox 45, which shipped with FlyWeb 1.0 and Pocket 1.0. We want to
+rollout FlyWeb 2.0 at a 10% sample rate. 10% of the time, AUS sends out:
+
+.. code-block:: xml
+
+  <updates>
+    <addons>
+      <addon id="flyweb@mozilla.org" URL="https://ftp.mozilla.org/pub/system-addons/flyweb/flyweb@mozilla.org-2.0.xpi" hashFunction="sha512" hashValue="abcdef123" size="1234" version="2.0"/>
+      <addon id="pocket@mozilla.org" URL="https://ftp.mozilla.org/pub/system-addons/pocket/pocket@mozilla.org-1.0.xpi" hashFunction="sha512" hashValue="abcdef123" size="1234" version="1.0"/>
+    </addons>
+  </updates>
+
+With this response, Firefox will download Pocket 1.0 and FlyWeb 2.0 and install
+them into the profile directory.
+
+The other 90% of the time, AUS sends out an empty response:
+
+.. code-block:: xml
+
+  <updates></updates>
+
+With the empty response, Firefox will not make any changes. This means users who
+haven’t seen the 10% update response will stay on FlyWeb 1.0, and users who have
+seen it will stay on FlyWeb 2.0.
+
+Once we’re happy with the rollout and want to switch to 100%, AUS will send the
+10% update response to 100% of users, upgrading everyone to FlyWeb 2.0.
+
+Rollback
+~~~~~~~~
+This example continues from the “Rollout” example. If, during the 10% rollout,
+we find a major issue with FlyWeb 2.0, we want to roll all users back to FlyWeb 1.0.
+AUS sends out the following:
+
+.. code-block:: xml
+
+  <updates>
+    <addons>
+      <addon id="flyweb@mozilla.org" URL="https://ftp.mozilla.org/pub/system-addons/flyweb/flyweb@mozilla.org-1.0.xpi" hashFunction="sha512" hashValue="abcdef123" size="1234" version="1.0"/>
+      <addon id="pocket@mozilla.org" URL="https://ftp.mozilla.org/pub/system-addons/pocket/pocket@mozilla.org-1.0.xpi" hashFunction="sha512" hashValue="abcdef123" size="1234" version="1.0"/>
+    </addons>
+  </updates>
+
+For users who have updated, Firefox will download FlyWeb 1.0 and Pocket 1.0 and
+install them into the profile directory. For users that haven’t yet updated,
+Firefox will see that the **default** add-on set matches the set in the update
+ping and clear the **update** add-on set.
diff --git a/toolkit/mozapps/extensions/docs/index.rst b/toolkit/mozapps/extensions/docs/index.rst
new file mode 100644
index 0000000000..551cdd0ebc
--- /dev/null
+++ b/toolkit/mozapps/extensions/docs/index.rst
@@ -0,0 +1,21 @@
+==============
+Add-on Manager
+==============
+
+Overview docs
+-------------
+
+.. toctree::
+   :maxdepth: 1
+
+   AMRemoteSettings-overview
+   SystemAddons
+
+API References
+--------------
+
+.. toctree::
+   :maxdepth: 1
+
+   AddonManager
+   AMRemoteSettings