diff options
Diffstat (limited to '')
| -rw-r--r-- | toolkit/components/search/docs/DefaultSearchEngines.rst | 66 | ||||
| -rw-r--r-- | toolkit/components/search/docs/SearchConfigurationSchema.rst | 515 | ||||
| -rw-r--r-- | toolkit/components/search/docs/SearchEngineConfiguration.rst | 72 | ||||
| -rw-r--r-- | toolkit/components/search/docs/Telemetry.rst | 23 | ||||
| -rw-r--r-- | toolkit/components/search/docs/index.rst | 28 |
5 files changed, 704 insertions, 0 deletions
diff --git a/toolkit/components/search/docs/DefaultSearchEngines.rst b/toolkit/components/search/docs/DefaultSearchEngines.rst new file mode 100644 index 0000000000..5668646648 --- /dev/null +++ b/toolkit/components/search/docs/DefaultSearchEngines.rst @@ -0,0 +1,66 @@ +====================== +Default Search Engines +====================== + +The search service specifies default search engines via the `configuration +schema`_. + +Changing Defaults +================= + +The default engine may change when: + +* The user has the default engine set and the configuration for the locale/region + changes. +* The user has the default engine set and their locale/region changes to one + which has a different default. +* The user chooses to set a different engine via preferences. +* The user installs an add-on which sets its default as one of the application + provided engines. +* The user installs an add-on which supplies a different engine and allows the + different engine to be set as default. + +Add-ons and App-provided Engines +-------------------------------- + +An add-on may set the name of the search provider in the manifest.json to be +the name of an app-provided engine. In this case: + +* If the add-on is a non-authorised partner, then we set the user's default + engine to be the name of the app-provided engine. +* If the add-on is from an authorised partner, then we set the users' default + engine to be the same as the app-provided engine, and we allow the + app-provided urls to be overridden with those of the add-on. + +If the specified engine is already default, then the add-on does +not override the app-provided engine, and it's settings are ignored and no +new engine is added. + +The list of authorised add-ons is stored in `remote settings`_ in the +`search-default-override-allowlist bucket`_. The list +includes records containing: + +* Third-party Add-on Id: The identifier of the third party add-on which will + override the app provided one. +* Add-on Id to Override: The identifier of the app-provided add-on to be + overridden. +* a list of the url / params that are authorised to be replaced. + +When an authorised add-on overrides the default, we record the add-on's id +with the app-provided engine in the ``overriddenBy`` field. This is used +when the engine is loaded on startup to known that it should load the parameters +from that add-on. + +The ``overriddenBy`` annotation may be removed when: + +* The associated authorised add-on is removed, disabled or can no longer be found. +* The user changes their default to another engine. + +If the ``overriddenBy`` annotation is present, but the add-on is not authorised, +then the annotation will be maintained in case the add-on is later re-authorised. +For example, a url is updated, but the update is performed before the allow list +is updated. + +.. _configuration schema: SearchConfigurationSchema.html +.. _remote settings: /services/common/services/RemoteSettings.html +.. _search-default-override-allowlist bucket: https://firefox.settings.services.mozilla.com/v1/buckets/main/collections/search-default-override-allowlist/records diff --git a/toolkit/components/search/docs/SearchConfigurationSchema.rst b/toolkit/components/search/docs/SearchConfigurationSchema.rst new file mode 100644 index 0000000000..ce28f0f2ec --- /dev/null +++ b/toolkit/components/search/docs/SearchConfigurationSchema.rst @@ -0,0 +1,515 @@ +=========================== +Search Configuration Schema +=========================== + +This document outlines the details of the schema and how the various sub-parts +interact. For the full fields and descriptions, please see the `schema itself`_. + +.. note:: + In the examples, only relevant properties are displayed. + +Overview +======== + +The configuration is a JSON blob which is object with a `data` property which +is an array of engines: + +.. code-block:: js + + { + data: [ + { + // engine 1 details + }, + { + // engine 2 details + } + ] + } + +Engine Objects +============== + +An engine's details are located in the properties of the object associated with it. +An engine that is deployed globally could be listed simply as: + +.. code-block:: js + + { + "default": "no", + "telemetryId": "engine1-telem", + "webExtension": { + "id": "web@ext" + }, + "appliesTo": [{ + "included": { + "everywhere": true + } + }] + } + +The ``appliesTo`` section is an array of objects. At least one object is required +to specify which regions/locales the engine is included within. If an +``appliesTo`` object lists additional attributes then these will override any +attributes at the top-level. + +For example, a more complex engine definition may be available only to users +located specific regions or with certain locales. For example: + +.. code-block:: js + + { + "webExtension": { + "id": "web@ext" + }, + "appliesTo": [{ + "included": { + "region": "us" + }, + "webExtension": { + "id": "web-us@ext" + } + }, { + "included": { + "region": "gb" + }, + "webExtension": { + "id": "web-gb@ext" + } + }] + } + +In this case users identified as being in the US region would use the WebExtension +with identifier ``web-us@ext``. GB region users would get +``web-gb@ext``, and all other users would get ``web@ext``. + +Special Attributes +================== + +$USER_LOCALE +------------ + +If a ``webExtension.locales`` property contains an element with the value +``"$USER_LOCALE"`` then the special value will be replaced in the +configuration object with the users locale. For example: + +.. code-block:: js + + { + "webExtension": { + "id": "web@ext" + }, + "appliesTo": [{ + "included": { + "locales": { + "matches": ["us", "gb"] + }, + "webExtension": { + "locales": ["$USER_LOCALE"], + } + } + }] + } + +Will report either ``[us]`` or ``[gb]`` as the ``webExtension.locales`` +depending on the user's locale. + +"default" +--------- + +You can specify ``"default"`` as a region in the configuration if +the engine is to be included when we do not know the user's region. + +"override" +---------- + +The ``"override"`` field can be set to true if you want a section to +only override otherwise included engines. ``"override"`` will only work for +sections which apply to distributions or experiments. The experiment case was +added in Firefox 81. + +Example: + +.. code-block:: js + + { + "webExtension": { + "id": "web@ext" + }, + "appliesTo": [{ + // Complicated and lengthy inclusion rules + }, { + "override": true, + "application": { "distributions": ["mydistrocode"]}, + "params": { + "searchUrlGetParams": [ + { "name": "custom", "value": "foobar" } + ] + } + }] + } + +Application Scoping +=================== + +An engine configuration may be scoped to a particular application. + +Name +---- + +One or more application names may be specified. Currently the only application +type supported is ``firefox``. If an application name is specified, then it +must be matched for the section to apply. If there are no application names +specified, then the section will match any consumer of the configuration. + +In the following example, ``web@ext`` would be included on any consumer +of the configuration, but ``web1@ext`` would only be included on Firefox desktop. + +.. code-block:: js + + { + "webExtension": { + "id": "web@ext" + }, + "appliesTo": [{ + "included": { + "everywhere": true + "application": { + "name": [] + } + } + ]} + }, + { + "webExtension": { + "id": "web1@ext" + }, + "appliesTo": [{ + "included": { + "everywhere": true + "application": { + "name": ["firefox"] + } + } + ]} + } + +Channel +------- + +One or more channels may be specified in an array to restrict a configuration +to just those channels. The current known channels are: + + - default: Self-builds of Firefox, or possibly some self-distributed versions. + - nightly: Firefox Nightly builds. + - aurora: Firefox Developer Edition + - beta: Firefox Beta + - release: The main Firefox release channel. + - esr: The ESR Channel. This will also match versions of Firefox where the + displayed version number includes ``esr``. We do this to include Linux + distributions and other manual builds of ESR. + +In the following example, ``web@ext`` would be set as default on the default +channel only, whereas ``web1@ext`` would be set as default on release and esr +channels. + +.. code-block:: js + + { + "webExtension": { + "id": "web@ext" + }, + "appliesTo": [{ + "included": { + "everywhere": true + "default": "yes", + "application": { + "channel": ["default"] + } + } + ]} + }, + { + "webExtension": { + "id": "web1@ext" + }, + "appliesTo": [{ + "included": { + "everywhere": true + "default": "yes", + "application": { + "channel": ["release", "esr"] + } + } + ]} + } + +Distributions +------------- + +Distributions may be specified to be included or excluded in an ``appliesTo`` +section. The ``distributions`` field in the ``application`` section is an array +of distribution identifiers. The identifiers match those supplied by the +``distribution.id`` preference. + +In the following, ``web@ext`` would be included in only the ``cake`` +distribution. ``web1@ext`` would be excluded from the ``apples`` distribution +but included in the main desktop application, and all other distributions. + +.. code-block:: js + + { + "webExtension": { + "id": "web@ext" + }, + "appliesTo": [{ + "included": { + "everywhere": true + "application": { + "distributions": ["cake"] + } + } + ]} + }, + { + "webExtension": { + "id": "web1@ext" + }, + "appliesTo": [{ + "included": { + "everywhere": true + "application": { + "excludedDistributions": ["apples"] + } + } + ]} + } + +Version +------- + +Minimum and Maximum versions may be specified to restrict a configuration to +specific ranges. These may be open-ended. Version comparison is performed +using `the version comparator`_. + +Note: comparison against ``maxVersion`` is a less-than comparison. The +``maxVersion`` won't be matched directly. + +In the following example, ``web@ext`` would be included for any version after +72.0a1, whereas ``web1@ext`` would be included only between 68.0a1 and 71.x +version. + +.. code-block:: js + + { + "webExtension": { + "id": "web@ext" + }, + "appliesTo": [{ + "included": { + "everywhere": true + "application": { + "minVersion": "72.0a1" + } + } + ]} + }, + { + "webExtension": { + "id": "web1@ext" + }, + "appliesTo": [{ + "included": { + "everywhere": true + "default": "yes", + "application": { + "minVersion": "68.0a1" + "maxVersion": "72.0a1" + } + } + ]} + } + +Experiments +=========== + +We can run experiments by giving sections within ``appliesTo`` a +``experiment`` value, the Search Service can then optionally pass in a +matching ``experiment`` value to match those sections. + +Sections which have a ``experiment`` will not be used unless a matching +``experiment`` has been passed in, for example: + +.. code-block:: js + + { + "webExtension": { + "id": "web@ext" + }, + "appliesTo": [{ + "included": { + "everywhere": true + }, + "experiment": "nov-16", + "webExtension": { + "id": "web-experimental@ext" + } + }, { + "included": { + "everywhere": true + }, + "webExtension": { + "id": "web-gb@ext" + } + }] + } + +Engine Defaults +=============== + +An engine may be specified as the default for one of two purposes: + +#. normal browsing mode, +#. private browsing mode. + +If there is no engine specified for private browsing mode for a particular region/locale +pair, then the normal mode engine is used. + +If the instance of the application does not support a separate private browsing mode engine, +then it will only use the normal mode engine. + +An engine may or may not be default for particular regions/locales. The ``default`` +property is a tri-state value with states of ``yes``, ``yes-if-no-other`` and +``no``. Here's an example of how they apply: + +.. code-block:: js + + { + "webExtension": { + "id": "engine1@ext" + }, + "appliesTo": [{ + "included": { + "region": "us" + }, + "default": "yes" + }, { + "excluded": { + "region": "us" + }, + "default": "yes-if-no-other" + }] + }, + { + "webExtension": { + "id": "engine2@ext" + }, + "appliesTo": [{ + "included": { + "region": "gb" + }, + "default": "yes" + }] + }, + "webExtension": { + "id": "engine3@ext" + }, + "default": "no" + "appliesTo": [{ + "included": { + "everywhere": true + }, + }] + }, + { + "webExtension": { + "id": "engine4@ext" + }, + "defaultPrivate": "yes", + "appliesTo": [{ + "included": { + "region": "fr" + } + }] + } + +In this example, for normal mode: + + - engine1@ext is default in the US region, and all other regions except for GB + - engine2@ext is default in only the GB region + - engine3@ext and engine4 are never default anywhere + +In private browsing mode: + + - engine1@ext is default in the US region, and all other regions execpt for GB and FR + - engine2@ext is default in only the GB region + - engine3@ext is never default anywhere + - engine4@ext is default in the FR region. + +Engine Ordering +=============== + +The ``orderHint`` field indicates the suggested ordering of an engine relative to +other engines when displayed to the user, unless the user has customized their +ordering. + +The default ordering of engines is based on a combination of if the engine is +default, and the ``orderHint`` fields. The ordering is structured as follows: + +#. Default engine in normal mode +#. Default engine in private browsing mode (if different from the normal mode engine) +#. Other engines in order from the highest ``orderHint`` to the lowest. + +Example: + +.. code-block:: js + + { + "webExtension": { + "id": "engine1@ext" + }, + "orderHint": 2000, + "default": "no", + }, + { + "webExtension": { + "id": "engine2@ext" + }, + "orderHint": 1000, + "default": "yes" + }, + { + "webExtension": { + "id": "engine3@ext" + }, + "orderHint": 500, + "default": "no" + } + +This would result in the order: ``engine2@ext, engine1@ext, engine3@ext``. + +Region Params +============= + +The ``regionParams`` field allows us to override query parameters used based on the users current Region without having to reload the engine list which is based on the users home Region. + +Example: + +.. code-block:: js + + { + "webExtension": { + "id": "engine1@ext" + }, + "params": { + "searchUrlGetParams": [{ "name": "param", "value": "default" }], + }, + "regionParams": { + "US": [{ "name": "param", "value": "custom" }] + } + }, + +Will send ``param=custom`` whenever we detect the user is in US. + +.. _schema itself: https://searchfox.org/mozilla-central/source/toolkit/components/search/schema/ +.. _the version comparator: https://developer.mozilla.org/en-US/docs/Mozilla/Toolkit_version_format diff --git a/toolkit/components/search/docs/SearchEngineConfiguration.rst b/toolkit/components/search/docs/SearchEngineConfiguration.rst new file mode 100644 index 0000000000..e9041affb8 --- /dev/null +++ b/toolkit/components/search/docs/SearchEngineConfiguration.rst @@ -0,0 +1,72 @@ +=========================== +Search Engine Configuration +=========================== + +The search engine configuration is a mapping that is used to determine the +list of search engines for each user. The mapping is primarily based on the +user's region and locale. + +Configuration Management +======================== + +The application stores a dump of the configuration that is used for first +initialisation. Subsequent updates to the configuration are either updates to the +static dump, or they may be served via remote servers. + +The mechanism of delivering the settings dumps to the Search Service is +`Remote Settings`_ + +Remote settings +--------------- + +The remote settings bucket for the search engine configuration list is +``search-config``. The version that is currently being delivered +to clients can be `viewed live`_. + +Configuration Schema +==================== + +The configuration format is defined via a `JSON schema`_. The search engine +configuration schema is `stored in mozilla-central`_ and is uploaded to the +Remote Settings server at convenient times after it changes. + +An outline of the schema may be found on the `Search Configuration Schema`_ page. + +Updating Search Engine WebExtensions +==================================== + +Updates for application provided search engine WebExtensions are provided via +`Normandy`_. + +It is likely that updates for search engine WebExtensions will be +received separately to configuration updates which may or may not be directly +related. As a result several situations may occur: + + - The updated WebExtension is for an app-provided engine already in-use by + the user. + + - In this case, the search service will apply the changes to the + app-provided engine's data. + + - A WebExtension addition/update is for an app-provided engine that is not + in-use by the user, or not in the configuration. + + - In this case, the search service will ignore the WebExtension. + - If the configuration (search or user) is updated later and the + new engine is added, then the Search Service will start to use the + new engine. + + - A configuration update is received that needs a WebExtension that is + not found locally. + + - In this case, the search service will ignore the missing engine and + continue without it. + - When the WebExtension is delivered, the search engine will then be + installed and added. + +.. _Remote Settings: /services/common/services/RemoteSettings.html +.. _JSON schema: https://json-schema.org/ +.. _stored in mozilla-central: https://searchfox.org/mozilla-central/source/toolkit/components/search/schema/ +.. _Search Configuration Schema: SearchConfigurationSchema.html +.. _viewed live: https://firefox.settings.services.mozilla.com/v1/buckets/main/collections/search-config/records +.. _Normandy: /toolkit/components/normandy/normandy/services.html diff --git a/toolkit/components/search/docs/Telemetry.rst b/toolkit/components/search/docs/Telemetry.rst new file mode 100644 index 0000000000..70f5271ed1 --- /dev/null +++ b/toolkit/components/search/docs/Telemetry.rst @@ -0,0 +1,23 @@ +========= +Telemetry +========= + +Scalars +------- + +browser.searchinit.init_result_status_code + Records the search service initialization code on startup. This is typically + one of the error values in https://searchfox.org/mozilla-central/source/xpcom/base/ErrorList.py + +Keyed Scalars +------------- + +browser.searchinit.engine_invalid_webextension + Records the WebExtension ID of a search engine where the saved search engine + settings do not match the WebExtension. + + The keys are the WebExtension IDs. The values are integers: + + 1. Associated WebExtension is not installed. + 2. Associated WebExtension is disabled. + 3. The submission URL of the associated WebExtension is different to that of the saved settings. diff --git a/toolkit/components/search/docs/index.rst b/toolkit/components/search/docs/index.rst new file mode 100644 index 0000000000..188f020ebf --- /dev/null +++ b/toolkit/components/search/docs/index.rst @@ -0,0 +1,28 @@ +============== +Search Service +============== + +This is documentation for the Search Service. + +Definitions +=========== + +* Application-provided engine (aka app-provided): This is an engine that is + provided by the application to the user as part of the configurations for the + user's locale/region. +* Default engine: This is the engine that is the one used by default when + doing searches from the address bar, search bar and other places. This may be + a default application provided engine or a user selected engine. +* Default private engine: Same as for the default engine, but this is used by + default when in private browsing mode. + +Contents +======== + +.. toctree:: + :maxdepth: 2 + + SearchEngineConfiguration + SearchConfigurationSchema + DefaultSearchEngines + Telemetry |