diff options
Diffstat (limited to 'browser/components/urlbar/docs')
47 files changed, 5816 insertions, 0 deletions
diff --git a/browser/components/urlbar/docs/.rstcheck.cfg b/browser/components/urlbar/docs/.rstcheck.cfg new file mode 100644 index 0000000000..741c90297a --- /dev/null +++ b/browser/components/urlbar/docs/.rstcheck.cfg @@ -0,0 +1,13 @@ +[rstcheck] +# Suppress some rstcheck messages. Unfortunately there isn't a better way to do +# this. See: https://github.com/myint/rstcheck#ignore-specific-errors +# +# Duplicate explicit target name: "[0-9]+" +# => Allow duplicate out-of-line definitions of links to bugs, like: +# .. _1689365: https://bugzilla.mozilla.org/show_bug.cgi?id=1689365 +# That way if a bug is referenced in more than one section, you can define +# it in every section it's used, which might be saner than making sure it's +# defined in only one place. +# Enumerated list start value not ordinal-1: "0" +# => Allow numbered lists to start at 0. +ignore_messages=(Duplicate explicit target name: "[0-9]+"|Enumerated list start value not ordinal-1: "0") diff --git a/browser/components/urlbar/docs/UrlbarController.rst b/browser/components/urlbar/docs/UrlbarController.rst new file mode 100644 index 0000000000..281aa9715d --- /dev/null +++ b/browser/components/urlbar/docs/UrlbarController.rst @@ -0,0 +1,5 @@ +UrlbarController Reference +========================== + +.. js:autoclass:: UrlbarController + :members: diff --git a/browser/components/urlbar/docs/UrlbarInput.rst b/browser/components/urlbar/docs/UrlbarInput.rst new file mode 100644 index 0000000000..3c74048830 --- /dev/null +++ b/browser/components/urlbar/docs/UrlbarInput.rst @@ -0,0 +1,5 @@ +UrlbarInput Reference +===================== + +.. js:autoclass:: UrlbarInput + :members: diff --git a/browser/components/urlbar/docs/UrlbarView.rst b/browser/components/urlbar/docs/UrlbarView.rst new file mode 100644 index 0000000000..1eacd53702 --- /dev/null +++ b/browser/components/urlbar/docs/UrlbarView.rst @@ -0,0 +1,5 @@ +UrlbarView Reference +==================== + +.. js:autoclass:: UrlbarView + :members: diff --git a/browser/components/urlbar/docs/assets/lifetime/lifetime.png b/browser/components/urlbar/docs/assets/lifetime/lifetime.png Binary files differnew file mode 100644 index 0000000000..17be253027 --- /dev/null +++ b/browser/components/urlbar/docs/assets/lifetime/lifetime.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/autofill.png b/browser/components/urlbar/docs/assets/nontechnical-overview/autofill.png Binary files differnew file mode 100644 index 0000000000..78587611ee --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/autofill.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/bookmark-keyword.png b/browser/components/urlbar/docs/assets/nontechnical-overview/bookmark-keyword.png Binary files differnew file mode 100644 index 0000000000..68dde6c88d --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/bookmark-keyword.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/bookmark.png b/browser/components/urlbar/docs/assets/nontechnical-overview/bookmark.png Binary files differnew file mode 100644 index 0000000000..ba4500d6c6 --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/bookmark.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/empty-placeholder.png b/browser/components/urlbar/docs/assets/nontechnical-overview/empty-placeholder.png Binary files differnew file mode 100644 index 0000000000..a885be7e06 --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/empty-placeholder.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/empty-url.png b/browser/components/urlbar/docs/assets/nontechnical-overview/empty-url.png Binary files differnew file mode 100644 index 0000000000..fd802258de --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/empty-url.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/form-history.png b/browser/components/urlbar/docs/assets/nontechnical-overview/form-history.png Binary files differnew file mode 100644 index 0000000000..ae4e236f99 --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/form-history.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/history.png b/browser/components/urlbar/docs/assets/nontechnical-overview/history.png Binary files differnew file mode 100644 index 0000000000..1b6de1e76f --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/history.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/intervention-clear.png b/browser/components/urlbar/docs/assets/nontechnical-overview/intervention-clear.png Binary files differnew file mode 100644 index 0000000000..56780bc169 --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/intervention-clear.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/intervention-refresh.png b/browser/components/urlbar/docs/assets/nontechnical-overview/intervention-refresh.png Binary files differnew file mode 100644 index 0000000000..1b8d87cc72 --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/intervention-refresh.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/intervention-update.png b/browser/components/urlbar/docs/assets/nontechnical-overview/intervention-update.png Binary files differnew file mode 100644 index 0000000000..41af8421d6 --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/intervention-update.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/non-empty.png b/browser/components/urlbar/docs/assets/nontechnical-overview/non-empty.png Binary files differnew file mode 100644 index 0000000000..3949d4c407 --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/non-empty.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/open-tab.png b/browser/components/urlbar/docs/assets/nontechnical-overview/open-tab.png Binary files differnew file mode 100644 index 0000000000..d063540981 --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/open-tab.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/prefs-privacy.png b/browser/components/urlbar/docs/assets/nontechnical-overview/prefs-privacy.png Binary files differnew file mode 100644 index 0000000000..cca5864dbb --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/prefs-privacy.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/prefs-show-suggestions.png b/browser/components/urlbar/docs/assets/nontechnical-overview/prefs-show-suggestions.png Binary files differnew file mode 100644 index 0000000000..4a0f019798 --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/prefs-show-suggestions.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/prefs-suggestions-first.png b/browser/components/urlbar/docs/assets/nontechnical-overview/prefs-suggestions-first.png Binary files differnew file mode 100644 index 0000000000..09ec455563 --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/prefs-suggestions-first.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/remote-tab.png b/browser/components/urlbar/docs/assets/nontechnical-overview/remote-tab.png Binary files differnew file mode 100644 index 0000000000..64c75da6c6 --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/remote-tab.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/search-heuristic.png b/browser/components/urlbar/docs/assets/nontechnical-overview/search-heuristic.png Binary files differnew file mode 100644 index 0000000000..6d84db2f04 --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/search-heuristic.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/search-mode.png b/browser/components/urlbar/docs/assets/nontechnical-overview/search-mode.png Binary files differnew file mode 100644 index 0000000000..42cb34d6e0 --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/search-mode.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/search-offers-selected.png b/browser/components/urlbar/docs/assets/nontechnical-overview/search-offers-selected.png Binary files differnew file mode 100644 index 0000000000..402f6cd19b --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/search-offers-selected.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/search-offers.png b/browser/components/urlbar/docs/assets/nontechnical-overview/search-offers.png Binary files differnew file mode 100644 index 0000000000..b61f54f432 --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/search-offers.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/search-suggestion.png b/browser/components/urlbar/docs/assets/nontechnical-overview/search-suggestion.png Binary files differnew file mode 100644 index 0000000000..0435615467 --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/search-suggestion.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/search-tip-onboard.png b/browser/components/urlbar/docs/assets/nontechnical-overview/search-tip-onboard.png Binary files differnew file mode 100644 index 0000000000..859b688e1a --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/search-tip-onboard.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/search-tip-redirect.png b/browser/components/urlbar/docs/assets/nontechnical-overview/search-tip-redirect.png Binary files differnew file mode 100644 index 0000000000..e34d53f12b --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/search-tip-redirect.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/tab-to-search-onboard.png b/browser/components/urlbar/docs/assets/nontechnical-overview/tab-to-search-onboard.png Binary files differnew file mode 100644 index 0000000000..204066e9ce --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/tab-to-search-onboard.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/tab-to-search-regular.png b/browser/components/urlbar/docs/assets/nontechnical-overview/tab-to-search-regular.png Binary files differnew file mode 100644 index 0000000000..de03d2f0eb --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/tab-to-search-regular.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/tail-suggestions.png b/browser/components/urlbar/docs/assets/nontechnical-overview/tail-suggestions.png Binary files differnew file mode 100644 index 0000000000..fd7f098a98 --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/tail-suggestions.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/top-sites.png b/browser/components/urlbar/docs/assets/nontechnical-overview/top-sites.png Binary files differnew file mode 100644 index 0000000000..6818b1c17d --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/top-sites.png diff --git a/browser/components/urlbar/docs/assets/nontechnical-overview/visit.png b/browser/components/urlbar/docs/assets/nontechnical-overview/visit.png Binary files differnew file mode 100644 index 0000000000..a0b182dd8f --- /dev/null +++ b/browser/components/urlbar/docs/assets/nontechnical-overview/visit.png diff --git a/browser/components/urlbar/docs/contact.rst b/browser/components/urlbar/docs/contact.rst new file mode 100644 index 0000000000..abd9947528 --- /dev/null +++ b/browser/components/urlbar/docs/contact.rst @@ -0,0 +1,9 @@ +Getting in Touch +================ + +For any questions regarding the Address Bar, the team is available through +the #search channel on Slack and the fx-search@mozilla.com mailing +list. + +Issues can be `filed in Bugzilla <https://bugzilla.mozilla.org/enter_bug.cgi?product=Firefox&component=Address%20Bar>`_ +under the Firefox / Address Bar component. diff --git a/browser/components/urlbar/docs/debugging.rst b/browser/components/urlbar/docs/debugging.rst new file mode 100644 index 0000000000..689657c068 --- /dev/null +++ b/browser/components/urlbar/docs/debugging.rst @@ -0,0 +1,4 @@ +Debugging & Logging +=================== + +*Content to be written* diff --git a/browser/components/urlbar/docs/dynamic-result-types.rst b/browser/components/urlbar/docs/dynamic-result-types.rst new file mode 100644 index 0000000000..a3bf24593f --- /dev/null +++ b/browser/components/urlbar/docs/dynamic-result-types.rst @@ -0,0 +1,806 @@ +Dynamic Result Types +==================== + +This document discusses a special category of address bar results called dynamic +result types. Dynamic result types allow you to easily add new types of results +to the address bar and are especially useful for extensions. + +The intended audience for this document is developers who need to add new kinds +of address bar results, either internally in the address bar codebase or through +extensions. + +.. contents:: + :depth: 2 + + +Motivation +---------- + +The address bar provides many different types of results in normal Firefox +usage. For example, when you type a search term, the address bar may show you +search suggestion results from your current search engine. It may also show you +results from your browsing history that match your search. If you typed a +certain phrase like "update Firefox," it will show you a tip result that lets +you know whether Firefox is up to date. + +Each of these types of results is built into the address bar implementation. If +you wanted to add a new type of result -- say, a card that shows the weather +forecast when the user types "weather" -- one way to do so would be to add a new +result type. You would need to update all the code paths in the address bar that +relate to result types. For instance, you'd need to update the code path that +handles clicks on results so that your weather card opens an appropriate +forecast URL when clicked; you'd need to update the address bar view (the panel) +so that your card is drawn correctly; you may need to update the keyboard +selection behavior if your card contains elements that can be independently +selected such as different days of the week; and so on. + +If you're implementing your weather card in an extension, as you might in an +add-on experiment, then you'd need to land your new result type in +mozilla-central so your extension can use it. Your new result type would ship +with Firefox even though the vast majority of users would never see it, and your +fellow address bar hackers would have to work around your code even though it +would remain inactive most of the time, at least until your experiment +graduated. + +Dynamic Result Types +-------------------- + +**Dynamic result types** are an alternative way of implementing new result +types. Instead of adding a new built-in type along with all that entails, you +add a new provider subclass and register a template that describes how the view +should draw your result type and indicates which elements are selectable. The +address bar takes care of everything else. (Or if you're implementing an +extension, you add a few event handlers instead of a provider subclass, although +we have a shim_ that abstracts away the differences between internal and +extension address bar code.) + +Dynamic result types are essentially an abstraction layer: Support for them as a +general category of results is built into the address bar, and each +implementation of a specific dynamic result type fills in the details. + +In addition, dynamic result types can be added at runtime. This is important for +extensions that implement new types of results like the weather forecast example +above. + +.. _shim: https://github.com/0c0w3/dynamic-result-type-extension/blob/master/src/shim.js + +Getting Started +--------------- + +To get a feel for how dynamic result types are implemented, you can look at the +`example dynamic result type extension <exampleExtension_>`__. The extension +uses the recommended shim_ that makes writing address bar extension code very +similar to writing internal address bar code, and it's therefore a useful +example even if you intend to add a new dynamic result type internally in the +address bar codebase in mozilla-central. + +The next section describes the specific steps you need to take to add a new +dynamic result type. + +.. _exampleExtension: https://github.com/0c0w3/dynamic-result-type-extension/blob/master/src/background.js + +Implementation Steps +-------------------- + +This section describes how to add a new dynamic result type in either of the +following cases: + +* You want to add a new dynamic result type in an extension using the + recommended shim_. +* You want to add a new dynamic result type internal to the address bar codebase + in mozilla-central. + +The steps are mostly the same in both cases and are described next. + +If you want to add a new dynamic result type in an extension but don't want to +use the shim, then skip ahead to `Appendix B: Using the WebExtensions API +Directly`_. + +1. Register the dynamic result type +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +First, register the new dynamic result type: + +.. code-block:: javascript + + UrlbarResult.addDynamicResultType(name); + +``name`` is a string identifier for the new type. It must be unique; that is, it +must be different from all other dynamic result type names. It will also be used +in DOM IDs, DOM class names, and CSS selectors, so it should not contain any +spaces or other characters that are invalid in CSS. + +2. Register the view template +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Next, add the view template for the new type: + +.. code-block:: javascript + + UrlbarView.addDynamicViewTemplate(name, viewTemplate); + +``name`` is the new type's name as described in step 1. + +``viewTemplate`` is an object called a view template. It describes in a +declarative manner the DOM that should be created in the view for all results of +the new type. For providers created in extensions, it also declares the +stylesheet that should be applied to results in the view. See `View Templates`_ +for a description of this object. + +3. Add the provider +~~~~~~~~~~~~~~~~~~~ + +As with any type of result, results for dynamic result types must be created by +one or more providers. Make a ``UrlbarProvider`` subclass for the new provider +and implement all the usual provider methods as you normally would: + +.. code-block:: javascript + + class MyDynamicResultTypeProvider extends UrlbarProvider { + // ... + } + +The ``startQuery`` method should create ``UrlbarResult`` objects with the +following two requirements: + +* Result types must be ``UrlbarUtils.RESULT_TYPE.DYNAMIC``. +* Result payloads must have a ``dynamicType`` property whose value is the name + of the dynamic result type used in step 1. + +The results' sources, other payload properties, and other result properties +aren't relevant to dynamic result types, and you should choose values +appropriate to your use case. + +If any elements created in the view for your results can be picked with the +keyboard or mouse, then be sure to implement your provider's ``onEngagement`` +method. + +For help on implementing providers in general, see the address bar's +`Architecture Overview`__. + +If you are creating the provider in the internal address bar implementation in +mozilla-central, then don't forget to register it in ``UrlbarProvidersManager``. + +If you are creating the provider in an extension, then it's registered +automatically, and there's nothing else you need to do. + +__ https://firefox-source-docs.mozilla.org/browser/urlbar/overview.html#urlbarprovider + +4. Implement the provider's getViewUpdate method +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +``getViewUpdate`` is a provider method particular to dynamic result type +providers. Its job is to update the view DOM for a specific result. It's called +by the view for each result in the view that was created by the provider. It +returns an object called a view update object. + +Recall that the view template was added earlier, in step 2. The view template +describes how to build the DOM structure for all results of the dynamic result +type. The view update object, in this step, describes how to fill in that +structure for a specific result. + +Add the ``getViewUpdate`` method to the provider: + +.. code-block:: javascript + + /** + * Returns a view update object that describes how to update the view DOM + * for a given result. + * + * @param {UrlbarResult} result + * The view update object describes how to update the view DOM for this + * particular result. + * @param {Map} idsByName + * A map from names in the view template to the IDs of their corresponding + * elements in the DOM. + */ + getViewUpdate(result, idsByName) { + let viewUpdate = { + // ... + }; + return viewUpdate; + } + +``result`` is the result from the provider for which the view update is being +requested. + +``idsByName`` is a map from names in the view template to the IDs of their +corresponding elements in the DOM. This is useful if parts of the view update +depend on element IDs, as some ARIA attributes do. + +The return value is a view update object. It describes in a declarative manner +the updates that should be performed on the view DOM. See `View Update Objects`_ +for a description of this object. + +5. Style the results +~~~~~~~~~~~~~~~~~~~~ + +If you are creating the provider in the internal address bar implementation in +mozilla-central, then add styling `urlbar-dynamic-results.css`_. + +.. _urlbar-dynamic-results.css: https://searchfox.org/mozilla-central/source/browser/themes/shared/urlbar-dynamic-results.css + +If you are creating the provider in an extension, then bundle a CSS file in your +extension and declare it in the top-level ``stylesheet`` property of your view +template, as described in `View Templates`_. Additionally, if any of your rules +override built-in rules, then you'll need to declare them as ``!important``. + +The rest of this section will discuss the CSS rules you need to use to style +your results. + +There are two DOM annotations that are useful for styling. The first is the +``dynamicType`` attribute that is set on result rows, and the second is a class +that is set on child elements created from the view template. + +dynamicType Row Attribute +......................... + +The topmost element in the view corresponding to a result is called a +**row**. Rows have a class of ``urlbarView-row``, and rows corresponding to +results of a dynamic result type have an attributed called ``dynamicType``. The +value of this attribute is the name of the dynamic result type that was chosen +in step 1 earlier. + +Rows of a specific dynamic result type can therefore be selected with the +following CSS selector, where ``TYPE_NAME`` is the name of the type: + +.. code-block:: css + + .urlbarView-row[dynamicType=TYPE_NAME] + +Child Element Class +................... + +As discussed in `View Templates`_, each object in the view template can have a +``name`` property. The elements in the view corresponding to the objects in the +view template receive a class named +``urlbarView-dynamic-TYPE_NAME-ELEMENT_NAME``, where ``TYPE_NAME`` is the name +of the dynamic result type, and ``ELEMENT_NAME`` is the name of the object in +the view template. + +Elements in dynamic result type rows can therefore be selected with the +following: + +.. code-block:: css + + .urlbarView-dynamic-TYPE_NAME-ELEMENT_NAME + +If an object in the view template does not have a ``name`` property, then it +won't receive the class and it therefore can't be selected using this selector. + +View Templates +-------------- + +A **view template** is a plain JS object that declaratively describes how to +build the DOM for a dynamic result type. When a result of a particular dynamic +result type is shown in the view, the type's view template is used to construct +the part of the view that represents the type in general. + +The need for view templates arises from the fact that extensions run in a +separate process from the chrome process and can't directly access the chrome +DOM, where the address bar view lives. Since extensions are a primary use case +for dynamic result types, this is an important constraint on their design. + +Properties +~~~~~~~~~~ + +A view template object is a tree-like nested structure where each object in the +nesting represents a DOM element to be created. This tree-like structure is +achieved using the ``children`` property described below. Each object in the +structure may include the following properties: + +``{string} name`` + The name of the object. This is required for all objects in the structure + except the root object and serves two important functions: + + 1. The element created for the object will automatically have a class named + ``urlbarView-dynamic-${dynamicType}-${name}``, where ``dynamicType`` is the + name of the dynamic result type. The element will also automatically have + an attribute ``name`` whose value is this name. The class and attribute + allow the element to be styled in CSS. + + 2. The name is used when updating the view, as described in `View Update + Objects`_. + + Names must be unique within a view template, but they don't need to be + globally unique. In other words, two different view templates can use the same + names, and other unrelated DOM elements can use the same names in their IDs + and classes. + +``{string} tag`` + The element tag name of the object. This is required for all objects in the + structure except the root object and declares the kind of element that will be + created for the object: ``span``, ``div``, ``img``, etc. + +``{object} [attributes]`` + An optional mapping from attribute names to values. For each name-value pair, + an attribute is set on the element created for the object. + + A special ``selectable`` attribute tells the view that the element is + selectable with the keyboard. The element will automatically participate in + the view's keyboard selection behavior. + + Similarly, the ``role=button`` ARIA attribute will also automatically allow + the element to participate in keyboard selection. The ``selectable`` attribute + is not necessary when ``role=button`` is specified. + +``{array} [children]`` + An optional list of children. Each item in the array must be an object as + described in this section. For each item, a child element as described by the + item is created and added to the element created for the parent object. + +``{array} [classList]`` + An optional list of classes. Each class will be added to the element created + for the object by calling ``element.classList.add()``. + +``{string} [stylesheet]`` + For dynamic result types created in extensions, this property should be set on + the root object in the view template structure, and its value should be a + stylesheet URL. The stylesheet will be loaded in all browser windows so that + the dynamic result type view may be styled. The specified URL will be resolved + against the extension's base URI. We recommend specifying a URL relative to + your extension's base directory. + + For dynamic result types created internally in the address bar codebase, this + value should not be specified and instead styling should be added to + `urlbar-dynamic-results.css`_. + +Example +~~~~~~~ + +Let's return to the weather forecast example from `earlier <Motivation_>`__. For +each result of our weather forecast dynamic result type, we might want to +display a label for a city name along with two buttons for today's and +tomorrow's forecasted high and low temperatures. The view template might look +like this: + +.. code-block:: javascript + + { + stylesheet: "style.css", + children: [ + { + name: "cityLabel", + tag: "span", + }, + { + name: "today", + tag: "div", + classList: ["day"], + attributes: { + selectable: true, + }, + children: [ + { + name: "todayLabel", + tag: "span", + classList: ["dayLabel"], + }, + { + name: "todayLow", + tag: "span", + classList: ["temperature", "temperatureLow"], + }, + { + name: "todayHigh", + tag: "span", + classList: ["temperature", "temperatureHigh"], + }, + }, + }, + { + name: "tomorrow", + tag: "div", + classList: ["day"], + attributes: { + selectable: true, + }, + children: [ + { + name: "tomorrowLabel", + tag: "span", + classList: ["dayLabel"], + }, + { + name: "tomorrowLow", + tag: "span", + classList: ["temperature", "temperatureLow"], + }, + { + name: "tomorrowHigh", + tag: "span", + classList: ["temperature", "temperatureHigh"], + }, + }, + }, + ], + } + +Observe that we set the special ``selectable`` attribute on the ``today`` and +``tomorrow`` elements so they can be selected with the keyboard. + +View Update Objects +------------------- + +A **view update object** is a plain JS object that declaratively describes how +to update the DOM for a specific result of a dynamic result type. When a result +of a dynamic result type is shown in the view, a view update object is requested +from the result's provider and is used to update the DOM for that result. + +Note the difference between view update objects, described in this section, and +view templates, described in the previous section. View templates are used to +build a general DOM structure appropriate for all results of a particular +dynamic result type. View update objects are used to fill in that structure for +a specific result. + +When a result is shown in the view, first the view looks up the view template of +the result's dynamic result type. It uses the view template to build a DOM +subtree. Next, the view requests a view update object for the result from its +provider. The view update object tells the view which result-specific attributes +to set on which elements, result-specific text content to set on elements, and +so on. View update objects cannot create new elements or otherwise modify the +structure of the result's DOM subtree. + +Typically the view update object is based on the result's payload. + +Properties +~~~~~~~~~~ + +The view update object is a nested structure with two levels. It looks like +this: + +.. code-block:: javascript + + { + name1: { + // individual update object for name1 + }, + name2: { + // individual update object for name2 + }, + name3: { + // individual update object for name3 + }, + // ... + } + +The top level maps object names from the view template to individual update +objects. The individual update objects tell the view how to update the elements +with the specified names. If a particular element doesn't need to be updated, +then it doesn't need an entry in the view update object. + +Each individual update object can have the following properties: + +``{object} [attributes]`` + A mapping from attribute names to values. Each name-value pair results in an + attribute being set on the element. + +``{object} [style]`` + A plain object that can be used to add inline styles to the element, like + ``display: none``. ``element.style`` is updated for each name-value pair in + this object. + +``{object} [l10n]`` + An ``{ id, args }`` object that will be passed to + ``document.l10n.setAttributes()``. + +``{string} [textContent]`` + A string that will be set as ``element.textContent``. + +Example +~~~~~~~ + +Continuing our weather forecast example, the view update object needs to update +several things that we declared in our view template: + +* The city label +* The "today" label +* Today's low and high temperatures +* The "tomorrow" label +* Tomorrow's low and high temperatures + +Typically, each of these, with the possible exceptions of the "today" and +"tomorrow" labels, would come from our results' payloads. There's an important +connection between what's in the view and what's in the payloads: The data in +the payloads serves the information shown in the view. + +Our view update object would then look something like this: + +.. code-block:: javascript + + { + cityLabel: { + textContent: result.payload.city, + }, + todayLabel: { + textContent: "Today", + }, + todayLow: { + textContent: result.payload.todayLow, + }, + todayHigh: { + textContent: result.payload.todayHigh, + }, + tomorrowLabel: { + textContent: "Tomorrow", + }, + tomorrowLow: { + textContent: result.payload.tomorrowLow, + }, + tomorrowHigh: { + textContent: result.payload.tomorrowHigh, + }, + } + +Accessibility +------------- + +Just like built-in types, dynamic result types support a11y in the view, and you +should make sure your view implementation is fully accessible. + +Since the views for dynamic result types are implemented using view templates +and view update objects, in practice supporting a11y for dynamic result types +means including appropriate `ARIA attributes <aria_>`_ in the view template and +view update objects, using the ``attributes`` property. + +Many ARIA attributes depend on element IDs, and that's why the ``idsByName`` +parameter to the ``getViewUpdate`` provider method is useful. + +Usually, accessible address bar results require the ARIA attribute +``role=group`` on their top-level DOM element to indicate that all the child +elements in the result's DOM subtree form a logical group. This attribute can be +set on the root object in the view template. + +.. _aria: https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA + +Example +~~~~~~~ + +Continuing the weather forecast example, we'd like for screen readers to know +that our result is labeled by the city label so that they announce the city when +the result is selected. + +The relevant ARIA attribute is ``aria-labelledby``, and its value is the ID of +the element with the label. In our ``getViewUpdate`` implementation, we can use +the ``idsByName`` map to get the element ID that the view created for our city +label, like this: + +.. code-block:: javascript + + getViewUpdate(result, idsByName) { + return { + root: { + attributes: { + "aria-labelledby": idsByName.get("cityLabel"), + }, + }, + // *snipping the view update object example from earlier* + }; + } + +Here we're using the name "root" to refer to the root object in the view +template, so we also need to update our view template by adding the ``name`` +property to the top-level object, like this: + +.. code-block:: javascript + + { + stylesheet: "style.css", + name: "root", + attributes: { + role: "group", + }, + children: [ + { + name: "cityLabel", + tag: "span", + }, + // *snipping the view template example from earlier* + ], + } + +Note that we've also included the ``role=group`` ARIA attribute on the root, as +discussed above. We could have included it in the view update object instead of +the view template, but since it doesn't depend on a specific result or element +ID in the ``idsByName`` map, the view template makes more sense. + +Mimicking Built-in Address Bar Results +-------------------------------------- + +Sometimes it's desirable to create a new result type that looks and behaves like +the usual built-in address bar results. Two conveniences are available that are +useful in this case. + +URL Navigation +~~~~~~~~~~~~~~ + +If a result's payload includes a string ``url`` property and a boolean +``shouldNavigate: true`` property, then picking the result will navigate to the +URL. The ``onEngagement`` method of the result's provider will still be called +before navigation. + +Text Highlighting +~~~~~~~~~~~~~~~~~ + +Most built-in address bar results emphasize occurrences of the user's search +string in their text by boldfacing matching substrings. Search suggestion +results do the opposite by emphasizing the portion of the suggestion that the +user has not yet typed. This emphasis feature is called **highlighting**, and +it's also available to the results of dynamic result types. + +Highlighting for dynamic result types is a fairly automated process. The text +that you want to highlight must be present as a property in your result +payload. Instead of setting the property to a string value as you normally +would, set it to an array with two elements, where the first element is the text +and the second element is a ``UrlbarUtils.HIGHLIGHT`` value, like the ``title`` +payload property in the following example: + +.. code-block:: javascript + + let result = new UrlbarResult( + UrlbarUtils.RESULT_TYPE.DYNAMIC, + UrlbarUtils.RESULT_SOURCE.OTHER_NETWORK, + { + title: [ + "Some result title", + UrlbarUtils.HIGHLIGHT.TYPED, + ], + // *more payload properties* + } + ); + +``UrlbarUtils.HIGHLIGHT`` is defined in the extensions shim_ and is described +below. + +Your view template must create an element corresponding to the payload +property. That is, it must include an object where the value of the ``name`` +property is the name of the payload property, like this: + +.. code-block:: javascript + + { + children: [ + { + name: "title", + tag: "span", + }, + // ... + ], + } + +In contrast, your view update objects must *not* include an update for the +element. That is, they must not include a property whose name is the name of the +payload property. + +Instead, when the view is ready to update the DOM of your results, it will +automatically find the elements corresponding to the payload property, set their +``textContent`` to the text value in the array, and apply the appropriate +highlighting, as described next. + +There are two possible ``UrlbarUtils.HIGHLIGHT`` values. Each controls how +highlighting is performed: + +``UrlbarUtils.HIGHLIGHT.TYPED`` + Substrings in the payload text that match the user's search string will be + emphasized. + +``UrlbarUtils.HIGHLIGHT.SUGGESTED`` + If the user's search string appears in the payload text, then the remainder of + the text following the matching substring will be emphasized. + +Appendix A: Examples +-------------------- + +This section lists some example and real-world consumers of dynamic result +types. + +`Example Extension`__ + This extension demonstrates a simple use of dynamic result types. + +`Weather Quick Suggest Extension`__ + A real-world Firefox extension experiment that shows weather forecasts and + alerts when the user performs relevant searches in the address bar. + +`Tab-to-Search Provider`__ + This is a built-in provider in mozilla-central that uses dynamic result types. + +__ https://github.com/0c0w3/dynamic-result-type-extension +__ https://github.com/mozilla-extensions/firefox-quick-suggest-weather/blob/master/src/background.js +__ https://searchfox.org/mozilla-central/source/browser/components/urlbar/UrlbarProviderTabToSearch.sys.mjs + +Appendix B: Using the WebExtensions API Directly +------------------------------------------------ + +If you're developing an extension, the recommended way of using dynamic result +types is to use the shim_, which abstracts away the differences between writing +internal address bar code and extensions code. The `implementation steps`_ above +apply to extensions as long as you're using the shim. + +For completeness, in this section we'll document the WebExtensions APIs that the +shim is built on. If you don't use the shim for some reason, then follow these +steps instead. You'll see that each step above using the shim has an analogous +step here. + +The WebExtensions API schema is declared in `schema.json`_ and implemented in +`api.js`_. + +.. _schema.json: https://github.com/0c0w3/dynamic-result-type-extension/blob/master/src/experiments/urlbar/schema.json +.. _api.js: https://github.com/0c0w3/dynamic-result-type-extension/blob/master/src/experiments/urlbar/api.js + +1. Register the dynamic result type +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +First, register the new dynamic result type: + +.. code-block:: javascript + + browser.experiments.urlbar.addDynamicResultType(name, type); + +``name`` is a string identifier for the new type. See step 1 in `Implementation +Steps`_ for a description, which applies here, too. + +``type`` is an object with metadata for the new type. Currently no metadata is +supported, so this should be an empty object, which is the default value. + +2. Register the view template +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Next, add the view template for the new type: + +.. code-block:: javascript + + browser.experiments.urlbar.addDynamicViewTemplate(name, viewTemplate); + +See step 2 above for a description of the parameters. + +3. Add WebExtension event listeners +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Add all the WebExtension event listeners you normally would in an address bar +extension, including the two required listeners, ``onBehaviorRequested`` and +and ``onResultsRequested``. + +.. code-block:: javascript + + browser.urlbar.onBehaviorRequested.addListener(query => { + return "active"; + }, providerName); + + browser.urlbar.onResultsRequested.addListener(query => { + let results = [ + // ... + ]; + return results; + }, providerName); + +See the address bar extensions__ document for help on the urlbar WebExtensions +API. + +__ https://firefox-source-docs.mozilla.org/browser/urlbar/experiments.html + +4. Add an onViewUpdateRequested event listener +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +``onViewUpdateRequested`` is a WebExtensions event particular to dynamic result +types. It's analogous to the ``getViewUpdate`` provider method described +earlier. + +.. code-block:: javascript + + browser.experiments.urlbar.onViewUpdateRequested.addListener((payload, idsByName) => { + let viewUpdate = { + // ... + }; + return viewUpdate; + }); + +Note that unlike ``getViewUpdate``, here the listener's first parameter is a +result payload, not the result itself. + +The listener should return a view update object. + +5. Style the results +~~~~~~~~~~~~~~~~~~~~ + +This step is the same as step 5 above. Bundle a CSS file in your extension and +declare it in the top-level ``stylesheet`` property of your view template. diff --git a/browser/components/urlbar/docs/experiments.rst b/browser/components/urlbar/docs/experiments.rst new file mode 100644 index 0000000000..b4fabc8f7a --- /dev/null +++ b/browser/components/urlbar/docs/experiments.rst @@ -0,0 +1,726 @@ +Extensions & Experiments +======================== + +This document describes address bar extensions and experiments: what they are, +how to run them, how to write them, and the processes involved in each. + +The primary purpose right now for writing address bar extensions is to run +address bar experiments. But extensions are useful outside of experiments, and +not all experiments use extensions. + +Like all Firefox extensions, address bar extensions use the WebExtensions_ +framework. + +.. _WebExtensions: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions + +.. contents:: + :depth: 2 + + +WebExtensions +------------- + +**WebExtensions** is the name of Firefox's extension architecture. The "web" +part of the name hints at the fact that Firefox extensions are built using Web +technologies: JavaScript, HTML, CSS, and to a certain extent the DOM. + +Individual extensions themselves often are referred to as *WebExtensions*. For +clarity and conciseness, this document will refer to WebExtensions as +*extensions*. + +Why are we interested in extensions? Mainly because they're a powerful way to +run experiments in Firefox. See Experiments_ for more on that. In addition, we'd +also like to build up a robust set of APIs useful to extension authors, although +right now the API can only be used by Mozilla extensions. + +WebExtensions are introduced and discussed in detail on `MDN +<WebExtensions_>`__. You'll need a lot of that knowledge in order to build +address bar extensions. + +Developing Address Bar Extensions +--------------------------------- + +Overview +~~~~~~~~ + +The address bar WebExtensions API currently lives in two API namespaces, +``browser.urlbar`` and ``browser.experiments.urlbar``. The reason for this is +historical and is discussed in the `Developing Address Bar Extension APIs`_ +section. As a consumer of the API, there are only two important things you need +to know: + +* There's no meaningful difference between the APIs of the two namespaces. + Their kinds of functions, events, and properties are similar. You should + think of the address bar API as one single API that happens to be split into + two namespaces. + +* However, there is a big difference between the two when it comes to setting up + your extension to use them. This is discussed next. + +The ``browser.urlbar`` API namespace is built into Firefox. It's a +**privileged API**, which means that only Mozilla-signed and temporarily +installed extensions can use it. The only thing your Mozilla extension needs to +do in order to use it is to request the ``urlbar`` permission in its +manifest.json, as illustrated `here <urlbarPermissionExample_>`__. + +In contrast, the ``browser.experiments.urlbar`` API namespace is bundled inside +your extension. APIs that are bundled inside extensions are called +**experimental APIs**, and the extensions in which they're bundled are called +**WebExtension experiments**. As with privileged APIs, experimental APIs are +available only to Mozilla-signed and temporarily installed extensions. +("WebExtension experiments" is a term of art and shouldn't be confused with the +general notion of experiments that happen to use extensions.) For more on +experimental APIs and WebExtension experiments, see the `WebExtensions API +implementation documentation <webextAPIImplBasicsDoc_>`__. + +Since ``browser.experiments.urlbar`` is bundled inside your extension, you'll +need to include it in your extension's repo by doing the following: + +1. The implementation consists of two files, api.js_ and schema.json_. In your + extension repo, create a *experiments/urlbar* subdirectory and copy the + files there. See `this repo`__ for an example. + +2. Add the following ``experiment_apis`` key to your manifest.json (see here__ + for an example in context):: + + "experiment_apis": { + "experiments_urlbar": { + "schema": "experiments/urlbar/schema.json", + "parent": { + "scopes": ["addon_parent"], + "paths": [["experiments", "urlbar"]], + "script": "experiments/urlbar/api.js" + } + } + } + +As mentioned, only Mozilla-signed and temporarily installed extensions can use +these two API namespaces. For information on running the extensions you develop +that use these namespaces, see `Running Address Bar Extensions`_. + +.. _urlbarPermissionExample: https://github.com/0c0w3/urlbar-top-sites-experiment/blob/ac1517118bb7ee165fb9989834514b1082575c10/src/manifest.json#L24 +.. _webextAPIImplBasicsDoc: https://firefox-source-docs.mozilla.org/toolkit/components/extensions/webextensions/basics.html +.. _api.js: https://searchfox.org/mozilla-central/source/browser/components/urlbar/tests/ext/api.js +.. _schema.json: https://searchfox.org/mozilla-central/source/browser/components/urlbar/tests/ext/schema.json +__ https://github.com/0c0w3/dynamic-result-type-extension/tree/master/src/experiments/urlbar +__ https://github.com/0c0w3/dynamic-result-type-extension/blob/0987da4b259b9fcb139b31d771883a2f822712b5/src/manifest.json#L28 + +browser.urlbar +~~~~~~~~~~~~~~ + +Currently the only documentation for ``browser.urlbar`` is its `schema +<urlbar.json_>`__. Fortunately WebExtension schemas are JSON and aren't too hard +to read. If you need help understanding it, see the `WebExtensions API +implementation documentation <webextAPIImplDoc_>`__. + +For examples on using the API, see the Cookbook_ section. + +.. _urlbar.json: https://searchfox.org/mozilla-central/source/browser/components/extensions/schemas/urlbar.json + +browser.experiments.urlbar +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +As with ``browser.urlbar``, currently the only documentation for +``browser.experiments.urlbar`` is its schema__. For examples on using the API, +see the Cookbook_ section. + +__ https://searchfox.org/mozilla-central/source/browser/components/urlbar/tests/ext/schema.json + +Workflow +~~~~~~~~ + +The web-ext_ command-line tool makes the extension-development workflow very +simple. Simply start it with the *run* command, passing it the location of the +Firefox binary you want to use. web-ext will launch your Firefox and remain +running until you stop it, watching for changes you make to your extension's +files. When it sees a change, it automatically reloads your extension — in +Firefox, in the background — without your having to do anything. It's really +nice. + +The `web-ext documentation <web-ext commands_>`__ lists all its options, but +here are some worth calling out for the *run* command: + +``--browser-console`` + Automatically open the browser console when Firefox starts. Very useful for + watching your extension's console logging. (Make sure "Show Content Messages" + is checked in the console.) + +``-p`` + This option lets you specify a path to a profile directory. + +``--keep-profile-changes`` + Normally web-ext doesn't save any changes you make to the profile. Use this + option along with ``-p`` to reuse the same profile again and again. + +``--verbose`` + web-ext suppresses Firefox messages in the terminal unless you pass this + option. If you've added some ``dump`` calls in Firefox because you're working + on a new ``browser.urlbar`` API, for example, you won't see them without this. + +web-ext also has a *build* command that packages your extension's files into a +zip file. The following *build* options are useful: + +``--overwrite-dest`` + Without this option, web-ext won't overwrite a zip file it previously created. + +web-ext can load its configuration from your extension's package.json. That's +the recommended way to configure it. Here's an example__. + +Finally, web-ext can also sign extensions, but if you're developing your +extension for an experiment, you'll use a different process for signing. See +`The Experiment Development Process`_. + +.. _web-ext: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/Getting_started_with_web-ext +.. _web-ext commands: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/web-ext_command_reference +__ https://github.com/0c0w3/urlbar-top-sites-experiment/blob/6681a7126986bc2565d036b888cb5b8807397ce5/package.json#L7 + +Automated Tests +~~~~~~~~~~~~~~~ + +It's possible to write `browser chrome mochitests`_ for your extension the same +way we write tests for Firefox. One of the example extensions linked throughout +this document includes a test_, for instance. + +See the readme in the example-addon-experiment_ repo for a workflow. + +.. _browser chrome mochitests: https://developer.mozilla.org/en-US/docs/Mozilla/Browser_chrome_tests +.. _test: https://github.com/0c0w3/urlbar-top-sites-experiment/blob/master/tests/tests/browser/browser_urlbarTopSitesExtension.js + +Cookbook +~~~~~~~~ + +*To be written.* For now, you can find example uses of ``browser.experiments.urlbar`` and ``browser.urlbar`` in the following repos: + +* https://github.com/mozilla-extensions/firefox-quick-suggest-weather +* https://github.com/0c0w3/urlbar-tips-experiment +* https://github.com/0c0w3/urlbar-top-sites-experiment +* https://github.com/0c0w3/urlbar-search-interventions-experiment + +Further Reading +~~~~~~~~~~~~~~~ + +`WebExtensions on MDN <WebExtensions_>`__ + The place to learn about developing WebExtensions in general. + +`Getting started with web-ext <web-ext_>`__ + MDN's tutorial on using web-ext. + +`web-ext command reference <web-ext commands_>`__ + MDN's documentation on web-ext's commands and their options. + +Developing Address Bar Extension APIs +------------------------------------- + +Built-In APIs vs. Experimental APIs +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Originally we developed the address bar extension API in the ``browser.urlbar`` +namespace, which is built into Firefox as discussed above. By "built into +Firefox," we mean that the API is developed in `mozilla-central +<urlbar.json_>`__ and shipped inside Firefox just like any other Firefox +feature. At the time, that seemed like the right thing to do because we wanted +to build an API that ultimately could be used by all extension authors, not only +Mozilla. + +However, there were a number of disadvantages to this development model. The +biggest was that it tightly coupled our experiments to specific versions of +Firefox. For example, if we were working on an experiment that targeted Firefox +72, then any APIs used by that experiment needed to land and ship in 72. If we +weren't able to finish an API by the time 72 shipped, then the experiment would +have to be postponed until 73. Our experiment development timeframes were always +very short because we always wanted to ship our experiments ASAP. Often we +targeted the Firefox version that was then in Nightly; sometimes we even +targeted the version in Beta. Either way, it meant that we were always uplifting +patch after patch to Beta. This tight coupling between Firefox versions and +experiments erased what should have been a big advantage of implementing +experiments as extensions in the first place: the ability to ship experiments +outside the usual cyclical release process. + +Another notable disadvantage of this model was just the cognitive weight of the +idea that we were developing APIs not only for ourselves and our experiments but +potentially for all extensions. This meant that not only did we have to design +APIs to meet our immediate needs, we also had to imagine use cases that could +potentially arise and then design for them as well. + +For these reasons, we stopped developing ``browser.urlbar`` and created the +``browser.experiments.urlbar`` experimental API. As discussed earlier, +experimental APIs are APIs that are bundled inside extensions. Experimental APIs +can do anything that built-in APIs can do with the added flexibility of not +being tied to specific versions of Firefox. + +Adding New APIs +~~~~~~~~~~~~~~~ + +All new address bar APIs should be added to ``browser.experiments.urlbar``. +Although this API does not ship in Firefox, it's currently developed in +mozilla-central, in `browser/components/urlbar/tests/ext/ <extDirectory_>`__ -- +note the "tests" subdirectory. Developing it in mozilla-central lets us take +advantage of our usual build and testing infrastructure. This way we have API +tests running against each mozilla-central checkin, against all versions of +Firefox that are tested on Mozilla's infrastructure, and we're alerted to any +breaking changes we accidentally make. When we start a new extension repo, we +copy schema.json and api.js to it as described earlier (or clone an example repo +with up-to-date copies of these files). + +Generally changes to the API should be reviewed by someone on the address bar +team and someone on the WebExtensions team. Shane (mixedpuppy) is a good +contact. + +.. _extDirectory: https://searchfox.org/mozilla-central/source/browser/components/urlbar/tests/ext/ + +Anatomy of an API +~~~~~~~~~~~~~~~~~ + +Roughly speaking, a WebExtensions API implementation comprises three different +pieces: + +Schema + The schema declares the functions, properties, events, and types that the API + makes available to extensions. Schemas are written in JSON. + + The ``browser.experiments.urlbar`` schema is schema.json_, and the + ``browser.urlbar`` schema is urlbar.json_. + + For reference, the schemas of built-in APIs are in + `browser/components/extensions/schemas`_ and + `toolkit/components/extensions/schemas`_. + + .. _browser/components/extensions/schemas: https://searchfox.org/mozilla-central/source/browser/components/extensions/schemas/ + .. _toolkit/components/extensions/schemas: https://searchfox.org/mozilla-central/source/toolkit/components/extensions/schemas/ + +Internals + Every API hooks into some internal part of Firefox. For the address bar API, + that's the Urlbar implementation in `browser/components/urlbar`_. + + .. _browser/components/urlbar: https://searchfox.org/mozilla-central/source/browser/components/urlbar/ + +Glue + Finally, there's some glue code that implements everything declared in the + schema. Essentially, this code mediates between the previous two pieces. It + translates the function calls, property accesses, and event listener + registrations made by extensions using the public-facing API into terms that + the Firefox internals understand, and vice versa. + + For ``browser.experiments.urlbar``, this is api.js_, and for + ``browser.urlbar``, it's ext-urlbar.js_. + + For reference, the implementations of built-in APIs are in + `browser/components/extensions`_ and `toolkit/components/extensions`_, in the + *parent* and *child* subdirecties. As you might guess, code in *parent* runs + in the main process, and code in *child* runs in the extensions process. + Address bar APIs deal with browser chrome and their implementations therefore + run in the parent process. + + .. _ext-urlbar.js: https://searchfox.org/mozilla-central/source/browser/components/extensions/parent/ext-urlbar.js + .. _browser/components/extensions: https://searchfox.org/mozilla-central/source/browser/components/extensions/ + .. _toolkit/components/extensions: https://searchfox.org/mozilla-central/source/toolkit/components/extensions/ + +Keep in mind that extensions run in a different process from the main process. +That has implications for your APIs. They'll generally need to be async, for +example. + +Further Reading +~~~~~~~~~~~~~~~ + +`WebExtensions API implementation documentation <webextAPIImplDoc_>`__ + Detailed info on implementing a WebExtensions API. + +.. _webextAPIImplDoc: https://firefox-source-docs.mozilla.org/toolkit/components/extensions/webextensions/ + +Running Address Bar Extensions +------------------------------ + +As discussed above, ``browser.experiments.urlbar`` and ``browser.urlbar`` are +privileged APIs. There are two different points to consider when it comes to +running an extension that uses privileged APIs: loading the extension in the +first place, and granting it access to privileged APIs. There's a certain bar +for loading any extension regardless of its API usage that depends on its signed +state and the Firefox build you want to run it in. There's yet a higher bar for +granting it access to privileged APIs. This section discusses how to load +extensions so that they can access privileged APIs. + +Since we're interested in extensions primarily for running experiments, there +are three particular signed states relevant to us: + +Unsigned + There are two ways to run unsigned extensions that use privileged APIs. + + They can be loaded temporarily using a Firefox Nightly build or + Developer Edition but not Beta or Release [source__], and the + ``extensions.experiments.enabled`` preference must be set to true [source__]. + You can load extensions temporarily by visiting + about:debugging#/runtime/this-firefox and clicking "Load Temporary Add-on." + `web-ext <Workflow_>`__ also loads extensions temporarily. + + __ https://searchfox.org/mozilla-central/rev/053826b10f838f77c27507e5efecc96e34718541/toolkit/components/extensions/Extension.jsm#1884 + __ https://searchfox.org/mozilla-central/rev/014fe72eaba26dcf6082fb9bbaf208f97a38594e/toolkit/mozapps/extensions/internal/AddonSettings.jsm#93 + + They can be also be loaded normally (not temporarily) in a custom build where + the build-time setting ``AppConstants.MOZ_REQUIRE_SIGNING`` [source__, source__] + and ``xpinstall.signatures.required`` pref are both false. As in the previous + paragraph, such builds include Nightly and Developer Edition but not Beta or + Release [source__]. In addition, your custom build must modify the + ``Extension.isPrivileged`` getter__ to return true. This getter determines + whether an extension can access privileged APIs. + + __ https://searchfox.org/mozilla-central/rev/053826b10f838f77c27507e5efecc96e34718541/toolkit/mozapps/extensions/internal/XPIProvider.jsm#2382 + __ https://searchfox.org/mozilla-central/rev/053826b10f838f77c27507e5efecc96e34718541/toolkit/mozapps/extensions/internal/AddonSettings.jsm#36 + __ https://searchfox.org/mozilla-central/search?q=MOZ_REQUIRE_SIGNING&case=false®exp=false&path= + __ https://searchfox.org/mozilla-central/rev/053826b10f838f77c27507e5efecc96e34718541/toolkit/components/extensions/Extension.jsm#1874 + + Extensions remain unsigned as you develop them. See the Workflow_ section for + more. + +Signed for testing (Signed for QA) + Signed-for-testing extensions that use privileged APIs can be run using the + same techniques for running unsigned extensions. + + They can also be loaded normally (not temporarily) if you use a Firefox build + where the build-time setting ``AppConstants.MOZ_REQUIRE_SIGNING`` is false and + you set the ``xpinstall.signatures.dev-root`` pref to true + [source__]. ``xpinstall.signatures.dev-root`` does not exist by default and + must be created. + + __ https://searchfox.org/mozilla-central/rev/053826b10f838f77c27507e5efecc96e34718541/toolkit/mozapps/extensions/internal/XPIInstall.jsm#262 + + You encounter extensions that are signed for testing when you are writing + extensions for experiments. See the Experiments_ section for details. + + "Signed for QA" is another way of referring to this signed state. + +Signed for release + Signed-for-release extensions that use privileged APIs can be run in any + Firefox build with no special requirements. + + You encounter extensions that are signed for release when you are writing + extensions for experiments. See the Experiments_ section for details. + +.. important:: + To see console logs from extensions in the browser console, select the "Show + Content Messages" option in the console's settings. This is necessary because + extensions run outside the main process. + +Experiments +----------- + +**Experiments** let us try out ideas in Firefox outside the usual release cycle +and on particular populations of users. + +For example, say we have a hunch that the top sites shown on the new-tab page +aren't very discoverable, so we want to make them more visible. We have one idea +that might work — show them every time the user begins an interaction with the +address bar — but we aren't sure how good an idea it is. So we test it. We write +an extension that does just that, make sure it collects telemetry that will help +us answer our question, ship it outside the usual release cycle to a small +percentage of Beta users, collect and analyze the telemetry, and determine +whether the experiment was successful. If it was, then we might want to ship the +feature to all Firefox users. + +Experiments sometimes are also called **studies** (not to be confused with *user +studies*, which are face-to-face interviews with users conducted by user +researchers). + +There are two types of experiments: + +Pref-flip experiments + Pref-flip experiments are simple. If we have a fully baked feature in the + browser that's preffed off, a pref-flip experiment just flips the pref on, + enabling the feature for users running the experiment. No code is required. + We tell the experiments team the name of the pref we want to flip, and they + handle it. + + One important caveat to pref-flip studies is that they're currently capable of + flipping only a single pref. There's an extension called Multipreffer_ that + can flip multiple prefs, though. + + .. _Multipreffer: https://github.com/mozilla/multipreffer + +Add-on experiments + Add-on experiments are much more complex but much more powerful. (Here + *add-on* is a synonym for extension.) They're the type of experiments that + this document has been discussing all along. + + An add-on experiment is shipped as an extension that we write and that + implements the experimental feature we want to test. To reiterate, the + extension is a WebExtension and uses WebExtensions APIs. If the current + WebExtensions APIs do not meet the needs of your experiment, then you must + create either experimental or built-in APIs so that your extension can use + them. If necessary, you can make any new built-in APIs privileged so that they + are available only to Mozilla extensions. + + An add-on experiment can collect additional telemetry that's not collected in + the product by using the privileged ``browser.telemetry`` WebExtensions API, + and of course the product will continue to collect all the telemetry it + usually does. The telemetry pings from users running the experiment will be + correlated with the experiment with no extra work on our part. + +A single experiment can deliver different UXes to different groups of users +running the experiment. Each group or UX within an experiment is called a +**branch**. Experiments often have two branches, control and treatment. The +**control branch** actually makes no UX changes. It may capture additional +telemetry, though. Think of it as the control in a science experiment. It's +there so we can compare it to data from the **treatment branch**, which does +make UX changes. Some experiments may require multiple treatment branches, in +which case the different branches will have different names. Add-on experiments +can implement all branches in the same extension or each branch in its own +extension. + +Experiments are delivered to users by a system called **Normandy**. Normandy +comprises a client side that lives in Firefox and a server side. In Normandy, +experiments are defined server-side in files called **recipes**. Recipes include +information about the experiment like the Firefox release channel and version +that the experiment targets, the number of users to be included in the +experiment, the branches in the experiment, the percentage of users on each +branch, and so on. + +Experiments are tracked by Mozilla project management using a system called +Experimenter_. + +Finally, there was an older version of the experiments program called +**Shield**. Experiments under this system were called **Shield studies** and +could be be shipped as extensions too. + +.. _Experimenter: https://experimenter.services.mozilla.com/ + +Further Reading +~~~~~~~~~~~~~~~ + +`Pref-Flip and Add-On Experiments <https://mana.mozilla.org/wiki/pages/viewpage.action?spaceKey=FIREFOX&title=Pref-Flip+and+Add-On+Experiments>`__ + A comprehensive document on experiments from the Experimenter team. See the + child pages in the sidebar, too. + +`Client Implementation Guidelines for Experiments <https://docs.telemetry.mozilla.org/cookbooks/client_guidelines.html>`_ + Relevant documentation from the telemetry team. + +#ask-experimenter Slack channel + A friendly place to get answers to your experiment questions. + +The Experiment Development Process +---------------------------------- + +This section describes an experiment's life cycle. + +1. Experiments usually originate with product management and UX. They're + responsible for identifying a problem, deciding how an experiment should + approach it, the questions we want to answer, the data we need to answer + those questions, the user population that should be enrolled in the + experiment, the definition of success, and so on. + +2. UX makes a spec that describes what the extension looks like and how it + behaves. + +3. There's a kickoff meeting among the team to introduce the experiment and UX + spec. It's an opportunity for engineering to ask questions of management, UX, + and data science. It's really important for engineering to get a precise and + accurate understanding of how the extension is supposed to behave — right + down to the UI changes — so that no one makes erroneous assumptions during + development. + +4. At some point around this time, the team (usually management) creates a few + artifacts to track the work and facilitate communication with outside teams + involved in shipping experiments. They include: + + * A page on `Experimenter <Experiments_>`__ + * A QA PI (product integrity) request so that QA resources are allocated + * A bug in `Data Science :: Experiment Collaboration`__ so that data science + can track the work and discuss telemetry (engineering might file this one) + + __ https://bugzilla.mozilla.org/enter_bug.cgi?assigned_to=nobody%40mozilla.org&bug_ignored=0&bug_severity=normal&bug_status=NEW&bug_type=task&cf_firefox_messaging_system=---&cf_fx_iteration=---&cf_fx_points=---&comment=%23%23%20Brief%20Description%20of%20the%20request%20%28required%29%3A%0D%0A%0D%0A%23%23%20Business%20purpose%20for%20this%20request%20%28required%29%3A%0D%0A%0D%0A%23%23%20Requested%20timelines%20for%20the%20request%20or%20how%20this%20fits%20into%20roadmaps%20or%20critical%20decisions%20%28required%29%3A%0D%0A%0D%0A%23%23%20Links%20to%20any%20assets%20%28e.g%20Start%20of%20a%20PHD%2C%20BRD%3B%20any%20document%20that%20helps%20describe%20the%20project%29%3A%0D%0A%0D%0A%23%23%20Name%20of%20Data%20Scientist%20%28If%20Applicable%29%3A%0D%0A%0D%0A%2APlease%20note%20if%20it%20is%20found%20that%20not%20enough%20information%20has%20been%20given%20this%20will%20delay%20the%20triage%20of%20this%20request.%2A&component=Experiment%20Collaboration&contenttypemethod=list&contenttypeselection=text%2Fplain&filed_via=standard_form&flag_type-4=X&flag_type-607=X&flag_type-800=X&flag_type-803=X&flag_type-936=X&form_name=enter_bug&maketemplate=Remember%20values%20as%20bookmarkable%20template&op_sys=Unspecified&priority=--&product=Data%20Science&rep_platform=Unspecified&target_milestone=---&version=unspecified + +5. Engineering breaks down the work and files bugs. There's another engineering + meeting to discuss the breakdown, or it's discussed asynchronously. + +6. Engineering sets up a GitHub repo for the extension. See `Implementing + Experiments`_ for an example repo you can clone to get started. Disable + GitHub Issues on the repo so that QA will file bugs in Bugzilla instead of + GitHub. There's nothing wrong with GitHub Issues, but our team's project + management tracks all work through Bugzilla. If it's not there, it's not + captured. + +7. Engineering or management fills out the Add-on section of the Experimenter + page as much as possible at this point. "Active Experiment Name" isn't + necessary, and "Signed Release URL" won't be available until the end of the + process. + +8. Engineering implements the extension and any new WebExtensions APIs it + requires. + +9. When the extension is done, engineering or management clicks the "Ready for + Sign-Off" button on the Experimenter page. That changes the page's status + from "Draft" to "Ready for Sign-Off," which allows QA and other teams to sign + off on their portions of the experiment. + +10. Engineering requests the extension be signed "for testing" (or "for + QA"). Michael (mythmon) from the Experiments team and Rehan (rdalal) from + Services Engineering are good contacts. Build the extension zip file using + web-ext as discussed in Workflow_. Attach it to a bug (a metabug for + implementing the extension, for example), needinfo Michael or Rehan, and ask + him to sign it. He'll attach the signed version to the bug. If neither + Michael nor Rehan is available, try asking in the #ask-experimenter Slack + channel. + +11. Engineering sends QA the link to the signed extension and works with them to + resolve bugs they find. + +12. When QA signs off, engineering asks Michael to sign the extension "for + release" using the same needinfo process described earlier. + +13. Paste the URL of the signed extension in the "Signed Release URL" textbox of + the Add-on section of the Experimenter page. + +14. Other teams sign off as they're ready. + +15. The experiment ships! 🎉 + + +Implementing Experiments +------------------------ + +This section discusses how to implement add-on experiments. Pref-flip +experiments are much simpler and don't need a lot of explanation. You should be +familiar with the concepts discussed in the `Developing Address Bar Extensions`_ +and `Running Address Bar Extensions`_ sections before reading this one. + +The most salient thing about add-on experiments is that they're implemented +simply as privileged extensions. Other than being privileged and possibly +containing bundled experimental APIs, they're similar to all other extensions. + +The `top-sites experiment extension <topSites_>`__ is an example of a real, +shipped experiment. + +.. _topSites: https://github.com/0c0w3/urlbar-top-sites-experiment + +Setup +~~~~~ + +example-addon-experiment_ is a repo you can clone to get started. It's geared +toward urlbar extensions and includes the stub of a browser chrome mochitest. + +.. _example-addon-experiment: https://github.com/0c0w3/example-addon-experiment + +browser.normandyAddonStudy +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +As discussed in Experiments_, an experiment typically has more than one branch +so that it can test different UXes. The experiment's extension(s) needs to know +the branch the user is enrolled in so that it can behave appropriately for the +branch: show the user the proper UX, collect the proper telemetry, and so on. + +This is the purpose of the ``browser.normandyAddonStudy`` WebExtensions API. +Like ``browser.urlbar``, it's a privileged API available only to Mozilla +extensions. + +Its schema is normandyAddonStudy.json_. + +It's a very simple API. The primary function is ``getStudy``, which returns the +study the user is currently enrolled in or null if there isn't one. (Recall that +*study* is a synonym for *experiment*.) One of the first things an experiment +extension typically does is to call this function. + +The Normandy client in Firefox will keep an experiment extension installed only +while the experiment is active. Therefore, ``getStudy`` should always return a +non-null study object. Nevertheless, the study object has an ``active`` boolean +property that's trivial to sanity check. (The example extension does.) + +The more important property is ``branch``, the name of the branch that the user +is enrolled in. Your extension should use it to determine the appropriate UX. + +Finally, there's an ``onUnenroll`` event that's fired when the user is +unenrolled in the study. It's not quite clear in what cases an extension would +need to listen for this event given that Normandy automatically uninstalls +extensions on unenrollment. Maybe if they create some persistent state that's +not automatically undone on uninstall by the WebExtensions framework? + +If your extension itself needs to unenroll the user for some reason, call +``endStudy``. + +.. _normandyAddonStudy.json: https://searchfox.org/mozilla-central/source/browser/components/extensions/schemas/normandyAddonStudy.json + +Telemetry +~~~~~~~~~ + +Experiments can capture telemetry in two places: in the product itself and +through the privileged ``browser.telemetry`` WebExtensions API. The API schema +is telemetry.json_. + +The telemetry pings from users running experiments are automatically correlated +with those experiments, no extra work required. That's true regardless of +whether the telemetry is captured in the product or though +``browser.telemetry``. + +The address bar has some in-product, preffed off telemetry that we want to +enable for all our experiments — at least that's the thinking as of August 2019. +It's called `engagement event telemetry`_, and it records user *engagements* +with and *abandonments* of the address bar [source__]. We added a +BrowserSetting_ on ``browser.urlbar`` just to let us flip the pref and enable +this telemetry in our experiment extensions. Call it like this:: + + await browser.urlbar.engagementTelemetry.set({ value: true }); + +.. _telemetry.json: https://searchfox.org/mozilla-central/source/toolkit/components/extensions/schemas/telemetry.json +.. _engagement event telemetry: https://bugzilla.mozilla.org/show_bug.cgi?id=1559136 +__ https://searchfox.org/mozilla-central/rev/7088fc958db5935eba24b413b1f16d6ab7bd13ea/browser/components/urlbar/UrlbarController.jsm#598 +.. _BrowserSetting: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/types/BrowserSetting + +Engineering Best Practices +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Clear up questions with your UX person early and often. There's often a gap +between what they have in their mind and what you have in yours. Nothing wrong +with that, it's just the nature of development. But misunderstandings can cause +big problems when they're discovered late. This is especially true of UX +behaviors, as opposed to visuals or styling. It's no fun to realize at the end +of a release cycle that you've designed the wrong WebExtensions API because some +UX detail was overlooked. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Related to the previous point, make builds of your extension for your UX person +so they can test it. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Taking the previous point even further, if your experiment will require a +substantial new API(s), you might think about prototyping the experiment +entirely in a custom Firefox build before designing the API at all. Give it to +your UX person. Let them disect it and tell you all the problems with it. Fill +in all the gaps in your understanding, and then design the API. We've never +actually done this, though. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +It's a good idea to work on the extension as you're designing and developing the +APIs it'll use. You might even go as far as writing the first draft of the +extension before even starting to implement the APIs. That lets you spot +problems that may not be obvious were you to design the API in isolation. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Your extension's ID should end in ``@shield.mozilla.org``. QA will flag it if it +doesn't. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Set ``"hidden": true`` in your extension's manifest.json. That hides it on +about:addons. (It can still be seen on about:studies.) QA will spot this if you +don't. + +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +There are drawbacks of hiding features behind prefs and enabling them in +experiment extensions. Consider not doing that if feasible, or at least weigh +these drawbacks against your expected benefits. + +* Prefs stay flipped on in private windows, but experiments often have special + requirements around private-browsing mode (PBM). Usually, they shouldn't be + active in PBM at all, unless of course the point of the experiment is to test + PBM. Extensions also must request PBM access ("incognito" in WebExtensions + terms), and the user can disable access at any time. The result is that part + of your experiment could remain enabled — the part behind the pref — while + other parts are disabled. + +* Prefs stay flipped on in safe mode, even though your extension (like all + extensions) will be disabled. This might be a bug__ in the WebExtensions + framework, though. + + __ https://bugzilla.mozilla.org/show_bug.cgi?id=1576997 diff --git a/browser/components/urlbar/docs/firefox-suggest-telemetry.rst b/browser/components/urlbar/docs/firefox-suggest-telemetry.rst new file mode 100644 index 0000000000..a692f45cfc --- /dev/null +++ b/browser/components/urlbar/docs/firefox-suggest-telemetry.rst @@ -0,0 +1,1533 @@ +Firefox Suggest Telemetry +========================= + +This document describes the telemetry that Firefox records for the Firefox +Suggest feature. That is, it describes Firefox Suggest telemetry recorded on the +client. It also discusses the data that Firefox sends to the Merino service. + +For information on other telemetry related to the address bar, see the general +address bar :doc:`telemetry` document. For information on all telemetry in +Firefox, see the toolkit :doc:`/toolkit/components/telemetry/index` document. + +.. contents:: + :depth: 2 + + +Histograms +---------- + +The following histograms are recorded for Firefox Suggest. For general +information on histogram telemetry in Firefox, see the +:doc:`/toolkit/components/telemetry/collection/histograms` document. + +FX_URLBAR_MERINO_LATENCY_MS +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This histogram records the latency in milliseconds of the Merino source for +suggestions, or in other words, the time from Firefox's request to the Merino +server to the time Firefox receives a response. It is an exponential histogram +with 50 buckets and values between 0 and 30000 (0s and 30s). + +Changelog + Firefox 93.0 + Introduced. [Bug 1727799_] + +.. _1727799: https://bugzilla.mozilla.org/show_bug.cgi?id=1727799 + +FX_URLBAR_MERINO_LATENCY_WEATHER_MS +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This histogram records the latency in milliseconds of weather suggestions from +Merino. It is updated in addition to ``FX_URLBAR_MERINO_LATENCY_MS`` and has the +same properties. It is an exponential histogram with 50 buckets and values +between 0 and 30000 (0s and 30s). + +Changelog + Firefox 110.0 + Introduced. [Bug 1804536_] + +.. _1804536: https://bugzilla.mozilla.org/show_bug.cgi?id=1804536 + +FX_URLBAR_MERINO_RESPONSE +~~~~~~~~~~~~~~~~~~~~~~~~~ + +This categorical histogram records a summary of each fetch from the Merino +server. It has the following categories: + +:0 "success": + The fetch completed without any error before the timeout elapsed and it + included at least one suggestion. (Before Firefox 110.0, this category meant + simply that the fetch completed without any error before the timeout elapsed + regardless of whether it included any suggestions.) +:1 "timeout": + The timeout elapsed before the fetch completed or otherwise failed. +:2 "network_error": + The fetch failed due to a network error before the timeout elapsed. e.g., the + user's network or the Merino server was down. +:3 "http_error": + The fetch completed before the timeout elapsed but the server returned an + error. +:4 "no_suggestion": + The fetch completed without any error before the timeout elapsed and it did + not include any suggestions. + +Changelog + Firefox 94.0.2 + Introduced. [Bug 1737923_] + + Firefox 110.0 + Added the ``no_suggestion`` category. The meaning of the ``success`` + category was changed from "The fetch completed without any error before the + timeout elapsed" to "The fetch completed without any error before the + timeout elapsed and it included at least one suggestion." [Bug 1804536_] + +.. _1737923: https://bugzilla.mozilla.org/show_bug.cgi?id=1737923 +.. _1804536: https://bugzilla.mozilla.org/show_bug.cgi?id=1804536 + +FX_URLBAR_MERINO_RESPONSE_WEATHER +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This categorical histogram records a summary of each fetch for weather +suggestions from the Merino server. It is updated in addition to +``FX_URLBAR_MERINO_RESPONSE`` and has the same categories. + +:0 "success": + The fetch completed without any error before the timeout elapsed and it + included at least one suggestion. +:1 "timeout": + The timeout elapsed before the fetch completed or otherwise failed. +:2 "network_error": + The fetch failed due to a network error before the timeout elapsed. e.g., the + user's network or the Merino server was down. +:3 "http_error": + The fetch completed before the timeout elapsed but the server returned an + error. +:4 "no_suggestion": + The fetch completed without any error before the timeout elapsed and it did + not include any suggestions. + +Changelog + Firefox 110.0 + Introduced. [Bug 1804536_] + +.. _1804536: https://bugzilla.mozilla.org/show_bug.cgi?id=1804536 + +FX_URLBAR_QUICK_SUGGEST_REMOTE_SETTINGS_LATENCY_MS +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This histogram records the latency in milliseconds of the remote settings source +for suggestions, or in other words, the time from when Firefox starts fetching a +suggestion from remote settings to the time the suggestion is retrieved. It is +an exponential histogram with 50 buckets and values between 0 and 30000 (0s and +30s). + +Note that unlike Merino, fetches from remote settings happen entirely on the +client, so remote settings latencies are expected to be much smaller than Merino +latencies. + +Changelog + Firefox 94.0.2 + Introduced. [Bug 1737651_] + +.. _1737651: https://bugzilla.mozilla.org/show_bug.cgi?id=1737651 + +Scalars +------- + +The following scalars are recorded for Firefox Suggest. For general information +on scalar telemetry in Firefox, see the +:doc:`/toolkit/components/telemetry/collection/scalars` document. + +browser.ui.interaction.preferences_panePrivacy +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar is incremented each time the user clicks a Firefox Suggest +checkbox or toggle switch in the preferences UI. Keys are the following: + +:firefoxSuggestBestMatch: + This key is incremented when the "Top pick" checkbox is clicked. +:firefoxSuggestBestMatchLearnMore: + This key is incremented when opening the learn more link for best match. +:firefoxSuggestDataCollectionToggle: + This key is incremented when the toggle switch for data collection + is clicked. +:firefoxSuggestNonsponsoredToggle: + This key is incremented when the toggle switch for non-sponsored suggestions + is clicked. +:firefoxSuggestSponsoredToggle: + This key is incremented when the toggle switch for sponsored suggestions + is clicked. + +Changelog + Firefox 94.0.2 + Introduced firefoxSuggestDataCollectionToggle, + firefoxSuggestNonsponsoredToggle and firefoxSuggestSponsoredToggle. + [Bug 1735976_] + + Firefox 99.0 + Introduced firefoxSuggestBestMatch. [Bug 1755100_] + Introduced firefoxSuggestBestMatchLearnMore. [Bug 1756917_] + +.. _1735976: https://bugzilla.mozilla.org/show_bug.cgi?id=1735976 +.. _1755100: https://bugzilla.mozilla.org/show_bug.cgi?id=1755100 +.. _1756917: https://bugzilla.mozilla.org/show_bug.cgi?id=1756917 + +contextual.services.quicksuggest.block_dynamic_wikipedia +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar is incremented each time the user dismisses ("blocks") a +dynamic wikipedia suggestion. Each key is the index at which a suggestion +appeared in the results (1-based), and the corresponding value is the number +of dismissals at that index. + +Changelog + Firefox 109.0 + Introduced. [Bug 1800993_] + +.. _1800993: https://bugzilla.mozilla.org/show_bug.cgi?id=1800993 + +contextual.services.quicksuggest.block_nonsponsored +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar is incremented each time the user dismisses ("blocks") a +non-sponsored suggestion, including both best matches and the usual +non-best-match suggestions. Each key is the index at which a suggestion appeared +in the results (1-based), and the corresponding value is the number of +dismissals at that index. + +Changelog + Firefox 101.0 + Introduced. [Bug 1761059_] + +.. _1761059: https://bugzilla.mozilla.org/show_bug.cgi?id=1761059 + +contextual.services.quicksuggest.block_nonsponsored_bestmatch +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar is incremented each time the user dismisses ("blocks") a +non-sponsored best match. Each key is the index at which a suggestion appeared +in the results (1-based), and the corresponding value is the number of +dismissals at that index. + +Changelog + Firefox 101.0 + Introduced. [Bug 1761059_] + +.. _1761059: https://bugzilla.mozilla.org/show_bug.cgi?id=1761059 + +contextual.services.quicksuggest.block_sponsored +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar is incremented each time the user dismisses ("blocks") a +sponsored suggestion, including both best matches and the usual non-best-match +suggestions. Each key is the index at which a suggestion appeared in the results +(1-based), and the corresponding value is the number of dismissals at that +index. + +Changelog + Firefox 101.0 + Introduced. [Bug 1761059_] + +.. _1761059: https://bugzilla.mozilla.org/show_bug.cgi?id=1761059 + +contextual.services.quicksuggest.block_sponsored_bestmatch +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar is incremented each time the user dismisses ("blocks") a +sponsored best match. Each key is the index at which a suggestion appeared in +the results (1-based), and the corresponding value is the number of dismissals +at that index. + +Changelog + Firefox 101.0 + Introduced. [Bug 1761059_] + +.. _1761059: https://bugzilla.mozilla.org/show_bug.cgi?id=1761059 + +contextual.services.quicksuggest.block_weather +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar is incremented each time the user dismisses ("blocks") a +Firefox Suggest weather suggestion. Each key is the index at which a suggestion +appeared in the results (1-based), and the corresponding value is the number of +dismissals at that index. + +Changelog + Firefox 110.0 + Introduced. [Bug 1804536_] + +.. _1804536: https://bugzilla.mozilla.org/show_bug.cgi?id=1804536 + +contextual.services.quicksuggest.click +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar is incremented each time the user picks a suggestion. Each key +is the index at which a suggestion appeared in the results (1-based), and the +corresponding value is the number of clicks at that index. + +Changelog + Firefox 87.0 + Introduced. [Bug 1693927_] + + Firefox 109.0 + Removed. [Bug 1800993_] + +.. _1693927: https://bugzilla.mozilla.org/show_bug.cgi?id=1693927 +.. _1800993: https://bugzilla.mozilla.org/show_bug.cgi?id=1800993 + +contextual.services.quicksuggest.click_dynamic_wikipedia +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar is incremented each time the user picks a dynamic +wikipedia suggestion. Each key is the index at which a suggestion appeared +in the results (1-based), and the corresponding value is the number of +clicks at that index. + +Changelog + Firefox 109.0 + Introduced. [Bug 1800993_] + +.. _1800993: https://bugzilla.mozilla.org/show_bug.cgi?id=1800993 + +contextual.services.quicksuggest.click_nav_notmatched +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar records how many times a heuristic result was clicked while a +navigational suggestion was absent. It is recorded only when the Nimbus variable +``recordNavigationalSuggestionTelemetry`` is true. (The variable is false by +default.) + +Each key is the type of heuristic result that was clicked. Key names are the +same as the heuristic result type names recorded in Glean telemetry. + +Changelog + Firefox 112.0 + Introduced. [Bug 1819797_] + +.. _1819797: https://bugzilla.mozilla.org/show_bug.cgi?id=1819797 + +contextual.services.quicksuggest.click_nav_shown_heuristic +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar records how many times a heuristic result was clicked while a +navigational suggestion was present. It is recorded only when the Nimbus +variable ``recordNavigationalSuggestionTelemetry`` is true. (The variable is +false by default.) + +Each key is the type of heuristic result that was clicked. Key names are the +same as the heuristic result type names recorded in Glean telemetry. + +Changelog + Firefox 112.0 + Introduced. [Bug 1819797_] + +.. _1819797: https://bugzilla.mozilla.org/show_bug.cgi?id=1819797 + +contextual.services.quicksuggest.click_nav_shown_nav +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar records how many times a navigational suggestion was clicked. +It is recorded only when the Nimbus variable +``recordNavigationalSuggestionTelemetry`` is true. (The variable is false by +default.) + +Each key is the type of heuristic result that was present at the time of the +engagement. Key names are the same as the heuristic result type names recorded +in Glean telemetry. + +Changelog + Firefox 112.0 + Introduced. [Bug 1819797_] + +.. _1819797: https://bugzilla.mozilla.org/show_bug.cgi?id=1819797 + +contextual.services.quicksuggest.click_nav_superceded +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar records how many times a heuristic result was clicked when a +navigational suggestion was matched but superseded by the heuristic. It is +recorded only when the Nimbus variable ``recordNavigationalSuggestionTelemetry`` +is true. (The variable is false by default.) + +Each key is the type of heuristic result that was clicked. Key names are the +same as the heuristic result type names recorded in Glean telemetry. + +Changelog + Firefox 112.0 + Introduced. [Bug 1819797_] + +.. _1819797: https://bugzilla.mozilla.org/show_bug.cgi?id=1819797 + +contextual.services.quicksuggest.click_nonsponsored +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar is incremented each time the user picks a non-sponsored +suggestion. Each key is the index at which a suggestion appeared in the +results (1-based), and the corresponding value is the number of clicks at +that index. + +Changelog + Firefox 109.0 + Introduced. [Bug 1800993_] + +.. _1800993: https://bugzilla.mozilla.org/show_bug.cgi?id=1800993 + +contextual.services.quicksuggest.click_nonsponsored_bestmatch +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar is incremented each time the user picks a non-sponsored best +match. Each key is the index at which a suggestion appeared in the results +(1-based), and the corresponding value is the number of clicks at that index. + +Changelog + Firefox 99.0 + Introduced. [Bug 1752953_] + +.. _1752953: https://bugzilla.mozilla.org/show_bug.cgi?id=1752953 + +contextual.services.quicksuggest.click_sponsored +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar is incremented each time the user picks a sponsored suggestion. +Each key is the index at which a suggestion appeared in the results (1-based), +and the corresponding value is the number of clicks at that index. + +Changelog + Firefox 109.0 + Introduced. [Bug 1800993_] + +.. _1800993: https://bugzilla.mozilla.org/show_bug.cgi?id=1800993 + +contextual.services.quicksuggest.click_sponsored_bestmatch +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar is incremented each time the user picks a sponsored best +match. Each key is the index at which a suggestion appeared in the results +(1-based), and the corresponding value is the number of clicks at that index. + +Changelog + Firefox 99.0 + Introduced. [Bug 1752953_] + +.. _1752953: https://bugzilla.mozilla.org/show_bug.cgi?id=1752953 + +contextual.services.quicksuggest.click_weather +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar is incremented each time the user picks a weather suggestion. +Each key is the index at which a suggestion appeared in the results (1-based), +and the corresponding value is the number of clicks at that index. + +Changelog + Firefox 110.0 + Introduced. [Bug 1804536_] + +.. _1804536: https://bugzilla.mozilla.org/show_bug.cgi?id=1804536 + +contextual.services.quicksuggest.exposure_weather +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar records weather suggestion exposures. It is incremented each +time the user is shown a weather suggestion. It can be compared to the +``urlbar.zeroprefix.exposure`` scalar (see :doc:`telemetry`) to determine the +percentage of zero-prefix exposures that included weather suggestions. + +Each key is the index at which a suggestion appeared in the results (1-based), +and the corresponding value is the number of exposures at that index. + +Changelog + Firefox 110.0 + Introduced. [Bug 1806765_] + + Firefox 114.0 + Removed since the weather suggestion is no longer triggered on zero prefix. + [Bug 1831971_] + +.. _1806765: https://bugzilla.mozilla.org/show_bug.cgi?id=1806765 +.. _1831971: https://bugzilla.mozilla.org/show_bug.cgi?id=1831971 + +contextual.services.quicksuggest.help +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar is incremented each time the user picks the help button in a +suggestion. Each key is the index at which a suggestion appeared in the results +(1-based), and the corresponding value is the number of help button clicks at +that index. + +Changelog + Firefox 87.0 + Introduced. [Bug 1693927_] + + Firefox 109.0 + Removed. [Bug 1800993_] + +.. _1693927: https://bugzilla.mozilla.org/show_bug.cgi?id=1693927 +.. _1800993: https://bugzilla.mozilla.org/show_bug.cgi?id=1800993 + +contextual.services.quicksuggest.help_dynamic_wikipedia +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar is incremented each time the user picks the help button in a +dynamic wikipedia suggestion. Each key is the index at which a suggestion +appeared in the results (1-based), and the corresponding value is the number +of help button clicks at that index. + +Changelog + Firefox 109.0 + Introduced. [Bug 1800993_] + +.. _1800993: https://bugzilla.mozilla.org/show_bug.cgi?id=1800993 + +contextual.services.quicksuggest.help_nonsponsored +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar is incremented each time the user picks the help button in a +non-sponsored suggestion. Each key is the index at which a suggestion appeared in the +results (1-based), and the corresponding value is the number of help button clicks +at that index. + +Changelog + Firefox 109.0 + Introduced. [Bug 1800993_] + +.. _1800993: https://bugzilla.mozilla.org/show_bug.cgi?id=1800993 + +contextual.services.quicksuggest.help_nonsponsored_bestmatch +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar is incremented each time the user picks the help button in a +non-sponsored best match. Each key is the index at which a suggestion appeared +in the results (1-based), and the corresponding value is the number of help +button clicks at that index. + +Changelog + Firefox 99.0 + Introduced. [Bug 1752953_] + +.. _1752953: https://bugzilla.mozilla.org/show_bug.cgi?id=1752953 + +contextual.services.quicksuggest.help_sponsored +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar is incremented each time the user picks the help button in a +sponsored suggestion. Each key is the index at which a suggestion appeared in the +results (1-based), and the corresponding value is the number of help button clicks +at that index. + +Changelog + Firefox 109.0 + Introduced. [Bug 1800993_] + +.. _1800993: https://bugzilla.mozilla.org/show_bug.cgi?id=1800993 + +contextual.services.quicksuggest.help_sponsored_bestmatch +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar is incremented each time the user picks the help button in a +sponsored best match. Each key is the index at which a suggestion appeared in +the results (1-based), and the corresponding value is the number of help button +clicks at that index. + +Changelog + Firefox 99.0 + Introduced. [Bug 1752953_] + +.. _1752953: https://bugzilla.mozilla.org/show_bug.cgi?id=1752953 + +contextual.services.quicksuggest.help_weather +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar is incremented each time the user picks the help button in a +weather suggestion. Each key is the index at which a suggestion appeared in the +results (1-based), and the corresponding value is the number of help button +clicks at that index. + +Changelog + Firefox 110.0 + Introduced. [Bug 1804536_] + +.. _1804536: https://bugzilla.mozilla.org/show_bug.cgi?id=1804536 + +contextual.services.quicksuggest.impression +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar records suggestion impressions. It is incremented each time +the user is shown a suggestion and the following two conditions hold: + +- The user has completed an engagement with the address bar by picking a result + in it or by pressing the Enter key. +- At the time the user completed the engagement, a suggestion was present in the + results. + +Each key is the index at which a suggestion appeared in the results (1-based), +and the corresponding value is the number of impressions at that index. + +Changelog + Firefox 87.0 + Introduced. [Bug 1693927_] + + Firefox 109.0 + Removed. [Bug 1800993_] + +.. _1693927: https://bugzilla.mozilla.org/show_bug.cgi?id=1693927 +.. _1800993: https://bugzilla.mozilla.org/show_bug.cgi?id=1800993 + +contextual.services.quicksuggest.impression_dynamic_wikipedia +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar records dynamic wikipedia impressions. It is incremented +each time the user is shown a dynamic wikipedia suggestion and the following +two conditions hold: + +- The user has completed an engagement with the address bar by picking a result + in it or by pressing the Enter key. +- At the time the user completed the engagement, a dynamic wikipedia suggestion + was present in the results. + +Each key is the index at which a suggestion appeared in the results (1-based), +and the corresponding value is the number of impressions at that index. + +Changelog + Firefox 109.0 + Introduced. [Bug 1800993_] + +.. _1800993: https://bugzilla.mozilla.org/show_bug.cgi?id=1800993 + +contextual.services.quicksuggest.impression_nav_notmatched +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar records how many times a urlbar engagement occurred while a +navigational suggestion was absent. It is recorded only when the Nimbus variable +``recordNavigationalSuggestionTelemetry`` is true. (The variable is false by +default.) + +Each key is the type of heuristic result that was present at the time of the +engagement. Key names are the same as the heuristic result type names recorded +in Glean telemetry. + +Changelog + Firefox 112.0 + Introduced. [Bug 1819797_] + +.. _1819797: https://bugzilla.mozilla.org/show_bug.cgi?id=1819797 + +contextual.services.quicksuggest.impression_nav_shown +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar records how many times a urlbar engagement occurred while a +navigational suggestion was present. It is recorded only when the Nimbus +variable ``recordNavigationalSuggestionTelemetry`` is true. (The variable is +false by default.) + +Each key is the type of heuristic result that was present at the time of the +engagement. Key names are the same as the heuristic result type names recorded +in Glean telemetry. + +Changelog + Firefox 112.0 + Introduced. [Bug 1819797_] + +.. _1819797: https://bugzilla.mozilla.org/show_bug.cgi?id=1819797 + +contextual.services.quicksuggest.impression_nav_superceded +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar records how many times a urlbar engagement occurred when a +navigational suggestion was matched but superseded by a heuristic result. It is +recorded only when the Nimbus variable ``recordNavigationalSuggestionTelemetry`` +is true. (The variable is false by default.) + +Each key is the type of heuristic result that was present at the time of the +engagement. Key names are the same as the heuristic result type names recorded +in Glean telemetry. + +Changelog + Firefox 112.0 + Introduced. [Bug 1819797_] + +.. _1819797: https://bugzilla.mozilla.org/show_bug.cgi?id=1819797 + +contextual.services.quicksuggest.impression_nonsponsored +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar records suggestion impressions. It is incremented each time +the user is shown a non-sponsored suggestion and the following two conditions hold: + +- The user has completed an engagement with the address bar by picking a result + in it or by pressing the Enter key. +- At the time the user completed the engagement, a suggestion was present in the + results. + +Each key is the index at which a suggestion appeared in the results (1-based), +and the corresponding value is the number of impressions at that index. + +Changelog + Firefox 109.0 + Introduced. [Bug 1800993_] + +.. _1800993: https://bugzilla.mozilla.org/show_bug.cgi?id=1800993 + +contextual.services.quicksuggest.impression_nonsponsored_bestmatch +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar records non-sponsored best match impressions. It is +incremented each time the user is shown a non-sponsored best match and the +following two conditions hold: + +- The user has completed an engagement with the address bar by picking a result + in it or by pressing the Enter key. +- At the time the user completed the engagement, a non-sponsored best match was + present in the results. + +Each key is the index at which a suggestion appeared in the results (1-based), +and the corresponding value is the number of impressions at that index. + +Changelog + Firefox 99.0 + Introduced. [Bug 1752953_] + +.. _1752953: https://bugzilla.mozilla.org/show_bug.cgi?id=1752953 + +contextual.services.quicksuggest.impression_sponsored +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar records suggestion impressions. It is incremented each time +the user is shown a sponsored suggestion and the following two conditions hold: + +- The user has completed an engagement with the address bar by picking a result + in it or by pressing the Enter key. +- At the time the user completed the engagement, a suggestion was present in the + results. + +Each key is the index at which a suggestion appeared in the results (1-based), +and the corresponding value is the number of impressions at that index. + +Changelog + Firefox 109.0 + Introduced. [Bug 1800993_] + +.. _1800993: https://bugzilla.mozilla.org/show_bug.cgi?id=1800993 + +contextual.services.quicksuggest.impression_sponsored_bestmatch +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar records sponsored best match impressions. It is incremented +each time the user is shown a sponsored best match and the following two +conditions hold: + +- The user has completed an engagement with the address bar by picking a result + in it or by pressing the Enter key. +- At the time the user completed the engagement, a sponsored best match was + present in the results. + +Each key is the index at which a suggestion appeared in the results (1-based), +and the corresponding value is the number of impressions at that index. + +Changelog + Firefox 99.0 + Introduced. [Bug 1752953_] + +.. _1752953: https://bugzilla.mozilla.org/show_bug.cgi?id=1752953 + +contextual.services.quicksuggest.impression_weather +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This keyed scalar records weather suggestion impressions. It is incremented each +time the user is shown a weather suggestion and the following two conditions +hold: + +- The user has completed an engagement with the address bar by picking a result + in it or by pressing the Enter key. +- At the time the user completed the engagement, a weather suggestion was + present in the results. + +Each key is the index at which a suggestion appeared in the results (1-based), +and the corresponding value is the number of impressions at that index. + +Changelog + Firefox 110.0 + Introduced. [Bug 1804536_] + +.. _1804536: https://bugzilla.mozilla.org/show_bug.cgi?id=1804536 + +Events +------ + +The following Firefox Suggest events are recorded in the +``contextservices.quicksuggest`` category. For general information on event +telemetry in Firefox, see the +:doc:`/toolkit/components/telemetry/collection/events` document. + +contextservices.quicksuggest.data_collect_toggled +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This event is recorded when the +``browser.urlbar.quicksuggest.dataCollection.enabled`` pref is toggled. The pref +can be toggled in the following ways: + +- The user can toggle it in the preferences UI. +- The user can toggle it in about:config. + +The event is also recorded when the user opts in to the online modal dialog, +with one exception: If the user has already enabled data collection using the +preferences UI or about:config, then the pref's user value is already +true. Opting in doesn't change the user value, so no event is recorded. + +The event's objects are the following: + +:enabled: + Recorded when the pref is flipped from false to true. +:disabled: + Recorded when the pref is flipped from true to false. + +Changelog + Firefox 94.0.2 + Introduced. [Bug 1735976_] + +.. _1735976: https://bugzilla.mozilla.org/show_bug.cgi?id=1735976 + +contextservices.quicksuggest.enable_toggled +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This event is recorded when the +``browser.urlbar.suggest.quicksuggest.nonsponsored`` pref is toggled. The pref +can be toggled in the following ways: + +- The user can toggle it in the preferences UI. +- The user can toggle it in about:config. + +The event's objects are the following: + +:enabled: + Recorded when the pref is flipped from false to true. +:disabled: + Recorded when the pref is flipped from true to false. + +Changelog + Firefox 87.0: + Introduced. The event corresponds to the + ``browser.urlbar.suggest.quicksuggest`` pref. [Bug 1693126_] + + Firefox 94.0.2: + ``browser.urlbar.suggest.quicksuggest`` is replaced with + ``browser.urlbar.suggest.quicksuggest.nonsponsored``, and this event now + corresponds to the latter pref. [Bug 1735976_] + + Firefox 96.0: + The event is no longer recorded when the user interacts with the online + modal dialog since the ``browser.urlbar.suggest.quicksuggest.nonsponsored`` + pref is no longer set when the user opts in or out. [Bug 1740965_] + +.. _1693126: https://bugzilla.mozilla.org/show_bug.cgi?id=1693126 +.. _1735976: https://bugzilla.mozilla.org/show_bug.cgi?id=1735976 +.. _1740965: https://bugzilla.mozilla.org/show_bug.cgi?id=1740965 + +contextservices.quicksuggest.engagement +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This event is recorded when an engagement occurs in the address bar while a +Firefox Suggest suggestion is present. In other words, it is recorded in two +cases: + +- The user picks a Firefox Suggest suggestion or a related UI element like its + help button. +- While a Firefox Suggest suggestion is present in the address bar, the user + picks some other row. + +The event's objects are the following possible values: + +:block: + The user dismissed ("blocked") the suggestion. +:click: + The user picked the suggestion. +:help: + The user picked the suggestion's help button. +:impression_only: + The user picked some other row. +:other: + The user engaged with the suggestion in some other way, for example by picking + a command in the result menu. This is a catch-all category and going forward + Glean telemetry should be preferred. + +The event's ``extra`` contains the following properties: + +:match_type: + "best-match" if the suggestion was a best match or "firefox-suggest" if it was + a non-best-match suggestion. +:position: + The index of the suggestion in the list of results (1-based). +:suggestion_type: + The type of suggestion, one of: "sponsored", "nonsponsored", + "dynamic-wikipedia", "navigational" +:source: + The source of suggestion, one of: "remote-settings", "merino" + +Changelog + Firefox 101.0 + Introduced. [Bug 1761059_] + + Firefox 109.0 + ``source`` is added. [Bug 1800993_] + ``dynamic-wikipedia`` is added as a value of ``suggestion_type``. [Bug 1800993_] + + Firefox 112.0 + ``navigational`` is added as a value of ``suggestion_type``. [Bug 1819797_] + + Firefox 114.0 + ``other`` is added as a value of the event object. [Bug 1827943_] + +.. _1761059: https://bugzilla.mozilla.org/show_bug.cgi?id=1761059 +.. _1800993: https://bugzilla.mozilla.org/show_bug.cgi?id=1800993 +.. _1819797: https://bugzilla.mozilla.org/show_bug.cgi?id=1819797 +.. _1827943: https://bugzilla.mozilla.org/show_bug.cgi?id=1827943 + +contextservices.quicksuggest.impression_cap +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This event is recorded when an event related to an impression cap occurs. The +event's objects are the following possible values: + +:hit: + Recorded when an impression cap is hit. +:reset: + Recorded when a cap's counter is reset because its interval period has + elapsed. The implementation may batch multiple consecutive reset events for a + cap in a single telemetry event; see the ``eventCount`` discussion below. + Reset events are reported only when a cap's interval period elapses while + Firefox is running. + +The event's ``extra`` contains the following properties: + +:count: + The number of impressions during the cap's interval period. +:eventCount: + The number of impression cap events reported in the telemetry event. This is + necessary because the implementation may batch multiple consecutive "reset" + events for a cap in a single telemetry event. When that occurs, this value + will be greater than 1, ``startDate`` will be the timestamp at which the + first event's interval period started, ``eventDate`` will be the timestamp at + which the last event's interval period ended, and ``count`` will be the number + of impressions during the first event's interval period. (The implementation + guarantees that reset events are batched only when the number of impressions + for all subsequent interval periods is zero.) For "hit" events, + ``eventCount`` will always be 1. +:eventDate: + The event's timestamp, in number of milliseconds since Unix epoch. For "reset" + events, this is the timestamp at which the cap's interval period ended. If + ``eventCount`` is greater than 1, it's the timestamp at which the last + interval period ended. For "hit" events, this is the timestamp at which the + cap was hit. +:impressionDate: + The timestamp of the most recent impression, in number of milliseconds since + Unix epoch. +:intervalSeconds: + The number of seconds in the cap's interval period. For lifetime caps, this + value will be "Infinity". +:maxCount: + The maximum number of impressions allowed in the cap's interval period. +:startDate: + The timestamp at which the cap's interval period started, in number of + milliseconds since Unix epoch. +:type: + The type of cap, one of: "sponsored", "nonsponsored" + +Changelog + Firefox 101.0 + Introduced. [Bug 1761058_, 1765881_] + +.. _1761058: https://bugzilla.mozilla.org/show_bug.cgi?id=1761058 +.. _1765881: https://bugzilla.mozilla.org/show_bug.cgi?id=1765881 + +contextservices.quicksuggest.opt_in_dialog +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This event is recorded when the user interacts with the online modal dialog. +The event's objects are the following: + +:accept: + The user accepted the dialog and opted in. This object was removed in Firefox + 96.0.2. +:accept_2: + The user accepted the dialog and opted in. +:close_1: + The user clicked close button or something similar link on the introduction + section. The user remains opted out in this case. +:dismiss_1: + The user dismissed the dialog by pressing the Escape key or some unknown way + on the introduction section. The user remains opted out in this case. +:dismiss_2: + The user dismissed the dialog by pressing the Escape key or some unknown way + on main section. The user remains opted out in this case. +:dismissed_escape_key: + The user dismissed the dialog by pressing the Escape key. The user remains + opted out in this case. This object was removed in Firefox 96.0.2. +:dismissed_other: + The dialog was dismissed in some unknown way. One case where this can happen + is when the dialog is replaced with another higher priority dialog like the + one shown when quitting the app. The user remains opted out in this case. + This object was removed in Firefox 96.0.2. +:learn_more: + The user clicked "Learn more". The user remains opted out in this case. This + object was removed in Firefox 96.0.2. +:learn_more_1: + The user clicked "Learn more" on the introduction section. The user remains + opted out in this case. +:learn_more_2: + The user clicked "Learn more" on the main section. The user remains opted out + in this case. +:not_now: + The dialog was dismissed in some way without opting in. This object was + removed in Firefox 94.0. +:not_now_2: + The user clicked "Not now" link on main section. The user remains opted out in + this case. +:not_now_link: + The user clicked "Not now". The user remains opted out in this case. This + object was removed in Firefox 96.0.2. +:reject_2: + The user rejected the dialog and opted out. +:settings: + The user clicked the "Customize" button. The user remains opted out in this + case. This object was removed in Firefox 96.0.2. + +Changelog + Firefox 92.0.1 + Introduced. Objects are: ``accept``, ``settings``, ``learn_more``, and + ``not_now``. ``not_now`` is recorded when the dialog is dismissed in any + manner not covered by the other objects. [Bug 1723860_] + + Firefox 94.0 + Objects changed to: ``accept``, ``dismissed_escape_key``, + ``dismissed_other``, ``learn_more``, ``not_now_link``, and ``settings``. + [Bug 1733687_] + + Firefox 96.0.2 + Objects changed to: ``accept_2``, ``reject_2``, ``learn_more_2``, + ``close_1``, ``not_now_2``, ``dismiss_1`` and ``dismiss_2``. + [Bug 1745026_] + + Firefox 100.0 + Objects changed to: ``accept_2``, ``reject_2``, ``learn_more_1``, + ``learn_more_2``, ``close_1``, ``not_now_2``, ``dismiss_1`` and + ``dismiss_2``. + [Bug 1761171_] + +.. _1723860: https://bugzilla.mozilla.org/show_bug.cgi?id=1723860 +.. _1733687: https://bugzilla.mozilla.org/show_bug.cgi?id=1733687 +.. _1745026: https://bugzilla.mozilla.org/show_bug.cgi?id=1745026 +.. _1761171: https://bugzilla.mozilla.org/show_bug.cgi?id=1761171 + +contextservices.quicksuggest.sponsored_toggled +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This event is recorded when the +``browser.urlbar.suggest.quicksuggest.sponsored`` pref is toggled. The pref can +be toggled in the following ways: + +- The user can toggle it in the preferences UI. +- The user can toggle it in about:config. + +The event's objects are the following: + +:enabled: + Recorded when the pref is flipped from false to true. +:disabled: + Recorded when the pref is flipped from true to false. + +Changelog + Firefox 92.0.1 + Introduced. [Bug 1728430_] + + Firefox 96.0: + The event is no longer recorded when the user interacts with the online + modal dialog since the ``browser.urlbar.suggest.quicksuggest.sponsored`` + pref is no longer set when the user opts in or out. [Bug 1740965_] + +.. _1728430: https://bugzilla.mozilla.org/show_bug.cgi?id=1728430 +.. _1740965: https://bugzilla.mozilla.org/show_bug.cgi?id=1740965 + +Environment +----------- + +The following preferences are recorded in telemetry environment data. For +general information on telemetry environment data in Firefox, see the +:doc:`/toolkit/components/telemetry/data/environment` document. + +browser.urlbar.quicksuggest.onboardingDialogChoice +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This pref records the user's choice in the online modal dialog. If the dialog +was shown multiple times, it records the user's most recent choice. It is a +string-valued pref with the following possible values: + +:<empty string>: + The user has not made a choice (e.g., because the dialog hasn't been shown). +:accept: + The user accepted the dialog and opted in. This object was removed in Firefox + 96.0.2. +:accept_2: + The user accepted the dialog and opted in. +:close_1: + The user clicked close button or something similar link on the introduction + section. The user remains opted out in this case. +:dismiss_1: + The user dismissed the dialog by pressing the Escape key or some unknown way + on the introduction section. The user remains opted out in this case. +:dismiss_2: + The user dismissed the dialog by pressing the Escape key or some unknown way + on main section. The user remains opted out in this case. +:dismissed_escape_key: + The user dismissed the dialog by pressing the Escape key. The user remains + opted out in this case. This object was removed in Firefox 96.0.2. +:dismissed_other: + The dialog was dismissed in some unknown way. One case where this can happen + is when the dialog is replaced with another higher priority dialog like the + one shown when quitting the app. The user remains opted out in this case. This + object was removed in Firefox 96.0.2. +:learn_more: + The user clicked "Learn more". The user remains opted out in this case. This + object was removed in Firefox 96.0.2. +:learn_more_1: + The user clicked "Learn more" on the introduction section. The user remains + opted out in this case. +:learn_more_2: + The user clicked "Learn more" on the main section. The user remains opted out + in this case. +:not_now_2: + The user clicked "Not now" link on main section. The user remains opted out in + this case. +:not_now_link: + The user clicked "Not now". The user remains opted out in this case. This + object was removed in Firefox 96.0.2. +:reject_2: + The user rejected the dialog and opted out. +:settings: + The user clicked the "Customize" button. The user remains opted out in this + case. This object was removed in Firefox 96.0.2. + +Changelog + Firefox 94.0 + Introduced. [Bug 1734447_] + + Firefox 96.0.2 + Added ``accept_2``, ``reject_2``, ``learn_more_2``, ``close_1``, + ``not_now_2``, ``dismiss_1``, ``dismiss_2`` and removed ``accept``, + ``dismissed_escape_key``, ``dismissed_other``, ``learn_more``, + ``not_now_link``, ``settings``. [Bug 1745026_] + + Firefox 100.0 + Added ``learn_more_1``. [Bug 1761171_] + +.. _1734447: https://bugzilla.mozilla.org/show_bug.cgi?id=1734447 +.. _1745026: https://bugzilla.mozilla.org/show_bug.cgi?id=1745026 +.. _1761171: https://bugzilla.mozilla.org/show_bug.cgi?id=1761171 + +browser.urlbar.quicksuggest.dataCollection.enabled +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This boolean pref records whether the user has opted in to data collection for +Firefox Suggest. It is false by default. It is set to true when the user opts in +to the online modal dialog. The user can also toggle it in the preferences UI +and about:config. + +Changelog + Firefox 94.0.2 + Introduced. [Bug 1735976_] + +.. _1735976: https://bugzilla.mozilla.org/show_bug.cgi?id=1735976 + +browser.urlbar.suggest.quicksuggest +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This pref no longer exists and is not recorded. It was replaced with +``browser.urlbar.suggest.quicksuggest.nonsponsored`` in Firefox 94.0.2. Prior to +94.0.2, this boolean pref recorded whether suggestions in general were enabled. + +Changelog + Firefox 92.0.1 + Introduced. [Bug 1730721_] + + Firefox 94.0.2 + Replaced with ``browser.urlbar.suggest.quicksuggest.nonsponsored``. [Bug + 1735976_] + +.. _1730721: https://bugzilla.mozilla.org/show_bug.cgi?id=1730721 +.. _1735976: https://bugzilla.mozilla.org/show_bug.cgi?id=1735976 + +browser.urlbar.suggest.quicksuggest.nonsponsored +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This boolean pref records whether non-sponsored suggestions are enabled. In both +the offline and online scenarios it is true by default. The user can also toggle +it in the preferences UI and about:config. + +Changelog + Firefox 94.0.2 + Introduced. It replaces ``browser.urlbar.suggest.quicksuggest``. [Bug + 1735976_] + + Firefox 96.0: + The pref is now true by default in the online scenario. Previously it was + false by default in online. For users who were enrolled in the online + scenario in older versions and who did not opt in or otherwise enable + non-sponsored suggestions, the pref will remain false when upgrading. For + all other users, it will default to true when/if they are enrolled in + online. [Bug 1740965_] + +.. _1735976: https://bugzilla.mozilla.org/show_bug.cgi?id=1735976 +.. _1740965: https://bugzilla.mozilla.org/show_bug.cgi?id=1740965 + +browser.urlbar.suggest.quicksuggest.sponsored +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This boolean pref records whether sponsored suggestions are enabled. In both the +offline and online scenarios it is true by default. The user can also toggle it +in the preferences UI and about:config. + +Changelog + Firefox 92.0.1 + Introduced. [Bug 1730721_] + + Firefox 96.0: + The pref is now true by default in the online scenario. Previously it was + false by default in online. For users who were enrolled in the online + scenario in older versions and who did not opt in or otherwise enable + sponsored suggestions, the pref will remain false when upgrading. For all + other users, it will default to true when/if they are enrolled in + online. [Bug 1740965_] + +.. _1730721: https://bugzilla.mozilla.org/show_bug.cgi?id=1730721 +.. _1740965: https://bugzilla.mozilla.org/show_bug.cgi?id=1740965 + +Contextual Services Pings +------------------------- + +The following custom telemetry pings are recorded for Firefox Suggest +suggestions. For general information on custom telemetry pings in Firefox, see +the `Custom Ping`_ document. + +.. _Custom Ping: https://docs.telemetry.mozilla.org/cookbooks/new_ping.html#sending-a-custom-ping + +Block +~~~~~ + +A block ping is recorded when the user dismisses ("blocks") a suggestion. Its +payload includes the following: + +:advertiser: + The name of the suggestion's advertiser. +:block_id: + A unique identifier for the suggestion (a.k.a. a keywords block). +:context_id: + A UUID representing this user. Note that it's not client_id, nor can it be + used to link to a client_id. +:iab_category: + The suggestion's category, either "22 - Shopping" or "5 - Education". +:improve_suggest_experience_checked: + A boolean indicating whether the user has opted in to improving the Firefox + Suggest experience. There are two ways for the user to opt in, either in an + opt-in modal experiment or by toggling a switch in Firefox's settings. +:match_type: + "best-match" if the suggestion was a best match or "firefox-suggest" if it was + a non-best-match suggestion. +:position: + The index of the suggestion in the list of results (1-based). +:request_id: + A request identifier for each API request to Merino. This is only included for + suggestions provided by Merino. +:source: + The source of the suggestion, either "remote-settings" or "merino". + +Changelog + Firefox 101.0 + Introduced. [Bug 1764669_] + + Firefox 103.0 + ``scenario`` is removed from the payload and + ``improve_suggest_experience_checked`` is added. [Bug 1776797_] + + Firefox 109.0 + ``source`` is added. [Bug 1800993_] + +.. _1764669: https://bugzilla.mozilla.org/show_bug.cgi?id=1764669 +.. _1776797: https://bugzilla.mozilla.org/show_bug.cgi?id=1776797 +.. _1800993: https://bugzilla.mozilla.org/show_bug.cgi?id=1800993 + +Click +~~~~~ + +A click ping is recorded when the user picks a suggestion. Its payload includes +the following: + +:advertiser: + The name of the suggestion's advertiser. +:block_id: + A unique identifier for the suggestion (a.k.a. a keywords block). +:context_id: + A UUID representing this user. Note that it's not client_id, nor can it be + used to link to a client_id. +:improve_suggest_experience_checked: + A boolean indicating whether the user has opted in to improving the Firefox + Suggest experience. There are two ways for the user to opt in, either in an + opt-in modal experiment or by toggling a switch in Firefox's settings. +:match_type: + "best-match" if the suggestion was a best match or "firefox-suggest" if it was + a non-best-match suggestion. +:position: + The index of the suggestion in the list of results (1-based). +:reporting_url: + The reporting URL of the suggestion, normally pointing to the ad partner's + reporting endpoint. +:request_id: + A request identifier for each API request to Merino. This is only included for + suggestions provided by Merino. +:source: + The source of the suggestion, either "remote-settings" or "merino". + +Changelog + Firefox 87.0 + Introduced. The payload is: ``advertiser``, ``block_id``, ``position``, and + ``reporting_url``. [Bug 1689365_] + + Firefox 92.0.1 + ``scenario`` is added to the payload. [Bug 1729576_] + + Firefox 94.0.2 + ``request_id`` is added to the payload. [Bug 1736117_] + + Firefox 99.0 + ``match_type`` is added to the payload. [Bug 1754622_] + + Firefox 103.0 + ``scenario`` is removed from the payload and + ``improve_suggest_experience_checked`` is added. [Bug 1776797_] + + Firefox 109.0 + ``source`` is added. [Bug 1800993_] + +.. _1689365: https://bugzilla.mozilla.org/show_bug.cgi?id=1689365 +.. _1729576: https://bugzilla.mozilla.org/show_bug.cgi?id=1729576 +.. _1736117: https://bugzilla.mozilla.org/show_bug.cgi?id=1736117 +.. _1754622: https://bugzilla.mozilla.org/show_bug.cgi?id=1754622 +.. _1776797: https://bugzilla.mozilla.org/show_bug.cgi?id=1776797 +.. _1800993: https://bugzilla.mozilla.org/show_bug.cgi?id=1800993 + +Impression +~~~~~~~~~~ + +An impression ping is recorded when the user is shown a suggestion and the +following two conditions hold: + +- The user has completed an engagement with the address bar by picking a result + in it or by pressing the Enter key. +- At the time the user completed the engagement, a suggestion was present in the + results. + +It is also recorded when the user dismisses ("blocks") a suggestion. + +The impression ping payload contains the following: + +:advertiser: + The name of the suggestion's advertiser. +:block_id: + A unique identifier for the suggestion (a.k.a. a keywords block). +:context_id: + A UUID representing this user. Note that it's not client_id, nor can it be + used to link to a client_id. +:improve_suggest_experience_checked: + A boolean indicating whether the user has opted in to improving the Firefox + Suggest experience. There are two ways for the user to opt in, either in an + opt-in modal experiment or by toggling a switch in Firefox's settings. +:is_clicked: + Whether or not the user also clicked the suggestion. When true, we will also + send a separate click ping. When the impression ping is recorded because the + user dismissed ("blocked") the suggestion, this will be false. +:match_type: + "best-match" if the suggestion was a best match or "firefox-suggest" if it was + a non-best-match suggestion. +:position: + The index of the suggestion in the list of results (1-based). +:reporting_url: + The reporting URL of the suggestion, normally pointing to the ad partner's + reporting endpoint. +:request_id: + A request identifier for each API request to Merino. This is only included for + suggestions provided by Merino. +:source: + The source of the suggestion, either "remote-settings" or "merino". + +Changelog + Firefox 87.0 + Introduced. The payload is: ``advertiser``, ``block_id``, ``is_clicked``, + ``matched_keywords``, ``position``, ``reporting_url``, and + ``search_query``. ``matched_keywords`` and ``search_query`` are always + included in the payload and are always identical: They both record the exact + search query as typed by the user. [Bug 1689365_] + + Firefox 91.0.1 (Release and ESR) + ``matched_keywords`` and ``search_query`` are always recorded as empty + strings. [Bug 1725492_] + + Firefox 92.0.1 + - When the user's scenaro is "online", ``matched_keywords`` records the full + keyword of the matching suggestion and ``search_query`` records the exact + search query as typed by the user; otherwise both are recorded as empty + strings. [Bug 1728188_, 1729576_] + - ``scenario`` is added to the payload. [Bug 1729576_] + + Firefox 94.0.2 + - When the user has opted in to data collection and the matching suggestion + is provided by remote settings, ``matched_keywords`` records the full + keyword of the suggestion and ``search_query`` records the exact search + query as typed by the user; otherwise both are excluded from the ping. + [Bug 1736117_, 1735976_] + - ``request_id`` is added to the payload. [Bug 1736117_] + + Firefox 97.0 + - Stop sending ``search_query`` and ``matched_keywords`` in the custom + impression ping for Firefox Suggest. [Bug 1748348_] + + Firefox 99.0 + ``match_type`` is added to the payload. [Bug 1754622_] + + Firefox 101.0 + The impression ping is now also recorded when the user dismisses ("blocks") + a suggestion. [Bug 1761059_] + + Firefox 103.0 + ``scenario`` is removed from the payload and + ``improve_suggest_experience_checked`` is added. [Bug 1776797_] + + Firefox 109.0 + ``source`` is added. [Bug 1800993_] + +.. _1689365: https://bugzilla.mozilla.org/show_bug.cgi?id=1689365 +.. _1725492: https://bugzilla.mozilla.org/show_bug.cgi?id=1725492 +.. _1728188: https://bugzilla.mozilla.org/show_bug.cgi?id=1728188 +.. _1729576: https://bugzilla.mozilla.org/show_bug.cgi?id=1729576 +.. _1736117: https://bugzilla.mozilla.org/show_bug.cgi?id=1736117 +.. _1735976: https://bugzilla.mozilla.org/show_bug.cgi?id=1735976 +.. _1748348: https://bugzilla.mozilla.org/show_bug.cgi?id=1748348 +.. _1754622: https://bugzilla.mozilla.org/show_bug.cgi?id=1754622 +.. _1761059: https://bugzilla.mozilla.org/show_bug.cgi?id=1761059 +.. _1776797: https://bugzilla.mozilla.org/show_bug.cgi?id=1776797 +.. _1800993: https://bugzilla.mozilla.org/show_bug.cgi?id=1800993 + +Nimbus Exposure Event +--------------------- + +A `Nimbus exposure event`_ is recorded once per app session when the user first +encounters the UI of an experiment in which they're enrolled. The timing of the +event depends on the experiment and branch. + +There are two Nimbus variables that determine the timing of the event: +``experimentType`` and the deprecated ``isBestMatchExperiment``. To determine +when the exposure event is recorded for a specific experiment and branch, +examine the experiment's recipe and look for one of these variables. + +Listed below are the supported values of ``experimentType`` and +``isBestMatchExperiment`` along with details on when their corresponding +exposure events are recorded. + +:experimentType = "best-match": + If the user is in a treatment branch and they did not disable best match, the + event is recorded the first time they trigger a best match; if the user is in + a treatment branch and they did disable best match, the event is not recorded + at all. If the user is in the control branch, the event is recorded the first + time they would have triggered a best match. (Users in the control branch + cannot "disable" best match since the feature is totally hidden from them.) +:experimentType = "modal": + If the user is in a treatment branch, the event is recorded when they are + shown an opt-in modal. If the user is in the control branch, the event is + recorded every time they would have been shown a modal, which is on every + startup where another non-Suggest modal does not appear. +:isBestMatchExperiment = true: + This is a deprecated version of ``experimentType == "best-match"``. +:All other experiments: + For all other experiments not listed above, the event is recorded the first + time the user triggers a Firefox Suggest suggestion. + +Changelog + Firefox 92.0 + Introduced. The event is always recorded the first time the user triggers + a Firefox Suggest suggestion regardless of the experiment they are enrolled + in. [Bug 1724076_, 1727392_] + + Firefox 99.0 + The ``isBestMatchExperiment = true`` case is added. [Bug 1752953_] + + Firefox 100.0 + The ``experimentType = "modal"`` case is added. + ``isBestMatchExperiment = true`` is deprecated in favor of + ``experimentType = "best-match"``. [Bug 1760596_] + +.. _Nimbus exposure event: https://experimenter.info/jetstream/jetstream/#enrollment-vs-exposure + +.. _1724076: https://bugzilla.mozilla.org/show_bug.cgi?id=1724076 +.. _1727392: https://bugzilla.mozilla.org/show_bug.cgi?id=1727392 +.. _1752953: https://bugzilla.mozilla.org/show_bug.cgi?id=1752953 +.. _1760596: https://bugzilla.mozilla.org/show_bug.cgi?id=1760596 + +Merino Search Queries +--------------------- + +Merino is a Mozilla service that provides Firefox Suggest suggestions. Along +with remote settings on the client, it is one of two possible sources for +Firefox Suggest. When Merino integration is enabled on the client and the user +has opted in to Firefox Suggest data collection, Firefox sends everything the +user types in the address bar to the Merino server. In response, Merino finds +relevant search results from its search providers and sends them to Firefox, +where they are shown to the user in the address bar. + +The user opts in to Firefox Suggest data collection when they either opt in to +the online modal dialog or they enable Firefox Suggest data collection in the +preferences UI. + +Merino queries are not telemetry per se but we include them in this document +since they necessarily involve data collection. + +Merino API +~~~~~~~~~~ + +Data that Firefox sends to the Merino server is summarized below. When Merino +integration is enabled on the client and the user has opted in to Firefox +Suggest data collection, this data is sent with every user keystroke in the +address bar. + +For details on the Merino API, see the `Merino documentation`_. + +.. _Merino documentation: https://mozilla-services.github.io/merino/api.html#suggest + +Search Query + The user's search query typed in the address bar. + + API parameter name: ``q`` + +Session ID + A UUID that identifies the user's current search session in the address bar. + This ID is unique per search session. A search session ends when the focus + leaves the address bar or a timeout of 5 minutes elapses, whichever comes + first. + + API parameter name: ``sid`` + +Sequence Number + A zero-based integer that is incremented after a response is received from + Merino. It is reset at the end of each search session along with the session + ID. + + API parameter name: ``seq`` + +Client Variants + Optional. A list of experiments or rollouts that are affecting the Firefox + Suggest user experience. If Merino recognizes any of them, it will modify its + behavior accordingly. + + API parameter name: ``client_variants`` + +Providers + Optional. A list of providers to use for this request. If specified, only + suggestions from the listed providers will be returned. Otherwise Merino will + use a default set of providers. + + API parameter name: ``providers`` diff --git a/browser/components/urlbar/docs/index.rst b/browser/components/urlbar/docs/index.rst new file mode 100644 index 0000000000..8abc9c974d --- /dev/null +++ b/browser/components/urlbar/docs/index.rst @@ -0,0 +1,56 @@ +Address Bar +=========== + +This document describes the implementation of Firefox's address bar, also known +as the quantumbar or urlbar. The address bar was also called the awesomebar +until Firefox 68, when it was substantially rewritten. + +The address bar is a specialized search access point that aggregates data from +several different sources, including: + + * Places (Firefox's history and bookmarks system) + * Search engines (including search suggestions) + * WebExtensions + * Open tabs + +Where to Start +-------------- + +If you want a high level, nontechnical summary of how the address bar works, +read :doc:`nontechnical-overview`. + +If you are interested in the technical details, you might want to skip ahead to +:doc:`overview`. + +Codebase +-------- + +The address bar code lives in `browser/components/urlbar <https://searchfox.org/mozilla-central/source/browser/components/urlbar/>`_. + +Table of Contents +----------------- + +.. toctree:: + + nontechnical-overview + overview + lifetime + utilities + telemetry + firefox-suggest-telemetry + debugging + ranking + experiments + dynamic-result-types + preferences + testing + contact + +API Reference +------------- + +.. toctree:: + + UrlbarController + UrlbarInput + UrlbarView diff --git a/browser/components/urlbar/docs/lifetime.rst b/browser/components/urlbar/docs/lifetime.rst new file mode 100644 index 0000000000..f12aba6e60 --- /dev/null +++ b/browser/components/urlbar/docs/lifetime.rst @@ -0,0 +1,109 @@ +Search Lifecycle +================ + +When a character is typed into the address bar, or the address bar is focused, +we initiate a search. What follows is a simplified version of the +lifetime of a search, describing the pipeline that returns results for a typed +string. Some parts of the query lifetime are intentionally omitted from this +document for clarity. + +The search described in this document is internal to the address bar. It is not +the search sent to the default search engine when you press Enter. Parts of this +process often occur multiple times per keystroke, as described below. + +It is recommended that you first read the :doc:`nontechnical-overview` to become +familiar with the terminology in this document. This document is current as +of April 2023. + +#. + The user types a query (e.g. "coffee near me") into the *UrlbarInput* + `<input> DOM element <https://searchfox.org/mozilla-central/rev/1f4f99a8f331cce8467a50742178b6d46914ab89/browser/base/content/navigator-toolbox.inc.xhtml#330-336>`_. + That DOM element `tells <https://searchfox.org/mozilla-central/rev/1f4f99a8f331cce8467a50742178b6d46914ab89/browser/components/urlbar/UrlbarInput.sys.mjs#3312>`_ + *UrlbarInput* that text is being input. + +#. + *UrlbarInput* `starts a search <https://searchfox.org/mozilla-central/rev/1f4f99a8f331cce8467a50742178b6d46914ab89/browser/components/urlbar/UrlbarInput.sys.mjs#3395>`_. + It `creates <https://searchfox.org/mozilla-central/rev/1f4f99a8f331cce8467a50742178b6d46914ab89/browser/components/urlbar/UrlbarInput.sys.mjs#1549>`_ + a `UrlbarQueryContext <https://firefox-source-docs.mozilla.org/browser/urlbar/overview.html#the-urlbarquerycontext>`_ + and `passes it to UrlbarController <https://searchfox.org/mozilla-central/rev/1f4f99a8f331cce8467a50742178b6d46914ab89/browser/components/urlbar/UrlbarInput.sys.mjs#1548>`_. + The query context is an object that will exist for the lifetime of the query + and it's how we keep track of what results to show. It contains information + like what kind of results are allowed, the search string ("coffee near me", + in this case), and other information about the state of the Urlbar. A new + *UrlbarQueryContext* is created every time the text in the input changes. + +#. + *UrlbarController* `tells UrlbarProvidersManager <https://searchfox.org/mozilla-central/rev/0ffaecaa075887ab07bf4c607c61ea2faa81b172/browser/components/urlbar/UrlbarController.sys.mjs#140>`_ + that the providers should fetch results. + +#. + *UrlbarProvidersManager* tells `each <https://searchfox.org/mozilla-central/rev/0ffaecaa075887ab07bf4c607c61ea2faa81b172/browser/components/urlbar/UrlbarProvidersManager.sys.mjs#408>`_ + provider to decide if it wants to provide results for this query by calling + their `isActive <https://searchfox.org/mozilla-central/rev/0ffaecaa075887ab07bf4c607c61ea2faa81b172/browser/components/urlbar/UrlbarProvidersManager.sys.mjs#422>`_ + methods. The provider can decide whether or not it will be active for this + query. Some providers are rarely active: for example, + *UrlbarProviderTopSites* `isn't active if the user has typed a search string <https://searchfox.org/mozilla-central/rev/0ffaecaa075887ab07bf4c607c61ea2faa81b172/browser/components/urlbar/UrlbarProviderTopSites.sys.mjs#97>`_. + +#. + *UrlbarProvidersManager* then tells the *active* providers to fetch results by + `calling their startQuery method <https://searchfox.org/mozilla-central/rev/0ffaecaa075887ab07bf4c607c61ea2faa81b172/browser/components/urlbar/UrlbarProvidersManager.sys.mjs#462>`_. + +#. + The providers fetch results for the query asynchronously. Each provider + fetches results in a different way. As one example, if the default search + engine is Google, *UrlbarProviderSearchSuggestions* would send the string + "coffee near me" to Google. Google would return a list of suggestions and + *UrlbarProviderSearchSuggestions* would create a *UrlbarResult* for each one. + +#. + The providers send their results back to *UrlbarProvidersManager*. They do + this one result at a time by `calling the addCallback callback <https://searchfox.org/mozilla-central/rev/0ffaecaa075887ab07bf4c607c61ea2faa81b172/browser/components/urlbar/UrlbarProviderSearchSuggestions.sys.mjs#292>`_ + passed into startQuery. *UrlbarProvidersManager* takes all the results from all the + providers and `puts them into the list of unsorted results <https://searchfox.org/mozilla-central/rev/0ffaecaa075887ab07bf4c607c61ea2faa81b172/browser/components/urlbar/UrlbarProvidersManager.sys.mjs#607>`_. + + Due to the asynchronous and parallel nature of providers, this and the + following steps may occur multiple times per search. Some providers may take + longer than others to return their results. We don't want to wait for slow + providers before showing results. To handle slow providers, + *UrlbarProvidersManager* gathers results from providers in "chunks". A timer + fires on an internal. Every time the timer fires, we take whatever results we + have from the active providers (the "chunk" of results) and perform the + following steps. + +#. + *UrlbarProvidersManager* `asks <https://searchfox.org/mozilla-central/rev/0ffaecaa075887ab07bf4c607c61ea2faa81b172/browser/components/urlbar/UrlbarProvidersManager.sys.mjs#648>`_ + *UrlbarMuxer* to sort the unsorted results. + +#. + *UrlbarMuxer* chooses the results that will be shown to the user. It groups + and sorts the results to determine the order in which the results will be + shown. This process usually involves discarding irrelevant and duplicate + results. We also cap results at a limit, defined in the + ``browser.urlbar.maxRichResults`` preference. + +#. + Once the results are sorted, *UrlbarProvidersManager* + `tells UrlbarController <https://searchfox.org/mozilla-central/rev/0ffaecaa075887ab07bf4c607c61ea2faa81b172/browser/components/urlbar/UrlbarProvidersManager.sys.mjs#675>`_ + that results are ready to be shown. + +#. + *UrlbarController* `sends out a notification <https://searchfox.org/mozilla-central/rev/0ffaecaa075887ab07bf4c607c61ea2faa81b172/browser/components/urlbar/UrlbarController.sys.mjs#213>`_ + that results are ready to be shown. *UrlbarView* was `listening <https://searchfox.org/mozilla-central/rev/0ffaecaa075887ab07bf4c607c61ea2faa81b172/browser/components/urlbar/UrlbarView.sys.mjs#662>`_ + for that notification. Once the view gets the notification, it `calls #updateResults <https://searchfox.org/mozilla-central/rev/0ffaecaa075887ab07bf4c607c61ea2faa81b172/browser/components/urlbar/UrlbarView.sys.mjs#670>`_ + to create `DOM nodes <https://searchfox.org/mozilla-central/rev/0ffaecaa075887ab07bf4c607c61ea2faa81b172/browser/components/urlbar/UrlbarView.sys.mjs#1185>`_ + for each *UrlbarResult* and `inserts them <https://searchfox.org/mozilla-central/rev/0ffaecaa075887ab07bf4c607c61ea2faa81b172/browser/components/urlbar/UrlbarView.sys.mjs#1156>`_ + into the view's DOM element. + + As described above, we may reach this step multiple times per search. That + means we may be updating the view multiple times per keystroke. A view that + visibly changes many times after a single keystroke is perceived as + "flickering" by the user. As a result, we try to limit the number of times + the view needs to update. + + + .. figure:: assets/lifetime/lifetime.png + :alt: A chart with boxes representing the various components of the + address bar. An arrow moves between the boxes to illustrate a query + moving through the components. + :scale: 80% + :align: center diff --git a/browser/components/urlbar/docs/nontechnical-overview.rst b/browser/components/urlbar/docs/nontechnical-overview.rst new file mode 100644 index 0000000000..e3fb6d7600 --- /dev/null +++ b/browser/components/urlbar/docs/nontechnical-overview.rst @@ -0,0 +1,628 @@ +Nontechnical Overview +===================== + +This document provides a high level, nontechnical overview of Firefox's address +bar, with a focus on the different types of results it shows and how it chooses +them. + +.. contents:: + :depth: 2 + + +Terminology +----------- + +This document uses a small number of terms of art that would be helpful to +understand up front. + +Input + The text box component of the address bar. In contrast, we use "address bar" + to refer to the whole system comprising the input, the view, and the logic + that determines the results that are shown in the view based on the text in + the input. + +Result + An individual item that is shown in the view. There are many different types + of results, including bookmarks, history, open tabs, and search suggestions. + +View + The panel that opens below the input when the input is focused. It contains + the results. + +Maximum Result Count +-------------------- + +The view shows a maximum of 10 results by default. This number is controlled by +a hidden preference, ``browser.urlbar.maxRichResults``. + +Search Strings +-------------- + +If the user has not modified the text in the input or the text in the input is +empty, we say that the user's **search string** is empty, or in other words, +there is no search string. In contrast, when the user has modified the text in +the input and the text is non-empty, then the search string is that non-empty +text. + +.. figure:: assets/nontechnical-overview/empty-url.png + :alt: Image of the address bar input showing a URL + :scale: 50% + :align: center + + Empty search string: The input text has not been modified + +.. figure:: assets/nontechnical-overview/empty-placeholder.png + :alt: Image of the address bar input showing its placeholder text + :scale: 50% + :align: center + + Empty search string: The input text is empty (and the input is showing its + placeholder text) + +.. figure:: assets/nontechnical-overview/non-empty.png + :alt: Image of the address bar input showing "porcupines" text + :scale: 50% + :align: center + + Non-empty search string: The input text has been modified and is non-empty + +The distinction between empty and non-empty search strings is helpful to +understand for the following sections. + +Top Sites +--------- + +When the search string is empty and the user focuses the input, the view opens +and shows the user's top sites. They are the same top sites that appear on the +new-tab page except their number is capped to the maximum number of address bar +results (10). If the user has fewer top sites than the maximum number of results +(as is the case in a new profile), then only that number of results is shown. + +.. figure:: assets/nontechnical-overview/top-sites.png + :alt: Image of the address bar view showing top sites + :scale: 50% + :align: center + + Top sites on a new en-US profile + +This behavior can be turned off by going to about:preferences#privacy and +unchecking “Shortcuts” in the “Address Bar” section. In that case, the view +closes when the search string is empty. + +Searches +-------- + +When the search string is non-empty, the address bar performs a search and +displays the matching results in the view. Multiple separate searches of +different sources are actually performed, and the results from each source are +combined, sorted, and capped to the maximum result count to display the final +list of results. In address bar terminology, each source is called a +**provider**. + +Each provider produces one or more types of results based on the search +string. The most common result types include the following (not exhaustive): + +.. figure:: assets/nontechnical-overview/search-suggestion.png + :alt: Image of a search suggestion result with text "porcupine meatballs" + :scale: 50% + :align: center + + Search suggestions from the user's default engine (magnifying glass icon) + +.. figure:: assets/nontechnical-overview/form-history.png + :alt: Image of a previous search result with text "porcupines" + :scale: 50% + :align: center + + Previous searches the user has performed from the address bar and search bar + (clock icon) + +.. figure:: assets/nontechnical-overview/bookmark.png + :alt: Image of a bookmark result with text "Porcupine - Wikipedia" + :scale: 50% + :align: center + + Bookmarks + +.. figure:: assets/nontechnical-overview/history.png + :alt: Image of a history result with text "Porcupines | National Geographic" + :scale: 50% + :align: center + + History + +.. figure:: assets/nontechnical-overview/open-tab.png + :alt: Image of an open tab result with text "Porcupines | San Diego Zoo + Animals & Plants" + :scale: 50% + :align: center + + Open tabs (switch to tab) + +.. figure:: assets/nontechnical-overview/remote-tab.png + :alt: Image of a remote tab result with text "Porcupine | rodent | + Britannica" + :scale: 50% + :align: center + + Remote tabs (via Sync) + +How the address bar combines and sorts results from different providers is +discussed below in `Result Composition`_. + +The Heuristic Result +-------------------- + +The first result in the view is special and is called the **heuristic** +result. As the user types each character in their search string, the heuristic +result is updated and automatically selected, and its purpose is to show the +user what will happen when they press the enter key without first selecting a +(non-heuristic) result. The heuristic result is so called because it shows +Firefox's best guess for what the user is trying to do based on their search +string. + +The heuristic result is determined by running through a number of different +heuristics and picking the one that first matches the search string. The most +important heuristics in the order that Firefox runs through them are: + +*Is the search string...* + +1. An omnibox extension keyword? Extensions using the omnibox API can register + keywords by which they become activated. +2. A bookmark keyword? The user can associate a keyword with each bookmark. + Typing a bookmark keyword plus an optional search string and pressing enter + will visit the bookmark. + + .. figure:: assets/nontechnical-overview/bookmark-keyword.png + :alt: Image of the address bar input with text "bug 1677126" and a + bookmark keyword heuristic result + :scale: 50% + :align: center + + Typing "bug" triggers a Bugzilla bookmark with the keyword "bug" + +3. A domain name or URL that should be autofilled? **Autofill** is the name of + the feature where the input completes the domain names and URLs of bookmarks + and frequently visited sites as the user is typing them. (Firefox autofills + “to the next slash”, meaning it first autofills domain names and then partial + paths.) + + .. figure:: assets/nontechnical-overview/autofill.png + :alt: Image of the address bar input with text "mozilla.org/" with + "illa.org/" selected and an autofill heuristic result + :scale: 50% + :align: center + + After typing "moz", the rest of mozilla.org is automatically completed + +4. A valid URL? If so, visit the URL. (This includes fixing common typos like + “mozilla..org” and “mozilla.ogr”. Valid URLs are based on the `Public Suffix + List`_. The user can also specify an allow-list using hidden preferences to + support domains like localhost.) + + .. figure:: assets/nontechnical-overview/visit.png + :alt: Image of the address bar input with text "porcupine-fancy.org" and a + visit heuristic result + :scale: 50% + :align: center + + Typing a URL that isn't bookmarked or in history + + .. _Public Suffix List: https://publicsuffix.org/ + +5. Ultimately fall back to performing a search using the default engine. (The + user can opt out of this fallback by setting the hidden preference + ``keyword.enabled`` to false. In that case, Firefox stops at the previous + step and attempts to visit the user's search string as if it were a URL.) + + .. figure:: assets/nontechnical-overview/search-heuristic.png + :alt: Image of the address bar input with text "porcupines" and a search + heuristic result + :scale: 50% + :align: center + + Typing a string that will perform a search using the default engine + +Result Composition +------------------ + +For a given search string, the address bar performs multiple separate searches +of different providers and then combines their results to display the final +list. The way in which results are combined and sorted is called **result +composition**. Result composition is based on the concept of result groups, one +group after another, with different types of results in each group. + +The default result composition is described next, starting with the first +result. + +1. Heuristic Result +~~~~~~~~~~~~~~~~~~~ + +The first result is always the heuristic result. + +2. Extension Omnibox Results +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The next group of results is those provided by extensions using the omnibox +API. Most users never encounter these results because they are provided only by +extensions that use this feature, and even then the user must type certain +extension-defined keywords to trigger them. There are at most 6 results in this +group. + +3. Search Suggestions +~~~~~~~~~~~~~~~~~~~~~ + +The next group is search suggestions. Typically this group contains 6 results, +but the exact number depends on certain factors described later in `Result +Composition Nuances`_. There are actually three types of search suggestions: + +* Previous searches the user has performed from the address bar and search bar + (denoted with a clock icon): + + .. image:: assets/nontechnical-overview/form-history.png + :alt: Image of a previous search result with text "porcupines" + :scale: 50% + :align: center + + This is the only type of search suggestion that is generated by Firefox alone, + without the help of a search engine. When the user performs a search using an + engine from the address bar or search bar (and only the address bar and search + bar), Firefox stores the search string, and then when the user starts to type + it again, Firefox includes it as a result to make it easy to perform past + searches. (Firefox does not store search strings used within web pages like + google.com.) + +* Suggestions from the user's default engine (denoted with a magnifying glass + icon): + + .. image:: assets/nontechnical-overview/search-suggestion.png + :alt: Image of a search suggestion result with text "porcupine meatballs" + :scale: 50% + :align: center + + These are fetched from the engine if the engine provides the necessary access + point. The ordering and total number of these suggestions is determined by the + engine. + +* Google-specific "tail" suggestions, which look like "... foo" and are provided + for long and/or specific queries to help the user narrow their search: + + .. image:: assets/nontechnical-overview/tail-suggestions.png + :alt: Image of a tail suggestion results with text "porcupine abc def" in + the input and two suggestions with text "... definition " and + "... defense" + :scale: 50% + :align: center + + These are fetched from Google when Google is the user's default engine. The + ordering and total number of these suggestions is determined by Google. + +The search suggestions group typically contains two previous searches followed +by four engine suggestions, but the exact numbers depend on the number of +matching previous searches and engine suggestions. Previous searches are limited +in number so that they don’t dominate this group, allowing remote suggestions to +provide content discovery benefits. Tail suggestions are shown only when there +are no other suggestions. + +The user can opt out of showing search suggestions in the address bar by +visiting about:preferences#search and unchecking "Provide search suggestions" or +"Show search suggestions in address bar results". + +4. General Results +~~~~~~~~~~~~~~~~~~ + +The final group of results is a general group that includes the following types: + +* Bookmarks +* History +* Open tabs (switch to tab) +* Remote tabs (via Sync) +* Sponsored and Firefox Suggest results (part of the Firefox Suggest feature) + +This general group is labeled "Firefox Suggest" in the Firefox Suggest feature. + +Typically this group contains 3 results, but as with search suggestions, the +exact number depends on certain factors (see `Result Composition Nuances`_). + +Most results within this group are first matched against the search string on +their titles and URLs and then sorted by a metric called **frecency**, a +combination of how frequently and how recently a page is visited. The top three +results are shown regardless of their specific types. + +This is the only group that is sorted by frecency. + +A few important complexities of this group are discussed in the next +subsections. The final subsection describes frecency in more detail. + +Adaptive History +................ + +The first few bookmark and history results in the general group may come from +**adaptive history**, a system that associates specific user search strings with +URLs. (It's also known as **input history**.) When the user types a search +string and picks a result, Firefox stores a database record that associates the +string with the result's URL. When the user types the string or a part of it +again, Firefox will try to show the URL they picked last time. This allows +Firefox to adapt to a user's habit of visiting certain pages via specific search +strings. + +This mechanism is mostly independent of frecency. URLs in the adaptive history +database have their own sorting score based on how many times they have been +used in the past. The score decays daily so that infrequently used search +strings and URLs aren't retained forever. (If two adaptive history results have +the same score, they are secondarily sorted by frecency.) + +Within the general group, the number of adaptive history results is not limited, +but typically there aren't many of them for a given search string. + +Open and Remote Tabs +.................... + +Unlike bookmarks and history, open and remote tabs don't have a "natural" +frecency, meaning a frecency that's updated in response to user actions as +described below in Frecency_. Tabs that match the search string are assigned +constant frecencies so they can participate in the sorting within the general +group. Open tabs are assigned a frecency of 1000, and remote tabs are assigned a +frecency of 1001. Picking appropriate frecencies is a bit of an art, but Firefox +has used these values for some time. + +Sponsored and Firefox Suggest Results +..................................... + +Sponsored and Firefox Suggest results are an exception within this group. They +are matched on predetermined keywords, and when present, they always appear last +in the general group. Frecency isn't involved at all. + +Frecency +........ + +Frecency is a complex topic on its own, but in summary, each URL stored in +Firefox's internal history database has a numeric score, the frecency, +associated with it. Larger numbers mean higher frecencies, and URLs with higher +frecencies are more likely to be surfaced to the user via the address bar. Each +time the user visits a URL, Firefox increases its frecency by a certain "boost" +amount that depends on how the visit is performed -- whether the user picked it +in the address bar, clicked its link on a page, clicked it in the history +sidebar, etc. In order to prevent frecencies from growing unbounded and to +penalize URLs that haven't been visited in a while, Firefox decays the +frecencies of all URLs over time. + +For details on frecency, see `The Frecency Algorithm`_. + +.. _The Frecency Algorithm: https://docs.google.com/document/d/10LRRXVGWWWcjEZIZ2YlEmuKkQqh2RaTclStFHNnPqQ8/edit#heading=h.588hanspexub + +Preferences that Affect Result Composition +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +There are a number of options in about:preferences that affect result +composition. + +The user can opt out of showing search suggestions in the address bar by +unchecking "Provide search suggestions" or "Show search suggestions in address +bar results" in about:preferences#search. (The first checkbox applies to both +the address bar and search bar, so it acts as a global toggle.) + +.. figure:: assets/nontechnical-overview/prefs-show-suggestions.png + :alt: Image of the preferences UI that allows the user to opt out of search + suggestions + :scale: 50% + :align: center + + Preferences allowing the user to opt out of search suggestions + +By default, the search suggestions group is shown before the general results +group, but unchecking "Show search suggestions ahead of browsing history in +address bar results" in about:preferences#search does the opposite. In that +case, typically the general results group will contain at most 6 results and the +search suggestions group will contain at most 3. In other words, regardless of +which group comes first, typically the first will contain 6 results and the +second will contain 3. + +.. figure:: assets/nontechnical-overview/prefs-suggestions-first.png + :alt: Image of the preferences UI that allows the user to choose whether + search suggestions are shown before general results + :scale: 50% + :align: center + + Preference allowing the user to choose which group is shown first + +The “Address Bar” section in about:preferences#privacy has several checkboxes +that allow for finer control over the types of results that appear in the view. +The top sites feature can be turned off by unchecking “Shortcuts” in this +section. + +.. figure:: assets/nontechnical-overview/prefs-privacy.png + :alt: Image of the preferences UI that allows the user to choose which + results are shown + :scale: 50% + :align: center + + Preferences allowing the user to choose which results are shown + +Result Composition Nuances +-------------------------- + +Among the search suggestions and general results groups, the group that's shown +first typically contains 6 results and the other group contains 3 results. The +exact number in each group depends on several factors: + +* The total maximum result count (controlled by the + ``browser.urlbar.maxRichResults`` hidden preference). + + The total number of results in the two groups scales up and down to + accommodate this number so that the view is always full of results. + +* The number of extension results. + + The extension results group comes before both groups, so if there are any + extension results, there are fewer available slots for search suggestions and + general results. + +* The number of matching results. + + The search string may match only one or two search suggestions or general + results, for example. + +* The number of results in the other group. + + The first group will try to contain 6 results and the second will try to + contain 3, but if either one is unable to fill up, then the other group will + be allowed to grow to make up the difference. + +Other Result Types +------------------ + +The most common result types are discussed above. This section walks through the +other types. + +An important trait these types have in common is that they do not belong to any +group. Most of them appear at specific positions within the view. + +Search Interventions +~~~~~~~~~~~~~~~~~~~~ + +Search interventions help the user perform a task based on their search string. +There are three kinds of interventions, and each is triggered by typing a +certain set of phrases_ in the input. They always appear as the second result, +after the heuristic result. + +The three kinds of interventions are: + +.. figure:: assets/nontechnical-overview/intervention-clear.png + :alt: Image of the clear intervention result with text "Clear your cache, + cookies, history and more" + :scale: 50% + :align: center + + Clear history, cache, and other data search intervention + +.. figure:: assets/nontechnical-overview/intervention-refresh.png + :alt: Image of the refresh intervention result with text "Restore default + settings and remove old add-ons for optimal performance" + :scale: 50% + :align: center + + Refresh Firefox search intervention + +.. figure:: assets/nontechnical-overview/intervention-update.png + :alt: Image of the update intervention result with text "The latest Firefox + is downloaded and ready to install" + :scale: 50% + :align: center + + Update Firefox search intervention + +Currently this feature is limited to English-speaking locales, but work is +ongoing to build a more sophisticated intent-matching platform to support other +locales, more complex search strings, and more kinds of interventions. + +.. _phrases: https://searchfox.org/mozilla-central/rev/c4d682be93f090e99d5f4049ceb7b6b6c03d0632/browser/components/urlbar/UrlbarProviderInterventions.jsm#64 + +Search Tips +~~~~~~~~~~~ + +Search tips inform the user they can perform searches directly from the +address bar. There are two kinds of search tips: + +.. figure:: assets/nontechnical-overview/search-tip-onboard.png + :alt: Image of the onboarding search tip with text "Type less, find more: + Search Google right from your address bar" + :scale: 50% + :align: center + + Onboarding search tip: Appears on the new-tab page + +.. figure:: assets/nontechnical-overview/search-tip-redirect.png + :alt: Image of the redirect search tip with text "Start your search in the + address bar to see suggestions from Google and your browsing history" + :scale: 50% + :align: center + + Redirect search tip: Appears on the home page of the user's default engine + (only for Google, Bing, and DuckDuckGo) + +In each case, the view automatically opens and shows the tip even if the user is +not interacting with the address bar. Each tip is shown at most four times, and +the user can stop them from appearing altogether by interacting with the address +bar or clicking the "Okay, Got It" button. + +Tab to Search +~~~~~~~~~~~~~ + +Tab to search allows the user to press the tab key to enter `search mode`_ while +typing the domain name of a search engine. There are two kinds of tab-to-search +results, and they always appear as the second result: + +.. figure:: assets/nontechnical-overview/tab-to-search-onboard.png + :alt: Image of the tab-to-search result with text "Search with Google" + :scale: 50% + :align: center + + Onboarding tab to search + +.. figure:: assets/nontechnical-overview/tab-to-search-regular.png + :alt: Image of the tab-to-search result with text "Search with Google" + :scale: 50% + :align: center + + Regular tab to search + +The onboarding type is shown until the user has interacted with it three times +over a period of at least 15 minutes, and after that the regular type is shown. + +Search Engine Offers +~~~~~~~~~~~~~~~~~~~~ + +Typing a single “@” shows a list of search engines. Selecting an engine enters +`search mode`_. + +.. figure:: assets/nontechnical-overview/search-offers.png + :alt: Image of the view showing search offer results + :scale: 50% + :align: center + + Search engine offers after typing “@” + +.. figure:: assets/nontechnical-overview/search-offers-selected.png + :alt: Image of the view showing search offer results with one selected + :scale: 50% + :align: center + + After pressing the down arrow key to select Google + +Search Mode +----------- + +**Search mode** is a feature that transforms the address bar into a search-only +access point for a particular engine. During search mode, search suggestions are +the only results shown in the view, and for that reason its result composition +differs from the usual composition. + +.. figure:: assets/nontechnical-overview/search-mode.png + :alt: Image of the view showing search mode + :scale: 50% + :align: center + + Search mode with Google as the selected engine + +Firefox shows suggestions in search mode even when the user has otherwise opted +out of them. Our rationale is that by entering search mode, the user has taken +an action that overrides their usual opt out. This allows the user to opt out +generally but opt back in at specific times. + +Search mode is an effective replacement for the legacy search bar and may +provide a good path forward for deprecating it. + +The user can enter search mode in many ways: + +* Picking a search shortcut button at the bottom of the view +* Typing an engine's keyword (which can be set in about:preferences#search, and + built-in engines have default keywords) +* Typing a single "?" followed by a space (to enter search mode with the default + engine) +* Typing a single "@" to list all engines and then picking one +* If the search bar is not also shown, pressing Ctrl+K (to enter search mode + with the default engine) + +To exit search mode, the user can backspace over the engine chiclet or click its +close button. diff --git a/browser/components/urlbar/docs/overview.rst b/browser/components/urlbar/docs/overview.rst new file mode 100644 index 0000000000..0ff1ee5ad7 --- /dev/null +++ b/browser/components/urlbar/docs/overview.rst @@ -0,0 +1,413 @@ +Architecture Overview +===================== + +The address bar is implemented as a *model-view-controller* (MVC) system. One of +the scopes of this architecture is to allow easy replacement of its components, +for easier experimentation. + +Each search is represented by a unique object, the *UrlbarQueryContext*. This +object, created by the *View*, describes the search and is passed through all of +the components, along the way it gets augmented with additional information. +The *UrlbarQueryContext* is passed to the *Controller*, and finally to the +*Model*. The model appends results to a property of *UrlbarQueryContext* in +chunks, it sorts them through a *Muxer* and then notifies the *Controller*. + +See the specific components below, for additional details about each one's tasks +and responsibilities. + + +The UrlbarQueryContext +---------------------- + +The *UrlbarQueryContext* object describes a single instance of a search. +It is augmented as it progresses through the system, with various information: + +.. highlight:: JavaScript +.. code:: + + UrlbarQueryContext { + allowAutofill; // {boolean} If true, providers are allowed to return + // autofill results. Even if true, it's up to providers + // whether to include autofill results, but when false, no + // provider should include them. + isPrivate; // {boolean} Whether the search started in a private context. + maxResults; // {integer} The maximum number of results requested. It is + // possible to request more results than the shown ones, and + // do additional filtering at the View level. + searchString; // {string} The user typed string. + userContextId; // {integer} The user context ID (containers feature). + + // Optional properties. + muxer; // {string} Name of a registered muxer. Muxers can be registered + // through the UrlbarProvidersManager. + providers; // {array} List of registered provider names. Providers can be + // registered through the UrlbarProvidersManager. + sources: {array} list of accepted UrlbarUtils.RESULT_SOURCE for the context. + // This allows to switch between different search modes. If not + // provided, a default will be generated by the Model, depending on + // the search string. + engineName: // {string} if sources is restricting to just SEARCH, this + // property can be used to pick a specific search engine, by + // setting it to the name under which the engine is registered + // with the search service. + currentPage: // {string} url of the page that was loaded when the search + // began. + prohibitRemoteResults: + // {boolean} This provides a short-circuit override for + // context.allowRemoteResults(). If it's false, then allowRemoteResults() + // will do its usual checks to determine whether remote results are + // allowed. If it's true, then allowRemoteResults() will immediately + // return false. Defaults to false. + + // Properties added by the Model. + results; // {array} list of UrlbarResult objects. + tokens; // {array} tokens extracted from the searchString, each token is an + // object in the form {type, value, lowerCaseValue}. + } + + +The Model +--------- + +The *Model* is the component responsible for retrieving search results based on +the user's input, and sorting them accordingly to their importance. +At the core is the `UrlbarProvidersManager <https://searchfox.org/mozilla-central/source/browser/components/urlbar/UrlbarProvidersManager.jsm>`_, +a component tracking all the available search providers, and managing searches +across them. + +The *UrlbarProvidersManager* is a singleton, it registers internal providers on +startup and can register/unregister providers on the fly. +It can manage multiple concurrent queries, and tracks them internally as +separate *Query* objects. + +The *Controller* starts and stops queries through the *UrlbarProvidersManager*. +It's possible to wait for the promise returned by *startQuery* to know when no +more results will be returned, it is not mandatory though. +Queries can be canceled. + +.. note:: + + Canceling a query will issue an interrupt() on the database connection, + terminating any running and future SQL query, unless a query is running inside + a *runInCriticalSection* task. + +The *searchString* gets tokenized by the `UrlbarTokenizer <https://searchfox.org/mozilla-central/source/browser/components/urlbar/UrlbarTokenizer.jsm>`_ +component into tokens, some of these tokens have a special meaning and can be +used by the user to restrict the search to specific result type (See the +*UrlbarTokenizer::TYPE* enum). + +.. caution:: + + The tokenizer uses heuristics to determine each token's type, as such the + consumer may want to check the value before applying filters. + +.. highlight:: JavaScript +.. code:: + + UrlbarProvidersManager { + registerProvider(providerObj); + unregisterProvider(providerObj); + registerMuxer(muxerObj); + unregisterMuxer(muxerObjOrName); + async startQuery(queryContext); + cancelQuery(queryContext); + // Can be used by providers to run uninterruptible queries. + runInCriticalSection(taskFn); + } + +UrlbarProvider +~~~~~~~~~~~~~~ + +A provider is specialized into searching and returning results from different +information sources. Internal providers are usually implemented in separate +*jsm* modules with a *UrlbarProvider* name prefix. External providers can be +registered as *Objects* through the *UrlbarProvidersManager*. +Each provider is independent and must satisfy a base API, while internal +implementation details may vary deeply among different providers. + +.. important:: + + Providers are singleton, and must track concurrent searches internally, for + example mapping them by UrlbarQueryContext. + +.. note:: + + Internal providers can access the Places database through the + *PlacesUtils.promiseLargeCacheDBConnection* utility. + +.. highlight:: JavaScript +.. code:: + + class UrlbarProvider { + /** + * Unique name for the provider, used by the context to filter on providers. + * Not using a unique name will cause the newest registration to win. + * @abstract + */ + get name() { + return "UrlbarProviderBase"; + } + /** + * The type of the provider, must be one of UrlbarUtils.PROVIDER_TYPE. + * @abstract + */ + get type() { + throw new Error("Trying to access the base class, must be overridden"); + } + /** + * Whether this provider should be invoked for the given context. + * If this method returns false, the providers manager won't start a query + * with this provider, to save on resources. + * @param {UrlbarQueryContext} queryContext The query context object + * @returns {boolean} Whether this provider should be invoked for the search. + * @abstract + */ + isActive(queryContext) { + throw new Error("Trying to access the base class, must be overridden"); + } + /** + * Gets the provider's priority. Priorities are numeric values starting at + * zero and increasing in value. Smaller values are lower priorities, and + * larger values are higher priorities. For a given query, `startQuery` is + * called on only the active and highest-priority providers. + * @param {UrlbarQueryContext} queryContext The query context object + * @returns {number} The provider's priority for the given query. + * @abstract + */ + getPriority(queryContext) { + // By default, all providers share the lowest priority. + return 0; + } + /** + * Starts querying. + * @param {UrlbarQueryContext} queryContext The query context object + * @param {function} addCallback Callback invoked by the provider to add a new + * result. A UrlbarResult should be passed to it. + * @note Extended classes should return a Promise resolved when the provider + * is done searching AND returning results. + * @abstract + */ + startQuery(queryContext, addCallback) { + throw new Error("Trying to access the base class, must be overridden"); + } + /** + * Cancels a running query, + * @param {UrlbarQueryContext} queryContext The query context object to cancel + * query for. + * @abstract + */ + cancelQuery(queryContext) { + throw new Error("Trying to access the base class, must be overridden"); + } + } + +UrlbarMuxer +~~~~~~~~~~~ + +The *Muxer* is responsible for sorting results based on their importance and +additional rules that depend on the UrlbarQueryContext. The muxer to use is +indicated by the UrlbarQueryContext.muxer property. + +.. caution:: + + The Muxer is a replaceable component, as such what is described here is a + reference for the default View, but may not be valid for other implementations. + +.. highlight:: JavaScript +.. code:: + + class UrlbarMuxer { + /** + * Unique name for the muxer, used by the context to sort results. + * Not using a unique name will cause the newest registration to win. + * @abstract + */ + get name() { + return "UrlbarMuxerBase"; + } + /** + * Sorts UrlbarQueryContext results in-place. + * @param {UrlbarQueryContext} queryContext the context to sort results for. + * @abstract + */ + sort(queryContext) { + throw new Error("Trying to access the base class, must be overridden"); + } + } + + +The Controller +-------------- + +`UrlbarController <https://searchfox.org/mozilla-central/source/browser/components/urlbar/UrlbarController.jsm>`_ +is the component responsible for reacting to user's input, by communicating +proper course of action to the Model (e.g. starting/stopping a query) and the +View (e.g. showing/hiding a panel). It is also responsible for reporting Telemetry. + +.. note:: + + Each *View* has a different *Controller* instance. + +.. highlight:: JavaScript +.. code:: + + UrlbarController { + async startQuery(queryContext); + cancelQuery(queryContext); + // Invoked by the ProvidersManager when results are available. + receiveResults(queryContext); + // Used by the View to listen for results. + addQueryListener(listener); + removeQueryListener(listener); + } + + +The View +-------- + +The View is the component responsible for presenting search results to the +user and handling their input. + +.. caution + + The View is a replaceable component, as such what is described here is a + reference for the default View, but may not be valid for other implementations. + +`UrlbarInput.jsm <https://searchfox.org/mozilla-central/source/browser/components/urlbar/UrlbarInput.jsm>`_ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Implements an input box *View*, owns an *UrlbarView*. + +.. highlight:: JavaScript +.. code:: + + UrlbarInput { + constructor(options = { textbox, panel }); + // Uses UrlbarValueFormatter to highlight the base host, search aliases + // and to keep the host visible on overflow. + formatValue(val); + openResults(); + // Converts an internal URI (e.g. a URI with a username or password) into + // one which we can expose to the user. + makeURIReadable(uri); + // Handles an event which would cause a url or text to be opened. + handleCommand(); + // Called by the view when a result is selected. + resultsSelected(); + // The underlying textbox + textbox; + // The results panel. + panel; + // The containing window. + window; + // The containing document. + document; + // An UrlbarController instance. + controller; + // An UrlbarView instance. + view; + // Whether the current value was typed by the user. + valueIsTyped; + // Whether the context is in Private Browsing mode. + isPrivate; + // Whether the input box is focused. + focused; + // The go button element. + goButton; + // The current value, can also be set. + value; + } + +`UrlbarView.jsm <https://searchfox.org/mozilla-central/source/browser/components/urlbar/UrlbarView.jsm>`_ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Represents the base *View* implementation, communicates with the *Controller*. + +.. highlight:: JavaScript +.. code:: + + UrlbarView { + // Manage View visibility. + open(); + close(); + // Invoked when the query starts. + onQueryStarted(queryContext); + // Invoked when new results are available. + onQueryResults(queryContext); + // Invoked when the query has been canceled. + onQueryCancelled(queryContext); + // Invoked when the query is done. This is invoked in any case, even if the + // query was canceled earlier. + onQueryFinished(queryContext); + // Invoked when the view opens. + onViewOpen(); + // Invoked when the view closes. + onViewClose(); + } + + +UrlbarResult +------------ + +An `UrlbarResult <https://searchfox.org/mozilla-central/source/browser/components/urlbar/UrlbarResult.jsm>`_ +instance represents a single search result with a result type, that +identifies specific kind of results. +Each kind has its own properties, that the *View* may support, and a few common +properties, supported by all of the results. + +.. note:: + + Result types are also enumerated by *UrlbarUtils.RESULT_TYPE*. + +.. code-block:: + + UrlbarResult { + constructor(resultType, payload); + + type: {integer} One of UrlbarUtils.RESULT_TYPE. + source: {integer} One of UrlbarUtils.RESULT_SOURCE. + title: {string} A title that may be used as a label for this result. + icon: {string} Url of an icon for this result. + payload: {object} Object containing properties for the specific RESULT_TYPE. + autofill: {object} An object describing the text that should be + autofilled in the input when the result is selected, if any. + autofill.value: {string} The autofill value. + autofill.selectionStart: {integer} The first index in the autofill + selection. + autofill.selectionEnd: {integer} The last index in the autofill selection. + suggestedIndex: {integer} Suggest a preferred position for this result + within the result set. Undefined if none. + isSuggestedIndexRelativeToGroup: {boolean} Whether the suggestedIndex + property is relative to the result's group + instead of the entire result set. + } + +The following RESULT_TYPEs are supported: + +.. highlight:: JavaScript +.. code:: + + // An open tab. + // Payload: { icon, url, userContextId } + TAB_SWITCH: 1, + // A search suggestion or engine. + // Payload: { icon, suggestion, keyword, query, providesSearchMode, inPrivateWindow, isPrivateEngine } + SEARCH: 2, + // A common url/title tuple, may be a bookmark with tags. + // Payload: { icon, url, title, tags } + URL: 3, + // A bookmark keyword. + // Payload: { icon, url, keyword, postData } + KEYWORD: 4, + // A WebExtension Omnibox result. + // Payload: { icon, keyword, title, content } + OMNIBOX: 5, + // A tab from another synced device. + // Payload: { icon, url, device, title } + REMOTE_TAB: 6, + // An actionable message to help the user with their query. + // Payload: { buttons, helpL10n, helpUrl, icon, titleL10n, type } + TIP: 7, + // A type of result created at runtime, for example by an extension. + // Payload: { dynamicType } + DYNAMIC: 8, diff --git a/browser/components/urlbar/docs/preferences.rst b/browser/components/urlbar/docs/preferences.rst new file mode 100644 index 0000000000..d26299b9b1 --- /dev/null +++ b/browser/components/urlbar/docs/preferences.rst @@ -0,0 +1,264 @@ +Preferences +=========== + +This document describes Preferences affecting the Firefox's address bar. +Preferences that are generated and updated by code won't be described here. + +User Exposed +------------ +These preferences are exposed through the Firefox UI. + +browser.urlbar.shortcuts.bookmarks (boolean, default: true) + Whether to show the bookmark search shortcut button in the view. + Can be controlled from Search Preferences. + +browser.urlbar.shortcuts.tabs (boolean, default: true) + Whether to show the tabs search shortcut button in the view. + Can be controlled from Search Preferences. + +browser.urlbar.shortcuts.history (boolean, default: true) + Whether to show the history search shortcut button in the view. + Can be controlled from Search Preferences. + +browser.urlbar.showSearchSuggestionsFirst (boolean, default: true) + Whether to show search suggestions before general results. + Can be controlled from Search Preferences. + +browser.urlbar.showSearchTerms.enabled (boolean, default: true) + Whether to show the search term in the urlbar + on a default search engine results page. + Can be controlled from Search Preferences. + +browser.urlbar.suggest.bookmark (boolean, default: true) + Whether results will include the user's bookmarks. + Can be controlled from Privacy Preferences. + +browser.urlbar.suggest.history (boolean, default: true) + Whether results will include the user's history. + Can be controlled from Privacy Preferences. + +browser.urlbar.suggest.openpage (boolean, default: true) + Whether results will include switch-to-tab results. + Can be controlled from Privacy Preferences. + +browser.urlbar.suggest.searches (boolean, default: true) + Whether results will include search suggestions. + Can be controlled from Search Preferences. + +browser.urlbar.suggest.engines (boolean, default: true) + Whether results will include search engines (e.g. tab-to-search). + Can be controlled from Privacy Preferences. + +browser.urlbar.suggest.topsites (boolean, default: true) + Whether results will include top sites and the view will open on focus. + Can be controlled from Privacy Preferences. + +browser.search.suggest.enabled (boolean, default: true) + Whether search suggestions are enabled globally, including the separate search + bar. + Can be controlled from Search Preferences. + +browser.search.suggest.enabled.private (boolean, default: false) + When search suggestions are enabled, controls whether they are provided in + Private Browsing windows. + Can be controlled from Search Preferences. + + +Hidden +------ +These preferences are normally hidden, and should not be used unless you really +know what you are doing. + +browser.urlbar.accessibility.tabToSearch.announceResults (boolean: default: true) + Whether we announce to screen readers when tab-to-search results are inserted. + +browser.urlbar.autoFill (boolean, default: true) + Autofill is the the feature that automatically completes domains and URLs that + the user has visited as the user is typing them in the urlbar textbox. + +browser.urlbar.autoFill.adaptiveHistory.enabled (boolean, default: false) + Whether adaptive history autofill feature is enabled. + +browser.urlbar.autoFill.adaptiveHistory.useCountThreshold (float, default: 1.0) + Threshold for use count of input history that we handle as adaptive history + autofill. If the use count is this value or more, it will be a candidate. + +browser.urlbar.bestMatch.enabled (boolean, default: false) + Whether the best match feature is enabled. + +browser.urlbar.autoFill.stddevMultiplier (float, default: 0.0) + Affects the frecency threshold of the autofill algorithm. The threshold is + the mean of all origin frecencies, plus one standard deviation multiplied by + this value. + +browser.urlbar.ctrlCanonizesURLs (boolean, default: true) + Whether using `ctrl` when hitting return/enter in the URL bar (or clicking + 'go') should prefix 'www.' and suffix browser.fixup.alternate.suffix to the + user value prior to navigating. + +browser.urlbar.decodeURLsOnCopy (boolean, default: false) + Whether copying the entire URL from the location bar will put a human + readable (percent-decoded) URL on the clipboard. + +browser.urlbar.delay (number, default: 50) + The amount of time (ms) to wait after the user has stopped typing before + fetching certain results. Reducing this doesn't make the Address Bar faster, + it will instead make it access the disk more heavily, and potentially make it + slower. Certain results, like the heuristic, always skip this timer anyway. + +browser.urlbar.dnsResolveSingleWordsAfterSearch (number, default: 0) + Controls when to DNS resolve single word search strings, after they were + searched for. If the string is resolved as a valid host, show a + "Did you mean to go to 'host'" prompt. + Set to 0. 0: Never resolve, 1: Use heuristics, 2. Always resolve. + +browser.urlbar.eventTelemetry.enabled (boolean, default: false) + Whether telemetry events should be recorded. This is expensive and should only + be enabled by experiments with a small population. + +browser.urlbar.extension.timeout (integer, default: 400) + When sending events to extensions, they have this amount of time in + milliseconds to respond before timing out. This affects the omnibox API. + +browser.urlbar.filter.javascript (boolean, default: true) + When true, `javascript:` URLs are not included in search results for safety + reasons. + +browser.urlbar.formatting.enabled (boolean, default: true) + Applies URL highlighting and other styling to the text in the urlbar input + field. This should usually be enabled for security reasons. + +browser.urlbar.maxCharsForSearchSuggestions (integer, default: 100) + As a user privacy measure, we don't fetch results from remote services for + searches that start by pasting a string longer than this. The pref name + indicates search suggestions, but this is used for all remote results. + +browser.urlbar.maxHistoricalSearchSuggestions (integer, default: 2) + The maximum number of form history results to include as search history. + +browser.urlbar.maxRichResults (integer, default: 10) + The maximum number of results in the urlbar popup. + +browser.urlbar.merino.clientVariants (string, default: "") + Comma separated list of client variants to send to send to Merino. See + `Merino API docs <https://mozilla-services.github.io/merino/api.html#suggest>`_ + for more details. This is intended to be used by experiments, not directly set + by users. + +browser.urlbar.merino.enabled (boolean, default: false) + Whether Merino is enabled as a quick suggest source. + +browser.urlbar.merino.providers (string, default: "") + Comma-separated list of providers to request from the Merino server. Merino + will return suggestions only for these providers. See `Merino API docs`_ for + more details. + +browser.urlbar.openintab (boolean, default: false) + Whether address bar results should be opened in new tabs by default. + +browser.urlbar.quicksuggest.enabled (boolean, default: false) + Whether the quick suggest feature is enabled, i.e., sponsored and recommended + results related to the user's search string. This pref can be overridden by + the ``quickSuggestEnabled`` Nimbus variable. If false, neither sponsored nor + non-sponsored quick suggest results will be shown. If true, then we look at + the individual prefs ``browser.urlbar.suggest.quicksuggest.nonsponsored`` and + ``browser.urlbar.suggest.quicksuggest.sponsored``. + +browser.urlbar.quicksuggest.remoteSettings.enabled (boolean, default: true) + Whether remote settings is enabled as a quick suggest source. + +browser.urlbar.quicksuggest.dataCollection.enabled (boolean, default: false) + Whether data collection is enabled for quick suggest results. + +browser.urlbar.quicksuggest.shouldShowOnboardingDialog (boolean, default: true) + Whether to show the quick suggest onboarding dialog. + +browser.urlbar.richSuggestions.tail (boolean, default: true) + If true, we show tail search suggestions when available. + +browser.urlbar.searchTips.test.ignoreShowLimits (boolean, default: false) + Disables checks that prevent search tips being shown, thus showing them every + time the newtab page or the default search engine homepage is opened. + This is useful for testing purposes. + +browser.urlbar.speculativeConnect.enabled (boolean, default: true) + Speculative connections allow to resolve domains pre-emptively when the user + is likely to pick a result from the Address Bar. This allows for faster + navigation. + +browser.urlbar.sponsoredTopSites (boolean, default: false) + Whether top sites may include sponsored ones. + +browser.urlbar.suggest.bestmatch (boolean, default: true) + Whether to show the best match result is enabled. This pref is ignored if + browser.urlbar.bestMatch.enabled is false. + +browser.urlbar.suggest.quicksuggest.nonsponsored (boolean, default: false) + Whether results will include non-sponsored quick suggest suggestions. + +browser.urlbar.suggest.quicksuggest.sponsored (boolean, default: false) + Whether results will include sponsored quick suggest suggestions. + +browser.urlbar.switchTabs.adoptIntoActiveWindow (boolean, default: false) + When using switch to tabs, if set to true this will move the tab into the + active window, instead of just switching to it. + +browser.urlbar.trimURLs (boolean, default: true) + Clean-up URLs when showing them in the Address Bar. + +keyword.enabled (boolean, default: true) + By default, when the search string is not recognized as a potential url, + search for it with the default search engine. If set to false any string will + be handled as a potential URL, even if it's invalid. + +browser.fixup.dns_first_for_single_words (boolean, default: false) + If true, any single word search string will be sent to the DNS server before + deciding whether to search or visit it. This may add a delay to the urlbar. + + +Experimental +------------ +These preferences are experimental and not officially supported. They could be +removed at any time. + +browser.urlbar.suggest.calculator (boolean, default: false) + Whether results will include a calculator. + +browser.urlbar.unitConversion.enabled (boolean, default: false) + Whether unit conversion is enabled. + +browser.urlbar.unitConversion.suggestedIndex (integer, default: 1) + The index where we show unit conversion results. + +browser.urlbar.experimental.expandTextOnFocus (boolean, default: false) + Whether we expand the font size when the urlbar is focused. + +browser.urlbar.experimental.searchButton (boolean, default: false) + Whether to displays a permanent search button before the urlbar. + +browser.urlbar.keepPanelOpenDuringImeComposition (boolean, default: false) + Whether the results panel should be kept open during IME composition. The + panel may overlap with the IME compositor panel. + +browser.urlbar.restyleSearches (boolean, default: false) + When true, URLs in the user's history that look like search result pages + are restyled to look like search engine results instead of history results. + +browser.urlbar.update2.emptySearchBehavior (integer, default: 0) + Controls the empty search behavior in Search Mode: 0. Show nothing, 1. Show + search history, 2. Show search and browsing history + +Deprecated +---------- +These preferences should not be used and may be removed at any time. + +browser.urlbar.autoFill.searchEngines (boolean, default: false) + If true, the domains of the user's installed search engines will be + autofilled even if the user hasn't actually visited them. + +browser.urlbar.usepreloadedtopurls.enabled (boolean, default: false) + Results will include a built-in set of popular domains when this is true. + +browser.urlbar.usepreloadedtopurls.expire_days (integer, default: 14) + After this many days from the profile creation date, the built-in set of + popular domains will no longer be included in the results. diff --git a/browser/components/urlbar/docs/ranking.rst b/browser/components/urlbar/docs/ranking.rst new file mode 100644 index 0000000000..a1c9d03c3c --- /dev/null +++ b/browser/components/urlbar/docs/ranking.rst @@ -0,0 +1,229 @@ +======= +Ranking +======= + +Before results appear in the UrlbarView, they are fetched from providers. + +Each `UrlbarProvider <https://firefox-source-docs.mozilla.org/browser/urlbar/overview.html#urlbarprovider>`_ +implements its own internal ranking and returns sorted results. + +Externally all the results are ranked by the `UrlbarMuxer <https://searchfox.org/mozilla-central/source/browser/components/urlbar/UrlbarMuxerUnifiedComplete.jsm>`_ +according to an hardcoded list of groups and sub-grups. + +.. NOTE:: Preferences can influence the groups order, for example by putting + Firefox Suggest before Search Suggestions. + +The Places provider, responsible to return history and bookmark results, uses +an internal ranking algorithm called Frecency. + +Frecency implementation +======================= + +Frecency is a term derived from `frequency` and `recency`, its scope is to provide a +ranking algorithm that gives importance both to how often a page is accessed and +when it was last visited. +Additionally, it accounts for the type of each visit through a bonus system. + +To account for `recency`, a bucketing system is implemented. +If a page has been visited later than the bucket cutoff, it gets the weight +associated with that bucket: + +- Up to 4 days old - weight 100 - ``places.frecency.firstBucketCutoff/Weight`` +- Up to 14 days old - weight 70 - ``places.frecency.secondBucketCutoff/Weight`` +- Up to 31 days old - weight 50 - ``places.frecency.thirdBucketCutoff/Weight`` +- Up to 90 days old - weight 30 - ``places.frecency.fourthBucketCutoff/Weight`` +- Anything else - weight 10 - ``places.frecency.defaultBucketWeight`` + +To account for `frequency`, the total number of visits to a page is used to +calculate the final score. + +The type of each visit is taken into account using specific bonuses: + +Default bonus + Any unknown type gets a default bonus. This is expected to be unused. + Pref ``places.frecency.defaultVisitBonus`` current value: 0. +Embed + Used for embedded/framed visits not due to user actions. These visits today + are stored in memory and never participate to frecency calculation. + Thus this is currently unused. + Pref ``places.frecency.embedVisitBonus`` current value: 0. +Framed Link + Used for cross-frame visits due to user action. + Pref ``places.frecency.framedLinkVisitBonus`` current value: 0. +Download + Used for download visits. It’s important to support link coloring for these + visits, but they are not necessarily useful address bar results (the Downloads + view can do a better job with these), so their frecency can be low. + Pref ``places.frecency.downloadVisitBonus`` current value: 0. +Reload + Used for reload visits (refresh same page). Low because it should not be + possible to influence frecency by multiple reloads. + Pref ``places.frecency.reloadVisitBonus`` current value: 0. +Redirect Source + Used when the page redirects to another one. + It’s a low value because we give more importance to the final destination, + that is what the user actually visits, especially for permanent redirects. + Pref ``places.frecency.redirectSourceVisitBonus`` current value: 25. +Temporary Redirect + Used for visits resulting from a temporary redirect (HTTP 307). + Pref ``places.frecency.tempRedirectVisitBonus`` current value: 40. +Permanent Redirect + Used for visits resulting from a permanent redirect (HTTP 301). This is the + new supposed destination for a url, thus the bonus is higher than temporary. + In this case it may be advisable to just pick the bonus for the source visit. + Pref ``places.frecency.permRedirectVisitBonus`` current value: 50. +Bookmark + Used for visits generated from bookmark views. + Pref ``places.frecency.bookmarkVisitBonus`` current value: 75. +Link + Used for normal visits, for example when clicking on a link. + Pref ``places.frecency.linkVisitBonus`` current value: 100. +Typed + Intended to be used for pages typed by the user, in reality it is used when + the user picks a url from the UI (history views or the Address Bar). + Pref ``places.frecency.typedVisitBonus`` current value: 2000. + +The above bonuses are applied to visits, in addition to that there are also a +few bonuses applied in case a page is not visited at all, both of these bonuses +can be applied at the same time: + +Unvisited bookmarked page + Used for pages that are bookmarked but unvisited. + Pref ``places.frecency.unvisitedBookmarkBonus`` current value: 140. +Unvisited typed page + Used for pages that were typed and now are bookmarked (otherwise they would + be orphans). + Pref ``places.frecency.unvisitedTypedBonus`` current value: 200. + +Two special frecency values are also defined: + +- ``-1`` represents a just inserted entry in the database, whose score has not + been calculated yet. +- ``0`` represents an entry for which a new value should not be calculated, + because it has a poor user value (e.g. place: queries) among search results. + +Finally, because calculating a score from all of the visits every time a new +visit is added would be expensive, only a sample of the last 10 +(pref ``places.frecency.numVisits``) visits is used. + +How frecency for a page is calculated +------------------------------------- + +.. mermaid:: + :align: center + :caption: Frecency calculation flow + + flowchart TD + start[URL] + a0{Has visits?} + a1[Get last 10 visit] + a2[bonus = unvisited_bonus + bookmarked + typed] + a3{bonus > 0?} + end0[Frecency = 0] + end1["frecency = age_bucket_weight * (bonus / 100)"] + a4[Sum points of all sampled visits] + a5{points > 0?} + end2[frecency = -1] + end3["Frecency = visit_count * (points / sample_size)"] + subgraph sub [Per each visit] + sub0[bonus = visit_type_bonus] + sub1{bookmarked?} + sub2[add bookmark bonus] + sub3["score = age_bucket_weight * (bonus / 100)"] + sub0 --> sub1 + sub1 -- yes --> sub2 + sub1 -- no --> sub3 + sub2 --> sub3; + end + start --> a0 + a0 -- no --> a2 + a2 --> a3 + a3 -- no --> end0 + a3 -- yes --> end1 + a0 -- yes --> a1 + a1 --> sub + sub --> a4 + a4 --> a5 + a5 -- no --> end2 + a5 -- yes --> end3 + +1. If the page is visited, get a sample of ``NUM_VISITS`` most recent visits. +2. For each visit get a transition bonus, depending on the visit type. +3. If the page is bookmarked, add to the bonus an additional bookmark bonus. +4. If the bonus is positive, get a bucket weight depending on the visit date. +5. Calculate points for the visit as ``age_bucket_weight * (bonus / 100)``. +6. Sum points for all the sampled visits. +7. If the points sum is zero, return a ``-1`` frecency, it will still appear in the UI. + Otherwise, frecency is ``visitCount * points / NUM_VISITS``. +8. If the page is unvisited and not bookmarked, or it’s a bookmarked place-query, + return a ``0`` frecency, to hide it from the UI. +9. If it’s bookmarked, add the bookmark bonus. +10. If it’s also a typed page, add the typed bonus. +11. Frecency is ``age_bucket_weight * (bonus / 100)`` + +When frecency for a page is calculated +-------------------------------------- + +Operations that may influence the frecency score are: + +* Adding visits +* Removing visits +* Adding bookmarks +* Removing bookmarks +* Changing the url of a bookmark + +Frecency is recalculated: + +* Immediately, when a new visit is added. The user expectation here is that the + page appears in search results after being visited. This is also valid for + any History API that allows to add visits. +* In background on idle times, in any other case. In most cases having a + temporarily stale value is not a problem, the main concern would be privacy + when removing history of a page, but removing whole history will either + completely remove the page or, if it's bookmarked, it will still be relevant. + In this case, when a change influencing frecency happens, the ``recalc_frecency`` + database field for the page is set to ``1``. + +Recalculation is done by the `PlacesFrecencyRecalculator <https://searchfox.org/mozilla-central/source/toolkit/components/places/PlacesFrecencyRecalculator.sys.mjs>`_ module. +The Recalculator is notified when ``PlacesUtils.history.shouldStartFrecencyRecalculation`` +value changes from false to true, that means there's values to recalculate. +A DeferredTask is armed, that will look for a user idle opportunity +in the next 5 minutes, otherwise it will run when that time elapses. +Once all the outdated values have been recalculated +``PlacesUtils.history.shouldStartFrecencyRecalculation`` is set back to false +until the next operation invalidating a frecency. +The recalculation task is also armed on the ``idle-daily`` notification. + +When the task is executed, it recalculates frecency of a chunk of pages. If +there are more pages left to recalculate, the task is re-armed. After frecency +of a page is recalculated, its ``recalc_frecency`` field is set back to ``0``. + +Frecency is also decayed daily during the ``idle-daily`` notification, by +multiplying all the scores by a decay rate of ``0.975`` (half-life of 28 days). +This guarantees entries not receiving new visits or bookmarks lose relevancy. + + +Adaptive Input History +====================== + +Input History (also known as Adaptive History) is a feature that allows to +find urls that the user previously picked. To do so, it associates search strings +with picked urls. + +Adaptive results are usually presented before frecency derived results, making +them appear as having an infinite frecency. + +When the user types a given string, and picks a result from the address bar, that +relation is stored and increases a use_count field for the given string. +The use_count field asymptotically approaches a max of ``10`` (the update is +done as ``use_count * .9 + 1``). + +On querying, all the search strings that start with the input string are matched, +a rank is calculated per each page as ``ROUND(MAX(use_count) * (1 + (input = :search_string)), 1)``, +so that results perfectly matching the search string appear at the top. +Results with the same rank are additionally sorted by descending frecency. + +On daily idles, when frecency is decayed, also input history gets decayed, in +particular the use_count field is multiplied by a decay rate of ``0.975``. +After decaying, any entry that has a ``use_count < 0.975^90 (= 0.1)`` is removed, +thus entries are removed if unused for 90 days. diff --git a/browser/components/urlbar/docs/telemetry.rst b/browser/components/urlbar/docs/telemetry.rst new file mode 100644 index 0000000000..dbc28db120 --- /dev/null +++ b/browser/components/urlbar/docs/telemetry.rst @@ -0,0 +1,769 @@ +Telemetry +========= + +This section describes existing telemetry probes measuring interaction with the +Address Bar. + +For telemetry specific to Firefox Suggest, see the +:doc:`firefox-suggest-telemetry` document. + +.. contents:: + :depth: 2 + + +Histograms +---------- + +PLACES_AUTOCOMPLETE_1ST_RESULT_TIME_MS +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + This probe tracks the amount of time it takes to get the first result. + It is an exponential histogram with values between 5 and 100. + +PLACES_AUTOCOMPLETE_6_FIRST_RESULTS_TIME_MS +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + This probe tracks the amount of time it takes to get the first six results. + It is an exponential histogram with values between 50 and 1000. + +FX_URLBAR_SELECTED_RESULT_METHOD +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + This probe tracks how a result was picked by the user from the list. + It is a categorical histogram with these values: + + - ``enter`` + The user pressed Enter without selecting a result first. + This most likely happens when the user confirms the default preselected + result (aka *heuristic result*), or when they select with the keyboard a + one-off search button and confirm with Enter. + - ``enterSelection`` + The user selected a result, but not using Tab or the arrow keys, and then + pressed Enter. This is a rare and generally unexpected event, there may be + exotic ways to select a result we didn't consider, that are tracked here. + Look at arrowEnterSelection and tabEnterSelection for more common actions. + - ``click`` + The user clicked on a result. + - ``arrowEnterSelection`` + The user selected a result using the arrow keys, and then pressed Enter. + - ``tabEnterSelection`` + The first key the user pressed to select a result was the Tab key, and then + they pressed Enter. Note that this means the user could have used the arrow + keys after first pressing the Tab key. + - ``rightClickEnter`` + Before QuantumBar, it was possible to right-click a result to highlight but + not pick it. Then the user could press Enter. This is no more possible. + +FX_URLBAR_ZERO_PREFIX_DWELL_TIME_MS +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + This probe records the amount of time the zero-prefix view was shown; that is, + the time from when it was opened to the time it was closed. "Zero-prefix" + means the search string was empty, so the zero-prefix view is the view that's + shown when the user clicks in the urlbar before typing a search string. Often + it's also called the "top sites" view since normally it shows the user's top + sites. This is an exponential histogram whose values range from 0 to 60,000 + with 50 buckets. Values are in milliseconds. This histogram was introduced in + Firefox 110.0 in bug 1806765. + +PLACES_FRECENCY_RECALC_CHUNK_TIME_MS +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + This records the time necessary to recalculate frecency of a chunk of pages, + as defined in the `PlacesFrecencyRecalculator <https://searchfox.org/mozilla-central/source/toolkit/components/places/PlacesFrecencyRecalculator.sys.mjs>`_ module. + +Scalars +------- + +urlbar.abandonment +~~~~~~~~~~~~~~~~~~ + + A uint recording the number of abandoned engagements in the urlbar. An + abandonment occurs when the user begins using the urlbar but stops before + completing the engagement. This can happen when the user clicks outside the + urlbar to focus a different part of the window. It can also happen when the + user switches to another window while the urlbar is focused. + +urlbar.autofill_deletion +~~~~~~~~~~~~~~~~~~~~~~~~ + + A uint recording the deletion count for autofilled string in the urlbar. + This occurs when the user deletes whole autofilled string by BACKSPACE or + DELETE key while the autofilled string is selected. + +urlbar.engagement +~~~~~~~~~~~~~~~~~ + + A uint recording the number of engagements the user completes in the urlbar. + An engagement occurs when the user navigates to a page using the urlbar, for + example by picking a result in the urlbar panel or typing a search term or URL + in the urlbar and pressing the enter key. + +urlbar.impression.* +~~~~~~~~~~~~~~~~~~~ + + A uint recording the number of impression that was displaying when user picks + any result. + + - ``autofill_about`` + For about-page type autofill. + - ``autofill_adaptive`` + For adaptive history type autofill. + - ``autofill_origin`` + For origin type autofill. + - ``autofill_other`` + Counts how many times some other type of autofill result that does not have + a specific scalar was shown. This is a fallback that is used when the code is + not properly setting a specific autofill type, and it should not normally be + used. If it appears in the data, it means we need to investigate and fix the + code that is not properly setting a specific autofill type. + - ``autofill_preloaded`` + For preloaded site type autofill. + - ``autofill_url`` + For url type autofill. + +urlbar.persistedsearchterms.revert_by_popup_count +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + A uint that is incremented when search terms are persisted in the Urlbar and + the Urlbar is reverted to show a full URL due to a PopupNotification. This + can happen when a user is on a SERP and permissions are requested, e.g. + request access to location. If the popup is persistent and the user did not + dismiss it before switching tabs, the popup will reappear when they return to + the tab. Thus, when returning to the tab with the persistent popup, this + value will be incremented because it should have persisted search terms but + instead showed a full URL. + +urlbar.persistedsearchterms.view_count +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + A uint that is incremented when search terms should be persisted in the + Urlbar. This will trigger when a user loads a SERP from any SAP that results + in the search terms persisting in the Urlbar, as well as switching to a tab + containing a SERP that should be persisting the search terms in the Urlbar, + regardless of whether a PopupNotification is present. Thus, for every + ``revert_by_popup_count``, there should be at least one corresponding + ``view_count``. + +urlbar.tips +~~~~~~~~~~~ + + This is a keyed scalar whose values are uints and are incremented each time a + tip result is shown, a tip is picked, and a tip's help button is picked. The + keys are: + + - ``intervention_clear-help`` + Incremented when the user picks the help button in the clear-history search + intervention. + - ``intervention_clear-picked`` + Incremented when the user picks the clear-history search intervention. + - ``intervention_clear-shown`` + Incremented when the clear-history search intervention is shown. + - ``intervention_refresh-help`` + Incremented when the user picks the help button in the refresh-Firefox + search intervention. + - ``intervention_refresh-picked`` + Incremented when the user picks the refresh-Firefox search intervention. + - ``intervention_refresh-shown`` + Incremented when the refresh-Firefox search intervention is shown. + - ``intervention_update_ask-help`` + Incremented when the user picks the help button in the update_ask search + intervention, which is shown when there's a Firefox update available but the + user's preference says we should ask them to download and apply it. + - ``intervention_update_ask-picked`` + Incremented when the user picks the update_ask search intervention. + - ``intervention_update_ask-shown`` + Incremented when the update_ask search intervention is shown. + - ``intervention_update_refresh-help`` + Incremented when the user picks the help button in the update_refresh search + intervention, which is shown when the user's browser is up to date but they + triggered the update intervention. We show this special refresh intervention + instead. + - ``intervention_update_refresh-picked`` + Incremented when the user picks the update_refresh search intervention. + - ``intervention_update_refresh-shown`` + Incremented when the update_refresh search intervention is shown. + - ``intervention_update_restart-help`` + Incremented when the user picks the help button in the update_restart search + intervention, which is shown when there's an update and it's been downloaded + and applied. The user needs to restart to finish. + - ``intervention_update_restart-picked`` + Incremented when the user picks the update_restart search intervention. + - ``intervention_update_restart-shown`` + Incremented when the update_restart search intervention is shown. + - ``intervention_update_web-help`` + Incremented when the user picks the help button in the update_web search + intervention, which is shown when we can't update the browser or possibly + even check for updates for some reason, so the user should download the + latest version from the web. + - ``intervention_update_web-picked`` + Incremented when the user picks the update_web search intervention. + - ``intervention_update_web-shown`` + Incremented when the update_web search intervention is shown. + - ``tabtosearch-shown`` + Increment when a non-onboarding tab-to-search result is shown, once per + engine per engagement. Please note that the number of times non-onboarding + tab-to-search results are picked is the sum of all keys in + ``urlbar.searchmode.tabtosearch``. Please also note that more detailed + telemetry is recorded about both onboarding and non-onboarding tab-to-search + results in urlbar.tabtosearch.*. These probes in ``urlbar.tips`` are still + recorded because ``urlbar.tabtosearch.*`` is not currently recorded + in Release. + - ``tabtosearch_onboard-shown`` + Incremented when a tab-to-search onboarding result is shown, once per engine + per engagement. Please note that the number of times tab-to-search + onboarding results are picked is the sum of all keys in + ``urlbar.searchmode.tabtosearch_onboard``. + - ``searchTip_onboard-picked`` + Incremented when the user picks the onboarding search tip. + - ``searchTip_onboard-shown`` + Incremented when the onboarding search tip is shown. + - ``searchTip_persist-picked`` + Incremented when the user picks the urlbar persisted search tip. + - ``searchTip_persist-shown`` + Incremented when the url persisted search tip is shown. + - ``searchTip_redirect-picked`` + Incremented when the user picks the redirect search tip. + - ``searchTip_redirect-shown`` + Incremented when the redirect search tip is shown. + +urlbar.searchmode.* +~~~~~~~~~~~~~~~~~~~ + + This is a set of keyed scalars whose values are uints incremented each + time search mode is entered in the Urlbar. The suffix on the scalar name + describes how search mode was entered. Possibilities include: + + - ``bookmarkmenu`` + Used when the user selects the Search Bookmarks menu item in the Library + menu. + - ``handoff`` + Used when the user uses the search box on the new tab page and is handed off + to the address bar. NOTE: This entry point was disabled from Firefox 88 to + 91. Starting with 91, it will appear but in low volume. Users must have + searching in the Urlbar disabled to enter search mode via handoff. + - ``keywordoffer`` + Used when the user selects a keyword offer result. + - ``oneoff`` + Used when the user selects a one-off engine in the Urlbar. + - ``shortcut`` + Used when the user enters search mode with a keyboard shortcut or menu bar + item (e.g. ``Accel+K``). + - ``tabmenu`` + Used when the user selects the Search Tabs menu item in the tab overflow + menu. + - ``tabtosearch`` + Used when the user selects a tab-to-search result. These results suggest a + search engine when the search engine's domain is autofilled. + - ``tabtosearch_onboard`` + Used when the user selects a tab-to-search onboarding result. These are + shown the first few times the user encounters a tab-to-search result. + - ``topsites_newtab`` + Used when the user selects a search shortcut Top Site from the New Tab Page. + - ``topsites_urlbar`` + Used when the user selects a search shortcut Top Site from the Urlbar. + - ``touchbar`` + Used when the user taps a search shortct on the Touch Bar, available on some + Macs. + - ``typed`` + Used when the user types an engine alias in the Urlbar. + - ``historymenu`` + Used when the user selects the Search History menu item in a History + menu. + - ``other`` + Used as a catchall for other behaviour. We don't expect this scalar to hold + any values. If it does, we need to correct an issue with search mode entry + points. + + The keys for the scalars above are engine and source names. If the user enters + a remote search mode with a built-in engine, we record the engine name. If the + user enters a remote search mode with an engine they installed (e.g. via + OpenSearch or a WebExtension), we record ``other`` (not to be confused with + the ``urlbar.searchmode.other`` scalar above). If they enter a local search + mode, we record the English name of the result source (e.g. "bookmarks", + "history", "tabs"). Note that we slightly modify the engine name for some + built-in engines: we flatten all localized Amazon sites (Amazon.com, + Amazon.ca, Amazon.de, etc.) to "Amazon" and we flatten all localized + Wikipedia sites (Wikipedia (en), Wikipedia (fr), etc.) to "Wikipedia". This + is done to reduce the number of keys used by these scalars. + +urlbar.picked.* +~~~~~~~~~~~~~~~ + + This is a set of keyed scalars whose values are uints incremented each + time a result is picked from the Urlbar. The suffix on the scalar name + is the result type. The keys for the scalars above are the 0-based index of + the result in the urlbar panel when it was picked. + + .. note:: + Available from Firefox 84 on. Use the *FX_URLBAR_SELECTED_** histograms in + earlier versions. + + .. note:: + Firefox 102 deprecated ``autofill`` and added ``autofill_about``, + ``autofill_adaptive``, ``autofill_origin``, ``autofill_other``, + ``autofill_preloaded``, and ``autofill_url``. + + Valid result types are: + + - ``autofill`` + This scalar was deprecated in Firefox 102 and replaced with + ``autofill_about``, ``autofill_adaptive``, ``autofill_origin``, + ``autofill_other``, ``autofill_preloaded``, and ``autofill_url``. Previously + it was recorded in each of the cases that the other scalars now cover. + - ``autofill_about`` + An autofilled "about:" page URI (e.g., about:config). The user must first + type "about:" to trigger this type of autofill. + - ``autofill_adaptive`` + An autofilled URL from the user's adaptive history. This type of autofill + differs from ``autofill_url`` in two ways: (1) It's based on the user's + adaptive history, a particular type of history that associates the user's + search string with the URL they pick in the address bar. (2) It autofills + full URLs instead of "up to the next slash" partial URLs. For more + information on this type of autofill, see this `adaptive history autofill + document`_. + - ``autofill_origin`` + An autofilled origin_ from the user's history. Typically "origin" means a + domain or host name like "mozilla.org". Technically it can also include a + URL scheme or protocol like "https" and a port number like ":8000". Firefox + can autofill domain names by themselves, domain names with schemes, domain + names with ports, and domain names with schemes and ports. All of these + cases count as origin autofill. For more information, see this `adaptive + history autofill document`_. + - ``autofill_other`` + Counts how many times some other type of autofill result that does not have + a specific keyed scalar was picked at a given index. This is a fallback that + is used when the code is not properly setting a specific autofill type, and + it should not normally be used. If it appears in the data, it means we need + to investigate and fix the code that is not properly setting a specific + autofill type. + - ``autofill_preloaded`` + An autofilled `preloaded site`_. The preloaded-sites feature (as it relates + to this telemetry scalar) has never been enabled in Firefox, so this scalar + should never be recorded. It can be enabled by flipping a hidden preference, + however. It's included here for consistency and correctness. + - ``autofill_url`` + An autofilled URL or partial URL from the user's history. Firefox autofills + URLs "up to the next slash", so to trigger URL autofill, the user must first + type a domain name (or trigger origin autofill) and then begin typing the + rest of the URL (technically speaking, its path). As they continue typing, + the URL will only be partially autofilled up to the next slash, or if there + is no next slash, to the end of the URL. This allows the user to easily + visit different subpaths of a domain. For more information, see this + `adaptive history autofill document`_. + - ``bookmark`` + A bookmarked URL. + - ``dynamic`` + A specially crafted result, often used in experiments when basic types are + not flexible enough for a rich layout. + - ``dynamic_wikipedia`` + A dynamic Wikipedia Firefox Suggest result. + - ``extension`` + Added by an add-on through the omnibox WebExtension API. + - ``formhistory`` + A search suggestion from previous search history. + - ``history`` + A URL from history. + - ``keyword`` + A bookmark keyword. + - ``navigational`` + A navigational suggestion Firefox Suggest result. + - ``quickaction`` + A QuickAction. + - ``quicksuggest`` + A Firefox Suggest (a.k.a. quick suggest) suggestion. + - ``remotetab`` + A tab synced from another device. + - ``searchengine`` + A search result, but not a suggestion. May be the default search action + or a search alias. + - ``searchsuggestion`` + A remote search suggestion. + - ``switchtab`` + An open tab. + - ``tabtosearch`` + A tab to search result. + - ``tip`` + A tip result. + - ``topsite`` + An entry from top sites. + - ``trending`` + A trending suggestion. + - ``unknown`` + An unknown result type, a bug should be filed to figure out what it is. + - ``visiturl`` + The user typed string can be directly visited. + - ``weather`` + A Firefox Suggest weather suggestion. + + .. _adaptive history autofill document: https://docs.google.com/document/d/e/2PACX-1vRBLr_2dxus-aYhZRUkW9Q3B1K0uC-a0qQyE3kQDTU3pcNpDHb36-Pfo9fbETk89e7Jz4nkrqwRhi4j/pub + .. _origin: https://html.spec.whatwg.org/multipage/origin.html#origin + .. _preloaded site: https://searchfox.org/mozilla-central/source/browser/components/urlbar/UrlbarProviderPreloadedSites.jsm + +urlbar.picked.searchmode.* +~~~~~~~~~~~~~~~~~~~~~~~~~~ + + This is a set of keyed scalars whose values are uints incremented each time a + result is picked from the Urlbar while the Urlbar is in search mode. The + suffix on the scalar name is the search mode entry point. The keys for the + scalars are the 0-based index of the result in the urlbar panel when it was + picked. + + .. note:: + These scalars share elements of both ``urlbar.picked.*`` and + ``urlbar.searchmode.*``. Scalar name suffixes are search mode entry points, + like ``urlbar.searchmode.*``. The keys for these scalars are result indices, + like ``urlbar.picked.*``. + + .. note:: + These data are a subset of the data recorded by ``urlbar.picked.*``. For + example, if the user enters search mode by clicking a one-off then selects + a Google search suggestion at index 2, we would record in **both** + ``urlbar.picked.searchsuggestion`` and ``urlbar.picked.searchmode.oneoff``. + +urlbar.tabtosearch.* +~~~~~~~~~~~~~~~~~~~~ + + This is a set of keyed scalars whose values are uints incremented when a + tab-to-search result is shown, once per engine per engagement. There are two + sub-probes: ``urlbar.tabtosearch.impressions`` and + ``urlbar.tabtosearch.impressions_onboarding``. The former records impressions + of regular tab-to-search results and the latter records impressions of + onboarding tab-to-search results. The key values are identical to those of the + ``urlbar.searchmode.*`` probes: they are the names of the engines shown in the + tab-to-search results. Engines that are not built in are grouped under the + key ``other``. + + .. note:: + Due to the potentially sensitive nature of these data, they are currently + collected only on pre-release version of Firefox. See bug 1686330. + +urlbar.zeroprefix.abandonment +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + A uint recording the number of abandonments of the zero-prefix view. + "Zero-prefix" means the search string was empty, so the zero-prefix view is + the view that's shown when the user clicks in the urlbar before typing a + search string. Often it's called the "top sites" view since normally it shows + the user's top sites. "Abandonment" means the user opened the zero-prefix view + but it was closed without the user picking a result inside it. This scalar was + introduced in Firefox 110.0 in bug 1806765. + +urlbar.zeroprefix.engagement +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + A uint recording the number of engagements in the zero-prefix view. + "Zero-prefix" means the search string was empty, so the zero-prefix view is + the view that's shown when the user clicks in the urlbar before typing a + search string. Often it's called the "top sites" view since normally it shows + the user's top sites. "Engagement" means the user picked a result inside the + view. This scalar was introduced in Firefox 110.0 in bug 1806765. + +urlbar.zeroprefix.exposure +~~~~~~~~~~~~~~~~~~~~~~~~~~ + + A uint recording the number of times the user was exposed to the zero-prefix + view; that is, the number of times it was shown. "Zero-prefix" means the + search string was empty, so the zero-prefix view is the view that's shown when + the user clicks in the urlbar before typing a search string. Often it's called + the "top sites" view since normally it shows the user's top sites. This scalar + was introduced in Firefox 110.0 in bug 1806765. + +urlbar.quickaction.impression +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + A uint recording the number of times the user was shown a quickaction, the + key is in the form $key-$n where $n is the number of characters the user typed + in order for the suggestion to show. See bug 1806024. + +urlbar.quickaction.picked +~~~~~~~~~~~~~~~~~~~~~~~~~ + + A uint recording the number of times the user selected a quickaction, the + key is in the form $key-$n where $n is the number of characters the user typed + in order for the suggestion to show. See bug 1783155. + +places.* +~~~~~~~~ + + This is Places related telemetry. + + Valid result types are: + + - ``sponsored_visit_no_triggering_url`` + Number of sponsored visits that could not find their triggering URL in + history. We expect this to be a small number just due to the navigation layer + manipulating URLs. A large or growing value may be a concern. + - ``pages_need_frecency_recalculation`` + Number of pages in need of a frecency recalculation. This number should + remain small compared to the total number of pages in the database (see the + `PLACES_PAGES_COUNT` histogram). It can be used to valuate the frequency + and size of recalculations, for performance reasons. + +Search Engagement Telemetry +--------------------------- + +The search engagement telemetry provided since Firefox 110 is is recorded using +Glean events. Because of the data size, these events are collected only for a +subset of the population, using the Glean Sampling feature. Please see the +following documents for the details. + + - `Engagement`_ : + It is defined as a completed action in urlbar, where a user picked one of + the results. + - `Abandonment`_ : + It is defined as an action where the user open the results but does not + complete an engagement action, usually unfocusing the urlbar. This also + happens when the user switches to another window, if the results popup was + opening. + - `Impression`_ : + It is defined as an action where the results had been shown to the user for + a while. In default, it will be recorded when the same results have been + shown and 1 sec has elapsed. The interval value can be modified through the + `browser.urlbar.searchEngagementTelemetry.pauseImpressionIntervalMs` + preference. + +.. _Engagement: https://dictionary.telemetry.mozilla.org/apps/firefox_desktop/metrics/urlbar_engagement +.. _Abandonment: https://dictionary.telemetry.mozilla.org/apps/firefox_desktop/metrics/urlbar_abandonment +.. _Impression: https://dictionary.telemetry.mozilla.org/apps/firefox_desktop/metrics/urlbar_impression + + +Custom pings for Contextual Services +------------------------------------ + +Contextual Services currently has two features involving the address bar, top +sites and Firefox Suggest. Top sites telemetry is described below. For Firefox +Suggest, see the :doc:`firefox-suggest-telemetry` document. + +Firefox sends the following `custom pings`_ to record impressions and clicks of +the top sites feature. + + .. _custom pings: https://docs.telemetry.mozilla.org/cookbooks/new_ping.html#sending-a-custom-ping + +Top Sites Impression +~~~~~~~~~~~~~~~~~~~~ + + This records an impression when a sponsored top site is shown. + + - ``context_id`` + A UUID representing this user. Note that it's not client_id, nor can it be + used to link to a client_id. + - ``tile_id`` + A unique identifier for the sponsored top site. + - ``source`` + The browser location where the impression was displayed. + - ``position`` + The placement of the top site (1-based). + - ``advertiser`` + The Name of the advertiser. + - ``reporting_url`` + The reporting URL of the sponsored top site, normally pointing to the ad + partner's reporting endpoint. + - ``version`` + Firefox version. + - ``release_channel`` + Firefox release channel. + - ``locale`` + User's current locale. + +Changelog + Firefox 108.0 + The impression ping is sent for Pocket sponsored tiles as well. Pocket sponsored tiles have different values for ``advertiser`` and ``reporting_url`` is null. [Bug 1794022_] + + Firefox 87.0 + Introduced. [Non_public_doc_] + +.. _Non_public_doc: https://docs.google.com/document/d/1qLb4hUwR8YQj5QnjJtwxQIoDCPLQ6XuAmJPQ6_WmS4E/edit +.. _1794022: https://bugzilla.mozilla.org/show_bug.cgi?id=1794022 + +Top Sites Click +~~~~~~~~~~~~~~~ + + This records a click ping when a sponsored top site is clicked by the user. + + - ``context_id`` + A UUID representing this user. Note that it's not client_id, nor can it be + used to link to a client_id. + - ``tile_id`` + A unique identifier for the sponsored top site. + - ``source`` + The browser location where the click was tirggered. + - ``position`` + The placement of the top site (1-based). + - ``advertiser`` + The Name of the advertiser. + - ``reporting_url`` + The reporting URL of the sponsored top site, normally pointing to the ad + partner's reporting endpoint. + - ``version`` + Firefox version. + - ``release_channel`` + Firefox release channel. + - ``locale`` + User's current locale. + +Changelog + Firefox 108.0 + The click ping is sent for Pocket sponsored tiles as well. Pocket sponsored tiles have different values for ``advertiser`` and ``reporting_url`` is null. [Bug 1794022_] + + Firefox 87.0 + Introduced. [Non_public_doc_] + + +Other telemetry relevant to the Address Bar +------------------------------------------- + +Search Telemetry +~~~~~~~~~~~~~~~~ + + Some of the `search telemetry`_ is also relevant to the address bar. + +contextual.services.topsites.* +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + These keyed scalars instrument the impressions and clicks for sponsored top + sites in the urlbar. + The key is a combination of the source and the placement of the top sites link + (1-based) such as 'urlbar_1'. For each key, it records the counter of the + impression or click. + Note that these scalars are shared with the top sites on the newtab page. + +Telemetry Environment +~~~~~~~~~~~~~~~~~~~~~ + + The following preferences relevant to the address bar are recorded in + :doc:`telemetry environment data </toolkit/components/telemetry/data/environment>`: + + - ``browser.search.suggest.enabled``: The global toggle for search + suggestions everywhere in Firefox (search bar, urlbar, etc.). Defaults to + true. + - ``browser.urlbar.autoFill``: The global preference for whether autofill in + the urlbar is enabled. When false, all types of autofill are disabled. + - ``browser.urlbar.autoFill.adaptiveHistory.enabled``: True if adaptive + history autofill in the urlbar is enabled. + - ``browser.urlbar.suggest.searches``: True if search suggestions are + enabled in the urlbar. Defaults to false. + +Firefox Suggest +~~~~~~~~~~~~~~~ + + Telemetry specific to Firefox Suggest is described in the + :doc:`firefox-suggest-telemetry` document. + +.. _search telemetry: /browser/search/telemetry.html + +Event Telemetry +--------------- + + .. note:: + This is a legacy event telemetry. For the current telemetry, please see + `Search Engagement Telemetry`_. These legacy events were disabled by default + and required enabling through a preference or a Urlbar WebExtension + experimental API. + +.. _Search Engagement Telemetry: #search-engagement-telemetry + +The event telemetry is grouped under the ``urlbar`` category. + +Event Method +~~~~~~~~~~~~ + + There are two methods to describe the interaction with the urlbar: + + - ``engagement`` + It is defined as a completed action in urlbar, where a user inserts text + and executes one of the actions described in the Event Object. + - ``abandonment`` + It is defined as an action where the user inserts text but does not + complete an engagement action, usually unfocusing the urlbar. This also + happens when the user switches to another window, regardless of urlbar + focus. + +Event Value +~~~~~~~~~~~ + + This is how the user interaction started + + - ``typed``: The text was typed into the urlbar. + - ``dropped``: The text was drag and dropped into the urlbar. + - ``pasted``: The text was pasted into the urlbar. + - ``topsites``: The user opened the urlbar view without typing, dropping, + or pasting. + In these cases, if the urlbar input is showing the URL of the loaded page + and the user has not modified the input’s content, the urlbar views shows + the user’s top sites. Otherwise, if the user had modified the input’s + content, the urlbar view shows results based on what the user has typed. + To tell whether top sites were shown, it's enough to check whether value is + ``topsites``. To know whether the user actually picked a top site, check + check that ``numChars`` == 0. If ``numChars`` > 0, the user initially opened + top sites, but then they started typing and confirmed a different result. + - ``returned``: The user abandoned a search, for example by switching to + another tab/window, or focusing something else, then came back to it + and continued. We consider a search continued if the user kept at least the + first char of the original search string. + - ``restarted``: The user abandoned a search, for example by switching to + another tab/window, or focusing something else, then came back to it, + cleared it and then typed a new string. + +Event Object +~~~~~~~~~~~~ + + These describe actions in the urlbar: + + - ``click`` + The user clicked on a result. + - ``enter`` + The user confirmed a result with Enter. + - ``drop_go`` + The user dropped text on the input field. + - ``paste_go`` + The user used Paste and Go feature. It is not the same as paste and Enter. + - ``blur`` + The user unfocused the urlbar. This is only valid for ``abandonment``. + +Event Extra +~~~~~~~~~~~ + + This object contains additional information about the interaction. + Extra is a key-value store, where all the keys and values are strings. + + - ``elapsed`` + Time in milliseconds from the initial interaction to an action. + - ``numChars`` + Number of input characters the user typed or pasted at the time of + submission. + - ``numWords`` + Number of words in the input. The measurement is taken from a trimmed input + split up by its spaces. This is not a perfect measurement, since it will + return an incorrect value for languages that do not use spaces or URLs + containing spaces in its query parameters, for example. + - ``selType`` + The type of the selected result at the time of submission. + This is only present for ``engagement`` events. + It can be one of: ``none``, ``autofill``, ``visiturl``, ``bookmark``, + ``history``, ``keyword``, ``searchengine``, ``searchsuggestion``, + ``switchtab``, ``remotetab``, ``extension``, ``oneoff``, ``keywordoffer``, + ``canonized``, ``tip``, ``tiphelp``, ``formhistory``, ``tabtosearch``, + ``help``, ``block``, ``quicksuggest``, ``unknown`` + In practice, ``tabtosearch`` should not appear in real event telemetry. + Opening a tab-to-search result enters search mode and entering search mode + does not currently mark the end of an engagement. It is noted here for + completeness. Similarly, ``block`` indicates a result was blocked or deleted + but should not appear because blocking a result does not end the engagement. + - ``selIndex`` + Index of the selected result in the urlbar panel, or -1 for no selection. + There won't be a selection when a one-off button is the only selection, and + for the ``paste_go`` or ``drop_go`` objects. There may also not be a + selection if the system was busy and results arrived too late, then we + directly decide whether to search or visit the given string without having + a fully built result. + This is only present for ``engagement`` events. + - ``provider`` + The name of the result provider for the selected result. Existing values + are: ``HeuristicFallback``, ``Autofill``, ``Places``, + ``TokenAliasEngines``, ``SearchSuggestions``, ``UrlbarProviderTopSites``. + Data from before Firefox 91 will also list ``UnifiedComplete`` as a + provider. This is equivalent to ``Places``. + Values can also be defined by `URLBar provider experiments`_. + + .. _URLBar provider experiments: experiments.html#developing-address-bar-extensions diff --git a/browser/components/urlbar/docs/testing.rst b/browser/components/urlbar/docs/testing.rst new file mode 100644 index 0000000000..a56bd297a7 --- /dev/null +++ b/browser/components/urlbar/docs/testing.rst @@ -0,0 +1,216 @@ +Testing +======= +This documentation discusses how to write a test for the address bar and the +different test utilities that are useful when writing a test for the address +bar. + +Common Tests +------------ +Mochitests +~~~~~~~~~~ +Some common tests for the address bar are the **mochitests**. The purpose of +a mochitest is to run the browser itself. Mochitests can be called +"browser tests", "mochitest-browser-chrome", or +"browser-chrome-mochitests". There are other types of mochitests that are not +for testing the browser and therefore can be ignored for the purpose of the +address bar. An example of a mochitest is +`tests/browser/browser_switchTab_currentTab.js <https://searchfox.org/mozilla- +central/source/browser/components/urlbar/tests/browser/browser_switchTab_ +currentTab.js>`_ + +XPCShell +~~~~~~~~ +`XPCShell Tests <https://firefox-source-docs.mozilla.org/testing/xpcshell/index +.html>`_ are another type of test relevant to the address bar. XPCShell tests +are often called unit tests because they tend to test specific modules or +components in isolation, as opposed the mochitest which have access to the full +browser chrome. + +XPCShell tests do not use the browser UI and are completely separate from +browser chrome. XPCShell tests are executed in a JavaScript shell that is +outside of the browser. For historical context, the "XPC" naming convention is +from XPCOM (Cross Platform Component Model) which is an older framework that +allows programmers to write custom functions in one language, such as C++, and +connect it to other components in another language, such as JavaScript. + +Each XPCShell test is executed in a new shell instance, therefore you will +see several Firefox icons pop up and close when XPCShell tests are executing. +These are two examples of XPCShell tests for the address bar +`test_providerHeuristicFallback <https://searchfox.org/mozilla-central/source +/browser/components/urlbar/tests/unit/test_providerHeuristicFallback.js>`_ +and +`test_providerTabToSearch <https://searchfox.org/mozilla-central/source/browser +/components/urlbar/tests/unit/test_providerTabToSearch.js>`_. + +When To Write a XPCShell or Mochitest? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Always default to writing an XPCShell test if it is possible. XPCShell +tests are faster to execute than browser tests. Although, most of the time you +will write a browser test because you could be modifying something in the UI or +testing a specific component in the UI. + +If you are writing a test for a urlbarProvider, you can test the Provider +through a XPCShell test. Providers do not modify the UI, instead what they do is +receive a url string query, search for the string and bring back the result. An +example is the `ProviderPlaces <https://searchfox.org/mozilla-central/sou +rce/browser/components/urlbar/UrlbarProviderPlaces.jsm>`_, which fetches +results from the Places database. Another component that’s good for writing +XPCShell test is the `urlbarMuxer <https://searchfox.org/mozilla-central/ +source/browser/components/urlbar/UrlbarMuxerUnifiedComplete.jsm>`_. + +There may be times where writing both an XPCShell test and browser test is +necessary. In these situations, you could be testing the result from a Provider +and also testing what appears in the UI is correct. + +How To Write a Test +------------------- + +Test Boilerplate +~~~~~~~~~~~~~~~~ +This basic test boilerplate includes a license code at the top and this license +code is present at the top of every test file, the ``"use strict"`` string is +to enable strict mode in JavaScript, and ``add_task`` function adds tests to be +executed by the test harness. + +.. code-block:: javascript + + /* Any copyright is dedicated to the Public Domain. + * http://creativecommons.org/publicdomain/zero/1.0/ */ + + /** + * This tests ensures that the urlbar ... + */ + + "use strict"; + + add_task(async function testOne() { + // testing code and assertions + }); + + add_task(async function testTwo() { + // testing code and assertions + }); + +In order to run a test use the ``./mach`` command, for example, ``./mach test <path to test +file>`` to run test locally. Use the command with ``--jsdebugger`` argument at +the end to open the DevTools debugger to step through the test, ``./mach test +<path to test> --jsdebugger``. + +Manifest +~~~~~~~~ +The manifest's purpose is to list all the test in the directory and dictate to +the test harness which files are test and how the test harness should run these +test. Anytime a test is created, the test file name needs to be added to the +manifest in alphabetical order. + +Start in the manifest file and add your test name in alphabetical +order. The manifest file we should add our test in is +`browser.ini <https://searchfox.org/mozilla-central/source/browser/components/ +urlbar/tests/browser/browser.ini>`_. The ``urlbar/test/browser/`` directory +is the main browser test directory for address bar, and the manifest file +linked above is the main browser test manifest. +The ``.ini`` file extension is an initialization file for Windows or MS-DOS. + +Manifest Metadata +~~~~~~~~~~~~~~~~~ +The manifest file can define common keys/metadata to influence the test's +behavior. For example, the metadata ``support-files`` are a list of additional +files required to run a test. Any values assigned to the key ``support-files`` +only applies to the single file directly above the ``support-files`` key. +If more files require ``support-files``, then ``support-files`` need to be +added directly under the other test file names. Another example of a manifest +metadata is ``[DEFAULT]``. Anything under ``[DEFAULT]`` will be picked up by +all tests in the manifest file. + +For information on all the manifest metadata available, please visit +:doc:`/build/buildsystem/test_manifests`. + +Common Test Utilities +--------------------- +This section describes common test utilities which may be useful when writing a +test for the address bar. Below are a description of common utils where you can +find helpful testing methods. + +Many test utils modules end with ``TestUtils.jsm``. However not every testing +function will end with ``TestUtils.jsm``. For example, `PlacesUtils <https:// +searchfox.org/mozilla-central/source/toolkit/components/places/PlacesUtils. +jsm>`_ does not have “Test” within its name. + +A critical function to remember is the ``registerCleanupFunction`` within +the ``head.js`` file mentioned below. This function's purpose may be to clean +up the history or any other clean ups that are necessary after your test is +complete. Cleaning up after a browser test is necessary because clean up +ensures what is done within one test will not affect subsequent tests. + +head.js and common-head.js +~~~~~~~~~~~~~~~~~~~~~~~~~~ +The `head.js <https://searchfox.org/mozilla-central/source/browser/components +/urlbar/tests/browser/head.js>`_ file is executed at the beginning before each +test and contains imports to modules which are useful for each test. +Any tasks ``head.js`` adds (via add_task) will run first for each test, and +any variables and functions it defines will be available in the scope of +each test. This file is small because most of our Utils are actually in other +`.jsm` files. + +The ``XPCOMUtils.defineLazyModuleGetters`` method within ``head.js`` sets up +modules names to where they can be found, their paths. ``Lazy`` means the files +are only imported if or when it is used. Any tests in this directory can use +these modules without importing it themselves in their own file. +The ``head.js`` provides a convenience for this purpose. The ``head.js`` file +imports `common-head.js <https://searchfox.org/mozilla-central/source/browser/components/urlbar/tests/browser/head-common.js>`_ +making everything within ``head-common.js`` available in ``head.js`` as well. + +The ``registerCleanupFunction`` is an important function in browser mochi tests +and it is part of the test harness. This function registers a callback function +to be executed when your test is complete. The purpose may be to clean up the +history or any other clean ups that are necessary after your test is complete. +For example, browser mochi tests are executed one after the other in the same +window instance. The global object in each test is the browser ``window`` +object, for example, each test script runs in the browser window. +If the history is not cleaned up it will remain and may affect subsequent +browser tests. For most test outside of address bar, you may not need to clear +history. In addition to cleanup, ``head.js`` calls the +``registerCleanupFunction`` to ensure the urlbar panel is closed after each +test. + +UrlbarTestUtils +~~~~~~~~~~~~~~~ +`UrlbarTestUtils.sys.mjs <https://searchfox.org/mozilla-central/source/browser/components/urlbar/tests/UrlbarTestUtils.sys.mjs>`_ is useful for url bar testing. This +file contains methods that can help with starting a new search in the url bar, +waiting for a new search to complete, returning the results in +the view, and etc. + +BrowserTestUtils +~~~~~~~~~~~~~~~~ +`BrowserTestUtils.sys.mjs <../../testing/browser-chrome/browsertestutils.html>`_ +is useful for browser window testing. This file contains methods that can help +with opening tabs, waiting for certain events to happen in the window, opening +new or private windows, and etc. + +TestUtils +~~~~~~~~~ +`TestUtils.jsm <../../testing/testutils.html>`_ is useful for general +purpose testing and does not depend on the browser window. This file contains +methods that are useful when waiting for a condition to return true, waiting for +a specific preference to change, and etc. + +PlacesTestUtils +~~~~~~~~~~~~~~~ +:searchfox:`PlacesTestUtils.sys.mjs <toolkit/components/places/tests/PlacesTestU +tils.sys.mjs>` is useful for adding visits, adding +bookmarks, waiting for notification of visited pages, and etc. + +EventUtils +~~~~~~~~~~ +`EventUtils.js <https://searchfox.org/mozilla-central/source/testing/mochitest +/tests/SimpleTest/EventUtils.js>`_ is an older test file and does not +need to be imported because it is not a ``.jsm`` file. ``EventUtils`` is only +used for browser tests, unlike the other TestUtils listed above which are +used for browser tests, XPCShell tests and other tests. + +All the functions within ``EventUtils.js`` are automatically available in +browser tests. This file contains functions that are useful for synthesizing +mouse clicks and keypresses. Some commonly used functions are +``synthesizeMouseAtCenter`` which places the mouse at the center of the DOM +element and ``synthesizeKey`` which can be used to navigate the view and start +a search by using keydown and keyenter arguments. diff --git a/browser/components/urlbar/docs/utilities.rst b/browser/components/urlbar/docs/utilities.rst new file mode 100644 index 0000000000..8fa3b8509e --- /dev/null +++ b/browser/components/urlbar/docs/utilities.rst @@ -0,0 +1,26 @@ +Utilities +========= + +Various modules provide shared utilities to the other components: + +`UrlbarPrefs.jsm <https://searchfox.org/mozilla-central/source/browser/components/urlbar/UrlbarPrefs.jsm>`_ +------------------------------------------------------------------------------------------------------------- + +Implements a Map-like storage or urlbar related preferences. The values are kept +up-to-date. + +.. highlight:: JavaScript +.. code:: + + // Always use browser.urlbar. relative branch, except for the preferences in + // PREF_OTHER_DEFAULTS. + UrlbarPrefs.get("delay"); // Gets value of browser.urlbar.delay. + +.. note:: + + Newly added preferences should always be properly documented in UrlbarPrefs. + +`UrlbarUtils.jsm <https://searchfox.org/mozilla-central/source/browser/components/urlbar/UrlbarUtils.jsm>`_ +------------------------------------------------------------------------------------------------------------- + +Includes shared utils and constants shared across all the components. |
