From 9e3c08db40b8916968b9f30096c7be3f00ce9647 Mon Sep 17 00:00:00 2001 From: Daniel Baumann Date: Sun, 21 Apr 2024 13:44:51 +0200 Subject: Adding upstream version 1:115.7.0. Signed-off-by: Daniel Baumann --- toolkit/components/extensions/docs/background.rst | 133 +++++ toolkit/components/extensions/docs/basics.rst | 275 ++++++++++ toolkit/components/extensions/docs/events.rst | 609 +++++++++++++++++++++ toolkit/components/extensions/docs/functions.rst | 201 +++++++ .../docs/generate_webidl_from_jsonschema.rst | 94 ++++ ...rate_webidl_from_jsonschema_dataflow.drawio.svg | 4 + toolkit/components/extensions/docs/incognito.rst | 78 +++ toolkit/components/extensions/docs/index.rst | 33 ++ toolkit/components/extensions/docs/lifecycle.rst | 60 ++ toolkit/components/extensions/docs/manifest.rst | 68 +++ toolkit/components/extensions/docs/other.rst | 140 +++++ toolkit/components/extensions/docs/reference.rst | 35 ++ toolkit/components/extensions/docs/schema.rst | 145 +++++ .../components/extensions/docs/webext-storage.rst | 227 ++++++++ .../components/extensions/docs/webidl_bindings.rst | 246 +++++++++ ..._backgroundWorker_apiRequestHandling.drawio.svg | 4 + .../docs/wiring_up_new_webidl_bindings.rst | 165 ++++++ 17 files changed, 2517 insertions(+) create mode 100644 toolkit/components/extensions/docs/background.rst create mode 100644 toolkit/components/extensions/docs/basics.rst create mode 100644 toolkit/components/extensions/docs/events.rst create mode 100644 toolkit/components/extensions/docs/functions.rst create mode 100644 toolkit/components/extensions/docs/generate_webidl_from_jsonschema.rst create mode 100644 toolkit/components/extensions/docs/generate_webidl_from_jsonschema_dataflow.drawio.svg create mode 100644 toolkit/components/extensions/docs/incognito.rst create mode 100644 toolkit/components/extensions/docs/index.rst create mode 100644 toolkit/components/extensions/docs/lifecycle.rst create mode 100644 toolkit/components/extensions/docs/manifest.rst create mode 100644 toolkit/components/extensions/docs/other.rst create mode 100644 toolkit/components/extensions/docs/reference.rst create mode 100644 toolkit/components/extensions/docs/schema.rst create mode 100644 toolkit/components/extensions/docs/webext-storage.rst create mode 100644 toolkit/components/extensions/docs/webidl_bindings.rst create mode 100644 toolkit/components/extensions/docs/webidl_bindings_backgroundWorker_apiRequestHandling.drawio.svg create mode 100644 toolkit/components/extensions/docs/wiring_up_new_webidl_bindings.rst (limited to 'toolkit/components/extensions/docs') diff --git a/toolkit/components/extensions/docs/background.rst b/toolkit/components/extensions/docs/background.rst new file mode 100644 index 0000000000..5d5dcd06b9 --- /dev/null +++ b/toolkit/components/extensions/docs/background.rst @@ -0,0 +1,133 @@ +Background +========== + +WebExtensions run in a sandboxed environment much like regular web content. +The purpose of extensions is to enhance the browser in a way that +regular content cannot -- WebExtensions APIs bridge this gap by exposing +browser features to extensions in a way preserves safety, reliability, +and performance. +The implementation of a WebExtension API runs with +:doc:`chrome privileges `. +Browser internals are accessed using +:ref:`XPCOM` +or :doc:`ChromeOnly WebIDL features `. + +The rest of this documentation covers how API implementations interact +with the implementation of WebExtensions. +To expose some browser feature to WebExtensions, the first step is +to design the API. Some high-level principles for API design +are documented on the Mozilla wiki: + +- `Vision for WebExtensions `_ +- `API Policies `_ +- `Process for creating new APIs `_ + +Javascript APIs +--------------- +Many WebExtension APIs are accessed directly from extensions through +Javascript. Functions are the most common type of object to expose, +though some extensions expose properties of primitive Javascript types +(e.g., constants). +Regardless of the exact method by which something is exposed, +there are a few important considerations when designing part of an API +that is accessible from Javascript: + +- **Namespace**: + Everything provided to extensions is exposed as part of a global object + called ``browser``. For compatibility with Google Chrome, many of these + features are also exposed on a global object called ``chrome``. + Functions and other objects are not exposed directly as properties on + ``browser``, they are organized into *namespaces*, which appear as + properties on ``browser``. For example, the + `tabs API `_ + uses a namespace called ``tabs``, so all its functions and other + properties appear on the object ``browser.tabs``. + For a new API that provides features via Javascript, the usual practice + is to create a new namespace with a concise but descriptive name. + +- **Environments**: + There are several different types of Javascript environments in which + extension code can execute: extension pages, content scripts, proxy + scripts, and devtools pages. + Extension pages include the background page, popups, and content pages + accessed via |getURL|_. + When creating a new Javascript feature the designer must choose + in which of these environments the feature will be available. + Most Javascript features are available in extension pages, + other environments have limited sets of API features available. + +.. |getURL| replace:: ``browser.runtime.getURL()`` +.. _getURL: https://developer.mozilla.org/en-US/Add-ons/WebExtensions/API/runtime/getURL + +- **Permissions**: + Many Javascript features are only present for extensions that + include an appropriate permission in the manifest. + The guidelines for when an API feature requires a permission are + described in (*citation needed*). + +The specific types of features that can be exposed via Javascript are: + +- **Functions**: + A function callable from Javascript is perhaps the most commonly + used feature in WebExtension APIs. + New API functions are asynchronous, returning a + `Promise `_. Even functions that do not return a result + use Promises so that errors can be indicated asynchronously + via a rejected Promise as opposed to a synchronously thrown Error. + This is due to the fact that extensions run in a child process and + many API functions require communication with the main process. + If an API function that needs to communicate in this way returned a + synchronous result, then all Javascript execution in the child + process would need to be paused until a response from the main process + was received. Even if a function could be implemented synchronously + within a child process, the standard practice is to make it + asynchronous so as not to constrain the implementation of the underlying + browser feature and make it impossible to move functionality out of the + child process. + Another consequence of functions using inter-process communication is + that the parameters to a function and its return value must all be + simple data types that can be sent between processes using the + `structured clone algorithm `_. + +- **Events**: + Events complement functions (which allow an extension to call into + an API) by allowing an event within the browser to invoke a callback + in the extension. + Any time an API requires an extension to pass a callback function that + gets invoked some arbitrary number of times, that API method should be + defined as an event. + +Manifest Keys +------------- +In addition to providing functionality via Javascript, WebExtension APIs +can also take actions based on the contents of particular properties +in an extension's manifest (or even just the presence of a particular +property). +Manifest entries are used for features in which an extension specifies +some static information that is used when an extension is installed or +when it starts up (i.e., before it has the chance to run any code to use +a Javascript API). +An API may handle a manifest key and implement Javascript functionality, +see the +`browser action `_ +API for an example. + +Other Considerations +-------------------- +In addition to the guidelines outlined above, +there are some other considerations when designing and implementing +a WebExtension API: + +- **Cleanup**: A badly written WebExtension should not be able to permanently + leak any resources. In particular, any action from an extension that + causes a resource to be allocated within the browser should be + automatically cleaned up when the extension is disabled or uninstalled. + This is described in more detail in the section on :ref:`lifecycle`. + +- **Performance**: A new WebExtension API should not add any new overhead + to the browser when the API is not used. That is, the implementation + of the API should not be loaded at all unless it is actively used by + an extension. In addition, initialization should be delayed when + possible -- extensions ared started relatively early in the browser + startup process so any unnecessary work done during extension startup + contributes directly to sluggish browser startup. diff --git a/toolkit/components/extensions/docs/basics.rst b/toolkit/components/extensions/docs/basics.rst new file mode 100644 index 0000000000..35d61561e2 --- /dev/null +++ b/toolkit/components/extensions/docs/basics.rst @@ -0,0 +1,275 @@ +.. _basics: + +API Implementation Basics +========================= +This page describes some of the pieces involved when creating +WebExtensions APIs. Detailed documentation about how these pieces work +together to build specific features is in the next section. + +The API Schema +-------------- +As described previously, a WebExtension runs in a sandboxed environment +but the implementation of a WebExtensions API runs with full chrome +privileges. API implementations do not directly interact with +extensions' Javascript environments, that is handled by the WebExtensions +framework. Each API includes a schema that describes all the functions, +events, and other properties that the API might inject into an +extension's Javascript environment. +Among other things, the schema specifies the namespace into which +an API should be injected, what permissions (if any) are required to +use it, and in which contexts (e.g., extension pages, content scripts, etc) +it should be available. The WebExtensions framework reads this schema +and takes care of injecting the right objects into each extension +Javascript environment. + +API schemas are written in JSON and are based on +`JSON Schema `_ with some extensions to describe +API functions and events. +The next section describes the format of the schema in detail. + +The ExtensionAPI class +---------------------- +Every WebExtensions API is represented by an instance of the Javascript +`ExtensionAPI `_ class. +An instance of its API class is created every time an extension that has +access to the API is enabled. Instances of this class contain the +implementations of functions and events that are exposed to extensions, +and they also contain code for handling manifest keys as well as other +part of the extension lifecycle (e.g., updates, uninstalls, etc.) +The details of this class are covered in a subsequent section, for now the +important point is that this class contains all the actual code that +backs a particular WebExtensions API. + +Built-in versus Experimental APIs +--------------------------------- +A WebExtensions API can be built directly into the browser or it can be +contained in a special type of extension called a privileged extension +that defines a WebExtensions Experiment (i.e. experimental APIs). +The API schema and the ExtensionAPI class are written in the same way +regardless of how the API will be delivered, the rest of this section +explains how to package a new API using these methods. + +Adding a built-in API +--------------------- +Built-in WebExtensions APIs are loaded lazily. That is, the schema and +accompanying code are not actually loaded and interpreted until an +extension that uses the API is activated. +To actually register the API with the WebExtensions framework, an entry +must be added to the list of WebExtensions modules in one of the following +files: + +- ``toolkit/components/extensions/ext-toolkit.json`` +- ``browser/components/extensions/ext-browser.json`` +- ``mobile/android/components/extensions/ext-android.json`` + +Here is a sample fragment for a new API: + +.. code-block:: js + + "myapi": { + "schema": "chrome://extensions/content/schemas/myapi.json", + "url": "chrome://extensions/content/ext-myapi.js", + "paths": [ + ["myapi"], + ["anothernamespace", "subproperty"] + ], + "scopes": ["addon_parent"], + "permissions": ["myapi"], + "manifest": ["myapi_key"], + "events": ["update", "uninstall"] + } + +The ``schema`` and ``url`` properties are simply URLs for the API schema +and the code implementing the API. The ``chrome:`` URLs in the example above +are typically created by adding entries to ``jar.mn`` in the mozilla-central +directory where the API implementation is kept. The standard locations for +API implementations are: + +- ``toolkit/components/extensions``: This is where APIs that work in both + the desktop and mobile versions of Firefox (as well as potentially any + other applications built on Gecko) should go +- ``browser/components/extensions``: APIs that are only supported on + Firefox for the desktop. +- ``mobile/android/components/extensions``: APIs that are only supported + on Firefox for Android. + +Within the appropriate extensions directory, the convention is that the +API schema is in a file called ``schemas/name.json`` (where *name* is +the name of the API, typically the same as its namespace if it has +Javascript visible features). The code for the ExtensionAPI class is put +in a file called ``ext-name.js``. If the API has code that runs in a +child process, that is conventionally put in a file called ``ext-c-name.js``. + +The remaining properties specify when an API should be loaded. +The ``paths``, ``scopes``, and ``permissions`` properties together +cause an API to be loaded when Javascript code in an extension references +something beneath the ``browser`` global object that is part of the API. +The ``paths`` property is an array of paths where each individual path is +also an array of property names. In the example above, the sample API will +be loaded if an extension references either ``browser.myapi`` or +``browser.anothernamespace.subproperty``. + +A reference to a property beneath ``browser`` only causes the API to be +loaded if it occurs within a scope listed in the ``scopes`` property. +A scope corresponds to the combination of a Javascript environment +(e.g., extension pages, content scripts, etc) and the process in which the +API code should run (which is either the main/parent process, or a +content/child process). +Valid ``scopes`` are: + +- ``"addon_parent"``, ``"addon_child``: Extension pages + +- ``"content_parent"``, ``"content_child``: Content scripts + +- ``"devtools_parent"``, ``"devtools_child"``: Devtools pages + +The distinction between the ``_parent`` and ``_child`` scopes will be +explained in further detail in following sections. + +A reference to a property only causes the API to be loaded if the +extension referencing the property also has all the permissions listed +in the ``permissions`` property. + +A WebExtensions API that is controlled by a manifest key can also be loaded +when an extension that includes the relevant manifest key is activated. +This is specified by the ``manifest`` property, which lists any manifest keys +that should cause the API to be loaded. + +Finally, APIs can be loaded based on other events in the WebExtension +lifecycle. These are listed in the ``events`` property and described in +more detail in :ref:`lifecycle`. + +Adding Experimental APIs in Privileged Extensions +------------------------------------------------- + +A new API may also be implemented within a privileged extension. An API +implemented this way is called a WebExtensions Experiment (or simply an +Experimental API). Experiments can be useful when actively developing a +new API, as they do not require building Firefox locally. These extensions +may be installed temporarily via ``about:debugging`` or, on browser that +support it (current Nightly and Developer Edition), by setting the preference +``xpinstall.signatures.required`` to ``false``. You may also set the +preference ``extensions.experiments.enabled`` to ``true`` to install the +addon normally and test across restart. + +.. note:: + Out-of-tree privileged extensions cannot be signed by addons.mozilla.org. + A different pipeline is used to sign them with a privileged certificate. + You'll find more information in the `xpi-manifest repository on GitHub `_. + +Experimental APIs have a few limitations compared with built-in APIs: + +- Experimental APIs can (currently) only be exposed to extension pages, + not to devtools pages or to content scripts. +- Experimental APIs cannot handle manifest keys (since the extension manifest + needs to be parsed and validated before experimental APIs are loaded). +- Experimental APIs cannot use the static ``"update"`` and ``"uninstall"`` + lifecycle events (since in general those may occur when an affected + extension is not active or installed). + +Experimental APIs are declared in the ``experiment_apis`` property in a +WebExtension's ``manifest.json`` file. For example: + +.. code-block:: js + + { + "manifest_version": 2, + "name": "Extension containing an experimental API", + "experiment_apis": { + "apiname": { + "schema": "schema.json", + "parent": { + "scopes": ["addon_parent"], + "paths": [["myapi"]], + "script": "implementation.js" + }, + + "child": { + "scopes": ["addon_child"], + "paths": [["myapi"]], + "script": "child-implementation.js" + } + } + } + } + +This is essentially the same information required for built-in APIs, +just organized differently. The ``schema`` property is a relative path +to a file inside the extension containing the API schema. The actual +implementation details for the parent process and for child processes +are defined in the ``parent`` and ``child`` properties of the API +definition respectively. Inside these sections, the ``scope`` and ``paths`` +properties have the same meaning as those properties in the definition +of a built-in API (though see the note above about limitations; the +only currently valid values for ``scope`` are ``"addon_parent"`` and +``"addon_child"``). The ``script`` property is a relative path to a file +inside the extension containing the implementation of the API. + +The extension that includes an experiment defined in this way automatically +gets access to the experimental API. An extension may also use an +experimental API implemented in a different extension by including the +string ``experiments.name`` in the ``permissions``` property in its +``manifest.json`` file. In this case, the string name must be replace by +the name of the API from the extension that defined it (e.g., ``apiname`` +in the example above. + +Globals available in the API scripts global +------------------------------------------- + +The API scripts aren't loaded as an JSM and so: + +- they are not fully isolated from each other (and they are going to be + lazy loaded when the extension does use them for the first time) and + be executed in a per-process shared global scope) +- the experimental APIs embedded in privileged extensions are executed + in a per-extension global (separate from the one used for the built-in APIs) + +The global scope where the API scripts are executed is pre-populated with +some useful globals: + +- ``AppConstants`` +- ``console`` +- ``CC``, ``Ci``, ``Cr`` and ``Cu`` +- ``ChromeWorker`` +- ``extensions``, ``ExtensionAPI``, ``ExtensionCommon`` and ``ExtensionUtils`` +- ``global`` +- ``MatchGlob``, ``MatchPattern`` and ``MatchPatternSet`` +- ``Services`` +- ``StructuredCloneHolder`` +- ``XPCOMUtils`` + +For a more complete and updated list of the globals available by default in +all API scripts look to the following source: + +- `SchemaAPIManager _createExtGlobal method `_ +- Only available in the parent Firefox process: + `toolkit/components/extensions/parent/ext-toolkit.js `_ +- Only available in the child Firefox process: + `toolkit/components/extensions/child/ext-toolkit.js `_ +- Only available in the Desktop builds: + `browser/components/extensions/parent/ext-browser.js `_ +- Only available in the Android builds: + `mobile/android/components/extensions/ext-android.js `_ + +.. warning:: + The extension API authors should never redefine these globals to avoid introducing potential + conflicts between API scripts (e.g. see `Bug 1697404 comment 3 `_ + and `Bug 1697404 comment 4 `_). + +WebIDL Bindings +--------------- + +In ``manifest_version: 3`` the extension will be able to declare a background service worker +instead of a background page, and the existing WebExtensions API bindings can't be injected into this +new extension global, because it lives off the main thread. + +To expose WebExtensions API bindings to the WebExtensions ``background.service_worker`` global +we are in the process of generating new WebIDL bindings for the WebExtensions API. + +An high level view of the architecture and a more in depth details about the architecture process +to create or modify WebIDL bindings for the WebExtensions API can be found here: + +.. toctree:: + :maxdepth: 2 + + webidl_bindings diff --git a/toolkit/components/extensions/docs/events.rst b/toolkit/components/extensions/docs/events.rst new file mode 100644 index 0000000000..d494155ffc --- /dev/null +++ b/toolkit/components/extensions/docs/events.rst @@ -0,0 +1,609 @@ +Implementing an event +===================== +Like a function, an event requires a definition in the schema and +an implementation in Javascript inside an instance of ExtensionAPI. + +Declaring an event in the API schema +------------------------------------ +The definition for a simple event looks like this: + +.. code-block:: json + + [ + { + "namespace": "myapi", + "events": [ + { + "name": "onSomething", + "type": "function", + "description": "Description of the event", + "parameters": [ + { + "name": "param1", + "description": "Description of the first callback parameter", + "type": "number" + } + ] + } + ] + } + ] + +This fragment defines an event that is used from an extension with +code such as: + +.. code-block:: js + + browser.myapi.onSomething.addListener(param1 => { + console.log(`Something happened: ${param1}`); + }); + +Note that the schema syntax looks similar to that for a function, +but for an event, the ``parameters`` property specifies the arguments +that will be passed to a listener. + +Implementing an event +--------------------- +Just like with functions, defining an event in the schema causes +wrappers to be automatically created and exposed to an extensions' +appropriate Javascript contexts. +An event appears to an extension as an object with three standard +function properties: ``addListener()``, ``removeListener()``, +and ``hasListener()``. +Also like functions, if an API defines an event but does not implement +it in a child process, the wrapper in the child process effectively +proxies these calls to the implementation in the main process. + +A helper class called +`EventManager `_ makes implementing +events relatively simple. A simple event implementation looks like: + +.. code-block:: js + + this.myapi = class extends ExtensionAPI { + getAPI(context) { + return { + myapi: { + onSomething: new EventManager({ + context, + name: "myapi.onSomething", + register: fire => { + const callback = value => { + fire.async(value); + }; + RegisterSomeInternalCallback(callback); + return () => { + UnregisterInternalCallback(callback); + }; + } + }).api(), + } + } + } + } + +The ``EventManager`` class is usually just used directly as in this example. +The first argument to the constructor is an ``ExtensionContext`` instance, +typically just the object passed to the API's ``getAPI()`` function. +The second argument is a name, it is used only for debugging. +The third argument is the important piece, it is a function that is called +the first time a listener is added for this event. +This function is passed an object (``fire`` in the example) that is used to +invoke the extension's listener whenever the event occurs. The ``fire`` +object has several different methods for invoking listeners, but for +events implemented in the main process, the only valid method is +``async()`` which executes the listener asynchronously. + +The event setup function (the function passed to the ``EventManager`` +constructor) must return a cleanup function, +which will be called when the listener is removed either explicitly +by the extension by calling ``removeListener()`` or implicitly when +the extension Javascript context from which the listener was added is destroyed. + +In this example, ``RegisterSomeInternalCallback()`` and +``UnregisterInternalCallback()`` represent methods for listening for +some internal browser event from chrome privileged code. This is +typically something like adding an observer using ``Services.obs`` or +attaching a listener to an ``EventEmitter``. + +After constructing an instance of ``EventManager``, its ``api()`` method +returns an object with with ``addListener()``, ``removeListener()``, and +``hasListener()`` methods. This is the standard extension event interface, +this object is suitable for returning from the extension's +``getAPI()`` method as in the example above. + +Handling extra arguments to addListener() +----------------------------------------- +The standard ``addListener()`` method for events may accept optional +addition parameters to allow extra information to be passed when registering +an event listener. One common application of this parameter is for filtering, +so that extensions that only care about a small subset of the instances of +some event can avoid the overhead of receiving the ones they don't care about. + +Extra parameters to ``addListener()`` are defined in the schema with the +the ``extraParameters`` property. For example: + +.. code-block:: json + + [ + { + "namespace": "myapi", + "events": [ + { + "name": "onSomething", + "type": "function", + "description": "Description of the event", + "parameters": [ + { + "name": "param1", + "description": "Description of the first callback parameter", + "type": "number" + } + ], + "extraParameters": [ + { + "name": "minValue", + "description": "Only call the listener for values of param1 at least as large as this value.", + "type": "number" + } + ] + } + ] + } + ] + +Extra parameters defined in this way are passed to the event setup +function (the last parameter to the ``EventManager`` constructor. +For example, extending our example above: + +.. code-block:: js + + this.myapi = class extends ExtensionAPI { + getAPI(context) { + return { + myapi: { + onSomething: new EventManager({ + context, + module: "myapi", + event: "onSomething", + register: (fire, minValue) => { + const callback = value => { + if (value >= minValue) { + fire.async(value); + } + }; + RegisterSomeInternalCallback(callback); + return () => { + UnregisterInternalCallback(callback); + }; + } + }).api() + } + } + } + } + +Handling listener return values +------------------------------- +Some event APIs allow extensions to affect event handling in some way +by returning values from event listeners that are processed by the API. +This can be defined in the schema with the ``returns`` property: + +.. code-block:: json + + [ + { + "namespace": "myapi", + "events": [ + { + "name": "onSomething", + "type": "function", + "description": "Description of the event", + "parameters": [ + { + "name": "param1", + "description": "Description of the first callback parameter", + "type": "number" + } + ], + "returns": { + "type": "string", + "description": "Description of how the listener return value is processed." + } + } + ] + } + ] + +And the implementation of the event uses the return value from ``fire.async()`` +which is a Promise that resolves to the listener's return value: + +.. code-block:: js + + this.myapi = class extends ExtensionAPI { + getAPI(context) { + return { + myapi: { + onSomething: new EventManager({ + context, + module: "myapi", + event: "onSomething", + register: fire => { + const callback = async (value) => { + let rv = await fire.async(value); + log(`The onSomething listener returned the string ${rv}`); + }; + RegisterSomeInternalCallback(callback); + return () => { + UnregisterInternalCallback(callback); + }; + } + }).api() + } + } + } + } + +Note that the schema ``returns`` definition is optional and serves only +for documentation. That is, ``fire.async()`` always returns a Promise +that resolves to the listener return value, the implementation of an +event can just ignore this Promise if it doesn't care about the return value. + +Implementing an event in the child process +------------------------------------------ +The reasons for implementing events in the child process are similar to +the reasons for implementing functions in the child process: + +- Listeners for the event return a value that the API implementation must + act on synchronously. + +- Either ``addListener()`` or the listener function has one or more + parameters of a type that cannot be sent between processes. + +- The implementation of the event interacts with code that is only + accessible from a child process. + +- The event can be implemented substantially more efficiently in a + child process. + +The process for implementing an event in the child process is the same +as for functions -- simply implement the event in an ExtensionAPI subclass +that is loaded in a child process. And just as a function in a child +process can call a function in the main process with +`callParentAsyncFunction()`, events in a child process may subscribe to +events implemented in the main process with a similar `getParentEvent()`. +For example, the automatically generated event proxy in a child process +could be written explicitly as: + +.. code-block:: js + + this.myapi = class extends ExtensionAPI { + getAPI(context) { + return { + myapi: { + onSomething: new EventManager( + context, + name: "myapi.onSomething", + register: fire => { + const listener = (value) => { + fire.async(value); + }; + + let parentEvent = context.childManager.getParentEvent("myapi.onSomething"); + parent.addListener(listener); + return () => { + parent.removeListener(listener); + }; + } + }).api() + } + } + } + } + +Events implemented in a child process have some additional methods available +to dispatch listeners: + +- ``fire.sync()`` This runs the listener synchronously and returns the + value returned by the listener + +- ``fire.raw()`` This runs the listener synchronously without cloning + the listener arguments into the extension's Javascript compartment. + This is used as a performance optimization, it should not be used + unless you have a detailed understanding of Javascript compartments + and cross-compartment wrappers. + +Event Listeners Persistence +--------------------------- + +Event listeners are persisted in some circumstances. Persisted event listeners can either +block startup, and/or cause an Event Page or Background Service Worker to be started. + +The event listener must be registered synchronously in the top level scope +of the background. Event listeners registered later, or asynchronously, are +not persisted. + +Currently only WebRequestBlocking and Proxy events are able to block +at startup, causing an addon to start earlier in Firefox startup. Whether +a module can block startup is defined by a ``startupBlocking`` flag in +the module definition files (``ext-toolkit.json`` or ``ext-browser.json``). +As well, these are the only events persisted for persistent background scripts. + +Events implemented only in a child process, without a parent process counterpart, +cannot be persisted. + +Persisted and Primed Event Listeners +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +In terms of terminology: + +- **Persisted Event Listener** is the set of data (in particular API module, API event name + and the parameters passed along with addListener call if any) related to an event listener + that has been registered by an Event Page (or Background Service Worker) in a previous run + and being stored in the StartupCache data + +- **Primed Event Listener** is a "placeholder" event listener created, from the **Persisted Event Listener** + data found in the StartupCache, while the Event Page (or Background Service Worker) is not running + (either not started yet or suspended after the idle timeout was hit) + +ExtensionAPIPersistent and PERSISTENT_EVENTS +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Most of the WebExtensions APIs promise some API events, and it is likely that most of those events are also +expected to be waking up the Event Page (or Background Service Worker) when emitted while the background +extension context has not been started yet (or it was suspended after the idle timeout was hit). + +As part of implementing a WebExtensions API that is meant to persist all or some of its API event +listeners: + +- the WebExtensions API namespace class should extend ``ExtensionAPIPersistent`` (instead of extending + the ``ExtensionAPI`` class) + +- the WebExtensions API namespace should have a ``PERSISTENT_EVENTS`` property, which is expected to be + set to an object defining methods. Each method should be named after the related API event name, which + are going to be called internally: + + - while the extension Event Page (or Background Service Worker) isn't running (either never started yet + or suspended after the idle timeout). These methods are called by the WebExtensions internals to + create placeholder API event listeners in the parent process for each of the API event listeners + persisted for that extension. These placeholder listeners are internally referred to as + ``primed listeners``). + + - while the extension Event Page (or Background Service Worker) is running (as well as for any other + extension context types they may have been created for the extension). These methods are called by the + WebExtensions internals to create the parent process callback that will be responsible for + forwarding the API events to the extension callbacks in the child processes. + +- in the ``getAPI`` method. For all the API namespace properties that represent API events returned by this method, + the ``EventManager`` instances created for each of the API events that is expected to persist its listeners + should include following options: + + - ``module``, to be set to the API module name as listed in ``"ext-toolkit.json"`` / ``"ext-browser.json"`` + / ``"ext-android.json"`` (which, in most cases, is the same as the API namespace name string). + - ``event``, to be set to the API event name string. + - ``extensionApi``, to be set to the ``ExtensionAPIPersistent`` class instance. + +Taking a look to some of the patches landed to introduce API event listener persistency on some of the existing +API as part of introducing support for the Event Page may also be useful: + +- Bug-1748546_ ported the browserAction and pageAction API namespace implementations to + ``ExtensionAPIPersistent`` and, in particular, the changes applied to: + + - ext-browserAction.js: https://hg.mozilla.org/integration/autoland/rev/08a3eaa8bce7 + - ext-pageAction.js: https://hg.mozilla.org/integration/autoland/rev/ed616e2e0abb + +.. _Bug-1748546: https://bugzilla.mozilla.org/show_bug.cgi?id=1748546 + +Follows an example of what has been described previously in a code snippet form: + +.. code-block:: js + + this.myApiName = class extends ExtensionAPIPersistent { + PERSISTENT_EVENTS = { + // @param {object} options + // @param {object} options.fire + // @param {function} options.fire.async + // @param {function} options.fire.sync + // @param {function} options.fire.raw + // For primed listeners `fire.async`/`fire.sync`/`fire.raw` will + // collect the pending events to be send to the background context + // and implicitly wake up the background context (Event Page or + // Background Service Worker), or forward the event right away if + // the background context is running. + // @param {function} [options.fire.wakeup = undefined] + // For primed listeners, the `fire` object also provide a `wakeup` method + // which can be used by the primed listener to explicitly `wakeup` the + // background context (Event Page or Background Service Worker) and wait for + // it to be running (by awaiting on the Promise returned by wakeup to be + // resolved). + // @param {ProxyContextParent} [options.context=undefined] + // This property is expected to be undefined for primed listeners (which + // are created while the background extension context does not exist) and + // to be set to a ProxyContextParent instance (the same got by the getAPI + // method) when the method is called for a listener registered by a + // running extension context. + // + // @param {object} [apiEventsParams=undefined] + // The additional addListener parameter if any (some API events are allowing + // the extensions to pass some parameters along with the extension callback). + onMyEventName({ context, fire }, apiEventParams = undefined) { + const listener = (...) { + // Wake up the EventPage (or Background ServiceWorker). + if (fire.wakeup) { + await fire.wakeup(); + } + + fire.async(...); + } + + // Subscribe a listener to an internal observer or event which will be notified + // when we need to call fire to either send the event to an extension context + // already running or wake up a suspended event page and accumulate the events + // to be fired once the extension context is running again and a callback registered + // back (which will be used to convert the primed listener created while + // the non persistent background extension context was not running yet) + ... + return { + unregister() { + // Unsubscribe a listener from an internal observer or event. + ... + } + convert(fireToExtensionCallback) { + // Convert gets called once the primed API event listener, + // created while the extension background context has been + // suspended, is being converted to a parent process API + // event listener callback that is responsible for forwarding the + // events to the child processes. + // + // The `fireToExtensionCallback` parameter is going to be the + // one that will emit the event to the extension callback (while + // the one got from the API event registrar method may be the one + // that is collecting the events to emit up until the background + // context got started up again). + fire = fireToExtensionCallback; + }, + }; + }, + ... + }; + + getAPI(context) { + ... + return { + myAPIName: { + ... + onMyEventName: new EventManager({ + context, + // NOTE: module is expected to be the API module name as listed in + // ext-toolkit.json / ext-browser.json / ext-android.json. + module: "myAPIName", + event: "onMyEventNAme", + extensionApi: this, + }), + }, + }; + } + }; + +Testing Persisted API Event Listeners +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +- ``extension.terminateBackground()`` / ``extension.terminateBackground({ disableResetIdleForTest: true})``: + + - The wrapper object returned by ``ExtensionTestUtils.loadExtension`` provides + a ``terminateBackground`` method which can be used to simulate an idle timeout, + by explicitly triggering the same idle timeout suspend logic handling the idle timeout. + - This method also accept an optional parameter, if set to ``{ disableResetIdleForTest: true}`` + will forcefully suspend the background extension context and ignore all the + conditions that would reset the idle timeout due to some work still pending + (e.g. a ``NativeMessaging``'s ``Port`` still open, a ``StreamFilter`` instance + still active or a ``Promise`` from an API event listener call not yet resolved) + +- ``ExtensionTestUtils.testAssertions.assertPersistentListeners``: + + - This test assertion helper can be used to more easily assert what should + be the persisted state of a given API event (e.g. assert it to not be + persisted, or to be persisted and/or primed) + +.. code-block:: js + + assertPersistentListeners(extension, "browserAction", "onClicked", { + primed: false, + }); + await extension.terminateBackground(); + assertPersistentListeners(extension, "browserAction", "onClicked", { + primed: true, + }); + +- ``extensions.background.idle.timeout`` preference determines how long to wait + (between API events being notified to the extension event page) before considering + the Event Page in the idle state and suspend it, in some xpcshell test this pref + may be set to 0 to reduce the amount of time the test will have to wait for the + Event Page to be suspended automatically + +- ``extension.eventPage.enabled`` pref is responsible for enabling/disabling + Event Page support for manifest_version 2 extension, technically it is + now set to ``true`` on all channels, but it would still be worth flipping it + to ``true`` explicitly in tests that are meant to cover Event Page behaviors + for manifest_version 2 test extension until the pref is completely removed + (mainly to make sure that if the pref would need to be flipped to false + for any reason, the tests will still be passing) + +Persisted Event listeners internals +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The ``ExtensionAPIPersistent`` class provides a way to quickly introduce API event +listener persistency to a new WebExtensions API, and reduce the number of code +duplication, the following section provide some more details about what the +abstractions are doing internally in practice. + +WebExtensions APIs classes that extend the ``ExtensionAPIPersistent`` base class +are still able to support non persisted listeners along with persisted ones +(e.g. events that are persisting the listeners registered from an event page are +already not persisting listeners registered from other extension contexts) +and can mix persisted and non-persisted events. + +As an example in ``toolkit/components/extensions/parent/ext-runtime.js``` the two +events ``onSuspend`` and ``onSuspendCanceled`` are expected to be never persisted +nor primed (even for an event page) and so their ``EventManager`` instances +receive the following options: + +- a ``register`` callback (instead of the one part of ``PERSISTED_EVENTS``) +- a ``name`` string property (instead of the two separate ``module`` and ``event`` + string properties that are used for ``EventManager`` instances from persisted + ones +- no ``extensionApi`` property (because that is only needed for events that are + expected to persist event page listeners) + +In practice ``ExtensionAPIPersistent`` extends the ``ExtensionAPI`` class to provide +a generic ``primeListeners`` method, which is the method responsible for priming +a persisted listener when the event page has been suspended or not started yet. + +The ``primeListener`` method is expected to return an object with an ``unregister`` +and ``convert`` method, while the ``register`` callback passed to the ``EventManager`` +constructor is expected to return the ``unregister`` method. + +.. code-block:: js + + function somethingListener(fire, minValue) => { + const callback = value => { + if (value >= minValue) { + fire.async(value); + } + }; + RegisterSomeInternalCallback(callback); + return { + unregister() { + UnregisterInternalCallback(callback); + }, + convert(_fire, context) { + fire = _fire; + } + }; + } + + this.myapi = class extends ExtensionAPI { + primeListener(extension, event, fire, params, isInStartup) { + if (event == "onSomething") { + // Note that we return the object with unregister and convert here. + return somethingListener(fire, ...params); + } + // If an event other than onSomething was requested, we are not returning + // anything for it, thus it would not be persistable. + } + getAPI(context) { + return { + myapi: { + onSomething: new EventManager({ + context, + module: "myapi", + event: "onSomething", + register: (fire, minValue) => { + // Note that we return unregister here. + return somethingListener(fire, minValue).unregister; + } + }).api() + } + } + } + } diff --git a/toolkit/components/extensions/docs/functions.rst b/toolkit/components/extensions/docs/functions.rst new file mode 100644 index 0000000000..f1727aceed --- /dev/null +++ b/toolkit/components/extensions/docs/functions.rst @@ -0,0 +1,201 @@ +Implementing a function +======================= +Implementing an API function requires at least two different pieces: +a definition for the function in the schema, and Javascript code that +actually implements the function. + +Declaring a function in the API schema +-------------------------------------- +An API schema definition for a simple function looks like this: + +.. code-block:: json + + [ + { + "namespace": "myapi", + "functions": [ + { + "name": "add", + "type": "function", + "description": "Adds two numbers together.", + "async": true, + "parameters": [ + { + "name": "x", + "type": "number", + "description": "The first number to add." + }, + { + "name": "y", + "type": "number", + "description": "The second number to add." + } + ] + } + ] + } + ] + +The ``type`` and ``description`` properties were described above. +The ``name`` property is the name of the function as it appears in +the given namespace. That is, the fragment above creates a function +callable from an extension as ``browser.myapi.add()``. +The ``parameters`` property describes the parameters the function takes. +Parameters are specified as an array of Javascript types, where each +parameter is a constrained Javascript value as described +in the previous section. + +Each parameter may also contain additional properties ``optional`` +and ``default``. If ``optional`` is present it must be a boolean +(and parameters are not optional by default so this property is typically +only added when it has the value ``true``). +The ``default`` property is only meaningful for optional parameters, +it specifies the value that should be used for an optional parameter +if the function is called without that parameter. +An optional parameter without an explicit ``default`` property will +receive a default value of ``null``. +Although it is legal to create optional parameters at any position +(i.e., optional parameters can come before required parameters), doing so +leads to difficult to use functions and API designers are encouraged to +use object-valued parameters with optional named properties instead, +or if optional parameters must be used, to use them sparingly and put +them at the end of the parameter list. + +.. XXX should we describe allowAmbiguousArguments? + +The boolean-valued ``async`` property specifies whether a function +is asynchronous. +For asynchronous functions, +the WebExtensions framework takes care of automatically generating a +`Promise `_ and then resolving the Promise when the function +implementation completes (or rejecting the Promise if the implementation +throws an Error). +Since extensions can run in a child process, any API function that is +implemented (either partially or completely) in the parent process must +be asynchronous. + +When a function is declared in the API schema, a wrapper for the function +is automatically created and injected into appropriate extension Javascript +contexts. This wrapper automatically validates arguments passed to the +function against the formal parameters declared in the schema and immediately +throws an Error if invalid arguments are passed. +It also processes optional arguments and inserts default values as needed. +As a result, API implementations generally do not need to write much +boilerplate code to validate and interpret arguments. + +Implementing a function in the main process +------------------------------------------- +If an asynchronous function is not implemented in the child process, +the wrapper generated from the schema automatically marshalls the +function arguments, sends the request to the parent process, +and calls the implementation there. +When that function completes, the return value is sent back to the child process +and the Promise for the function call is resolved with that value. + +Based on this, an implementation of the function we wrote the schema +for above looks like this: + +.. code-block:: js + + this.myapi = class extends ExtensionAPI { + getAPI(context) { + return { + myapi: { + add(x, y) { return x+y; } + } + } + } + } + +The implementations of API functions are contained in a subclass of the +`ExtensionAPI `_ class. +Each subclass of ExtensionAPI must implement the ``getAPI()`` method +which returns an object with a structure that mirrors the structure of +functions and events that the API exposes. +The ``context`` object passed to ``getAPI()`` is an instance of +`BaseContext `_, +which contains a number of useful properties and methods. + +If an API function implementation returns a Promise, its result will +be sent back to the child process when the Promise is settled. +Any other return type will be sent directly back to the child process. +A function implementation may also raise an Error. But by default, +an Error thrown from inside an API implementation function is not +exposed to the extension code that called the function -- it is +converted into generic errors with the message "An unexpected error occurred". +To throw a specific error to extensions, use the ``ExtensionError`` class: + +.. code-block:: js + + this.myapi = class extends ExtensionAPI { + getAPI(context) { + return { + myapi: { + doSomething() { + if (cantDoSomething) { + throw new ExtensionError("Cannot call doSomething at this time"); + } + return something(); + } + } + } + } + } + +The purpose of this step is to avoid bugs in API implementations from +exposing details about the implementation to extensions. When an Error +that is not an instance of ExtensionError is thrown, the original error +is logged to the +`Browser Console `_, +which can be useful while developing a new API. + +Implementing a function in a child process +------------------------------------------ +Most functions are implemented in the main process, but there are +occasionally reasons to implement a function in a child process, such as: + +- The function has one or more parameters of a type that cannot be automatically + sent to the main process using the structured clone algorithm. + +- The function implementation interacts with some part of the browser + internals that is only accessible from a child process. + +- The function can be implemented substantially more efficiently in + a child process. + +To implement a function in a child process, simply include an ExtensionAPI +subclass that is loaded in the appropriate context +(e.g, ``addon_child``, ``content_child``, etc.) as described in +the section on :ref:`basics`. +Code inside an ExtensionAPI subclass in a child process may call the +implementation of a function in the parent process using a method from +the API context as follows: + +.. code-block:: js + + this.myapi = class extends ExtensionAPI { + getAPI(context) { + return { + myapi: { + async doSomething(arg) { + let result = await context.childManager.callParentAsyncFunction("anothernamespace.functionname", [arg]); + /* do something with result */ + return ...; + } + } + } + } + } + +As you might expect, ``callParentAsyncFunction()`` calls the given function +in the main process with the given arguments, and returns a Promise +that resolves with the result of the function. +This is the same mechanism that is used by the automatically generated +function wrappers for asynchronous functions that do not have a +provided implementation in a child process. + +It is possible to define the same function in both the main process +and a child process and have the implementation in the child process +call the function with the same name in the parent process. +This is a common pattern when the implementation of a particular function +requires some code in both the main process and child process. diff --git a/toolkit/components/extensions/docs/generate_webidl_from_jsonschema.rst b/toolkit/components/extensions/docs/generate_webidl_from_jsonschema.rst new file mode 100644 index 0000000000..ea29d6b7b6 --- /dev/null +++ b/toolkit/components/extensions/docs/generate_webidl_from_jsonschema.rst @@ -0,0 +1,94 @@ +Generating WebIDL definitions from WebExtensions API JSONSchema +=============================================================== + +In ``toolkit/components/extensions/webidl-api``, a python script named ``GenerateWebIDLBindings.py`` +helps to generation of the WebIDL definitions for the WebExtensions API namespaces based on the existing +JSONSchema data. + +.. figure:: generate_webidl_from_jsonschema_dataflow.drawio.svg + :alt: Diagram of the GenerateWebIDLBindings.py script data flow + +.. + This svg diagram has been created using https://app.diagrams.net, + the svg file also includes the source in the drawio format and so + it can be edited more easily by loading it back into app.diagrams.net + and then re-export from there (and include the updated drawio format + content into the exported svg file). + +Example: how to execute GenerateWebIDLBindings.py +------------------------------------------------- + +As an example, the following shell command generates (or regenerates if one exists) the webidl bindings +for the `runtime` API namespace: + +.. code-block:: sh + + $ export SCRIPT_DIR="toolkit/components/extensions/webidl-api" + $ mach python $SCRIPT_DIR/GenerateWebIDLBindings.py -- runtime + +this command will generates a `.webdil` file named `dom/webidl/ExtensionRuntime.webidl`. + +.. warning:: + This python script uses some python libraries part of mozilla-central ``mach`` command + and so it has to be executed using ``mach python`` and any command line options that has + to the passed to the ``GenerateWebIDLBindings.py`` script should be passed after the ``--`` + one that ends ``mach python`` own command line options. + +* If a webidl file with the same name already exist, the python script will ask confirmation and + offer to print a diff of the changes (or just continue without changing the existing webidl file + if the content is exactly the same): + +.. code-block:: + + $ mach python $SCRIPT_DIR/GenerateWebIDLBindings.py -- runtime + + Generating webidl definition for 'runtime' => dom/webidl/ExtensionRuntime.webidl + Found existing dom/webidl/ExtensionRuntime.webidl. + + (Run again with --overwrite-existing to allow overwriting it automatically) + + Overwrite dom/webidl/ExtensionRuntime.webidl? (Yes/No/Diff) + D + --- ExtensionRuntime.webidl--existing + +++ ExtensionRuntime.webidl--updated + @@ -24,6 +24,9 @@ + [Exposed=(ServiceWorker), LegacyNoInterfaceObject] + interface ExtensionRuntime { + // API methods. + + + + [Throws, WebExtensionStub="Async"] + + any myNewMethod(boolean aBoolParam, optional Function callback); + + [Throws, WebExtensionStub="Async"] + any openOptionsPage(optional Function callback); + + + Overwrite dom/webidl/ExtensionRuntime.webidl? (Yes/No/Diff) + +* By convention each WebExtensions API WebIDL binding is expected to be paired with C++ files + named ``ExtensionMyNamespace.h`` and ``ExtensionMyNamespace.cpp`` and located in + ``toolkit/components/extensions/webidl-api``: + + * if no files with the expected names is found the python script will generate an initial + boilerplate files and will store them in the expected mozilla-central directory. + * The Firefox developers are responsible to fill this initial boilerplate as needed and + to apply the necessary changes (if any) when the webidl definitions are updated because + of changes to the WebExtensions APIs JSONSchema. + +``ExtensionWebIDL.conf`` config file +------------------------------------ + +TODO: + +* mention the role of the "webidl generation" script config file in handling + special cases (e.g. mapping types and method stubs) + +* notes on desktop-only APIs and API namespaces only partially available on Android + + +``WebExtensionStub`` WebIDL extended attribute +---------------------------------------------- + +TODO: + +* mention the special webidl extended attribute used in the WebIDL definitions diff --git a/toolkit/components/extensions/docs/generate_webidl_from_jsonschema_dataflow.drawio.svg b/toolkit/components/extensions/docs/generate_webidl_from_jsonschema_dataflow.drawio.svg new file mode 100644 index 0000000000..aaa5a4c3e0 --- /dev/null +++ b/toolkit/components/extensions/docs/generate_webidl_from_jsonschema_dataflow.drawio.svg @@ -0,0 +1,4 @@ + + + +
generate initial
boilerplate
generate initial...
Script config file
Script config file
ExtensionWebIDL.conf
ExtensionWebIDL.conf
toolkit/components/extensions/webidl-api/
GenerateWebIDLBindings.py
toolkit/components/extensions/webidl-api/...
generate / update
generate / update
ExtensionMyAPI.webidl
ExtensionMyAPI.webidl
dom/webidl
dom/webidl
Toolkit-level schema
files
Toolkit-level schema...
WebExtension JSONSChema files
WebExtension JSONSChema files
Desktop-level schema
files
Desktop-level schema...
Mobile-level schema
files
Mobile-level schema...
WebExtensions API
namespace name
WebExtensions API...
Script CLI options
Script CLI options
ExtensionAPI.webidl.in
ExtensionAPI.webidl.in
ExtensionAPI.[cpp|h].in
ExtensionAPI.[cpp|h].in
Jinja-based templates
Jinja-based templates
ExtensionMyAPI.h/cpp
ExtensionMyAPI.h/cpp
toolkit/components/extensions/webidl-api
toolkit/components/extensi...
  • mapping JSONSchema to WebIDL types
  • mapping API method to webidl WebExtensionStub extended attribute
  • mapping schema group (toolkit, desktop, mobile) to mozilla-central dir paths
mapping JSONSchema to WebIDL types...Viewer does not support full SVG 1.1 \ No newline at end of file diff --git a/toolkit/components/extensions/docs/incognito.rst b/toolkit/components/extensions/docs/incognito.rst new file mode 100644 index 0000000000..7df71e77c4 --- /dev/null +++ b/toolkit/components/extensions/docs/incognito.rst @@ -0,0 +1,78 @@ +.. _incognito: + +Incognito Implementation +======================== + +This page provides a high level overview of how incognito works in +Firefox, primarily to help in understanding how to test the feature. + +The Implementation +------------------ + +The incognito value in manifest.json supports ``spanning`` and ``not_allowed``. +The other value, ``split``, may be supported in the future. The default +value is ``spanning``, however, by default access to private windows is +not allowed. The user must turn on support, per extension, in ``about:addons``. + +Internally this is handled as a hidden extension permission called +``internal:privateBrowsingAllowed``. This permission is reset when the +extension is disabled or uninstalled. The permission is accessible in +several ways: + +- extension.privateBrowsingAllowed +- context.privateBrowsingAllowed (see BaseContext) +- WebExtensionPolicy.privateBrowsingAllowed +- WebExtensionPolicy.canAccessWindow(DOMWindow) + +Testing +------- + +The goal of testing is to ensure that data from a private browsing session +is not accessible to an extension without permission. + +In Firefox 67, the feature will initially be disabled, however the +intention is to enable the feature on in 67. The pref controlling this +is ``extensions.allowPrivateBrowsingByDefault``. When this pref is +``true``, all extensions have access to private browsing and the manifest +value ``not_allowed`` will produce an error. To enable incognito.not_allowed +for tests you must flip the pref to false. + +Testing EventManager events +--------------------------- + +This is typically most easily handled by running a test with an extension +that has permission, using ``incognitoOverride: spanning`` in the call to +ExtensionTestUtils.loadExtension. You can then use a second extension +without permission to try and catch any events that would typically be passed. + +If the events can happen without calls produced by an extension, you can +also use BrowserTestUtils to open a private window, and use a non-permissioned +extension to run tests against it. + +There are two utility functions in head.js, getIncognitoWindow and +startIncognitoMonitorExtension, which are useful for some basic testing. + +Example: `browser_ext_windows_events.js `_ + +Testing API Calls +----------------- + +This is easily done using an extension without permission. If you need +an ID of a window or tab, use getIncognitoWindow. In most cases, the +API call should throw an exception when the window is not accessible. +There are some cases where API calls explicitly do not throw. + +Example: `browser_ext_windows_incognito.js `_ + +Privateness of window vs. tab +----------------------------- + +Android does not currently support private windows. When a tab is available, +the test should prefer tab over window. + +- PrivateBrowsingUtils.isBrowserPrivate(tab.linkedBrowser) +- PrivateBrowsingUtils.isContentWindowPrivate(window) + +When WebExtensionPolicy is handy to use, you can directly check window access: + +- policy.canAccessWindow(window) diff --git a/toolkit/components/extensions/docs/index.rst b/toolkit/components/extensions/docs/index.rst new file mode 100644 index 0000000000..63a7c3685c --- /dev/null +++ b/toolkit/components/extensions/docs/index.rst @@ -0,0 +1,33 @@ +WebExtensions API Development +============================= + +This documentation covers the implementation of WebExtensions inside Firefox. +Documentation about existing WebExtension APIs and how to use them +to develop WebExtensions is available +`on MDN `_. + +To use this documentation, you should already be familiar with +WebExtensions, including +`the anatomy of a WebExtension `_ +and `permissions `_. +You should also be familiar with concepts from +`Firefox development `_ +including `e10s `_ +in particular. + +.. toctree:: + :caption: WebExtension API Developers Guide + :maxdepth: 2 + + background + basics + schema + functions + events + manifest + lifecycle + incognito + webidl_bindings + webext-storage + other + reference diff --git a/toolkit/components/extensions/docs/lifecycle.rst b/toolkit/components/extensions/docs/lifecycle.rst new file mode 100644 index 0000000000..8f08b34b68 --- /dev/null +++ b/toolkit/components/extensions/docs/lifecycle.rst @@ -0,0 +1,60 @@ +.. _lifecycle: + +Managing the Extension Lifecycle +================================ +The techniques described in previous pages allow a WebExtension API to +be loaded and instantiated only when an extension that uses the API is +activated. +But there are a few other events in the extension lifecycle that an API +may need to respond to. + +Extension Shutdown +------------------ +APIs that allocate any resources (e.g., adding elements to the browser's +user interface, setting up internal event listeners, etc.) must free +these resources when the extension for which they are allocated is +shut down. An API does this by using the ``callOnClose()`` +method on an `Extension `_ object. + +Extension Uninstall and Update +------------------------------ +In addition to resources allocated within an individual browser session, +some APIs make durable changes such as setting preferences or storing +data in the user's profile. +These changes are typically not reverted when an extension is shut down, +but when the extension is completely uninstalled (or stops using the API). +To handle this, extensions can be notified when an extension is uninstalled +or updated. Extension updates are a subtle case -- consider an API that +makes some durable change based on the presence of a manifest property. +If an extension uses the manifest key in one version and then is updated +to a new version that no longer uses the manifest key, +the ``onManifestEntry()`` method for the API is no longer called, +but an API can examine the new manifest after an update to detect that +the key has been removed. + +Handling lifecycle events +------------------------- + +To be notified of update and uninstall events, an extension lists these +events in the API manifest: + +.. code-block:: js + + "myapi": { + "schema": "...", + "url": "...", + "events": ["update", "uninstall"] + } + +If these properties are present, the ``onUpdate()`` and ``onUninstall()`` +methods will be called for the relevant ``ExtensionAPI`` instances when +an extension that uses the API is updated or uninstalled. + +Note that these events can be triggered on extensions that are inactive. +For that reason, these events can only be handled by extension APIs that +are built into the browser. Or, in other words, these events cannot be +handled by APIs that are implemented in WebExtension experiments. If the +implementation of an API relies on these events for correctness, the API +must be built into the browser and not delivered via an experiment. + +.. Should we even document onStartup()? I think no... diff --git a/toolkit/components/extensions/docs/manifest.rst b/toolkit/components/extensions/docs/manifest.rst new file mode 100644 index 0000000000..194dc43a8d --- /dev/null +++ b/toolkit/components/extensions/docs/manifest.rst @@ -0,0 +1,68 @@ +Implementing a manifest property +================================ +Like functions and events, implementing a new manifest key requires +writing a definition in the schema and extending the API's instance +of ``ExtensionAPI``. + +The contents of a WebExtension's ``manifest.json`` are validated using +a type called ``WebExtensionManifest`` defined in the namespace +``manifest``. +The first step when adding a new property is to extend the schema so +that manifests containing the new property pass validation. +This is done with the ``"$extend"`` property as follows: + +.. code-block:: js + + [ + "namespace": "manifest", + "types": [ + { + "$extend": "WebExtensionManifest", + "properties": { + "my_api_property": { + "type": "string", + "optional": true, + ... + } + } + } + ] + ] + +The next step is to inform the WebExtensions framework that this API +should be instantiated and notified when extensions that use the new +manifest key are loaded. +For built-in APIs, this is done with the ``manifest`` property +in the API manifest (e.g., ``ext-toolkit.json``). +Note that this property is an array so an extension can implement +multiple properties: + +.. code-block:: js + + "myapi": { + "schema": "...", + "url": "...", + "manifest": ["my_api_property"] + } + +The final step is to write code to handle the new manifest entry. +The WebExtensions framework processes an extension's manifest when the +extension starts up, this happens for existing extensions when a new +browser session starts up and it can happen in the middle of a session +when an extension is first installed or enabled, or when the extension +is updated. +The JSON fragment above causes the WebExtensions framework to load the +API implementation when it encounters a specific manifest key while +starting an extension, and then call its ``onManifestEntry()`` method +with the name of the property as an argument. +The value of the property is not passed, but the full manifest is +available through ``this.extension.manifest``: + +.. code-block:: js + + this.myapi = class extends ExtensionAPI { + onManifestEntry(name) { + let value = this.extension.manifest.my_api_property; + /* do something with value... */ + } + } diff --git a/toolkit/components/extensions/docs/other.rst b/toolkit/components/extensions/docs/other.rst new file mode 100644 index 0000000000..85a9b6db41 --- /dev/null +++ b/toolkit/components/extensions/docs/other.rst @@ -0,0 +1,140 @@ +Utilities for implementing APIs +=============================== + +This page covers some utility classes that are useful for +implementing WebExtension APIs: + +WindowManager +------------- +This class manages the mapping between the opaque window identifiers used +in the `browser.windows `__ API. +See the reference docs `here `__. + +TabManager +---------- +This class manages the mapping between the opaque tab identifiers used +in the `browser.tabs `__ API. +See the reference docs `here `__. + +ExtensionSettingsStore +---------------------- +ExtensionSettingsStore (ESS) is used for storing changes to settings that are +requested by extensions, and for finding out what the current value +of a setting should be, based on the precedence chain or a specific selection +made (typically) by the user. + +When multiple extensions request to make a change to a particular +setting, the most recently installed extension will be given +precedence. + +It is also possible to select a specific extension (or no extension, which +infers user-set) to control a setting. This will typically only happen via +ExtensionPreferencesManager described below. When this happens, precedence +control is not used until either a new extension is installed, or the controlling +extension is disabled or uninstalled. If user-set is specifically chosen, +precedence order will only be returned to by installing a new extension that +takes control of the setting. + +ESS will manage what has control over a setting through any +extension state changes (ie. install, uninstall, enable, disable). + +Notifications: +^^^^^^^^^^^^^^ + +"extension-setting-changed": +**************************** + + When a setting changes an event is emitted via the apiManager. It contains + the following: + + * *action*: one of select, remove, enable, disable + + * *id*: the id of the extension for which the setting has changed, may be null + if the setting has returned to default or user set. + + * *type*: The type of setting altered. This is defined by the module using ESS. + If the setting is controlled through the ExtensionPreferencesManager below, + the value will be "prefs". + + * *key*: The name of the setting altered. + + * *item*: The new value, if any that has taken control of the setting. + + +ExtensionPreferencesManager +--------------------------- +ExtensionPreferencesManager (EPM) is used to manage what extensions may control a +setting that results in changing a preference. EPM adds additional logic on top +of ESS to help manage the preference values based on what is in control of a +setting. + +Defining a setting in an API +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +A preference setting is defined in an API module by calling EPM.addSetting. addSetting +allows the API to use callbacks that can handle setting preferences as needed. Since +the setting is defined at runtime, the API module must be loaded as necessary by EPM +to properly manage settings. + +In the api module definition (e.g. ext-toolkit.json), the api must use `"settings": true` +so the management code can discover which API modules to load in order to manage a +setting. See browserSettings[1] as an example. + +Settings that are exposed to the user in about:preferences also require special handling. +We typically show that an extension is in control of the preference, and prevent changes +to the setting. Some settings may allow the user to choose which extension (or none) has +control of the setting. + +Preferences behavior +^^^^^^^^^^^^^^^^^^^^ + +To actually set a setting, the module must call EPM.setSetting. This is typically done +via an extension API, such as browserSettings.settingName.set({ ...value data... }), though +it may be done at other times, such as during extension startup or install in a modules +onManifest handler. + +Preferences are not always changed when an extension uses an API that results in a call +to EPM.setSetting. When setSetting is called, the values are stored by ESS (above), and if +the extension currently has control, or the setting is controllable by the extension, then +the preferences would be updated. + +The preferences would also potentially be updated when installing, enabling, disabling or +uninstalling an extension, or by a user action in about:preferences (or other UI that +allows controlling the preferences). If all extensions that use a preference setting are +disabled or uninstalled, the prior user-set or default values would be returned to. + +An extension may watch for changes using the onChange api (e.g. browserSettings.settingName.onChange). + +[1] https://searchfox.org/mozilla-central/rev/04d8e7629354bab9e6a285183e763410860c5006/toolkit/components/extensions/ext-toolkit.json#19 + +Notifications: +^^^^^^^^^^^^^^ + +"extension-setting-changed:*name*": +*********************************** + + When a setting controlled by EPM changes an event is emitted via the apiManager. It contains + no other data. This is used primarily to implement the onChange API. + +ESS vs. EPM +----------- +An API may use ESS when it needs to allow an extension to store a setting value that +affects how Firefox works, but does not result in setting a preference. An example +is allowing an extension to change the newTab value in the newTab service. + +An API should use EPM when it needs to allow an extension to change a preference. + +Using ESS/EPM with experimental APIs +------------------------------------ + +Properly managing settings values depends on the ability to load any modules that +define a setting. Since experimental APIs are defined inside the extension, there +are situations where settings defined in experimental APIs may not be correctly +managed. The could result in a preference remaining set by the extension after +the extension is disabled or installed, especially when that state is updated during +safe mode. + +Extensions making use of settings in an experimental API should practice caution, +potentially unsetting the values when the extension is shutdown. Values used for +the setting could be stored in the extensions locale storage, and restored into +EPM when the extension is started again. diff --git a/toolkit/components/extensions/docs/reference.rst b/toolkit/components/extensions/docs/reference.rst new file mode 100644 index 0000000000..f88c0b872e --- /dev/null +++ b/toolkit/components/extensions/docs/reference.rst @@ -0,0 +1,35 @@ +WebExtensions Javascript Component Reference +============================================ +This page contains reference documentation for the individual classes +used to implement WebExtensions APIs. This documentation is generated +from jsdoc comments in the source code. + +ExtensionAPI class +------------------ +.. js:autoclass:: ExtensionAPI + :members: + +Extension class +--------------- +.. js:autoclass:: Extension + :members: + +EventManager class +------------------ +.. js:autoclass:: EventManager + :members: + +BaseContext class +----------------- +.. js:autoclass:: BaseContext + :members: + +WindowManager class +------------------- +.. js:autoclass:: WindowManagerBase + :members: + +TabManager class +---------------- +.. js:autoclass:: TabManagerBase + :members: diff --git a/toolkit/components/extensions/docs/schema.rst b/toolkit/components/extensions/docs/schema.rst new file mode 100644 index 0000000000..b55e918588 --- /dev/null +++ b/toolkit/components/extensions/docs/schema.rst @@ -0,0 +1,145 @@ +API Schemas +=========== +Anything that a WebExtension API exposes to extensions via Javascript +is described by the API's schema. The format of API schemas uses some +of the same syntax as `JSON Schema `_. +JSON Schema provides a way to specify constraints on JSON documents and +the same method is used by WebExtensions to specify constraints on, +for example, parameters passed to an API function. But the syntax for +describing functions, namespaces, etc. is all ad hoc. This section +describes that syntax. + +An individual API schema consists of structured descriptions of +items in one or more *namespaces* using a structure like this: + +.. code-block:: js + + [ + { + "namespace": "namespace1", + // declarations for namespace 1... + }, + { + "namespace": "namespace2", + // declarations for namespace 2... + }, + // other namespaces... + ] + +Most of the namespaces correspond to objects available to extensions +Javascript code under the ``browser`` global. For example, entries in the +namespace ``example`` are accessible to extension Javascript code as +properties on ``browser.example``. +The namespace ``"manifest"`` is handled specially, it describes the +structure of WebExtension manifests (i.e., ``manifest.json`` files). +Manifest schemas are explained in detail below. + +Declarations within a namespace look like: + +.. code-block:: js + + { + "namespace": "namespace1", + "types": [ + { /* type definition */ }, + ... + ], + "properties": { + "NAME": { /* property definition */ }, + ... + }, + "functions": [ + { /* function definition */ }, + ... + ], + "events": [ + { /* event definition */ }, + ... + ] + } + +The four types of objects that can be defined inside a namespace are: + +- **types**: A type is a re-usable schema fragment. A common use of types + is to define in one place an object with a particular set of typed fields + that is used in multiple places in an API. + +- **properties**: A property is a fixed Javascript value available to + extensions via Javascript. Note that the format for defining + properties in a schema is different from the format for types, functions, + and events. The next subsection describes creating properties in detail. + +- **functions** and **events**: + These entries create functions and events respectively, which are + usable from Javascript by extensions. Details on how to implement + them are later in this section. + +Implementing a fixed Javascript property +---------------------------------------- +A static property is made available to extensions via Javascript +entirely from the schema, using a fragment like this one: + +.. code-block:: js + + [ + "namespace": "myapi", + "properties": { + "SOME_PROPERTY": { + "value": 24, + "description": "Description of my property here." + } + } + ] + +If a WebExtension API with this fragment in its schema is loaded for +a particular extension context, that extension will be able to access +``browser.myapi.SOME_PROPERTY`` and read the fixed value 24. +The contents of ``value`` can be any JSON serializable object. + +Schema Items +------------ +Most definitions of individual items in a schema have a common format: + +.. code-block:: js + + { + "type": "SOME TYPE", + /* type-specific parameters... */ + } + +Type-specific parameters will be described in subsequent sections, +but there are some optional properties that can appear in many +different types of items in an API schema: + +- ``description``: This string-valued property serves as documentation + for anybody reading or editing the schema. + +- ``permissions``: This property is an array of strings. + If present, the item in which this property appears is only made + available to extensions that have all the permissions listed in the array. + +- ``unsupported``: This property must be a boolean. + If it is true, the item in which it appears is ignored. + By using this property, a schema can define how a particular API + is intended to work, before it is implemented. + +- ``deprecated``: This property must be a boolean. If it is true, + any uses of the item in which it appears will cause a warning to + be logged to the browser console, to indicate to extension authors + that they are using a feature that is deprecated or otherwise + not fully supported. + + +Describing constrained values +----------------------------- +There are many places where API schemas specify constraints on the type +and possibly contents of some JSON value (e.g., the manifest property +``name`` must be a string) or Javascript value (e.g., the first argument +to ``browser.tabs.get()`` must be a non-negative integer). +These items are defined using `JSON Schema `_. +Specifically, these items are specified by using one of the following +values for the ``type`` property: ``boolean``, ``integer``, ``number``, +``string``, ``array``, ``object``, or ``any``. +Refer to the documentation and examples at the +`JSON Schema site `_ for details on how these +items are defined in a schema. diff --git a/toolkit/components/extensions/docs/webext-storage.rst b/toolkit/components/extensions/docs/webext-storage.rst new file mode 100644 index 0000000000..23005bb679 --- /dev/null +++ b/toolkit/components/extensions/docs/webext-storage.rst @@ -0,0 +1,227 @@ +======================== +How webext storage works +======================== + +This document describes the implementation of the the `storage.sync` part of the +`WebExtensions Storage APIs +`_. +The implementation lives in the `toolkit/components/extensions/storage folder `_ + +Ideally you would already know about Rust and XPCOM - `see this doc for more details <../../../../writing-rust-code/index.html>`_ + +At a very high-level, the system looks like: + +.. mermaid:: + + graph LR + A[Extensions API] + A --> B[Storage JS API] + B --> C{magic} + C --> D[app-services component] + +Where "magic" is actually the most interesting part and the primary focus of this document. + + Note: The general mechanism described below is also used for other Rust components from the + app-services team - for example, "dogear" uses a similar mechanism, and the sync engines + too (but with even more complexity) to manage the threads. Unfortunately, at time of writing, + no code is shared and it's not clear how we would, but this might change as more Rust lands. + +The app-services component `lives on github `_. +There are docs that describe `how to update/vendor this (and all) external rust code <../../../../build/buildsystem/rust.html>`_ you might be interested in. + +To set the scene, let's look at the parts exposed to WebExtensions first; there are lots of +moving part there too. + +WebExtension API +################ + +The WebExtension API is owned by the addons team. The implementation of this API is quite complex +as it involves multiple processes, but for the sake of this document, we can consider the entry-point +into the WebExtension Storage API as being `parent/ext-storage.js `_ + +This entry-point ends up using the implementation in the +`ExtensionStorageSync JS class `_. +This class/module has complexity for things like migration from the earlier Kinto-based backend, +but importantly, code to adapt a callback API into a promise based one. + +Overview of the API +################### + +At a high level, this API is quite simple - there are methods to "get/set/remove" extension +storage data. Note that the "external" API exposed to the addon has subtly changed the parameters +for this "internal" API, so there's an extension ID parameter and the JSON data has already been +converted to a string. +The semantics of the API are beyond this doc but are +`documented on MDN `_. + +As you will see in those docs, the API is promise-based, but the rust implementation is fully +synchronous and Rust knows nothing about Javascript promises - so this system converts +the callback-based API to a promise-based one. + +xpcom as the interface to Rust +############################## + +xpcom is old Mozilla technology that uses C++ "vtables" to implement "interfaces", which are +described in IDL files. While this traditionally was used to interface +C++ and Javascript, we are leveraging existing support for Rust. The interface we are +exposing is described in `mozIExtensionStorageArea.idl `_ + +The main interface of interest in this IDL file is `mozIExtensionStorageArea`. +This interface defines the functionality - and is the first layer in the sync to async model. +For example, this interface defines the following method: + +.. code-block:: rust + + interface mozIExtensionStorageArea : nsISupports { + ... + // Sets one or more key-value pairs specified in `json` for the + // `extensionId`... + void set(in AUTF8String extensionId, + in AUTF8String json, + in mozIExtensionStorageCallback callback); + +As you will notice, the 3rd arg is another interface, `mozIExtensionStorageCallback`, also +defined in that IDL file. This is a small, generic interface defined as: + +.. code-block:: + + interface mozIExtensionStorageCallback : nsISupports { + // Called when the operation completes. Operations that return a result, + // like `get`, will pass a `UTF8String` variant. Those that don't return + // anything, like `set` or `remove`, will pass a `null` variant. + void handleSuccess(in nsIVariant result); + + // Called when the operation fails. + void handleError(in nsresult code, in AUTF8String message); + }; + +Note that this delivers all results and errors, so must be capable of handling +every result type, which for some APIs may be problematic - but we are very lucky with this API +that this simple XPCOM callback interface is capable of reasonably representing the return types +from every function in the `mozIExtensionStorageArea` interface. + +(There's another interface, `mozIExtensionStorageListener` which is typically +also implemented by the actual callback to notify the extension about changes, +but that's beyond the scope of this doc.) + +*Note the thread model here is async* - the `set` call will return immediately, and later, on +the main thread, we will call the callback param with the result of the operation. + +So under the hood, what happens is something like: + +.. mermaid:: + + sequenceDiagram + Extension->>ExtensionStorageSync: call `set` and give me a promise + ExtensionStorageSync->>xpcom: call `set`, supplying new data and a callback + ExtensionStorageSync-->>Extension: your promise + xpcom->>xpcom: thread magic in the "bridge" + xpcom-->>ExtensionStorageSync: callback! + ExtensionStorageSync-->>Extension: promise resolved + +So onto the thread magic in the bridge! + +webext_storage_bridge +##################### + +The `webext_storage_bridge `_ +is a Rust crate which, as implied by the name, is a "bridge" between this Javascript/XPCOM world to +the actual `webext-storage `_ crate. + +lib.rs +------ + +Is the entry-point - it defines the xpcom "factory function" - +an `extern "C"` function which is called by xpcom to create the Rust object +implementing `mozIExtensionStorageArea` using existing gecko support. + +area.rs +------- + +This module defines the interface itself. For example, inside that file you will find: + +.. code-block:: rust + + impl StorageSyncArea { + ... + + xpcom_method!( + set => Set( + ext_id: *const ::nsstring::nsACString, + json: *const ::nsstring::nsACString, + callback: *const mozIExtensionStorageCallback + ) + ); + /// Sets one or more key-value pairs. + fn set( + &self, + ext_id: &nsACString, + json: &nsACString, + callback: &mozIExtensionStorageCallback, + ) -> Result<()> { + self.dispatch( + Punt::Set { + ext_id: str::from_utf8(&*ext_id)?.into(), + value: serde_json::from_str(str::from_utf8(&*json)?)?, + }, + callback, + )?; + Ok(()) + } + + +Of interest here: + +* `xpcom_method` is a Rust macro, and part of the existing xpcom integration which already exists + in gecko. It declares the xpcom vtable method described in the IDL. + +* The `set` function is the implementation - it does string conversions and the JSON parsing + on the main thread, then does the work via the supplied callback param, `self.dispatch` and a `Punt`. + +* The `dispatch` method dispatches to another thread, leveraging existing in-tree `moz_task `_ support, shifting the `Punt` to another thread and making the callback when done. + +Punt +---- + +`Punt` is a whimsical name somewhat related to a "bridge" - it carries things across and back. + +It is a fairly simple enum in `punt.rs `_. +It's really just a restatement of the API we expose suitable for moving across threads. In short, the `Punt` is created on the main thread, +then sent to the background thread where the actual operation runs via a `PuntTask` and returns a `PuntResult`. + +There's a few dances that go on, but the end result is that `inner_run() `_ +gets executed on the background thread - so for `Set`: + +.. code-block:: rust + + Punt::Set { ext_id, value } => { + PuntResult::with_change(&ext_id, self.store()?.get()?.set(&ext_id, value)?)? + } + +Here, `self.store()` is a wrapper around the actual Rust implementation from app-services with +various initialization and mutex dances involved - see `store.rs`. +ie, this function is calling our Rust implementation and stashing the result in a `PuntResult` + +The `PuntResult` is private to that file but is a simple struct that encapsulates both +the actual result of the function (also a set of changes to send to observers, but that's +beyond this doc). + +Ultimately, the `PuntResult` ends up back on the main thread once the call is complete +and arranges to callback the JS implementation, which in turn resolves the promise created in `ExtensionStorageSync.jsm` + +End result: +----------- + +.. mermaid:: + + sequenceDiagram + Extension->>ExtensionStorageSync: call `set` and give me a promise + ExtensionStorageSync->>xpcom - bridge main thread: call `set`, supplying new data and a callback + ExtensionStorageSync-->>Extension: your promise + xpcom - bridge main thread->>moz_task worker thread: Punt this + moz_task worker thread->>webext-storage: write this data to the database + webext-storage->>webext-storage: done: result/error and observers + webext-storage-->>moz_task worker thread: ... + moz_task worker thread-->>xpcom - bridge main thread: PuntResult + xpcom - bridge main thread-->>ExtensionStorageSync: callback! + ExtensionStorageSync-->>Extension: promise resolved diff --git a/toolkit/components/extensions/docs/webidl_bindings.rst b/toolkit/components/extensions/docs/webidl_bindings.rst new file mode 100644 index 0000000000..93556e9296 --- /dev/null +++ b/toolkit/components/extensions/docs/webidl_bindings.rst @@ -0,0 +1,246 @@ +WebIDL WebExtensions API Bindings +================================= + +While on ``manifest_version: 2`` all the extension globals (extension pages and content scripts) +that lives on the main thread and the WebExtensions API bindings can be injected into the extension +global from the JS privileged code part of the WebExtensions internals (`See Schemas.inject defined in +Schemas.jsm `_), +in ``manifest_version: 3`` the extension will be able to declare a background service worker +instead of a background page, and the existing WebExtensions API bindings can't be injected into this +new extension global, because it lives off of the main thread. + +To expose WebExtensions API bindings to the WebExtensions ``background.service_worker`` global +we are in the process of generating new WebIDL bindings for the WebExtensions API. + +.. warning:: + + For more general in depth details about WebIDL in Gecko: + + - :doc:`/dom/bindings/webidl/index` + - :doc:`/dom/webIdlBindings/index` + +Review process on changes to webidl definitions +----------------------------------------------- + +.. note:: + + When new webidl definitions are being introduced for a WebExtensions API, or + existing ones need to be updated to stay in sync with changes applied to the + JSONSchema definitions of the same WebExtensions API, the resulting patch + will include a **new or changed WebIDL located at dom/webidl** and that part of the + patch **will require a mandatory review and sign-off from a peer part of the** + webidl_ **phabricator review group**. + +This section includes a brief description about the special setup of the +webidl files related to WebExtensions and other notes useful to the +WebIDL peers that will be reviewing and signing off these webidl files. + +.. _webidl: https://phabricator.services.mozilla.com/tag/webidl/ + +How/Where are these webidl interfaces restricted to the extensions background service workers? +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +All the webidl interfaces related to the extensions API are only visible in +specific extension globals: the WebExtensions background service worker +(a service worker declared in the extension ``manifest.json`` file, through +the ``background.service_worker`` manifest field). + +All webidl interfaces related to the WebExtensions API interfaces are exposed +through the ``ExtensionBrowser`` interface, which gets exposed into the +``ServiceWorkerGlobalScope`` through the ``ExtensionGlobalsMixin`` interface and +restricted to the WebExtensions background service worker through the +``mozilla::extensions::ExtensionAPIAllowed`` helper function. + +See ``ExtensionBrowser`` and ``ExtensionGlobalsMixin`` interfaces defined from +ExtensionBrowser.webidl_ and ``mozilla::extensions::ExtensionAPIAllowed`` defined in +ExtensionBrowser.cpp_. + +.. _ExtensionBrowser.webidl: https://searchfox.org/mozilla-central/source/dom/webidl/ExtensionBrowser.webidl +.. _ExtensionBrowser.cpp: https://searchfox.org/mozilla-central/source/toolkit/components/extensions/webidl-api/ExtensionBrowser.cpp + +Why do all the webidl interfaces for WebExtensions API use LegacyNoInterfaceObject? +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The existing WebExtensions API bindings are not exposing any constructor in the +globals where they are available (e.g. the webidl bindings for the ``browser.alarms`` +API namespace is defined by the ``ExtensionAlarms`` webidl interface, but there +shouldn't be any ``ExtensionAlarms`` constructor available as a global to extension +code running in the background service worker). + +A previous attempt to create W3C specs for the WebExtensions APIs described in WebIDL +syntaxes (https://browserext.github.io/browserext) was also using the same +``NoInterfaceObject`` WebIDL attribute on the definitions of the API namespace +with the same motivations (eg. see ``BrowserExtBrowserRuntime`` as defined here: +https://browserext.github.io/browserext/#webidl-definition-4). + +Bug 1713877_ is tracking a followup to determine a long term replacement for the +``LegacyNoInterfaceObject`` attribute currently being used. + +.. _1713877: https://bugzilla.mozilla.org/1713877 + +Background Service Workers API Request Handling +----------------------------------------------- + +.. figure:: webidl_bindings_backgroundWorker_apiRequestHandling.drawio.svg + :alt: High Level Diagram of the Background Service Worker API Request Handling + +.. + This svg diagram has been created using https://app.diagrams.net, + the svg file also includes the source in the drawio format and so + it can be edited more easily by loading it back into app.diagrams.net + and then re-export from there (and include the updated drawio format + content into the exported svg file). + +Generating WebIDL definitions from WebExtensions API JSONSchema +--------------------------------------------------------------- + +WebIDL definitions for the extension APIs are being generated based on the WebExtensions API JSONSchema +data (the same metadata used to generate the "privilged JS"-based API bindings). + +Most of the API methods in generated WebIDL are meant to be implemented using stub methods shared +between all WebExtensions API classes, a ``WebExtensionStub`` webidl extended attribute specify +which shared stub method should be used when the related API method is called. + +For more in depth details about how to generate or update webidl definition for an Extension API +given its API namespace: + +.. toctree:: + :maxdepth: 2 + + generate_webidl_from_jsonschema + +Wiring up new WebExtensions WebIDL files into mozilla-central +------------------------------------------------------------- + +After a new WebIDL definition has been generated, there are a few more steps to ensure that +the new WebIDL binding is wired up into mozilla-central build system and to be able to +complete successfully a full Gecko build that include the new bindings. + +For more in depth details about these next steps: + +.. toctree:: + :maxdepth: 2 + + wiring_up_new_webidl_bindings + +Testing WebExtensions WebIDL bindings +------------------------------------- + +Once the WebIDL definition for an WebExtensions API namespace has been +implemented and wired up, the following testing strategies are available to +cover them as part of the WebExtensions testing suites: + +``toolkit/components/extensions/test/xpcshell/webidl-api`` +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The xpcshell tests added to this group of xpcshell tests are meant to provide testing coverage +related to lower level components and behaviors (e.g. when making changes to the shared C++ +helpers defined in ``toolkit/components/extensions/webidl-api``, or adding new ones). + +These tests will often mock part of the internals and use a ``browser.mockExtensionAPI`` +API namespace which is only available in tests and not mapped to any actual API implementation +(instead it is being mocked in the test cases to recreate the scenario that the test case is meant +to cover). + +And so **they are not meant to provide any guarantees in terms of consistency of the behavior +of the two different bindings implementations** (the new WebIDL bindings vs. the current implemented +bindings), instead the other test suites listed in the sections below should be used for that purpose. + +All tests in this directory are skipped in builds where the WebExtensions WebIDL API bindings +are being disabled at build time (e.g. beta and release builds, where otherwise +the test would permafail while riding the train once got on those builds). + + +``toolkit/components/extensions/test/xpcshell/xpcshell-serviceworker.ini`` +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +When a new or existing xpcshell tests added to this xpcshell-test manifest, all test extensions +will be generated with a background service worker instead of a background page. + +.. warning:: + **Unless the background page or scripts are declared as part of the test extension manifest**, + the test file added to this manifest should be explicitly reviewed to make sure all tests + are going to provide the expected test coverage in all modes. + +.. note:: + In a background script that runs in both a background page and a background + service worker it may be necessary to run different code for part of the + test, ``self !== self.window`` is a simple check that can be used to detect if + the script is being executed as a background service worker. + +Test tasks that should be skipped when running in "background service worker mode", but temporarily +until a followup fixes the underlying issue can use the ``ExtensionTestUtils.isInBackgroundServiceWorkerTests()`` in the optional +``add_task``'s ``skip_if`` parameter: + +.. code-block:: js + + add_task( + { + // TODO(Bug TBF): remove this once ... + skip_if: () => ExtensionTestUtils.isInBackgroundServiceWorkerTests(), + }, + async function test_someapi_under_scenario() { + ... + } + ); + +On the contrary if the test tasks is covering a scenario that is specific to a background page, +and it would need to be permanently skipped while the background script is running as a service worker, +it may be more appropriate to split out those tests in a separate test file not included in this +manifest. + +.. warning:: + Make sure that all tests running in multiple modes (in-process, + remote, and "background service worker mode") do not assume that the WebIDL + bindings and Background Service Worker are enabled and to skip them when appropriate, + otherwise the test will become a permafailure once it gets to a channel + where the "Extensions WebIDL API bindings" are disabled by default at build + time (currently on **beta** and **release**). + +While running the test files locally they will be executed once for each manifest file +where they are included, to restrict the run to just the "background service +workers mode" specify the ``sw-webextensions`` tag: + +.. code-block:: + + mach xpcshell-test --tag sw-webextensions path/to/test/file.js + +``toolkit/components/extensions/test/mochitest/mochitest-serviceworker.ini`` +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Same as the xpcshell-serviceworker.ini manifest but for the mochitest-plain tests. + +.. warning:: + The same warnings described in the xpcshell-serviceworker.ini subsection do + also apply to this manifest file. + +Test tasks that should be skipped when not running in "background service worker +mode" can be split into separate test file or skipped inside the ``add_task`` +body, but mochitests' ``add_task`` does not support a ``skip_if`` option and so +that needs to be done manually (and it may be good to also log a message to make +it visible when a test is skipped): + +.. code-block:: js + + add_task(async function test_someapi_in_background_service_worker() { + if (!ExtensionTestUtils.isInBackgroundServiceWorkerTests()) { + is( + ExtensionTestUtils.getBackgroundServiceWorkerEnabled(), + false, + "This test should only be skipped with background service worker disabled" + ) + info("Test intentionally skipped on 'extensions.backgroundServiceWorker.enabled=false'"); + return; + } + + + ... + }); + +While executing the test files locally they will run once for each manifest file +where they are included, to restrict the run to just the "background service +workers mode" specify the ``sw-webextensions`` tag: + +.. code-block:: + + mach mochitest --tag sw-webextensions path/to/test/file.js diff --git a/toolkit/components/extensions/docs/webidl_bindings_backgroundWorker_apiRequestHandling.drawio.svg b/toolkit/components/extensions/docs/webidl_bindings_backgroundWorker_apiRequestHandling.drawio.svg new file mode 100644 index 0000000000..ff1fc003ff --- /dev/null +++ b/toolkit/components/extensions/docs/webidl_bindings_backgroundWorker_apiRequestHandling.drawio.svg @@ -0,0 +1,4 @@ + + + +
IPC
IPC
Extension API namespace
(WebIDL - C++)
Extension API namesp...
mozIExtensionAPI
RequestHandler
(XPCOM - JS)
mozIExtensionAPI...mozIExtensionAPIRequest
  • apiNamespace
  • apiName
  • apiObjectType
  • apiObjectId
  • callerSavedFrame
  • serviceWorkerInfo
  • args
  • normalizedArgs (R/W)
apiNamespaceapiName...
(XPCOM - C++)
(XPCOM - C++)
  • retrieve WorkerContextChild
  • validate and normalize arguments
  • check permissions
retrieve WorkerContextChild...
WebIDL
ChildAPIManager
(extends ChildAPIManager)
WebIDL...
WebIDL
ChildLocalAPIImpl
(extends ChildLocalAPIImpl)
WebIDL...
WebIDL
ChildObjectTypeIImpl
(extends ChildLocalAPIImpl)
WebIDL...
ProxyAPIImplementation
ProxyAPIImplementation
ProcessConduitsChild
ProcessConduitsChild
ProcessConduitsParent
ProcessConduitsParent
ext-APINAMESPACE.js
ExtensionAPI subclass
ext-APINAMESPACE.js...
PARENT
PROCESS
PARENT...
EXTENSIONS
CHILD PROCESS
EXTENSIONS...
DOM Worker
Thread
DOM Worker...
Main
Thread
Main...Viewer does not support full SVG 1.1 \ No newline at end of file diff --git a/toolkit/components/extensions/docs/wiring_up_new_webidl_bindings.rst b/toolkit/components/extensions/docs/wiring_up_new_webidl_bindings.rst new file mode 100644 index 0000000000..59088d7079 --- /dev/null +++ b/toolkit/components/extensions/docs/wiring_up_new_webidl_bindings.rst @@ -0,0 +1,165 @@ +Wiring up new WebExtensions WebIDL files into mozilla-central +============================================================= + +Add a new entry in ``dom/bindings/Bindings.conf`` +------------------------------------------------- + +New WebIDL bindings should be added as new entries in ``dom/bindings/Bindings.conf``. The new entry should be +added in alphabetic order and nearby the other WebExtensions API bindings already listed in this config file +(look for the ``ExtensionBrowser`` webidl definition and the other existing WebIDL bindings related to the +WebExtensions APIs): + +.. code-block:: + + # WebExtension API + ... + 'ExtensionRuntime': { + 'headerFile': 'mozilla/extensions/ExtensionRuntime.h', + 'nativeType': 'mozilla::extensions::ExtensionRuntime', + }, + +.. warning:: + + `mach build` will fail if the entries in `dom/bindings/Bindings.conf` are not in alphabetic order, + or if the `headerFile` referenced does not exist yet. + +Add a new entry in ``dom/webidl/moz.build`` +------------------------------------------- + +The new ``.webidl`` file has to be also listed in "dom/webidl/moz.build", it should be added in + +- the existing group of ``WEBIDL_FILES`` entries meant specifically for the WebExtensions API bindings +- or in the group of ``PREPROCESSED_WEBIDL_FILES`` entries meant specifically for the WebExtensions + API bindings, **if the generated `.webidl` includes preprocessing macros** (e.g. when part of an API + is not available in all builds, e.g. subset of APIs that are only available in Desktop builds). + +.. code-block:: + + # WebExtensions API. + WEBIDL_FILES += [ + ... + "ExtensionRuntime.webidl" + ... + ] + + PREPROCESSED_WEBIDL_FILES += [ + ... + ] + +.. warning:: + + The group of PREPROCESSED_WERBIDL_FILES meant to list WebExtensions APIs ``.webidl`` files + may not exist yet (one will be added right after the existing `WEBIDL_FILES` when the first + preprocessed `.webidl` will be added). + + +Add new entries in ``toolkit/components/extensions/webidl-api/moz.build`` +------------------------------------------------------------------------- + +The new C++ files for the WebExtensions API binding needs to be added to ``toolkit/components/extensions/webidl-api/moz.build`` +to make them part of the build, The new ``.cpp`` file has to be added into the ``UNIFIED_SOURCES`` group +where the other WebIDL bindings are being listed. Similarly, the new ``.h`` counterpart has to be added to +``EXPORTS.mozilla.extensions`` (which ensures that the header file will be placed into the path set earlier +in ``dom/bindings/Bindings.conf``): + +.. code-block:: + + # WebExtensions API namespaces. + UNIFIED_SOURCES += [ + ... + "ExtensionRuntime.cpp", + ... + ] + + EXPORTS.mozilla.extensions += [ + ... + "ExtensionRuntime.h", + ... + ] + +Wiring up the new API into ``dom/webidl/ExtensionBrowser.webidl`` +----------------------------------------------------------------- + +To make the new WebIDL bindings part of the ``browser`` global, a new attribute has to be added to +``dom/webidl/ExtensionBrowser.webidl``: + +.. code-block:: + + // `browser.runtime` API namespace. + [Replaceable, SameObject, BinaryName="GetExtensionRuntime", + Func="mozilla::extensions::ExtensionRuntime::IsAllowed"] + readonly attribute ExtensionRuntime runtime; + +.. note:: + ``chrome`` is defined as an alias of the ``browser`` global, and so by adding the new attribute + into ``ExtensionBrowser` the same attribute will also be available in the ``chrome`` global. + Unlike the "Privileged JS"-based WebExtensions API, the ``chrome`` and ``browser`` APIs are + exactly the same and a the async methods return a Promise if no callback has been passed + (similarly to Safari versions where the WebExtensions APIs are supported). + +The additional attribute added into ``ExtensionBrowser.webidl`` will require some addition to the ``ExtensionBrowser`` +C++ class as defined in ``toolkit/components/extensions/webidl-api/ExtensionBrowser.h``: + +- the definition of a new corresponding **public method** (by convention named ``GetExtensionMyNamespace``) +- a ``RefPtr`` as a new **private data member named** (by convention named ``mExtensionMyNamespace``) + +.. code-block:: + + ... + namespace extensions { + + ... + class ExtensionRuntime; + ... + + class ExtensionBrowser final : ... { + ... + RefPtr mExtensionRuntime; + ... + + public: + ... + ExtensionRuntime* GetExtensionRuntime(); + } + ... + + +And then in its ``toolkit/components/extensions/webidl-api/ExtensionBrowser.cpp`` counterpart: + +- the implementation of the new public method +- the addition of the new private member data ``RefPtr`` in the ``NS_IMPL_CYCLE_COLLECTION_UNLINK`` + and ``NS_IMPL_CYCLE_COLLECTION_TRAVERSE`` macros + +.. code-block:: + + ... + #include "mozilla/extensions/ExtensionRuntime.h" + ... + NS_IMPL_CYCLE_COLLECTION_UNLINK_BEGIN(ExtensionBrowser) + ... + NS_IMPL_CYCLE_COLLECTION_UNLINK(mExtensionRuntime) + ... + NS_IMPL_CYCLE_COLLECTION_UNLINK_END + + NS_IMPL_CYCLE_COLLECTION_TRAVERSE_BEGIN(ExtensionBrowser) + ... + NS_IMPL_CYCLE_COLLECTION_TRAVERSE(mExtensionRuntime) + ... + NS_IMPL_CYCLE_COLLECTION_TRAVERSE_END + ... + + ExtensionRuntime* ExtensionBrowser::GetExtensionRuntime() { + if (!mExtensionRuntime) { + mExtensionRuntime = new ExtensionRuntime(mGlobal, this); + } + + return mExtensionRuntime + } + +.. warning:: + + Forgetting to add the new ``RefPtr`` into the cycle collection traverse and unlink macros + will not result in a build error, but it will result into a leak. + + Make sure to don't forget to double-check these macros, especially if some tests are failing + because of detected shutdown leaks. -- cgit v1.2.3