diff options
Diffstat (limited to 'remote/doc')
29 files changed, 2445 insertions, 0 deletions
diff --git a/remote/doc/Building.md b/remote/doc/Building.md new file mode 100644 index 0000000000..75880f01f9 --- /dev/null +++ b/remote/doc/Building.md @@ -0,0 +1,51 @@ +Building +======== + +The Remote Agent is included in the default Firefox build, but only +ships on the Firefox Nightly release channel: + + % ./mach run --remote-debugging-port + +The source code can be found under [remote/ in central]. + +There are two build modes to choose from: + +Full build mode +--------------- + +The Remote Agent is included when you build in the usual way: + + % ./mach build + +When you make changes to XPCOM component files you need to rebuild +in order for the changes to take effect. The most efficient way to +do this, provided you haven’t touched any compiled code (C++ or Rust): + + % ./mach build faster + +Component files include the likes of components.conf, +RemoteAgent.manifest, moz.build files, and jar.mn. +All the JS modules (files ending with `.jsm`) are symlinked into +the build and can be changed without rebuilding. + +You may also opt out of building all the WebDriver specific components +([Marionette], and the Remote Agent) by setting the following flag in +your [mozconfig]: + + ac_add_options --disable-webdriver + +Artifact mode +------------- + +You may also use [artifact builds] when working on the Remote Agent. +This fast build mode downloads pre-built components from the Mozilla +build servers, rendering local compilation unnecessary. To use +them, place this in your [mozconfig]: + + ac_add_options --enable-artifact-builds + + +[remote/ in central]: https://searchfox.org/mozilla-central/source/remote +[mozconfig]: /build/buildsystem/mozconfigs.rst +[artifact builds]: /contributing/build/artifact_builds.rst +[Marionette]: /testing/marionette/index.rst diff --git a/remote/doc/CodeStyle.md b/remote/doc/CodeStyle.md new file mode 100644 index 0000000000..4a6571324c --- /dev/null +++ b/remote/doc/CodeStyle.md @@ -0,0 +1,73 @@ +Style guide +=========== + +Like other projects, we also have some guidelines to keep to the code. +For the overall Remote Agent project, a few rough rules are: + + * Make your code readable and sensible, and don’t try to be + clever. Prefer simple and easy solutions over more convoluted + and foreign syntax. + + * Fixing style violations whilst working on a real change as a + preparatory clean-up step is good, but otherwise avoid useless + code churn for the sake of conforming to the style guide. + + * Code is mutable and not written in stone. Nothing that + is checked in is sacred and we encourage change to make + this a pleasant ecosystem to work in. + + * We never land any code that is unnecessary or unused. + + +Documentation +------------- + +We keep our documentation (what you are reading right now!) in-tree +under [remote/doc]. Updates and minor changes to documentation should +ideally not be scrutinised to the same degree as code changes to +encourage frequent updates so that documentation does not go stale. +To that end, documentation changes with `r=me a=doc` from anyone +with commit access level 3 are permitted. + +Use fmt(1) or an equivalent editor specific mechanism (such as +<kbd>Meta-Q</kbd> in Emacs) to format paragraphs at a maximum of +75 columns with a goal of roughly 65. This is equivalent to `fmt +-w75 -g65`, which happens to the default on BSD and macOS. + +The documentation can be built locally this way: + + % ./mach doc remote + +[remote/doc]: https://searchfox.org/mozilla-central/source/remote/doc + + +Linting +------- + +The Remote Agent consists mostly of JavaScript code, and we lint that +using [mozlint], which is harmonises different linters including [eslint]. + +To run the linter and get sensible output: + + % ./mach lint -funix remote + +For certain classes of style violations, eslint has an automatic +mode for fixing and formatting code. This is particularly useful +to keep to whitespace and indentation rules: + + % ./mach eslint --fix remote + +The linter is also run as a try job (shorthand `ES`) which means +any style violations will automatically block a patch from landing +(if using Autoland) or cause your changeset to be backed out (if +landing directly on inbound). + +If you use git(1) you can [enable automatic linting] before +you push to a remote through a pre-push (or pre-commit) hook. +This will run the linters on the changed files before a push and +abort if there are any problems. This is convenient for avoiding +a try run failing due to a simple linting issue. + +[mozlint]: /code-quality/lint/mozlint.rst +[eslint]: /code-quality/lint/linters/eslint.rst +[enable automatic linting]: /tools/lint/usage.html#using-a-vcs-hook diff --git a/remote/doc/Debugging.md b/remote/doc/Debugging.md new file mode 100644 index 0000000000..86df75022d --- /dev/null +++ b/remote/doc/Debugging.md @@ -0,0 +1,53 @@ +Debugging +========= + +For other debugging resources, see also: Remote project [wiki] + +Increasing the logging verbosity +-------------------------------- + +To increase the internal logging verbosity you can use the +`remote.log.level` [preference]. + +If you use mach to start Firefox: + + ./mach run --setpref "remote.log.level=Trace" --remote-debugging-port + +By default, long log lines are truncated. To print long lines in full, you +can set `remote.log.truncate` to false. + +Enabling logging of emitted events +---------------------------------- + +To dump events produced by EventEmitter, +including CDP events produced by the Remote Agent, +you can use the `toolkit.dump.emit` [preference]: + + ./mach run --setpref "toolkit.dump.emit=true" --remote-debugging-port + + +Logging observer notifications +------------------------------ + +Observer notifications are used extensively throughout the +code and it can sometimes be useful to log these to see what is +available and when they are fired. + +The `MOZ_LOG` environment variable controls the C++ logs and takes +the name of the subsystem along with a verbosity setting. See +[prlog.h] for more details. + + MOZ_LOG=ObserverService:5 + +You can optionally redirect logs away from stdout to a file: + + MOZ_LOG_FILE=service.log + +This enables `LogLevel::Debug` level information and places all +output in the file service.log in your current working directory. + + +[preference]: Prefs.md +[System observer notifications]: https://developer.mozilla.org/en-US/docs/Archive/Add-ons/Overlay_Extensions/XUL_School/Observer_Notifications +[prlog.h]: https://searchfox.org/mozilla-central/source/nsprpub/pr/include/prlog.h +[wiki]: https://wiki.mozilla.org/Remote/Developer_Resources diff --git a/remote/doc/Prefs.md b/remote/doc/Prefs.md new file mode 100644 index 0000000000..68ff234550 --- /dev/null +++ b/remote/doc/Prefs.md @@ -0,0 +1,43 @@ +Preferences +=========== + +There are a couple of preferences associated with the Remote Agent: + + +Configurable preferences +------------------------ + +### `remote.active-protocols` + +Defines the remote protocols that are active. Available protocols are, +WebDriver BiDi (`1`), and CDP (`2`). Multiple protocols can be activated +at the same time by using bitwise or with the values. Defaults to `3` (WebDriver +BiDi and CDP). + +### `remote.experimental.enabled` + +Defines if WebDriver BiDi experimental commands and events are available for usage. +Defaults to `true` in Nightly builds, and `false` otherwise. + +### `remote.log.level` + +Defines the verbosity of the internal logger. Available levels +are, in descending order of severity, `Trace`, `Debug`, `Config`, +`Info`, `Warn`, `Error`, and `Fatal`. Note that the value is +treated case-sensitively. + +### `remote.log.truncate` + +Defines whether long log messages should be truncated. Defaults to true. + +### `remote.prefs.recommended` + +By default remote protocols attempts to set a range of preferences deemed +suitable in automation when it starts. These include the likes of +disabling auto-updates, Telemetry, and first-run UX. Set this preference to +`false` to skip setting those preferences, which is mostly useful for internal +Firefox CI suites. + +The user preference file takes precedence over the recommended +preferences, meaning any user-defined preference value will not be +overridden. diff --git a/remote/doc/Security.md b/remote/doc/Security.md new file mode 100644 index 0000000000..70510d975a --- /dev/null +++ b/remote/doc/Security.md @@ -0,0 +1,117 @@ +Security aspects of the Remote Agent +==================================== + +The Remote Agent is not a web-facing feature and as such has different +security characteristics than traditional web platform APIs. The +primary consumers are out-of-process programs that connect to the +agent via a remote protocol, but can theoretically be extended to +facilitate browser-local clients communicating over IPDL. + +Design considerations +--------------------- + +The Remote Agent allows consumers to interface with Firefox through +an assorted set of domains for inspecting the state and controlling +execution of documents running in web content, injecting arbitrary +scripts to documents, do browser service instrumentation, simulation +of user interaction for automation purposes, and for subscribing +to updates in the browser such as network- and console logs. + +The remote interfaces are served over an HTTP wire protocol, by a +server listener hosted in the Firefox binary. This can only be +started by passing the `--remote-debugging-port` +flag. Connections are restricted to loopback devices +(such as localhost and 127.0.0.1). + +Since the Remote Agent is not an in-document web feature, the +security concerns we have for this feature are essentially different +to other web platform features. The primary concern is that the +HTTPD is not spun up without passing one of the command-line flags. +It is out perception that if a malicious user has the capability +to execute arbitrary shell commands, there is little we can do to +prevent the browser being turned into an evil listening device. + +User privacy concerns +--------------------- + +There are no user privacy concerns beyond the fact that the offered +interfaces will give the client access to all browser internals, +and thereby follows all browser-internal secrets. + +How the Remote Agent works +-------------------------- + +When the `--remote-debugging-port` flag is used, +it spins up an HTTPD on the desired port, or defaults to +localhost:9222. The HTTPD serves WebSocket connections via +`nsIWebSocket.createServerWebSocket` that clients connect to in +order to give the agent remote instructions. Hereby the HTTPD only +accepts system-local loopback connections from clients: + + if (!LOOPBACKS.includes(host)) { + throw new Error("Restricted to loopback devices"); + } + +The Remote Agent implements a large subset of the Chrome DevTools +Protocol (CDP). This protocol allows a client to: + +- take control over the user session for automation purposes, for + example to simulate user interaction such as clicking and typing; + +- instrument the browser for analytical reasons, such as intercepting + network traffic; + +- and extract information from the user session, including cookies + and local storage. + +There are no web-exposed features in the Remote Agent whatsoever. + +Security model +-------------- + +It shares the same security model as DevTools and Marionette, in +that there is no other mechanism for enabling the Remote Agent than +by passing a command-line flag. + +It is our assumption that if an attacker has shell access to the +user account, there is little we can do to prevent secrets from +being accessed or leaked. + +The Remote Agent is available on all release channels. + +Remote Hosts and Origins +------------------------ + +By default RemoteAgent only accepts connections with no `Origin` header and a +`Host` header set to an IP address or a localhost loopback address. + +Other `Host` or `Origin` headers can be allowed by starting Firefox with the +`--remote-allow-origins` and `--remote-allow-hosts` arguments: + +- `--remote-allow-hosts` expects a comma separated list of hostnames + +- `--remote-allow-origins` expects a comma separated list of origins + +Note: Users are strongly discouraged from using the Remote Agent in a way that +allows it to be accessed by untrusted hosts e.g. by binding it to a publicly +routeable interface. + +The Remote Agent does not provide message encryption, which means that all +protocol messages are subject to eavesdropping and tampering. It also does not +provide any authentication system. This is acceptable in an isolated test +environment, but not to be used on an untrusted network such as the internet. +People wishing to provide remote access to Firefox sessions via the Remote Agent +must provide their own encryption, authentication, and authorization. + +Security reviews +---------------- + +More details can be found in the security reviews conducted for Remote Agent and +WebDriver BiDi: + +- [Remote Agent security review] (November 2019) + +- [WebDriver BiDi security review] (April 2022) + +[Remote Agent security review]: https://bugzilla.mozilla.org/show_bug.cgi?id=1542229 +[WebDriver BiDi security review]: https://bugzilla.mozilla.org/show_bug.cgi?id=1753997 diff --git a/remote/doc/Testing.md b/remote/doc/Testing.md new file mode 100644 index 0000000000..38fb13760a --- /dev/null +++ b/remote/doc/Testing.md @@ -0,0 +1,164 @@ +Testing +======= + +The Remote Protocol has unit- and functional tests located under different folders: + + - CDP: `remote/cdp` + - Marionette: `remote/marionette`. + - Shared Modules: `remote/shared/` + - WebDriver BiDi: `remote/webdriver-bidi` + +You may want to run all the tests under a particular subfolder locally like this: + + % ./mach test remote + + +Unit tests +---------- + +Because tests are run in parallel and [xpcshell] itself is quite +chatty, it can sometimes be useful to run the tests in sequence: + + % ./mach xpcshell-test --sequential remote/cdp/test/unit/test_DomainCache.js + +The unit tests will appear as part of the `X` (for _xpcshell_) jobs +on Treeherder. + +[xpcshell]: /testing/xpcshell/index.rst + + +Browser Chrome Mochitests +------------------------- + +We also have a set of functional browser-chrome mochitests located +under several components, ie. _remote/shared/messagehandler/test/browser_: + + % ./mach mochitest remote/shared/messagehandler/test/browser/browser_* + +The functional tests will appear under the `M` (for _mochitest_) +category in the `remote` jobs on Treeherder. + +As the functional tests will sporadically pop up new Firefox +application windows, a helpful tip is to run them in headless +mode: + + % ./mach mochitest --headless remote/shared/messagehandler/test/browser + +The `--headless` flag is equivalent to setting the `MOZ_HEADLESS` +environment variable. You can additionally use `MOZ_HEADLESS_WIDTH` +and `MOZ_HEADLESS_HEIGHT` to control the dimensions of the virtual +display. + +The `add_task()` function used for writing asynchronous tests is +replaced to provide some additional test setup and teardown useful +for writing tests against the Remote Agent and the targets. + +There are also specific browser-chrome tests for CDP. + +Before such a task is run, the `nsIRemoteAgent` listener is started +and a [CDP client] is connected. You will use this CDP client for +interacting with the agent just as any other CDP client would. + +Also target discovery is getting enabled, which means that targetCreated, +targetDestroyed, and targetInfoChanged events will be received by the client. + +The task function you provide in your test will be called with the +three arguments `client`, `CDP`, and `tab`: + + - `client` is the connection to the `nsIRemoteAgent` listener, + and it provides the a client CDP API + + - `CDP` is the CDP client class + + - `tab` is a fresh tab opened for each new test, and is automatically + removed after the test has run + +This is what it looks like all put together: + + add_task(async function testName({client, CDP, tab}) { + // test tab is implicitly created for us + info("Current URL: " + tab.linkedBrowser.currentURI.spec); + + // manually connect to a specific target + const { mainProcessTarget } = RemoteAgent.cdp.targetList; + const target = mainProcessTarget.wsDebuggerURL; + const client = await CDP({ target }); + + // retrieve the Browser domain, and call getVersion() on it + const { Browser } = client; + const version = await Browser.getVersion(); + + await client.close(); + + // tab is implicitly removed + }); + +You can control the tab creation behaviour with the `createTab` +option to `add_task(taskFunction, options)`: + + add_task(async function testName({client}) { + // tab is not implicitly created + }, { createTab: false }); + +If you want to write an asynchronous test _without_ this implicit +setup you may instead use `add_plain_task()`, which works exactly like the +original `add_task()`. + +[CDP client]: https://github.com/cyrus-and/chrome-remote-interface + + +Puppeteer tests +--------------- + +In addition to our own Firefox-specific tests, we run the upstream +[Puppeteer test suite] against our implementation to [track progress] +towards achieving full [Puppeteer support] in Firefox. The tests are written +in the behaviour-driven testing framework [Mocha]. + +Puppeteer tests are vendored under _remote/test/puppeteer/_ and are +run locally like this: + + % ./mach puppeteer-test + +You can also run them against Chrome as: + + % ./mach puppeteer-test --product=chrome + +By default the mach command will automatically install Puppeteer but that's +only needed for the very first time, or when a new Puppeteer release has been +vendored in. To skip the install step use the `--no-install` option. + +To run only some specific tests from the whole test suite the appropriate +test files have to be updated first. To select specific tests or test +groups within a file define [exclusive tests] by adding the `.only` suffix +like `it.only()` or `describe.only()`. Make sure to specify as well the +`--subset` option to the mach command to disable the check for missing or +skipped tests in our log parsing. + +More customizations for [Mocha] can be found in its own documentation. + +Test expectation metadata is collected in _remote/test/puppeteer-expected.json_ +via log parsing and a custom Mocha reporter under +_remote/test/puppeteer/json-mocha-reporter.js_ + +Check the upstream [Puppeteer test suite] documentation for instructions on +how to skip tests, run only one test or a subsuite of tests. + +Testing on Try +-------------- + +To schedule all the Remote Protocol tests on try, you can use the +`remote-protocol` [try preset]: + + mach try --preset remote-protocol + +But you can also schedule tests by selecting relevant jobs yourself: + + mach try fuzzy + +[Puppeteer test suite]: https://github.com/puppeteer/puppeteer/blob/master/test/README.md +[Puppeteer support]: https://bugzilla.mozilla.org/show_bug.cgi?id=puppeteer +[Mocha]: https://mochajs.org/ +[exclusive tests]: https://mochajs.org/#exclusive-tests +[track progress]: https://puppeteer.github.io/ispuppeteerfirefoxready/ +[try preset]: /tools/try/presets diff --git a/remote/doc/cdp/Architecture.md b/remote/doc/cdp/Architecture.md new file mode 100644 index 0000000000..f9465e3c71 --- /dev/null +++ b/remote/doc/cdp/Architecture.md @@ -0,0 +1,163 @@ +Remote Agent overall architecture +================================= + +This document will cover the Remote Agent architecture by following the sequence of steps needed to start the agent, connect a client and debug a target. + +Remote Agent startup +-------------------- + +Everything starts with the `RemoteAgent` implementation, which handles command line +arguments (--remote-debugging-port) to eventually +start a server listening on the TCP port 9222 (or the one specified by the command line). +The browser target websocket URL will be printed to stderr. +To do that this component glue together three main high level components: + + * `server/HTTPD` + This is a copy of httpd.js, from /netwerk/ folder. This is a JS implementation of an http server. + This will be used to implement the various http endpoints of CDP. + There is a few static URL implemented by `JSONHandler` and one dynamic URL per target. + + * `cdp/JSONHandler` + This implements the following three static http endpoints: + * /json/version: + Returns information about the runtime as well as the url of the browser target websocket url. + * /json/list: + Returns a list of all debuggable targets with, for each, their dynamic websocket URL. + For now it only reports tabs, but will report workers and addons as soon as we support them. + The main browser target is the only one target not listed here. + * /json/protocol: + Returns a big dictionary describing the supported protocol. + This is currently hard coded and returns the full CDP protocol schema, including APIs we don’t support. + We have a future intention to fix this and report only what Firefox implements. + You can connect to these websocket URL in order to debug things. + + * `cdp/targets/TargetList` + This component is responsible of maintaining the list of all debuggable targets. + For now it can be either: + * The main browser target + A special target which allows to inspect the browser, but not any particular tab. + This is implemented by `cdp/targets/MainProcessTarget` and is instantiated on startup. + * Tab targets + Each opened tab will have a related `cdp/targets/TabTarget` instantiated on their opening, + or on server startup for already opened ones. + Each target aims at focusing on one particular context. This context is typically running in one + particular environment. This can be a particular process or thread. + In the future, we will most likely support targets for workers and add-ons. + All targets inherit from `cdp/targets/Target`. + +Connecting to Websocket endpoints +--------------------------------- + +Each target's websocket URL will be registered as a HTTP endpoint via `server/HTTPD:registerPathHandler` (This registration is done from `RemoteAgentParentProcess:#listen`). +Once a HTTP request happens, `server/HTTPD` will call the `handle` method on the object passed to `registerPathHandler`. +For static endpoints registered by `JSONHandler`, this will call `JSONHandler:handle` and return a JSON string as http body. +For target's endpoint, it is slightly more complicated as it requires a special handshake to morph the HTTP connection into a WebSocket one. +The WebSocket is then going to be long lived and be used to inspect the target over time. +When a request is made to a target URL, `cdp/targets/Target:handle` is called and: + + * delegate the complex HTTP to WebSocket handshake operation to `server/WebSocketHandshake:upgrade` + In return we retrieve a WebSocket object. + + * hand over this WebSocket to `server/WebSocketTransport` + and get a transport object in return. The transport implements a basic JSON stream over WebSocket. With that, you can send and receive JSON objects over a WebSocket connection. + + * hand over the transport to a freshly instantiated `Connection` + The Connection has two goals: + * Interpret incoming CDP packets by reading the JSON object attribute (`id`, `method`, `params` and `sessionId`) + This is done in `Connection:onPacket`. + * Format outgoing CDP packets by writing the right JSON object for command response (`id`, `result` and `sessionId`) and events (`method`, `params` and `sessionId`) + * Redirect CDP packet from/to the right session. + A connection may have more than one session attached to it. + + * instantiate the default session + The session is specific to each target kind and all of them inherit from `cdp/session/Session`. + For example, tabs targets uses `cdp/session/TabSession` and the main browser target uses `cdp/session/MainProcessSession`. + Which session class is used is defined by the Target subclass’ constructor, which pass a session class reference to `cdp/targets/Target:constructor`. + A session is mostly responsible of accommodating the eventual cross process/cross thread aspects of the target. + The code we are currently describing (`cdp/targets/Target:handle`) is running in the parent process. + The session class receive CDP commands from the connection and first try to execute the Domain commands in the parent process. + Then, if the target actually runs in some other context, the session tries to forward this command to this other context, which can be a thread or a process. + Typically, the `cdp/sessions/TabSession` forward the CDP command to the content process where the tab is running. + It also redirects back the command response as well as Domain events from that process back to the parent process in order to + forward them to the connection. + Sessions will be using the `DomainCache` class as a helper to manage a list of Domain implementations in a given context. + +Debugging additional Targets +---------------------------- + +From a given connection you can know about the other potential targets. +You typically do that via `Target.setDiscoverTargets()`, which will emit `Target.targetCreated` events providing a target ID. +You may create a new session for the new target by handing the ID to `Target.attachToTarget()`, which will return a session ID. +"Target" here is a reference to the CDP Domain implemented in `cdp/domains/parent/Target.jsm`. That is different from `cdp/targets/Target` +class which is an implementation detail of the Remote Agent. + +Then, there is two ways to communicate with the other targets: + + * Use `Target.sendMessageToTarget()` and `Target.receivedMessageFromTarget` + You will manually send commands via the `Target.sendMessageToTarget()` command and receive command's response as well as events via `Target.receivedMessageFromTarget`. + In both cases, a session ID attribute is passed in the command or event arguments in order to select which additional target you are communicating with. + + * Use `Target.attachToTarget({ flatten: true })` and include `sessionId` in CDP packets + This requires a special client, which will use the `sessionId` returned by `Target.attachToTarget()` in order to spawn a distinct client instance. + This client will re-use the same WebSocket connection, but every single CDP packet will contain an additional `sessionId` attribute. + This helps distinguish packets which relate to the original target as well as the multiple additional targets you may attach to. + +In both cases, `Target.attachToTarget()` is special as it will spawn `cdp/session/TabSession` for the tab you are attaching to. +This is the codepath creating non-default session. The default session is related to the target you originally connected to, +so that you don't need any ID for this one. When you want to debug more than one target over a single connection +you need additional sessions, which will have a unique ID. +`Target.attachToTarget` will compute this ID and instantiate a new session bound to the given target. +This additional session will be managed by the `Connection` class, which will then redirect CDP packets to the +right session when you are using flatten session. + +Cross Process / Layers +---------------------- + +Because targets may runs in different contexts, the Remote Agent code runs in different processes. +The main and startup code of the Remote Agent code runs in the parent process. +The handling of the command line as well as all the HTTP and WebSocket work is all done in the parent process. +The browser target is also all implemented in the parent process. +But when it comes to a tab target, as the tab runs in the content process, we have to run code there as well. +Let's start from the `cdp/sessions/TabSession` class, which has already been described. +We receive here JSON packets from the WebSocket connection and we are in the parent process. +In this class, we route the messages to the parent process domains first. +If there is no implementation of the domain or the particular method, +we forward the command to a `cdp/session/ContentProcessSession` which runs in the tab's content process. +These two Session classes will interact with each other in order to forward back the returned value +of the method we just called, as well as piping back any event being sent by a Domain implemented in any +of the two processes. + +Organizational chart of all the classes +---------------------------------------- +``` + ┌─────────────────────────────────────────────────┐ + │ │ + 1 ▼ │ + ┌───────────────┐ 1 ┌───────────────┐ 1..n┌───────────────┐ + │ RemoteAgent │──────▶│ HttpServer │◀───────▶│ JsonHandler │ + └───────────────┘ └───────────────┘ 1 └───────────────┘ + │ + │ + │ 1 ┌────────────────┐ 1 + └───────────────▶│ TargetList │◀─┐ + └────────────────┘ │ + │ │ + ▼ 1..n │ + ┌────────────┐ │ + ┌─────────────────│ Target [1]│ │ + │ └────────────┘ │ + │ ▲ 1 │ + ▼ 1..n │ │ + ┌────────────┐ 1..n┌────────────┐ │ + │ Connection │◀─────────▶│ Session [2]│──────┘ + └────────────┘ 1 └────────────┘ + │ 1 ▲ + │ │ + ▼ 1 ▼ 1 +┌────────────────────┐ ┌──────────────┐ 1..n┌────────────┐ +│ WebSocketTransport │ │ DomainCache | │──────────▶│ Domain [3]│ +└────────────────────┘ └──────────────┘ └────────────┘ +``` + [1] Target is inherited by TabTarget and MainProcessTarget. + [2] Session is inherited by TabSession and MainProcessSession. + [3] Domain is inherited by Log, Page, Browser, Target.... i.e. all domain implementations. From both cdp/domains/parent and cdp/domains/content folders. diff --git a/remote/doc/cdp/PuppeteerVendor.md b/remote/doc/cdp/PuppeteerVendor.md new file mode 100644 index 0000000000..4de3298fdc --- /dev/null +++ b/remote/doc/cdp/PuppeteerVendor.md @@ -0,0 +1,83 @@ +Vendoring Puppeteer +=================== + +As mentioned in the chapter on [Testing], we run the full [Puppeteer +test suite] on try. These tests are vendored in central under +_remote/test/puppeteer/_ and we have a script to pull in upstream changes. + +We periodically perform a manual two-way sync. Below is an outline of the +process interspersed with some tips. + +1. Clone the Puppeteer git repository and checkout the release tag you want + to vendor into mozilla-central. + + % git checkout tags/v10.0 -b sync-v10.0 + +2. Apply any recent changes in `remote/test/puppeteer` to the Puppeteer branch + created above. + + You might want to [install the project] at this point and make sure unit + tests pass. Check the project's `package.json` for relevant testing commands. + + You should use this as basis for a PR to the Puppeteer project once you are + satisfied that the two-way sync will be successful in mozilla-central. See + their [contributing.md]. + + Typically, the changes we push to Puppeteer include unskipping newly passing + unit tests for Firefox along with minor fixes to the tests or + to Firefox-specific browser-fetching and launch code. + + Be sure to [run tests against both Chromium and Firefox] in the Puppeteer + repo. You can specify your local Firefox build when you do so: + + % BINARY=<path-to-objdir-binary> npm run funit + +3. Now back in mozilla-central, you can run the following mach command to + copy over the Puppeteer branch you just prepared. The mach command has + flags to specify a local or remote repository as well as a commit. + + % ./mach remote vendor-puppeteer + + By default, this command also installs the newly-pulled Puppeteer package + in order to generate a new `package-lock.json` file for the purpose of + pinning Puppeteer dependencies for our CI. There is a `--no-install` option + if you want to skip this step; for example, if you want to run installation + separately at a later point. + +4. Go through the changes under `remote/test/puppeteer/test` and [unskip] any + newly-skipped tests (e.g. change `itFailsFirefox` to `it`). + A mass-change with `awk` might be useful here. + + Why do we do this? The Puppeteer team runs their unit tests against Firefox + in their CI with many tests skipped. In contrast, we leave these tests + unskipped in Mozilla CI and track test expectation metadata + in [puppeteer-expected.json] instead. + +5. Use `./mach puppeteer-test` (see [Testing]) to run Puppeteer tests against + both Chromium and Firefox in headless mode. Again, only running a subset of + tests against Firefox is fine -- at this point you just want to check that + the typescript compiles and the browser binaries are launched successfully. + +6. Next you want to update the test expectation metadata: test results might + have changed, tests may have been renamed, removed or added. The + easiest way to do this is to run the Puppeteer test job on try + (see [Testing]). You will find the new test metadata as an artifact on that + job and you can copy it over into your sync patch if it looks reasonable. + + Examine the job logs and makes sure the run didn't get interrupted early + by a crash or a hang, especially if you see a lot of + `TEST-UNEXPECTED-MISSING` in the Treeherder Failure Summary. You might need + to add new test skips (especially for these tests as designed for Chrome only) + or fix some new bug in the unit tests. This is the fun part. + +7. Once you are happy with the metadata and are ready to submit the sync patch + up for review, run the Puppeteer test job on try again with `--rebuild 10` + to check for stability. + +[Testing]: ../Testing.md +[Puppeteer test suite]: https://github.com/GoogleChrome/puppeteer/tree/master/test +[install the project]: https://github.com/puppeteer/puppeteer/blob/main/docs/contributing.md#getting-code +[run tests against both Chromium and Firefox]: https://github.com/puppeteer/puppeteer/blob/main/test/README.md#running-tests +[puppeteer-expected.json]: https://searchfox.org/mozilla-central/source/remote/test/puppeteer-expected.json +[contributing.md]: https://github.com/puppeteer/puppeteer/blob/main/docs/contributing.md +[unskip]: https://github.com/puppeteer/puppeteer/blob/main/test/README.md#skipping-tests-in-specific-conditions diff --git a/remote/doc/cdp/RequiredPreferences.md b/remote/doc/cdp/RequiredPreferences.md new file mode 100644 index 0000000000..26b4b54d90 --- /dev/null +++ b/remote/doc/cdp/RequiredPreferences.md @@ -0,0 +1,14 @@ +Required Preferences for Fission +================================ + +Fission (site isolation for Firefox) introduced some architectural changes that are incompatible with our CDP implementation. To keep using CDP for Firefox, make sure the following preferences are set in the profile before starting Firefox with `--remote-debugging-port`: + + * `fission.bfcacheInParent` should be set to `false`. + + * `fission.webContentIsolationStrategy` should be set to `0`. + +Without those preferences, expect issues related to navigation in several domains (Page, Runtime, ...). + +Third party tools relying on CDP such as Puppeteer ensure that those preferences are correctly set before starting Firefox. + +The work to lift those restrictions is tracked in [Bug 1732263](https://bugzilla.mozilla.org/show_bug.cgi?id=1732263) and [Bug 1706353](https://bugzilla.mozilla.org/show_bug.cgi?id=1706353). diff --git a/remote/doc/cdp/Usage.md b/remote/doc/cdp/Usage.md new file mode 100644 index 0000000000..747bcb8fe2 --- /dev/null +++ b/remote/doc/cdp/Usage.md @@ -0,0 +1,62 @@ +Usage +===== + +When using the CDP-based Remote Agent in Firefox, there are +three different programs/components running simultaneously: + + * the __client__, being the out-of-process script or library + (such as Puppeteer) or web inspector frontend you use to control + and retrieve information out of Firefox; + + * the __agent__ that the client connects to which is an HTTPD living + inside Firefox, facilitating communication between clients + and targets; + + * and the __target__, which is the web document being debugging. + +Since Firefox 86 the Remote Agent ships in all Firefox releases by default. + +To check if your Firefox binary has the Remote Agent enabled, you +can look in its help message for this: + + % ./firefox -h + … + --remote-debugging-port [<port>] Start the Firefox Remote Agent, which is + a low-level debugging interface based on the CDP protocol. + Defaults to listen on localhost:9222. + … + +When used, the Remote Agent will start an HTTP server and print a +message on stderr with the location of the main target’s WebSocket +listener: + + % firefox --remote-debugging-port + DevTools listening on ws://localhost:9222/devtools/browser/7b4e84a4-597f-4839-ac6d-c9e86d16fb83 + +`--remote-debugging-port` takes an optional port as input: + + [<port>] + +You can use this to instruct the Remote Agent to bind to a particular +port on your system. port is optional, +which means `firefox --remote-debugging-port` will bind the HTTPD to +the default `localhost:9222`. + +If port has been specified the default port will be overridden: + + % firefox --remote-debugging-port 9989 + DevTools listening on ws://localhost:9989/devtools/browser/b49481af-8ad3-9b4d-b1bf-bb0cdb9a0620 + +When you ask the Remote Agent to listen on port 0, +the system will atomically allocate an arbitrary free port: + + % firefox --remote-debugging-port 0 + DevTools listening on ws://localhost:59982/devtools/browser/a12b22a9-1b8b-954a-b81f-bd31552d3f1c + +Allocating an atomic port can be useful if you want to avoid race +conditions. The atomically allocated port will be somewhere in the +ephemeral port range, which varies depending on your system and +system configuration, but is always guaranteed to be free thus +eliminating the risk of binding to a port that is already in use. + +[Firefox Nightly]: https://www.mozilla.org/firefox/channel/desktop/#nightly diff --git a/remote/doc/cdp/index.rst b/remote/doc/cdp/index.rst new file mode 100644 index 0000000000..4892c5c457 --- /dev/null +++ b/remote/doc/cdp/index.rst @@ -0,0 +1,27 @@ +===================== +Remote Protocol (CDP) +===================== + +The Firefox **remote protocol (CDP)** is a low-level debugging interface +you can use to inspect the state and control execution of documents +running in web content, instrument the browser in interesting ways, +simulate user interaction for automation purposes, and for subscribing +to updates in the browser such as network- or console logs. + +It complements the existing Firefox Developer Tools :ref:`Remote Debugging +Protocol <Remote Debugging Protocol>` (RDP) by implementing a subset of the +`Chrome DevTools Protocol`_ (CDP). + +To use Firefox remote protocol with Fission, CDP client authors should read the +`Required Preferences`_ page. + +.. _Chrome DevTools Protocol: https://chromedevtools.github.io/devtools-protocol/ +.. _Required Preferences: /remote/cdp/RequiredPreferences.html + +.. toctree:: + :maxdepth: 1 + + Usage.md + Architecture.md + PuppeteerVendor.md + RequiredPreferences.md diff --git a/remote/doc/index.rst b/remote/doc/index.rst new file mode 100644 index 0000000000..64bb0a8d71 --- /dev/null +++ b/remote/doc/index.rst @@ -0,0 +1,94 @@ +================ +Remote Protocols +================ + +Firefox supports several remote protocols, which allow to inspect and control +the browser, usually for automation purposes: + +* :ref:`marionette-header` + +* :ref:`remote-protocol-cdp-header` + +* :ref:`webdriver-bidi-header` + + +Common documentation +==================== + +The following documentation pages apply to all remote protocols + +.. toctree:: + :maxdepth: 1 + + Building.md + Debugging.md + Prefs.md + Testing.md + CodeStyle.md + Security.md + +Protocols +========= + +.. _marionette-header: + +Marionette +---------- + +Marionette is used both by internal tools and testing solutions, but also by +geckodriver to implement the `WebDriver (HTTP) specification`_. The documentation +for Marionette can be found under `testing/marionette`_. + +.. _WebDriver (HTTP) specification: https://w3c.github.io/webdriver/ +.. _testing/marionette: /testing/marionette + + +.. _remote-protocol-cdp-header: + +Remote Protocol (CDP) +--------------------- + +Firefox implements a subset of the `Chrome DevTools Protocol`_ (CDP) in order to +support third party automation tools such as `puppeteer`. The documentation for +the remote protocol (CDP) implement can be found at `remote/cdp`_. + +.. _Chrome DevTools Protocol: https://chromedevtools.github.io/devtools-protocol/ +.. _remote/cdp: cdp/ + + +.. _webdriver-bidi-header: + +WebDriver BiDi +-------------- + +`The WebDriver BiDi specification <https://w3c.github.io/webdriver-bidi>`_ +extends WebDriver HTTP to add bidirectional communication. Dedicated +documentation will be added as the Firefox implementation makes progress. + +Architecture +============ + +Message Handler +--------------- + +The documentation for the framework used to build WebDriver BiDi modules can be +found at `remote/messagehandler`_. + +.. _remote/messagehandler: messagehandler/ + + +Bugs +==== + +Bugs are tracked under the `Remote Protocol product`_. + +.. _Remote Protocol product: https://bugzilla.mozilla.org/describecomponents.cgi?product=Remote%20Protocol + + +Communication +============= + +See `Communication`_ on `our project wiki`_. + +.. _Communication: https://wiki.mozilla.org/Remote#Communication +.. _our project wiki: https://wiki.mozilla.org/Remote diff --git a/remote/doc/marionette/Building.md b/remote/doc/marionette/Building.md new file mode 100644 index 0000000000..1f55697d2e --- /dev/null +++ b/remote/doc/marionette/Building.md @@ -0,0 +1,56 @@ +# Building + +Marionette is built into Firefox by default and ships in the official +Firefox binary. As Marionette is written in [XPCOM] flavoured +JavaScript, you may choose to rely on so called [artifact builds], +which will download pre-compiled Firefox blobs to your computer. +This means you don’t have to compile Firefox locally, but does +come at the cost of having a good internet connection. To enable +[artifact builds] you may choose ‘Firefox for Desktop Artifact +Mode’ when bootstrapping. + +Once you have a clone of [mozilla-unified], you can set up your +development environment by running this command and following the +on-screen instructions: + + % ./mach bootstrap + +When you're getting asked to choose the version of Firefox you want to build, +you may want to consider choosing "Firefox for Desktop Artifact Mode". This +significantly reduces the time it takes to build Firefox on your machine +(from 30+ minutes to just 1-2 minutes) if you have a fast internet connection. + +To perform a regular build, simply do: + + % ./mach build + +You can clean out the objdir using this command: + + % ./mach clobber + +Occasionally a clean build will be required after you fetch the +latest changes from mozilla-central. You will find that the the +build will error when this is the case. To automatically do clean +builds when this happens you may optionally add this line to the +[mozconfig] file in your top source directory: + + mk_add_options AUTOCLOBBER=1 + +If you compile Firefox frequently you will also want to enable +[ccache] and [sccache] if you develop on a macOS or Linux system: + + mk_add_options 'export RUSTC_WRAPPER=sccache' + mk_add_options 'export CCACHE_CPP2=yes' + ac_add_options --with-ccache + +You may also opt out of building all the WebDriver specific components +(Marionette, and the [Remote Agent]) by setting the following flag: + + ac_add_options --disable-webdriver + +[mozilla-unified]: https://mozilla-version-control-tools.readthedocs.io/en/latest/hgmozilla/unifiedrepo.html +[artifact builds]: /contributing/build/artifact_builds.rst +[mozconfig]: /build/buildsystem/mozconfigs.rst +[ccache]: https://ccache.samba.org/ +[sccache]: https://github.com/mozilla/sccache +[Remote Agent]: /remote/index.rst diff --git a/remote/doc/marionette/CodeStyle.md b/remote/doc/marionette/CodeStyle.md new file mode 100644 index 0000000000..ad6f03092f --- /dev/null +++ b/remote/doc/marionette/CodeStyle.md @@ -0,0 +1,229 @@ +# Style guide + +Like other projects, we also have some guidelines to keep to the code. +For the overall Marionette project, a few rough rules are: + +* Make your code readable and sensible, and don’t try to be + clever. Prefer simple and easy solutions over more convoluted + and foreign syntax. + +* Fixing style violations whilst working on a real change as a + preparatory clean-up step is good, but otherwise avoid useless + code churn for the sake of conforming to the style guide. + +* Code is mutable and not written in stone. Nothing that + is checked in is sacred and we encourage change to make + remote/marionette a pleasant ecosystem to work in. + +## JavaScript + +Marionette is written in JavaScript and ships +as part of Firefox. We have access to all the latest ECMAScript +features currently in development, usually before it ships in the +wild and we try to make use of new features when appropriate, +especially when they move us off legacy internal replacements +(such as Promise.jsm and Task.jsm). + +One of the peculiarities of working on JavaScript code that ships as +part of a runtime platform is, that unlike in a regular web document, +we share a single global state with the rest of Firefox. This means +we have to be responsible and not leak resources unnecessarily. + +JS code in Gecko is organised into _modules_ carrying _.js_ or _.jsm_ +file extensions. Depending on the area of Gecko you’re working on, +you may find they have different techniques for exporting symbols, +varying indentation and code style, as well as varying linting +requirements. + +To export symbols to other Marionette modules, remember to assign +your exported symbols to the shared global `this`: + + const EXPORTED_SYMBOLS = ["PollPromise", "TimedPromise"]; + +When importing symbols in Marionette code, try to be specific about +what you need: + + const { TimedPromise } = ChromeUtils.import( + "chrome://remote/content/marionette/sync.js" + ); + +We prefer object assignment shorthands when redefining names, +for example when you use functionality from the `Components` global: + + const {classes: Cc, interfaces: Ci, utils: Cu, results: Cr} = Components; + +When using symbols by their own name, the assignment name can be +omitted: + + const {TYPE_ONE_SHOT, TYPE_REPEATING_SLACK} = Ci.nsITimer; + +In addition to the default [Mozilla eslint rules], we have [our +own specialisations] that are stricter and enforce more security. +A few notable examples are that we disallow fallthrough `case` +statements unless they are explicitly grouped together: + + switch (x) { + case "foo": + doSomething(); + + case "bar": // <-- disallowed! + doSomethingElse(); + break; + + case "baz": + case "bah": // <-- allowed (-: + doCrazyThings(); + } + +We disallow the use of `var`, for which we always prefer `let` and +`const` as replacements. Do be aware that `const` does not mean +that the variable is immutable: just that it cannot be reassigned. +We require all lines to end with semicolons, disallow construction +of plain `new Object()`, require variable names to be camel-cased, +and complain about unused variables. + +For purely aesthetic reasons we indent our code with two spaces, +which includes switch-statement `case`s, and limit the maximum +line length to 78 columns. When you need to wrap a statement to +the next line, the second line is indented with four spaces, like this: + + throw new TypeError( + "Expected an element or WindowProxy, " + + pprint`got: ${el}`); + +This is not normally something you have to think to deeply about as +it is enforced by the [linter]. The linter also has an automatic +mode that fixes and formats certain classes of style violations. + +If you find yourself struggling to fit a long statement on one line, +this is usually an indication that it is too long and should be +split into multiple lines. This is also a helpful tip to make the +code easier to read. Assigning transitive values to descriptive +variable names can serve as self-documentation: + + let location = event.target.documentURI || event.target.location.href; + log.debug(`Received DOM event ${event.type} for ${location}`); + +On the topic of variable naming the opinions are as many as programmers +writing code, but it is often helpful to keep the input and output +arguments to functions descriptive (longer), and let transitive +internal values to be described more succinctly: + + /** Prettifies instance of Error and its stacktrace to a string. */ + function stringify(error) { + try { + let s = error.toString(); + if ("stack" in error) { + s += "\n" + error.stack; + } + return s; + } catch (e) { + return "<unprintable error>"; + } + } + +When we can, we try to extract the relevant object properties in +the arguments to an event handler or a function: + + const responseListener = ({name, target, json, data}) => { … }; + +Instead of: + + const responseListener = msg => { + let name = msg.name; + let target = msg.target; + let json = msg.json; + let data = msg.data; + … + }; + +All source files should have `"use strict";` as the first directive +so that the file is parsed in [strict mode]. + +Every source code file that ships as part of the Firefox bundle +must also have a [copying header], such as this: + + /* This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this file, + * You can obtain one at http://mozilla.org/MPL/2.0/. */ + +New xpcshell test files _should not_ have a license header as all +new Mozilla tests should be in the [public domain] so that they can +easily be shared with other browser vendors. We want to re-license +existing tests covered by the [MPL] so that they can be shared. +We very much welcome your help in doing version control archeology +to make this happen! + +The practical details of working on the Marionette code is outlined +in [Contributing.md], but generally you do not have to re-build +Firefox when changing code. Any change to remote/marionette/*.js +will be picked up on restarting Firefox. The only notable exception +is remote/components/Marionette.jsm, which does require +a re-build. + +[strict mode]: https://developer.mozilla.org/docs/Web/JavaScript/Reference/Strict_mode +[Mozilla eslint rules]: https://searchfox.org/mozilla-central/source/.eslintrc.js +[our own specialisations]: https://searchfox.org/mozilla-central/source/remote/marionette/.eslintrc.js +[linter]: #linting +[copying header]: https://www.mozilla.org/en-US/MPL/headers/ +[public domain]: https://creativecommons.org/publicdomain/zero/1.0/ +[MPL]: https://www.mozilla.org/en-US/MPL/2.0/ +[Contributing.md]: ./Contributing.md + +## Python + +TODO + +## Documentation + +We keep our documentation in-tree under [remote/doc/marionette] +and [testing/geckodriver/doc]. Updates and minor changes to +documentation should ideally not be scrutinised to the same degree +as code changes to encourage frequent updates so that the documentation +does not go stale. To that end, documentation changes with `r=me` +from module peers are permitted. + +Use fmt(1) or an equivalent editor specific mechanism (such as Meta-Q +in Emacs) to format paragraphs at a maximum width of 75 columns +with a goal of roughly 65. This is equivalent to `fmt -w 75 -g 65`, +which happens to be the default on BSD and macOS. + +We endeavour to document all _public APIs_ of the Marionette component. +These include public functions—or command implementations—on +the `GeckoDriver` class, as well as all exported symbols from +other modules. Documentation for non-exported symbols is not required. + +[remote/doc/marionette]: https://searchfox.org/mozilla-central/source/remote/marionette/doc +[testing/geckodriver/doc]: https://searchfox.org/mozilla-central/source/testing/geckodriver/doc + +## Linting + +Marionette consists mostly of JavaScript (server) and Python (client, +harness, test runner) code. We lint our code with [mozlint], +which harmonises the output from [eslint] and [flake8]. + +To run the linter with a sensible output: + + % ./mach lint -funix remote/marionette + +For certain classes of style violations the eslint linter has +an automatic mode for fixing and formatting your code. This is +particularly useful to keep to whitespace and indentation rules: + + % ./mach eslint --fix remote/marionette + +The linter is also run as a try job (shorthand `ES`) which means +any style violations will automatically block a patch from landing +(if using Autoland) or cause your changeset to be backed out (if +landing directly on mozilla-inbound). + +If you use git(1) you can [enable automatic linting] before you push +to a remote through a pre-push (or pre-commit) hook. This will +run the linters on the changed files before a push and abort if +there are any problems. This is convenient for avoiding a try run +failing due to a stupid linting issue. + +[mozlint]: /code-quality/lint/mozlint.rst +[eslint]: /code-quality/lint/linters/eslint.rst +[flake8]: /code-quality/lint/linters/flake8.rst +[enable automatic linting]: /code-quality/lint/usage.html#using-a-vcs-hook diff --git a/remote/doc/marionette/Contributing.md b/remote/doc/marionette/Contributing.md new file mode 100644 index 0000000000..13fe137905 --- /dev/null +++ b/remote/doc/marionette/Contributing.md @@ -0,0 +1,69 @@ +# Contributing + +If you are new to open source or to Mozilla, you might like this +[tutorial for new Marionette contributors](NewContributors.md). + +We are delighted that you want to help improve Marionette! +‘Marionette’ means different a few different things, depending +on who you talk to, but the overall scope of the project involves +these components: + +* [_Marionette_] is a Firefox remote protocol to communicate with, + instrument, and control Gecko-based applications such as Firefox + and Firefox for mobile. It is built in to the application and + written in JavaScript. + + It serves as the backend for the geckodriver WebDriver implementation, + and is used in the context of Firefox UI tests, reftesting, + Web Platform Tests, test harness bootstrapping, and in many + other far-reaching places where browser instrumentation is required. + +* [_geckodriver_] provides the HTTP API described by the [WebDriver + protocol] to communicate with Gecko-based applications such as + Firefox and Firefox for mobile. It is a standalone executable + written in Rust, and can be used with compatible W3C WebDriver clients. + +* [_webdriver_] is a Rust crate providing interfaces, traits + and types, errors, type- and bounds checks, and JSON marshaling + for correctly parsing and emitting the [WebDriver protocol]. + +By participating in this project, you agree to abide by the Mozilla +[Community Participation Guidelines]. Here are some guidelines +for contributing high-quality and actionable bugs and code. + +[_Marionette_]: ./index.rst +[_geckodriver_]: /testing/geckodriver/index.rst +[_webdriver_]: https://searchfox.org/mozilla-central/source/testing/webdriver/README.md +[WebDriver protocol]: https://w3c.github.io/webdriver/webdriver-spec.html#protocol +[Community Participation Guidelines]: https://www.mozilla.org/en-US/about/governance/policies/participation/ + +## Writing code + +Because there are many moving parts involved remote controlling +a web browser, it can be challenging to a new contributor to know +where to start. Please don’t hesitate to [ask questions]! + +The canonical source code repository is [mozilla-central]. Bugs are +filed in the [Testing :: Marionette] component on Bugzilla. We also +have a curated set of [good first bugs] you may consider attempting first. + +We have collected a lot of good advice for working on Marionette +code in our [code style document], which we highly recommend you read. + +[ask questions]: ./index.rst#communication +[mozilla-central]: https://searchfox.org/mozilla-central/source/remote/marionette/ +[Testing :: Marionette]: https://bugzilla.mozilla.org/buglist.cgi?resolution=---&component=Marionette +[good first bugs]: https://codetribute.mozilla.org/projects/automation?project%3DMarionette +[code style document]: CodeStyle.md + +## Next steps + +* [Building](Building.md) +* [Debugging](Debugging.md) +* [Testing](Testing.md) +* [Patching](Patches.md) + +## Other resources + +* [Code style](CodeStyle.md) +* [New Contributor Tutorial](NewContributors.md) diff --git a/remote/doc/marionette/Debugging.md b/remote/doc/marionette/Debugging.md new file mode 100644 index 0000000000..682d7adec5 --- /dev/null +++ b/remote/doc/marionette/Debugging.md @@ -0,0 +1,86 @@ +# Debugging + +## Redirecting the Gecko output + +The most common way to debug Marionette, as well as chrome code in +general, is to use `dump()` to print a string to stdout. In Firefox, +this log output normally ends up in the gecko.log file in your current +working directory. With Fennec it can be inspected using `adb logcat`. + +`mach marionette-test` takes a `--gecko-log` option which lets +you redirect this output stream. This is convenient if you want to +“merge” the test harness output with the stdout from the browser. +Per Unix conventions you can use `-` (dash) to have Firefox write +its log to stdout instead of file: + + % ./mach marionette-test --gecko-log - + +It is common to use this in conjunction with an option to increase +the Marionette log level: + + % ./mach test --gecko-log - -vv TEST + +A single `-v` enables debug logging, and a double `-vv` enables +trace logging. + +This debugging technique can be particularly effective when combined +with using [pdb] in the Python client or the JS remote debugger +that is described below. + +[pdb]: https://docs.python.org/3/library/pdb.html + +## JavaScript debugger + +You can attach the [Browser Toolbox] JavaScript debugger to the +Marionette server using the `--jsdebugger` flag. This enables you +to introspect and set breakpoints in Gecko chrome code, which is a +more powerful debugging technique than using `dump()` or `console.log()`. + +To automatically open the JS debugger for `Mn` tests: + + % ./mach marionette-test --jsdebugger + +It will prompt you when to start to allow you time to set your +breakpoints. It will also prompt you between each test. + +You can also use the `debugger;` statement anywhere in chrome code +to add a breakpoint. In this example, a breakpoint will be added +whenever the `WebDriver:GetPageSource` command is called: + + GeckoDriver.prototype.getPageSource = async function() { + debugger; + … + } + +To be prompted at the start of the test run or between tests, +you can set the `marionette.debugging.clicktostart` preference to +`true` this way: + + % ./mach marionette-test --setpref='marionette.debugging.clicktostart=true' --jsdebugger + +For reference, below is the list of preferences that enables the +chrome debugger for Marionette. These are all set implicitly when +`--jsdebugger` is passed to mach. In non-official builds, which +are the default when built using `./mach build`, you will find that +the chrome debugger won’t prompt for connection and will allow +remote connections. + +* `devtools.browsertoolbox.panel` -> `jsdebugger` + + Selects the Debugger panel by default. + +* `devtools.chrome.enabled` → true + + Enables debugging of chrome code. + +* `devtools.debugger.prompt-connection` → false + + Controls the remote connection prompt. Note that this will + automatically expose your Firefox instance to localhost. + +* `devtools.debugger.remote-enabled` → true + + Allows a remote debugger to connect, which is necessary for + debugging chrome code. + +[Browser Toolbox]: /devtools-user/browser_toolbox/index.rst diff --git a/remote/doc/marionette/Intro.md b/remote/doc/marionette/Intro.md new file mode 100644 index 0000000000..8c34eef700 --- /dev/null +++ b/remote/doc/marionette/Intro.md @@ -0,0 +1,70 @@ +# Introduction to Marionette + +Marionette is an automation driver for Mozilla's Gecko engine. +It can remotely control either the UI or the internal JavaScript of +a Gecko platform, such as Firefox. It can control both the chrome +(i.e. menus and functions) or the content (the webpage loaded inside +the browsing context), giving a high level of control and ability +to replicate user actions. In addition to performing actions on the +browser, Marionette can also read the properties and attributes of +the DOM. + +If this sounds similar to [Selenium/WebDriver] then you're +correct! Marionette shares much of the same ethos and API as +Selenium/WebDriver, with additional commands to interact with +Gecko's chrome interface. Its goal is to replicate what Selenium +does for web content: to enable the tester to have the ability to +send commands to remotely control a user agent. + +[Selenium/WebDriver]: https://dvcs.w3.org/hg/webdriver/raw-file/tip/webdriver-spec.html + +## How does it work? + +Marionette consists of two parts: a server which takes requests and +executes them in Gecko, and a client. The client sends commands to +the server and the server executes the command inside the browser. + +## When would I use it? + +If you want to perform UI tests with browser chrome or content, +Marionette is the tool you're looking for! You can use it to +control either web content, or Firefox itself. + +A test engineer would typically import the Marionette client package +into their test framework, import the classes and use the class +functions and methods to control the browser. After controlling +the browser, Marionette can be used to return information about +the state of the browser which can then be used to validate that +the action was performed correctly. + +## Using Marionette + +Marionette combines a gecko component (the Marionette server) with an +outside component (the Marionette client), which drives the tests. +The Marionette server ships with Firefox, and to use it you will +need to download a Marionette client or use the in-tree client. + +* [Download and setup the Python client for Marionette][1] +* [Run Tests with Python][2] – How to run tests using the + Python client +* You might want to experiment with [using Marionette interactively + at a Python command prompt][2] +* Start [writing and running][3] tests +* Tips on [debugging][4] Marionette code +* [Download and setup the Marionette JS client][5] +* [Protocol definition][6] + +[1]: /python/marionette_driver.rst +[2]: /python/marionette_driver.rst +[3]: PythonTests.md +[4]: Debugging.md +[5]: https://github.com/mozilla-b2g/marionette_js_client +[6]: Protocol.md + +## Bugs + +Please file any bugs you may find in the `Testing :: Marionette` +component in Bugzilla. You can view a [list of current bugs] +to see if your problem is already being addressed. + +[list of current bugs]: https://bugzilla.mozilla.org/buglist.cgi?product=Testing&component=Marionette diff --git a/remote/doc/marionette/NewContributors.md b/remote/doc/marionette/NewContributors.md new file mode 100644 index 0000000000..06051090f4 --- /dev/null +++ b/remote/doc/marionette/NewContributors.md @@ -0,0 +1,84 @@ +# New contributors + +This page is aimed at people who are new to Mozilla and want to contribute +to Mozilla source code related to Marionette Python tests, WebDriver +spec tests and related test harnesses and tools. Mozilla has both +git and Mercurial repositories, but this guide only describes Mercurial. + +If you run into issues or have doubts, check out the [Resources](#resources) +section below and **don't hesitate to ask questions**. :) The goal of these +steps is to make sure you have the basics of your development environment +working. Once you do, we can get you started with working on an +actual bug, yay! + +## Accounts, communication + + 1. Set up a [Bugzilla] account (and, if you like, a [Mozillians] profile). + Please include your Element nickname in both of these accounts so we can work + with you more easily. For example, Eve Smith would set the Bugzilla name + to "Eve Smith (:esmith)", where "esmith" is the Element nick. + + 2. For a direct communication with us it will be beneficial to setup [Element]. + Make sure to also register your nickname as described in the linked document. + + 3. Join our [#webdriver:mozilla.org] channel, and introduce yourself to the + team. :whimboo, :jdescottes, and :jgraham are all familiar with Marionette. + We're nice, I promise, but we might not answer right away due to different + time zones, time off, etc. So please be patient. + + 4. When you want to ask a question on Element, just go ahead an ask it even if + no one appears to be around/responding. + Provide lots of detail so that we have a better chance of helping you. + If you don't get an answer right away, check again in a few hours -- + someone may have answered you in the mean time. + + 5. If you're having trouble reaching us over Element, you are welcome to send an + email to our [mailing list](index.html#communication) instead. It's a good + idea to include your Element nick in your email message. + +[Element]: https://chat.mozilla.org +[#webdriver:mozilla.org]: https://chat.mozilla.org/#/room/#webdriver:mozilla.org +[Bugzilla]: https://bugzilla.mozilla.org/ +[Mozillians]: https://mozillians.org/ + +## Getting the code, running tests + +Follow the documentation on [Contributing](Contributing.md) to get a sense of +our projects, and which is of most interest for you. You will also learn how to +get the Firefox source code, build your custom Firefox build, and how to run the +tests. + +## Work on bugs and get code review + +Once you are familiar with the code of the test harnesses, and the tests you might +want to start with your first contribution. The necessary steps to submit and verify +your patches are laid out in [Patches](Patches.md). + +## Resources + +* Search Mozilla's code repository with searchfox to find the [code for + Marionette] and the [Marionette client/harness]. + +* Another [guide for new contributors]. It has not been updated in a long + time but it's a good general resource if you ever get stuck on something. + The most relevant sections to you are about Bugzilla, Mercurial, Python and the + Development Process. + +* [Mercurial for Mozillians] + +* More general resources are available in this little [guide] :maja_zf wrote + in 2015 to help a student get started with open source contributions. + +* Textbook about general open source practices: [Practical Open Source Software Exploration] + +* If you'd rather use git instead of hg, see [git workflow for + Gecko development] and/or [this blog post by :ato]. + +[code for Marionette]: https://searchfox.org/mozilla-central/source/remote/marionette/ +[Marionette client/harness]: https://searchfox.org/mozilla-central/source/testing/marionette/ +[guide for new contributors]: https://ateam-bootcamp.readthedocs.org/en/latest/guide/index.html#new-contributor-guide +[Mercurial for Mozillians]: https://mozilla-version-control-tools.readthedocs.org/en/latest/hgmozilla/index.html +[guide]: https://gist.github.com/mjzffr/d2adef328a416081f543 +[Practical Open Source Software Exploration]: https://quaid.fedorapeople.org/TOS/Practical_Open_Source_Software_Exploration/html/index.html +[git workflow for Gecko development]: https://github.com/glandium/git-cinnabar/wiki/Mozilla:-A-git-workflow-for-Gecko-development +[this blog post by :ato]: https://sny.no/2016/03/geckogit diff --git a/remote/doc/marionette/Patches.md b/remote/doc/marionette/Patches.md new file mode 100644 index 0000000000..acccb7c45f --- /dev/null +++ b/remote/doc/marionette/Patches.md @@ -0,0 +1,32 @@ +# Submitting patches + +You can submit patches by using [Phabricator]. Walk through its documentation +in how to set it up, and uploading patches for review. Don't worry about which +person to select for reviewing your code. It will be done automatically. + +Please also make sure to follow the [commit creation guidelines]. + +Once you have contributed a couple of patches, we are happy to +sponsor you in [becoming a Mozilla committer]. When you have been +granted commit access level 1 you will have permission to use the +[Firefox CI] to trigger your own “try runs” to test your changes. + +You can use the `remote-protocol` [try preset]: + + mach try --preset remote-protocol + +This preset will schedule tests related to the Remote Protocol component on +various platforms. You can reduce the number of tasks by filtering on platforms +(e.g. linux) or build type (e.g. opt): + + mach try --preset remote-protocol -xq "'linux 'opt" + +But you can also schedule tests by selecting relevant jobs yourself: + + mach try fuzzy + +[Phabricator]: https://moz-conduit.readthedocs.io/en/latest/phabricator-user.html +[commit creation guidelines]: https://mozilla-version-control-tools.readthedocs.io/en/latest/devguide/contributing.html#submitting-patches-for-review +[becoming a Mozilla committer]: https://www.mozilla.org/en-US/about/governance/policies/commit/ +[Firefox CI]: https://treeherder.mozilla.org/ +[try preset]: /tools/try/presets diff --git a/remote/doc/marionette/Prefs.md b/remote/doc/marionette/Prefs.md new file mode 100644 index 0000000000..2f464054c3 --- /dev/null +++ b/remote/doc/marionette/Prefs.md @@ -0,0 +1,24 @@ +# Preferences + +There are a couple of [Remote Agent preferences] associated with the Gecko remote +protocol. Those listed below are additional ones uniquely used for Marionette. + +[Remote Agent preferences]: /remote/Prefs.md + +## `marionette.debugging.clicktostart` + +Delay server startup until a modal dialogue has been clicked to +allow time for user to set breakpoints in the [Browser Toolbox]. + +[Browser Toolbox]: /devtools-user/browser_toolbox/index.rst + +## `marionette.port` + +Defines the port on which the Marionette server will listen. Defaults +to port 2828. + +This can be set to 0 to have the system atomically allocate a free +port, which can be useful when running multiple Marionette servers +on the same system. The effective port is written to the user +preference file when the server has started and is also logged to +stdout. diff --git a/remote/doc/marionette/Protocol.md b/remote/doc/marionette/Protocol.md new file mode 100644 index 0000000000..cd42c453b6 --- /dev/null +++ b/remote/doc/marionette/Protocol.md @@ -0,0 +1,114 @@ +# Protocol + +Marionette provides an asynchronous, parallel pipelining user-facing +interface. Message sequencing limits chances of payload race +conditions and provides a uniform way in which payloads are serialised. + +Clients that deliver a blocking WebDriver interface are still +expected to not send further command requests before the response +from the last command has come back, but if they still happen to do +so because of programming error, no harm will be done. This guards +against [mixing up responses]. + +Schematic flow of messages: + + client server + | | + msgid=1 |----------->| + | command | + | | + msgid=2 |<-----------| + | command | + | | + msgid=2 |----------->| + | response | + | | + msgid=1 |<-----------| + | response | + | | + +The protocol consists of a `command` message and the corresponding +`response` message. A `response` message must always be sent in +reply to a `command` message. + +This means that the server implementation does not need to send +the reply precisely in the order of the received commands: if it +receives multiple messages, the server may even reply in random order. +It is therefore strongly advised that clients take this into account +when imlpementing the client end of this wire protocol. + +This is required for pipelining messages. On the server side, +some functions are fast, and some less so. If the server must +reply in order, the slow functions delay the other replies even if +its execution is already completed. + +[mixing up responses]: https://bugzil.la/1207125 + +## Command + +The request, or `command` message, is a four element JSON Array as shown +below, that may originate from either the client- or server remote ends: + + [type, message ID, command, parameters] + +* _type_ must be 0 (integer). This indicates that the message + is a `command`. + +* _message ID_ is a 32-bit unsigned integer. This number is + used as a sequencing number that uniquely identifies a pair of + `command` and `response` messages. The other remote part will + reply with a corresponding `response` with the same message ID. + +* _command_ is a string identifying the RPC method or command + to execute. + +* _parameters_ is an arbitrary JSON serialisable object. + +## Response + +The response message is also a four element array as shown below, +and must always be sent after receiving a `command`: + + [type, message ID, error, result] + +* _type_ must be 1 (integer). This indicates that the message is a + `response`. + +* _message ID_ is a 32-bit unsigned integer. This corresponds + to the `command`’s message ID. + +* _error_ is null if the command executed correctly. If the + error occurred on the server-side, then this is an [error] object. + +* _result_ is the result object from executing the `command`, if + it executed correctly. If an error occurred on the server-side, + this field is null. + +The structure of the result field can vary, but is documented +individually for each command. + +## Error object + +An error object is a serialisation of JavaScript error types, +and it is structured like this: + + { + "error": "invalid session id", + "message": "No active session with ID 1234", + "stacktrace": "" + } + +All the fields of the error object are required, so the stacktrace and +message fields may be empty strings. The error field is guaranteed +to be one of the JSON error codes as laid out by the [WebDriver standard]. + +## Clients + +Clients may be implemented in any language that is capable of writing +and receiving data over TCP socket. A [reference client] is provided. +Clients may be implemented both synchronously and asynchronously, +although the latter is impossible in protocol levels 2 and earlier +due to the lack of message sequencing. + +[WebDriver standard]: https://w3c.github.io/webdriver/#dfn-error-code +[reference client]: https://searchfox.org/mozilla-central/source/testing/marionette/client/ diff --git a/remote/doc/marionette/PythonTests.md b/remote/doc/marionette/PythonTests.md new file mode 100644 index 0000000000..c3497d6272 --- /dev/null +++ b/remote/doc/marionette/PythonTests.md @@ -0,0 +1,69 @@ +# Mn Python tests + +_Marionette_ is the codename of a [remote protocol] built in to +Firefox as well as the name of a functional test framework for +automating user interface tests. + +The in-tree test framework supports tests written in Python, using +Python’s [unittest] library. Test cases are written as a subclass +of `MarionetteTestCase`, with child tests belonging to instance +methods that have a name starting with `test_`. + +You can additionally define [`setUp`] and [`tearDown`] instance +methods to execute code before and after child tests, and +[`setUpClass`]/[`tearDownClass`] for the parent test. When you use +these, it is important to remember calling the `MarionetteTestCase` +superclass’ own [`setUp`]/[`tearDown`] methods since they handle +setup/cleanup of the session. + +The test structure is illustrated here: + +```python +from marionette_harness import MarionetteTestCase + +class TestSomething(MarionetteTestCase): + def setUp(self): + # code to execute before any tests are run + MarionetteTestCase.setUp(self) + + def test_foo(self): + # run test for 'foo' + + def test_bar(self): + # run test for 'bar' + + def tearDown(self): + # code to execute after all tests are run + MarionetteTestCase.tearDown(self) +``` + +[remote protocol]: Protocol.md +[unittest]: https://docs.python.org/3/library/unittest.html +[`setUp`]: https://docs.python.org/3/library/unittest.html#unittest.TestCase.setUp +[`setUpClass`]: https://docs.python.org/3/library/unittest.html#unittest.TestCase.setUpClass +[`tearDown`]: https://docs.python.org/3/library/unittest.html#unittest.TestCase.tearDown +[`tearDownClass`]: https://docs.python.org/3/library/unittest.html#unittest.TestCase.tearDownClass + +## Test assertions + +Assertions are provided courtesy of [unittest]. For example: + +```python +from marionette_harness import MarionetteTestCase + +class TestSomething(MarionetteTestCase): + def test_foo(self): + self.assertEqual(9, 3 * 3, '3 x 3 should be 9') + self.assertTrue(type(2) == int, '2 should be an integer') +``` + +## The API + +The full API documentation is found [here], but the key objects are: + +* `MarionetteTestCase`: a subclass for `unittest.TestCase` + used as a base class for all tests to run. + +* {class}`Marionette <marionette_driver.marionette.Marionette>`: client that speaks to Firefox + +[here]: /python/marionette_driver.rst diff --git a/remote/doc/marionette/SeleniumAtoms.md b/remote/doc/marionette/SeleniumAtoms.md new file mode 100644 index 0000000000..3788de2c54 --- /dev/null +++ b/remote/doc/marionette/SeleniumAtoms.md @@ -0,0 +1,90 @@ +# Selenium atoms + +Marionette uses a small list of [Selenium atoms] to interact with +web elements. Initially those have been added to ensure a better +reliability due to a wider usage inside the Selenium project. But +by adding full support for the [WebDriver specification] they will +be removed step by step. + +Currently the following atoms are in use: + +- `getElementText` +- `isElementDisplayed` +- `isElementEnabled` + +To use one of those atoms Javascript modules will have to import +[atom.sys.mjs]. + +[Selenium atoms]: https://github.com/SeleniumHQ/selenium/tree/master/javascript/webdriver/atoms +[WebDriver specification]: https://w3c.github.io/webdriver/webdriver-spec.html +[atom.sys.mjs]: https://searchfox.org/mozilla-central/source/remote/marionette/atom.sys.mjs + +## Update required Selenium atoms + +In regular intervals the atoms, which are still in use, have to +be updated. Therefore they have to be exported from the Selenium +repository first, and then updated in [atom.sys.mjs]. + +### Export Selenium Atoms + +The canonical GitHub repository for Selenium is + + <https://github.com/SeleniumHQ/selenium.git> + +so make sure to have an up-to-date local copy of it. If you have to clone +it first, it is recommended to specify the `--depth=1` argument, so only the +last changeset is getting downloaded (which itself might already be +more than 100 MB). + +```bash +git clone --depth=1 https://github.com/SeleniumHQ/selenium.git +``` + +To export the correct version of the atoms identify the changeset id (SHA1) of +the Selenium repository in the [index section] of the WebDriver specification. + +Fetch that changeset and check it out: + +```bash +git fetch --depth=1 origin SHA1 +git checkout SHA1 +``` + +Now you can export all the required atoms by running the following +commands. Make sure to [install bazelisk] first. + +```bash +bazel build //javascript/atoms/fragments:get-text +bazel build //javascript/atoms/fragments:is-displayed +bazel build //javascript/atoms/fragments:is-enabled +``` + +For each of the exported atoms a file can now be found in the folder +`bazel-bin/javascript/atoms/fragments/`. They contain all the +code including dependencies for the atom wrapped into a single function. + +[index section]: <https://w3c.github.io/webdriver/#index> +[install bazelisk]: <https://github.com/bazelbuild/bazelisk#installation> + +### Update atom.sys.mjs + +To update the atoms for Marionette the `atoms.js` file has to be edited. For +each atom to be updated the steps as laid out below have to be performed: + +1. Open the Javascript file of the exported atom. See above for + its location. + +2. Remove the contained license header, which can be found somewhere + in the middle of the file. + +3. Update the parameters of the wrapper function (at the very top) + so that those are equal with the used parameters in `atom.sys.mjs`. + +4. Copy the whole content of the file, and replace the existing + code for the atom in `atom.sys.mjs`. + +### Test the changes + +To ensure that the update of the atoms doesn't cause a regression +a try build should be run including Marionette unit tests, Firefox +ui tests, and all the web-platform-tests. diff --git a/remote/doc/marionette/Taskcluster.md b/remote/doc/marionette/Taskcluster.md new file mode 100644 index 0000000000..c5c9832523 --- /dev/null +++ b/remote/doc/marionette/Taskcluster.md @@ -0,0 +1,88 @@ +# Testing with one-click loaners + +[Taskcluster] is the task execution framework that supports Mozilla's +continuous integration and release processes. + +Build and test jobs (like Marionette) are executed across all supported +platforms, and job results are pushed to [Treeherder] for observation. + +The best way to debug issues for intermittent test failures of +Marionette tests for Firefox and Fennec (Android) is to use a +one-click loaner as provided by Taskcluster. Such a loaner creates +an interactive task you can interact with via a shell and VNC. + +To create an interactive task for a Marionette job which is shown +as failed on Treeherder, select the job, click the ellipse in the lower +left pane, and choose `Create Interactive Task`. + +Please note that you need special permissions to actually request +such a loaner. + +When the task has been created you will receive an email with the connection +details. Open the referenced shell and you will be connected via a WebSocket. +Once that has been done a wizard will automatically launch and +provide some options. Best here is to choose the second option, +which will run all the setup steps, installs the Firefox or Fennec +binary, and then exits. + +[Taskcluster]: https://docs.taskcluster.net/ +[Treeherder]: https://treeherder.mozilla.org + +## Setting up the Marionette environment + +Best here is to use a virtual environment, which has all the +necessary packages installed. If no modifications to any Python +package will be done, the already created environment by the +wizard can be used: + + % cd /builds/worker/workspace/build + % source venv/bin/activate + +Otherwise a new virtual environment needs to be created and +populated with the mozbase and marionette packages installed: + + % cd /builds/worker/workspace/build && rm -r venv + % virtualenv venv && source venv/bin/activate + % cd tests/mozbase && ./setup_development.py + % cd ../marionette/client && python setup.py develop + % cd ../harness && python setup.py develop + % cd ../../../ + +## Running Marionette tests + +### Firefox + +To run the Marionette tests execute the `runtests.py` script. For all +the required options as best search in the log file of the failing job +the interactive task has been created from. Then copy the complete +command and run it inside the already sourced virtual environment: + + % /builds/worker/workspace/build/venv/bin/python -u /builds/worker/workspace/build/tests/marionette/harness/marionette_harness/runtests.py --gecko-log=- -vv --binary=/builds/worker/workspace/build/application/firefox/firefox --address=127.0.0.1:2828 --symbols-path=https://queue.taskcluster.net/v1/task/GSuwee61Qyibujtxq4UV3A/artifacts/public/build/target.crashreporter-symbols.zip /builds/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit-tests.ini + +### Fennec + +The Marionette tests for Fennec are executed by using an Android +emulator which runs on the host platform. As such some extra setup +steps compared to Firefox on desktop are required. + +The following lines set necessary environment variables before +starting the emulator in the background, and to let Marionette +know of various Android SDK tools. + + % export ADB_PATH=/builds/worker/workspace/build/android-sdk-linux/platform-tools/adb + % export ANDROID_AVD_HOME=/builds/worker/workspace/build/.android/avd/ + % /builds/worker/workspace/build/android-sdk-linux/tools/emulator -avd test-1 -show-kernel -debug init,console,gles,memcheck,adbserver,adbclient,adb,avd_config,socket & + +The actual call to `runtests.py` is different per test job because +those are using chunks on Android. As best search for the command +and its options in the log file of the failing job the interactive +task has been created from. Then copy the complete command and run +it inside the already sourced virtual environment. + +Here an example for chunk 1 which runs all the tests in the current +chunk with some options for logs removed: + + % /builds/worker/workspace/build/venv/bin/python -u /builds/worker/workspace/build/tests/marionette/harness/marionette_harness/runtests.py --emulator --app=fennec --package=org.mozilla.fennec_aurora --address=127.0.0.1:2828 /builds/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit-tests.ini --gecko-log=- --symbols-path=/builds/worker/workspace/build/symbols --startup-timeout=300 --this-chunk 1 --total-chunks 10 + +To execute a specific test only simply replace `unit-tests.ini` +with its name. diff --git a/remote/doc/marionette/Testing.md b/remote/doc/marionette/Testing.md new file mode 100644 index 0000000000..a40443dccb --- /dev/null +++ b/remote/doc/marionette/Testing.md @@ -0,0 +1,192 @@ +# Testing + +We verify and test Marionette in a couple of different ways, using +a combination of unit tests and functional tests. There are three +distinct components that we test: + +- the Marionette **server**, using a combination of xpcshell +unit tests and functional tests written in Python spread across +Marionette- and WPT tests; + +- the Python **client** is tested with the same body of functional +Marionette tests; + +- and the **harness** that backs the Marionette, or `Mn` job on +try, tests is verified using separate mock-styled unit tests. + +All these tests can be run by using [mach]. + +[mach]: https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/mach + +## xpcshell unit tests + +Marionette has a set of [xpcshell] unit tests located in +_remote/marionette/test/xpcshell. These can be run this way: + + % ./mach test remote/marionette/test/unit + +Because tests are run in parallel and xpcshell itself is quite +chatty, it can sometimes be useful to run the tests sequentially: + + % ./mach test --sequential remote/marionette/test/xpcshell/test_error.js + +These unit tests run as part of the `X` jobs on Treeherder. + +[xpcshell]: https://developer.mozilla.org/en-US/docs/Mozilla/QA/Writing_xpcshell-based_unit_tests + +## Marionette functional tests + +We also have a set of [functional tests] that make use of the Marionette +Python client. These start a Firefox process and tests the Marionette +protocol input and output, and will appear as `Mn` on Treeherder. +The following command will run all tests locally: + + % ./mach marionette-test + +But you can also run individual tests: + + % ./mach marionette-test testing/marionette/harness/marionette_harness/tests/unit/test_navigation.py + +In case you want to run the tests with another binary like [Firefox Nightly]: + + % ./mach marionette-test --binary /path/to/nightly/firefox TEST + +When working on Marionette it is often useful to surface the stdout +from Gecko, which can be achieved using the `--gecko-log` option. +See [Debugging](Debugging.md) for usage instructions, but the gist is that +you can redirect all Gecko output to stdout: + + % ./mach marionette-test --gecko-log - TEST + +Our functional integration tests pop up Firefox windows sporadically, +and a helpful tip is to suppress the window can be to use Firefox’ +headless mode: + + % ./mach marionette-test -z TEST + +`-z` is an alias for the `--headless` flag and equivalent to setting +the `MOZ_HEADLESS` output variable. In addition to `MOZ_HEADLESS` +there is also `MOZ_HEADLESS_WIDTH` and `MOZ_HEADLESS_HEIGHT` for +controlling the dimensions of the no-op virtual display. This is +similar to using Xvfb(1) which you may know from the X windowing system, +but has the additional benefit of also working on macOS and Windows. + +[functional tests]: PythonTests.md +[Firefox Nightly]: https://nightly.mozilla.org/ + +### Android + +Prerequisites: + +- You have [built Fennec](https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/Simple_Firefox_for_Android_build). +- You can run an Android [emulator](https://wiki.mozilla.org/Mobile/Fennec/Android/Testing#Running_tests_on_the_Android_emulator), + which means you have the AVD you need. + +When running tests on Fennec, you can have Marionette runner take care of +starting Fennec and an emulator, as shown below. + + % ./mach marionette-test --emulator --app fennec + --avd-home /path/to/.mozbuild/android-device/avd + --emulator-binary /path/to/.mozbuild/android-sdk/emulator/emulator + --avd=mozemulator-x86 + +For Fennec tests, if the appropriate `emulator` command is in your `PATH`, you may omit the `--emulator-binary` argument. See `./mach marionette-test -h` +for additional options. + +Alternately, you can start an emulator yourself and have the Marionette runner +start Fennec for you: + + % ./mach marionette-test --emulator --app='fennec' --address=127.0.0.1:2828 + +To connect to an already-running Fennec in an emulator or on a device, +you will need to have it started with the `-marionette` command line argument, +or by setting the environment variable `MOZ_MARIONETTE=1` for the process. + +Make sure port 2828 is forwarded: + + % adb forward tcp:2828 tcp:2828 + +If Fennec is already started: + + % ./mach marionette-test --app='fennec' --address=127.0.0.1:2828 + +If Fennec is not already started on the emulator/device, add the `--emulator` +option. Marionette Test Runner will take care of forwarding the port and +starting Fennec with the correct prefs. (You may need to run +`adb forward --remove-all` to allow the runner to start.) + + % ./mach marionette-test --emulator --app='fennec' --address=127.0.0.1:2828 --startup-timeout=300 + +If you need to troubleshoot the Marionette connection, the most basic check is +to start Fennec with `-marionette` or the environment variable `MOZ_MARIONETTE=1`, +make sure that the port 2828 is forwarded, and then see if you get any response from +Marionette when you connect manually: + + % telnet 127.0.0.1:2828 + +You should see output like `{"applicationType":"gecko","marionetteProtocol":3}` + +[geckodriver]: /testing/geckodriver/index.rst + +## WPT functional tests + +Marionette is also indirectly tested through [geckodriver] with WPT +(`Wd` on Treeherder). To run them: + + % ./mach wpt testing/web-platform/tests/webdriver + +WPT tests conformance to the [WebDriver] standard and uses +[geckodriver]. Together with the Marionette remote protocol in +Gecko, they make up Mozilla’s WebDriver implementation. + +This command supports a `--webdriver-arg='-vv'` argument that +enables more detailed logging, as well as `--jsdebugger` for opening +the Browser Toolbox. + +A particularly useful trick is to combine this with the headless +mode for Firefox: + + % ./mach wpt --webdriver-arg='-vv' --headless testing/web-platform/tests/webdriver + +[WebDriver]: https://w3c.github.io/webdriver/ + +## Harness tests + +The Marionette harness Python package has a set of mock-styled unit +tests that uses the [pytest] framework. The following command will +run all tests: + + % ./mach python-test testing/marionette + +To run a specific test specify the full path to the module: + + % ./mach python-test testing/marionette/harness/marionette_harness/tests/harness_unit/test_serve.py + +[pytest]: https://docs.pytest.org/en/latest/ + +## One-click loaners + +Additionally, for debugging hard-to-reproduce test failures in CI, +one-click loaners from [Taskcluster](Taskcluster.md) can be particularly useful. + +## Out-of-tree testing + +All the above examples show tests running _in-tree_, with a local +checkout of _central_ and a local build of Firefox. It is also +possibly to run the Marionette tests _without_ a local build and +with a downloaded test archive from [Taskcluster](Taskcluster.md) + +If you want to run tests from a downloaded test archive, you will +need to download the `target.common.tests.tar.gz` artifact attached to +Treeherder [build jobs] `B` for your system. Extract the archive +and set up the Python Marionette client and harness by executing +the following command in a virtual environment: + + % pip install -r config/marionette_requirements.txt + +The tests can then be found under +_marionette/tests/testing/marionette/harness/marionette_harness/tests_ +and can be executed with the command `marionette`. It supports +the same options as described above for `mach`. + +[build jobs]: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&filter-searchStr=build diff --git a/remote/doc/marionette/index.rst b/remote/doc/marionette/index.rst new file mode 100644 index 0000000000..7272aff162 --- /dev/null +++ b/remote/doc/marionette/index.rst @@ -0,0 +1,64 @@ +========== +Marionette +========== + +Marionette is a remote `protocol`_ that lets out-of-process programs +communicate with, instrument, and control Gecko-based browsers. + +It provides interfaces for interacting with both the internal JavaScript +runtime and UI elements of Gecko-based browsers, such as Firefox +and Fennec. It can control both the chrome- and content documents, +giving a high level of control and ability to emulate user interaction. + +Within the central tree, Marionette is used in most TaskCluster +test jobs to instrument Gecko. It can additionally be used to +write different kinds of functional tests: + + * The `Marionette Python client`_ is used in the `Mn` job, which + is generally what you want to use for interacting with web documents + +Outside the tree, Marionette is used by `geckodriver`_ to implement +`WebDriver`_. + +Marionette supports to various degrees all the Gecko based applications, +including Firefox, Thunderbird, Fennec, and Fenix. + +.. _protocol: Protocol.html +.. _Marionette Python client: /python/marionette_driver.html +.. _geckodriver: /testing/geckodriver/ +.. _WebDriver: https://w3c.github.io/webdriver/ + +Some further documentation can be found here: + +.. toctree:: + :maxdepth: 1 + + Intro.md + Building.md + PythonTests.md + Protocol.md + Contributing.md + NewContributors.md + Patches.md + Debugging.md + Testing.md + Taskcluster.md + CodeStyle.md + SeleniumAtoms.md + Prefs.md + + +Bugs +==== + +Bugs are tracked in the `Testing :: Marionette` component. + + +Communication +============= + +The mailing list for Marionette discussion is +https://groups.google.com/a/mozilla.org/g/dev-webdriver. + +If you prefer real-time chat, ask your questions +on `#webdriver:mozilla.org <https://chat.mozilla.org/#/room/#webdriver:mozilla.org>`__. diff --git a/remote/doc/messagehandler/Intro.md b/remote/doc/messagehandler/Intro.md new file mode 100644 index 0000000000..3a3369fba1 --- /dev/null +++ b/remote/doc/messagehandler/Intro.md @@ -0,0 +1,81 @@ +# Introduction + +## Overview + +When developing browser tools in Firefox, you need to reach objects or APIs only available in certain layers (eg. processes or threads). There are powerful APIs available to communicate across layers (JSWindowActors, JSProcessActors) but they don't usually match all the needs from browser tool developers. For instance support for sessions, for events, ... + +### Modules + +The MessageHandler framework proposes to organize your code in modules, with the restriction that a given module can only run in a specific layer. Thanks to this, the framework will instantiate the modules where needed, and will provide easy ways to communicate between modules across layers. The goal is to take away all the complexity of routing information so that developers can simply focus on implementing the logic for their modules. + +### Commands and Events + +The framework is also designed around commands and events. Each module developed for the MessageHandler framework should expose commands and/or events. Commands follow a request/response model, and are conceptually similar to function calls where the caller could live in a different process than the callee. Events are emitted at the initiative of the module, and can reach listeners located in other layers. The role of modules is to implement the logic to handle commands (eg "click on an element") or generate events. The role of the framework is to send commands to modules, or to bubble events from modules. Commands and events are both used to communicate internally between modules, as well as externally with the consumer of your tooling modules. + +The "MessageHandler" name comes from this role of "handling" commands and events, aka "messages". + +### Summary + +As a summary, the MessageHandler framework proposes to write tooling code as modules, which will run in various processes or threads, and communicate across layers using commands and events. + +## Basic Architecture + +### MessageHandler Network + +Modules created for the MessageHandler framework need to run in several processes, threads, ... + +To support this the framework will dynamically create a network of [MessageHandler](https://searchfox.org/mozilla-central/source/remote/shared/messagehandler/MessageHandler.sys.mjs) instances in the various layers that need to be accessed by your modules. The MessageHandler class is obviously named after the framework, but the name is appropriate because its role is mainly to route commands and events. + +On top of routing duties, the MessageHandler class is also responsible for instantiating and managing modules. Typically, processing a command has two possible outcomes. Either it's not intended for this particular layer, in which case the MessageHandler will analyze the command and send it towards the appropriate recipient. But if it is intended for this layer, then the MessageHandler will try to delegate the command to the appropriate module. This means instantiating the module if it wasn't done before. So each node of a MessageHandler network also contains module instances. + +The root of this network is the [RootMessageHandler](https://searchfox.org/mozilla-central/source/remote/shared/messagehandler/RootMessageHandler.sys.mjs) and lives in the parent process. For consumers, this is also the single entry point exposing the commands and events of your modules. It can also own module instances, if you have modules which are supposed to live in the parent process (aka root layer). + +At the moment we only support another type of MessageHandler, the [WindowGlobalMessageHandler](https://searchfox.org/mozilla-central/source/remote/shared/messagehandler/WindowGlobalMessageHandler.sys.mjs) which will be used for the windowglobal layer and lives in the content process. + +### Simplified architecture example + +Let's imagine a very simple example, with a couple of modules: +- a root module called "version" with a command returning the current version of the browser +- a windowglobal module called "location" with a command returning the location of the windowglobal + +Suppose the browser has 2 tabs, running in different processes. If the consumer used the "version" module, and the "location" module but only for one of the two tabs, the network will look like: + +``` + parent process content process 1 +┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ + ╔═══════════════════════╗ ┌─────────────┐ │ +│ ╔═══════════════════════╗ │ │ ║ WindowGlobal ╠──────┤ location │ + ║ RootMessageHandler ║◀ ─ ─ ─ ─▶║ MessageHandler ║ │ module │ │ +│ ╚══════════╦════════════╝ │ │ ╚═══════════════════════╝ └─────────────┘ + │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┘ +│ │ │ + ┌──────┴──────┐ content process 2 +│ │ version │ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ + │ module │ │ +│ └─────────────┘ │ │ + │ +│ │ │ + ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┘ +``` + +But if the consumer sends another command, to retrieve the location of the other tab, the network will then evolve to: + +``` + parent process content process 1 +┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ + ╔═══════════════════════╗ ┌─────────────┐ │ +│ ╔═══════════════════════╗ │ │ ║ WindowGlobal ╠──────┤ location │ + ║ RootMessageHandler ║◀ ─ ┬ ─ ─▶║ MessageHandler ║ │ module │ │ +│ ╚══════════╦════════════╝ │ │ ╚═══════════════════════╝ └─────────────┘ + │ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┘ +│ │ │ + ┌──────┴──────┐ │ content process 2 +│ │ version │ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ + │ module │ │ ╔═══════════════════════╗ ┌─────────────┐ │ +│ └─────────────┘ │ │ ║ WindowGlobal ╠──────┤ location │ + └ ─ ▶ ║ MessageHandler ║ │ module │ │ +│ │ │ ╚═══════════════════════╝ └─────────────┘ + ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┘ +``` + +We can already see here that while RootMessageHandler is connected to both WindowGlobalMessageHandler(s), they are not connected with each other. There are restriction on the way messages can travel on the network both for commands and events, which will be the topic for other documentation pages. diff --git a/remote/doc/messagehandler/SimpleExample.md b/remote/doc/messagehandler/SimpleExample.md new file mode 100644 index 0000000000..249e5446c2 --- /dev/null +++ b/remote/doc/messagehandler/SimpleExample.md @@ -0,0 +1,142 @@ +# Simple Example + +As a tutorial, let's create a very simple example, with a couple of modules: +- a root (parent process) module to retrieve the current version of the browser +- a windowglobal (content process) module to retrieve the location of a given tab + +Some concepts used here will not be explained in details. More documentation should follow to clarify those. + +We will not use events in this example, only commands. + +## Create a root `version` module + +First let's create the root module. + +```javascript +import { Module } from "chrome://remote/content/shared/messagehandler/Module.sys.mjs"; + +class VersionModule extends Module { + destroy() {} + + getVersion() { + return Services.appinfo.platformVersion; + } +} + +export const version = VersionModule; +``` + +All modules should extend Module.sys.mjs and must define a destroy method. +Each public method of a Module class will be exposed as a command for this module. +The name used to export the module class will be the public name of the module, used to call commands on it. + +## Create a windowglobal `location` module + +Let's create the second module. + +```javascript +import { Module } from "chrome://remote/content/shared/messagehandler/Module.sys.mjs"; + +class LocationModule extends Module { + #window; + + constructor(messageHandler) { + super(messageHandler); + + // LocationModule will be a windowglobal module, so `messageHandler` will + // be a WindowGlobalMessageHandler which comes with a few helpful getters + // such as a `window` getter. + this.#window = messageHandler.window; + } + + destroy() { + this.#window = null; + } + + getLocation() { + return this.#window.location.href; + } +} + +export const location = LocationModule; +``` + +We could simplify the module and simply write `getLocation` to return `this.messageHandler.window.location.href`, but this gives us the occasion to get a glimpse at the module constructor. + +## Register the modules as Firefox modules + +Before we register those modules for the MessageHandler framework, we need to register them as Firefox modules first. For the sake of simplicity, we can assume they are added under a new folder `remote/example`: +- `remote/example/modules/root/version.sys.mjs` +- `remote/example/modules/windowglobal/location.sys.mjs` + +Register them in the jar.mn so that they can be loaded as any other Firefox module. + +The paths contain the corresponding layer (root, windowglobal) only for clarity. We don't rely on this as a naming convention to actually load the modules so you could decide to organize your folders differently. However the name used to export the module's class (eg `location`) will be the official name of the module, used in commands and events, so pay attention and use the correct export name. + +## Define a ModuleRegistry + +We do need to instruct the framework where each module should be loaded however. + +This is done via a ModuleRegistry. Without getting into too much details, each "set of modules" intended to work with the MessageHandler framework needs to provide a ModuleRegistry module which exports a single `getModuleClass` helper. This method will be called by the framework to know which modules are available. For now let's just define the simplest registry possible for us under `remote/example/modules/root/ModuleRegistry.sys.mjs` + +```javascript +export const getModuleClass = function(moduleName, moduleFolder) { + if (moduleName === "version" && moduleFolder === "root") { + return ChromeUtils.importESModule( + "chrome://remote/content/example/modules/root/version.sys.mjs" + ).version; + } + if (moduleName === "location" && moduleFolder === "windowglobal") { + return ChromeUtils.importESModule( + "chrome://remote/content/example/modules/windowglobal/location.sys.mjs" + ).location; + } + return null; +}; +``` + +Note that this can (and should) be improved by defining some naming conventions or patterns, but for now each set of modules is really free to implement this logic as needed. + +Add this module to jar.mn as well so that it becomes a valid Firefox module. + +### Temporary workaround to use the custom ModuleRegistry + +With this we have a set of modules which is almost ready to use. Except that for now MessageHandler is hardcoded to use WebDriver BiDi modules only. Once [Bug 1722464](https://bugzilla.mozilla.org/show_bug.cgi?id=1722464) is fixed we will be able to specify other protocols, but at the moment, the only way to instruct the MessageHandler framework to use non-bidi modules is to update the [following line](https://searchfox.org/mozilla-central/rev/08f7e9ef03dd2a83118fba6768d1143d809f5ebe/remote/shared/messagehandler/ModuleCache.sys.mjs#25) to point to `remote/example/modules/ModuleRegistry.sys.mjs`. + +Now with this, you should be able to create a MessageHandler network and use your modules. + +## Try it out + +For instance, you can open the Browser Console and run the following snippet: +```javascript +(async function() { + const { RootMessageHandlerRegistry } = ChromeUtils.importESModule( + "chrome://remote/content/shared/messagehandler/RootMessageHandlerRegistry.sys.mjs" + ); + const messageHandler = RootMessageHandlerRegistry.getOrCreateMessageHandler("test-session"); + const version = await messageHandler.handleCommand({ + moduleName: "version", + commandName: "getVersion", + params: {}, + destination: { + type: "ROOT", + }, + }); + console.log({ version }); + + const location = await messageHandler.handleCommand({ + moduleName: "location", + commandName: "getLocation", + params: {}, + destination: { + type: "WINDOW_GLOBAL", + id: gBrowser.selectedBrowser.browsingContext.id, + }, + }); + console.log({ location }); +})(); +``` + +This should print a version number `{ version: "109.0a1" }` and a location `{ location: "https://www.mozilla.org/en-US/" }` (actual values should of course be different for you). + +We are voluntarily skipping detailed explanations about the various parameters passed to `handleCommand`, as well as about the `RootMessageHandlerRegistry`, but this should give you some idea already of how you can start creating modules and using them. diff --git a/remote/doc/messagehandler/index.rst b/remote/doc/messagehandler/index.rst new file mode 100644 index 0000000000..68cf4aef24 --- /dev/null +++ b/remote/doc/messagehandler/index.rst @@ -0,0 +1,11 @@ +============== +MessageHandler +============== + +MessageHandler is the framework used to implement WebDriver BiDi modules in Firefox. + +.. toctree:: + :maxdepth: 1 + + Intro.md + SimpleExample.md |