diff options
Diffstat (limited to 'remote/doc')
| -rw-r--r-- | remote/doc/Architecture.md | 164 | ||||
| -rw-r--r-- | remote/doc/Building.md | 53 | ||||
| -rw-r--r-- | remote/doc/CodeStyle.md | 73 | ||||
| -rw-r--r-- | remote/doc/Debugging.md | 53 | ||||
| -rw-r--r-- | remote/doc/Prefs.md | 32 | ||||
| -rw-r--r-- | remote/doc/PuppeteerVendor.md | 83 | ||||
| -rw-r--r-- | remote/doc/Security.md | 99 | ||||
| -rw-r--r-- | remote/doc/Testing.md | 140 | ||||
| -rw-r--r-- | remote/doc/Usage.md | 62 | ||||
| -rw-r--r-- | remote/doc/index.rst | 54 | ||||
| -rw-r--r-- | remote/doc/moz.build | 8 |
11 files changed, 821 insertions, 0 deletions
diff --git a/remote/doc/Architecture.md b/remote/doc/Architecture.md new file mode 100644 index 0000000000..e1569d0a04 --- /dev/null +++ b/remote/doc/Architecture.md @@ -0,0 +1,164 @@ +Remote agent overall architecture +================================= + +This document will cover the Remote Agent architecture by following the sequence of steps needed to start the agent, connect a client and debug a target. + +Remote Agent startup +-------------------- + +Everything starts with the `RemoteAgent` implementation, which handles command line +arguments (--remote-debugging-port) to eventually +start a server listening on the TCP port 9222 (or the one specified by the command line). +The browser target websocket URL will be printed to stderr. +To do that this component glue together three main high level components: + + * `server/HTTPD` + This is a copy of httpd.js, from /netwerk/ folder. This is a JS implementation of an http server. + This will be used to implement the various http endpoints of CDP. + There is a few static URL implemented by `JSONHandler` and one dynamic URL per target. + + * `JSONHandler` + This implements the following three static http endpoints: + * /json/version: + Returns information about the runtime as well as the url of the browser target websocket url. + * /json/list: + Returns a list of all debuggable targets with, for each, their dynamic websocket URL. + For now it only reports tabs, but will report workers and addons as soon as we support them. + The main browser target is the only one target not listed here. + * /json/protocol: + Returns a big dictionary describing the supported protocol. + This is currently hard coded and returns the full CDP protocol schema, including APIs we don’t support. + We have a future intention to fix this and report only what Firefox implements. + You can connect to these websocket URL in order to debug things. + + * `targets/TargetList` + This component is responsible of maintaining the list of all debuggable targets. + For now it can be either: + * The main browser target + A special target which allows to inspect the browser, but not any particular tab. + This is implemented by `targets/MainProcessTarget` and is instantiated on startup. + * Tab targets + Each opened tab will have a related `targets/TabTarget` instantiated on their opening, + or on server startup for already opened ones. + Each target aims at focusing on one particular context. This context is typically running in one + particular environment. This can be a particular process or thread. + In the future, we will most likely support targets for workers and add-ons. + All targets inherit from `targets/Target`. + +Connecting to Websocket endpoints +--------------------------------- + +Each target's websocket URL will be registered as a HTTP endpoint via `server/HTTPD:registerPathHandler`. +(This registration is done from `RemoteAgentClass:listen`) +Once a HTTP request happens, `server/HTTPD` will call the `handle` method on the object passed to `registerPathHandler`. +For static endpoints registered by `JSONHandler`, this will call `JSONHandler:handle` and return a JSON string as http body. +For target's endpoint, it is slightly more complicated as it requires a special handshake to morph the HTTP connection into a WebSocket one. +The WebSocket is then going to be long lived and be used to inspect the target over time. +When a request is made to a target URL, `targets/Target:handle` is called and: + + * delegate the complex HTTP to WebSocket handshake operation to `server/WebSocketHandshake:upgrade` + In return we retrieve a WebSocket object. + + * hand over this WebSocket to `server/WebSocketTransport` + and get a transport object in return. The transport implements a basic JSON stream over WebSocket. With that, you can send and receive JSON objects over a WebSocket connection. + + * hand over the transport to a freshly instantiated `Connection` + The Connection has two goals: + * Interpret incoming CDP packets by reading the JSON object attribute (`id`, `method`, `params` and `sessionId`) + This is done in `Connection:onPacket`. + * Format outgoing CDP packets by writing the right JSON object for command response (`id`, `result` and `sessionId`) and events (`method`, `params` and `sessionId`) + * Redirect CDP packet from/to the right session. + A connection may have more than one session attached to it. + + * instantiate the default session + The session is specific to each target kind and all of them inherit from `session/Session`. + For example, tabs targets uses `session/TabSession` and the main browser target uses `session/MainProcessSession`. + Which session class is used is defined by the Target subclass’ constructor, which pass a session class reference to targets/Target:constructor. + A session is mostly responsible of accommodating the eventual cross process/cross thread aspects of the target. + The code we are currently describing (`targets/Target:handle`) is running in the parent process. + The session class receive CDP commands from the connection and first try to execute the Domain commands in the parent process. + Then, if the target actually runs in some other context, the session tries to forward this command to this other context, which can be a thread or a process. + Typically, the `sessions/TabSession` forward the CDP command to the content process where the tab is running. + It also redirects back the command response as well as Domain events from that process back to the parent process in order to + forward them to the connection. + Sessions will be using the `DomainCache` class as a helper to manage a list of Domain implementations in a given context. + +Debugging additional Targets +---------------------------- + +From a given connection you can know about the other potential targets. +You typically do that via `Target.setDiscoverTargets()`, which will emit `Target.targetCreated` events providing a target ID. +You may create a new session for the new target by handing the ID to `Target.attachToTarget()`, which will return a session ID. +"Target" here is a reference to the CDP Domain implemented in `domains/parent/Target.jsm`. That is different from `targets/Target` +class which is an implementation detail of the Remote Agent. + +Then, there is two ways to communicate with the other targets: + + * Use `Target.sendMessageToTarget()` and `Target.receivedMessageFromTarget` + You will manually send commands via the `Target.sendMessageToTarget()` command and receive command's response as well as events via `Target.receivedMessageFromTarget`. + In both cases, a session ID attribute is passed in the command or event arguments in order to select which additional target you are communicating with. + + * Use `Target.attachToTarget({ flatten: true })` and include `sessionId` in CDP packets + This requires a special client, which will use the `sessionId` returned by `Target.attachToTarget()` in order to spawn a distinct client instance. + This client will re-use the same WebSocket connection, but every single CDP packet will contain an additional `sessionId` attribute. + This helps distinguish packets which relate to the original target as well as the multiple additional targets you may attach to. + +In both cases, `Target.attachToTarget()` is special as it will spawn `session/TabSession` for the tab you are attaching to. +This is the codepath creating non-default session. The default session is related to the target you originally connected to, +so that you don't need any ID for this one. When you want to debug more than one target over a single connection +you need additional sessions, which will have a unique ID. +`Target.attachToTarget` will compute this ID and instantiate a new session bound to the given target. +This additional session will be managed by the `Connection` class, which will then redirect CDP packets to the +right session when you are using flatten session. + +Cross Process / Layers +---------------------- + +Because targets may runs in different contexts, the remote agent code runs in different processes. +The main and startup code of the Remote agent code runs in the parent process. +The handling of the command line as well as all the HTTP and WebSocket work is all done in the parent process. +The browser target is also all implemented in the parent process. +But when it comes to a tab target, as the tab runs in the content process, we have to run code there as well. +Let's start from the `sessions/TabSession` class, which has already been described. +We receive here JSON packets from the WebSocket connection and we are in the parent process. +In this class, we route the messages to the parent process domains first. +If there is no implementation of the domain or the particular method, +we forward the command to a `session/ContentProcessSession` which runs in the tab's content process. +These two Session classes will interact with each other in order to forward back the returned value +of the method we just called, as well as piping back any event being sent by a Domain implemented in any +of the two processes. + +Organizational chart of all the classes +---------------------------------------- +``` + ┌─────────────────────────────────────────────────┐ + │ │ + 1 ▼ │ + ┌───────────────┐ 1 ┌───────────────┐ 1..n┌───────────────┐ + │ RemoteAgent │──────▶│ HttpServer │◀───────▶│ JsonHandler │ + └───────────────┘ └───────────────┘ 1 └───────────────┘ + │ + │ + │ 1 ┌────────────────┐ 1 + └───────────────▶│ TargetList │◀─┐ + └────────────────┘ │ + │ │ + ▼ 1..n │ + ┌────────────┐ │ + ┌─────────────────│ Target [1]│ │ + │ └────────────┘ │ + │ ▲ 1 │ + ▼ 1..n │ │ + ┌────────────┐ 1..n┌────────────┐ │ + │ Connection │◀─────────▶│ Session [2]│──────┘ + └────────────┘ 1 └────────────┘ + │ 1 ▲ + │ │ + ▼ 1 ▼ 1 +┌────────────────────┐ ┌──────────────┐ 1..n┌────────────┐ +│ WebSocketTransport │ │ DomainCache | │──────────▶│ Domain [3]│ +└────────────────────┘ └──────────────┘ └────────────┘ +``` + [1] Target is inherited by TabTarget and MainProcessTarget. + [2] Session is inherited by TabSession and MainProcessSession. + [3] Domain is inherited by Log, Page, Browser, Target.... i.e. all domain implementations. From both domains/parent and domains/content folders. diff --git a/remote/doc/Building.md b/remote/doc/Building.md new file mode 100644 index 0000000000..93b8ef8cdb --- /dev/null +++ b/remote/doc/Building.md @@ -0,0 +1,53 @@ +Building +======== + +The remote agent is included in the default Firefox build, but only +ships on the Firefox Nightly release channel: + + % ./mach run --remote-debugging-port + +The source code can be found under [remote/ in central]. + +There are two build modes to choose from: + +Full build mode +--------------- + +The remote agent supports only Firefox, and is included when you +build in the usual way: + + % ./mach build + +When you make changes to XPCOM component files you need to rebuild +in order for the changes to take effect. The most efficient way to +do this, provided you haven’t touched any compiled code (C++ or Rust): + + % ./mach build faster + +Component files include the likes of components.conf, +RemoteAgent.manifest, moz.build files, and jar.mn. +All the JS modules (files ending with `.jsm`) are symlinked into +the build and can be changed without rebuilding. +The remote agent’s startup code found under remote/startup/ +is written in Rust and requires rebuilds when changed. + +You may also opt out of building the remote agent entirely by setting +the `--disable-cdp` build flag in your [mozconfig]: + + ac_add_options --disable-cdp + + +Artifact mode +------------- + +You may also use [artifact builds] when working on the remote agent. +This fast build mode downloads pre-built components from the Mozilla +build servers, rendering local compilation unnecessary. To use +them, place this in your [mozconfig]: + + ac_add_options --enable-artifact-builds + + +[remote/ in central]: https://searchfox.org/mozilla-central/source/remote +[mozconfig]: ../build/buildsystem/mozconfigs.html +[artifact builds]: https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/Artifact_builds diff --git a/remote/doc/CodeStyle.md b/remote/doc/CodeStyle.md new file mode 100644 index 0000000000..eade28e2b6 --- /dev/null +++ b/remote/doc/CodeStyle.md @@ -0,0 +1,73 @@ +Style guide +=========== + +Like other projects, we also have some guidelines to keep to the code. +For the overall remote agent project, a few rough rules are: + + * Make your code readable and sensible, and don’t try to be + clever. Prefer simple and easy solutions over more convoluted + and foreign syntax. + + * Fixing style violations whilst working on a real change as a + preparatory clean-up step is good, but otherwise avoid useless + code churn for the sake of conforming to the style guide. + + * Code is mutable and not written in stone. Nothing that + is checked in is sacred and we encourage change to make + this a pleasant ecosystem to work in. + + * We never land any code that is unnecessary or unused. + + +Documentation +------------- + +We keep our documentation (what you are reading right now!) in-tree +under [remote/doc]. Updates and minor changes to documentation should +ideally not be scrutinised to the same degree as code changes to +encourage frequent updates so that documentation does not go stale. +To that end, documentation changes with `r=me a=doc` from anyone +with commit access level 3 are permitted. + +Use fmt(1) or an equivalent editor specific mechanism (such as +<kbd>Meta-Q</kbd> in Emacs) to format paragraphs at a maximum of +75 columns with a goal of roughly 65. This is equivalent to `fmt +-w75 -g65`, which happens to the default on BSD and macOS. + +The documentation can be built locally this way: + + % ./mach doc remote + +[remote/doc]: https://searchfox.org/mozilla-central/source/remote/doc + + +Linting +------- + +The remote agent consists mostly of JavaScript code, and we lint that +using [mozlint], which is harmonises different linters including [eslint]. + +To run the linter and get sensible output: + + % ./mach lint -funix remote + +For certain classes of style violations, eslint has an automatic +mode for fixing and formatting code. This is particularly useful +to keep to whitespace and indentation rules: + + % ./mach eslint --fix remote + +The linter is also run as a try job (shorthand `ES`) which means +any style violations will automatically block a patch from landing +(if using Autoland) or cause your changeset to be backed out (if +landing directly on inbound). + +If you use git(1) you can [enable automatic linting] before +you push to a remote through a pre-push (or pre-commit) hook. +This will run the linters on the changed files before a push and +abort if there are any problems. This is convenient for avoiding +a try run failing due to a simple linting issue. + +[mozlint]: /tools/lint/usage.html +[eslint]: https://eslint.org/ +[enable automatic linting]: https://firefox-source-docs.mozilla.org/tools/lint/usage.html#using-a-vcs-hook diff --git a/remote/doc/Debugging.md b/remote/doc/Debugging.md new file mode 100644 index 0000000000..b0e8e8d47d --- /dev/null +++ b/remote/doc/Debugging.md @@ -0,0 +1,53 @@ +Debugging +========= + +For other debugging resources, see also: Remote project [wiki] + +Increasing the logging verbosity +-------------------------------- + +To increase the internal logging verbosity you can use the +`remote.log.level` [preference]. + +If you use mach to start Firefox: + + ./mach run --setpref "remote.log.level=Trace" --remote-debugging-port + +By default, long log lines are truncated. To print long lines in full, you +can set `remote.log.truncate` to false. + +Enabling logging of emitted events +---------------------------------- + +To dump events produced by EventEmitter, +including CDP events produced by the remote agent, +you can use the `toolkit.dump.emit` [preference]: + + ./mach run --setpref "toolkit.dump.emit=true" --remote-debuggiing-port + + +Logging observer notifications +------------------------------ + +[System observer notifications] are used extensively throughout the +code and it can sometimes be useful to log these to see what is +available and when they are fired. + +The `MOZ_LOG` environment variable controls the C++ logs and takes +the name of the subsystem along with a verbosity setting. See +[prlog.h] for more details. + + MOZ_LOG=ObserverService:5 + +You can optionally redirect logs away from stdout to a file: + + MOZ_LOG_FILE=service.log + +This enables `LogLevel::Debug` level information and places all +output in the file service.log in your current working directory. + + +[preference]: ./Prefs.html +[System observer notifications]: https://developer.mozilla.org/en-US/docs/Archive/Add-ons/Overlay_Extensions/XUL_School/Observer_Notifications +[prlog.h]: https://searchfox.org/mozilla-central/source/nsprpub/pr/include/prlog.h +[wiki]: https://wiki.mozilla.org/Remote/Developer_Resources diff --git a/remote/doc/Prefs.md b/remote/doc/Prefs.md new file mode 100644 index 0000000000..44b29b37e2 --- /dev/null +++ b/remote/doc/Prefs.md @@ -0,0 +1,32 @@ +Preferences +=========== + +There are a couple of preferences associated with the remote agent: + + +Configurable preferences +------------------------ + +### `remote.enabled` + +Indicates whether the remote agent is enabled. When the remote +agent is enabled, it exposes a [`--remote-debugging-port` flag] for Firefox. +When set to false, the remote agent will not be loaded on startup. + +[`--remote-debugging` flag]: Usage.html + +### `remote.force-local` + +Limits the remote agent to be allowed to listen on loopback devices, +e.g. 127.0.0.1, localhost, and ::1. + +### `remote.log.level` + +Defines the verbosity of the internal logger. Available levels +are, in descending order of severity, `Trace`, `Debug`, `Config`, +`Info`, `Warn`, `Error`, and `Fatal`. Note that the value is +treated case-sensitively. + +### `remote.log.truncate` + +Defines whether long log messages should be truncated. Defaults to true. diff --git a/remote/doc/PuppeteerVendor.md b/remote/doc/PuppeteerVendor.md new file mode 100644 index 0000000000..27ec606c4c --- /dev/null +++ b/remote/doc/PuppeteerVendor.md @@ -0,0 +1,83 @@ +Vendoring Puppeteer +=================== + +As mentioned in the chapter on [Testing], we run the full [Puppeteer +test suite] on try. These tests are vendored in central under +_remote/test/puppeteer/_ and we have a script to pull in upstream changes. + +We periodically perform a manual two-way sync. Below is an outline of the +process interspersed with some tips. + +1. Clone the Puppeteer git repository and checkout the release tag you want + to vendor into mozilla-central. + + % git checkout tags/v10.0 -b sync-v10.0 + +2. Apply any recent changes in `remote/test/puppeteer` to the Puppeteer branch + created above. + + You might want to [install the project] at this point and make sure unit + tests pass. Check the project's `package.json` for relevant testing commands. + + You should use this as basis for a PR to the Puppeteer project once you are + satisfied that the two-way sync will be successful in mozilla-central. See + their [CONTRIBUTING.md]. + + Typically, the changes we push to Puppeteer include unskipping newly passing + unit tests for Firefox along with minor fixes to the tests or + to Firefox-specific browser-fetching and launch code. + + Be sure to [run tests against both Chromium and Firefox] in the Puppeteer + repo. You can specify your local Firefox build when you do so: + + % BINARY=<path-to-objdir-binary> npm run funit + +3. Now back in mozilla-central, you can run the following mach command to + copy over the Puppeteer branch you just prepared. The mach command has + flags to specify a local or remote repository as well as a commit. + + % ./mach remote vendor-puppeteer + + By default, this command also installs the newly-pulled Puppeteer package + in order to generate a new `package-lock.json` file for the purpose of + pinning Puppeteer dependencies for our CI. There is a `--no-install` option + if you want to skip this step; for example, if you want to run installation + separately at a later point. + +4. Go through the changes under `remote/test/puppeteer/test` and [unskip] any + newly-skipped tests (e.g. change `itFailsFirefox` to `it`). + A mass-change with `awk` might be useful here. + + Why do we do this? The Puppeteer team runs their unit tests against Firefox + in their CI with many tests skipped. In contrast, we leave these tests + unskipped in Mozilla CI and track test expectation metadata + in [puppeteer-expected.json] instead. + +5. Use `./mach puppeteer-test` (see [Testing]) to run Puppeteer tests against + both Chromium and Firefox in headless mode. Again, only running a subset of + tests against Firefox is fine -- at this point you just want to check that + the typescript compiles and the browser binaries are launched successfully. + +6. Next you want to update the test expectation metadata: test results might + have changed, tests may have been renamed, removed or added. The + easiest way to do this is to run the Puppeteer test job on try + (see [Testing]). You will find the new test metadata as an artifact on that + job and you can copy it over into your sync patch if it looks reasonable. + + Examine the job logs and makes sure the run didn't get interrupted early + by a crash or a hang, especially if you see a lot of + `TEST-UNEXPECTED-MISSING` in the Treeherder Failure Summary. You might need + to add new test skips or fix some new bug in the unit tests. This is the + fun part. + +7. Once you are happy with the metadata and are ready to submit the sync patch + up for review, run the Puppeteer test job on try again with `--rebuild 10` + to check for stability. + +[Testing]: ./Testing.html +[Puppeteer test suite]: https://github.com/GoogleChrome/puppeteer/tree/master/test +[re-install the project]: https://github.com/puppeteer/puppeteer/blob/main/CONTRIBUTING.md#getting-code +[run tests against both Chromium and Firefox]: https://github.com/puppeteer/puppeteer/blob/main/test/README.md#running-tests +[puppeteer-expected.json]: https://searchfox.org/mozilla-central/source/remote/puppeteer-expected.json +[CONTRIBUTING.md]: https://github.com/puppeteer/puppeteer/blob/main/CONTRIBUTING.md +[unskip]: https://github.com/puppeteer/puppeteer/blob/main/test/README.md#skipping-tests-in-specific-conditions diff --git a/remote/doc/Security.md b/remote/doc/Security.md new file mode 100644 index 0000000000..6f95ac9463 --- /dev/null +++ b/remote/doc/Security.md @@ -0,0 +1,99 @@ +Security aspects of the remote agent +==================================== + +The remote agent is not a web-facing feature and as such has different +security characteristics than traditional web platform APIs. The +primary consumers are out-of-process programs that connect to the +agent via a remote protocol, but can theoretically be extended to +facilitate browser-local clients communicating over IPDL. + + +Design considerations +--------------------- + +The remote agent allows consumers to interface with Firefox through +an assorted set of domains for inspecting the state and controlling +execution of documents running in web content, injecting arbitrary +scripts to documents, do browser service instrumentation, simulation +of user interaction for automation purposes, and for subscribing +to updates in the browser such as network- and console logs. + +The remote interfaces are served over an HTTP wire protocol, by a +server listener hosted in the Firefox binary. This can only be +started by passing the `--remote-debugging-port` +flag. Connections are by default restricted to loopback devices +(such as localhost and 127.0.0.1), but this can be overridden with +the `remote.force-local` preference. + +The feature as a whole is guarded behind the `remote.enabled` +preference. This preference serves as a way to gate the remote +agent component through release channels, and potentially for +remotely disabling the remote agent through Normandy if the need +should arise. + +Since the remote agent is not an in-document web feature, the +security concerns we have for this feature are essentially different +to other web platform features. The primary concern is that the +HTTPD is not spun up without passing one of the command-line flags. +It is out perception that if a malicious user has the capability +to execute arbitrary shell commands, there is little we can do to +prevent the browser being turned into an evil listening device. + + +User privacy concerns +--------------------- + +There are no user privacy concerns beyond the fact that the offered +interfaces will give the client access to all browser internals, +and thereby follows all browser-internal secrets. + + +How the remote agent works +-------------------------- + +When the `--remote-debugging-port` flag is used, +it spins up an HTTPD on the desired port, or defaults to +localhost:9222. The HTTPD serves WebSocket connections via +`nsIWebSocket.createServerWebSocket` that clients connect to in +order to give the agent remote instructions. + +The `remote.force-local` preference controls whether the HTTPD +accepts connections from non-loopback clients. System-local loopback +connections are the default: + + if (Preferences.get(FORCE_LOCAL) && !LOOPBACKS.includes(host)) { + throw new Error("Restricted to loopback devices"); + } + +The remote agent implements a large subset of the Chrome DevTools +Protocol (CDP). This protocol allows a client to: + + - take control over the user session for automation purposes, for + example to simulate user interaction such as clicking and typing; + + - instrument the browser for analytical reasons, such as intercepting + network traffic; + + - and extract information from the user session, including cookies + and local strage. + +There are no web-exposed features in the remote agent whatsoever. + + +Security model +-------------- + +It shares the same security model as DevTools and Marionette, in +that there is no other mechanism for enabling the remote agent than +by passing a command-line flag. + +It is our assumption that if an attacker has shell access to the +user account, there is little we can do to prevent secrets from +being accessed or leaked. + +The preference `remote.enabled` is true on the Firefox Nightly +release channel. The [security review] was completed in November +2019. + + +[security review]: https://bugzilla.mozilla.org/show_bug.cgi?id=1542229 diff --git a/remote/doc/Testing.md b/remote/doc/Testing.md new file mode 100644 index 0000000000..358acb85d2 --- /dev/null +++ b/remote/doc/Testing.md @@ -0,0 +1,140 @@ +Testing +======= + +The remote agent has unit- and functional tests located under +`remote/test/{unit,browser}`. + +You may run all the tests under a particular subfolder like this: + + % ./mach test remote + + +Unit tests +---------- + +Because tests are run in parallel and [xpcshell] itself is quite +chatty, it can sometimes be useful to run the tests in sequence: + + % ./mach xpcshell-test --sequential remote/test/unit/test_DomainCache.js + +The unit tests will appear as part of the `X` (for _xpcshell_) jobs +on Treeherder. + +[xpcshell]: https://developer.mozilla.org/en-US/docs/Mozilla/QA/Writing_xpcshell-based_unit_tests + + +Browser chrome tests +-------------------- + +We also have a set of functional [browser chrome] tests located +under _remote/test/browser_: + + % ./mach mochitest remote/test/browser/browser_cdp.js + +The functional tests will appear under the `M` (for _mochitest_) +category in the `remote` jobs on Treeherder. + +As the functional tests will sporadically pop up new Firefox +application windows, a helpful tip is to run them in [headless +mode]: + + % ./mach mochitest --headless remote/test/browser + +The `--headless` flag is equivalent to setting the `MOZ_HEADLESS` +environment variable. You can additionally use `MOZ_HEADLESS_WIDTH` +and `MOZ_HEADLESS_HEIGHT` to control the dimensions of the virtual +display. + +The `add_task()` function used for writing [asynchronous tests] is +replaced to provide some additional test setup and teardown useful +for writing tests against the remote agent and the targets. + +Before the task is run, the `nsIRemoteAgent` listener is started +and a [CDP client] is connected. You will use this CDP client for +interacting with the agent just as any other CDP client would. + +Also target discovery is getting enabled, which means that targetCreated, +targetDestroyed, and targetInfoChanged events will be received by the client. + +The task function you provide in your test will be called with the +three arguments `client`, `CDP`, and `tab`: + + - `client` is the connection to the `nsIRemoteAgent` listener, + and it provides the a client CDP API + + - `CDP` is the CDP client class + + - `tab` is a fresh tab opened for each new test, and is automatically + removed after the test has run + +This is what it looks like all put together: + + add_task(async function testName({client, CDP, tab}) { + // test tab is implicitly created for us + info("Current URL: " + tab.linkedBrowser.currentURI.spec); + + // manually connect to a specific target + const { mainProcessTarget } = RemoteAgent.targets; + const target = mainProcessTarget.wsDebuggerURL; + const client = await CDP({ target }); + + // retrieve the Browser domain, and call getVersion() on it + const { Browser } = client; + const version = await Browser.getVersion(); + + await client.close(); + + // tab is implicitly removed + }); + +You can control the tab creation behaviour with the `createTab` +option to `add_task(taskFunction, options)`: + + add_task(async function testName({client}) { + // tab is not implicitly created + }, { createTab: false }); + +If you want to write an asynchronous test _without_ this implicit +setup you may instead use `add_plain_task()`, which works exactly like the +original `add_task()`. + +[browser chrome]: https://developer.mozilla.org/en-US/docs/Mozilla/Browser_chrome_tests +[headless mode]: https://developer.mozilla.org/en-US/Firefox/Headless_mode +[asynchronous tests]: https://developer.mozilla.org/en-US/docs/Mozilla/Browser_chrome_tests#Test_functions +[CDP client]: https://github.com/cyrus-and/chrome-remote-interface + + +Puppeteer tests +--------------- + +In addition to our own Firefox-specific tests, we run the upstream +[Puppeteer test suite] against our implementation to [track progress] +towards achieving full [Puppeteer support] in Firefox. The tests are written +in the behaviour-driven testing framework [Mocha]. + +Check the upstream [Puppeteer test suite] documentation for instructions on +how to skip tests, run only one test or a subsuite of tests. + +Puppeteer tests are vendored under _remote/test/puppeteer/_ and are +run locally like this: + + % ./mach puppeteer-test + +You can also run them against Chrome as: + + % ./mach puppeteer-test --product=chrome --subset + +`--subset` disables a check for missing or skipped tests in our log parsing. +This check is typically not relevant when running against Chrome. + +To schedule the tests on try, look for `source-test-remote-puppeteer` in +`./mach try fuzzy`. On try they appear under the `remote(pup)` symbol. + +Test expectation metadata is collected in _remote/puppeteer-expected.json_ +via log parsing and a custom Mocha reporter under +_remote/test/puppeteer/json-mocha-reporter.js_ + +[Puppeteer test suite]: https://github.com/puppeteer/puppeteer/blob/master/test/README.md +[track progress]: https://puppeteer.github.io/ispuppeteerfirefoxready/ +[Puppeteer support]: https://bugzilla.mozilla.org/show_bug.cgi?id=puppeteer +[Mocha]: https://mochajs.org/ diff --git a/remote/doc/Usage.md b/remote/doc/Usage.md new file mode 100644 index 0000000000..f93f4cd858 --- /dev/null +++ b/remote/doc/Usage.md @@ -0,0 +1,62 @@ +Usage +===== + +When using the CDP-based remote agent in Firefox, there are +three different programs/components running simultaneously: + + * the __client__, being the out-of-process script or library + (such as Puppeteer) or web inspector frontend you use to control + and retrieve information out of Firefox; + + * the __agent__ that the client connects to which is an HTTPD living + inside Firefox, facilitating communication between clients + and targets; + + * and the __target__, which is the web document being debugging. + +The remote agent ships in [Firefox Nightly] only. + +To check if your Firefox binary has the remote agent enabled, you +can look in its help message for this: + + % ./firefox -h + … + --remote-debugging-port [<port>] Start the Firefox remote agent, which is + a low-level debugging interface based on the CDP protocol. + Defaults to listen on localhost:9222. + … + +When used, the remote agent will start an HTTP server and print a +message on stderr with the location of the main target’s WebSocket +listener: + + % firefox --remote-debugging-port + DevTools listening on ws://localhost:9222/devtools/browser/7b4e84a4-597f-4839-ac6d-c9e86d16fb83 + +`--remote-debugging-port` takes an optional port as input: + + [<port>] + +You can use this to instruct the remote agent to bind to a particular +port on your system. port is optional, +which means `firefox --remote-debugging-port` will bind the HTTPD to +the default `localhost:9222`. + +If port has been specified the default port will be overridden: + + % firefox --remote-debugging-port 9989 + DevTools listening on ws://localhost:9989/devtools/browser/b49481af-8ad3-9b4d-b1bf-bb0cdb9a0620 + +When you ask the remote agent to listen on port 0, +the system will atomically allocate an arbitrary free port: + + % firefox --remote-debugging-port 0 + DevTools listening on ws://localhost:59982/devtools/browser/a12b22a9-1b8b-954a-b81f-bd31552d3f1c + +Allocating an atomic port can be useful if you want to avoid race +conditions. The atomically allocated port will be somewhere in the +ephemeral port range, which varies depending on your system and +system configuration, but is always guaranteed to be free thus +eliminating the risk of binding to a port that is already in use. + +[Firefox Nightly]: https://www.mozilla.org/en-GB/firefox/channel/desktop/#nightly diff --git a/remote/doc/index.rst b/remote/doc/index.rst new file mode 100644 index 0000000000..d7d1265ff1 --- /dev/null +++ b/remote/doc/index.rst @@ -0,0 +1,54 @@ +=============== +Remote Protocol +=============== + +The Firefox **remote protocol** is a low-level debugging interface +you can use to inspect the state and control execution of documents +running in web content, instrument the browser in interesting ways, +simulate user interaction for automation purposes, and for subscribing +to updates in the browser such as network- or console logs. + +It complements the existing Firefox Developer Tools :ref:`Remote Debugging +Protocol <Remote Debugging Protocol>` (RDP) by implementing a subset of the +`Chrome DevTools Protocol`_ (CDP). + +.. _Chrome DevTools Protocol: https://chromedevtools.github.io/devtools-protocol/ + +.. toctree:: + :maxdepth: 1 + + Usage.md + Prefs.md + + +Internals +========= + +.. toctree:: + :maxdepth: 1 + + Building.md + Debugging.md + Testing.md + Architecture.md + Prefs.md + CodeStyle.md + PuppeteerVendor.md + Security.md + + +Bugs +==== + +Bugs are tracked under the `Remote Protocol product`_. + +.. _Remote Protocol product: https://bugzilla.mozilla.org/describecomponents.cgi?product=Remote%20Protocol + + +Communication +============= + +See `Communication`_ on `our project wiki`_. + +.. _Communication: https://wiki.mozilla.org/Remote#Communication +.. _our project wiki: https://wiki.mozilla.org/Remote diff --git a/remote/doc/moz.build b/remote/doc/moz.build new file mode 100644 index 0000000000..fe70012822 --- /dev/null +++ b/remote/doc/moz.build @@ -0,0 +1,8 @@ +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. + +SPHINX_TREES["remote"] = "." + +with Files("**"): + SCHEDULES.exclusive = ["docs"] |