diff options
Diffstat (limited to 'devtools/docs')
98 files changed, 8265 insertions, 0 deletions
diff --git a/devtools/docs/.gitignore b/devtools/docs/.gitignore new file mode 100644 index 0000000000..9a8882dd2b --- /dev/null +++ b/devtools/docs/.gitignore @@ -0,0 +1 @@ +_book
\ No newline at end of file diff --git a/devtools/docs/README.md b/devtools/docs/README.md new file mode 100644 index 0000000000..37eabda0e2 --- /dev/null +++ b/devtools/docs/README.md @@ -0,0 +1,3 @@ +# Firefox Developer Tools: Contributor Docs + +This is a guide to working on the code for Firefox Developer Tools. If you're looking for help with using the tools, see the [user docs](https://developer.mozilla.org/en-US/docs/Tools). For other ways to get involved, check out our [community site](https://firefox-dev.tools). diff --git a/devtools/docs/about-documentation.md b/devtools/docs/about-documentation.md new file mode 100644 index 0000000000..fece8a6ee4 --- /dev/null +++ b/devtools/docs/about-documentation.md @@ -0,0 +1,27 @@ +# About this documentation + +This guide is built with MarkDown files and [GitBook](https://github.com/GitbookIO/gitbook). + +The source code for this documentation is distributed with the source code for the tools, in the `docs/` folder. + +If you want to contribute to the documentation, [clone the repository](./getting-started/build.md#getting-the-code), make your changes locally, and then regenerate the book to see how it looks like before submitting a patch: + +```bash +# Install GitBook locally +npm install -g gitbook-cli + +# Go into the docs directory +cd /path/to/mozilla-central/devtools/docs/ + +# Install plugins +gitbook install + +# Generate the docs and start a local server +gitbook serve + +# You can now navigate to localhost:4000 in your browser to see the output +open http://localhost:4000 # this would work in MacOS + +# Or build the book only (this places the output into `docs/_book`) +gitbook build +``` diff --git a/devtools/docs/backend/actor-best-practices.md b/devtools/docs/backend/actor-best-practices.md new file mode 100644 index 0000000000..3e29f51f60 --- /dev/null +++ b/devtools/docs/backend/actor-best-practices.md @@ -0,0 +1,38 @@ +# Actor Best Practices + +Some aspects of front and actor design can be tricky to understand, even for experienced engineers. +The following are several best practices you should keep in mind when adding new actors and fronts. + +## Actor Should Clean Up Itself, Don't Wait For the Client + +In the past, some actors would wait for the client to send a "you are done now" message when the toolbox closes to shutdown the actor. +This seems reasonable at first, but keep in mind that the connection can disappear at any moment. +It may not be possible for the client to send this message. + +A better choice is for the actor to do all clean up itself when it's notified that the connection goes away. +Then there's no need for the client to send any clean up message, and we know the actor will be in a good state no matter what. + +## Actor Destruction + +Ensure that the actor's destroy is really destroying everything that it should. Here's an example from the animation actor: + +```js +destroy: function() { + Actor.prototype.destroy.call(this); + this.targetActor.off("will-navigate", this.onWillNavigate); + this.targetActor.off("navigate", this.onNavigate); + + this.stopAnimationPlayerUpdates(); + this.targetActor = this.observer = this.actors = null; +}, +``` + +## Child Actors + +With protocol.js actors, if your creates child actors for further functionality, in most cases you should call: + +```js +this.manage(child); +``` + +in the parent after constructing the child, so that the child is destroyed when the parent is. diff --git a/devtools/docs/backend/actor-e10s-handling.md b/devtools/docs/backend/actor-e10s-handling.md new file mode 100644 index 0000000000..af532b0ac5 --- /dev/null +++ b/devtools/docs/backend/actor-e10s-handling.md @@ -0,0 +1,108 @@ +# How to handle E10S in actors + +In multi-process environments, most devtools actors are created and initialized in the child content process, to be able to access the resources they are exposing to the toolbox. But sometimes, these actors need to access things in the parent process too. Here's why and how. + +{% hint style="danger" %} + +This documentation page is **deprecated**. `setupInParent` relies on the message manager which is being deprecated. Furthermore, communications between parent and content processes should be avoided for security reasons. If possible, the client should be responsible for calling actors both on the parent and content process. + +This page will be removed when all actors relying on this API are removed. + +{% endhint %} + +## Use case and examples + +Some actors need to exchange messages between the parent and the child process (typically when some components aren't available in the child process). + +To that end, there's a parent/child setup mechanism at `DevToolsServer` level that can be used. + +When the actor is loaded for the first time in the `DevToolsServer` running in the child process, it may decide to run a setup procedure to load a module in the parent process with which to communicate. + +Example code for the actor running in the child process: + +``` + const {DevToolsServer} = require("devtools/server/devtools-server"); + + // Setup the child<->parent communication only if the actor module + // is running in a child process. + if (DevToolsServer.isInChildProcess) { + setupChildProcess(); + } + + function setupChildProcess() { + // `setupInParent` is defined on DevToolsServerConnection, + // your actor receives a reference to one instance in its constructor. + conn.setupInParent({ + module: "devtools/server/actors/module-name", + setupParent: "setupParentProcess" + }); + // ... + } +``` + +The `setupChildProcess` helper defined and used in the previous example uses the `DevToolsServerConnection.setupInParent` to run a given setup function in the parent process DevTools Server. + +With this, the `DevToolsServer` running in the parent process will require the requested module and call its `setupParentProcess` function (which should be exported on the module). + +The `setupParentProcess` function will receive a parameter that contains a reference to the **MessageManager** and a prefix that should be used to send/receive messages between the child and parent processes. + +See below an example implementation of a `setupParent` function in the parent process: + +``` +exports.setupParentProcess = function setupParentProcess({ mm, prefix }) { + // Start listening for messages from the actor in the child process. + setMessageManager(mm); + + function handleChildRequest(msg) { + switch (msg.json.method) { + case "get": + return doGetInParentProcess(msg.json.args[0]); + break; + case "list": + return doListInParentProcess(); + break; + default: + console.error("Unknown method name", msg.json.method); + throw new Error("Unknown method name"); + } + } + + function setMessageManager(newMM) { + if (mm) { + // Remove listener from old message manager + mm.removeMessageListener("debug:some-message-name", handleChildRequest); + } + // Switch to the new message manager for future use + // Note: Make sure that any other functions also use the new reference. + mm = newMM; + if (mm) { + // Add listener to new message manager + mm.addMessageListener("debug:some-message-name", handleChildRequest); + } + } + + return { + onDisconnected: () => setMessageManager(null), + }; +}; +``` + +The server will call the `onDisconnected` method returned by the parent process setup flow to give the actor modules the chance to cleanup their handlers registered on the disconnected message manager. + +## Summary of the setup flow + +In the child process: + +* The `DevToolsServer` loads an actor module, +* the actor module checks `DevToolsServer.isInChildProcess` to know whether it runs in a child process or not, +* the actor module then uses the `DevToolsServerConnection.setupInParent` helper to start setting up a parent-process counterpart, +* the `DevToolsServerConnection.setupInParent` helper asks the parent process to run the required module's setup function, +* the actor module uses the `DevToolsServerConnection.parentMessageManager.sendSyncMessage` and `DevToolsServerConnection.parentMessageManager.addMessageListener` helpers to send or listen to message. + +In the parent process: + +* The DevToolsServer receives the `DevToolsServerConnection.setupInParent` request, +* tries to load the required module, +* tries to call the `module[setupParent]` function with the frame message manager and the prefix as parameters `{ mm, prefix }`, +* the `setupParent` function then uses the mm to subscribe the message manager events, +* the `setupParent` function returns an object with a `onDisconnected` method which the server can use to notify the module of various lifecycle events diff --git a/devtools/docs/backend/actor-hierarchy.md b/devtools/docs/backend/actor-hierarchy.md new file mode 100644 index 0000000000..123f86a8fe --- /dev/null +++ b/devtools/docs/backend/actor-hierarchy.md @@ -0,0 +1,183 @@ +# How actors are organized + +To start with, actors are living within devtools/server/actors folder. +They are organized in a hierarchy for easier lifecycle and memory management: +once a parent is removed from the pool, its children are removed as well. +(See actor-registration.md for more information about how to implement one) + +The overall hierarchy of actors looks like this: + +``` +RootActor: First one, automatically instantiated when we start connecting. + | Mostly meant to instantiate new actors. + | + |-- Global-scoped actors: + | Actors exposing features related to the main process, that are not + | specific to any particular target (document, tab, add-on, or worker). + | These actors are registered with `global: true` in + | devtools/server/main.js. + | Examples include: + | PreferenceActor (for Firefox prefs) + | + \-- Target actors: + Actors that represent the main "thing" being targeted by a given toolbox, + such as a tab, frame, worker, add-on, etc. and track its lifetime. + Generally, there is a target actor for each thing you can point a + toolbox at. + Examples include: + FrameTargetActor (for a frame, such as a tab) + WorkerTargetActor (for various kind of workers) + | + \-- Target-scoped actors: + Actors exposing one particular feature set. They are children of a + given target actor and the data they return is filtered to reflect + the target. + These actors are registered with `target: true` in + devtools/server/main.js. + Examples include: + WebConsoleActor + InspectorActor + These actors may extend this hierarchy by having their own children, + like LongStringActor, WalkerActor, etc. +``` + +## RootActor + +The root actor is special. It is automatically created when a client connects. +It has a special `actorID` which is unique and is "root". +All other actors have an `actorID` which is computed dynamically, +so that you need to ask an existing actor to create an Actor +and returns its `actorID`. That's the main role of RootActor. + +``` +RootActor (root.js) + | + |-- TabDescriptorActor (descriptors/tab.js) + | Targets frames (such as a tab) living in the parent or child process. + | Note that this is just a proxy for FrameTargetActor, which is loaded via + | the frame's message manager as a frame script in the process containing + | the frame content. This proxy via message manager is always used, even + | when the content happens to be in the same process. + | Returned by "listTabs" or "getTab" requests. + | | + | \-- FrameTargetActor (frame.js) + | The "real" target actor for a frame (such as a tab) which runs in + | whichever process holds the content. FrameTargetActorProxy + | communicates with this via the frame's message manager. + | Extends the abstract class BrowsingContextTargetActor. + | Returned by "connect" server method on FrameTargetActorProxy. + | + |-- WorkerTargetActor (worker.js) + | Targets a worker (applies to various kinds like web worker, service + | worker, etc.). + | Returned by "listWorkers" request to the root actor to get all workers. + | Returned by "listWorkers" request to a FrameTargetActorProxy to get + | workers for a specific frame. + | Returned by "listWorkers" request to a ContentProcessTargetActor to get + | workers for the chrome of the child process. + | + |-- ChromeWindowTargetActor (chrome-window.js) + | Targets a single window, such as a browser window in Firefox, but it can + | be used to reach any window in the parent process. + | Extends the abstract class BrowsingContextTargetActor. + | Returned by "getWindow" request to the root actor. + | + |-- ParentProcessTargetActor (parent-process.js) + | Targets all resources in the parent process of Firefox (chrome documents, + | JSMs, JS XPCOM, etc.). + | Extends the abstract class BrowsingContextTargetActor. + | Extended by WebExtensionTargetActor. + | Returned by "getProcess" request without any argument. + | + |-- ContentProcessTargetActor (content-process.js) + | Targets all resources in a content process of Firefox (chrome sandboxes, + | frame scripts, documents, etc.) + | Returned by "getProcess" request with a id argument, matching the + | targeted process. + | + \-- WebExtensionActor (addon/webextension.js) + Represents a WebExtension add-on in the parent process. This gives some + metadata about the add-on and watches for uninstall events. This uses a + proxy to access the actual WebExtension in the WebExtension process via + the message manager. + Returned by "listAddons" request. + | + \-- WebExtensionTargetActor (targets/webextension.js) + Targets a WebExtension add-on. This runs in the WebExtension process. + The client issues an additional "connect" request to + WebExtensionActor to get this actor, which is different from the + approach used for frame target actors. + Extends ParentProcessTargetActor. + Returned by "connect" request to WebExtensionActor. +``` + +## Target Actors + +Those are the actors exposed by the root actors which are meant to track the +lifetime of a given target: tab, process, add-on, or worker. It also allows to +fetch the target-scoped actors connected to this target, which are actors like +console, inspector, thread (for debugger), style inspector, etc. + +Some target actors inherit from BrowsingContextTargetActor (defined in +browsing-context.js) which is meant for "browsing contexts" which present +documents to the user. It automatically tracks the lifetime of the targeted +browsing context, but it also tracks its iframes and allows switching the +target to one of its iframes. + +For historical reasons, target actors also handle creating the ThreadActor, used +to manage breakpoints in the debugger. Actors inheriting from +BrowsingContextTargetActor expose `attach`/`detach` requests, that allows to +start/stop the ThreadActor. + +Target-scoped actors are accessed via the target actor's RDP form which contains +the `actorID` for each target-scoped actor. + +The target-scoped actors expect to find the following properties on the target +actor: + - threadActor: + ThreadActor instance for the given target, + only defined once `attach` request is called, or on construction. + - isRootActor: (historical name) + Always false, except on ParentProcessTargetActor. + Despite the attribute name, it is being used to accept all resources + (like chrome one) instead of limiting only to content resources. + - makeDebugger: + Helper function used to create Debugger object for the target. + (See actors/utils/make-debugger.js for more info) + +In addition to this, the actors inheriting from BrowsingContextTargetActor, +expose many other attributes and events: + - window: + Reference to the window global object currently targeted. + It can change over time if we switch target to an iframe, so it + shouldn't be stored in a variable, but always retrieved from the actor. + - windows: + List of all document globals including the main window object and all + iframes. + - docShell: + Primary docShell reference for the targeted document. + - docShells: + List of all docShells for the targeted document and all its iframes. + - chromeEventHandler: + The chrome event handler for the current target. Allows to listen to events + that can be missing/cancelled on this document itself. + +See BrowsingContextTargetActor documentation for more details. + +## Target-scoped actors + +Each of these actors focuses on providing one particular feature set. They are +children of a given target actor. + +The data they return is filtered to reflect the target. For example, the +InspectorActor that you fetch from a FrameTargetActor gives you information +about the markup and styles for only that frame. + +These actors may extend this hierarchy by having their own children, like +LongStringActor, WalkerActor, etc. + +To improve performance, target-scoped actors are created lazily. The target +actor lists the actor ID for each one, but the actor modules aren't actually +loaded and instantiated at that point. Once the first request for a given +target-scoped actor is received by the server, that specific actor is +instantiated just in time to service the request. diff --git a/devtools/docs/backend/actor-registration.md b/devtools/docs/backend/actor-registration.md new file mode 100644 index 0000000000..c43fef29a7 --- /dev/null +++ b/devtools/docs/backend/actor-registration.md @@ -0,0 +1,39 @@ +# How to register an actor + +## Target-scoped actors vs. global actors + +Target-scoped actors are the most common types of actors. That's the type of actors you will most probably be adding. + +Target-scoped actors target a document, this could be a tab in Firefox or a remote document in Firefox for Android. + +Global actors however are for the rest, for things not related to any particular document but instead for things global to the whole Firefox/Chrome/Safari instance the toolbox is connected to (e.g. the preference actor). + +## The ActorRegistry.registerModule function + +To register a target-scoped actor: + +``` +ActorRegistry.registerModule("devtools/server/actors/webconsole", { + prefix: "console", + constructor: "WebConsoleActor", + type: { target: true } +}); +``` + +To register a global actor: + +``` +ActorRegistry.registerModule("devtools/server/actors/preference", { + prefix: "preference", + constructor: "PreferenceActor", + type: { global: true } +}); +``` + +If you are adding a new built-in actor, you should be registering it using `ActorRegistry.registerModule` in `addBrowserActors` or `addTargetScopedActors` in `/devtools/server/actors/utils/actor-registry.js`. + +## A note about lazy registration + +The `ActorRegistry` loads and creates all of the actors lazily to keep the initial memory usage down (which is extremely important on lower end devices). + +It becomes especially important when debugging pages with e10s when there are more than one process, because that's when we need to spawn a `DevToolsServer` per process (it may not be immediately obvious that the server in the main process is mostly only here for piping messages to the actors in the child process). diff --git a/devtools/docs/backend/backward-compatibility.md b/devtools/docs/backend/backward-compatibility.md new file mode 100644 index 0000000000..8166d75dd8 --- /dev/null +++ b/devtools/docs/backend/backward-compatibility.md @@ -0,0 +1,90 @@ +# Backward Compatibility + +## Overview + +When making changes to the DevTools, there are certain backward compatibility requirements that we should keep in mind. + +In general, we should strive to maintain feature support for existing servers as we continue to make changes to the code base. At times, this can be difficult to achieve, however. + +## Specific Guidelines + +The important compatibility scenarios are: + +- Nightly desktop client **MUST** maintain existing compatibility back to release channel servers. + +This is mainly to simplify cross-platform use cases, i.e. desktop Nightly with release Fennec. + +- Servers **MAY** use traits to state a feature is not supported yet. + +This helps us support alternate environments, which does not implement every possible server feature. + +Certainly when a new feature needs a new actor method to function, it won't work with servers that don't support it. But we should still ensure the client doesn't explode when using unrelated, existing features, at least until the above time windows have elapsed. + +## Testing + +The harder part of this currently is that there is no automated testing to ensure the above guidelines have been met. While we hope to have this at some point, for now manual testing is needed here. + +The easiest way to test this is to check your work against a Firefox for Android device on release channel to ensure existing features in the area you are changing continue to function. That doesn't cover every case, but it's a great start. + +Alternatively, you can connect to a Firefox release server. This can be done in multiple steps: + +1. Start Firefox release from the command line, specifying the `--start-debugger-server` with an available port (e.g. `/Applications/Firefox.app/Contents/MacOS/firefox --start-debugger-server 6081`) +2. Navigate to a page where you can check that the part of DevTools which is impacted by the patch still works. +3. Build and run Firefox locally with the patch you want to check +4. In this build, open an `about:debugging` tab +5. On the `Network Location` section, fill in the host with localhost and the debugger server port you started the Firefox release instance with (e.g. `localhost:6081`) and hit Enter (or the `Add` button) +6. A new item will appear in the sidebar, click on its `Connect` button. +7. Accept the `Incoming connection` popup that appears +8. Click on the on sidebar item again. You will now see a list of the tabs and workers running in the Firefox release instance. Click on the `Inspect` button next to them to open a toolbox that is connected to the older server. + +## Feature Detection + +Starting with Firefox 36 (thanks to [bug 1069673](https://bugzilla.mozilla.org/show_bug.cgi?id=1069673)), you can use actor feature detection to determine which actors exist. + +### Target hasActor helper + +Detecting if the server has an actor: all you need is access to the `Toolbox` instance, which all panels do, when they get instantiated. Then you can do: + +```js +let hasPerformanceActor = toolbox.target.hasActor("performance"); +``` + +The `hasActor` method returns a boolean synchronously. + +### Traits + +Expose traits on an Actor in order to flag certain features as available or not. For instance if a new method "someMethod" is added to an Actor, expose a "supportsSomeMethod" flag in the traits object for the Actor, set to true. When debugging older servers, the flag will be missing and will default to false. + +Traits need to be forwarded to the client, and stored or used by the corresponding Front. There is no unique way of exposing traits, but there are still a few typical patterns found in the codebase. + +For Actors using a "form()" method, for which the Front is automatically created by protocol.js, the usual pattern is to add a "traits" property to the form, that contains all the traits for the actor. The Front can then read the traits in its corresponding "form()" method. Example: + +- [NodeActor form method](https://searchfox.org/mozilla-central/rev/e75e8e5b980ef18f4596a783fbc8a36621de7d1e/devtools/server/actors/inspector/node.js#209) +- [NodeFront form method](https://searchfox.org/mozilla-central/rev/e75e8e5b980ef18f4596a783fbc8a36621de7d1e/devtools/client/fronts/node.js#145) + +For other Actors, there are two options. First option is to define the trait on the Root actor. Those traits will be available both via TargetMixin::getTrait(), and on DevToolsClient.traits. The second option is to implement a "getTraits()" method on the Actor, which will return the traits for the Actor. Example: + +- [CompatibilityActor getTraits method](https://searchfox.org/mozilla-central/rev/e75e8e5b980ef18f4596a783fbc8a36621de7d1e/devtools/shared/specs/compatibility.js#40) +- [CompatibilitySpec getTraits definition](https://searchfox.org/mozilla-central/rev/e75e8e5b980ef18f4596a783fbc8a36621de7d1e/devtools/shared/specs/compatibility.js#40-43) +- [CompatibilityFront getTraits method](https://searchfox.org/mozilla-central/rev/e75e8e5b980ef18f4596a783fbc8a36621de7d1e/devtools/client/fronts/compatibility.js#41-47) + +Ironically, "getTraits" needs to be handled with backwards compatibility. But there is no way to check that "getTraits" is available on the server other than performing a try catch around the method. See the CompatibilityFront example. + +Whenever traits are added, make sure to add a relevant backward compatibility comment so that we know when the trait can be removed. + +## Maintaining backward compatibility code + +When introducing backward compatibility code, a comment should be added for extra information. +In order to simplify future code cleanups, the comment should follow the following syntax: +`// @backward-compat { version XX } Detailed comment`, where `XX` is the Firefox version this code was added in. + +Below is a made-up example of what it should look like: + +```js +// @backward-compat { version 85 } For older server which don't have the AwesomeActor, +// we have to do this another way. +if (!toolbox.target.hasActor("awesome")) { +``` + +Backward compatibility code can be safely removed when the revision it was added in reaches the release channel. +So if something landed in Firefox Nightly 85, it can be removed when Firefox 85 is released, i.e. when Firefox Nightly is 87. Search for the corresponding `@backward-compat` entries to retrieve all the code that can be removed. diff --git a/devtools/docs/backend/client-api.md b/devtools/docs/backend/client-api.md new file mode 100644 index 0000000000..818ab54bcd --- /dev/null +++ b/devtools/docs/backend/client-api.md @@ -0,0 +1,252 @@ +# Client API + +DevTools has a client module that allows applications to be written that debug or inspect web pages using the [Remote Debugging Protocol](protocol.md). + +## Starting communication + +In order to communicate, a client and a server instance must be created and a protocol connection must be established. The connection can be either over a TCP socket or an nsIPipe. The `start` function displayed below establishes an nsIPipe-backed connection: + +```javascript +const { DevToolsServer } = require("devtools/server/devtools-server"); +const { DevToolsClient } = require("devtools/client/devtools-client"); + +function start() { + // Start the server. + DevToolsServer.init(); + DevToolsServer.registerAllActors(); + + // Listen to an nsIPipe + let transport = DevToolsServer.connectPipe(); + + // Start the client. + client = new DevToolsClient(transport); + + client.connect((type, traits) => { + // Now the client is connected to the server. + debugTab(); + }); +} +``` + +If a TCP socket is required, the function should be split in two parts, a server-side and a client-side, like this: + +```javascript +const { DevToolsServer } = require("devtools/server/devtools-server"); +const { DevToolsClient } = require("devtools/client/devtools-client"); + +function startServer() { + // Start the server. + DevToolsServer.init(); + DevToolsServer.registerAllActors(); + + // For an nsIServerSocket we do this: + DevToolsServer.openListener(2929); // A connection on port 2929. +} + +async function startClient() { + let transport = await DevToolsClient.socketConnect({ host: "localhost", port: 2929 }); + + // Start the client. + client = new DevToolsClient(transport); + + client.connect((type, traits) => { + // Now the client is connected to the server. + debugTab(); + }); +} +``` + +## Shutting down + +When the application is finished, it has to notify the client to shut down the protocol connection. This makes sure that memory leaks are avoided and the server is terminated in an orderly fashion. Shutting down is as simple as it gets: + +```javascript +function shutdown() { + client.close(); +} +``` + +## Attaching to a browser tab + +Attaching to a browser tab requires enumerating the available tabs and attaching to one: + +```javascript +function attachToTab() { + // Get the list of tabs to find the one to attach to. + client.mainRoot.listTabs().then(tabs => { + // Find the active tab. + let targetFront = tabs.find(tab => tab.selected); + + // Attach to the tab. + targetFront.attach().then(() => { + // Now the targetFront is ready and can be used. + + // Attach listeners for client events. + targetFront.on("tabNavigated", onTab); + }); + }); +} +``` + +The devtools client will send event notifications for a number of events the application may be interested in. These events include state changes in the debugger, like pausing and resuming, stack frames or source scripts being ready for retrieval, etc. + +## Handling location changes + +When the user navigates away from a page, a `tabNavigated` event will be fired. The proper way to handle this event is to detach from the previous thread and tab and attach to the new ones: + +```javascript +async function onTab() { + // Detach from the previous tab. + await targetFront.detach(); + // Start debugging the new tab. + start(); +} +``` + +## Debugging JavaScript running in a browser tab + +Once the application is attached to a tab, it can attach to its thread in order to interact with the JavaScript debugger: + +```javascript +// Assuming the application is already attached to the tab, and response is the first +// argument of the attachTarget callback. + +client.attachThread(response.threadActor).then(function(threadFront) { + if (!threadFront) { + return; + } + + // Attach listeners for thread events. + threadFront.on("paused", onPause); + threadFront.on("resumed", fooListener); + + // Debugger is now ready and debuggee is running. +}); +``` + +## Debugger application example + +Here is the source code for a complete debugger application: + +```javascript +/* + * Debugger API demo. + */ +const { DevToolsServer } = require("devtools/server/devtools-server"); +const { DevToolsClient } = require("devtools/client/devtools-client"); + +let client; +let threadFront; + +function startDebugger() { + // Start the server. + DevToolsServer.init(); + DevToolsServer.registerAllActors(); + // Listen to an nsIPipe + let transport = DevToolsServer.connectPipe(); + // For an nsIServerSocket we do this: + // DevToolsServer.openListener(port); + // ...and this at the client: + // let transport = debuggerSocketConnect(host, port); + + // Start the client. + client = new DevToolsClient(transport); + client.connect((type, traits) => { + // Now the client is connected to the server. + debugTab(); + }); +} + +function shutdownDebugger() { + client.close(); +} + +/** + * Start debugging the current tab. + */ +async function debugTab() { + // Get the list of tabs to find the one to attach to. + const tabs = await client.mainRoot.listTabs(); + // Find the active tab. + let targetFront = tabs.find(tab => tab.selected); + // Attach to the tab. + await targetFront.attach(); + // Attach to the thread (context). + const threadFront = await targetFront.attachThread(); + // Attach listeners for thread events. + threadFront.on("paused", onPause); + threadFront.on("resumed", fooListener); + // Debugger is now ready and debuggee is running. +} + +/** + * Handler for location changes. + */ +function onTab() { + // Detach from the previous tab. + client.detach().then(() => { + // Start debugging the new tab. + debugTab(); + }); +} + +/** + * Helper function to inspect the provided frame. + */ +function inspectFrame(frame) { + // Get the "this" object. + if (frame["this"]) { + getObjectProperties(frame["this"]); + } + + // Add "arguments". + if (frame.arguments && frame.arguments.length > 0) { + // frame.arguments is a regular Array. + dump("frame.arguments: " + frame.arguments.toSource() + "\n"); + + // Add variables for every argument. + let objClient = client.activeThread.pauseGrip(frame.callee); + objClient.getSignature(response => { + for (let i = 0; i < response.parameters.length; i++) { + let name = response.parameters[i]; + let value = frame.arguments[i]; + + if (typeof value == "object" && value.type == "object") { + getObjectProperties(value); + } + } + }); + } +} + +/** + * Helper function that retrieves the specified object's properties. + */ +function getObjectProperties(object) { + let thisClient = client.activeThread.pauseGrip(object); + thisClient.getPrototypeAndProperties(response => { + // Get prototype as a protocol-specified grip. + if (response.prototype.type != "null") { + dump("__proto__: " + response.prototype.toSource() + "\n"); + } + + // Get the rest of the object's own properties as protocol-specified grips. + for (let prop of Object.keys(response.ownProperties)) { + dump(prop + ": " + response.ownProperties[prop].toSource() + "\n"); + } + }); +} + +/** + * Generic event listener. + */ +function fooListener(event) { + dump(event + "\n"); +} + +// Run the program. +startDebugger(); + +// Execute the following line to stop the program. +//shutdownDebugger(); +``` diff --git a/devtools/docs/backend/debugger-api.md b/devtools/docs/backend/debugger-api.md new file mode 100644 index 0000000000..344cf7eddc --- /dev/null +++ b/devtools/docs/backend/debugger-api.md @@ -0,0 +1,5 @@ +# Debugger API + +The Debugger API is a low-level API that provides methods for introspecting and affecting a target environment like a page. You can find JavaScript sources, set breakpoints on them, and more. + +This API is completely documented on the [Debugger API](https://developer.mozilla.org/en-US/docs/Tools/Debugger-API) MDN page. diff --git a/devtools/docs/backend/protocol.js.md b/devtools/docs/backend/protocol.js.md new file mode 100644 index 0000000000..568344209e --- /dev/null +++ b/devtools/docs/backend/protocol.js.md @@ -0,0 +1,577 @@ +Writing an Actor +================ + +A Simple Hello World +-------------------- + +Here's a simple Hello World actor. It is a global actor (not associated with a given browser tab). +It has two parts: a spec and an implementation. The spec would go somewhere like +`devtools/shared/specs/hello-world.js` and would look like: + + const {Arg, RetVal, generateActorSpec} = require("devtools/shared/protocol"); + + const helloWorldSpec = generateActorSpec({ + typeName: "helloWorld", // I'll explain types later, I promise. + + methods: { + sayHello: { + // The request packet template. There are no arguments, so + // it is empty. The framework will add the "type" and "to" + // request properties. + request: {}, + + // The response packet template. The return value of the function + // will be plugged in where the RetVal() appears in the template. + response: { + greeting: RetVal("string") // "string" is the return value type. + } + }, + }, + }); + + // Expose the spec so it can be imported by the implementation. + exports.helloWorldSpec = helloWorldSpec; + +The actor implementation would go somewhere like +`devtools/server/actors/hello-world.js` and would look like: + + const protocol = require("devtools/shared/protocol"); + const {helloWorldSpec} = require("devtools/shared/specs/hello-world"); + + const HelloActor = protocol.ActorClassWithSpec(helloWorldSpec, { + initialize: function (conn) { + protocol.Actor.prototype.initialize.call(this, conn); // This is the worst part of heritage. + }, + + sayHello: function () { + return "hello"; + }, + }); + + // You also need to export the actor class in your module for discovery. + exports.HelloActor = HelloActor; + +To activate your actor, register it in the `addBrowserActors` method in `server/actors/utils/actor-registry.js`. +The registration code would look something like this: + + this.registerModule("devtools/server/actors/hello-world", { + prefix: "hello", + constructor: "HelloActor", + type: { global: true } + }); + +Your spec allows the actor to support a `sayHello` request. +A request/reply will look like this: + + -> { to: <actorID>, type: "sayHello" } + <- { from: <actorID>, greeting: "hello" } + +Now we can create a client side object. We call these *front* objects and +they typically go in `devtools/client/fronts/`. + +Here's the front for the HelloActor: + + const HelloFront = protocol.FrontClassWithSpec(helloWorldSpec, { + initialize: function (client, form) { + protocol.Front.prototype.initialize.call(this, client, form); + // This call may not be required but it's a good idea. It will + // guarantee that your instance is managed in the pool. + this.manage(this); + } + }); + +Note that there is no `sayHello` method. The FrontClass will generate a method on the Front object that matches the method declaration in the Actor class. + +The generated methods will return a Promise. That promise will resolve to the RetVal of the actor method. + +So if we have a reference to a HelloFront object, we can issue a `sayHello` request: + + hello.sayHello().then(greeting => { + console.log(greeting); + }); + +How do you get an initial reference to the front? That's a bit tricky, but basically there are two ways: + +* Manually +* Magically + +Manually - If you're using a DevToolsClient instance, you can discover the actorID manually and create a Front for it: + + let hello = new HelloFront(this.client, { actor: <hello actorID> }); + +Magically - Once you have an initial reference to a protocol.js object, it can return other protocol.js objects and fronts will automatically be created. + +Arguments +--------- + +`sayHello` has no arguments, so let's add a method that does take arguments. +Here's an adjustment to the spec: + + methods: { + echo: { + request: { echo: Arg(0, "string") }, + response: { echoed: RetVal("string") } + } + } + +Here's an adjustment to the implementation: + + echo: function (str) { + return str + "... " + str + "..."; + } + +This tells the library to place the 0th argument, which should be a string, in the `echo` property of the request packet. + + +This will generate a request handler whose request and response packets look like this: + + { to: <actorID>, type: "echo", echo: <str> } + { from: <actorID>, echoed: <str> } + +The client usage should be predictable: + + hello.echo("hello").then(str => { assert(str === "hello... hello...") }) + +The library tries hard to make using fronts feel like natural javascript (or as natural as you believe promises are, I guess). When building the response it will put the return value of the function where RetVal() is specified in the response template, and on the client side it will use the value in that position when resolving the promise. + +Returning JSON +-------------- + +Maybe your response is an object. Here's an example of a spec: + + methods: { + addOneTwice: { + request: { a: Arg(0, "number"), b: Arg(1, "number") }, + response: { ret: RetVal("json") } + } + } + +Here's an example implementation: + + addOneTwice: function (a, b) { + return { a: a + 1, b: b + 1 }; + } + +This will generate a response packet that looks like: + + { from: <actorID>, ret: { a: <number>, b: <number> } } + +That's probably unnecessary nesting (if you're sure you won't be returning an object with 'from' as a key!), so you can just replace `response` with: + + response: RetVal("json") + +and now your packet will look like: + + { from: <actorID>, a: <number>, b: <number> } + +Types and Marshalling +--------------------- + +Things have been pretty simple up to this point - all the arguments we've passed in have been javascript primitives. But for some types (most importantly Actor types, which I'll get to eventually), we can't just copy them into a JSON packet and expect it to work, we need to marshal things ourselves. + +Again, the protocol lib tries hard to provide a natural API to actors and clients, and sometime that natural API might involve object APIs. I'm going to use a wickedly contrived example, bear with me. Let's say I have a small object that contains a number and has a few methods associated with it: + + let Incrementor = function (i) { + this.value = value; + } + Incrementor.prototype = { + increment: function () { this.value++ }, + decrement: function () { this.value-- } + }; + + +and I want to return it from a backend function: + + // spec: + methods: { + getIncrementor: { + request: { number: Arg(0, "number") }, + response: { value: RetVal("incrementor") } // We'll define "incrementor" below. + } + } + + // implementation: + getIncrementor: function (i) { + return new Incrementor(i) + } + +I want that response to look like `{ from: <actorID>, value: <number> }`, but the client side needs to know to return an Incrementor, not a primitive number. So let's tell the protocol lib about Incrementors: + + protocol.types.addType("incrementor", { + // When writing to a protocol packet, just send the value + write: (v) => v.value, + + // When reading from a protocol packet, wrap with an Incrementor + // object. + read: (v) => new Incrementor(v) + }); + +And now our client can use the API as expected: + + front.getIncrementor(5).then(incrementor => { + incrementor.increment(); + assert(incrementor.value === 6); + }); + +You can do the same thing with arguments: + + // spec: + methods: { + passIncrementor: { + request: { Arg(0, "incrementor") }, + } + } + + // implementation: + passIncrementor: function (inc) { + w.increment(); + assert(incrementor.value === 6); + } + + front.passIncrementor(new Incrementor(5)); + +The library provides primitiive `boolean`, `number`, `string`, and `json` types. + +Moving right along, let's say you want to pass/return an array of Incrementors. You can just prepend `array:` to the type name: + + // spec: + methods: { + incrementAll: { + request: { incrementors: Arg(0, "array:incrementor") }, + response: { incrementors: RetVal("array:incrementor") } + } + } + + // implementation: + incrementAll: function (incrementors) { + incrementors.forEach(incrementor => { + incrementor.increment(); + } + return incrementors; + } + +You can use an iterator in place of an array as an argument or return value, and the library will handle the conversion automatically. + +Or maybe you want to return a dictionary where one item is a incrementor. To do this you need to tell the type system which members of the dictionary need custom marshallers: + + protocol.types.addDictType("contrivedObject", { + incrementor: "incrementor", + incrementorArray: "array:incrementor" + }); + + // spec: + methods: { + reallyContrivedExample: { + response: RetVal("contrivedObject") + } + } + + // implementations: + reallyContrivedExample: function () { + return { + /* a and b are primitives and so don't need to be called out specifically in addDictType */ + a: "hello", b: "world", + incrementor: new Incrementor(1), + incrementorArray: [new Incrementor(2), new Incrementor(3)] + } + } + + front.reallyContrivedExample().then(obj => { + assert(obj.a == "hello"); + assert(obj.b == "world"); + assert(incrementor.i == 1); + assert(incrementorArray[0].i == 2); + assert(incrementorArray[1].i == 3); + }); + +Nullables +--------- + +If an argument, return value, or dict property can be null/undefined, you can prepend `nullable:` to the type name: + + "nullable:incrementor", // Can be null/undefined or an incrementor + "array:nullable:incrementor", // An array of incrementors that can have holes. + "nullable:array:incrementor" // Either null/undefined or an array of incrementors without holes. + + +Actors +------ + +Probably the most common objects that need custom martialing are actors themselves. These are more interesting than the Incrementor object, but by default they're somewhat easy to work with. Let's add a ChildActor implementation that will be returned by the HelloActor (which is rapidly becoming the OverwhelminglyComplexActor): + + // spec: + const childActorSpec = generateActorSpec({ + actorType: "childActor", + methods: { + getGreeting: { + response: { greeting: RetVal("string") }, + } + } + }); + + // implementation: + const ChildActor = protocol.ActorClassWithSpec(childActorSpec, { + initialize: function (conn, id) { + protocol.Actor.prototype.initialize.call(this, conn); + this.greeting = "hello from " + id; + }, + getGreeting: function () { + return this.greeting; + }, + }); + + exports.ChildActor = ChildActor; + + const ChildFront = protocol.FrontClassWithSpec(childActorSpec, { + initialize: function (client, form) { + protocol.Front.prototype.initialize.call(this, client, form); + }, + }); + +The library will register a marshaller for the actor type itself, using typeName as its tag. + +So we can now add the following code to HelloActor: + + // spec: + methods: { + getChild: { + request: { id: Arg(0, "string") }, + response: { child: RetVal("childActor") } + } + } + + // implementation: + getChild: function (id) { + return ChildActor(this.conn, id); + } + + front.getChild("child1").then(childFront => { + return childFront.getGreeting(); + }).then(greeting => { + assert(id === "hello from child1"); + }); + +The conversation will look like this: + + { to: <actorID>, type: "getChild", id: "child1" } + { from: <actorID>, child: { actor: <childActorID> }} + { to: <childActorID>, type: "getGreeting" } + { from: <childActorID>, greeting: "hello from child1" } + +But the ID is the only interesting part of this made-up example. You're never going to want a reference to a ChildActor without checking its ID. Making an extra request just to get that id is wasteful. You really want the first response to look like `{ from: <actorID>, child: { actor: <childActorID>, greeting: "hello from child1" } }` + +You can customize the marshalling of an actor by providing a `form` method in the `ChildActor` class: + + form: function () { + return { + actor: this.actorID, + greeting: this.greeting + } + }, + +And you can demarshal in the `ChildFront` class by implementing a matching `form` method: + + form: function (form) { + this.actorID = form.actor; + this.greeting = form.greeting; + } + +Now you can use the id immediately: + + front.getChild("child1").then(child => { assert(child.greeting === "child1) }); + +You may come across a situation where you want to customize the output of a `form` method depending on the operation being performed. For example, imagine that ChildActor is a bit more complex, with a, b, c, and d members: + + ChildActor: + form: function () { + return { + actor: this.actorID, + greeting: this.greeting, + a: this.a, + b: this.b, + c: this.c, + d: this.d + } + } + ChildFront: + form: function (form) { + this.actorID = form.actorID; + this.id = form.id; + this.a = form.a; + this.b = form.b; + this.c = form.c; + this.d = form.d; + } + +And imagine you want to change 'c' and return the object: + + // Oops! If a type is going to return references to itself or any other + // type that isn't fully registered yet, you need to predeclare the type. + types.addActorType("childActor"); + + ... + + // spec: + methods: { + changeC: { + request: { newC: Arg(0) }, + response: { self: RetVal("childActor") } + } + } + + // implementation: + changeC: function (newC) { + c = newC; + return this; + } + + ... + + childFront.changeC('hello').then(ret => { assert(ret === childFront); assert(childFront.c === "hello") }); + +Now our response will look like: + + { from: <childActorID>, self: { actor: <childActorID>, greeting: <id>, a: <a>, b: <b>, c: "hello", d: <d> } + + +Lifetimes +--------- + +No, I don't want to talk about lifetimes quite yet. + +Events +------ + +Your actor has great news! + +Actors are subclasses of jetpack `EventTarget`, so you can just emit events. +Here's how you'd set it up in a spec: + + events: { + "good-news": { + type: "goodNews", // event target naming and packet naming are at odds, and we want both to be natural! + news: Arg(0) + } + } + + methods: { + giveGoodNews: { + request: { news: Arg(0) } + } + } + +Here's how the implementation would look: + + const EventEmitter = require("devtools/shared/event-emitter"); + + // In your protocol.ActorClassWithSpec definition: + giveGoodNews: function (news) { + EventEmitter.emit(this, "good-news", news); + } + +Now you can listen to events on a front: + + front.on("good-news", news => { + console.log(`Got some good news: ${news}\n`); + }); + front.giveGoodNews().then(() => { console.log("request returned.") }); + +If you want to modify the argument that will be passed to event listeners callbacks, you +can use `before(eventName, fn)` in the front definition. This can only be used once for a +given `eventName`. The `fn` function will be called before emitting the event via +the EventEmitter API on the Front, and its return value will be passed to the event +listener callbacks. If `fn` is async, the event will only be emitted after `fn` call resolves. + + // In front file, most probably in the constructor: + this.before("good-news", function(news) { + return news.join(" - "); + }); + + // In any consumer + front.on("good-news", function(news) { + console.log(news); + }); + +So if the server sent the following array: `[1, 2, 3]`, the console.log in the consumer +would print `1 - 2 - 3`. + +On a somewhat related note, not every method needs to be request/response. Just like an actor can emit a one-way event, a method can be marked as a one-way request. Maybe we don't care about giveGoodNews returning anything: + + // spec: + methods: { + giveGoodNews: { + request: { news: Arg(0, "string") }, + oneway: true + } + } + + // implementation: + giveGoodNews: function (news) { + emit(this, "good-news", news); + } + +Lifetimes +--------- + +No, let's talk about custom front methods instead. + +Custom Front Methods +-------------------- + +You might have some bookkeeping to do before issuing a request. Let's say you're calling `echo`, but you want to count the number of times you issue that request. Just use the `custom` tag in your front implementation: + + echo: custom(function (str) { + this.numEchos++; + return this._echo(str); + }, { + impl: "_echo" + }) + +This puts the generated implementation in `_echo` instead of `echo`, letting you implement `echo` as needed. If you leave out the `impl`, it just won't generate the implementation at all. You're on your own. + +Lifetimes +--------- + +OK, I can't think of any more ways to put this off. The remote debugging protocol has the concept of a *parent* for each actor. This is to make distributed memory management a bit easier. Basically, any descendents of an actor will be destroyed if the actor is destroyed. + +Other than that, the basic protocol makes no guarantees about lifetime. Each interface defined in the protocol will need to discuss and document its approach to lifetime management (although there are a few common patterns). + +The protocol library will maintain the child/parent relationships for you, but it needs some help deciding what the child/parent relationships are. + +The default parent of an object is the first object that returns it after it is created. So to revisit our earlier HelloActor `getChild` implementation: + + // spec: + methods: { + getChild: { + request: { id: Arg(0) }, + response: { child: RetVal("childActor") } + } + } + + // implementation: + getChild: function (id) { + return new ChildActor(this.conn, id); + } + +The ChildActor's parent is the HelloActor, because it's the one that created it. + +You can customize this behavior in two ways. The first is by defining a `marshallPool` property in your actor. Imagine a new ChildActor method: + + // spec: + methods: { + getSibling: { + request: { id: Arg(0) }, + response: { child: RetVal("childActor") } + } + } + + // implementation: + getSibling: function (id) { + return new ChildActor(this.conn, id); + } + +This creates a new child actor owned by the current child actor. But in this example we want all actors created by the child to be owned by the HelloActor. So we can define a `defaultParent` property that makes use of the `parent` property provided by the Actor class: + + get marshallPool() { return this.parent } + +The front needs to provide a matching `defaultParent` property that returns an owning front, to make sure the client and server lifetimes stay synced. diff --git a/devtools/docs/backend/protocol.md b/devtools/docs/backend/protocol.md new file mode 100644 index 0000000000..199a0f5ffe --- /dev/null +++ b/devtools/docs/backend/protocol.md @@ -0,0 +1,1609 @@ +# Remote Debugging Protocol + +The Mozilla debugging protocol allows a debugger to connect to a browser, discover what sorts of things are present to debug or inspect, select JavaScript threads to watch, and observe and modify their execution. The protocol provides a unified view of JavaScript, DOM nodes, CSS rules, and the other technologies used in client-side web applications. The protocol ought to be sufficiently general to be extended for use with other sorts of clients (profilers, say) and servers (mail readers; random XULrunner applications). + +All communication between debugger (client) and browser (server) is in the form of JSON objects. This makes the protocol directly readable by humans, capable of graceful evolution, and easy to implement using stock libraries. In particular, it should be easy to create mock implementations for testing and experimentation. + +The protocol operates at the JavaScript level, not at the C++ or machine level, and assumes that the JavaScript implementation itself is healthy and responsive. The JavaScript program being executed may well have gone wrong, but the JavaScript implementation's internal state must not be corrupt. Bugs in the implementation may cause the debugger to fail; bugs in the interpreted program must not. + +## General Conventions + +### Actors + +An **actor** is something on the server that can exchange JSON packets with the client. Every packet from the client specifies the actor to which it is directed, and every packet from the server indicates which actor sent it. + +Each server has a root actor, with which the client first interacts. The root actor can explain what sort of thing the server represents (browser; mail reader; etc.), and enumerate things available to debug: tabs, chrome, and so on. Each of these, in turn, is represented by an actor to which requests can be addressed. Both artifacts of the program being debugged, like JavaScript objects and stack frames, and artifacts of the debugging machinery, like breakpoints and watchpoints, are actors with whom packets can be exchanged. + +For example, a debugger might connect to a browser, ask the root actor to list the browser's tabs, and present this list to the developer. If the developer chooses some tabs to debug, then the debugger can send `attach` requests to the actors representing those tabs, to begin debugging. + +Actor names are JSON strings, containing no spaces or colons. The name of the root actor is `"root"`. + +To allow the server to reuse actor names and the resources they require, actors have limited lifetimes. All actors in a server form a tree, whose root is the root actor. Closing communications with an actor automatically closes communications with its descendants. For example, the actors representing a thread's stack frames are children of the actor representing the thread itself, so that when a debugger detaches from a thread, which closes the thread's actor, the frames' actors are automatically closed. This arrangement allows the protocol to mention actors liberally, without making the client responsible for explicitly closing every actor that has ever been mentioned. + +When we say that some actor *A* is a child of some actor *B*, we mean that *A* is a direct child of *B*, not a grandchild, great-grandchild, or the like. Similarly, **parent** means "direct parent". We use the terms **ancestor** and **descendent** to refer to those looser relationships. + +The root actor has no parent, and lives as long as the underlying connection to the client does; when that connection is closed, all actors are closed. + +Note that the actor hierarchy does not, in general, correspond to any particular hierarchy appearing in the debuggee. For example, although web workers are arranged in a hierarchy, the actors representing web worker threads are all children of the root actor: one might want to detach from a parent worker while continuing to debug one of its children, so it doesn't make sense to close communications with a child worker simply because one has closed communications with its parent. + +*(We are stealing the "actor" terminology from Mozilla's IPDL, to mean, roughly, "things participating in the protocol". However, IPDL does much more with the idea than we do: it treats both client and server as collections of actors, and uses that detail to statically verify properties of the protocol. In contrast, the debugging protocol simply wants a consistent way to indicate the entities to which packets are directed.)* + +### Packets + +The protocol is carried by a reliable, bi-directional byte stream; data sent in both directions consists of JSON objects, called packets. A packet is a top-level JSON object, not contained inside any other value. + +Every packet sent from the client has the form: + +``` +{ "to":actor, "type":type, ... } +``` + +where `actor` is the name of the actor to whom the packet is directed and `type` is a string specifying what sort of packet it is. Additional properties may be present, depending on `type`. + +Every packet sent from the server has the form: + +``` +{ "from":actor, ... } +``` + +where `actor` is the name of the actor that sent it. The packet may have additional properties, depending on the situation. + +If a packet is directed to an actor that no longer exists, the server sends a packet to the client of the following form: + +``` +{ "from":actor, "error":"noSuchActor" } +``` + +where `actor` is the name of the non-existent actor. (It is strange to receive messages from actors that do not exist, but the client evidently believes that actor exists, and this reply allows the client to pair up the error report with the source of the problem.) + +Clients should silently ignore packet properties they do not recognize. We expect that, as the protocol evolves, we will specify new properties that can appear in existing packets, and experimental implementations will do the same. + +### Common Patterns of Actor Communication + +Each type of actor specifies which packets it can receive, which it might send, and when it can do each. Although in principle these interaction rules could be complex, in practice most actors follow one of two simple patterns: + +* **Request/Reply**: Each packet sent to the actor ("request") elicits a single packet in response ("reply"). +* **Request/Reply/Notify**: Like Request/Reply, but the actor may send packets that are not in response to any specific request ("notification"), perhaps announcing events that occur spontaneously in the debuggee. + +These patterns are described in more detail below. + +Some actors require more complicated rules. For example, the set of packets accepted by a [Thread-like actor](#interacting-with-thread-like-actors) depends on which one of four states it occupies. The actor may spontaneously transition from one state to another, and not all state transitions produce notification packets. Actors like this require careful specification. + +#### The Request/Reply Pattern + +In this specification, if we call a packet a **request**, then it is a packet sent by the client, which always elicits a single packet from the actor in return, the **reply**. These terms indicate a simple pattern of communication: the actor processes packets in the order they are received, and the client can trust that the *i*'th reply corresponds to the *i*'th request. + +An [error reply packet](#error-packets) from a request/reply actor constitutes a reply. + +Note that it is correct for a client to send several requests to a request/reply actor without waiting for a reply to each request before sending the next; requests can be pipelined. However, as the pending requests consume memory, the client should ensure that only a bounded number of requests are outstanding at any one time. + +#### The Request/Reply/Notify Pattern + +Some actors follow the request/reply pattern, but may also send the client ***notification*** packets, not in reply to any particular request. For example, if the client sends the root actor a `["listTabs"](#listing-browser-tabs)` request, then the root actor sends a reply. However, since the client has now expressed an interest in the list of open tabs, the root actor may subsequently send the client a `"tabListChanged"` notification packet, indicating that the client should re-fetch the list of tabs if it is interested in the latest state. + +There should be a small upper bound on the number of notification packets any actor may send between packets received from the client, to ensure that the actor does not flood the client. In the example above, the root actor sends at most one `"tabListChanged"` notification after each `"listTabs"` request. + +#### Error Packets + +Any actor can reply to a packet it is unable to process with an **error reply** of the form: + +``` +{ "from":actor, "error":name, "message":message } +``` + +where *name* is a JSON string naming what went wrong, and *message* is an English error message. Error *names* are specified by the protocol; the client can use the name to identify which error condition arose. The *message* may vary from implementation to implementation, and should only be displayed to the user as a last resort, as the server lacks enough information about the user interface context to provide appropriate messages. + +If an actor receives a packet whose type it does not recognize, it sends an error reply of the form: + +``` +{ "from":actor, "error":"unrecognizedPacketType", "message":message } +``` + +where *message* provides details to help debugger developers understand what went wrong: what kind of actor actor is; the packet received; and so on. + +If an actor receives a packet which is missing needed parameters (say, an `"autocomplete"` packet with no `"text"` parameter), it sends an error reply of the form: + +``` +{ "from":actor, "error":"missingParameter", "message":message } +``` + +where *message* provides details to help debugger developers fix the problem. + +If an actor receives a packet with a parameter whose value is inappropriate for the operation, it sends an error reply of the form: + +``` +{ "from":actor, "error":"badParameterType", "message":message } +``` + +where *message* provides details to help debugger developers fix the problem. (Some packets' descriptions specify more specific errors for particular circumstances.) + +### Grips + +A grip is a JSON value that refers to a specific JavaScript value in the debuggee. Grips appear anywhere an arbitrary value from the debuggee needs to be conveyed to the client: stack frames, object property lists, lexical environments, `paused` packets, and so on. + +For mutable values like objects and arrays, grips do not merely convey the value's current state to the client. They also act as references to the original value, by including an actor to which the client can send messages to modify the value in the debuggee. + +A grip has one of the following forms: + +``` +value +``` + +where value is a string, a number, or a boolean value. For these types of values, the grip is simply the JSON form of the value. + +``` +{ "type":"null" } +``` + +This represents the JavaScript `null` value. (The protocol does not represent JavaScript `null` simply by the JSON `null`, for the convenience of clients implemented in JavaScript: this representation allows such clients to use `typeof(grip) == "object"` to decide whether the grip is simple or not.) + +``` +{ "type":"undefined" } +``` + +This represents the JavaScript `undefined` value. (`undefined` has no direct representation in JSON.) + +``` +{ "type":"Infinity" } +``` + +This represents the JavaScript `Infinity` value. (`Infinity` has no direct representation in JSON.) + +``` +{ "type":"-Infinity" } +``` + +This represents the JavaScript `-Infinity` value. (`-Infinity` has no direct representation in JSON.) + +``` +{ "type":"NaN" } +``` + +This represents the JavaScript `NaN` value. (`NaN` has no direct representation in JSON.) + +``` +{ "type":"-0" } +``` + +This represents the JavaScript `-0` value. (`-0` stringifies to JSON as 0.) + +``` +{ "type":"object", "class":className, "actor":actor } +``` + +This represents a JavaScript object whose class is `className`. (Arrays and functions are treated as objects for the sake of forming grips.) Actor can be consulted for the object's contents, as explained below. + +If the class is "Function", the grip may have additional properties: + +``` +{ "type":"object", "class":"Function", "actor":actor, + "name":name, "displayName":displayName, + "userDisplayName":userDisplayName, + "url":url, "line":line, "column":column } +``` + +These additional properties are: + +***Name*** + +The function's name (as given in the source code, following the `function` keyword), as a string. If the function is anonymous, the `name` property is omitted. + +***displayName*** + +A name the system has inferred for the function (say, `"Foo.method"`). If the function has a given name (appearing in the grip as the `"name"` property), or if the system was unable to infer a suitable name for it, the `displayName` property is omitted. + +***userDisplayName*** + +If the function object has a `"displayName"` value property whose value is a string, this is that property's value. (Many JavaScript development tools consult such properties, to give developers a way to provide their own meaningful names for functions.) + +***url*** + +The URL of the function's source location (see [Source Locations](#source-locations)); + +***line*** + +The line number of the function's source location (see [Source Locations](#source-locations)); + +***column*** + +The column number of the function's source location (see [Source Locations](#source-locations)); + +``` +{ "type":"longString", "initial":initial, "length":length, "actor":actor } +``` + +This represents a very long string, where "very long" is defined at the server's discretion. `Initial` is some initial portion of the string, `length` is the string's full length, and actor can be consulted for the rest of the string, as explained below. + +For example, the following table shows some JavaScript expressions and the grips that would represent them in the protocol: + +| JavaScript Expression | Grip | +|:--------------------------------------------------------:|:---------------------------------------------------------------------------------------------:| +| 42 | 42 | +| true | true | +| "nasu" | "nasu" | +| (void 0) | `{ "type":"undefined" }` | +| ({x:1}) | `{ "type":"object", "class":"Object", "actor":"24" }` | +| "Arms and the man I sing, who, *[much, much more text]*" | `{ "type":"longString", "initial":"Arms and the man I sing", "length":606647, "actor":"25" }` | + +Garbage collection will never free objects visible to the client via the protocol. Thus, actors representing JavaScript objects are effectively garbage collection roots. + +#### Objects + +While a thread is paused, the client can send requests to the actors appearing in object grips to examine the objects they represent in more detail. + +##### Property Descriptors + +Protocol requests that describe objects' properties to the client often use **descriptors**, JSON values modeled after ECMAScript 5's property descriptors, to describe individual properties. + +A descriptor has the form: + +``` +{ "enumerable":<enumerable>, "configurable":<configurable>, ... } +``` + +where *enumerable* and *configurable* are boolean values indicating whether the property is enumerable and configurable, and additional properties are present depending on what sort of property it is. + +A descriptor for a data property has the form: + +``` +{ "enumerable":<enumerable>, "configurable":<configurable>, + "value":<value>, "writeable":<writeable> } +``` + +where *value* is a grip on the property's value, and *writeable* is a boolean value indicating whether the property is writeable. + +A descriptor for an accessor property has the form: + +``` +{ "enumerable":<enumerable>, "configurable":<configurable>, + "get":<getter>, "set":<setter> } +``` + +where *getter* and *setter* are grips on the property's getter and setter functions. These may be `{ "type":"undefined" }` if the property lacks the given accessor function. + +A **safe getter value descriptor** provides a value that an inherited accessor returned when applied to an instance. (See [Finding An Object's Prototype And Properties](#finding-an-objects-prototype-and-properties) for an explanation of why and when such descriptors are used.) Such a descriptor has the form: + +``` +{ "getterValue": <value>, "getterPrototypeLevel": <level>, + "enumerable":<enumerable>, "writable":<writable> } +``` + +where *value* is a grip on the value the getter returned, *level* is the number of steps up the object's prototype chain one must take to find the object on which the getter appears as an own property. If the getter appears directly on the object, *level* is zero. The *writable* property is true if the inherited accessor has a setter, and false otherwise. + +For example, if the JavaScript program being debugged evaluates the expression: + +``` +({x:10, y:"kaiju", get a() { return 42; }}) +``` + +then a grip on this value would have the form: + +``` +{ "type":"object", "class":"Object", "actor":<actor> } +``` + +and sending a ["prototypeAndProperties"](#finding-an-objects-prototype-and-properties) request to *actor* would produce the following reply: + +``` +{ "from":<actor>, "prototype":{ "type":"object", "class":"Object", "actor":<objprotoActor> }, + "ownProperties":{ "x":{ "enumerable":true, "configurable":true, "writeable":true, "value":10 }, + "y":{ "enumerable":true, "configurable":true, "writeable":true, "value":"kaiju" }, + "a":{ "enumerable":true, "configurable":true, + "get":{ "type":"object", "class":"Function", "actor":<getterActor> }, + "set":{ "type":"undefined" } + } + } +} +``` + + +Sending a ["prototypeAndProperties"](#finding-an-objects-prototype-and-properties) request to an object actor referring to a DOM mouse event might produce the following reply: + +``` +{ "from":<mouseEventActor>, "prototype":{ "type":"object", "class":"MouseEvent", "actor":<mouseEventProtoActor> }, + "ownProperties":{ } + "safeGetterValues":{ "screenX": { "getterValue": 1000, "getterPrototypeLevel": 1, + "enumerable": true, "writable": false }, + "screenY": { "getterValue": 1000, "getterPrototypeLevel": 1, + "enumerable": true, "writable": false }, + "clientX": { "getterValue": 800, "getterPrototypeLevel": 1, + "enumerable": true, "writable": false }, + "clientY": { "getterValue": 800, "getterPrototypeLevel": 1, + "enumerable": true, "writable": false }, + ... + } +} +``` + +##### Finding An Object's Prototype And Properties + +To examine an object's prototype and properties, a client can send the object's grip's actor a request of the form: + +``` +{ "to":<gripActor>, "type":"prototypeAndProperties" } +``` + +to which the grip actor replies: + +``` +{ "from":<gripActor>, "prototype":<prototype>, "ownProperties":<ownProperties> } +``` + +where *prototype* is a grip on the object's prototype (possibly `{ "type":"null" }`), and *ownProperties* has the form: + +``` +{ name:<descriptor>, ... } +``` + +with a *name*:<descriptor> pair for each of the object's own properties. + +The web makes extensive use of inherited accessor properties; for example, the `clientX` and `clientY`> properties of a mouse click event are actually accessor properties which the event object inherits from its prototype chain. It can be very valuable to display such properties' values directly on the object (taking care to distinguish them from true "own" properties), if the server can determine that the getters can be called without side effects. + +To this end, when possible, the server may provide safe getter value descriptors for an object, as described in [Property Descriptors](#property-descriptors) above, reporting the values that getter functions found on the object's prototype chain return when applied to that object. If the server chooses to provide any, the reply includes a `"safeGetterValues"` property of the form: + +``` +{ name:<descriptor>, ... } +``` + +with a *name*:<descriptor> pair for each safe getter the object inherits from its prototype chain, or that appears directly on the object. Each *descriptor* here is a safe getter value descriptor. + +*TODO: What about objects with many properties?* + +##### Finding an Object's Prototype + + +To find an object's prototype, a client can send the object's grip's actor a request of the form: + +``` +{ "to":<gripActor>, "type":"prototype" } +``` + +to which the grip actor replies: + +``` +{ "from":<gripActor>, "prototype":<prototype> } +``` + +where *prototype* is a grip on the object's prototype (possibly `{ "type":"null" }`). + + +##### Listing an Object's Own Properties' Names + +To list an object's own properties' names, a client can send the object's grip's actor a request of the form: + +``` +{ "to":<gripActor>, "type":"ownPropertyNames" } +``` + +to which the grip actor replies: + +``` +{ "from":<gripActor>, "ownPropertyNames":[ <name>, ... ] } +``` + +where each *name* is a string naming an own property of the object. + +##### Finding Descriptors For Single Properties + +To obtain a descriptor for a particular property of an object, a client can send the object's grip's actor a request of the form: + +``` +{ "to":<gripActor>, "type":"property", "name":<name> } +``` + +to which the grip actor replies: + +``` +{ "from":<gripActor>, "descriptor":<descriptor> } +``` + +where *descriptor* is a descriptor for the own property of the object named *name*, or `null` if the object has no such own property. + +A property descriptor has the form: + +``` +{ "configurable":<configurable>, "enumerable":<enumerable>, ... } +``` + +where *configurable* and *enumerable* are boolean values. *Configurable* is true if the property can be deleted or have its attributes changed. *Enumerable* is true if the property will be enumerated by a `for-in` enumeration. + +Descriptors for value properties have the form: + +``` +{ "configurable":<configurable>, "enumerable":<enumerable>, + "writable":<writable>, "value":<value> } +``` + +where *writable* is `true` if the property's value can be written to; *value* is a grip on the property's value; and *configurable* and *enumerable* are as described above. + +Descriptors for accessor properties have the form: + +``` +{ "configurable":<configurable>, "enumerable":<enumerable>, + "get":<get>, "set":<set> } +``` + +where *get* and *set* are grips on the property's getter and setter functions; either or both are omitted if the property lacks the given accessor function. *Configurable* and *enumerable* are as described above. + +*TODO: assign to value property* + +*TODO: special stuff for arrays* + +*TODO: special stuff for functions* + +*TODO: find function's source position* + +*TODO: get function's named arguments, in order* + +*TODO: descriptors for Harmony proxies* + +##### Functions + +If an object's class as given in the grip is `"Function"`, then the grip's actor responds to the messages given here. + +``` +{ "to":<functionGripActor>, "type":"parameterNames" } +``` + +This requests the names of the parameters of the function represented by *functionGripActor*. The reply has the form: + +``` +{ "from":<functionGripActor>, "parameterNames":[ <parameter>, ... ] } +``` + +where each *parameter* is the name of a formal parameter to the function as a string. If the function takes destructuring arguments, then *parameter* is a structure of JSON array and object forms matching the form of the destructuring arguments. + +``` +{ "to":<functionGripActor>, "type":"scope" } +``` + +Return the lexical environment over which the function has closed. The reply has the form: + +``` +{ "from":<functionGripActor>, "scope":<environment> } +``` + +where *environment* is a [lexical environment](#lexical-environments). Note that the server only returns environments of functions in a context being debugged; if the function's global scope is not the browsing context to which we are attached, the function grip actor sends an error reply of the form: + +``` +{ "from":<functionGripActor>, "error":"notDebuggee", "message":<message> } +``` + +where *message* is text explaining the problem. + +``` +{ "to":<functionGripActor>, "type":"decompile", "pretty":<pretty> } +``` + +Return JavaScript source code for a function equivalent to the one represented by *functionGripActor*. If the optional `pretty` parameter is present and *pretty* is `true`, then produce indented source code with line breaks. The reply has the form: + +``` +{ "from":<functionGripActor>, "decompiledCode":<code> } +``` + +where *code* is a string. + +If *functionGripActor*'s referent is not a function, or is a function proxy, the actor responds to these requests with an error reply of the form: + +``` +{ "from":<functionGripActor>, "error":"objectNotFunction", message:<message> } +``` + +where *message* is a string containing any additional information that would be helpful to debugger developers. + +#### Long Strings + +The client can find the full contents of a long string by sending a request to the long string grip actor of the form: + +``` +{ "to":<gripActor>, "type":"substring", "start":<start>, "end":<end> } +``` + +where *start* and *end* are integers. This requests the substring starting at the *start*'th character, and ending before the *end*'th character. The actor replies as follows: + +``` +{ "from":<gripActor>, "substring":<string> } +``` + +where *string* is the requested portion of the string the actor represents. Values for *start* less than zero are treated as zero; values greater than the length of the string are treated as the length of the string. Values for *end* are treated similarly. If *end* is less than *start*, the two values are swapped. (This is meant to be the same behavior as JavaScript's `String.prototype.substring`.) + +As with any other actor, the client may only send messages to a long string grip actor while it is alive: for [pause-lifetime grips](#grip-lifetimes), until the debuggee is resumed; or for [thread-lifetime grips](#grip-lifetimes), until the thread is detached from or exits. However, unlike object grip actors, the client may communicate with a long string grip actor at any time the actor is alive, regardless of whether the debuggee is paused. (Since strings are immutable values in JavaScript, the responses from a long string grip actor cannot depend on the actions of the debuggee.) + +#### Grip Lifetimes + +Most grips are **pause-lifetime** grips: they last only while the JavaScript thread is paused, and become invalid as soon as the debugger allows the thread to resume execution. (The actors in pause-lifetime grips are children of an actor that is closed when the thread resumes, or is detached from.) This arrangement allows the protocol to use grips freely in responses without requiring the client to remember and close them all. + +However, in some cases the client may wish to retain a reference to an object or long string while the debuggee runs. For example, a panel displaying objects selected by the user must update its view of the objects each time the debuggee pauses. To carry this out, the client can promote a pause-lifetime grip to a **thread-lifetime** grip, which lasts until the thread is detached from or exits. Actors in thread-lifetime grips are children of the thread actor. When the client no longer needs a thread-lifetime grip, it can explicitly release it. + +Both pause-lifetime and thread-lifetime grips are garbage collection roots. + +To promote a pause-lifetime grip to a thread-lifetime grip, the client sends a packet of the form: + +``` +{ "to":<gripActor>, "type":"threadGrip" } +``` + +where *gripActor* is the actor from the existing pause-lifetime grip. The grip actor will reply: + +``` +{ "from":<gripActor>, "threadGrip":<threadGrip> } +``` + +where *threadGrip* is a new grip on the same object, but whose actor is parented by the thread actor, not the pause actor. + +The client can release a thread-lifetime grip by sending the grip actor a request of the form: + +``` +{ "to":<gripActor>, "type":"release" } +``` + +The grip actor will reply, simply: + +``` +{ "from":<gripActor> } +``` + +This closes the grip actor. The `"release"` packet may only be sent to thread-lifetime grip actors; if a pause-lifetime grip actor receives a `"release"` packet, it sends an error reply of the form: + +``` +{ "from":<gripActor>, "error":"notReleasable", "message":<message> } +``` + +where each *gripActor* is the name of a child of *thread* that should be freed. The thread actor will reply, simply: + +``` +{ "from":<thread> } +``` + +Regardless of the lifetime of a grip, the client may only send messages to object grip actors while the thread to which they belong is paused; the client's interaction with mutable values cannot take place concurrently with the thread. + +### Completion Values + +Some packets describe the way a stack frame's execution completed using a **completion value**, which takes one of the following forms: + +``` +{ "return":<grip> } +``` + +This indicates that the frame completed normally, returning the value given by *grip*. + +``` +{ "throw":<grip> } +``` + +This indicates that the frame threw an exception; *grip* is the exception value thrown. + +``` +{ "terminated":true } +``` + +This indicates that the frame's execution was terminated, as by a "slow script" dialog box or running out of memory. + +### Source Locations + +Many packets refer to particular locations in source code: breakpoint requests specify where the breakpoint should be set; stack frames show the current point of execution; and so on. + +Descriptions of source code locations (written as *location* in packet descriptions) can take one of the following forms: + +``` +{ "url":<url>, "line":<line>, "column":<column> } +``` + +This refers to line *line*, column *column* of the source code loaded from *url*. Line and column numbers start with 1. If *column* or *line* are omitted, they default to 1. + +``` +{ "eval":<location>, "id":<id>, "line":<line>, "column":<column> } +``` + +This refers to line *line*, column *column* of the source code passed to the call to eval at *location*. To distinguish the different texts passed to eval, each is assigned a unique integer, *id*. + +``` +{ "function":<location>, "id":<id>, "line":<line>, "column":<column> } +``` + +This refers to line *line*, column *column* of the source code passed to the call to the `Function` constructor at *location*. To distinguish the different texts passed to the `Function` constructor, each is assigned a unique integer, *id*. + +As indicated, locations can be nested. A location like this one: + +``` +{ "eval":{ "eval":{ "url":"file:///home/example/sample.js", "line":20 } + "id":300, "line":30 } + "id":400, "line":40 } +``` + +refers to line 40 of the code passed to the call to eval occurring on line 30 of the code passed to the call to eval on line 20 of `file:///home/example/sample.js`. + +## The Root Actor + +When the connection to the server is opened, the root actor opens the conversation with the following packet: + +``` +{ "from":"root", "applicationType":<appType>, "traits":<traits>, ...} +``` + +The root actor's name is always `"root"`. *appType* is a string indicating what sort of program the server represents. There may be more properties present, depending on *appType*. + +*traits* is an object describing protocol variants this server supports that are not convenient for the client to detect otherwise. The property names present indicate what traits the server has; the properties' values depend on their names. If *traits* would have no properties, the `"traits"` property of the packet may be omitted altogether. This version of the protocol defines no traits, so if the `"traits"` property is present at all, its value must be an object with no properties, `{}`. + +For web browsers, the introductory packet should have the following form: + +``` +{ "from":"root", "applicationType":"browser", "traits":<traits> } +``` + +### Listing Browser Tabs + +To get a list of the tabs currently present in a browser, a client sends the root actor a request of the form: + +``` +{ "to":"root", "type":"listTabs" } +``` + +The root actor replies: + +``` +{ "from":"root", "tabs":[<tab>, ...], "selected":<selected> } +``` + +where each *tab* describes a single open tab, and *selected* is the index in the array of tabs of the currently selected tab. This form may have other properties describing other global actors; for one example, see [Chrome Debugging](#chrome-debugging). + +Each *tab* has the form: + +``` +{ "actor":<targetActor>, "title":<title>, "url":<URL> } +``` + +where *targetActor* is the name of an actor representing the tab, and *title* and *URL* are the title and URL of the web page currently visible in that tab. This form may have other properties describing other tab-specific actors. + +To attach to a *targetActor*, a client sends a message of the form: + +``` +{ "to":<targetActor>, "type":"attach" } +``` + +The target actor replies: + +``` +{ "from":<targetActor>, "threadActor":<tabThreadActor> } +``` + +where *tabThreadActor* is the name of a thread-like actor representing the tab's current content. If the user navigates the tab, *tabThreadActor* switches to the new content; we do not create a separate thread-like actor each page the tab visits. + +If the user closes the tab before the client attaches to it, *targetActor* replies: + +``` +{ "from":<targetActor>, "error":"exited" } +``` + +When the client is no longer interested in interacting with the tab, the client can request: + +``` +{ "to":<targetActor>, "type":"detach" } +``` + +The *targetActor* replies: + +``` +{ "from":<targetActor>, "type":"detached" } +``` + +If the client was not already attached to *targetActor*, *targetActor* sends an error reply of the form: + +``` +{ "from":<targetActor>, "error":"wrongState" } +``` + +While the client is attached, *targetActor* sends notifications to the client whenever the user navigates the tab to a new page. When navigation begins, *targetActor* sends a packet of the form: + +``` +{ "from":<targetActor>, "type":"tabNavigated", "state":"start", + "url":<newURL> } +``` + +This indicates that the tab has begun navigating to *newURL*; JavaScript execution in the tab's prior page is suspended. When navigation is complete, *targetActor* sends a packet of the form: + +``` +{ "from":<targetActor>, "type":"tabNavigated", "state":"stop", + "url":<newURL>, "title":<newTitle> } +``` + +where *newURL* and *newTitle* are the URL and title of the page the tab is now showing. The *tabThreadActor* given in the response to the original `"attach"` packet is now debugging the new page's code. + +If the user closes a tab to which the client is attached, its *targetActor* sends a notification packet of the form: + +``` +{ "from":<targetActor>, "type":"tabDetached" } +``` + +The client is now detached from the tab. + +### Chrome Debugging + +If the server supports debugging chrome code, the root actor's reply to a `"listTabs"` request includes a property named `"chromeDebugger"`, whose value is the name of a thread-like actor to which the client can attach to debug chrome code. + +## Interacting with Thread-Like Actors + +Actors representing independent threads of JavaScript execution, like browsing contexts and web workers, are collectively known as "threads". Interactions with actors representing threads follow a more complicated communication pattern. + +A thread is always in one of the following states: + +* **Detached**: the thread is running freely, and not presently interacting with the debugger. Detached threads run, encounter errors, and exit without exchanging any sort of messages with the debugger. A debugger can attach to a thread, putting it in the **Paused** state. Or, a detached thread may exit on its own, entering the **Exited** state. + +* **Running**: the thread is running under the debugger's observation, executing JavaScript code or possibly blocked waiting for input. It will report exceptions, breakpoint hits, watchpoint hits, and other interesting events to the client, and enter the **Paused** state. The debugger can also interrupt a running thread; this elicits a response and puts the thread in the **Paused** state. A running thread may also exit, entering the **Exited** state. + +* **Paused**: the thread has reported a pause to the client and is awaiting further instructions. In this state, a thread can accept requests and send replies. If the client asks the thread to continue or step, it returns to the **Running** state. If the client detaches from the thread, it returns to the **Detached** state. + +* **Exited**: the thread has ceased execution, and will disappear. The resources of the underlying thread may have been freed; this state merely indicates that the actor's name is not yet available for reuse. When the actor receives a "release" packet, the name may be reused. + + + +These interactions are meant to have certain properties: + +* At no point may either client or server send an unbounded number of packets without receiving a packet from its counterpart. This avoids deadlock without requiring either side to buffer an arbitrary number of packets per actor. +* In states where a transition can be initiated by either the debugger or the thread, it is always clear to the debugger which state the thread actually entered, and for what reason.<br>For example, if the debugger interrupts a running thread, it cannot be sure whether the thread stopped because of the interruption, paused of its own accord (to report a watchpoint hit, say), or exited. However, the next packet the debugger receives will either be "paused", or "exited", resolving the ambiguity.<br>Similarly, when the debugger attaches to a thread, it cannot be sure whether it has succeeded in attaching to the thread, or whether the thread exited before the "attach" packet arrived. However, in either case the debugger can expect a disambiguating response: if the attach succeeded, it receives an "attached" packet; and in the second case, it receives an "exit" packet.<br>To support this property, the thread ignores certain debugger packets in some states (the "interrupt" packet in the **Paused** and **Exited** states, for example). These cases all handle situations where the ignored packet was preempted by some thread action. + +Note that the rules here apply to the client's interactions with each thread actor separately. A client may send an "interrupt" to one thread actor while awaiting a reply to a request sent to a different thread actor. + +*TODO: What about user selecting nodes in displayed content? Should those be eventy things the client can receive in the "paused" state? What does that mean for the "request"/"reply" pattern?* + +### Attaching To a Thread + +To attach to a thread, the client sends a packet of the form: + +``` +{ "to":<thread>, "type":"attach" } +``` + +Here, *thread* is the actor representing the thread, perhaps a browsing context from a "listContexts" reply. This packet causes the thread to pause its execution, if it does not exit of its own accord first. The thread responds in one of two ways: + +``` +{ "from":<thread>, "type":"paused", "why":{ "type":"attached" }, ... } +``` + +The thread is now in the **Paused** state, because the client has attached to it. The actor name *thread* remains valid until the client detaches from the thread or acknowledges a thread exit. This is an ordinary `"paused"` packet, whose form and additional properties are as described in [Thread Pauses](#thread-pauses), below. + +``` +{ "from":<thread>, "type":"exited" } +``` + +This indicates that the thread exited on its own before receiving the "attach" packet. The thread is now in the **Exited** state. The client should follow by sending a "release" packet; see [Exiting Threads](#exiting-threads), below. + +If the client sends an `"attach"` packet to a thread that is not in the **Detached** or **Exited** state, the actor sends an error reply of the form: + +``` +{ "from":<thread>, "error":"wrongState", "message":<message> } +``` + +where *message* details which state the thread was in instead (to make debugging debuggers easier). In this case, the thread's state is unaffected. + +### Detaching From a Thread + +To detach from a thread, the client sends a packet of the form: + +``` +{ "to":<thread>, "type":"detach" } +``` + +The thread responds in one of three ways: + +``` +{ "from":<thread>, "type":"detached" } +``` + +This indicates that the client has detached from the thread. The thread is now in the **Detached** state: it can run freely, and no longer reports events to the client. Communications with *thread* are closed, and the actor name is available for reuse. If the thread had been in the **Paused** state, the pause actor is closed (because the pause actor is a child of *thread*). + +``` +{ "from":<thread>, "type":"paused", ... } +{ "from":<thread>, "type":"detached" } +``` + +This series of packets indicates that the thread paused of its own accord (for the reason given by the additional properties of the "paused" packet), and only then received the "detach" packet. As above, this indicates that the thread is in the **Detached** state, the just-created pause actor is closed, and the actor name is available for reuse. + +``` +{ "from":<thread>, "type":"exited" } +``` + +This indicates that the thread exited on its own before receiving the "detach" packet. The client should follow by sending a "release" packet; see [Exiting Threads](#exiting-threads), below. + +Detaching from a thread causes all breakpoints, watchpoints, and other debugging-related state to be forgotten. + +If the client sends a `"detach"` packet to a thread that is not in the **Running**, **Paused**, or **Exited** state, the actor sends an error reply of the form: + +``` +{ "from":<thread>, "error":"wrongState", "message":<message> } +``` + +where *message* details which state the thread was in instead (to make debugging debuggers easier). In this case, the thread's state is unaffected. + +### Running Threads + +Once the client has attached to a thread, it is in the **Running** state. In this state, four things can happen: + +* The thread can hit a breakpoint or watchpoint, or encounter some other condition of interest to the client. +* The thread can exit. +* The client can detach from the thread. +* The client can interrupt the running thread. + +Note that a client action can occur simultaneously with a thread action. The protocol is designed to avoid ambiguities when both client and thread act simultaneously. + +### Thread Pauses + +If the thread pauses to report an interesting event to the client, it sends a packet of the form: + +``` +{ "from":<thread>, "type":"paused", "actor":<pauseActor>, "why":<reason>, + "currentFrame":<frame>, "poppedFrames":[<poppedFrame>...] } +``` + +This indicates that the thread has entered the **Paused** state, and explains where and why. + +*PauseActor* is a "pause actor", representing this specific pause of the thread; it lives until the thread next leaves the **Paused** state. The pause actor parents actors referring to values and other entities uncovered during this pause; when the thread resumes, those actors are automatically closed. This relieves the client from the responsibility to explicitly close every actor mentioned during the pause. + +Since actors in value grips are parented by the pause actor, this means that those grips become invalid when the thread resumes, or is detached from; it is not possible to take a grip from one pause and use it in the next. To create a grip that remains valid between pauses, see [Grip Lifetimes](#grip-lifetimes). + +The *currentFrame* value describes the top frame on the JavaScript stack; see [Listing Stack Frames](#listing-stack-frames), below. + +The `"poppedFrames"` property is an array of frame actor names, listing the actors for all frames that were live as of the last pause, but have since been popped. If no frames have been popped, or if this is the first pause for this thread, then this property's value is the empty array. + +The *reason* value describes why the thread paused. It has one of the following forms: + +``` +{ "type":"attached" } +``` + +The thread paused because the client attached to it. + +``` +{ "type":"interrupted" } +``` + +The thread stopped because it received an "interrupt" packet from the client. + +``` +{ "type":"resumeLimit" } +``` + +The client resumed the thread with a `"resume"` packet that included a `resumeLimit` property, and the thread paused because the given *limit* was met. Execution remains in the frame the thread was resumed in, and that frame is not about to be popped. + +``` +{ "type":"resumeLimit", "frameFinished":<completion> } +``` + +The client resumed the thread with a `"resume"` packet that included a `resumeLimit` property, and the thread paused because the frame is about to be popped. *Completion* is a [completion value](#completion-values) describing how the frame's execution ended. The frame being popped is still the top frame on the stack, but subsequent `"resume"` operations will run in the calling frame. + +``` +{ "type":"debuggerStatement" } +``` + +The thread stopped because it executed a JavaScript "debugger" statement. + +``` +{ "type":"breakpoint", "actors":[<breakpointActor>...] } +``` + +The thread stopped at the breakpoints represented by the given actors. + +``` +{ "type":"watchpoint", "actors":[<watchpointActor>...] } +``` + +The thread stopped at the watchpoints represented by the given actors. + +*TODO: This should provide more details about the watchpoint in the packet, instead of incurring another round-trip before we can display anything helpful.* + +``` +{ "type":"clientEvaluated", "frameFinished":<completion> } +``` + +The expression given in the client's prior `clientEvaluate` command has completed execution; *completion* is a [completion value](#completion-values) describing how it completed. The frame created for the `clientEvaluate` resumption has been popped from the stack. See [Evaluating Source-Language Expressions](#evaluating-source-language-expressions) for details. + +### Resuming a Thread + +If a thread is in the **Paused** state, the client can resume it by sending a packet of the following form: + +``` +{ "to":<thread>, "type":"resume" } +``` + +This puts the thread in the **Running** state. The thread will pause again for breakpoint hits, watchpoint hits, throw watches, frame pop watches, and other standing pause requests. + +To step a thread's execution, the client can send a packet of the form: + +``` +{ "to":<thread>, "type":"resume", "resumeLimit":<limit> } +``` + +*Limit* must have one of the following forms: + +``` +{ "type":"next" } +``` + +The thread should pause: + +* just before the current frame is popped, whether by throwing an exception or returning a value; or +* when control in the current frame reaches a different statement than the one it is currently at. + +Note that execution in frames younger than the current frame never meets these conditions, so a `"next"` limit steps over calls, generator-iterator invocations, and so on. + +``` +{ "type":"step" } +``` + +The thread should pause: + +* just before the current frame is popped, whether by throwing an exception or returning a value; or +* just after a new frame is pushed; or +* when control in the current frame reaches a different statement than the one it is currently at. + +This is the same as `"next"`, except that it steps into calls. + +To resume the thread but have it stop when the current frame is about to be popped, the client can send a packet of the form: + +``` +{ "to":<thread>, "type":"resume", "resumeLimit":{ "type":"finish" } } +``` + +Here, the thread should pause just before the current frame is popped, whether by throwing an exception, returning a value, or being terminated. + +When a thread pauses because a limit was reached, the "paused" packet's *reason* will have a type of `"resumeLimit"`. + +A resume limit applies only to the current resumption; once the thread pauses, whether because the limit was reached or some other event occurred—a breakpoint hit, for example—the resume limit is no longer in effect. + +If no `"resumeLimit"` property appears in the `"resume"` packet, then the thread should run until some standing pause condition is met (a breakpoint is hit; a watchpoint triggers; or the like). + +To force the current frame to end execution immediately, the client can send a packet of the form: + +``` +{ "to":<thread>, "type":"resume", "forceCompletion":<completion> } +``` + +where *completion* is a [completion value](#completion-values) indicating whether the frame should return a value, throw an exception, or be terminated. Execution resumes in the current frame's caller, in the manner appropriate for *completion*. + +To request that execution pause when an exception is thrown, the client may send a request of the form: + +``` +{ "to":<thread>, "type":"resume", "pauseOnExceptions": true } +``` + +If `pauseOnExceptions` has the value `false` or is omitted, execution will continue in the face of thrown exceptions. When a thread pauses because an exception was thrown, the "paused" packet's *reason* will have the following form: + +``` +{ "type":"exception", "exception":<exception> } +``` + +where *exception* is a grip on the exception object. + +To request that execution pause on a DOM event, the client may send a request of the form: + +If a `"forceCompletion"` property is present in a `"resume"` packet, along with `"resumeLimit"`, or `"pauseOnExceptions"`, the thread will respond with an error: + +``` +{ "from":<thread>, "error":"badParameterType", "message":<message> } +``` + +A `"resume"` packet closes the pause actor the client provided in the "paused" packet that began the pause. + +If the client sends a `"resume"` packet to a thread that is not in the **Paused** state, the actor sends an error reply of the form: + +``` +{ "from":<thread>, "error":"wrongState", "message":<message> } +``` + +where *message* details which state the thread was in instead (to make debugging debuggers easier). In this case, the thread's state is unaffected. + +### Interrupting a Thread + +If a thread is in the **Running** state, the client can cause it to pause where it is by sending a packet of the following form: + +``` +{ "to":<thread>, "type":"interrupt" } +``` + +The thread responds in one of two ways: + +``` +{ "from":<thread>, "type":"paused", "why":<reason>, ... } +``` + +This indicates that the thread stopped, and is now in the **Paused** state. If *reason* is `{ "type":"interrupted" }`, then the thread paused due to the client's *interrupt* packet. Otherwise, the thread paused of its own accord before receiving the *interrupt* packet, and will ignore the *interrupt* packet when it receives it. In either case, this is an ordinary `"paused"` packet, whose form and additional properties are as described in [Thread Pauses](#thread-pauses), above. + +``` +{ "from":<thread>, "type":"exited" } +``` + +This indicates that the thread exited before receiving the client's *interrupt* packet, and is now in the **Exited** state. See [Exiting Threads](#exiting-threads), below. + +If the client sends an `"interrupt"` packet to a thread that is not in the **Running**, **Paused**, or **Exited** state, the actor sends an error reply of the form: + +``` +{ "from":<thread>, "error":"wrongState", "message":<message> } +``` + +where *message* details which state the thread was in instead (to make debugging debuggers easier). In this case, the thread's state is unaffected. + +### Exiting Threads + +When a thread in the **Running** state exits, it sends a packet of the following form: + +``` +{ "from":<thread>, "type":"exited" } +``` + +At this point, the thread can no longer be manipulated by the client, and most of the thread's resources may be freed; however, the thread actor name must remain alive, to handle stray `interrupt` and `detach` packets. To allow the last trace of the thread to be freed, the client should send a packet of the following form: + +``` +{ "to":<thread>, "type":"release" } +``` + +This acknowledges the exit and allows the thread actor name, *thread*, to be reused for other actors. + +## Inspecting Paused Threads + +When a thread is in the **Paused** state, the debugger can make requests to inspect its stack, lexical environment, and values. + +Only those packets explicitly defined to do so can cause the thread to resume execution. JavaScript features like getters, setters, and proxies, which could normally lead inspection operations like enumerating properties and examining their values to run arbitrary JavaScript code, are disabled while the thread is paused. If a given protocol request is not defined to let the thread run, but carrying out the requested operation would normally cause it to do so—say, fetching the value of a getter property—the actor sends an error reply of the form: + +``` +{ "from":<actor>, "error":"threadWouldRun", "message":<message>, "cause":<cause> } +``` + +where *message* is text that could be displayed to users explaining why the operation could not be carried out. *Cause* is one of the following strings: + +| *cause* value | meaning | +|:-------------:|:------------------------------------------------------------------------:| +| "proxy" | Carrying out the operation would cause a proxy handler to run. | +| "getter" | Carrying out the operation would cause an object property getter to run. | +| "setter" | Carrying out the operation would cause an object property setter to run. | + +(Taken together, the `"threadWouldRun"` error name and the *cause* value should allow the debugger to present an appropriately localized error message.) + +### Loading Script Sources + +To get a snapshot of all sources currently loaded by the thread actor, the client can send the following packet: + +``` +{ to: <threadActorID>, type: "sources" } +``` + +The response packet has the form: + +``` +{ from: <threadActorID>, sources: [<sourceForm1>, <sourceForm2>, ..., <sourceFormN>] } +``` + +Where each *sourceForm* has the following form: + +``` +{ actor: <sourceActorID>, + url: <sourceURL>, + isBlackBoxed: <isBlackBoxed> } +``` + +* *sourceActorID* is the source actor's id +* *sourceURL* is the URL of the source represented by the source actor +* *isBlackBoxed* is a boolean specifying whether the source actor's 'black-boxed' flag is set. See [Black Boxing Sources](#black-boxing-sources). + +Each source actor exists throughout the thread's whole lifetime. + +To get the contents of a source, send the corresponding source actor the following packet: + +``` +{ to: <sourceActorID>, type: "source" } +``` + +And the source actor replies with a packet of the following form: + +``` +{ from: <sourceActorID>, source: <contentsOfSource> } +``` + +where *contentsOfSource* is a grip representing the string of source code: either a JSON string, or a long string grip. (See [Grips](#grips) for a description of long string grips.) + +#### Black-Boxing Sources + +When debugging a web application that uses large off-the-shelf JavaScript libraries, it may help the developer focus on their own code to treat such libraries as "black boxes", whose internal details are omitted or simplified in the user interface. For example, the user interface could display a sub-chain of stack frames within a black-boxed library as a single element; breakpoints set in a black-boxed library could be disabled; and so on. + +Each source actor has a 'black-boxed' flag, and understands requests to set and clear the flag. When a source actor is black-boxed, the debugger does not pause when it hits breakpoints or `debugger` statements inside that source. If pausing on exceptions is enabled and an exception is thrown inside a black-boxed source, the debugger does not pause until the stack has unwound to a frame in a source that is not black-boxed. + +Thread actors still list black-boxed source actors in `"sources"` replies; and include stack frames running black-boxed code in `"frames"` requests. However, each *sourceForm* includes an `"isBlackBoxed"` property, giving the client all the information it needs to implement the black-boxing behavior in the user interface. + +To set a source actor's 'black-boxed' flag: + +``` +{ "to": <sourceActor>, "type": "blackbox" } +``` + +The *sourceActor* responds with a blank response on success: + +``` +{ "from": <sourceActor> } +``` + +Or an error response on failure: + +``` +{ "from": <sourceActor>, "error": <reason> } +``` + +To clear a source actor's 'black-boxed' flag: + +``` +{ "to": <sourceActor>, "type": "unblackbox" } +``` + +And once again, the *sourceActor* responds with a blank response on success: + +``` +{ "from": <sourceActor> } +``` + +Or an error response on failure: + +``` +{ "from": <sourceActor>, "error": <reason> } +``` + +### Listing Stack Frames + +To inspect the thread's JavaScript stack, the client can send the following request: + +``` +{ "to":<thread>, "type":"frames", "start":<start>, "count":<count> } +``` + +The `start` and `count` properties are optional. If present, *start* gives the number of the youngest stack frame the reply should describe, where the youngest frame on the stack is frame number zero; if absent, *start* is taken to be zero. If present, *count* specifies the maximum number of frames the reply should describe; if absent, it is taken to be infinity. (Clients should probably avoid sending `frames` requests with no *count*, to avoid being flooded by frames from unbounded recursion.) + +The thread replies as follows: + +``` +{ "from":<thread>, "frames":[<frame> ...] } +``` + +where each *frame* has the form: + +``` +{ "actor": <actor>, + "depth": <depth>, + "type": <type>, + "this": <this>, + ... } +``` + +where: + +* *actor* is the name of an actor representing this frame; +* *depth* is the number of this frame, starting with zero for the youngest frame on the stack; +* *type* is a string indicating what sort of frame this is; and +* *this* is a grip on the value of `this` for this call. + +The frame may have other properties, depending on *type*. + +All actors mentioned in the frame or grips appearing in the frame (*actor*, *callee*, *environment*, and so on) are parented by the thread actor. + +#### Global Code Frames + +A frame for global code has the form: + +``` +{ "actor":<actor>, + "depth":<depth>, + "type":"global", + "this":<this>, + "where":<location>, + "source":<source>, + "environment":<environment> } +``` + +where: + +* *location* is the source location of the current point of execution in the global code (see [Source Locations](#source-locations)); +* *environment* is a value representing the lexical environment of the current point of execution (see [Lexical Environments](#lexical-environments)); +* *source* is a source form as described in [Loading Script Sources](#loading-script-sources) + +and other properties are as above. + +#### Function Call Frames + +A frame for an ordinary JavaScript function call has the form: + +``` +{ "actor":<actor>, "depth":<depth>, "type":"call", "this":<this>, + "where":<location>, "environment":<environment>, + "callee":<callee>, "arguments":<arguments> } +``` + +where: + +* *callee* is a grip on the function value being called; +* *arguments* is an array of grips on the actual values passed to the function; + +and other properties are as above. + +If the callee is a host function, or a function scoped to some global other than the one to which we are attached, the `"where"` and `"environment"` properties are absent. + +The argument list may be incomplete or inaccurate, for various reasons. If the program has assigned to its formal parameters, the original values passed may have been lost, and compiler optimizations may drop some argument values. + +#### Eval Frames + +A frame for a call to `eval` has the form: + +``` +{ "actor":<actor>, "depth":<depth>, "type":"eval", "this":<this>, + "where":<location>, "environment":<environment> } +``` + +where the properties are as defined above. + +#### Client Evaluation Frames + +When the client evaluates an expression with an `clientEvaluate` packet, the evaluation appears on the stack as a special kind of frame, of the form: + +``` +{ "actor":<actor>, "depth":<depth>, "type":"clientEvaluate", "this":<this>, + "where":<location>, "environment":<environment> } +``` + +where the properties are as defined above. In this case, *where* will be a location inside the expression provided by the debugger. + +### Popping Stack Frames + +The client can remove frames from the stack by sending a request of the form: + +``` +{ "to":<frameActor>, "type":"pop", "completionValue":<completion> } +``` + +where *frameActor* is the actor representing the stack frame to pop, and *completion* is a [completion value](#completion-values) describing how the frame should appear to have finished execution. All younger stack frames are also popped. The thread remains paused. The frame actor will reply: + +``` +{ "from":<frameActor>, "watches":[<watchActor> ...] } +``` + +where each *watchActor* is the name of a frame pop watch actor that has been triggered in the process of popping the given frame. If no frame pop watches are triggered, the `watches` property may be omitted. + +*TODO: specify the error to return if the frame cannot be popped --- can host (C++) function frames be popped?* + +### Evaluating Source-Language Expressions + +To evaluate a source-language expression in a thread, the client sends a specialized `"resume"` packet of the form: + +``` +{ "to":<thread>, "type":"clientEvaluate", "expression":<expr>, "frame":<frame> } +``` + +This resumes the thread just as an ordinary `"resume"` packet does, but, rather than continuing execution where the pause took place, has the thread begin evaluation of the source-language expression given by *expr*, a string. The evaluation takes place in a new [Client Evaluation Frame](#client-evaluation-frames), pushed on top of *thread*'s current stack, using the environment of *frame*. *Frame* must be a live actor for one of *thread*'s frames, and the given frame must be one from which we can retrieve a lexical environment; that is, it must not be the frame for a call to a non-debuggee function. When evaluation of *expr* completes, the client will report a `clientEvaluate` pause containing the expression's value. + +If evaluating *expr* completes abruptly, this outcome is still reported via an `clientEvaluated` pause, so it is not necessary for the client to take explicit steps to catch exceptions thrown by the expression. + +If *frame* is not the name of an actor for a frame currently on *thread*'s stack, the thread actor sends a reply of the form: + +``` +{ "from":<thread>, "error":"unknownFrame", "message":<message> } +``` + +where *message* provides any details that would be helpful to the debugger developers. In this case, the thread's state is unaffected. + +If *frame* is not a frame whose environment we can access, the thread actor sends an error reply of the form: + +``` +{ "from":<thread>, "error":"notDebuggee", "message":<message> } +``` + +where *message* provides further appropriate details. + +If the client sends a `"clientEvaluate"` packet to a thread that is not in the **Paused** state, the actor sends an error reply of the form: + +``` +{ "from":<thread>, "error":"wrongState", "message":<message> } +``` + +where *message* details which state the thread was in instead (to make debugging debuggers easier). In this case, the thread's state is unaffected. + +*TODO: evaluate with given grips bound to given identifiers* + +## Lexical Environments + +A lexical environment (written as *environment* in packet descriptions) records the identifier bindings visible at a particular point in the program. An environment has one of the following forms: + +``` +{ "type":"object", "actor":<actor>, "object":<object>, "parent":<parentEnvironment> } +``` + +This represents a scope chain element whose identifier bindings reflect the properties of *object* (a grip). This could be the global object (`window` in a browser), or a DOM element (for event handler content attributes, which have the input element, form, and document on their scope chain along with the `window`). + +*Actor* is the name of an actor representing this lexical environment. The requests it can answer are described below. + +*ParentEnvironment* is a lexical environment describing the next enclosing environment; the `parent` property is omitted on the outermost environment. + +``` +{ "type":"function", "actor":<actor>, "function":<function>, + "bindings":<bindings>, "parent":<parentEnvironment> } +``` + +This represents the variable environment created by a call to *function* (a grip). *Bindings* describes the bindings in scope, including the function's arguments, the `arguments` object, and local `var` and function bindings; its form is described in detail below. The other properties are as described above. + +``` +{ "type":"with", "actor":<actor>, "object":<object>, "parent":<parentEnvironment> } +``` + +This represents an environment introduced by a `with` statement whose operand is *object* (a grip). The other properties are as described above. + +``` +{ "type":"block", "actor":<actor>, "bindings":<bindings>, "parent":<parentEnvironment> } +``` + +This represents an environment introduced by a `let` block, `for-in` statement, `catch` block, or the like. The properties are as described above. + +A *bindings* value has the form: + +``` +{ "arguments":[ { name:<descriptor> }, ... ], + "variables":{ name:<descriptor>, ... } } +``` + +Each *name* is the name of a bound identifier, as a string. Each *descriptor* is a [property descriptor](#property-descriptors) for the variable, presenting the variable's value as the descriptor's `"value"` property, and the variable's mutability as the descriptor's `"writable"` property. The descriptor's `"configurable"` property reflects whether the environment supports deleting and adding variables. Each descriptor's `"enumerable"` property is `true`. + +The `"arguments"` list appears only in bindings for `"function"` environments. It lists the arguments in the order they appear in the function's definition. (The same name may appear several times in the list, as permitted by JavaScript; the name's last appearance is the one in scope in the function.) + +Note that language implementations may omit some environment records from a function's scope if it can determine that the function would not use them. This means that it may be impossible for a debugger to find all the variables that ought to be in scope. + +To fully enumerate the bindings introduced by any lexical environment, the client can send a request of the following form to the environment's actor: + +``` +{ "to":<envActor>, "type":"bindings" } +``` + +The actor will reply as follows: + +``` +{ "from":<envActor>, "bindings":<bindings> } +``` + +Note that this request elicits a `"threadWouldRun"` error reply when *envActor* refers to an object environment whose object is a proxy. + +To change the value of a variable bound in a particular lexical environment, the client can send a request to the environment's actor: + +``` +{ "to":<envActor>, "type":"assign", "name":<name>, "value":<value> } +``` + +This changes the value of the identifier whose name is *name* (a string) to that represented by *value* (a grip). The actor will reply as follows, simply: + +``` +{ "from":<envActor> } +``` + +If the named identifier is immutable, the actor will send an error reply of the form: + +``` +{ "from":<envActor>, "error":"immutableBinding", "message":<message> } +``` + +If *envActor* refers to an object environment whose object is a proxy, or whose property named *name* has a setter function, this request elicits a `"threadWouldRun"` error reply. + +### Lexical Environment Examples + +For example, if we have the following JavaScript code: + +``` +function f(x) { + function g(y) { + var z = "value of z"; + alert(x + y); + } +} +``` + +we set a breakpoint on the line containing the call to `alert`, and then evaluate the expression: + +``` +f("argument to f")("argument to g") +``` + +then we would hit that breakpoint, eliciting a packet like the following: + +``` +{ "from":<thread>, "type":"paused", "actor":<pauseActor>, + "why":{ "type":"breakpoint", "actors":[<breakpointActor>] }, + "frame":{ "actor":<frameActor>, "depth":1, + "type":"call", "where":{ "url":"sample.js", "line":3 }, + "environment":{ "type":"function", "actor":<gFrameActor>, + "function":{ "type":"object", "class":"Function", "actor":<gActor> }, + "functionName":"g", + "bindings":{ arguments: [ { "y": { "value":"argument to g", "configurable":"false", + "writable":true, "enumerable":true } } ] }, + "parent":{ "type":"function", "actor":<fFrameActor>, + "function":{ "type":"object", "class":"Function", "actor":<fActor> }, + "functionName":"f", + "bindings": { arguments: [ { "x": { "value":"argument to f", "configurable":"false", + "writable":true, "enumerable":true } } ], + variables: { "z": { "value":"value of z", "configurable":"false", + "writable":true, "enumerable":true } } }, + "parent":{ "type":"object", "actor":<globalCodeActor>, + "object":{ "type":"object", "class":"Global", + "actor":<globalObjectActor> } + } + } + }, + "callee":<gActor>, "calleeName":"g", + "this":{ "type":"object", "class":"Function", "actor":<gActor> }, + "arguments":["argument to g"] + } +} +``` + +You can see here the three nested environment forms, starting with the `environment` property of the top stack frame, reported in the pause: + +* The first environment form shows the environment record created by the call to `g`, with the string `"argument to g"` passed as the value of `y`. +* Because `g` is nested within `f`, each function object generated for `g` captures the environment of a call to the enclosing function `f`. Thus, the next thing on `g`'s scope chain is an environment form for the call to `f`, where `"argument to f"` was passed as the vale of `x`. +* Because `f` is a top-level function, the (only) function object for `f` closes over the global object. This is the "type":"object" environment shown as the parent of `f`'s environment record. +* Because the global object is at the end of the scope chain, its environment form has no `parent` property. + +## Breakpoints + +While a thread is paused, a client can set breakpoints in the thread's code by sending requests of the form: + +``` +{ "to":<thread>, "type":"setBreakpoint", "location":<location> } +``` + +where *location* is a [source location](#source-locations). If the thread is able to establish a breakpoint at the given location, it replies: + +``` +{ "from":<thread>, "actor":<actor>, "actualLocation":<actualLocation> } +``` + +where *actor* is an actor representing the breakpoint (a child of the thread actor), and *actualLocation* is the location at which the breakpoint was really set. If *location* and *actualLocation* are the same, then the `actualLocation` property can be omitted. + +If the thread cannot find the script referred to in *location*, it sends an error reply of the form: + +``` +{ "from":<thread>, "error":"noScript" } +``` + +If *location* refers to a line and column at which the given script has no program code, and no reasonable alternative location can be chosen (say, by skipping forward), then the thread sends an error reply of the form: + +``` +{ "from":<thread>, "error":"noCodeAtLineColumn" } +``` + +To delete a breakpoint, the client can send the breakpoint's actor a message of the form: + +``` +{ "to":<breakpointActor>, "type":"delete" } +``` + +to which the breakpoint actor will reply, simply: + +``` +{ "from":<breakpointActor> } +``` + +This closes communications with *breakpointActor*. + +## Event Listeners + +To request a list of all the event listeners and event handlers (see [DOM Event Handlers](https://developer.mozilla.org/docs/Web/Guide/DOM/Events/Event_handlers#Definitions) for definitions of the two terms) attached to the page, the client sends a request of the form: + +``` +{ "to":<thread>, "type":"eventListeners" } +``` + +The thread replies with a response of the form: + +``` +{ "from":<thread>, "listeners":[ <listener>, ... ] } +``` + +Such requests can be sent when the thread is either paused or running. A *listener* value has the form: + +``` +{ "node":{ "selector":<node-selector>, "object":<node> }, + "type":<type>, + "capturing":<capturing>, + "allowsUntrusted":<allowsUntrusted>, + "inSystemEventGroup":<inSystemEventGroup>, + "isEventHandler":<isEventHandler>, + "function":<function> } +``` + +The values for these properties are: + +***node-selector*** + +A unique CSS selector of the DOM element on which the event handler is attached, or `"window"` if the handler is attached on the window. + +***node*** + +A grip on the DOM element on which the event handler is attached. + +***type*** + +The type of the DOM event as specified in the DOM specification (see [nsIEventListenerInfo](https://developer.mozilla.org/docs/XPCOM_Interface_Reference/nsIEventListenerInfo#Attributes)). + +***capturing*** + +A boolean flag indicating whether the event listener is in capture mode (see [nsIEventListenerInfo](https://developer.mozilla.org/docs/XPCOM_Interface_Reference/nsIEventListenerInfo#Attributes)). + +***allowsUntrusted*** + +A boolean flag that indicates whether the listener allows untrusted events (see [nsIEventListenerInfo](https://developer.mozilla.org/docs/XPCOM_Interface_Reference/nsIEventListenerInfo#Attributes)). + +***inSystemEventGroup*** + +A boolean flag that indicates whether or not the event listener is in the system event group (see [nsIEventListenerInfo](https://developer.mozilla.org/docs/XPCOM_Interface_Reference/nsIEventListenerInfo#Attributes)). + +***isEventHandler*** + +A boolean flag indicating whether this is an event handler or an event listener (see [DOM Event Handlers](https://developer.mozilla.org/docs/Web/Guide/DOM/Events/Event_handlers#Definitions) for definitions of the two terms). For HTML attribute handlers or assignments to WebIDL properties this flag would be true. + +***function*** + +A grip on the function object. + +## Stream Transport + +The debugging protocol is specified in terms of packets exchanged between a client and server, where each packet is either a JSON text or a block of bytes (a "bulk data" packet). The protocol does not specify any particular mechanism for carrying packets from one party to the other. Implementations may choose whatever transport they like, as long as packets arrive reliably, undamaged, and in order. + +This section describes the Mozilla Remote Debugging Protocol Stream Transport, a transport layer suitable for carrying Mozilla debugging protocol packets over a reliable, ordered byte stream, like a TCP/IP stream or a pipe. Debugger user interfaces can use it to exchange packets with debuggees in other processes (say, for debugging Firefox chrome code), or on other machines (say, for debugging Firefox OS apps running on a phone or tablet). + +(The Stream Transport is not the only transport used by Mozilla. For example, when using Firefox's built-in script debugger, the client and server are in the same process, so for efficiency they use a transport that simply exchanges the JavaScript objects corresponding to the JSON texts specified by the protocol, and avoid serializing packets altogether.) + +### Packets + +Once the underlying byte stream is established, transport participants may immediately begin sending packets, using the forms described here. The transport requires no initial handshake or setup, and no shutdown exchange: the first bytes on the stream in each direction are those of the first packet, if any; the last bytes on the stream in each direction are the final bytes of the last packet sent, if any. + +The transport defines two types of packets: JSON and bulk data. + +#### JSON Packets + +A JSON packet has the form: + +``` +length:JSON +``` + +where *length* is a series of decimal ASCII digits, *JSON* is a well-formed JSON text (as defined in [RFC 4627](http://www.ietf.org/rfc/rfc4627.txt)) encoded in UTF-8, and *length*, interpreted as a number, is the length of *JSON* in bytes. + +#### Bulk Data Packets + +A bulk data packet has the form: + +``` +bulk actor type length:data +``` + +where: + +* The keyword `bulk` is encoded in ASCII, and the spaces are always exactly one ASCII space +* *actor* is a sequence of Unicode characters, encoded in UTF-8, containing no spaces or colons +* *type* is a sequence of Unicode characters, encoded in UTF-8, containing no spaces or colons +* *length* is a sequence of decimal ASCII digits +* *data* is a sequence of bytes whose length is *length* interpreted as a number + +The *actor* field is the name of the actor sending or receiving the packet. (Actors are server-side entities, so if the packet was sent by the client, *actor* names the recipient; and if the packet was sent by the server, *actor* names the sender.) The protocol imposes the same syntactic restrictions on actor names that we require here. + +Which actor names are valid at any given point in an exchange is established by the remote debugging protocol. + +The *type* field defines the type of the packet, which may be used with the actor name to route the packet to its destination properly. The protocol provides more detail about the type, which remains in effect here. + +The content of a bulk data packet is exactly the sequence of bytes appearing as *data*. Data is not UTF-8 text. + +### Stream Requirements + +The Stream Transport requires the underlying stream to have the following properties: + +* It must be **transparent**: each transmitted byte is carried to the recipient without modification. Bytes whose values are ASCII control characters or fall outside the range of ASCII altogether must be carried unchanged; line terminators are left alone. +* It must be **reliable**: every transmitted byte makes it to the recipient, or else the connection is dropped altogether. Errors introduced by hardware, say, must be detected and corrected, or at least reported (and the connection dropped). The Stream Transport includes no checksums of its own; those are the stream's responsibility. (So, for example, a plain serial line is not suitable for use as an underlying stream.) +* It must be **ordered**: bytes are received in the same order they are transmitted, and bytes are not duplicated. (UDP packets, for example, may be duplicated or arrive out of order.) + +TCP/IP streams and USB streams meet these requirements. + +### Implementation Notes + +#### Constant-Overhead Bulk Data + +Mozilla added bulk data packets to the protocol to let devices with limited memory upload performance profiling and other large data sets more efficiently. Profiling data sets need to be as large as possible, as larger data sets can cover a longer period of time or more frequent samples. However, converting a large data set to a JavaScript object, converting that object to a JSON text, and sending the text over the connection entails making several temporary complete copies of the data; on small devices, this limits how much data the profiler can collect. Avoiding these temporary copies would allow small devices to collect and transmit larger profile data sets. Since it seemed likely that other sorts of tools would need to exchange large binary blocks efficiently as well, we wanted a solution usable by all protocol participants, rather than one tailored to the profiler's specific case. + +In our implementation of this Stream Transport, when a participant wishes to transmit a bulk data packet, it provides the actor name, the type, the data's length in bytes, and a callback function. When the underlying stream is ready to send more data, the transport writes the packet's `bulk actor type length:` header, and then passes the underlying `nsIOutputStream` to the callback, which then writes the packet's data portion directly to the stream. Similarly, when a participant receives a bulk data packet, the transport parses the header, and then passes the actor name, type, and the transport's underlying `nsIInputStream` to a callback function, which consumes the data directly. Thus, while the callback functions may well use fixed-size buffers to send and receive data, the transport imposes no overhead proportional to the full size of the data. diff --git a/devtools/docs/book.json b/devtools/docs/book.json new file mode 100644 index 0000000000..a0a738e0b7 --- /dev/null +++ b/devtools/docs/book.json @@ -0,0 +1,6 @@ +{ + "plugins": ["hints"], + "structure": { + "summary": "index.md" + } +} diff --git a/devtools/docs/bugs-issues.md b/devtools/docs/bugs-issues.md new file mode 100644 index 0000000000..254e1fc81e --- /dev/null +++ b/devtools/docs/bugs-issues.md @@ -0,0 +1,6 @@ +# Bugs and issue trackers + +Since we have code in two different places, issues and bugs are to be found in two different places: + +* For code in `m-c`: [http://firefox-dev.tools/](http://firefox-dev.tools/) which also lets you filter by good bugs for beginners. +* For code in `devtools-html`: [this page](https://github.com/search?l=&q=org%3Adevtools-html+state%3Aopen&type=Issues) lists all the issues across the organisation and these are [available issues](https://github.com/search?l=&q=org%3Adevtools-html+state%3Aopen+label%3Aavailable&type=Issues) i.e. ready to be worked on. diff --git a/devtools/docs/contributing.md b/devtools/docs/contributing.md new file mode 100644 index 0000000000..094efd0c47 --- /dev/null +++ b/devtools/docs/contributing.md @@ -0,0 +1,47 @@ +# Contributing + +Thank you for taking the time to contribute! There are several areas where you can help: code, UX, bugs, talking about the DevTools, etc... + +--- + +## 👉 Code of conduct 👈 + +We strive for collaboration with [mutual respect](https://searchfox.org/mozilla-central/source/devtools/CODE_OF_CONDUCT.md) for each other. Mozilla also has a set of [participation guidelines](https://www.mozilla.org/en-US/about/governance/policies/participation/) which goes into greater detail specific to Mozilla employees and contributors. + +Please read the two links above before getting involved. **Contributions that don't abide by the code of conduct and participation guidelines will be rejected**. + +--- + +## Help with code + +Whether you're an external contributor or a Mozilla employee, the process to get your code into the repository is essentially the same: + +* You [find a bug to work on](./contributing/find-bugs.md) (*note: we use bugs to track 'broken' things, new features and even discussions*). +* [Work on the bug](./contributing/fixing-bugs.md). +* [Request a review](./contributing/making-prs.md) for your code. +* Land the code in the repository. +* And you've contributed—well done 😀 + +## Help with design and UX + +If you're more interested in user experience (think: wireframes, workflows, navigations... and not necessarily implementation details), please have a look at the [UX](https://github.com/firefox-devtools/ux) repository—our friendly designers will be more than happy to welcome you onboard. You can also have a look at the [issues](https://github.com/firefox-devtools/ux/issues) they are considering right now, to get an idea of how it works. + +## Help with BUGS! 🐛🐞 <!--TODO: we might want to split this out to another page with more detail, in addition to this introductory section--> + +Not less importantly, we also love **when people file bugs**. They help us a lot and are very valuable (specially when they come with reproducible steps, e.g. in the case of crashes or malfunctions). Here is a short [guide on how to file good bugs](./filing-good-bugs.md) if you've never done it before (or if you need a reminder). + +Another thing that is super valuable is **reproducing** bugs (to validate they're happening in more than one environment), and also **completing** bugs, i.e. ensuring the bug has steps to reproduce, a test case, etc, as [mentioned on the guide](./filing-good-bugs.md). This saves time for the person(s) who will work on the bug, as then they can jump straight to fixing or implementing whatever is needed, instead of doing research work. If you can do any of these for a given bug, add a comment with the additional data that you found out. + +Likewise, if you think that a bug is solved, because you can't reproduce it and doesn't happen any more, this is also useful to know. We can always do with closing more bugs, so please leave a comment detailing as much information as you can provide 😀 + +## Talking about the tools <!--TODO: same as above, might want a separate page on talking and maybe collecting talks?--> + +We really love when people talk about our work. Be it in a blog post, or in your favourite social media network, or user group, conference, you name it! We'd love to read/watch your article/talk, so please get in touch if you do. + +Answering other people's questions in [our Discourse forum](https://discourse.mozilla.org/c/devtools) or [developer mailing list](https://groups.google.com/forum/#!forum/mozilla.dev.developer-tools) is also helpful. + +## Other areas + +The above is not a comprehensive list; if you think you can help some other way that is not here, feel free to do it! + +**Whatever you choose to do, thank you so much for helping us** ❤️ diff --git a/devtools/docs/contributing/code-reviews-checklist.md b/devtools/docs/contributing/code-reviews-checklist.md new file mode 100644 index 0000000000..3b63f92bf2 --- /dev/null +++ b/devtools/docs/contributing/code-reviews-checklist.md @@ -0,0 +1,59 @@ +# Code reviews checklist + +This checklist is primarily aimed at reviewers, as it lists important points to check while reviewing a patch. + +It can also be useful for patch authors: if the changes comply with these guidelines, then it's more likely the review will be approved. + +## Bug status and patch file + +* Bug status is assigned, and assignee is correctly set. +* Commit title and message follow [the conventions](https://developer.mozilla.org/en-US/docs/Mercurial/Using_Mercurial#Commit_Message_Conventions). +* Commit message says [what is being changed and why](http://mozilla-version-control-tools.readthedocs.org/en/latest/mozreview/commits.html#write-detailed-commit-messages). +* Patch applies locally to current sources with no merge required. +* Check that every new file introduced by the patch has the proper Mozilla license header: https://www.mozilla.org/en-US/MPL/headers/ + +## Manual testing + +* Verify: + * if it's a new feature, the patch implements it. + * if it's a fix, the patch fixes the bug it addresses. +* Report any problems you find in the global review comment. +* Decide if any of those problems should block landing the change, or if they can be filed as follow-up bugs instead, to be fixed later. + +## Automated testing + +* Run new/modified tests, [with and without e10s](../tests/writing-tests.md#electrolysis). +* Watch out for tests that pass but log exceptions or end before protocol requests are handled. +* Watch out for slow/long tests: suggest many small tests rather than single long tests. +* Watch out for new tests written as integration tests instead of as unit tests: unit tests should be the preferred option, when possible. + +## Code review + +* Code changes: + * Review only what was changed by the contributor. + * Code formatting follows [our ESLint rules](eslint.md) and [coding standards](./coding-standards.md). + * Code is properly commented, JSDoc is updated, new "public" methods all have JSDoc, see the [comment guidelines](./javascript.md#comments). + * If Promise code was added/modified, the right promise syntax is used and rejections are handled. See [asynchronous code](./javascript.md#asynchronous-code). + * If a CSS file is added/modified, it follows [the CSS guidelines](./css.md). + * If a React or Redux module is added/modified, it follows the [React/Redux guidelines](./javascript.md#react--redux). + * If DevTools server code that should run in a worker is added/modified then it shouldn't use Services +* Test changes: + * The feature or bug is [tested by new tests, or a modification of existing tests](../tests/writing-tests.md). + * [Test logging](../tests/writing-tests.md#logs-and-comments) is sufficient to help investigating test failures/timeouts. + * [Test is e10s compliant](../tests/writing-tests.md#e10s-electrolysis) (doesn't try to access web content from the parent process, etc…). + * Tests are [clean and maintainable](../tests/writing-tests.md#writing-clean-maintainable-test-code). + * A try push has started (or even better, is green already). +* User facing changes: + * If any user-facing interfaces are added/modified, double-check the changes with the UX mockups or specs, if available. If there's any confusion, need-info the UX designer.<!--TODO this needs updating with the new process--> + * If a user facing string has been added, it is localized and follows [the localization guidelines](../files/adding-files.md#localization-l10n). + * If a user-facing string has changed meaning, [the key has been updated](https://developer.mozilla.org/en-US/docs/Mozilla/Localization/Localization_content_best_practices#Changing_existing_strings). + * If a new image is added, it is a SVG image or there is a reason for not using a SVG. + * If a SVG is added/modified, it follows [the SVG guidelines](../frontend/svgs.md). + * If a documented feature has been modified, the keyword `dev-doc-needed` is present on the bug. + +## Finalize the review + +* R+: the code should land as soon as possible. +* R+ with comments: there are some comments, but they are minor enough, or don't require a new review once addressed, trust the author. +* R cancel / R- / F+: there is something wrong with the code, and a new review is required. + diff --git a/devtools/docs/contributing/code-reviews-find-reviewer.md b/devtools/docs/contributing/code-reviews-find-reviewer.md new file mode 100644 index 0000000000..fc1a8d3c8e --- /dev/null +++ b/devtools/docs/contributing/code-reviews-find-reviewer.md @@ -0,0 +1,7 @@ +# Finding suitable reviewers + +The code in DevTools is divided into modules (because there's just so much code it's impossible for one person to know *everything* off the top of their head). + +Each module has one or more "owners", who know a lot about it. And they're the people you want to ask to review your code 😀 + +You might already know who is the reviewer (perhaps the person that is mentoring the bug, perhaps your team mate), but in case of doubt, you can [consult this list of people and modules](https://firefox-dev.tools/#people-and-modules). diff --git a/devtools/docs/contributing/code-reviews-setup.md b/devtools/docs/contributing/code-reviews-setup.md new file mode 100644 index 0000000000..8813e58ddb --- /dev/null +++ b/devtools/docs/contributing/code-reviews-setup.md @@ -0,0 +1,27 @@ +# Set up for code reviews + +There are two things you need to do before you can get a code review, although you only need to do this once 😃 + +## Set up to get code reviews in Phabricator + +We use an online tool called Phabricator for code reviews. To create an account in Phabricator, you first need the Bugzilla account that you created earlier. If you don't have one, [create it now](../getting-started/bugzilla.md). + +--- + +⚠️ *IMPORTANT:* ⚠️️️ + +It's helpful to have the same user name in both Bugzilla and Phabricator, so that people always know how to find you. + +Bugzilla's `Real name` field can be edited after the fact, but you cannot change Phabricator's username once the account has been created. + +If you added an `:ircnickname` in your Bugzilla's `Real name`, Phabricator will use that to pre-fill the username field when you create the account. **Please double check you like the proposed username, and make any corrections before you register**. + +--- + +Once you understand the above, please [create a Phabricator account](https://moz-conduit.readthedocs.io/en/latest/phabricator-user.html#creating-an-account). + + + +## Set up to send code for review + +In order to push your commit to Phabricator, you need to install [moz-phab](https://moz-conduit.readthedocs.io/en/latest/phabricator-user.html#using-moz-phab). diff --git a/devtools/docs/contributing/code-reviews.md b/devtools/docs/contributing/code-reviews.md new file mode 100644 index 0000000000..9ddb03b5bb --- /dev/null +++ b/devtools/docs/contributing/code-reviews.md @@ -0,0 +1,104 @@ +# Code reviews + +A review is required before code is added to Mozilla's repository. In addition, contrary to what you might have seen in other open source projects, code is not pushed directly to the repository. Instead, once the code is approved by reviewers, they will _request_ the code to be _landed_ on the repository. + +All this can be done somehow manually, but we have infrastructure in place to help us with reviews and landing code. + +Learn about how to get started with getting your code reviewed and landed in our [setting up](./code-reviews-setup.md) guide. + +And read on to learn about why and how we do code reviews. + +## Why do we do code reviews? + +### Quality + +Doing code reviews helps with **correctness**. It gives us a chance to check that the code: + +- fixes the problem, +- doesn't have bugs, +- doesn't regress other things, +- covers edge cases. + +A review is also a chance for the reviewer to check that the right **test coverage** is here, that the change doesn't have **performance** problems, that it **simplifies hard to understand code** and that it comes with **code comments** and **adequate documentation**. + +### Learning and sharing knowledge + +While going through the process of doing a code review, **both the reviewer and the reviewee** will be learning something (a new part of the code, a new efficient way to write code, tests, localization, etc.). + +Making the code easy to understand by the reviewer helps everybody. + +Doing reviews also helps people working on DevTools feel more comfortable and learn about new parts of the codebase. + +It helps build consensus around ideas and practices. + +### Maintenance + +Doing reviews gives us an opportunity to check we are willing to maintain and support the new code that being introduced for a feature or bug fix. + +### Code consistency + +It is also a great opportunity for us to check that whatever new code is being written is consistent with what already exists in the code base. + +### Shared responsibility + +Approving a code review means that you are the second person who thinks this change is correct and a good idea. Doing this makes you responsible for the code change just as much as the author. + +It is the entire DevTools group who owns the code, not just the author. We write code as a group, not as individuals, because on the long-term it's the group that maintains it. + +Having a review discussion shares the ownership of the code, because authors and reviewers all put a little bit of themselves in it, and the code that results isn't just the result of one person's work + +## What should be avoided in code reviews? + +- Style nits that a linter should ideally handle. +- Requests that are outside of the scope of the bug. + +## What are some good practices for code reviews? + +### Use empathetic language. + +More reading: + +- [Mindful Communication in Code Reviews](http://amyciavolino.com/assets/MindfulCommunicationInCodeReviews.pdf) +- [How to Do Code Reviews Like a Human](https://mtlynch.io/human-code-reviews-1/) + +### Prefer smaller commits over large monolithic commits. + +It makes it easier for the reviewer to provide a quality review. It's also easier to get consensus on the change if that change is small. Finally, it's easier to understand the risk, and find regressions. + +### Be explicit + +Specifically, be explicit about required versus optional changes. + +Reviews are conversations, and the reviewee should feel comfortable on discussing and pushing back on changes before making them. + +### Be quick + +As a reviewer, strive to answer in a couple of days, or at least under a week. + +If that is not possible, please update the bug to let the requester know when you'll get to the review. Or forward it to someone else who has more time. + +### Ask for help + +Know when you're out of your comfort zone as a reviewer, and don't hesitate to ask for an extra review to someone else. + +It's fine, you can't know everything, and the more people participate, the more everybody learns, and the more bugs we avoid. + +## How do we communicate? + +First and foremost, like in any Mozilla-run platforms or events, please abide by [the Community Participation Guidelines](https://www.mozilla.org/en-US/about/governance/policies/participation/). + +Maintainers should **lead by example through their tone and language**. We want to build an inclusive and safe environment for working together. + +As a reviewer, **double-check your comments**. Just because you're a reviewer and think you have a better solution doesn't mean that's true. Assume **the code author has spent more time thinking about this part of the code than you have** (if you're the reviewer) and might actually be right, even if you originally thought something was wrong. It doesn't take long to look up the code and double-check. + +**Being inclusive** is highly important. We shouldn't make any assumptions about the person requesting a review, or about the person you're asking a review from, and always provide as much information as required, in a way that is as inclusive as possible. + +The bug will live forever online, and many people will read it long after the author and reviewers are done. + +Talking over video/IRC/Slack is a great way to get buy-in and share responsibility over a solution. It is also helpful to resolve disagreement and work through issues in a more relaxed environment. + +**Do not criticize** the reviewee, talk about the quality of the code on its own, and not directly how the reviewee wrote the code. + +Take the time to thank and point out good code changes. + +**Using "please" and “what do you think?”** goes a long way in making others feel like colleagues, and not subordinates. diff --git a/devtools/docs/contributing/coding-standards.md b/devtools/docs/contributing/coding-standards.md new file mode 100644 index 0000000000..553f1f622c --- /dev/null +++ b/devtools/docs/contributing/coding-standards.md @@ -0,0 +1,10 @@ +# Coding standards + +Our code base is quite large, and a lot of different people contribute to it all the time. Therefore, it's important to share standards to keep the code consistent and written in a predictable style. This also helps avoid common mistakes. + +We have pages defining standards, best practices and tips for various languages used in our tools: + +* [JavaScript](./javascript.md) +* [CSS](./css.md) +* [SVG](../frontend/svgs.md) + diff --git a/devtools/docs/contributing/css.md b/devtools/docs/contributing/css.md new file mode 100644 index 0000000000..26ab276d58 --- /dev/null +++ b/devtools/docs/contributing/css.md @@ -0,0 +1,132 @@ +# CSS + +This page is for information about CSS used by DevTools. Wondering about the Dev Edition theme? See this page for more information about the [Developer Edition theme](https://wiki.mozilla.org/DevTools/Developer_Edition_Theme). + +## Basics + +The CSS code is in `devtools/client/themes`. + +Here are some basic tips that can optimize reviews if you are changing CSS: + +* Avoid `!important` but if you have to use it, make sure it's obvious why you're using it (maybe with a comment). +* Avoid magic numbers, prefer automatic sizing. +* Avoid platforms specific styles, put everything in the `shared` directory. +* Avoid preprocessor variables, use CSS variables instead. +* Avoid setting styles in JavaScript. It's generally better to set a class and then specify the styles in CSS +* `classList` is generally better than `className`. There's less chance of over-writing an existing class. + +### Boilerplate + +Make sure each file starts with the standard copyright header (see [License Boilerplate](https://www.mozilla.org/MPL/headers/)). + +### Testing + +CSS changes should generally be similar across platforms since they used a shared implementation, but there can still be differences worth checking out. Check major changes on Windows, OS X and Ubuntu. + +## Formatting + +We use 2-spaces indentation for the CSS. + +In general the formatting looks like this: + +```css +selector, +alternate-selector { + property: value; + other-property: other-value; +} +``` +<!--TODO: add examples for long shorthand properties, and multi-valued properties (background, font-family, ...)--> +Also: + +* Omit units on 0 values. + * Example: Use `margin: 0;`, not `margin: 0px;`. +* Add a space after each comma, **except** within color functions. + * Example: `linear-gradient(to bottom, black 1px, rgba(255,255,255,0.2) 1px)`. +* Always add a space before ` !important`. +* Assume `="true"` in attribute selectors. + * Example: Use `option[checked]`, not `option[checked="true"]`. +* Use longhand versions of properties so it's clear what you're changing. + * Example: Use `border-color: red`, not `border: red;`. + +Naming standards for class names: + +* `lower-case-with-dashes` is the most common. +* But `camelCase` is also used sometimes. Try to follow the style of existing or related code. + +## Light and Dark theme support + +DevTools supports 2 different themes: the dark theme and the light theme. In order to support them, there are 2 class names available (`theme-dark` and `theme-light`). + +* Use [pre-defined CSS variables](https://searchfox.org/mozilla-central/source/devtools/client/themes/variables.css) instead of hardcoding colors when possible. +* If you need to support themes and the pre-defined variables don't fit, define a variable with your custom colors at the beginning of the CSS file. This avoids selector duplication in the code. + +Example: + +```css +.theme-light { + --some-variable-name: <color-for-light-theme>; +} +.theme-dark { + --some-variable-name: <color-for-dark-theme>; +} +#myElement { + background-color: var(--some-variable-name); +} +``` + +## HDPI support + +It's recommended to use SVG since it keeps the CSS clean when supporting multiple resolutions. However, if only 1x and 2x PNG assets are available, you can use this `@media` query to target higher density displays (HDPI): `@media (min-resolution: 1.1dppx)`. <!--TODO an example would be good here--> + +## Performance + +* Read [Writing Efficient CSS](https://developer.mozilla.org/en-US/docs/Web/Guide/CSS/Writing_efficient_CSS). +* Use an iframe where possible so your rules are scoped to the smallest possible set of nodes.<!--TODO: is this still true? and also refine exactly when it is appropriate to use an iframe. Examples might help--> +* If your CSS is used in `browser.xhtml`, you need to take special care with performance: + * Descendent selectors should be avoided. + * If possible, find ways to use **only** id selectors, class selectors and selector groups. + +## Localization + +### Text Direction +* For margins, padding and borders, use `inline-start`/`inline-end` rather than `left`/`right`. + * Example: Use `margin-inline-start: 3px;` not `margin-left: 3px`. +* For RTL-aware positioning (left/right), use `inset-inline-start/end`. +* When there is no special RTL-aware property (eg. `float: left|right`) available, use the pseudo `:-moz-locale-dir(ltr|rtl)` (for XUL files) or `:dir(ltr|rtl)` (for HTML files). +* Remember that while a tab content's scrollbar still shows on the right in RTL, an overflow scrollbar will show on the left. +* Write `padding: 0 3px 4px;` instead of `padding: 0 3px 4px 3px;`. This makes it more obvious that the padding is symmetrical (so RTL won't be an issue). + +### RTL support for html modules + +By default, new HTML modules support only left-to-right (LTR) and do not reuse the current direction of the browser. + +To enable right-to-left (RTL) support in a module, set the `[dir]` attribute on the document element of the module: +* Example: `<html xmlns="http://www.w3.org/1999/xhtml" dir="">`. + +The appropriate value for the `dir` attribute will then be set when the toolbox loads this module. + +### Testing + +The recommended workflow to test RTL on DevTools is to use the [Force RTL extension](https://addons.mozilla.org/en-US/firefox/addon/force-rtl/). After changing the direction using Force RTL, you should restart DevTools to make sure all modules apply the new direction. A future version of Force RTL will be able to update dynamically all DevTools documents.<!--TODO: update when the fate of this addon/webextension is known--> + +## Toggles + +Sometimes you have a style that you want to turn on and off. For example a tree twisty (a expand-collapse arrow), a tab background, etc. + +The Mozilla way is to perform the toggle using an attribute rather than a class: + +```css +.tree-node { + background-image: url(right-arrow.svg); +} +.tree-node[open] { + background-image: url(down-arrow.svg); +} +``` + +## Tips + +* Use `:empty` to match a node that doesn't have children. +* Usually, if `margin` or `padding` has 4 values, something is wrong. If the left and right values are asymmetrical, you're supposed to use `-start` and `-end`. If the values are symmetrical, use only 3 values (see localization section). + diff --git a/devtools/docs/contributing/eslint-atom-settings.png b/devtools/docs/contributing/eslint-atom-settings.png Binary files differnew file mode 100644 index 0000000000..1108c206dc --- /dev/null +++ b/devtools/docs/contributing/eslint-atom-settings.png diff --git a/devtools/docs/contributing/eslint-atom.png b/devtools/docs/contributing/eslint-atom.png Binary files differnew file mode 100644 index 0000000000..52350a31ec --- /dev/null +++ b/devtools/docs/contributing/eslint-atom.png diff --git a/devtools/docs/contributing/eslint-sublimetext3.png b/devtools/docs/contributing/eslint-sublimetext3.png Binary files differnew file mode 100644 index 0000000000..8a76f06cda --- /dev/null +++ b/devtools/docs/contributing/eslint-sublimetext3.png diff --git a/devtools/docs/contributing/eslint-vscode.png b/devtools/docs/contributing/eslint-vscode.png Binary files differnew file mode 100644 index 0000000000..41a58fe9d4 --- /dev/null +++ b/devtools/docs/contributing/eslint-vscode.png diff --git a/devtools/docs/contributing/eslint.md b/devtools/docs/contributing/eslint.md new file mode 100644 index 0000000000..73f50c75de --- /dev/null +++ b/devtools/docs/contributing/eslint.md @@ -0,0 +1,158 @@ +# Using ESLint in DevTools +<!--TODO paths, executables and everything here should be reviewed when we go to GitHub--> + +The main rule set is in `devtools/.eslintrc`. It is meant to be used with ESLint 3.5.0. + +Note that the file `.eslintignore` at the root of the repository contains a list of paths to ignore. This is because a lot of the code isn't ESLint compliant yet. We're in the process of making code free of ESLint warnings and errors, but this takes time. In the meantime, make sure the file or folder you are running ESLint on isn't ignored. + +## Installing + +From the root of the project type: + +`./mach eslint --setup` + +ESLint, `eslint-plugin-html`, `eslint-plugin-mozilla` and `eslint-plugin-react` will be automatically downloaded and installed. + +## Running ESLint + +### From the command line + +The preferred way of running ESLint from the command line is by using `mach` like this: + +```bash +./mach eslint path/to/directory +``` + +This ensures that ESLint runs with the same configuration that our CI environment (see the next section). + +### In continuous integration + +Relying only on people to run ESLint isn't enough to guarantee new warnings or errors aren't introduced in the code. Therefore, ESLint also runs automatically in our Continuous Integration environment. + +This means that every time a commit is pushed to one of the repositories, a job runs ESLint on the whole code. + +If you are pushing a patch to the [`try` repository](https://wiki.mozilla.org/ReleaseEngineering/TryServer) to run the tests, then you can also tell it to run the ESLint job and therefore verify that you did not introduce new errors. + +If you build on all platforms, then the ESLint job will run by default, but if you selected a few platforms only in your [trysyntax](https://wiki.mozilla.org/Build:TryChooser), then you need to also add `eslint-gecko` as a target platform for ESLint to run. + +### Running ESLint in SublimeText + +SublimeText is a popular code editor and it supports ESLint via a couple of plugins. Here are some pointers to get started: + +* make sure you have [SublimeText 3](http://www.sublimetext.com/3), the linter plugin doesn't work with ST2, +* install [SublimeLinter 3](http://www.sublimelinter.com/en/latest/installation.html), this is a framework for linters that supports, among others, ESLint. Installing SublimeLinter via [Package Control](https://packagecontrol.io/installation) is the easiest way) +* with SublimeLinter installed, you can now [install the specific ESLint plugin](https://github.com/roadhump/SublimeLinter-eslint#linter-installation). The installation instructions provide details about how to install node, npm, eslint which are required). +* make sure to configure SublimeLinter with the `--no-ignore` option so that errors are also shown for source files that are ignored. To do this, open the SublimeLinter user configuration at: Preferences / Package Settings / SublimeLinter / Settings - User, and add `"args": "--no-ignore"` to the eslint linter object. + +You will also need to point SublimeLinter at the local eslint installation by setting the path to whatever `./mach eslint --setup` gives you when you run it (include a trailing slash but remove the eslint binary filename) e.g. + +NOTE: Your local eslint binary is at /some-project-path/tools/lint/eslint/node_modules/.bin/eslint + +``` + "paths": { + "linux": [], + "osx": [ + "/some-project-path/tools/lint/eslint/node_modules/.bin" + ], + "windows": [ + "C:\\some-project-path\\tools\\lint\\eslint\\node_modules\\.bin" + ] + }, +``` + +Once done, open the mozilla project in SublimeText and open any JS file in the `/devtools` directory. You can then trigger the linter via the contextual menu (right click on the file itself) or with a keyboard shortcut (ctrl+option+L on Mac). + +You should see errors and warnings in the gutter as shown in the screenshot below. You can also see all errors listed with ctrl+option+A, and individual errors by clicking in the gutter marker. + + + +### Running ESLint in Emacs + +* First, install the flycheck package (flymake doesn't support ESLint yet). You can get flycheck from the [marmalade](https://marmalade-repo.org/) or [melpa-stable](http://stable.melpa.org/#/) repositories. + +* Tell flycheck to disable jslint, and enable flycheck in your javascript mode. Some care is needed to find the eslint installed in the source tree. This snippet assumes the built-in javascript mode, but with minor changes (just the name of the hook) should work fine with js2-mode as well: +```lisp +(defun my-js-mode-hacks () + (setq-local mode-name "JS") + ;; Set this locally so that the head.js rule continues to work + ;; properly. In particular for a mochitest we want to preserve the + ;; "browser_" prefix. + (when (buffer-file-name) + (let ((base (file-name-nondirectory (buffer-file-name)))) + (when (string-match "^\\([a-z]+_\\)" base) + (setq-local flycheck-temp-prefix (match-string 1 base)))) + (let ((base-dir (locate-dominating-file (buffer-file-name) + ".eslintignore"))) + (when base-dir + (let ((eslint (expand-file-name + "tools/lint/eslint/node_modules/.bin/eslint" base-dir))) + (when (file-exists-p eslint) + (setq-local flycheck-javascript-eslint-executable eslint)))))) + (flycheck-mode 1)) +(require 'flycheck) +(setq-default flycheck-disabled-checkers + (append flycheck-disabled-checkers + '(javascript-jshint))) +(add-hook 'js-mode-hook #'my-js-mode-hacks) +``` + +* flycheck puts its bindings on `C-c !` by default, so use `C-c ! C-h` to see what is available. There are key bindings to list all the errors and to move through the errors, among other things. +* To make sure flycheck is finding eslint, open a .js file and run `M-x flycheck-verify-setup`. It should show the path to your eslint installation. + +### Running ESLint in Atom + +From the root of the project type: + +`./mach eslint --setup` + +Install the [linter-eslint](https://atom.io/packages/linter-eslint) package v.8.00 or above. Then go to the package settings and enable the following options: + + + +Once done, you should see errors and warnings as shown in the screenshot below. + + + +### Running ESLint in ViM + +If you don't use Syntastic yet, the instructions here should get you started: https://wiki.mozilla.org/WebExtensions/Hacking#Vim + +Alternatively, if you do use Syntastic already, add this to your `.vimrc` to get ESLint working where the path contains `mozilla-central` (adjust the path to reflect the one in your computer): + +```vim + autocmd FileType javascript,html + \ if stridx(expand("%:p"), "/mozilla-central/") != -1 | + \ let b:syntastic_checkers = ['eslint'] | + \ let b:syntastic_eslint_exec = '/path/to/mozilla-central/tools/lint/eslint/node_modules/.bin/eslint' | + \ let b:syntastic_html_eslint_args = ['--plugin', 'html'] | + \ endif +``` + +You probably need to close and reopen ViM for the changes to take effect. Then, open any file and try to edit it to cause an error, then save it. If all goes well, you will get some distinctive arrows pointing to the error. Hovering with the mouse will produce a sort of tooltip with more information about the error. + +### Running ESLint in Visual Studio Code + +From the root of the project type: + +`./mach eslint --setup` + +Install the [dbaeumer.vscode-eslint](https://marketplace.visualstudio.com/items?itemName=dbaeumer.vscode-eslint) package. Then go to the package settings and set the following option: + +`"eslint.nodePath": "tools/lint/eslint/node_modules/.bin"` + +Once done, you should see errors and warnings as shown in the screenshot below: + + + +### Fixing ESLint Errors + +This should help you write eslint-clean code: + +* When moving or refactoring a piece of code, consider this as an opportunity to remove all ESlint errors from this piece of code. In fact, it may even be a good opportunity to remove all ESLint errors from the entire file. +* When doing ESLint-only changes, please do them in a separate patch from the actual functionality changes or bug fix. This helps make the review easier, and isolate the actual changes when looking at the source history. +* When cleaning an entire file or folder from ESLint errors, do not forget to remove the corresponding entry from the `.eslintignore` file. +* When writing new code, from scratch, please make it ESLint compliant from the start. This is a lot easier than having to revisit it later. +* ESLint also runs on `<script>` tags in HTML files, so if you create new HTML test files for mochitests for example, make sure that JavaScript code in those files is free of ESLint errors. +* Depending on how a dependency is loaded into a file, the symbols this dependency exports might not be considered as defined by ESLint. For instance, using `Cu.import("some.jsm")` doesn't explicitly say which symbols are now available in the scope of the file, and so using those symbols will be consider by ESLint as using undefined variables. When this happens, please avoid using the `/* globals ... */` ESLint comment (which tells it that these variables are defined). Instead, please use `/* import-globals-from relative/path/to/file.js */`. This way, you won't have a list of variables to maintain manually, the globals are going to be imported dynamically instead. +* In test files (xpcshell and mochitest), all globals from the corresponding `head.js` file are imported automatically, so you don't need to define them using a `/* globals ... */` comment or a `/* import-globals-from head.js */` comment. + diff --git a/devtools/docs/contributing/filing-good-bugs.md b/devtools/docs/contributing/filing-good-bugs.md new file mode 100644 index 0000000000..c57979dff7 --- /dev/null +++ b/devtools/docs/contributing/filing-good-bugs.md @@ -0,0 +1,11 @@ +# Filing good bugs + +Getting started working on a bug can be hard, specially if you lack context. + +This guide is meant to provide a list of steps to provide the necessary information to resolve a bug. + +* Use a descriptive title. Avoid jargon and abbreviations where possible, they make it hard for other people to find existing bugs, and to understand them. +* Explain the problem in depth and provide the steps to reproduce. Be as specific as possible, and include things such as operating system and version if reporting a bug. +* If you can, list files and lines of code that may need to be modified. Ideally provide a patch for getting started. +* If applicable, provide a test case or document that can be used to test the bug is solved. For example, if the bug title was "HTML inspector fails when inspecting a page with one million of nodes", you would provide an HTML document with one million of nodes, and we could use it to test the implementation, and make sure you're looking at the same thing we're looking at. You could use services like jsfiddle, codepen or jsbin to share your test cases. Other people use GitHub, or their own web server. +* If it's a bug that new contributors can work on, add the keyword `good-first-bug`. diff --git a/devtools/docs/contributing/find-bugs.md b/devtools/docs/contributing/find-bugs.md new file mode 100644 index 0000000000..b756904844 --- /dev/null +++ b/devtools/docs/contributing/find-bugs.md @@ -0,0 +1,6 @@ +# Find bugs to work on + +* Choose something from [the list of existing bugs](https://bugs.firefox-dev.tools/?easy&tool=all). You can filter by tools (e.g. only `Console` bugs), and also by good bugs for beginners. +* Or if you would like to work on something that is not listed there, [file a bug in Bugzilla](https://bugzilla.mozilla.org/enter_bug.cgi?product=DevTools) (you'll need the Bugzilla account [you created earlier](../getting-started/bugzilla.md)) and ask for it to be assigned to you. Please also try to initiate a conversation in the bug first, to ensure that you don't work on something that will not be accepted (for example, if you think you found a bug, but the feature worked that way by design). + +<!-- TODO: mention finding potential work that is captured as a TODO or FIXME comments, but doesn't have an associated filed bug --> diff --git a/devtools/docs/contributing/fixing-bugs.md b/devtools/docs/contributing/fixing-bugs.md new file mode 100644 index 0000000000..d2df3e377d --- /dev/null +++ b/devtools/docs/contributing/fixing-bugs.md @@ -0,0 +1,170 @@ +# How to fix a bug + +## Make sure you understand what needs to be done + +If you're not quite sure of this, please add a comment requesting more information in the bug itself. It is absolutely fine to also talk to someone via other means (e.g. irc, email, in the office kitchen...), but it is good to come back to the bug and add the extra information, just in case you stop working on the bug at some point and someone else has to pick work where you left it. + +## Find out where are the files that need changing + +In an ideal world, the bug has this information from the start, but very often it doesn't. + +If you're not yet familiar with the codebase, the [files and directories overview](../files/README.md) might help you find where things are. + +If you know what you're looking for (e.g. a string that has a typo and needs correcting, or a function name that needs modifying), you can use a source code search engine: + +* [Searchfox](http://searchfox.org/mozilla-central/source) + +It is a good idea to [add smart keyword searches](https://support.mozilla.org/en-US/kb/how-search-from-address-bar) for DXR to search faster. + +You can also use your operating system's command line. For example, let's search for occurrences of `TODO` in the code base. + +Within your command line prompt, `cd` to the `devtools` directory: + +```bash +cd ~/mozilla-central/devtools # use your actual folder name! +grep -r 'TODO' . +``` + +This will list all the places in the DevTools code that contain the `TODO` string. If there are too many instances, you can combine the above with `less`, to scroll and paginate the output of `grep`: + +```bash +grep -r 'TODO' . | less +``` + +Press `q` to exit. + +If after all of this you still can't find your bearings, add a comment in the bug asking for more information. + +## How do you know that you have found what you were looking for? + +There are a few options to confirm that you found the right files: + +### If this is about changing a string... + +Edit the file(s), and change the string (e.g. fix a typo). Rebuild, and run again: + +```bash +./mach build +./mach run +``` +Then go to the panel that displays the string you wanted to change. + +Does the typo still occur? Or is the string being displayed the correct one now? + +### If this is about changing JavaScript code... + +If you think you found the right file to edit, add some logging statement on a place of the code which is likely to be executed (for example, on a class constructor): + +```javascript +// For front-end code +console.log('hello friends\n'); + +// Sometimes it's hard to spot your output. Emojis can help here really well. +console.log('👗👗👗', 'This is your logged output!'); +``` + +Or... + +```javascript +// For server code +dump('hello friends\n'); +``` + +TIP: Whether to use one or another depends on the type of bug you're working on, but if you've just started in DevTools, it's highly likely you'll take a front-end bug first. + +Then rebuild and run again: + +```bash +./mach build +./mach run +``` + +Go to the panel or initiate the action that is likely to result on the code being executed, and pay close attention to the output in your console. + +Can you see `hello friends`? Then you found the file that you were looking for. + +It's possible that you'll get a lot of other messages you don't care about now, but we can use `grep` to filter: + +```bash +./mach run | grep hello friends +``` + +This will only show lines that contain `hello friends`. If you get an empty output after trying to get the code to execute, maybe this isn't the right file, or maybe you didn't trigger the action. + +And that means it's time to ask for more information in the bug or from your colleagues. Tell them what you tried, so they don't have to figure that out themselves (saves everyone some time!). + +### If this is about changing CSS code... + +If you think you have found the right file and the right CSS selector, you could try to edit the file to insert some outrageously colourful rule (for example, a really thick bright blue border), then restart DevTools to see if the changes show up. + +```css +border: 4px solid blue; +``` + +## NEXT: do whatever needs doing + +This will always depend on the specific bug you're working on, so it's hard to provide guidance here. + +The key aspect here is that if you have questions, you should not hesitate to ask. Ask your mentor, your manager, or [get in touch](https://firefox-dev.tools/#getting-in-touch). **You should just not get stuck**. + +Some people find it difficult to recognise or even admit they're in such a situation, so some ways to describe 'stuck' could be: + +* you've tried everything you could think of, nothing works, and do not know what else to do. +* you have various ideas for things that can be done, and do not know which one to go for. +* you have not learned anything new about the problem in the last one or two days. +* you're starting to think about abandoning this bug and doing something else instead. +* you don't know what to do, but are afraid of asking for help. + +If you think *any* of the above describes you, ask for help! + +Another tip you can use if you're afraid that you're wasting people's time is to timebox. For example, give yourself 2 hours to investigate. If you can't figure anything after that time has elapsed, stop and ask for help. It might be that you needed a key piece of information that was missing in the bug description, or you misunderstood something, or maybe even that you found a bug and that's why things didn't work even if they should! This is why it's important to call for help sooner rather than later. + +### Useful references + +#### Coding standards + +If it's your first time contributing, the documentation on [coding standards](./coding-standards.md) might have answers to questions such as what style of code to use, how to name new files (if you have to add any), tools we use to run automated checks, etc. + +#### Specialised guides + +We also have a few guides explaining how to work on specific types of problems, for example: [investigating performance issues](./performance.md), or [writing efficient React code](./react-performance-tips.md). Please have a look at the sidebar or use the search box on top of the sidebar to see if there's something written about the type of bug you're working on. + +If not, maybe you'll be able to contribute with one by the time you fix your bug! + +#### MDN + +[MDN Web Docs](http://developer.mozilla.org/) (also known as *MDN*) has a lot of information about HTML, CSS, JS, DOM, Web APIs, Gecko-specific APIs, and more. + +## Run tests + +We have several types of automated tests to help us when developing. + +Some, like the linting tests, address coding style; others address functionality, such as unit and integration tests. This page has more [details on types of tests and how to run them](../tests/). + +You might want to run the unit and integration types of tests quite frequently, to confirm you're not breaking anything else. Depending on what you're doing, it might be even possible to run just one test file which addresses the specific change you're implementing: + +```bash +./mach test devtools/path/to/test.js +``` + +Sometimes you might want to run a number of tests which are related to the bug you're fixing: + +```bash +./mach test devtools/path/to/test-thing-*.js +``` + +At the beginning, it is entirely possible that you have no idea of where the tests are for the thing you're working on. Please ask for help! You will eventually learn your way around. + +It is good etiquette to ensure the tests pass locally before asking for a code review. This includes linting tests. To run them, please [configure your system to run ESlint](./eslint.md), and then you can execute: + +```bash +./mach eslint devtools/path/to/files/you/changed +``` + +Our tool for code review will run the linter automatically as well, but if you run this locally you'll get instant feedback, and avoid having to send an updated commit again. + +## Time for a review + +When you think you have fixed the bug, first let's celebrate a bit! Yay! Well done 😀 + +And now it's time to [get your code reviewed](./code-reviews.md). diff --git a/devtools/docs/contributing/javascript.md b/devtools/docs/contributing/javascript.md new file mode 100644 index 0000000000..f912a1a9da --- /dev/null +++ b/devtools/docs/contributing/javascript.md @@ -0,0 +1,89 @@ +# JavaScript coding standards + +Probably the best piece of advice is **to be consistent with the rest of the code in the file**. + +We use [ESLint](http://eslint.org/) to analyse JavaScript files automatically, either from within a code editor or from the command line. Here's [our guide to install and configure it](./eslint.md). + +For quick reference, here are some of the main code style rules: + +* file references to browser globals such as `window` and `document` need `/* eslint-env browser */` at the top of the file, +* lines should be 90 characters maximum, +* indent with 2 spaces (no tabs!), +* `camelCasePlease`, +* don't open braces on the next line, +* don't name function expressions: `let o = { doSomething: function doSomething() {} };`, +* use a space before opening paren for anonymous functions, but don't use one for named functions: + * anonymous functions: `function () {}` + * named functions: `function foo() {}` + * anonymous generators: `function* () {}` + * named generators: `function* foo() {}` +* aim for short functions, 24 lines max (ESLint has a rule that checks for function complexity too), +* `aArguments aAre the aDevil` (don't use them please), +* `"use strict";` globally per module, +* `semicolons; // use them`, +* no comma-first, +* consider using async/await for nice-looking asynchronous code instead of formatting endless `.then` chains, +* use ES6 syntax: + * `function setBreakpoint({url, line, column}) { ... }`, + * `(...args) => { }` rest args are awesome, no need for `arguments`, + * `for..of` loops, +* don't use non-standard SpiderMonkey-only syntax: + * no `for each` loops, + * no `function () implicitReturnVal`, + * getters / setters require { }, +* only import specific, explicitly-declared symbols into your namespace: + * `const { foo, bar } = require("foo/bar");`, + * `const { foo, bar } = ChromeUtils.import("...");`, +* use Maps, Sets, WeakMaps when possible, +* use [template strings](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals) whenever possible to avoid concatenation, allow multi-line strings, and interpolation. + + +## Comments + +Commenting code is important, but bad comments can hurt too, so it's important to have a few rules in mind when commenting: + +* If the code can be rewritten to be made more readable, then that should be preferred over writing an explanation as a comment. +* Instead of writing a comment to explain the meaning of a poorly chosen variable name, then rename that variable. +* Avoid long separator comments like `// ****************** another section below **********`. They are often a sign that you should split a file in multiple files. +* Line comments go above the code they are commenting, not at the end of the line. +* Sentences in comments start with a capital letter and end with a period. +* Watch out for typos. +* Obsolete copy/pasted code hurts, make sure you update comments inside copy/pasted code blocks. +* A global comment at the very top of a file explaining what the file is about and the major types/classes/functions it contains is a good idea for quickly browsing code. +* If you are forced to employ some kind of hack in your code, and there's no way around it, then add a comment that explains the hack and why it is needed. The reviewer is going to ask for one anyway. +* Bullet points in comments should use stars aligned with the first comment to format each point +```javascript +// headline comment +// * bullet point 1 +// * bullet point 2 +``` + +## Asynchronous code + +A lot of code in DevTools is asynchronous, because a lot of it relies on connecting to the DevTools server and getting information from there in an asynchronous fashion. + +It's easy to make mistakes with asynchronous code, so here are a few guidelines that should help: + +* Prefer [promises](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise) over callbacks. +* Use the `new Promise(() => {})` syntax. +* Don't forget to catch rejections by defining a rejection handler: `promise.then(() => console.log("resolved"), () => console.log("rejected"));` or `promise.catch(() => console.log("rejected"));`. +* Make use of [`Tasks` and generator functions](https://developer.mozilla.org/en-US/docs/Mozilla/JavaScript_code_modules/Task.jsm) to make asynchronous code look synchronous. <!--TODO async/await as in the TODO above?--> + +## React & Redux + +There are React-specific code style rules in the .eslintrc file. + +### Components + +* Default to creating components as [stateless function components](https://facebook.github.io/react/docs/reusable-components.html#stateless-functions). +* If you need local state or lifecycle methods, use `React.createClass` instead of functions. +* Use React.DOM to create native elements. Assign it to a variable named `dom`, and use it like `dom.div({}, dom.span({}))`. You may also destructure specific elements directly: `const { div, ul } = React.DOM`. + +### PropTypes + +* Use [PropTypes](https://facebook.github.io/react/docs/reusable-components.html#prop-validation) to define the expected properties of your component. Each directly accessed property (or child of a property) should have a corresponding PropType. +* Use `isRequired` for any required properties. +* Place the propTypes definition at the top of the component. If using a stateless function component, place it above the declaration of the function. +* Where the children property is used, consider [validating the children](http://www.mattzabriskie.com/blog/react-validating-children). + + diff --git a/devtools/docs/contributing/landing-code.md b/devtools/docs/contributing/landing-code.md new file mode 100644 index 0000000000..4f9f1d0fa8 --- /dev/null +++ b/devtools/docs/contributing/landing-code.md @@ -0,0 +1,20 @@ +# Landing code (i.e. getting code into Mozilla's repository) + +Code changes (patches) in Mozilla are not 'merged' in a sequential way, as it's the fashion in other popular projects. Here, the patches will be *applied* on top of the latest code, and will stay there if + +1. the patch applies cleanly, without conflicts +2. the patch doesn't cause 'bustage' (i.e. breaks the build) + +Therefore, it's good to try and do smaller changes rather than bigger, specially if you're modifying files that many other people are working on simultaneously, to avoid conflicts and your patch being rejected. Otherwise you might need to rebase from the latest changes, try to write your changes on top of it, and submit this new diff. + +Leaving potential conflicts aside, a patch can make its way into the repository in two ways: + +## From Phabricator + +Once a review has been approved, someone with enough privileges can request the code be merged, using the [Lando](https://moz-conduit.readthedocs.io/en/latest/lando-user.html) interface. These 'privileges' are "commit level access 3". You get these once you have successfully contributed with a number of patches. See [levelling up](./levelling-up.md) for more details. + +If you don't have the privileges, you can also ask your mentor to land the code. In fact, they might even initiate that for you once the code review is approved. + +To request the landing, add the *Check-in Needed* Tags to the review (*Edit Revision*). + +Someone looks at review marked with `Check-in Needed` a few times a day, verifies they have an associated **approved** review, and then attempts to land them. diff --git a/devtools/docs/contributing/levelling-up.md b/devtools/docs/contributing/levelling-up.md new file mode 100644 index 0000000000..fc0c8bab01 --- /dev/null +++ b/devtools/docs/contributing/levelling-up.md @@ -0,0 +1,23 @@ +# Levelling up + +Once you've fixed a few bugs, it's quite likely that we'll ask you to complete one or more of the following steps. They will grant you access to useful Mozilla infrastructure for testing or landing code automatically, so you can be more autonomous and able to contribute with less supervision. + +If you're curious, the [Mozilla Commit Access Policy](https://www.mozilla.org/en-US/about/governance/policies/commit/access-policy/) page explains what does each level mean. + +## Request commit access level 1 + +Once this is granted, you can use [the `Try` server](https://wiki.mozilla.org/ReleaseEngineering/TryServer) for running automated tests of your code. + +To request this, [follow the instructions here](https://www.mozilla.org/en-US/about/governance/policies/commit/) to file a bug with the title "Commit Access (Level 1) for ${your name}", and continue with steps such as adding the public SSH key, etc (it's not enough with just filing the bug!). + +The person that asked you to request this can also vouch for you. + +## Request commit access level 3 + +After you landed a few bugs, that are more advanced than the `good-first-bugs` and you feel confident about your contributions as well as the process, you can request level 3 access. Once this is granted, you will get access to Lando, the autoland feature. + +To request this, [follow the instructions here](https://www.mozilla.org/en-US/about/governance/policies/commit/) to file a bug with the title "Commit Access (Level 3) for ${your name}". + +The person that asked you to request this can also vouch for you, and at this point you might already know a few other people who will vouch for you. You need two vouchers. + +If you reach this level, well done! That's amazing, and we're glad to have you on board 😄 diff --git a/devtools/docs/contributing/making-prs.md b/devtools/docs/contributing/making-prs.md new file mode 100644 index 0000000000..fdaf827d15 --- /dev/null +++ b/devtools/docs/contributing/making-prs.md @@ -0,0 +1,82 @@ +# Sending your code for review (also known as "sending patches") + +First, commit your changes. For example: + +```bash +hg add /path/to/file/changed +hg commit -m "Bug 1234567 - [devtools] Implement feature XYZ. r=name,name2!" +``` + + The commit message explained in detail: + - `Bug 1234567` - The number of the bug in bugzilla. + - `- [devtools] Implement feature XYZ.` - The commit message, with a "devtools" prefix to quickly identify DevTools changesets. + - `r=name` - The short form to request a review. Enter the name you found using the + instructions in the [previous step](./code-reviews-find-reviewer.md). + - `,name2!` - You can have more than one reviewer. The `!` makes the review a *blocking* review (Patch can not land without accepted review). + +Then create a revision in Phabricator using `moz-phab`: + +```bash +moz-phab submit +``` + +A revision will be created including that information and the difference in code between your changes and the point in the repository where you based your work (this difference is sometimes called "a patch", as it's what you'd need to apply on the repository to get to the final state). + +If you click on the provided URL for the revision, it'll bring you to Phabricator's interface, which the reviewer will visit as well in order to review the code. They will look at your changes and decide if they need any additional work, based on whether the changes do fix what the bug describes or not. To get a better idea of the types of things they will look at and verify, read the [code reviews checklist](./code-reviews-checklist.md). + +For more information on using moz-phab, you can run: + +```bash +moz-phab -h +``` + +or to get information on a specific command (here `submit`): + +```bash +moz-phab submit -h +``` + +The reviewer might suggest you do additional changes. For example, they might recommend using a helper that already exists (but you were not aware of), or they might recommend renaming things to make things clearer. Or they might recommend you do *less* things (e.g. if you changed other things that are out of scope for the bug). Or they might simply ask questions if things aren't clear. You can also ask questions if the comments are unclear or if you're unsure about parts of the code you're interacting with. Something that looks very obvious to one person might confuse others. + +Hence, you might need to go back to the code and do some edits to address the issues and recommendations. Once you have done this, you must update the existing commit: + +```bash +hg commit --amend +``` + +And submit the change again: + +```bash +moz-phab submit +``` + +You might have to go through this cycle of submitting changes and getting it reviewed several times, depending on the complexity of the bug. + +Once your code fixes the bug, and there are no more blocking issues, the reviewer will approve the changes, and the code can be landed in the repository now. + + +# Squashing commits + +Sometimes you may be asked to squash your commits. Squashing means merging multiple commits into one in case you created multiple commits while working on a bug. Squashing bugs is easy! + +We will use the histedit extension for squashing commits in Mercurial. You can check if this extension is enabled in your Mercurial installation following these steps: + +* Open `.hgrc` (Linux/OSX) or `Mercurial.ini` (Windows) –this is the default configuration file of Mercurial– located in your home directory, using your favourite editor. +* Then add `histedit= ` under the `[extensions]` list present in file, if not present already. + +Then, run the following command: + +`hg histedit` + +You will see something like this on your terminal: + +``` +pick 3bd22d1cc59a 0 "First-Commit-Message" +pick 81c4d40e57d3 1 "Second-Commit-Message" +``` + +These lines represent your commits. Suppose we want to merge `81c4d40e57d3` to `3bd22d1cc59a`. Then replace **pick** in front of `81c4d40e57d3` with **fold** (or simply 'f'). Save the changes. + +You will see that `81c4d40e57d3` has been combined with `3bd22d1cc59a`. You can verify this using the `hg log` command. + +You can fold as many commits you want, and they will be combined with the first commit above them which does not use fold. diff --git a/devtools/docs/contributing/performance.md b/devtools/docs/contributing/performance.md new file mode 100644 index 0000000000..1b0f374260 --- /dev/null +++ b/devtools/docs/contributing/performance.md @@ -0,0 +1,152 @@ +# Writing efficient code + +When debugging a page, tools get to slow down the website because of the added instrumentation. +While working on Developer Tools we should strive to be the less impactful. +First, because it is painful to work with laggy UI, but also because some tools record timings. +For example, the network monitor records HTTP request timings. +If the tools are slowing down Firefox significantly, it will make these measurements be wrong. + +To be efficient while working on performance, you should always focus on one precise user scenario. +It could be: +* a bug report where someone reports a precise interaction being slow, +* or you could be trying to improve overall tools performance by looking at the most common usages. +The important point here is to have some steps to reproduce, that you can redo manually in order to record a profile. +And also, it is even better if you can replay via a test script. Test script that you can save as a new performance test. + +## Don't guess — profile. + +The very first thing to do is to record a profile while reproducing the scenario. + +Here's the Firefox documentation for [how to install the profiler and record a profile](https://developer.mozilla.org/docs/Mozilla/Performance/Reporting_a_Performance_Problem) and also [how to interpret the profiles](https://developer.mozilla.org/docs/Mozilla/Performance/Profiling_with_the_Built-in_Profiler#Understanding_Profiles) + +There are some peculiarities about DevTools architecture that are worth knowing about when looking at a profile: + +### Tweak profiler default settings + +The default buffer size (9MB) is too small. If you don't increase it, you may easily miss data and only see last couple of seconds of your recording. +To increase the buffer size, click on the profiler add-on icon, in the Firefox toolbar, and set it to 360MB, like this: + + <img src="performance/profiler-buffer-size.png" alt="Profiler buffer size" style="width: 300px" /> + +The other setting worth mentioning for DevTools debugging is the `Interval` +The profiler records only samples, based on this `Interval`. +If you want to see more fine-grained stack traces, you may reduce this interval to 0.1ms, +but do that only if you really need it, as it will make Firefox much even slower when recording, +and the measured times will be even slower. + +### The DevTools UI runs on the parent process + +When you are debugging tool front-ends (e.g. panels), always ensure you select the `Main Thread` line. +It should have a light blue background like this: + + <img src="performance/profiler-main-thread.png" alt="Select main process" style="width: 300px" /> + +Otherwise, the vast majority of DevTools backend (DevToolsServer, actors, ...) lives in content processes. +So if you are debugging them, you should select one of the `Content` lines. + +### Most of the DevTools codebase is in Javascript + +In the call tree, it is easier to filter by `JS`, via this menu list: + <img src="performance/profiler-filter-js.png" alt="JS Filtering" style="width: 200px" /> + +But note that you may have to switch back to `Combined` in order to understand why some particular Javascript method is slow. + +### Handy filter strings for DevTools: + + * `require` + Helps highlighting the cost of module loading +  + * DevTools uses two kind of URLs: + * `chrome://devtools/` for all panel documents. Filter with this to see the cost of all panel documents: +  + * `resource://devtools/` for all javascript modules. Filter with this to see the cost of all modules: +  + +### Record durations manually + +Sometimes it is handy to focus on a very precise piece of code and record its time manually. +For example when you identified one slow running method and think you can speed it up. +It saves your from having to: record the profile, wait for the profiler to display and search for the precise method durations. + +#### Print durations in your Terminal and in the Browser Console + +You can use the [`Performance`](https://developer.mozilla.org/docs/Web/API/Performance) API, like this: +``` +let start = window.performance.now(); + +// Run the code you want to measure + +// Once it is done, do: +console.log("my function took", window.performance.now() - start, "ms"); +``` + +#### Use markers + +The Performance API also allows recording markers, like this: +``` +window.performance.mark("my-function-start"); + +// Run the code you want to measure + +// Once it is done, do: +window.performance.measure("my-function", "my-function-start"); +``` + +This marker will appear in the `Marker Chart` section in [profiler.firefox.com](https://profiler.firefox.com), in the `UserTiming` lines: +  + +You can double click on it to make [profiler.firefox.com](https://profiler.firefox.com) display the record during this precise moment in time, +and the call tree will only display what was executed during this measurement. + +### Prototype quickly + +Sometimes the best way to find what is slow is to comment blocks of code out +and uncomment them one by one until you identify the culprit. And then focus on it. + +There are few things worse than spending a long time refactoring the piece of code that was not slow to begin with! + +## Assess your improvement. + +Once you have a patch that you think improves the performance, you have to assess whether it actually improves it. + +### Record another profile + +Compare the two profiles, without and with your patch. +Then see if the call tree reports a significant difference: +* A function call completely disappears in the new profile, with your fix. + For example you were loading a big module, and you got a frame for `require("my/big/module")` call, and no longer see it. +* The same function call takes xxx ms less with your patch. + +This [lazy loading of modules in netmonitor](https://bugzilla.mozilla.org/show_bug.cgi?id=1420289) is a good example. +Without this patch, App.js was loading in 91ms and was loading MonitorPanel.js and StatisticsPanel.js as dependencies: +  + +With the patch, App.js loads in 47ms and only loads MonitorPanel.js: +  + +It highlights that: + * we no longer load StatisticsPanel, + * App is faster to load. + +### Run performance tests + +See if any subtest reports a improvement. Ensure that the improvement makes any sense. +For example, if the test is 50% faster, maybe you broke the performance test. +This might happen if the test no longer waits for all the operations to finish executing before completing. + +To push your current patch to try, execute: +```bash +./mach try fuzzy --query "'linux64-shippable/ 'damp" --rebuild 5 +``` +It will print in your Terminal a link to perfherder like this one: +[https://treeherder.mozilla.org/perf.html#/comparechooser?newProject=try&newRevision=9bef6cb13c43bbce21d40ffaea595e082a4c28db](https://treeherder.mozilla.org/perf.html#/comparechooser?newProject=try&newRevision=9bef6cb13c43bbce21d40ffaea595e082a4c28db) +Running performance tests takes time, so you should open it 30 minutes up to 2 hours later to see your results. +See [Performance tests (DAMP)](../tests/performance-tests.md) for more information about PerfHerder/try. + +Let's look at how to interpret an actual real-life [set of perfherder results](https://treeherder.mozilla.org/perf.html#/comparesubtest?originalProject=mozilla-central&newProject=try&newRevision=9bef6cb13c43bbce21d40ffaea595e082a4c28db&originalSignature=edaec66500db21d37602c99daa61ac983f21a6ac&newSignature=edaec66500db21d37602c99daa61ac983f21a6ac&showOnlyImportant=1&framework=1&selectedTimeRange=172800): + + + +These results are related to [lazy loading of modules in netmonitor](https://bugzilla.mozilla.org/show_bug.cgi?id=1420289). +It is interesting to see that this patch is a trade-off. It makes netmonitor opening significantly faster, by preventing loading many modules during its opening. +But it makes the page reload a bit slower as some modules that used to be loaded during netmonitor open, now have to be loaded during page reload. diff --git a/devtools/docs/contributing/performance/perfherder-results.png b/devtools/docs/contributing/performance/perfherder-results.png Binary files differnew file mode 100644 index 0000000000..8f35a47c8c --- /dev/null +++ b/devtools/docs/contributing/performance/perfherder-results.png diff --git a/devtools/docs/contributing/performance/profiler-buffer-size.png b/devtools/docs/contributing/performance/profiler-buffer-size.png Binary files differnew file mode 100644 index 0000000000..626fcc5c3f --- /dev/null +++ b/devtools/docs/contributing/performance/profiler-buffer-size.png diff --git a/devtools/docs/contributing/performance/profiler-chrome-url.png b/devtools/docs/contributing/performance/profiler-chrome-url.png Binary files differnew file mode 100644 index 0000000000..fa6209f2fd --- /dev/null +++ b/devtools/docs/contributing/performance/profiler-chrome-url.png diff --git a/devtools/docs/contributing/performance/profiler-custom-markers.png b/devtools/docs/contributing/performance/profiler-custom-markers.png Binary files differnew file mode 100644 index 0000000000..be40060a8d --- /dev/null +++ b/devtools/docs/contributing/performance/profiler-custom-markers.png diff --git a/devtools/docs/contributing/performance/profiler-filter-js.png b/devtools/docs/contributing/performance/profiler-filter-js.png Binary files differnew file mode 100644 index 0000000000..e59242aea8 --- /dev/null +++ b/devtools/docs/contributing/performance/profiler-filter-js.png diff --git a/devtools/docs/contributing/performance/profiler-filter-require.png b/devtools/docs/contributing/performance/profiler-filter-require.png Binary files differnew file mode 100644 index 0000000000..44c89b0907 --- /dev/null +++ b/devtools/docs/contributing/performance/profiler-filter-require.png diff --git a/devtools/docs/contributing/performance/profiler-main-thread.png b/devtools/docs/contributing/performance/profiler-main-thread.png Binary files differnew file mode 100644 index 0000000000..f7e8f5541f --- /dev/null +++ b/devtools/docs/contributing/performance/profiler-main-thread.png diff --git a/devtools/docs/contributing/performance/profiler-netmon-open-fixed.png b/devtools/docs/contributing/performance/profiler-netmon-open-fixed.png Binary files differnew file mode 100644 index 0000000000..007a923713 --- /dev/null +++ b/devtools/docs/contributing/performance/profiler-netmon-open-fixed.png diff --git a/devtools/docs/contributing/performance/profiler-netmonitor-open.png b/devtools/docs/contributing/performance/profiler-netmonitor-open.png Binary files differnew file mode 100644 index 0000000000..31c78845f5 --- /dev/null +++ b/devtools/docs/contributing/performance/profiler-netmonitor-open.png diff --git a/devtools/docs/contributing/performance/profiler-resource-url.png b/devtools/docs/contributing/performance/profiler-resource-url.png Binary files differnew file mode 100644 index 0000000000..d4631f5317 --- /dev/null +++ b/devtools/docs/contributing/performance/profiler-resource-url.png diff --git a/devtools/docs/contributing/react-performance-tips.md b/devtools/docs/contributing/react-performance-tips.md new file mode 100644 index 0000000000..db6f920895 --- /dev/null +++ b/devtools/docs/contributing/react-performance-tips.md @@ -0,0 +1,527 @@ +# Writing efficient React code + +In this article we'll discuss about the various component types we can use, as +well as discuss some tips to make your React application faster. + +## TL;DR tips + +* Prefer props and state immutability and use `PureComponent` components as a default +* As a convention, the object reference should change **if and only if** the inner data + changes. + * Be careful to never use new instance of functions as props to a Component (it's fine to use + them as props to a DOM element). + * Be careful to not update a reference if the inner data doesn't change. +* [Always measure before optimizing](./performance.md) to have a real impact on + performance. And always measure _after_ optimizing too, to prove your change + had a real impact. + +## How React renders normal components + +### What's a normal component? +As a start let's discuss about how React renders normal plain components, that +don't use `shouldComponentUpdate`. What we call plain components here are either: +* classes that extend [`Component`](https://reactjs.org/docs/react-component.html) + ```jsx + class Application extends React.Component { + render() { + return <div>{this.props.content}</div>; + } + } + ``` +* normal functions that take some `props` as parameter and return some JSX. We + call these functions either Stateless Components or Functional Components. + This is important to understand that these Stateless Components are _not_ + especially optimized in React. + ```jsx + function Application(props) { + return <div>{props.content}</div>; + } + ``` + These functions are equivalent to classes extending `Component`. In + the rest of the article we'll especially focus on the latter. Unless otherwise + stated everything about classes extending `Component` is also true for + Stateless/Functional Components. + +#### Notes on the use of JSX +Because we don't use a build step in mozilla-central yet, some of our +tools don't use JSX and use [factories](https://reactjs.org/docs/react-api.html#createfactory) +instead: +```javascript +class Application extends React.Component { + render() { + return dom.div(null, this.props.content); + } +} +``` + +We'll use JSX in this documentation for more clarity but this is strictly +equivalent. You can read more on [React documentation](https://reactjs.org/docs/react-without-jsx.html). + +### The first render +There's only one way to start a React application and trigger a first render: +calling `ReactDOM.render`: + +```jsx +ReactDOM.render( + <Application content='Hello World!'/>, + document.getElementById('root') +); +``` + +React will call that component's `render` method, and then recursively call +every child's `render` method, generating a rendering tree and then a virtual +DOM tree. It will then render actual DOM elements to the specified container. + +### Subsequent rerenders + +There are several ways to trigger a rerender: +1. We call `ReactDOM.render` again with the same component. + ```jsx + ReactDOM.render( + <Application content='Good Bye, Cruel World!'/>, + document.getElementById('root') + ); + ``` +2. One component's state changes, through the use of [`setState`](https://reactjs.org/docs/react-component.html#setstate). + If the application is using Redux, this is how Redux-connected components + trigger updates too. +3. One component's props change. But note that this can't happen by itself, this + is always a consequence of the case 1 or 2 in one of its parents. So we'll + ignore this case for this chapter. + +When one of these happens, just like the initial render, React will call that +component's `render` method, and then recursively call every child's `render` +method, but this time possibly with changed props compared to the previous render. + +These recursive calls produce a new rendering tree. That's where React uses an +algorithm called _virtual diffing_ or +[_reconciliation_](https://reactjs.org/docs/reconciliation.html) to find the +minimal set of updates to apply to the DOM. This is good because the less +updates to the DOM the less work the browser has to do to reflow and repaint the +application. + +### Main sources of performance issues + +From this explanation we can gather that the main performance issues can +come from: +1. triggering the render process **too frequently**, +2. **expensive** render methods, +3. the reconciliation algorithm itself. The algorithm is O(n) according to React + authors, which means the processing duration increases linearly with **the number + of elements in the tree** we compare. So a larger tree means a longer time to + process. + +Let's dive more into each one of these issues. + +#### Do not render too often + +A rerender will happen after calling `setState` to change the +local state. + +Everything that's in the state should be used in `render`. +Anything in the state that's not used in `render` shouldn't be in the state, but +rather in an instance variable. This way you won't trigger an update if you +change some internal state that you don't want to reflect in the UI. + +If you call `setState` from an event handler you may call it too often. +This is usually not a problem because React is smart enough to merge close +setState calls and trigger a rerender only once per frame. Yet if your `render` +is expensive (see below as well) this could lead to problems and you may want to +use `setTimeout` or other similar techniques to throttle the renders. + +#### Keep `render` methods as lean as possible + +When rendering a list, it's very common that we'll map this list to a list of +components. This can be costly and we might want to cut this list in several +chunks of items or to +[virtualize this list](https://reactjs.org/docs/optimizing-performance.html#virtualize-long-lists). +Although this is not always possible or easy. + +Do not do heavy computations in your `render` methods. Rather do them before +setting the state, and set the state to the result of these computations. +Ideally `render` should be a direct mirror of the component's props and state. + +Note that this rule also applies to the other methods called as part of the +rendering process: `componentWillUpdate` and `componentDidUpdate`. In +`componentDidUpdate` especially avoid synchronous reflows by getting DOM +measurements, and do not call `setState` as this would trigger yet another +update. + +#### Help the reconciliation algorithm be efficient + +The smaller the tree is, the faster the algorithm is. So it's +useful to limit the changes to a subtree of the full tree. Note that the use of +`shouldComponentUpdate` or `PureComponent` alleviates this issue by cutting off +entire branches from the rendering tree, [we discuss this in more details +below](shouldcomponentupdate-and-purecomponent-avoiding-renders-altogether). + +Try to change the state as close as possible to where your UI +should change (close in the components tree). + +Do not forget to [set `key` attributes when rendering a list of +things](https://reactjs.org/docs/lists-and-keys.html), which shouldn't be the +array's indices but something that identifies the item in a predictable, unique +and stable way. This helps the algorithm +a lot by skipping parts that likely haven't changed. + +### More documentation + +The React documentation has [a very well documented page](https://reactjs.org/docs/implementation-notes.html#mounting-as-a-recursive-process) +explaining the whole render and rerender process. + +## `shouldComponentUpdate` and `PureComponent`: avoiding renders altogether + +React has an optimized algorithm to apply changes. But the fastest algorithm is +an algorithm that isn't executed at all. + +[React's own documentation about performance](https://reactjs.org/docs/optimizing-performance.html#shouldcomponentupdate-in-action) +is quite complete on this subject. + +### Avoiding rerenders with `shouldComponentUpdate` + +As the first step of a rerender process, React calls your component's +[`shouldComponentUpdate`](https://reactjs.org/docs/react-component.html#shouldcomponentupdate) +method with 2 parameters: the new props, and the new +state. If this method returns false, then React will skip the render process for this +component, **and its whole subtree**. + +```jsx +class ComplexPanel extends React.Component { + // Note: this syntax, new but supported by Babel, automatically binds the + // method with the object instance. + onClick = () => { + this.setState({ detailsOpen: true }); + } + + // Return false to avoid a render + shouldComponentUpdate(nextProps, nextState) { + // Note: this works only if `summary` and `content` are primitive data + // (eg: string, number) or immutable data + // (keep reading to know more about this) + return nextProps.summary !== this.props.summary + || nextProps.content !== this.props.content + || nextState.detailsOpen !== this.state.detailsOpen; + } + + render() { + return ( + <div> + <ComplexSummary summary={this.props.summary} onClick={this.onClick}/> + {this.state.detailsOpen + ? <ComplexContent content={this.props.content} /> + : null} + </div> + ); + } +} +``` + +__This is a very efficient way to improve your application speed__, because this +avoids everything: both calling render methods for this component _and_ the +whole subtree, and the reconciliation phase for this subtree. + +Note that just like the `render` method, `shouldComponentUpdate` is called once +per render cycle, so it needs to be very lean and return as fast as possible. So +it should execute some cheap comparisons only. + +### `PureComponent` and immutability + +A very common implementation of `shouldComponentUpdate` is provided by React's +[`PureComponent`](https://reactjs.org/docs/react-api.html#reactpurecomponent): +it will shallowly check the new props and states for reference equality. + +```jsx +class ComplexPanel extends React.PureComponent { + // Note: this syntax, new but supported by Babel, automatically binds the + // method with the object instance. + onClick = () => { + // Running this repeatidly won't render more than once. + this.setState({ detailsOpen: true }); + } + + render() { + return ( + <div> + <ComplexSummary summary={this.props.summary} onClick={this.onClick}/> + {this.state.detailsOpen + ? <ComplexContent content={this.props.content} /> + : null} + </div> + ); + } +} +``` + +This has a very important consequence: for non-primitive props and states, that is +objects and arrays that can be mutated without changing the reference itself, +PureComponent's inherited `shouldComponentUpdate` will yield wrong results and will +skip renders where it shouldn't. + +So you're left with one of these two options: +* either implement your own `shouldComponentUpdate` in a `Component` +* or (__preferred__) decide to make all your data structure immutable. + +The latter is recommended because: +* It's much simpler to think about. +* It's much faster to check for equality in `shouldComponentUpdate` and in other + places (like Redux' selectors). + +Note you could technically implement your own `shouldComponentUpdate` in a +`PureComponent` but this is quite useless because `PureComponent` is nothing +more than `Component` with a default implementation for `shouldComponentUpdate`. + +### About immutability +#### What it doesn't mean +It doesn't mean you need to enforce the immutability using a library like +[Immutable](https://github.com/facebook/immutable-js). + +#### What it means +It means that once a structure exists, you don't mutate it. + +**Every time some data changes, the object reference must change as well**. This +means a new object or a new array needs to be created. This gives the nice +reverse guarantee: if the object reference has changed, the data has changed. + +It's good to go one step further to get a **strict equivalence**: if the data +doesn't change, the object reference mustn't change. This isn't necessary for +your app to work, but this is a lot better for performance as this avoids +spurious rerenders. + +Keep reading to learn how to proceed. + +#### Keep your state objects simple + +Updating your immutable state objects can be difficult if the objects used are +complex. That's why it's a good idea to keep the objects simple, especially keep +them not nested, so that you don't need to use a library like +[immutability-helper](https://github.com/kolodny/immutability-helper), +[updeep](https://github.com/substantial/updeep), or even +[Immutable](https://github.com/facebook/immutable-js). Be especially careful +with Immutable as it's easy to create performance problems by misusing +its API. + +If you're using Redux ([see below as well](#a-few-words-about-redux)) this +advice applies to your individual reducers as well, even if Redux tools make +it easy to have a nested/combined state. + +#### How to update an object + +Updating an object is quite easy. + +You must not change/add/delete inner properties directly: + +```javascript +// Note that in the following examples we use the callback version +// of `setState` everywhere, because we build the new state from +// the current state. + +// Please don't do this as this will likely induce bugs. +this.setState(state => { + state.stateObject.details = details; + return state; +}); + +// This is wrong too: `stateObject` is still mutated. +this.setState(({ stateObject }) => { + stateObject.details = details; + return { stateObject }; +}); +``` + +Instead **you must create a new object** for this property. In this example +we'll use the object spread operator, already implemented in Firefox, Chrome and Babel. + +However here we take care to return the same object if it doesn't need an update. The +comparison happens inside the callback because it depends on the state as +well. This is a good thing to do so that the shallow equality check doesn't +return false if nothing changes. + +```javascript +// Updating one property in the state +this.setState(({ stateObject }) => ({ + stateObject: stateObject.content === newContent + ? stateObject + : { ...stateObject, content: newContent }, +}); + +// This is very similar if 2 properties need an update: +this.setState(({ stateObject1, stateObject2 }) => ({ + stateObject1: stateObject1.content === newContent + ? stateObject1 + : { ...stateObject1, content: newContent }, + stateObject2: stateObject2.details === newDetails + ? stateObject2 + : { ...stateObject2, details: newDetails }, +}); + +// Or if one of the properties needs to update 2 of it's own properties: +this.setState(({ stateObject }) => ({ + stateObject: stateObject.content === newContent && stateObject.details === newDetails + ? stateObject + : { ...stateObject, content: newContent, details: newDetails }, +}); +``` + +Note that this isn't about the returned `state` object, but its properties. +The returned object is always merged into the current state, and React creates +a new component's state object at each update cycle. + +#### How to update an array +Updating an array is easy too. + +You must avoid methods that mutate the array like push/splice/pop/shift and you +must not change directly an item. + +```javascript +// Please don't do this as this will likely induce bugs. +this.setState(({ stateArray }) => { + stateArray.push(newItem); // This is wrong + stateArray[1] = newItem; // This is wrong too + return { stateArray }; +}); +``` + +Instead here again you need to **create a new array instance**. + +```javascript +// Adding an element is easy. +this.setState(({ stateArray }) => ({ + stateArray: [...stateArray, newElement], +})); + +this.setState(({ stateArray }) => { + // Removing an element is more involved. + const newArray = stateArray.filter(element => element !== removeElement); + // or + const newArray = [...stateArray.slice(0, index), ...stateArray.slice(index + 1)]; + // or do what you want on a new clone: + const newArray = stateArray.slice(); + return { + // Because we want to keep the old array if removeElement isn't in the + // filtered array, we compare the lengths. + // We still start a render phase because we call `setState`, but thanks to + // PureComponent's shouldComponentUpdate implementation we won't actually render. + stateArray: newArray.length === stateArray.length ? stateArray : newArray, + }; + + // You can also return a falsy value to avoid the render cycle at all: + return newArray.length === stateArray.length + ? null + : { stateArray: newArray }; +}); +``` + +#### How to update Maps and Sets +The process is very similar for Maps and Sets. Here is a quick example: + +```javascript +// For a Set +this.setState(({ stateSet }) => { + if (!stateSet.has(value)) { + stateSet = new Set(stateSet); + stateSet.add(value); + } + return { stateSet }; +}); + +// For a Map +this.setState(({ stateMap }) => { + if (stateMap.get(key) !== value) { + stateMap = new Map(stateMap); + stateMap.set(key, value); + } + return { stateMap }; +})); +``` + +#### How to update primitive values + +Obviously, with primitive types like boolean, number or string, that are +comparable with the operator `===`, it's much easier: + +```javascript +this.setState({ + stateString: "new string", + stateNumber: 42, + stateBool: false, +}); +``` + +Note that we don't use the callback version of `setState` here. That's because +for primitive values we don't need to use the previous state to generate a new +state. + +#### A few words about Redux + +When working with Redux, the rules stay the same, except all of this +happens in your reducers instead of in your components. With Redux comes the +function [`combineReducers`](https://redux.js.org/docs/api/combineReducers.html) +that obeys all the rules we outlined before while making it possible to have a +nested state. + +### `shouldComponentUpdate` or `PureComponent`? + +It is highly recommended to go the full **PureComponent + immutability** route, +instead of writing custom `shouldComponentUpdate` implementations for +components. This is more generic, more maintainable, less error-prone, faster. + +Of course all rules have exceptions and you're free to implement a +`shouldComponentUpdate` method if you have specific cases to take care of. + +### Some gotchas with `PureComponent` + +Because `PureComponent` shallowly checks props and state, you need to take care +to not create a new reference for something that's otherwise identical. Some +common cases are: + +* Using a new instance for a prop at each render cycle. Especially, do not use + a bound function or an anonymous function (both classic functions or + arrow functions) as a prop: + + ```jsx + render() { + return <MyComponent onUpdate={() => this.update()} />; + } + ``` + + Each time the `render` method runs, a new function will be created, and in + `MyComponent`'s `shouldComponentUpdate` the shallow check will always fail + defeating its purpose. + +* Using another reference for the same data. One very common example is the empty + array: if you use a new `[]` for each render, you won't skip render. A solution + is to reuse a common instance. Be careful as this can very well be hidden + within some complicated Redux reducers. + +* A similar issue can arise if you use sets or maps. If you add an element in a + `Set` that's already in there, you don't need to return a new `Set` as it will be + identical. + +* Be careful with array's methods, especially `map` or `filter`, as they always + return a new array. So even with the same inputs (same input array, same + function), you'll get a new output, even if it contains the same data. If + you're using Redux, [reselect](https://github.com/reactjs/reselect) is + recommended. + [memoize-immutable](https://github.com/memoize-immutable/memoize-immutable) + can be useful in some cases too. + +## Diagnosing performance issues with some tooling + +[You can read about it in the dedicated +page](./performance.md#diagnosing-performance-issues-in-react-based-applications). + +## Breaking the rules: always measure first + +You should generally follow these rules because they bring a consistent +performance in most cases. + +However you may have specific cases that will need that you break the rules. In +that case the first thing to do is to **measure** using a profiler so that you +know where your problem are. + +Then and only then you can decide to break the rules by using some mutable state +and/or custom `shouldComponentUpdate` implementation. + +And remember to measure again after you did your changes, to check and prove +that your changes actually made an impact. Ideally you should always give links +to profiles when requesting a review for a performance patch. diff --git a/devtools/docs/files/README.md b/devtools/docs/files/README.md new file mode 100644 index 0000000000..50372e82e2 --- /dev/null +++ b/devtools/docs/files/README.md @@ -0,0 +1,13 @@ +# Directories Overview + +This page provides a very top level overview of what is on each directory in the DevTools source code: + +* `devtools/shared`: Code shared by both the client (front-end UI) and server. If we are using any third party libraries, or importing external repositories into our tree, those libraries generally live here (eg, `devtools/shared/acorn`), assuming they are used by both client and server. + * `devtools/shared/client`: Code for the [Remote Debugging Protocol](../backend/protocol.md) (RDP) client. You may wonder why this is not in `devtools/client` below: it's mainly because tests in server also need access to the RDP client. + * `devtools/shared/locales`: Strings used in either the server only, or shared with both the client and server. +* `devtools/server`: Code for the [RDP](../backend/protocol.md) server and transport layer. + * `devtools/server/actors`: [RDP Actors](../backend/protocol.md#actors). Note that if you're modifying actors, you may need to worry about [backwards compatibility](../backend/backward-compatibility.md) with older clients. +* `devtools/client`: Code for the front-end side of our tools. In theory, each directory corresponds to a panel, but this is not always the case. This directory is only shipped with desktop Firefox, as opposed to other directories above, which are shipped with all Gecko products (Firefox for Android, etc.) + * `devtools/client/locales`: Strings used in the client front-end. + * `devtools/client/themes`: CSS and images used in the client front-end. + diff --git a/devtools/docs/files/adding-files.md b/devtools/docs/files/adding-files.md new file mode 100644 index 0000000000..4795e8f8ba --- /dev/null +++ b/devtools/docs/files/adding-files.md @@ -0,0 +1,163 @@ +# Adding New Files - Various DevTools Resource Types + +This page lists the various DevTools resource types and how they can be created and loaded. + +## JavaScript Modules + +### Build Configuration + +JavaScript modules are installed by our build system using `moz.build` files. If you add a new JavaScript module, you'll need to update (or add) one of these files to make the build system aware of your new module. See the example below. + +A `moz.build` file must live in the same directory as the files to be installed. Don't list files from a subdirectory in a `moz.build` from a parent directory. + +Following these steps ensures that `require()` and `resource://` paths map directly to locations in the source tree, instead of being totally arbitrary. + +Example: + +* File: `/devtools/server/actors/layout.js` +* In `/devtools/server/actors/moz.build`: + +``` +DevToolsModules( + 'layout.js' +) +``` + +### `require()` + +Most DevTools JS code is in the form of CommonJS modules that loaded with `require()`. + +To `require()` a file, the module ID is exactly its source tree path. + +Example: + +* File: `/devtools/server/actors/layout.js` +* Usage (prefer lazy in most cases): + * `loader.lazyRequireGetter(this, "layout", "devtools/server/actors/layout")` + * `require("devtools/server/actors/layout")` + +### `ChromeUtils.import()` + +Some older DevTools JS modules use the Gecko [JavaScript code module](https://developer.mozilla.org/en-US/docs/Mozilla/JavaScript_code_modules/Using) format with the file extension `.jsm`. We are trying to move away from this format, so it's unlikely you would add a new one, but you might need to import an existing one in your code. + +These modules are loaded using `ChromeUtils.import()`. To `import()` a file, you provide a `resource://` URL, which is exactly the source tree path. + +In more detail: + +* `/devtools/client/<X>: resource://devtools/client/<X>` +* `/devtools/server/<X>: resource://devtools/server/<X>` +* `/devtools/shared/<X>: resource://devtools/shared/<X>` + +Example: + +* File: `/devtools/shared/Loader.jsm` +* Usage: + * `const { loader } = ChromeUtils.import("resource://devtools/shared/Loader.jsm")` + +Example: + +* File: `/toolkit/mozapps/extensions/AddonManager.jsm` +* Usage (prefer lazy in most cases): + * `loader.lazyImporter(this, "AddonManager", "resource://gre/modules/AddonManager.jsm")` + * `const { AddonManager } = ChromeUtils.import("resource://gre/modules/AddonManager.jsm")` + +## Chrome Content + +Much of the DevTools front-end / UI is currently loaded using `chrome://` URLs, which allow those files to have privileged access to platform internals. + +This is typically used to load XUL, HTML, and JS files in the UI. + +Note: "Chrome" here means "browser chrome", as in the UI, and bears no relation to "Chrome" as in the browser. We'd like to move away from this on DevTools and be more like regular web sites, but most tools are using `chrome://` URLs for now. + +### Packaging + +If you add a new file that should be loaded via a `chrome://` URL, you need to update a manifest file at `/devtools/client/jar.mn` so that it's packaged correctly. + +Please ensure that any new files are added so their entire source tree path is part of the URL. To do so, the `jar.mn` entry should look like: + +``` +content/<X> (<X>) +``` + +where `<X>` is the path to your file after removing the `/devtools/client/` prefix. + +Example: + +* File: `/devtools/client/webaudioeditor/models.js` +* Entry: `content/webaudioeditor/models.js (webaudioeditor/models.js)` + +### Usage + +Chrome content URLs almost match their source tree path, with one difference: the segment `client` is replaced by `content`. This is a requirement of the `chrome://` protocol handler. + +Example: + +* File: `/devtools/client/webaudioeditor/models.js` +Usage: `chrome://devtools/content/webaudioeditor/models.js` + +For files within a single tool, consider relative URLs. They're shorter! + +## Chrome Themes + +Similar to the chrome content section above, we also use chrome themes (or `skin` URLs) in the DevTools UI. These are typically used to load CSS and images. + +### Packaging + +If you add a new file that should be loaded via `chrome://` (such as a new CSS file for a tool UI), you need to update a manifest file at `/devtools/client/jar.mn` so that it's packaged correctly. + +Please ensure that any new files are added so their entire source tree path is part of the URL. To do so, the `jar.mn` entry should look like: + +``` +skin/<X> (themes/<X>) +``` + +where `<X>` is the path to your file after removing the `/devtools/client/themes/` prefix. + +Example: + +* File: `/devtools/client/themes/images/add.svg` +* Entry: `skin/images/add.svg (themes/images/add.svg)` + +### Usage + +Chrome theme URLs almost match their source tree path, with one difference: the segment `client/themes` is replaced by `skin`. This is a requirement of the `chrome://` protocol handler. + +Example: + +* File: `/devtools/client/themes/images/add.svg` +* Usage: `chrome://devtools/skin/images/add.svg` + +## Localization (l10n) + +Similar to the other chrome sections above, we also use `locale` URLs in the DevTools UI to load localized strings. This section applies to `*.dtd` (for use as entities within XUL / XHTML files) and `*.properties` (for use via runtime APIs) files. + +We currently have two sets of localized files: + +* `devtools/client/locales`: Strings used in the DevTools client (front-end UI) +* `devtools/shared/locales`: Strings used in either the DevTools server only, or shared with both the client and server + +### Packaging + +If you add a new l10n file (such as a new `*.dtd` or `*.properties` file), there should not be any additional packaging steps to perform, assuming the new file is placed in either of the 2 directories mentioned above. Each one contains a `jar.mn` which uses wildcards to package all files in the directory by default. + +### Usage + +Locale URLs differ somewhat based on whether they are in `client` or `shared`. While we would have preferred them to match the source tree path, the requirements of the `chrome://` protocol don't make that easy to do. + +Example: + +* File: `/devtools/client/locales/en-US/debugger.dtd` +* Usage: `chrome://devtools/locale/debugger.dtd` + +Example: + +* File: `/devtools/shared/locales/en-US/screenshot.properties` +* Usage: `chrome://devtools-shared/locale/screenshot.properties` + +### Guidelines + +Localization files should follow a set of guidelines aimed at making it easier for people to translate the labels in these files in many languages. + +[Find these guidelines on MDN](https://developer.mozilla.org/en-US/docs/Mozilla/Localization/Localization_content_best_practices). + +In particular, it's important to write self-explanatory comments for new keys, deleting unused keys, changing the key name when changing the meaning of a string, and more. So make sure you read through these guidelines should you have to modify a localization file in your patch. diff --git a/devtools/docs/frontend/csp.md b/devtools/docs/frontend/csp.md new file mode 100644 index 0000000000..7357a13ba2 --- /dev/null +++ b/devtools/docs/frontend/csp.md @@ -0,0 +1,56 @@ + +The DevTools toolbox is loaded in an iframe pointing to about:devtools-toolbox. This iframe has a [Content Security Policy](https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP) (CSP) applied, which will mitigate potential attacks. However this may limit the resources that can be loaded in the toolbox documenth. + +# Current DevTools CSP + +The current policy for about:devtools-toolbox is: +``` +default-src chrome: resource:; img-src chrome: resource: data:; object-src 'none' +``` + +This means: +- `chrome://` and `resource://` are allowed for any resource +- `chrome://` and `resource://` and `data://` are allowed for images + +For more information about which resources and requests are in scope of the CSP, you can read the [default-src documentation on MDN](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/default-src). + +# Scope of the DevTools CSP + +This content security policy only applies to the toolbox document for now. If you are working within the document of a panel or if you are working on the server, those limitations should not apply. + +Note that even when working in the document of a panel, we are sometimes interacting with the toolbox document, for instance to show tooltips. So typically any resource created for a tooltip will be subject to the CSP limitations. + +# Recognizing CSP issues + +Open the Browser Toolbox, if you see errors such as + +``` +JavaScript Error: "Content Security Policy: The page’s settings blocked the loading of a resource [...]" +``` + +it means you are trying to load a resource with a forbidden scheme. + +# Fixing CSP issues + +If your implementation hits a CSP issue, the first suggestion is to try to use a supported scheme. If this is not an option, check if you can perform your request or load your resource outside of the toolbox document. For instance if the resource you are loading is related to the debugged target, the request can (and probably should) be made from an actor in the DevTools server and then forwarded from the server to the client. Requests made by the server will not be impacted by the CSP. + +If it seems like the only solution is to update the CSP, get in touch with security peers in order to discuss about your use case. You can [file a bug in Core/DOM: security](https://bugzilla.mozilla.org/enter_bug.cgi?product=Core&component=DOM%3A%20Security). + +# Fixing CSP issues in tests + +If the issue comes from test code, it should be possible to update the test to use a supported scheme. A typical issue might be trying to load an iframe inside of the toolbox with a data-uri. Instead, you can create an HTML support file and load it from either a chrome:// or a resource:// URL. + +In general once a support file is added you can access it via: +- `https://example.com/browser/[path_to_file]` +- or `chrome://mochitests/content/browser/[path_to_file]` + +For instance [devtools/client/aboutdebugging/test/browser/resources/service-workers/controlled-sw.html](https://searchfox.org/mozilla-central/source/devtools/client/aboutdebugging/test/browser/resources/service-workers/controlled-sw.html) is accessed in tests via `http://example.com/browser/devtools/client/aboutdebugging/test/browser/resources/service-workers/controlled-sw.html`. + +If you absolutely have to use an unsupported scheme, you can turn off CSPs for the test only. To do so, you need to temporarily update two preferences: + +``` +await pushPref("security.csp.enable", false); +await pushPref("dom.security.skip_about_page_has_csp_assert", true); +``` + +The `pushPref` helper will ensure the preferences come back to their initial value at the end of the test. diff --git a/devtools/docs/frontend/react-guidelines.md b/devtools/docs/frontend/react-guidelines.md new file mode 100644 index 0000000000..b6fe32fb89 --- /dev/null +++ b/devtools/docs/frontend/react-guidelines.md @@ -0,0 +1,73 @@ + +# Guidelines for Writing React + +These are soft rules for writing react devtools code. Try to stick to +these for consistency, and if you disagree, file a bug to change these +docs and we can talk about it. + +**Please also read** the [coding + standards](https://wiki.mozilla.org/DevTools/CodingStandards#React_.26_Redux) +for react and redux code. The guidelines here are more general +patterns not specific to code style. + +### Why no JSX? + +You probably already noticed we don't use JSX. The answer isn't +complicated: we don't build our JS code, and we write directly for our +JS engine, SpiderMonkey. It already supports much of ES6, but it does +not support JSX (which is not a standard). + +This may change if we ever adopt a build step. Even so, the author is +not convinced that JSX helps enough to warrant all the syntax. It is +clearer sometimes, but it can be noisy switching between JSX and JS a +lot. + +It's not as bad as you may think! If you are used to JSX it may be an +adjustment, but you won't miss it too much. + +### One component per file + +Try to only put one component in a file. This helps avoid large files +full of components, but it's also technically required for how we wrap +components with factories. See the next rule. + +It also makes it easier to write tests because you might not export +some components, so tests can't access them. + +You can include small helper components in the same file if you really +want to, but note that they won't be directly tested and you will have +to use `React.createElement` or immediately wrap them in factories to +use them. + +### Export the component directly and create factory on import + +Modules are the way components interact. Ideally every component lives +in a separate file and they require whatever they need. This allows +tests to access all components and use module boundaries to wrap +components. + +For example, we don't use JSX, so we need to create factories for +components to use them as functions. A simple way to do this is on +import: + +```js +const Thing1 = React.createFactory(require('./thing1')); +const Thing2 = React.createFactory(require('./thing2')); +``` + +It adds a little noise, but then you can do `Thing1({ ... })` instead +of `React.createElement(Thing1, { ... })`. Definitely worth it. + +Additionally, make sure to export the component class directly: + +```js +const Thing1 = React.createClass({ ... }); +module.exports = Thing1; +``` + +Do not export `{ Thing1 }` or anything like that. This is required for +the factory wrapping as well as hot reloading. + +### More to Come + +This is just a start. We will add more to this document.
\ No newline at end of file diff --git a/devtools/docs/frontend/react.md b/devtools/docs/frontend/react.md new file mode 100644 index 0000000000..55882ebab2 --- /dev/null +++ b/devtools/docs/frontend/react.md @@ -0,0 +1,157 @@ + +We use [React](http://facebook.github.io/react/) to write our user +interfaces. In here you can find an explanation of why we chose React +and a short primer on it. Additionally, we list best practices that +all devtools code should adhere to when writing React. + +# Quick Intro + +This is a very quick introduction on how to *use* React, but does not +explain in-depth the concepts behind it. If you want more in-depth +articles, I recommend the following links: + +* http://facebook.github.io/react/docs/tutorial.html - the official tutorial +* https://github.com/petehunt/react-howto - how to learn React +* http://jlongster.com/Removing-User-Interface-Complexity,-or-Why-React-is-Awesome - long read but explains the concepts in depth + +React embraces components as a way of thinking about UIs. Components +are the center of everything: they are composable like functions, +testable like JSON data, and provide lifecycle APIs for more complex +scenarios. + +A component can represent anything from a single item in a list to a +complete virtualized grid that is made up of sub-components. They can +be used to abstract out "behaviors" instead of UI elements (think of a +`Selectable` component). React's API makes it easy to break up your UI +into whatever abstractions you need. + +The core idea of a component is simple: it's something that takes +properties and returns a DOM-like structure. + +```js +function Item({ name, iconURL }) { + return div({ className: "item" }, + img({ className: "icon", href: iconURL }), + name); +} +``` + +The `div` and `span` functions refer to `React.DOM.div` and +`React.DOM.span`. React provides constructors for all DOM elements on +`React.DOM`. These conform to the standard API for creating elements: +the first argument takes properties, and the rest are children. + +You can see component composition kick in when using `Item`: + +```js +const Item = React.createFactory(require('./Item')); + +function List({ items }) { + return div({ className: "list" }, + items.map(item => Item({ name: item.name, icon: item.iconURL))); +} +``` + +You can use custom components exactly the same way you use native +ones! The only difference is we wrapped it in a factory when importing +instead of using the React.DOM functions. Factories are just a way of +turning a component into a convenient function. Without factories, you +need to do do `React.createElement(Item, { ... })`, which is exactly +the same as `Item({ ... })` if using a factory. + +## Rendering and Updating Components + +Now that we have some components, how do we render them? You use +`React.render` for that: + +```js +let items = [{ name: "Dubois", iconURL: "dubois.png" }, + { name: "Ivy", iconURL: "ivy.png" }]; + +React.render(List({ items: items }), + document.querySelector("#mount")); +``` + +This renders a `List` component, given `items`, to a DOM node with an +id of `mount`. Typically you have a top-level `App` component that is +the root of everything, and you would render it like so. + +What about updating? First, let's talk about data. The above +components take data from above and render out DOM structure. If any +user events were involved, the components would call callbacks passed +as props, so events walk back up the hierarchy. The conceptual model +is data goes down, and events come up. + +You usually want to change data in response to events, and rerender +the UI with the new data. What does that look like? There are two +places where React will rerender components: + +1\. Any additional `React.render` calls. Once a component is mounted, +you can call `React.render` again to the same place and React will see +that it's already mounted and perform an update instead of a full +render. For example, this code adds an item in response to an event +and updates the UI, and will perform optimal incremental updates: + +```js +function addItem(item) { + render([...items, item]); +} + +function render(items) { + React.render(List({ items: items, + onAddItem: addItem }), + document.querySelector("#mount")); +} + +render(items); +``` + +2\. Changing component local state. This is much more common. React +allows components to have local state, and whenever the state is +changed with the `setState` API it will rerender that specific +component. If you use component local state, you need to create a +component with `createClass`: + +```js +const App = React.createClass({ + getInitialState: function() { + return { items: [] }; + }, + + handleAddItem: function(item) { + const items = [...this.props.items, item]; + this.setState({ items: items }); + }, + + render: function() { + return List({ items: this.state.items, + onAddItem: this.handleAddItem }); + } +}); + ``` + +If you are using something like Redux to manage state this is handled +automatically for you with the library you use to bind Redux with +React. See more in [Redux](redux.html). + +## DOM Diffing + +What does it mean when React "updates" a component, and how does it +know which DOM to change? React achieves this with a technique called +DOM diffing. This alleviates the need for the programmer to worry +about how updates are actually applied to the DOM, and components can +render DOM structure declaratively in response to data. In the above +examples, when adding an item, React knows to only add a new DOM node +instead of recreating the whole list each time. + +DOM diffing is possible because our components return what's called +"virtual DOM": a lightweight JSON structure that React can use to diff +against previous versions, and generate minimal changes to the real DOM. + +This also makes it really east to test components with a real DOM: +just make sure the virtual DOM has what it should. + +## Next + +Read the [React Guidelines](react-guidelines.md) next to learn how to +write React code specifically for the devtools.
\ No newline at end of file diff --git a/devtools/docs/frontend/redux-guidelines.md b/devtools/docs/frontend/redux-guidelines.md new file mode 100644 index 0000000000..1782a6de47 --- /dev/null +++ b/devtools/docs/frontend/redux-guidelines.md @@ -0,0 +1,52 @@ +### Getting data from the store + +To get data from the store, use `connect()`. + +When using connect, you'll break up your component into two parts: + +1. The part that displays the data (presentational component) + + // todos.js + const Todos = React.createClass({ + propTypes: { + todos: PropTypes.array.isRequired + } + + render: function() {...} + }) + + module.exports = Todos; + +2. The part that gets the data from the store (container component) + + // todos-container.js + const Todos = require("path/to/todos"); + + function mapStateToProps(state) { + return { + todos: state.todos + }; + } + + module.exports = connect(mapStateToProps)(Todos); + + +`connect()` generates the container component. It wraps around the presentational component that was passed in (e.g. Todos). + +The `mapStateToProps` is often called a selector. That's because it selects data from the state object. When the container component is rendering, the the selector will be called. It will pick out the data that the presentational component is going to need. Redux will take this object and pass it in to the presentational component as props. + +With this setup, a presentational component is easy to share across different apps. It doesn't have any dependencies on the app, or any hardcoded expectations about how to get data. It just gets props that are passed to it and renders them. + +For more advanced use cases, you can pass additional parameters into the selector and `connect()` functions. Read about those in the [`connect()`](https://github.com/reactjs/react-redux/blob/master/docs/api.md#connectmapstatetoprops-mapdispatchtoprops-mergeprops-options) docs. + +--- + +Need to answer the following questions: + +* How do I do I load asynchronous data? +* How do I do optimistic updates or respond to errors from async work? +* Do I use Immutable.js for my state? +* What file structure should I use? +* How do I test redux code? + +And more. diff --git a/devtools/docs/frontend/redux.md b/devtools/docs/frontend/redux.md new file mode 100644 index 0000000000..5aab53caf5 --- /dev/null +++ b/devtools/docs/frontend/redux.md @@ -0,0 +1,160 @@ + +We use [Redux](https://github.com/reactjs/redux) to manage application +state. The [docs](http://redux.js.org/) do a good job explaining the +concepts, so go read them. + +# Quick Intro + +Just like the [React introduction](react.html), this is a quick +introduction to redux, focusing on how it fits into React and why we +chose it. + +One of the core problems that React does not address is managing +state. In the React intro, we talked about data flowing down and +events flowing up. Conceptually this is nice, but you quickly run into +awkward situations in large apps. + +Let's look at an example. Say you have a page with a tabbed interface. +Here, `Tab1` is managing a list of items, so naturally it uses local +state. `Tab2` renders different stuff. + +```js +const Tab1 = React.createClass({ + getInitialState: function() { + return { items: [] }; + }, + + handleAddItem: function(item) { + this.setState({ items: [...this.state.items, item]}); + }, + + render: function() { + /* ... Renders the items and button to add new item ... */ + } +}); + +const Tab2 = React.createClass({ + render: function() { + /* ... Renders other data ... */ + } +}); + +// Assume `Tab1` and `Tab2` are wrapped with a factory when importing +const Tabs = React.createClass({ + render: function() { + return div( + { className: 'tabs' }, + // ... Render the tab buttons ... + Tab1(), + Tab2() + ); + } +}); +``` + +What happens when `Tab2` needs the list of items though? This scenario +comes up all time: components that aren't directly related need access +to the same state. A small change would be to move the `items` state +up to the `Tabs` component, and pass it down to both `Tab1` and `Tab2`. + +But now `Tabs` has to implement the `handleAddItem` method to add an +item because it's managing that state. This quickly gets ugly as the +end result is the root component ends up with a ton of state and +methods to manage it: a [god +component](https://en.wikipedia.org/wiki/God_object) is born. + +Additionally, how do we know what data each tab needs? We end up +passing *all* the state down because we don't know. This is not a +modular solution: one object managing the state and every component +receiving the entire state is like using tons of global variables. + +## Evolution of Flux + +Facebook addressed this with the +[flux](https://facebook.github.io/flux/) architecture, which takes the +state out of the components and into a "store". Redux is the latest +evolution of this idea and solves a lot of problems previous flux +libraries had (read it's documentation for more info). + +Because the state exists outside the component tree, any component can +read from it. Additionally, **state is updated with +[actions](http://redux.js.org/docs/basics/Actions.html)** that any +component can fire. We have [guidelines](redux-guidelines) for where +to read/write state, but it completely solves the problem described +above. Both `Tab1` and `Tab2` would be listening for changes in the +`item` state, and `Tab1` would fire actions to change it. + +With redux, **state is managed modularly with +[reducers](http://redux.js.org/docs/basics/Reducers.html)** but tied +together into a single object. This means a single JS object +represents most* of your state. It may sound crazy at first, but think +of it as an object with references to many pieces of state; that's all +it is. + +This makes it very easy to test, debug, and generally think about. You +can log your entire state to the console and inspect it. You can even +dump in old states and "replay" to see how the UI changed over time. + +I said "most*" because it's perfectly fine to use both component local +state and redux. Be aware that any debugging tools will not see local +state at all though. It should only be used for transient state; we'll +talk more about that in the guidelines. + +## Immutability + +Another important concept is immutability. In large apps, mutating +state makes it very hard to track what changed when. It's very easy to +run into situations where something changes out from under you, and +the UI is rendered with invalid data. + +Redux enforces the state to be updated immutably. That means you +always return new state. It doesn't mean you do a deep copy of the +state each time: when you need to change some part of the tree you +only need to create new objects to replace the ones your changing (and +walk up to the root to create a new root). Unchanged subtrees will +reference the same objects. + +This removes a whole class of errors, almost like Rust removing a +whole class of memory errors by enforcing ownership. + +## Order of Execution + +One of best things about React is that **rendering is synchronous**. That +means when you render a component, given some data, it will fully +render in the same tick. If you want the UI to change over time, you +have to change the *data* and rerender, instead of arbitrary UI +mutations. + +The reason this is desired is because if you build the UI around +promises or event emitters, updating the UI becomes very brittle +because anything can happen at any time. The state might be updated in +the middle of rendering it, maybe because you resolved a few promises +which made your rendering code run a few ticks later. + +Redux embraces the synchronous execution semantics as well. What this +means is that everything happens in a very controlled way. When +updating state through an action, all reducers are run and a new state +is synchronously generated. At that point, the new state is handed off +to React and synchronously rendered. + +Updating and rendering happen in two phases, so the UI will *always* +represent consistent state. The state can never be in the middle of +updating when rendering. + +What about asynchronous work? That's where +[middleware](http://redux.js.org/docs/advanced/Middleware.html) come +in. At this point you should probably go study our code, but +middleware allows you to dispatch special actions that indicate +asynchronous work. The middleware will catch these actions and do +something async, dispatching "raw" actions along the way (it's common +to emit a START, DONE, and ERROR action). + +**Ultimately there are 3 "phases" or level of abstraction**: the async +layer talks to the network and may dispatch actions, actions are +synchronously pumped through reducers to generate state, and state is +rendered with react. + +## Next + +Read the [Redux Guidelines](redux-guidelines.md) next to learn how to +write React code specifically for the devtools.
\ No newline at end of file diff --git a/devtools/docs/frontend/svgs.md b/devtools/docs/frontend/svgs.md new file mode 100644 index 0000000000..65603126df --- /dev/null +++ b/devtools/docs/frontend/svgs.md @@ -0,0 +1,42 @@ +# Panel SVGs +These are the guidelines for creating devtools SVGs to make sure they're as small and neatly formatted as possible. The Mozilla Developer SVG guidelines can be found [here](https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/SVG_Guidelines). + +## Explanation of Pixel Grid +Since so many of our SVGs appear so small, designing them on the pixel grid will help them not appear fuzzy when they're sized down to 16x16 pixels. There is program-specific documentation in both the [Illustrator](#illustrator) and [Sketch](#sketch) sections. + +## Panel Icon Requirements +The devtools panel icons do a couple of things in a specific way; following these guidelines will help stick your patch: + +1. **Inline fill colors.** Devtools panel icons all use ```fill="#0b0b0b"``` in the ```<svg>``` tag. +2. **Inline opacities.** Devtools panel icons also inline opacities on their relevant path. + +## Illustrator +For Illustrator you'll want the following document settings: + +- **Document settings**: ```Units: pixels```, ```Advanced``` > check ```Align New Objects to Pixel Grid``` +- **Transform Panel**: for existing artwork not on pixel grid, select shape and then within ```Transform``` > ```Advanced``` > check ```Align to Pixel Grid``` + +You can get a more detailed breakdown with images [here](http://medialoot.com/blog/3-valuable-pixel-perfect-illustrator-techniques/). + +You can download a sample Illustrator file [here](https://www.dropbox.com/home/Mozilla_MobileUX_Share/Internal%20Assets/Templates/Firefox?preview=pixel-grid-illustrator.ai). + +### Tips for Object Creation +When you're designing your icons in a graphics editor like Adobe Illustrator, there are a lot of things you can do that will bring down the size of the file and make your SVGs easier for the developers to work with. Here are some of them: + +- **Expand paths**: Instead of having multiple shapes overlapping each other, expand shapes using the pathfinder. + +- Simplify paths (```Object``` > ```Path``` > ```Simplify```) +- Expand objects so that strokes become objects. This has the added benefit of keeping the stroke size intact as the SVG is resized. + + +## Sketch +Sketch vector work is a little different but the fundamentals (keeping your SVG small, expanding all paths) is the same. Here's what we've found helps to build clean icons: + +- **Build your icon at 16x16 with the Pixel Grid turned on.** You can turn the pixel grid on at ```View > Canvas > Show Pixels``` + +- **Make sure that all x/y coordinates are full pixels for lines/rectangles.** Sub-pixels = not on pixel grid. + + +- **Expand all your paths so strokes expand properly as the SVG gets resized.** You can do this at ```Layer > Paths > Vectorize Stroke```. + +- **Align anything that isn't boxy to the pixel grid with item selected then ```Layer > Round to Nearest Pixel Edge```.**
\ No newline at end of file diff --git a/devtools/docs/frontend/telemetry.md b/devtools/docs/frontend/telemetry.md new file mode 100644 index 0000000000..b0a424e64c --- /dev/null +++ b/devtools/docs/frontend/telemetry.md @@ -0,0 +1,446 @@ +# Telemetry + +We use telemetry to get metrics of usage of the different features and panels in DevTools. This will help us take better, informed decisions when prioritising our work. + +## Adding metrics to a tool + +The process to add metrics to a tool roughly consists in: + +1. Adding the probe to Firefox +2. Using Histograms.json probes in DevTools code +3. Using Scalars.yaml probes in DevTools code +4. Using Events.yaml probes in DevTools code for analysis in Amplitude. +5. Getting approval from the data team + +### 1. Adding the probe to Firefox + +The first step involves creating entries for the probe in one of the files that contain declarations for all data that Firefox might report to Mozilla. + +These files are: + +- `toolkit/components/telemetry/Histograms.json` +- `toolkit/components/telemetry/Scalars.yaml` +- `toolkit/components/telemetry/Events.yaml` + +Scalars allow collection of simple values, like counts, booleans and strings and are to be used whenever possible instead of histograms. + +Histograms allow collection of multiple different values, but aggregate them into a number of buckets. Each bucket has a value range and a count of how many values we recorded. + +Events allow collection of a number of properties keyed to a category, method, object and value. Event telemetry helps us tell a story about how a user is interacting with the browser. + +Both scalars & histograms allow recording by keys. This allows for more flexible, two-level data collection. + +#### The different file formats + +The data team chose YAML for `Scalars.yaml` and `Events.yaml` because it is easy to write and provides a number of features not available in JSON including comments, extensible data types, relational anchors, strings without quotation marks, and mapping types preserving key order. + +While we previously used JSON for similar purposes in histograms.json, we have used YAML here because it allows for comments and is generally easier to write. + +The data team are considering moving the histograms over to YAML format at some point. + +If it's the first time you add one of these, it's advised to follow the style of existing entries. + +New data types have been added over the years, so it's quite feasible that some of our probes are not the most suitable nowadays. + +There's more information about types (and telemetry in general) on [this page](https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/start/adding-a-new-probe.html) and [this other page](https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/collection/index.html). + +And of course, in case of doubt, ask! + +### Adding probes to `Histograms.json` + +Our entries are prefixed with `DEVTOOLS_`. For example: + +```json + "DEVTOOLS_DOM_OPENED_COUNT": { + "alert_emails": ["dev-developer-tools@lists.mozilla.org"], + "expires_in_version": "never", + "kind": "count", + "bug_numbers": [1343501], + "description": "Number of times the DevTools DOM Inspector has been opened.", + "releaseChannelCollection": "opt-out" + }, + "DEVTOOLS_DOM_TIME_ACTIVE_SECONDS": { + "alert_emails": ["dev-developer-tools@lists.mozilla.org"], + "expires_in_version": "never", + "kind": "exponential", + "bug_numbers": [1343501], + "high": 10000000, + "n_buckets": 100, + "description": "How long has the DOM inspector been active (seconds)" + }, +``` + +There are different types of probes you can use. These are specified by the `kind` field. Normally we use `count` for counting how many times the tools are opened, and `exponential` for how many times a panel is active. + +### Adding probes to `Scalars.yaml` + +Our entries are prefixed with `devtools.`. For example: + +```yaml +devtools.toolbar.eyedropper: + opened: + bug_numbers: + - 1247985 + - 1352115 + description: Number of times the DevTools Eyedropper has been opened via the inspector toolbar. + expires: never + kind: uint + notification_emails: + - dev-developer-tools@lists.mozilla.org + release_channel_collection: opt-out + record_in_processes: + - 'main' + +devtools.copy.unique.css.selector: + opened: + bug_numbers: + - 1323700 + - 1352115 + description: Number of times the DevTools copy unique CSS selector has been used. + expires: "57" + kind: uint + notification_emails: + - dev-developer-tools@lists.mozilla.org + release_channel_collection: opt-out + record_in_processes: + - 'main' +``` + +### Adding probes to `Events.yaml` + +Our entries are prefixed with `devtools.`. For example: + +```yaml +devtools.main: + open: + objects: ["tools"] + bug_numbers: [1416024] + notification_emails: ["dev-developer-tools@lists.mozilla.org", "hkirschner@mozilla.com"] + record_in_processes: ["main"] + description: User opens devtools toolbox. + release_channel_collection: opt-out + expiry_version: never + extra_keys: + entrypoint: How was the toolbox opened? CommandLine, ContextMenu, HamburgerMenu, KeyShortcut, SessionRestore or SystemMenu + first_panel: The name of the first panel opened. + host: "Toolbox host (positioning): bottom, side, window or other." + splitconsole: Indicates whether the split console was open. + width: Toolbox width (px). +``` + +### 2. Using Histograms.json probes in DevTools code + +Once the probe has been declared in the `Histograms.json` file, you'll need to actually use it in our code. + +First, you need to give it an id in `devtools/client/shared/telemetry.js`. Similarly to the `Histograms.json` case, you'll want to follow the style of existing entries. For example: + +```js +dom: { + histogram: "DEVTOOLS_DOM_OPENED_COUNT", + timerHistogram: "DEVTOOLS_DOM_TIME_ACTIVE_SECONDS" +}, +``` + +... would correspond to the probes we declared in the previous section. + +Then, include that module on each tool that requires telemetry: + +```js +let Telemetry = require("devtools/client/shared/telemetry"); +``` + +Create a telemetry instance on the tool constructor: + +```js +this._telemetry = new Telemetry(); +``` + +And use the instance to report e.g. tool opening... + +```js +this._telemetry.toolOpened("mytoolname", sessionId, this); +``` + +... or closing: + +```js +this._telemetry.toolClosed("mytoolname", sessionId, this); +``` + +Note that `mytoolname` is the id we declared in the `telemetry.js` module. + +### 3. Using Scalars.yaml probes in DevTools code + +Once the probe has been declared in the `Scalars.yaml` file, you'll need to actually use it in our code. + +First, you need to give it an id in `devtools/client/shared/telemetry.js`. You will want to follow the style of existing lowercase histogram entries. For example: + +```js +toolbareyedropper: { + scalar: "devtools.toolbar.eyedropper.opened", // Note that the scalar is lowercase +}, +copyuniquecssselector: { + scalar: "devtools.copy.unique.css.selector.opened", +}, +``` + +... would correspond to the probes we declared in the previous section. + +Then, include that module on each tool that requires telemetry: + +```js +let Telemetry = require("devtools/client/shared/telemetry"); +``` + +Create a telemetry instance on the tool constructor: + +```js +this._telemetry = new Telemetry(); +``` + +And use the instance to report e.g. tool opening... + +```js +this._telemetry.toolOpened("mytoolname", sessionId, this); +``` + +Notes: + +- `mytoolname` is the id we declared in the `Scalars.yaml` module. +- Because we are not logging tool's time opened in `Scalars.yaml` we don't care + about toolClosed. Of course, if there was an accompanying `timerHistogram` + field defined in `telemetry.js` and `histograms.json` then `toolClosed` should + also be added. + +### 4. Using Events.yaml probes in DevTools code + +Once the probe has been declared in the `Events.yaml` file, you'll need to actually use it in our code. + +It is crucial to understand that event telemetry have a string identifier which is constructed from the `category`, `method`, `object` (name) and `value` on which the event occurred. This key points to an "extra" object that contains further information about the event (we will give examples later in this section). + +Because these "extra" objects can be from completely independent code paths we +can send events and leave them in a pending state until all of the expected extra properties have been received. + +First, include the telemetry module in each tool that requires telemetry: + +```js +let Telemetry = require("devtools/client/shared/telemetry"); +``` + +Create a telemetry instance on the tool constructor: + +```js +this._telemetry = new Telemetry(); +``` + +And use the instance to report e.g. tool opening... + +```js +// Event telemetry is disabled by default so enable it for your category. +this._telemetry.setEventRecordingEnabled(true); + +// If you already have all the properties for the event you can send the +// telemetry event using: +// this._telemetry.recordEvent(method, object, value, extra) e.g. +this._telemetry.recordEvent("open", "tools", null, { + "entrypoint": "ContextMenu", + "first_panel": "Inspector", + "host": "bottom", + "splitconsole": false, + "width": 1024, + "session_id": this.toolbox.sessionId +}); + +// If your "extra" properties are in different code paths you will need to +// create a "pending event." These events contain a list of expected properties +// that can be populated before or after creating the pending event. + +// Use the category, method, object, value combinations above to add a +// property... we do this before creating the pending event simply to +// demonstrate that properties can be sent before the pending event is created. +this._telemetry.addEventProperty( + this, "open", "tools", null, "entrypoint", "ContextMenu"); + +// In this example `"open", "tools", null` make up the +// signature of the event and needs to be sent with all properties. + +// Create the pending event using +// this._telemetry.preparePendingEvent(this, method, object, value, +// expectedPropertyNames) e.g. +this._telemetry.preparePendingEvent(this, "open", "tools", null, + ["entrypoint", "first_panel", "host", "splitconsole", "width", "session_id"] +); + +// Use the category, method, object, value combinations above to add each +// property. +this._telemetry.addEventProperty( + this, "open", "tools", null, "first_panel", "inspector"); +this._telemetry.addEventProperty( + this, "open", "tools", null, "host", "bottom"); +this._telemetry.addEventProperty( + this, "open", "tools", null, "splitconsole", false); +this._telemetry.addEventProperty( + this, "open", "tools", null, "width", 1024); + +// You can also add properties in batches using e.g.: +this._telemetry.addEventProperties(this, "open", "tools", null, { + "first_panel": "inspector", + "host": "bottom", + "splitconsole": false, + "width": 1024 +}); + +``` + +Notes: + +- `mytoolname` is the id we declared in the `Scalars.yaml` module. +- Because we are not logging tool's time opened in `Scalars.yaml` we don't care + about toolClosed. Of course, if there was an accompanying `timerHistogram` + field defined in `telemetry.js` and `histograms.json` then `toolClosed` should + also be added. + +#### Note on top level panels + +The code for the tabs uses their ids to automatically report telemetry when you switch between panels, so you don't need to explicitly call `toolOpened` and `toolClosed` on top level panels. + +You will still need to call those functions on subpanels, or tools such as about:debugging which are not opened as tabs. + +#### Testing + +The telemetry module will print warnings to stdout if there are missing ids. It is strongly advisable to ensure this is working correctly, as the module will attribute usage for undeclared ids to a generic `custom` bucket. This is not good for accurate results! + +To see these warnings, you need to have the `browser.dom.window.dump.enabled` browser preference set to `true` in `about:config` (and restart the browser). + +Then, try doing things that trigger telemetry calls (e.g. opening a tool). Imagine we had a typo when reporting the tool was opened: + +```js +this._telemetry.toolOpened('mytoolnmae', sessionId, this); + ^^^^ typo, should be *mytoolname* +``` + +Would report an error to stdout: + +```text +Warning: An attempt was made to write to the mytoolnmae histogram, which is not defined in Histograms.json +``` + +So watch out for errors. + +#### Testing Event Telemetry + +This is best shown via an example: + +```js +/* Any copyright is dedicated to the Public Domain. + * http://creativecommons.org/publicdomain/zero/1.0/ */ + +"use strict"; + +const { Toolbox } = require("devtools/client/framework/toolbox"); +const { TelemetryTestUtils } = ChromeUtils.import("resource://testing-common/TelemetryTestUtils.jsm"); + +const URL = "data:text/html;charset=utf8,browser_toolbox_telemetry_close.js"; +const { RIGHT, BOTTOM } = Toolbox.HostType; +const DATA = [ + { + category: "devtools.main", + method: "close", + object: "tools", + value: null, + extra: { + host: "right", + width: w => w > 0, + } + }, + { + category: "devtools.main", + method: "close", + object: "tools", + value: null, + extra: { + host: "bottom", + width: w => w > 0, + } + } +]; + +add_task(async function() { + // Let's reset the counts. + Services.telemetry.clearEvents(); + + // Ensure no events have been logged + TelemetryTestUtils.assertNumberOfEvents(0); + + await openAndCloseToolbox("webconsole", SIDE); + await openAndCloseToolbox("webconsole", BOTTOM); + + checkResults(); +}); + +async function openAndCloseToolbox(toolId, host) { + const tab = await addTab(URL); + const target = await TargetFactory.forTab(tab); + const toolbox = await gDevTools.showToolbox(target, toolId); + + await toolbox.switchHost(host); + await toolbox.destroy(); +} + +function checkResults() { + TelemetryTestUtils.assertEvents(DATA, {category: "devtools.main", method: "close", object: "tools"}); +} +``` + +#### Compile it + +You need to do a full Firefox build if you have edited either `Histograms.json` or `Events.yaml`, as they are processed at build time, and various checks will be run on them to guarantee they are valid. + +```bash +./mach build +``` + +If you use `mach build faster` or artifact builds, the checks will not be performed, and your try builds will fail ("bust") when the checks are run there. + +Save yourself some time and run the checks locally. + +NOTE: Changes to `Scalars.yaml` *are* processed when doing an artifact build. + +### 4. Getting approval from the data team + +This is required before the changes make their way into `mozilla-central`. + +To get approval, attach your patch to the bug in Bugzilla, and set two flags: + +- a `review?` flag for a data steward. +- a `needinfo?` flag to hkirschner (our product manager, so he vouches that we're using the data) + +Be sure to explain very clearly what is the new probe for. E.g. "We're seeking approval for tracking opens of a new panel for debugging Web API ABCD" is much better than just asking for feedback without background info. + +This review shouldn't take too long: if there's something wrong, they should tell you what to fix. If you see no signs of activity after a few days, you can ask in `#developers`. + +Note that this review is *in addition* to normal colleague reviews. + +Click [here](https://wiki.mozilla.org/Firefox/Data_Collection#Requesting_Data_Collection) for more details. + +## Accessing existing data + +### Local data + +Go to [about:telemetry](about:telemetry) to see stats relating to your local instance. + +### Global data + +Data aggregated from large groups of Firefox users is available at [telemetry.mozilla.org](https://telemetry.mozilla.org). + +Reports are written with SQL. For example, here's one comparing [usage of some DevTools panels](https://sql.telemetry.mozilla.org/queries/1000#table). + +If you want to get better understanding of how people are using the tools, you are encouraged to explore this data set by writing your own reports. + +The easiest way to get started is to *fork* an existing report and modify it to get used to the syntax, as SQL for massive data tables is very different from SQL for a humble blog engine, and you'll find some new operators that might look unfamiliar. + +It's also recommended to take small steps and run the queries often to detect errors before they're too complicated to solve, particularly if you're not experienced with this (yet). + +Slow queries will be interrupted by the system, so don't worry about "fetching too much data" or "using too many resources". There's built-in protection to avoid your code eating up the Telemetry database. + +Funnily, if you're based in Europe, you might be in luck, as the website tends to be more responsive during European working hours than it is at Pacific working hours, as seemingly there's less people in Europe interacting with it. diff --git a/devtools/docs/getting-started/README.md b/devtools/docs/getting-started/README.md new file mode 100644 index 0000000000..9a43b4d7b3 --- /dev/null +++ b/devtools/docs/getting-started/README.md @@ -0,0 +1,29 @@ +# Getting started + +Hello, and thanks for your interest in contributing to Firefox DevTools! + +DevTools is a complex web app, but if you're familiar with either HTML/CSS or JavaScript, you can contribute! The process goes like this: + +- Claim a bug +- Set up your dev environment +- Write the patch and get it reviewed + +Feel free to ask questions at any point on [Matrix](https://chat.mozilla.org/#/room/#devtools:mozilla.org). + +## Claim a bug + +Visit the [Codetribute](https://codetribute.mozilla.org/projects/devtools) bug tracker and find a bug you like. Anything labeled **good-first-bug** is perfect for a newcomer. Many of these tasks will make a visible impact to the DevTools UI. + +Claim the bug by creating a Bugzilla account and posting a comment on the bug’s page to say that you’d like to work on it. Ask questions if you have any uncertainty about what the bug means. + +## Set up your dev environment + +Follow the steps of Firefox’s [contributor guide](https://firefox-source-docs.mozilla.org/contributing/how_to_contribute_firefox.html) to install and run Firefox locally. During installation, follow the steps for “Artifact Mode.” + +If you run into errors about missing libraries, search the web to learn how to install whatever is missing. If you get stuck, ask for help on [Matrix](https://chat.mozilla.org/#/room/#devtools:mozilla.org). + +## Write the patch and get it reviewed + +The rest of this documentation has info on coding standards as well as specifics of DevTools architecture. + +When you’re ready to commit your changes, you can ask the bug’s mentor to review, or refer to the list of [DevTools team members](https://firefox-dev.tools/#about-devtools). diff --git a/devtools/docs/getting-started/architecture-overview.md b/devtools/docs/getting-started/architecture-overview.md new file mode 100644 index 0000000000..9c35a13252 --- /dev/null +++ b/devtools/docs/getting-started/architecture-overview.md @@ -0,0 +1,13 @@ +# Architecture overview + +Broadly speaking, the tools are divided in two parts: the server and the client. A **server** is anything that can be debugged: for example, your browser, but it could also be Firefox for Android, running on another device. The **client** is the front-end side of the tools, and it is what developers interact with when using the tools. + +Since these two parts are decoupled, we can connect to any server using the same client. This enables us to debug multiple types of servers, using the same protocol to communicate. + +You will often hear about `actors`. Each feature that can be debugged (for example, network) is exposed via an `actor`, which provides data about that specific feature. It's up to each server to implement some or all actors; the client needs to find out and decide what it can render on the front-side when it connects to the server. So when we want to debug a new feature, we might need to do work in two parts of the code: the server (perhaps implementing a new actor, or extending existing ones) and the client (to display the debugging data returned by the actor). + +Often, an actor will correspond to a panel. But a panel might want to get data from multiple actors. + +You might also hear about `the toolbox`. The toolbox is what everyone else calls `developer tools` i.e. the front-end that you see when you open the tools in your browser. + + diff --git a/devtools/docs/getting-started/bugzilla.md b/devtools/docs/getting-started/bugzilla.md new file mode 100644 index 0000000000..fe68491477 --- /dev/null +++ b/devtools/docs/getting-started/bugzilla.md @@ -0,0 +1,9 @@ +# Get a Bugzilla account + +Mozilla's bug tracker is at [https://bugzilla.mozilla.org/](https://bugzilla.mozilla.org/), which is often abbreviated as `BMO`. + +You don't need an account if you simply want to build the code and modify it, but you will need an account in Bugzilla if you want to file or comment on bugs, send patches, get assigned to bugs (so you can 'claim' them), etc. + +**Note**: if you are a Mozilla employee, don’t use an email alias to sign up, use your full LDAP account. + +To make yourself easier to find by other colleagues (for example when they're trying to set a reviewer for a patch), you can [edit the *real name* field](https://bugzilla.mozilla.org/userprefs.cgi?tab=account) to add your alias or any other word they might use to search for you there. The convention is to use something like `Your Name :alias :ldap/:ircnick`. For example: `Mary Smith :mary :msmith` diff --git a/devtools/docs/getting-started/build.md b/devtools/docs/getting-started/build.md new file mode 100644 index 0000000000..eb93367fca --- /dev/null +++ b/devtools/docs/getting-started/build.md @@ -0,0 +1,159 @@ +# Set up to build Firefox Developer Tools + +Since the tools are part of Firefox, we need to get and build Firefox's source code. + +These are the steps we're going to look at: + +* [Installing Mercurial](#installing-mercurial) +* [Getting the code](#getting-the-code) +* [Building and running locally](#building-and-running-locally) + * [Rebuilding](#rebuilding) + * [Artifact builds](#building-even-faster-with-artifact-builds) for even faster builds + * [Maybe you don't even need to build](#maybe-you-dont-even-need-to-build) + +## Installing Mercurial + +Firefox's source code is hosted on [a Mercurial repository](https://hg.mozilla.org/mozilla-central/). + +Please [install Mercurial](https://www.mercurial-scm.org/) if it's not already installed in your system. Its website provides [files for downloading](https://www.mercurial-scm.org/downloads) and also instructions for the most popular package managers. For example, if using Homebrew in macOS: + +```bash +brew install mercurial +``` + +Or in Debian/Ubuntu Linux with apt-get: + +```bash +apt-get install mercurial +``` + +## Getting the code + +This will take a long time, because the repository is **B I G** (Firefox is more than 10 years old and there are lots of commits every day!). So please be patient. + +You will also need a lot of disk space (about ~40Gb free). + +```bash +cd ~ # or the folder where you store your projects, for example ~/projects +hg clone https://hg.mozilla.org/mozilla-central +``` + +## Building and running locally + +Although DevTools is written in JavaScript, the core of Firefox is not—we need tools such as compilers to build the C/C++/Rust code and turn it into binaries that our computer can run. + +If you're a web developer, you might not be familiar with these tools, but fortunately, the Firefox team has made a very good job of automating the process of installing them with bootstrap scripts, and putting [documentation](https://developer.mozilla.org/docs/Mozilla/Developer_guide/Build_Instructions/Simple_Firefox_build) together. You don't *need* to read it now; as we'll provide a simplified way to build the code, but if you have any issues, please refer to the documentation. + +Try building Firefox and downloading dependencies by running the following commands: + +```bash +./mach bootstrap +``` + +This script might ask you questions, so it's recommended to keep an eye on the terminal while it's running—otherwise it will never finish! + +*Please note, when you are prompted for your name and add unicode characters this can crash the process. The workaround here is to use ascii-friendly characters and later on edit your `~/.hgrc` file manually to use the unicode characters in your name.* + +After it finishes, you might be asked to add some lines to a `mozconfig` file. Create this file in the repository folder (i.e. in `~/mozilla-central`), then add the lines. + +Then run this: + +```bash +./mach configure +./mach build +``` + +Please note, if this fails it might be possible you need to run the `bootstrap.py` script first. Download the [bootstrap.py script](https://hg.mozilla.org/mozilla-central/raw-file/default/python/mozboot/bin/bootstrap.py) and save it in your project directory. Then run `python bootstrap.py` and follow the prompted steps. + +**Note:** if using Windows, you might need to type the commands without the `./`: + +```bash +mach bootstrap +mach configure +mach build +``` + +If your system needs additional dependencies installed (for example, Python, or a compiler, etc) the above commands might fail, and various diagnostic messages will be printed to your screen. Follow their advice and then try running the command that failed again, until the three of them complete successfully. + +Some error messages can be quite cryptic. It is a good idea to consult the [documentation](https://developer.mozilla.org/docs/Mozilla/Developer_guide/Build_Instructions/Simple_Firefox_build) specific to the platform you're using. Sometimes searching in the internet for the specific error message you get can help, and you can also [get in touch](https://firefox-dev.tools/#getting-in-touch) if you get stuck. + +Once you complete a first successful build, you should be able to build again by running only this command: + +```bash +./mach build +``` + +By the way, building takes a long time (specially on slow computers). + +### Running your own compiled version of Firefox + +To run the Firefox you just compiled: + +```bash +./mach run +``` + +This will run using an empty temporary profile which is discarded when you close the browser. We will look into [persistent development profiles later](./development-profiles.md). But first... + +⭐️ **Time for some congratulations!** You managed to get Firefox's code, build tools and dependencies, and just run your very own copy of Firefox! Well done! ⭐ ️ + +### Rebuilding + +<!--TODO: it would be valuable to explain how to pull changes! --> + +Suppose you pulled the latest changes from the remote repository (or made some changes, to experiment and see what happens) and want to build again. + +You can ask the `mach` script to build only changed files: + +```bash +./mach build faster +``` + +This should be faster (a matter of seconds). + +Sometimes, if you haven't updated in a while, you'll be told that you need to *clobber*, or basically delete precompiled stuff and start from scratch, because there are too many changes. The way to do it is: + +```bash +./mach clobber +``` + +It is a bit tedious to do this manually, but fortunately you can add an entry to `mozconfig` to have this done automatically for you each time it's required. Add this and save the file: + +``` +# Automatically clobber when an incremental build is not possible +mk_add_options AUTOCLOBBER=1 +``` + +### Building even faster with artifact builds + +It is unusual to modify C/C++ code when working on DevTools. This means that we can use *artifact builds*. This method downloads prebuilt binary components, and then the build process becomes faster. + +Add the following content to `mozconfig`: + +```bash +# Automatically download and use compiled C++ components: +ac_add_options --enable-artifact-builds + +# Write build artifacts to: +mk_add_options MOZ_OBJDIR=./objdir-frontend +``` + +And then you can follow the normal build process again (only *faster*!) + +**Note**: On macOS you might want to use `MOZ_OBJDIR=./objdir-frontend.noindex` instead. Using the `.noindex` file extension prevents the Spotlight from indexing your `objdir`, which is slow. + +For more information on aspects such as technical limitations of artifact builds, read the [Artifact Builds](https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/Artifact_builds) page. + +### Updating worker code requires a few extra steps + +#### Prerequisite + +1. Install yarn 1.x globally via `npm install -g yarn` if you do not have it installed globally already. +2. Go to the `client/debugger/` directory and run `npm install`. + +#### Regenerating work bundle + +1. Save the modified files. +2. Back in your terminal, under `client/debugger/` directory, execute `node bin/build`. +3. After completion, use the `Restart (Developer)` option that is available under the `File` menu in local builds (or its respective shortcut). + diff --git a/devtools/docs/getting-started/development-profiles.md b/devtools/docs/getting-started/development-profiles.md new file mode 100644 index 0000000000..41990eda94 --- /dev/null +++ b/devtools/docs/getting-started/development-profiles.md @@ -0,0 +1,90 @@ +# Setting up a development profile + +You can have various [Firefox profiles](https://developer.mozilla.org/en-US/Firefox/Multiple_profiles) (think of something like "user accounts"), each one with different settings, addons, appearance, etc. + +This page will guide you through configuring a new profile to enable development features such as additional logging, dumping of network packets, remote debugging, etc. which will help when working in DevTools. + +Many of these changes are achieved by modifying preferences in `about:config`, a special page you can access by typing in `about:config` in Firefox's URL bar. The first time, it will show you a warning page. Click through or disable the warning for the future, and then you can start searching for preferences to modify. + +(If you're curious, here's more information about [about:config](https://support.mozilla.org/en-US/kb/about-config-editor-firefox)) + +## Create a permanent profile + +We were using a temporary profile in the previous step, [building DevTools](./build.md). The contents of this profile are deleted each time the browser is closed, which means any preferences we set will not persist. + +The solution is to create a permanent profile: + +``` +./mach run -P development +``` + +If this profile doesn't exist yet (quite likely), a window will open offering you options to create a new profile, and asking you for the name you want to use. + +Create a new profile, and name it `development`. Then start Firefox by clicking on `Start Nightly`. + +Next time you start Firefox with `./mach run -P development`, the new profile will be automatically used, and settings will persist between browser launches. + +It's now time to [start contributing](../contributing.html)! 😃 + +--- + +## Advanced settings + +The following section describes how to enable additional development features; don't worry if you don't understand what some of these are or what they're for. Feel free to skip these if you're new; you probably don't need them yet. + +### Enable additional logging + +You can change the value of these preferences by going to `about:config`: + +| Preference name | Value | Comments | +| --------------- | --------------- | -------- | +| `browser.dom.window.dump.enabled` | `true` | Adds global `dump` function to log strings to `stdout` | +| `devtools.console.stdout.chrome` | `true` | Allows console API to write to `stdout` when used by chrome content | +| `devtools.console.stdout.content` | `true` | Allows console API to write to `stdout` when used by content | +| `devtools.debugger.log` (*) | `true` | Dump packets sent over remote debugging protocol to `stdout`.<!-- TODO: I think this is outdated and there isn't a compatible addon anymore <br /><br />The [remote protocol inspector add-on](https://github.com/firebug/rdp-inspector/wiki) might be useful too.--> | +| `devtools.dump.emit` (*) | `true` | Log event notifications from the EventEmitter class<br />(found at `devtools/shared/event-emitter.js`). | + +Preferences marked with a (`*`) also require `browser.dom.window.dump.enabled` in order to work. You might not want to enable *all* of those all the time, as they can cause the output to be way too verbose, but they might be useful if you're working on a server actor, for example<!--TODO link to actors doc-->. + +Restart the browser to apply configuration changes. + +### Enable remote debugging and the Browser Toolbox + +<!--TODO: aren't some of these preferences enabled by default now in local builds? --> + +These settings allow you to use the [browser toolbox](https://developer.mozilla.org/docs/Tools/Browser_Toolbox) to inspect the DevTools themselves, set breakpoints inside of DevTools code in the *Browser* environment. + +Open DevTools, and click the "Toolbox Options" gear icon in the top right (the image underneath is outdated). <!--TODO update image--> + +Make sure the following two options are checked: + +- Enable browser chrome and add-on debugging toolboxes +- Enable remote debugging + + + +In `about:config`, set `devtools.debugger.prompt-connection` to `false`. + +This will get rid of the prompt displayed every time you open the browser toolbox. + +### Enable DevTools assertions + +When assertions are enabled, assertion failures are fatal, log console warnings, and throw errors. + +When assertions are not enabled, the `assert` function is a no-op. + +It also enables the "debug" builds of certain third party libraries, such as React. + +To enable assertions, add this to your `mozconfig` file: + +``` +ac_add_options --enable-debug-js-modules +``` + +And assert your own invariants like this: + +``` +const { assert } = require("devtools/shared/DevToolsUtils"); +// ... +assert(1 + 1 === 2, "I really hope this is true..."); +``` diff --git a/devtools/docs/getting-started/restart.png b/devtools/docs/getting-started/restart.png Binary files differnew file mode 100644 index 0000000000..a4611e116a --- /dev/null +++ b/devtools/docs/getting-started/restart.png diff --git a/devtools/docs/getting-started/setting-up-mozilla-central-for-macOS.md b/devtools/docs/getting-started/setting-up-mozilla-central-for-macOS.md new file mode 100644 index 0000000000..4159bedea6 --- /dev/null +++ b/devtools/docs/getting-started/setting-up-mozilla-central-for-macOS.md @@ -0,0 +1,191 @@ +## Table of contents + +To set up `mozilla-central`, follow parts 1-6. Part 7 is for debugging common errors that may pop up during the process. + +1) Before you start +2) Bootstrapping your modules +3) Cloning mozilla-central +4) Bootstrapping mozilla-central +5) Setting up arcanist and moz-phab +6) Making your first practice patch +7) Common errors + +## Before you start + +1) Install [Xcode](https://developer.apple.com/xcode/). +2) Install [Homebrew](https://brew.sh/). + +## Bootstrapping your modules + +1) Create a "src" directory for yourself under your home directory: +``` +mkdir src && cd src +``` + +2) Download the [bootstrap.py](https://hg.mozilla.org/mozilla-central/raw-file/default/python/mozboot/bin/bootstrap.py) script and save it in your `src/` directory. + +3) Run the bootstrap script with: +``` +python bootstrap.py +``` + +4) Follow the prompts, which should be mostly Yes (`Y`). One of the prompts will ask you if you want to clone mozilla-unified. Leave the destination field empty and hit Enter. + +## Cloning mozilla-central + +In your `src` folder run the following: +``` +hg clone https://hg.mozilla.org/mozilla-central/ mozilla-central +``` + +Your directory should now look something like this: +``` +src + |--- mozilla-central + |--- bootstrap.py +``` + +Go ahead and open your cloned `mozilla-central` folder: +``` +cd mozilla-central +``` + +## Bootstrapping mozilla-central + +1. Run the command below to bootstrap your mozilla-central: +``` +./mach bootstrap +``` + +2. At the end of the bootstrapping process, it will ask you to create a `mozconfig` file. Create a file named `mozconfig` in the `mozilla-central` folder, and paste the following. Afterwards, save that file: +``` +ac_add_options --enable-artifact-builds +``` + +3. Run the command below to build mozilla-central +``` +./mach build +``` + +4. If your build is successful, it should tell you that you can now run `./mach run`. Go ahead and do that to run Firefox! +``` +./mach run +``` + +## Setting up arcanist and moz-phab + +Follow these instructions here: https://moz-conduit.readthedocs.io/en/latest/arcanist-macos.html. + +If at the end of the instructions you try to run `moz-phab -h` and it can't find the command, edit your `.bash_profile`: +``` +sudo nano ~/.bash_profile +``` + +Enter your password, and then paste the string below into that file: +``` +export PATH="$HOME/.mozbuild/arcanist/bin:$HOME/.mozbuild/moz-phab:$PATH" +``` + +Restart your terminal, and `moz-phab -h` should now work as expected. + +## Making your first practice patch + +To verify that everything works correctly, try submitting a practice patch. These steps are primarily based off of https://moz-conduit.readthedocs.io/en/latest/arcanist-user.html. + +Before proceeding, make sure to create a [Phabricator account](https://phabricator.services.mozilla.com). + +1) Go into your mozilla-central folder and setup arc: +``` +cd mozilla-central +arc install-certificate +``` + +2) Using your editor, create a test file named `testfile`. + +3) Check to see if your `testfile` is detected and identified as an untracked file: +``` +hg status +# `? testfile` should be logged in PINK +``` + +4) Track your `testfile`: +``` +hg add testfile +``` + +5) Check that the `testfile` is now identified as a tracked file: +``` +hg status +# `A testfile` should be logged in GREEN +``` + +6) Commit the tracked changes: +``` +hg commit -m "Bug N/A - first test patch, please ignore r=davidwalsh" +``` + +7) Verify the commit by checking your commit logs: +``` +hg log +# Your "Added test file" should appear at the top of the log +``` + +8) Submit your patch: +``` +arc diff +# this should open a text file with VIM, just fill it similarly to the one below +``` + +``` +Bug N/A - first test patch, please ignore r=davidwalsh + +Summary: Test patch to see if I set MC correctly. Referring to a benign bug that's CLOSED WONTFIX + +Test Plan: N/A + +Reviewers: davidwalsh + +Subscribers: + +Bug #: 1395261 +``` + +### Finished submitting your patch? +Double check in Phabricator that it did go through: https://phabricator.services.mozilla.com + +## Common Errors + +### Checking for rustc... not found +``` +checking for rustc... not found +checking for cargo... not found +``` + +Install rust and cargo with `curl https://sh.rustup.rs -sSf | sh` and then restart your terminal. Afterwards, install cbindgen with `cargo install cbindgen`. + +### Baldrdash +``` +`Common error: failed to run custom build command for `baldrdash v0.1.0 (directory here)` +``` +First, delete any existing `gecko-dev` folder that you might have cloned. Afterwards, delete your `mozilla-central` folder and repeat the **Cloning Mozilla Central** steps above. + +If you still run into the same error, reinstall llvm with `brew uninstall llvm && brew install llvm`. Again, delete your `mozilla-central` folder and repeat the **Cloning Mozilla Central** steps above. + +### Error running mach +``` + 0:00.73 /usr/bin/make -f client.mk -s + 0:16.13 Error running mach: + 0:16.13 ['--log-no-times', 'artifact', 'install'] +``` +Delete your `mozilla-central` folder and repeat the **Cloning Mozilla Central** steps above. + +### Packaging specialpowers@mozilla.org.xpi... +``` +1:28.71 Packaging specialpowers@mozilla.org.xpi... + 1:29.14 Traceback (most recent call last): + 1:29.14 File "/Users/jarilvalenciano/Desktop/src/mozilla-central/config/nsinstall.py", line 188, in <module> + 1:29.14 sys.exit(_nsinstall_internal(argv[1:])) + 1:29.14 File "/Users/jarilvalenciano/Desktop/src/mozilla-central/config/nsinstall.py", line 149, in _nsinstall_internal + 1:29.14 copy_all_entries(args, target) +``` +Just run `./mach build` again.
\ No newline at end of file diff --git a/devtools/docs/getting-started/where-is-the-code.md b/devtools/docs/getting-started/where-is-the-code.md new file mode 100644 index 0000000000..b89d9cf292 --- /dev/null +++ b/devtools/docs/getting-started/where-is-the-code.md @@ -0,0 +1,14 @@ +# Where is the code? (or: `mozilla-central` vs `devtools-html`) + +Most of the code is hosted in the Firefox repository (we call it `mozilla-central`, often abbreviated as `m-c`), in the [devtools](https://searchfox.org/mozilla-central/source/devtools) folder. Development of some pieces of the tools is happening in GitHub, on the [firefox-devtools](https://github.com/firefox-devtools/) organisation. + +<!--TODO: table listing components and locations (m-c vs github)--> + +Code in `m-c` takes longer to obtain and build, as it involves checking out Firefox's repository and installing tools such as a compiler to build a version of Firefox in your machine. + +On the other hand, the repositories in `devtools-html` are more straightforward if you're used to *the GitHub workflow*: you clone them, and then run `npm install && npm run` or similar. Roughly, you can work with each repository individually, and we periodically generate JavaScript bundles that are then copied into `m-c`. + +Even if you only want to work on a tool whose code is on `devtools-html`, you might still need to go through the step of getting and compiling the code from `mozilla-central` in order to do integration work (such as updating a tool bundle). + +From now on, this guide will focus on building the full DevTools within Firefox. Please refer to individual project instructions for tools hosted in `devtools-html`. + diff --git a/devtools/docs/index.md b/devtools/docs/index.md new file mode 100644 index 0000000000..37313269d5 --- /dev/null +++ b/devtools/docs/index.md @@ -0,0 +1,66 @@ +# Devtools + +* [Getting started](getting-started/README.md) + * [Get a Bugzilla account](getting-started/bugzilla.md) + * [Get and build the code](getting-started/build.md) + * [Create a development profile](getting-started/development-profiles.md) +* [Contributing](./contributing.md) + * [Find bugs to work on](./contributing/find-bugs.md) + * [How to fix a bug](./contributing/fixing-bugs.md) + * [Code reviews](./contributing/code-reviews.md) + * [Set up for code reviews](./contributing/code-reviews-setup.md) + * [Find a reviewer](./contributing/code-reviews-find-reviewer.md) + * [Send your code for review](./contributing/making-prs.md) + * [Reviews checklist](./contributing/code-reviews-checklist.md) + * [Landing code](./contributing/landing-code.md) + * [Levelling up](./contributing/levelling-up.md) + * [Coding standards](./contributing/coding-standards.md) + * [JavaScript](./contributing/javascript.md) + * [ESLint](./contributing/eslint.md) + * [CSS](./contributing/css.md) + * [Filing good bugs](./contributing/filing-good-bugs.md) + * [Investigating performance issues](./contributing/performance.md) + * [Writing efficient React code](./contributing/react-performance-tips.md) +* [Automated tests](tests/README.md) + * Running tests + * [`xpcshell`](tests/xpcshell.md) + * [Chrome mochitests](tests/mochitest-chrome.md) + * [DevTools mochitests](tests/mochitest-devtools.md) + * [Node tests](tests/node-tests.md) + * [Writing tests](tests/writing-tests.md) + * [Debugging intermittent failures](tests/debugging-intermittents.md) + * [Performance tests (DAMP)](tests/performance-tests.md) + * [Writing a new test](tests/writing-perf-tests.md) + * [Example](tests/writing-perf-tests-example.md) + * [Advanced tips](tests/writing-perf-tests-tips.md) +* [Files and directories](files/README.md) + * [Adding New Files](files/adding-files.md) +* [Tool Architectures](tools/tools.md) + * [Inspector](tools/inspector.md) + * [Panel Architecture](tools/inspector-panel.md) + * [Highlighters](tools/highlighters.md) + * [Memory](tools/memory-panel.md) + * [Debugger](tools/debugger-panel.md) + * [Responsive Design Mode](tools/responsive-design-mode.md) + * [Console](tools/console-panel.md) +* [Frontend](frontend/frontend.md) + * [Panel SVGs](frontend/svgs.md) + * [React](frontend/react.md) + * [Guidelines](frontend/react-guidelines.md) + * [Redux](frontend/redux.md) + * [Guidelines](frontend/redux-guidelines.md) + * [Telemetry](frontend/telemetry.md) + * [Content Security Policy](frontend/csp.md) +* [Backend](backend/backend.md) + * [Remote Debugging Protocol](backend/protocol.md) + * [Client API](backend/client-api.md) + * [Debugger API](backend/debugger-api.md) + * [Backward Compatibility](backend/backward-compatibility.md) + * Actors + * [Actors Organization](backend/actor-hierarchy.md) + * [Handling Multi-Processes in Actors](backend/actor-e10s-handling.md) + * [Writing Actors With protocol.js](backend/protocol.js.md) + * [Registering A New Actor](backend/actor-registration.md) + * [Actor Best Practices](backend/actor-best-practices.md) +* [Preferences](preferences.md) +* [About this documentation](about-documentation.md) diff --git a/devtools/docs/index.rst b/devtools/docs/index.rst new file mode 100644 index 0000000000..d339b8a89f --- /dev/null +++ b/devtools/docs/index.rst @@ -0,0 +1,118 @@ +================================= +Firefox DevTools Contributor Docs +================================= + +This is a guide to working on the code for Firefox Developer Tools. If you're looking for help with using the tools, see the `user docs <https://developer.mozilla.org/en-US/docs/Tools>`__. For other ways to get involved, check out our `community site <https://firefox-dev.tools/>`__. + + +Getting Started +=============== +.. toctree:: + :maxdepth: 1 + + Getting Started <getting-started/README.md> + Get a Bugzilla account <getting-started/bugzilla.md> + Get and build the code <getting-started/build.md> + Create a development profile <getting-started/development-profiles.md> + + +Contributing +============ +.. toctree:: + :maxdepth: 1 + + Contributing <contributing.md> + Find bugs to work on <contributing/find-bugs.md> + How to fix a bug <contributing/fixing-bugs.md> + Code reviews <contributing/code-reviews.md> + Landing code <contributing/landing-code.md> + Leveling up <contributing/levelling-up.md> + Coding standards <contributing/coding-standards.md> + Filing good bugs <contributing/filing-good-bugs.md> + Investigating performance issues <contributing/performance.md> + Writing efficient React code <contributing/react-performance-tips.md> + + +Automated tests +=============== +.. toctree:: + :maxdepth: 1 + + Automated tests <tests/README.md> + xpcshell <tests/xpcshell.md> + Chrome mochitests <tests/mochitest-chrome.md> + DevTools mochitests <tests/mochitest-devtools.md> + Node tests <tests/node-tests.md> + Writing tests <tests/writing-tests.md> + Debugging intermittent failures <tests/debugging-intermittents.md> + Performance tests (DAMP) <tests/performance-tests.md> + Writing a new test <tests/writing-perf-tests.md> + Example <tests/writing-perf-tests-example.md> + Advanced tips <tests/writing-perf-tests-tips.md> + +Files and directories +===================== +.. toctree:: + :maxdepth: 1 + + Files and directories <files/README.md> + Adding New Files <files/adding-files.md> + + +Tool Architectures +================== +.. toctree:: + :maxdepth: 1 + + Inspector Panel Architecture <tools/inspector-panel.md> + Inspector Highlighters <tools/highlighters.md> + Memory <tools/memory-panel.md> + Debugger <tools/debugger-panel.md> + Responsive Design Mode <tools/responsive-design-mode.md> + Console <tools/console-panel.md> + Network </devtools/netmonitor/architecture.md> + + +Frontend +======== +.. toctree:: + :maxdepth: 1 + + Panel SVGs <frontend/svgs.md> + React <frontend/react.md> + React Guidelines <frontend/react-guidelines.md> + Redux <frontend/redux.md> + Redux Guidelines <frontend/redux-guidelines.md> + Telemetry <frontend/telemetry.md> + Content Security Policy <frontend/csp.md> + + +Backend +======= +.. toctree:: + :maxdepth: 1 + + Remote Debugging Protocol <backend/protocol.md> + Client API <backend/client-api.md> + Debugger API <backend/debugger-api.md> + Backward Compatibility <backend/backward-compatibility.md> + Actors Organization <backend/actor-hierarchy.md> + Handling Multi-Processes in Actors <backend/actor-e10s-handling.md> + Writing Actors With protocol.js <backend/protocol.js.md> + Registering A New Actor <backend/actor-registration.md> + Actor Best Practices <backend/actor-best-practices.md> + +Preferences +=========== +.. toctree:: + :maxdepth: 1 + + Preferences <preferences.md> + + +About this documentation +======================== +.. toctree:: + :maxdepth: 1 + + About this documentation <about-documentation.md> diff --git a/devtools/docs/preferences.md b/devtools/docs/preferences.md new file mode 100644 index 0000000000..93ebcf5395 --- /dev/null +++ b/devtools/docs/preferences.md @@ -0,0 +1,96 @@ +# Preferences + +This documentation aims at giving an overview of the preferences API used in DevTools, it +is not an actual documentation about the list of preferences available in DevTools. + +## Overview + +Preferences allows you to save and read strings, numbers, booleans to the preferences +store, which is tied to a profile. A preference can also have a default value. + +The technical solution for handling preferences differs depending whether you are +testing DevTools as Firefox panel, or a standalone tool running with Launchpad. + +## Preference types + +DevTools relies on nsIPrefBranch for preferences, which supports different types of +preferences: +* `Int` +* `Boolean` +* `Char` +* `String` + +Choose the appropriate type depending on the data you need to store. If you need to store +a JavaScript object or array, the recommended way is to: +* use a `String` type preference +* use JSON.stringify to save +* use JSON.parse to read + +Note that nsIPrefBranch also supports a `Complex` type, but this type is not supported +when running in Launchpad. + +## Reading and updating preferences + +### API docs for nsIPrefBranch and nsIPrefService + +DevTools relies on Services.pref to handle preferences. You can access the API docs for +this service at: +* [API docs for nsIPrefBranch](https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/Interface/nsIPrefBranch) +* [API docs for nsIPrefService](https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/Interface/nsIPrefService) + +If you are using Launchpad, note that only a subset of nsIPrefService methods are +implemented (addObserver and removeObserver). Launchpad relies on a Services shim file +provided by devtools-module ([code on GitHub](https://github.com/firefox-devtools/devtools-core/blob/master/packages/devtools-modules/src/Services.js)). + +### Requiring Services.pref + +To require Services and use Services.pref, you can normally use the following snippet: + +```javascript +const Services = require("Services"); +``` + +In the rare event where you don't have access to the DevTools' require method, you can use + +```javascript +const { Services } = Components.utils.import("resource://gre/modules/Services.jsm"); +``` + +### Services.pref.get* and Services.pref.set* + +The main APIs you will have to know and use are getters and setters. +* `Services.pref.getIntPref(prefName, defaultValue);` This method will throw if the +preference cannot be found and you didn't pass a default value! +* `Services.pref.setIntPref(prefName, prefValue)` This method will throw if the provided +value does not match the preference type! + +These APIs are very similar for each preference type. + +## Create a new preference + +Debugger-specific preferences should go in +devtools/client/preferences/debugger.js. Beyond that, most new preferences +should go in browser/app/profile/firefox.js, which is for desktop Firefox only. +If a preference should be available even when the client for DevTools is not +shipped (for instance on Fennec) it should go in modules/libpref/init/all.js, +which is for preferences that go in all products. + +### Projects using Launchpad + +At the time of writing this doc, projects using Launchpad have to duplicate the default +definition of a preference. +* debugger.html: update [src/utils/prefs.js](https://github.com/firefox-devtools/debugger.html/blob/master/src/utils/prefs.js) +* netmonitor: update [index.js](http://searchfox.org/mozilla-central/source/devtools/client/netmonitor/index.js) +* webconsole: update [local-dev/index.js](http://searchfox.org/mozilla-central/source/devtools/client/webconsole/local-dev/index.js) + +## Inspect preferences + +Depending on the project you are working on, preferences are stored differently but can +always be inspected. + +In Firefox, you can open a tab to about:config and search by preference name. + +In Launchpad, preferences are actually saved to localStorage. Open DevTools on your +Launchpad application and inspect the local storage content. You should see entries +prefixed by `Services.prefs:`. You will only see preferences where a user-specific value +has overridden the default value. diff --git a/devtools/docs/resources/DevToolsDeveloperSettings.png b/devtools/docs/resources/DevToolsDeveloperSettings.png Binary files differnew file mode 100644 index 0000000000..4ed135e626 --- /dev/null +++ b/devtools/docs/resources/DevToolsDeveloperSettings.png diff --git a/devtools/docs/resources/box-model-highlighter-screenshot.png b/devtools/docs/resources/box-model-highlighter-screenshot.png Binary files differnew file mode 100644 index 0000000000..1c64b38768 --- /dev/null +++ b/devtools/docs/resources/box-model-highlighter-screenshot.png diff --git a/devtools/docs/resources/expand-strokes.gif b/devtools/docs/resources/expand-strokes.gif Binary files differnew file mode 100644 index 0000000000..9e15767c5f --- /dev/null +++ b/devtools/docs/resources/expand-strokes.gif diff --git a/devtools/docs/resources/pathfinder.gif b/devtools/docs/resources/pathfinder.gif Binary files differnew file mode 100644 index 0000000000..73d376b103 --- /dev/null +++ b/devtools/docs/resources/pathfinder.gif diff --git a/devtools/docs/resources/sketch-position.png b/devtools/docs/resources/sketch-position.png Binary files differnew file mode 100644 index 0000000000..bcc9ac6689 --- /dev/null +++ b/devtools/docs/resources/sketch-position.png diff --git a/devtools/docs/resources/thread-states.png b/devtools/docs/resources/thread-states.png Binary files differnew file mode 100644 index 0000000000..1808345270 --- /dev/null +++ b/devtools/docs/resources/thread-states.png diff --git a/devtools/docs/styles/website.css b/devtools/docs/styles/website.css new file mode 100644 index 0000000000..eeab68a9aa --- /dev/null +++ b/devtools/docs/styles/website.css @@ -0,0 +1,67 @@ +/* This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ + +.book .book-summary ul.summary li { + cursor: pointer; +} + +.book .book-body .page-wrapper .page-inner section.normal p, +.book .book-body .page-wrapper .page-inner section.normal pre { + margin: 0.85em 0; +} + +.book .book-body .page-wrapper .page-inner section.normal pre { + line-height: 1.25em; +} + +/* Resets default style on the main page container */ +.page-inner { + max-width: unset !important; + margin: 0 10px !important; +} + +/* + * Sections are displayed on the grid. By default everything goes into the + * second column, and we use left and right columns to expand content when needed, + * for example for images, diagrams, code example, … + */ +.normal.markdown-section { + display: grid; + grid-template-columns: 1fr minmax(min-content, 800px) 1fr; + margin: 0 50px; +} + +.normal.markdown-section > * { + grid-column: 2 / 3; +} + +/* Hero element take the whole row */ +.normal.markdown-section > .hero { + grid-column: 1 / -1; + width: max-content; + max-width: 100%; + justify-self: center; +} + +.diagram, +pre.diagram { + width: max-content; + max-width: 100%; + background-color: #f9f9fa; + border: 3px solid #d7d7db; + margin: 0 auto !important; + padding: 2em; + overflow-x: auto; + white-space: pre; + font-size: 11px; +} + +figcaption { + max-width: 800px; + font-size: 0.85em !important; + text-align: center; + font-style: italic; + margin: 10px auto 0; + line-height: 1.25; +} diff --git a/devtools/docs/tests/README.md b/devtools/docs/tests/README.md new file mode 100644 index 0000000000..2b15e87d2d --- /dev/null +++ b/devtools/docs/tests/README.md @@ -0,0 +1,23 @@ +# Automated tests + +When working on a patch for DevTools, there's almost never a reason not to add a new test. If you are fixing a bug, you probably should write a new test to prevent this bug from occurring again. If you're implementing a new feature, you should write new tests to cover the aspects of this new feature. + +Ask yourself: +* Are there enough tests for my patch? +* Are they the right types of tests? + +We use three suites of tests: + +* [`xpcshell`](xpcshell.md): Unit-test style of tests. No browser window, only a JavaScript shell. Mostly testing APIs directly. +* [Chrome mochitests](mochitest-chrome.md): Unit-test style of tests, but with a browser window. Mostly testing APIs that interact with the DOM. +* [DevTools mochitests](mochitest-devtools.md): Integration style of tests. Fires up a whole browser window with every test and you can test clicking on buttons, etc. + +More information about the different types of tests can be found on the [automated testing page](https://developer.mozilla.org/en-US/docs/Mozilla/QA/Automated_testing) at MDN. + +To run all DevTools tests, regardless of suite type: + +```bash +./mach test devtools/* +``` + +Have a look at the child pages for more specific commands for running only a single suite or single test in a suite. diff --git a/devtools/docs/tests/debugging-intermittents.md b/devtools/docs/tests/debugging-intermittents.md new file mode 100644 index 0000000000..626a9d93ea --- /dev/null +++ b/devtools/docs/tests/debugging-intermittents.md @@ -0,0 +1,84 @@ +# Debugging Intermittent Test Failures + +## What are Intermittents (aka Oranges)? + +Intermittents are test failures which happen intermittently, in a seemingly random way. Often you'll write a test that passes fine locally on your computer, but when ran thousands of times on various CI environments (some of them under heavy load) it may start to fail randomly. + +Intermittents are also known as Oranges, because the corresponding test jobs are rendered orange on [treeherder](http://treeherder.mozilla.org/). + +These intermittent failures are tracked in Bugzilla. When a test starts being intermittent a bug is filed in Bugzilla (usually by a Mozilla code sheriff). + +Once the bug exists for a given test failure, all further similar failures of that test will be reported as comments within that bug. +These reports are usually posted weekly and look like this: + +> 5 failures in 2740 pushes (0.002 failures/push) were associated with this bug in the last 7 days. + +See [an example here](https://bugzilla.mozilla.org/show_bug.cgi?id=1250523#c4). + +Sometimes, tests start failing more frequently and these reports are then posted daily. + +To help with the (unfortunately) ever-growing list of intermittents, the Stockwell project was initiated a while ago (read more about the goals of that project on [their wiki](https://wiki.mozilla.org/Auto-tools/Projects/Stockwell)). + +This project defines a scenario where very frequently failing tests get disabled. +Ideally, we should try to avoid this, because this means reducing our test coverage, but sometimes we do not have time to investigate the failure, and disabling it is the only remaining option. + +## Finding Intermittents + +You will have no trouble finding out that a particular test is intermittent, because a bug for it will be filed and you will see it in Bugzilla ([watching the Bugzilla component of your choice](https://bugzilla.mozilla.org/userprefs.cgi?tab=component_watch) is a good way to avoid missing the failure reports). + +However, it can still be useful to see intermittents in context. The [Intermittent Failures View on Treeherder](https://treeherder.mozilla.org/intermittent-failures.html) shows intermittents ranked by frequency. + +You can also see intermittents in Bugzilla. Go to [the settings page](https://bugzilla.mozilla.org/userprefs.cgi?tab=settings) and enable "When viewing a bug, show its corresponding Orange Factor page". + +## Reproducing Test Failures locally + +The first step to fix an intermittent is to reproduce it. + +Sometimes reproducing the failure can only be done in automation, but it's worth trying locally, because this makes it much simpler to debug. + +First, try running the test in isolation. You can use the `--repeat` and `--run-until-failure` flags to `mach mochitest` to automate this a bit. It's nice to do this sort of thing in headless mode (`--headless`) or in a VM (or using Xnest on Linux) to avoid locking up your machine. Mozilla provides an [easy-to-use VM](https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Using_the_VM). + +Sometimes, though, a test will only fail if it is run in conjunction with one or more other tests. You can use the `--start-at` and `--end-at` flags with `mach mochitest` to run a group of tests together. + +For some jobs, but not all, you can get an [interactive shell from TaskCluster](https://jonasfj.dk/2016/03/one-click-loaners-with-taskcluster/). + +There's also a [handy page of e10s test debugging tips](https://wiki.mozilla.org/Electrolysis/e10s_test_tips) that is worth a read. + +Because intermittents are often caused by race conditions, it's sometimes useful to enable Chaos Mode. This changes timings and event orderings a bit. The simplest way to do this is to enable it in a specific test, by +calling `SimpleTest.testInChaosMode`. You can also set the `MOZ_CHAOSMODE` environment variable, or even edit `mfbt/ChaosMode.cpp` directly. + +Some tests leak intermittently. Use `ac_add_options --enable-logrefcnt` in your mozconfig to potentially find them.<!--TODO: how? add more detail about this --> + +The `rr` tool has [its own chaos mode](http://robert.ocallahan.org/2016/02/introducing-rr-chaos-mode.html). This can also sometimes reproduce a failure that isn't ordinarily reproducible. While it's difficult to debug JS bugs using `rr`, often if you can reliably reproduce the failure you can at least experiment (see below) to attempt a fix. + +## That Didn't Work + +If you couldn't reproduce locally, there are other options. + +One useful approach is to add additional logging to the test, then push again. Sometimes log buffering makes the output weird; you can add a call to `SimpleTest.requestCompleteLog()` to fix this. + +You can run a single directory of tests on try using `mach try DIR`. You can also use the `--rebuild` flag to retrigger test jobs multiple times; or you can also do this easily from treeherder.<!--TODO: how? and why is it easy?--> + +## Solving + +If a test fails at different places for each failure it might be a timeout. The current mochitest timeout is 45 seconds, so if successful runs of an intermittent are ~40 seconds, it might just be a +real timeout. This is particularly true if the failure is most often seen on the slower builds, for example Linux 32 debug. In this case you can either split the test or call `requestLongerTimeout` somewhere at the beginning of the test (here's [an example](https://searchfox.org/mozilla-central/rev/c56977420df7a1b692ce0f7e499ddb364d9fd7b2/devtools/client/framework/test/browser_toolbox_tool_remote_reopen.js#12)). + +Sometimes the problem is a race at a specific spot in the test. You can test this theory by adding a short wait to see if the failure goes away, like: +```javascript +yield new Promise(r => setTimeout(r, 100)); +``` + +See the `waitForTick` and `waitForTime` functions in `DevToolsUtils` for similar functionality. + +You can use a similar trick to "pause" the test at a certain point. This is useful when debugging locally because it will leave Firefox open and responsive, at the specific spot you've chosen. Do this +using `yield new Promise(r => r);`. + +`shared-head.js` also has some helpers, like `once`, to bind to events with additional logging. + +You can also binary search the test by either commenting out chunks of it, or hacking in early `return`s. You can do a bunch of these experiments in parallel without waiting for the first to complete. + +## Verifying + +It's difficult to verify that an intermittent has truly been fixed. +One thing you can do is push to try, and then retrigger the job many times in treeherder. Exactly how many times you should retrigger depends on the frequency of the failure. diff --git a/devtools/docs/tests/mochitest-chrome.md b/devtools/docs/tests/mochitest-chrome.md new file mode 100644 index 0000000000..13956ca799 --- /dev/null +++ b/devtools/docs/tests/mochitest-chrome.md @@ -0,0 +1,14 @@ +# Automated tests: chrome mochitests + +To run the whole suite of chrome mochitests: + +```bash +./mach mochitest -f chrome --tag devtools +``` + +To run a specific chrome mochitest: + +```bash +./mach mochitest devtools/path/to/the/test_you_want_to_run.html +``` + diff --git a/devtools/docs/tests/mochitest-devtools.md b/devtools/docs/tests/mochitest-devtools.md new file mode 100644 index 0000000000..e5f44ba1d6 --- /dev/null +++ b/devtools/docs/tests/mochitest-devtools.md @@ -0,0 +1,36 @@ +# Automated tests: DevTools mochitests + +To run the whole suite of browser mochitests for DevTools (sit back and relax): + +```bash +./mach mochitest --subsuite devtools --tag devtools +``` +To run a specific tool's suite of browser mochitests: + +```bash +./mach mochitest devtools/client/<tool> +``` + +For example, run all of the debugger browser mochitests: + +```bash +./mach mochitest devtools/client/debugger +``` +To run a specific DevTools mochitest: + +```bash +./mach mochitest devtools/client/path/to/the/test_you_want_to_run.js +``` +Note that the mochitests *must* have focus while running. The tests run in the browser which looks like someone is magically testing your code by hand. If the browser loses focus, the tests will stop and fail after some time. (Again, sit back and relax) + +In case you'd like to run the mochitests without having to care about focus and be able to touch your computer while running: + +```bash +./mach mochitest --headless devtools/client/<tool> +``` + +You can also run just a single test: + +```bash +./mach mochitest --headless devtools/client/path/to/the/test_you_want_to_run.js +``` diff --git a/devtools/docs/tests/node-tests.md b/devtools/docs/tests/node-tests.md new file mode 100644 index 0000000000..0152d64cab --- /dev/null +++ b/devtools/docs/tests/node-tests.md @@ -0,0 +1,82 @@ +# DevTools node tests + +In addition to mochitests and xpcshell tests, some panels in DevTools are using node test libraries to run unit tests. For instance, several panels are using [Jest](https://jestjs.io/) to run React component unit tests. + +## Find the node tests on Try + +The DevTools node tests are split in two different test suites on try: +- `node(devtools)`: all the DevTools node tests, except the ones for the debugger +- `node(debugger)`: only the Debugger node tests + +They are running on the `Linux 64 opt` platform. They are both tier 1 jobs, which means that any failure will lead to a backout. + +## Run Tests On Try + +To run the DevTools node tests on try, you can use `./mach try fuzzy` and look for the jobs named `source-test-node-debugger-tests` and `source-test-node-devtools-tests`. + +They are also run when using the "devtools" preset: `./mach try --preset devtools`. + +### Node tests try job definition + +The definition of those try jobs can be found at [taskcluster/ci/source-test/node.yml](https://searchfox.org/mozilla-central/source/taskcluster/ci/source-test/node.yml). + +The definition also contains the list of files that will trigger the node test jobs. Currently the debugger tests run when any file is modified under `devtools/client/debugger`, the devtools tests run when any file is modified under `devtools/client` or `devtools/shared`. + +## Run Tests Locally + +### Prerequisite: yarn + +You will need yarn to be installed in order to run both the debugger and the DevTools tests. See https://yarnpkg.com/docs/install/. + +### Debugger + +To run the debugger node tests: +``` +> cd devtools/client/debugger/ +> yarn && node bin/try-runner.js +``` + +Note that the debugger is running other tasks than just unit tests: `flow`, `eslint`, `stylelint` etc... +Using `yarn && yarn test` would only run the Jest tests, while `node bin/try-runner` will run the same tests and scripts as the ones used on try. + +### DevTools + +To run the other (non-debugger) DevTools tests, the easiest is to rely on the same script as the one used to run the tests on try: +``` +> node devtools/client/bin/devtools-node-test-runner.js --suite={suitename} +``` + +At the moment of writing, the supported suites for this script are: +- `aboutdebugging` +- `accessibility` +- `application` +- `compatibility` +- `framework` +- `netmonitor` +- `performance` +- `shared_components` +- `webconsole` + +(You can see the full list and the associated configuration in devtools/client/bin/devtools-node-test-runner.js) + +Alternatively, you can also locate the `package.json` corresponding to a given suite, and run `yarn && yarn test`. + +## Updating snapshots + +Some of the node tests are snapshot tests, which means they compare the output of a given component to a previous text snapshot. They might break if you are legitimately modifying a component and it means the snapshots need to be updated. + +A snapshot failure will show up as follows: +``` +› 1 snapshot failed from 1 test suite +``` + +It should also mention the command you can run to update the snapshots: +``` +Inspect your code changes or run `yarn run test-ci -u` to update them. +``` + +For example, if you need to update snapshots in a specific panel, first locate the package.json corresponding to the node test folder of the panel. In theory it should be under `devtools/client/{panelname}/test/node/` but it might be slightly different depending on each panel. Then run `yarn run test-ci -u` in this folder and add the snapshot changes to your commit. + +## TypeScript + +The "performance" suite performs TypeScript checks. The TypeScript usage in the performance panel is documented at [devtools/client/performance-new/typescript.md](https://searchfox.org/mozilla-central/source/devtools/client/performance-new/typescript.md) ([see rendered version on GitHub](https://github.com/mozilla/gecko-dev/blob/master/devtools/client/performance-new/typescript.md)). diff --git a/devtools/docs/tests/perfherder-compare-link.png b/devtools/docs/tests/perfherder-compare-link.png Binary files differnew file mode 100644 index 0000000000..8e253bf363 --- /dev/null +++ b/devtools/docs/tests/perfherder-compare-link.png diff --git a/devtools/docs/tests/perfherder-compare.png b/devtools/docs/tests/perfherder-compare.png Binary files differnew file mode 100644 index 0000000000..55ba3b3c5d --- /dev/null +++ b/devtools/docs/tests/perfherder-compare.png diff --git a/devtools/docs/tests/perfherder-create-gecko-profile.png b/devtools/docs/tests/perfherder-create-gecko-profile.png Binary files differnew file mode 100644 index 0000000000..a7526bb25f --- /dev/null +++ b/devtools/docs/tests/perfherder-create-gecko-profile.png diff --git a/devtools/docs/tests/perfherder-damp.png b/devtools/docs/tests/perfherder-damp.png Binary files differnew file mode 100644 index 0000000000..e8b853adb7 --- /dev/null +++ b/devtools/docs/tests/perfherder-damp.png diff --git a/devtools/docs/tests/perfherder-filter-subtests.png b/devtools/docs/tests/perfherder-filter-subtests.png Binary files differnew file mode 100644 index 0000000000..c33187d556 --- /dev/null +++ b/devtools/docs/tests/perfherder-filter-subtests.png diff --git a/devtools/docs/tests/perfherder-subtests.png b/devtools/docs/tests/perfherder-subtests.png Binary files differnew file mode 100644 index 0000000000..fbe90299ac --- /dev/null +++ b/devtools/docs/tests/perfherder-subtests.png diff --git a/devtools/docs/tests/performance-tests.md b/devtools/docs/tests/performance-tests.md new file mode 100644 index 0000000000..01ecd18b92 --- /dev/null +++ b/devtools/docs/tests/performance-tests.md @@ -0,0 +1,192 @@ +# Performance Tests: DAMP + +DAMP (DevTools At Maximum Performance) is our test suite to track performance. + +## How to run it locally? + +```bash +./mach talos-test --activeTests damp +``` +Note that the first run is slower as it pulls a large tarball with various website copies. +This will run all DAMP tests, you can filter by test name with: +```bash +./mach talos-test --activeTests damp --subtests console +``` +This command will run all tests which contains "console" in their name. + +### Command line options + +#### Running tests only once + +```bash +./mach talos-test --activeTests damp --cycles 1 --tppagecycles 1 +``` +`--cycles` will limit the number of Firefox restart to only one, while +`--tppagecycles` will limit the number of test re-run in each firefox start to one. +This is often helpful when debugging one particular subtest. + +#### Taking screenshots + +```bash +DEBUG_DEVTOOLS_SCREENSHOTS=1 ./mach talos-test --activeTests damp +``` +When passing `DEBUG_DEVTOOLS_SCREENSHOTS` env variable, screenshots will be taken after each subtest +was run. The screenshot will be opened in new tabs and their title +includes the subtest label. Firefox won't automatically close so that you can view the screenshots. + +#### Recording a profile + +```bash +./mach talos-test --activeTests damp --gecko-profile --gecko-profile-entries 100000000 +``` +This will automatically record the tests and open the profile. You may use the following command in order +to focus on just one subtest run: +```bash +./mach talos-test --activeTests damp --subtests custom.webconsole --cycles 1 --tppagecycles 1 --gecko-profile --gecko-profile-entries 100000000 +``` + +## How to run it on try? + +```bash +./mach try fuzzy --query "'linux64-shippable/ 'damp" --rebuild 6 +``` +* Linux appears to build and run quickly, and offers quite stable results over the other OSes. +The vast majority of performance issues for DevTools are OS agnostic, so it doesn't really matter which one you run them on. +* "damp" is the talos bucket in which we run DAMP. +* And 6 is the number of times we run DAMP tests. That's to do averages between all the 6 runs and helps filtering out the noise. + +## How to get performance profiles on try? + +Once you have a successful try job for `damp`: +* select this job in treeherder +* click on the `...` menu in the bottom left +* select "Create Gecko Profile" + + + +This should start a new damp job called `damp-p`. Once `damp-p` is finished: +* select the `damp-p` job +* click on `Job Details` tab +* click on `open in Firefox Profiler` + +## What does it do? + +DAMP measures three important operations: +* Open a toolbox +* Reload the web page +* Close the toolbox +It measures the time it takes to do each of these operations for the following panels: + +inspector, console, netmonitor debugger, memory, performance. + +It runs all these three tests two times. Each time against a different web page: +* "simple": an empty webpage. This test highlights the performance of all tools against the simplest possible page. +* "complicated": a copy of bild.de website. This is supposed to represent a typical website to debug via DevTools. + +Then, there are a couple of extra tests: +* "cold": we run the three operations (open toolbox, page reload and close toolbox) first with the inspector. +This is run first after Firefox's startup, before any other test. +This test allows to measure a "cold startup". When a user first interacts with DevTools, many resources are loaded and cached, +so that all next interactions will be significantly faster. +* and many other smaller tests, focused on one particular feature or possible slowness for each panel. + +## How to see the results from try? + +First, open TreeHerder. A link is displayed in your console when executing `./mach try`. +You should also receive a mail with a link to it. + +Look for "T-e10s(+6)", click on "+6", then click on "damp": + + +On the bottom panel that just opened, click on "Compare result against another revision". + + +You are now on PerfHerder, click on "Compare", + + +Next to "Talos" select menu, in the filter textbox, type "damp". +Under "damp opt e10s" item, mouse over the "linux64" line, click on "subtests" link. + + +And here you get the results for each DAMP test: + + +On this page, you can filter by test name with the filter box on top of the result table. +This table has the following columns: +* Base: + Average time it took to run the test on the base build (by default, the last 2 days of DAMP runs on mozilla-central revisions) +* New: + Average time it took to run the test on the new build, the one with your patches. + Both "Base" and "New" have a "± x.xx%" suffix which tells you the variance of the timings. + i.e. the average difference in percent between the median timing and both the slowest and the fastest. +* Delta: + Difference in percent between the base and new runs. + The color of this can be red, orange or green: + * Red means "certainly regressing" + * Orange means "possibly regressing" + * Green means "certainly improving" + * No colored background means "nothing to conclude" + The difference between certainly and possibly is explained by the next column. +* Confidence: + If there is a significant difference between the two runs, tells if the results is trustworthy. + * "low" either means there isn't a significant difference between the two runs, or the difference is smaller than the typical variance of the given test. + If the test is known to have an execution time varying by 2% between two runs of the same build, and you get a 1% difference between your base and new builds, + the confidence will be low. You really can't make any conclusion. + * "med" means medium confidence and the delta is around the size of the variance. It may highlight a regression, but it can still be justified by the test noise. + * "high" means that this is a high confidence difference. The delta is significantly higher than the typical test variance. A regression is most likely detected. + +There is also "Show only important changes" checkbox, which helps seeing if there is any significant regression. +It will only display regressions and improvements with a medium or high confidence. + +## How to contribute to DAMP? + +DAMP is based on top of a more generic test suite called [Talos](https://wiki.mozilla.org/Buildbot/Talos). +Talos is a Mozilla test suite to follow all Firefox components performance. +It is written in Python and here are [the sources](https://searchfox.org/mozilla-central/source/testing/talos/) in mozilla-central. +Compared to the other test suites, it isn't run on the cloud, but on dedicated hardware. +This is to ensure performance numbers are stable over time and between two runs. +Talos runs various types of tests. More specifically, DAMP is a [Page loader test](https://wiki.mozilla.org/Buildbot/Talos/Tests#Page_Load_Tests). +The [source code](http://searchfox.org/mozilla-central/source/testing/talos/talos/tests/devtools/) for DAMP is also in mozilla-central. +See [Writing new performance test](./writing-perf-tests.md) for more information about the implementation of DAMP tests. + +## How to see the performance trends? + +You can find the dedicated performance dashboard for DevTools at http://firefox-dev.tools/performance-dashboard. You will find links to trend charts for various tools: +* [Inspector dashboard](http://firefox-dev.tools/performance-dashboard/tools/inspector.html?days=60&filterstddev=true) +* [Console dashboard](http://firefox-dev.tools/performance-dashboard/tools/console.html?days=60&filterstddev=true) +* [Netmonitor dashboard](http://firefox-dev.tools/performance-dashboard/tools/netmonitor.html?days=60&filterstddev=true) +* [Debugger dashboard](http://firefox-dev.tools/performance-dashboard/tools/debugger.html?days=60&filterstddev=true) + +Each tool page displays charts for all the subtests relevant for a given panel. + +Each circle on the chart is a push to mozilla-central. You can hover on a circle to see some additional information about the push, such as the date, the performance impact for the subtest, and the push id. Clicking on a circle will take you to the pushlog. + +Colored circles indicate that the push contains a change that was identified as having a performance impact. Those can be categorized as: +- hardware: hardware change for the machines used to run Talos +- platform: non-DevTools change that impacts DevTools performance +- damp: test change in DAMP that impacts test results +- devtools: identified DevTools change that introduced an improvement or regression + +This data is synchronized from a [shared Google doc](https://docs.google.com/spreadsheets/d/12Goo3vq-0X0_Ay-J6gfV56pUB8GC0Nl62I4p8G-UsEA/edit#gid=0). + +There is a PerfHerder link on each chart that will take you to the PerfHerder page corresponding to this subtest. + +## How to use PerfHerder charts + +On PerfHerder charts, each circle is a push on mozilla-central. +When you see a spike or a drop, you can try to identify the patch that relates to it by clicking the circles. +It will show a black popup. Then click on the changeset hash like "cb717386aec8" and you will get a mercurial changelog. +Then it is up to you to read the changelog and see which changeset may have hit the performance. + +For example, open [this page](https://treeherder.mozilla.org/perf.html#/graphs?timerange=31536000&series=mozilla-central,1417969,1,1&series=mozilla-central,1417971,1,1&series=mozilla-central,1417966,1,1&highlightedRevisions=a06f92099a5d&zoom=1482734645161.3916,1483610598216.4773,594.756508587898,969.2883437938906). +This is tracking inspector opening performance against the "Simple" page. + + +See the regression on Dec 31th? +Now, click on the first yellow circle of this spike. +You will get a black popup like this one: + + +Click on the [changelog link](https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=9104708cc3ac0ccfe4cf5d518e13736773c565d7&tochange=a06f92099a5d8edeb05e5971967fe8d6cd4c593c) to see which changesets were added during this run. Here, you will see that the regression comes from these patches: + * Bug 1245921 - Turn toolbox toolbar into a React component + * Bug 1245921 - Monkey patch ReactDOM event system for XUL diff --git a/devtools/docs/tests/regression-graph.png b/devtools/docs/tests/regression-graph.png Binary files differnew file mode 100644 index 0000000000..3212324012 --- /dev/null +++ b/devtools/docs/tests/regression-graph.png diff --git a/devtools/docs/tests/regression-popup.png b/devtools/docs/tests/regression-popup.png Binary files differnew file mode 100644 index 0000000000..e64d55df0d --- /dev/null +++ b/devtools/docs/tests/regression-popup.png diff --git a/devtools/docs/tests/writing-perf-tests-example.md b/devtools/docs/tests/writing-perf-tests-example.md new file mode 100644 index 0000000000..80b28c59d2 --- /dev/null +++ b/devtools/docs/tests/writing-perf-tests-example.md @@ -0,0 +1,67 @@ +# Performance test example: performance of click event in the inspector + +Let's look at a trivial but practical example and add a simple test to measure the performance of a click in the inspector. + +First we create a file under [tests/inspector](https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/devtools/addon/content/tests/inspector) since we are writing an inspector test. We call the file `click.js`. + +We will use a dummy test document here: `data:text/html,click test document`. + +We prepare the imports needed to write the test, from head.js and inspector-helper.js: +- `testSetup`, `testTeardown`, `openToolbox` and `runTest` from head.js +- `reloadInspectorAndLog` from inspector-helper.js + +The full code for the test looks as follows: +``` +const { + reloadInspectorAndLog, +} = require("devtools/docs/tests/inspector-helpers"); + +const { + openToolbox, + runTest, + testSetup, + testTeardown, +} = require("devtools/docs/head"); + +module.exports = async function() { + // Define here your custom document via a data URI: + const url = "data:text/html,click test document"; + + await testSetup(url); + const toolbox = await openToolbox("inspector"); + + const inspector = toolbox.getPanel("inspector"); + const window = inspector.panelWin; // Get inspector's panel window object + const body = window.document.body; + + await new Promise(resolve => { + const test = runTest("inspector.click"); + body.addEventListener("click", function () { + test.done(); + resolve(); + }, { once: true }); + body.click(); + }); + + // Check if the inspector reload is impacted by click + await reloadInspectorAndLog("click", toolbox); + + await testTeardown(); +} +``` + +Finally we add an entry in [damp-tests.js](https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/devtools/addon/content/damp-tests.js): +``` + { + name: "inspector.click", + path: "inspector/click.js", + description: + "Measure the time to click in the inspector, and reload the inspector", + }, +``` + +Then we can run our test with: +``` +./mach talos-test --activeTests damp --subtest inspector.click +``` + diff --git a/devtools/docs/tests/writing-perf-tests-tips.md b/devtools/docs/tests/writing-perf-tests-tips.md new file mode 100644 index 0000000000..7381b206e5 --- /dev/null +++ b/devtools/docs/tests/writing-perf-tests-tips.md @@ -0,0 +1,41 @@ +# How to write a good performance test? + +## Verify that you wait for all asynchronous code + +If your test involves asynchronous code, which is very likely given the DevTools codebase, please review carefully your test script. +You should ensure that _any_ code ran directly or indirectly by your test is completed. +You should not only wait for the functions related to the very precise feature you are trying to measure. + +This is to prevent introducing noise in the test run after yours. If any asynchronous code is pending, +it is likely to run in parallel with the next test and increase its variance. +Noise in the tests makes it hard to detect small regressions. + +You should typically wait for: +* All RDP requests to finish, +* All DOM Events to fire, +* Redux action to be dispatched, +* React updates, +* ... + + +## Ensure that its results change when regressing/fixing the code or feature you want to watch. + +If you are writing the new test to cover a recent regression and you have a patch to fix it, push your test to try without _and_ with the regression fix. +Look at the try push and confirm that your fix actually reduces the duration of your perf test significantly. +If you are introducing a test without any patch to improve the performance, try slowing down the code you are trying to cover with a fake slowness like `setTimeout` for asynchronous code, or very slow `for` loop for synchronous code. This is to ensure your test would catch a significant regression. + +For our click performance test, we could do this from the inspector codebase: +``` +window.addEventListener("click", function () { + + // This for loop will fake a hang and should slow down the duration of our test + for (let i = 0; i < 100000000; i++) {} + +}, true); // pass `true` in order to execute before the test click listener +``` + + +## Keep your test execution short. + +Running performance tests is expensive. We are currently running them 25 times for each changeset landed in Firefox. +Aim to run tests in less than a second on try.
\ No newline at end of file diff --git a/devtools/docs/tests/writing-perf-tests.md b/devtools/docs/tests/writing-perf-tests.md new file mode 100644 index 0000000000..da0cc92bd5 --- /dev/null +++ b/devtools/docs/tests/writing-perf-tests.md @@ -0,0 +1,145 @@ +# Writing new performance test + +See [Performance tests (DAMP)](performance-tests.md) for an overall description of our performance tests. +Here, we will describe how to write a new test and register it to run in DAMP. + +{% hint style="tip" %} + +**Reuse existing tests if possible!** + +If a `custom` page already exists for the tool you are testing, try to modify the existing `custom` test rather than adding a new individual test. + +New individual tests run separately, in new tabs, and make DAMP slower than just modifying existing tests. Complexifying `custom` test pages should also help cover more scenarios and catch more regressions. For those reasons, modifying existing tests should be the preferred way of extending DAMP coverage. + +`custom` tests are using complex documents that should stress a particular tool in various ways. They are all named `custom.${tool}` (for instance `custom.inspector`). The test pages for those tests can be found in [pages/custom](https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/devtools/addon/content/pages/custom). + +If your test case requires a dedicated document or can't run next to the other tests in the current `custom` test, follow the instructions below to add a new individual test. + +{% endhint %} + +This page contains the general documentation for writing DAMP tests. See also: +- [Performance test writing example](./writing-perf-tests-example.html) for a practical example of creating a new test +- [Performance test writing tips](./writing-perf-tests-tips.html) for detailed tips on how to write a good and efficient test + +## Test location + +Tests are located in [testing/talos/talos/tests/devtools/addon/content/tests](https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/devtools/addon/content/tests). You will find subfolders for panels already tested in DAMP (debugger, inspector, …) as well as other subfolders for tests not specific to a given panel (server, toolbox). + +Tests are isolated in dedicated files. Some examples of tests: +- [tests/netmonitor/simple.js](https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/devtools/addon/content/tests/netmonitor/simple.js) +- [tests/inspector/mutations.js](https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/devtools/addon/content/tests/inspector/mutations.js) + +## Basic test + +The basic skeleton of a test is: + +``` +const { + testSetup, + testTeardown, + SIMPLE_URL, +} = require("devtools/docs/head"); + +module.exports = async function() { + await testSetup(SIMPLE_URL); + + // Run some measures here + + await testTeardown(); +}; +``` + +* always start the test by calling `testSetup(url)`, with the `url` of the document to use +* always end the test with `testTeardown()` + + +## Test documents + +DevTools performance heavily depends on the document against which DevTools are opened. There are two "historical" documents you can use for tests for any panel: +* "Simple", an empty webpage. This one helps highlighting the load time of panels, +* "Complicated", a copy of bild.be, a German newspaper website. This allows us to examine the performance of the tools when inspecting complicated, big websites. + +The URL of those documents are exposed by [tests/head.js](https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/devtools/addon/content/tests/head.js). The Simple page can be found at [testing/talos/talos/tests/devtools/addon/content/pages/simple.html](https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/devtools/addon/content/pages/simple.html). The Complicated page is downloaded via [tooltool](https://wiki.mozilla.org/ReleaseEngineering/Applications/Tooltool) automatically the first time you run the DAMP tests. + +You can create also new test documents under [testing/talos/talos/tests/devtools/addon/content/pages](https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/devtools/addon/content/pages). See the pages in the `custom` subfolder for instance. If you create a document in `pages/custom/mypanel/index.html`, the URL of the document in your tests should be `PAGES_BASE_URL + "custom/mypanel/index.html"`. The constant `PAGES_BASE_URL` is exposed by head.js. + +Note that modifying any existing test document will most likely impact the baseline for existing tests. + +Finally you can also create very simple test documents using data urls. Test documents don't have to contain any specific markup or script to be valid DAMP test documents, so something as simple as `testSetup("data:text/html,my test document");` is valid. + + +## Test helpers + +Helper methods have been extracted in shared modules: +* [tests/head.js](https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/devtools/addon/content/tests/head.js) for the most common ones +* tests/{subfolder}/{subfolder}-helpers.js for folder-specific helpers ([example](https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/devtools/addon/content/tests/inspector/inspector-helpers.js)) + +To measure something which is not covered by an existing helper, you should use `runTest`, exposed by head.js. + +``` +module.exports = async function() { + await testSetup(SIMPLE_URL); + + // Calling `runTest` will immediately start recording your action duration. + // You can execute any necessary setup action you don't want to record before calling it. + const test = runTest(`mypanel.mytest.mymeasure`); + + await doSomeThings(); // <== Do an action you want to record here + + // Once your action is completed, call `runTest` returned object's `done` method. + // It will automatically record the action duration and appear in PerfHerder as a new subtest. + // It also creates markers in the profiler so that you can better inspect this action in + // profiler.firefox.com. + test.done(); + + await testTeardown(); +}; +``` + +If your measure is not simply the time spent by an asynchronous call (for instance computing an average, counting things…) there is a lower level helper called `logTestResult` which will directly log a value. See [this example](https://searchfox.org/mozilla-central/rev/325c1a707819602feff736f129cb36055ba6d94f/testing/talos/talos/tests/devtools/addon/content/tests/webconsole/streamlog.js#62). + + +## Test runner + +If you need to dive into the internals of the DAMP runner, most of the logic is in [testing/talos/talos/tests/devtools/addon/content/damp.js](https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/devtools/addon/content/damp.js). + + +# How to name your test and register it? + +If a new test file was created, it needs to be registered in the test suite. To register the new test, add it in [damp-tests.js](https://searchfox.org/mozilla-central/source/testing/talos/talos/tests/devtools/addon/content/damp-tests.js). This file acts as the manifest for the DAMP test suite. + +If your are writing a test executing against Simple and Complicated documents, your test name will look like: `(simple|complicated).${tool-name}.${test-name}`. +So for our example, it would be `simple.inspector.click` and `complicated.inspector.click`. +For independent tests that don't use the Simple or Complicated documents, the test name only needs to start with the tool name, if the test is specific to that tool +For the example, it would be `inspector.click`. + +In general, the test name should try to match the path of the test file. As you can see in damp-tests.js this naming convention is not consistently followed. We have discrepancies for simple/complicated/custom tests, as well as for webconsole tests. This is largely for historical reasons. + + +# How to run your new test? + +You can run any performance test with this command: +``` +./mach talos-test --activeTests damp --subtest ${your-test-name} +``` + +By default, it will run the test 25 times. In order to run it just once, do: +``` +./mach talos-test --activeTests damp --subtest ${your-test-name} --cycles 1 --tppagecycles 1 +``` +`--cycles` controls the number of times Firefox is restarted +`--tppagecycles` defines the number of times we repeat the test after each Firefox start + +Also, you can record a profile while running the test. To do that, execute: +``` +./mach talos-test --activeTests damp --subtest ${your-test-name} --cycles 1 --tppagecycles 1 --gecko-profile --gecko-profile-entries 100000000 +``` +`--gecko-profile` enables the profiler +`--gecko-profile-entries` defines the profiler buffer size, which needs to be large while recording performance tests + +Once it is done executing, the profile lives in a zip file you have to uncompress like this: +``` +unzip testing/mozharness/build/blobber_upload_dir/profile_damp.zip +``` +Then you have to open [https://profiler.firefox.com/](https://profiler.firefox.com/) and manually load the profile file that lives here: `profile_damp/page_0_pagecycle_1/cycle_0.profile` + diff --git a/devtools/docs/tests/writing-tests.md b/devtools/docs/tests/writing-tests.md new file mode 100644 index 0000000000..8571847401 --- /dev/null +++ b/devtools/docs/tests/writing-tests.md @@ -0,0 +1,250 @@ +# Automated tests: writing tests + +<!--TODO this file might benefit from being split in other various files. For now it's just taken from the wiki with some edits--> + +## Adding a new browser chrome test + +It's almost always a better idea to create a new test file rather than to add new test cases to an existing one. + +This prevents test files from growing up to the point where they timeout for running too long. Test systems may be under lots of stress at time and run a lot slower than your regular local environment. + +It also helps with making tests more maintainable: with many small files, it's easier to track a problem rather than in one huge file. + +### Creating the new file + +The first thing you need to do is create a file. This file should go next to the code it's testing, in the `tests` directory. For example, an inspector test would go into `devtools/inspector/test/`. + +### Naming the new file + +Naming your file is pretty important to help other people get a feeling of what it is supposed to test. +Having said that, the name shouldn't be too long either. + +A good naming convention is `browser_<panel>_<short-description>[_N].js` + +where: + +* `<panel>` is one of `debugger`, `markupview`, `inspector`, `ruleview`, etc. +* `<short-description>` should be about 3 to 4 words, separated by hyphens (-) +* and optionally add a number at the end if you have several files testing the same thing + +For example: `browser_ruleview_completion-existing-property_01.js` + +Note that not all existing tests are consistently named. So the rule we try to follow is to **be consistent with how other tests in the same test folder are named**. + +### Basic structure of a test + +```javascript +/* Any copyright is dedicated to the Public Domain. +http://creativecommons.org/publicdomain/zero/1.0/ */ +"use strict"; + +// A detailed description of what the test is supposed to test + +const TEST_URL = TEST_URL_ROOT + "doc_some_test_page.html"; + +add_task(function*() { +yield addTab(TEST_URL_ROOT); +let {toolbox, inspector, view} = yield openRuleView(); +yield selectNode("#testNode", inspector); +yield checkSomethingFirst(view); +yield checkSomethingElse(view); +}); + +function* checkSomethingFirst(view) { +/* ... do something ... this function can yield */ +} + +function* checkSomethingElse(view) { +/* ... do something ... this function can yield */ +} +``` + +### Referencing the new file + +For your test to be run, it needs to be referenced in the `browser.ini` file that you'll find in the same directory. For example: `browser/devtools/debugger/test/browser.ini` + +Add a line with your file name between square brackets, and make sure that the list of files **is always sorted by alphabetical order** (some lists can be really long, so the alphabetical order helps in finding and reasoning about things). + +For example, if you were to add the test from the previous section, you'd add this to `browser.ini`: + +```ini +[browser_ruleview_completion-existing-property_01.js] +``` + +### Adding support files + +Sometimes your test may need to open an HTML file in a tab, and it may also need to load CSS or JavaScript. For this to work, you'll need to... + +1. place these files in the same directory, and also +2. reference them in the `browser.ini` file. + +There's a naming convention for support files: `doc_<support-some-test>.html` + +But again, often names do not follow this convention, so try to follow the style of the other support files currently in the same test directory. + +To reference your new support file, add its filename in the `support-files` section of `browser.ini`, also making sure this section is in alphabetical order. + +Support files can be accessed via a local server that is started while tests are running. This server is accessible at [http://example.com/browser/](http://example.com/browser/). See the `head.js section` below for more information. + +## Leveraging helpers in `head.js` + +`head.js` is a special support file that is loaded in the scope the test runs in, before the test starts. It contains global helpers that are useful for most tests. Read through the head.js file in your test directory to see what functions are there and therefore avoid duplicating code. + +Each panel in DevTools has its own test directory with its own `head.js`, so you'll find different things in each panel's `head.js` file. + +For example, the head.js files in the `markupview` and `styleinspector` test folders contain these useful functions and constants: + +* Base URLs for support files: `TEST_URL_ROOT`. This avoids having to duplicate the http://example.com/browser/browser/devtools/styleinspector/ URL fragment in all tests, +* `waitForExplicitFinish()` is called in `head.js` once and for all<!--TODO: what does this even mean?-->. All tests are asynchronous, so there's no need to call it again in each and every test, +* `auto-cleanup`: the toolbox is closed automatically and all tabs are closed, +* `tab addTab(url)` +* `{toolbox, inspector} openInspector()` +* `{toolbox, inspector, view} openRuleView()` +* `selectNode(selectorOrNode, inspector)` +* `node getNode(selectorOrNode)` +* ... + +## Shared head.js file + +A [shared-head.js](https://searchfox.org/mozilla-central/source/devtools/client/shared/test/shared-head.js) file has been introduced to avoid duplicating code in various `head.js` files. + +It's important to know whether or not the `shared.js` in your test directory already imports `shared-head.js` (look for a <code>Services.scriptloader.loadSubScript</code> call), as common helpers in `shared-head.js` might be useful for your test. + +If you're planning to work on a lot of new tests, it might be worth the time actually importing `shared-head.js` in your `head.js` if it isn't here already. + +## Electrolysis + +E10S is the codename for Firefox multi-process, and what that means for us is that the process in which the test runs isn't the same as the one in which the test content page runs. + +You can learn more about E10S [from this blog post](https://timtaubert.de/blog/2011/08/firefox-electrolysis-101/), [the Electrolysis wiki page](https://wiki.mozilla.org/Electrolysis) and the page on [tests and E10s](https://wiki.mozilla.org/Electrolysis/e10s_test_tips). + +One of the direct consequences of E10S on tests is that you cannot retrieve and manipulate objects from the content page as you'd do without E10S. + +So when creating a new test, if this test needs to access the content page in any way, you can use [the message manager](https://developer.mozilla.org/en-US/docs/The_message_manager) to communicate with a script loaded in the content process to do things for you instead of accessing objects in the page directly. + +You can use the helper `ContentTask.spawn()` for this. See [this list of DevTools tests that use that helper for examples](https://searchfox.org/mozilla-central/search?q=ContentTask.spawn%28&path=devtools%2Fclient). + +Note that a lot of tests only need to access the DevTools UI anyway, and don't need to interact with the content process at all. Since the UI lives in the same process as the test, you won't need to use the message manager to access it. + +## Asynchronous tests + +Most browser chrome DevTools tests are asynchronous. One of the reasons why they are asynchronous is that the code needs to register event handlers for various user interactions in the tools and then simulate these interactions. Another reason is that most DevTools operations are done asynchronously via the debugger protocol. + +Here are a few things to keep in mind with regards to asynchronous testing: + +* `head.js` already calls `waitForExplicitFinish()` so there's no need for your new test to do it too. +* Using `add_task` with a generator function means that you can yield calls to functions that return promises. It also means your main test function can be written to almost look like synchronous code, by adding `yield` before calls to asynchronous functions. For example: + +```javascript +for (let test of testData) { + yield testCompletion(test, editor, view); +} +``` + +Each call to `testCompletion` is asynchronous, but the code doesn't need to rely on nested callbacks and maintain an index, a standard for loop can be used. + +<!--TODO: I think the following paragraph might be slightly outdated--> + +* Define your test functions as generators that yield, no need for them to be tasks since they are called from one already. In some cases you'll need to return promises anyway (if you're adding a new helper function to head.js for example). If this is the case, it sometimes is best to define your function like so: + +```javascript +let myHelperFunction = Task.async(function*() { + ... +}); +``` + +## Writing clean, maintainable test code + +Test code is as important as feature code itself, it helps avoiding regressions of course, but it also helps understanding complex parts of the code that would be otherwise hard to grasp. + +Since we find ourselves working with test code a large portion of our time, we should spend the time and energy it takes to make this time enjoyable. + +### Logs and comments + +Reading test output logs isn't exactly fun and it takes time but is needed at times. Make sure your test generates enough logs by using: + +``` +info("doing something now") +``` + +it helps a lot knowing around which lines the test fails, if it fails. + +One good rule of thumb is if you're about to add a JS line comment in your test to explain what the code below is about to test, write the same comment in an `info()` instead. + +Also add a description at the top of the file to help understand what this test is about. The file name is often not long enough to convey everything you need to know about the test. Understanding a test often teaches you about the feature itself. + +Not really a comment, but don't forget to "use strict"; + +### Callbacks and promises + +Avoid multiple nested callbacks or chained promises. They make it hard to read the code. + +Thanks to `add_task`, it's easy to write asynchronous code that looks like flat, synchronous, code. + +### Clean up after yourself + +Do not expose global variables in your test file, they may end up causing bugs that are hard to track. Most functions in `head.js` return useful instances of the DevTools panels, and you can pass these as arguments to your sub functions, no need to store them in the global scope. +This avoids having to remember nullifying them at the end. + +If your test needs to toggle user preferences, make sure you reset these preferences when the test ends. Do not reset them at the end of the test function though because if your test fails, the preferences will never be reset. Use the `registerCleanupFunction` helper instead. + +It may be a good idea to do the reset in `head.js`. + +### Write small, maintainable code + +Split your main test function into smaller test functions with self explanatory names. + +Make sure your test files are small. If you are working on a new feature, you can create a new test each time you add a new functionality, a new button to the UI for instance. This helps having small, incremental tests and can also help writing test while coding. + +If your test is just a sequence of functions being called to do the same thing over and over again, it may be better to describe the test steps in an array instead and just have one function that runs each item of the array. See the following example + +```javascript +const TESTS = [ + {desc: "add a class", cssSelector: "#id1", makeChanges: function*() {...}}, + {desc: "change href", cssSelector: "a.the-link", makeChanges: function*() {...}}, + ... +]; + +add_task(function*() { + yield addTab("..."); + let {toolbox, inspector} = yield openInspector(); + for (let step of TESTS) { + info("Testing step: " + step.desc); + yield selectNode(step.cssSelector, inspector); + yield step.makeChanges(); + } +}); +``` + +As shown in this code example, you can add as many test cases as you want in the TESTS array and the actual test code will remain very short, and easy to understand and maintain (note that when looping through test arrays, it's always a good idea to add a "desc" property that will be used in an info() log output). + +### Avoid exceptions + +Even when they're not failing the test, exceptions are bad because they pollute the logs and make them harder to read. +They're also bad because when your test is run as part of a test suite and if an other, unrelated, test fails then the exceptions may give wrong information to the person fixing the unrelated test. + +After your test has run locally, just make sure it doesn't output exceptions by scrolling through the logs. + +Often, non-blocking exceptions may be caused by hanging protocol requests that haven't been responded to yet when the tools get closed at the end of the test. Make sure you register to the right events and give time to the tools to update themselves before moving on. + +### Avoid test timeouts + +<!--TODO: this recommendation is conflicting with the above recommendation. What? --> +When tests fail, it's far better to have them fail and end immediately with an exception that will help fix it rather than have them hang until they hit the timeout and get killed. + +## Adding new helpers + +In some cases, you may want to extract some common code from your test to use it another another test. + +* If this is very common code that all tests could use, then add it to `devtools/client/shared/test/shared-head.js`. +* If this is common code specific to a given tool, then add it to the corresponding `head.js` file. +* If it isn't common enough to live in `head.js`, then it may be a good idea to create a helper file to avoid duplication anyway. Here's how to create a helper file: + * Create a new file in your test director. The naming convention should be `helper_<description_of_the_helper>.js` + * Add it to the browser.ini support-files section, making sure it is sorted alphabetically + * Load the helper file in the tests + * `browser/devtools/markupview/test/head.js` has a handy `loadHelperScript(fileName)` function that you can use. + * The file will be loaded in the test global scope, so any global function or variables it defines will be available (just like `head.js`). + * Use the special ESLint comment `/* import-globals-from helper_file.js */` to prevent ESLint errors for undefined variables. + +In all cases, new helper functions should be properly commented with an jsdoc comment block. + diff --git a/devtools/docs/tests/xpcshell.md b/devtools/docs/tests/xpcshell.md new file mode 100644 index 0000000000..1e51714a4f --- /dev/null +++ b/devtools/docs/tests/xpcshell.md @@ -0,0 +1,14 @@ +# Automated tests: `xpcshell` tests + +To run all of the xpcshell tests: + +```bash +./mach xpcshell-test --tag devtools +``` + +To run a specific xpcshell test: + +```bash +./mach xpcshell-test devtools/path/to/the/test_you_want_to_run.js +``` + diff --git a/devtools/docs/tools/console-panel.md b/devtools/docs/tools/console-panel.md new file mode 100644 index 0000000000..cc7c688e81 --- /dev/null +++ b/devtools/docs/tools/console-panel.md @@ -0,0 +1,159 @@ +# Console Tool Architecture
+
+The Console panel is responsible for rendering all logs coming from the current page.
+
+## Architecture
+
+Internal architecture of the Console panel (the client side) is described
+on the following diagram.
+
+<figure class="hero">
+ <pre class="diagram">
+┌──────────────────────────────┐ ┌────────────────────────┐
+│ DevTools │ │ WebConsolePanel │
+│[client/framework/devtools.js]│ │ [panel.js] │
+└──────────────────────────────┘ └────────────────────────┘
+ │ │
+ │
+ openBrowserConsole() or │
+ toggleBrowserConsole() │
+ │ │
+ ▼ │
+┌──────────────────────────────┐ {hud}
+│ BrowserConsoleManager │ │
+│ [browser-console-manager.js] │ │
+└──────────────────────────────┘ │
+ │ │
+ │ │
+ {_browserConsole} │
+ │ │
+ ▼ 0..1 ▼ 1
+┌──────────────────────────────┐ ┌────────────────────────┐
+│ BrowserConsole │ │ WebConsole │
+│ [browser-console.js] │─ ─ extends ─ ▶│ [webconsole.js] │
+└──────────────────────────────┘ └──────────────1─────────┘
+ │
+ {ui}
+ │
+ ▼ 1
+ ┌────────────────────────┐ ┌─────────────────────────────────┐
+ │ WebConsoleUI │ 1│ WebConsoleConnectionProxy │
+ │ [webconsole-ui.js] │───{proxy}──▶│[webconsole-connection-proxy.js] │
+ └────────────────────────┘ └─────────────────────────────────┘
+ │ │
+ {wrapper}
+ │ │
+ │
+ │ │
+ ▼ 1
+ ┌────────────────────────┐ │
+ │ WebConsoleWrapper │
+ │[webconsole-wrapper.js] │◀ ─ ─ ─calls methods from ─ ─ ┘
+ └────────────────────────┘
+ │
+ <renders>
+ │
+ ▼
+ ┌────────────────────────┐
+ │ App │
+ └────────────────────────┘
+ </pre>
+ <figcaption>Elements between curly bracket on arrows represent the property name of the reference (for example, the WebConsolePanel as a `hud` property that is a reference to the WebConsole instance)</figcaption>
+</figure>
+
+## Components
+
+The Console panel UI is built on top of [React](../frontend/react.md). It defines set of React components in `components` directory
+The React architecture is described on the following diagram.
+
+<figure class="hero">
+ <pre class="diagram">
+┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─
+ WebConsole React components │
+│ [/components] ┌────────────────────────┐
+ │ App │ │
+│ └────────────────────────┘
+ │ │
+│ │
+ ┌───────────────────┬──────────────────────┬───────────────────┬───────────┴─────────┬───────────────────────┬────────────────────┬─────────────────┐ │
+│ │ │ │ │ │ │ │ │
+ ▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼ │ ┌────────────────────────────────────────┐
+│ ┌──────────┐ ┌────────────────┐ ┌────────────────┐ ┌───────────┐ ┌────────────────────┐ ┌──────────────┐ ┌──────────────┐ ┌─────────┐ │ Editor │
+ │ SideBar │ │NotificationBox │ │ ConfirmDialog │ │ FilterBar │ │ ReverseSearchInput │ │ConsoleOutput │ │EditorToolbar │ │ JSTerm │──.editor───▶│ <CodeMirror> │
+│ └──────────┘ └────────────────┘ │ <portal> │ └───────────┘ └────────────────────┘ └──────────────┘ └──────────────┘ └─────────┘ │ [client/shared/sourceeditor/editor.js] │
+ │ └────────────────┘ │ │ │ └────────────────────────────────────────┘
+│ │ ┌─────────┴─────────────┐ │
+ │ │ │ │ │
+│ │ ▼ ▼ ▼
+ │ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │
+│ │ │ FilterButton │ │ FilterCheckbox │ │ MessageContainer │
+ │ └──────────────────┘ └──────────────────┘ └──────────────────┘ │
+│ │ │
+ │ │ │
+│ │ │
+ │ ▼ │
+│ │ ┌──────────────────┐
+ │ │ Message │ │
+│ │ └──────────────────┘
+ │ │ │ ┌─────────────────────────────────────┐
+│ │ │ │ Frame │
+ │ ┌─────────────────────┬─────────────────────┬─────────────────────┬───────┴─────────────┬─────────────────────┬─────────────┼─────┬──▶│ [client/shared/components/Frame.js] │
+│ │ │ │ │ │ │ │ │ └─────────────────────────────────────┘
+ │ ▼ ▼ ▼ ▼ ▼ ▼ │ │
+│ │ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │ ┌────────────────────────────────────────┐
+ │ │ MessageIndent │ │ MessageIcon │ │ CollapseButton │ │ GripMessageBody │ │ ConsoleTable │ │ MessageRepeat │ │ │ │ SmartTrace │
+│ │ └──────────────────┘ └──────────────────┘ └──────────────────┘ └──────────────────┘ └──────────────────┘ └──────────────────┘ ├──▶│[client/shared/components/SmartTrace.js]│
+ │ │ │ │ │ └────────────────────────────────────────┘
+└ ─ ─ ─ ┼ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─│─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─│─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │
+ │ │ │ │ ┌──────────────────────────────────────────────────┐
+ │ │ │ │ │ TabboxPanel │
+ │ ├─────────────────────┘ └──▶│[client/netmonitor/src/components/TabboxPanel.js] │
+ │ │ └──────────────────────────────────────────────────┘
+ │ │
+ │ │
+ │ ▼
+ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─
+ │ Reps ┌──────────────────────┐ │
+ │ │ [client/shared/components/reps/reps.js] │ ObjectInspector │
+ │ └──────────────────────┘ │
+ │ │ │
+ │ ▼ │
+ │ │ ┌──────────────────────┐
+ │ │ ObjectInspectorItem │ │
+ │ │ └──────────────────────┘
+ └───────────────────────────────────────────────────────────────▶ │ │
+ │ ▼
+ ┌──────────────────────┐ │
+ │ ┌─▶│ Rep │
+ │ └──────────────────────┘ │
+ │ │ │
+ │ │ │
+ │ └──────────────┘
+ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┘
+ </pre>
+</figure>
+
+There are several external components we use from the WebConsole:
+- ObjectInspector/Rep: Used to display a variable in the console output and handle expanding the variable when it's not a primitive.
+- Frame: Used to display the location of messages.
+- SmartTrace: Used to display the stack trace of messages and errors
+- TabboxPanel: Used to render a network message detail. This is directly using the component from the Netmonitor so we are consistent in how we display a request internals.
+
+## Actions
+
+The Console panel implements a set of actions divided into several groups.
+
+- **Filters** Actions related to content filtering.
+- **Messages** Actions related to list of messages rendered in the panel.
+- **UI** Actions related to the UI state.
+
+### State
+
+The Console panel manages the app state via [Redux](../frontend/redux.md).
+
+There are following reducers defining the panel state:
+
+- `reducers/filters.js` state for panel filters. These filters can be set from within the panel's toolbar (e.g. error, info, log, css, etc.)
+- `reducers/messages.js` state of all messages rendered within the panel.
+- `reducers/prefs.js` Preferences associated with the Console panel (e.g. logLimit)
+- `reducers/ui.js` UI related state (sometimes also called a presentation state). This state contains state of the filter bar (visible/hidden), state of the time-stamp (visible/hidden), etc.
diff --git a/devtools/docs/tools/debugger-panel.md b/devtools/docs/tools/debugger-panel.md new file mode 100644 index 0000000000..19d769eee8 --- /dev/null +++ b/devtools/docs/tools/debugger-panel.md @@ -0,0 +1 @@ +You can find the debugger documentation on [the debugger.html GitHub repository](https://devtools-html.github.io/debugger.html/).
\ No newline at end of file diff --git a/devtools/docs/tools/highlighters.md b/devtools/docs/tools/highlighters.md new file mode 100644 index 0000000000..65905dc46a --- /dev/null +++ b/devtools/docs/tools/highlighters.md @@ -0,0 +1,184 @@ +# Highlighters + +This article provides technical documentation about DevTools highlighters. + +By highlighter, we mean anything that DevTools displays on top of the content page, in order to highlight an element, a set of elements or shapes to users. + +The most obvious form of highlighter is the box-model highlighter, whose job is to display the 4 box-model regions on top of a given element in the content page, as illustrated in the following screen capture: + + + +But there can be a wide variety of highlighters. In particular, highlighters are a pretty good way to give detailed information about: + +* the exact form of a css shape, +* how a css transform applied to an element, +* which are all the elements that match a given selector, +* ... + +## Using highlighters + +Highlighters run on the debuggee side, not on the toolbox side. This is so that it's possible to highlight elements on a remote device for instance. This means you need to go through the [Remote Debugging Protocol](protocol.md) to use a highlighter. + +The InspectorFront provides the following method: + +| Method | Description | +|----------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `getHighlighterByType(typeName)` | Instantiate a new highlighter, given its type (as a String). At the time of writing, the available types of highlighters are: `CssGridHighlighter`, `BoxModelHighlighter`, `CssTransformHighlighter`, `FlexboxHighlighter`, `FontsHighlighter`, `GeometryEditorHighlighter`, `MeasuringToolHighlighter`, `PausedDebuggerOverlay`, `RulersHighlighter`, `SelectorHighlighter` and `ShapesHighlighter`. This returns a promise that resolves to the new instance of [protocol.js](https://wiki.mozilla.org/DevTools/protocol.js) actor. | + +### The highlighter API + +When getting a highlighter via `InspectorFront.getHighlighterByType(typeName)`, the right type of highlighter will be instantiated on the server-side and will be wrapped into a `CustomHighlighterActor` and that's what will be returned to the caller. This means that all types of highlighters share the same following API: + +| Method | Description | +|------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `show(NodeActor node[, Object options])` | Highlighters are hidden by default. Calling this method is what makes them visible. The first, mandatory, parameter should be a NodeActor. NodeActors are what the WalkerActor return. It's easy to get a NodeActor for an existing DOM node. For example `toolbox.walker.querySelector(toolbox.walker.rootNode, "css selector")` resolves to a NodeFront (the client-side version of the NodeActor) which can be used as the first parameter. The second, optional, parameter depends on the type of highlighter being used. | +| `hide()` | Hides the highlighter. | +| `finalize()` | Destroys the highlighter. | + +## Creating new highlighters + +Before digging into how one goes about adding a new type of highlighter to the DevTools, it is worth understanding how are highlighters displayed in the page. + +### Inserting content in the page + +Highlighters use web technology themselves to display the required information on screen. For instance, the box-model highlighter uses SVG to draw the margin, border, padding and content regions over the highlighted node. + +This means the highlighter content needs to be inserted in the page, but in a non-intrusive way. Indeed, the DevTools should never alter the page unless the alteration was done by the user (like changing the DOM using the inspector or a CSS rule via the style-editor for example). So simply appending the highlighter's markup in the content document is not an option. + +Furthermore, highlighters not only need to work with Firefox Desktop, but they should work just as well on Firefox OS, Firefox for Android, and more generally anything that runs the Gecko rendering engine. Therefore appending the highlighter's markup to the browser chrome XUL structure isn't an option either. + +To this end, DevTools highlighters make use of a (chrome-only) API: + +``` + /** + * Chrome document anonymous content management. + * This is a Chrome-only API that allows inserting fixed positioned anonymous + * content on top of the current page displayed in the document. + * The supplied content is cloned and inserted into the document's CanvasFrame. + * Note that this only works for HTML documents. + */ + partial interface Document { + /** + * Deep-clones the provided element and inserts it into the CanvasFrame. + * Returns an AnonymousContent instance that can be used to manipulate the + * inserted element. + */ + [ChromeOnly, NewObject, Throws] + AnonymousContent insertAnonymousContent(Element aElement); + + /** + * Removes the element inserted into the CanvasFrame given an AnonymousContent + * instance. + */ + [ChromeOnly, Throws] + void removeAnonymousContent(AnonymousContent aContent); + }; +``` + +Using this API, it is possible for chrome-privileged JS to insert arbitrary DOM elements on top of the content page. + +Technically, the DOM element is inserted into the `CanvasFrame` of the document. The `CanvasFrame` is part of the rendered frame tree and the DOM element is part of the native anonymous elements of the `CanvasFrame`. + +Consider the following simple example: + +```js + let el = document.createElement("div"); + el.textContent = "My test element"; + let insertedEl = document.insertAnonymousContent(el); +``` + +In this example, the test DIV will be inserted in the page, and will be displayed on top of everything else, in a way that doesn't impact the current layout. + +### The AnonymousContent API + +In the previous example, the returned `insertedEl` object isn't a DOM node, and it certainly is not `el`. It is a new object, whose type is `AnonymousContent` ([see the WebIDL here](https://searchfox.org/mozilla-central/source/dom/webidl/AnonymousContent.webidl)). + +Because of the way content is inserted into the page, it isn't wanted to give consumers a direct reference to the inserted DOM node. This is why `document.insertAnonymousContent(el)` actually **clones** `el` and returns a new object whose API lets consumers make changes to the inserted element in a way that never gives back a reference to the inserted DOM node. + +### CanvasFrameAnonymousContentHelper + +In order to help with the API described in the previous section, the `CanvasFrameAnonymousContentHelper` class was introduced. + +Its goal is to provide a simple way for highlighters to insert their content into the page and modify it dynamically later. One of its goal is also to re-insert the highlighters' content on page navigation. Indeed, the frame tree is destroyed when the page is navigated away from since it represents the document element. One thing to note is that highlighter content insertion is asynchronous and `CanvasFrameAnonymousContentHelper` users must call and wait for its `initialize` method to resolve. + +Using this helper is quite simple: + +```js +let helper = new CanvasFrameAnonymousContentHelper(targetActor, this.buildMarkup.bind(this)); +await helper.initialize(); +``` + +It only requires a `targetActor`, which highlighters get when they are instantiated, and a callback function that will be used to create and insert the content the first time the highlighter is shown, and every time there's a page navigation. + +The returned object provides the following API: + +| Method | Description | +|-------------------------------------------|------------------------------------------------------------| +| `getTextContentForElement(id)` | Get the textContent of an element given its ID. | +| `setTextContentForElement(id, text)` | Set the textContent of an element given its ID. | +| `setAttributeForElement(id, name, value)` | Set an attribute value of an element given its ID. | +| `getAttributeForElement(id, name)` | Get an attribute value of an element given its ID. | +| `removeAttributeForElement(id, name)` | Remove an attribute of an element given its ID. | +| `content` | This property returns the wrapped AnonymousContent object. | +| `destroy()` | Destroy the helper instance. | + + ### Creating a new highlighter class + +A good way to get started is by taking a look at [existing highlighters here](https://searchfox.org/mozilla-central/rev/1a973762afcbc5066f73f1508b0c846872fe3952/devtools/server/actors/highlighters.js#519-530). + +Here is some boilerplate code for a new highlighter class: + +```js + function MyNewHighlighter(targetActor) { + this.doc = targetActor.window.document; + this.markup = new CanvasFrameAnonymousContentHelper(targetActor, this._buildMarkup.bind(this)); + this.markup.initialize(); + } + + MyNewHighlighter.prototype = { + destroy: function() { + this.doc = null; + this.markup.destroy(); + }, + + _buildMarkup: function() { + let container = this.markup.anonymousContentDocument.createElement("div"); + container.innerHTML = '<div id="new-highlighted-" style="display:none;">'; + return container; + }, + + show: function(node, options) { + this.markup.removeAttributeForElement("new-highlighted-el", "style"); + }, + + hide: function() { + this.markup.setAttributeForElement("new-highlighted-el", "style", "display:none;"); + } + }; +``` + +In most situations, the `container` returned by `_buildMarkup` will be absolutely positioned, and will need to contain elements with IDs, so that these can then later be moved, resized, hidden or shown in `show` and `hide` using the AnonymousContent API. + +### The AutoRefreshHighlighter parent class + +It is worth mentioning this class as it may be a useful parent class to inherit a new highlighter from in some situations. + +If the new highlighter's job is to highlight an element in the DOM, then it most likely should inherit from `AutoRefreshHighlighter`. + +The `AutoRefreshHighlighter` class updates itself in a loop, checking if the currently highlighted node's geometry has changed since the last iteration. This is useful to make sure the highlighter **follows** the highlighted node around, in case the layout around it changes, or in case it is an animated node. + +Sub classes must implement the following methods: + +| Method | Description | +|-------------|-------------------------------------------------------------------------------------| +| `_show()` | Called when the highlighter should be shown. | +| `_update()` | Called while the highlighter is shown and the geometry of the current node changes. | +| `_hide()` | Called when the highlighter should be hidden. | + +Sub classes will have access to the following properties: + +| Property | Description | +|---------------------|-------------------------------------------| +| `this.currentNode` | The node to be shown. | +| `this.currentQuads` | All of the node's box model region quads. | +| `this.win` | The current window | diff --git a/devtools/docs/tools/inspector-panel.md b/devtools/docs/tools/inspector-panel.md new file mode 100644 index 0000000000..7aaf20eb27 --- /dev/null +++ b/devtools/docs/tools/inspector-panel.md @@ -0,0 +1,93 @@ +# High-Level Inspector Architecture
+
+## UI structure
+The Inspector panel is a tab in the toolbox. Like all tabs, it's in its own iframe.
+
+The high-level hierarchy looks something like this:
+
+ Toolbox
+ |
+ InspectorPanel
+ |
+ +-------------+------------------+---------------+
+ | | | |
+ MarkupView SelectorSearch HTMLBreadcrumbs ToolSidebar widget (iframes)
+ |
+ +- RuleView
+ |
+ +- ComputedView
+ |
+ +- LayoutView
+ |
+ +- FontInspector
+ |
+ +- AnimationInspector
+
+## Server dependencies
+- The inspector panel relies on a series of actors that live on the server.
+- Some of the most important actors are actually instantiated by the toolbox, because these actors are needed for other panels to preview and link to DOM nodes. For example, the webconsole needs to output DOM nodes, highlight them in the page on mouseover, and open them in the inspector on click. This is achieved using some of the same actors that the inspector panel uses.
+- See Toolbox.prototype.initInspector: This method instantiates the InspectorActor, WalkerActor and HighlighterActor lazily, whenever they're needed by a panel.
+
+## Panel loading overview
+- As with other panels, this starts with Toolbox.prototype.loadTool(id)
+- For the inspector though, this calls Toolbox.prototype.initInspector
+- When the panel's open method is called:
+ - It uses the WalkerActor for the first time to know the default selected node (which could be a node that was selected before on the same page).
+ - It starts listening to the WalkerActor's "new-root" events to know when to display a new DOM tree (when there's a page navigation).
+ - It creates the breadcrumbs widget, the sidebar widget, the search widget, the markup-view
+- Sidebar:
+ - When this widget initializes, it loads its sub-iframes (rule-view, ...)
+ - Each of these iframes contain panel that, in turn, listen to inspector events like "new-node-front" to know when to retrieve information about a node (i.e the rule-view will fetch the css rules for a node).
+- Markup-view:
+ - This panel initializes in its iframe, and gets a reference to the WalkerActor. It uses it to know the DOM tree to display. It knows when nodes change (markup-mutations), and knows what root node to start from.
+ - It only displays the nodes that are supposed to be visible at first (usually html, head, body and direct children of body).
+ - Then, as you expand nodes, it uses the WalkerActor to get more nodes lazily. It only ever knows data about nodes that have already been expanded once in the UI.
+
+## Server-side structure
+Simplified actor hierarchy
+
+ InspectorActor
+ |
+ +---------------+
+ | |
+ WalkerActor PageStyleActor (for rule-view/computed-view)
+ | |
+ NodeActor StyleRuleActor
+
+__InspectorActor__
+
+This tab-level actor is the one the inspector-panel connects to. It doesn't do much apart from creating and returning the WalkerActor and PageStyleActor.
+
+__WalkerActor__
+
+- Single most important part of the inspector panel.
+- Responsible for walking the DOM on the current page but:
+ - also walks iframes
+ - also finds pseudo-elements ::before and ::after
+ - also finds anonymous content (e.g. in the BrowserToolbox)
+- The actor uses an instance of inIDeepTreeWalker to walk the DOM
+- Provides a tree of NodeActor objects that reflects the DOM.
+- But only has a partial knowledge of the DOM (what is currently displayed/expanded in the MarkupView). It doesn't need to walk the whole tree when you first instantiate it.
+- Reflects some of the usual DOM APIs like querySelector.
+- Note that methods like querySelector return arbitrarily nested NodeActors, in which case the WalkerActor also sends the list of parents to link the returned nodes to the closest known nodes, so the UI can display the tree correctly.
+- Emits events when there are DOM mutations. These events are sent to the front-end and used to, for example refresh the markup-view. This uses an instance of MutationObserver (https://developer.mozilla.org/en-US/docs/Web/API/MutationObserver) configured with, in particular, nativeAnonymousChildList set to true, so that mutation events are also sent when pseudo elements are added/removed via css.
+
+__NodeActor__
+
+- Representation of a single DOM node (tagname, namespace, attributes, parent, sibblings, ...), which panels use to display previews of nodes.
+- Also provide useful methods to:
+ - change attributes
+ - scroll into view
+ - get event listeners data
+ - get image data
+ - get unique css selector
+
+## Highlighters
+
+One of the important aspects of the inspector is the highlighters.
+You can find a lot more [documentation about highlighters here](highlighters.md).
+
+We don't just have 1 highlighter, we have a framework for highlighters:
+- a (chrome-only) platform API to inject markup in a native-anonymous node in content (that works on all targets)
+- a number of specific highlighter implementations (css transform, rect, selector, geometry, rulers, ...)
+- a CustomHighlighterActor to get instances of specific highlighters
diff --git a/devtools/docs/tools/inspector.md b/devtools/docs/tools/inspector.md new file mode 100644 index 0000000000..6a3a223da5 --- /dev/null +++ b/devtools/docs/tools/inspector.md @@ -0,0 +1,2 @@ + +These files provide information about the Inspector panel architecture. diff --git a/devtools/docs/tools/memory-panel.md b/devtools/docs/tools/memory-panel.md new file mode 100644 index 0000000000..6e1bdbc531 --- /dev/null +++ b/devtools/docs/tools/memory-panel.md @@ -0,0 +1,226 @@ +# Memory Tool Architecture + +The memory tool is built of three main elements: + +1. The live heap graph exists in memory, and is managed by the C++ allocator and + garbage collector. In order to get access to the structure of this graph, a + specialized interface is created to represent its state. The `JS::ubi::Node` + is the basis for this representation. This interface can be created from the + live heap graph, or a serialized, offline snapshot from a previous moment in + time. Our various heap analyses (census, dominator trees, shortest paths, + etc) run on top of `JS::ubi::Node` graphs. The `ubi` in the name stands for + "ubiquitous" and provides a namespace for memory analyses in C++ code. + +2. The `HeapAnalysesWorker` runs in a worker thread, performing analyses on + snapshots and translating the results into something the frontend can render + simply and quickly. The `HeapAnalysesClient` is used to communicate between + the worker and the main thread. + +3. Finally, the last element is the frontend that renders data received from the + `HeapAnalysesClient` to the DOM and translates user input into requests for + new data with the `HeapAnalysesClient`. + +Unlike other tools (such as the JavaScript debugger), the memory tool makes very +little use of the Remote DevTools Server and the actors that reside in it. Use +of the [`MemoryActor`](devtools/server/actors/memory.js) is limited to toggling +allocation stack recording on and off, and transferring heap snapshots from the +debuggee (which is on the server) to the `HeapAnalysesWorker` (which is on the +client). A nice benefit that naturally emerges, is that supporting "legacy" +servers (eg, using Firefox Developer Edition as a client to remote debug a +release Firefox for Android server) is a no-op. As we add new analyses, we can +run them on snapshots taken on old servers no problem. The only requirement is +that changes to the snapshot format itself remain backwards compatible. + +## `JS::ubi::Node` + +`JS::ubi::Node` is a lightweight serializable interface that can represent the +current state of the heap graph. For a deeper dive into the particulars of how +it works, it is very well documented in the `js/public/UbiNode.h` + +A "heap snapshot" is a representation of the heap graph at some particular past +instance in time. + +A "heap analysis" is an algorithm that runs on a `JS::ubi::Node` heap graph. +Generally, analyses can run on either the live heap graph or a deserialized +snapshot. Example analyses include "census", which aggregates and counts nodes +into various user-specified buckets; "dominator trees", which compute the +[dominates](https://en.wikipedia.org/wiki/Dominator_%28graph_theory%29) relation +and retained size for all nodes in the heap graph; and "shortest paths" which +finds the shortest paths from the GC roots to some subset of nodes. + +### Saving Heap Snapshots + +Saving a heap snapshot has a few requirements: + +1. The binary format must remain backwards compatible and future extensible. + +2. The live heap graph must not mutate while we are in the process of + serializing it. + +3. The act of saving a heap snapshot should impose as little memory overhead as + possible. If we are taking a snapshot to debug frequent out-of-memory errors, + we don't want to trigger an OOM ourselves! + +To solve (1), we use the [protobuf](https://developers.google.com/protocol-buffers/) +message format. The message definitions themselves are in +`devtools/shared/heapsnapshot/CoreDump.proto`. We always use `optional` fields +so we can change our mind about what fields are required sometime in the future. +Deserialization checks the semantic integrity of deserialized protobuf messages. + +For (2), we rely on SpiderMonkey's GC rooting hazard static analysis and the +`AutoCheckCannotGC` dynamic analysis to ensure that neither JS nor GC runs and +modifies objects or moves them from one address in memory to another. There is +no equivalent suppression and static analysis technique for the +[cycle collector](https://developer.mozilla.org/en/docs/Interfacing_with_the_XPCOM_cycle_collector), +so care must be taken not to invoke methods that could start cycle collection or +mutate the heap graph from the cycle collector's perspective. At the time of +writing, we don't yet support saving the cycle collector's portion of the heap +graph in snapshots, but that work is deemed Very Important and Very High +Priority. + +Finally, (3) imposes upon us that we do not build the serialized heap snapshot +binary blob in memory, but instead stream it out to disk while generating it. + +Once all of that is accounted for, saving snapshots becomes pretty straight +forward. We traverse the live heap graph with `JS::ubi::Node` and +`JS::ubi::BreadthFirst`, create a protobuf message for each node and each node's +edges, and write these messages to disk before continuing the traversal to the +next node. + +This functionality is exposed to chrome JavaScript as the +`ChromeUtils.saveHeapSnapshot` function. See `dom/webidl/ChromeUtils.webidl` for +API documentation. + +### Reading Heap Snapshots + +Reading heap snapshots has less restrictions than saving heap snapshots. The +protobuf messages that make up the core dump are deserialized one by one, stored +as a set of `DeserializedNode`s and a set of `DeserializedEdge`s, and the result +is a `HeapSnapshot` instance. + +The `DeserializedNode` and `DeserializedEdge` classes implement the +`JS::ubi::Node` interface. Analyses running on offline heap snapshots rather +than the live heap graph operate on these classes (unknowingly, of course). + +For more details, see the +[`mozilla::devtools::HeapSnapshot`](devtools/shared/heapsnapshot/HeapSnapshot.cpp) +and +[`mozilla::devtools::Deserialized{Node,Edge}`](devtools/shared/heapsnapshot/DeserializedNode.h) +classes. + +### Heap Analyses + +Heap analyses operate on `JS::ubi::Node` graphs without knowledge of whether +that graph is backed by the live heap graph or an offline heap snapshot. They +must make sure never to allocate GC things or modify the live heap graph. + +In general, analyses are implemented in their own `js/public/Ubi{AnalysisName}.h` +header (eg `js/public/UbiCensus.h`), and are exposed to chrome JavaScript code +via a method on the [`HeapSnapshot`](dom/webidl/HeapSnapshot.webidl) webidl +interface. + +For each analysis we expose to chrome JavaScript on the `HeapSnapshot` webidl +interface, there is a small amount of glue code in Gecko. The +[`mozilla::devtools::HeapSnapshot`](devtools/shared/heapsnapshot/HeapSnapshot.h) +C++ class implements the webidl interface. The analyses methods (eg +`ComputeDominatorTree`) take the deserialized nodes and edges from the heap +snapshot, create `JS::ubi::Node`s from them, call the analyses from +`js/public/Ubi*.h`, and wrap the results in something that can be represented in +JavaScript. + +For API documentation on running specific analyses, see the +[`HeapSnapshot`](dom/webidl/HeapSnapshot.webidl) webidl interface. + +### Testing `JS::ubi::Node`, Snapshots, and Analyses + +The majority of the tests reside within `devtools/shared/heapsnapshot/tests/**`. +For reading and saving heap snapshots, most tests are gtests. The gtests can be +run with the `mach gtest DevTools.*` command. The rest are integration sanity +tests to make sure we can read and save snapshots in various environments, such +as xpcshell or workers. These can be run with the usual `mach test $PATH` +commands. + +There are also `JS::ubi::Node` related unit tests in +`js/src/jit-test/tests/heap-analysis/*`, `js/src/jit-test/tests/debug/Memory-*`, +and `js/src/jsapi-tests/testUbiNode.cpp`. See +https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/Running_Automated_JavaScript_Tests#Running_jit-tests +for running the JIT tests. + +## `HeapAnalysesWorker` + +The `HeapAnalysesWorker` orchestrates running specific analyses on snapshots and +transforming the results into something that can simply and quickly be rendered +by the frontend. The analyses can take some time to run (sometimes on the order +of seconds), so doing them in a worker thread allows the interface to stay +responsive. The `HeapAnalysisClient` provides the main thread's interface to the +worker. + +The `HeapAnalysesWorker` doesn't actually do much itself; mostly just shuffling +data and transforming it from one representation to another or calling C++ +utility functions exposed by webidl that do those things. Most of these are +implemented as traversals of the resulting census or dominator trees. + +See the following files for details on the various data transformations and +shuffling that the `HeapAnalysesWorker` delegates to. + +* `devtools/shared/heapsnapshot/CensusUtils.js` +* `devtools/shared/heapsnapshot/CensusTreeNode.js` +* `devtools/shared/heapsnapshot/DominatorTreeNode.js` + +### Testing the `HeapAnalysesWorker` and `HeapAnalysesClient` + +Tests for the `HeapAnalysesWorker` and `HeapAnalysesClient` reside in +`devtools/shared/heapsnapshot/tests/**` and can be run with the usual `mach test +$PATH` command. + +## Frontend + +The frontend of the memory tool is built with React and Redux. + +[React has thorough documentation.](https://facebook.github.io/react/) + +[Redux has thorough documentation.](http://rackt.org/redux/index.html) + +We have React components in `devtools/client/memory/components/*`. + +We have Redux reducers in `devtools/client/memory/reducers/*`. + +We have Redux actions and action-creating tasks in +`devtools/client/memory/actions/*`. + +React components should be pure functions from their props to the rendered +(virtual) DOM. Redux reducers should also be observably pure. + +Impurity within the frontend is confined to the tasks that are creating and +dispatching actions. All communication with the outside world (such as the +`HeapAnalysesWorker`, the Remote DevTools Server, or the file system) is +restricted to within these tasks. + +### Snapshots State + +On the JavaScript side, the snapshots represent a reference to the underlying +heap dump and the various analyses. The following diagram represents a finite +state machine describing the snapshot states. Any of these states may go to the +ERROR state, from which they can never leave. + +``` +SAVING → SAVED → READING → READ + ↗ + IMPORTING +``` + +Each of the report types (census, diffing, tree maps, dominators) have their own states as well, and are documented at `devtools/client/memory/constants.js`. +These report states are updated as the various filtering and selecting options +are updated in the UI. + +### Testing the Frontend + +Unit tests for React components are in `devtools/client/memory/test/chrome/*`. + +Unit tests for actions, reducers, and state changes are in +`devtools/client/memory/test/xpcshell/*`. + +Holistic integration tests for the frontend and the whole memory tool are in +`devtools/client/memory/test/browser/*`. + +All tests can be run with the usual `mach test $PATH` command. diff --git a/devtools/docs/tools/responsive-design-mode.md b/devtools/docs/tools/responsive-design-mode.md new file mode 100644 index 0000000000..fc348fb28b --- /dev/null +++ b/devtools/docs/tools/responsive-design-mode.md @@ -0,0 +1,70 @@ +# Responsive Design Mode Architecture + +## Context + +You have a single browser tab that has visited several pages, and now has a +history that looks like, in oldest to newest order: + +1. https://newsblur.com +2. https://mozilla.org (← current page) +3. https://convolv.es + +## Opening RDM During Current Firefox Session + +When opening RDM, the browser tab's history must preserved. Additionally, we +strive to preserve the exact state of the currently displayed page (effectively +any in-page state, which is important for single page apps where data can be +lost if they are reloaded). + +This seems a bit convoluted, but one advantage of this technique is that it +preserves tab state since the same tab is reused. This helps to maintain any +extra state that may be set on tab by add-ons or others. + +1. Create a temporary, hidden tab to load the tool UI. +2. Mark the tool tab browser's docshell as active so the viewport frame is + created eagerly and will be ready to swap. +3. Create the initial viewport inside the tool UI. +4. Swap tab content from the regular browser tab to the browser within the + viewport in the tool UI, preserving all state via + `gBrowser._swapBrowserDocShells`. +5. Force the original browser tab to be non-remote since the tool UI must be + loaded in the parent process, and we're about to swap the tool UI into + this tab. +6. Swap the tool UI (with viewport showing the content) into the original + browser tab and close the temporary tab used to load the tool via + `swapBrowsersAndCloseOther`. +7. Start a tunnel from the tool tab's browser to the viewport browser + so that some browser UI functions, like navigation, are connected to + the content in the viewport, instead of the tool page. + +## Closing RDM During Current Firefox Session + +To close RDM, we follow a similar process to the one from opening RDM so we can +restore the content back to a normal tab. + +1. Stop the tunnel between outer and inner browsers. +2. Create a temporary, hidden tab to hold the content. +3. Mark the content tab browser's docshell as active so the frame is created + eagerly and will be ready to swap. +4. Swap tab content from the browser within the viewport in the tool UI to the + regular browser tab, preserving all state via + `gBrowser._swapBrowserDocShells`. +5. Force the original browser tab to be remote since web content is loaded in + the child process, and we're about to swap the content into this tab. +6. Swap the content into the original browser tab and close the temporary tab + used to hold the content via `swapBrowsersAndCloseOther`. + +## Session Restore + +When restarting Firefox and restoring a user's browsing session, we must +correctly restore the tab history. If the RDM tool was opened when the session +was captured, then it would be acceptable to either: + +* A: Restore the tab content without any RDM tool displayed **OR** +* B: Restore the RDM tool the tab content inside, just as before the restart + +We currently follow path A (no RDM after session restore), which seems more in +line with how the rest of DevTools currently functions after restore. To do so, +we watch for `beforeunload` events on the tab at shutdown and quickly exit RDM +so that session restore records only the original page content during its final +write at shutdown. |
