diff options
| author | Daniel Baumann <daniel.baumann@progress-linux.org> | 2024-04-07 17:32:43 +0000 |
|---|---|---|
| committer | Daniel Baumann <daniel.baumann@progress-linux.org> | 2024-04-07 17:32:43 +0000 |
| commit | 6bf0a5cb5034a7e684dcc3500e841785237ce2dd (patch) | |
| tree | a68f146d7fa01f0134297619fbe7e33db084e0aa /docs/contributing | |
| parent | Initial commit. (diff) | |
| download | thunderbird-6bf0a5cb5034a7e684dcc3500e841785237ce2dd.tar.xz thunderbird-6bf0a5cb5034a7e684dcc3500e841785237ce2dd.zip | |
Adding upstream version 1:115.7.0.upstream/1%115.7.0upstream
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'docs/contributing')
59 files changed, 6440 insertions, 0 deletions
diff --git a/docs/contributing/Code_Review_FAQ.rst b/docs/contributing/Code_Review_FAQ.rst new file mode 100644 index 0000000000..18fe85a6e2 --- /dev/null +++ b/docs/contributing/Code_Review_FAQ.rst @@ -0,0 +1,93 @@ +Code Review FAQ +=============== + +What is the purpose of code review? +----------------------------------- + +Code review is our basic mechanism for validating the design and +implementation of patches. It also helps us maintain a level of +consistency in design and implementation practices across the many +hackers and among the various modules of Mozilla. + +Of course, code review doesn't happen instantaneously, and so there is +some latency built into the system. We're always looking for ways to +reduce the wait, while simultaneously allowing reviewers to do a good +chunk of hacking themselves. We don't have a perfect system, and we +never will. It's still evolving, so let us know if you have suggestions. + +Mozilla used to have the concept of "super-review", but `a consensus was +reached in +2018 <https://groups.google.com/forum/#!topic/mozilla.governance/HHU0h-44NDo>`__ +to retire this process. + +Who must review my code? +------------------------ + +You must have an approval ("r={{ mediawiki.external('name') }}") from +the module owner or designated "peer" of the module where the code will +be checked in. If your code affects several modules, then generally you +should have an "r={{ mediawiki.external('name') }}" from the owner or +designated peer of each affected module. We try to be reasonable here, +so we don't have an absolute rule on when every module owner must +approve. For example, tree-wide changes such as a change to a string +class or a change to text that is displayed in many modules generally +doesn't get reviewed by every module owner. + +You may wish to ask others as well. + + +What do reviewers look for? +--------------------------- + +A review is focused on a patch's design, implementation, usefulness in +fixing a stated problem, and fit within its module. A reviewer should be +someone with domain expertise in the problem area. A reviewer may also +utilize other areas of his or her expertise and comment on other +possible improvements. There are no inherent limitations on what +comments a reviewer might make about improving the code. + +Reviewers will probably look at the following areas of the code: + +- “goal” review: is the issue being fixed actually a bug? Does the + patch fix the fundamental problem? +- API/design review. Because APIs define the interactions between + modules, they need special care. Review is especially important to + keep APIs balanced and targeted, and not too specific or + overdesigned. There are a `WebIDL review + checklist <https://wiki.mozilla.org/WebAPI/WebIDL_Review_Checklist>`__. + There are also templates for emails that should be sent when APIs are + going to be exposed to the Web and general guidance around naming on + `this wiki + page <https://wiki.mozilla.org/WebAPI/ExposureGuidelines>`__. +- Maintainability review. Code which is unreadable is impossible to + maintain. If the reviewer has to ask questions about the purpose of a + piece of code, then it is probably not documented well enough. Does + the code follow the :ref:`Coding style` ? Be careful when + reviewing code using modern C++ features like auto. +- Security review. Does the design use security concepts such as input + sanitizers, wrappers, and other techniques? Does this code need + additional security testing such as fuzz-testing or static analysis? +- Integration review. Does this code work properly with other modules? + Is it localized properly? Does it have server dependencies? Does it + have user documentation? +- Testing review. Are there tests for correct function? Are there tests + for error conditions and incorrect inputs which could happen during + operation? +- Performance review. Has this code been profiled? Are you sure it's + not negatively affecting performance of other code? +- License review. Does the code follow the `code licensing + rules <http://www.mozilla.org/hacking/committer/committers-agreement.pdf>`__? + + +How can I tell the status of reviews? +------------------------------------- + +When a patch has passed review you'll see "Accepted" in green at the top +of a Phabricator revision, under the title. In Bugzilla (which is +deprecated in favour of Phabricator), this is indicated by "{{ +mediawiki.external('name') }}:review+" in the attachment table in the +bug report. If it has failed review then you'll see "Needs Revision" in +red at the top of the revision, or, in Bugzilla, "{{ +mediawiki.external('name') }}:review-". Most of the time that a reviewer +sets a review flag, they will also add a comment to the bug explaining +the review. diff --git a/docs/contributing/build/artifact_builds.rst b/docs/contributing/build/artifact_builds.rst new file mode 100644 index 0000000000..7a92a559f6 --- /dev/null +++ b/docs/contributing/build/artifact_builds.rst @@ -0,0 +1,173 @@ +Understanding Artifact Builds +============================= + +Firefox for Desktop and Android supports a **fast build mode** called +*artifact mode*. The resulting builds are called *artifact builds*. +Artifact mode downloads pre-built C++ components rather than building them +locally, trading bandwidth for time. + +Artifact builds will be useful to many developers who are not working +with compiled code (see "Restrictions" below). Artifacts are typically +fetched from `mozilla-central <https://hg.mozilla.org/mozilla-central/>`__. + +To automatically download and use pre-built binary artifacts, add the +following lines into your :ref:`mozconfig <Configuring Build Options>` +file: + +.. code-block:: shell + + # Automatically download and use compiled C++ components: + ac_add_options --enable-artifact-builds + + # Write build artifacts to: + mk_add_options MOZ_OBJDIR=./objdir-frontend + +To automatically download and use the debug version of the pre-built +binary artifact (currently supported for Linux, OSX and Windows +artifacts), add ``ac_add_options --enable-debug`` to your mozconfig file +(with artifact builds option already enabled): + +.. code-block:: shell + + # Enable debug versions of the pre-built binary artifacts: + ac_add_options --enable-debug + + # Automatically download and use compiled C++ components: + ac_add_options --enable-artifact-builds + + # Download debug info so that stack traces refers to file and columns rather than library and Hex address + ac_add_options --enable-artifact-build-symbols + + # Write build artifacts to: + mk_add_options MOZ_OBJDIR=./objdir-frontend-debug-artifact + + +Prerequisites +------------- + +Artifact builds are supported for users of Mercurial and Git. Git +artifact builds require a mozilla-central clone made with the help of +`git-cinnabar <https://github.com/glandium/git-cinnabar>`__. Please +follow the instructions on the git-cinnabar project page to install +git-cinnabar. Further information about using git-cinnabar to interact +with Mozilla repositories can be found on `the project +wiki <https://github.com/glandium/git-cinnabar/wiki/Mozilla:-A-git-workflow-for-Gecko-development>`__. + +Building +-------- + +If you've added ``--enable-artifact-builds`` to your ``mozconfig``, each +time you run ``mach build`` and ``mach build path/to/subdirectory`` the +build system will determine what the best pre-built binary artifacts +available are, download them, and put them in place for you. The +computations are cached, so the additional calculations should be very +fast after the up-to-date artifacts are downloaded -- just a second or +two on modern hardware. Most Desktop developers should find that + +.. code-block:: shell + + ./mach build + ./mach run + +just works. + +To only rebuild local changes (to avoid re-checking for pushes and/or +unzipping the downloaded cached artifacts after local commits), you can +use: + +.. code-block:: shell + + ./mach build faster + +which only "builds" local JS, CSS and packaged (e.g. images and other +asset) files. + +Most Firefox for Android developers should find that + +.. code-block:: shell + + ./mach build + ./mach package + ./mach install + +just works. + +Pulling artifacts from a try build +---------------------------------- + +To only accept artifacts from a specific revision (such as a try build), +set ``MOZ_ARTIFACT_REVISION`` in your environment to the value of the +revision that is at the head of the desired push. Note that this will +override the default behavior of finding a recent candidate build with +the required artifacts, and will cause builds to fail if the specified +revision does not contain the required artifacts. + +Restrictions +------------ + +Oh, so many. Artifact builds are rather delicate: any mismatch between +your local source directory and the downloaded binary artifacts can +result in difficult to diagnose incompatibilities, including unexplained +crashes and catastrophic XPCOM initialization and registration +failures. These are rare, but do happen. + +Things that are supported +------------------------- + +- Modifying JavaScript, (X)HTML, and CSS resources; and string + properties and FTL files. +- Modifying Android Java code, resources, and strings. +- Running mochitests and xpcshell tests. +- Modifying ``Scalars.yaml`` to add Scalar Telemetry (since {{ + Bug("1425909") }}, except artifact builds on try). +- Modifying ``Events.yaml`` to add Event Telemetry (since {{ + Bug("1448945") }}, except artifact builds on try). + +Essentially everything updated by ``mach build faster`` should work with +artifact builds. + +Things that are not supported +----------------------------- + +- Support for products other than Firefox for Desktop and + Android are not supported and are unlikely to ever be supported. + Other projects like Thunderbird may provide + `their own support <https://developer.thunderbird.net/thunderbird-development/building-thunderbird/artifact-builds>`__ + for artifact builds. +- You cannot modify C, C++, or Rust source code anywhere in the tree. + If it’s compiled to machine code, it can't be changed. +- You cannot modify ``histograms.json`` to add Telemetry histogram + definitions.(But see `Bug 1206117 <https://bugzilla.mozilla.org/show_bug.cgi?id=1206117>`__). +- Modifying build system configuration and definitions does not work in + all situations. + +Things that are not **yet** supported +------------------------------------- + +- Tests other than mochitests, xpcshell, and Marionette-based tests. + There aren’t inherent barriers here, but these are not known to work. +- Modifying WebIDL definitions, even ones implemented in JavaScript. + +Troubleshooting +--------------- + +There are two parts to artifact mode: +the ``--disable-compile-environment`` option, and the ``mach artifact`` +command that implements the downloading and caching. Start by running + +.. code-block:: shell + + ./mach artifact install --verbose + +to see what the build system is trying to do. There is some support for +querying and printing the cache; run ``mach artifact`` to see +information about those commands. + +Downloaded artifacts are stored in +``$MOZBUILD_STATE_PATH/package-frontend``, which is almost always +``~/.mozbuild/package-frontend``. + +Discussion is best started on the `dev-builds mailing +list <https://lists.mozilla.org/listinfo/dev-builds>`__. Questions are +best raised in `#build <https://chat.mozilla.org/#/room/#build:mozilla.org>`__ on `Matrix <https://chat.mozilla.org/>`__. Please +file bugs in *Firefox Build System :: General*, blocking `Bug 901840 <https://bugzilla.mozilla.org/show_bug.cgi?id=901840>`__ diff --git a/docs/contributing/build/building_mobile_firefox.rst b/docs/contributing/build/building_mobile_firefox.rst new file mode 100644 index 0000000000..90e922af8a --- /dev/null +++ b/docs/contributing/build/building_mobile_firefox.rst @@ -0,0 +1,34 @@ +Firefox for Mobile Devices +-------------------------- + +We have several different mobile products aimed at different tasks, +devices, and audiences: + +- Building **Firefox for Android** (codename: fenix). Our general-purpose + mobile browser is split into several different artifact layers: + + - `The fenix Android application </mobile/android/fenix.html>`_ + - `The android-components Android library <https://github.com/mozilla-mobile/firefox-android/tree/main/android-components>`_ + - `The GeckoView platform </mobile/android/geckoview>`_ + +- `Firefox for iOS <https://github.com/mozilla-mobile/firefox-ios>`_, + our general-purpose browser for iOS with desktop sync built-in. +- Building **Firefox Focus**, our privacy-focused browser for + + - `iOS <https://github.com/mozilla-mobile/focus-ios>`_ + - `Android <https://github.com/mozilla-mobile/firefox-android/tree/main/focus-android>`_. This browser + also uses the android-components library and GeckoView platform, like Firefox for Android + +For both Desktop and Mobile development, please bear the following in +mind: + +- While you can build Firefox on older hardware it can take quite a bit + of time to compile on slower machines. Having at least 8GB of RAM is + recommended, and more is always better. The build process is both CPU + and I/O intensive, so building on a machine with an SSD is also + strongly preferred. +- Fast broadband internet is strongly recommended as well. Both the + development environment and the source code repository are quite + large. +- Though you can build Firefox to run on 32-bit machines, the build + process for almost all of our products requires a 64-bit OS. diff --git a/docs/contributing/build/supported.rst b/docs/contributing/build/supported.rst new file mode 100644 index 0000000000..4223f812ff --- /dev/null +++ b/docs/contributing/build/supported.rst @@ -0,0 +1 @@ +.. include:: ../../build/buildsystem/supported-configurations.rst diff --git a/docs/contributing/committing_rules_and_responsibilities.rst b/docs/contributing/committing_rules_and_responsibilities.rst new file mode 100644 index 0000000000..c3edcba79f --- /dev/null +++ b/docs/contributing/committing_rules_and_responsibilities.rst @@ -0,0 +1,198 @@ +Committing rules and responsibilities +===================================== + ++--------------------------------------------------------------------+ +| This page is an import from MDN and the contents might be outdated | ++--------------------------------------------------------------------+ + +Preparation +----------- + +There are things you need to be sure of before you even attempt to check +in: + +- Your code must + :ref:`compile <Building Firefox On Linux>` and `pass all the automated tests <https://developer.mozilla.org/docs/Mozilla/QA/Automated_testing>`__ + before you consider pushing changes. If you are at all unsure, verify + your changes with the + `mozilla-central <https://wiki.mozilla.org/Build:TryServer>`__. + try server, as appropriate. +- You need :ref:`code review <Code Review FAQ>`. +- Depending on the stage of the development process, you may need + `approval <https://wiki.mozilla.org/Tree_Rules>`__. Commits to trees + where approval is required must have "a=" in the commit message + followed by the name of the approver. +- Code should be factored in such a way such that we can disable + features which cause regressions, either by backout or via a kill + switch/preference. Be especially careful when landing features which + depend on other new features which may be disabled. Ask + mozilla.dev.planning for assistance if there are any questions. + +Checkin comment +--------------- + +The checkin comment for the change you push should include the bug +number, the names of the reviewers, and a clear explanation of the fix. +Please say what changes are made, not what problem was fixed, e.g.: + +Good: "Bug 123456 - Null-check presentation shell so we don't crash when a +button removes itself during its own onclick handler. r=paul, a=ringo." + +Bad: "Bug 123456 - crash clicking button on www.example.com" + +If you are not the author of the code, use ``hg commit -u`` to specify +the actual author in the Mercurial changeset: + +:: + + hg commit -u "Pat Chauthor <pat@chauthor.com>" + +Commit message restrictions +--------------------------- + +The purpose of these new restrictions, implemented via a mercurial hook, +is to prevent commit messages that do not have a bug number. We will +still allow a small set of special commits lacking bugs numbers, like +merges and backouts. + +This hook will be enabled on mozilla-central and every major branch that +directly merges into it, such as autoland or integration +branches, team branches, or established project branches. + +An example for a passing commit message would be, + +:: + + Bug 577872 - Create WebM versions of Ogg reftests. r=kinetik + +Note the *Bug ####*, you at least need that. You also can't commit +bustage-fixes without a bug number anymore. This is intentional to keep +track of the bug which caused it. + +Allowed are: + +- Commit messages containing "bug" or "b=" followed by a bug number +- Commit messages containing "no bug" (please use this sparingly) +- Commit message indicating backout of a given 12+ digit changeset ID, + starting with (back out|backing out|backed out|backout)( of)? + (rev|changeset|cset)s? [0-9a-f]{12} +- Commit messages that start with "merge" or "merging" and are actually + for a merge changeset. + +Special exceptions: + +- Commits by the special users "ffxbld", "seabld", "tbirdbld", or + "cltbld". +- When the commit is older then some date shortly after the hook has + been enabled, to allow merges from other branches. This exception + will be lifted after a short period of time (probably a few months) + after the hooks is enabled. +- You can also specify "IGNORE BAD COMMIT MESSAGES" in the tip (latest) + commit message to override all the restrictions. This is an extreme + measure, so you should only do this if you have a very good reason. + +Explicitly disallowed: + +- Commit messages containing "try: " to avoid unintentional commits + that were meant for the try server. + +All tests for allowed or excluded messages are case-insensitive. The +hook, +`commit-message.py <https://hg.mozilla.org/hgcustom/version-control-tools/file/tip/hghooks/mozhghooks/commit-message.py>`__, +was added in `bug 506949 <https://bugzilla.mozilla.org/show_bug.cgi?id=506949>`__. + + +Check the tree +-------------- + +TaskCluster is a continuous build system that builds and tests every change +checked into autoland/mozilla-central and related source trees. +`Treeherder <https://treeherder.mozilla.org/>`__ displays the progress +and results of all the build and test jobs for a given tree. For a +particular job, green means all is well, orange means tests have failed, +and red means the build itself broke. Purple means that a test was +interrupted, possibly by a problem with the build system or the +network. Blue means that a test was interrupted in a known way and will +be automatically restarted. You can click on the "Help" link in the top +right corner of Treeherder for a legend to help you decode all the other +colors and letters. + +If the tree is green, it is okay to check in. If some builds are orange +or red, you can either wait, or make sure all the failures are +classified with annotations/comments that reference bug numbers or +fixes. + +If the tree is marked as "closed", or if you have questions about any +oranges or reds, you should contact the sheriff before checking in. + + +Failures and backouts +--------------------- + +Patches which cause unit test failures (on :ref:`tier 1 +platforms <Supported Build Hosts and Targets>`) will be backed out. +Regressions on tier-2 platforms and in performance are not cause for a +direct backout, but you will be expected to help fix them if quickly. + +*Note: Performance regressions require future data points to ensure a +sustained regression and can take anywhere from 3 hours to 30 hours +depending on the volume of the tree and build frequency. All regression +alerts do get briefly investigated and bugs are filed if necessary.* + + +Dealing with test failures +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If a build or a test job fails, you can click on the red or orange or +purple symbol for the job on Treeherder to display more information. +The information will appear in the footer, including a summary of any +error messages, a "+" icon to re-trigger the job (schedule it to run +again), and links to the log files and to possibly-related bugs. + +Here are some steps you can follow to figure out what is causing most +failures, `and "star" them +appropriately <http://ehsanakhgari.org/blog/2010-04-09/assisted-starring-oranges>`__: + +#. Click on the failing job to see a list of suggested bugs. If the + failure clearly matches a known bug, **click on the star** next to + that bug and then click "Add a comment" and then submit the comment. + This is referred to as "starring the build;" you'll see this phrase + or ones like it in IRC a lot. +#. If the failure might match a known bug but you are not sure, click + the bug number to open the Bugzilla report, and click the failing job + to open its log. If the log and the bug do match, add a comment as + in step 1 (above). +#. If the summary does not seem to match any suggested bugs, search + Bugzilla for the name of the failing test or the error message. If + you find a matching bug, add a comment in the bug in Bugzilla, and + another to the job in Treeherder. +#. If you can't figure out whether a known bug exists (for example, + because you can't figure out what part of the log you should search + for), look on Treeherder to see if there are other similar failures + nearby, or ask on #developers to see if anyone recognizes it as a + known failure. For example, many Android tests fail frequently in + ways that do not produce useful log messages. You can often find the + appropriate bug just by looking at other Android failures that are + already starred. +#. If there is no matching bug, you can back out the change (if you + suspect the failure was caused by your changeset) or re-trigger the + job (if you suspect it's an unrelated intermittent failure). After + more test runs it should become clear whether it is a new regression + or just an unknown intermittent failure. +#. If it turns out to be an unknown intermittent failure, file a new bug + with "intermittent-failure" in the keywords. Include the name of the + test file and an one-line summary of the log messages in the Summary + field. In the description, include an excerpt of the error messages + from the log, and a link to the log file itself. + +At any point if you are not sure or can't figure out what to do, ask for +advice or help in `#developers <https://chat.mozilla.org>`__. +If a large number of jobs are failing and you suspect an infrastructure problem, you can also ask +about it in `#releng <https://chat.mozilla.org>`__. + + +Dealing with performance regressions +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Under some circumstances, if your patch causes a performance regression +that is not acceptable, it will get backed out. diff --git a/docs/contributing/contributing_to_mozilla.rst b/docs/contributing/contributing_to_mozilla.rst new file mode 100644 index 0000000000..0e12922ece --- /dev/null +++ b/docs/contributing/contributing_to_mozilla.rst @@ -0,0 +1,63 @@ +Contributing to Mozilla projects +================================ + +This page is here to help you get from "I want to build Firefox" +to "I'm building my own Firefox" to "I can contribute to Firefox". +So if you'd like to help Mozilla build the best web browsers in the +world, you're in the right place. + +.. rubric:: Need help? + :name: Need_help + +The Mozilla community prides itself on being an open, accessible, and +friendly community for new participants. If you have any difficulties +getting involved or finding answers to your questions, please `come and +ask your questions in our +chatroom <https://chat.mozilla.org/#/room/#introduction:mozilla.org>`_, +where we can help you get started. + +We know even before you start contributing that getting set up to work +on Firefox and finding a bug that's a good fit for your skills can be a +challenge, and we're always looking for ways to improve this process: making +Mozilla more open, accessible, and easier to participate with. If you're +having any trouble following this documentation, or hit a barrier you +can't get around, please join us in the the Introduction room on Matrix. + +What skills do I need? +---------------------- + +Mozilla maintains small and large projects and we are thrilled to have contributors with +very diverse skills: + +- If you know **C++,** **Rust,** **JavaScript,** **HTML** or **CSS**, + you can :ref:`contribute to the core layers <Firefox Contributors' Quick Reference>` of + Firefox and many other Mozilla projects. +- If you know **Rust**, you can also contribute to the `Rust programming + language <https://github.com/rust-lang/rust>`_ itself, numerous crates like `grcov <https://github.com/mozilla/grcov/>`_ + or `Servo <https://servo.org/>`_, the web browser engine designed for parallelism and safety. +- If you know **Kotlin**, you can contribute to `Firefox + for Android <https://github.com/mozilla-mobile/fenix>`_ (code name: + "Fenix"). +- If you know **Swift**, you can contribute to `Firefox for + iOS <https://github.com/mozilla-mobile/firefox-ios>`_ and `Firefox + Focus for iOS <https://github.com/mozilla-mobile/focus-ios>`_. +- If you know **C++**, you can contribute to our `VPN client <https://github.com/mozilla-mobile/mozilla-vpn-client>`_. +- If you know **Python**, you can contribute to our web services, + including Firefox Sync and Firefox Accounts. +- If you know **Make**, **shell**, **Perl**, or **Python**, you can + contribute to our build systems, release engineering, and automation. +- If you know **Go** or **JavaScript**, you can contribute to `TaskCluster + <https://github.com/taskcluster/taskcluster>`_ our CI infrastructure. +- If you know **C**, you can contribute to `NSS <https://developer.mozilla.org/docs/Mozilla/Projects/NSS>`_, + `Opus <https://opus-codec.org/>`_, and `Daala <https://wiki.xiph.org/Daala>`_. +- There are even many ways to contribute to the Mozilla mission without + programming. If getting involved in design, support, translation, + testing, or other types of contributions sparks your interest please + see the `Volunteer Opportunities + wiki <https://contribute.mozilla.org>`_ or the `Mozilla + community <https://mozilla.community/>`_ site. + +Perhaps you do not know programming yet, but you want to start learning? +There are `plenty of +resources <https://developer.mozilla.org/learn>`_ available on +the MDN Web Docs! diff --git a/docs/contributing/contribution_quickref.rst b/docs/contributing/contribution_quickref.rst new file mode 100644 index 0000000000..dcaa969c97 --- /dev/null +++ b/docs/contributing/contribution_quickref.rst @@ -0,0 +1,369 @@ +Firefox Contributors' Quick Reference +===================================== + +Some parts of this process, including cloning and compiling, can take a long time even on modern hardware. +If at any point you get stuck, please don't hesitate to ask at `https://chat.mozilla.org <https://chat.mozilla.org>`__ +in the `#introduction <https://chat.mozilla.org/#/room/#introduction:mozilla.org>`__ channel. + +Don’t hesitate to look at the :ref:`Getting Set Up To Work On The Firefox Codebase<Getting Set Up To Work On The Firefox Codebase>` for a more detailed tutorial. + +Before you start +---------------- +Please register and create your account for + +`Bugzilla <https://bugzilla.mozilla.org/>`__ : web-based general-purpose bug tracking system. +To register with Phabricator, make sure you enable Two-Factor Authentication (My Profile >> Edit Profile & Preferences >> Two-Factor Authentication) in Bugzilla. + +`Phabricator <https://phabricator.services.mozilla.com/>`__: web-based software development collaboration tools, mainly for code review. +Please obtain an API Token (Settings >> Conduit API Tokens) + +Windows dependencies +-------------------- + +#. You need a :ref:`supported version of Windows<tier_1_hosts>`. +#. Download and install `Visual Studio Community Edition. <https://visualstudio.microsoft.com/downloads/>`__ +#. Finally download the `MozillaBuild Package. <https://ftp.mozilla.org/pub/mozilla/libraries/win32/MozillaBuildSetup-Latest.exe>`__ Installation directory should be: + + .. code-block:: shell + + $ c:\mozilla-build\ + +#. Before moving on to the next steps, make sure to fulfill the :ref:`Windows prerequisites <Building Firefox On Windows>` + +.. note:: + + All the commands of this tutorial must be run in the shell provided with the MozillaBuild Package (start-shell.bat) + +:ref:`More information <Building Firefox On Windows>` + +Bootstrap a copy of the Firefox source code +------------------------------------------- + +You can download the source code and have Firefox automatically download and install the other dependencies it needs. The below command as per your Operating System, will download a lot of data (years of Firefox history!) then guide you through the interactive setup process. + +Downloading can take from 40 minutes to two hours (depending on your connection) and the repository should be less than 5GB (~ 20GB after the build). + +The default options are recommended. +If you're not planning to write C++ or Rust code, select :ref:`Artifact Mode <Understanding Artifact Builds>` +and follow the instructions at the end of the bootstrap for creating a mozconfig file. + +To Setup Firefox On Windows +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: shell + + $ cd c:/ + $ mkdir mozilla-source + $ cd mozilla-source + $ wget https://hg.mozilla.org/mozilla-central/raw-file/default/python/mozboot/bin/bootstrap.py + $ python3 bootstrap.py + +More information :ref:`for Windows <Building Firefox On Windows>` + +To Setup Firefox On macOS and Linux +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: shell + + $ curl https://hg.mozilla.org/mozilla-central/raw-file/default/python/mozboot/bin/bootstrap.py -O + $ python3 bootstrap.py + +More information :ref:`for Linux <Building Firefox On Linux>` and :ref:`for MacOS <Building Firefox On MacOS>` + +To set up your editor +--------------------- + +.. note:: + + Visual Studio Code is the recommended editor for Firefox development. + Not because it is better than the other editors but because we decided to + focus our energy on a single editor. + +Setting up your editor is an important part of the contributing process. Having +linting and other features integrated, saves you time and will help with reducing +build and reviews cycles. + +See our :ref:`editor page for more information about how to set up your favorite editor <Editor / IDE integration>`. + +To build & run +-------------- + +Once the System is bootstrapped, run: + +.. code-block:: shell + + $ cd mozilla-unified + $ ./mach build + +which will check for dependencies and start the build. +This will take a while; a few minutes to a few hours depending on your hardware. + +.. note:: + + The default build is a compiled build with optimizations. Check out the + :ref:`mozconfig file documentation <Configuring Build Options>` + to see other build options. If you don't plan to change C++ or Rust code, + an :ref:`artifact build <Understanding Artifact Builds>` will be faster. + +To run it: + +.. code-block:: shell + + $ ./mach run + +:ref:`More information about Linux <Building Firefox On Linux>` / :ref:`More information about MacOS <Building Firefox On MacOS>` + +.. _write_a_patch: + +To write a patch +---------------- + +Make the changes you need in the codebase. You can look up UI text in `Searchfox <https://searchfox.org>`__ to find the right file. + +Then: + +.. code-block:: shell + + # Mercurial + $ hg commit + + # Git + $ git commit + +.. _Commit message: + +The commit message should look like: + +.. code-block:: text + + Bug xxxx - Short description of your change. r?reviewer + + Optionally, a longer description of the change. + +**Make sure you include the bug number and at least one reviewer (or reviewer group) in this format.** + +To :ref:`find a reviewer or a review group <Getting reviews>`, the easiest way is to run +``hg log <modified-file>`` (or ``git log <modified-file>``, if +you're using git) on the relevant files, and look who usually is +reviewing the actual changes (ie not reformat, renaming of variables, etc). + + +To visualize your patch in the repository, run: + +.. code-block:: shell + + # Mercurial + $ hg wip + + # Git + $ git show + +:ref:`More information on how to work with stack of patches <Working with stack of patches Quick Reference>` + +:ref:`More information <Mercurial Overview>` + +To make sure the change follows the coding style +------------------------------------------------ + +To detect coding style violations, use mach lint: + +.. code-block:: shell + + $ ./mach lint path/to/the/file/or/directory/you/changed + + # To get the autofix, add --fix: + $ ./mach lint path/to/the/file/or/directory/you/changed --fix + +:ref:`More information <Code quality>` + +To test a change locally +------------------------ + +To run the tests, use mach test with the path. However, it isn’t +always easy to parse the results. + +.. code-block:: shell + + $ ./mach test dom/serviceworkers + +To run tests based on :ref:`GTest` (C/C++ based unit tests), run: + +.. code-block:: shell + + $ ./mach gtest 'QuotaManager.*' + +To test a change remotely +------------------------- + +Running all the tests for Firefox takes a very long time and requires multiple +operating systems with various configurations. To build Firefox and run its +tests on continuous integration servers (CI), multiple :ref:`options to select tasks <Selectors>` +are available. + +To automatically select the tasks that are most likely to be affected by your changes, run: + +.. code-block:: shell + + $ ./mach try auto + +To select tasks manually using a fuzzy search interface, run: + +.. code-block:: shell + + $ ./mach try fuzzy + +To rerun the same tasks: + +.. code-block:: shell + + $ ./mach try again + +From `Treeherder <https://treeherder.mozilla.org/>`__ (our continuous integration system), it is also possible to attach new jobs. As every review has +a try CI run associated, it makes this work easier. See :ref:`attach-job-review` for +more information. + +.. note:: + + This requires `level 1 commit access <https://www.mozilla.org/about/governance/policies/commit/access-policy/>`__. + + You can ask your reviewer to submit the patch for you if you don't have that + level of access. + +:ref:`More information <Pushing to Try>` + + +To submit a patch +----------------- + +To submit a patch for review, we use a tool called `moz-phab <https://pypi.org/project/MozPhab/>`__. +To install it, run: + +.. code-block:: shell + + $ ./mach install-moz-phab + +Once you want to submit your patches (make sure you :ref:`use the right commit message <Commit message>`), run: + +.. code-block:: shell + + $ moz-phab + +It will publish all the currently applied patches to Phabricator and inform the reviewer. + +If you wrote several patches on top of each other: + +.. code-block:: shell + + $ moz-phab submit <first_revision>::<last_revision> + +`More +information <https://moz-conduit.readthedocs.io/en/latest/phabricator-user.html>`__ + +To update the working directory +------------------------------- + +If you're finished with a patch and would like to return to the tip to make a new patch: + +.. code-block:: shell + + $ hg pull central + $ hg up central + +To update a submitted patch +--------------------------- + +It is rare that a reviewer will accept the first version of patch. Moreover, +as the code review bot might suggest some improvements, changes to your patch +may be required. + +If your patch is not loaded in your working directory, you first need to re-apply it: + +.. code-block:: shell + + $ moz-phab patch D<revision_id> + +Make your changes in the working folder and run: + +.. code-block:: shell + + # Or, if you need to pass arguments, e.g., changing the commit message: + $ hg commit --amend + + # Git + $ git commit --amend + +After amending the patch, you will need to submit it using moz-phab again. + +.. warning:: + + Don't use ``hg commit --amend -m`` or ``git commit --amend -m``. + + Phabricator tracks revision by editing the commit message when a + revision is created to add a special ``Differential Revision: + <url>`` line. + + When ``--amend -m`` is used, that line will be lost, leading to + the creation of a new revision when re-submitted, which isn't + the desired outcome. + +If you wrote many changes, you can squash or edit commits with the +command: + +.. code-block:: shell + + # Mercurial + $ hg histedit + + # Git + $ git rebase -i + +The submission step is the same as for the initial patch. + +:ref:`More information on how to work with stack of patches <Working with stack of patches Quick Reference>` + +Retrieve new changes from the repository +---------------------------------------- + +To pull changes from the repository, run: + +.. code-block:: shell + + # Mercurial + $ hg pull --rebase + + # Git + $ git pull --rebase + +.. _push_a_change: + +To push a change in the code base +--------------------------------- + +Once the change has been accepted and you've fixed any remaining issues +the reviewer identified, the reviewer should land the patch. + +If the patch has not landed on "autoland" (the integration branch) after a few days, +feel free to contact the reviewer and/or +@Aryx or @Sylvestre on the `#introduction <https://chat.mozilla.org/#/room/#introduction:mozilla.org>`__ +channel. + +The landing procedure will automatically close the review and the bug. + +:ref:`More information <How to submit a patch>` + +Contributing to GeckoView +------------------------- + +Note that the GeckoView setup and contribution processes are different from those of Firefox; +GeckoView setup and contribution docs live in `geckoview.dev <https://geckoview.dev>`__. + +More documentation about contribution +------------------------------------- + +:ref:`Contributing to Mozilla projects` + +https://mozilla-version-control-tools.readthedocs.io/en/latest/devguide/contributing.html + +https://moz-conduit.readthedocs.io/en/latest/phabricator-user.html + +https://mikeconley.github.io/documents/How_mconley_uses_Mercurial_for_Mozilla_code diff --git a/docs/contributing/debugging/capturing_minidump.rst b/docs/contributing/debugging/capturing_minidump.rst new file mode 100644 index 0000000000..4c8f7591f5 --- /dev/null +++ b/docs/contributing/debugging/capturing_minidump.rst @@ -0,0 +1,259 @@ +Capturing a minidump +==================== + +*Minidumps* are files created by various Windows tools which record the +complete state of a program as it's running, or as it was at the moment +of a crash. Small minidumps are created by the Breakpad :ref:`crash +reporting <Crash Reporter>` tool, but sometimes that's not +sufficient to diagnose a problem. For example, if the application is +hanging (not responding to input, but hasn't crashed) then Breakpad is +not triggered, and it can be difficult to determine where the problem +lies. Sometimes a more complete form of minidump is needed to see +additional details about a crash, in which case manual capture of a +minidump is desired. + +This page describes how to capture these minidumps on Windows, to permit +better debugging. + + +Privacy and minidumps +--------------------- + +.. warning:: + + **Warning!** Unlike the minidumps submitted by Breakpad, these + minidumps contain the **complete** contents of program memory. They + are therefore much more likely to contain private information, if + there is any in the browser. For this reason, you may prefer to + generate minidumps against a `clean + profile <http://support.mozilla.com/en-US/kb/Managing%20profiles>`__ + where possible. + + +Capturing a minidump: application crash +--------------------------------------- + +To capture a full minidump for an application crash, you can use a tool called +**Debugging Tools for Windows**, which is provided by Microsoft for free. + + +Install Debugging Tools for Windows +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Please follow `these instructions +<https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/>`__. +You can install the Debugging Tools for Windows alone, but you cannot +download it alone. To install it, download Windows Software Development Kit +(SDK) first, start the installer, and then select only **Debugging Tools for +Windows** in the list of features to install. + +.. image:: img/sdk-installer.png + +The latest installer installs the tools for all CPU architectures (X86, +X64, ARM, and ARM64). You need to choose a tool of the architecture +matching Firefox you want to capture a minidump from. For example, if +you want to capture a minidump from 32-bit Firefox on 64-bit Windows, +use the X86 version of tools, not X64 tools. + +The default install path of SDK for 64-bit Windows is +``%ProgramFiles(x86)%\Windows Kits\10``. The debugging tools can be found in +the folder named **Debuggers** under your install path of SDK. + +The Debugging Tools for Windows contains both graphical and command line +debugging tools that can be used to capture a minidump. If you prefer +a graphical tool, please follow `Capture a minidump in a graphical way +<#capture-a-minidump-in-a-graphical-way>`__. If you prefer a command +line tool, please follow `Capture a minidump from the command line +<#capture-a-minidump-from-the-command-line>`__. + + +Capture a minidump in a graphical way +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +#. Launch the graphical debugger named WinDbg from the Start Menu. WinDbg + icons are registered under "Windows Kits" as below. + + |WinDbg in Start Menu| + +#. Connect Firefox to the debugger. + + a. If Firefox is not already running, open the **"File"** menu on WinDbg + and choose **"Open Executable..."**. In the file chooser window that + appears, open the firefox.exe executable. If you're not sure about where + it is, please see `How to find the location of firefox.exe + <#how-to-find-the-location-of-firefox-exe>`__. + + b. If Firefox is already running and you know which process you want to + capture a minidump from, open the **"File"** menu on WinDbg and choose + **"Attach to a Process..."**. In the "Attach to Process" dialog that + appears, select the process. To identify a process, please see + `Identify a process to attach a debugger to + <#identify-a-process-to-attach-a-debugger-to>`__. + +#. You should now see a "Command" text window with debug output at the + top and an input box at the bottom. From the menu, select + ``Debug → Go``, and Firefox should start. If the debugger spits out + some text right away and Firefox doesn't come up, select + ``Debug → Go`` again. + +#. When the program is about to crash, WinDbg will spit out more data, + and the prompt at the bottom will change from saying "``*BUSY*``" to + having a number in it. At this point, you should type + "``.dump /ma c:\temp\firefoxcrash.dmp``" -- without the quotes, but + don't forget the dot at the beginning. Once it completes, which can + take a fair while, you will have a very large file at + ``c:\temp\firefoxcrash.dmp`` that can be used to help debug your + problem. File size will depend on this size of Firefox running in + your environment, which could several GB. + +#. Ask in the relevant bug or thread how best to share this very large + file! + + +Capture a minidump from the command line +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If Firefox is not already running, open the Command Prompt and run the following +command. This command launches all Firefox processes under a debugger. This +technique is useful when you have a startup crash or when you're not sure about +which process will crash. + +To find out where firefox.exe is, please see `How to find the location +of firefox.exe <#how-to-find-the-location-of-firefox-exe>`__. + +.. code:: + + <path to debugger>\cdb.exe -g -G -o <path to firefox>\firefox.exe + + +For example, if both the debugging tools and Firefox are installed in the +default folder and you want to capture a minidump of 64-bit Firefox, +the command will be like this. Please note that you need to add double +quotes when a path contains one or more whitespaces. + +.. code:: + + "C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\cdb.exe" -g -G -o "C:\Program Files\Mozilla Firefox\firefox.exe" + + +If a Firefox process you want to capture a minidump from is already running, +attach the debugger to it with the following command. To identify a process, +please see `Identify a process to attach a debugger to +<#identify-a-process-to-attach-a-debugger-to>`__. + +.. code:: + + <path to debugger>\cdb.exe -g -G -p <PID> + +When the process crashes, the debugger tool captures it and waits for your +command. At this point, you should type ``.dump /ma c:\temp\firefoxcrash.dmp`` +-- don't forget the dot at the beginning. Once it completes, which can take +a fair while, you will have a very large file at ``c:\temp\firefoxcrash.dmp`` +that can be used to help debug your problem. File size will depend on this +size of Firefox running in your environment, which could several GB. + +After a minidump is generated, type ``q`` and press Enter to quit the debugger. + + +Capturing a minidump: application hang +-------------------------------------- + +On Windows Vista and Windows 7, you can follow `these +instructions <http://support.microsoft.com/kb/931673>`__ to capture a +dump file and locate it after it's been saved. + + +Identify a process to attach a debugger to +------------------------------------------ + +When you're running Firefox, even if you have only a single tab, you may have +noticed a bunch of firefox.exe instances in Task Manager. This means Firefox +consists of multiple processes. Since an application crash happens per process +and a minidump is generated per process, you need to identify which process will +crash before starting a debugger. + +Identify a process type +~~~~~~~~~~~~~~~~~~~~~~~ + +Each instance of firefox.exe has a type. To identify a process to attach +a debugger to, you need to know its process type first. + +When a crash happens, if all Firefox windows are suddenly gone and Mozilla +Crash Reporter window is opend, a process that crashed is the main process. + +.. image:: img/crashreporter.png + +When a crash happens, if you see the following page, a process that crashed +is a tab (content) process. + +.. image:: img/tabcrashed.png + +There are more process types, but there isn't an easy way to detect a crash in +a process of those types because the symptom varies. If you cannot be sure +about the type of a crashing process, terminate Firefox and launch a new +instance of Firefox under a debugger in the way described above. + +If a GPU process crashes, you may see a window is not rendered correctly as +below. Since the main process relaunches a GPU process, this symptom will be +transient and the window will be rendered correctly again. + +.. image:: img/crash-gpu.png + +If a GMP (= Gecko Media Plugin) process crashes, you will see an information +bar will be displayed below the address bar. + +.. image:: img/crash-gmp.png + +If an RDD (= Remote Data Decoder) process crashes, Firefox may stop playing +a video as below, but not limited to this symptom. + +.. image:: img/crash-rdd.png + + +Identify a process ID (PID) +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Once you understand what type of process crashes, the next step is to get a +process ID (PID), which is a value to specify in the debugger command we +discussed above. We present two ways to get a PID here. + +The first way is to use Firefox itself. Open a new tab and go to the +**about:processes** page. This page shows the list of all processes and their +PIDs. In the example below, the PID of the main process is **6308** and the +PID of the tab process hosting a page of mozilla.org is **6748**. + +.. image:: img/about-processes.png + +The second way is to use `Process Explorer +<https://docs.microsoft.com/en-us/sysinternals/downloads/process-explorer>`__, +which is provided by Microsoft for free. You may need this technique to attach +a debugger to a hung process or when you cannot open a new tab in the existing +Firefox for some reason. + +Process Explorer is basically an advanced version of Task Manager. Since it +displays processes in a hierarchical tree view, you can easily locate the main +process, which is the parent of all child processes. In the example below, the +PID of the main process is **6308** and all processes, including +plugin-container.exe, under the main process is child processes. + +Another helpful feature of Process Explorer is that when you hover the mouse +cursor on a process, it displays a tooltip window showing the process's command +line string. For a child process of firefox.exe, the command line's last +keyword shows the process type, so you can tell which process is which process +type. In the example below, the tooltip window displays a command line string +of a GPU process. + +.. image:: img/process-explorer.png + + +How to find the location of firefox.exe +--------------------------------------- + +If you're not sure about the location of the executable file (firefox.exe) of +Firefox you run, you can find it in the **about:support** page. In the +"Application Basics" section, the path to firefox.exe is displayed in the row +of "Application Binary". + +.. image:: img/about-support.png + +.. |WinDbg in Start Menu| image:: img/windbg-in-startmenu.png + :width: 50% diff --git a/docs/contributing/debugging/debugging_a_hang_on_macos.rst b/docs/contributing/debugging/debugging_a_hang_on_macos.rst new file mode 100644 index 0000000000..ee2bed16b2 --- /dev/null +++ b/docs/contributing/debugging/debugging_a_hang_on_macos.rst @@ -0,0 +1,10 @@ +Debugging A Hang On macOS +========================= + +See `How to Report a Hung +Firefox <https://developer.mozilla.org/en-US/docs/Mozilla/How_to_report_a_hung_Firefox>`_. + +See also +~~~~~~~~ + +`Debugging on macOS <https://developer.mozilla.org/en-US/docs/Mozilla/Debugging/Debugging_on_macOS>`__ diff --git a/docs/contributing/debugging/debugging_a_minidump.rst b/docs/contributing/debugging/debugging_a_minidump.rst new file mode 100644 index 0000000000..5638c29d85 --- /dev/null +++ b/docs/contributing/debugging/debugging_a_minidump.rst @@ -0,0 +1,202 @@ +Debugging A Minidump +==================== + ++--------------------------------------------------------------------+ +| This page is an import from MDN and the contents might be outdated | ++--------------------------------------------------------------------+ + +The +`minidump <http://msdn.microsoft.com/en-us/library/windows/desktop/ms680369%28v=vs.85%29.aspx>`__ +file format contains data about a crash on Windows. It is used by +`rust-minidump <https://github.com/luser/rust-minidump>`__, +`Breakpad <https://wiki.mozilla.org/Breakpad>`__, and also by various +Windows debugging tools. Each minidump includes the following data. + +- Details about the exception which led to the crash. +- Information about each thread in the process: the address which was + executing and the register state at the time the process stopped. +- A list of shared libraries loaded into the process at the time of the + crash. +- The stack memory of each thread. +- The memory right around the crashing address. +- (Optional) Other memory regions, if requested by the application. +- (Optional) Other platform-specific data. + +Accessing minidumps from crash reports +-------------------------------------- + +Minidumps are not available to everyone. For details on how to gain +access and where to find minidump files for crash reports, consult the +:ref:`crash report documentation <Understanding Crash Reports>` + +Using rust-minidump's tooling +----------------------------------- + +Most of our crash-reporting infrastructure is based on rust-minidump. +The primary tool for this is the +`minidump-stackwalk <https://github.com/luser/rust-minidump/tree/master/minidump-stackwalk>`__ +CLI application, which includes extensive user documentation. + +That documentation includes a +`dedicated section <https://github.com/luser/rust-minidump/tree/master/minidump-stackwalk#analyzing-firefox-minidumps>`__ +on locally analyzing Firefox crashreports and minidumps. + +If you're looking for minidump_dump, it's included as part of +minidump-stackwalk. + +Using the MS Visual Studio debugger +----------------------------------- + +#. Set up the debugger to :ref:`use the Mozilla symbol + server <Using The Mozilla Symbol Server>` and + :ref:`source server <Using The Mozilla Source Server>`. +#. Double-click on the minidump file to open it in the debugger. +#. When it loads, click the green icon in the visual studio debugger + toolbar that looks like a play button. + +For Firefox releases older than Firefox 41, you will also need to +install the relevant release of Firefox (for example from +`here <https://ftp.mozilla.org/pub/mozilla.org/firefox/releases/>`__), +and add the directory it is in (e.g., "C:\Program Files\Mozilla +Firefox 3.6 Beta 1\") to the same dialog in which you set up the +symbol server (in case the binary location in the minidump is not the +same as the one on your machine). Note that you can install the +relevant release anywhere. Just make sure to configure the symbol +server to the directory where you installed it. For releases from 41 +onward, the binaries are available on the symbol server. + +If this doesn't work, downloading the exact build and crashreporter +symbols full files. These can be found in treeherder / build folder. +Load Visual Studio, and go to file -> open -> minidump location. Click +on "Run Native", and Visual Studio will ask for the corresponding symbol +files. For each .dll you wish to have symbols for, you must go to a +console and go to the corresponding directory. E.g. (xul.dll should go +to xul.pdf in the crashreporter symbols directory). Each directory will +have a .pd\_ file. In a command shell run: "expand /r foo.pd\_". Then +point Visual Studio to this directory. + +Then you'll be able to examine: + ++------------------+-------------------------------------------------------------------------+ +| stack trace | The debugger shows the stack trace. You can right-click on any frame | +| | in the stack, and then choose "go to disassembly" or "go to source". | +| | (Choosing "go to disassembly" from the source view might not get you | +| | to the right place due to optimizations.) When looking at the | +| | source, beware that the debugging information will associate all | +| | inlined functions as part of the line into which they were inlined, | +| | and compiler (with PGO) inlines *very* aggressively (including | +| | inlining virtual functions). You can often figure out where you | +| | *really* are by reading the disassembly. | ++------------------+-------------------------------------------------------------------------+ +| registers | In the Registers tab (Debug->Windows->Registers if you don't have | +| | it open), you can look at the registers associated with each stack | +| | frame, but only at the current state (i.e., the time of the crash). | +| | Registers that Visual Studio can't figure out will be grayed-out and | +| | have the value 00000000. | ++------------------+-------------------------------------------------------------------------+ +| stack memory | You open a window (Memory 1, etc.) that shows contiguous segments of | +| | memory using the Debug->Windows->Memory menu item. You can then | +| | enter the address of the stack pointer (ESP register) in this window | +| | and look at the memory on the stack. (The minidump doesn't have the | +| | memory on the heap.) It's a good idea to change the "width" dropdown | +| | in the top right corner of the window from its default "Auto" to | +| | either "8" or "16" so that the memory display is word-aligned. If | +| | you're interested in pointers, which is usually the case, you can | +| | right click in this window and change the display to show 4-byte | +| | words (so that you don't have to reverse the order due to | +| | little-endianness). This view, combined with the disassembly, can | +| | often be used to reconstruct information beyond what in shown the | +| | function parameters. | ++------------------+-------------------------------------------------------------------------+ +| local variables | In the Watch 1 (etc.) window (which, if you don't have open, you can | +| | iget from Debug->Windows->Watch), you can type an expression | +| | (e.g., the name of a local variable) and the debugger will show you | +| | its value (although it sometimes gets confused). If you're looking | +| | at a pointer to a variable that happens to be on the stack, you can | +| | even examine member variables by just typing expressions. If Visual | +| | Studio can't figure something out from the minidump, it might show | +| | you 00000000 (is this true?). | ++------------------+-------------------------------------------------------------------------+ + +Using minidump-2-core on Linux +------------------------------ + +The `Breakpad +source <https://chromium.googlesource.com/breakpad/breakpad/+/master/>`__ +contains a tool called +`minidump-2-core <https://chromium.googlesource.com/breakpad/breakpad/+/master/src/tools/linux/md2core/>`__, +which converts Linux minidumps into core files. If you checkout and +build Breakpad, the binary will be at +``src/tools/linux/md2core/minidump-2-core``. Running the binary with the +path to a Linux minidump will generate a core file on stdout which can +then be loaded in gdb as usual. You will need to manually download the +matching Firefox binaries, but then you can use the :ref:`GDB Python +script <Downloading symbols on Linux / Mac OS X>` to download symbols. + +The ``minidump-2-core`` source does not currently handle processing +minidumps from a different CPU architecture than the system it was +built for. If you want to use it on an ARM dump, for example, you may +need to build the tool for ARM and run it under QEMU. + +Using other tools to inspect minidump data +------------------------------------------ + +Ted has a few tools that can be built against an already-built copy of +Breakpad to do more targeted inspection. All of these tools assume you +have checked out their source in a directory next to the breakpad +checkout, and that you have built Breakpad in an objdir named +``obj-breakpad`` at the same level. + +- `stackwalk-http <https://hg.mozilla.org/users/tmielczarek_mozilla.com/stackwalk-http/>`__ + is a version of the breakpad's minidump_stackwalk that can fetch symbols + over HTTP, and also has the Mozilla symbol server URL baked in. If you + run it like ``stackwalk /path/to/dmp /tmp/syms`` it will print the stack + trace and save the symbols it downloaded in ``/tmp/syms``. Note that + symbols are only uploaded to the symbol server for nightly and + release builds, not per-change builds. +- `dump-lookup <https://hg.mozilla.org/users/tmielczarek_mozilla.com/dump-lookup/>`__ + takes a minidump and prints values on the stack that are potential + return addresses. This is useful when a stack trace looks truncated + or otherwise wrong. It needs symbol files to produce useful output, + so you will generally want to have run ``stackwalk-http`` to download + them first. +- `get-minidump-instructions <https://hg.mozilla.org/users/tmielczarek_mozilla.com/get-minidump-instructions/>`__ + retrieves and displays the memory range surrounding the faulting + instruction pointer from a minidump. You will almost always want to + run it with the ``--disassemble`` option, which will make it send the + bytes through ``objdump`` to display the disassembled instructions. + If you also give it a path to symbols (see ``stackwalk-http`` above) + it can download the matching source files from hg.mozilla.org and + display source interleaved with the disassembly. +- `minidump-modules <http://hg.mozilla.org/users/tmielczarek_mozilla.com/minidump-modules>`__ + takes a minidump and prints the list of modules from the crashed + process. It will print the full path to each module, whereas the + Socorro UI only prints the filename for each module for privacy + reasons. It also accepts a -v option to print the debug ID for each + module, and a -d option to print relative paths to the symbol files + that would be used instead of the module filenames. + +Getting a stack trace from a crashed B2G process +------------------------------------------------ + +#. Get the minidump file in the phone at + /data/b2g/mozilla/\*.default/minidump/. You can use `adb + pull <http://developer.android.com/tools/help/adb.html>`__ for that. +#. Build the debug symbols using the command ./build.sh buildsymbols + inside the B2G tree. The symbol files will be generated in + $OBJDIR/dist/crashreporter-symbols. +#. Build and install + `google-breakpad <https://code.google.com/p/google-breakpad/>`__. +#. Use the + `minidump-stackwalk <https://github.com/luser/rust-minidump/tree/master/minidump-stackwalk>`__ + tool to get the stack trace. + +.. code:: bash + + Example: + + $ cd B2G + $ adb pull /data/b2g/mozilla/*.default/minidump/*.dmp . + $ls *.dmp + 71788789-197e-d769-67167423-4e7aef32.dmp + $ minidump-stackwalk 71788789-197e-d769-67167423-4e7aef32.dmp objdir-debug/dist/crashreporter-symbols/ diff --git a/docs/contributing/debugging/debugging_firefox_with_gdb.rst b/docs/contributing/debugging/debugging_firefox_with_gdb.rst new file mode 100644 index 0000000000..8b188edc85 --- /dev/null +++ b/docs/contributing/debugging/debugging_firefox_with_gdb.rst @@ -0,0 +1,501 @@ +Debugging Firefox with GDB +========================== + ++--------------------------------------------------------------------+ +| This page is an import from MDN and the contents might be outdated | ++--------------------------------------------------------------------+ + +This page details how you can more easily debug Firefox and work around +some GDB problems. + +Use GDB 5, or higher. A more recent version of GDB can be obtained from +`sourceware <https://sourceware.org/gdb/>`__ or your Linux distro repo. +If you are running less than 256 MB of RAM, be sure to see `Using gdb on +wimpy computers <https://developer.mozilla.org/en/Using_gdb_on_wimpy_computers>`__. + +Where can I find general gdb documentation? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Using GDB is beyond the scope of this document. Documentation is likely +available on your system if you have GDB installed, in the form of +**info,** **man** pages, or the gnome help browser. Additionally, you +can use a graphical front-end to GDB like +`ddd <https://www.gnu.org/software/ddd/>`__ or +`insight <https://sourceware.org/insight/>`__. For more information see +https://sourceware.org/gdb/current/onlinedocs/gdb/ + +How do I run Firefox under gdb? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The preferred method, is using the +:ref:`mach` command-line tool to run the +debugger, which can bypass several optional defaults. Use "mach help +run" to get more details. If inside the source directory, you would use +"./mach". Please note that :ref:`mach is aware of mozconfigs <mach_and_mozconfigs>`. + +.. code:: bash + + $ ./mach run --debug [arguments to pass to firefox] + +If you need to direct arguments to gdb, you can use '--debugger-args' +options via the command line parser, taking care to adhere to shell +splitting rules. For example, if you wanted to run the command 'show +args' when gdb starts, you would use: + +.. code:: bash + + $ ./mach run --debug --debugger-args "-ex 'show args'" + +Alternatively, you can run gdb directly against Firefox. However, you +won't get some of the more useful capabilities this way. For example, +mach sets an environment variable (see below) to stop the JS engine from +generating synthetic segfaults to support the slower script dialoging +mechanism. + +.. code:: + + (gdb) OBJDIR/dist/bin/firefox + +How do I pass arguments in prun? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Set the arguments in GDB before calling prun. Here's an example on how +to do that: + +.. code:: + + (gdb) set args https://www.mozilla.org + (gdb) prun + +How do I set a breakpoint in a library that hasn't been loaded? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +GDB 6.1 and above has support for "pending breakpoints". This is +controlled by the "``set breakpoint pending``" setting, and is enabled +by default. If a breakpoint cannot be immediately resolved, it will be +re-checked each time a shared library is loaded, by the process being +debugged. If your GDB is older than this, you should upgrade. + +In older versions, there isn't a way to set breakpoints in a library +that has not yet been loaded. See more on `setting a breakpoint when a +component is +loaded <#How_do_I_set_a_breakpoint_when_a_component_is_loaded.3F>`__. If +you have to set a breakpoint you can set a breakpoint in ``_dl_open``. +This function is called when a new library is loaded, when you can +finally set your breakpoint. + +How do I set a breakpoint when a component is loaded? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In Firefox Version 57 (and possibly earlier) XPCOM_BREAK_ON_LOAD does +not seem to exist. + +There's a facility in XPCOM which allows you to set an environment +variable to drop into the debugger when loading a certain component. You +have to set ``XPCOM_BREAK_ON_LOAD`` variable before you run Firefox, +setting it to a string containing the names of libraries you want to +load. For example, if you wish to stop when a library named ``raptor`` +or ``necko`` is loaded, you set the variable to ``raptor:necko``. Here's +an example: + +.. code:: + + (gdb) set env XPCOM_BREAK_ON_LOAD raptor:necko + (gdb) prun + +Why can't I set a breakpoint? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You probably can't set a breakpoint because its library hasn't been +loaded. Most Firefox functionality is in libraries loaded mid-way +through the ``main()``\ function. If you break on ``main(),``\ and step +through until the libraries are loaded, with a call to +``InitCOMGlue()``, you should be able to set breakpoints on many more +symbols, source files, and continue running. + +.. code:: + + (gdb) break main + (gdb) run + Breakpoint 1, main(argc=4, argv=0x7fffffffde98, envp=0x7ffffffffdec0) ..... + 256 { + (gdb) next + ... + 293 nsresult rv = InitXPCOMGlue() + (gdb) next + +If you still can't set the breakpoints, you need to confirm the library +has loaded. You can't proceed until the library loads. See more on +`loading shared libraries <#How_do_I_load_shared_libraries.3F>`__. If +you wish to break as soon as the library is loaded, see the section on +`breaking when a component is +loaded <#How_do_I_set_a_breakpoint_when_a_component_is_loaded.3F>`__ and +`breaking on a library +load <#How_do_I_set_a_breakpoint_when_a_component_is_loaded.3F>`__. + +How do I display PRUnichar's? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +One suggestion is this: + +.. code:: + + (gdb) print ((PRUnichar*)uri.mBuffer)[0]@16 + $47 = {114, 100, 102, 58, 110, 117, 108, 108, 0, 0, 8, 0, 0, 0, 37432, + 16514} + + + +.. code:: + + (gdb) print aURI + $1 = (const PRUnichar *) 0x855e6e0 + (gdb) x/32ch aURI + 0x855e6e0: 104 'h' 116 't' 116 't' 112 'p' 58 ':' 47 '/' 47 '/' 119 'w' + 0x855e6f0: 119 'w' 119 'w' 46 '.' 109 'm' 111 'o' 122 'z' 105 'i' 108 'l' + 0x855e700: 108 'l' 97 'a' 46 '.' 111 'o' 114 'r' 103 'g' 47 '/' 115 's' + 0x855e710: 116 't' 97 'a' 114 'r' 116 't' 47 '/' 0 '\0' 25 '\031' 0 '\0' + (gdb) + +- Define helper functions in your .gdbinit + +.. code:: + + # Define a "pu" command to display PRUnichar * strings (100 chars max) + # Also allows an optional argument for how many chars to print as long as + # it's less than 100. + def pu + set $uni = $arg0 + if $argc == 2 + set $limit = $arg1 + if $limit > 100 + set $limit = 100 + end + else + set $limit = 100 + end + # scratch array with space for 100 chars plus null terminator. Make + # sure to not use ' ' as the char so this copy/pastes well. + set $scratch = "____________________________________________________________________________________________________" + set $i = 0 + set $scratch_idx = 0 + while (*$uni && $i++ < $limit) + if (*$uni < 0x80) + set $scratch[$scratch_idx++] = *(char*)$uni++ + else + if ($scratch_idx > 0) + set $scratch[$scratch_idx] = '\0' + print $scratch + set $scratch_idx = 0 + end + print /x *(short*)$uni++ + end + end + if ($scratch_idx > 0) + set $scratch[$scratch_idx] = '\0' + print $scratch + end + end + + # Define a "ps" command to display subclasses of nsAC?String. Note that + # this assumes strings as of Gecko 1.9 (well, and probably a few + # releases before that as well); going back far enough will get you + # to string classes that this function doesn't work for. + def ps + set $str = $arg0 + if (sizeof(*$str.mData) == 1 && ($str.mFlags & 1) != 0) + print $str.mData + else + pu $str.mData $str.mLength + end + end + +`This is hard. Give me a .gdbinit that already has the +functions. <#This_is_hard._Give_me_a_.gdbinit_that_works.>`__ + +- Define a small helper function "punichar" in #ifdef NS_DEBUG code + somewhere. + +How do I display an nsString? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You can call the ToNewCString() method on the nsString. It leaks a +little memory but it shouldn't hurt anything if you only do it a few +times in one gdb session. (via akkana@netscape.com) + +.. code:: + + (gdb) p string.ToNewCString() + +Another method (via bent) is the following (replace ``n`` with: the +returned length of your string): + +.. code:: + + (gdb) p string.Length() + $1 = n + (gdb) x/ns string.BeginReading() + +You can of course use any of the above unichar-printing routines instead +of x/s. + +This is hard. Give me a .gdbinit that works. +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +See `Boris Zbarsky's +.gdbinit <http://web.mit.edu/bzbarsky/www/gdbinit>`__. It contained +several function definitions including: + +- "prun" to start the browser and disable library loading. +- "pu" which will display a (PRUnichar \*) string. +- "ps" which will display a nsString. + +How do I determine the concrete type of an object pointed to by an interface pointer? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You can determine the concrete type of any object pointed to, by an +XPCOM interface pointer, by looking at the mangled name of the symbol +for the object's vtable: + +.. code:: + + (gdb) p aKidFrame + $1 = (nsIFrame *) 0x85058d4 + (gdb) x/wa *(void**)aKidFrame + 0x4210d380 <__vt_14nsRootBoxFrame>: 0x0 + (gdb) p *(nsRootBoxFrame*)aKidFrame + [ all the member variables of aKidFrame ] + +If you're using gcc 3.x, the output is slightly different from the gcc +2.9x output above. Pay particular attention to the vtable symbol, in +this case ``__vt_14nsRootBoxFrame``. You won't get anything useful if +the shared library containing the object is not loaded. See `How do I +load shared libraries? <#How_do_I_load_shared_libraries.3F>`__ and `How +do I see what libraries I already have +loaded? <#How_do_I_see_what_libraries_I_already_have_loaded.3F>`__ + +Or use the gdb command ``set print object on``. + +How can I debug JavaScript from gdb? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you have JavaScript Engine code on the stack, you'll probably want a +JS stack in addition to the C++ stack. + +.. code:: + + (gdb) call DumpJSStack() + +See `https://developer.mozilla.org/en-US/docs/Mozilla/Debugging/Debugging_JavaScript <https://developer.mozilla.org/en-US/docs/Mozilla/Debugging/Debugging_JavaScript>`__ +for more JS debugging tricks. + +How can I debug race conditions and/or how can I make something different happen at NS_ASSERTION time? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +| [submitted by Dan Mosedale] +| As Linux is unable to generate useful core files for multi-threaded + applications, tracking down race-conditions which don't show up under + the debugger can be a bit tricky. Unless you've given the + ``--enable-crash-on-assert`` switch to ``configure``, you can now + change the behavior of ``NS_ASSERTION`` (nsDebug::Break) using the + ``XPCOM_DEBUG_BREAK`` environment variable. + +How do I run the debugger in emacs/xemacs? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Emacs and XEmacs contain modes for doing visual debugging. However, you +might want to set up environment variables, specifying the loading of +symbols and components. The easiest way to set up these is to use the +``run-mozilla.sh`` script, located in the dist/bin directory of your +build. This script sets up the environment to run the editor, shell, +debugger, or defining a preferred setup and running any commands you +wish. For example: + +.. code:: bash + + $ ./run-mozilla.sh /bin/bash + MOZILLA_FIVE_HOME=/home/USER/src/mozilla/build/dist/bin + LD_LIBRARY_PATH=/home/USER/src/mozilla/build/dist/bin + LIBRARY_PATH=/home/USER/src/mozilla/build/dist/bin + SHLIB_PATH=/home/USER/src/mozilla/build/dist/bin + LIBPATH=/home/USER/src/mozilla/build/dist/bin + ADDON_PATH=/home/USER/src/mozilla/build/dist/bin + MOZ_PROGRAM=/bin/bash + MOZ_TOOLKIT= + moz_debug=0 + moz_debugger= + +GDB 5 used to work for me, but now Firefox won't start. What can I do? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +A recent threading change (see `bug +57051 <https://bugzilla.mozilla.org/show_bug.cgi?id=57051>`__ for +details) caused a problem on some systems. Firefox would get part-way +through its initialization, then stop before showing a window. A recent +change to gdb has fixed this. Download and build `the latest version of +Insight <https://sources.redhat.com/insight/>`__, or if you don't want a +GUI, `the latest version of gdb <https://sources.redhat.com/gdb/>`__. + +"run" or "prun" in GDB fails with "error in loading shared libraries." +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Running mozilla-bin inside GDB fails with an error message like: + +.. code:: + + Starting program: + /u/dmose/s/mozilla/mozilla-all/mozilla/dist/bin/./mozilla-bin + /u/dmose/s/mozilla/mozilla-all/mozilla/dist/bin/./mozilla-bin: error + in loading shared libraries: libraptorgfx.so: cannot open shared + object file: No such file or directory + +Your LD_LIBRARY_PATH is probably being reset by your .cshrc or .profile. +From the GDB manual: + +*\*Warning:\* GDB runs your program using the shell indicated by your +'SHELL' environment variable if it exists (or '/bin/sh' if not). If your +'SHELL' variable names a shell that runs an initialization file -- such +as '.cshrc' for C-shell, or '.bashrc' for BASH--any variables you set in +that file affect your program. You may wish to move the setting of +environment variables to files that are only run when you sign on, such +as '.login' or '.profile'.* + +Debian's GDB doesn't work. What do I do? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Debian's unstable distribution currently uses glibc 2.1 and GDB 4.18. +However, there is no package of GDB for Debian with the appropriate +threads patches that will work with glibc 2.1. I was able to get this to +work by getting the GDB 4.18 RPM from Red Hat's rawhide server and +installing that. It has all of the patches necessary for debugging +threaded software. These fixes are expected to be merged into GDB, which +will fix the problem for Debian Linux. (via `Bruce +Mitchener <mailto:bruce@cybersight.com>`__) + +Firefox is aborting. Where do I set a breakpoint to find out where it is exiting? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +On Linux there are two possible symbols that are causing this: +``PR_ASSERT()`` and ``NS_ASSERTION()``. To see where it's asserting you +can stop at two places: + +.. code:: + + (gdb) b abort + (gdb) b exit + +I keep getting a SIGSEGV in JS/JIT code under gdb even though there is no crash when gdb is not attached. How do I fix it? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Set the JS_DISABLE_SLOW_SCRIPT_SIGNALS environment variable (in FF33, +the shorter and easier-to-remember JS_NO_SIGNALS). For an explanation, +read `Jan's blog +post <https://www.jandemooij.nl/blog/2014/02/18/using-segfaults-to-interrupt-jit-code/>`__. + +I keep getting a SIG32 in the debugger. How do I fix it? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you are getting a SIG32 while trying to debug Firefox you might have +turned off shared library loading before the pthreads library was +loaded. For example, ``set auto-solib-add 0`` in your ``.gdbinit`` file. +In this case, you can either: + +- Remove it and use the method explained in the section about `GDB's + memory + usage <#The_debugger_uses_a_lot_of_memory._How_do_I_fix_it.3F>`__ +- Use ``handle SIG32 noprint`` either in gdb or in your ``.gdbinit`` + file + +Alternatively, the problem might lie in your pthread library. If this +library has its symbols stripped, then GDB can't hook into thread +events, and you end up with SIG32 signals. You can check if your +libpthread is stripped in ``file /lib/libpthread*`` and looking for +``'stripped'.``\ To fix this problem on Gentoo Linux, you can re-emerge +glibc after adding ``"nostrip"`` to your ``FEATURES`` in +``/etc/make.conf``. + +How do I get useful stack traces inside system libraries? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Many Linux distributions provide separate packages with debugging +information for system libraries, such as gdb, Valgrind, profiling +tools, etc., to give useful stack traces via system libraries. + +Fedora +^^^^^^ + +On Fedora, you need to enable the debuginfo repositories, as the +packages are in separate repositories. Enable them permanently, so when +you get updates you also get security updates for these packages. A way +to do this is edit ``/etc/yum.repos.d/fedora.repo`` and +``fedora-updates.repo`` to change the ``enabled=0`` line in the +debuginfo section to ``enabled=1``. This may then flag a conflict when +upgrading to a new distribution version. You would the need to perform +this edit again. + +You can finally install debuginfo packages with yum or other package +management tools. The best way is install the ``yum-utils`` package, and +then use the ``debuginfo-install`` command to install all the debuginfo: + +.. code:: bash + + $ yum install yum-utils + $ debuginfo-install firefox + +This can be done manually using: + +.. code:: bash + + $ yum install GConf2-debuginfo ORBit2-debuginfo atk-debuginfo \ + cairo-debuginfo dbus-debuginfo dbus-glib-debuginfo expat-debuginfo \ + fontconfig-debuginfo freetype-debuginfo gcc-debuginfo glib2-debuginfo \ + glibc-debuginfo gnome-vfs2-debuginfo gtk2-debuginfo gtk2-engines-debuginfo \ + hal-debuginfo libX11-debuginfo libXcursor-debuginfo libXext-debuginfo \ + libXfixes-debuginfo libXft-debuginfo libXi-debuginfo libXinerama-debuginfo \ + libXrender-debuginfo libbonobo-debuginfo libgnome-debuginfo \ + libselinux-debuginfo pango-debuginfo popt-debuginfo scim-bridge-debuginfo + +Debugging electrolysis (e10s) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +``mach run`` and ``mach test`` both accept a ``--disable-e10s`` +argument. Some debuggers can't catch child-process crashes without it. + +You can find some (outdated) information on +https://wiki.mozilla.org/Electrolysis/Debugging. You may also like to +read +https://mikeconley.ca/blog/2014/04/25/electrolysis-debugging-child-processes-of-content-for-make-benefit-glorious-browser-of-firefox +for a more up-to-date blog post. + +To get the child process id use: + +.. code:: + + MOZ_DEBUG_CHILD_PROCESS=1 mach run + +See also +~~~~~~~~~ + +- `Debugging <https://developer.mozilla.org/En/Debugging>`__ +- `Performance tools <https://wiki.mozilla.org/Performance:Tools>`__ +- `Fun with + gdb <https://blog.mozilla.com/sfink/2011/02/22/fun-with-gdb/>`__ by + Steve Fink +- `Archer pretty printers for + SpiderMonkey <https://hg.mozilla.org/users/jblandy_mozilla.com/archer-mozilla>`__ + (`blog + post <https://itcouldbesomuchbetter.wordpress.com/2010/12/20/debugging-spidermonkey-with-archer-2/>`__) +- `More pretty + printers <https://hg.mozilla.org/users/josh_joshmatthews.net/archer-mozilla/>`__ + for Gecko internals (`blog + post <https://www.joshmatthews.net/blog/2011/06/nscomptr-has-never-been-so-pretty/>`__) + +.. container:: originaldocinfo + + .. rubric:: Original Document Information + :name: Original_Document_Information + + - `History <http://bonsai-www.mozilla.org/cvslog.cgi?file=mozilla-org/html/unix/debugging-faq.html&rev=&root=/www/>`__ + - Copyright Information: © 1998-2008 by individual mozilla.org + contributors; content available under a `Creative Commons + license <https://www.mozilla.org/foundation/licensing/website-content.html>`__ diff --git a/docs/contributing/debugging/debugging_firefox_with_lldb.rst b/docs/contributing/debugging/debugging_firefox_with_lldb.rst new file mode 100644 index 0000000000..99ae5a60c0 --- /dev/null +++ b/docs/contributing/debugging/debugging_firefox_with_lldb.rst @@ -0,0 +1,80 @@ +Debugging Firefox with LLDB +=========================== + +See http://lldb.llvm.org/index.html. + +Mozilla-specific lldb settings +------------------------------ + +There's an +``.lldbinit`` `file <https://searchfox.org/mozilla-central/source/.lldbinit>`_ +in the Mozilla source tree, which applies recommended settings and +includes a few type summaries and Mozilla-specific debugging commands +via the lldbutils module (see +`python/lldbutils/README.txt <https://searchfox.org/mozilla-central/source/python/lldbutils/README.txt>`__). +For information about available features see the links above and the `Using +LLDB to debug Gecko <http://mcc.id.au/blog/2014/01/lldb-gecko>`__ blog +post. + +The in-tree ``.lldbinit`` should be loaded automatically in most cases +when running lldb from the command line (e.g. using +:ref:`mach`), but **not** +when using Xcode. See :ref:`Debugging on macOS` for information on setting up +Xcode. + +.. warning:: + + LLDB warning: Xcode 5 only comes with lldb (gdb is gone). The + introduction and use of UNIFIED_SOURCES in the source starting around + November 2013 has broken the default LLDB configuration so that it + will not manage to resolve breakpoints in files that are build using + UNIFIED_SOURCES (the breakpoints will be listed as "pending", and + lldb will not stop at them). To fix this add the following to your + $HOME/.lldbinit file: + + .. code:: + + # Mozilla's use of UNIFIED_SOURCES to include multiple source files into a + # single compiled file breaks lldb breakpoint setting. This works around that. + # See http://lldb.llvm.org/troubleshooting.html for more. + settings set target.inline-breakpoint-strategy always + + Restart Xcode/lldb and restart your debugging session. If that still + doesn't fix things then try closing Xcode/lldb, doing a clobber + build, reopening Xcode/lldb, and restarting your debugging session. + +Starting a debugging session +---------------------------- + +Attaching to an existing process +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You can attach to Firefox with following command: + +.. code:: + + (lldb) process attach --name firefox + +Some versions of lldb causes crashes after attaching to Firefox. + +Running a new process +~~~~~~~~~~~~~~~~~~~~~ + +To start Firefox under the debugger, run ``lldb`` followed by "--", +followed by the command line you'd like to run, like this: + +.. code:: bash + + $ lldb -- obj-ff-dbg/dist/Nightly.app/Contents/MacOS/firefox-bin -no-remote -profile /path/to/profile + +Then set breakpoints you need and start the process: + +.. code:: + + (lldb) breakpoint set --name nsInProcessTabChildGlobal::InitTabChildGlobal + Breakpoint created: 1: name = 'nsInProcessTabChildGlobal::InitTabChildGlobal', locations = 0 (pending) + WARNING: Unable to resolve breakpoint to any actual locations. + + (lldb) r + Process 7602 launched: '/.../obj-ff-opt/dist/Nightly.app/Contents/MacOS/firefox-bin' (x86_64) + 1 location added to breakpoint 1 diff --git a/docs/contributing/debugging/debugging_firefox_with_rr.rst b/docs/contributing/debugging/debugging_firefox_with_rr.rst new file mode 100644 index 0000000000..ef68d9c6ea --- /dev/null +++ b/docs/contributing/debugging/debugging_firefox_with_rr.rst @@ -0,0 +1,98 @@ +Debugging Firefox with rr +========================= + +This page is intended to help Firefox/Gecko developers get started using rr to debug Firefox. + +Prerequisites +------------- + +You must have Linux installed with a recent kernel. If you're not running Linux already, an option is set up a virtual machine in which to record Firefox. Be forewarned though that + + * rr requires a VM hypervisor that virtualizes CPU performance counters. VMWare Workstation supports that. + * there's a 20% or so performance hit from running in a VM; generally speaking recorder overhead increases from ~1.2x to ~1.4x. (It's a feather in the cap of the hypervisor authors that the hit is that small, though!) + * Some features (reverse execution) `may not work well in VMWare <https://robert.ocallahan.org/2014/09/vmware-cpuid-conditional-branch.html>`__ due to a VMWare optimization that can be disabled `this way <http://robert.ocallahan.org/2015/11/rr-in-vmware-solved.html>`__. + +When using VSCode, consider adding the `Midas <https://github.com/farre/midas>`__ addon to debug rr traces from the editor. You can use Midas to install / build rr and to configure the :code:`auto-load safe path` via the setup commands. + +Ensure that you've `installed <http://rr-project.org/>`__ or `built <https://github.com/mozilla/rr/wiki/Building-And-Installing>`__ rr and have `used it successfully <https://github.com/mozilla/rr/wiki/Usage>`__. Check that you have the latest version. + +You likely need to configure your :code:`auto-load safe path` for rr / gdb to work correctly. If you are on Linux and your builds are in ~/moz, then add the following to ~/.gdbinit + +.. code:: + + add-auto-load-safe-path ~/moz + +Firefox developers are strongly encouraged to build rr from source. If your Firefox patch triggers a bug in rr, rr developers will fix that bug with high priority. You might be able to pull a fix within a few hours or days instead of waiting for the next release. + +Recording Firefox +----------------- + + +To record Firefox running normally, simply launch it under rr as you would if running it under valgrind or gdb + +.. code:: bash + + $ rr $ff-objdir/dist/bin/firefox ... + +or use mach + +.. code:: bash + + $ ./mach run --debugger=rr + +This will save a trace to your working directory as described in the `usage instructions <https://github.com/mozilla/rr/wiki/Usage>`__. Please refer to `those instructions <https://github.com/mozilla/rr/wiki/Usage>`__ for details on how to debug the recording, which isn't covered in this document. + +Sandboxing reduces recording performance because of the SIGSYS signals and extra syscall. Disabling it can help. + +The background hang monitor might also be making things worse by causing a lot of extra syscalls. It can be disabled by setting +toolkit.content-background-hang-monitor.disabled=true. + +SIGSYS +------ + +When recording and replaying Firefox running with the Linux sandbox, you will get SIGSYS signals frequently. This is expected behavior caused by the sandbox. In gdb, use handle SIGSYS noprint nostop to suppress the signals. + +Recording test suites +--------------------- + +You can use the test runners' --debugger feature to punch rr down through the layers of python script to where Firefox is launched. This is used in the same way you would use --debugger to run valgrind or gdb, for example: + +.. code:: bash + + $ ./mach mochitest --debugger=rr ... + +The test harnesses disable the slow-script timeout when the --debugger argument is passed. That's usually sensible, because you don't want those warnings being generated while Firefox is stopped in gdb. However, this has been `observed to change Gecko behavior <https://bugzilla.mozilla.org/show_bug.cgi?id=986673>`__. rr doesn't need to have the slow-script timeout disabled, so to avoid those kinds of pitfalls, pass the --slowscript argument to the test harness. + +To run rr in chaos mode: + +.. code:: bash + + $ ./mach mochitest --debugger=rr --debugger-args="record --chaos" + +You can also run the entire test harness in rr: + +.. code:: bash + + $ rr ./mach mochitest ... + +The trace will contain many processes, so to debug the correct one, you'll want to use rr ps or rr replay -p firefox etc. + +Working with multiple processes +------------------------------ + +rr should work out of the box with multi-process Firefox. Once you have a recording you can use rr ps to show all the process that were recorded and rr replay -p <pid> to attach to a particular process. + +If you want to debug a particular part of code, you can use the :code:`MOZ_DBG` macro and :code:`getpid()` function to write the process id to stderr. `MOZ_LOG <https://firefox-source-docs.mozilla.org/xpcom/logging.html>`__ will include the pid in log messages by default. + +You can combine that with the -M and -g flags to jump to a particular point in a particular process's lifetime. + +Get help! +--------- + +If you encounter a problem with rr, please `file an issue <https://github.com/mozilla/rr/issues>`__. Firefox bugs are high priority, so usually your issue can be fixed very quickly. + +If you want to chat with rr developers, because you need more help or want to contribute or want to complain, we hang out in the `#rr channel <https://chat.mozilla.org/#/room/#rr:mozilla.org>`__. There is also a channel for `#midas <https://chat.mozilla.org/#/room/#midas:mozilla.org>`__. + +You also may find `these debugging protips <https://github.com/mozilla/rr/wiki/Debugging-protips>`__ helpful, though many are for rr developers, not users. + +Happy debugging! diff --git a/docs/contributing/debugging/debugging_firefox_with_valgrind.rst b/docs/contributing/debugging/debugging_firefox_with_valgrind.rst new file mode 100644 index 0000000000..33aab3638a --- /dev/null +++ b/docs/contributing/debugging/debugging_firefox_with_valgrind.rst @@ -0,0 +1,177 @@ +Debugging Firefox with Valgrind +=============================== + ++--------------------------------------------------------------------+ +| This page is an import from MDN and the contents might be outdated | ++--------------------------------------------------------------------+ + +This page describes how to use Valgrind (specifically, its Memcheck +tool) to find memory errors. + +Supported platforms +------------------- + +Valgrind runs desktop Firefox fine on Linux, especially on x86 and +x86-64. Firefox for Android and Firefox OS on ARMv7 should also run, +though perhaps not as smoothly. The other architectures supported by +Valgrind on Linux (AARCH64, PPC{32,64}, MIPS{32,64}, S390X) should also +work, in theory. + +MacOS X 10.10 (Yosemite), 64-bit only, works, although it can be a bit +of a rough ride. + +- Expect lower performance and a somewhat higher false positive error + rate than on Linux. +- Valgrind's handling of malloc zones on Yosemite is imperfect. Regard + leak reports with caution. +- Valgrind has been known to cause kernel panics, for unknown reasons. + +Where to get Valgrind +--------------------- + +Linux: Download `Valgrind <https://valgrind.org/>`__ directly, or use +your distribution's package manager (if it has a recent enough version). + +MacOSX: `Get Valgrind trunk from +SVN <https://valgrind.org/downloads/repository.html>`__ and build it. +Don't use 3.10.x or any other tarball. + +Make sure you have Valgrind 3.14 or later, version 3.16.1 is known to work, +3.13.0 did not. Newer versions tend to have better compatibility with both +Firefox's JITs and newer toolchain components (compiler, libc and linker +versions). + +Basics +------ + +Build +~~~~~ + +Build Firefox with the following options, which maximize speed and +accuracy. + +.. code:: + + ac_add_options --disable-jemalloc + ac_add_options --disable-strip + ac_add_options --enable-valgrind + ac_add_options --enable-optimize="-g -O2" + ac_add_options --disable-sandbox + +Run +~~~ + +Note that programs run *much* more slowly under Valgrind than they do +natively. Slow-downs of 20x or 30x aren't unexpected, and it's slower on +Mac than on Linux. Don't try this on an underpowered machine. + +Linux +^^^^^ + +On Linux, run Valgrind with the following options. + +.. code:: + + --smc-check=all-non-file --vex-iropt-register-updates=allregs-at-mem-access --show-mismatched-frees=no --read-inline-info=yes + +The ``--smc-check`` and ``--vex-iropt-register-updates`` options are +necessary to avoid crashes in JIT-generated code. + +The ``--show-mismatched-frees`` option is necessary due to inconsistent +inlining of ``new`` and ``delete`` -- i.e. one gets inlined but the +other doesn't -- which lead to false-positive mismatched-free errors. + +The ``--read-inline-info`` option improves stack trace readability in +the presence of inlining. + +Also, run with the following environment variable set. + +.. code:: + + G_SLICE=always-malloc + +This is necessary to get the Gnome system libraries to use plain +``malloc`` instead of pool allocators. + +Mac +^^^ + +On Mac, run Valgrind with the following options. + +.. code:: + + --smc-check=all-non-file --vex-iropt-register-updates=allregs-at-mem-access --show-mismatched-frees=no --dsymutil=yes + +The ``--dsymutil`` option ensures line number information is present in +stack traces. + +Advanced usage +-------------- + +Shared suppression files +~~~~~~~~~~~~~~~~~~~~~~~~ + +`/build/valgrind/ <https://searchfox.org/mozilla-central/source/build/valgrind/>`__ +contains the suppression files used by the periodic Valgrind jobs on +Treeherder. Some of these files are platform-specific. + +Running mochitests under Valgrind? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To run a mochitest under Valgrind, use the following command. + +.. code:: bash + + $ ./mach mochitest-plain --debugger="valgrind" --debugger-args="$VALGRIND_OPTIONS" relative/path/to/tests + +Where ``$VALGRIND_OPTIONS`` are the options described +:ref:`above <Debugging Firefox With Valgrind>`. You might also +need ``--trace-children=yes`` to trace into child processes. + +As of December 2014 it is possible to do a complete run of +mochitests-plain on Valgrind in about 8 CPU hours on a Core i4910 +(Haswell) machine. Maximum process size is 5.4G, of which about 80% is +in memory. Runs of small subsets of mochitests take far less memory. + +Bits and pieces +~~~~~~~~~~~~~~~ + +For un-released Linux distros (Fedora Rawhide, etc.) you'll need to use +a version of Valgrind trunk build, because fixes for the latest gcc and +glibc versions appear there first. Without them you'll be flooded with +false errors from Memcheck, and have debuginfo reading problems. + +On Linux, code compiled by LLVM at high optimisation levels can cause +Memcheck to report false uninitialised value errors. See +`here <https://bugs.kde.org/show_bug.cgi?id=242137#c3>`__ for an easy +workaround. On Mac, Valgrind has this workaround built in. + +You can make stack traces easier to read by asking for source file names +to be given relative to the root of your source tree. Do this by using +``--fullpath-after=`` to specify the rightmost part of the absolute path +that you don't want to see. For example, if your source tree is rooted +at ``/home/sewardj/MC-20-12-2014``, use ``--fullpath-after=2014/`` to +get path names relative to the source directory. + +The ``--track-origins=yes`` slows down Valgrind greatly, so don't use it +unless you are hunting down a specific uninitialised value error. But if +you are hunting down such an error, it's extremely helpful and worth +waiting for. + +Additional help +--------------- + +The `Valgrind Quick Start +Guide <http://www.valgrind.org/docs/manual/quick-start.html>`__ is short +and worth reading. The `User +Manual <https://valgrind.org/docs/manual/manual.html>`__ is also useful. + +If Valgrind asserts, crashes, doesn't do what you expect, or otherwise +acts up, first of all read this page and make sure you have both Firefox +and Valgrind correctly configured. If that's all OK, try using the +`Valgrind trunk from +SVN <http://www.valgrind.org/downloads/repository.html>`__. Oftentimes +bugs are fixed in the trunk before most users fall across them. If that +doesn't help, consider `filing a bug +report <http://www.valgrind.org/support/bug_reports.html>`__, and/or +mailing Julian Seward or Nick Nethercote. diff --git a/docs/contributing/debugging/debugging_on_macos.rst b/docs/contributing/debugging/debugging_on_macos.rst new file mode 100644 index 0000000000..ef42c162f9 --- /dev/null +++ b/docs/contributing/debugging/debugging_on_macos.rst @@ -0,0 +1,359 @@ +Debugging On macOS +================== + +This document explains how to debug Gecko-based applications such as +Firefox, Thunderbird, and SeaMonkey on macOS using Xcode. If you want to +debug from the terminal see :ref:`Debugging Mozilla with +lldb <Debugging Firefox with LLDB>`. For specific +information on a way to debug hangs, see :ref:`Debugging a hang on macOS <Debugging A Hang On macOS>`. + +Creating a debuggable build +--------------------------- + +First, you need to build the application you're going to debug using +this in your .mozconfig + +.. code:: + + ac_add_options --disable-optimize + ac_add_options --enable-debug-symbols + +you can also add this flag if you want assertions etc. compiled in + +.. code:: + + ac_add_options --enable-debug + +See :ref:`Building Firefox for macOS <Building Firefox On MacOS>` +if you need help creating your own build. + +Debugging Firefox on macOS 10.14+ +--------------------------------- + +macOS 10.14 introduced Notarization and Hardened Runtime features for +improved application security. macOS 10.15 went further, requiring +applications to be Notarized with Hardened Runtime enabled in order to +launch (ignoring workarounds). When run on earlier macOS versions, +Notarization and Hardened Runtime settings have no effect. + +Official Builds +~~~~~~~~~~~~~~~ + +At this time, official builds of Firefox 69 and later are Notarized. +**As a result, it is not possible to attach a debugger to these official +Firefox releases on macOS 10.14+ without disabling System Integrity +Protection (SIP).** This is due to Notarization requiring Hardened +Runtime to be enabled with the ``com.apple.security.get-task-allow`` +entitlement disallowed. **Rather than disabling SIP (which has security +implications), it is recommended to debug with try builds or local +builds. The differences are explained below.** + +try Server Builds +~~~~~~~~~~~~~~~~~ + +In most cases, developers needing to debug a build as close as possible +to the production environment should use a :ref:`try +build <Pushing to Try>`. These +builds enable Hardened Runtime and only differ from production builds in +that they are not Notarized which should not otherwise affect +functionality, (other than the ability to easily launch the browser on +macOS 10.15+ -- see quarantine note below). At this time, developers can +obtain a Hardened Runtime build with the +``com.apple.security.get-task-allow`` entitlement allowed by submitting +a try build and downloading the dmg generated by the "Rpk" shippable +build job. A debugger can be attached to Firefox processes of these +builds. try builds use the ``developer.entitlements.xml`` file from the +source tree while production builds use ``production.entitlements.xml``. +**On macOS 10.15+, downloaded try builds will not launch by default +because Notarization is required. To workaround this problem, remove the +quarantine extended attribute from the downloaded Nightly:** + + ``$ xattr -r -d com.apple.quarantine /Path/to/Nightly.app`` + +Local Builds +~~~~~~~~~~~~ + +Local builds of mozilla-central do not enable Hardened Runtime and hence +do not have debugging restrictions. As a result, some functionality will +be permitted on local builds, but blocked on production builds which +have Hardened Runtime enabled. `Bug +1522409 <https://bugzilla.mozilla.org/show_bug.cgi?id=1522409>`__ was +filed to automate codesigning local builds to enable Hardened Runtime by +default and eliminate this discrepancy. + +To obtain a Hardened Runtime build without using try infrastructure, a +developer can manually codesign builds using the macOS ``codesign(1)`` +command with the ``developer.entitlements.xml`` file from the tree. This +requires creating a codesigning identity. + +Disabling System Integrity Protection (SIP) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If debugging a production build is required, follow Apple's documented +steps for disabling System Integrity Protection (SIP). Note that +disabling SIP bypasses Hardened Runtime restrictions which can mask some +bugs that only occur with Hardened Runtime so it is recommended to test +fixes with SIP enabled. **Disabling SIP has system security implications +that should be understood before taking this step.** + +Creating an Xcode project +------------------------- + +If you try to create a new Xcode project in an existing directory +then Xcode will delete its existing contents (Xcode will warn you +beforehand). To work around that, the steps below have you initialize +the project outside the Mozilla source tree, close the project, copy +the .xcodeproj project "file" into the source tree, and then reopen +the project to finish setting it up. + +Note also that since Xcode 7.3.1 it doesn't seem to be possible to +have the Xcode project live outside the source tree. If you try to do +that then Xcode will simply **copy** the source files under the +project directory rather than link to them which breaks debugging and the +possibility to modify-rebuild-relaunch from inside Xcode. + +These steps were last updated for Xcode 10.3: + +#. Open Xcode, and create a new Project with File > New Project. Select + the "Cross-platform" tab then under the "Other" template group select + the "Empty" project type. the click Next. Name the project and click + Next. Create/select a temporary directory to contain the project and + then click Create. +#. Before going any further, close the project (File > Close Project) + and open Finder. Find the \*.xcodejproj directory in the temporary + directory, move it into your Mozilla source tree, and then + double-click on it to reopen it. +#. In the left-hand pane in Xcode you should see a tree item where the + root item has the project name. If the temporary directory that you + originally created the Xcode project in is under that, right click it + and delete it. Now, right click on the root item, select 'Add files + to "<project-name>"', select all the files and directories in your + source directory, untick "Copy items if needed", then click Add. + (These will then be progressively added under the root item + <project-name> in the left-hand pane. Note that subdirectories may + initially appear to be empty, but they too will progressively be + populated as Xcode processes the sourse files. Once done, you should + be able to open any file quickly by hitting Cmd-Shift-O and typing in + the name of a file.) +#. In the Product menu, select Scheme > New Scheme and name your scheme + (for example, "Debug"). After you click OK, Xcode should open the + settings window for the new scheme. (If not, then open its settings + from the Product > Edit Scheme menu.) +#. Select "Run" on the left-hand side of the settings window, then + select the "Info" tab. Set the Executable by clicking on "None" and + selecting "Other...". A new dialog titled "Choose an executable to + launch" will pop up. Browse to the ``.app`` file that you want to + debug (``Firefox.app``, ``Nightly``\ ``Debug.app`` etc). The ``.app`` + file is typically found inside the ``dist`` folder in your build + directory. +#. If you are debugging Firefox, Thunderbird, or some other application + that supports multiple profiles, using a separate profile for + debugging purposes is recommended. See "Having a profile for + debugging purposes" below. Select the "Arguments" tab in the scheme + editor, and click the '+' below the "Arguments passed on launch" + field. Add "-P *profilename*", where *profilename* is the name of a + profile you created previously. Repeat that to also add the argument + "-no-remote". +#. Also in the "Arguments" panel, you may want to add an environment + variable MOZ_DEBUG_CHILD_PROCESS set to the value 1 to help with + debugging e10s. +#. Select "Build" from the left of the scheme editor window, and check + that there is nothing listed under Targets (otherwise it may cause + problems when you try to run the executable for debugging since you + will get build errors). +#. Click "Close" to close the scheme editor. + +At this point you can run the application from Xcode, and when you pause +or hit breakpoints it should show open the correct source file at the +correct line. + +Setting up lldb +--------------- + +``lldb`` is the debugger Xcode provides/uses. + +.. warning:: + + One important issue that the Mozilla .lldbinit file fixes is that by + default some breakpoints will be listed as "pending", and Xcode will + not stop at them. If you don't include the Mozilla's .lldbinit, you + must at least put + ``settings set target.inline-breakpoint-strategy always`` in your + ``$HOME/.lldbinit`` as recommended on :ref:`Debugging Firefox with + lldb <Debugging Firefox with LLDB>`. + +The +`.lldbinit <http://searchfox.org/mozilla-central/source/.lldbinit>`__ +file in the source tree imports many useful `Mozilla specific lldb +settings, commands and +formatters <https://searchfox.org/mozilla-central/source/python/lldbutils/README.txt>`__ +into ``lldb``, but you may need to take one of the following steps to +make sure this file is used. + +If you are using ``lldb`` on the command line (independently of Xcode) +and you will always run it from either the top source directory, the +object directory or else the dist/bin subdirectory of the object +directory, then adding the following setting to your ``$HOME/.lldbinit`` +is sufficient: + +:: + + settings set target.load-cwd-lldbinit true + +*However*, if you will run lldb from a different directory, or if you +will be running it indirectly by debugging in Xcode (Xcode always runs +lldb from "/"), then this setting will not help you. Instead, add the +following to your ``$HOME/.lldbinit``: + +:: + + # This automatically sources the Mozilla project's .lldbinit as soon as lldb + # starts or attaches to a Mozilla app (that's in an object directory). + # + # This is mainly a workaround for Xcode not providing a way to specify that + # lldb should be run from a given directory. (Xcode always runs lldb from "/", + # regardless of what directory Xcode was started from, and regardless of the + # value of the "Custom working directory" field in the Scheme's Run options. + # Therefore setting `settings set target.load-cwd-lldbinit true` can't help us + # without Xcode providing that functionality.) + # + # The following works by setting a one-shot breakpoint to break on a function + # that we know will both run early (which we want when we start first start the + # app) and run frequently (which we want so that it will trigger ASAP if we + # attach to an already running app). The breakpoint runs some commands to + # figure out the object directory path from the attached target and then + # sources the .lldbinit from there. + # + # NOTE: This scripts actions take a few seconds to complete, so the custom + # formatters, commands etc. that are added may not be immediately available. + # + breakpoint set --name nsThread::ProcessNextEvent --thread-index 1 --auto-continue true --one-shot true + breakpoint command add -s python + # This script that we run does not work if we try to use the global 'lldb' + # object, since it is out of date at the time that the script runs (for + # example, `lldb.target.executable.fullpath` is empty). Therefore we must + # get the following objects from the 'frame' object. + target = frame.GetThread().GetProcess().GetTarget() + debugger = target.GetDebugger() + + # Delete our breakpoint (not actually necessary with `--one-shot true`): + target.BreakpointDelete(bp_loc.GetBreakpoint().GetID()) + + # For completeness, find and delete the dummy breakpoint (the breakpoint + # lldb creates when it can't initially find the method to set the + # breakpoint on): + # BUG WORKAROUND! GetID() on the *dummy* breakpoint appears to be returning + # the breakpoint index instead of its ID. We have to add 1 to correct for + # that! :-( + dummy_bp_list = lldb.SBBreakpointList(target) + debugger.GetDummyTarget().FindBreakpointsByName("nsThread::ProcessNextEvent", dummy_bp_list) + dummy_bp_id = dummy_bp_list.GetBreakpointAtIndex(0).GetID() + 1 + debugger.GetDummyTarget().BreakpointDelete(dummy_bp_id) + + # "source" the Mozilla project .lldbinit: + os.chdir(target.executable.fullpath.split("/dist/")[0]) + debugger.HandleCommand("command source -s true " + os.path.join(os.getcwd(), ".lldbinit")) + DONE + +see :ref:`Debugging Mozilla with +lldb <Debugging Firefox with LLDB>`. for more information. + +Having a profile for debugging purposes +--------------------------------------- + +It is recommended to create a separate profile to debug with, whatever +your task, so that you don't lose precious data like Bookmarks, saved +passwords, etc. So that you're not bothered with the profile manager +every time you start to debug, expand the "Executables" branch of the +"Groups & Files" list and double click on the Executable you added for +Mozilla. Click the plus icon under the "Arguments" list and type "-P +<profile name>" (e.g. "-P MozillaDebug"). Close the window when you're +done. + +Running a debug session +----------------------- + +Make sure breakpoints are active (which implies running under the +debugger) by opening the Product menu and selecting "Debug / Activate +Breakpoints" (also shown by the "Breakpoints" button in the top right +section of the main window). Then click the "Run" button or select "Run" +from the Product menu. + +Setting breakpoints +~~~~~~~~~~~~~~~~~~~ + +Setting a breakpoint is easy. Just open the source file you want to +debug in Xcode, and click in the margin to the left of the line of code +where you want to break. + +During the debugging session, each time that line is executed, the +debugger will break there, and you will be able to debug it. + +.. warning:: + + Note that with the default configuration, some breakpoints will be + listed as "pending", and Xcode will not stop at them. If you don't + include the Mozilla's .lldbinit, you must at least put + ``settings set target.inline-breakpoint-strategy always`` in your + ``$HOME/.lldbinit`` as recommended on :ref:`Debugging Mozilla with + lldb <Debugging Firefox with LLDB>`. + +Using Firefox-specific lldb commands +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you included the .lldbinit when `Setting up +lldb <#setting-up-lldb>`__, you can use Mozilla-specific lldb commands +in the console, located in the Debug area of Xcode. For example, type +``js`` to see the JavaScript stack. For more information, see :ref:`Debugging +Mozilla with lldb <Debugging Firefox with LLDB>`. + +Debugging e10s child processes +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Using Xcode to debug child processes created by an e10s-enabled browser +is a little trickier than debugging a single-process browser, but it can +be done. These directions were written using Xcode 6.3.1 + +#. Complete all the steps above under "Creating the Project" +#. From the "Product" menu, ensure the scheme you created is selected + under "Scheme", then choose "Scheme > Edit Scheme" +#. In the resulting popup, click "Duplicate Scheme" +#. Give the resulting scheme a more descriptive name than "Copy of + Scheme" +#. Select "Run" on the left-hand side of the settings window, then + select the "Info" tab. Set the Executable by clicking on the + "Executable" drop-down, and selecting the ``plugin-container.app`` + that is inside the app bundle of the copy of Firefox you want to + debug. +#. On the same tab, under "Launch" select "Wait for executable to be + launched" +#. On the "Arguments" tab, remove all arguments passed on launch. + +Now you're ready to start debugging: + +#. From the "Product" menu, ensure the scheme you created above is + selected under "Scheme" +#. Click the "Run" button. The information area at the top of the window + will show "Waiting for plugin-container to launch" +#. From a command line, run your build of Firefox. When that launches a + child process (for example, when you start to load a webpage), Xcode + will notice and attach to that child process. You can then debug the + child process like you would any other process. +#. When you are done debugging, click the "Stop" button and quit the + instance of Firefox that you were debugging in the normal way. + +For some help on using lldb see :ref:`Debugging Mozilla with +lldb <Debugging Firefox with LLDB>`. + +Other resources +--------------- + +Apple has an extensive list of `debugging tips and +techniques <https://developer.apple.com/library/mac/#technotes/tn2124/_index.html>`__. + +Questions? Problems? +~~~~~~~~~~~~~~~~~~~~ + +Try asking in our Element channels +`#developers <https://chat.mozilla.org/#/room/#developers:mozilla.org>`__ or +`#macdev <https://chat.mozilla.org/#/room/#macdev:mozilla.org>`__. diff --git a/docs/contributing/debugging/debugging_on_windows.rst b/docs/contributing/debugging/debugging_on_windows.rst new file mode 100644 index 0000000000..3213299199 --- /dev/null +++ b/docs/contributing/debugging/debugging_on_windows.rst @@ -0,0 +1,330 @@ +Debugging On Windows +==================== + +This document explains how to debug Gecko based applications such as +Firefox, Thunderbird, and SeaMonkey on Windows using the Visual Studio IDE. + +If VS and your Gecko application hang shortly after you launch the +application under the debugger, see `Problems Loading Debug +Symbols <#problems-loading-debug-symbols>`__. + +Ways to start the debugger +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +First of all, it's necessary to install a Visual Studio extension to be +able to follow child processes as they are created. Firefox, in general, +and even in non-e10s mode, does not start the main process directly, it +starts it via a Launcher Process. This means that Visual Studio will +only attach to the first process it finds, and will not hit any +break-point (and even notifies you that it cannot find their location). +`Microsoft Child Process Debugging Power +Tool <https://marketplace.visualstudio.com/items?itemName=vsdbgplat.MicrosoftChildProcessDebuggingPowerTool>`__ +allows automatically attaching to child processes, such as Web Content +process, GPU process, etc. Enable it by going its configuration menu in +"Debug > Other debugging targets > Child process debugging settings", +and ticking the box. + +If you have followed the steps in :ref:`Building Firefox for +Windows <Building Firefox On Windows>` +and have a local debug build, you can **execute this command from same command line.** + +.. code:: + + ./mach run --debug + +It would open Visual Studio with Firefox's +run options configured. You can **click "Start" button** to run Firefox +then, already attached in the debugger. + +Alternatively, if you have generated the Visual Studio solution, via +``./mach build-backend -b VisualStudio``, opening this solution allows +you to run ``firefox.exe`` directly in the debugger. To make it the +startup project, right click on the project and select ``Set As Startup +Project``. It appears bold when it's the case. Breakpoints are kept +across runs, this can be a good way to debug startup issues. + +**Run the program until you hit an assertion.** You will get a dialog +box asking if you would like to debug. Hit "Cancel". The MSDEV IDE will +launch and load the file where the assertion happened. This will also +create a Visual Studio Mozilla project in the directory of the executable +by default. + +**Attach the debugger to an existing Mozilla process**. In the Visual +Studio, select Debug > Attach to Process. If you want to debug a content +process, you can **hover on the tab** of page you want to debug, which +would show the pid. You can then select the process from dialog opened +from "Attach to Process". You can open ``about:processes`` to see the pid +for all subprocesses, including tabs but also GPU, networking etc. +For more information, see `Attach to Running Processes with the Visual Studio +Debugger <http://msdn.microsoft.com/en-us/library/vstudio/3s68z0b3.aspx>`__. + +**Starting an MSIX installed Firefox with the debugger**. In Visual +Studio, select Debug -> Other Debug Targets -> Debug Installed App Package. +In the dialog, select the installed Firefox package you wish to debug +and click "Start". + +Debugging Release and Nightly Builds +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Refer to the steps to :ref:`use the Mozilla symbol +server <Using The Mozilla Symbol Server>` and :ref:`source +server <Using The Mozilla Source Server>` + +Creating a Visual Studio project for Firefox +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Please refer to :ref:`this <Visual Studio Projects>`. + +Changing/setting the executable to debug +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To change or set the executable to debug, go to Project > Properties > +Debugging > Command. (As of Visual Studio 2022.) + +It should show the executable you are debugging. If it is empty or +incorrect, manually add the correct path to the executable. + +Command line parameters and environment variables +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To change or set the command line options, go to Project > Properties > +Debugging > Command Arguments. + +Some common options would be the URL of the file you want the browser to +open as soon as it starts, starting the Profile Manager, or selecting a +profile. You can also redirect the console output to a file (by adding +"``> filename.txt``" for example, without the quotes). + +Customizing the debugger's variable value view +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You can customize how Visual Studio displays classes in the variable view. +By default VS displays "{...}" and you need to click the small + icon +to expand the members. You can change this behaviour, and make Visual +Studio display whatever data member you want in whatever order, formatted +however you like instead of just "{...}". + +You need to locate a file called "gecko.natvis" under toolkit/library. +The file contains a list of types and how they should be displayed in +the debugger. It is XML and after a little practice you should be well +on your way. + +To understand the file in detail refer to `Create custom views of C++ +objects in the debugger using the Natvis framework +<https://docs.microsoft.com/en-us/visualstudio/debugger/create-custom-views-of-native-objects>`__ + +The file already comes with a number of entries that will make your life +easier, like support for several string types. If you need to add a custom +type, or want to change an existing entry for debugging purposes, you can +easily edit the file. For your convenience it is included in all generated +Visual Studio projects, and if you edit and save it within Visual Studio, it +will pick up the changes immediately. + +Handling multiple processes in Visual Studio +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Turn off "Break all processes when one process breaks" to single step a single +process. + +Turning off "Break all processes when one process breaks" adds "Step Into +Current Process", "Step Over Current Process" and "Step Out Current Process" to +the "Debug" menu. + +To single step a single process with the other processes paused: + +- Turn on "Break all processes when one process breaks" +- Hit a breakpoint which stops all processes +- Turn off "Break all processes when one process breaks" +- Now using "Step Into Current Process" will leave the other processes stopped + and just advance the current one. + +Obtaining ``stdout`` and other ``FILE`` handles +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Running the following command in the Command Window in Visual Studio +returns the value of ``stdout``, which can be used with various +debugging methods (such as ``nsGenericElement::List``) that take a +``FILE*`` param: + +.. code:: + + Debug.EvaluateStatement {,,msvcr80d}(&__iob_func()[1]) + +(Alternatively you can evaluate ``{,,msvcr80d}(&__iob_func()[1])`` in +the Immediate window) + +Similarly, you can open a file on the disk using ``fopen``: + +.. code:: + + >Debug.EvaluateStatement {,,msvcr80d}fopen("c:\\123", "w") + 0x10311dc0 { ..snip.. } + >Debug.EvaluateStatement ((nsGenericElement*)0x03f0e710)->List((FILE*)0x10311dc0, 1) + <void> + >Debug.EvaluateStatement {,,msvcr80d}fclose((FILE*)0x10311dc0) + 0x00000000 + +Note that you may not see the debugging output until you flush or close +the file handle. + +Disabling ASSERTIONS +~~~~~~~~~~~~~~~~~~~~ + +There are basically two ways to disable assertions. One requires setting +an environment variable, while the other affects only the currently +running program instance in memory. + +Environment variable +^^^^^^^^^^^^^^^^^^^^ + +There is an environment variable that can disable breaking for +assertions. This is how you would normally set it: + +.. code:: + + set XPCOM_DEBUG_BREAK=warn + +The environment variable takes also other values besides ``warn``, see +``XPCOM_DEBUG_BREAK`` for more details. + +Note that unlike Unix, the default for Windows is not warn, it's to pop +up a dialog. To set the environment variable for Visual Studio, use +Project > Properties > Debugging > Environment and click the little box. +Then use + +.. code:: + + XPCOM_DEBUG_BREAK=warn + +Changing running code +^^^^^^^^^^^^^^^^^^^^^ + +You normally shouldn't need to do this (just quit the application, set +the environment variable described above, and run it again). And this +can be **dangerous** (like **trashing your hard disc and corrupting your +system**). So unless you feel comfortable with this, don't do it. **You +have been warned!** + +It is possible to change the interrupt code in memory (which causes you +to break into debugger) to be a NOP (no operation). + +You do this by running the program in the debugger until you hit an +assertion. You should see some assembly code. One assembly code +instruction reads "int 3". Check the memory address for that line. Now +open memory view. Type/copy/drag the memory address of "int 3" into the +memory view to get it to update on that part of the memory. Change the +value of the memory to "90", close the memory view and hit "F5" to +continue. + +Automatically handling ASSERTIONS without a debugger attached +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +When an assertion happens and there is not a debugger attached, a small +helper application +(```windbgdlg.exe`` </En/Automatically_Handle_Failed_Asserts_in_Debug_Builds>`__) +is run. That application can automatically select a response to the "Do +you want to debug" dialog instead of prompting if you configure it, for +more info, see +```windbgdlg.exe`` </En/Automatically_Handle_Failed_Asserts_in_Debug_Builds>`__. + +Debugging optimized builds +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To effectively debug optimized builds, you should enable debugging +information which effectively leaves the debug symbols in optimized code +so you can still set breakpoints etc. Because the code is optimized, +stepping through the code may occasionally provide small surprises when +the debugger jumps over something. + +You need to make sure this configure parameter is set: + +.. code:: + + ac_add_options --enable-debug + +You can also choose to include or exclude specific modules. + +Console debugging +~~~~~~~~~~~~~~~~~ + +When printing to STDOUT from a content process, the console message will +not appear on Windows. One way to view it is simply to disable e10s +(``./mach run --disable-e10s``) but in order to debug with e10s enabled +one can run + +:: + + ./mach run ... 2>&1 | tee + +It may also be necessary to disable the content sandbox +(``MOZ_DISABLE_CONTENT_SANDBOX=1 ./mach run ...``). + +Running two instances of Mozilla simultaneously +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You can run two instances of Mozilla (e.g. debug and optimized) +simultaneously by setting the environment variable ``MOZ_NO_REMOTE``: + +.. code:: + + set MOZ_NO_REMOTE=1 + +Or, starting with Firefox 2 and other Gecko 1.8.1-based applications, +you can use the ``-no-remote`` command-line switch instead (implemented +in +`bug 325509 <https://bugzilla.mozilla.org/show_bug.cgi?id=325509>`__). + +You can also specify the profile to use with the ``-P profile_name`` +command-line argument. + +Debugging JavaScript +~~~~~~~~~~~~~~~~~~~~ + +You can use helper functions from +`nsXPConnect.cpp <https://searchfox.org/mozilla-central/source/js/xpconnect/src/nsXPConnect.cpp>`__ +to inspect and modify the state of JavaScript code from the MSVS +debugger. + +For example, to print current JavaScript stack to stdout, evaluate this +in Immediate window: + +.. code:: + + {,,xul}DumpJSStack() + +Visual Studio will show you something in the quick watch window, but +not the stack, you have to look in the OS console for the output. + +Also this magical command only works when you have JS on the VS stack. + +Debugging minidumps +~~~~~~~~~~~~~~~~~~~ + +See :ref:`debugging a minidump <Debugging A Minidump>`. + +Problems post-mortem debugging on Windows 7 SP1 x64? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you attempt to use ``NS_DebugBreak`` etc to perform post-mortem +debugging on a 64bit Windows 7, but as soon as you try and continue +debugging the program crashes with an Access Violation, you may be +hitting a Windows bug relating to AVX support. For more details, +including a work-around see `this blog +post <http://www.os2museum.com/wp/?p=960>`__ or `this social.msdn +thread <http://social.msdn.microsoft.com/Forums/vstudio/en-US/392ca62c-e502-42d9-adbc-b4e22d5da0c3/jit-debugging-32bit-app-crashing-with-access-violation>`__. +(And just in-case those links die, the work-around is to execute + +:: + + bcdedit /set xsavedisable 1 + +from an elevated command-prompt to disable AVX support.) + +Got a tip? +~~~~~~~~~~ + +If you think you know a cool Mozilla debugging trick, feel free to +discuss it with `#developers <https://chat.mozilla.org/#/room/#developers:mozilla.org>`__ and +then post it here. + +.. |Screenshot of disabling assertions| image:: https://developer.mozilla.org/@api/deki/files/420/=Win32-debug-nop.png + :class: internal diff --git a/docs/contributing/debugging/img/about-processes.png b/docs/contributing/debugging/img/about-processes.png Binary files differnew file mode 100644 index 0000000000..6f1563bcd1 --- /dev/null +++ b/docs/contributing/debugging/img/about-processes.png diff --git a/docs/contributing/debugging/img/about-support.png b/docs/contributing/debugging/img/about-support.png Binary files differnew file mode 100644 index 0000000000..2b12d1027d --- /dev/null +++ b/docs/contributing/debugging/img/about-support.png diff --git a/docs/contributing/debugging/img/crash-gmp.png b/docs/contributing/debugging/img/crash-gmp.png Binary files differnew file mode 100644 index 0000000000..595e439307 --- /dev/null +++ b/docs/contributing/debugging/img/crash-gmp.png diff --git a/docs/contributing/debugging/img/crash-gpu.png b/docs/contributing/debugging/img/crash-gpu.png Binary files differnew file mode 100644 index 0000000000..9e40b63917 --- /dev/null +++ b/docs/contributing/debugging/img/crash-gpu.png diff --git a/docs/contributing/debugging/img/crash-rdd.png b/docs/contributing/debugging/img/crash-rdd.png Binary files differnew file mode 100644 index 0000000000..5548672d27 --- /dev/null +++ b/docs/contributing/debugging/img/crash-rdd.png diff --git a/docs/contributing/debugging/img/crashlist.jpg b/docs/contributing/debugging/img/crashlist.jpg Binary files differnew file mode 100644 index 0000000000..4cb376d0f2 --- /dev/null +++ b/docs/contributing/debugging/img/crashlist.jpg diff --git a/docs/contributing/debugging/img/crashreporter.png b/docs/contributing/debugging/img/crashreporter.png Binary files differnew file mode 100644 index 0000000000..3137c71c78 --- /dev/null +++ b/docs/contributing/debugging/img/crashreporter.png diff --git a/docs/contributing/debugging/img/process-explorer.png b/docs/contributing/debugging/img/process-explorer.png Binary files differnew file mode 100644 index 0000000000..26e7837a8d --- /dev/null +++ b/docs/contributing/debugging/img/process-explorer.png diff --git a/docs/contributing/debugging/img/sdk-installer.png b/docs/contributing/debugging/img/sdk-installer.png Binary files differnew file mode 100644 index 0000000000..c7dcc1f1a4 --- /dev/null +++ b/docs/contributing/debugging/img/sdk-installer.png diff --git a/docs/contributing/debugging/img/tabcrashed.png b/docs/contributing/debugging/img/tabcrashed.png Binary files differnew file mode 100644 index 0000000000..9982e527ed --- /dev/null +++ b/docs/contributing/debugging/img/tabcrashed.png diff --git a/docs/contributing/debugging/img/windbg-in-startmenu.png b/docs/contributing/debugging/img/windbg-in-startmenu.png Binary files differnew file mode 100644 index 0000000000..0819d4b424 --- /dev/null +++ b/docs/contributing/debugging/img/windbg-in-startmenu.png diff --git a/docs/contributing/debugging/local_symbols.rst b/docs/contributing/debugging/local_symbols.rst new file mode 100644 index 0000000000..875717e1a1 --- /dev/null +++ b/docs/contributing/debugging/local_symbols.rst @@ -0,0 +1,64 @@ +Symbolicating TreeHerder stacks locally +======================================= + +When using tools like the :ref:`Dark Matter Detector (DMD)` or +:ref:`refcount logging<Refcount Tracing and Balancing>` to +investigate issues occurring on TreeHerder that you can't reproduce locally, you +can often end up with unsymbolicated stacks. Fortunately, there is a way to +symbolicate these stacks on your own machine. + +These instructions are for a Linux TreeHerder build for MacOS, so they might +require some modifications for other combinations of platforms. + +Download ``target.tar.bz2`` and ``target.crashreporter-symbols.zip`` from the +Build job. **Note that these files are very large so you'll want to delete +them and the extracted files when you are done.** + +These files each contain a large number of files, so I'd recommend creating +a directory for each of them. Call these ``<TARGET_DIR>`` and ``<SYMB_DIR>``, +and move the prior two files into these two directories, respectively. + +Go to ``<TARGET_DIR>`` and run something like + +.. code-block:: shell + + tar xf target.tar.bz2 + +then go to ``<SYMB_DIR>`` and run something like + +.. code-block:: shell + + unzip target.crashreporter-symbols.zip + +You should be able to delete the two original files now. + +Next we need to ensure that the locations of binaries are rewritten from +where they are on TreeHerder to where we have them locally. We'll do this by +editing ``fix_stacks.py``. This file is located in the ``tools/rb/`` directory of +the Firefox source directory. You need to add these two lines to the function +``fixSymbols``, after ``line_str`` is defined and before it is written to +``fix_stacks.stdin``. I've done this right before the definition of +``is_missing_newline``. + +.. code-block:: python + + line_str = line_str.replace("/builds/worker/workspace/build/application/firefox/firefox", + "<TARGET_DIR>/firefox/firefox-bin") + line_str = line_str.replace("/builds/worker/workspace/build/application/firefox/libxul.so", + "<TARGET_DIR>/firefox/libxul.so") + +The initial locations should appear verbatim in the stack you are trying to +symbolicate, so double check that they match. Also, ``<TARGET_DIR>`` of course +needs to be replaced with the actual local directories where those files are +located. Note that the ``firefox`` executable is changed to ``firefox-bin``. +I don't know why that is necessary, but only the latter existed for me. + +Finally, we need to make it so that the stack fixer can find the location of +the breakpad symbols we downloaded. If you are running ``fix_stacks.py`` via +``dmd.py`` or directly (in a recent version), you can do this by running with the +environment variable ``BREAKPAD_SYMBOLS_PATH`` set to the ``<SYMB_DIR>`` from above. +If that doesn't work, you'll have to edit ``initFixStacks`` in ``fix_stacks.py`` to +set ``breakpadSymsDir`` to ``<SYMB_DIR>``. + +With all of that done, you should now be able to run ``dmd.py`` or ``fix_stacks.py`` +to fix the stacks. Note that the stack fixing process can take a minute or two. diff --git a/docs/contributing/debugging/process_dump_task_manager.rst b/docs/contributing/debugging/process_dump_task_manager.rst new file mode 100644 index 0000000000..d345171856 --- /dev/null +++ b/docs/contributing/debugging/process_dump_task_manager.rst @@ -0,0 +1,69 @@ +How to get a process dump with Windows Task Manager +=================================================== + +Introduction +------------ + +When tracking down the causes of process hangs, it is often helpful to +obtain a process dump while the process is experiencing a hang. This +article describes how to get a process dump with Task Manager on +Windows. (To get a process dump for Thunderbird or some other product, +substitute the product name where ever you see Firefox in these +instructions.) + + +Caution +------- + +The memory dump that will be created through this process is a complete +snapshot of the state of Firefox when you create the file, so it +contains URLs of active tabs, history information, and possibly even +passwords depending on what you are doing when the snapshot is taken. It +is advisable to create a new, blank profile to use when reproducing the +hang and capturing the memory dump. Please ask for help doing this! + + +Requirements +------------ + +Windows + To get a process dump, you need to be using Windows Vista or above. +A Firefox nightly or release + You need a Firefox version for which symbols are available from the + :ref:`symbol server <Using The Mozilla Symbol Server>`. You + can use any `official nightly + build <https://ftp.mozilla.org/pub/firefox/nightly/>`__ or released + version of Firefox from Mozilla. You can find the latest trunk + nightly builds under + `http://ftp.mozilla.org/pub/mozilla.o.../latest-trunk/ <http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-trunk/>`__. + + +Creating the Dump File +---------------------- + +Ensure that Firefox is not already running. + + +Run Firefox, reproduce the hang +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Start Firefox and perform whatever steps are necessary to cause Firefox +to hang. Once the browser hangs, continue with the steps below. + + +After the hang +~~~~~~~~~~~~~~ + +#. Open Windows Task Manager (CTRL+SHIFT+ESC). +#. Find Firefox.exe among the list of processes. +#. Right-click Firefox.exe and select "Create dump file". Task manager + should indicate where the dump file was written to. + + +See also +-------- + +- :ref:`How to get a stacktrace for a bug report <How to get a stacktrace for a bug report>` +- `How to create a user-mode process dump file in Windows Vista and in + Windows 7 + (MSDN) <https://docs.microsoft.com/en-us/windows/client-management/generate-kernel-or-complete-crash-dump#manually-generate-a-memory-dump-file>`__ diff --git a/docs/contributing/debugging/stacktrace_report.rst b/docs/contributing/debugging/stacktrace_report.rst new file mode 100644 index 0000000000..2cae230745 --- /dev/null +++ b/docs/contributing/debugging/stacktrace_report.rst @@ -0,0 +1,153 @@ +How to get a stacktrace for a bug report +======================================== + +If you file a bug report in Bugzilla about a crash you should include a +stacktrace (call stack) in your report. A stacktrace will tell Mozilla +developers what crashed and provide a starting point for investigating +its cause. This article describes how to use the Mozilla Crash Reporter +(Breakpad) to get a crash ID, which our engineers can use to get a +stacktrace, and alternative ways to get a stacktrace if you can't get a +crash ID. + +Requirements +------------ + +You need a binary build of Firefox from +`Mozilla.org <https://www.mozilla.org/firefox/>`__. SeaMonkey and +Thunderbird also support crash reporting. + +Mozilla's crash report server currently only has debug information for +Mozilla builds and thus the crash reporter cannot work if you use a +build from a Linux distribution or if you compile from source code. In +these cases you will need to use one of the :ref:`alternative +methods <Alternative ways to get a stacktrace>` listed below. + +.. note:: + + **Note:** When filing a crash report, it is important to know whether + the crash occurs with `Firefox safe + mode <http://support.mozilla.com/kb/Safe+Mode>`__. This helps + engineers determine whether a particular + `extension <http://support.mozilla.com/kb/Troubleshooting+extensions+and+themes>`__ + or + `plugin <http://support.mozilla.com/kb/Troubleshooting+plugins>`__ + is the cause of the crash. + + +How to get a crash ID with the Mozilla Crash Reporter +----------------------------------------------------- + +1 - Crash and submit a report to the system. + +.. image:: img/crashreporter.png + +The Mozilla Crash Reporter window should automatically come up after Firefox crashes. +If you have any additional information about the crash, such as additional detail on +what you were doing at the time that may have triggered the crash, please enter it +into the comments box. Be sure that you **check the "Tell Mozilla about this crash"** +checkbox and click the restart button. The crash reporter should now submit the +crash report and Firefox should open again. + +.. note:: + + The "Details" button gives additional data about the incident, + however this is not useful in a bug report. + + +2 - Tell us the ID of the report you submitted. + +.. image:: img/crashlist.jpg + +To access all of your submitted reports type "about:crashes" into the Firefox address bar +and press enter. Firefox should open a list of IDs for your submitted crash reports. +Copy two or three of the IDs for the appropriate crashes and paste them into your +Bugzilla report. Please check the listed times to avoid copying the ID of an unrelated +crash report. + +.. note:: + + You can prefix a "bp-" to the beginning of an ID to make Bugzilla turn it + into a link: bp-a70759c6-1295-4160-aa30-bc4772090918 + + +How to get the crash ID if Firefox crashes on startup +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If Firefox crashes on startup you can still access your submitted crash +reports. Crash reports are accessible from all Firefox profiles, so if a +`new +profile <https://support.mozilla.org/kb/profile-manager-create-remove-switch-firefox-profiles>`__ +does not crash you can use it to access them through "about:crashes" as above. + + +Accessing crash report IDs outside of Firefox +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +If you cannot load Firefox at all you can find the crash report files at +this location depending on your operating system: + +* Windows : ``%APPDATA%\Mozilla\Firefox\Crash Reports\submitted\`` +* macOS : ``~/Library/Application Support/Firefox/Crash Reports/submitted/`` +* Linux : ``~/.mozilla/firefox/Crash Reports/submitted/`` + +Each file in this folder contains one submitted crash report ID. You can +check the modified or creation time for each file to discern which crash +reports are relevant to your bug report. + +.. _Alternative ways to get a stacktrace: + +Alternative ways to get a stacktrace +------------------------------------ + +If the Mozilla crash reporter doesn't come up or isn't available you +will need to obtain a stacktrace manually: + + +Windows +~~~~~~~ + +See the article :ref:`Create a stacktrace with Windbg <How to get a stacktrace with WinDbg>` for information +on how to do this. + +For a full process dump, see :ref:`How to get a process dump with Windows +Task Manager`. + + +macOS +~~~~~ + +Run /Applications/Utilities/Console.app. Expand "~/Library/Logs" and +"CrashReporter", then look for logs for "firefox-bin". + + +Linux +~~~~~ + +Note that for most distros, the package you need to get symbols for will +be something like "xulrunner", not "firefox". + + +Crash reports files on your computer +------------------------------------ + +When Breakpad initially catches a crash it first writes crash report +files (e.g. .dump and .extra files) into the 'pending' subdirectory of +its 'Crash Reports' directory. + +If Breakpad successfully sends the crash report to the reporting server +then, by default, the files added to the 'pending' subdirectory for the +crash are removed, and a .txt file is placed in the 'submitted' +directory containing the crash ID created by the reporting server. + +If you want Breakpad to leave the .dump and .extra files on your +computer so that you can examine them locally, then set the +MOZ_CRASHREPORTER_NO_DELETE_DUMP environment variable to 1. + +- Ubuntu: `Instructions from the Ubuntu + Team <https://wiki.ubuntu.com/MozillaTeam/Bugs#Obtain%20a%20backtrace%20from%20an%20apport%20crash%20report%20(using%20gdb)>`__ +- openSUSE: `General instructions from + openSUSE <https://en.opensuse.org/openSUSE:Bugreport_application_crashed>`__ +- Fedora: `Capturing Stack + Traces <https://fedoraproject.org/wiki/StackTraces>`__ +- Gentoo: `Debugging using + GDB <https://wiki.gentoo.org/wiki/Debugging_with_GDB>`__ diff --git a/docs/contributing/debugging/stacktrace_windbg.rst b/docs/contributing/debugging/stacktrace_windbg.rst new file mode 100644 index 0000000000..a0d2abfc25 --- /dev/null +++ b/docs/contributing/debugging/stacktrace_windbg.rst @@ -0,0 +1,232 @@ +How to get a stacktrace with WinDbg +=================================== + ++--------------------------------------------------------------------+ +| This page is an import from MDN and the contents might be outdated | ++--------------------------------------------------------------------+ + +Introduction +------------ + +Sometimes you need to get a stacktrace (call stack) for a crash or hang +but `Breakpad <http://kb.mozillazine.org/Breakpad>`__ fails because it's +a special crash or a hang. This article describes how to get a +stacktrace in those cases with WinDbg on Windows. (To get a stacktrace +for Thunderbird or some other product, substitute the product name where +ever you see Firefox in this instructions.) + +Requirements +------------ + +To get such a stacktrace you need to install the following software: + +Debugging Tools for Windows +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Microsoft distributes the Debugging Tools for Windows for free, those +include WinDbg which you will need here. Download it from `Install +Debugging Tools for +Windows <https://docs.microsoft.com/en-us/windows-hardware/drivers/download-the-wdk>`__. +(*You'll want the 32-bit version*, even if you are using a 64-bit +version of Windows) Then install it, the standard settings in the +installation process are fine. + +A Firefox nightly or release +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You need a Firefox version for which symbols are availables from the +:ref:`symbol server <Using The Mozilla Symbol Server>` to use +with WinDbg. You can use any `official nightly +build <https://ftp.mozilla.org/pub/firefox/nightly/>`__ or released +version of Firefox from Mozilla. You can find the latest trunk nightly +builds under +`http://ftp.mozilla.org/pub/mozilla.o.../latest-trunk/ <https://ftp.mozilla.org/pub/firefox/nightly/latest-mozilla-central/>`__. + + +Debugging +--------- + +To begin debugging, ensure that Firefox is not already running and open +WinDbg from the Start menu. (Start->All Programs->Debugging Tools for +Windows->WinDbg) Next, open the **"File"** menu and choose **"Open +Executable..."**. In the file chooser window that appears, open the +firefox.exe executable in your Firefox program folder (C:\Program +Files\Mozilla Firefox). + +You should now see a "Command" text window with debug output at the top +and an input box at the bottom. Before debugging can start, several +commands must be entered into the one-line input box at the bottom of +the Command window. + +.. note:: + + Tip: All commands must be entered exactly as written, one line at a + time, into the bottom of the Command box. + + - Copying and pasting each line is the easiest method to avoid + mistakes + - Some commands start with a period (.) or a pipe character (|), + which is required. (The keystroke for a pipe character on US + keyboards is SHIFT+\) + - Submit the log file on a bug or via the support site, even if + nothing seems to happen during the debug process + + +Start debugging +~~~~~~~~~~~~~~~ + +Now that Firefox is opened in the debugger, you need to configure your +WinDbg to download symbols from the Mozilla symbol server. To load the +symbols, enter the three commands below, pressing enter after each one. +(More details are available at :ref:`symbol server <Using The Mozilla Symbol Server>`.) + +:: + + .sympath SRV*c:\symbols*http://symbols.mozilla.org/firefox;SRV*c:\symbols*http://msdl.microsoft.com/download/symbols + .symfix+ c:\symbols + .reload /f + +Now wait for the symbols to download. This may take some time depending +on your connection speed; the total size of the Mozilla and Microsoft +symbols download is around 1.4GB. WinDbg will show "Busy" at the bottom +of the application window until the download is complete. + +Once the download is complete, you need to configure WinDbg to examine +child processes, ignore a specific event caused by Flash Player, and +record a log of loaded modules. You will also want to open a log file to +save data you collect. To do this, enter these four commands, pressing +enter after each one. + +:: + + .logopen /t c:\temp\firefox-debug.log + .childdbg 1 + .tlist + sxn gp + lm + +If you see firefox.exe listed in the output from .tlist more than once, +then you are already running the application and need to close the +running instance first before you start debugging, otherwise you won't +get useful results. + +Now run Firefox by opening the **Debug** menu and clicking **Go**. +**While Firefox is running, you will not be able to type any commands +into the debugger.** After it starts, try to reproduce the crash or +hanging issue that you are seeing. + +.. note:: + + If Firefox fails to start, and you see lines of text followed by a + command prompt in the debugger, a "breakpoint" may have been + triggered. If you are prompted for a command but don't see an error + about a crash, go back to the **Debug** menu and press **Go**. + +Once the browser crashes, you will see an error (such as "Access +violation") in the WinDbg Command window. If Firefox hangs and there is +no command prompt available in the debugger, open the **Debug** menu and +choose **Break.** Once the browser has crashed or been stopped, continue +with the steps below. + + +After the crash or hang +~~~~~~~~~~~~~~~~~~~~~~~ + +You need to capture the debug information to include in a bug comment or +support request. Enter these three commands, one at a time, to get the +stacktrace, crash/hang analysis and log of loaded modules. (Again, press +Enter after each command.) + +:: + + ~* kp + !analyze -v -f + lm + +After these steps are completed, find the file +**c:\temp\firefox-debug-(Today's Date).txt** on your hard drive. To +provide the information to the development community, submit this file +with a `support request <https://support.mozilla.com/>`__ or attach it +to a related bug on `Bugzilla <https://bugzilla.mozilla.org/>`__. + + +Producing a minidump +~~~~~~~~~~~~~~~~~~~~ + +Sometimes the stacktrace alone is not enough information for a developer +to figure out what went wrong. A developer may ask you for a "minidump" +or a "full memory dump", which are files containing more information +about the process. :ref:`You can easily produce minidumps from WinDBG and +provide them to developers <Capturing a minidump>`. + +FAQ + +Q: I am running Windows 7 (32-bit or 64-bit) and I see an exception in +the WinDbg command window that says 'ntdll32!LdrpDoDebuggerBreak+0x2c' +or 'ntdll32!LdrpDoDebuggerBreak+0x30'. What do I do now? + +A: If you see 'int 3' after either of those exceptions, you will need to +execute the following commands in WinDbg. + +:: + + bp ntdll!LdrpDoDebuggerBreak+0x30 + bp ntdll!LdrpDoDebuggerBreak+0x2c + eb ntdll!LdrpDoDebuggerBreak+0x30 0x90 + eb ntdll!LdrpDoDebuggerBreak+0x2c 0x90 + +| Make sure you enter them one at a time and press enter after each one. + If you use the 64-bit version of Windows, you need to replace "ntdll" + in these commands with "ntdll32". +| Q: The first four frames of my stack trace look like this: + +:: + + 0012fe20 7c90e89a ntdll!KiFastSystemCallRet + 0012fe24 7c81cd96 ntdll!ZwTerminateProcess+0xc + 0012ff20 7c81cdee kernel32!_ExitProcess+0x62 + + 0012ff34 6000179e kernel32!ExitProcess+0x14 + +This looks wrong to me?! + +A: You ran the application without the "Debug child processes also" +check box being checked. You need to detach the debugger and open the +application again, this time with the check box being checked. + +Q: WinDbg tells me that it is unable to verify checksum for firefox.exe. +Is this normal? + +A: Yes, this is normal and can be ignored. + +Q: Should I click yes or no when WinDbg asks me to "Save information for +workspace?" + +A: Click yes and WinDbg will save you from having to enter in the symbol +location for Firefox.exe in the future. Click no if you'd rather not +having WinDbg save this information. + +Q: I'm seeing "wow64" on top of each thread, is that ok ? + +A: No, you are running a 64 bit version of Windbg and trying to debug a +32 bit version of the mozilla software. Redownload and install the 32 +bit version of windbg. + + +Troubleshooting: Symbols will not download +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If symbols will not download no matter what you do, the problem may be +that Internet Explorer has been set to the **Work Offline** mode. You +will not receive any warnings of this in Windbg, Visual C++ or Visual +Studio. Even using the command line with symchk.exe to download symbols +will fail. This is because Microsoft uses Internet Explorer's internet & +proxy settings to download the symbol files. Check the File menu of +Internet Explorer to ensure "Work Offline" is unchecked. + + +See also +-------- + +- :ref:`symbol server <Using The Mozilla Symbol Server>` Maps addresses to human readable strings. +- :ref:`source server <Using The Mozilla Source Server>` Maps addresses to source code lines diff --git a/docs/contributing/debugging/understanding_crash_reports.rst b/docs/contributing/debugging/understanding_crash_reports.rst new file mode 100644 index 0000000000..9f8e5bbe03 --- /dev/null +++ b/docs/contributing/debugging/understanding_crash_reports.rst @@ -0,0 +1,325 @@ +Understanding Crash Reports +=========================== + ++--------------------------------------------------------------------+ +| This page is an import from MDN and the contents might be outdated | ++--------------------------------------------------------------------+ + +If a user experiences a crash they will be prompted to submit a raw +crash report, which is generated by Breakpad. The raw crash report is +received by `Socorro <https://github.com/mozilla/socorro>`__ which +`creates <https://github.com/mozilla/socorro/blob/master/socorro/processor/mozilla_processor_2015.py>`__ +a processed crash report. The processed crash report is based on the raw +crash report but also has a signature, classifications, and a number of +improved fields (e.g. OS, product, version). Many of the fields in both +the raw crash report and the processed crash report are viewable and +searchable on `crash-stats <https://crash-stats.mozilla.org/>`__. +Although there are two distinct crash reports, the raw and the +processed, people typically talk about a single "crash report" because +crash-stats mostly presents them in a combined way. + +Each crash report contains a wealth of data about the crash +circumstances. Despite this, many crash reports lack sufficient data for +a developer to understand why the crash occurred. As well as providing a +general overview, this page aims to highlight parts of a crash report +that may provide non-obvious insights. + +Note that most crash report fields are visible, but a few +privacy-sensitive parts of it are only available to users who are logged +in and have "minidump access". A relatively small number of users have +minidump access, and they are required to follow certain rules. For +access, see the `Protected Data Access docs on Crash Stats +<https://crash-stats.mozilla.org/documentation/protected_data_access/>`__. + +Each crash report has the following tabs: Details, Metadata, Modules, +Raw Dump, Extensions, and (optional) Correlations. + +Details tab +----------- + +The Details tab is the first place to look because it contains the most +important pieces of information. + +Primary fields +~~~~~~~~~~~~~~ + +| The first part of the Details tab shows a table containing the most + important crash report fields. It includes such things as when the + crash occurred, in which product and version, the crash kind, and + various details about the OS and configuration of the machine on which + the crash occurred. The following screenshot shows some of these + fields. +| |Example fields in the "Details" tab of a crash report| + +All fields have a tool-tip. For many fields, the tool-tip describes its +meaning. For all fields, the tool-tip indicates the key to use when you +want to do searches involving this field. (The field name is usually but +not always similar to the search key. E.g. the field "Adapter Device ID" +has the search key "adapter_device_id".) These descriptions are shown in +the `SuperSearchFields +API <https://crash-stats.mozilla.org/api/SuperSearchFields/>`__ and can be +`modified in super_search_fields.py <https://github.com/mozilla-services/socorro/blob/main/socorro/external/es/super_search_fields.py>`__ +or by writing up a `bug in Socorro <https://bugzilla.mozilla.org/enter_bug.cgi?format=__standard__&product=Socorro>`__. + +The fields present in this tab vary depending on the crash kind. Not all +fields are always present. + +The "Signature" field is the main identifier or label for a crash report. +Rather than considering each crash report in isolation, we want to put +crash reports into clusters so we can deal with groups of them at once. +An ideal clustering algorithm would put all crash reports with the same +root cause into a single cluster, and all crash reports with different +root causes into different clusters. The crash signature is our +imperfect but still useful attempt at such an algorithm. Most crash +signatures are based on the crashing stack trace, but some +special-purpose annotations are used to indicate particular kinds of +crashes. + +- ``Abort``: A controlled abort, e.g. via ``NS_RUNTIMEABORT``. + (Controlled aborts that occur via ``MOZ_CRASH`` or + ``MOZ_RELEASE_ASSERT`` currently don't get an ``Abort`` annotation, + but they do get a "MOZ_CRASH Reason" field.) +- ``OOM | <size>``, where ``<size>`` is one of ``large``, ``small``, + ``unknown``: an out-of-memory (OOM) abort. The ``<size>`` annotation + is determined by the "OOM Allocation Size" field; if that field is + missing ``<size>`` will be ``unknown``. +- ``hang``: a hang prior to shutdown. +- ``shutdownhang``: a hang during shutdown. +- ``IPCError-browser``: a problem involving IPC. If the parent Firefox + process detects that the child process has sent broken or + unprocessable IPDL data, or is not shutting down in a timely manner, + it kills the child process with a crash report. These crashes will + now have a signature that indicates why the process was killed, + rather than the child stack at the moment. + +When no special-purpose annotation is present and the signature begins +with a stack frame, it's usually a vanilla uncontrolled crash. The crash +cause can be determined from the "Crash Reason" field. Most commonly +it's a bad memory access. In that case, on Windows you can tell from the +reason field if the crash occurred while reading, writing or executing +memory (e.g. ``EXCEPTION_VIOLATION_ACCESS_READ`` indicates a bad memory +read). On Mac and Linux the reason will be SIGSEGV or SIGBUS and you +cannot tell from this field what kind of memory access it was. + +See `this +file <https://github.com/mozilla-services/socorro/blob/master/socorro/signature/README.rst>`__ +for a detailed explanation of the crash report signature generation +procedure, and for information on how modify this procedure. + +There are no fields that uniquely identify the user that a crash report +came from, but if you want to know if multiple crashes come from a +single user the "Install Time" field is a good choice. Use it in +conjunction with other fields that don't change, such as those +describing the OS or graphics card, for additional confidence. + +For bad memory accesses, the "Crash Address" field can give additional +indications what went wrong. + +- 0x0 is probably a null pointer deference[*]. +- Small addresses like 0x8 can indicate an object access (e.g. + ``this->mFoo``) via a null ``this`` pointer. +- Addresses like 0xfffffffffd8 might be stack accesses, depending on + the platform[*]. +- Addresses like 0x80cdefd3 might be heap accesses, depending on the + platform. +- Addresses may be poisoned: 0xe4 indicates the address comes from + memory that has been allocated by jemalloc but not yet initialized; + 0xe5 indicates the address comes from memory freed by jemalloc. The + JS engine also has multiple poison values defined in + ``js/src/jsutil.h``. + +[*] Note that due to the way addressing works on x86-64, if the crash +address is 0x0 for a Linux/macOS crash report, or 0xffffffffffffffff for +a Windows crash report, it's highly likely that the value is incorrect. +(There is a `bug +report <https://bugzilla.mozilla.org/show_bug.cgi?id=1493342>`__ open +for this problem.) You can sanity-check these crashes by looking at the +raw dump or minidump in the Raw Dump tab (see below). + +Note that for non-release builds the "Version" field represents multiple +different builds since nightly and beta version numbers are reused for +builds created over a series of days until the version number is bumped. +(The "Build ID" field can disambiguate.) It's not currently possible to +`restrict searches to a given version or +later <https://bugzilla.mozilla.org/show_bug.cgi?id=1401517>`__ (using +>= with a build ID and a given release channel may work around this). + +Some fields, such as "URL" and "Email Address", are privacy-sensitive +and are only visible to users with minidump access. + +The Windows-only "Total Virtual Memory" field indicates if the Firefox +build and OS are 32-bit or 64-bit. + +- A value of 2 GiB indicates 32-bit Firefox on 32-bit Windows. +- A value of 3 or 4 GiB indicates 32-bit Firefox on 64-bit Windows + (a.k.a. "WoW64"). Such a user could switch to 64-bit Firefox. +- A value much larger than 4 GiB (e.g. 128 TiB) indicates 64-bit + Firefox. (The "Build Architecture" field should be "amd64" in this + case.) + +Some crash reports might contain a memory report. This memory report will +have been made some time before the crash, at a time when available +memory was low. In this case, a number of key measurements from the +memory report are shown in the Details tab, each one having a field name +starting with "MR:", short for "memory report". The full memory report +can be obtained in the Raw Dump tab (see below). + +Bug-related information +~~~~~~~~~~~~~~~~~~~~~~~ + +The second part of the Details tab shows bug-related information, as the +following screenshot shows. + +|Information relating to bug reports in the "Details" tab of a crash +report| + +The "Report this bug in" links can be used to easily file bug reports. +Each one links to a Bugzilla bug report creation page that has various +fields pre-filled, such as the crash signature. + +The "Related Bugs" section shows related bug reports, as determined by +the crash signature. + +Stack traces +~~~~~~~~~~~~ + +The third part of the Details tab shows the stack trace and thread +number of the crashing thread, as the following screenshot shows. + +|Information relating to threads in the "Details" tab of a crash report| + +Each stack frame has a link to the source code, when possible. If a +crash is new, the regressing changeset can often be identified by +looking for recent changes in the blame annotations for one or more of +the top stack frames. Blame annotations are also good for identifying +who might know about the code in question. + +Sometimes the highlighted source code is puzzling, e.g. the identified +line may not touch memory even though the crash is memory-related. This +can be caused by compiler optimizations. It's often better to look at +the disassembly (e.g. in a minidump) to understand exactly what code is +being executed. + +Stack frame entries take on a variety of forms. + +- The simplest are functions names, such as ``NS_InitXPCOM2``. +- Name/address pairs such as ``nss3.dll@0x1eb720`` are within system + libraries. +- Names such as ``F1398665248_____________________________`` ('F' + followed by many numbers then many underscores) are in Flash. +- Addresses such as ``@0xe1a850ac`` may indicate an address that wasn't + part of any legitimate code. If an address such as this occurs in the + first stack frame, the crash may be + `exploitable <https://developer.mozilla.org/en-US/docs/Mozilla/Security/Exploitable_crashes>`__. + +Stack traces for other threads can be viewed by clicking on the small +"Show other threads" link. + +If the crash report is for a hang, the crashing thread will be the +"watchdog" thread, which exists purely to detect hangs; its top stack +frame will be something +like\ :literal:`mozilla::`anonymous namespace'::RunWatchdog`. In that +case you should look at the other threads' stack traces to determine the +problem; many of them will be waiting on some kind of response, as shown +by a top stack frame containing a function like +``NtWaitForSingleObject`` or ``ZwWaitForMultipleObjects``. + +Metadata tab +------------ + +The Metadata tab is similar to the first part of the Details tab, +containing a table with various fields. These are the fields from the +raw crash report, ordered alphabetically by field name, but with +privacy-sensitive fields shown only to users with minidump access. There +is some overlap with the fields shown in the Details tab. + +Modules tab +----------- + +The modules tab shows all the system libraries loaded at the time of the +crash, as the following screenshot shows. + +|Table of modules in the "Modules" tab of a crash report| + +On Windows these are mostly DLLs, on Mac they are mostly ``.dylib`` +files, and on Linux they are mostly ``.so`` files. + +This information is most useful for Windows crashes, because DLLs loaded +by antivirus software or malware often cause Firefox to crash. +Correlations between loaded modules and crash signatures can be seen in +the "Correlations" tab (see below). + +`This page <https://support.mozilla.org/en-US/kb/helping-crashes>`__ +says that files lacking version/debug identifier/debug filename are +likely to be malware. + +Raw Dump tab +------------ + +The first part of the Raw Dump tab shows the raw crash report, in JSON +format. Once again, privacy-sensitive fields are shown only to users +with minidump access. + +|JSON data in the "Raw Dump" tab of a crash report| + +For users with minidump access, the second part of the Raw Dump tab has +some links, as the following screenshot shows. + +|Links to downloadable files in the "Raw Dump" tab of a crash report| + +These links are to the following items. + +#. A minidump. Minidumps can be extremely useful in understanding a + crash report; see :ref:`this page <Debugging A Minidump>` for an + explanation how to use them. +#. The aforementioned JSON raw crash report. +#. The memory report contained within the crash report. +#. The unredacted crash report, which has additional information. + +Extensions tab +-------------- + +The Extensions tab shows which extensions are installed and enabled. + +|Table of extensions in the "Extensions" tab of a crash report| + +Usually it just shows an ID rather than the proper extension name. + +Note that several extensions ship by default with Firefox and so will be +present in almost all crash reports. (The exact set of default +extensions depends on the release channel.) The least obvious of these +has an Id of ``{972ce4c6-7e08-4474-a285-3208198ce6fd}``, which is the +default Firefox theme. Some (but not all) of the other extensions +shipped by default have the following Ids: ``webcompat@mozilla.org``, +``e10srollout@mozilla.org``, ``firefox@getpocket.com``, +``flyweb@mozilla.org``, ``loop@mozilla.org``. + +If an extension only has a hexadecimal identifier, a Google search of +that identifier is usually enough to identify the extension's name. + +This information is useful because some crashes are caused by +extensions. Correlations between extensions and crash signatures can be +seen in the "Correlations" tab (see below). + +Correlations tab +---------------- + +This tab is only shown when crash-stats identifies correlations between +a crash and modules or extensions that are present, which happens +occasionally. + +See also +-------- + +- `A talk about understanding crash + reports <https://air.mozilla.org/a-talk-about-understanding-crash-reports/>`__, + by David Baron, from March 2016. +- :ref:`A guide to searching crash reports` + +.. |Example fields in the "Details" tab of a crash report| image:: https://mdn.mozillademos.org/files/13579/Details1.png +.. |Information relating to bug reports in the "Details" tab of a crash report| image:: https://mdn.mozillademos.org/files/13581/Details2.png +.. |Information relating to threads in the "Details" tab of a crash report| image:: https://mdn.mozillademos.org/files/13583/Details3.png +.. |Table of modules in the "Modules" tab of a crash report| image:: https://mdn.mozillademos.org/files/13593/Modules1.png +.. |JSON data in the "Raw Dump" tab of a crash report| image:: https://mdn.mozillademos.org/files/13595/RawDump1.png +.. |Links to downloadable files in the "Raw Dump" tab of a crash report| image:: https://mdn.mozillademos.org/files/14047/raw-dump-links.png +.. |Table of extensions in the "Extensions" tab of a crash report| image:: https://mdn.mozillademos.org/files/13599/Extensions1.png diff --git a/docs/contributing/directory_structure.rst b/docs/contributing/directory_structure.rst new file mode 100644 index 0000000000..fd447278c2 --- /dev/null +++ b/docs/contributing/directory_structure.rst @@ -0,0 +1,565 @@ +Firefox Source Code Directory Structure +======================================= + +This article provides an overview of what the various directories contain. + +To simply take a look at the Firefox source code, you do not need to +download it. You can look at the source directly with your web browser +using Searchfox (start at https://searchfox.org/mozilla-central/source for +the complete firefox source code of branch HEAD). + +In order to modify the source, you have to acquire it either by +downloading a :ref:`snapshot <Mercurial Overview>` of the sources or +by checking out the current sources from +:ref:`the repository <Firefox Contributors' Quick Reference>`. + +This document describes the directory structure -- i.e., directories that +are used by at least some of the +Mozilla project's client products. There are other directories in the +other Mozilla repository, such as those for Web tools and those for the +Classic codebase. + +See the `more detailed overview of the pieces of Gecko <https://wiki.mozilla.org/Gecko:Overview>`__. + +.cargo +------ + +Configuration files for the `Cargo package +manager <https://crates.io/>`__. + +.vscode +------- + +Configuration files used by the `Visual Studio Code +IDE <https://code.visualstudio.com/>`__ when working in the +mozilla-central tree. + +accessible +---------- + +Files for accessibility (i.e., MSAA (Microsoft Active Accessibility), +ATK (Accessibility Toolkit, used by GTK) support files). See +`Accessibility <https://developer.mozilla.org/docs/Web/Accessibility>`__. + + +browser +------- + +Contains the front end code (in XUL, Javascript, XBL, and C++) for the +Firefox desktop browser. Many of these files started off as a copy of files in +`xpfe <#xpfe>`__. + +browser/extensions +------------------ + +Contains `PDF.js <https://mozilla.github.io/pdf.js/>`__ and +`WebCompat <https://github.com/mozilla/webcompat-addon>`__ built-in extensions. + +browser/themes +-------------- + +Contains images and CSS files to skin the browser for each OS (Linux, +Mac and Windows) + +build +----- + +Miscellaneous files used by the build process. See also `config <#config>`__. + +caps +---- + +Capability-based web page security management. It contains C++ interfaces +and code for determining the capabilities of content based on the +security settings or certificates (e.g., VeriSign). See `Component +Security <https://www.mozilla.org/projects/security/components/>`__ . + +chrome +------ + +:ref:`Chrome registry <Chrome Registration>` used with `toolkit <#toolkit>`__/. +These files were originally copies of files in `rdf/chrome/`. + +config +------ + +More files used by the build process, common includes for the makefiles, +etc. + + +devtools +-------- + +The Firefox Developer Tools server and client components. See :ref:`contributor <devtools-contributor-doc>` and :ref:`user <devtools-user-doc>` documentation. + + +docs +---- + +Contains the documentation configuration (`Sphinx <http://www.sphinx-doc.org/>`__ based), the index page +and the contribution pages. + + +docshell +-------- + +Implementation of the docshell, the main object managing things related +to a document window. Each frame has its own docshell. It contains +methods for loading URIs, managing URI content listeners, etc. It is the +outermost layer of the embedding API used to embed a Gecko browser into +an application. + +dom +--- + +- :ref:`IDL definitions <XPIDL>` of the interfaces defined by + the DOM specifications and Mozilla extensions to those interfaces + (implementations of these interfaces are primarily, but not + completely, in `content <#content>`__). +- The parts of the connection between JavaScript and the + implementations of DOM objects that are specific both to JavaScript + and to the DOM. +- Implementations of a few of the core "DOM Level 0" objects, such as + `window <https://developer.mozilla.org/docs/Web/API/Window>`__ , `window.navigator <https://developer.mozilla.org/docs/Web/API/Window/navigator>`__, `window.location <https://developer.mozilla.org/docs/Web/API/Window/location>`__, etc. + +editor +------ + +The editor directory contains XUL/Javascript for the embeddable editor +component, which is used for the HTML Editor("Composer"), for plain and +HTML mail composition, and for text fields and text areas throughout the +product. The editor is designed like a +"browser window with editing features": it adds some special classes for +editing text and managing transaction undo/redo, but reuses browser code +for nearly everything else. + +extensions +---------- + +Contains several extensions to mozilla, which can be enabled at +compile-time using the ``--enable-extensions`` configure argument. + +Note that some of these are now built specially and not using the +``--enable-extensions`` option. For example, disabling xmlextras is done +using ``--disable-xmlextras``. + + +extensions/auth +--------------- + +Implementation of the negotiate auth method for HTTP and other +protocols. Has code for SSPI, GSSAPI, etc. See `Integrated +Authentication <https://www.mozilla.org/projects/netlib/integrated-auth.html>`__. + + +extensions/pref +--------------- + +Preference-related extensions. + +extensions/spellcheck +--------------------- + +Spellchecker for mailnews and composer. + +extensions/universalchardet +--------------------------- + +Detects the character encoding of text. + +gfx +--- + +Contains interfaces that abstract the capabilities of platform specific +graphics toolkits, along with implementations on various platforms. +These interfaces provide methods for things like drawing images, text, +and basic shapes. It also contains basic data structures such as points +and rectangles used here and in other parts of Mozilla. + +gradle +------ + +Containing files related to a Java build system. + +hal +--- + +Contains platform specified functions (e.g. obtaining battery status, +sensor information, memory information, Android +alarms/vibrate/notifications/orientation, etc) + +image +----- + +Image rendering library. Contains decoders for the image formats Firefox +supports. + +intl +---- + +Internationalization and localization support. See +`L10n:NewProjects <https://wiki.mozilla.org/L10n:NewProjects>`__. + +intl/locale +----------- + +Code related to determination of locale information from the operating +environment. + +intl/lwbrk +---------- + +Code related to line breaking and word breaking. + +intl/strres +----------- + +Code related to string resources used for localization. + +intl/uconv +---------- + +Code that converts (both ways: encoders and decoders) between UTF-16 and +many other character encodings. + +intl/unicharutil +---------------- + +Code related to implementation of various algorithms for Unicode text, +such as case conversion. + +ipc +--- + +Container for implementations of IPC (Inter-Process Communication). + +js/src +------ + +The JavaScript engine, also known as +:ref:`SpiderMonkey <SpiderMonkey>`. +See also `JavaScript <https://developer.mozilla.org/docs/JavaScript>`__. + +js/xpconnect +------------ + +Support code for calling JavaScript code from C++ code and C++ code from +JavaScript code, using XPCOM interfaces. See +`XPConnect <https://developer.mozilla.org/docs/XPConnect>`__. + +layout +------ + +Code that implements a tree of rendering objects that describe the types +and locations of the objects that are displayed on the screen (such as +CSS boxes, tables, form controls, XUL boxes, etc.), and code that +manages operations over that rendering tree (such as creating and +destroying it, doing layout, painting, and event handling). See +`documentation <https://www.mozilla.org/newlayout/doc/>`__ and `other +information <https://www.mozilla.org/newlayout/>`__. + +layout/base +----------- + +Code that deals with the rendering tree. + +layout/forms +------------ + +Rendering tree objects for HTML form controls. + +layout/generic +-------------- + +The basic rendering object interface and the rendering tree objects for +basic CSS boxes. + +layout/mathml +------------- + +Rendering tree objects for `MathML <https://developer.mozilla.org/docs/Web/MathML>`__. + +layout/svg +---------- + +Rendering tree objects for `SVG <https://developer.mozilla.org/docs/Web/SVG>`__. + +layout/tables +------------- + +Rendering tree objects for CSS/HTML tables. + +layout/xul +---------- + +Additional rendering object interfaces for `XUL <https://developer.mozilla.org/docs/XUL>`__ and +the rendering tree objects for XUL boxes. + +media +----- + +Contains sources of used media libraries for example *libpng*. + +memory +------ + +Cross-platform wrappers for *memallocs* functions etc. + +mfbt +---- + +Implementations of classes like *WeakPtr*. Multi-platform *assertions* +etc. + +mobile +------ + +mobile/android +-------------- + +Firefox for Android and Geckoview + +modules +------- + +Compression/Archiving, math library, font (and font compression), +Preferences Library + +modules/libjar +-------------- + +Code to read zip files, used for reading the .jar files that contain the +files for the mozilla frontend. + +modules/libpref +--------------- + +Library for reading and writing preferences. + +modules/zlib +------------ + +Source code of zlib, used at least in the networking library for +compressed transfers. + +mozglue +------- + +Glue library containing various low-level functionality, including a +dynamic linker for Android, a DLL block list for Windows, etc. + +netwerk +------- + +:ref:`Networking library <Networking>`, also known as Necko. +Responsible for doing actual transfers from and to servers, as well as +for URI handling and related stuff. + +netwerk/cookie +-------------- + +Permissions backend for cookies, images, etc., as well as the user +interface to these permissions and other cookie features. + +nsprpub +------- + +Netscape Portable Runtime. Used as an abstraction layer to things like +threads, file I/O, and socket I/O. See :ref:`NSPR`. + +nsprpub/lib +----------- + +Mostly unused; might be used on Mac? + +other-licenses +-------------- + +Contains libraries that are not covered by the MPL but are used in some +Firefox code. + +parser +------ + +Group of structures and functions needed to parse files based on +XML/HTML. + +parser/expat +------------ + +Copy of the expat source code, which is the XML parser used by mozilla. + +parser/html +----------- + +The HTML parser (for everything except about:blank). + +parser/htmlparser +----------------- + +The legacy HTML parser that's still used for about:blank. Parts of it +are also used for managing the conversion of the network bytestream into +Unicode in the XML parsing case. + +parser/xml +---------- + +The code for integrating expat (from parser/expat) into Gecko. + +python +------ + +Cross module python code. + +python/mach +----------- + +The code for the :ref:`Mach` building tool. + +security +-------- + +Contains NSS and PSM, to support cryptographic functions in mozilla +(like S/MIME, SSL, etc). See :ref:`Network Security Services (NSS)` +and +`Personal Security Manager +(PSM) <https://www.mozilla.org/projects/security/pki/psm/>`__. + +services +-------- + +Firefox accounts and sync (history, preferences, tabs, bookmarks, +telemetry, startup time, which addons are installed, etc). See +`here <https://docs.services.mozilla.com/>`__. + +servo +----- + +`Servo <https://servo.org/>`__, the parallel browser engine project. + +startupcache +------------ + +XXX this needs a description. + +storage +------- + +`Storage <https://developer.mozilla.org/docs/Mozilla/Tech/XPCOM/Storage>`__: XPCOM wrapper for sqlite. Wants to +unify storage of all profile-related data. Supersedes mork. See also +`Unified Storage <https://wiki.mozilla.org/Mozilla2:Unified_Storage>`__. + +taskcluster +----------- + +Scripts and code to automatically build and test Mozilla trees for the +continuous integration and release process. + +testing +------- + +Common testing tools for mozilla codebase projects, test suite +definitions for automated test runs, tests that don't fit anywhere else, +and other fun stuff. + +third_party +----------- + +Vendored dependencies maintained outside of Mozilla. + +toolkit +------- + +The "new toolkit" used by Thunderbird, Firefox, etc. This contains +numerous front-end components shared between applications as well as +most of the XBL-implemented parts of the XUL language (most of which was +originally forked from versions in `xpfe/`). + +toolkit/mozapps/extensions/test/xpinstall +----------------------------------------- + +The installer, which contains code for installing Mozilla and for +installing XPIs/extensions. This directory also contains code needed to +build installer packages. See `XPInstall <https://developer.mozilla.org/docs/XPInstall>`__ and +the `XPInstall project +page <https://www.mozilla.org/projects/xpinstall/>`__. + +tools +----- + +Some tools which are optionally built during the mozilla build process. + +tools/lint +---------- + +The linter declarations and configurations. +See `linting documentation </code-quality/lint/>`_ + +uriloader +--------- + +uriloader/base +-------------- + +Content dispatch in Mozilla. Used to load uris and find an appropriate +content listener for the data. Also manages web progress notifications. +See `Document Loading: From Load Start to Finding a +Handler <https://www.mozilla.org/docs/docshell/uri-load-start.html>`__ +and `The Life Of An HTML HTTP +Request <https://www.mozilla.org/docs/url_load.html>`__. + + +uriloader/exthandler +-------------------- + +Used to handle content that Mozilla can't handle itself. Responsible for +showing the helper app dialog, and generally for finding information +about helper applications. + +uriloader/prefetch +------------------ + +Service to prefetch documents in order to have them cached for faster +loading. + +view +---- + +View manager. Contains cross-platform code used for painting, scrolling, +event handling, z-ordering, and opacity. Soon to become obsolete, +gradually. + +widget +------ + +A cross-platform API, with implementations on each platform, for dealing +with operating system/environment widgets, i.e., code related to +creation and handling of windows, popups, and other native widgets and +to converting the system's messages related to painting and events into +the messages used by other parts of Mozilla (e.g., `view/` and +`content/`, the latter of which converts many of the +messages to yet another API, the DOM event API). + +xpcom +----- + +`Cross-Platform Component Object Model </en-US/docs/XPCOM>`__. Also +contains data structures used by the rest of the mozilla code. See also +`XPCOM Project <https://www.mozilla.org/projects/xpcom/>`__. + +xpfe +---- + +XPFE (Cross Platform Front End) is the SeaMonkey frontend. It contains +the XUL files for the browser interface, common files used by the other +parts of the mozilla suite, and the XBL files for the parts of the XUL +language that are implemented in XBL. Much of this code has been copied +to `browser/` and `toolkit/` for use in +Firefox, Thunderbird, etc. + + +xpfe/components +--------------- + +Components used by the Mozilla frontend, as well as implementations of +interfaces that other parts of mozilla expect. diff --git a/docs/contributing/editor.rst b/docs/contributing/editor.rst new file mode 100644 index 0000000000..1ce083d4fc --- /dev/null +++ b/docs/contributing/editor.rst @@ -0,0 +1,26 @@ +Editor / IDE integration +======================== + +You can use any editor or IDE to contribute to Firefox, as long as it can edit +text files. However, there are some steps specific to mozilla-central that may +be useful for a better development experience. This page attempts to document +them. + +.. note:: + + Visual Studio Code is the recommended editor for Firefox development. + Not because it is better than the other editors but because we decided to + focus our energy on a single editor. + +.. note:: + + This page is a work in progress. Please enhance this page with instructions + for your favourite editor. + +.. toctree:: + :maxdepth: 1 + + editors/vscode + editors/emacs + editors/vim + editors/others diff --git a/docs/contributing/editors/emacs.rst b/docs/contributing/editors/emacs.rst new file mode 100644 index 0000000000..fbd41195a1 --- /dev/null +++ b/docs/contributing/editors/emacs.rst @@ -0,0 +1,121 @@ +Emacs +===== + +ESLint +------ + +See `the devtools documentation <https://wiki.mozilla.org/DevTools/CodingStandards#Running_ESLint_in_Emacs>`__ +that describes how to integrate ESLint into Emacs. + +C/C++ Development Packages +-------------------------- + +General Guidelines to Emacs C++ Programming +------------------------------------------- + +The following guides give an overview of the C++ editing capabilities of emacs. + +It is worth reading through these guides to see what features are available. +The rest of this section is dedicated to Mozilla/Gecko specific setup for +packages. + + + * `C/C++ Development Environment for Emacs <https://tuhdo.github.io/c-ide.html>`__ + * `Emacs as C++ IDE <https://syamajala.github.io/c-ide.html>`__ + +rtags (LLVM/Clang-based Code Indexing) +-------------------------------------- + +Instructions for the installation of rtags are available at the +`rtags github repo <https://github.com/Andersbakken/rtags>`__. + +rtags requires a :ref:`compilation database <CompileDB back-end-compileflags>`. + +In order for rtags to index correctly, included files need to be copied and +unified compilation files need to be created. Either run a full build of the +tree, or if you only want indexes to be generated for the moment, run the +following commands (assuming you're in the gecko repo root): + +.. code:: + + cd gecko_build_directory + make export + ./config.status + +To increase indexing speed, it's best to remove unified build files and test +files from database updates. This can be done by creating a :code:`~/.rdmrc` +file with the following contents, with :code:`[src_dir]` replaced with either +the repo or build directory for your checkout: + +.. code:: + + -X */[src_dir]/*Unified*;*/[src_dir]/*/test/*;*/[src_dir]/*/tests/* + +Once the rdm daemon is running, the compilation database can be added to rtags +like so: + +.. code:: + + rc -J [gecko_build_directory]/compile_commands.json + +Note that this process will take a while initially. However, once the database +is built, it will only require incremental updates. As long as the rdm daemon +is running in the background, the database will be updated based on changes to +files. + +irony (LLVM/Clang-based Code Completion) +---------------------------------------- + +Instructions on the installation of irony-mode are available at the +`irony-mode github repo <https://github.com/Sarcasm/irony-mode>`__. + +irony-mode requires a :ref:`compilation database <CompileDB back-end-compileflags>`. + +Note that irony-mode, by default, uses elisp to parse the +:code:`compile_commands.json` file. As gecko is a very large codebase, this +file can easily be multiple megabytes, which can make irony-mode take multiple +seconds to load on a gecko file. + +It is recommended to use `this fork of irony-mode <https://github.com/Hylen/irony-mode/tree/compilation-database-guessing-4-pull-request>`__, +which requires the boost System and Filesystem libraries. + +`Checking the bug to get this patch into the mainline of irony-mode <https://github.com/Sarcasm/irony-mode/issues/176>`__ +is recommended, to see if the fork can be used or if the mainline repo can be +used. Using the Boost version of the irony-mode server brings file load times +to under 1s. + +Projectile (Project Management) +------------------------------- + +Instructions on the installation of projectile are available at the +`projectile github repo <https://github.com/bbatsov/projectile>`__. + +Projectile comes preconfigured for many project types. Since, gecko uses its +own special build system (mach), a new project type needs to be added. This can +be done via adding the following elisp configuration command to your emacs +configuration file. + +.. code:: + + (projectile-register-project-type 'gecko + '("mach" "moz.build") + "python mach --log-no-times build" + "python mach mochitest" + "python mach run") + +Assuming projectile-global-mode is on, this will allow projectile to run the +correct commands whenever it is working in a gecko repo. + +gdb +^^^ + +Emacs comes with great integration with gdb, especially when using +`gdb-many-windows <https://www.gnu.org/software/emacs/manual/html_node/emacs/GDB-User-Interface-Layout.html>`__. + +However, when gdb is invoked via mach, some special arguments +need to be passed in order to make sure the correct display mode is used. To +use M-x gdb with mach on firefox, use the following command: + +.. code:: + + gecko_repo_directory/mach run --debug --debugparams=-i=mi diff --git a/docs/contributing/editors/others.rst b/docs/contributing/editors/others.rst new file mode 100644 index 0000000000..b753ff1042 --- /dev/null +++ b/docs/contributing/editors/others.rst @@ -0,0 +1,41 @@ +Eclipse +======= + +You can generate an Eclipse project by running: + +.. code:: + + ./mach ide eclipse + +See also the `Eclipse CDT <https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Eclipse/Eclipse_CDT>`__ docs on MDN. + +Visual Studio +============= + +You can run a Visual Studio project by running: + +.. code:: + + ./mach ide visualstudio + +.. _CompileDB back-end-compileflags: + +CompileDB back-end / compileflags +================================= + +You can generate a :code:`compile_commands.json` in your object directory by +running: + +.. code:: + + ./mach build-backend --backend=CompileDB + +This file, the compilation database, is understood by a variety of C++ editors / IDEs +to provide auto-completion capabilities. You can also get an individual compile command by +running: + +.. code:: + + ./mach compileflags path/to/file + +This is how the :ref:`VIM <VIM>` integration works, for example. diff --git a/docs/contributing/editors/vim.rst b/docs/contributing/editors/vim.rst new file mode 100644 index 0000000000..0cce21d4e9 --- /dev/null +++ b/docs/contributing/editors/vim.rst @@ -0,0 +1,75 @@ +Vim / Neovim +============ + +AutoCompletion +-------------- + +For C++, anything that can use an LSP like :code:`coc.nvim`, +:code:`nvim-lspconfig`, or what not, should work as long as you generate a +:ref:`compilation database <CompileDB back-end-compileflags>` and point to it. + +Additionally, `YouCompleteMe <https://github.com/ycm-core/YouCompleteMe/>`__ +works without the need of a C++ compilation database as long as you have run +:code:`./mach build` or :code:`./mach configure`. Configuration for this lives +in :searchfox:`.ycm_extra_conf <.ycm_extra_conf>` at the root of the repo. + +Rust auto-completion should work both with Rust's LSP :code:`rust-analyzer`. + +Make sure that the LSP is configured in a way that it detects the root of the +tree as a workspace, not the crate you happen to be editing. For example, the +default of :code:`nvim-lspconfig` is to search for the closest +:code:`Cargo.toml` file, which is not what you want. You'd want something like: + +.. code :: + + root_dir = lspconfig.util.root_pattern(".git", ".hg") + +You also need to set some options to get full diagnostics: + +.. code :: + + "rust-analyzer.server.extraEnv": { + "CARGO_TARGET_DIR": "/path/to/objdir" + }, + "rust-analyzer.check.overrideCommand": [ "/path/to/mach", "--log-no-times", "cargo", "check", "--all-crates", "--message-format-json" ], + "rust-analyzer.cargo.buildScripts.overrideCommand": [ "/path/to/mach", "--log-no-times", "cargo", "check", "--all-crates", "--message-format-json" ], + +The easiest way to make these work out of the box is using +`neoconf <https://github.com/folke/neoconf.nvim/>`__, which +automatically supports importing VSCode configuration files. +:code:`./mach ide vscode --no-interactive` will then generate the right +configuration for you. + +ESLint +------ + +The easiest way to integrate ESLint with VIM is using the `Syntastic plugin +<https://github.com/vim-syntastic/syntastic>`__. + +In order for VIM to detect jsm files as JS you might want something like this +in your :code:`.vimrc`: + +.. code:: + + autocmd BufRead,BufNewFile *.jsm set filetype=javascript + +:code:`mach eslint --setup` installs a specific ESLint version and some ESLint +plugins into the repositories' :code:`node_modules`. + +You need something like this in your :code:`.vimrc` to run the checker +automatically on save: + +.. code:: + + autocmd FileType javascript,html,xhtml let b:syntastic_checkers = ['javascript/eslint'] + +You need to have :code:`eslint` in your :code:`PATH`, which you can get with +:code:`npm install -g eslint`. You need at least version 6.0.0. + +You can also use something like `eslint_d +<https://github.com/mantoni/eslint_d.js#editor-integration>`__ which should +also do that automatically: + +.. code:: + + let g:syntastic_javascript_eslint_exec = 'eslint_d' diff --git a/docs/contributing/editors/vscode.rst b/docs/contributing/editors/vscode.rst new file mode 100644 index 0000000000..9830dcbe4f --- /dev/null +++ b/docs/contributing/editors/vscode.rst @@ -0,0 +1,179 @@ +Visual Studio Code +================== + +.. toctree:: + :maxdepth: 1 + :glob: + +General Knowledge +----------------- + +`VSCode <https://code.visualstudio.com/>`__ is a multi-platform open-source programming editor developed by Microsoft and volunteers. +It has support for many programming languages using extensions. +This is the recommended editor for Firefox development. + +For more general information on the VSCode project see the `repository <https://github.com/Microsoft/vscode/>`__. + +Recommended extensions +---------------------- + +VS Code provides number of extensions for JavaScript, Rust, etc. By default, +Firefox source tree comes with its own set of recommendations of Visual Studio +Code extensions. They will be offered when you first open the project. + +If you need to refer to them later, the extensions are listed in +`.vscode/extensions.json <https://searchfox.org/mozilla-central/source/.vscode/extensions.json>`__. + +For Rust development, the `rust-analyzer <https://marketplace.visualstudio.com/items?itemName=matklad.rust-analyzer>`__ extension is recommended. +`See the manual <https://rust-analyzer.github.io/manual.html>`__ for more information. + +Getting setup +------------- + +Close `VS Code` if it is already open, then build the configuration for `VS Code` +by simplying running from the terminal: + +.. code:: + + ./mach ide vscode + +This will automatically set some of the recommended preferences for the workspace, +and if you are set up for a full build, it will enable clangd and rust integrations. + +If successful, `VS Code` will open at the end. You do not need to run this command +every time to open `VS Code`, you may open it in the normal way. + +If you are running full builds, the command above will set up the `Clangd` +integration so that subsequent invocations of ``./mach build`` run and update the +integration. + +.. note:: + + If `VS Code` is already open with a previous configuration generated, please make sure to + restart `VS Code` otherwise the new configuration will not be used, and the `compile_commands.json` + needed by `clangd` server will not be refreshed. This is a known `bug <https://github.com/clangd/vscode-clangd/issues/42>`__ + in `clangd-vscode` extension + +Ignore Files in Mercurial Repositories +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +When using Mercurial in mozilla-central, VS Code will treat your build directories as ordinary directories by default, causing some undesirable behavior including long indexing times, Go to Definition will open files in the build directory instead of the source tree, and Search Files by Name will find duplicate files from the source tree and the build directory (note: when using Git, VS Code will not do this since it reads ``.gitignore``). You can follow these directions to have VS Code largely ignore your build directories: + +#. Go to Preferences -> Settings +#. Search "exclude" in the Settings +#. (optional) Select "Workspace" below the search bar to only change this setting for the mozilla-central repository +#. Under "Files: Exclude", click "Add Pattern", type ``obj-*`` (assuming your build directory names start with the default text, ``obj-``), and click "OK" +#. Repeat the step above for the "Files: Watcher Exclude" setting +#. Reload VS Code: the easiest way to do this is to quit and reopen it. + +Despite excluding the build directories above, Go to Definition will still correctly open files that only appear in the build directory such as generated source code. See `Bug 1790517 <https://bugzilla.mozilla.org/show_bug.cgi?id=1790517>`_ for our effort to automatically exclude the build directories. + +Recommended Preferences +~~~~~~~~~~~~~~~~~~~~~~~ + +.. note:: + + These are automatically set when running ``./mach ide vscode`` but may be + changed manually. These are set only for particular file types. + +* ``"editor.formatOnSave": true`` + * This will turn on automatically fixing formatting issues when you save a file. +* ``"editor.defaultFormatter": "esbenp.prettier-vscode"`` + * This sets the default formatter to prettier using the recommended prettier + extension. + +``*.jsm`` and ``*.sjs`` file extensions should also be associated with JavaScript: + +.. code:: + + "files.associations": { + "*.jsm": "javascript", + "*.sjs": "javascript", + }, + +C/C++ Features and Support +-------------------------- + +For C++ support we offer an out of the box configuration based on +`clangd <https://clangd.llvm.org>`__. + +Leveraging the `clang` toolchain compiler we now have support in the IDE for the following features: + +**1.** Syntax highlighting + +**2.** IntelliSense with comprehensive code completion and suggestion + +.. image:: ../img/auto_completion.gif + +**3.** Go-to definition and Go-to declaration + +.. image:: ../img/goto_definition.gif + +**4.** Find all references + +.. image:: ../img/find_references.gif + +**5.** Open type hierarchy + +.. image:: ../img/type_hierarchy.gif + +**6.** Rename symbol, all usages of the symbol will be renamed, including declaration, definition and references + +.. image:: ../img/rename_symbol.gif + +**7.** Code formatting, based on `clang-format` that respects our coding standard using the `.clang-format` and `.clang-format-ignore` files. Format can be performed on an entire file or on a code selection + +.. image:: ../img/format_selection.gif + +**8.** Inline parsing errors with limited auto-fix hints + +.. image:: ../img/diagnostic_error.gif + +**9.** Basic static-code analysis using `clang-tidy` and our list of enabled checkers. (This is still in progress not all checkers are supported by `clangd`) + +Clangd-specific Commands +------------------------ + +Clangd supports some commands that are specific to C/C++: + +.. code:: + + "clangd.switchheadersource" + +This command navigates from the currently open header file to its corresponding source file (if there is one), or vice versa. + +This command can be invoked from the command menu (activated via ``F1``), or using its keybinding of ``Alt+o`` (``Alt+cmd+o`` on Mac). The keybinding can also be customized in ``Keyboard Shortcuts``. + +Remote Development over SSH +--------------------------- + +VS Code provides an `extension <https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-ssh>`__ that lets you use any remote machine with a SSH server as your development environment. No matter if it's Linux based, macOS or Windows, as long as the target machine offers a SSH connection, it can be used for development. + +No source code needs to be on your local machine to use VS Code remotely since the extension runs commands and other extensions directly on the remote machine. + +In order to setup a connection please follow these steps: + +**1.** Open VS Code and select from the left side panel ``Remote Explorer`` + +.. image:: ../img/remote_explorer.png + +**2.** From the ``Remote Explorer`` panel select ``SSH Targets`` and click on ``Add`` and enter the connection details + +.. image:: ../img/remote_explorer_add.png + +.. image:: ../img/remote_explorer_add_wind.png + +**3.** Click on the connection that you just configured at the previous step + +**4.** Finally you should be connected to the desired remote SSH server + +.. image:: ../img/connection_done.png + +Please note that during the first connection VS Code will install itself remotely and also install all of the needed dependencies. + + + +Filing Bugs +----------- + +Bugs should be filed in the `Firefox Build System` product under `Developer Environment Integration`, preferably blocking `Bug 1662709 <https://bugzilla.mozilla.org/show_bug.cgi?id=1662709>`__. diff --git a/docs/contributing/engineering_show_and_tell.rst b/docs/contributing/engineering_show_and_tell.rst new file mode 100644 index 0000000000..87c2013bb8 --- /dev/null +++ b/docs/contributing/engineering_show_and_tell.rst @@ -0,0 +1,83 @@ +Engineering Show and Tell +========================= + +The engineering teams at Mozilla have put together a series of 5 minute long +tips and tricks to help boost productivity. You can find the recordings below. + +August 2021 +----------- + +`Link to the August 2021 recording <https://mozilla.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=bd0c503d-903f-4829-9c9d-ad7c011cee9b>`_ + +Links + +- Mossop: Mercurial Tips n Tricks + + - https://github.com/Mossop/dotfiles/blob/main/shared/hg/hgrc + - https://mikeconley.github.io/documents/How_mconley_uses_Mercurial_for_Mozilla_code + +- mak: Gmail filtering with Google Apps Script + + - https://script.google.com/home + - https://paste.mozilla.org/9zEDnAiA#L9 + - https://developers.google.com/apps-script/reference/ + +- mhowell: Windows Tools You Might Not Know About + + - https://docs.google.com/document/d/1DyeTPIEz3SIcw7ldvSGKUVp4XP34Ue-_pV8lYmRJX6I/ + +- bryce: Debugging Remote Resources Via Proxy + + - https://www.telerik.com/fiddler + - https://www.charlesproxy.com/ + - https://mitmproxy.org/ + +- jody: CircleCI for VS Code + + - https://marketplace.visualstudio.com/items?itemName=jodyh.circleci-vscode + +- sfink: Moz workflow aids for post-landing pruning, taskcluster job replication & more + + - https://github.com/hotsphink/sfink-tools + - https://hg.sr.ht/~sfink/sfink-tools + +- mconley: Multiple mozconfigs with mozconfigwrapper + + - https://github.com/ahal/mozconfigwrapper + +February 2021 +------------- + +`Link to the February 2021 recording <https://mozilla.zoom.us/rec/share/RvN62-Y3ByGmeyQChmsqXy6WXAE3iJETeB2yNj1xq4-z3c80ewwFaz-EAVnoHSo-.8FbmZF3nQEWBPFSr>`_ + +Slide decks: + +- `bash scripts and keyboard shortcuts <https://docs.google.com/presentation/d/1T8z99Hy0rI-_W3wJIZsG-edKRiicxgl-QeBoJuM90qQ/edit?usp=sharing>`_ +- `sccache-dist <https://docs.google.com/presentation/d/1_mN5rgV2LrzRKEOn06j4uaryYC9zRQJOdomgeBrJ8RA/edit#slide=id.g832b271044_1_1173>`_ +- `How to animate things smoothly <https://docs.google.com/presentation/d/11csNTR1GnVs2BdjN1alJcTbjXMhKyhpv9GIx2jNMJ6U/edit#slide=id.p>`_ +- `How I learned to love the M-C <https://docs.google.com/presentation/d/13O06nyDWqfbZyLeGRdLztd234zJ2SQcJu7LkBFDLnYg/edit?usp=sharing>`_ + +Links for "Editor, Lint, & More" talk: + +- :ref:`Linting` +- :ref:`lint-vcs-hook` +- https://eslint.org/docs/user-guide/integrations + +Links for "Stupid Mac Tricks" talk: + +- https://www.obdev.at/products/launchbar/index.html +- Moom: https://itunes.apple.com/app/id496437906?mt=12&ls=1 +- https://kapeli.com/dash +- https://git-fork.com/ +- https://www.usboverdrive.com/ +- https://manytricks.com/download/butler/ +- https://bjango.com/mac/istatmenus/ +- https://iterm2.com/ + +Other links: + +- https://twitter.com/asutherland/status/1274942012316319744 +- https://scootersoftware.com/ +- https://pasteapp.io/ +- https://github.com/acreskeyMoz/browsertime_scripts +- https://chuttenblog.wordpress.com/2021/01/18/doubling-the-speed-of-windows-firefox-builds-using-sccache-dist/ diff --git a/docs/contributing/how_to_submit_a_patch.rst b/docs/contributing/how_to_submit_a_patch.rst new file mode 100644 index 0000000000..0ceba7f39c --- /dev/null +++ b/docs/contributing/how_to_submit_a_patch.rst @@ -0,0 +1,245 @@ +How to submit a patch +===================== + ++--------------------------------------------------------------------+ +| This page is an import from MDN and the contents might be outdated | ++--------------------------------------------------------------------+ + +Submitting a patch, getting it reviewed, and committed to the Firefox +source tree involves several steps. This article explains how. + +.. note:: + + We are also providing a :ref:`Firefox Contributors Quick Reference <Firefox Contributors' Quick Reference>` for contributors. + +The process of submission is illustrated by the following diagram, and +each step is detailed below: + +.. mermaid:: + + graph TD; + Preparation --> c[Working on a patch]; + c[Working on a patch] --> Testing; + Testing --> c[Working on a patch]; + Testing --> e[Submit the patch]; + e[Submit the patch] --> d[Getting Reviews] + d[Getting Reviews] -- Addressing Review comment --> c[Working on a patch]; + d[Getting Reviews] --> h[Push the change]; + + + + +Preparation +----------- + +Every change to the code is tracked by a bug report +in `bugzilla.mozilla.org <https://bugzilla.mozilla.org/>`__. Without a +bug, code will not be reviewed, and without review, code will not be +accepted. To avoid duplication, `search for an existing +bug <https://bugzilla.mozilla.org/query.cgi?format=specific>`__ about +your change, and only if none exists, file a new one. Most communication +about code changes take place in the associated code +review, so be sure the bug describes the exact problem being solved. + +Please verify the bug is for the correct product and component. For more +information, ask questions on the newsgroups, or on the #developers room +on `chat.mozilla.org <https://chat.mozilla.org>`__. + +The person working on a bug should be the 'assignee' of that bug in +Bugzilla. If somebody else is currently the assignee of a bug, email +this person to coordinate changes. If the bug is unassigned, leave a +message in the bug's comments, stating that you intend working on it, +and suggest that someone with bug-editing privileges assign it to you. + +Some teams wait for new contributors to attach their first patch before +assigning a bug. This makes it available for other contributors, in case +the new contributor is unable to level up to patch creation. By +expressing interest in a bug comment, someone from that team should +guide you through their process. + + +Module ownership +---------------- + +All code is supervised by a `module +owner <https://www.mozilla.org/en-US/about/governance/policies/module-ownership/>`__. +This person will be responsible for reviewing and accepting the change. +Before writing your code, determine the module owner, verifying your +proposed change is considered acceptable. They may want to look over any +new user interface (UI review), functions (API review), or testcases for +the proposed change. + +If module ownership is not clear, ask on the newsgroups or `on +Matrix <https://chat.mozilla.org>`__. The revision log for the relevant +file might also be helpful. For example, see the change log for +``browser/base/content/browser.js``, by clicking the "Hg Log" +link at the top of `Searchfox <https://searchfox.org/mozilla-central/source/>`__, or +by running ``hg log browser/base/content/browser.js``. The corresponding +checkin message will contain something like "r=nickname", identifying +active code submissions, and potential code reviewers. + + +Working on a patch +------------------ + +Changes to the Firefox source code are presented in the form of a patch. +A patch is a commit to version control. Firefox and related code is +stored in our `Mercurial +server <https://hg.mozilla.org/mozilla-central>`__. We have extensive +documentation on using Mercurial in our guide, :ref:`Mercurial Overview`. + +Each patch should represent a single complete change, separating +distinct changes into multiple individual patches. If your change +results in a large, complex patch, seek if it can be broken into +`smaller, easy to understand patches representing complete +steps <https://secure.phabricator.com/book/phabflavor/article/writing_reviewable_code/#many-small-commits>`__, +applied on top of each other. This makes it easier to review your +changes, `leading to quicker +reviews, <https://groups.google.com/group/mozilla.dev.planning/msg/2f99460f57f776ef?hl=en>`__ +and improved confidence in this review outcome. + +Also ensure that your commit message is formatted appropriately. A +simple commit message should look like this: + +:: + + Bug 123456 - Change this thing to work better by doing something. r=reviewers + +The ``r=reviewers`` part is optional; if you are using Phabricator, +Lando will add it automatically based on who actually granted review, +and in any case the person who does the final check-in of the patch will +make sure it's added. + +The text of the message should be what you did to fix the bug, not a +description of what the bug was. If it is not obvious why this change is +appropriate, then `explain why in the commit +message <https://mozilla-version-control-tools.readthedocs.io/en/latest/mozreview/commits.html#write-detailed-commit-messages>`__. +If this does not fit on one line, then leave a blank line and add +further lines for more detail and/or reasoning. + +You can edit the message of the current commit at any time using +``hg commit --amend`` or ``hg histedit``. + +Also look at our :ref:`Reviewer Checklist` for a list +of best practices for patch content that reviewers will check for or +require. + + +Testing +------- + +All changes must be tested. In most cases, an `automated +test <https://developer.mozilla.org/docs/Mozilla/QA/Automated_testing>`__ is required for every +change to the code. + +While we desire to have automated tests for all code, we also have a +linter tool which runs static analysis on our JavaScript, for best +practices and common mistakes. See :ref:`ESLint` for more information. + +Ensure that your change has not caused regressions, by running the +automated test suite locally, or using the `Mozilla try +server <https://wiki.mozilla.org/Build:TryServer>`__. Module owners, or +developers `on Matrix <https://chat.mozilla.org>`__ may be willing to +submit jobs for those currently without try server privileges. + + +Submit the patch +---------------- + +.. note:: + + Make sure you rebase your patch on top of the latest build before you + submit to prevent any merge conflicts. + +Mozilla uses Phabricator for code review. See the `Mozilla Phabricator +User +Guide <https://moz-conduit.readthedocs.io/en/latest/phabricator-user.html>`__ +for instructions. + +Don't be shy in posting partial patches, demonstrating potential +approaches, and asking for preliminary feedback. It is easier for others +to comment, and offer suggestions, when a question is accompanied by +some code. + + +Getting reviews for my patch +---------------------------- + +See the dedicated page :ref:`Getting reviews` + + +Addressing review comments +-------------------------- + +It is unusual for patches to be perfect the first time around. The +reviewer may use the ‘Request Changes’ +`action <http://moz-conduit.readthedocs.io/en/latest/phabricator-user.html#reviewing-patches>`__ +and list problems that must be addressed before the patch can be +accepted. Please remember that requesting revisions is not meant to +discourage participation, but rather to encourage the best possible +resolution of a bug. Carefully work through the changes that the +reviewer recommends, attach a new patch, and request review again. + +Sometimes a reviewer will grant conditional review with the ‘Accept +Revision’ action but will also indicate minor necessary changes, such as +spelling, or indentation fixes. All recommended corrections should be +made, but a re-review is unnecessary. Make the changes and submit a new +patch. If there is any confusion about the revisions, another review +should be requested. + +Sometimes, after a patch is reviewed, but before it can be committed, +someone else makes a conflicting change. If the merge is simple, and +non-invasive, post an updated version of the patch. For all non-trivial +changes, another review is necessary. + +If at any point the review process stalls for more than two weeks, see +the previous 'Getting attention' section. + +In many open source projects, developers will accept patches in an +unfinished state, finish them, and apply the completed code. In +Mozilla's culture, **the reviewer will only review and comment on a +patch**. If a submitter declines to make the revisions, the patch will +sit idle, until someone chooses to take it on. + + +Pushing the change +------------------ + +A patch can be pushed (aka. 'landed') after it has been properly +reviewed. + +.. note:: + + Note: Be sure to build the application with the patch applied. This + ensures it runs as expected, passing automated tests, and/or runs + through the `try + server <https://wiki.mozilla.org/Build:TryServerAsBranch>`__. In the + bug, please also mention you have completed this step. + + Submitting untested patches wastes the committer's time, and may burn + the release tree. Please save everyone's time and effort by + completing all necessary verifications. + + +Ask the reviewer to land the patch for you. +For more details, see :ref:`push_a_change` + +If pushing the patch yourself, please follow :ref:`Committing rules and responsibilities`. +`Lando <https://moz-conduit.readthedocs.io/en/latest/lando-user.html>`__ is used +to automatically land your code. + + +Regressions +----------- + +It is possible your code causes functional or performance regressions. +There is a tight +`policy <https://www.mozilla.org/about/governance/policies/regressions/>`__ on +performance regressions, in particular. This means your code may be +dropped, leaving you to fix and resubmit it. Regressions, ultimately +mean the tests you ran before checking in are not comprehensive enough. +A resubmitted patch, or a patch to fix the regression, should be +accompanied by appropriate tests. + +After authoring a few patches, consider `getting commit access to +Mozilla source code <https://www.mozilla.org/about/governance/policies/commit/>`__. diff --git a/docs/contributing/img/auto_completion.gif b/docs/contributing/img/auto_completion.gif Binary files differnew file mode 100644 index 0000000000..4d545c29fd --- /dev/null +++ b/docs/contributing/img/auto_completion.gif diff --git a/docs/contributing/img/connection_done.png b/docs/contributing/img/connection_done.png Binary files differnew file mode 100644 index 0000000000..fc87571e4e --- /dev/null +++ b/docs/contributing/img/connection_done.png diff --git a/docs/contributing/img/diagnostic_error.gif b/docs/contributing/img/diagnostic_error.gif Binary files differnew file mode 100644 index 0000000000..25d50a7d7c --- /dev/null +++ b/docs/contributing/img/diagnostic_error.gif diff --git a/docs/contributing/img/example-stack.png b/docs/contributing/img/example-stack.png Binary files differnew file mode 100644 index 0000000000..c12265ae5f --- /dev/null +++ b/docs/contributing/img/example-stack.png diff --git a/docs/contributing/img/find_references.gif b/docs/contributing/img/find_references.gif Binary files differnew file mode 100644 index 0000000000..e986894566 --- /dev/null +++ b/docs/contributing/img/find_references.gif diff --git a/docs/contributing/img/format_selection.gif b/docs/contributing/img/format_selection.gif Binary files differnew file mode 100644 index 0000000000..28c88b3fa3 --- /dev/null +++ b/docs/contributing/img/format_selection.gif diff --git a/docs/contributing/img/goto_definition.gif b/docs/contributing/img/goto_definition.gif Binary files differnew file mode 100644 index 0000000000..9b3dcbf90f --- /dev/null +++ b/docs/contributing/img/goto_definition.gif diff --git a/docs/contributing/img/remote_explorer.png b/docs/contributing/img/remote_explorer.png Binary files differnew file mode 100644 index 0000000000..3f7a97bdc0 --- /dev/null +++ b/docs/contributing/img/remote_explorer.png diff --git a/docs/contributing/img/remote_explorer_add.png b/docs/contributing/img/remote_explorer_add.png Binary files differnew file mode 100644 index 0000000000..9faf9750fb --- /dev/null +++ b/docs/contributing/img/remote_explorer_add.png diff --git a/docs/contributing/img/remote_explorer_add_wind.png b/docs/contributing/img/remote_explorer_add_wind.png Binary files differnew file mode 100644 index 0000000000..6aa9f8d54d --- /dev/null +++ b/docs/contributing/img/remote_explorer_add_wind.png diff --git a/docs/contributing/img/rename_symbol.gif b/docs/contributing/img/rename_symbol.gif Binary files differnew file mode 100644 index 0000000000..d41fcd7ed6 --- /dev/null +++ b/docs/contributing/img/rename_symbol.gif diff --git a/docs/contributing/img/type_hierarchy.gif b/docs/contributing/img/type_hierarchy.gif Binary files differnew file mode 100644 index 0000000000..139bdf88cb --- /dev/null +++ b/docs/contributing/img/type_hierarchy.gif diff --git a/docs/contributing/index.rst b/docs/contributing/index.rst new file mode 100644 index 0000000000..6dca72aa02 --- /dev/null +++ b/docs/contributing/index.rst @@ -0,0 +1,41 @@ +Working on Firefox +================== + +Welcome to the Firefox codebase. This is the home of the Firefox +development process and source code documentation. + +.. toctree:: + :caption: Making Changes To Firefox + :maxdepth: 1 + + contribution_quickref + stack_quickref + pocket-guide-shipping-firefox + editor + reviews + +.. toctree:: + :caption: The Mercurial Version Control System + :maxdepth: 1 + :glob: + + vcs/* + + +.. toctree:: + :caption: Debugging + :maxdepth: 1 + :glob: + + debugging/* + + +.. toctree:: + :caption: Additional Information + :maxdepth: 1 + + directory_structure + build/artifact_builds + build/building_mobile_firefox + build/supported + engineering_show_and_tell diff --git a/docs/contributing/pocket-guide-shipping-firefox.rst b/docs/contributing/pocket-guide-shipping-firefox.rst new file mode 100644 index 0000000000..fdd37b9620 --- /dev/null +++ b/docs/contributing/pocket-guide-shipping-firefox.rst @@ -0,0 +1,523 @@ +Pocket Guide: Shipping Firefox +============================== + +*Estimated read time:* 15min + + +Introduction +------------ + +The purpose of this document is to provide a high level understanding of +how Mozilla ships Firefox. With the intention of helping new Mozillians +(and those who would like a refresher) understand the basics of our +release process, tools, common terms, and mechanisms employed in +shipping Firefox to our users. Often this document will introduce a +concept, explain how it fits into the process, and then provide a link +to learn more if interested. + +Repositories & Channels +----------------------- + +Shipping Firefox follows a software release :ref:`train model <train model>` +along 3 primary code :ref:`repositories <repositories>`; mozilla-central +(aka “m-c”), mozilla-beta, and mozilla-release. Each of these repositories are +updated within a defined cadence and built into one of our Firefox +products which are released through what is commonly referred to as +:ref:`Channels <channels>`: Firefox Nightly, Firefox Beta, and Firefox Release. + +`Firefox Nightly <https://whattrainisitnow.com/release/?version=nightly>`__ offers access to the latest cutting edge features +still under active development. Released every 12 hours with all the +changes that have :ref:`landed <landing>` on mozilla-central for Desktop and on +`main in firefox-android <https://github.com/mozilla-mobile/firefox-android/tree/main>`__ for Android. + +Every `4 weeks <https://whattrainisitnow.com/calendar/>`__, we +:ref:`merge <merge>` the code from mozilla-central to our +mozilla-beta branch. +For Android, we branch from main on firefox-android to a release branch. +New code or features can be added to mozilla-beta +outside of this 4 week cadence but will be required to land in +mozilla-central and then be :ref:`uplifted <uplift>` into +mozilla-beta. +Similarly for Android, uplifts are required to land in main on firefox-android before +backporting to the firefox-android release branch. + +`Firefox Beta <https://whattrainisitnow.com/release/?version=beta>`__ is for developers and early adopters who want to see +and test what’s coming next in Firefox. We release a new Desktop/Android Beta version +three times a week. + +.. note:: + + The first and second beta builds of a new cycle are shipped to a + subset of our Beta population. The full Beta population gets updated + starting with Beta 3 only.* + +Each Beta cycle lasts a total of 4 weeks where a final build is +validated by our QA and tagged for release into the mozilla-release +branch for Desktop. On Android we release from the same release branch +used during the Beta cycle. + +.. note:: + + **Firefox Developer Edition** *is a separate product based on + the mozilla-beta repo and is specifically tailored for Web Developers.* + +`Firefox Release <https://whattrainisitnow.com/release/?version=release>`__ is released every 4 weeks and is the end result +of our Beta cycle. This is our primary product shipping to hundreds of +millions of users. While a release is live, interim updates (dot releases) +are used to ship important bug fixes to users prior to the next major release. +These can happen on an as-needed basis when there is an important-enough +:ref:`driver <dot release drivers>` to do so (such as a critical bug severely +impairing the usability of the product for some users). In order to provide +better predictability, there is also a planned dot release scheduled for two +weeks after the initial go-live for less-critical fixes and other +:ref:`ride-along fixes <ride alongs>` deemed low-risk enough to include. + +.. note:: + `Firefox ESR (Extended Support Release) <https://whattrainisitnow.com/release/?version=esr>`__ *is a separate + product intended for Enterprise use. Major updates are rolled out once + per year to maintain stability and predictability. ESR also + contains a number of policy options not available in the standard + Firefox Release. Minor updates are shipped in sync with the Firefox + Release schedule for security and select quality fixes only.* + +Further Reading/Useful links: + +- `Firefox + Trains <https://whattrainisitnow.com/>`__ +- `Release + Calendar <https://whattrainisitnow.com/calendar/>`__ +- `Firefox Release + Process <https://wiki.mozilla.org/Release_Management/Release_Process>`__ +- `Firefox Delivery + dashboard <https://mozilla.github.io/delivery-dashboard/>`__ + +Landing Code and Shipping Features +---------------------------------- + +Mozillians (those employed by MoCo and the broader community) land lots +of code in the Mozilla repositories: fixes, enhancements, compatibility, +new features, etc. and is managed by :ref:`Mercurial <Mercurial Overview>` (aka +hg). All new code is tracked in :ref:`Bugzilla <bugzilla>`, reviewed +in :ref:`Phabricator <Phabricator>`, and then checked into the +mozilla-central repository using :ref:`Lando <Lando>`. + +.. note:: + + Some teams use :ref:`GitHub <github>` during development + but will still be required to use Phabricator (tracked in Bugzilla) to + check their code into the mozilla-central hg repository. + +The standard process for code to be delivered to our users is by ‘riding +the trains’, meaning that it’s landed in mozilla-central where it waits +for the next Beta cycle to begin. After merging to Beta the code will +stabilize over a 4 week period (along with everything else that merged +from mozilla-central). At the end of the beta cycle a release candidate +(:ref:`RC <rc>`) build will be generated, tested thoroughly, and +eventually become the next version of Firefox. + +Further Reading/Useful links: + +- `Phabricator and why we use it <https://wiki.mozilla.org/Phabricator>`__ +- `Firefox Release Notes Process <https://wiki.mozilla.org/Release_Management/Release_Notes>`__ +- `Firefox Release Notes Nomination <https://wiki.mozilla.org/Release_Management/Release_Notes_Nomination>`__ + +An exception to this process... +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Not all code can simply wait for the normal train model to be included +in a Firefox build. There are a variety of reasons for this; critical +fixes, security concerns, stabilizing a feature that’s already in Beta, +shipping high priority features faster, and so on. + +In these situations an uplift can be requested to take a recent landing +in mozilla-central and merge specific bits to another repository outside +the standard train model. After the request is made within Bugzilla, +:ref:`Release Management <release management>` will assess the potential risk +and will make a decision on whether it’s accepted. + +Further Reading/Useful links: + +- `Patch uplifting + rules <https://wiki.mozilla.org/Release_Management/Uplift_rules>`__ +- `Requesting an + uplift <https://wiki.mozilla.org/Release_Management/Requesting_an_Uplift>`__ + +Ensuring build stability +~~~~~~~~~~~~~~~~~~~~~~~~ + +Throughout the process of landing code in mozilla-central to riding the +trains to Firefox Release, there are many milestones and quality +checkpoints from a variety of teams. This process is designed to ensure +a quality and compelling product will be consistently delivered to our +users with each new version. See below for a distilled list of those +milestones. + +=========================================== ================ ================= =============================================================================== +Milestone Week Day of Week +------------------------------------------- ---------------- ----------------- ------------------------------------------------------------------------------- +Merge Day Nightly W1 Monday Day 1 of the new Nightly Cycle +PI Request deadline Nightly W1 Friday Manual QA request deadline for high risk features +Feature technical documentation due Nightly W2 Friday Deadline for features requiring manual QA +Beta release notes draft Nightly W4 Wednesday +Nightly features Go/No-Go decisions Nightly W4 Wednesday +Feature Complete Milestone Nightly W4 Wednesday Last day to land risky patches and/or enable new features +Nightly soft code freeze start Nightly W4 Thursday Stabilization period in preparation to merge to Beta +String freeze Nightly W4 Thursday Modification or deletion of strings exposed to the end-users is not allowed +QA pre-merge regression testing completed Nightly W4 Friday +Merge Day Beta W1 Monday Day 1 of the new Beta cycle +Pre-release sign off Beta W3 Friday Final round of QA testing prior to Release +Firefox RC week Beta W4 Monday Validating Release Candidate builds in preparation for the next Firefox Release +Release Notes ready Beta W4 Tuesday +What’s new page ready Beta W4 Wednesday +Firefox go-live @ 6am PT Release W1 Tuesday Day 1 of the new Firefox Release to 25% of Release users +Firefox Release bump to 100% Release W1 Thursday Increase deployment of new Firefox Release to 100% of Release users +Scheduled dot release approval requests due Release W2 Friday All requests required by EOD +Scheduled dot release go-live Release W3 Tuesday By default, ships when ready. Specific time available upon request. +=========================================== ================ ================= =============================================================================== + + +The Release Management team (aka “Relman”) monitors and enforces this +process to protect the stability of Firefox. Each member of Relman +rotates through end-to-end ownership of a given :ref:`release +cycle <release cycle>`. The Relman owner of a cycle will focus on the +overall release, blocker bugs, risks, backout rates, stability/crash +reports, etc. Go here for a complete overview of the `Relman Release +Process +Checklist <https://wiki.mozilla.org/Release_Management/Release_Process_Checklist_Documentation>`__. + +.. note:: + + While Relman will continually monitor the overall health of each + Release it is the responsibility of the engineering organization to + ensure the code they are landing is of high quality and the potential + risks are understood. Every Release has an assigned :ref:`Regression + Engineering Owner <reo>` (REO) to ensure a decision is made + about each regression reported in the release.* + +Further Reading/Useful links: + +- `Release Tracking + Rules <https://wiki.mozilla.org/Release_Management/Tracking_rules>`__ +- `Release + Owners <https://wiki.mozilla.org/Release_Management/Release_owners>`__ +- `Regression Engineering + Owners <https://wiki.mozilla.org/Platform#Regression_Engineering_Owner_.28REO.29>`__ +- `Commonly used Bugzilla queries for all + Channels <https://trainqueries.herokuapp.com/>`__ + +Enabling/Disabling code (Prefs) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Within Firefox we allow the ability to Enable/Disable bits of code or +entire features using `Preferences <preferences>`. There are many +reasons why this is useful. Here are some examples: + +- Continual development over multiple release cycles without exposing + partially completed features to our users +- Provide the ability to quickly disable a feature if there is a + problem found during the release process +- Control features which are experimental or not ready to be shown to a + specific channel population (e.g. enabled for Beta but disabled for + Release) +- A/B testing via :ref:`telemetry <telemetry>` experiments + +.. note:: + + :ref:`Normandy <normandy>` Pref Rollout is a feature that + allows Mozilla to change the state of a preference for a targeted set of + users, without deploying an update to Firefox. This is especially useful + when conducting experiments or a gradual rollout of high risk features + to our Release population. + +Further Reading/Useful links: + +- `Brief guide to Mozilla + preferences <https://developer.mozilla.org/en-US/docs/Mozilla/Preferences/A_brief_guide_to_Mozilla_preferences>`__ +- `Normandy Pref + rollout <https://wiki.mozilla.org/Firefox/Normandy/PreferenceRollout>`__ + +Release & Feature QA +~~~~~~~~~~~~~~~~~~~~ + +Release QA is performed regularly and throughout the Release Cycle. +Organized in two-week sprints its primary goals are: + +- Qualifying builds for release +- Feature testing +- Product Integrity requests +- Bug work +- Community engagement + +Features that can have significant impact and/or pose risk to the code +base should be nominated for QA support by the :ref:`feature +owner <feature owner>` in its intended release. This process is kicked +off by filing a :ref:`Product Integrity <product integrity>` team request +:ref:`PI request <pi request>`. These are due by the end of week 2 +of the Nightly cycle. + +.. note:: + + Manual QA testing is only required for features as they go + through the Beta cycle. Nightly Feature testing is always optional. + +Further Reading/Useful links: + +- `QA Feature + Testing <https://wiki.mozilla.org/QA/Feature_Testing_v2>`__ +- `Release QA + overview <https://docs.google.com/document/d/1ic_3TO9-kNmZr11h1ZpyQbSlgiXzVewr3kSAP5ML4mQ/edit#heading=h.pvvuwlkkvtc4>`__ +- `PI Request template and + overview <https://mana.mozilla.org/wiki/pages/viewpage.action?spaceKey=PI&title=PI+Request>`__ + +Experiments +~~~~~~~~~~~ + +As we deliver new features to our users we continually ask ourselves +about the potential impacts, both positive and negative. In many new +features we will run an experiment to gather data around these impacts. +A simple definition of an experiment is a way to measure how a change to +our product affects how people use it. + +An experiment has three parts: + +1. A new feature that can be selectively enabled +2. A group of users to test the new feature +3. Telemetry to measure how people interact with the new feature + +Experiments are managed by an in-house tool called +`Experimenter <https://experimenter.services.mozilla.com/>`__. + +Further Reading/Useful links: + +- `More about experiments and + Experimenter <https://github.com/mozilla/experimenter>`__ +- `Requesting a new + Experiment <https://experimenter.services.mozilla.com/experiments/new/>`__ + (Follow the ‘help’ links to learn more) +- `Telemetry <https://wiki.mozilla.org/Telemetry>`__ + +Definitions +----------- + +.. _approval flag: + +**Approval Flag** - A flag that represents a security approval or uplift +request on a patch. + +.. _bugzilla: + +**Bugzilla** - Web-based general purpose bug tracking system and testing +tool. + +.. _channel: + +**Channel** - Development channels producing concurrent releases of +Firefox for Windows, Mac, Linux, and Android. + +.. _chemspill: + +**Chemspill** - Short for Chemical Spill. A chemspill is a rapid +security-driven or critical stsbility dot release of our product. + +.. _channel meeting: + +**Channel Meeting** - A twice weekly time to check in on the status +of the active releases with the release team. + +.. _dot release drivers: + +**Dot Release Drivers** - Issues/Fixes that are significant enough to +warrant a minor dot release to the Firefox Release Channel. Usually to +fix a stability (top-crash) or Security (Chemspill) issue. + +.. _early beta: + +**Early Beta** - Beta releases with the features gated by EARLY_BETA_OR_EARLIER +enabled. The first 2 weeks of Beta releases during the cycle are early beta releases. + +.. _feature owner: + +**Feature Owner** - The person who is ultimately responsible for +developing a high quality feature. This is typically an Engineering +Manager or Product Manager. + +.. _fenix: + +**Fenix** - Also known as Firefox Preview is an all-new browser for +Android based on GeckoView and Android Components + +.. _github: + +**Github** - Web-based version control and collaboration platform for +software developers + +.. _gtb: + +**GTB** - Acronym for Go to build. Mostly used in the release schedule +communication ("Go to build on March 18"), this means that we initiate the +building of a specific release. + +.. _landing: + +**Landing** - A general term used for when code is merged into a +particular source code repository + +.. _lando: + +**Lando** - Automated code lander for Mozilla. It is integrated with +our `Phabricator instance <https://phabricator.services.mozilla.com>`__ +and can be used to land revisions to various repositories. + +.. _mercurial: + +**Mercurial** - A source-code management tool (just like git) +which allows users to keep track of changes to the source code +locally and share their changes with others. It is also called hg. + +.. _merge: + +**Merge** - General term used to describe the process of integrating and +reconciling file changes within the mozilla repositories + +.. _nightly soft code freeze: + +**Nightly Soft Code Freeze** - Last week of the nightly cycle on mozilla-central +just before the merge to beta during which landing risky or experimental code +in the repository is discouraged. + +.. _normandy: + +**Normandy** - Normandy is a collection of servers, workflows, and +Firefox components that enables Mozilla to remotely control Firefox +clients in the wild based on precise criteria + +.. _nucleus: + +**Nucleus** - Name of the internal application used by release managers +to prepare and publish release notes. The data in this application is +fetched by mozilla.org. + +.. _orange_factor: + +**Orange** - Also called flaky or intermittent tests. Describes a state +when a test or a testsuite can intermittently fail. + +.. _phabricator: + +**Phabricator** - Mozilla’s instance of the web-based software +development collaboration tool suite. Read more about `Phabricator as a +product <https://phacility.com/phabricator/>`__. + +.. _pi request: + +**PI Request** - Short for Product Integrity Request is a form +submission request that’s used to engage the PI team for a variety of +services. Most commonly used to request Feature QA it can also be used +for Security, Fuzzing, Performance, and many other services. + +.. _preferences: + +**Preferences** - A preference is any value or defined behavior that can +be set (e.g. enabled or disabled). Preference changes via user interface +usually take effect immediately. The values are saved to the user’s +Firefox profile on disk (in prefs.js). + +.. _rc: + +**Release Candidate** - Beta version with potential to be a final +product, which is ready to release unless significant bugs emerge. + +.. _rc week: + +**RC Week** - The week prior to release go-live is known as RC week. +During this week an RC is produced and tested. + +.. _release cycle: + +**Release Cycle** - The sum of stages of development and maturity for +the Firefox Release Product. + +.. _reo: + +**Regression Engineering Owner** - A partner for release management +assigned to each release. They both keep a mental state of how we are +doing and ensure a decision is made about each regression reported in +the release. AKA *REO*. + +.. _release engineering: + +**Release engineering** - Team primarily responsible for maintaining +the build pipeline, the signature mechanisms, the update servers, etc. aka *releng* + +.. _release management: + +**Release Management** - Team primarily responsible for the process of +managing, planning, scheduling and controlling a software build through +different stages and environments. aka *relman*. + +.. _relnotes: + +**Relnotes** - Short for release notes. Firefox Nightly, Beta, and Release each ship +with release notes. + +.. _Repository: + +**Repository** - a collection of stored data from existing databases +merged into one so that it may be shared, analyzed or updated throughout +an organization. + +.. _ride alongs: + +**Ride Alongs** - Bug fixes that are impacting release users but not +considered severe enough to ship without an identified dot release +driver. + +.. _rollout: + +**Rollout** - Shipping a release to a percentage of the release population. + +.. _status flags: + +**Status Flags** - A flag that represents the status of the bug with +respect to a Firefox release. + +.. _string freeze: + +**String Freeze** - Period during which the introduction, modification, or +deletion of strings exposed to the end-users is not allowed so as to allow our +localizers to translate our product. + +.. _taskcluster: + +**taskcluster** - Our execution framework to build, run tests on multiple +operating system, hardware and cloud providers. + +.. _telemetry: + +**Telemetry** - Firefox measures and collects non-personal information, +such as performance, hardware, usage and customizations. This +information is used by Mozilla to improve Firefox. + +.. _train model: + +**Train model** - a form of software release schedule in which a number +of distinct series of versioned software releases are released as a +number of different "trains" on a regular schedule. + +.. _tracking flags: + +**Tracking Flags** - A Bugzilla flag that shows whether a bug is being investigated +for possible resolution in a Firefox release. Bugs marked tracking-Firefox XX are +bugs that must be resolved one way or another before a particular release ship. + +.. _throttle unthrottle: + +**Throttle/Unthrottle a rollout** - Throttle is restricting a release rollout to 0% +of the release population, users can still choose to update but are not updated +automatically. Unthrottle is removing the release rollout restriction. + +.. _uplift: + +**Uplift** - the action of taking parts from a newer version of a +software system (mozilla-central or mozilla-beta) and porting them to an +older version of the same software (mozilla-beta, mozilla-release or ESR) diff --git a/docs/contributing/reviewer_checklist.rst b/docs/contributing/reviewer_checklist.rst new file mode 100644 index 0000000000..cfe772dba9 --- /dev/null +++ b/docs/contributing/reviewer_checklist.rst @@ -0,0 +1,181 @@ +Reviewer Checklist +================== + + Submitting patches to Mozilla source code needn't be complex. This + article provides a list of best practices for your patch content that + reviewers will check for or require. Following these best practices + will lead to a smoother, more rapid process of review and acceptance. + + +Good web citizenship +-------------------- + +- Make sure new web-exposed APIs actually make sense and are either + standards track or preffed off by default. +- In C++, wrapper-cache as needed. If your object can be gotten from + somewhere without creating it in the process, it needs to be + wrapper-cached. + + +Correctness +----------- + +- The bug being fixed is a valid bug and should be fixed. +- The patch fixes the issue. +- The patch is not unnecessarily complicated. +- The patch does not add duplicates of existing code ('almost + duplicates' could mean a refactor is needed). Commonly this results + in "part 0" of a bug, which is "tidy things up to make the fix easier + to write and review". +- If QA needs to verify the fix, you should provide steps to reproduce + (STR). + + +Quality +------- + +- If you can unit-test it, you should unit-test it. +- If it's JS, try to design and build so that xpcshell can exercise + most functionality. It's quicker. +- Make sure the patch doesn't create any unused code (e.g., remove + strings when removing a feature) +- All caught exceptions should be logged at the appropriate level, + bearing in mind personally identifiable information, but also + considering the expense of computing and recording log output. + [Fennec: Checking for log levels is expensive unless you're using + Logger.] + + +Style +----- + +- Follow the `style + guide <https://firefox-source-docs.mozilla.org/code-quality/coding-style/index.html>`__ + for the language and module in question. +- Follow local style for the surrounding code, even if that local style + isn't formally documented. +- New files have license declarations and modelines. +- New JS files should use strict mode. +- Trailing whitespace (git diff and splinter view both highlight this, + as does hg with the color extension enabled). Whitespace can be fixed + easily in Mercurial using the `CheckFiles + extension <https://www.mercurial-scm.org/wiki/CheckFilesExtension>`__. + In git, you can use git rebase --whitespace=fix. + + +Security issues +--------------- + +- There should be no writing to arbitrary files outside the profile + folder. +- Be careful when reading user input, network input, or files on disk. + Assume that inputs will be too big, too short, empty, malformed, or + malicious. +- Tag for sec review if unsure. +- If you're writing code that uses JSAPI, chances are you got it wrong. + Try hard to avoid doing that. + + +Privacy issues +-------------- + +- There should be no logging of URLs or content from which URLs may be + inferred. +- [Fennec: Android Services has Logger.pii() for this purpose (e.g., + logging profile dir)]. +- Tag for privacy review if needed. + + +Resource leaks +-------------- + +- In Java, memory leaks are largely due to singletons holding on to + caches and collections, or observers sticking around, or runnables + sitting in a queue. +- In C++, cycle-collect as needed. If JavaScript can see your object, + it probably needs to be cycle-collected. +- [Fennec: If your custom view does animations, it's better to clean up + runnables in onDetachFromWindow().] +- Ensure all file handles and other closeable resources are closed + appropriately. +- [Fennec: When writing tests that use PaintedSurface, ensure the + PaintedSurface is closed when you're done with it.] + + +Performance impact +------------------ + +- Check for main-thread IO [Fennec: Android may warn about this with + strictmode]. +- Remove debug logging that is not needed in production. + + +Threading issues +---------------- + +- Enormous: correct use of locking and volatility; livelock and + deadlock; ownership. +- [Fennec: All view methods should be touched only on UI thread.] +- [Fennec: Activity lifecycle awareness (works with "never keep + activities"). Also test with oom-fennec + (`https://hg.mozilla.org/users/blassey_mozilla.com/oom-fennec/) <https://hg.mozilla.org/users/blassey_mozilla.com/oom-fennec/%29>`__]. + + +Compatibility +------------- + +- Version files, databases, messages +- Tag messages with ids to disambiguate callers. +- IDL UUIDs are updated when the interface is updated. +- Android permissions should be 'grouped' into a common release to + avoid breaking auto-updates. +- Android APIs added since Froyo should be guarded by a version check. + + +Preffability +------------ + +- If the feature being worked on is covered by prefs, make sure they + are hooked up. +- If working on a new feature, consider adding prefs to control the + behavior. +- Consider adding prefs to disable the feature entirely in case bugs + are found later in the release cycle. +- [Fennec: "Prefs" can be Gecko prefs, SharedPreferences values, or + build-time flags. Which one you choose depends on how the feature is + implemented: a pure Java service can't easily check Gecko prefs, for + example.] + + +Strings +------- + +- There should be no string changes in patches that will be uplifted + (including string removals). +- Rev entity names for string changes. +- When making UI changes, be aware of the fact that strings will be + different lengths in different locales. + + +Documentation +------------- + +- The commit message should describe what the patch is changing (not be + a copy of the bug summary). The first line should be a short + description (since only the first line is shown in the log), and + additional description, if needed, should be present, properly + wrapped, in later lines. +- Adequately document any potentially confusing pieces of code. +- Flag a bug with dev-doc-needed if any addon or web APIs are affected. +- Use Javadocs extensively, especially on any new non-private methods. +- When moving files, ensure blame/annotate is preserved. + + +Accessibility +------------- + +- For HTML pages, images should have the alt attribute set when + appropriate. Similarly, a button that is not a native HTML button + should have role="button" and the aria-label attribute set. +- [Fennec: Make sure contentDescription is set for parts of the UI that + should be accessible] diff --git a/docs/contributing/reviews.rst b/docs/contributing/reviews.rst new file mode 100644 index 0000000000..0cb0dfb59c --- /dev/null +++ b/docs/contributing/reviews.rst @@ -0,0 +1,145 @@ +Getting reviews +=============== + + +Thorough code reviews are one of Mozilla's ways of ensuring code quality. +Every patch must be reviewed by the module owner of the code, or one of their designated peers. + +To request a review, you will need to specify a review group (starts with #). If there is not, you should select one or more usernames either when you submit the patch, or afterward in the UI. +If you have a mentor, the mentor can usually either also review or find a suitable reviewer on your behalf. + +For example, the syntax to request review from a group should be: + +.. code-block:: + + Bug xxxx - explain what you are doing and why r?#group-name + + or + + Bug xxxx - explain what you are doing and why r?developer-nickname + +Getting attention: If a reviewer doesn't respond within a week, or so of the review request: + + * Contact the reviewer directly (either via e-mail or on Matrix). + * Join developers on `Mozilla's Matrix server <https://chat.mozilla.org>`_, and ask if anyone knows why a review may be delayed. Please link to the bug too. + * If the review is still not addressed, mail the reviewer directly, asking if/when they'll have time to review the patch, or might otherwise be able to review it. + +For simple documentation changes, reviews are not required. + +For more information about the review process, see the :ref:`Code Review FAQ`. + +Review groups +------------- + + +.. list-table:: + :header-rows: 1 + + * - Name + - Owns + - Members + * - #anti-tracking + - `Core: Anti-Tracking </mots/index.html#core-anti-tracking>`__ + - `Member list <https://phabricator.services.mozilla.com/project/members/157/>`__ + * - #build or #firefox-build-system-reviewers + - The configure & build system + - `Member list <https://phabricator.services.mozilla.com/project/members/20/>`__ + * - #cookies + - `Core: Cookies </mots/index.html#core-cookies>`__ + - `Member list <https://phabricator.services.mozilla.com/project/members/177/>`__ + * - #dom-workers-and-storage-reviewers + - DOM Workers & Storage + - `Member list <https://phabricator.services.mozilla.com/project/members/115/>`__ + * - #devtools-reviewers + - Firefox DevTools + - `Member list <https://phabricator.services.mozilla.com/project/members/153/>`__ + * - #fluent-reviewers + - Changes to Fluent (FTL) files (translation). + - `Member list <https://phabricator.services.mozilla.com/project/members/105/>`__ + * - #firefox-source-docs-reviewers + - Documentation files and its build + - `Member list <https://phabricator.services.mozilla.com/project/members/118/>`__ + * - #firefox-ux-team + - User experience (UX) + - `Member list <https://phabricator.services.mozilla.com/project/members/91/>`__ + * - #firefox-svg-reviewers + - SVG-related changes + - `Member list <https://phabricator.services.mozilla.com/project/members/97/>`__ + * - #geckoview-reviewers + - Changes to GeckoView + - `Member list <https://phabricator.services.mozilla.com/project/members/92/>`__ + * - #gfx-reviewers + - Changes to Graphics code + - `Member list <https://phabricator.services.mozilla.com/project/members/122/>`__ + * - #webgpu-reviewers + - Changes to WebGPU code + - `Member list <https://phabricator.services.mozilla.com/project/members/170/>`__ + * - #intermittent-reviewers + - Test manifest changes + - `Member list <https://phabricator.services.mozilla.com/project/members/110/>`__ + * - #layout-reviewers + - Layout changes. + - `Member list <https://phabricator.services.mozilla.com/project/members/126/>`__ + * - #linter-reviewers + - tools/lint/* + - `Member list <https://phabricator.services.mozilla.com/project/members/119/>`__ + * - #mac-reviewers + - Changes to Mac-specific code + - `Member list <https://phabricator.services.mozilla.com/project/members/149/>`__ + * - #mozbase + - Changes to Mozbase + - `Member list <https://phabricator.services.mozilla.com/project/members/113/>`__ + * - #mozbase-rust + - Changes to Mozbase in Rust + - `Member list <https://phabricator.services.mozilla.com/project/members/114/>`__ + * - #necko-reviewers + - Changes to network code (aka necko, aka netwerk) + - `Member list <https://phabricator.services.mozilla.com/project/members/127/>`__ + * - #nss-reviewers + - Changes to Network Security Services (NSS) + - `Member list <https://phabricator.services.mozilla.com/project/members/156/>`__ + * - #perftest-reviewers + - Perf Tests + - `Member list <https://phabricator.services.mozilla.com/project/members/102/>`__ + * - #permissions or #permissions-reviewers + - `Permissions </mots/index.html#core-permissions>`__ + - `Member list <https://phabricator.services.mozilla.com/project/members/158/>`__ + * - #platform-i18n-reviewers + - Platform Internationalization + - `Member list <https://phabricator.services.mozilla.com/project/members/150/>`__ + * - #preferences-reviewers + - Firefox for Desktop Preferences (Options) user interface + - `Member list <https://phabricator.services.mozilla.com/project/members/132/>`__ + * - #remote-debugging-reviewers + - Remote Debugging UI & tools + - `Member list <https://phabricator.services.mozilla.com/project/members/108/>`__ + * - #spidermonkey-reviewers + - SpiderMonkey JS/Wasm Engine + - `Member list <https://phabricator.services.mozilla.com/project/members/173/>`__ + * - #static-analysis-reviewers + - Changes related to Static Analysis + - `Member list <https://phabricator.services.mozilla.com/project/members/120/>`__ + * - #style or #firefox-style-system-reviewers + - Firefox style system (servo, layout/style). + - `Member list <https://phabricator.services.mozilla.com/project/members/90/>`__ + * - #webcompat-reviewers + - System addons maintained by the Web Compatibility team + - `Member list <https://phabricator.services.mozilla.com/project/members/124/>`__ + * - #webdriver-reviewers + - Marionette and geckodriver (including MozBase Rust), and Remote Protocol with WebDriver BiDi, and CDP. + - `Member list <https://phabricator.services.mozilla.com/project/members/103/>`__ + * - #webidl + - Changes related to WebIDL + - `Member list <https://phabricator.services.mozilla.com/project/members/112/>`__ + * - #xpcom-reviewers + - Changes related to XPCOM + - `Member list <https://phabricator.services.mozilla.com/project/members/125/>`__ + * - #media-playback-reviewers + - `Media playback <https://wiki.mozilla.org/Modules/All#Media_Playback>`__ + - `Member list <https://phabricator.services.mozilla.com/project/profile/159/>`__ + * - #cubeb-reviewers + - Changes related to cubeb, Gecko's audio input/output library and associated projects (audioipc, cubeb-rs, rust cubeb backends) + - `Member list <https://phabricator.services.mozilla.com/project/profile/129/>`__ + +To create a new group, fill a `new bug in Conduit::Administration <https://bugzilla.mozilla.org/enter_bug.cgi?product=Conduit&component=Administration>`__. +See `bug 1613306 <https://bugzilla.mozilla.org/show_bug.cgi?id=1613306>`__ as example. diff --git a/docs/contributing/stack_quickref.rst b/docs/contributing/stack_quickref.rst new file mode 100644 index 0000000000..cd298fde1a --- /dev/null +++ b/docs/contributing/stack_quickref.rst @@ -0,0 +1,166 @@ +Working with stack of patches Quick Reference +============================================= + +Working on Firefox, we strongly recommend working with stack of patches. +Patches should be small and could be landed in the order used to push them. +This also helps to breakdown the work for different reviewers. + +As it can be complex for new comers, this documentation explains the +various commands. + +In Phabricator, the stack can be seen in the `Revision Contents` section. +The top of the stack (most recent change) is first in the list. + +This is also sometimes called "stack of revisions", "stack of commits" or "series of commits". + +**Example:** + +.. image:: img/example-stack.png + + +For the overall quick reference guide, see the :ref:`Firefox Contributors Quick Reference <Firefox Contributors' Quick Reference>` + +Visualize the stack +------------------- + +.. code-block:: shell + + # Mercurial + $ hg wip + + # Git + $ git log + + +Merge two patches +----------------- + +It can happen that, instead of updating a patch, a new revision is +created on Phabricator. For this, merge the patches locally: + +.. code-block:: shell + + # Mercurial + # Mark the patch to be merged with "roll" (key: "r") + # or "fold" (key: "f") + $ hg histedit + + # Git + # Replace "pick" by "squash" or "fixup" + $ git rebase -i + +Then, push to Phabricator and abandon the old change. + + +Submitting the first patch on the stack +--------------------------------------- + +There are times when you are working on multiple patches and +just want to submit the first one. For this, you can use: + +.. code-block:: shell + + $ moz-phab submit . + + +Reorder the stack +----------------- + +Sometimes, we want to change the order the patches in the stack. +Fortunately, VCS support this easily. + +.. code-block:: shell + + # Mercurial + # Just change the order the patch. The tool should highlight + # potential risks of conflicts. + # Note that ctrl+c works well if used + $ hg histedit + + # Git + # In the editor, just move the line below/above + # Remove everything if you want to cancel the operation + $ git rebase -i + + +Make a change on a patch at the beginning of the stack +------------------------------------------------------ + +In some cases, the reviewer is asking for a change at the bottom of the stack (ie not at the top). +So, a simple `hg/git commit --amend` would not work. + +In such case, the following approach can be used: + +.. code-block:: shell + + # Mercurial + # hg will try to guess in which an unambiguous prior commit + $ hg absorb + + # if this doesn't work, create a temporary commit + # and merge it using "fold" or "roll" + $ hg histedit + + # Git + $ git commit --fixup <hash of the commit> + + +Removing patches in the stack +----------------------------- + +To remove a patch in the stack: + +.. code-block:: shell + + # Mercurial + # select "drop" (letter "d") + $ hg histedit + + # Git + # Replace "pick" by "drop" + # Or simply remove the line for this commit + $ git rebase -i + + +Rebasing the stack +------------------ + +As the codebase moves fast, it can be necessary to pull changes from +mozilla-central before landing the changes. + +.. code-block:: shell + + # Mercurial + # First, see where your patches are in the stack + $ hg wip + # Then, rebase it: + # If you are a beginner, don't hesitate to add "--dry-run" + $ hg pull + $ hg rebase -b . -d central + + + # Git + $ git remote update + $ git rebase mozilla/central + + +Reorganizing the stack in Phabricator +------------------------------------- + +.. code-block:: shell + + $ moz-phab reorg [start_rev] [end_rev] + +allows you to reorganize the stack in Phabricator. + +If you've changed the local stack by adding, removing or moving the commits around, you need to change the parent/child relation of the revisions in Phabricator. + +.. code-block:: shell + + $ moz-phab reorg + +command will compare the stack, display what will be changed and ask for permission before taking any action. + +.. note:: + + Note that reviewbot will not restart the analysis. diff --git a/docs/contributing/vcs/mercurial.rst b/docs/contributing/vcs/mercurial.rst new file mode 100644 index 0000000000..f5f7cef39d --- /dev/null +++ b/docs/contributing/vcs/mercurial.rst @@ -0,0 +1,194 @@ +Mercurial Overview +================== + +Mercurial is a source-code management tool which allows users to keep track of changes to the source code locally and share their changes with others. +We use it for the development of Firefox. + +Installation +------------ + +See `Mercurial Page <https://www.mercurial-scm.org/downloads>`__ for installation. + + +Using `hg clone` +---------------- + +If you are not worried about network interruptions, then you can simply +use Mercurial to directly clone the repository you're interested in +using its URL, as given below. For example, to use the command line to +clone ``mozilla-central`` into a directory called ``firefox-source``, +you would use the following: + +.. code-block:: shell + + hg clone https://hg.mozilla.org/mozilla-central/ firefox-source + cd firefox-source + +Using Mercurial bundles +----------------------- + +If you are worried that your Internet connection is not fast or robust +enough to download such a large amount of data all in one go without +being interrupted and cannot clone using the command given above, then you are recommended to try :ref:`Mercurial bundles <Mercurial bundles>`. If interrupted, they can be resumed (continued without downloading +from the beginning) if the app you're using to download supports it. For +example, in Firefox you would right click on the download and select +`Resume` once your connection to the Internet was reestablished. + +Basic configuration +------------------- + +You should configure Mercurial before submitting patches to Mozilla. + +If you will be pulling the Firefox source code or one of the derived repositories, the easiest way to configure Mercurial is to run the vcs-setup mach command: + +.. code-block:: shell + + $ ./mach vcs-setup + +This command starts an interactive wizard that will help ensure your Mercurial is configured with the latest recommended settings. This command will not change any files on your machine without your consent. + + +Other configuration tips +------------------------ + +If you don't have the Firefox source code available, you should edit your Mercurial configuration file to look like the following: + +.. code-block:: shell + + [ui] + username = Your Real Name <user@example.com> + merge = your-merge-program (or internal:merge) + + [diff] + git = 1 + showfunc = 1 + unified = 8 + + [defaults] + commit = -v + +These settings can be added to ``$HOME/.hgrc`` (Linux/macOS) or ``$HOME\Mercurial.ini`` (Windows). + +You can configure the editor to use for commit messages using the `editor` option in the `[ui]` section or by setting the `EDITOR` environment variable. + +If you are trying to access the repository through a proxy server, see `these +instructions <http://www.selenic.com/mercurial/hgrc.5.html#http-proxy>`__ + + +Selecting a repository (tree) +----------------------------- + +There are multiple hg repositories hosted at mozilla.org to choose from. +A summary of the main trees is given below, but see +https://hg.mozilla.org/ for the full list. + +mozilla-central +--------------- + +This is the main development tree for Firefox. Most developers write +patches against mozilla-central. + +URL: https://hg.mozilla.org/mozilla-central/ + + +mozilla-beta +------------ + +The source for the current beta version of Firefox (and the next and all +previous betas). This code represents the expected next release of the +Firefox browser, and should be pretty stable. + +URL: https://hg.mozilla.org/releases/mozilla-beta/ + +mozilla-release +--------------- + +The source for the current release of Firefox (and the next and all +previous releases). + +URL: https://hg.mozilla.org/releases/mozilla-release/ + +autoland +-------- + +This is the integration tree for Firefox. Patches land in this repository first, +and then are merged by the sheriffs in mozilla-central. + +URL: https://hg.mozilla.org/integration/autoland/ + +L10n repos +---------- + +Mainly useful for localizers working on localizing Firefox. Code for all +l10n projects lives here and is organized into separate repos that (in +most cases) have the locale's two character ISO code. To get the repo +that you need look for the repo you're interested in on the following +page. + +URL: https://hg.mozilla.org/l10n-central/ + +Unified Repositories +-------------------- + +It is common for advanced users to want to interact with more than one +firefox repository. If you get to the point where having individual +copies of repositories is annoying you, then see +https://mozilla-version-control-tools.readthedocs.org/en/latest/hgmozilla/unifiedrepo.html +for instructions on doing this efficiently. + +Selecting a revision to build +----------------------------- + +Most of the time the `tip` revision of most repositories will build +without issue. If you are worried about it not, then you may want to +stick to mozilla-central. + +Building +-------- + +By default with no configuration a similar-to-release build is done. If +you wish you can :ref:`configure <Configuring Build Options>` the build using a ``.mozconfig`` file +and ``mach build``. +Different OSs have different prerequisites for a successful build, +please refer to the :ref:`build documentation <Getting Set Up To Work On The Firefox Codebase>` +to verify they are available on your build machine. + +Extensions +---------- + +There's a number of extensions you can enable. See http://mercurial.selenic.com/wiki/UsingExtensions. Almost everyone should probably enable the following, most of them are enabled by ``mach boostrap``: + +#. color - Colorize terminal output +#. histedit - Provides git rebase --interactive behavior. +#. rebase - Ability to easily rebase patches on top of other heads. +#. evolve - Enable and enhance the inprogress ChangesetEvolution work. +#. firefoxtree - Enhances the interaction with Firefox repositories. + +These can all be turned on by just adding this to your `.hgrc` file: + +.. code-block:: shell + + [extensions] + color = + rebase = + histedit = + firefoxtree = + evolve = + +In addition, there are some 3rd party extensions that are incredibly +useful for basic development: + +`mozext <https://hg.mozilla.org/hgcustom/version-control-tools/file/default/hgext/mozext>`__ + Mozilla-specific functionality to aid in developing Firefox/Gecko. + +Configuring the try repository +------------------------------ + +About :ref:`Pushing to Try <Pushing to Try>`. + +Learning to use Mercurial +------------------------- + +If you are new to Mercurial, you should start with the `official guide <https://www.mercurial-scm.org/guide>`__. + +Then, move on to the `version control tool docs <https://mozilla-version-control-tools.readthedocs.io/en/latest/hgmozilla/>`__ for Mozilla-centric Mercurial information. diff --git a/docs/contributing/vcs/mercurial_bundles.rst b/docs/contributing/vcs/mercurial_bundles.rst new file mode 100644 index 0000000000..66d6c26c72 --- /dev/null +++ b/docs/contributing/vcs/mercurial_bundles.rst @@ -0,0 +1,65 @@ +Mercurial Bundles +================= + +If you have a poor network connection that is preventing ``hg clone`` from completing, you may want to try downloading a bundle of the repository you're interested in. This is useful since a file download, unlike ``hg clone``, can be resumed if the connection is interrupted. Once you have the bundle, staying up-to-date shouldn't take much time at all, if you keep up with it regularly. + +This document explains the steps to setup the `mozilla-unified <https://hg.mozilla.org/mozilla-unified/>`__ repository using a bundle file. Be sure to replace "``mozilla-unified``" with the project you're working with as appropriate. + +Download the bundle +------------------- + +1. Open https://hg.cdn.mozilla.net/ : + +It lists up-to-date bundles for some of the repositories listed at https://hg.mozilla.net/ . +Each row corresponds to each repository, and each column corresponds to each compression format. + +2. Download the bundle file for the ``mozilla-unified`` repository: + +Click the link in the "mozilla-unified" row, the "zstd (max)" column. + +Setting up the repository +------------------------- + +Once you have downloaded the repository bundle, follow the steps below to recreate the repository locally based upon that bundle. + +1. Initialize a new repository (in a directory called ``mozilla-unified`` here): + +.. code-block:: shell + + mkdir mozilla-unified + hg init mozilla-unified + +2. Un-bundle the bundle file to that repository: + +Move the bundle file next to ``mozilla-unified`` directory, and rename it to ``bundle.hg``. + +.. code-block:: shell + + cd mozilla-unified + hg unbundle ../bundle.hg + +Get comfortable. Grab a coffee (or your favorite tasty beverage). Maybe a nap. This unbundling process is going to take quite a lot of time. + +3. Create the repository's config file ``.hg/hgrc``, and add the following lines, so that Mercurial will automatically know where to pull changes from future updates. You can open the template config file in your editor by running ``hg config --local --edit`` or ``EDITOR=<editor-of-your-choice> hg config --local --edit`` + +.. code-block:: shell + + [paths] + default = https://hg.mozilla.org/mozilla-unified/ + +4. Update the repository to get all the changes since the bundle was created (this step also doubles as a check of the bundle integrity since if its contents are not exactly the same as what's in the official repository then the ``hg pull`` will fail): + +.. code-block:: shell + + hg pull + +5. Check out a working copy from your new up to date repository: + +.. code-block:: shell + + hg update + +You now have a clone of ``mozilla-unified`` that is identical to one made via ``hg clone``. You can adjust your build settings, or you can go straight ahead and build Firefox! + +If at any point you are stuck, feel free to ask on Riot/Matrix at `https://chat.mozilla.org <https://chat.mozilla.org>`__ +in `#introduction <https://chat.mozilla.org/#/room/#introduction:mozilla.org>`__ channel. |
